From 227609f8025972fb2db0565dc286b1418400457a Mon Sep 17 00:00:00 2001 From: DonislawDev Date: Sun, 2 Aug 2026 18:24:02 +0200 Subject: [PATCH 1/2] chore(repo): stop tracking the internal changelog CHANGELOG-INTERNAL.md is ours: it records ADRs, rejected variants, measured dead ends and every added test. It belongs next to the notes it argues with, not in a public checkout. Owner's decision; older revisions stay in git history and that is accepted. Consequences handled here rather than discovered later: - tests/test_version_and_release.py can no longer read it, so the shared CHANGELOG_FILES tuple and test_breaking_sections_come_first now cover only the shipped changelog. A test that reads an absent file would redden CI for a reason no CI could fix. The dated-section test lost the word "both" from its name because it is no longer true. - release.yml checks one changelog at tag time instead of two. - The PR template no longer asks outside contributors to edit a file they cannot see. - Two comments pointed readers at it from engine.py and test_inject_batch.py; both now say what happened instead of where to read about it. - The repository scanners skip it, so they keep measuring the same set here and in CI. Its structure is still guarded, just not from here: BREAKING-comes-first and the dated section for VERSION.txt moved into the Stop hook, which runs where the file exists. What that costs is worth stating: at the moment of tagging, nothing in CI can see this file any more. Co-Authored-By: Claude Opus 5 --- .github/PULL_REQUEST_TEMPLATE.md | 2 +- .github/workflows/release.yml | 4 +- .gitignore | 4 + CHANGELOG-INTERNAL.md | 5230 ----------------------------- beantester/engine.py | 2 +- tests/test_inject_batch.py | 4 +- tests/test_repo_conventions.py | 6 +- tests/test_version_and_release.py | 13 +- 8 files changed, 24 insertions(+), 5241 deletions(-) delete mode 100644 CHANGELOG-INTERNAL.md diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 4287788..4fddbb4 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -7,6 +7,6 @@ - [ ] `python -m pytest tests` passes locally. - [ ] New behaviour has tests (see `tests/` for the style). - [ ] UI text goes through i18n keys, with **both** `lang/en.json` and `lang/pl.json` updated. -- [ ] User-facing changes noted in `CHANGELOG.md`; technical ones and new tests in `CHANGELOG-INTERNAL.md`, under `[Unreleased]`. +- [ ] User-facing changes noted in `CHANGELOG.md`, under `[Unreleased]`. - [ ] Commits follow Conventional Commits (`type(scope): summary`). - [ ] No version bump - the owner closes a version via `VERSION.txt`. diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index eba8a3b..b4996ef 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -64,7 +64,9 @@ jobs: version="$(cat VERSION.txt)" escaped="${version//./\\.}" fail=0 - for f in CHANGELOG.md CHANGELOG-INTERNAL.md; do + # CHANGELOG-INTERNAL.md is not in the repository, so this job cannot + # see it. Its structure is guarded locally by .claude/hooks/check_notes.py. + for f in CHANGELOG.md; do grep -qE "^## \[${escaped}\] +- +[0-9]{4}-[0-9]{2}-[0-9]{2}" "$f" \ || { echo "$f: no dated '## [$version] - YYYY-MM-DD' section"; fail=1; } open="$(awk '/^## \[Unreleased\]/{u=1;next} /^## \[/{u=0} u && /^- /{n++} END{print n+0}' "$f")" diff --git a/.gitignore b/.gitignore index 22d8002..ccc6a0b 100644 --- a/.gitignore +++ b/.gitignore @@ -31,6 +31,10 @@ BeanNetworkTester-*.zip CLAUDE.md PROJECT_NOTES.md HISTORY_NOTES.md +# The technical changelog is ours: it records ADRs, rejected variants and +# every added test. It stays next to the notes it argues with, not in the +# release. Its structure is guarded by .claude/hooks/check_notes.py. +CHANGELOG-INTERNAL.md # Internal scripts kept between sessions: measurement rigs, probes, diagnostics. # Never shipped, never a CI dependency, backed up by the owner separately. diff --git a/CHANGELOG-INTERNAL.md b/CHANGELOG-INTERNAL.md deleted file mode 100644 index c95ecd1..0000000 --- a/CHANGELOG-INTERNAL.md +++ /dev/null @@ -1,5230 +0,0 @@ -# Internal changelog - -Technical, developer-facing history for Bean Network Tester. This file is NOT shipped to -users - the user-facing log is `CHANGELOG.md`. Keep entries technical: which -modules/registries were touched, decisions, format migrations, CI changes, and NEW TESTS. -This is the one place we record added tests (file/test name + what it guards). Entries may -reference conventions (for example "convention 24") and code symbols. - -The format follows [Keep a Changelog](https://keepachangelog.com/); versions follow SemVer. -Entries go under `[Unreleased]`; the owner closes a version by setting `VERSION.txt` (never -bump the version here). Plain hyphen only, no em/en dashes (convention 33). - -Breaking changes must be visible at a glance: a change that breaks a public contract (CLI -flags, exit codes, the NDJSON schema, on-disk file formats, or the facade's public API) gets -a `### BREAKING` section placed FIRST in that version, and each such line is prefixed with -`**BREAKING:**`. A breaking change also requires a version bump by the owner (convention 34). - -## [Unreleased] - -### Fixed - -- **A START-only field rendered by a WINDOW never locked.** `App._sync_running_ui` refreshed - `self.form` only, so `narrow_filter` - which lives on `SettingsWindow`'s own `ControlForm` - (`surface="settings"`, convention 42) - stayed `state="normal"` for the whole session whenever - the window was open BEFORE START. Opened AFTER a start it came up correctly disabled, because - the build path reads `is_locked` and nothing re-read it afterwards. Measured on the fake Tk both - ways before and after the fix. Two halves, both guarded: `SettingsWindow.refresh()` now calls - `form.refresh_field_states()` (self-healing on the 700 ms tick, whatever moved the state), and - `_sync_running_ui` ticks the open windows so the lock is immediate rather than a tick late. - Conventions 7, 21 and 42. - -### Tests - -- **The hot path's "zero allocations" rule got a gate.** New - `tests/test_hot_path_allocations.py` pins that `decide()` RETAINS nothing per packet: at most - 64 blocks and 4096 bytes over 5000 calls, against a measured floor of 13 blocks and 608 bytes - that stays flat with duplication, latency, the NAT flow table and 200 ports. `test_hot_path.py` - guards the neighbouring rule (no syscalls on the packet threads); this catches the other class, - where something starts being kept once per packet. - 🔴 **It reads two meters because one of them has a hole, and the hole was found by mutation - rather than by reasoning.** The first version counted `sys.getallocatedblocks()` only, and a - mutation making `decide()` append to an ever-growing list **survived**: the appended value was - a cached small int, so no object was created and blocks moved from 6 to 7. `tracemalloc` - current sees the same case as 42 032 bytes against 208. The meter-canary test now proves both - meters can rise, including the references-only case specifically. Still not caught, stated in - the file: transient garbage, since both meters are net. -- **Size ceilings, as a ratchet.** New `tests/test_code_shape.py` caps a function at 167 logic - lines and a module at 1299 - today's maxima (`theme.py::init_style`, `gui/app.py`), so nothing - had to be rewritten to make it pass. Lowering them is routine work, raising either is the - owner's call. **Comments and docstrings do not count**, and that is measured rather than - assumed: 9 776 of the package's 17 770 lines are logic, so 45% is explanation, and a raw-line - cap would have been a cap on explaining. A second test pins that property directly (ninety - lines of comment measure the same as none). Both mutation-checked, plus the empty-scan canary. -- **"Verified by mutation" became checkable.** New `tests/test_mutation_registry.py` holds the - claim as data in three states that say three different things: `MUTATIONS` (re-runnable now, - five entries), `PROVEN_BY_HAND` (a dated session did it, no patch was written down, so no - machine repeats it) and `NOT_PROVEN` (no mutation, said out loud). Until now the phrase - appeared over twenty times in the notes and in a dozen docstrings with nothing behind it - - convention 5's own evidence was the prose convention 5 warns about. The suite checks the - bookkeeping only: every named test exists, none is filed under two states, and every search - pattern still occurs exactly once, so a registry entry rots the day the code moves rather - than the day someone runs it. The mutations themselves run from `internal_tools/mutate.py` - (outside git, one subprocess suite run each), with a mandatory canary entry that must report - BROKEN - a tree that fails to compile also exits non-zero, so without it a whole run can - report "all caught" and mean nothing. First full run: 5 caught, 0 survived, canary BROKEN. -- **The repository scanners now prove they read something.** - `test_repo_conventions.py::test_the_repository_scanners_actually_read_files` names one file - each collector must return plus a floor on the count, for all seven collectors across - `test_repo_conventions`, `test_code_hygiene`, `test_layering` and `test_readme_guards`. A - glob or walk that comes back EMPTY satisfies every assertion built on it and looks exactly - like a working guard, so renaming `beantester/` would have silenced four guards at once in - silence. Mutation-checked: emptying a collector and pointing the walk at a missing root each - turn it red. -- **The whole-tree scans stopped measuring files that are not in the repository.** - New `repo_text_files()` collector skips `internal_tools/`, `.claude/`, `crashes/` and the - private notes, and `::test_the_repository_scanners_stay_out_of_what_is_not_in_the_repository` - keeps them out. Measured before the change: the dash scan covered 183 files, 12 of them - git-ignored, so the same test measured a different set locally than in CI - including - `crashes/latest-crash.txt`, whose text comes from OS exceptions. Convention 33's coverage of - the private notes moves to the Stop hook, which runs where those files exist. -- `tests/test_gui_release_fixes.py::test_start_only_fields_are_locked_while_a_session_runs` - rewritten to DERIVE its subjects from `fields.FIELD_DEFS` instead of naming `duration` and the - filter combobox by hand, and to resolve each field to the surface that renders it - (`SECTIONS[].surface`, with `CHOICE` belonging to `App`, not to a form). The old shape was a test - of two examples, so `narrow_filter` was added to the registry, shipped unlocked and left the - suite green - PROJECT_NOTES rule 2.6 (one value, several consumers, a guard on one of them) and - the same mistake convention 16 records. It now opens the Settings window BEFORE the session - starts, which is the order that was broken. Verified by mutation: removing either half of the - fix turns it red. - -## [0.4.0] - 2026-08-01 - -### BREAKING - -- **BREAKING:** **`load_config_file` rejects unknown keys** (`settings.py`), the same rule - `scenario.py` already applies to step keys and settings names. Was - `s.update({... if k in DEFAULT_SETTINGS})` - a silent filter. Touches a frozen on-disk format - ("Kontrakty publiczne"), so it needs the owner's version bump. - New i18n keys in BOTH files: `errors.config_unknown_setting`, - `errors.config_unknown_setting_hint` (the latter used only when there is exactly one unknown key - and `difflib.get_close_matches` finds a near miss - `latancy` -> `latency`, which is the whole - failure mode this closes). - **The recorded counter-argument was checked before changing this, and it is about a different - file.** `CHANGELOG-INTERNAL` on the `ui.json` store says unknown keys are kept on purpose, - because dropping them "would silently discard state written by a newer version". That reasoning - holds THERE: `ui.json` is written by the program and read key by key, so a typo is impossible and - forward compatibility is the only thing at stake. The traffic config file is hand-written (README - recipe 2 exists to validate hand-written ones in a pipeline), so a typo is the common case and - silence is the expensive answer. The forward-compat cost is real and is stated in the public - changelog rather than glossed. - `save_config_file` emits exactly the `DEFAULT_SETTINGS` key set, so nothing this tool writes is - affected - guarded now by `::test_every_file_this_tool_writes_still_loads`, which is what keeps - that true when a setting is renamed or dropped. - Consumers swept (rule 2): `cli.py:205` (already wraps `ValueError` -> `CONFIG(3)`, so free) and - `gui/app.py:1280` (already inside `try/except Exception` -> `dialogs.show_error`, so the dialog - path is free too - but it got its own guard, since a hand-written file is most likely to be - opened there). Profiles do NOT go through this path (`gui/profiles.py` + `PROFILE_FILE`), so they - are untouched. - Tests: `test_settings_config_scenario.py:: - test_a_misspelled_setting_in_a_config_file_is_an_error_not_a_silent_default` (REPLACES - `::test_config_file_unknown_keys_ignored`, which pinned the old behaviour - and whose own check - message already said "rejected" while the code ignored), - `::test_every_file_this_tool_writes_still_loads`; - `test_cli_runtime.py::test_dry_run_catches_a_misspelled_setting_in_a_config_file`; - `test_gui_file_actions.py::test_a_misspelled_setting_reaches_the_user_as_a_dialog`. - -- **BREAKING:** **a finished scenario ends the CLI run** (`stop_reason: "scenario_done"`, a new - value of an existing NDJSON field, so it needs the owner's version bump). - `ScenarioRunner.finished` is set on the timeline-complete branch only (`scenario_runner.py`), - and `BeanEngine.scenario_finished()` exposes it; `cli._report_loop` returns `"scenario_done"` - when `cfg["stop_on_scenario"]` is set. **The end of a timeline is ASKED OF THE RUNNER, not - recomputed from `Scenario.duration` in the CLI** - the tail past the last step (`duration + 0.1`) - lives in the runner, and a second reader of the same fact drifts on the first edit ("Jedno - zrodlo prawdy"). - `finished` deliberately means *the timeline ran out* and NOT *the runner stopped*: `stop()` and a - dead engine also end that loop, and reporting either as `scenario_done` would dress a faulted run - up as a clean one. `cli._plan_the_end_of_the_scenario` decides and announces; it declines for a - looping scenario and for `duration == 0` (a single step at `at: 0` is settings, not a timeline, - and would otherwise end the session inside 0.1 s), warning in both cases. An injected engine - without `scenario_finished` (public `engine=` seam) falls back to the old behaviour but SAYS so - at debug level, because a silent fallback would quietly restore the hang. - Measured before: a two-step non-looping scenario with no `--duration` was still sampling when a - hard timeout killed it at 12 s (exit 124). After: `exit 0, reason=scenario_done`. - Note the detection inherits the report loop's wake-up granularity - the run ends at the first - wake AFTER the timeline, so up to `--interval` late. That bound is the sleep-cap item, not this - one. - New tests: `tests/test_scenario_runner.py::test_a_completed_timeline_reports_that_it_finished`, - `::test_a_looping_runner_never_reports_finished`, - `::test_a_runner_the_engine_shut_down_did_not_finish`; - `tests/test_cli_runtime.py::test_a_finished_scenario_ends_the_run_even_without_a_duration`, - `::test_an_explicit_duration_still_wins_over_the_scenario`, - `::test_a_scenario_with_no_timeline_does_not_cut_the_run_short`, - `::test_a_looping_scenario_runs_to_its_duration_not_to_its_timeline`, - `::test_a_scenario_that_cannot_end_the_run_says_so_up_front`, - `::test_the_loop_flag_takes_the_derived_ending_away_again` (``--loop`` is folded into - ``scen.loop`` BEFORE the end is planned, so reading the file's own flag instead of the effective - one would end a run the user asked to repeat), - `::test_an_engine_that_cannot_report_the_scenario_end_says_so`. - Those last two were written AFTER the code and so were never red; both were verified by mutation - instead. The second one paid for itself immediately: `_plan_the_end_of_the_scenario` used to log - the injected-engine case and then FALL THROUGH to the shared warning, telling the reader a - two-step file "has a single step, so there is no timeline". The branch is now an `if/elif/else` - where every reason is true for its case, and the test asserts the reason given is that one rather - than merely that some warning appeared. (Mutation also showed the two chunks composing: without - the guard the double raises `TypeError` into the new session handler and comes back as a coded - `RUNTIME` instead of a traceback.) - The CLI tests cannot use `FakeClock` (the runner reads the wall clock on its own thread), so they - run on real time with a budgeted `sleep` that raises `_NeverEnded` - a **`BaseException`**, since - the new session handler below catches `Exception` and would otherwise swallow the signal and make - a hang look like a clean `RUNTIME`. That is a hang turned into a named failure instead of a test - that stalls the suite. - Stability pass (convention/rule 8, the flag is read off-thread): 4.8M cold reads before any - runner exists - all `False`, no exceptions; 30 cycles x 8 concurrent readers - no within-runner - `True -> False`; runner swap and `stop_scenario` both leave `False`. (The first version of that - check counted transitions ACROSS runners and "found" hundreds of regressions - `start()` resets - the flag by design. The script was wrong, not the code.) - -- **BREAKING:** **`capture_narrowed` column in the statistics CSV.** New `App.CSV_SESSION_COLUMNS` - (keyed by `session_info()` key, the way `CSV_COLUMNS` is keyed by counter key) and - `_csv_session_value`, written between the timestamp and the counters. Booleans go out as - `yes`/`no` in English, like `impaired` in the connections export - a CSV is read by scripts, not - in the interface language. Touches a frozen on-disk format ("Kontrakty publiczne"), so it needs - the owner's version bump; the existing file rotates through the header-change path that already - exists in `export_csv`. - Why it is worth breaking: the file's own comment says an append log whose columns mean different - things in different rows is worse than useless, and that argument was applied to the VIEW - preference (both totals carried, reader narrows them) while capture narrowing - the stronger - version, which changes `seen` itself and cannot be undone by picking a column - had no marker at - all. The CLI's JSON summary has carried the same fact since narrowing shipped, and the repro - report embeds the whole `session_info`; this was the last surface that could not say. - `test_readme_guards.py::test_both_readmes_document_every_csv_column` now unions the session - columns, and both READMEs document it (table row plus the paragraph explaining why this file - ignores the view switch but cannot ignore this). - Guard: `test_view_scope.py::test_the_stats_csv_records_which_world_each_row_was_measured_in` - exports twice across a changed verdict and reads the file back. Four mutants, all caught: the - column removed, its value pinned, the row desynchronised from the header, and the README entry - renamed. Two earlier mutants were BAD mutants rather than results - deleting the guard line - itself (nothing can catch a deleted test) and renaming the column to a SUPERSTRING (the docs - guard matches by substring, so `capture_narrowed_TYPO` still contains `capture_narrowed`). - Two consumers pin the header literally and went red on the full run, which is the change proving - itself: `test_conns_export.py::test_export_csv_stats_appends_then_rotates_on_a_column_change` and - the CSV section of `smoke_gui.py`. Neither was in the pre-change sweep - the sweep found the - readers of the FACT and missed the readers of the HEADER. - -- **BREAKING:** **`reset_now` removed from `scenario.ACTIONS`**, leaving `reset_tcp` as the only - action. Touches the frozen scenario-file format ("Kontrakty publiczne"), so it needs the owner's - version bump. Thanks to the step validation added earlier in this release, a file still using the - old name fails with a message naming the step rather than being silently ignored. -- **`scenario_runner` now reads `ACTIONS` instead of carrying its own copy of the names.** It - matched against a hand-written `("reset_tcp", "reset_now")` one module away from the tuple that - decides which names are legal. The two agreeing was a coincidence maintained by hand, and the - drift is silent BOTH ways: an action dropped from `ACTIONS` keeps firing here, and one added - there validates and then does nothing. Same class as the preset mapping the CLI used to duplicate. - Guard: `test_scenario_runner.py::test_the_runner_fires_exactly_the_actions_the_validator_accepts` - drives every name in `ACTIONS` through the real runner loop and asserts a rejected name fires - nothing. Verified by mutation - restoring the hardcoded tuple fails with - `the runner ignores a name the validator would reject`. - 🔴 **`reset_now` still exists as a METHOD name** (`core.reset_now`, `engine.reset_now`, - `App.reset_now_click`, the `buttons.reset_now` / `tips.reset_now` keys) - that is the GUI's - "Reset TCP now" button and the engine's public API, unrelated to the scenario action. A - grep-and-delete would have removed a working feature. Noted in `PROJECT_NOTES` beside the action - list. - Three fixtures in `test_settings_config_scenario.py` used `reset_now` as a scenario action and - went red on the change, which is the removal proving itself - they now use `reset_tcp`. (My - pre-change survey missed them: the grep output was cut by `head`, and I reported "no test uses it - as a scenario action". The tests were right and the summary was wrong.) - -- **BREAKING:** **`scenario.py` validates the keys it always accepted silently.** New `STEP_KEYS` - and `FILE_KEYS` registries; an unknown key in a step or at the file level raises a translated - error naming it, exactly as an unknown `settings` name already did. `duration` is validated as a - number `>= 0` and rejected without an `action`. - The bug this closes was a SILENT one, which is why it was worth a breaking change: `duration` - reached the engine as `float(step.get("duration", 3.0))` straight off the unvalidated dict, so a - string raised on the SCENARIO THREAD (killing the timeline mid-run while the session carried on, - looking exactly like a file being ignored) and a misspelling fell back to 3 s. Same shape for - `"lop"` instead of `"loop"`. Touches a frozen on-disk format (`PROJECT_NOTES`, "Kontrakty - publiczne"), so it needs the owner's version bump. - New i18n keys in both files: `errors.scenario_unknown_key`, `errors.scenario_unknown_file_key`, - `errors.scenario_step_duration`, `errors.scenario_duration_without_action`. - Tests (`test_settings_config_scenario.py`): `::test_a_scenario_step_rejects_a_bad_duration`, - `::test_a_scenario_step_rejects_a_duration_with_nothing_to_apply_to`, - `::test_a_misspelled_scenario_key_is_an_error_not_a_silent_default` - the last one asserts the - offending key is NAMED in the message, since "invalid scenario" without a name would leave the - user hunting. Verified by mutation: disabling the two checks turns all three red. - -- **BREAKING:** **every preset was re-tuned against measured sources, and five were added** - (`presets.py`). The table is now 17 entries. Two systematic errors ran through the old values: - - **Latency was dialled as if it were a round trip.** `lat` is added to EVERY packet, and with - the default two-way filter that is both the request and the reply, so the delivered ping is - `2 x lat`. `presets.satellite` at 600 delivered a **1200 ms** ping against a real GEO figure of - 683-684 ms (Ookla Q1 2025 medians for HughesNet and Viasat) - it is now 340. Same correction - for `lte` 45 -> 30, `roaming` 300 -> 200, `modem56k` 200 -> 100 and `3g` 150 -> 90 (UMTS is - 100-200 ms RTT, not 300). `jit` deliberately was NOT halved: each packet draws its own - uniform, so the spread grows by sqrt(2), not 2 (measured: std 32.7 ms of RTT wobble at - `jit=40` against 23.1 ms per packet). - - **Three presets carried kbit/s numbers in a KB/s field, so they ran 8x fast.** `dsl` 1536/256 - is the canonical ADSL pair in kbit/s; `3g` 384/128 is UMTS R99's. `3g` therefore delivered - 3.1 Mbit/s - HSPA+, not the experience the preset is picked to reproduce. Also corrected: - `5g` upload (67 -> 21 Mbit/s; 5G upload runs 50-120% above 4G, not 6x), `weak_wifi` and `cafe` - (both were faster than most home DSL), and `terrible`, whose 2.1 Mbit/s made the WORST preset - faster than the 3G one. - - **New:** `presets.leo` (low-orbit satellite), `presets.distant` (fast pipe, high RTT, no loss - - the shape every other high-ping preset misses), `presets.bufferbloat` (idle-good link whose - queue is the impairment - the only preset that exercises `buffer`), `presets.metro` (periodic - tunnel outages; the only one that takes the link fully down long enough to force a RECONNECT) - and `presets.inflight` (satellite in-flight: 750 ms RTT and 7% median loss, measured over 45 - flight-hours in "Mile High WiFi", WWW 2018). - - 🔴 **LEO carries a spike, NOT a flap, and that reversal came from the sources.** The first pass - described Starlink's 15 s handover as a link outage and justified the whole `flap` field with - it. Measurement says the reconfiguration stops transmission for 100 ms but the packets are - **QUEUED, not dropped** ("Making Sense of Constellations"), with the latency peak averaging - +74 ms ("A Multifaceted Look at Starlink Performance", WWW 2024). 100 ms out of every 15 000 is - 0.67% of the timeline, hence `spike_prob=0.7, spike_ms=100` and no flap. Modelling it as loss - would have made the tool impair harder than the network it names. - - Every value in `PRESETS` now carries either a named source or an explicit **JUDGEMENT** marker - in the comment beside it (`weak_wifi`, `cafe`, `roaming`, `terrible` and the flap duty cycles - of `metro`/`inflight` are judgement - no canonical measurement exists for them). The source - block above `PRESETS` lists what was actually read. Rationale in PROJECT_NOTES rule 5: a number - describing the world outside this repo cannot be falsified by any test here, so it needs a - citation or an admission. - - Tests: `test_cli.py::test_cli_parsing_and_override` and - `test_cli_runtime.py::test_print_config_dumps_the_effective_settings` read the expected latency - from `PRESETS` instead of a hardcoded `150`. Both are about PRECEDENCE and about a preset - REACHING the dump; a copy of the value only ever fails when a preset is retuned. - - Names in both language files, and `presets.satellite` renamed to say **geostationary** so the - contrast with the new low-orbit entry is visible in the picker (the id is unchanged, so a - stored `ui.json` selection and any saved config keep working; `--preset "Satellite link"` and - `--preset "Lacze satelitarne"` do not, which is why this line is BREAKING). `presets.dsl` says - VDSL for the same reason - the numbers moved and the name should say which DSL it means. - - 🔴 **New guard: `test_presets_filters.py::test_every_preset_has_a_name_in_every_language`.** - `PRESETS` had no link to the language files at all - `fields.py` has had one since forever - (`test_field_registry.py::test_labels_and_tips_exist_in_every_language`), this registry did - not. Five presets were added, rendered as raw `presets.leo` in the picker and in `--preset`, - and the suite stayed entirely green. - The guard reads the language FILES rather than calling `translate()`, and that distinction was - found by mutation, not by reasoning: a key missing from Polish **falls back to the English - text**, which is not equal to the key, so the first version of this test passed unchanged when - the Polish `presets.leo` line was deleted. Re-verified after the fix - the same deletion now - fails with `presets: every id has a pl name (['presets.leo'])`. (`test_i18n_coverage` catches - the same deletion from the other side, by comparing key SETS; this one catches a preset id that - reaches neither file.) - - **All seven shipped `scenarios/*.json` recalculated onto the same scale.** They carried the two - errors the presets did - rates on the 8x scale (`down: 8000` as an "LTE baseline") and - latencies dialled as round trips - so a scenario and a preset described the same network with - different numbers. Latencies halved, rates mapped onto KB/s. The pleasant consequence: the - DELIVERED ping is now the number the file says, because halving `lat` is exactly what makes - `2 x lat` equal the author's intent - `cafe-wifi.json` said 40 and delivered 80; it now says 20 - and delivers 40. `mobile-lte-to-3g.json` walks `presets.lte` (ping 60, 33.6 Mbit/s) down to - `presets.3g`'s bandwidth (0.8 Mbit/s) and back, so the file and the preset finally agree about - what "LTE" and "3G" mean. **Jitter was NOT halved**, in scenarios or presets: each packet draws - its own uniform, so the spread grows by sqrt(2), not 2, and the existing values remain - plausible read as per-packet figures. The shape of every story (the relative progression, the - reset points, the loop) is untouched - this was a recalculation, not a redesign. - Covered by the existing `test_shipped_scenarios.py`, which drives every file through the real - validator. - - **The profile picker was a hardcoded `width=24` characters** (`gui/pages/control.py`), against a - longest name of 34 (EN) and 35 (PL), so the popdown truncated in silence - ttk sizes a combobox - popdown from the widget and never from its contents. New `theme.popdown_width(values)` is the - other half of `popdown_height`: that one takes rows from how MANY values there are, this takes - characters from how LONG they are, capped at `POPDOWN_MAX_CHARS = 44` because a profile name is - whatever the user typed and the row also holds two buttons. Applied at build AND in - `App._sync_profile_widgets`, since saving a profile is how a long name enters the list. - Tests (`test_gui_layout.py`): `::test_the_profile_picker_fits_its_longest_name` and - `::test_the_profile_picker_regrows_when_a_long_profile_is_saved`. - **What each actually catches, measured rather than assumed.** Mutating ONE path leaves the - picker correct, because the other still sets the width - so the fit test only goes red when - BOTH are broken, which is the pre-fix state and exactly the reported bug: it fails with - `(24, 'Zapchane lacze domowe (bufferbloat)')`. The regrow test is the one that pins the sync - path on its own. The first version of the regrow test was also wrong and its own assertion said - so: it saved a profile SHORTER than the longest preset and expected the picker to grow - (`37 -> 37`), so the name in it is now deliberately longer than any built-in. - - New guard `test_readme_guards.py::test_both_readmes_list_every_preset_id`: the `--preset` id - list is typed by hand in both READMEs and nothing tied it to `PRESETS` - the same shape as the - project-layout guard right above it. Checks BOTH directions, because a renamed-away id leaves a - line pointing at a `--preset` value the CLI rejects. Verified by mutation (deleting - `presets.leo` from README.md fails with `missing: ['presets.leo']`). - - The two tests that pinned the old Polish name kept their POINT rather than being deleted: they - exist for the stroke-letter fold (`ł` does not decompose under NFD), so they moved to - `"Odlegly serwer (inny kontynent)"`, `"Zapchane lacze domowe (bufferbloat)"` and - `"Pociag / metro (tunele)"`. The `STROKE_LETTERS` comment in `presets.py` cited the old name as - its example and was updated with them. - -- **BREAKING:** **the profile scope grew from 7 fields to 12**, and the shape a profile is stored - in is now DERIVED from the field registry instead of a second hand-written table. `spike_prob`, - `spike_ms`, `flap_period`, `flap_down` and `buffer` are marked `in_profile=True` in - `fields.FIELD_DEFS`; `presets.PRESET_TO_SETTING` is built from `Field.in_profile` + - the new `Field.preset_key` (the frozen short keys `lat`/`jit` are now declared by the registry, - not translated by a second table). This reverses the "Variant A" decision that deliberately kept - a profile to the seven classic preset fields - flapping is PERIODIC and phase-locked to the - session start (`BeanCore.decide` step 5), so it is the one impairment that lets a profile - describe a link that drops out on a cadence, which no other profile field can express. - Consequences worth naming: - - `PROFILE_FIELDS` had **no production consumer** before this (only `test_field_registry.py`); - what a profile actually stored came from `PRESET_TO_SETTING` in another module, so the two - could drift with nothing to notice. `in_profile` is now the only switch. - - `preset_to_settings()` is **always complete**: a preset names only the fields it means, and - every other profile field comes back at its `DEFAULT_SETTINGS` value. A partial answer would - leave the previous profile's flapping on screen after picking a clean link. - - The fill is the field's **default, not zero**. `buffer` defaults to 1000 ms and 0 means - UNBOUNDED there, so a blanket zero-fill would have quietly reinstated the runaway token bucket - (`BeanCore.decide` step 11) on every preset pick. `settings_to_preset()` lost its hardcoded - `0` fallback for the same reason. - - `buffer` is the only profile field that cannot impair anything on its own: both of its readers - sit inside `if rate > 0` (`decide` steps 11 and 12), so with no speed limit it touches no - packet. It is in the profile because `down`/`up` are - the buffer is what decides whether that - same limit shows up as DELAY or as LOSS. - - Visible effect for existing presets: none of the twelve names a buffer, so picking one now sets - the buffer to 1000 ms explicitly where it previously left the widget alone. - - Tests: `test_field_registry.py::test_the_stored_profile_shape_follows_the_registry` (new - the - stored shape covers exactly `PROFILE_FIELDS`, the short keys stay short, every stored field has - a default); `test_validators_settings.py::test_a_preset_always_yields_every_profile_field` - (new - unnamed fields come back as defaults, not zero and not missing); - `test_passthrough.py::PRESET_OFF` is now derived from `IMPAIRMENT_OFF` intersected with the - profile scope, so an impairment joining the scope cannot be forgotten there, and `buffer` is - correctly excluded (demanding `buffer == 0` of a harmless preset would demand an unbounded - buffer). `test_profile_scope_is_derived` updated to the 12 fields. - - `presets.py` gained module-level imports of `fields` and `settings`. No cycle: nothing in - `settings`'s dependency chain imports `presets` (checked), and `test_layering.py` is green. - -- **BREAKING:** **`--preset` applies a preset through the shared mapping** (`cli.py`), instead of - the copy of it that hand-wrote the seven classic keys. The GUI has always used - `preset_to_settings`, so the two front ends applied different subsets of the same preset name; - the only guard, `test_cli.py::test_cli_english`, asserts `latency` and `down` and nothing else. - With the profile scope widened this also stopped being a silent difference: the copy indexed - `p["corrupt"]` directly, so the first preset naming only the fields it means would have taken the - CLI down with a `KeyError` (reproduced against the future `presets.leo` shape before the fix). - Documented precedence is unchanged (defaults < file < preset < flags), but a preset now carries - five more keys, so `--config f.json --preset presets.3g` resets `buffer`, `spike_*` and `flap_*` - to their defaults where it previously left the file's values in place - the same rule that has - always applied to `loss` and `down`. - Tests: `test_cli.py::test_a_preset_carries_every_profile_field_into_the_settings` (new) drives a - PROBE preset whose every profile field is non-default, because run against the twelve real - presets the check is vacuous - none of them names a buffer, a spike or a flap, so the old copy - left exactly the defaults the shared mapping fills in, and the test would pass against the code - it exists to reject. `::test_a_preset_that_names_only_some_fields_does_not_break_the_cli` (new) - covers the partial-preset `KeyError`. Both verified by mutation: reverting `cli.py` turns them - red with `buffer 1000 != 1007` and `KeyError: 'corrupt'` respectively. - -- **`ProfileStore._clean` falls back to each field's DEFAULT, not to zero** (`gui/profiles.py`). - `float(values.get(key, 0) or 0)` was correct only while every profile field defaulted to zero. - It does not any more: a `profiles.json` written before the scope widened has no `buffer` key, and - `buffer = 0` means an UNBOUNDED link buffer, so the old fill would have loaded every profile - saved by an earlier version as the runaway token bucket the bounded buffer exists to prevent - - silently, on settings the user had already saved. Absence (`null` and `""` included) now takes - `presets.PRESET_DEFAULTS[key]`; an explicit `0` in the file stays 0, because that is what "no cap - on the queue" looks like when it is meant. The on-disk format therefore needs NO migration and no - version field: it stays backward compatible (fewer keys read fine) and forward compatible (an - older build ignores keys it does not know, losing only the new fields). - Side effect of dropping `or 0`: a falsy non-numeric value (`[]`, `{}`) now drops the profile and - reports it via `store.problem`, where it used to load silently as 0. That matches what the module - promises - validate every entry, drop what it cannot use, and say so. - Tests (all new, `test_release_fixes.py`): - `::test_a_profile_from_before_the_wider_scope_loads_with_field_defaults` - a seven-key file loads - with its own values, `buffer` at the default and the impairments it never named off; verified by - mutation (restoring the blanket zero turns it red with `buffer 0.0`). - `::test_an_explicit_zero_buffer_is_not_the_same_as_a_missing_one` - the paired constraint that - makes the fallback usable; it is deliberately NOT a regression guard (it survives the mutation, - because both paths yield 0 there). - `::test_a_profile_round_trips_the_widened_scope` - save and load carry `flap_*`, `spike_*` and - `buffer`, through the same `settings_to_preset` call `App.save_profile` makes. - -- **BREAKING:** the effective-loss figure is redefined at both ends (audit F4). `gui/pages/stats.py` - and `repro.py` computed `100 * drop_loss / seen`. The numerator was the configured Loss alone, - ignoring `drop_rate`, `drop_flap`, `drop_block`, `drop_syn`, `drop_mtu`, `drop_nat`, `drop_rst` - and `drop_lan`; the denominator was every captured packet, including traffic outside the - targeting scope. Measured on this branch, driven through `BeanEngine` with `FakeDivert`: - a 50 KB/s cap with **zero** configured loss went from **0.0% to 99.9%** (`drop_rate=999` of - 1000 packets), and 50% loss with a third of the traffic in scope went from **17.7% to 53.0%** - (`seen=900`, `scoped_seen=300`, `drop_loss=159`). Both reproduce the audit's shape (0.0% against - a real 90%, and 16.7% against the target application's 50.1%). `metrics.effective_loss_pct` and - `metrics.effective_corruption_pct` therefore change meaning; reports across this line are not - comparable. Additive alongside them: `metrics.packets_in_scope`, the `scoped_seen` counter in - `st` (so it also appears in NDJSON `summary.counters`), and the CSV column `packets_in_scope` - (`App.CSV_COLUMNS`). `_sample_record` in `cli.py` was deliberately left alone - it is a curated - subset, and the summary already carries the full counter dict. -- **New single source in `engine.py`.** `DROP_BY_REASON` (the reason -> counter map, lifted out of - the capture loop), `IMPAIRMENT_DROP_KEYS` **derived from it**, `TOOL_DROP_KEYS`, and two pure - functions `impairment_loss_pct(stats)` / `corruption_pct(stats)` used by both the GUI and the - repro report. They take a stats dict, not an engine, so the GUI computes from the snapshot it - already holds and the tests need no session. `repro.py` and `gui/pages/stats.py` import them; - no layering rule is touched (`engine` imports neither). -- **Why the tool's own drops are excluded**, recorded here because it is a judgement call: - `README.md` defines the term around a congested link ("that is how a congested link behaves"), - and `tips.stat_shutdown` says of queued-at-stop packets "They were not lost in the network". - There is also a hard reason: the delay queue holds out-of-scope packets too, so counting - `drop_overflow` against a `scoped_seen` denominator could produce a figure above 100%. As - defined, every counted drop happened to a packet that is in the denominator, so the result - cannot exceed 100%. -- **Hot path measured, not assumed** (Win11 AMD64, CPython 3.14.6, 150k 1500 B packets through - `FakeDivert`, median of 5, benchmarked back to back against a `git worktree` of master). - Nothing-impaired path 149.7k -> **157.1k pkt/s** (ranges 145.7-155.4 vs 150.6-163.3: overlapping, - so read as no regression while gaining a third counter). 100%-loss path 234.5k -> - **267.8k pkt/s** (188.2-238.2 vs 234.1-274.2: the new minimum sits at the old maximum), which is - the dict literal no longer being built for every dropped packet. -- **Two deliberate rearrangements in the capture loop.** (1) `seen`, `bytes_*_total` and - `scoped_seen` now share ONE `_slock` acquisition, placed after `decide()` because scope is not - knowable before it - fewer acquisitions per packet than before the counter was added. If - `decide()` ever raised, the packet would go uncounted where it used to be counted; it does not - raise (`Matcher.matches()` is documented never to) and if it did, the capture thread dies and - the session fail-stops. (2) `DROP_BY_REASON` at module scope instead of a literal per drop. -- New tests in `tests/test_engine.py`, all verified by mutation (four mutants, each caught with - the pre-fix number): `test_effective_loss_counts_every_impairment_not_just_the_loss_setting` - (reverting the numerator gives `pct=0.0` against `drop_rate=999`), - `test_effective_loss_measures_the_traffic_that_was_targeted` (reverting the denominator gives - `pct=17.7`; never bumping `scoped_seen` gives `scoped_seen=0`), and - `test_every_drop_counter_and_drop_reason_is_classified` - a mechanical guard that every - `drop_*` counter in `reset_stats()` is in `IMPAIRMENT_DROP_KEYS` or `TOOL_DROP_KEYS`, and that - every reason `Decision(True, ...)` can carry in `core.py` routes through `DROP_BY_REASON`. That - last one is the guard against F4's actual failure mode: a counter that exists, is correct, and - is simply not part of the sum. -- i18n: new key `tips.eff_loss` in `lang/en.json` + `lang/pl.json` (the session row had **no** - tooltip at all, which is part of why the figure could lie unnoticed); `SESSION_ROWS` in - `gui/pages/stats.py` now names it. README EN + PL gained a bullet defining the figure; the - existing prose about a speed limit causing loss above the set percentage was left untouched - - it was already true, and now the number finally agrees with it. - -- **BREAKING:** `--gui` combined with any other option now exits `USAGE(2)`. `args.gui` was - parsed and then **never read anywhere** - `cli.py::main` routes to the GUI only when argv is - empty or exactly `["--gui"]`, so every other combination fell through to a full CLI session - while `--help` advertised "force the GUI". On real WinDivert that meant - `--gui --loss 30 --duration 600` impaired the machine's network with no window and no STOP - control. The guard sits at the top of `run_cli`'s `try` block (before `--license` / - `--doctor` / `--cleanup-driver`) and uses the existing `CliError` path, so the message goes - to stderr and stdout stays clean. -- Blast radius checked before the change: nothing in `tests/`, `smoke_gui.py`, `tools/` or the - launcher facade passes `--gui`; `test_cli_fuzz.py` builds `FLAGS` from `FIELD_DEFS`, so the - fuzzer never generates it; `test_cli_docs.py` compares flag NAMES, not help text. `USAGE` was - already in the fuzzer's `ACCEPTABLE` set, so the new outcome fits the CLI contract rather - than widening it. -- Rejected alternatives: opening the GUI and silently dropping the other flags (asking for 30% - loss and getting zero without being told is the class of quiet lie this project removes), and - opening the GUI with the form PREFILLED from the flags. The second is genuinely nicer and is - still open - it needs `gui/app.py`, which is due for decomposition, so it is deferred rather - than declined. -- New test: `tests/test_cli_runtime.py::test_gui_flag_combined_with_settings_is_a_usage_error` - - asserts `USAGE(2)`, the reason on stderr, and an empty stdout (the data-channel invariant). -- Help text and the flag tables in both READMEs now state that the flag is valid on its own. -- Version bump deliberately NOT taken (convention 34): the owner closes it in `VERSION.txt`. - -### Changed - -- **`internal_tools/` added to `.gitignore`** - a home for scripts kept between sessions - (measurement rigs, probes against the real driver, expensive diagnostics). Not shipped, never a - CI dependency, backed up by the owner separately; index and entry rule in - `internal_tools/README.md`. Only the `.gitignore` line is in this repo. - The alternative was the scratchpad, and the scratchpad does not survive a session: `PROJECT_NOTES` - rule 5 has been pointing at "`bench_ab.py` from that session's scratchpad" - a file that no longer - existed - so the same A/B rig kept being rebuilt without its interleaving or its drift canary. - **`tools/` is deliberately NOT covered by this and stays tracked**: `ci.yml` runs - `tools/ci_gui_render.py` on the Linux runner, so ignoring it would break the GUI render check on - a fresh clone. A script that becomes a CI dependency moves to `tools/`. - -- **Coverage gate raised 80 -> 83** (convention 32). Measured with `COVERAGE_PROCESS_START` on - 2026-08-01: **87.43%**, up from 83.03% on 2026-07-21. The comment above `fail_under` now also - records what the number is NOT: it reports lines as missing that mutation proves are guarded - (coverage cannot see inside `threading.excepthook`) and as covered where nothing checks them (a - short-circuited `and`, a line executed only while the object is built). It gauges the trend for - the package; a behaviour is proved guarded by mutation. -- **`tooltip._grab_active` docstring corrected.** It said `grab_current` was the fallback "for - environments without `.tk` (the test double)". The double answers `call` like the real - interpreter now, so tests take the branch production takes; the fallback is for a widget with no - `tk` handle at all. - -- **What the on-screen numbers COVER is now derived in one place: `gui/scope.py`.** The notes on - Statistics and Connections answer "what do these figures cover?", but both read a single input - - the `scope_view_to_target` preference (`stats.py` line 149, `conns.py` line 122). With - `narrow_filter` on and effective, the driver's filter has the destination folded into it, so both - notes stated the opposite of the truth ("ALL captured traffic ... targeting decides what gets - impaired, not what gets listed") while `tips.narrow_filter`, one window away, promised the two - tabs "then show only that traffic". Both READMEs carry the same contradiction (the - `--narrow-filter` row against the behaviour list). Nothing could go red over it: prose has no - guard. - This commit adds the fact and the single decider, and wires up no surface yet: - `BeanEngine.capture_narrowed()` (the start-only verdict as a plain bool, beside the - `session_info()` dict the repro report and the NDJSON summary already carry - deliberately NOT - cleared by `stop()`, because the counters it describes stay on screen); - `BeanCore.process_target_active()` / `BeanEngine.process_target_active()` (the process half of - `targeting_active()` alone - the half `processId` can never push into a NETWORK-layer filter, so - it still separates CAPTURED from IMPAIRED when the capture has been narrowed); pure, Tk-free - `scope.coverage()` returning one of `ALL` / `CAPTURE` / `CAPTURE_PROCESS` / `VIEW`; and - `App.coverage()`, which is to every sentence about the figures what `App.scoped_stat` already is - to the figures themselves. - `VIEW` outranks `CAPTURE` on purpose: with the view scoped every counter with a twin IS its - scoped twin, so the figures cover what targeting selected however wide the capture behind them - was. `CAPTURE_PROCESS` is a state rather than a hedge in the wording because with no process - target everything captured is also impaired, and "your targeting still decides what gets impaired - inside it" would then be true, useless, and would imply a narrowing that is not happening. - Five new guards in `test_view_scope.py` (all eight input combinations, the precedence, `None` - reading as off, the two readers of the narrowing fact agreeing, the process half on its own). - Verified by mutation: `stop()` clearing the flag, the precedence flipped, the process half - widened to `targeting_active()`, and the reader hardcoded to `False` are each caught. - -- **Every surface that states what the figures cover now renders the verdict.** `stats.py` and - `conns.py` each hold a `state -> i18n key` table built through `scope.keys_for_states()`, which - refuses a table that is missing a state (or carries an obsolete one) when the page module is - imported - so a fifth state breaks the build rather than one screen falling back to "everything". - Wired up: both scope notes, their tooltips, the chart caption (`THROUGHPUT_TITLES`) and a new - `session.capture` row in `SESSION_ROWS`. - Ten new keys in both language files (`stats.scope_note_capture`, - `stats.scope_note_capture_process`, the two `conns.` twins, `frames.throughput_capture`, - `session.capture` / `_all` / `_narrowed`, `tips.scope_note_capture`, `tips.session_capture`). - 🔴 **The session row reads `session_info()["narrowed"]`, NOT the coverage state** - it is about - what the driver handed over, which a view preference cannot change. That distinction survived - only because a mutant exposed it: deriving the row from `state != ALL` passed the whole file, and - the case that separates them (view scoped, capture wide) was missing from the test. It is there - now. - New in `tooltip.py`: `retip(widget, key)`. `add_tooltip` binds `Tooltip.text` once, so every - re-worded label kept a bubble explaining the state it had just left - the scope notes had been - doing this since the view preference shipped. `Tooltip.text` is read when the bubble is shown, so - swapping the attribute is the whole update. - Six new guards in `test_view_scope.py` (both notes across all four states, note/bubble agreement, - the chart caption, the session row, table completeness plus every key resolving in EN **and** PL, - and `keys_for_states` refusing an incomplete or stale table). Eight mutants, all caught: each note - reverted to reading the preference, the `retip` call dropped, the caption frozen, the session row - pinned wide, the session row rederived from the coverage state, `keys_for_states` stripped of its - check, and a misspelled note key. - The `conns.py` module docstring claimed "these are ALL captured connections ... targeting decides - what gets broken, not what gets seen" as unconditional fact; corrected. - -- **One "Scope" card in the Settings window, holding both switches and a live verdict.** The - `capture` section is renamed `scope` (label `frames.scope`, `frames.capture` retired) and gains - `extra="scope"`; `SettingsWindow._build_scope_extra` fills the rest of that card. The two - switches cannot share a registry - `narrow_filter` is a `Field` with a CLI flag that travels in a - config file, `scope_view_to_target` is a ui.json `Pref` (convention 42) - so the CARD is the only - place they can meet, and the `extra` hook is the sanctioned way to put a page's own widgets - inside a registry section. - `Pref` gains `section: str = ""`, naming the registry section that renders it instead of a - preference group, plus `prefs.prefs_in_section()`. Declared on the `Pref` rather than in a second - list, so "where is this rendered" keeps one answer. `PREF_GROUPS` no longer lists it. - Two guards had to change, and both were registry PROXIES for "is this reachable in the window": - `test_every_pref_is_grouped_exactly_once` becomes - `test_every_pref_is_rendered_in_exactly_one_place` (group XOR section, never both, never - neither), and the old `any(... in PREF_GROUPS)` line in `test_view_scope.py` is replaced by - `test_both_scope_switches_are_rendered_in_one_card`, which opens the real window and walks the - widget parents. New `test_a_pref_can_only_name_a_section_that_will_actually_render_it` - a typo - in `Pref.section` fails silently, exactly like a stray id in `FIRST_RUN_COLLAPSED`. - **The verdict line answers before START.** `_narrowing_verdict()` asks - `filters.narrowed_filter` - the same call, and therefore the same driver-parser answer, - `start()` will make - over matchers built from the current destination fields; while a session - runs it returns `engine.capture_narrowed()` instead, because the handle's filter is already - fixed and the fields on screen may describe a session that does not exist. Memoised on its - inputs, so the driver's parser is asked when something moves, not every tick. - Guards: the card answers before a session, a running session outranks the preview, and the line - is right at BUILD time rather than one tick later. `narrowed_filter` is PATCHED in those tests - - it reaches `WinDivertHelperCompileFilter`, which is absent on the Linux half of the CI matrix, - where "cannot prove it" is the honest answer for every destination. Seven mutants, all caught; - the build-time one survived the first pass (every test called `refresh()` first) and produced - the third guard. - -- **The prose that contradicted itself is corrected, in every place it lived.** Both READMEs said - "Statistics and Connections show ALL captured traffic ... targeting decides only what gets broken, - not what is visible" a few hundred lines away from the `--narrow-filter` row saying "statistics - and connections cover the narrowed traffic only". Both behaviour sections now describe the two - scope switches as the different things they are, name where the answer is shown (the Scope card - before START, the log at start, the Session row for the run), and cover the - capture-narrowed-plus-process-target case. - `tips.scope_note_scoped` was the last UI text still false in a reachable state: shown for `VIEW`, - which can sit on top of a narrowed capture, it claimed "the traffic filter still decides what is - captured" and offered "turn it off to see everything again" - you would see everything CAPTURED, - which is already narrow. Reworded to be true either way rather than adding a fifth state for a - tooltip. `tips.narrow_filter` and `tips.scope_note` were CHECKED and left alone: the first - already promised what the code now delivers, and the second is only ever shown for `ALL`, where - every word of it is true. - -- **Two layout defects the Scope card exposed, both structural rather than cosmetic.** Reported from - a screenshot; neither was visible to any existing test. - 1. **`ControlForm` built section bodies as fields -> note -> error -> extra.** The note is packed - EMPTY and kept mapped on purpose (a `start_only` or `overridden_by` section must not jump when - the reason text appears mid-session), so it reserves a blank line wherever it sits - and it sat - between the registry field and everything the extra added. In the Scope card that wedged a - blank line between the two checkboxes that exist to be read as a pair. The extra now runs - BEFORE the note: content first, commentary after. **Only `scope` changes today** - it is the - only extra-bearing section with a `start_only` or overridden field (checked against the - registry, not assumed), and the error label was never affected because it is created here but - PACKED later, and `pack` appends, so it already landed below the extra everywhere. - 2. **`SettingsWindow` had no scroller.** The footer is packed first so the CONTENT is what runs - out of room, which meant one card too many pushed the last preference group off the bottom - edge - "Behaviour" rendered as a bare header, with no scrollbar and no hint anything was - missing. Body is now a `ScrollableFrame` (the Control page's proven combination: `ControlForm` - + `CollapsibleSection` + a combobox inside a scroller), with Close deliberately OUTSIDE it. - `SIZE` 520x520 -> 560x620, and the comment says why that is not the fix: `_restore_geometry` - prefers a saved geometry, so anyone who has opened this window keeps the old size. The height - of this window can only ever be right by accident - it grows with every preference, in two - languages, at every DPI. - `scope.narrow_has_no_effect` shortened in both languages (EN 230 -> 194, PL 214 -> 176 chars): - three red sentences under a checkbox is the wall convention 1b exists to prevent, and the third - listed the forms that do NOT work, which `tips.narrow_filter` already spells out. The positive - half carries the same information and is the actionable one. - New guards: `test_nothing_is_wedged_between_the_two_scope_checkboxes` (walks the card's children - and asserts the pref checkbox is the NEXT one after the field checkbox, and that the reserved note - is still present, below both) and `test_the_settings_window_can_reach_every_group_at_any_height` - (every section and the last pref group's rows descend from the scroller, Close does not). Four - mutants, all caught: the extra moved back after the note, the scroller removed, the groups built - outside it, and Close moved inside it. - -- **`App._log_capture_scope()`: the start-time narrowing verdict reaches the GUI log, both ways.** - `cli._run_session` has reported this since the option shipped (`log.info` on success, - `log.warn` on "asked for and did not get it"); the GUI passed `narrow=` to `engine.start()` and - never read the answer back - a grep for `narrow` across `beantester/gui/` returned exactly one - line, the call itself. Called from `_finish_start`, so it runs on the UI thread after a start - that actually succeeded, and only when the checkbox was ticked. Not a dialog: the session started - fine, and a modal would interrupt the run that was just asked for. - New keys `log.narrow_applied` / `log.narrow_no_effect` in both languages. - Guard: `test_view_scope.py::test_the_window_says_whether_the_narrowing_actually_happened` drives - all three cases (not asked, asked and got it, asked and did not). Four mutants, all caught: the - call removed, only the success branch kept, the request reported instead of the outcome, and the - "was it even asked for" gate defeated. - -- **`tips.narrow_filter` rewritten in both languages, and the cause of its shape removed.** - `narrow_filter` declared `hint="fields.narrow_filter_hint"` - a 300-character explanation in two - languages that **was never rendered**: `gui/form.py::_place_one` handles `BOOL` and returns at - line 190, and the hint is drawn at line 249. The detail therefore had nowhere to live, so the - tooltip had swollen into a wall carrying all of it at once. The `hint` and both i18n keys are - gone, and the tooltip now leads with what the option does, names the on-screen places it affects - (the Statistics and Connections tabs), and states the limitation: **wildcards, `re:` patterns and - process-only targeting cannot be narrowed, so the option does nothing.** That last part was - verified by running `windivert_fragment` over each form (`10.0.0.*` and `re:^10\.` produce no - fragment; a plain address, a list, a range and a CIDR all do) rather than repeated from a code - comment. - New guard `test_field_registry.py::test_only_fields_that_can_show_a_hint_declare_one`: `BOOL` and - `CHOICE` return before the hint is drawn, so declaring one on either kind is text nobody can ever - read, and **nothing raises when it happens**. Verified by mutation - putting a hint back on the - checkbox fails with `[('narrow_filter', 'bool')]`. - Writing rules for user-facing text are now in `PROJECT_NOTES` (convention 1b), including the - measured reason NOT to build a "no code tokens in tooltips" guard: of its 5 hits, 3 are - legitimate (`re:` is syntax the user types, the CSV names are real filenames). - `fields.narrow_filter` renamed from "Narrow the driver filter to the target" to **"Capture only - the targeted traffic"** in both languages, by the same rule: the old label named the mechanism. - The `--narrow-filter` flag, the settings key and the config format are untouched. - -- **Semicolons swept out of every user-facing text, and the rule is now enforced.** 34 replacements - across `lang/en.json` and `lang/pl.json` (21 tooltips plus `dialogs.*` and `errors.*`), and 81 - lines of prose across both READMEs. Each case was decided individually rather than regexed: a - full stop where the semicolon joined two independent clauses (which is what a semicolon is for), - a comma only inside a parenthetical aside. **Every rewritten line is the same length as the - original**, which is the invariant that proves nothing was mangled - and it caught the one line - that was: a Polish bullet with two semicolons got its tail duplicated by the sweep script, fixed - by hand. - Two new guards, both mutation-verified: `test_i18n.py::test_no_semicolons_in_ui_text` (fails with - `['tips.jitter']` when one is put back) and - `test_readme_guards.py::test_no_semicolons_in_readme_prose` (fenced blocks, indented blocks and - inline code spans are cut out first, because a semicolon there is syntax). Unlike the readability - lint rejected above, this one has **no false positives**, which is precisely why it is worth - having. - -- **`block` added to `FIRST_RUN_COLLAPSED`** (`gui/app.py`), and the list reordered to match form - order. It only applies when `ui.json` has no saved `collapsed` state, so existing users see no - change. - New guard `test_prefs.py::test_the_first_run_collapse_list_names_real_sections`: every id must be - a real **Control-page** section, with no duplicates, and the panels a first-time user needs - (`profiles`, `traffic`, `latency`, `impairments`) must not be in it. The list is consumed by - `form.py` as `sec.id not in app.collapsed_sections`, so **nothing ever looks an id up** - a typo - raises nothing, it just leaves that panel expanded, which is invisible. Verified by mutation: - `"blocking"` instead of `"block"` fails with `assert not ['blocking']`. - -- **`spike_prob` and `spike_ms` moved from the `advanced` section to `latency`** (`fields.py`): the - two `Field` entries were relocated inside `FIELD_DEFS` so the registry reads in form order, and - the two `Section` tuples were updated. **One entry per field really was enough** - grepped every - reader first, and `core.py`, `settings.py`, `summary.py`, `repro.py`, `presets.py` and the GUI's - variable wiring all go by the field KEY, so none of them knew which section the fields lived in. - `test_prefs.py` and `test_windows.py` use `"advanced"` as a collapsed-section id; the section - still exists with its remaining five fields, so they are unaffected. - Moving the entries changes the KEY ORDER of `PRESET_TO_SETTING` and `PRESET_DEFAULTS` (both are - derived from `FIELD_DEFS` order) and therefore the key order written into `profiles.json`. Checked - rather than assumed: every test over those compares sets or whole dicts, never order, and JSON - object order carries no meaning here. - Docs: the option walkthrough in both READMEs is organised by GUI section, so the *Latency spike* - bullet moved out of "Advanced (NAT / connections)" and into the delay text. **Nothing guards - that** - the bullet structure is prose, not a registry view - so it is the one part of this change - a test could not have caught. - -### Fixed - -- **`--dry-run`'s prose narrowed to what it actually checks.** The comment above it called it "the - one thing a CI/CD pipeline runs to find out whether the next command will work", and the success - line said "Configuration is valid" full stop. Measured with `is_admin()` patched false: exit `0` - from the preflight, exit `7` from the very next real run. **The behaviour was deliberately left - alone** - `_run_session` keeps the elevation and pydivert gates, because validating a config on a - build agent and running it elsewhere is a legitimate pattern that widening the check would break. - Only the sentence changed, plus the `--doctor` + `--dry-run` pair in both READMEs. - Guard: `test_cli_runtime.py::test_dry_run_says_what_it_did_not_check`. -- **`run_cli` no longer lets an unforeseen exception escape as a traceback.** Convention 18 and the - `CliError` docstring both promise "never a raw traceback", and that held for the parse/validate - surface only: `test_cli_fuzz.py` runs every one of its cases under `--dry-run` ("pure contract - surface"), so the SESSION path - engine, threads, driver, report loop - had no guard at all. - Measured: a `RuntimeError` raised in `_report_loop` escaped `run_cli` with no `[bean] error:` - line, no `summary` record, and CPython's exit `1` (colliding with `RUNTIME`). - Two layers, deliberately, because they can do different amounts for the caller: - 1. `except Exception` inside `_run_session` around the report loop - the engine is still alive - and the counters readable, so the run sets `RUNTIME`/`"fault"` and **still emits a complete - `summary`**, instead of leaving a truncated NDJSON file; - 2. `except Exception` in `run_cli` as the last resort, for everything outside the session - (config load, building the result, a bug in this module) where nothing can be summarised. - Both go through `crashlog.record` (convention 30 - the only sanctioned swallow). `CliError` is a - `SystemExit`, so it passes both handlers untouched to its own, as before. The outer message names - its PHASE ("while finishing the run") because a fault in something the summary also needs trips - both handlers, and two identical lines read as two separate bugs. - New tests: `tests/test_cli_runtime.py:: - test_an_unexpected_session_fault_is_a_coded_exit_with_a_full_summary` (asserts the complete - summary and `stop_reason == "fault"`) and `::test_a_fault_outside_the_session_is_still_a_coded_ - exit` (asserts the two messages are distinguishable). -- **The shared-port warning named the target's own program among the "other processes".** Reported - from a real session targeting `chrome`. Not a regression from the handle-read change - verified - field for field on both checkouts (matched pids, target ports, shared ports, co-owners and the - full name map for all 37 socket pids were **identical**), and the warning itself predates it - (`a1336ad`, on master). - - **Cause.** `ports_shared_with_others` subtracts by PID (`targeting.py:379`), while - `ProcessTargeting.refresh` only ever enumerates pids that WON a port in the collapsed - `port -> pid` map (`targeting.py:117`). A second process of the same program that lost every - collapse is therefore invisible to targeting, survives the subtraction, and prints under the - target's own name. Reproduced against the real table: targeting `msedge` matched pid 55120 - while pid 47664 - also `msedge.exe`, `in_collapsed_map=False` - came back as a stranger. - - Co-owners whose name matches one the target resolved are now marked inline - (`log.shared_port_same_app`) rather than listed as strangers. Marked, not hidden: they really - do hold the port, and they really are not in the target set. - - **The message also offered both outcomes when the tool already knew which applied.** The - collapse WINNER decides it: in the target set means the port is in scope and everything on it - is impaired (`log.shared_port_hits`), otherwise the target's own traffic there is skipped - (`log.shared_port_misses`, which names the holder). Measured on the same session: 5353 was - **not in the target's port set at all** (14 ports, none of them 5353), so only the second half - was ever true there. - - One line per port instead of one line for all of them, since the verdict is per port, plus a - single trailing `log.shared_port_footer`. The winner is read from ONE `snapshot()`, and a port - missing from it is skipped rather than described from a different walk. - - **`settings.describe_port`** turns `5353` into `5353 (mDNS)`. Names come from the machine's own - services file via `socket.getservbyport` (upper-cased - a services file is lower-case by - convention), so they cannot drift from what the OS believes. `_PORT_LABELS` overlays only what - that file leaves unhelpful, checked here: 5353 is absent from it on Windows (IANA / RFC 6762) - and DHCP is registered as `bootps`/`bootpc`. - - i18n: `log.targeting_shared_ports` replaced by `log.shared_port_hits`, `log.shared_port_misses`, - `log.shared_port_same_app`, `log.shared_port_footer` in `lang/en.json` + `lang/pl.json`. - - **Fixture repair that mattered.** The existing fakes had no `snapshot()` and no `names()`, and - `_warn_about_shared_ports` runs inside `crashlog.quiet` - so the new code would have raised, - been swallowed, produced zero lines, and left a test asserting `len(...) == 1` to fail for the - wrong reason. Both fakes now carry the full read surface. - -- **The tool was stalling its own capture thread and then blaming WinDivert for it.** Reported as a - spurious warning on the first START ("WinDivert held a packet for 105 ms ... the driver's queue is - backing up ... narrow the traffic filter"), which was true about the wait and wrong about the - cause: nothing outside the tool was congested, and narrowing the filter could not have helped. - - **Cause.** `PortTable.warm_names()` runs on the WATCHDOG thread and resolves every socket-owning - PID. The resolve went through `psutil.Process(pid)`, whose `ppid()` on Windows is `ppid_map()` - - a snapshot of EVERY process, taken afresh per call, because the C extension exports only - `ppid_map` and no `proc_ppid` (checked, psutil 7.2.2). So a pass took **one full system scan per - PID**: measured 9.36 ms/PID, **308.9 ms for 33 PIDs**, against 9.6 ms for a single snapshot - covering all 453 processes. The name was never the expensive part - `name()` measured - 0.02-0.06 ms, and the docstring blaming NAME resolution for the cold resolve was wrong. - - **Not a startup wart.** `_expire_info` drops an entry `INFO_TTL_S` after it was WRITTEN and a - cache hit deliberately does not renew that stamp, so cold passes RECUR. Measured over 95 s: - spikes at t=0.3, 30.5, 60.9, 91.1 s, peaks 457/317/206/202 ms. With process churn (new PIDs - arriving continuously) it is worse still - 13 of 19 windows over the warn line, **peak 508 ms, - 7 warnings in 95 s**. - - **Fix: `portmap._process_info` - one handle instead of one snapshot per PID.** New - `_native_process_info` reads name, parent and creation stamp through a single - `OpenProcess` + `NtQueryInformationProcess` + `GetProcessTimes` + `QueryFullProcessImageNameW`, - with `_psutil_process_info` kept behind it as the fallback. **0.03 ms/PID, 1.0 ms for the set.** - Measured end to end on the shipped code, churn on, 95 s: **peak 508 -> 16.6 ms, 0 of 20 windows - over the warn line, 7 warnings -> 0**, process names still 36/36. `warm_names()` cold: - 344-458 -> 24 ms; warm 0.22 ms. - - **Why the handle and not a cached `{pid: ppid}` table**, which measured 0.31 ms/PID and would - also have fixed the symptom: the handle PINS the identity, so name, parent and creation stamp - describe the one process the kernel gave us. A cached table has a staleness window in which a - recycled PID lets one process's name meet another's parent - the mixture `_looks_recycled` - exists to prevent. Speed was not the deciding argument. - - **Verified against churn** (2149 lookups, 489 PIDs, continuous spawn/exit): 0 ppid - disagreements with psutil, 0 with the toolhelp snapshot (461/461 identical). **NOT verified:** - PID recycling could not be forced (0 observed), so that property rests on the mechanism, not a - measurement; 32-bit/WOW64 and ARM64 are untested. Every failure path returns `None` and falls - back, so if a future Windows withdraws `NtQueryInformationProcess` this degrades to today's - behaviour and today's cost, not to a broken tool. - - **Stability, since this opens a kernel handle per PID and runs forever.** 20 000 calls over a - mix of live, dead and nonsense PIDs: **handles 185 -> 185, RSS flat**, 35.6 us each. The two - failure routes leak nothing either (5 000 calls each with the binding dead, and with the query - failing *after* `OpenProcess` succeeded: handle count unchanged) - the `finally` gives the - handle back on every path. A 5-minute session with traffic and continuous process churn: - **handles 231 -> 232** (range 225-234), RSS +1.4 MB, worst wait in any 20 s window **19.8 ms, - 0 of 15 windows over the warn line, 0 warnings**, names resolved 100% throughout. - Degenerate input returns `None` rather than raising: `None`, 0, negative, non-numeric text, - values past a DWORD, and dead PIDs. - - **A concurrency fault that a lock did NOT fix, found by measuring twice.** The lazy bind - published an "in progress" marker in the shared slot before loading the DLLs. The fast path - reads that slot unlocked and `False is not None`, so every other thread read it as "unavailable" - and returned without ever queueing - **adding a lock changed nothing, 200 of 1600 calls, 5 - trials of 5**, and only a state trace showed why. Nothing is published now until the binding has - an answer: **1600 of 1600**. It matters because the two threads that hit this from cold are the - watchdog warming names and the resolver matching a target, i.e. session start - the exact moment - this change exists to make cheap. - - **One bug found by the suite, worth recording.** The first version gated - `_ALLOW_NATIVE_PROCESSES` while BINDING the ctypes entry points and cached the result, so the - flag was read once per process: `test_a_target_restarting_onto_a_recycled_pid_is_still_impaired` - asked its fake world for pid 5000 and got `wslhost.exe` off the real machine. The gate is now - read per call, like `_toolhelp_process_table` does it. Caching ABILITY is fine; caching POLICY - is not, and it fails in both directions. - -- **The Live stats tab is a `ScrollableFrame`** (`gui/pages/stats.py`). The counter grid reflows its - column count with the window WIDTH, so a narrow window turns it into five tall rows; it packs at - its natural height and the chart, packed `expand=True`, took the leftover - about ten pixels. The - canvas `height=scaled(180)` is a REQUEST, and pack shrinks below a request when it has nothing to - hand out, so it was never a floor. This is the same squeeze the module docstring records at page - level ("pack simply gave the last two panels no space at all"), reappearing one level down. - Convention 14 is satisfied - no Treeview/Text/Listbox on the tab, the chart canvas is a drawing - surface with no scrolling of its own. - 🔴 **Known trade, verified rather than assumed:** `ScrollableFrame._on_canvas_configure` only - stretches the inner window's WIDTH, so content is laid out at its requested height. The chart - therefore stops growing on a tall window and keeps its 180 px. Making the scroller stretch height - to the viewport when content is shorter would restore it, but that is shared with the Control page - and was not touched here. -- **Panel footers are packed FIRST** (`gui/panels/settings.py`, `gui/panels/about.py`). pack hands - out height in call order, so a footer packed last gets what the content left - which for Settings - was nothing, and "Close" was clipped by the window edge. About had the identical shape and only - fits today because its content is shorter. -- **`dialogs._center` takes the widget to focus** and sets it after `focus_force()`; `ask_string` - passes its entry. The old `entry.focus_set()` failed twice over: it ran while the dialog was still - `withdraw()`n (`_shell`), where focus does not stick, and `_center` then forced focus to the - window and discarded it. That is why "Save profile..." needed a click before typing. - Guards (`test_gui_layout.py`): `::test_the_live_stats_tab_is_scrolled_so_the_chart_cannot_vanish` - and `::test_a_panel_window_reserves_its_footer_before_its_content`, which walks - `body.pack_slaves()` and asserts the bottom-packed footer is at index 0. Both mutation-verified - - reverting them gives "the Live tab is not scrolled" and "the footer is packed at position 5 of 6". - -### Tests - -- **New `tests/test_gui_file_actions.py` - Save file / Load file / Save repro were the three - largest uncovered blocks in `gui/app.py`.** The modules behind them are well covered and both - write formats frozen by "Kontrakty publiczne"; what had no test is the App side - which values - the form hands over, what comes back into the widgets, and what happens when the write fails. - Eight tests: the config round-trips through the real on-disk format, a cancelled dialog (an empty - path, which is how the user backs out of every one of these) writes nothing and clears nothing, - an unwritable path and a broken or wrong-shaped JSON are reported through `dialogs.show_error` - with the form left intact, a repro without a session refuses before it even opens the dialog, and - a saved report carries the seed plus the CLI line to replay it - into the log, or the report is - half useless. -- **`fakes.wait_until` replaces the fixed waits that raced an assertion.** Ten sites in - `test_engine.py`, `test_release_fixes.py` and `test_scenario_runner.py` now poll to a deadline - instead of sleeping a guessed interval. The worst was `test_scenario_integration`: it read - `core.loss` 0.15 s into a 0.3 s window, so a stall longer than the gap made the "before" read - land after the scenario step and the test failed for a reason it was not about. It now reads the - first value immediately and polls for the second. The remaining fixed waits are the ones that - assert something does NOT happen (the session clock does not move after STOP; a looping scenario - is still alive past its duration) - absence cannot be polled for, so they stay, each with a - comment saying why. This buys reliability, not speed: the waits removed total ~1.3 s of a ~368 s - run. -- **The flow-table guards live in one place again.** Four tests moved from `test_audit_fixes.py` - into `test_core.py`. The table had guards in three files because two of them are named after the - EPISODE that produced them rather than the subject they cover, so finding "what protects the flow - table" needed knowledge of the project's history. Nothing referenced the four by name, so the - move was free. The wider scatter was MEASURED before touching anything and mostly is not scatter: - targeting appears in eight files because there are eight distinct mechanisms, and - `test_release_fixes.py` / `test_gui_release_fixes.py` are cited by name from six places in the - handover note plus the regression-surface table, so renaming them would cost more than the - discoverability it buys. -- **`crashlog.install()` claims every failure path; only one of the three was guarded.** - Mutation-checked 2026-08-01: gutting the main-thread hook and the Tk-callback hook left 106 - tests green, while the same treatment of the worker hook reddens - `test_crashlog.py::test_a_worker_thread_exception_is_recorded` at once. New: - `::test_an_unhandled_main_thread_exception_is_recorded` (also checks the previous - `sys.excepthook` still runs - we record, we do not swallow), - `::test_a_tk_callback_crash_is_recorded` and - `::test_attaching_the_tk_hook_to_a_hostile_root_is_not_a_crash`. Both new guards were confirmed - red against their mutants. -- **New `tests/test_license_surface.py` - `--license` had zero test coverage**, the one finding of - the 2026-08-01 review with legal rather than functional weight (convention 35). What was tested - is that `LICENSE` and `THIRD-PARTY-NOTICES.md` EXIST in the tree; the code reads them through - `resource_path()`, a different resolution, and `legal._read` answers an `OSError` with an empty - string. So a frozen build that stopped resolving them would print an empty licence and keep the - suite green. Six tests: the texts resolve and are non-empty, every component carries a version - and a source URL, the report names all of them plus the no-telemetry line, the flag writes to - stdout and exits OK without touching the driver, and `--format json` is one parsable record whose - `components[]` matches `legal.COMPONENTS` exactly (the NDJSON schema is a frozen contract). -- **New `tests/test_wheel_and_scroll.py` - the three user-visible fixes in `gui/scrollable.py` - had no guard at all** (44.5% line coverage). The one existing test replaced `_resolve` with a - lambda and the one touching `ensure_visible` replaced it with a spy: both good wiring tests, but - between them the behaviour never ran. Nine tests covering the master-chain walk (a nested control - scrolls the page it sits on), a self-scrolling Treeview keeping its own wheel only while it has - something to scroll, a bounded walk, the end-to-end `_on_wheel`, the disarmed combobox class - binding, and `ensure_visible` as maths in four cases including "taller than the viewport shows - its START". Three mutants planted in `scrollable.py` (broken walk, broken tall-widget branch, - removed combobox disarm) were each caught by the matching test. -- **New `tests/test_tooltip_bubble.py` - the module's whole reason for existing was unguarded** - (39.9% coverage; the entire show/hide lifecycle untouched). One reused bubble per toplevel is - what stops Windows flashing the taskbar button on every hover, and a rewrite to one `Toplevel` - per hover would have passed the suite. Seven tests: the bubble is reused across widgets, a - destroyed one is rebuilt, a Tk grab suppresses it (it would cover the dropdown it describes), - empty text builds nothing, the Enter/Leave state machine, `retip` re-wording, and the shortcut - line. -- **`SortableTree._clicked` - sorting by clicking a column header - was called by nothing.** Three - tests in `test_virtual_tables.py` go through the heading command the way a click does: the same - column flips direction, a different column starts from its own `default_reverse` instead of - inheriting the previous one, every click reaches `on_sort`, sorting returns the viewport to the - top, and `copy_text(header=True)` adds exactly one line while leaving the rows unchanged. -- **Pure `winenv` helpers in `test_failsafe.py`.** `_quote` builds the parameter string for the - elevated re-launch (a mis-quote means the elevated copy starts with the wrong settings, or with - extra ones) and `elevation_disabled` reads `BEAN_NO_ELEVATE`, which the whole automated - GUI/screenshot workflow depends on. Both pure, both previously at zero coverage. The quoting - assertions build their paths with `chr(92)` so the test's own escaping cannot be what they - measure. -- **Both branches of `gui/icon.py::_running_variant`.** The primary branch (copy the idle artwork, - stamp the dot) only became reachable once the tkinter double grew a working `tk.call`; before - that every GUI test silently exercised the fallback instead. Neither had a guard, so the swap - went unnoticed - two tests now pin both. - -- **A green GUI test can no longer hide a swallowed fault.** `gui_harness.run_gui` now ends every - subprocess by reading `crashlog.recent()` back and failing the test on anything not declared via - the new `run_gui(allow_faults=(...))`. `crashlog.quiet`/`note` exist so a failure stops being - invisible to US while staying invisible to the user (convention 30) - but nothing ever read them - back, so the swallowing was invisible to us too. The check lives in the subprocess rather than in - a test that scans `crashes/`, so the failure carries the name of the test that caused it and it - keeps working now that the crash log is redirected per subprocess. Verified in both directions: - a planted `crashlog.quiet` fault fails the test, and the same fault named in `allow_faults` - passes. -- **The suite no longer writes the developer's own files into the working tree.** New - `tests/user_files.py::redirect_to_temp` sends UI state, profiles and the crash log to a temp - directory; `gui_harness` and `smoke_gui.py` both call it, and a session-scoped fixture in - `conftest.py` does the same for the in-process tests. The redirect existed only inside the - harness, so `smoke_gui.py` - run by `test_gui_smoke.py` and by CI - overwrote the real - `bean_network_tester_ui.json` (window geometry, language, sort order, preferences) and left a - `crashes/` folder next to the sources on every `pytest tests`. Both are git-ignored, so nothing - ever showed it. A full run now leaves the tree untouched. -- **Three more fidelity gaps in `fake_tk`, all the same class as `winfo_exists`** (production - reaching THROUGH an attribute of the double), all found by writing the tests that needed them: - - `W.__getattr__` now refuses `_bnt_*` names instead of fabricating a function for them. Those - are the app's own marker attributes, and `getattr(w, "_bnt_scroll_owner", None)` asks whether - a widget was STAMPED - a no-op function is not `None`, so `WheelDispatcher._resolve` treated - every widget as an owning scroll container and resolved the wheel to a function. - - `W.yview`/`yview_scroll`/`yview_moveto` are explicit and read a settable `_yview`; - `Treeview.yview` no longer hard-codes `(0.0, 1.0)`. A double that always answers "nothing to - scroll" cannot express the other variant, so the dispatcher's "does this widget keep its own - wheel" branch was unreachable and `_can_scroll` was constant. - - `W.bind_class` records into `CLASS_BINDINGS` instead of vanishing, which is the only way to - check that `_disarm_combobox_wheel` is still in place. -- **`fakes.check` sets `__tracebackhide__`**, so a failure points at the calling test instead of at - the `assert` inside the helper - the one frame that is never interesting. Its docstring now also - says when the third argument matters: 750 of 1564 call sites omit it, and pytest can only rewrite - an `assert` it can see, so `check("ports match", a == b)` reports the label and nothing else. - -- **Three test doubles had drifted from the interfaces they stand in for, and the suite was - recording it 809 times per run without anyone reading it.** `crashlog` writes every swallowed - exception to `crashes/crashes.ndjson` (convention 30), so the drift was fully visible - just - never looked at. Measured 2026-08-01 on a full green run; after this change the same run leaves - **6** entries, all of them faults the tests inject on purpose. - - `fake_tk.W.tk` is now an explicit shared `Interp` with a recording `call()`. It was reaching - `W.__getattr__`, which answers any unknown attribute with a function, and a function has no - `.call` - so `gui/scaling.py`'s `root.tk.call("tk", "scaling", dpi/72.0)` raised into - `crashlog` on every App build (x526) and **Tk font scaling never ran in a single GUI test**. - `Interp.call` answers `grab current` from the existing `GRAB` slot, which also puts - `tooltip._grab_active` back on its PRIMARY path instead of the `grab_current()` fallback. - - `fake_tk.Text` is a real class instead of an alias for bare `W`, modelling the line - accounting behind `insert`/`delete`/`index`/`get`. `App._append_log_line` reads - `index("end-1c").split(".")` to bound the WIDGET as well as the in-memory list; against the - old double `index()` returned `None` and the `.split` raised (x264), so the widget-side trim - never executed. Verified after the fix: 900 appends with `log_lines=500` leave 502 lines in - the widget. - - `warm_names()` added to the three port-table doubles (`test_target_resolver._CountingTable` - and both `FakeTable`s in `test_gui_state`), plus the `cheap=` keyword `PortTable.name_of` - actually takes. The watchdog calls `warm_names()` every tick; without it those tests ran - against a silently degraded engine - the exact mechanism the `warm_names` ADR is about. - This is the same class of bug the note already records for `winfo_exists`: production reaching - THROUGH an attribute of the double. The generalised rule (any attribute whose RESULT is called - must be explicit in the fake) is now in the note beside it. - -- **Four new guards in `test_processes.py` for the per-PID handle read**, filling a gap that had - been open for as long as `ppid` has mattered: it feeds `PortTable.ancestors`, which is how - `targeting.py` matches a process TREE, and every existing targeting test drives a FAKE table with - its own `ancestors` - so all of them stay green no matter what the real resolver returns. - - `::test_the_handle_read_agrees_with_an_independent_oracle_on_the_parent` - checks the parent - against `os.getppid()`, which shares no code with psutil or toolhelp, and cross-checks name and - parent against the psutil path it replaced. - - `::test_a_process_that_will_not_name_itself_is_declined` - PID 4 (`System`) never yields an - image name and must resolve to `None`, not to a nameless cache entry. - - `::test_a_name_that_comes_back_empty_is_declined_not_cached` - the succeeded-but-EMPTY name, the - state a just-exited process is in (measured under churn: pid 45336, identical parent and start - time across two reads, name `'cmd.exe'` then `''`). Drives the five ctypes entry points through - an injected fake, which also pins the FILETIME epoch conversion and runs the parsing on - **every platform**, not only where the API exists. - - `::test_the_native_policy_gate_is_read_per_call_not_cached` - regression test for the bind-time - gate described above. - - `::test_a_cold_binding_does_not_push_other_threads_onto_the_slow_path` - 8 threads released - together onto a never-bound surface; every one of the 400 calls must take the handle route. - Regression test for the marker-in-the-shared-slot fault, and it catches its mutant (putting the - marker back) while the lock alone does not. - - **Verified by mutation, and one had to be rewritten to earn it.** Four mutants: policy gate - removed, parent replaced by the process's own pid, creation stamp dropped, empty name cached. - The first version of the empty-name guard **SURVIVED** its mutant - PID 4 leaves by the - failed-call branch, so deleting the empty-string check changed nothing it could see. Splitting - it into the two tests above catches all four. -- `test_hot_path.py::OS_FUNCTIONS` gained `_native_process_info`. Without it the guard would have - gone on passing while no longer watching the main resolve route - the failure mode where a test - stays green because it stopped looking. Its docstring's call tally is re-measured. - -- **Four new guards in `test_processes.py` for the shared-port warning**, all verified by mutation - (5 mutants, 5 caught): - - `::test_the_targets_own_program_is_not_reported_as_a_stranger` - the fixture deliberately gives - pids 11 and 14 the SAME program name, because a fake that cannot tell them apart cannot catch - this at all. - - `::test_the_warning_says_which_of_the_two_outcomes_applies` - asserts each branch says its own - verdict AND does not also say the opposite. - - `::test_a_port_that_left_the_map_is_not_described_from_stale_data` - - `::test_known_ports_are_named_and_unknown_ones_are_left_alone` - shape plus the two overlay - entries, since the rest comes from the machine's services file. - -- **`test_failsafe.py::test_a_dead_capture_thread_fails_open` was racy and CI caught it** (green ~5/5 - locally, red on the runner). `_stop_locked` clears `_running` as its SECOND statement - deliberate, - so a watchdog mid-maintenance sees nothing to fire - and closes the divert several statements - later, logging the STOP event after that. So `not is_running()` is true well before all three - things the test then asserted, and a loaded runner can be descheduled inside that window. **The - product is correct: the fail-open invariant holds**, the divert is closed inside the same critical - section. The test now waits for the outcome (running cleared AND divert closed AND the event - logged), which is exactly the fix `::test_the_engine_stops_itself_after_its_duration` already - carries a few lines above - complete with the comment "Wait for the promise, not for the flag". - That test was fixed after CI caught the same shape; this one was never given the same treatment. - -- **Three guards on the ACT of releasing, in `tests/test_version_and_release.py`.** The tree was - heavily guarded and the release was not: every rule about it lived in prose, and three of them - had already been broken in practice (see the `### CI` entry below and the folded `[Unreleased]` - block this version opens with). - - `::test_version_txt_has_a_dated_section_in_both_changelogs` - VERSION.txt must name a - `## [x.y.z] - YYYY-MM-DD` section in both files. Safe to run on every commit because - VERSION.txt carries the LAST released version while new entries collect under `[Unreleased]`. - - `::test_the_mutable_changelog_sections_have_no_duplicate_headings` - one `### Added` per - version, and only convention 39's names. Scoped to `CHANGELOG.md` and to the sections still - mutable (`[Unreleased]` and the VERSION.txt one): this file's headings carry information a type - vocabulary cannot hold, and `[0.3.0]` is a published release note whose duplicate `### Changed` - is not worth rewriting. The duplicates had been merged by hand twice (`61601ad`, `0aaa86a`) and - came back both times, because the only structural guard reads exactly one index of the section - list. - - `::test_ci_and_release_freeze_the_same_python` - `ci.yml`'s `build` job and `release.yml` must - freeze the same interpreter. Parsed by line scan, not PyYAML: PyYAML is not in - `requirements-dev.txt`, so importing it would make the test an ERROR on a fresh CI checkout. - **All three verified by mutation, seven mutants, all caught**: VERSION.txt bumped without its - section, the heading stripped of its date, a second `### Added`, a `### Security` outside the - vocabulary, `release.yml` drifted to 3.13, the `build` job renamed, and `python-version` turned - into a variable. The last two matter most - a guard that silently passes when the file moves - under it is not a guard. - -### Docs - -- **Three registries gained a documented mirror, and a guard to keep it honest.** The scenario - format, both CSV headers and the 17 Connections columns were undocumented in both READMEs; the - column meanings existed only as GUI tooltips (`tips.col_*`), and the CSV headers existed nowhere - but the code. New guards in `test_readme_guards.py`: - `::test_both_readmes_document_every_csv_column` (over `App.CONN_CSV_HEADER` and - `App.CSV_COLUMNS`), `::test_both_readmes_document_every_connections_column` (over - `conns.COLUMNS`, checked by i18n LABEL, since the label is what a reader has to find), and - `::test_both_readmes_list_every_scenario_action_and_shipped_file` (over `ACTIONS`, `STEP_KEYS`, - `FILE_KEYS` and the actual `scenarios/*.json` set). - -- **Four README drifts found by reading the prose the guards do NOT cover.** `test_readme_guards` - pins the layout tree, the `decide()` order, the presets, both CSV headers, the Connections - columns and the scenario set; `test_cli_docs` pins every flag. What is left is ordinary prose, - and that is where all four were: - - **An "Enable" checkbox in both READMEs that no longer exists.** Verified before changing it, - not assumed: no i18n key for it anywhere in `lang/*.json`, and no section passes `toggle=` in - `fields.py` (the `Section.toggle` mechanism survives, unused). Replaced with the case that - really happens, `overridden_by` on `down`/`up` under a schedule. - - **"Target process ... requires `psutil`" is false on Windows.** Measured by blocking the - import with a `MetaPathFinder` and re-running: 310 ports mapped, 37 of 37 names resolved, - `make_targeting("claude")` found 99 ports. `psutil` is the off-Windows fallback, and it stays - in `requirements.txt` unconditionally. **A first attempt at this probe proved nothing** - it - used the removed `find_module` hook, which 3.14 ignores, so `psutil` imported anyway. The - probe printed that it had failed, which is the only reason it was not written up as a result. - - **The Tests section listed roughly half of `ci.yml`**, omitting the coverage gate, `--doctor`, - `--license`, the "driver ships next to the exe" check and, most notably, the **GUI render - check on real Tk under Xvfb in both languages** - the one step with no unit-test equivalent. - - **Neither README mentioned `release.yml`**, while telling users to verify the - `SHA256SUMS.txt` that only that workflow produces. - No new guard: the fix here is prose accuracy, and the honest mechanical guard for prose does not - exist (PROJECT_NOTES rule 5). Both READMEs changed together, as convention requires. - -- **The user-facing `[0.4.0]` section was rewritten at a third of the length.** Measured before - touching it: **817 lines, 92 entries, 11 338 words, median 115 per entry, longest 342**. That is - not a release note, it is an internal log wearing one - convention 39 already says CHANGELOG.md - carries the EFFECT for a tester and CHANGELOG-INTERNAL.md carries the reasoning, and this file - holds **50 729 words** for the same version, so nothing was lost by cutting. Verified that claim - before cutting rather than assuming it: eight topics from the longest user-facing entries were - checked here first, including two that needed a second look under different names - (`effective_loss_pct`, `switch interval` / `THREAD_SWITCH`). - After: **346 lines, 67 entries, 4 023 words, median 65, longest 93.** Related fixes merged into - one entry each where they shared a subject (the 55 `Fixed` entries became 22), and a **short - lead block** was added above the first section - deliberately NOT a `### Highlights`, so it - cannot collide with convention 39's section vocabulary or the guards that read it. - 🔴 **Six entries carrying `**BREAKING:**` were sitting inside `### Changed`** (profiles, the - connections table, the repro report's reset count, "Effective loss", the CSV columns, `--gui`). - They are now in `### BREAKING`, which is the whole point of that section existing first. - Past versions were left alone: `[0.3.0]` and `[0.2.0]` are published release notes. - The CSV guard **failed on its first run and caught a real gap** - the docs abbreviated - `delivered_in_scope_bytes_down / ..._up`, so the second name appeared nowhere - and - `test_cli_docs.py::test_no_stale_app_flags_in_readmes` caught the new prose writing a literal - `--flag` that no parser has. Both are the guards working before the commit, not after. - -### CI - -- **CI and the release build run on Python 3.14 only.** The test matrix dropped `3.10` and `3.13` - for a single `3.14` (it still varies the OS, so it is 2 cells instead of 4), and both PyInstaller - jobs - `ci.yml`'s `build` and `release.yml` - moved from 3.13 to 3.14. That is the version the - project is developed, built and released on; the old matrix was proving a configuration nobody - receives. Verified against `actions/python-versions` before the change rather than assumed: - 3.14.6 is stable and published for `linux-x64` and `win32-x64`, which is exactly what - `ubuntu-latest` and `windows-latest` need. - 🔴 **`requires-python` stays at `>=3.10`, and both READMEs still say "Python 3.10+"** (owner's - decision): installing from source on an older Python remains supported, it just stops being - CI-proven. The mismatch is deliberate and is written down in both workflows - do NOT close it by - raising `requires-python`; if 3.10 needs guarding again, add the version back to the MATRIX. - The two PyInstaller jobs must keep the same interpreter as each other, because PyInstaller - freezes it into the bundle: letting them drift means CI smoke-tests one artefact and users - download another. - **That last sentence is now a test** - `test_ci_and_release_freeze_the_same_python`. It was a - true, load-bearing statement with nothing behind it, which in this project means it survives - exactly until a session that never read it edits one of the two files. - -- **`release.yml` refuses to publish a release whose changelogs are still open.** The existing - step proves the tag is LABELLED right (its base equals VERSION.txt). The new one proves the - release NOTES are: both changelogs must carry a dated `## [x.y.z] - YYYY-MM-DD` section for - VERSION.txt, and `[Unreleased]` must hold no entries. - Placed in the workflow rather than in pytest because the second half cannot be always-on - during - ordinary development `[Unreleased]` is supposed to have entries - and because the tag push is the - one moment the check must not be skippable. An EMPTY `[Unreleased]` heading passes, since that is - the normal shape between releases. - Exercised against four trees before landing: the current one (pass), an `[Unreleased]` reopened - with one entry (fail), the dated heading stripped of its date (fail), and an empty `[Unreleased]` - (pass). - -- **The GitHub release body is now the changelog section, not `--generate-notes`.** New - `tools/release_notes.py` extracts `## [x.y.z]` from `CHANGELOG.md` (stdlib only - the release job - installs `requirements.txt` and PyInstaller, nothing else) and `release.yml` passes it as - `--notes-file`. `--generate-notes` lists merged PR titles, so the release page showed something - nobody wrote and never showed the changelog at all; for 0.4.0 that would have been 43 PR titles. - The same command is what you paste into a blog post, so there is no second copy to drift. - 🔴 **It crashed on `[0.3.0]` and the crash is the interesting part.** `release.yml` runs on - **windows-latest**, where stdout still defaults to the ANSI code page, and `[0.3.0]` carries - U+25CF - so writing the notes as `str` raised `UnicodeEncodeError` and would have failed the - publish step, at the one moment nobody wants a surprise. `0.4.0` is pure ASCII and printed fine, - which is exactly why testing only the current version proves nothing. It writes UTF-8 bytes now. - -- **A word cap on user-facing changelog entries: `test_no_user_facing_entry_grows_into_an_essay`.** - 100 words, `CHANGELOG.md` only, mutable sections only - the same scope and the same reasons as the - duplicate-heading guard. 100 rather than the ~40 most entries manage, because the few that carry a - behaviour change a reader must act on need the room, and splitting those into three entries is - worse than one long one. Verified by mutation both ways: a 130-word entry goes red, and `[0.3.0]` - keeps a 171-word entry while the suite stays green, which proves the scoping is real rather than - accidental. - -- **`test_release_notes_extract_every_released_version`** pins the extractor: every version yields a - non-empty body, the `## [x.y.z]` heading is dropped (gh sets the title), the body stops before the - next version, an unknown version returns `None` and the script exits non-zero with empty stdout. - 🔴 **Two mutation-methodology traps, both hit here, both producing a false "caught".** - (1) `tools/release_notes.py` was UNTRACKED, so `git checkout -- ` restored nothing and the - four mutants accumulated - only the first was cleanly attributed. Re-run from an explicit - known-good copy, **one of the four turned out to SURVIVE**. (2) That survivor survived because the - test ran the subprocess only for `VERSION.txt`, which is ASCII, so it never fed the code the - character that breaks it. The test now runs the real process for **every** released version and - asserts that at least one section carries non-ASCII, so the encoding path cannot quietly stop - being exercised. A surviving mutant is information about the TEST, not only about the code. - -### Fixed - -- **`SocketWatcher.reconcile()` let a STALE poller snapshot overwrite a newer SOCKET event - (audit F2).** The merge was unconditional (`merged[port] = pid`) and the docstring called the - snapshot "authoritative" - reading "complete" as "newest". It is collected up to - `portmap.REFRESH_S` (0.30 s) before it is handed over and applied up to a watchdog tick (0.20 s) - later; **measured: 10 of 20 ticks hand over a snapshot the previous tick already applied**, - because those two intervals interleave that way. Consumers of the damage: `engine._pid_for` - (connection-log `proc`/`pid`), `engine._process_for`, `targeting.owner_targeted` - the gate for a - TCP SYN and for EVERY UDP datagram - and `ProcessTargeting.refresh` via `table.snapshot()`. - - **Fix: every map entry carries the time of the evidence behind it.** New `_evidence` - (`local_port -> monotonic`), written by `apply()` on both branches, including CLOSE, where it is - a TOMBSTONE: without a mark, "closed a moment ago" and "never seen" are the same absence, which - is what let an older snapshot resurrect a closed port. A snapshot entry is applied only when the - snapshot was COLLECTED after that stamp. - - **The stamp gates the PRUNE too**, not only the merge: a port an event touched after the - collection does not count as absent, because that snapshot never had a chance to see it. - Previously its survival rested on `WATCHDOG_TICK_S` (0.2) and `portmap.REFRESH_S` (0.3) - interleaving so two stale reconciles never ran back to back - arithmetic nothing stated or - tested. The two-pass grace SURVIVES alongside it (the collection itself takes time). - - **`_evidence` is rebuilt in `reconcile`**, keeping a tombstone only while it can still veto a - snapshot. Steady state is (open sockets + one refresh interval of churn), not every socket the - session ever saw. - - **Rejected: "events always win"** (the smaller change). With a missed CLOSE *and* a missed new - CONNECT the map would keep a dead pid that the prune can never take, since the port is in every - snapshot. Guarded by `test_a_snapshot_taken_after_the_event_still_heals_a_stale_entry`. - - **New `portmap.PortTable.collected()` -> `(dict(ports), _last)` from ONE lock hold.** - `_last` is when the collection STARTED (`refresh()` reads `now` before the `iphlpapi` calls), - so it under-states freshness - the safe direction. Taking the map and the stamp separately - would let a refresh land between them and stamp OLD data with a NEW time, which is the one - error direction that silently reintroduces this bug. `snapshot()` is unchanged, so the shared - read surface (`snapshot`/`name_of`/`ancestors`/`refresh`/`pid_for`) that `ProcessTargeting` - resolves against does not grow a value only one of its two implementations could fill. - - **`collected_at` has no default, deliberately.** Both call sites sit inside exception handlers - (`crashlog.quiet` at bootstrap, the watchdog's per-tick `except`), so a missing argument would - become "the safety net silently stopped running" against a session that looks healthy. This is - not hypothetical: it happened during verification to a diagnostic script left on the old - signature, which reported 0 reconciles for a whole run. - - **`pid_for` is untouched** - `_evidence` is read only by `reconcile`, under `_lock`. The - lock-free capture-thread read stays a plain int->int dict lookup, so the hot path pays nothing - by construction, not by measurement. - - MEASURED on a live session (2026-07-29, real SOCKET handle, 887 connections / 5232 events / - 123 reconciles in 25 s): **919 writes the old rule would have made are refused**, each for a - port an event touched 0-170 ms after that snapshot was walked; **0 stale writes survive**. An - earlier count of "1394 resurrections + 11 reverts" is superseded - that detector could not tell - a resurrected socket from the same process reopening a recycled port and over-reported by about - a third. A separate check confirmed the socket table drops a port immediately on `close()` - (pid `None` at +0.0 s), so lingering TCP teardown is NOT an explanation for what remains. -- **New tests.** `tests/test_socketwatch.py`: `test_a_newer_event_is_not_undone_by_an_older_snapshot`, - `test_a_port_known_only_from_a_connect_still_outranks_an_older_snapshot` (written because a - MUTANT SURVIVED - the handover test is carried by the CLOSE tombstone, so the ADD branch's stamp - had no guard), `test_a_closed_port_is_not_resurrected_by_an_older_snapshot`, - `test_a_snapshot_taken_after_the_event_still_heals_a_stale_entry` (guards against over-fixing), - `test_a_port_an_event_touched_after_the_collection_is_never_counted_absent` (the prune gate), - `test_the_evidence_map_does_not_grow_with_every_connection_ever_seen` (tombstone bound), plus a - hand-driven `_Clock` so the event/collection ordering is a fact of the test rather than a race - against the real `time.monotonic()`. `tests/test_socketwatch_wiring.py::test_the_watchdog_keeps_reconciling_the_live_map` - (the safety net is seen running, since both call sites are inside handlers). - `tests/test_processes.py::test_collected_hands_over_the_map_and_when_it_was_gathered_together` - (the stamp is never newer than the moment the collection began). - `test_reconcile_bootstraps_and_prunes_only_after_a_two_pass_grace` now states that its snapshots - are collected AFTER the event on purpose - that is the residual case the grace is for; the - before-the-event case is a different scenario with its own test. - **Mutation-checked**: nine mutants (merge gate, prune gate, both `apply` stamps, tombstone - expiry, the "events always win" over-fix, `collected()` stamping `clock()` instead of `_last`, - the watchdog call removed, `owner_targeted` returning False) - all nine RED. -- Four rows added to PROJECT_NOTES "Powierzchnia regresji". - -### Fixed: the last three audit findings (F3, F4, F5) - -- **`_FlowTable.keep_for` could only ever RAISE the age window (F3/T5).** The rotation deadline was - cached as an absolute `_next_rotate` and `keep_for` changed `_rotate_s` without touching it, so - once `set_nat(>0)` had pushed the window to infinity, `set_nat(0)` could not bring it down for the - rest of the session and `_flow_last` was freed only by its SIZE ceiling. Every GUI "Apply" runs - every setter, so one round trip through the NAT field was enough. **Fixed by removing the class, - not the symptom**: the table now stores `_last_rotate` (when the last rotation HAPPENED) and - derives the deadline live, so any change to `_rotate_s` takes effect on the very next call - no - reset to remember, in either direction. Impact was memory only: a retired record reads back as - "never seen", which the NAT check treats as "pass", so it can lose an impairment and never invent - one. Guard: `test_the_flow_table_age_window_can_be_lowered_again_not_only_raised` (both the unit - and the real `set_nat` path). -- **`_trim_conns` could empty the connection log instead of trimming it (F4/T6).** The cutoff is a - SAMPLED estimate compared inclusively (`c["last"] <= cutoff`), so rows sharing a timestamp all - land on the same side of it; with every row on one stamp it takes the whole table. Measured - against a 1000-row cap: **1200 rows trimmed to 0 instead of 900**. Not reachable from ordinary - traffic here - `time.monotonic()` resolves to ~100 ns on this machine (26 204 ties in 200 000 - reads), and a 1200-row burst still produced 865 distinct stamps and trimmed correctly - but the - clock's resolution is a platform property this code should not lean on. **Fixed with a floor**: - never evict more than `len(conns) - MAX_CONNS * EVICT_KEEP`, whatever the estimate says. Guard: - `test_evicting_the_connection_log_can_never_empty_it`, which also asserts the SURVIVORS are the - newest rows, so the floor cannot be satisfied by keeping the wrong ones. -- **BREAKING: `duplicated` counted the DECISION to duplicate, not the copy (F5/T8).** `tips.stat_duplicated` - says "Packets sent twice"; the counter was bumped whenever `decide()` returned a second release, - whatever `_enqueue` then did with it. Measured with a queue too small to hold the copy: **seen=40, - duplicated=40, and ZERO packets sent** - a session that put nothing on the wire reporting forty - duplicates. - - **Decision: fix the COUNTER, not the tooltip.** The project's own convention is that counters - report what happened - `corrupted` counts "successful payload flips only", and `rst_reset` / - `rst_sent` were deliberately SPLIT because they answer different questions. `duplicated` was the - exception. And "Packets sent twice" is the sentence a tester actually wants; rewording it to - "packets the tool decided to duplicate" would be accurate and less useful. - - `_enqueue` now returns **whether it QUEUED the packet**, which is a different question from - whether an overflow was counted - the two part company for a copy, since a refused copy is - deliberately not an overflow. The capture loop counts one duplicate per copy the queue accepted. - - Counted at ENQUEUE, not at send, which is the same standard `corrupted` is held to: the work was - really done, though a STOP can still strand the copy afterwards. - - `duplicated` is part of the NDJSON `sample` schema (`cli._sample_record`), hence BREAKING: - reports across this line are not comparable on that field. - - `test_overflow_counts_packets_lost_not_queue_entries` had `duplicated == n` as a SETUP sanity - check pinning the old meaning; it now asserts the new one (`queue // 2`, five copies fitting in - ten slots) with a note saying why. Its actual subject - `drop_overflow` never exceeding `seen` - - is untouched and still passes unchanged. - - New guard: `test_duplicated_counts_copies_that_were_queued_not_decisions`. -- **Mutation-checked**: caching the rotation deadline again, removing the eviction floor, counting - duplicates per decision again, and making `_enqueue` report overflow instead of queued - all RED. - -### Added: the port -> pid collapse stops being silent (ADR, no behaviour change) - -`portmap`'s map is keyed by the port NUMBER alone and merges four tables (tcp/v4, tcp/v6, udp/v4, -udp/v6) in that order, so a port with several rows keeps the LAST: UDP overwrites TCP, v6 -overwrites v4, and within one table the winner is whatever `iphlpapi` returned. - -- **MEASURED 2026-07-30** (Win11, 50 samples, reading RAW rows - an earlier probe counted 2 of 91 - because `_table` collapses inside itself and hid the rest): **4 of 127 port numbers had - conflicting owners** - 68 (DHCP, two svchosts), 1900 (SSDP, svchost + Spotify), **5353 (mDNS, - FIVE at once**: svchost, Spotify, adb, msedge twice) and 49664 (RPC: TCP lsass vs UDP svchost). - Under load (600 extra sockets, 718 port numbers) **no new ephemeral collision appeared**; 66 - ports >= 49152 had exactly one owner. -- **Consequences DEMONSTRATED through `ProcessTargeting`, not reasoned:** target `Spotify.exe` -> - 1900 and 5353 OUT of its port set (its own traffic escapes); target `msedge.exe` -> 5353 IN, - together with svchost's, Spotify's and adb's traffic; target `lsass.exe` -> its TCP 49664 escapes - because UDP svchost won the row. The connection log shows one arbitrary owner for every row on - those ports. On 5353 the winner is not even stable: 40 walks 0.25 s apart gave Spotify x32, adb - x6, msedge x2, so scope flickers at the resolver's rebuild rate. -- **REJECTED - a `(proto, port)` key** (two frozensets rather than a tuple, so no per-packet - allocation): fixes 1 of the 4, because three are several processes on the SAME protocol and - family (`SO_REUSEADDR`), where the local port does not carry the answer at all. Packet-path work - for one port in 127, in a budget where disabling the WHOLE connection log is 1.012x. -- **REJECTED - `port -> {pids}` matching any owner**: not a fix, a POLICY change from "one - arbitrary owner" to "all of them". On 5353 that means targeting msedge also impairs svchost, - Spotify and adb - trading a coin-toss false negative for a guaranteed false positive, exactly - where the OS multiplexes. -- **TAKEN: behaviour unchanged, silence removed.** New `portmap._put(out, owners, port, pid)` - - ONE source for the row-installing rule, used by the native walk and the psutil fallback, so the - two cannot drift about which ports are shared. It records the owner a row evicts and allocates - nothing unless a port really has two. `PortTable._shared` is installed beside `_ports`, under the - same lock and the same generation guard (a shared list from an older walk beside a newer map - would name processes that own nothing), and read via `shared_ports()`. - `targeting.ports_shared_with_others(pids, table)` intersects that with the TARGET's pids and - names the OTHERS; `settings._warn_about_shared_ports` says it once from `apply_targeting`'s - announcing path (never per resolver tick), via the new `log.targeting_shared_ports` key in both - language files. It asks the POLLING table on purpose even in a session where targeting resolves - against the `SocketWatcher`: the question is about the OS socket table, not about how the tool - learned a mapping. -- A **global** "N ports are shared" counter was considered and dropped: a number about the machine, - not about the user's test, and nothing they can act on. -- **New tests** in `tests/test_processes.py`: - `test_a_port_several_processes_hold_is_recorded_instead_of_silently_collapsed` (the flat map is - UNCHANGED - last row still wins - while every evicted owner is recorded; drives the REAL `_put`, - because the first version gave its fake table a copy of the rule and a mutant deleting the - recording SURVIVED), `test_shared_ports_are_published_with_the_map_they_belong_to`, - `test_only_ports_the_TARGET_holds_are_reported_as_shared` (a port shared between two strangers is - not reported; a table that cannot answer degrades to silence), - `test_a_target_sharing_a_port_is_said_out_loud_once` (and a target that shares nothing stays - quiet), and `test_the_shared_port_warning_is_actually_wired_into_apply_targeting` - written - because deleting the call site left the direct-call test green, which is a diagnostic nobody - invokes. - Mutation-checked: `_put` not recording, `_shared` not installed, the warning not filtered by - target, the target named among "the others", the call site deleted, and the warning firing with - nothing to say - all RED. -- `_Native.port_pid_map` / `_psutil_port_pid_map` / `_Native._table` take an optional `owners` - dict. Four test doubles updated to match: with the old signature the call RAISED, the `except` in - `refresh()` swallowed it, and the whole collection fell through to the real system - the tests - caught exactly that. -- **NOT measured**: the `SocketWatcher` side has the same shape (`_ports[port] = pid`, last event - wins) and was seen holding ports 138 and 1900 with two pids at once, but was not swept as - thoroughly as the poller. - -### Fixed - -- **`BeanEngine._start_locked` swallowed a failed `divert.open()` (audit F1).** The `except` did - `crashlog.note(_exc, "engine")` and fell through, so `_running` went True, three workers were - spawned, and the capture thread's first `recv()` raised - `RuntimeError("WinDivert handle is not open")`. THAT became `self.fault`, the log line, the event - log and the repro report; the real cause never left crashlog at `severity=debug`. - MEASURED against the real driver with a filter it rejects - before: `start()` returned normally, - `is_running=True`, `fault='WinDivert handle is not open'`, 3 swallowed crashlog records, and the - log printed START *after* STOP. After: `start()` raises `OSError [WinError 87] Parametr jest - niepoprawny`, `is_running=False`, `fault=None` (there was no session to fault), **0 crashlog - records**. - - Both callers already handled this and NEITHER could be reached: `cli._run_session` - ([cli.py](beantester/cli.py)) wraps `start()` to `_fail(RUNTIME, "cannot start the capture: - {e}")`, and the GUI's `_finish_start` shows `dialogs.start_failed` WITH - `dialogs.run_as_admin` - the hint a non-elevated user needs. The exit code was right anyway - (via `if engine.fault` further down), but the message was the symptom. - - `except BaseException` and a bare `raise`, so the original exception object reaches the caller - unwrapped. `self._divert = None` first: `_running` is still False here and the engine never - reached `_LIVE_ENGINES`, so dropping the reference is the whole cleanup. Deliberately NO - `close()` - a pydivert handle that never opened raises from `close()` too (measured), which - would replace the real error with a second meaningless one. - - `session_info()['driver_queue']` stops lying as a side effect: it documents `None` as "the - simulate path, which has no driver queue", and an unopened handle used to produce exactly that - on a REAL session. Residual, NOT fixed here: a real handle whose `get_param` fails for some - other reason still reports `None` and still reads as "simulate". -- **The START banner moved ABOVE the thread spawn (audit F6).** `self.log(log.start_filter)` and - `log_event("START", ...)` sat below the `except` block, so an early fault printed the live log - BACKWARDS (measured: recv error, fault, *then* "Start. Filter: ...", then Stop). The event log was - always ordered correctly - a worker-initiated stop blocks on `_stop_lock` until `start()` returns - - so only the log a tester actually watches was lying. Announcing first also reads correctly when - the spawn itself fails: START, fault, STOP. -- **New tests.** `tests/test_failsafe.py::test_a_divert_that_cannot_open_fails_the_start_instead_of_faulting_later` - (the real cause reaches the caller; the engine is not left "running"; no fault is recorded because - there was no session; the handle is dropped; and a later START still works) and - `::test_the_start_banner_is_logged_before_a_worker_can_fault`, with a new `UnopenableDivert` - double whose `recv()` still "works" - like the real thing, where an unopened handle fails at recv - with a message naming nothing. `tests/test_cli_runtime.py::test_exit_code_runtime_without_pydivert` - gained one assertion: the REASON must survive the trip to stderr, not just the exit code - that - branch was unreachable before this fix. - The banner test matches on the seed and on the interpolated exception text, NOT on translated - words: an earlier version looked for "fault" and passed or failed by the machine's UI language - (green on an English CI runner, red on this Polish one), which is a test reporting the locale. - Mutation-checked: restoring the swallow -> RED; moving the banner back below the spawn -> RED. -- **`_watchdog_loop` read `self._socketwatch` TWICE per tick** - once for the `is not None` guard, - once to call `reconcile`, with `self._ports.collected()` in between. A concurrent `stop()` - landing in that gap made an ordinary STOP raise `AttributeError` into the tick's `except`, - filing a crash record for a clean shutdown. Now read ONCE into a local, the idiom `_live_pid` - already uses for the same attribute and the same reason. - Guard: `tests/test_socketwatch_wiring.py::test_a_stop_landing_mid_tick_does_not_fault_the_watchdog` - - the window is a few instructions wide, so it is STAGED rather than raced: the port-table double - clears the engine's reference from inside `collected()`. The first version of that test staged - nothing and a mutant restoring the double read SURVIVED, because `_start_socketwatch` bootstraps - through the same method while `_socketwatch` is still `None` and consumed the one shot; the - double now only fires once the watchdog is the caller. Mutation-checked RED afterwards. - -### Tests: the audit's coverage gaps, the ones whose subject already works - -Gaps T2/T4/T7 from the engine stability audit. T1/T5/T6/T8 are deliberately NOT here: their -subjects (F1 swallowed `open()`, F3 `_FlowTable.keep_for`, F4 `_trim_conns` on tied stamps, F5 the -`duplicated` counter) are still defective, and a test written now would either be red or would pin -a bug as expected behaviour. They land with their fixes. - -- **T2 `tests/test_engine.py::test_every_captured_packet_is_accounted_for_under_every_impairment`** - (parametrised over 19 combinations). The only balance test was - `test_inject_batch.py::test_the_balance_holds_across_an_ordinary_session`, which drives a - PASS-THROUGH session and sums four of the twelve loss counters - the other eight being zero when - nothing is configured. So `seen == delivered + losses` was guarded exactly where it is hardest to - break. This arms every gate in turn (loss, corrupt, dup, latency+jitter, bandwidth, - bandwidth+dup, schedule, SYN, MTU, NAT, RST, flap, LAN, block, destination target) plus two - all-at-once runs, over mixed TCP/SYN/UDP/ICMP traffic in both directions. Delivered is counted as - DISTINCT packet objects, not sends, so duplication does not confuse the question; the test also - asserts `drop_overflow == 0`, because an overflowed original whose duplicate still got through is - a documented double-count (see `_enqueue`) and a different subject. -- **T4 `::test_a_dropped_packet_is_always_in_impairment_scope`** - the premise - `impairment_loss_pct`'s docstring rests on ("cannot exceed 100%"), which nothing checked. Sweeps - 400 randomised core configurations x 6 packets and asserts no `Decision` is ever - `drop=True, scoped=False`, plus that the sweep really armed several gates. A new impairment placed - ABOVE the targeting gates would break the FIGURE rather than the pipeline - silently, because the - number would still look like a percentage. - **`::test_the_loss_figure_stays_a_percentage_with_a_target_set`** is the end-to-end half, and its - docstring says plainly that it is the weaker of the two: mutation shows the unscoped-LAN-drop - mutant is caught by the structural test and NOT by this one. -- **T7 `::test_nat_and_rst_survive_being_switched_off_and_on_mid_session`** - every GUI "Apply" - re-runs every setter, so a live session toggles these constantly, and nothing exercised the second - and third call. Covers NAT off->on (the flow table must not be stranded) and an RST cooldown - shortened mid-session (a flow reset after the change gets the SHORT hold; one already held keeps - the deadline it was given). An earlier version of the RST half could not tell "still held" from - "hit again", because `rst_prob=100` re-arms the flow the instant its cooldown lapses. -- **Mutation-checked**: stranded packets uncounted at STOP -> RED (3 combos); a drop placed outside - targeting scope -> RED; `set_nat` made sticky -> RED; `set_rst` cooldown made sticky -> RED. One - mutant (removing the `drop_send` bump) survived the balance test because `FakeDivert` never fails - a send - checked, and it is already guarded by - `::test_a_packet_the_tool_could_not_re_inject_is_counted_as_a_drop`, so no gap. - -### Tests: the recycled-PID window is measured and bounded, not just asserted (handoff point C) - -- The last "known edge, never reproduced" from the handoff. `owner_targeted` trusts the pid the live - map reports without verifying identity (verifying = `create_time()` in the packet path, convention - 20), so between a target exiting and the next rebuild a socket Windows hands that pid number is - treated as the target's. The docstring claimed "up to one resolver cycle (0.30 s)" - a design - statement with nothing behind it. -- **Measured against the REAL socket table** (no WinDivert and no admin needed: `portmap` reads - `iphlpapi` and `ProcessTargeting` resolves against it). Seven rounds per transport, probe holding - a real socket, killed, time until its pid leaves `_pids`: - - | transport | median | range | - |---|---|---| - | TCP | **309 ms** | 271-327 | - | UDP | **315 ms** | 290-325 | - - i.e. the 0.30 s routine tick and no more. **TIME_WAIT does not stretch it** - worth checking - rather than assuming, since a TCP socket can outlive its owner. Upper bound: with no traffic every - rebuild is the routine tick, while a live session's constant misses shorten it toward the 0.05 s - floor. -- **Deliberately NOT done: chasing real Windows PID reuse.** Hitting the same number inside a 0.30 s - window needs a spawn storm at ~30-50 ms per process - a dozen low-probability attempts, and - "we tried and it did not reproduce" is an expensive non-result. The window is the bound that - matters, and it is measurable directly. -- New guard `test_targeting_socketwatch.py::test_a_recycled_pid_is_in_scope_until_the_next_rebuild_and_no_longer` - owns BOTH halves: the false positive is real (or the docs promise a hazard that does not exist) - and it ENDS at the next rebuild (or the bound is fiction). Adding identity verification later - fails the first half, which is the intended way of being sent back to these docs. -- Three mutants, all caught: `owner_targeted` always False (no window at all), `_pids` never - forgetting a pid (window never closes), and the rebuild re-adopting a pid on its stale cached name. - -### Fixed: the first packet of a fresh UDP flow is in targeting scope (handoff point 3) - -- **The gap.** `decide()` step 1 asked `syn_covers` only when `is_syn`, and `is_syn` is set for TCP - alone (`engine.py`). UDP has no SYN, so it was never asked: a fresh flow was judged against - `_ports`, which the resolver had not rebuilt yet. For a long-lived flow that costs one packet; - for **DNS over UDP and QUIC, which take a fresh ephemeral port per exchange, it costs all of them**. -- **The fix is `(is_syn or not is_tcp)`** - a TCP SYN, or anything that is not TCP. No new parameter - on `decide()`; ICMP lands here too and exits on `port is None` inside the callback. - `ProcessTargeting.syn_covers` is renamed **`owner_targeted`**, because the old name became a lie - about when it runs (`core._syn_covers` -> `_owner_targeted`). -- **The shape was chosen by MEASUREMENT, and the measurement reversed the first recommendation.** - Variant (a) "ask on every miss" was proposed on the strength of a microbenchmark of one call - (185 ns) plus reasoning that the difference was negligible. Measured properly - each variant as a - real byte patch of `core.py`, fresh subprocess, three traffic mixes x three map sizes, median - of 5: - - | map / mix | today | (a) every miss | (b) syn-or-not-tcp | (a) - (b) | - |---|---|---|---|---| - | 400 / tcp-bulk | 799 | 997 | 788 | **+209** | - | 10 000 / tcp-bulk | 796 | 1056 | 790 | **+266** | - | 100 000 / tcp-bulk | 834 | 1052 | 833 | **+219** | - | 100 000 / mixed | 865 | 1030 | 934 | +96 | - | 100 000 / udp-heavy | 819 | 1028 | 1017 | +11 | - - (ns per `decide()`, every packet missing.) (b) is **free** on TCP-heavy traffic - within noise of - today - while (a) costs ~26% of `decide()`. **Map SIZE barely matters** (400 -> 100 000 ports is - ~35 ns), so this is about how often each variant asks, not about the lookup. The lesson went into - PROJECT_NOTES rule 5: measuring a COMPONENT is not measuring the DIFFERENCE BETWEEN VARIANTS. -- **Acceptance, master worktree against this tree on the same machine.** 8 DNS queries from FRESH - sockets (the realistic shape: a new ephemeral port each), the probe holding one socket open so - `_pids` cannot be the hidden variable, `--target --dst-ip 8.8.8.8 --dst-port 53 - --loss 100 --filter out` (outbound only, so the reply comes back untouched and the result stays - binary): - - | tree | queries answered | `scoped_seen` | `drop_loss` | - |---|---|---|---| - | master | **8 of 8** | **0** of 1944 captured | 0 | - | this branch | **0 of 8** | 8 | 8 | - - The master column is the finding in its purest form: with a process target set, **not one packet - of 1944 was ever considered in scope**. The branch column cross-checks exactly - 8 queries, - 8 scoped, 8 dropped. -- **The price, named rather than implied:** covering UDP widens the recycled-PID false positive - from one SYN per connection to every UDP datagram of such a socket until the next rebuild (<=0.30 s). - Ordinary TCP data is still never asked, so an established connection cannot be dragged in. -- Two new guards. `test_core.py::test_a_fresh_udp_flow_of_a_targeted_process_is_in_scope` pins both - halves - UDP covered, ordinary TCP data NOT asked - so nobody "simplifies" it into the expensive - form. `test_targeting_socketwatch.py::test_owner_targeted_says_no_for_portless_traffic` guards the - `port is None` line, which **this change put on a live path for the first time**: ICMP is not TCP, - so every ping now calls `owner_targeted(None)` on the capture thread. -- **Four mutants, all caught - after the run exposed a real hole.** Mutating the REAL - `owner_targeted` to ignore the portless case was caught by NOTHING at first: the core test drives - a `_FakePorts` double with its own implementation, so it never touched the code under test. That - is the same trap F16 recorded, hit again from a different direction; the portless guard above - exists because of it, and the core test now says which half it actually pins. - -### ADR 2026-07-29: Nuitka instead of PyInstaller - BUILT, measured and REJECTED - -- **Really built, twice**: Nuitka 4.1.3 on CPython 3.14.6 with MSVC (`cl 14.5`), `--standalone`, - once with defaults and once with `--lto=yes --python-flag=no_asserts`. Both executables WORK - - `--version`, `--license` (so bundled resources resolve), `--simulate` with correct NDJSON, and a - real WinDivert capture. -- **And both are slower where this tool lives.** Real capture throughput, alternating windows with - the ratio taken inside each pair: **0.777x untuned, 0.824x tuned, 0 of 8 pairs across the two - builds**. Start-up goes the other way and is the one real win: `--version` in 71 ms against 112, - a repeatable **1.57x**. Build time 271 s cold / 159 s with ccache against ~9-16 s; the release - folder 46.7 MB in 982 files against 22.5 MB in 263; the exe 23.1 MB against 3.3 MB. -- **The direction is settled (8/8), the magnitude is NOT**: the drift canary in the tuned run came - out at 0.82x, so "18% slower" is not a number to quote. `--lto` and `no_asserts` moved nothing, - which fits the bottleneck being the cross-thread handoff rather than generated-code quality - - but that is a hypothesis, not a finding. -- **Three real defects found on the way**, listed because they are the work anyone returning would - have to do FIRST: - 1. `--include-package-data=pydivert` **skips DLLs**, so `WinDivert64.dll` was missing and capture - was dead with `exit 1` (the spec collects it via `collect_dynamic_libs`). Fail-open behaved - exactly as designed: a loud `reason=fault`, not a silent zero. - 2. **Nuitka sets neither `sys.frozen` nor `sys._MEIPASS`, and `sys.executable` points at a - `python.exe` that does not exist** (verified with a separately compiled probe). From that one - root: `winenv.elevate_self()` silently fails to raise the UAC prompt and the GUI carries on - unelevated; the reproduction command names `python bean_network_tester.py` instead of the exe; - and `paths.app_dir()` / `resource_path()` work only BY ACCIDENT, through `PROJECT_ROOT` - derived from a fabricated `__file__`. Nothing guards any of it - the suite runs from sources, - where `is_frozen()` is False and all of it is correct. - 3. The `tk-inter` plugin ships **837 Tcl + 89 Tk data files** - the ~750 tzdata/msgs files the - spec deliberately trims. -- **The one argument that survives is UNMEASURED**: antivirus false positives, which cannot be - tested on the build machine. If that ever becomes a real problem for users, it is the only reason - to come back - and the price is the three defects above plus the throughput. **Source protection - is NOT an argument**: the program is GPLv3 with public sources. - -### Changed: checksums are recomputed only for packets this tool actually edited (handoff point 2) - -- **The rule:** `_inject_loop` now calls `send(packet, recalculate_checksum=modified)`, where - `modified` is set in `_capture_loop` when `core.corrupt_packet` reports it changed the bytes, and - rides through the release queue with the entry. `_send_rst` passes `recalculate_checksum=True` - explicitly - that packet was BUILT, never captured, so nothing has ever computed a checksum for it. -- **Correctness first, because this one can fail silently.** WinDivert delivers each packet with - its checksum-valid flags in `WINDIVERT_ADDRESS`, and pydivert exposes them - (`packet.ip_checksum` / `.tcp_checksum` / `.udp_checksum`). MEASURED here, sniff-only: - `udp_checksum` was **0 on 120 of 120 packets** on loopback AND over a real interface - outbound - traffic carries a PSEUDO checksum for the hardware to finish. Re-injecting the same bytes with - the same address hands that flag back to the stack, which then does what it would have done - anyway. Verified end to end where it can actually differ (loopback is a special path and lenient - about checksums it never had to compute): 24 MiB of TCP through the engine to a sink inside a WSL - guest, over 192.168.48.1, **25 165 824 B received, SHA-256 match, `drop_send` 0**, with - recalculation on and off alike. A bad checksum does not corrupt data quietly - the receiver - discards the segment - so the byte count is the guard. -- **Worth 1.122x end to end** (8 of 8 paired windows, spread 1.087-1.156, toggled mid-session with - the order swapped every pair), `send()` itself 54.5 -> 45.9 us. - 🔴 **The prediction going in was ~1.00x and it was wrong, for a reason worth keeping.** The - release heap sits at depth 0-2 since PR #74, so the inject thread has slack, and making a thread - with slack faster should buy nothing. The gain does not come from the injector's throughput - it - comes from the work no longer competing with the CAPTURE thread, which is what sets the rate. - Same shape as the earlier "send() is slow" misreading: the unit that matters is the PAIR of - threads, not either one of them. -- **A side effect that is really the point:** with nothing configured, the tool now passes traffic - through byte for byte in the literal sense. Recomputing meant an untouched packet went out with - different bytes than it came in - a pseudo checksum turned into a real one. -- 🔴 **`send()` on the divert protocol gained a keyword, and that was the deliberate choice.** - It touches `SyntheticDivert` plus seven test doubles (`tests/fakes.py`, - `test_engine.py`, `test_failsafe.py` x2, `test_rst_local.py`, `test_concurrency_chaos.py`, - `test_socketwatch_wiring.py`). The smaller alternative - wrapping the real handle in - `_start_locked` so the protocol stays one-argument - was rejected: with it, `_send_rst` would - silently inherit "never recalculate" and inject RSTs the local stack drops, and the symptom - ("RST does not reset anything") points nowhere near the cause. The explicit keyword puts the - decision at the call site where the next reader meets it. -- **Tests** (all verified by mutation - five mutants, five deaths): - `tests/test_checksums.py::test_an_untouched_packet_is_re_injected_without_recomputing_its_checksums`, - `::test_a_corrupted_packet_IS_recomputed`, - `::test_a_duplicate_of_a_corrupted_packet_is_recomputed_too` (the duplicate release is a separate - queue entry, and the mutant that dropped `modified` from it died here with `[True, False]`), and - `tests/test_rst_local.py::test_an_injected_rst_is_always_recomputed`. The RST guard lives with - the RST fixtures on purpose: a `FakePacket` has no TCP layer, so a "reset" test written next to - the others would have passed without ever building one. - -### Measured and NOT changed: the locks, the connection log and the address conversion - -- With the handoff cheap (PR #74), the three obvious remaining targets were re-measured the same - paired way, inside one session. **All three are noise:** nulling `_slock` 1.011x (6/6 pairs), - nulling `_clock` 1.002x (4/6 - a coin toss), both together 1.011x, and the watchdog's per-tick - work (`refresh_if_stale` + `warm_names`) 1.003x. The 1.71x / 1.25x those same ablations gave at - the 5 ms switch interval was the handoff showing up ON the locks, not the cost of the locks. -- **Memoising pydivert's per-packet `inet_ntop`** (`packet/ip.py:43-66` converts the address to a - string on EVERY read and caches nothing): 1.017x, 6/6 pairs. Real but small, and the change would - either duplicate pydivert internals or make the engine read raw address bytes and reshape - `_flowkey`, whose string keys are consumed by the connection log, the CSV, the repro report and - the NDJSON schema. Not worth it at 1.7%. -- **The whole connection log removed**: 1.012x. There is nothing to win by optimising it. -- 🔴 **Sending inline on the capture thread (zero handoffs) is 0.72x - 28% SLOWER, 0 of 8 pairs.** - Worth recording because it is counter-intuitive and cheap to try again: `recv` really does get - faster (32.6 -> 25.2 us) once nothing competes with it, but one thread then has to do both jobs. - The two threads genuinely overlap, and what looks like "handoff overhead" in the budget is the - price of that overlap, not waste. -- 🔴 **The list of performance levers is EXHAUSTED, and that is a result rather than a shrug.** - Everything measured is above; **every remaining candidate is <=1.02x**, and the strongest proof - is that removing the ENTIRE connection log - a feature nobody is going to remove - buys 1.012x. - That is the ceiling for optimising anything on the Python side; the rest of the budget is - syscalls (recv ~20 us + send ~32 us of ~70 us per packet) and the price of parallelism. -- ⬜ **The one thing NOT explored: free-threaded CPython (3.14t).** This session established that - the pipeline is bound by GIL HANDOFFS rather than by work - shortening the switch interval bought - 1.33-1.36x while nulling the locks bought 1.011x - and free-threading attacks exactly that - mechanism. **There is no measurement. That means "nobody checked", not "it does not work"** - - do not cite this either way. What would have to be settled first: whether `pydivert` and `psutil` - have free-threaded wheels or build, whether the win survives `recv`/`send` sitting in the kernel - anyway, and what happens to the lock-free assumptions in `socketwatch.pid_for` and - `targeting.__contains__`, which today rest on the GIL as an implicit lock. - -### Fixed: the engine's ceiling was CPython's thread-switch interval, not any stage (handoff point 1) - -- **The question.** The previous session narrowed the bottleneck by elimination - not the read - (a sniff-only loop does 92 944 pkt/s), not `decide()` (~800 ns) - and asked for a PER-STAGE - budget rather than another throughput benchmark, because this rig's absolute numbers are not - reproducible (see "What this actually sustains" in `engine.py`). -- **Method: measure in place, never re-implement the loop.** A proxy on the divert timed `recv` - and `send` (and the interval between them, which IS the loop body); a proxy on the packet timed - every attribute the loop reads BY NAME, so pydivert's parsing was measured where it happens; the - four engine methods were wrapped. `engine.start(divert=...)` is the only seam that needed to - exist, and it already did. -- **What the split showed, in one run:** the Python stages are small and not where the time goes - - `decide()` 6.2 us, `src_port` (which pulls the whole transport header) 5.5 us, `dst_addr` - (an `inet_ntop` per packet, uncached) 2.8 us, `_log_conn` 1.8 us, `_enqueue` 2.4 us. The two - syscall-bracketed stages were 3x their uncontended cost, on BOTH threads at once. -- 🔴 **The cause, isolated by ablation** (paired against a minimal 2-thread reference, drift canary - 1.03x): not the shared handle - a separate `SEND_ONLY` injection handle made it WORSE (send - 1.71x, recv 1.97x, rate 1.26x instead of 1.59x). Not the locks - nulling `_slock` bought 1.71x - and `_clock` 1.25x, both partial. **`sys.setswitchinterval` bought 2.38x at 0.5 ms and 2.68x at - 0.05 ms**, and collapsed `peak_queue` from 253 to 8-62. The capture and inject threads hand every - packet to each other and both leave the interpreter for a syscall, so each packet costs two waits - for the interpreter lock - and CPython lets a waiter sleep up to the switch interval, 5 ms by - default, before it insists. -- **The number that is trustworthy, and the one that is not.** A phase-to-phase throughput ratio is - worthless here: the 5 ms canary alone ranged 10 370 to 15 450 pkt/s inside one run. So the final - measurement flips the interval INSIDE one session, counts packets over alternating windows and - swaps the order every pair: **median 1.35x (loopback), 1.33x (loopback, repeat), 1.36x (real - interface to a WSL guest) - 24 of 24 pairs**. The per-call readings repeat exactly and are the - same finding without a stopwatch: recv 41.7 -> 19.9 us, send 56.7 -> 32.0 us. -- **Priced, not assumed.** Process CPU per packet FELL, 103.1 -> 92.9 us. Delay accuracy (the thing - the tool actually promises) is unchanged: against a configured 10 ms, lateness held a median of - 0.73 vs 0.76 ms and a p95 of 1.71 vs 1.65 ms, the shorter interval ahead in 3 pairs of 6. 0.5 ms - is the KNEE, not the floor - 1 ms still behaves like 5 ms, and 0.1 / 0.05 / 0.01 buy nothing. -- **NOT measured:** GUI smoothness at 0.5 ms, TCP traffic (this was UDP), and machines with fewer - than 16 cores. -- **Implementation:** `winenv.request_fast_thread_switch()` / `release_fast_thread_switch()` plus - `THREAD_SWITCH_S = 0.0005`, taken in `BeanEngine._start_locked` beside `request_fine_timers` and - given back in `_stop_locked` after the workers are joined - the same session-scoped shape, for - the same reason. It carries its OWN refcount and saved value: unlike `timeBeginPeriod` the OS - does not count these, so two overlapping sessions would otherwise let the first one to stop - restore a value out from under the second. Guarded by `_SWITCH_LOCK`, taken on start/stop only. - This is the one function in `winenv.py` that is not Windows-specific - it tunes CPython - and the - module docstring now says so. -- **Tests** (`tests/test_failsafe.py`, all four verified by mutation): - `test_the_shortened_switch_interval_is_in_force_only_while_a_session_runs`, - `test_the_switch_interval_is_restored_on_every_session_path` (clean stop, double stop, failed - start), `test_two_overlapping_sessions_do_not_restore_each_other_s_interval` (the refcount), and - `test_releasing_a_switch_interval_nobody_took_changes_nothing`. -- 🔴 **The mutation run found the hole in the TESTS, not in the code, and it is worth recording.** - The deleted-release mutant survived at first. Two reasons, both the same shape: the four tests - share one process-global holder count, so a leak in one made the next one's `start()` a no-op; - and "the interval came back to what it was when I started" is a TAUTOLOGY once anything leaks - - with the release deleted every engine session in the file leaks, so the process was already - sitting at the shortened value and "restored" was true by accident. The fix is `_switch_baseline` - / `_switch_restore`: assert against a sentinel value chosen by the test (0.004 - neither the - CPython default nor `THREAD_SWITCH_S`), and read the post-stop value INSIDE the try, before the - cleanup can paper over it. A second miss on the way: the first mutation run used `-k switch`, - which silently skipped the one test whose name says "interval" - the only guard on the refcount - - so that mutant came back green. - -### Corrected: "the bottleneck is INJECTION" and the batched injector's shallow-heap premise - -- **`PROJECT_NOTES` hot-paths section said the constraint was the `send()` side**, on the evidence - that the release heap grew ~1100 entries/s. The premise is right and the inference was wrong: - `send()` costs 26.9 us with nobody to contend with, and the 56.7 us it showed in the engine was - the wait for the interpreter lock. Both threads were inflated by the same factor, which is what - should have been the tell - a slow `send()` cannot slow down `recv()` on another thread. -- **The batching ADR below records `peak_queue` 30 and a mean batch of 2.13** and forbids - re-opening the topic without re-measuring the mean batch size under the workload in question. - Re-measured: under a saturating flood the heap sits 15-320 deep and reaches `max_queue` (20 000) - over a real interface, so a `SendEx` call would have been handed tens of packets, not two. That - condition IS met - and the topic still should not be re-opened, because the switch interval takes - `peak_queue` back down to 8-62. The ADR's conclusion survives; the reason under it changes from - "the injector keeps up" to "the injector keeps up ONCE THE HANDOFF IS CHEAP". - -### ADR 2026-07-29: the batched injector was BUILT, measured and REVERTED - -- **What was built.** `_inject_loop` taking every already-due packet (cap 32) and sending them with - one `WinDivertSendEx`; per-packet accounting that walked `pSendLen` back over the batch rather - than assuming atomicity; a `batch_sender` seam on `start()`; six guards including a staggered - timing test. All of it worked and was green. -- **Why it went anyway.** The bare-loop measurement promised 1.62-1.75x. The ENGINE measured - **1.00x** - 7 of 14 pairs, median 1.00, range 0.93-1.05, paired with alternating order. An earlier - 6-pair run had said 1.60x; the longer one shows that was drift (the unbatched half sat in a slower - phase and "caught up" in the last two pairs). The decision rule was fixed BEFORE the run: below - 1.2x end to end, revert rather than merge. -- 🔴 **The mechanism, which is the part worth keeping.** Instrumenting the real engine under a - saturating flood: **mean 2.13 packets per batched call** (distribution 1:3966, 2:10658, 3:3889, - 4:931, tail to 30) and `peak_queue` **30** against a 20 000 limit. The release heap barely - accumulates, because the inject thread keeps up with capture - so the batcher was handed pairs, - not batches. The earlier sweep already showed batch 8 giving only 1.27x and the full 1.6x needing - 32. At 2 there is nothing to amortise, and assembling the blob costs what the syscall saves. -- **Do not re-open this without first measuring the MEAN BATCH SIZE** under the workload in - question. If a configuration exists where the heap genuinely backs up (deep buffers, heavy rate - limiting), the number to check is that mean - not the throughput ratio, which drifts. -- **Kept from the work:** a real accounting hole it exposed. A packet popped off the heap and then - found without a `_divert` - STOP cleared it in between - vanished with NO counter: gone from - `_heap`, so `stop()`'s stranded sweep could not see it either. One packet wide already, a whole - batch wide had this shipped. It is charged to `drop_shutdown` now, with - `tests/test_inject_batch.py` pinning it and the seen/delivered/dropped balance around it. -- **Also kept, as a lesson rather than code:** the premature-release mutant (batching packets not - yet due) was caught by **zero tests in the whole suite** until a guard was written for it, and - that guard only worked once the arrivals were staggered AND the latency exceeded the stagger - - two conditions, each of which silently made the test vacuous. Recorded in PROJECT_NOTES rule 5. - -### Added: "show only the targeted traffic" - a VIEW preference (chunk 3) - -- `gui/prefs.py::scope_view_to_target` (BOOL, default False - convention 42's second kind: a - cross-restart GUI preference, so no CLI flag and never inside a traffic config file). -- **One decider, not five.** `App.scoped_stat(snap, key)` + `App.SCOPED_TWIN` is the single place - that knows which counters have a narrowed twin; the counter grid, the session panel, the chart, - the rate window, the Connections model and the connections CSV all go through it, so the - preference cannot mean one thing on one tab and something else on the next. -- **Engine side stays lock-free.** `bytes_in_scoped` / `bytes_out_scoped` are bumped in - `_log_delivered` from the row's own sticky `scoped` flag - the flag is already on the row by then, - so no change to the heap tuple was needed (its three positional consumers stay untouched). Written - without `_slock` on the same argument the sibling `sent`/`sent_in`/`sent_out` counters already - rest on: the inject thread is the only writer and an int rebind is atomic. Taking the lock there - was measured at a 5% regression when the sibling counters were added. -- 🔴 **Three deliberate omissions, each with a test, because "we decided not to" is what gets tidied - away by somebody making things consistent:** - - `drop_overflow` / `drop_shutdown` / `drop_send` never narrow - they count what THIS TOOL lost, - including untargeted traffic, and hiding that is precisely the convention-20 failure. Their - tooltips now say so, in both languages. - - the stats CSV does not follow the preference (append log; it gains both totals as columns). - - NDJSON and the reproduction report are untouched - they already carry `seen` AND `scoped_seen`, - so a saved run never depends on how the window was set. -- **The notes re-word themselves on the tick**, not only at build: the preference can be toggled - while a page is already built, and a note reading "ALL captured traffic" over narrowed numbers is - exactly the misleading sentence it exists to prevent. Same for the chart caption - (`frames.throughput_scoped`), which is redrawn every tick - a chart is what people screenshot, so - the scope has to be readable from the picture. -- New `tests/test_view_scope.py`. **Five mutants, all caught**, and the two that matter are the - good-faith "let's be consistent" edits: giving a tool-loss counter a scoped twin, and counting - scoped bytes for every flow. Also caught: ignoring the preference, defaulting it ON, and building - the note once without re-wording it. -- Both READMEs updated; the old flat claim "Statistics and Connections show ALL captured traffic" - now says "by default" and spells out what does not follow the switch. - -### Tests: every gate that judges the REMOTE end is pinned in BOTH directions - -- `engine._capture_loop` reads the remote endpoint as the packet's DESTINATION when outbound and - as its SOURCE when inbound. **Five** features consume it: destination IP, destination port, LAN - mode, block by IP, block by port. -- 🔴 **Measured, not suspected: that rule was effectively unguarded.** Mutating the inbound branch - to read `dst_addr` - the tidy-up anyone would make while cleaning that block - was caught by - **three tests, all about the CONNECTION LOG**. Not one test about targeting, LAN mode or blocking - failed. The tool would have kept impairing outbound traffic and quietly stopped impairing - everything coming back, every counter healthy, with the only red in rows somebody could - plausibly have "fixed" by adjusting the expected output. -- New `tests/test_remote_endpoint_directions.py`, table-driven on the CONSUMERS rather than on one - feature (same shape as `GATES` in `test_core_properties.py`): adding a sixth reader of the remote - endpoint and forgetting the inbound case now fails immediately. A mirror test keeps the first - honest - a gate that simply said "yes" would satisfy "fires in both directions". -- 🔴 **The fixture was hiding an entire axis.** `FakePacket` set `src_port` and `dst_port` to the - SAME value, so swapping the inbound branch's two ports changed nothing anywhere in the suite - - the port half of the assertion looked sound and tested nothing. `FakePacket` now takes - `src_port`/`dst_port` separately (defaulting to `port`, so every existing test is unchanged), and - the new file uses a local port that differs from the peer's. Three direction mutants are caught - now: inbound address, inbound ports, outbound ports. -- Also worth recording because it cost a run: `is_local_ip` counts the TEST-NET documentation - ranges (203.0.113.x, 198.51.100.x) as LOCAL - they are not globally routable - so reaching for - one of them, the obvious choice in a test, makes LAN mode do nothing and reads as a bug in the - gate. The file uses genuinely routable addresses and says why. - -### Measured (no code): WinDivertSendEx rejects a bad batch ATOMICALLY - -- Open question left over from the batching work, and the thing that decided whether `drop_send` - could stay honest under a batched injector: does `SendEx` send a PREFIX of the batch before it - hits a bad entry, or nothing at all? -- Measured 2026-07-29 by calling the DLL directly (pydivert's wrapper raises before `pSendLen` can - be read), with a deliberately truncated packet placed in the MIDDLE of a batch of four: - - | batch | rc | `pSendLen` | last error | - |---|---|---|---| - | four good packets | 1 | 240 of 240 | 0 | - | one truncated entry in the middle | 0 | **0** of 190 | 122 (`ERROR_INSUFFICIENT_BUFFER`) | - - 120 bytes of perfectly good packets sat before the bad one and **none of them went out**. So the - call is all-or-nothing: on failure a batched injector charges every packet in the batch to - `drop_send` and there is no partial-send ambiguity to account for. That removes the one blocker - named against chunk 5. -- Recorded here rather than in code because no code changed yet - the next session should not have - to re-derive it. - -### Added: `--narrow-filter` folds the destination into the driver's filter (chunk 2) - -- `filters.narrowed_filter(base, ip_matcher, port_matcher)` -> `(text, narrowed)`, and - `filters.filter_compiles(text)` asking `WinDivertHelperCompileFilter` (lazy `pydivert` import - - win32-only dependency, and `False` when it cannot be asked, because "cannot prove" must mean - "keep the wide filter"). -- Registry field `narrow_filter` (BOOL, `start_only`, `surface="settings"`), so the CLI flag, the - widget, validation and the config file all fall out of one entry (convention 11). Wired through - `engine.start(..., narrow=)` for the same reason `duration` is a `start()` parameter: a handle's - filter is fixed when it opens. -- **Only on the REAL path.** An injected divert (`--simulate`, tests) never reads the filter - string, so narrowing there would move `session.narrowed` in the report without changing a single - packet - a cosmetic lie, which is the class of thing this audit has been removing. -- 🔴 **The mid-session trap, and the guard for it.** `dst_ip`/`dst_port` are live-appliable while - the handle's filter is not. Accepting a destination change during a narrowed session would leave - the DRIVER filtering by the old expression while `decide()` judged by the new one - traffic the - user just asked to impair would never arrive, every counter healthy. `apply_settings` now refuses - a CHANGED destination while narrowed and says so (`log.dest_frozen_while_narrowed`); re-applying - the same value is untouched, because that is what every "Apply changes" does. -- **Reported, not just done.** `session_info()["narrowed"]` (so the repro report carries it - it - embeds the whole session dict) and a new `capture_narrowed` key in the NDJSON summary. Additive - to a frozen contract, and load-bearing: with narrowing on, `packets` no longer counts every - packet on the machine, so two reports with the same key describe two different worlds. The CLI - also says at START whether the narrowing took effect - a user who asked for the throughput and - silently got the wide filter would otherwise believe they had it. -- **A mutant survived and found a real hole in the tests.** Removing the compile check went GREEN: - nothing in the file produced a fragment the driver would refuse. It exists - the grammar has a - length limit, and a plausible expression hits it. Measured here: a 50-port list (4 698 chars) - compiles, a 100-port list (9 398) does not. Without the check the tool would hand WinDivert an - unparseable filter and fail to open the handle at START. Now guarded by - `test_a_fragment_the_driver_refuses_falls_back_instead_of_being_used`. -- Six mutants: five caught, one green ON PURPOSE (the `filter_compiles` exception path is - unreachable where pydivert exists, which is where the suite runs). -- **Consumers found the slow way, worth recording:** `engine.start`'s signature has five test - doubles across `test_failsafe.py`, `test_gui_stack_chaos.py` and `test_gui_state.py`. They took - `**kw` afterwards. The "who consumes this" sweep in PROJECT_NOTES rule 2 should have listed test - doubles, and now it does by example. -- 🔴 **ACCEPTANCE on a real capture (2026-07-29), and it is a CORRECTNESS result, not a speed one.** - A flood to the targeted destination plus a decoy flood outside it: - - | run | `capture_narrowed` | `seen` | `scoped_seen` | `drop_loss` | sent to target | receiver got | - |---|---|---|---|---|---|---| - | wide, no impairment | False | 75 056 | 15 768 | 0 | 28 050 | **15 768** | - | narrow, no impairment | True | 27 950 | 27 950 | 0 | 27 950 | **27 950** | - | wide, `--loss 100` | False | 88 943 | 21 540 | 21 540 | 28 050 | 0 | - | narrow, `--loss 100` | True | 27 900 | 27 900 | 27 900 | 27 900 | 0 | - - Without narrowing the driver was overloaded by traffic the tool was never going to touch and - **discarded 43% of the TARGETED traffic before the tool saw it** - the session impaired less than - it claimed AND computed its numbers over the survivors. Narrowed: nothing lost, `seen` equals - `scoped_seen`. The driver-wait warning fired in both wide runs (52 / 184 ms) and in neither - narrowed one, which is the F18 overload seen from the other side. -- **The regression worth fearing did not happen:** with `--loss 100` the narrowed run dropped - 27 900 of 27 900 and the receiver got nothing, exactly as the wide run did. A narrowing that also - stopped the impairing would have read as a win in every counter. - -### Added: expressions compile down into the DRIVER's filter (chunk 1 of the narrowing work) - -- **Why.** With a destination target set the tool captures everything and re-injects almost all of - it untouched - measured 1944 packets diverted with 0 impairable, and 1632 with 8. Each cost a - recv plus a send for nothing. The WinDivert filter runs in the driver, so whatever can be pushed - into it never reaches this process. -- `matchers.windivert_fragment(matcher)` emits a filter fragment or `None`. Every `_Term` now - carries a `shape` - what the parser concluded (`("eq", 443)`, `("ip_range", 4, lo, hi)`, `None` - for glob / `re:` / process name). The parsers return `(predicate, shape)` instead of throwing the - parse away, so the compiler **reads the parse result rather than parsing the text again** - (convention 10: a second reader of the same syntax drifts at the first edit). The hot path is - untouched - `matches()` still calls the closure. -- **The invariant is the whole design: the fragment must be a SUPERSET of the matcher.** - Over-capture is free, under-capture is the silent regression. So: only POSITIVES are compiled - (dropping negatives can only widen), one shapeless term voids the whole expression (an unbounded - member makes the OR unbounded), and no address-dependent term (`outbound`, `loopback`) is emitted - - those live in `WINDIVERT_ADDRESS`, so keeping them out is what lets the guard evaluate a - synthetic packet and still mean something. -- 🔴 **A real bug caught while tracing writes, not while reading names.** `engine._capture_loop` - reads the remote endpoint as the packet's DESTINATION when outbound and as its SOURCE when - inbound. A fragment testing only `DstAddr`/`DstPort` would have kept every INBOUND packet away - from the tool - impairing one direction, silently dropping the other, all counters healthy. - Both directions are emitted, and `test_both_directions_are_covered_because_the_remote_end_swaps` - exists so it cannot come back. -- **Verified against the driver's own evaluator, not against my reading of it.** - `WinDivertHelperEvalFilter` was first checked for fidelity on **40 real captured packets across 9 - filters, zero disagreements**, including the address-dependent terms that a blank address struct - would have got wrong (sniff-only handle - it cannot touch traffic). Then every emitted fragment - was compile-tested and swept: **27 648 packet views over 16 expressions, zero superset - violations**. -- New guards in `tests/test_matchers_windivert.py`. The oracle test needs `pydivert` and is skipped - where it is missing - **half the CI matrix**, so a green Linux run has not checked the invariant. -- **Five mutants. Four caught; the fifth green ON PURPOSE:** emitting only a range's lower bound is - a WIDENING, and the tests must not reject it. That one is the check that the suite encodes the - asymmetry rather than just "any change is bad". The four caught: dst-only fields, negatives - compiled, a shapeless term no longer voiding the expression, and an `outbound` term leaking in. - -### Docs: the throughput rig is not reproducible, and the prose now says so (audit F18, follow-up) - -- Re-measuring the same bare `recv`+`send` loop over the same loopback flood, same machine, three - times, gave **14 488 -> 30 189 -> 15 004 pkt/s**. A factor of two, unexplained. Ratios moved too: - batched-vs-unbatched ran from 1.23x to 3.65x, and "real NIC vs loopback" **changed sign** between - runs (0.67x -> 1.32x). One run produced a physically impossible ordering - a strategy doing - strictly LESS work measured slower - which is itself a reading of the noise floor. -- `engine.py`'s "What this actually sustains" section now carries a **HOW MUCH OF THIS TO TRUST** - paragraph: a number from this rig is evidence only against another number from the SAME run. -- **Withdrew a derived claim that was never sound:** the section previously divided a sniff-only - reader's 92 944 pkt/s by the engine's ~14k, concluded "five sixths of the per-packet budget is - spent after the read", and pointed at the inject side. Those two figures come from different - runs. What remains is the release heap growing under ZERO configured delay within one run - (`peak_queue` 4080 -> 7020) - a direction, not a proportion. -- **Kept, with the distinction made explicit:** direct OBSERVATIONS survive where timings do not - - a loss percentage against a control that lost 0.00%, a queue depth, `scoped_seen`, and the 1.4 - packets per `RecvEx` call that carries the batching ADR. The ADR is amended in place rather than - reversed, because its argument never rested on the timing. -- The measurement-method lesson went into PROJECT_NOTES rule 5 (third variant of the same trap): - repeat the WHOLE run before believing a number; interleave strategies instead of running each in - a block; carry a drift canary that makes the script REFUSE a conclusion rather than print a - median; report spread, not just the median. - -### ADR 2026-07-28: batched WinDivert I/O MEASURED and REJECTED (audit F18, part c) - -- **The question.** The engine does one `recv()` and one `send()` syscall per packet, and WinDivert - exposes `WinDivertRecvEx`/`SendEx`, which can carry many packets per call. `pydivert` binds both - (its own wrapper uses them only for overlapped IO, single packet), so raw ctypes could batch - - the project already writes raw ctypes in `portmap.py`. Rule 6 says price the primitive before - building anything around it. -- **The bar was set BEFORE the run**, so the result could not be argued into: below 2x, do not touch - the capture loop (the most safety-critical code here, convention 20); 3x or better, worth a real - design; between the two, record and leave open. -- **Measured** (elevated, sniff-only `SNIFF|RECV_ONLY` handle on `loopback and udp and - udp.DstPort == 39999`, same 64 B flood for all three, median of 5): - - | how the same traffic is read | pkt/s | vs today | packets per call | - |---|---|---|---| - | `handle.recv()` - syscall + `pydivert` Packet object | 92 944 | 1.00x | 1.0 | - | raw `WinDivertRecv` - syscall, no Packet object | 120 356 | 1.29x | 1.0 | - | raw `WinDivertRecvEx` - up to 64 packets per call | 130 528 | **1.40x** | **1.4** | - -- **Rejected, and the reason is not the wrapper:** the driver hands back ~1.4 packets per call even - under a saturating flood against a full queue. There is no batch there to exploit, so the syscall - count is not what to attack. **Do not reopen this without new evidence that the driver will - actually fill a batch.** -- **AMENDED 2026-07-28 (same day):** the packets-per-call figure is a direct observation and - stands; the 1.29x / 1.30x TIMINGS in the table came from a rig later shown to be unreliable - - the same bare loop re-measured 14 488 / 30 189 / 15 004 pkt/s across three runs. The conclusion - does not depend on them (1.4 packets per call is the whole argument), but do not quote those - ratios. See the "HOW MUCH OF THIS TO TRUST" paragraph in `engine.py`. -- **What the run redirected the question to.** A sniff-only reader sustains 92 944 packets/s while - the whole engine manages ~14k - **6.6x**. Reading is not the ceiling; roughly five sixths of the - per-packet budget goes to `decide()`, the connection log, the release heap and the inject - thread's `send()`. Which of those dominates is NOT measured, and saying so is the point: the - evidence supports "the inject side trails" (the heap grows under zero configured delay) and - nothing finer. -- **Anomaly recorded rather than smoothed over:** runs 4 and 5 of the RecvEx strategy collapsed to - 29 010 / 29 769 pkt/s against 130-132k in runs 1-3. Unexplained. The median absorbs it and the - conclusion does not turn on it (even the best RecvEx run is 1.42x), but a future session - re-running this should expect it rather than treat it as a new discovery. -- Prediction log: `recv()` was predicted at 30-80k and came in at 93k; the ordering 3 > 2 > 1 held, - the magnitudes did not. - -### Docs: "150 000 packets a second" is a SYNTHETIC number, and now says so (audit F18, part b) - -- The figure appears in five places (`core.py`, `crashlog.py` x2, `engine.py`, `targeting.py`) as - the working assumption for hot-path cost. All of it came from benchmarks against - `SyntheticDivert`, where a packet costs no syscall and no `pydivert` parsing. A real session was - measured at **~13.8-15.6k packets/s** end to end (capture + re-injection, nothing configured to - impair) - an order of magnitude lower, with nothing in the text distinguishing the two. -- The number now lives in ONE place: a "What this actually sustains" section in `engine.py`'s module - docstring, with the measurement conditions and, deliberately, what was NOT measured (idle machine, - real NIC, full-size frames - 14k packets/s is ~7 Mbit/s at 64 B but ~170 Mbit/s at 1500 B; the - limit is packets, not bits). -- **The ceiling is FLAT**, which is its own control: three times the offered load moved throughput - by 13%. The load generators share the machine, so CPU contention is a confound - but if it were - the binding one, MORE senders would push the tool's rate down, and it went slightly up. -- **The bottleneck is INJECTION, not `decide()`.** With no impairment configured the release heap - still grew ~1100 entries/s (`peak_queue` 4080 -> 7020), which can only happen if the inject thread - trails the capture thread. Every hot-path optimisation recorded in this package targets `decide()` - and the capture thread; the measured constraint is the `send()` side. -- **Deliberately NOT rewritten:** `core.py::_enforce_ceiling` ("measured at 150 000 new flows/s") - describes a benchmark's INPUT rate, not a claim about production throughput, and `crashlog.quiet` - / `engine.OVERFLOW_WARN_S` use the figure as a conditional cost bound ("would be a lot at that - rate"), which holds either way. Rule 5 cuts both directions: prose that is still true for a - different reason than the one being corrected does not get deleted along with it. - -### Fixed: the driver-wait warning names the LOSS, not just the delay (audit F18, part a) - -- **Measured first** (2026-07-28, elevated, real WinDivert, `--filter loopback` so nothing but - loopback is ever diverted, 64 B UDP flood, no impairment configured). Control passes with no tool - at all lost **0.00%** at both 138k and 303k packets/s, so the socket path itself is not the story: - - | offered | arrived | lost | `seen` | `driver_wait_peak_ms` | `peak_queue` | - |---|---|---|---|---|---| - | 666 000 | 54 957 | **91.75%** | 55 201 | 198.3 | 4 080 | - | 1 263 600 | 60 677 | **95.20%** | 60 977 | 345.6 | 4 722 | - | 1 767 600 | 58 706 | **96.68%** | 62 315 | 220.5 | 7 020 | - - `drop_overflow` / `drop_shutdown` / `drop_send` / `drop_loss` were **0 in every row**. The tool - reported a healthy session while over nine tenths of the user's traffic was destroyed. -- **No counter can exist for it, and that is now checked rather than assumed** (rule 6): - `WinDivert64.dll` exports `WinDivertGetParam`/`SetParam` and no statistics call; params 0-4 answer - (queue length / time / size, version major / minor) and 5-7 are refused. So the driver-wait - warning is the ONLY signal the tool has for this state - which is why its wording is load-bearing. -- Changed `log.driver_wait`, `events.driver_wait` and `log.driver_queue` in `lang/en.json` + - `lang/pl.json`: a full queue means DROPPED packets, not just late ones, and the advice is to - narrow the traffic filter. `_warn_driver_wait`'s docstring carries the measurement. -- **Why the threshold is where it is, now written down:** under saturation the queue sits at its - limit, so the wait converges on `QUEUE_LEN / service rate`; at 4096 and ~14k/s that is ~290 ms, - and the run measured 198-346 ms. `DRIVER_WAIT_WARN_MS = 50` is therefore crossed whenever the - tool serves below ~82k packets/s. Recorded so nobody raises it without redoing that arithmetic. -- Prediction log, since being wrong is the useful part: two of four predictions failed. Loopback - was expected to be counted TWICE (`seen` ~ 2x arrived) - measured `seen` ~ arrived, one capture - event per datagram. And the warning was predicted NOT to fire (wait pinned under 50 ms) - it - fired in all three runs, because the service rate put into that formula was 10x too high. - -### Docs: what a target restart actually costs, and one README claim it falsifies (audit F17) - -- `targeting.py::syn_covers` documented the `_pids` limit with 19/20 (holding sockets) and 6/20 - (closing them), but said nothing about a RESTART, which is the same mechanism from the other - side. Measured 2026-07-28 (elevated, real WinDivert, `--target beanprobe.exe --dst-ip 8.8.8.8 - --dst-port 53 --syn-drop 100`, three lives of four held connections): `OK FAIL FAIL FAIL` in - **3 of 3** lives, so a restart costs exactly ONE connection and then recovers by itself. Same - probe by PID, killed and restarted: **5 of 5** untouched, because the number no longer exists. - The recovery belongs to the EXPRESSION, not to `syn_covers`, and the docstring now says which. -- **The "cannot be closed" claim is now argued rather than asserted** (rule 6): the SOCKET event - leads the SYN by 0.018-0.027 ms, covering a brand-new process needs pid -> name plus the matcher - inside that window, and a COLD name resolve is milliseconds. Moving it to the watcher thread does - not change the window. The only design that closes it holds the SYN until the answer is in, which - is added delay in a tool that exists to inject a precise amount of it. Named so the next session - does not re-derive it - and so the alternative that WOULD work is on record. -- **A README sentence measured false and fixed in both languages.** "A connection is in scope the - moment it opens - for an outbound connection, before its first packet even leaves" is true for a - process already in `_pids` and false for that process's first connection, which is precisely the - case the measurement isolates. Both READMEs now state the exception and carry a new bullet on - name-vs-pid under restart. `test_readme_guards.py` / `test_cli_docs.py` green (this touches - neither the pipeline order nor the flag tables). - -### Tests: the GUI target banner is pinned to the PROCESS, not to the field (audit F17) - -- **The gap.** `test_gui_state.py::test_a_gui_session_keeps_the_target_banner_honest` only ever - moves the EXPRESSION - the user types something that matches nothing, then something that does. - The case a tester actually hits moves the other end: the field is left alone and the targeted - program exits, or a harness restarts it onto a new pid. Nothing covered that, and a handoff note - had concluded from reading the code that the verdict was taken only at session start. It is not: - `App._refresh_target` re-reads `targeting.matched` on every tick and only the APPLY half is - gated on the expression having changed. -- New guard: `::test_a_target_that_dies_mid_session_raises_the_banner_without_being_retyped`. Real - engine, real resolver, `SyntheticDivert`, injected `FakeTable`; the test empties the socket table - under a running session and asserts `_applied_target` did NOT move (otherwise it is the old test - again), then that the banner rises, then that it comes back DOWN when the process reappears under - a NEW pid with the same name - the recovery measured against a real capture the same day. -- **Three mutants, all caught, each on its intended assertion.** The one worth recording is the - first: `_refresh_target` made to read the verdict ONLY when the expression changed, i.e. exactly - the "banner appears at session start" behaviour the note had assumed already existed. It goes - red, so the guard really does pin the live re-read. The other two: `ProcessTargeting.matched` - forced to `True`, and the banner never taken back down. -- Test-only chunk, so no `CHANGELOG.md` entry (convention 39): nothing on screen changed, this - fixes the fact that nothing was watching it. - -### Added: the CLI reports a target that stops matching, and what share was in scope (audit F17) - -- **Symptom, measured before writing anything** (2026-07-28, elevated, real WinDivert, probe - narrowed to `--target --dst-ip 8.8.8.8 --dst-port 53 --syn-drop 100`). A process was - targeted BY PID, impaired correctly (1 of 6 connections slipped, the rest timed out), then killed - and restarted under a new pid: **5 of 5 fresh connections untouched**, `scoped_seen` flat, and the - only targeting line in the whole run was the one `apply_targeting` prints at start. `exit=OK`. -- **Same probe, targeting BY NAME**, 3 lives x 4 held connections: `OK FAIL FAIL FAIL` in **3 of 3** - lives. So the name path recovers by itself and the restart costs exactly one connection - the one - opened before the process owns any socket, which is the `_pids` limitation F16 already documents, - not a new one. `drop_syn` cross-checks at exactly 2 per caught connection (SYN + one retransmit - inside the 2.5 s connect timeout). -- **The asymmetry this exposes.** `gui/app.py::_refresh_target` re-reads `targeting.matched` on - every tick and raises `fields.target_no_match`, so the GUI has always shouted about this. The CLI - - the CI/CD interface (convention 18) - resolved once in `apply_targeting(announce=True)` and - never looked again; `_report_loop` did not mention targeting at all. -- **Fix, all in `cli.py`.** `_targeting_state(engine)` returns `(matched, describe())` or `None`, - and `_report_loop` logs only the TRANSITION: `warn` when the target stops matching, `info` when it - comes back. Reading `matched` is a plain bool on the live `ProcessTargeting` - no lock, no - syscall, no socket table - so the loop can ask on every pass. `getattr` on `engine.targeting` - because `run_cli(engine=...)` is a public seam. Sampled, not continuous: a verdict that flips and - flips back between two passes is not seen, and the comment says so rather than implying otherwise. -- **End of run:** a `warn` when a target was set, traffic WAS captured and `scoped_seen` is 0, plus - an `In scope: X of Y captured packets` line in the text summary. Guarded by `stats["seen"]` on - purpose - with nothing captured at all the capture filter is the story and `--min-packets` is the - flag that tells it, so saying both would point at the wrong thing. -- **Deliberately NOT done:** no new exit code and no `--min-scoped` flag. A target with no traffic - of its own is a legitimate run, and making it an assertion would change the exit-code contract - under everyone already running one. The number is in the JSON summary's `counters.scoped_seen` for - a pipeline that wants to assert on it itself. The NDJSON `sample` schema is untouched (the frozen - contract); the new line goes down the TEXT channel only. -- Two new guards in `tests/test_cli_runtime.py`: - `::test_the_run_says_when_the_process_target_stops_matching` (transition reported, reported ONCE, - and silent while the target keeps matching) and - `::test_a_target_that_caught_nothing_is_called_out_at_the_end` (zero scope called out, non-zero - scope not accused, no-traffic-at-all not blamed on the target). Both drive a `_TargetedEngine` - fake: a real engine cannot play this part, since `--target` is stripped under `--simulate` and a - real capture needs WinDivert plus elevation. `winenv.is_admin` is monkeypatched so the pair cannot - become a THIRD environment-dependent result in this file. -- The fake's stats dict is copied from a real `BeanEngine().st`, so a counter added to the engine - cannot leave it answering with a key the CLI reads. -- **Five mutants, all caught** (source rewritten as BYTES - `write_text` would flip the file to CRLF - and trip the changelog hook): transition logging removed; transition logging fired every pass - instead of on change; the zero-scope warning removed; its `stats["seen"]` guard removed; the - `In scope` line removed. Each went red on its intended assertion. - -### Fixed: the first packet of a fresh connection can be in targeting scope (audit F16) - -- **Symptom, measured end to end before writing anything.** `ProcessTargeting.__contains__` answers - from `_ports`, a frozenset rebuilt on the resolver's thread, so a socket opened microseconds ago - is unknown there - and the first packet of a connection is judged BEFORE any rebuild it triggers. - 20 fresh connections against a process target with `--syn-drop 100`: **20 established, `drop_syn` - 0**. `--syn-drop` combined with `--target` was a complete no-op, and every other impairment missed - one packet per connection. -- **This is NOT the 0.02 ms.** The SOCKET-layer margin measured the same day is real, but nothing - consumed it: the live map fed the REBUILD, not the packet-path test. The order being in our favour - is worth nothing until something reads it at the right moment. -- **Fix.** `ProcessTargeting.syn_covers(port)` - a lock-free `pid_for` read checked against `_pids`, - the set the last rebuild concluded - and `BeanCore.decide` step 1 calls it **for a TCP SYN only**, - i.e. once per connection rather than once per packet. `_syn_covers` is bound in `set_target`, so - the packet path does not even do a `getattr`, and a plain port set (tests, one-shot resolution) - binds `None` and keeps exactly the behaviour it had. -- **It drags a second fix with it, and this is the part that would have been missed.** Step 4 arms a - reset on the first in-scope TCP packet. With SYNs now in scope that becomes the SYN - and the RST - forged from a SYN copies its `ack_num` as the sequence, which a SYN does not have, so it goes out - with `seq=0` and no ACK. RFC 793 lets a stack in SYN_SENT ignore that, and **it was measured doing - exactly that** (2026-07-28: the client hung until its own timeout, `rst_sent` reported 1). So a - reset is no longer ARMED from a SYN, while a SYN arriving inside an existing cooldown is still - held down. Without this, F16 would have traded a working reset for a hang. -- **Acceptance, and it corrected the limitation as written.** Predicted 19 of 20 blocked; the first - run gave **6**. The gap is not the 0.02 ms: `_pids` is rebuilt from pids owning CURRENTLY OPEN - sockets, and the probe connected, closed at once and idled 0.2 s, so a rebuild landing in that - gap dropped the process out again. A second probe **holding its sockets open** was blocked **19 - of 20** (`drop_syn` 38 - each connection's SYN plus its retransmit), the single escape being - attempt 0, before the process had any socket at all. Two candidate causes existed and this - separated them: it is `_pids` churn, not the watcher failing to process the event in time. -- **So the documented limit was too narrow and is now precise:** not "the first connection of a - freshly started target" but "any target with no open socket when a rebuild runs". A browser or an - app under test is covered; a script opening one connection, closing it and pausing keeps slipping - through. Both measured numbers (6/20 closing, 19/20 holding) are in the docstring, because the - difference between them IS the limitation. -- Also: `_pids` is up to one resolver cycle stale, so a recycled PID can pull one packet of an - unrelated socket into scope - a DIFFERENT false positive from the stale `_ports` this code already - lived with. UDP has no SYN and is not covered. -- **Contract widened on purpose:** `set_table` documented `snapshot`/`name_of`/`ancestors`/`refresh`; - `pid_for` is now part of it. Both real tables always had it, but `syn_covers` reads it through - `getattr` so a table that does not - a test double, an older implementation - answers False - instead of raising `AttributeError` **on the capture thread**. -- **Hot path measured** against a worktree of master (150k 1500 B packets, median of 5). No - targeting: 135.9k -> 140.3k pkt/s. **Targeting on with every packet missing** - the common path - when a target is set, and where the new test lives: 164.0/164.7k -> 164.5/170.4k. No regression - either way; the added cost is one boolean that short-circuits on `is_syn`, below the noise floor. -- Six new guards across `tests/test_core.py` and `tests/test_targeting_socketwatch.py`; **six - mutants, every one caught after a repair worth recording** - two of them first pointed at - `test_core.py`, which drives a local fake rather than `ProcessTargeting`, so the real `getattr` - guard was not covered by anything. It has its own test now. -- `test_core_properties.py::test_an_armed_gate_wins_over_every_later_step` drove every gate with - `is_syn=True`; the rst gate now needs an ordinary packet. The test states why, rather than being - quietly relaxed. - -### Docs: the SOCKET-layer timing now says what was measured (prose only, no behaviour change) - -The whole SOCKET-layer targeting design (PR #38) is justified by two numbers that came from a -2026-07-22 spike note and were never re-measured. Both are wrong, in different ways, and the -conclusion drawn from the first claimed more than it supports. - -**Re-measured 2026-07-28** (Win11, elevated, three sniff-only handles opened SEQUENTIALLY and -compared on one clock - the QPC stamp WinDivert puts on every event - across 10 outbound TCP -connections to 8.8.8.8:53): - -| claim in the code | measured | -|---|---| -| `SOCKET_CONNECT` ~0.1 ms before the SYN | before in **10/10**, by **0.018-0.027 ms** (median 0.020) | -| `FLOW_ESTABLISHED` ~28 ms after | **37.7-41.3 ms** after (median 38.7) | - -- **The ORDER holds, the margin does not.** `SOCKET_CONNECT` really does precede the SYN every - time, so choosing the SOCKET layer over polling stands. But the margin is **five times smaller** - than the docstring said. -- **"The race is closed at the source" is now cut.** Those tens of microseconds are the gap between - the two events AT THE DRIVER. Whether `socketwatch`'s own handling - thread wake, parse, dict - insert - finishes inside that gap has **not** been measured, and at 20 us it is no longer - self-evident the way it looked at 100 us. The prose now guarantees ordering and explicitly does - not guarantee slack. -- **The FLOW number was a property of somebody's network, not of the FLOW layer.** 38.7 ms tracks - the round trip to the peer (ping to the same host on this link: 23-47 ms), because the flow is - established once the handshake completes. Recorded as such, so nobody reads it as a constant. - The reason for rejecting FLOW is unchanged: it lands after the handshake either way. - -Corrected in `socketwatch.py` (module docstring), `engine.py::_pid_for`, `targeting.py` (module -docstring) and PROJECT_NOTES in three places. **Deliberately NOT touched:** the `~0.1 ms` in -`model_worker.py`, `gui/panels/event_log.py` and `gui/pages/conns.py` - that is the virtualised -table's repaint cost, a different measurement that happens to share a number. Historical changelog -entries are left as written; they record what was believed at the time. - -No test guards this and none can - it is prose. What can be done was done: the numbers now carry -their conditions, so the next session can tell whether they still apply to its machine. - -### Fixed: the forged RST now lands on loopback connections too (audit F15) - -Measured on the owner's machine (2026-07-28, elevated, real driver) by watching what the -APPLICATION sees rather than what a counter says: - -- **Non-loopback: the RST works.** DNS over TCP to 8.8.8.8:53, connection established and - exchanging, tool started after 6 s -> the client got `ConnectionResetError` (WinError 10054) at - **6.6 s** after 26 answered queries, with `rst=1/1`. This closes the audit's oldest unverified - claim, that `rst_sent` only proves `send()` did not raise. It proves more than that: Windows - accepts the packet `_build_rst_packet` forges, and the application's connection really dies. -- **Loopback: it does not.** The same shape against an own echo server on 127.0.0.1 gave 26 - exchanges, then silence, then the client's own 3 s timeout - `TIMED OUT`, never a reset, while - `rst=5/1` (five packets blackholed during the cooldown, one RST "sent"). So with the `loopback` - traffic filter - which the GUI offers - "Reset connections" silently degrades to a blackhole and - still reports an RST as sent. -- **A measurement trap worth keeping.** The first attempt fired on the SYN, because `--rst-prob 100` - catches the first packet of a flow, and a bare RST with `seq=0` and no ACK is ignored in SYN_SENT - (RFC 793). That run measured the SYN case, not the established one. When testing behaviour ON a - connection: establish it, pass some traffic, and only then switch the impairment on. - -**Two hypotheses, one measurement each, and the first was wrong.** - -1. `_build_rst_packet` builds a fresh `pydivert.Packet`, whose address starts with `Loopback=0`, so - the flag was carried over from the provoking packet. **One line on purpose** - changing the - direction at the same time would have made a success ambiguous. Re-ran the probe: - `TIMED OUT at 9.5s after 26 exchanges`, byte for byte the baseline. **Falsified.** -2. Rather than guess again, the mechanism was measured: a sniff-only handle on `loopback and tcp` - printed every packet of a real 127.0.0.1 conversation as **`outbound=1, loopback=1`, exactly - once each - the server's replies included.** Loopback has no inbound presentation at all, so an - RST injected as `Direction.INBOUND` was put on a path the stack never reads. The RST is now - built to look exactly like those captured rows: OUTBOUND, loopback flag set, for loopback - packets only. Ordinary traffic keeps INBOUND, which is measured to work. - -**Result:** `CONNECTION RESET at 6.5s after 26 exchanges` (WinError 10054), and the tool's own -counters moved from `rst=5/1` to `rst=1/1` - independent confirmation, since a connection that dies -at once has no further packets to swallow during the cooldown. - -New test: `tests/test_rst_local.py::test_a_loopback_rst_is_injected_the_way_loopback_packets_travel`, -asserting direction and flag for both cases against a hand-built IPv4+TCP ACK. Conditional on -pydivert (win32-only), the same shape as `test_the_driver_queue_param_numbers_match_pydivert`. -Three mutants, every one caught - including "every RST becomes a loopback one", which would have -broken the ordinary path this fix must not touch. - -### Added: the wait inside the driver is measured, not guessed (audit F10, part b) - -- **The audit asked for an "overload heuristic". None was written, because the dependency already - answers the question.** `pydivert.Packet.timestamp` is the raw QueryPerformanceCounter value - WinDivert stamps each packet with (`packet/__init__.py:216`), so `QPC_now - stamp` IS the time - that packet spent in the driver's queue. Reading the capability surface first (convention 6) - turned a heuristic into a measurement. -- `winenv.qpc_now()` / `qpc_frequency()` (Windows-only, `None` elsewhere). Deliberately not - `time.perf_counter()`: it is derived from QPC but carries an arbitrary epoch offset, so - subtracting a raw stamp from it is meaningless. -- **Sampled on TIME, not on a packet count.** A 1-in-N sample never fires on a quiet link - 24 - packets in 12 seconds would never reach 1-in-256 - and a quiet link is exactly where a 2 s driver - queue would go unnoticed. `DRIVER_WAIT_SAMPLE_S = 0.05` gives 20 samples a second at any rate, - and the per-packet cost is one float compare against a `now` the loop had already read. -- `driver_wait_peak_ms` in `st` (the only float in there), a Session-tab row, the repro report, the - stats CSV. `DRIVER_WAIT_WARN_MS = 50.0` - chosen against the MEASURED idle value on the owner's - machine (0.049-0.163 ms, median ~0.08), so the threshold is several hundred times normal rather - than a guess at one. Warning rate-limited at 5 s like the other two, with the first occurrence in - the event log so the repro report carries it. -- **The QPC pair is an injected dependency**, like the clock and sleep `run_cli` takes: `start()` - reads the frequency only `if self._qpc_freq is None`, so a test can drive the capture loop with a - known clock. -- **Hot path measured** against a worktree of the F10a branch (150k packets, median of 5, twice - each): 148.4 / 149.6k -> 154.1 / 154.4k pkt/s. A float compare cannot make anything faster, so - that spread is noise and the honest reading is **no measurable cost**. NOT covered by this - benchmark: the QPC call itself - synthetic packets have no `timestamp`, so the sampler returns at - the `getattr`. On the real path that call happens 20 times a second regardless of traffic. -- Five new tests; **five mutants, four caught, and the fifth is recorded rather than papered over**: - - mutating away `if waited_ms <= 0` changes nothing observable, because the peak comparison and - the warn comparison both already reject a negative. The branch is a cheap skip of the stats - lock, not a guard, and its comment now says exactly that instead of claiming it prevents a - negative reading. - - two mutants were caught only after repairs. The direct-call tests said nothing about whether - the capture loop ever REACHES the sampler - deleting the call site left them all green - so - `test_the_capture_loop_actually_takes_the_sample` runs a real session with the QPC injected. - And it injects **1 MHz, not this machine's real 10 MHz**: with a matching frequency, a mutant - that lets `start()` overwrite the injected value is invisible on Windows and only breaks on the - Linux runner. Same trap as the pydivert import in part a, from the other side. - -### Added: the driver's own queue is read and reported (audit F10, part a) - -- **Symptom.** `pydivert.WinDivert(filt)` never received a `set_param()`, and the package contained - no `get_param()` at all, so WinDivert's queue was invisible: with `QUEUE_TIME` at its default a - packet may be held for up to **two seconds**, which is latency this tool adds while being blind - to it, and no session artefact said what queue produced its numbers. -- **Measured on the owner's machine before designing anything** (2026-07-28, elevated, real driver, - throwaway ICMP handle): `QUEUE_LEN=4096`, `QUEUE_TIME=2000`, `QUEUE_SIZE=4194304`, QPC frequency - 10 MHz. `QUEUE_SIZE` binds before `QUEUE_LEN` for full frames - 4 MiB / 1500 B = **2796 packets** - - so at a saturated gigabit (83k pkt/s of 1500 B) a frozen capture thread has roughly 34 ms - before the driver starts discarding. -- **Deliberately NOT retuned.** The trade is real in both directions and both are invisible today: - a large `QUEUE_TIME` is latency the tool hides, a small one is LOSS the tool cannot even count, - because a packet the driver drops never reaches us. Picking values before measuring the actual - queue delay would be the "machinery around the wrong primitive" convention 6 warns about. The - measurement is part b. -- `BeanEngine._read_driver_queue()` reads the three params after the handle opens (guarded by - `crashlog.quiet`), stores them, and `session_info()` carries them into the repro report as - `session.driver_queue`; one `log.driver_queue` line at START. `None` - not zeroes - on the - simulate path, because "no driver" and "a queue of nothing" are different claims. -- **The param numbers are spelled out, not imported, and that is the interesting bit.** Reaching - for `pydivert.consts.Param` here would import a **win32-only** dependency: the read would return - None on the Linux half of the CI matrix while passing on Windows - the exact bug shape that only - appears on the runner you did not run. `DRIVER_QUEUE_PARAMS` holds the ABI numbers, and - `tests/test_engine.py::test_the_driver_queue_param_numbers_match_pydivert` compares them with the - real enum wherever pydivert IS importable, so they cannot drift in silence. -- `--doctor` gains a line that says where the values live and why it does not read them itself: - they need an open handle, and opening one loads the driver, falsifying the "windivert driver" - check printed two lines above it in the same report. -- Three new tests, three mutants, every one caught (including the ABI numbers being swapped). - No hot-path change: this is three calls, once per session. - -### BREAKING: metrics.connections_reset counts connections, not packets (audit F7) - -- **Symptom.** `repro.py` had `connections_reset=stats["drop_rst"]`, and `drop_rst` counts every - PACKET dropped while a connection sits in its RST cooldown. With `--rst-cooldown 30` on a busy - flow that is thousands against a handful of resets; the guard test now pins the smallest possible - case - one reset connection reported itself as **50**. -- **Verified before designing the fix**, because the same lie could have been on screen: it is not. - `tips.stat_rst` already reads "Packets of reset connections (RST)", so the live tile has always - been honest. The repro report was the only surface claiming connections. Nothing in either README - mentions the key. -- **Fix.** New engine counter `rst_reset`, bumped in the capture loop when `dec.emit_rst` is set - - i.e. when a connection is actually torn down. Counted there rather than inside `_send_rst` - deliberately: the flow is put into cooldown and its traffic dropped whether or not an RST can be - built and injected for it, so `rst_reset >= rst_sent`, and the gap means "held down without an - RST going out". The report now carries three keys for the three questions: `connections_reset`, - `rst_packets_dropped` (the old value of the misnamed key) and `rst_sent`. CSV column - `connections_reset` (`App.CSV_COLUMNS`); NDJSON `summary.counters` gets it for free. -- **No GUI tile added.** `drop_rst` and `rst_sent` already have tiles with correct tooltips, and - `rst_reset` differs from `rst_sent` only when an injection could not be built or sent - a third - tile for that is noise on a grid that is already 18 wide. The precise accounting belongs in the - report, which is where the wrong number was. -- **Hot path:** one `_bump` behind `if dec.emit_rst`, which is per RESET, not per packet. Nothing - on the common path, so not benchmarked. -- Two mutants, both caught - **after two repairs to the tests, both worth recording.** The first - run reported "CAUGHT" on a pytest **exit code 4**, which is a usage error (I had guessed the test - id), not a failing assertion. The second attempt put the report assertion in - `test_summary_repro_views.py::test_build_repro_report`, whose session never resets anything: the - mutant left `connections_reset` and `rst_reset` both at 0 and the test passed while guarding - nothing. It lives in `tests/test_rst_local.py::test_rst_cooldown_sends_once_then_drops_silently` - now, the one session where the three numbers genuinely differ, and the mutant fails there with - `assert 50 == 1`. - -### BREAKING: connection rows separate captured from delivered, and count the queue's drops (audit F5) - -Two independent divergences in one table, fixed together because both need the same missing -mechanism: per-flow attribution of what happens AFTER the capture thread has moved on. - -- **(a) `dropped` was recorded a step too early.** `_log_conn(dropped=dec.drop)` ran before - `_enqueue`, so a packet the QUEUE refused counted nowhere in its row: measured - `drop_overflow=5500` against `dropped=0` on the row it happened to, and `drop_shutdown=4000` with - `dropped=0` and `bytes_in=4.8 MB` on a row that received nothing. `_enqueue` now returns whether - it refused, and `stop()` charges every stranded ORIGINAL to the row that was expecting it. -- **(b) `bytes`/`bytes_in`/`bytes_out` were CAPTURED bytes under headings every other surface uses - for DELIVERED** (session panel "Downloaded (MB)", repro report). Measured: a row reading - `bytes_in = 5 122 600 B` whose application received `409 600 B`, a factor of 12.5. Rows now carry - both: `bytes*` (captured, unchanged) and `sent`/`sent_in`/`sent_out` (delivered, new). The GUI's - `down`/`up`/`kb` columns are the DELIVERED pair - so they finally agree with the session panel - - and `down_seen`/`up_seen` are new columns for captured. `traffic_totals` (the footer) follows the - columns above it and sums delivered; `conns.totals` says "Delivered" outright. -- **The mechanism.** Queue entries gained the flow key as a fifth element - `(release, counter, packet, copy, key)`; the injector credits delivered bytes with it, `stop()` - charges stranded packets with it, and the send-failure path charges the row too (F14's counter - gets its per-flow half here). -- **Ordering is load-bearing, and the first version got it wrong.** Moving `_log_conn` AFTER - `_enqueue` (so it could log the refusal in one call) opens a race: with no latency set the - injector can deliver a packet before the row exists, and `_log_delivered` then has nowhere to - credit it. It usually loses that race - it must wake on `_cv` first - which is exactly the - "usually" that becomes a flake later. The row is created first; a refusal is charged afterwards - through `_charge_flow`, off the common path. -- **`_log_delivered` runs WITHOUT `_clock`**, on the same reasoning as `SocketWatcher.pid_for` - (convention 20): `sent`/`sent_in`/`sent_out` have exactly one writer, the inject thread, because - the capture thread only ever initialises them at row creation. Readers only read, and an int - rebind is atomic. `_charge_flow` (used for `dropped`, which the capture thread also writes) does - take the lock. Guarded by - `tests/test_engine.py::test_only_the_injector_writes_the_delivered_counters`, a source scan - - nothing else can enforce a single-writer invariant, and a future edit crediting delivered bytes - from another thread would drop updates silently and only under load. -- **Hot path measured, and it costs something** (Win11 AMD64, CPython 3.14.6, 150k 1500 B packets, - median of 5, three repetitions against a worktree of the parent branch): parent 157.4 / 161.4 / - 160.9k pkt/s, this branch 153.8 / 153.5 / 153.8k - about **4% down**, consistently, on the - synthetic path. Dropping the lock recovered roughly 1 point of the 5 the locked version cost - (160.4 -> 152.0k with it); the rest is the dict lookup and the adds themselves, i.e. the price of - the feature. Note the synthetic `send()` is a list append, so a real WinDivert syscall dilutes - this - unmeasured, and not claimed. -- New tests: `tests/test_engine.py::test_a_connection_row_records_the_drops_the_queue_made` - (asserts the row before AND after STOP: 190 refused, then 200 once the stranded ten are charged), - `::test_an_undisturbed_row_has_delivered_equal_to_captured` (so the new columns are not a - permanent discrepancy), and `tests/test_views.py::test_delivered_and_captured_are_different_columns`. - Five mutants, every one caught **after one repair**: the first version of the views test only - compared sort ORDER, and the mutant that put captured back under `down` kept the same order, so - it passed. It now reads the `DERIVED` cells directly and uses a row pair that leads on opposite - columns. -- Existing tests updated to the new contract rather than to green: `_conn()` in `test_views.py` - defaults `sent_* == bytes_*` (an undisturbed flow), and the export test now asserts an impaired - row where the two pairs genuinely differ. -- i18n: new `conns.down_seen` / `conns.up_seen` / `tips.col_down_seen` / `tips.col_up_seen`; - `tips.col_down` / `col_up` / `col_kb` and `conns.totals` reworded to say which quantity they hold - and to point at the other pair. Both READMEs describe the split with the measured example. - -### Fixed: three small ones - stray .tmp, orphaned scenario runner, STOP racing a capture fault (audit F11, F12, F13) - -**F11 - `export_connections_csv` left its temp file on failure.** It writes `path + ".tmp"` and -`os.replace`s it, but the `except` only logged: the half-written temp file stayed next to the real -one, and the next export silently overwrote it. `jsonfile.write_json` has had the cleanup (and a -test asserting no `*.tmp` after a failed write) for a while; this is the same guarantee. -New test: `tests/test_conns_export.py::test_a_failed_connections_export_leaves_no_tmp_file_behind`, -forcing the failure from inside the row loop (unsubtractable timestamps) so the temp file -definitely exists when it blows up. NOT changed: the stats CSV appends without an atomic write, so -a process death mid-flush can still truncate a line. Rewriting an append log through a temp file -costs the whole file per export; the exposure is a partial final line, and it is recorded here -rather than engineered away. - -**F12 - `start_scenario()` orphaned the previous runner.** It overwrote `self._scenario_runner` -without stopping it, and `stop_scenario()` only ever knew about the current object, so the old -thread kept applying its own steps to the same engine. `self.stop_scenario()` first. Found by -reading; NOT reproduced in the running program, because the GUI and CLI both start one scenario per -session - which is a property of today's callers, not of the engine. New test: -`tests/test_engine.py::test_a_second_scenario_stops_the_first_instead_of_orphaning_it`. - -**F13 - the capture-fault path waited on a stop that was joining it.** `_fail_stop(blocking=True)` -called `stop()`, which blocks on `_stop_lock`. When a `recv()` failed for its own reason in the few -instructions between the `_running` check and that call, an external STOP already held the lock and -was joining this very thread with a 2.0 s timeout: no deadlock, but STOP took the full 2 s. The -docstring claimed it "cannot deadlock against an external STOP: that path closes the divert first, -so the capture loop sees `_running` already False and never reaches here" - true when the fault IS -that stop closing the divert, and that sentence is why nobody looked further. - -- **First attempt was wrong and the suite caught it.** Routing the capture path through - `_worker_stop` (the audit's first suggestion, one line) broke - `test_a_dead_capture_thread_fails_open`: a divert that fails on its very first reads faults while - `start()` still holds the lock, which is not a corner case but the case the blocking path exists - for. It would have handed every such teardown to the watchdog a tick later. -- **What landed:** `_fault_stop_blocking()` polls `acquire(timeout=FAULT_LOCK_POLL_S)` (0.05 s) and - bails only when `_running` has gone False. No new state is needed to tell the two holders apart: - `_stop_locked` clears `_running` as its second statement, so "held and still running" is a start - and "held and not running" is a stop that already owns the teardown. -- `_fail_stop` now keeps the FIRST fault (`if not self.fault`). The watchdog's "worker thread died - unexpectedly" is a symptom; letting it overwrite the cause blanks out the only useful half of the - report. Both are still logged. -- New tests in `tests/test_failsafe.py`, all structural rather than wall-clock so they cannot - flake: `::test_a_capture_fault_racing_an_external_stop_does_not_wait_for_it`, - `::test_a_capture_fault_still_waits_for_a_start_that_holds_the_lock` (the other half - a mutant - that bows out on a start is caught too), and `::test_the_first_fault_is_the_one_kept_for_the_report`. -- **One of those tests was itself wrong, and mutation is what found it.** The fault test held - `_stop_lock` and called `_fail_stop` on the SAME thread - `_stop_lock` is an `RLock`, so the - nested `_worker_stop` re-entered, the stop completed, `_running` went False and the second fault - returned at the guard. It passed for the wrong reason and guarded nothing. Driving the faults - from another thread fixed it. Worth remembering when writing anything that leans on this lock. - -Five mutants across the three, every one caught after that repair. - -### Fixed: a failed injection is counted, and stops flooding the log (audit F14) - -- **Symptom.** `_inject_loop`'s `except` around `self._divert.send(packet)` logged and moved on. - The packet is already off the heap at that point: `bytes_in`/`bytes_out` are not bumped, no drop - counter is bumped, and `seen` counted it at capture - so it left the seen/delivered/dropped - balance entirely. That balance is the only mechanism keeping these numbers honest, and - `drop_shutdown` was added for precisely this class of hole ("instead of letting them vanish from - the balance"); the send-failure path was missed at the time. -- **Found by a question, not by a test.** The owner asked what happens to statistics if the - connection breaks mid-session. Answering it properly meant reading the path. -- **Fix (`engine.py`).** New `drop_send` counter bumped in the `except`, classified into - `TOOL_DROP_KEYS` - the tool failing, not the simulated link, so it stays out of - `impairment_loss_pct` while entering the balance. `SEND_WARN_S = 5.0` and `_warn_send_failed()` - mirror `OVERFLOW_WARN_S` / `_warn_overflow()` exactly, including the first-occurrence - `log_event("WARN", ...)` so the repro report carries it. -- **Why the rate limit is part of the same fix.** The comment above `OVERFLOW_WARN_S` already - states the rule - a per-packet line "becomes the second bug" - and this path had no limit at - all. `App._drain_log` drains the whole queue per tick and inserts EVERY line into the Tk widget - on the UI thread with no per-tick bound, so a burst of failures freezes the window on top of - losing the packets. Measured by mutation: 400 failures produced **400 log lines** before, at - most 2 after. -- **The F4 classification guard did its job on the first new counter since it was written.** - `test_every_drop_counter_and_drop_reason_is_classified` goes red with - `unclassified: ['drop_send']` if the counter is added to `st` without being classified - checked - by mutation, not assumed. -- **NOT verified:** no send failure was reproduced on a live WinDivert, so how often the real - driver refuses a packet, and whether it arrives in bursts, is unknown. The balance hole does not - depend on that number - it is a hole at one failure as much as at a thousand. -- Surfaced additively: live tile `stats.send_failed` (`gui/pages/stats.py::CELLS`), CSV column - `dropped_send_failed` (`App.CSV_COLUMNS`), and `_drain_engine_warning` now also fires for it, - with overflow keeping precedence (that one the user can act on by lowering the latency or rate). - `drop_send` reaches NDJSON `summary.counters` and the repro report's `counters` for free. -- i18n: `log.send_error` **retired** (its only caller is gone) and replaced by `log.send_failed` - carrying `{n}` and `{e}`; new `events.send_failed`, `warn.send_failed`, `stats.send_failed`, - `tips.stat_send_failed`. Net +4 keys in both lang files, key sets identical. Both READMEs list - the new counter and now state as a group that the three tool-loss counters are excluded from - "Effective loss" on purpose. -- **Hot path:** every added line is inside the `except`, so it cannot execute on a successful send - - a structural argument, but measured anyway against a `git worktree` of master (150k 1500 B - packets, median of 5): 157.6k -> 162.6k pkt/s, no regression. -- New tests, all mutation-verified (four mutants, every one caught): - `tests/test_engine.py::test_a_packet_the_tool_could_not_re_inject_is_counted_as_a_drop` (a - `RefusingDivert` whose `send` always raises; asserts the balance closes and that the figure stays - at 0.0% because this is not impairment damage), - `::test_failed_injections_do_not_flood_the_log`, and - `tests/test_gui_release_fixes.py::test_the_banner_also_fires_when_the_tool_cannot_re_inject`. - -### Docs: the effective-loss figure names its boundary (prose only, no behaviour change) - -- **Where it came from.** The owner ran the F9 acceptance (ping 30, one reply lost out on the - network) and asked why the tool showed 59 packets and zero loss. Both numbers are correct - 30 - requests out, 29 replies back, nothing broken here - but `tips.eff_loss`, added one commit - earlier, opened with "how much of the traffic you aimed at never arrived", which reads as "never - reached the far end". That is the wrong quantity, and it is the second reading of the same - sentence, so the sentence is at fault, not the reader. -- `tips.eff_loss` (both lang files) now leads with "how much of the traffic you aimed at THIS TOOL - broke" and states outright that a packet lost out in the network never arrives here, so nothing - here can count it. Same for the README bullet in EN and PL, which gained the worked ping example - (59 packets, zero drops, and why both are right). Values only - no new keys, no code touched. -- **No test guards this**, and none can: it is prose. What can be said is that nothing else in the - repo repeats the claim - `tips.stat_loss`, `tips.col_dropped` and `conns.scope_note` all already - scope themselves to the configured impairments, checked before writing this. - -### Fixed: portless traffic (ICMP) reaches the connection log (audit F9) - -- **Symptom.** `core._flowkey()` returns `None` when any of local port / peer address / peer port - is missing, and `_log_conn()` returns immediately on a `None` key. ICMP has no ports, so it was - counted in `seen` and in the byte totals and then dropped on the floor. Measured: 500 ICMP - packets -> `seen=500`, `bytes_out_total=49000`, **0 rows**. Confirmed on live WinDivert by the - owner (traffic filter "Ping (ICMP)", 30 s of pinging, empty tab) while `conns.scope_note` says - "All captured connections" and `README.md` says "Statistics and Connections show ALL captured - traffic". Both statements are now true; neither was edited, because neither was wrong about the - intent. -- **Fix (`engine.py`, capture loop).** When the flow key is `None` but a peer address is known, - the row is keyed `(proto, remote_ip)`. Both directions land on the same key (`remote_ip` is - `dst_addr` outbound, `src_addr` inbound), so a ping is one row, not two. A 2-tuple cannot - collide with the 3-tuple flow keys. **`_flowkey` itself is deliberately untouched**: it is also - the key of `core`'s flow table, which drives NAT expiry, RST cooldown and flapping, so making - ICMP a flow there would change what gets IMPAIRED rather than what gets listed. -- **The dependency was read before the design, not after** (convention 6). Installed pydivert, - `packet/__init__.py`: `src_port` (line 553) **returns None rather than raising** when there is - no TCP/UDP header - load-bearing, because the ports are assigned in the same statement BEFORE - `remote_ip`, so a raise would leave `remote_ip` at `None` and the fallback key would have - nothing to key on. Also `packet.icmp` (line 483) is `icmpv4 or icmpv6`, so the existing - `proto = "ICMP"` detection in the capture loop is right for real packets; had pydivert exposed - only `icmpv4`, ping rows would have been labelled "IP" and the fix would have looked like it - worked. -- **Three consumers assumed ports always exist.** All three were fixed in the same change, and - the first was a crash: `cli.py::_print_conns` pads with `{...:<6}`, and `format(None, '<6')` - raises `TypeError: unsupported format string passed to NoneType.__format__` (verified), so - `--log-conns` in text mode would have died on the first ping row - a regression this fix would - have INTRODUCED. It now prints `-`. `gui/pages/conns.py::_render` passed the value straight to - Tk, where `None` renders as the literal string "None"; new `port_cell()` blanks it. - `views.py::_connection_blob` used `c.get('remote_port', '')`, but a portless row HAS the key - holding `None`, so the default never fired and the blob read `8.8.8.8:none` - every ping row a - hit for the search term "none". Now `or ''`. -- **Checked and NOT broken** (verified rather than assumed): sorting by a port column - - `filter_sort_connections` casts inside `try/except (TypeError, ValueError)` -> 0.0, and the text - branch goes through `str()`; the connections CSV export - `csv.writer` writes `None` as an empty - cell, now pinned by `test_conns_export.py::test_export_connections_csv_writes_a_portless_row_with_empty_port_cells`; - the context menu - `_selected()` reads the DISPLAYED cells, so "Limit to this IP:port" on a ping - row fills the address and leaves the port blank, and "Target this process" is already disabled - for a row with no process. -- **A fourth consumer, found by re-reading this change rather than by a test failing.** - `ConnsPage._key_of` joined local port, address and remote port - unique for every row while - every row had ports, and no longer: two portless rows to the SAME address (ping plus, say, ESP - or GRE to a VPN gateway) both stringify to `"None|10.8.0.1|None"`. `SortableTree._ensure_index` - builds `{key: position}` as a **dict**, so one of the two rows becomes unselectable and - unscrollable-to. The protocol is now part of the identity. This is a defect this change would - have INTRODUCED, like the `--log-conns` crash; guarded by - `test_conns_columns.py::test_two_portless_rows_to_one_address_keep_separate_identities` - (mutation: restoring the old key collapses the index to one entry). -- **Hot path:** one extra `key is None` comparison per packet. Measured against a `git worktree` - of master (150k 1500 B packets, median of 5): 153.4k -> 158.8k pkt/s, i.e. no measurable cost - (the difference is run-to-run spread, the two paths differ by one comparison). -- New tests, all verified by mutation (five mutants, every one caught): - `tests/test_engine.py::test_portless_traffic_reaches_the_connection_log` (removing the fallback - gives `rows=0`) with an `IcmpPacket` double shaped after the pydivert reading above; - `::test_traffic_with_no_peer_address_still_gets_no_row` (dropping the `remote_ip is not None` - guard invents a row keyed on nothing); - `tests/test_cli_runtime.py::test_the_connection_listing_survives_a_row_with_no_ports` (reverting - it reproduces the `TypeError` above); and - `tests/test_conns_columns.py::test_a_portless_row_renders_empty_port_cells_and_is_not_searchable_as_none`, - which covers the render blanking and the search blob in one page-level run. -- i18n: `tips.col_proto` gained ICMP to its protocol list; `tips.col_remote_port` and - `tips.col_local_port` now say the cell is empty for portless traffic (and why the process column - is empty with it). Values only in both lang files - **no new keys**. README EN + PL gained one - clause in the Connections description. - -### Fixed: drop_overflow and drop_shutdown count packets, not queue entries (audit F6) - -- **Symptom.** `_enqueue()` runs once per element of `dec.releases`, and pipeline step 12 adds a - second element for a duplicate. Both `drop_overflow` (in `_enqueue`) and `drop_shutdown` (in - `stop()`, from `len(self._heap)`) therefore counted queue ENTRIES while `tips.stat_overflow` and - `tips.stat_shutdown` both promise "Packets". Measured before the fix: `seen=1000`, - `duplicated=1000` -> `drop_overflow=1900`, `drop_shutdown=100`, so the "Buffer overflow" tile - could read higher than the "Packets" tile in the same session. -- **Fix (`engine.py` only).** `_enqueue(release, packet, copy=False)`; heap entries carry the flag - as a fourth element `(release, counter, packet, copy)` (ordering unaffected, `counter` is unique - so comparison never reaches index 2 or 3); `_inject_loop` unpacks four; `stop()` uses - `sum(1 for entry in self._heap if not entry[3])`. The capture loop enqueues `releases[0]` - directly and only branches into a loop for the duplicates, so the single-release path (every - session without duplication) does not pay for an `enumerate`. -- **The warning follows the counter.** A refused copy no longer warns either, because - `log.queue_overflow` interpolates the counter: firing it for a copy would print "the TOOL is now - dropping packets you did not ask to lose (0 so far)". Only originals count and warn, which keeps - the counter, the log line and the GUI banner (`_drain_engine_warning`, which reads - `st["drop_overflow"]`) telling the same story. A queue that refuses only copies is losing the - user nothing, so silence is correct. -- **Known, deliberate gap** (recorded in `_enqueue`'s docstring rather than engineered away): if - the original is refused and the injector then frees a slot so the duplicate fits, the packet is - delivered up to 20 ms late but counted once as lost. Closing it needs copy/original pairing - through the heap. -- **Hot path measured, not assumed** (Win11 AMD64, CPython 3.14.6, 150k pre-built 1500 B packets - through `FakeDivert`, median of 5, both trees benchmarked back to back from a `git worktree` of - master): no duplication 153.8k -> 156.0k pkt/s, 100% duplication 100.9k -> 102.4k pkt/s. The - medians moved about 1.4% in the new code's favour, but the per-run ranges overlap - (147.8-155.5 vs 149.3-157.7), so the honest reading is NO MEASURABLE REGRESSION, not a speedup. -- New tests, both verified by mutation (reverting each half of the fix turns them red with exactly - the pre-fix numbers): `tests/test_engine.py::test_overflow_counts_packets_lost_not_queue_entries` - (100% duplication into a 10-slot queue: `drop_overflow == 390` before, `195` after, against - `seen=200`) and `::test_drop_shutdown_counts_packets_never_delivered_not_queue_entries` - (`drop_shutdown == 400` before, `200` after). Both are deterministic, not statistical: - `dup=100%` fires on every packet and the 60 s latency releases nothing before STOP. - -### Fixed: two tooltips that described counters they do not describe (audit F8) - -Both in `lang/en.json` and `lang/pl.json`, both user-visible, neither catchable by a test - this is -the failure mode rule 5 is about: true-sounding prose next to correct code. - -* `tips.stat_loss` said "Packets dropped because of the configured Loss (or link outages)". Link - outages have had their own counter (`drop_flap`) since they stopped inflating "Dropped", and - `tips.stat_flap` says so in the next cell: "Counted separately from loss". Two tooltips - contradicting each other, with the wrong one attached to the number a tester reads first. -* `tips.data_down` promised "Hovering also shows how much the app tried to download". - `add_tooltip(widget, key)` renders one static translated string (`gui/tooltip.py`); there is no - dynamic path and never was. The offered figure exists only as `metrics.offered_mb` in the repro - report - and note it is the SUM of both directions, so it is not "how much the app tried to - download" either. Replaced with something true and useful in its place: dropped packets are not - counted in this figure, which is the difference between it and the connections table's bytes. - -No code change, so no new test; `test_i18n_coverage` already pins key parity (465 keys, identical -sets) and `test_no_em_or_en_dashes_in_repo_text` the punctuation. - -### Fixed: an impairment no longer expires when its flow record does (audit F2, and F1's tail) - -`_reset_until` and `_flow_last` are `_FlowTable`s that retired a generation every `FLOW_ROTATE_S` -(30 s), so a record survived 30-60 s - and both tables hold the state that IS an impairment. Two -settings were therefore capped by a constant nobody had connected them to: - -* **`rst_cooldown`** accepts up to 3600 s. Measured at 120 s, the same flow was reset again after - 30.3, then 60.8, then 60.8 s. -* **NAT expiry** was worse than capped. A retired record reads back as "never seen", so the next - inbound packet reopens the mapping with nothing sent. Measured: a 5 s timeout blackholed for 20 s - and then passed traffic; **a 30 s and a 120 s timeout dropped ZERO packets**, because the record - died at the rotation just before the first inbound packet arrived. - -`_FlowTable.keep_for(seconds)` raises the age window; `set_rst` passes the cooldown, and `set_nat` -switches ageing OFF for `_flow_last` (`float("inf")`) while NAT is on. Matching the window to the -timeout is not enough for NAT and was measured failing exactly as above - the record has to outlive -the timeout AND the blackhole after it, and that blackhole is meant to last until the application -sends. The RST cooldown is a fixed span, so there the window is simply the cooldown. - -**Why raising the window is safe, and the property that says so:** the SIZE ceiling is enforced on -every write, independently of the age window, so a longer window makes the table older and never -bigger. Measured through the real setter: 250k flows with `nat_timeout=3600` peaked at 199,999 -against the 200,000 ceiling. - -After: cooldowns of 60 / 120 / 300 s reset at exactly those intervals; a NAT blackhole held for the -full 900 s of the probe at 5, 30 and 120 s timeouts (1800 packets dropped) and ended on the first -outbound packet, which is the documented contract. - -New tests in `tests/test_core.py`: -`test_a_long_rst_cooldown_is_honoured_not_truncated_by_the_flow_table`, -`test_an_expired_nat_mapping_stays_shut_until_the_application_sends` and -`test_the_size_ceiling_holds_even_with_the_age_rotation_switched_off` - the last one drives the -table through `set_nat` on purpose, because the existing churn test assigns `nat_timeout_s` -directly and so never exercises the ageing-off path. The first two were verified by mutation: -making `keep_for` a no-op reproduces the audit's numbers exactly (`gaps=[30.3, 60.8, 60.8, ...]` -and `dropped=0, passed=720`). - -Prose corrected in the same commit (convention: a behaviour change updates the sentence that -justified the old one): the `_FlowTable` docstring said eviction "can lose an impairment, never -invent one" as if that covered both paths. It covers the SIZE path; on the AGE path it was not a -trade but a silent cap. The step-3 comment added by the F1 fix said the blackhole ends at the first -rotation - that is now what the fix prevents. - -### Fixed: the injector asks Windows for a fine timer tick, and for it to be honoured (audit F3) - -`_inject_loop` holds a delayed packet with `Condition.wait(timeout=...)`, and Windows rounds that -timeout UP to the system timer tick - 15.6 ms unless the process asks for better. That rounding -was the entire added-latency error. Measured on the REAL capture path (ping through live -WinDivert, `--dst-ip 8.8.8.8`, two independent runs): a **constant +12.6 ms of round-trip -overshoot, independent of the setting** - +12.2 ms at `--latency 10` and +12.8 ms at `--latency -50`, where a proportional error would have been ~62 ms at the higher setting. The control run at -`--latency 0` measured +0.4 and -0.3 ms, so it was not the capture path, the driver or the link. - -`BeanEngine.start()` now takes a fine tick for the life of the SESSION and `_stop_locked` gives it -back after the injector thread is joined; `winenv.request_fine_timers` / `release_fine_timers` are -the (Windows-only, no-op elsewhere) wrappers. Session-scoped rather than process-scoped: the pair -costs ~1.3 us (measured), so nothing needs to hold a finer tick while the tool sits idle. - -**Two things this cost, both found by measuring rather than by reading docs:** - -* **`timeBeginPeriod` alone is a fix that works and then stops.** Windows 11 throttles a - BACKGROUND process's timer resolution: the request kept returning success with a perfectly - balanced request/release log, while the effect vanished after roughly ten seconds - in one - process, `Condition.wait(10 ms)` went 10.1, 10.3, then 15.6 ms for every later session. This - tool lives in the background (start a session, switch to the app under test), so shipping only - the obvious call would have regressed silently on users while measuring clean here. - `winenv._allow_fine_timers_in_background()` opts out via `SetProcessInformation` - (`ProcessPowerThrottling` + `PROCESS_POWER_THROTTLING_IGNORE_TIMER_RESOLUTION`, state 0), once - per process. With it, the same wait held 10.1-10.7 ms across 40 s of sampling and across eight - back-to-back sessions. -* **Querying the system-wide resolution proves nothing.** `NtQueryTimerResolution` reported a - current tick of 1.0 ms - something else on the machine was holding it - while our own waits were - still being rounded to 15.6 ms, because since Windows 10 2004 the tick is per-process. A session - that checks the global number and concludes "the timer is already fine" is reading a number that - does not apply to it. That is written into the `request_fine_timers` docstring. - -Result through the engine, eight sessions in one process (sparse traffic, the case the audit -measured at +8.3 ms): overshoot **+0.22 to +0.55 ms**, worst case down from ~25 ms to ~11 ms. - -**Accepted on the real capture path** (ping, 40 packets per setting, live WinDivert): - - krok nominal min p50 p90 max nadwyzka p50 (przed) - latency 10 45 45 46 50 61 +1 ms +12.6 ms - latency 50 125 125 126 130 135 +1 ms +12.6 ms - baseline 25 24 25 28 112 - -Same +1 ms at both settings, so the surcharge is gone rather than scaled; p90 is +5 ms at both. -Two measurement lessons worth keeping, because the first acceptance run (10 packets per setting) -read as a HALF fix at +6.3 / +9.5 ms: - -* **Average over ten pings is the wrong statistic here.** About 1% of packets sit in a tail, a ping - carries two impaired packets, and one outlier moves a ten-sample mean by 4-8 ms. The median was - right all along - the ten-sample run's own minima were already exactly nominal (45 and 125). -* **The tail is the LINK, not us.** In the 40-packet run the untouched baseline produced the worst - outlier of the whole session (max 112 ms, against 61 and 135 with the tool in the path). An - earlier reading that blamed a "worse tail" on this change did not survive a bigger sample. - -New tests in `tests/test_failsafe.py`: -`test_the_fine_timer_request_is_balanced_on_every_session_path` (clean stop, double stop, second -session and a start that raises - an unbalanced pair is invisible from inside the program, it just -means the process keeps a finer system timer for life), `test_a_refused_fine_timer_request_is_never -_released` (releasing one we never took decrements somebody else's refcount), -`test_the_background_timer_opt_out_is_asked_for_once_per_process` and -`test_the_fine_timer_calls_are_safe_to_make_anywhere`. The first two were verified by mutation: -dropping the release turns the first red with `['request']`, releasing unconditionally turns the -second red with `['request', 'release']`. - -### Fixed: a dropped packet no longer revives the NAT mapping it was dropped for (audit F1) - -`BeanCore.decide()` step 3 read and wrote the activity stamp in one `_FlowTable.touch()`, so the -write happened BEFORE the expiry verdict and applied to the drop path too. The packet rejected -with reason `nat` therefore stamped the flow as active, and the mapping was back: the direction -lost exactly one packet per `nat_timeout_s` and carried traffic in between, with nothing outbound -involved. Measured before the fix (timeout 5 s, one outbound at t=0, inbound only afterwards): -`t=10 DROP, t=11 pass, t=12 pass, t=13 pass, t=20 DROP`. The impairment exists to test whether an -application sends keep-alives, and in that shape the test could not fail. - -Split into `get()` + a `set()` placed after the verdict, so the drop path skips the write. Same -cost - `touch()` was a `get` plus this same write - measured 160 ns/op both ways at 200k -iterations, difference below the noise floor. `_FlowTable.touch()` had exactly one caller and is -removed with it. - -How long the blackhole holds is bounded by the flow table, and the comment in `core.py` now says -so with numbers instead of a general claim: the drop path returns above the `_prune()` call, so it -never rotates anything itself. Measured with the same setup - this flow alone: still blackholed at -t=200; one other flow driving `_prune`: reopened at t=30, the first rotation. That is the table's -documented safe direction (it can lose an impairment, never invent one) and is a different thing -from the resurrection above, where the dropped packet did the reopening itself. - -New test: `tests/test_core.py::test_a_dropped_packet_does_not_revive_an_expired_nat_mapping` - -asserts every inbound packet after expiry is dropped (not just the first) and that an OUTBOUND -packet is what brings the mapping back. Verified by mutation, not assumed: restoring the -write-before-verdict order turns it red with `drops=[True, False, False, False]`, the exact -symptom above. The two existing NAT tests (`test_nat_expiry`, `test_nat_outbound_refreshes`) pass -unchanged - neither of them pinned the buggy behaviour, which is why it survived. - -### Performance: PortTable.refresh collects the socket table outside its lock - -The capture thread takes `PortTable._lock` too - `name_of(cheap=True)` -> `info()` in -`engine._process_for` - so whatever `refresh()` holds it for, the packet path can be made to wait -for. It used to hold it across the four `iphlpapi` calls, the port->pid dict build AND the -departed-pid diff, all O(number of sockets). - -MEASURED first, 2026-07-25 (Win11, CPython 3.14, elevated, medians with a control), because "the hot -path waits N ms" is exactly the kind of claim convention 5 says to measure rather than estimate: - -- Lock hold with everything inside: **0.495 ms median at 119 sockets** (p99 0.843, max 0.950). - Python-side work scales linearly: 0.011 ms at 100 sockets, 0.303 at 10 000, **3.555 at 100 000** - (syscalls excluded, so a floor). -- The capture thread's own `name_of(cheap=True)` is **0.4 us median** - and with a thread hammering - `refresh(force=True)` alongside, its median, p95 and p99 did not move at all. Only **2 calls in - 20 000** waited, worst 1.5 ms. At the real rate the lock is held ~0.15% of the time. -- So at desktop scale this was NOT an observable problem, and the honest conclusion could have been - "measured, rejected". What decided it was the scaling: a network tester is the thing that gets - pointed at 100 000 connections, and this project's history is a series of "it was fine until - someone ran a load test" (the flow table once settled at 3.2 million entries). -- Also learned while checking WHO refreshes: since chunk 2c the resolver resolves against the - `SocketWatcher`, whose `refresh()` is a no-op - so in a real session only the watchdog refreshes - the `PortTable`, ~3-5x a second. The up-to-20x a second case is the poller FALLBACK - (`--simulate`, tests, non-Windows), which has no real capture thread to stall. - -After the change: **the hold is flat at ~0.018 ms** whatever the table size (188x shorter at 100 000 -sockets, and no longer a function of n). The whole call costs the same - the work left the lock -rather than disappearing. Kept internal-only deliberately: nothing a tester can observe changes, and -claiming a user-visible win would be unmeasured prose. - -- Collection and the departed diff now run unlocked; only the reference swap, `_last`, the - `_info` pops and `_expire_info` are taken under the lock. -- **Overlapping refreshes cannot move the map backwards.** Two threads can now collect at once, so - each call takes a GENERATION under the lock before starting and installs only if nothing newer - landed. A counter, not a timestamp: `time.monotonic()` has ~15 ms granularity on Windows, so two - refreshes inside one tick would compare equal and both install. The hazard was real, not - theoretical - the mutation below installs the stale map without the guard. -- The `native` handle is captured per call and the "it stopped answering" flip is re-checked under - the lock (`self._native is native`), so a concurrent refresh cannot have its own conclusion undone. -- New tests in `tests/test_processes.py`, both mutation-checked: - `::test_the_socket_table_is_collected_without_holding_the_lock` - probes the lock from ANOTHER - thread, because `_lock` is an RLock and a same-thread acquire would succeed while held and prove - nothing; putting the collection back inside turns it red. - `::test_an_older_collection_does_not_overwrite_a_newer_map` - gate-driven rather than sleep-raced; - removing the generation guard turns it red with the stale map installed. - -### Changed: start() binds the targeting under the lock that protects it - -`_start_locked` read `self._targeting` bare and then acted on that reference three times -(`set_table`, a synchronous `refresh`, `retarget`), while every other access to the field - in -`set_target` and `target_for` - is under `_target_lock`. A concurrent `set_target()` landing in the -middle would leave the resolver pointed at an ORPHAN while the core tested against the object that -replaced it, which is the same class of mismatch `set_target()` itself was fixed for. - -- **This is consistency of access, NOT a fixed symptom, and the commit says so.** The race was not - reproduced: 25 start/stop cycles against a concurrent applier in `test_concurrency_chaos.py` stayed - green, and the window is narrow today because the CLI applies settings before start, so only a GUI - "Apply" landing inside start could hit it. Recorded this way deliberately, so a later session does - not read it as "we fixed a race we had seen". -- Lock order checked before widening the hold, not after: `_stop_lock` -> `_target_lock` -> - `ProcessTargeting._lock` -> `PortTable._lock`, and nothing walks it the other way - the resolver - never calls back into the engine, and `retarget()` is only reached with `_target_lock` already - held. In this spot there is not even contention: `_resolver.start()` comes after the block and - `stop()` joins the resolver, so the synchronous `refresh()` has no contender. Cost is one cold - resolve (~36 ms elevated, per `portmap.info`) at session start. -- Rejected: locking only the READ. It narrows the window without closing it, and the point is that - start binds ONE coherent object rather than that a mismatch becomes less likely. -- New test: `tests/test_target_resolver.py::test_start_binds_the_targeting_under_the_lock_that_protects_it` - - a mechanical guard (an instrumented `_target_lock` that records acquisitions), not a timing test, - because a timing test for this window would be flaky and prove nothing either way. Mutation-checked: - reverting to the bare read turns it red. Its docstring states what it does not claim - that the - race was real. - -### Tests: the concurrency chaos suite now includes the SocketWatcher - -The suite's own charter is "many threads hammering one engine" and it names the failure it exists to -catch - threads tested in isolation. Since the SOCKET-layer work the session runs a FOURTH thread -and, since the connection-log fix, the capture thread reads the watcher's live map WITHOUT a lock - -and no test in that file ever passed the engine a `socket_source`, so neither the watcher's lifecycle -nor that lock-free read took part in any chaos. - -- `test_the_socket_watcher_survives_start_stop_cycles` - `CYCLES` start/stop rounds with a real - event stream running: fail-open (running implies a live capture thread AND a watcher), the watcher - cleared and its handle released on stop, and no `bean-socket-watcher` thread outliving its session. - Checked after a 0.3 s grace, because `SocketWatcher.stop()` only joins for 0.25 s. -- `test_the_capture_thread_reads_the_live_socket_map_under_churn` - the real new surface: the capture - thread resolving pids from the map while the watcher thread mutates it, the watchdog republishes it - wholesale, and settings and targeting churn underneath. Runs on `FastDivert` and on the SAME ports - the event source announces, or `pid_for` would always miss and a green run would prove nothing. -- **The crashlog watch is the test, not decoration.** `_pid_for` / `_process_for` swallow into - `crashlog.once` by design, so a read that started raising would leave every other assertion green. - Patching `once` also defeats its `_once_seen` dedupe, so repeated failures stay visible instead of - collapsing into one entry. -- Conclusiveness is a CONDITION the test waits for (`MIN_STAMPED` rows carrying the event stream's - pid), never a duration - the same reasoning as `MIN_BUILDS`/`MIN_ROWS`, and for the same reason: a - wall-clock budget lets machine speed decide whether the test proved anything. -- Both claims MUTATION-CONFIRMED: a `SocketWatcher.pid_for` that raises turns it red through the - crashlog watch, and an engine reverted to poller-only stamps 0 rows out of 2.8 million packets. - The first mutation also independently confirmed the failure-domain split below - it reported - `engine.ports` AND `engine.ports.pid`, not one collapsed entry. -- Stated in the docstring, so nobody assumes otherwise: this does NOT catch putting the lock back - into `pid_for`. A lock contends, it does not raise, and this test does not measure contention - - that property has its own guard in `test_socketwatch.py`. -- `_LiveSocketSource` is paced on purpose. Unpaced it saturates a core and starves the very threads - the test is about, which would make a green run meaningless. - -### Fixed: the connection log resolves the owner from the live socket map, not the poller - -`_pid_for` / `_process_for` asked `portmap.PortTable` - a snapshot refreshed a few times a second - -so a flow that opened AND finished inside one refresh interval left a row with no owner at all. -Short-lived connections are what this tool gets pointed at, so that was the common case rather than -an edge one. The engine now asks the SOCKET-event map first (`SocketWatcher.pid_for`) and falls back -to the poller, so a row is stamped from its FIRST packet: the CONNECT event lands ~0.1 ms before the -SYN reaches the NETWORK layer (measured 2026-07-22). - -- **The read had to become LOCK-FREE first, and that is the substance of this change.** `pid_for` - used to take `_lock` - which the watcher thread holds on every socket event, and which - `reconcile` holds across a whole snapshot merge. Calling that from `_log_conn` would have let the - CAPTURE THREAD queue behind maintenance, which is exactly the stall convention 20 exists to - prevent. `pid_for` now reads the reference once and does a C-level `get` on int keys (the same - idiom as `PortTable.pid_for`), and `reconcile` builds the new state to the side and publishes it - by REASSIGNMENT, so its O(n) pass is atomic to a reader instead of being observed half-applied. -- Name resolution is unchanged and still `cheap=True` (cache or nothing), so a brand-new pid can - reach a row BEFORE its name does - names are warmed by the watchdog. That is written into the - docstring rather than glossed over: a PID with no name yet is still an answer, and `_log_conn` - keeps retrying the name while packets arrive. -- Deliberately NOT done: warming names from the watcher's map as well. It would close the remaining - name lag, but it adds per-pid OS calls to the watchdog and deserves its own measurement first. -- The shared lookup lives in `_live_pid`, which has NO handler of its own on purpose. The first - attempt had `_process_for` delegate to `_pid_for`, and because that one swallows and records under - `engine.ports.pid`, a broken port table reported ONE failure instead of two - the name domain - could no longer speak for itself. `test_processes.py::test_engine_records_a_broken_port_table_instead_of_going_quiet` - caught it. The two callers are two failure domains and each wraps the raising helper itself, which - is the same principle the watchdog already applies to refresh-vs-trim ("different jobs, different - failure domains"). -- `_Broken` in that test now models what the engine actually calls (`pid_for` + `name_of(cheap=)`) - instead of `process_for_port`, which the engine no longer touches. The insight in its comment - - that a fake missing the keyword raises TypeError and the test then passes while exercising the - wrong failure - still holds, so it moved to the keyword that now matters rather than being - deleted. -- New tests, each MUTATION-CHECKED rather than trusted: - `test_socketwatch_wiring.py::test_a_fresh_socket_stamps_the_connection_row_from_the_live_map` - (reverting the engine to poller-only turns it red), plus in `test_socketwatch.py` - `::test_pid_for_takes_no_lock_because_the_capture_thread_calls_it` (restoring the lock turns it - red) and `::test_reconcile_publishes_a_new_map_instead_of_mutating_in_place` (mutating in place - turns it red). -- `test_socketwatch.py::test_a_lock_free_reader_survives_writes_in_flight` RUNS the safety claim - instead of asserting it in prose: a reader hammering `pid_for` while another thread inserts, - deletes and republishes the whole map never raised and only ever saw the real pid. -- The engine test drives a GATED divert, so it asserts that the live map is consulted rather than - that the capture thread happened to lose a race with the watcher thread. -- `test_hot_path.py` (packet threads must never reach the OS) was run explicitly and stays green: - both lookups are dict reads. - -### Changed: SocketEvent carries only what the map is for (closes socket-event-fields) - -The SOCKET-layer event used to carry `proto`, `remote_ip`, `remote_port` and `outbound` "for the -connection log later". Nothing ever read them, and an engineering review found out why that never -became a problem worth solving: for all four, the NETWORK-layer packet the engine already holds is -a strictly better source. The connection log takes `remote_ip` / `remote_port` / `proto` / direction -straight off the packet, and the packet even distinguishes ICMP, which the SOCKET layer does not. -The one thing a packet cannot tell us is the owning pid - which is precisely what is left. - -- `socketwatch.py`: `SocketEvent` is now `kind pid local_port`. `_ipv4()` goes with it, so the real - Windows source no longer decodes an address per event for nobody. -- Dropping `_ipv4()` also removes a live TRAP. It decodes IPv4 only (`addr[0]` read as a 32-bit - int), so the first person to "just wire up the field we already have" would have shipped garbage - for IPv6 into a user-visible column, in a tool whose traffic filters all cover v4 AND v6. -- PRESERVED from the deleted `test_ipv4_decodes_high_byte_first`, because the code is gone but the - knowledge should not be: the 2026-07-22 spike established that WinDivert stores the IPv4 address - MSB-first in `addr[0]`, and that the naive low-byte-first decode rendered 192.168.1.29 as - 29.1.168.192. If remote addresses are ever wanted again, start there - and extend it to IPv6. -- Helpers updated in `test_socketwatch.py`, `test_socketwatch_wiring.py` and - `test_targeting_socketwatch.py`. `OPEN_PENDING` is now empty, which is its healthy state. -- Guard fix, found BY this closing: `test_no_stale_pending_markers` now skips `CHANGELOG*.md`. A - changelog records what HAPPENED and is dated by its nature, so the entry below that announced the - marker stays true after the marker is gone - flagging it was a false positive, and the first real - closing walked straight into it. Re-verified by mutation after the fix: a marker naming an - unlisted stage in an ordinary `.md` still turns the test red, so the scan was narrowed, not - broken. - -### Tests: a mechanical guard for prose with an expiry date (convention 44) - -The 2b/2c drift fixed below was not a set of FALSE claims - it was four claims that were true when -written and were supposed to die when a stage landed, with nothing enforcing the expiry. -Convention 5 covers claims that are wrong; `check_notes.py` deliberately does not check prose at -all. So the sentences outlived the code they described by several PRs. This adds the missing -mechanical step, because discipline alone produced four stale sites in a single transition. - -- New test: `tests/test_repo_conventions.py::test_no_stale_pending_markers` - scans every `.py` - and `.md` in the repo for `PENDING()` markers and checks BOTH directions. A marker whose id - is not in `OPEN_PENDING` fails (the stage closed, so the prose beside the marker is now a lie), - and an id in `OPEN_PENDING` that nothing references fails (a leftover entry). Closing a stage is - therefore ONE deletion from `OPEN_PENDING`, which turns the guard red on every marker still - pointing at it - so the prose gets corrected in the same commit as the code that outdated it. -- `OPEN_PENDING` (same file) is the single source of open stage ids, and is deliberately NOT a - roadmap: it is the set of ids that PROSE points at, which is what makes the second check - meaningful rather than bureaucratic. -- MUTATION-CHECKED both ways instead of asserted (convention 5): a probe file carrying a marker - for an unlisted stage, and an unreferenced id added to the set, were each confirmed to turn the - test red with the offending `file:line` in the message. The probe was then removed. -- First real marker: the `SocketEvent` comment in `socketwatch.py`, which was itself a leftover - ("carried for the connection log later (2c)"). `proto` / `remote_ip` / `remote_port` / - `outbound` are still unconsumed - the comment now says so plainly and carries - `PENDING(socket-event-fields)`, so whichever way that decision goes, the guard forces the - sentence to be revisited rather than quietly kept. -- The guard skips its own file (that file holds the ids rather than pointing at them), and the - `` placeholder form used in prose does not match the pattern, so docs and changelogs can - name the token without registering it. -- Rejected, MEASURED not guessed: scanning for the word "yet". 25 hits across `beantester/` and - `tests/`, of which roughly 21 are permanently true ("width 1 == not laid out yet", "no honest - answer yet"). A guard with that false-positive rate is switched off within a week. -- Convention 44, a sub-rule under process rule 5, and a new definition-of-done step (grep for the - id when closing a stage) live in the private notes, which this test cannot see - the notes are - not in this repo, so that half stays manual on purpose. - -### Docs: de-stale the 2b/2c prose - targeting DOES resolve against the live socket map - -Prose drift of the kind convention 5 exists for, with a twist: every one of these sentences was -TRUE when written and acquired an expiry date nobody enforced. Chunks 2b/2c/2d landed (PR #38), -so "not wired yet" became a lie sitting next to correct code - and `check_notes.py` deliberately -does not check prose, so nothing went red. Found during an engineering review; no behaviour -change, comments and docstrings only. - -- `engine.py`: the `_socketwatch` attribute comment ("NOT read by targeting yet - that is 2c") - and the `_start_socketwatch` docstring ("targeting does not read it yet") now point at - `_targeting_table()` / `_start_locked`, which is where the poller-vs-watcher choice is actually - made. -- `socketwatch.py`: the module docstring no longer claims the module is "in isolation... NOT - wired into the engine or targeting yet (2b/2c)"; the lifecycle section header drops "in 2b". -- `tests/test_socketwatch_wiring.py`: the docstring kept the part that is still true (this file - covers the plumbing only) and lost the false framing; it now names - `tests/test_targeting_socketwatch.py` as the guard for the resolution contract. -- Deliberately NOT touched: the `SocketEvent` comment saying `remote_ip`/`remote_port` are - "carried for the connection log later (2c)". Those fields are still unconsumed, and whether - they get used or cut is a separate decision - fixing the sentence now would only mean writing - prose that decision will rewrite. -- Follow-up on its own branch: a mechanical guard for expiring prose (`PENDING()` markers - checked against one list of open stage ids), because discipline alone produced four stale - sites in a single transition. - -### Docs: correct the resolver recycle-check cost - it is elevation-bound, not "~180 ms" - -Measured 2026-07-24 with Chrome open, elevated AND non-elevated (`runas /trustlevel:0x20000`), -because the deferred "~180 ms per rebuild" perf follow-up from the socket-layer work did not -reproduce elevated. It is real prose drift (convention 5): two docstrings disagreed - `_psutil_created` -claimed 0.005 ms, `info` claimed ~5 ms - and both were right for a different elevation. - -- `create_time()` is ~0.005 ms per PID when `OpenProcess` succeeds and ~5.7 ms when it is DENIED - (psutil then scans the whole system for that one PID). Denial is an ELEVATION question, not the - "hardened process / Chrome renderer" one the older note assumed - renderers do not own sockets, - so they never reach this loop. ELEVATED: 0 of 27 socket-owning PIDs denied, warm recycle check - ~0.16 ms, cold `resolve('chrome')` ~36 ms. NON-ELEVATED: 16 of 27 denied, ~90-180 ms warm, - ~381 ms cold - which is where the handoff's ~365/180 ms came from. -- Real impairment ALWAYS runs elevated (WinDivert will not open without admin; a non-admin START - fails), so the check is cheap in every real session. The only non-elevated path that runs the - resolver at all is `--simulate` (synthetic packets; the cost is on the resolver thread, never - the capture one). -- DECISION, rejected: batching the denied PIDs in one `NtQuerySystemInformation`. It speeds only - the non-elevated `--simulate` warm check, does nothing for the elevated hot path, and nothing - for the cold resolve (name resolution dominates that, not create_time). Not worth the ctypes - surface for a demo mode. This closes the "known perf follow-up" / "handed to a follow-up" notes - in the entries below. -- Corrected in `portmap.py` (`_psutil_created` and `PortTable.info` docstrings) and PROJECT_NOTES. - No code behaviour changed - nothing a user notices - so CHANGELOG.md is untouched. - -### Fixed: the UI rebuild no longer piles up `` handlers on the root - -Follow-up to the teardown-crash fix. `App._build_ui` runs on every language switch, and the root -window outlives it, so binding `` on the root inside `_build_ui` (with `add="+"`, nothing -removing it) accumulated one handler per rebuild. Measured on the fake tkinter: 2 after the first -build, 8 after three switches, linear. Each is cheap - and after the teardown fix the dead ones are -no-ops, not crash records - but it is O(rebuilds) work on every resize for the life of the process. - -- The earlier "not fixed here" note named only `_on_root_configure` and undercounted (convention 5): - the two banners built with `wrapping_label(root, ...)` (`engine_warning` always, `admin_warning` - when non-admin) each bound their OWN `` on the same persistent root and multiplied - identically - half the handlers. A `wrapping_label` on a SHORT-LIVED container does not leak (its - binding dies with the container); only the two whose container is the root do. -- `_on_root_configure` is now bound ONCE in `__init__`, before the first `_build_ui`, and the line is - gone from `_build_ui`. It reaches its widgets through `self`, so a single binding always drives the - freshly rebuilt ones - no unbind needed, and none was safe (`Misc.unbind(seq, funcid)` still clears - the whole sequence on the oldest Python in the CI matrix, and the root carries other `` - bindings). -- The two banners are now plain `ttk.Label`s wrapped by that same single handler, not `wrapping_label`. - Same look (left/anchored, an initial `wraplength` refined on the first resize). `gui/labels.py` and - its short-lived-container callers are untouched; `wrapping_label` is no longer imported by - `gui/app.py`. -- Not user-visible (identical look and wrapping, no behaviour change), so CHANGELOG.md is deliberately - left alone - a user-facing line for an imperceptible change would be the filler convention 4 forbids. -- New test: `tests/test_gui_release_fixes.py::test_the_ui_rebuild_does_not_pile_up_configure_handlers_on_the_root` - - asserts exactly one `` handler on the root after several rebuilds, and that the banner - still wraps to the width-derived value (not just its build-time default, which would pass even with - the fold gone). Both halves verified by MUTATION: reintroducing the per-rebuild bind, and dropping - the banner wrap, each turns it red. - -### Fixed: two swallowed GUI teardown crashes, and the window geometry they were losing - -From a field `crashes.ndjson`: two `swallowed`/`debug` records, `TclError: bad window path name -".!toplevel2"` at `gui/windows.py::_save_geometry` and `TclError: invalid command name ".!label"` -at `gui/labels.py::_resize`. They share one cause: `App._build_ui` rebuilds the main UI by -destroying every child of the root window, and a `Toplevel` is a child of the root. - -- `App._build_ui` now skips the Toplevels the registry owns (new `WindowManager.toplevels()`). It - used to tear the open panel windows down behind the registry's back, so the `windows.rebuild()` - that follows a language switch ran `close()` on windows that no longer existed. That cost two - things, not one: the TclError above, and the geometry was never saved - the window reopened where - it had last been CLOSED, not where the user had just put it. -- `PanelWindow._save_geometry` returns early when the window is already gone (`winfo_exists`). - Nothing in the app should destroy a registry window behind its back now, but the root still takes - every Toplevel with it when the app quits. -- `bind_wraplength._resize` returns early when the label is gone. The `` binding lives on - the CONTAINER, which routinely outlives the label: the two banners built with - `wrapping_label(root, ...)` in `gui/app.py` hang off the root window itself, and every rebuild - destroys them and leaves their handler behind. Nothing unbinds it (it is added with `add="+"`, and - `Misc.unbind(seq, funcid)` still clears the WHOLE sequence on the oldest Python in the CI matrix - - verified only that 3.14 removes just the one binding). So this was never a one-off teardown race: - it is one dead handler per rebuild, each recording an entry on every resize from then on. -- `on_close` already closed the windows before persisting the UI state, so their geometry survives a - quit. That ORDER had no test; it has one now. -- `tests/fake_tk.py` made honest about destruction, or none of this could be caught: `destroy()` now - destroys children and leaves the widget DEAD, `winfo_exists()` is real, and `configure()` / - `geometry()` raise `TclError` afterwards, the way Tk does. `winfo_exists` HAD to be added rather - than inherited: `W.__getattr__` answers any unknown attribute with a no-op returning `None`, so a - `winfo_exists()` guard would have read as "destroyed" for every widget on the fake and silently - switched off the code it guards in every GUI test. -- New tests: `tests/test_windows.py::test_a_language_switch_keeps_the_window_alive_and_saves_its_geometry`, - `tests/test_windows.py::test_closing_a_window_that_is_already_gone_is_not_a_crash`, - `tests/test_windows.py::test_closing_the_app_saves_an_open_window_before_the_root_goes`, - `tests/test_gui_release_fixes.py::test_a_resize_after_the_label_is_gone_is_not_a_crash`. All four - verified by MUTATION (convention 5): each guard was removed in turn and the test that claims to - catch it went red. The `on_close` mutation is what showed the fourth test was originally guarding - a duplicate call added in this chunk rather than the real one; the duplicate was removed. -- Left for a follow-up (now DONE, see the entry above): `App._build_ui` also re-bound `` - on the root every rebuild, so that handler multiplied too - as did the banners, which this note - missed. - -### Fixed: the connections "impaired?" column and its row highlight now use ONE signal - -- Field report: rows showed orange with the column reading "no" (and "yes" rows with no colour) - - "something is wrong with the connections table". Cause: the column read the stored per-flow - `scoped` record (chunk 1) while `ConnsPage._tag_of` still asked the engine LIVE (`in_scope_now` - -> `local_port in target_ports`). For a closed or idle flow those answer differently - the live - check flips to False the instant the socket closes - so the colour and the column disagreed. - `_tag_of` now reads the SAME stored `c["scoped"]`; column, highlight, sort (`views.py`) and CSV - (`gui/app.py`) are one signal and can never diverge. `ConnsPage._in_scope` removed; - `engine.in_scope_now` / `core.in_scope` are now unused (flagged for a follow-up removal). -- Updated `tests/test_conns_columns.py`: a flow with `scoped=True` whose port is OUT of the - current target is now both "yes" AND highlighted (it used to be "yes" with an empty tag). -- Not fixed here (separate, measured, handed to a follow-up): a target LAUNCHED after START has - its first connections logged before the resolver matches it (the resolve costs ~180 ms, the - create_time recycle check - see PortTable.info), so they read a truthful "no". Shrinking that - window means speeding up the resolve, which touches the recycle logic; deliberately deferred. - -### Fixed: SocketWatcher STOP crash + slow, unreliable target-by-name (native toolhelp snapshot) - -Follow-up to the chunk 2 rollout, from a field crash log + two reports: a "first target-start takes -seconds" pause AND targeting `chrome` BY NAME resolving to nothing (`BRAK pasującego procesu`) while -targeting its PID worked. Both trace to one slow, fragile path: process-name resolution. - -- **SocketWatcher stop recorded a spurious crash.** `_loop` caught the `WinError 995` that `stop()` - induces - closing the SOCKET handle unblocks the parked `recv()` with "I/O aborted" - and wrote it - to crashes/ via `crashlog.once`, so every STOP left a `socketwatch.loop` entry. Now guarded by - `if not self._stopping.is_set()`, exactly like the capture loop's `if self._running`; a real - socket-stream failure while running is still recorded. Test: - `test_socketwatch.py::test_stop_does_not_record_the_close_induced_error_as_a_crash`. -- **Root cause of both other symptoms: the process-name bulk fallback was `psutil.process_iter` - (measured ~2.6-2.9 s).** When `psutil.Process(pid)` cannot open a HARDENED process (Chrome's - network service and renderers refuse OpenProcess; the individual lookup also fetches ppid + - create_time, any of which can fail), `portmap.info` falls back to that scan. That is why (a) the - first synchronous start/apply resolve blocked for ~2 s, and (b) targeting BY NAME missed Chrome - - no openable name, no match - while BY PID worked (a PID match needs no name). New - `portmap._toolhelp_process_table()` reads every process's name+ppid via `CreateToolhelp32Snapshot` - WITHOUT opening any of them: **measured ~6 ms for 350 procs, and it names hardened processes.** - `_process_table()` prefers it (psutil fallback off Windows / in tests). Cold - `ProcessTargeting.refresh('chrome')` 2148 ms -> ~365 ms, and it now matches Chrome by name. This - also relieves the yes/no flicker: the resolver's periodic refreshes hit the same slow scan, so - targeting was effectively stale/empty for ~2 s and connections in that window slipped to "no". -- The earlier `allow_bulk=False` attempt (previous commit) was REVERTED: it made the resolve fast by - SKIPPING the bulk, which is exactly what broke target-by-name for Chrome (skipping the only path - that can name a hardened process). Making the bulk fast is the correct fix; there is no `allow_bulk` - flag any more. -- Docstring corrected (convention 5): `PortTable.info` claimed `create_time()` = 0.005 ms and - 0.13 ms per rebuild; measured ~5 ms/PID (~180 ms/rebuild with a browser open), because it opens a - handle. The per-refresh recycle-verification cost is now stated honestly and flagged as a perf - follow-up (batch the create times, or verify by a cached snapshot) - it runs on the resolver - thread, not the capture one. -- Tests: `test_processes.py`'s fake psutil now provides `Process` (individual resolution) to match - real psutil, and both `fake_psutil` + `_World` disable `_ALLOW_NATIVE_PROCESSES` so the bulk - fallback reads their fake table, not the real machine's toolhelp snapshot. - -### Added: socketwatch.py - live local_port->pid map from WinDivert SOCKET events (chunk 2a) - -- New module `beantester/socketwatch.py` (`SocketWatcher`): the event-driven replacement for - polling the socket table. WinDivert 2.2 SOCKET layer (sniff-only, `SNIFF|RECV_ONLY`) delivers - BIND/CONNECT/ACCEPT/LISTEN/CLOSE with the owning ProcessId; the map adds on the first four and - removes on CLOSE, pid-checked so a late CLOSE cannot evict a port the OS has recycled to a - different process. `reconcile()` seeds from a `portmap` snapshot and prunes a port absent for - TWO passes (grace against evicting a socket opened microseconds before the snapshot was taken). - Names/ancestors delegate to `portmap` (no duplication); the event source is injected, so the map - is unit-tested without WinDivert. -- Why SOCKET, not FLOW (measured spike 2026-07-22, elevated, sniff-only): SOCKET_CONNECT arrives - ~0.1 ms BEFORE the outbound SYN reaches the NETWORK layer (closes the race); FLOW_ESTABLISHED - arrives ~28 ms AFTER (post-handshake, the SYN already slipped). Two sniff handles (NETWORK+SOCKET) - were confirmed to coexist. The real `_WinDivertSocketSource` was smoke-verified end to end: a - known outbound connection's local port mapped to `os.getpid()` and was removed on close. -- Scope: chunk 2a is the module in ISOLATION. It is NOT wired into `BeanEngine` or - `ProcessTargeting` yet (2b wires the lifecycle + bootstrap + fallback; 2c makes targeting read - the live map). The polling path (`portmap` / `target_resolver`) is untouched and stays as the - fallback for `--simulate` / tests / non-Windows. -- New tests: `tests/test_socketwatch.py` - map add/remove, pid-checked recycled-port removal, junk - rejection, the reconcile two-pass grace (both prune and reappear-resets-grace), name delegation, - the refresh no-op, the reader thread on an injected fake source, and the MSB-first IPv4 decode - the spike corrected. -- `import beantester` still does not import pydivert: the real source constructs it lazily inside - `start()`, so the package import and every unit test stay WinDivert-free. - -### Changed: BeanEngine drives the SocketWatcher lifecycle (chunk 2b) - -- `BeanEngine` now creates and starts a `SocketWatcher` in `_start_locked` and stops it in - `_stop_locked`, next to the `TargetResolver` (both hold OS handles; both are session-length). A - new `start(..., socket_source=None)` parameter injects the event source for tests. Bootstrap: - start reconciles the watcher from a forced `portmap` snapshot, so connections open BEFORE the - session are known from the first packet; the watchdog folds a fresh snapshot in each tick as the - safety net (a missed CLOSE ages out, a dropped event is recovered). -- Started ONLY on the real-WinDivert path (`divert is None`) or with an injected source; on the - synthetic/simulate/test path `self._socketwatch` stays None and the poller stands - the - testable-without-WinDivert contract is intact. `_start_socketwatch` DEGRADES to the poller if the - SOCKET handle cannot open (recorded via `crashlog.once`) rather than failing the session - (convention 20 spirit: a second-handle failure must not take the user's network down). -- NO behaviour change yet: targeting still reads the polling table; the watcher is kept live but - unused until 2c. Verified end to end by a real-path smoke (elevated, narrow pass-through filter): - the engine opened the NETWORK impairing handle AND the SOCKET watcher together - retiring the - coexistence risk flagged in the 2a design - and a known connection's local port mapped to - `os.getpid()` in the watcher, cleared on stop. -- New tests: `tests/test_socketwatch_wiring.py` - bootstrap+run on an injected source, no watcher - on the synthetic path, degrade-not-kill on a source that fails to open, and no watcher thread - left after stop. Driven on an idle `FakeDivert` (the session stays up) with a fake port table. - -### Changed: process targeting resolves against the live socket map (chunk 2c) - -- `ProcessTargeting` gained `set_table()`, and the engine now points it at the `SocketWatcher`'s - live map when a session has one (`_targeting_table()` returns the watcher, else `portmap`), - rebinding in `_start_locked` (targeting is often built before start, against the poller). - `_start_locked` was reordered so the watcher is created BEFORE the initial synchronous resolve - - the very first resolve already reads the live map. -- Effect (the point of chunk 2): a connection of the targeted process is in impairment scope the - instant its SOCKET_CONNECT event arrives, not at the next poll - and since SOCKET_CONNECT precedes - the SYN, before its first packet. Verified end to end on real WinDivert (elevated, narrow - pass-through filter): targeting bound to the watcher, and a fresh outbound connection read - `in_scope` in ~0 ms. The polling path stays as the fallback, unchanged, so short-lived - connections only still escape when there is no real WinDivert. -- Only the LIVE path changed: `engine.target_for` now builds against `_targeting_table()`. The - one-shot reporting helpers (`processes.find_process_ports` -> `resolve_ports`, `make_targeting`) - keep resolving against `portmap` - they are display snapshots, not session targeting. -- Prose corrected (rules 5/6): the `targeting.py` docstring claimed the race "cannot be closed, - only made small" - true for the poller, false for the watcher. It now names which table closes - it. PROJECT_NOTES targeting bullet + the targeting ADR moved from "Chunk 2 (planned)" to done. -- New tests: `tests/test_targeting_socketwatch.py` - the set_table swap, an end-to-end resolve of a - CONNECT event through a watcher + resolver (no poll), and the engine binding targeting to the - watcher (present) vs the poller (synthetic path). - -### Fixed: the connections "impaired?" column is a session record, not a live port lookup - -- **Symptom (reported from the field, Chrome):** targeting `chrome.exe` showed a connections - table where the large majority of rows read "no" in the impaired? column, so the tool looked - like it was missing most of the traffic. It was not - the column was misreporting FINISHED - connections. -- **Cause:** `gui/pages/conns.py::_render` computed the column LIVE via - `engine.in_scope_now` -> `BeanCore.in_scope` -> `local_port in target_ports`. A closed or idle - flow's ephemeral port has left the socket table, so the live test returns False for every - connection that is no longer open - which is most of them on a browser. Meanwhile the stored - per-flow `scoped` flag (`engine._log_conn`) tracked only the LATEST packet, and both the CSV - export (`gui/app.py`) and the column's sort key (`views.py::_SORT["scoped"]`) already read - that stored flag - so the on-screen cell, its sort order and the CSV disagreed (two semantics, - three call sites). -- **Fix:** `engine._log_conn` now keeps `scoped` STICKY per flow - (`c["scoped"] = c["scoped"] or bool(scoped)`) - a session-long "was ever in impairment scope" - record - and `conns.py::_render` reads that stored flag instead of the live lookup. Cell, sort - key and CSV now all read the one stored flag. The LIVE "in scope now" signal is unchanged: it - is the row HIGHLIGHT (`_tag_of`, still via `in_scope`), so a chrome->firefox narrowing drops - the highlight without erasing the record. -- **Reversed decision (recorded so it is not re-reversed):** the column was deliberately made - live once, to stop an idle flow keeping a stale "yes" after the target was narrowed. That - concern is now carried by the HIGHLIGHT (live), while the COLUMN is the audit trail (sticky) - - the two signals were conflated into one before. -- New test: `tests/test_engine.py::test_scoped_is_a_sticky_session_record` - three packets on one - flow (in scope, then twice out) keep the flag True, and a never-scoped flow stays False, driven - straight through `_log_conn` (no thread timing). -- Updated test: `tests/test_conns_columns.py::test_connection_columns_tag_and_footer` - the - out-of-current-target `svchost` row now asserts column "yes" (stored record) with NO highlight - tag (live), locking the two-signal split. `tests/test_conns_export.py` and - `tests/test_engine.py::test_connection_records_scope_and_dropped` were already consistent with - the stored flag and pass unchanged. -- i18n: `tips.col_scoped` reworded in `lang/en.json` + `lang/pl.json` (values only; key set and - sort order unchanged, so `test_i18n.py` parity holds). -- Scope: this is the display/coherence half (Chunk 1). The underlying port->PID resolution is - still a periodic socket-table snapshot, so short-lived connections that open and close inside a - refresh window can still escape impairment (`tests/test_target_resolver.py` documents the race); - closing that at the source with the WinDivert FLOW/SOCKET layer is tracked separately. - -### Tests: property-based coverage for the two packet-mutating functions (F6) - -Engineering-review finding F6: the property suites covered matchers and `decide()`, but the -only two functions that reach into packet BYTES - `BeanCore.corrupt_packet` (flips a payload -bit, on the capture thread) and `BeanCore.build_rst_fields` (forges the RST injected onto the -user's live connection) - had example tests only. New file -`tests/test_packet_mutation_properties.py` (Hypothesis, 7 properties): - -- **corrupt_packet:** flips EXACTLY one bit and preserves length; is deterministic for a seed - (reproducibility contract); touches no header field; an empty payload -> False untouched; and - is TOTAL - a packet whose payload cannot even be read comes back False, never an exception - (it runs on the capture thread, convention 20). -- **build_rst_fields:** the endpoint/seq logic restated as ONE invariant across both directions - - the forged RST is always sent from the remote peer TO the local socket, marked inbound, with - the sequence the local end expects next; a non-TCP packet yields `None`. -- Mutation-checked (all three substantive properties bite): an 8-bit flip fails the one-bit - property, dropping the src/dst swap fails the endpoint invariant, and narrowing the `except` - lets the hostile-packet case raise. No production code changed. - -### Tests: cover the driver STOP+DELETE path and the in-app dialogs (F5) - -Engineering-review finding F5: the two lowest-covered spots were both error/teardown -code that only ever runs on a user's machine. No production code changed. - -- **`driver.stop_and_remove` was 0%** - it STOPS and DELETES a Windows service and runs on - the way out of every real-capture session (`release_on_exit`), so it must not be exercised - for the first time in the field. It is pure Service-Manager glue, so `tests/test_driver_windows.py` - now drives every branch through a fake advapi (`_FakeAdvapi` + a real `ctypes.Structure` so - `byref` has a target, `is_windows` forced True so it runs identically on the Linux CI): stop+ - delete, a delete that will not take, no SCM handle, access-denied vs not-installed vs an - unexpected error, and the off-Windows no-op. Plus four `cleanup_driver` orchestration cases - (per-service loop, stale `_MEI*` temp dirs, the admin gate, nothing installed). driver.py - 75% -> 88%; the stop_and_remove block (was fully uncovered) is now exercised. Mutation-checked: - forcing the delete to read as failed turns the success test red. -- **`gui/dialogs.py` was 16%** - the dark in-app modals. New `tests/test_dialogs.py` drives them - on the fake tkinter (where `wait_window` is a no-op, so each modal builds and returns its - dismissal default) and exercises `_close` directly. 16% -> 85%; the remaining lines are the - `crashlog.note` except-branches that only fire when a real Tk call raises, which the fake - cannot provoke. - -### Fixed: STOP no longer blocks for 2 s when it races the duration deadline (F2) - -Engineering-review finding F2, measured before and after: a user STOP colliding with the -session deadline took **2091 ms**; it is now **~160 ms** (40-trial worst case). - -- **The deadlock:** `stop()` holds `_stop_lock` and joins the worker threads with a 2.0 s - timeout. When the watchdog fired the duration deadline at the same instant, its - `stop(reason="duration")` blocked waiting for that same lock while the user's `stop()` was - blocked waiting to join the watchdog - a lock-ordering inversion broken only by the join - timeout. The capture thread's `_fail_stop` -> `stop()` had the same shape. -- **The fix:** `stop()` is split into `stop()` (external callers - GUI/CLI/atexit/tests - - which BLOCK on `_stop_lock`, preserving start/stop serialisation), `_worker_stop()` (worker - threads, which take the lock NON-blocking and bow out under contention), and the shared - `_stop_locked()` body. The watchdog's deadline and liveness stops go through `_worker_stop`; - `_deadline` is now cleared at the TOP of the stop body so a watchdog finishing a slow op sees - nothing to fire. -- **Subtlety that cost a round:** `_fail_stop` is called by BOTH the capture thread and the - watchdog. Routing both through `_worker_stop` regressed `test_a_dead_capture_thread_fails_open` - - a divert that faults on its first reads does so while `start()` still holds `_stop_lock`, so - the capture thread's non-blocking stop no-opped and the watchdog stopped it a tick later with a - generic "died unexpectedly" instead of the real "driver went away". So `_fail_stop` grew a - `blocking` flag: the capture thread blocks (safe - an external STOP closes the divert first, so - the capture loop sees `_running` False and never reaches `_fail_stop`), only the watchdog's - liveness path is non-blocking. -- New test `tests/test_failsafe.py::test_a_worker_stop_never_blocks_on_a_held_stop_lock` - holds - `_stop_lock` and asserts `_worker_stop` returns anyway (structural, not wall-clock, so it cannot - flake). Mutation-checked: making `_worker_stop` blocking turns it red. - -### Fixed: BeanEngine.start() is now atomic - a partial start fails OPEN (convention 20) - -Engineering-review finding F1, confirmed by experiment before the fix (forced `Thread.start()` -to raise after N workers and inspected the engine state), not by reading. - -- **The hole:** `_start_locked` set `self._running = True` and opened the divert BEFORE spawning - the resolver + capture/inject/watchdog threads, and called `_LIVE_ENGINES.add(self)` only AFTER - all three were up. A failing `Thread.start()` (thread/memory exhaustion - most likely under the - load this tool is aimed at) therefore left a "running" engine with an OPEN divert, no capture - thread draining it, and **invisible to the `atexit` hook** - WinDivert queueing the user's - packets into a void while the UI said "running". Worse, `_running` stayed True, so every later - `start()` hit the `RuntimeError("already running")` guard: START was wedged for the process - lifetime. GUI `_finish_start(err)` only shows a dialog and resets the button; it never calls - `stop()`. -- **The fix:** `_LIVE_ENGINES.add(self)` moved to BEFORE the worker-spawn block (the moment the - divert is open + `_running` is the moment atexit must be able to find it), and the spawn block - wrapped in `try/except BaseException` that logs the fault, calls `self.stop(reason="fault")` - (closes the divert, stops/joins whatever DID start, clears `_running`, discards from - `_LIVE_ENGINES`) and re-raises. `_stop_lock` is an `RLock`, so the nested `stop()` from inside - `start()` re-enters cleanly. -- New test: `tests/test_failsafe.py::test_a_failed_start_never_leaves_an_open_divert` - monkeypatches - `threading.Thread.start` to fail after the resolver thread, then asserts: the error propagates, - the engine is not left running, the divert is closed, it is gone from `_LIVE_ENGINES`, and a - later `start()` succeeds (no longer wedged). - -### Fixed: corrupt_packet() records its failures (F3); a core-scoped guard so it cannot swallow again (F4) - -Engineering-review findings F3 + F4. - -- **F3:** `BeanCore.corrupt_packet`'s `except Exception: return False` swallowed a REAL - failure (a raising `packet.payload` setter, a foreign packet type) in a way - indistinguishable from its legitimate empty-payload `return False`. A broken corruptor - therefore read as `corrupted == 0` - "the traffic had no payloads" - and the tester - would blame their traffic, not the tool. It now calls `crashlog.once("core.corrupt", - exc)` before returning False. `crashlog` is imported LAZILY inside the handler, so - core.py still imports only utils/matchers at load (layering contract) and stays free of - logging/print in the hot path; `once()` caps the cost at one traceback. Verified by - experiment: a raising setter now lands one `core.corrupt` record and returns False, - while the empty-payload path stays a quiet False (no crash-log spam). -- **F4:** `test_no_silently_swallowed_exceptions` only recognises a `pass`/`...` body, so - the `return False` swallow above passed it for as long as it existed. New guard - `tests/test_code_hygiene.py::test_the_decision_core_never_swallows_an_exception_silently` - asserts the stronger property for core.py ALONE: every broad `except` must reach - `crashlog` (quiet/once/note/record) or re-raise. Scoped to the decision core on purpose - - the wider package's 50-odd broad handlers are legitimate control-flow fallbacks - (parse -> None, `matches()` -> False by hot-path contract, a DPI probe -> default), so - holding them to this rule would fire on correct code. Mutation-checked: reverting F3 - turns the new guard red on `core.py:626`. - -### Fixed: gitignore coverage artefacts (F7), drop stale numbers from the CI comment (F8) - -Engineering-review findings F7 + F8. Neither ships; both are the "prose nothing guards" -class convention 5 warns about. - -- **F7:** `.gitignore` matched only the bare `.coverage`, but `[tool.coverage.run] - parallel = true` (pyproject) GUARANTEES per-process `.coverage...` - files, and the CI coverage step writes `coverage.xml`. Both appear as untracked after a - coverage run, and `git add -A` would have committed them (against convention 3). Added - `.coverage.*` and `coverage.xml`. Verified with `git check-ignore`. -- **F8:** the comment on the coverage step claimed "the same suite reads 51% instead of - 77%". 77 was the PREVIOUS gate value (it has since moved 75 -> 77 -> 80), and the real - measured split lives in pyproject (45% vs 83.03% when measured). The comment no longer - restates any number - it points at pyproject, the single source, so it cannot drift again. - -### PROJECT_NOTES audit, part 2: measured numbers, and the coverage gate to 80 - -Every "costs N ms" in the audit's blast radius was re-measured instead of trusted. Conditions -are now attached to each figure (Win11 AMD64, CPython 3.14.6, median of 7), because a number -without conditions cannot be re-verified and the next session cannot tell drift from hardware. - -- **`engine.connections_snapshot(limit=None)` was documented at ~25 ms "at the cap"; it is - 0.7 ms** (2.4 ms at 500k). `conns.py` repeated the same claim as "~70 ms for a 500 000-row - copy" - wrong by ~30x under every interpretation (`list(values())` 2.3 ms, `dict()` 6.4 ms, - per-row copy 222 ms at 500k). Left as an argument for moving the snapshot back onto the UI - thread, which would have been a real regression. -- The decision survives for a **different and verified** reason, now written down instead: - `connections_snapshot()` acquires the engine's `_clock`, the same lock the capture thread - takes on every logged packet, so taking it on the UI thread makes the UI queue behind the - capture thread. Cheap to copy, still wrong to copy there. -- **`views.filter_sort_connections` kept its heap-vs-sort ratio test** - re-measurement confirms - the crossover (top 400: 12.6 ms heap vs 27.7 ms sort; top 50 000: 130.6 ms heap vs 28.0 ms - sort). Only the absolute figures were dated. The docstring now carries a table plus a warning - that first bit this audit: benchmark it with keys from a tiny range and Timsort exploits the - runs, making the sort column look artificially fast and the optimisation look pointless. -- **Coverage gate raised 77 -> 80.** Measured with `COVERAGE_PROCESS_START`: **83.03%** - (83.07% on re-run), so the gate keeps its ~3-point margin for subprocess-coverage variance. - Also measured the counterfactual the comment asserted without evidence: **without** the env - var the same suite reports **45%**, not the 51% claimed. Both numbers, and their conditions, - now live in `pyproject.toml` only. -- Notes-side fixes with no code change: the connection table's "max 400 rows" (stated twice) - is a limit removed when the tables were virtualised - `row_limit` defaults to 50 000, ranges - 0-1 000 000 and 0 means no limit; the scroll cost now has one source (`sortable_tree.py`) - instead of two that disagreed (0.8 ms vs ~1 ms). - -### PROJECT_NOTES audit, part 3: a mechanical guard for the note itself - -`PROJECT_NOTES.md` is git-ignored (private Doc repo), so a pytest test would be skipped in CI -forever and would name a private file inside the public suite. The guard is therefore a Stop -hook, `.claude/hooks/check_notes.py`, next to the existing `check_changelog.py`; it exits -silently when the file is absent. Neither the hook nor the note is part of this repository - -this entry is the public record that they exist. - -It refuses to end a turn when the note drifts in a way a machine can see: a named `tests/*.py` -that does not exist, a named `file.py::test_name` that does not exist, a package module the -note never mentions (the rule `test_readme_guards.py` already applies to the READMEs, which is -exactly why the README tree stayed right while the note's lost `crashlog.py`, `gui/labels.py` -and `gui/rates.py`), and a registered window the note never mentions (`event_log` - the window -whose docstring says "COPY THIS FILE to make a new one" - was undocumented, so the note pointed -newcomers at a worse template). All four checks verified by mutation: green on the real note, -red on each injected drift. It deliberately does not try to check prose. - -### PROJECT_NOTES audit, part 1: prose that would have made the next session write a bug - -A full claim-by-claim audit of `PROJECT_NOTES.md` against the code. Convention 5 ("every -`because` is a claim - check it or do not write it") applied to the note itself. This part -covers the findings that actively mis-instruct; numbers, stale lists and undocumented -mechanisms follow in their own commits. - -- **Convention 16 was backwards about labels.** It told the next session to set - `state="disabled"` on field labels. The code deliberately does the opposite: a - state-disabled `ttk.Label` paints a FILLED BOX, so `ControlForm._apply_toggle_state` and - `apply_overrides` swap the style to `CardOff.TLabel` instead, and - `test_an_overridden_field_is_visibly_disabled` even asserts `state is None` on the label. - The convention also cited `test_gui_layout.py::test_disabled_fields_are_visibly_disabled` - as its guard - **that test has never existed**. Rewritten to separate the field rule - (state + a `disabled` map) from the label rule (style swap, never state). -- **The same stale claim lived in `theme.py`**, five lines below the correct one: the comment - above the label `disabled` maps said field labels "are set to state=disabled together with - their entries". Nothing in the GUI sets `state` on a label. Comment corrected to say what - the maps actually are: defensive, and free. -- Measured, so the note can stop guessing: removing the `disabled` foreground maps for EVERY - label style leaves the whole suite AND `smoke_gui.py` green. The convention now says it has - no guard instead of naming one. -- **Convention 40's guard covered half of what it claimed.** - `test_shortcut_buttons_advertise_their_key` asserted on `btn_start` and `btn_apply` only, so - dropping `shortcut="Ctrl+S"` from the Save button kept the suite green - verified by - mutation. The test now drives a table of all four shortcut buttons and fails naming the - offender; re-run against the same mutation it goes red. "Save file" / "Load file" moved from - local variables to `App.btn_save` / `App.btn_load` so the guard can reach them. -- **Convention 42 described a replaced implementation:** `icon.make_gear_icon` is an - anti-aliased RGBA PNG built with stdlib `zlib`/`struct`, not "plain `PhotoImage.put`". The - per-pixel `put` version had no alpha and rasterised jagged teeth; it survives only as a - fallback for a Tk build that cannot read PNG. - -### The hot-path guard now covers the route Linux takes, on every machine - -`test_hot_path.py` shipped with an explicit "NOT verified" note: `PortTable` reads the socket table -through `iphlpapi` when `_make_native()` succeeds and through `psutil.net_connections` when it does -not, and on Windows the first always wins. So `_psutil_port_pid_map` was watched by the guard and -had never once fired locally - the Linux behaviour was covered only by the ubuntu leg of CI, and -only by accident of which platform happened to run. - -`_make_native` returning `None` IS what a non-Windows platform does, so substituting it exercises -that route anywhere. New `test_the_psutil_socket_table_path_is_just_as_clean` does exactly that. -Measured with the substitution in place, against the same session (traffic, targeting that matches -nothing, five seconds): - -| | native path | forced fallback | -|---|---|---| -| `_Native._table` | 36 calls | **0** | -| `_psutil_port_pid_map` | **0** | 12 calls | -| from a packet thread | nothing | **nothing** | - -The test asserts its own conclusiveness before asserting the invariant - the table really took the -psutil route (`table.native is False`), the fallback lookup really ran, and no native call leaked -through - so a substitution that silently did nothing fails instead of passing quietly. - -Verified by mutation, and this is the part that makes it more than a duplicate: reopening -`_process_for`'s refresh (`allow_refresh=True`) makes it fail naming the OTHER function - -`[('_psutil_port_pid_map', 'Thread-1 (_capture_loop)')]` - where the Windows test names -`_Native._table`. Same regression, second route, and now it is caught on both without waiting for -a particular runner. - -The module docstring's "NOT verified" paragraph is replaced rather than left to rot; it now says -what was measured. - -### One stray lang/*.json stopped the program from starting (audit item #10, the edges) - -`lang/*.json` is the one on-disk format with its own `json.load`, outside `jsonfile`, and it was -left out of the first #10 pass as a shipped file rather than a user file. It is not only shipped: -translations are meant to be added, and `load_languages` promises in its docstring that "a broken -or unreadable file is skipped so it can never break app startup". - -`meta = data.pop("_meta", None) or {}` rescued a FALSY `_meta` - `null`, `0`, `""` - and nothing -else. A non-empty one of the wrong type (`"_meta": "en"`, a list, a number, `true`) sailed past -`or {}` and died on `meta.get()`, which sits OUTSIDE the per-file `try`. The AttributeError escaped -`load_languages`, which runs at startup. Measured with one such file dropped into the real `lang/`: -**`python -m beantester --version` exited 1 with a traceback.** One stray file, no program - CLI or -GUI. - -Fixed with an `isinstance(meta, dict)` check. The file then behaves exactly like one carrying no -`_meta` at all, which is a supported case: the filename supplies the language code and the -translations are kept. That is deliberately NOT "skip the file" - discarding a translator's work -over a typo in one metadata field would be the wrong trade, and the first draft of the test -asserted the wrong thing here before the behaviour was thought through. - -Two more edges measured, neither a bug, both now covered so nobody has to re-derive them: - -- **a directory where a file belongs**: `open()` raises `IsADirectoryError` on Linux and - `PermissionError` on Windows; both are `OSError`, `read_json` reports it, nothing raises. -- **a file that cannot be read**: reported the same way. The test asserts the portable invariant - (it returns, with data or with a message) because `chmod` genuinely blocks reads on POSIX while - on Windows it only toggles the read-only bit. - -An unreadable file is also QUARANTINED, because `quarantine()` renames and renaming needs no read -access. That first looked like a wart worth splitting - `OSError` (do not quarantine, it may be -readable next time) versus `ValueError` (quarantine, the content is unusable) - and this file said -so. **Checking it reversed the conclusion, so the suggestion is withdrawn rather than left as a -trap.** `UiStateStore.persist()` runs unconditionally (on close, and on every window-state change) -and `write_json` ends in `os.replace`. Leave an unreadable file in place and the first save of the -session OVERWRITES it, destroying precisely the content nobody could read. The quarantine is what -preserves it. Current behaviour is correct and must stay. - -CI is what forced the check: `test_an_unreadable_file_is_reported_not_crashed` passed on Windows, -where `chmod` only toggles the read-only bit, and failed on Linux, where it genuinely denies the -read - the test's cleanup chmod'd a fixed path that the quarantine had already renamed away. The -test now restores whatever is actually in the directory, and asserts the preservation half only on -platforms that really denied the read (root ignores `chmod` everywhere). - -Verified by mutation: with `or {}` restored the suite fails with the original -`AttributeError: 'str' object has no attribute 'get'`. - -### --dry-run called a scenario valid without ever opening it (audit item #10) - -`--dry-run` is the gate a CI/CD pipeline runs before the real command. It returned `OK` for every -broken scenario file tried - a bare list, a string, a number, truncated JSON, an empty file - and -printed "Configuration is valid", because the scenario was loaded only inside `_run_session`, -which `--dry-run` returns before reaching. The same files correctly gave `SCENARIO(4)` on a real -run. A gate whose verdict disagrees with the thing it gates is worse than no gate. - -`run_cli` now loads and validates the scenario inside the `--dry-run` branch, failing with -`SCENARIO(4)` and the parse error. `--print-config` and `--save-config` return early as before and -are deliberately left alone: neither claims the configuration is valid, they report or store the -SETTINGS, and a scenario is not part of those. - -**Owner's call: this is a fix, not a BREAKING change** (`--dry-run` on a broken scenario goes from -`OK` to `SCENARIO(4)`). A script relying on the old outcome is relying on the gate lying to it. No -`### BREAKING` section, no version bump. - -Tests: every shape in `BROKEN_JSON` must be rejected by `--dry-run` *and* the output must not -contain the word "valid"; the dry run and the real run must return the SAME code for the same -file; and all seven shipped `scenarios/*.json` must pass `--dry-run` - the check that the fix does -not start rejecting real files. - -Worth recording, because it briefly looked like a regression: the first draft of that test invented -a scenario shape (`{"duration": 1, "loss": 5}`) instead of using the documented one -(`{"at": seconds, "settings": {...}}`). `--dry-run` rejected it, correctly, and for a moment that -read as the fix rejecting good files. The shipped-scenario loop was added as the answer - it -cannot be argued with. - -Verified by mutation: without the change the suite fails with the original symptom, `code=0` and -`Configuration is valid` for a broken scenario. - -### --config with valid JSON of the wrong shape was a traceback, not an exit code (audit item #10) - -`settings.load_config_file` does a raw `json.load` and goes straight to `data.items()`. For -`[1, 2, 3]`, `"x"`, `42`, `null` or `true` the parse succeeds and the type error lands one line -later as an **AttributeError** - which `cli.py` does not catch, since it catches `ValueError` and -`OSError`. Measured on all five shapes: a Python traceback on stderr and exit **1 (RUNTIME)**, -where a bad config file is **CONFIG(3)**. - -Two contracts at once: convention 18 (every way of ending has a code from `exitcodes.py`) and the -comment sitting directly above that `try` in `cli.py`, which promises "a clear CLI error, never a -raw traceback". For a CI/CD pipeline reading the exit code, the difference is being told the tool -crashed instead of being told its config is wrong. - -`load_config_file` now checks the parsed value is a dict and raises `ValueError` otherwise, which -the existing handler already turns into `CONFIG(3)`. Deliberately NOT routed through -`jsonfile.read_json`: quarantine is right for the app's own state files, but silently moving aside -a file the user named explicitly on the command line would be a surprise. - -The GUI path is unaffected - `App.load_config_file` catches `Exception` and shows a dialog - so -this was CLI-only, which is where the exit-code contract lives. - -Tests in `tests/test_ondisk_formats.py`: every shape in `BROKEN_JSON` through `--config` must give -`CONFIG(3)`, an `error:` line on stderr and a clean stdout, plus the other direction - a -well-formed config file still loads. Run in-process, so an exception escaping `run_cli` fails the -test with its own traceback, which is the failure mode being guarded. Verified by mutation: -without the check the suite fails with the original `AttributeError: 'list' object has no -attribute 'items'`. - -### ui.json: a valid dict with the wrong types stopped the app from starting (audit item #10) - -`jsonfile.read_json` guarantees the file parses and is a dict. Nothing beyond that - and -`UiStateStore` trusted the rest, while its own module docstring promises that corruption "must -never break startup, so every failure degrades to the defaults". Measured, by building a real -`App` over a poisoned `bean_network_tester_ui.json`, three keys broke that promise outright: - -| key | value | result | -|---|---|---| -| `page` | `[1, 2, 3]` | `TypeError: cannot use 'list' as a dict key (unhashable type: 'list')` | -| `conn_sort` | `[1, 2]` | `TypeError: object is not iterable` | -| `event_sort` | `"kb"` | `ValueError: dictionary update sequence element #0 has length 1; 2 is required` | - -The window never appeared, and the traceback named none of the files involved. - -`UiStateStore._clean` now drops values whose TYPE is not the one `DEFAULTS` promises, records -which keys it ignored in `self.problem` (already surfaced by `App._report_storage_problems` -through the existing `log.ui_state_problem` key, so no new i18n), and keeps everything else. It is -deliberately the same shape as `ProfileStore._clean`, which has always done this for the other -user file - the mechanism existed, it just was not applied here. - -**Only the TYPE is checked, and that is a measured decision, not caution.** Every wrong VALUE of -the right type was tried first and already degrades gracefully: an unknown page id, an unknown -stats sub-page, an unknown language code, a missing profile name, a nonsense geometry string, a -negative or absurd sash position, a sort column that does not exist, `collapsed` holding ints or -nested lists, `conn_sort` with a list under `col`. Validating further would add rules that catch -nothing. Unknown keys are kept on purpose: `get` only reads keys it knows, and dropping them would -silently discard state written by a newer version. - -New `tests/test_ondisk_formats.py` (4 tests, more coming for the config and scenario paths): -per-key type fuzzing at the store level, the whole poison set driven through a real `App` in one -subprocess, and the unparseable-file half - every shape in `BROKEN_JSON` must leave usable state, -a reported problem and a `.corrupt-` file rather than a clobbered one. - -Verified by mutation: with `_clean` removed the suite fails with the original symptom, -`the app did not start: page=[1, 2, 3]: TypeError: cannot use 'list' as a dict key`. - -### The chaos test was measuring the machine, not the code - -CI failed `test_the_model_worker_survives_a_live_connection_table` with -`traffic really flowed while it did (8342)`. The test asserted `seen > 10_000` after a fixed three -seconds - a threshold read off a dev machine, where the same three seconds produce hundreds of -thousands of packets. A CI runner under coverage managed 8342, about fifteen times less. - -The interesting part is that the test had already reached the state it exists for: the -`rows > 1000` assertion, checked one line earlier, PASSED. The table was big enough for the sort to -be real work. Only the packet count - a proxy for the same thing, and a worse one - was out of -range. A green run on a fast machine and a red one on a slow machine, with identical behaviour -under test. - -Fixed by asserting the CONDITION and waiting for it, instead of assuming a duration produces it: - -- the request/poll loop now runs for at least `STRESS_SECONDS` and then keeps going until - `MIN_BUILDS` rebuilds have completed over a table of at least `MIN_ROWS` rows, with a 30 s hard - cap so a broken run still ends; -- the packet-count assertion is gone. The row count already implies traffic - a thousand distinct - flows cannot exist without it - and counting packets measured the runner; -- `FastDivert`'s docstring now says its measured throughput is a dev-machine number and not a - promise, so nobody turns it back into a threshold. - -Verified by simulating the slow runner rather than by hoping: throttled to ~3000 packets/s (below -what CI managed), the old assertions fail with the exact CI symptom -(`traffic really flowed while it did (2429)`) and the new ones pass. Unthrottled, both pass. - -### A hot-path guard that watches the routes, not one object (audit item #8) - -The rule is one sentence: nothing on the capture or inject thread may ask the OS a question. It -already had a guard - `test_target_resolver.py::test_the_capture_thread_never_touches_the_socket_table` -- but a narrow one, and its first limitation has already bitten this project: - -- **it watches an object, so it can watch the WRONG object.** An earlier version gave the counting - table only to the targeting and left the engine on `portmap.default_table()`. It passed while - `_log_conn` -> `_process_for` -> `process_for_port` rebuilt the real table ~16 times a second on - the capture thread. A live run caught what the test could not. -- **it only knows about the socket table.** Targeting is one route to the OS; `_log_conn` is a - second, independent one; a third would be invisible to it. - -New `tests/test_hot_path.py` watches the ROUTES instead. `portmap` is the only module in the -package that touches `psutil` or `iphlpapi`, through five entry points (`_psutil_port_pid_map`, -`_psutil_process_table`, `_psutil_created`, `_psutil_process_info`, `_Native._table`). Wrapping -all five catches any caller, including one nobody has written yet. Threads are compared by -IDENTITY against `engine._t_cap` / `_t_inj`, not by name substring, so it does not depend on how -CPython happens to name a thread. - -The target expression deliberately matches nothing. With no matching port every packet is a miss, -so the resolver is woken continuously and the surface gets hammered - and the test stops depending -on which processes exist, so it means the same on a CI runner as here. Measured during a session -with traffic and targeting active: - -| calls | function | thread | -|---|---|---| -| 5232 | `_psutil_created` | bean-target-resolver | -| 1431 | `_psutil_process_info` | bean-target-resolver | -| 448 | `_psutil_created` | watchdog | -| 212 | `_Native._table` | bean-target-resolver | -| 3 | `_psutil_process_table` | bean-target-resolver | -| **0** | **anything** | **capture / inject** | - -The work is real and heavy, and none of it is where the user's packets wait. That is the design, -now asserted. - -**Found no bug** - the invariant holds today. Verified by mutation, negative included: - -- **caught:** `_process_for` reopening the refresh (`allow_refresh=True`), the regression that has - already happened twice. The failure names both ends: - `[('_Native._table', 'Thread-1 (_capture_loop)')]`. -- **not caught:** a name lookup on the capture thread that HITS the warm info cache. That is the - guard's boundary, not a hole - it watches trips to the OS and a cache hit is not one - but a - regression that only misses the cache occasionally will only be caught occasionally. - -A second test asserts the RECORDER records: a wrapper that silently failed to install would leave -the guard permanently and invisibly green. - -**Deliberately not a wall-clock budget.** The suite's existing timing assertions -(`test_failsafe`'s "start did not block the UI thread", `test_target_resolver`'s "stop did not wait -for the scan") all separate outcomes differing by an order of magnitude. "The hot path costs under -N microseconds" has no such separation: on a shared CI runner it measures the runner, and the first -thing anybody does with such a test is widen the bound until it stops failing. - -**Not verified:** the Linux path. On this machine `_Native` initialises, so the `psutil` fallback -for the socket table never runs; `_psutil_port_pid_map` is watched but was never seen to fire here. -CI runs ubuntu too, where it is the main path. The assertions are written against the TOTAL over -the surface rather than any single function so they hold either way, but the ubuntu behaviour is -unverified locally. - -Stability: 10 consecutive runs, 0 failures, median 3.8 s. - -### Chaos through the whole stack (audit item #11, part 2) - -New `tests/test_gui_stack_chaos.py`: the real `App` on the fake tkinter, a real engine on -synthetic traffic, the real connections page and its `AsyncModel`, and the `_tick` loop running -throughout - while a simulated user switches pages, types in the search box, flips sort columns, -toggles "freeze", and stops and restarts the session mid-rebuild. - -The combination is what matters. The off-main-thread Tk call in the old target refresher survived -every test precisely because nothing ran the pieces together, and the fake tkinter is single -threaded so it could not have seen it. - -Three choices worth recording: - -- **The off-main-thread check watches the fake's widget base class**, not a handful of named - widgets. A named spy only catches the widget somebody already suspected; patching `W.configure`, - `W.pack`, `W.after` and friends covers every widget in the app, including ones added later. - Note `config = configure` in the fake binds the ORIGINAL function at class creation, so both - names need patching - patching one silently misses half the calls. -- **A failed tick is detected through the LOG, not through the loop surviving.** `_tick` catches - everything by design (the loop must outlive a broken tick) and reports `log.ui_error`, so - "the loop kept running" is true even when every single tick failed. The test takes the literal - part of the translated template, ahead of the `{e}` placeholder, and asserts no line carries it. -- **Scope is stated in the docstring:** this is about thread boundaries, not volume. The traffic - is `SyntheticDivert` on a twelve-row table; making the sort big enough to matter is part 1's - job. Saying so keeps the next session from reading it as a load test. - -Every assertion was verified by injecting the failure it exists for, and confirming the run goes -red: a widget touched from a worker thread, a tick that raises (injected into `_sample`, which -only `_tick` calls - the first attempt broke `conns.refresh`, which the test body also calls -directly, so it blew up on the wrong line and proved nothing), and a wedged model worker whose -`busy()` never clears. - -Stability, which is the risk with a test like this: run 10 times consecutively, 0 failures, -median 3.4 s. Suite 591 -> 593 tests, +5.2 s (157.6 s -> 162.8 s). - -### The model worker meets a live engine (audit item #11, part 1) - -`test_concurrency_chaos.py` was engine-only, and the seven `AsyncModel` tests all feed the worker -a fake `build`. Nothing put the two together - which matters because `ConnectionsPage.refresh()` -hands the worker **the engine itself**, not a snapshot of it (a snapshot is ~70 ms at half a -million rows, most of what moving the sort off the UI thread bought back). So `_build_model` calls -`connections_snapshot()` on the worker thread, and that returns `list(self._conns.values())`: -the outer list is a copy taken under the lock, but every row in it is the live dict the capture -thread keeps updating. `model_worker.py` asserts in prose that this is safe. Nothing checked it. - -New `test_the_model_worker_survives_a_live_connection_table` runs the real pipeline - snapshot, -filter, sort, totals, scope - on the real `AsyncModel` against a real engine under load, while -settings and targeting churn underneath. - -**The traffic had to be built for it, and the reason is measured.** `SyntheticDivert` sleeps once -per packet, and Windows timer granularity turns that into a ceiling: it delivers **~1900 -packets/s whatever `gen_kbps` says** (2000 kbps and 1 Gbps both land there), over a flow space of -three local ports against three hard-coded remote addresses - so the connection table stops at -**12 rows** however long a test runs. A model-worker test on that table sorts twelve rows and -proves nothing. `FastDivert` (test-local, unthrottled) measures **~126 000 packets/s and ~125 000 -connection rows in three seconds**. It stays in the test file on purpose: production has no use -for an unthrottled generator, and widening `SyntheticDivert` to make a test look better would be -changing the tool to suit the test. - -Verified by mutation, and the negatives are recorded in the test docstring so nobody re-derives -them: - -- **caught:** `connections_snapshot` returning the live `dict.values()` view instead of a copy - under the lock - the tempting optimisation here, and the one that turns every rebuild into a - race with flow creation. -- **not caught:** taking the copy without the lock (window too narrow to hit in a few seconds); - iterating a row (`dict(c)`, `**c`, `.items()`). The second is harmless only because `_log_conn` - builds each row with its full key set and never adds one later, so a row never changes size - - if that ever stops being true, this test will not warn anybody. - -The test watches `crashlog.note` as well as the thread excepthook. `AsyncModel._run` catches -everything, records it and keeps the previous table on screen, so a worker raising on every build -would otherwise leave a green test and a quietly frozen table. It also asserts it ran in the -regime it claims (builds completed, table over 1000 rows, over 10 000 packets seen): a green run -that never got there would be decorative. - -Stability: the file was run 10 times consecutively, 0 failures. - -### stop() releases the divert before anything that can block - -CI caught `test_failsafe.py::test_engine_stops_itself_when_the_duration_elapses` failing on -master. Not a flake to silence: measured, it failed **10 runs out of 30**, and the cause was a -real ordering problem in the stop path. - -`stop()` sets `_running = False` first, which it must - `_capture_loop` runs `while self._running` -and that flag is how it ends. But the divert was closed sixteen lines further down, after -`stop_scenario()`, `_resolver.stop()`, `log_event()` and `notify_all()`. Between those two points -the capture thread is already gone and the divert is still open, so WinDivert keeps diverting into -a queue nobody drains - the exact failure FAIL-OPEN exists to prevent (convention 20). It is -invisible on a synthetic divert, whose `recv()` blocks until close, and real on the live one, -whose `recv()` returns immediately under traffic. - -The window was not theoretical. `_resolver.stop()` joins with a 0.25 s timeout and a resolve in -flight uses it: an earlier session measured STOP at 252 ms with a scan running against ~100 ms -idle, and recorded that as STOP latency. It was also a quarter of a second of the user's packets -queued into a void. - -Measured here with a divert whose `recv()` returns immediately and a 200 ms resolver join - time -the divert stayed OPEN after the capture thread had left: - -| | before | after | -|---|---|---| -| idle resolver | +0.04 ms | -0.36 ms | -| resolver mid-scan (200 ms join) | **+200.06 ms** | -0.40 ms | - -Negative means the divert was closed before the capture thread finished leaving, which is the -point - closing it is what ends that thread. - -- `engine.stop()`: the `_divert.close()` block moves up, directly after the stop bookkeeping and - ahead of `stop_scenario()` / `_resolver.stop()` / `log_event()`. -- It deliberately does NOT move above `self._running = False`. Checked: `recv()` would then raise - while the session still looked live, so `_capture_loop` would take the `_fail_stop` path and - report a fault for an ordinary stop - which would also break `test_concurrency_chaos`'s - `engine.fault is None`. A microscopic window is unavoidable; the point is that no join sits - inside it. - -Two changes on the test side: - -- `test_engine_stops_itself_when_the_duration_elapses` stopped treating `not is_running()` as - "stop() has finished". It is not: the flag drops at the top of `stop()` and every promise (the - divert closed, the STOP event logged, the workers joined) lands afterwards, so waiting on the - flag and asserting a post-condition in the next statement is a race by construction. The test - now waits for the post-conditions themselves. Worth recording: reordering the close alone did - NOT make it green - the failure simply MOVED to the STOP-event assertion, which is how the - wider problem surfaced. -- New `test_stop_releases_the_divert_before_anything_that_can_block` asserts the ORDER (close - before the resolver join) rather than elapsed time, so it cannot flake. Verified by mutation: - with the production change reverted it fails with `['resolver.stop', 'divert.close']`. - -Verified: `tests/test_failsafe.py` run 40 times consecutively, 0 failures (10/30 before). - -### Property tests for the decision pipeline (audit item #9) - -`BeanCore.decide()` is a twelve-step pipeline over twenty-odd interacting fields, and every test -it had pinned ONE step at a time. `test_passthrough.py` already drove it with Hypothesis, but -only in the "everything switched off" configuration, so the INTERACTION between armed steps was -untested. New file `tests/test_core_properties.py`, 8 tests: - -- **Totality.** `decide()` never raises, across the settings space x packet shapes. An exception - there kills the capture thread with the divert still open, which is the fail-open failure of - convention 20. `flap_enabled` is generated INDEPENDENTLY of `flap_period`, so "enabled with a - zero period" stays covered - that combination is reachable only through the setter, because - `apply_settings` derives the flag from the period. -- **Structural coherence.** `drop` and `releases` are the same statement made twice: a dropped - packet has no release times, a delivered one has one or two, a duplicate never precedes its - original, nothing is released before it arrived, and `emit_rst` implies a drop. The engine - injects straight from `releases`, so an incoherent Decision is a lost or a doubled packet. -- **Pipeline order.** Each deterministic gate (lan, block, nat, rst, flap, mtu, syn), armed alone - against 100% loss/corruption/duplication/spike, still names its own reason; and an earlier gate - beats a later one. The order is documented as a contract in the module docstring - this states - it as a test. -- **An unnamed drop belongs to loss.** With `loss = 0` every drop carries a reason. That is how - the engine picks the counter, and `test_passthrough`'s DAMAGE_COUNTERS assertions rest on it. -- **Out of scope means untouched, and leaves no trace.** An off-target packet neither charges the - token bucket nor writes a flow-table entry. Not covered anywhere before: if observation charged - the bucket, merely watching a busy machine would eat the shaped link of the application under - test, and the measurement would be wrong invisibly. -- **A bounded buffer bounds the added delay.** Generalises `test_bandwidth_buffer` across rates, - packet sizes and buffer depths. - -Two details worth recording, because they cost time: - -- The bound in the buffer property is `max(buffer_s, size / rate)`, NOT `buffer_s`. A packet - arriving into an EMPTY buffer is always accepted, even when its own serialisation takes longer - than the whole buffer - deliberate, and guarded by - `test_bandwidth_buffer.test_empty_buffer_never_blacks_out_the_link`. Writing the bound as plain - `buffer_s` yields a test that is RED against correct code: measured, a 10 ms buffer with a - 65535 B packet at 1 KB/s leaves the bucket 64 s ahead of `now`, 6400x the buffer. -- `set_schedule()` reads `time.monotonic()` directly, so a core carrying a schedule is only - deterministic once `reset_buckets(t)` has run after it. Production always does - (`BeanEngine.start`); a property test that skips it flakes on the schedule position. Recorded - in the test module docstring rather than changed - the coupling is harmless in production, and - a rewrite here buys no stability. - -**These tests found no bug.** The pipeline survived every attempt to falsify them before the file -existed: 3000 Hypothesis examples across the settings space and 300-seed sweeps per gate. Their -value is the regression net. The pipeline GROWS - step 2c (blocking) was added after the pipeline -was first documented - and a step inserted at the wrong position is invisible to example tests -that each arm a single knob. Every property was then verified by MUTATION: each guard was -confirmed to go RED against a deliberately broken `core.py` (the `rate > 0` guard removed, a drop -carrying a release time, MTU moved ahead of LAN mode, flapping dropping unnamed, the targeting -gate marking a packet in scope, the flow table written before that gate, and the tail drop -disabled). A guard that stays green under its own mutation is decoration. - -Suite: 582 -> 590 tests, +4.9 s (149.3 s -> 154.2 s, measured on this machine, not estimated). - -### The capture thread could still reach psutil - and the fix for that broke the process column - -Two findings from reviewing the PID-reuse diff, both verified by running them. - -- **Identity verification put a psutil call back on the capture thread.** `engine._process_for` - reads with `allow_refresh=False`, but that only gated the socket-table rebuild - the NAME lookup - underneath it went on to `info()`, which now verifies. Measured with a port table that actually - resolves: 12 `create_time()` calls from `Thread-1 (_capture_loop)`. Once per NEW FLOW rather - than per packet, so 3/s here - but this tool gets pointed at load generators and port scans, - where new flows arrive in thousands per second. -- **Worse, the same hole predates this branch.** Checked against `master`: on a cache MISS the old - `info()` called `_psutil_process_info` from whatever thread asked, so the capture thread could - already trigger a resolve (~5 ms) and even a full `process_iter()` (~1.7 s). The previous chunk - stopped `process_for_port` from refreshing the socket table and left that path open. -- Fixed with an explicit `cheap=True` mode on `info()` / `name_of()`, wired from - `process_for_port(allow_refresh=False)`: **answer from the cache or not at all.** Resolving a - name and verifying an identity are both psutil calls, and gating only one of them is what left - the packet path making the other. Re-measured warm and cold: zero psutil calls from the capture - thread in both. -- **That fix then emptied the connection log's process column** - the regression is only visible - with no target set, which is most sessions: the capture thread no longer resolves, and the - resolver only fills the cache for PIDs it matches, so with no target nothing filled it at all. - Measured: 6 rows, 0 names. The column exists precisely because it used to read "?"; shipping - that back would have undone a fixed bug. `PortTable.warm_names()` now runs on the WATCHDOG next - to the socket-table refresh - cheap in the steady state (one identity check per PID, ~0.13 ms), - paying the real resolve once. Re-measured: 6 rows, 6 names, with and without targeting, and - still zero psutil from the capture thread. - -### portmap: a PID is a number, not an identity (audit item P2) - -- **The `pid -> (name, ppid)` cache could not expire, by any route.** `_expire_info` returned - early below 512 entries (a normal machine holds 26-343, so it never ran), and `info()` bumped - the timestamp on every cache HIT - which made the entry of a busily-read PID immortal, i.e. - exactly the entry decisions rest on. Both reproduced against the real table: a target - restarting onto a recycled PID was **not impaired**, and an innocent process inheriting the - target's old PID **was**. The second is the serious one: this tool breaks networking, and - breaking an application the user never named is the worst thing it can do quietly. -- **Fixed by verifying identity, not by guessing at ages.** Each entry now carries the process - START TIME (`create_time`), and every cache hit checks it. The analysis had rejected this as - "costs as much as re-resolving" - **that was wrong by three orders of magnitude**, and measuring - it is what found the right design: - - create_time() for 2 PIDs : 0.01 ms full re-resolve: 9.8 ms - create_time() for 8 PIDs : 0.03 ms full re-resolve: 38.9 ms - - `name()` is expensive because it must open the process and read its image path; `create_time()` - does not. On this machine it succeeds for **24/24** socket-owning PIDs, including the protected - ones that make `name()` fall back to a full `process_iter()`. -- **"Cannot tell" is not "recycled".** Treating a missing start time as proof of reuse looked like - the safe reading and was in fact a way to destroy the cache wholesale on every fallback path: - each lookup evicted, re-resolved, failed to stamp, and evicted again, so process names came back - empty. Caught by the suite going red on the psutil fake. `_looks_recycled` now returns True only - when both stamps are known AND differ; unverifiable environments fall back to the TTL exactly as - before. Hardening must not degrade what it cannot harden. -- Two cheaper mechanisms kept as backstops: the TTL now counts from INSERTION and runs - unconditionally (2.2 us a sweep, measured), and a PID that loses every socket is forgotten at - once (2.5 us, measured) - a PID can only be reissued after its owner exits, and exiting closes - its sockets. -- **Cost, measured properly on a second pass.** The first figure recorded here (1.4 -> 2.96 ms, - ~5% of a core) was an AVERAGE polluted by a single outlier and roughly double the truth. Isolated - by stubbing `_psutil_created` and comparing medians over 40 runs each, with a control run to - confirm reproducibility: - - with verification 1.29 ms (control: 1.28 ms) - without 0.93 ms - delta +0.35 ms (+38%) - - At the resolver's measured 17 rebuilds/s that is **22 ms/s, 2.2% of one core** - and 2.6% of the - 0.05 s floor it has to fit inside. On the RESOLVER thread; the capture thread is untouched, which - is what the previous chunk bought. A batch verification in `PortTable.refresh()` would shave that - to ~0.2%, and is deliberately not taken: it splits one mechanism into two and leaves ancestors on - the TTL, to save two percent of a background thread that is not short of time. -- Two more things checked rather than assumed. The 0.001 s tolerance in `_looks_recycled` is never - actually needed here - across 342 processes, `process_iter` and `Process.create_time()` agreed to - **0.000000000 s**, so there are no false "recycled" verdicts; the tolerance stays as defence on - platforms that are less exact. And PIDs that lose every socket and come back do exist (2 of them - oscillated 3-4 times in 10 s of observation), costing about 0.8 extra resolves a second - noise - against the numbers above. -- Verified beyond the suite: a **real** child process with a **real** socket resolves to - `python.exe` while alive and to `""` the moment it exits - no stale name survives. A 10 s live - session with targeting: 9020 packets, 171 rebuilds, 23 targeted ports, STOP 18 ms, no thread - left behind. -- New tests in `tests/test_processes.py`, on a controllable `_World` (ports, processes, start - times): the restarted target is impaired, the innocent inheritor is not, a living process keeps - its entry (verifying must not become re-resolving), an unverifiable environment still resolves - names, expiry works below the old 512 threshold and a busily-read entry no longer renews itself, - and a PID that loses every socket is forgotten at once. - -### STOP no longer waits for a resolve, and the number that explains why - -- **Measured, because nobody here knew it: a COLD resolve costs 1.7 SECONDS.** On this desktop - 25 PIDs own sockets but the process-info cache ends up with 346 entries - the expensive part is - one full `psutil.process_iter()`, triggered the moment a protected PID refuses `psutil.Process`. - Once warm the same resolve is **1.4 ms**. A thousandfold difference that every fake in the - suite hides, because fakes answer instantly. -- **That made `stop()` slow, and STOP is the control this tool may never make slow.** The - resolver joined with a 2 s timeout, so pressing STOP while a cold scan was in flight blocked - for **1647 ms** (measured; with an artificially slow table it ate the full 2000 ms and still - left the thread running). The old GUI refresher was an unjoined daemon, so this was a - regression introduced by the rewrite. -- Fixed with `TargetResolver.JOIN_S = 0.25`: long enough that an IDLE resolver is always joined - (it is parked in `wait()` and exits in microseconds), short enough that a scan in flight can - never hold STOP up. Not joining a straggler is safe - `stop()` has already cleared the target - and set the stop flag, so it finishes at most one more scan into an object nobody reads and - then exits; it is a daemon either way. Re-measured: **252 ms** with a 1.7 s scan in flight, - **265 ms** with a 5 s one, **100 ms** idle (and that 100 ms is the engine's other joins). -- Guards: `test_stop_never_waits_for_a_scan_in_flight` (deliberately slow table, asserts under - 900 ms) and `test_stop_does_join_an_idle_resolver` (the other half - the common case must be - clean, not merely fast). -- **A full GUI session was driven end to end for the first time** - real engine, real resolver, - synthetic traffic, real GUI code - because everything until then had exercised the engine - directly and left `_tick`'s new wiring unverified. Verified: resolver up for a targetless - session, a non-matching target raises the banner, a matching one takes it down, clearing the - field drops targeting, traffic never stalls, STOP stays under 900 ms and leaks no thread. - Pinned as `test_gui_state.py::test_a_gui_session_keeps_the_target_banner_honest`, on a fake - table so it stays fast and deterministic. -- Two false alarms during that work, recorded so they are not re-chased: `ProcessTargeting` - defines `__len__`, so an object with an empty port set is FALSY - a diagnostic printing - `"y" if tg else "N"` reported a live target as missing. And `python` owns no sockets on this - machine, so a test using it as a "should match" target was wrong, not the code. Production uses - `is None` throughout, which is why neither reached the program. - -### Review pass over the whole targeting diff (four more findings) - -Read line by line before merge, on the principle that a green suite had already missed three -things in this branch. Each one below was verified by running it, not by reasoning about it. - -- **The watchdog's new port refresh could cancel the memory work.** `refresh_if_stale()` was put - FIRST inside the tick's existing `try`, so a socket-table failure aborted the block and - `_trim_conns()` plus `core.drain_retired()` never ran for that tick - the connection log would - grow unbounded because a NAME lookup failed. Now its own `try`: cosmetic work and memory safety - are different failure domains. Verified with a table that raises on every refresh: `_trim_conns` - still ran 6 times in 1.5 s and the row count stayed under the cap. -- **A failed resolve in `apply_targeting` left the engine and the core disagreeing.** The - synchronous announce-path refresh sat inside the `try` whose `except Exception` returns without - calling `set_target`, so `engine._targeting` held a new object the core had never been pointed - at. Moved out and wrapped in `crashlog.quiet`: a stale announcement is a far smaller problem - than two halves disagreeing about what is being impaired, and the resolver corrects it within a - tick. Verified with a table that always raises: engine, core and resolver all end up on the - same object. -- **`TargetResolver.stop()` signalled `_stopping` outside its lock**, leaving a window where a - concurrent `start()` could clear the flag, spawn a thread, and have the late `set()` kill it on - its first check. `BeanEngine` serialises start/stop under `_stop_lock` so it could not happen - today, but a threading primitive should not depend on its caller for safety. Verified with 200 - lifecycle cycles plus 300 start-immediately-after-stop pairs: no thread killed on arrival, no - leak, no dangling `on_miss`. -- **Dead knobs removed from `ProcessTargeting`.** `interval`, `miss_interval` and `_last` were - still written but no longer read by anything - pacing lives in `TargetResolver` now. Leaving - constructor parameters that control nothing invites somebody to tune them. Also fixed the fake - in `test_engine_records_a_broken_port_table_instead_of_going_quiet`, whose `process_for_port` - lacked the `allow_refresh` keyword: it was raising `TypeError` instead of the `RuntimeError` the - test meant to exercise, and passing for the wrong reason. - -### The connection log was a SECOND socket-table scan on the capture thread - -- **Moving targeting off the hot path did nothing for this one, and a green test suite said - otherwise.** `_log_conn` -> `_process_for` -> `PortTable.process_for_port` calls - `refresh_if_stale(miss=True)` whenever the port is unknown - four iphlpapi calls, sometimes a - psutil walk - **on the capture thread**, for the connection log's process column. Measured live - with a real port table and synthetic traffic: **47 rebuilds in 3 s from - `Thread-1 (_capture_loop)`**, alongside the resolver's own 48. -- **The end-to-end test missed it because it watched the wrong object.** It injected a counting - table into the `ProcessTargeting` but left the engine on `portmap.default_table()`, so it - asserted on a table the capture thread never used and passed vacuously. The test now sets - `engine._ports` to the same table and the fake grew the engine-side surface - (`process_for_port`, `pid_for`, `refresh_if_stale`). Found by instrumenting a live run, not by - the suite - which is the lesson worth keeping. -- Fix follows the pattern the project already uses for eviction and flow rotation: `_process_for` - reads with `allow_refresh=False` (a pure lookup), and the **watchdog** calls - `self._ports.refresh_if_stale()` on its 200 ms tick. Maintenance belongs on the maintenance - thread. Cost: a brand-new socket can read as `""` for up to one refresh interval, and - `_log_conn` already retries while packets keep coming, so the row fills itself in. -- Re-measured after the fix: socket-table refreshes come from `bean-target-resolver` (48) and - `MainThread` (2). **Zero from the capture thread.** - -### Targeting resolves off the capture thread (new `target_resolver.py`) - -- **`ProcessTargeting.__contains__` used to call `refresh()` inline** - i.e. from - `BeanCore.decide()`, on the CAPTURE THREAD, holding `core._lock`. One rebuild is four - `iphlpapi` calls, an O(n) dict copy, a `psutil.Process()` per distinct PID and, whenever a - protected PID refuses to open, a whole `psutil.process_iter()`. **And it was the normal case, - not an edge one:** targeting exists to narrow traffic to one application, so every packet from - every OTHER application is a miss, and a miss triggered the rebuild - a steady ~20 Hz of - syscalls in the packet path whenever a target was set. A stalled capture thread is precisely - what fail-open (convention 20), the watchdog, the eviction move and the table-sort move all - exist to prevent: WinDivert keeps diverting into a queue nobody drains, so the user loses - connectivity while the UI says "running". Targeting was the last place still doing it. -- **`__contains__` is now a frozenset lookup and nothing else.** A miss sets a plain bool - (atomic under the GIL, free) and, only on the FALSE -> TRUE transition, calls the resolver's - wake-up. That guard is the point: `Event.set()` takes a lock, so waking per packet would have - moved the problem rather than removed it. `refresh()` stays public and synchronous for - one-shot callers (`resolve_ports`, `make_targeting`) and tests. -- **New `beantester/target_resolver.py`.** Deliberately the same shape as `scenario_runner.py`: - a small class owning one thread, lifecycle driven explicitly by `BeanEngine`. Two differences - on purpose: `stop()` JOINS (it holds OS handles), and it waits on an `Event` rather than - sleeping, so a miss is picked up in milliseconds instead of at the next tick. **One resolver - per engine with a swappable target** - retargeting is a reference swap, not a thread restart, - because the GUI applies settings repeatedly and `test_concurrency_chaos` does it hundreds of - times. Wake ordering is clear-then-refresh-then-wait, so a miss arriving DURING a rebuild - re-arms instead of being swallowed by it. -- **`engine.set_target` is now the single place the resolver is pointed at a target.** - `self._targeting` was previously assigned only by `target_for`, so installing a live targeting - directly left the engine believing it had none while the core tested against it. `target_for` - keeps its memoisation (one live object per expression, so the port and process caches survive) - but no longer resolves; `start()` reconciles the two and does one synchronous pass so the first - packet meets a populated port set; `stop()` joins the thread. -- **The resolver's life matches a SESSION's**, not a target's: configuring a target without - starting must not leave something scanning the socket table in the background. -- **`apply_targeting` refreshes only when `announce=True`.** It has to, because the log line - reports what was matched and an unresolved target would always read as "matches nothing" - the - very message this project made loud on purpose. That is the explicit user-applied path; the - periodic path passes `announce=False` and never blocks. Strictly less work than before, where - `target_for` refreshed on every call including the GUI's 2 s loop. -- **Found while re-reading, fixed by removal: the GUI refresher thread leaked on fast restart.** - `_finish_start` spawned `_target_thread` unconditionally and nothing ever joined or signalled - it, while `_target_refresher` looped on `while self.running` with a 2 s sleep. STOP followed by - START inside that sleep left the OLD thread looping as well - one extra permanent scanner, each - doing a full OS scan every 2 s, per fast restart cycle. Not reproduced live (driving the async - start/stop on the fake-tk harness is awkward); `test_repeated_start_stop_cycles_do_not_stack_resolver_threads` - is the guard that would have caught it. -- **A FLOOR under miss-driven rebuilds, found by re-reading the design rather than by a test.** - Moving `miss_interval` out of `__contains__` removed the rate limit without putting it back - anywhere: targeting narrows traffic to one application, so every packet from every OTHER - application is a miss, misses arrive continuously, and the wake-up was re-armed as fast as it - was consumed. Measured with a 5 s routine tick: **63 rebuilds a second**, bounded only by the - GIL - with a real socket table that is a thread pegged at 100% scanning the OS. The resolver now - enforces `min_interval` (`portmap.MISS_REFRESH_S`, the same 0.05 s the old code used), in ONE - place instead of on the capture thread. Re-measured: 14 rebuilds/s with the floor, 33/s with it - disabled. The cost is the worst-case delay before a brand-new socket starts being impaired - - up to 50 ms, exactly the trade the old code made. -- **Dynamic process trees verified, not assumed.** A child spawned mid-session opens its own - socket; the first packet slips through (the documented, unclosable race) and the miss wakes the - resolver, which matches the child through its ancestor chain. Measured pick-up: ~3 ms without - the floor, bounded by `min_interval` with it. Grandchildren (two levels) work the same way, and - `myapp, !myapp-helper` keeps excluding a respawning helper despite its matching parent. -- **Caught in review, before merge: a target applied MID-SESSION got a frozen port set.** The - resolver was started only when a target already existed at `start()`. Press START, watch, then - type a process name - an ordinary workflow - and nobody was keeping the port set fresh: it - froze at whatever the first resolve produced and sockets opened afterwards were never picked - up. Precisely the failure live targeting exists to prevent, reintroduced by the fix for it. - The resolver's life is now the SESSION's, unconditionally; with nothing to resolve it blocks on - its event and costs nothing. Guarded by - `test_a_target_applied_mid_session_still_gets_a_live_port_set`. -- **The GUI does not resolve on the UI thread while a session runs.** `_refresh_target` resolves - inline only when the engine is STOPPED (no resolver to do it, and no session to stall); while - running it lets the banner wait for the next 700 ms tick. Four syscalls and a psutil walk on - the UI thread would be a frozen window, and a frozen window here is the user unable to press - STOP on their own broken network. -- `TargetResolver.stop()` detaches the old targeting's `on_miss`, so a late packet cannot poke - the event of a worker that is no longer listening. -- New `tests/test_target_resolver.py`: miss wakes the resolver and the new port is picked up - (long interval, so only the WAKE can explain it), `stop()` joins rather than signals, - retargeting does not churn threads, an orphaned targeting is detached, a failing table leaves - the resolver alive, **the capture thread never touches the socket table** (end to end over - synthetic traffic, asserting on the THREAD NAMES that made it look), no thread outlives a - session, and five start/stop cycles stack nothing. - `tests/test_release_fixes.py::test_an_unknown_port_forces_an_early_refresh` is rewritten as - `..._asks_for_a_rebuild_without_scanning_inline`: it now asserts the socket table is NOT - touched from the packet path and that 50 misses wake the resolver exactly once. - -### AsyncModel: a build returning None no longer wedges the worker for good - -- **`poll()` used `None` for two different things** - "no result arrived" and "the result". It - started with `rows = None` and returned early on `rows is None`, so a build that genuinely - produced `None` looked identical to an empty queue and **`_pending` was never cleared**. From - that moment `request()` coalesced into `_latest` for ever and nothing ran again: the table - stopped rebuilding for the rest of the session, and `busy()` stayed True, which leaves - `conns._poll_soon()` rescheduling its 40 ms catch-up timer indefinitely on the UI thread. -- Fixed with a module-level `_NOTHING` sentinel. The caller's contract is unchanged (`poll()` - still returns `None` for "nothing new to show"); what changed is that a result for the request - in flight now clears `_pending` whatever its value. -- Latent, not live: `conns._build_model` always returns a dict. But convention 29 makes - `AsyncModel` the mechanism every future heavy table is meant to use, so the contract had to - hold before something is built on it. -- **Deliberately NOT fixed in the same pass:** the exception path in `_run` clears `_pending` but - drops a request that queued into `_latest` while the build was failing. It self-heals - the page - calls `request()` on every tick, so a newer payload starts within about a second - and - re-submitting would mean calling `request()` (documented UI-thread only) from the worker thread, - outside the lock to avoid deadlocking on it. Threading complexity for a case that already - recovers is the wrong trade in a tool whose STOP button has to keep working. -- New test: `tests/test_model_worker.py::test_a_build_returning_none_does_not_wedge_the_worker`. - Verified non-vacuous by restoring the old collision (`_NOTHING = None`) and confirming the - worker wedges. - -### portmap/engine/processes: port-resolution failures stop being invisible - -- **`_Native.port_pid_map` accepted a PARTIAL socket table as the truth.** `ok |= self._table(...)` - over the four (proto, family) combinations left `ok` True when three of four answered, and - `refresh()` cached the result as authoritative. A missing table means sockets the tool cannot - see, and an unseen socket is traffic the user asked to impair sailing through untouched - - which on screen looks exactly like "the application coped". The failures are now counted and - named: all four failing still returns `None` (psutil fallback, unchanged), a partial result is - still returned but goes through `crashlog.once("portmap.native.")`, with the failing - tables in the key so a different failure is recorded too. -- **The stricter option (any failure -> psutil) was rejected on purpose.** Measured on the dev - machine, all four tables answer `rc=0` (tcp/v4 103 rows, tcp/v6 10, udp/v4 90, udp/v6 23), so - the failure mode is NOT reproducible here. Trading a possible gap for a certain order-of- - magnitude slowdown, on a path that cannot be tested, is the wrong bet; when a real machine - reports it, `crashes/` will hold the evidence and the decision can be made on data. -- **`_Native._table` no longer pretends to reuse its buffer.** The comment claimed "grow and KEEP - one buffer per table", but a fresh `create_string_buffer` was allocated on every call and the - stored buffer was never read back - the cache only pinned memory. `self._buffers` becomes - `self._sizes` (the size hint is the part that was doing work). Real reuse was considered and - rejected: four allocations a few times a second against aliasing between calls in ctypes code. -- **`engine._process_for` / `_pid_for` now use `crashlog.once("engine.ports*")`.** They swallowed - silently while the same file, 200 lines up, already used `crashlog.once("engine.packet")` for - the same class of event on the same thread. `once()` and not `note()` because this is the - capture path: a port table that starts failing turns every row's process into "?", which is - worth one traceback, not one per packet. -- **`processes.port_process_map` uses `crashlog.quiet("processes.port_map")`.** Best-effort for - the caller (an empty map still just means "?"), recorded for us. -- New tests in `tests/test_processes.py`: `test_port_process_map_records_a_failure_instead_of_swallowing_it`, - `test_a_partial_socket_table_is_reported_not_silently_trusted`, - `test_every_socket_table_failing_falls_back_to_psutil`, - `test_engine_records_a_broken_port_table_instead_of_going_quiet`. They spy on `crashlog.record` - (and reset `_once_seen`) instead of reading the crash directory, so they touch no disk. - -### Changelog structure: `### BREAKING` first, now guarded - -- Convention 39 requires `### BREAKING` to be the FIRST section of a version in both changelogs. - The `--doctor` entry was added ABOVE it in both files, pushing it to second place - the exact - drift the convention exists to prevent, committed two chunks after writing the convention down. - Nothing caught it: `test_no_em_or_en_dashes` reads changelog TEXT, never its structure. -- Fixed in both files, and `tests/test_version_and_release.py::test_breaking_sections_come_first` - now enforces it: in every version block of either changelog, if a `### BREAKING` heading exists - it must be the first `###` under its `##`. - -### Hygiene guard: measured, then deliberately NOT tightened - -- The audit proposed extending `test_code_hygiene` to catch `except ...: return `, not - only `except ...: pass`. A prototype was run across the package first. Result: **66 silent - handlers, of which 26 catch a NARROW type** (`OSError`, `(TypeError, ValueError)`) and are - idiomatic, and 40 are broad. Of the 40: 7 are `crashlog.py` (already exempt), 12 sit in modules - whose docstring states a "never raises" contract (`portmap` 6, `winenv` 4, `matchers.matches()`, - `utils.is_local_ip`), 2 in `legal.py` already carry `# noqa: BLE001` with a reason, and 14 are - in `gui/` - against roughly 100 correct `crashlog.*` uses in the same directory. -- **Conclusion: the codebase is disciplined and the guard would mostly encode the status quo**, - at the cost of a wide diff and future false positives. Tightening was dropped; the three - handlers that were genuinely inconsistent with their own neighbours were fixed above instead. - If it is ever revisited, the mechanism to use is the one `legal.py` already established - - `# noqa: BLE001 - ` at the handler - not a central allowlist. - -### Deferred: PID reuse in the portmap info cache (audit item P2) - -- `PortTable._expire_info` returns early below 512 entries, so on a normal machine (50-250 - socket-owning PIDs) the `pid -> (name, ppid)` cache never expires and a recycled PID keeps the - dead process's name. That matters beyond a wrong column: `ProcessTargeting.refresh()` matches on - `name_of(pid)`, so the tool can impair a process the user did not target. -- **The obvious fix does not work.** `info()` refreshes `last_seen` on every cache HIT, so the - dangerous case - a recycled PID that is being actively looked up - never expires no matter what - the TTL is. Real fixes (TTL from INSERTION, `create_time()` validation, or evicting PIDs that - vanish from the socket table) all add work to `PortTable.refresh()`, which today runs **on the - capture thread** via `ProcessTargeting.__contains__`. TTL-from-insertion additionally gives a - thundering herd: entries created together expire together, so one refresh re-resolves dozens of - PIDs at once, in the packet path. -- Therefore P2 is scheduled straight after the targeting rewrite, when the cost no longer sits on - the capture thread. Designing around a constraint that is about to be removed would be wasted work. - -### driver.py: read a service with read rights, not ALL_ACCESS - -- **`service_state` opened services with `SERVICE_ALL_ACCESS` (0xF01FF) just to read their - state, and mapped the resulting failure to `None` = "not installed".** Measured on Windows 11 - from an ELEVATED shell, so this was never a "needs admin" problem: - - OpenServiceW(Schedule, SERVICE_ALL_ACCESS) -> NULL, error 5 (ACCESS_DENIED) - OpenServiceW(Schedule, SERVICE_QUERY_STATUS) -> handle, QueryServiceStatus = running - - Same for `Dnscache`; `EventLog` grants both, which is why the path looked fine. Any service - whose security descriptor withholds full control read back as absent. Now - `SC_MANAGER_CONNECT` + `SERVICE_QUERY_STATUS`, which also makes the read work unelevated. -- **Third return value `NO_ACCESS`**, distinct from a state label and from `None`. "I cannot - tell" and "it is not there" lead to opposite conclusions, so they no longer share a value. - `installed_drivers()` keeps such a service in the dict (absence from that dict has to keep - meaning "not installed"); `doctor()` renders it `warn` with a "re-run as Administrator" hint - instead of `ok / not loaded`. Exit codes are untouched: `warn` is not `fail`, and - `ok = all(state != "fail")` is unchanged. -- **`_advapi()` now loads advapi32 with `use_last_error=True`.** `ctypes.get_last_error()` in - `stop_and_remove` read a thread-local ctypes never populated, so it was always 0 and both - branches of the `if` returned the same string - dead code pretending to discriminate. With the - flag it works, so a refusal is reported as `access denied` rather than `not installed`. -- **`stop_and_remove` deliberately keeps `SERVICE_ALL_ACCESS`.** Narrowing it to - `SERVICE_STOP|DELETE|SERVICE_QUERY_STATUS` (0x10024) was measured and does NOT help: a - hardened service denies `DELETE` itself. The only honest improvement there is the message. -- **`_advapi()` and the `SERVICE_STATUS` structure are cached** in module-level slots. - `installed_drivers()` asks about three service names, and each call used to rebuild the - binding, re-assign six sets of prototypes and define a fresh `ctypes.Structure` subclass. - `ctypes.WinDLL(...)` (unlike `ctypes.windll.advapi32`) returns a NEW object per call, so - without the cache the `use_last_error` change would have been a small regression. Both stay - lazy: `ctypes.wintypes` does not import on Linux and CI runs on ubuntu too. -- New tests in `tests/test_driver_windows.py`: - `test_reading_a_service_state_asks_only_for_the_right_to_read` (the regression guard - probes - `Schedule`/`Dnscache`/`EventLog` on Windows and requires a real state back, plus a genuinely - absent service still returning `None`), `test_advapi_and_status_type_are_built_once`, - `test_doctor_says_it_could_not_look_rather_than_not_loaded` and - `test_doctor_still_calls_a_clean_machine_not_loaded` (both directions of the doctor row). -- Not proven, stated plainly: no WinDivert driver was loaded on the test machine, so this is a - correctness and robustness fix rather than a reproduced WinDivert failure. WinDivert's own - service descriptor is probably permissive today; the point is that `--doctor` no longer - depends on it staying that way. - -### CI: one run of the test suite, under coverage - -- **`.github/workflows/ci.yml`: the `tests` job ran the whole suite twice over, plus two - overlapping subsets.** Four steps executed: `pytest tests`, then the - `test_matchers_properties.py` + `test_cli_fuzz.py` subset, then `test_concurrency_chaos.py`, - then `pytest tests --cov` over everything again. `testpaths = ["tests"]` (pyproject) already - pulls both subsets into every full run, so the middle steps re-executed tests that had just - passed - on ubuntu and windows, on 3.10 and 3.13, four cells deep. -- **Now a single step:** `pytest tests --cov=beantester` with `COVERAGE_PROCESS_START`, keeping - the `fail_under = 77` gate and the `coverage.xml` artifact. Nothing changed about WHICH tests - run. The rationale each deleted step carried (why the property/fuzz suites and the chaos suite - earn their keep) moved into a comment on the surviving step, so the reasoning outlived the - checkmarks it was attached to. -- **Accepted trade:** a failure now surfaces as one red step instead of a named one (pytest - still names the file and test, so diagnosis is unaffected), and the wall-clock assertions - (`test_failsafe.py` start/stop under 0.2 s, `test_model_worker.py`, `test_audit_fixes.py`) lose - their uninstrumented reference run. They already ran under coverage in the old gate step and - passed; if one starts flaking, split the clean run back out. - -## [0.3.0] - 2026-07-20 - -### GUI fix: numeric preferences went red without a reason - -- **`gui/panels/settings.py`: the `Pref` NUMBER rows grew the error line the registry fields - already had.** `_on_pref_number` caught the `ValueError` from `parse_number` and dropped it, - keeping only `style="Bad.TEntry"` - yet that exception already carries the translated - `errors.field_range` / `errors.field_number` text, min and max included. The same window - rendered the row limit through `ControlForm`, which does show it (`form.py::validate_section`), - so one dialog answered the user's "what is allowed here?" for one field and stonewalled for the - other two. -- **Shape copied from `ControlForm`, not invented:** one `Bad.TLabel` per `PREF_GROUPS` group - (`wrapping_label`, packed only while non-empty so the card keeps its height), reasons joined - with the same `" • "` separator, live messages kept per pref key in `_pref_messages` so - fixing one field clears only its own reason. `_pref_errors[group] = (label, number_keys)`. -- No new i18n keys and no registry change: `prefs.py` is untouched, the text comes from the - `errors.*` keys that already exist in both languages (convention 9 needs nothing here). - Persisting is unchanged - an invalid value still never reaches `App.set_pref`. -- Tests: `test_prefs.py::test_settings_window_number_field_says_why_it_is_red` asserts the reason - appears with its bounds, that a second bad field in the group ADDS a reason instead of replacing - it, that fixing one field clears only its own, and that the last fix unpacks the line again. - -### GUI fix: the running-state icon never reached the main window (Tk `-default` trap) - -- **`gui/icon.py`: new `show_running_icon` / `show_idle_icon`** (over `_set_icon`), called from - `App._sync_running_ui` in place of the bare `root.iconphoto(True, icon)`. -- **Root cause:** `iconphoto(True, img)` is Tk's `-default` - the icon for toplevels created - from then on. On Windows it lands on the window CLASS, and a window owning an icon of its own - keeps that one; the main window owns `bean.ico` from `apply_window_icon`'s `iconbitmap`. So - the swap was a no-op where it mattered and DID paint the dot on the next Toplevel opened - (the close-confirmation dialog), which is how the owner spotted it. Measured, not guessed: - `WM_GETICON` on the toplevel returned the same `HICON` before and after `iconphoto(True, ...)` - and a different one after `iconphoto(False, ...)`. Both calls are kept - `False` for this - window, `True` so panels opened later carry the state too. -- **Idle restores through `iconbitmap(bean.ico)`, not the photo.** `bean.ico` ships 16/24/32/48/ - 64/128/256 px frames; `bean.png` is 256 px only, so restoring through the photo would leave - the taskbar on a downscale of it permanently after the first capture. Windows-only, guarded, - falls back to the photo. -- The 0.2.0 entry below ("swaps `root.iconphoto` between an idle and a running icon") described - a feature that only half-worked on the one platform this tool targets. -- Tests: `test_gui_state.py::test_the_running_icon_lands_on_the_window_not_just_the_default` - asserts the swap hits BOTH the window and the default. Needed a fake that can see it - - `fake_tk.Root` now records `iconphoto`/`iconbitmap` into `kw["icons"]` instead of swallowing - them in `W.__getattr__`. Verified to fail pre-fix with `[('default', ...)]` alone. Limits: - the fake can only prove which call we make - that Windows repaints the taskbar is not - testable here (convention 41: confirmed by render). - -### GUI fix: a widened throughput chart crept into its new window instead of filling it - -- **`App._reconcile_chart_len` now zero-pads when GROWING** (new `App._resized_hist(hist, n)`). - Two paths build the history and only one padded: `__init__` creates - `deque([0] * n, maxlen=n)`, while the reconcile did `deque(hist, maxlen=n)` - correct when - shrinking (the deque drops the oldest itself), but on a grow it left `len` at the OLD value - and only raised `maxlen`. -- **Why it was visible:** `chart.draw_throughput_chart` labels the X axis from - `len(down_hist) * sample_interval_s`, so raising `chart_seconds` from ~20 s to 250 s left the - axis reading "-28 s" and counting up one sample per `TICK_MS`, ~4 minutes to fill, while - `stats._throughput_title` reads the preference directly and said 250 immediately - breaking - the invariant its own docstring promises ("never drifts from the live X-axis label"). The - series is also drawn across the full plot width (`x = i / (len - 1)`), so the horizontal - scale crept with every tick. `chart.py` and `stats.py` are unchanged: with `len == maxlen` - restored as an invariant, both are already right. -- Tests: `test_prefs.py::test_a_resized_chart_spans_its_whole_window_at_once` (len matches the - window after growing AND shrinking, newest sample stays newest, padding lands on the left). - Verified to fail on the pre-fix code with `(171, 357)`. The existing - `test_chart_history_length_follows_the_preference` asserted `maxlen` only, which was correct - throughout - the bug lived in `len`, which is what the axis is computed from. - -### GUI: dark mode for the parts Windows draws itself (system menu, menu frames) - -- **New `theme.apply_dark_app_mode()`**, called once from the top of - `theme.apply_dark_titlebar` (module flag `_app_mode_applied`, one attempt per process). - Piggy-backed there deliberately: `App`, every `PanelWindow` in the registry and `dialogs.py` - already call `apply_dark_titlebar`, so no window can ask for a dark frame and still get a - white system menu - and no caller needed changing. -- **Why a second mechanism at all:** `DWMWA_USE_IMMERSIVE_DARK_MODE` is a PER-WINDOW attribute - and only covers the DWM-drawn frame. The system menu (title-bar icon / Alt+Space) and the - frame user32 puts around a classic `tk.Menu` popup follow a PROCESS-WIDE flag in undocumented - `uxtheme` exports instead, which nothing in the package was setting - so every window had a - white system menu, and the Connections context menu kept the light rim noted in convention 41 - ("Tk reaches the entries but not the system-drawn frame"). -- **Implementation:** `uxtheme` ordinals 135 (`SetPreferredAppMode`, `AllowDarkModeForApp` on - 1809) and 136 (`FlushMenuThemes`). `ForceDark` (2), not `AllowDark` (1): the UI is dark - unconditionally, so following the system theme would leave a light menu for a user running - Windows in light mode. The flush is required - the menu theme is cached per process and is - already light by the time we get here. Gated on `sys.getwindowsversion().build >= 17763` - (first build with these exports), wrapped in `crashlog.note` (convention 30): the exports are - undocumented, and the worst case on failure is the light menu we had before. -- Side effect, accepted by the owner: the native `filedialog` pickers render dark now. They stay - native on purpose (see the `dialogs.py` docstring) and dark is the consistent look. -- **No test guard.** This is pixels painted by the OS outside the widget tree - the tkinter - fake cannot observe it and `tools/ci_gui_render.py` only sees the client area. Verified by - render on Windows 11 build 26200 (convention 41: check live, not from the code). - -### GUI fixes: truncated About text, a button left highlighted, and a render check that lied - -- **`panels/about.py` uses `labels.wrapping_label` for every prose line** (author, copyright, - licence, licence terms, the no-telemetry line, the third-party heading). A plain `ttk.Label` - never wraps - it is CUT at the frame edge - and the helper written for exactly this was not - being used here. `pad` is `2 * 12 + 16`: the `padx` on both sides plus the few pixels a wrapped - `ttk.Label` requests on top of its `wraplength` (measured against the render check, not - guessed - at `pad=30` the widest wrapped line still overhung by 12 px). -- **`App._release_focus()`, called from `App.open_window`.** ttk gives a button keyboard focus - when it is clicked and `theme.py` paints `focus` exactly like `active` (both -> `BTN_HOVER` + - `ACC` border), so closing a window handed focus back to the button that opened it and it kept - looking hovered. Focus goes to the toplevel instead, and the invoking widget's `active`/`focus` - flags are cleared - the same remedy `theme.unhighlight_combobox` applies to a readonly combobox. -- **`tools/ci_gui_render.py` now FAILS on a truncated label** instead of filing every clipped - label under "it probably wraps". The split is `wraplength > 0` -> note (it re-wraps), no - `wraplength` -> `TRUNCATED LABEL`, which is a real defect. This check had been printing the two - About lines as harmless notes for as long as they had been broken. -- **The render check also opens EVERY window in the `WINDOWS` registry** (it only ever opened - About), and runs against an **empty user state** - `UiStateStore`/`ProfileStore` are pointed at - a temp dir like `tests/gui_harness.py` does. It was reading the developer's own - `bean_network_tester_ui.json`, so the `--lang en` pass rendered whatever language that file - remembered (the "en" run was reporting Polish strings), and a saved geometry could have hidden - the very clipping the check exists to find. -- **`tests/fake_tk.py`** models keyboard focus (`FOCUS`, `focus_set`/`focus_get`) and ttk state - flags (`W.states`, `state(["!active"])` sets/clears; `Root.state()` still answers `"normal"`, - since a toplevel's `state()` is the window state, not ttk flags). -- **Tests:** `tests/test_windows.py::test_opening_a_window_takes_the_highlight_off_the_button_that_opened_it` - and `::test_every_prose_label_in_the_about_window_can_wrap`. Both were confirmed to fail against - the pre-fix code, and the render check was confirmed to report `2 truncated label(s)` on it. - -### GUI: focus is a ring, hover is a fill (they used to be the same picture) - -- **`gui/theme.py`: every `("focus", )` entry that duplicated the style's `("active", - )` is gone** (`TButton`, `Accent`, `Stop`, `Dirty`, `Help`, `Donate`, `Section`, - `Gear`). Hover keeps the fill; focus is drawn by clam's **`Button.focus` element** through - `focuscolor` (`focusthickness=1`, `focussolid=True` on `TButton`, inherited by the derived - styles; the coloured buttons keep their own ink colour, because an accent ring on an - accent-blue button is invisible). Measured, not assumed: at thickness 1 the ring costs no - space (a button is 82x31 either way) and at 3 it grows to 86x35 - which is why this one - number is not `scaled()`. -- **`tools/ci_gui_render.py` fails when a style paints `focus` and `active` the same.** The - styles it checks come from two places, neither hand-kept: the widgets actually on screen, plus - every name `theme.py` configures or maps (regex over the module source) - `Stop.TButton` only - exists while a capture runs and `Dirty.TButton` only while the form is dirty, so a screen walk - alone missed exactly the styles nobody looks at. Against the pre-fix theme it reports all 8 - offending styles; after the fix, none. -- Theme module docstring gained the rule as a third invariant, next to "no hard pixels" and - "a disabled widget must look disabled". This is the other half of the "button left - highlighted" fix above: that one stops focus LANDING on the button, this one stops focus from - being painted as hover in the first place. - -### GUI fix: a tooltip covered the whole row, not the text - -- **A tooltip belongs to a WIDGET, so a label packed `fill`/`expand` shows its bubble over the - blank space next to the sentence.** Measured on real Tk at 1366x768: `App.summary` was **508 px - wider and 17 px taller than its own text** (it filled the fixed-height summary strip), so the - bubble fired over empty header background nowhere near the line it explains. Same shape, smaller - numbers, on the two `wrapping_label` scope notes (`pages/stats.py` 118 px, `pages/conns.py`). -- **Fix:** pack them to their content - `App.summary` -> `side="left", anchor="nw"`, both scope - notes -> `anchor="w"` instead of `fill="x"`. A `wrapping_label` does NOT need `fill` to wrap: - `labels.bind_wraplength` follows the PARENT's ``, so the wrap width is unchanged. -- **Test:** `tests/test_gui_layout.py::test_a_tooltip_never_covers_empty_space` - walks all three - pages and fails on any LEAF widget that has a tooltip, carries `text`, no `command`, and is - packed with `fill`/`expand`. Containers are exempt on purpose (a stat tile or a `LabelFrame` - with a tooltip does answer for everything inside it), and so are entries/comboboxes/buttons, - where the whole box is the control. Confirmed to report all three offenders before the fix. - -### GUI: the profile picker is a ttk.Combobox again (convention 41) - -- **`gui/pages/control.py::_build_profiles`: `ttk.Menubutton` + `tk.Menu` -> `ttk.Combobox`** - (readonly, no named style - the shared `TCombobox` look), bound to - `App.on_profile_selected` (`unhighlight_combobox` + `load_selected_profile`), i.e. exactly - how the traffic filter is built in `form.py`. The menu was introduced so group headings - could be rendered non-pickable, but the headings had already been dropped from the menu, so - all it still bought was a dropdown that could not be made to match: **on Windows a `tk.Menu` - is a native Win32 popup**, so its frame (a light system border), its width (no `-width` - option, so it is sized to the longest label instead of to the button) and the highlight on - the current entry are outside Tk's reach - no amount of styling closes that gap. -- **Group headings dropped entirely** (owner's decision, convention 41: every row in a list - must DO something). `App.profile_names()` is now `presets + own profiles`, full stop; - `App._profile_separators` and the snap-back branch in `App.load_selected_profile` are gone, - `_is_reserved_profile_name` is down to the preset check (a user profile may now be called - "Presets"), and the `profiles.presets_separator` / `profiles.mine_separator` keys are - deleted from `lang/en.json` + `lang/pl.json`. `smoke_gui.py`'s separator check is replaced - by one asserting the picker offers presets then own profiles and nothing else. -- **Removed:** `App._rebuild_profile_menu`, `App._post_profile_menu` (a workaround for the - Menubutton's post-on-mouse-down toggle - a combobox has no such problem), the - `App.profile_menu` attribute, the `Profile.TMenubutton` layout/configure/map and the bare - `TMenubutton` styles in `gui/theme.py`, and the `like_combobox` parameter of - `theme.style_menu` (context menus were its only other caller). `App.profile_mb` -> - `App.profile_cb`. -- **`theme.popdown_height(values)`** (+ `POPDOWN_ROWS = 20`) is now the single source for the - "a list that fits must not spawn the popdown scrollbar" rule, used by the profile picker, - the traffic filter (`form.py`) and the language box (`panels/settings.py`), which each had - their own `height=len(...)`. The profile list is the only one the user can grow without - limit, hence the cap at ttk's own default rather than a dropdown taller than the screen. - `App._sync_profile_widgets` now refills `values=`/`height=` instead of rebuilding a menu. -- **Tests:** `test_gui_release_fixes.py::test_profile_menu_has_no_indicator_gutter` and - `::test_profile_picker_uses_the_combobox_field_style` (both about the retired Menubutton) - replaced by `::test_profile_picker_is_the_same_widget_as_the_traffic_filter`, which checks - the built widget (readonly, no style override, `values` == `profile_names()`, `height` == - item count) AND greps `gui/pages/control.py` for `Menubutton`/`tk.Menu(`, so the imitation - cannot come back. -- **PROJECT_NOTES convention 41** rewritten with two lessons: same role -> same widget (do not - imitate a sibling widget with styles - the imitation has a ceiling that is invisible in the - code), and every row in a list must do something. The stale "405 keys per lang file" figure - in the repo-structure section (really 465) was replaced by the command that counts them - - a number copied out of its source file drifts, which is exactly what convention "one fact, - one source" is about. - -### GUI fix: "restore the last profile" ignored the user's own profiles - -- **`App._set_profile_key(key)` is now the single writer of `_profile_key`** (`gui/app.py`). - Three paths change the current profile - `select_profile`, `save_profile`, `delete_profile` - - but only the first also wrote `ui["profile"]`, the key the `restore_profile` preference reads - on startup. Since **saving** is how a user ends up on their own profile, the preference - restored the preset picked before the save; picking an own profile from the list already - worked, which is why this looked like "it does not work for custom profiles". `delete_profile` - now remembers the fallback (`DEFAULT_PROFILE`) instead of the deleted name. -- **The key is persisted on the spot** (`ui.persist()` inside `_set_profile_key`), the rule - `set_pref` already follows: a deliberate user choice must survive an unclean exit, unlike - session state written in `on_close`. One small atomic write per profile change. -- **`App.__init__` keeps a plain `self._profile_key = DEFAULT_PROFILE`** (commented): routing it - through `_set_profile_key` would write the default into `ui.json` before - `_restore_last_profile` reads the remembered one. -- **`_restore_last_profile` clears a dead pointer**: a name that resolves to neither a preset nor - a stored profile (deleted by hand, `profiles.json` quarantined as corrupt, removed by another - instance) is still ignored without an error, but `ui["profile"]` is reset so the file stops - carrying a ghost. -- **Test:** `tests/test_prefs.py::test_restore_last_profile_covers_the_users_own_profiles` - - save remembers, delete falls back, a vanished profile is ignored AND forgotten. - -### Docs: intro wording and third-party links - -- **README intro (EN + PL)** reworded: leads with the product name (branding + the auto-snippet), - compares to Clumsy/NetLimiter by what the tool *does* rather than by the driver - NetLimiter does - not use WinDivert, and the old phrasing implied it did - and names WinDivert **via PyDivert**, both - linked. -- **Third-party section (EN + PL):** each named component now links to its homepage/source - (WinDivert, PyDivert, psutil, CPython, Tcl/Tk, PyInstaller). - -### CI: one run per commit - -- **`ci.yml` `push` trigger scoped to `master`.** With `on: [push, pull_request]` a branch that - had an open PR ran the whole matrix twice (a `push` event and a `pull_request` event; the - concurrency group only dedupes within one event). Now feature branches run once via their PR, - and `master` runs on push (after a merge). Halves the Actions runs on PR branches. - -### Relicense: GPLv3 - -- **Relicensed from the proprietary Bean Network Tester License to the GNU GPL v3.** `LICENSE` is - now the verbatim GPLv3 text (byte-identical to `licenses/GPL-3.0.txt`, so GitHub detects it and - the copyleft terms actually apply). Touchpoints updated in one pass: `appinfo.LICENSE_NAME` and - `COPYRIGHT` (dropped "All rights reserved"), the `pyproject` classifier - (`Other/Proprietary` -> `OSI Approved :: GNU General Public License v3 (GPLv3)`), - `about.license_terms` in `lang/en.json` + `lang/pl.json`, the License and third-party sections of - both READMEs (removed the "closed source" line), and the `THIRD-PARTY-NOTICES.md` header. Exe - metadata (`.spec` LegalCopyright) and the About window follow automatically via the `appinfo` - constants. No version bump (owner closes the version). -- **Test:** `tests/test_version_and_release.py` - `test_license_is_not_mit_anymore` rewritten as - `test_license_is_gplv3` (asserts the verbatim GPLv3 text is present and the old "may not be sold" - wording is gone); `test_no_mit_references_left_in_metadata` renamed to - `test_no_stale_license_references_in_metadata` (also asserts the Proprietary classifier is gone - and the GPLv3 classifier is present). - -### Docs: English README is now the default - -- **Swapped the README language default.** `README.en.md` (English) is now `README.md` - the file - GitHub renders on the project page - and the Polish text moved to `README.pl.md`. Cross-links at - the top/bottom of each file updated to point at the new names; `pyproject.readme` now points at - `README.md`. -- **Tests:** `tests/test_readme_guards.py` and `tests/test_cli_docs.py` - `READMES` tuple and the - per-language pipeline guards retargeted (`test_english_readme_pipeline_matches_core_decide` reads - `README.md`; `test_polish_readme_pipeline_keeps_lan_and_blocking` reads `README.pl.md`). No new - tests, same guarantees against the new filenames. - -### CI: release workflow and GitHub repo furniture - -- **New `.github/workflows/release.yml`.** Tag push `v*` -> assert the tag matches - `VERSION.txt`, build the onedir exe from `BeanNetworkTester.spec`, - smoke it, zip it as `BeanNetworkTester--windows-x64.zip`, write `SHA256SUMS.txt` (the - checksum the README tells users to verify), and publish a GitHub Release via the preinstalled - `gh` (job token, `contents: write` scoped to the workflow; no third-party action). A - `v-rc.N` (or `-beta.N` / `-alpha.N`) tag publishes as a GitHub Pre-release; a plain - `v` tag as Latest. The tag's base version must equal `VERSION.txt`. `ci.yml` is - unchanged. -- **Repo furniture:** `SECURITY.md` (private vulnerability reporting, tool-specific scope), - `.github/FUNDING.yml` (Sponsor button -> the project support page), - `.github/ISSUE_TEMPLATE/` (bug-report + feature-request forms tailored to the tool: version, - Windows, GUI/CLI/simulate, elevation, `--doctor`; plus `config.yml` disabling blank issues and - linking support + security advisories), and `PULL_REQUEST_TEMPLATE.md` (checklist keyed to the - project conventions: tests, both lang files, both changelogs, Conventional Commits, no bump). -- `dependabot.yml` already covered `pip` + `github-actions` - left as is. - -### Repo: line endings, code of conduct, README badges - -- **`.gitattributes`** pins text files to LF (`* text=auto eol=lf`; `*.png` / `*.ico` binary), - ending the "LF -> CRLF" checkout churn on Windows and giving the Linux/Windows CI runners - identical bytes. `git add --renormalize` was a no-op (the repo already stored LF), so no - content changed. -- **`CODE_OF_CONDUCT.md`** (Contributor Covenant 2.1) completes the GitHub community profile; - README, LICENSE, CONTRIBUTING, SECURITY and the issue/PR templates were already present. -- **README badges** in both languages (CI status, latest release, downloads, GPLv3, Windows). - -### Repo: release-note grouping, downloads script, WinDivert link - -- **`.github/release.yml`** groups the auto-generated release notes by PR label - (New features / Bug fixes / Performance / Documentation / CI / Other) instead of one - flat list. -- **`tools/downloads.py`** prints per-release, per-asset GitHub download counts via the - public API (stdlib only, no token). The README downloads badge shows the same total live. -- **README (EN + PL):** the first WinDivert mention now links to its homepage - (`reqrypt.org/windivert.html`). -- CodeQL and `dependency-review-action` are deferred to just after the repo goes public - (both need a public repo or GitHub Advanced Security); steps are in the Doc repo runbook. - -### Docs: README polish for the public repo - -- Table of contents is now **expanded by default** (removed the `
` fold) in both READMEs. -- Added a **Contributing** section linking `CONTRIBUTING.md`, `CODE_OF_CONDUCT.md` and `SECURITY.md`. -- Added a short **AI-assisted-workflow disclosure** to the Author section. -- Section order left intact - a full read confirmed it was already logical and public-ready, so no - reshuffle (would have been churn against the guards for no reader benefit). - -### GUI: Settings window - -- **New "surface" split of the field registry.** `fields.Section` gains `surface` - (default `"control"`); the `tables` section is marked `surface="settings"`. Added views - `CONTROL_SECTIONS` / `SETTINGS_SECTIONS`. The Control page and the Settings window are now - both renderers of one registry: a new preference is one entry with `surface="settings"` and - it renders itself (widget, label, unit, live validation) - no second code path. `row_limit` - (still `ui_only`, convention 37) is the first field to move. -- **`gui/form.py::ControlForm` takes `sections=`** (defaults to `CONTROL_SECTIONS`); all its - `SECTIONS` loops now read `self._sections`. `SECTION_BY_ID` stays a full lookup. The Control - page passes the default; the Settings window passes `SETTINGS_SECTIONS`. Shared `app.vars` - keep both forms in sync (a config-file load updates the Settings entry live). -- **New window `gui/panels/settings.py` (`SettingsWindow`, ID `settings`)** via the window - registry (convention 25): language combobox (bound to the App's `lang_var`, locked mid-session - like before) + `ControlForm(sections=SETTINGS_SECTIONS)`. Registered in `panels/__init__.py`. -- **`gui/app.py`**: header language combobox+label replaced by a gear button opening the - Settings window (`Gear.TButton` in `theme.py`, icon from new `icon.make_gear_icon`, pure - `PhotoImage.put`, DPI-scaled). `lang_var` / `_lang_name2code` still built in `_build_ui` - (smoke and `_switch_language` depend on them); `lang_cb` is now owned by the Settings window - (set on build, `None` on close). `_sync_running_ui` skips `lang_cb` when `None`. -- New i18n keys (both files): `windows.settings`, `buttons.settings`, `tips.settings`. -- `theme.apply_dark_titlebar` now forces a non-client repaint (`SetWindowPos` SWP_FRAMECHANGED) - after setting the DWM attribute, and `windows.PanelWindow.open` re-asserts it once the window - is mapped: a Toplevel shown without being activated (opened while the main window keeps focus) - used to keep a white title bar until first click. Fixes it for every registered window. -- Dead-entry cleanup: `gui/app.py::FIRST_RUN_COLLAPSED` no longer lists `"tables"` (that section - is settings-surface, rendered `collapsible=False`, so the collapse hint did nothing). -- Tests: `test_windows.py::test_settings_window_holds_the_language_box_and_the_view_fields` - (row_limit renders in Settings, not on the Control page; no Control field leaks in; language - box owned + rebound across a language switch + dropped on close). Updated - `test_gui_layout.py::test_stop_button_and_language_picker_reflect_the_session` and - `test_gui_release_fixes.py::test_short_dropdowns_do_not_spawn_a_popdown_scrollbar` to open the - Settings window before asserting on `lang_cb` (it left the header). Added - `test_windows.py::test_settings_sections_render_open_and_do_not_touch_collapse_state` (the - clobber guard: two ControlForms must not fight over `app.collapsed_sections`) and, in - `test_field_registry.py`, `test_sections_split_cleanly_by_surface` + - `test_ui_only_fields_live_on_the_settings_surface` (convention 42 / 37 invariants). - -### GUI preferences (ui.json-backed) - -- **New `gui/prefs.py`** - a small preference registry *separate* from `fields.FIELD_DEFS`. - `Pref(kind=NUMBER|BOOL|ACTION)`; `PREFS` + `PREF_GROUPS`. These persist in `*_ui.json` under - `pref.` (never in a traffic config file, never a CLI flag). `App.pref(key)` reads+validates, - `App.set_pref(key, v)` writes through and persists immediately (a preference must survive an - unclean exit). `SettingsWindow` renders the groups (numbers with live `parse_number` validation, - checkbuttons, and an action button). See convention 42 for the two-kind model (registry - `surface="settings"` field vs a `Pref`). -- Wired behaviours in `gui/app.py`: - - `chart_seconds` -> `App.chart_samples()` (seconds / tick period) sizes the throughput deques; - `_reconcile_chart_len()` resizes them live in `_sample()` (keeps the most recent samples). - - `log_lines` -> `_append_log_line` trims `_log_lines` and the Text widget to the preference - (with a +100 hysteresis so it is not an every-line reslice); the rebuild-restore uses it too. - - `confirm_close` -> `on_close` only prompts when the switch is on. - - `restore_profile` -> `_restore_last_profile()` (startup only, never on a language rebuild) - refills the form from the saved `ui.json` "profile"; `select_profile` now persists that key. - Fills the form only, never auto-applies (convention 15). - - `reset_ui_layout()` (the `reset_layout` action) clears geometry / page / collapsed / sorts / - sash and the `window.*` geometries back to `ui_state.DEFAULTS`, then recentres and rebuilds. - Note: driver cleanup on exit was NOT added - `driver.release_on_exit` (called from `on_close`) - already unloads the driver when a run loaded it, so a toggle would only disable useful behaviour. -- `gui/app.py::FIRST_RUN_COLLAPSED` unchanged here; `README.md`/`README.en.md` project layout lists - `prefs.py`. New i18n keys (both files, and both files re-sorted to the documented key order): - `prefs.*` (labels/units/hints/groups), `tips.chart_seconds|log_lines|confirm_close|restore_profile|reset_layout`, - `dialogs.reset_layout_title|body`, `log.layout_reset`. -- Tests: `tests/test_prefs.py` - registry (every pref grouped once; all texts resolve in en+pl; - `coerce` clamps/falls back), accessors (round-trip + persistence), and each wired behaviour - (chart resize, log trim, confirm-close honoured, restore-on-start fills only when enabled, - reset-layout forgets window state). Extended - `test_windows.py::test_settings_window_holds_the_language_box_and_the_view_fields` to assert the - pref widgets render. - -### Startup / performance - -- `bean_network_tester.py`: the launcher facade now resolves the GUI (`App`, `Tooltip`, - `add_tooltip`, `make_bean_icon`) LAZILY via a module `__getattr__` (PEP 562), mirroring - `beantester/gui/__init__.py`, instead of an eager `from beantester.gui import App` at module - load. Importing the launcher no longer pulls in `tkinter` or the `gui/` package. This matters - on the GUI-launch path: with `asInvoker` + `winenv.elevate_self` (convention 19) a non-admin - start spawns a second elevated process, and the doomed pre-elevation process used to import - all of `gui/` (and Tk) for nothing before relaunching. Verified: `import bean_network_tester` - leaves `tkinter` out of `sys.modules` until `App` is first accessed; `_HAS_TK` is still - exposed (computed lazily). Guard `test_launcher_compat.py` stays green (it asserts the - engine/CLI API surface, not GUI symbols). - -- `gui/app.py`: START/STOP no longer block the Tk main thread. The blocking parts - the psutil - target resolution + `engine.start()` (WinDivert driver load, ~0.5-1 s) and `engine.stop()` - (worker-thread joins) - now run on a short-lived worker via `_begin_transition(kind, work)`. - The worker leaves `(kind, err)` on a new `self._ui_queue`; `_tick` drains it (`_drain_ui_queue`) - and applies the result on the main thread (`_finish_start` / `_finish_stop`), exactly like the - log queue and the target-warning handoff (convention 26 - no widget touch off the main thread). - While in flight, `self._transition` is `"starting"`/`"stopping"`; the button keeps showing - START/STOP with NO transitional label (owner UX decision: the work is normally milliseconds - - measured driver open ~6 ms on a warm driver - and a second click is a no-op while a transition - is set, so there is nothing to relabel or disable). `_poll_transition` re-arms via - `root.after(30 ms)` so the button flips as soon as the worker finishes instead of on the next - `_tick` (the fake-tk `after` never fires, so tests drive it through `_settle_transition`). - `_refresh_start_enabled` early-returns during a transition, and the `_on_engine_stopped` trigger - in `_tick` is gated on `_transition is None` so a mid-stop tick does not fire it early. `on_close` sets - `self._closing` before `engine.stop()`; a start finishing after that does not resurrect the UI, - and `engine.stop()` still serialises on the engine's `_stop_lock`, so no divert can leak - (fail-open, convention 20). `_settle_transition()` (join worker + drain) lets a headless test - drive the async path deterministically. - -### Engine / filters - -- New `drop_shutdown` counter (`engine.py`). `BeanEngine.stop()` records `len(self._heap)` - before clearing the delay queue, so packets captured but never injected (parked in the queue at - STOP) are accounted for instead of vanishing from the seen/delivered/dropped balance. Seeded in - `reset_stats`; bumped once under `_slock` after releasing `_cv` (the cv->slock order matches - `_enqueue`). Flows into the NDJSON summary `counters` automatically - `cli.py` builds - `counters=stats_snapshot()` AFTER `engine.stop()`. NOT added to the per-interval `sample` record - (only ever nonzero at STOP). Additive to the NDJSON schema (a new key; existing keys unchanged), - so NOT a `### BREAKING` change - same as earlier counter additions (`drop_rate`, `drop_block`). - -- New `loopback` traffic filter: one entry in `filters.py::FILTER_DEFS` - (`loopback and (ip or ipv6) and (tcp or udp or icmp or icmpv6)`). Combobox order and - `--filter` choices derive from the registry, so no GUI/CLI code changed; i18n key - `filters.loopback` added to both lang files; both READMEs' filter prose + `--filter` row updated. - Covered by the existing `test_presets_filters.py::test_every_filter_covers_ipv4_and_ipv6`. - Confirmed on real WinDivert (elevated Windows run): `--filter loopback --latency 200` took - `ping 127.0.0.1` RTT from <1 ms to ~408 ms avg (200 ms each direction) and the packet counter - tracked the loopback traffic, so WinDivert captures and reinjects 127.0.0.1/::1 correctly. - -### GUI - -- `drop_shutdown` shown in the live counters grid: entry in `gui/pages/stats.py::STAT_ROWS` - (after `drop_overflow`) plus the key added to the hardcoded tuple in `refresh_counters`; i18n - keys `stats.shutdown_dropped` + `tips.stat_shutdown` in both lang files. Label/tip deliberately - reassuring, because it is routinely nonzero on any stop that used latency or a speed limit - (packets were queued, not lost in transit). CSV column `dropped_at_stop` added to - `App.CSV_COLUMNS` (every `drop_*` counter has a friendly `dropped_*` name). - -- `tips.stat_corrupted` reworded (both lang files). `corrupted` counts successful payload - bit-flips only: `BeanCore.corrupt_packet` returns False for a payload-less packet (bare ACK) - and the engine never bumps the counter, so on real TCP traffic it trails the configured - corruption percentage. The tooltip now says so. Behaviour unchanged - correct, since there is - no payload to corrupt. - -- Per-field "?" help sheet generalised beyond filter expressions. `fields.Field` gained - `help_title` / `help_body` (i18n keys, default ""); `gui/form.py::ControlForm._place_one` - renders the existing `Help.TButton` for any field declaring `help_body` (new `_show_field_help`, - opens `dialogs.show_help`), in an `elif` after the `kind == EXPR` branch. Hover shows the field's - own `tip`, a click opens the sheet. The `buffer` field now declares - `dialogs.buffer_help` / `dialogs.buffer_help_title` (new keys in `lang/en.json` + `lang/pl.json`). - `settings.DEFAULT_SETTINGS["buffer"]` changed 2000 -> 1000 ms (rate-limiter link buffer, - `BeanCore.buffer_s`): not a public contract (no test asserts the default; `--buffer` and - `core.set_buffer` unchanged). Chosen after a measurement sweep - delivered rate is accurate at - every buffer and under sustained overload loss converges regardless, so the buffer only trades - added latency against onset-of-loss; 1000 ms halves worst-case added latency and tracks sub-2 s - schedule steps. Both READMEs updated. - -- Control-page jitter on START/STOP: `form.py::ControlForm` no longer `pack`/`pack_forget`s the - per-section override/lock note. The label is packed ONCE at build time (in `_place_fields`) and - kept mapped; `apply_overrides` now only sets its `text` ("" when idle). An empty `ttk.Label` - reserves the same one-line height as a full one (measured), so the section height is constant and - the scrolled form stops reflowing/jumping when the `fields.locked_running` note appears at START. - Guard already present: `test_gui_layout.py::test_schedule_overrides_greys_the_constant_limits` - asserts the note text is `""` when idle (unchanged by this fix). - -- Profile picker styling (`theme.py`, `pages/control.py`): `Profile.TMenubutton` now uses the flat - `Menubutton.indicator` arrow inside `Combobox.field` instead of `Combobox.downarrow`. clam draws - the downarrow as a bordered, sunken button (a lighter `BORDER`/#39404e box) which read as a - "white arrow" next to the flat traffic-filter combobox; the indicator is a bare triangle. - `style_menu(menu, like_combobox=True)` paints the profile dropdown on `FIELD` (matching the - `*TCombobox*Listbox` popdown colour) instead of the `BG2` card colour; context menus keep `BG2`. - -- Connections avg column: extracted `views.avg_packet_bytes(c)` (rounds `bytes / max(1, packets)`) - and routed BOTH `conns.py::_render` and `App.export_connections_csv` through it. They had - duplicated the formula and diverged - the table rounded (`f"{avg:.0f}"`), the CSV floored - (`bytes // packets`), so 767.5 B/pkt read 768 on screen and 767 in the file. Tests: - `test_views.py::test_avg_packet_bytes_rounds_like_the_table` and - `test_conns_export.py::test_export_connections_csv_avg_matches_the_table_rounding`. - -- Session average throughput: extracted `rates.average_kbps(total_bytes, elapsed_s)` (pure) and - used it in `pages/stats.py`; the figure was computed inline from a MB value already rounded to - two decimals, now it divides the full-precision byte count. Test: - `test_gui_helpers.py::test_average_kbps_is_total_bytes_over_elapsed`. - -- Release-polish pass (bug fixes + UI cleanup): - -- Running-state chrome: `App._sync_running_ui` now also sets the window title - (`APP_NAME` + `T("app.title.running")` tag) and swaps `root.iconphoto` between an idle and a - running icon. Both flow through the ONE place already called after every start/stop and every - language-switch rebuild, so the tag/icon never desync (same reason status/filter live there). - `gui/icon.py`: `make_bean_icon(active=)` stamps a red recording dot (`_put_dot`); `_running_variant` - copies the idle PhotoImage (keeps a user `bean.png`'s art) and stamps the dot, falling back to a - drawn active bean. `apply_window_icon` now returns `(idle, running)`; `App` keeps both refs - (`_icon_idle`/`_icon_running`) so Tk does not GC them. New i18n key `app.title.running` - (en `"● RUNNING"`, pl `"● DZIAŁA"`). CLI is untouched (GUI-only, `test_layering` still holds). - Test: `test_gui_state.py::test_language_switch_keeps_running_state` now asserts the title carries - the tag while running and reverts to bare `APP_NAME` on stop. - -- Release-polish pass (bug fixes + UI cleanup): - - Native crash capture is now armed LAZILY (`crashlog.arm_native`), not at launch: - `install(native=True)` only records intent (`_arm_wanted`) and registers - `atexit(_cleanup_native)`; it no longer opens the file. `BeanEngine._start_locked` calls - `crashlog.arm_native()` right after `driver.mark_driver_used()` - the one moment a native - crash becomes possible (real WinDivert loaded), so `--simulate`/tests never arm it. faulthandler - must hold its file open before a hard crash, so it cannot be created purely on-demand, but this - means opening the GUI leaves NO `crashes/` folder. `_cleanup_native` (atexit) closes the stream, - removes the empty `native-crash.txt` and `os.rmdir`s an empty `crashes/`; a real segfault skips - atexit so a genuine report survives. `reset()` clears the new native flags for test isolation. - DEBUG-severity records still persist (owner decision). - - Profile picker (`gui/app.py`, `gui/pages/control.py`): the `Menubutton` is now posted - explicitly via `App._post_profile_menu` bound to `` (returns `"break"`), fixing the - intermittent press/release toggle where the dropdown reopened shut. `_rebuild_profile_menu` was - simplified to a plain `add_command` list (presets, `add_separator`, then user profiles): the - disabled group-heading entries rendered as muddy "blurry" text and the selected-item tick (both - the native radiobutton indicator and a hand-drawn glyph) looked wrong, so both are gone - the - current profile shows on the button via `textvariable`. The `profiles.presets_separator`/ - `mine_separator` i18n keys are kept (still used by `profile_names`/`_profile_separators` for the - reserved-name guard), just no longer shown in the menu. - - Connections tint (`gui/pages/conns.py`, `gui/theme.py`): `_tag_of` is an instance method gated - on `_scope_active` and the flow's CURRENT scope (see the dropdown/scope pass below), so the - "impaired" tag never floods when no target narrows. - `CONN_COLORS["impaired"]` switched from a muddy `background` to an amber `foreground`. - - `SortableTree._on_select` (`gui/widgets/sortable_tree.py`) drops blank-slot iids from the - widget selection (re-`selection_set`/`selection_remove`), so a click below the last real row - no longer leaves an empty row visibly selected. - - Chart (`gui/chart.py`): new `_axis_label(value, peak)` gives adaptive precision (int >=10, - 1 dp >=1, else 2 dp) so an idle `peak=1` axis no longer collapses to "0 0 0 1 1"; five Y - ticks only when `ph >= scaled(70)`, otherwise two (floor + peak). i18n `frames.throughput` - caption "~80 s" -> "~84 s" to match the axis (120 samples x 0.7 s). - - Windows (`gui/windows.py`): `PanelWindow.open` now sets `maxsize` (`_max_size` = `SIZE * - MAX_FACTOR` clamped by `scaling.max_window_size`) and calls `theme.disable_maximize`, so every - registered window is capped and non-maximisable like the main window (convention 25, updated - in PROJECT_NOTES). - -- Tooltips are suppressed while a Tk grab is held (`gui/tooltip.py`): new `_grab_active(widget)` - guards the shared `_show_bubble`, so a field's bubble no longer draws over an open combobox - popdown (the list the user just opened). Detection uses the raw `grab current` Tcl call - the - ttk popdown is a Tcl-only window, so `Misc.grab_current`/`_nametowidget` raises on it; - `grab_current()` stays as a fallback for the test double (no `.tk`). Same pattern as - `WheelDispatcher._popdown_open`. Silence goes through `crashlog.quiet` (convention 30), not - `except: pass`. Modal dialogs also grab but carry no tooltips and block background hover events, - so nothing regresses. - -- Dropdown/combobox polish pass (screenshot-driven, on the Connections/Control pages): - - Scenario dialog (`gui/app.py::load_scenario`) passes `initialdir=paths.scenarios_dir()`; new - `paths.scenarios_dir()` resolves the bundled `scenarios/` via `_resource_bases()` (same pattern - as `lang_dir`), so it points at `_MEIPASS`/`_internal` when frozen. Single source for the path. - - `theme.unhighlight_combobox` now also hands focus to the widget's `master` after a pick: a - readonly combobox kept keyboard focus after a mouse selection, so the accent focus ring - (`TCombobox` map `bordercolor=[("focus", ACC)]`) lingered as a stuck highlight. Applies to the - traffic filter (`form.py::_on_choice`) and the language picker (`app.py`). Silence via - `crashlog.quiet` (convention 30). - - Readonly comboboxes get `height=len(values)` (`form.py` CHOICE field, `app.py` language picker): - a list that fits no longer spawns the popdown scrollbar, which renders as a light `SCROLL_BG` - bar over the near-black listbox. Confirmed by pixel probe: the bar was the themed scrollbar - (#3a4150), not an unstyled one - it just should not appear for a 6-item list. - - `_rebuild_profile_menu` (`app.py`) adds `hidemargin=True` to every `add_command`: tk.Menu - reserved an indicator gutter for the check/radio tick that was removed, leaving a stray indent. - - Profile picker now uses `Profile.TMenubutton` (`gui/theme.py`, applied in `gui/pages/control.py`): - a custom layout that borrows the combobox's own `Combobox.field` + `Combobox.downarrow` elements - (label from `Menubutton.label`), so it renders pixel-identical to a readonly combobox while still - posting the grouped menu. The bare `TMenubutton` looked flat next to the traffic filter. - -- Live targeting scope for the Connections view (`core.py`, `engine.py`, `gui/pages/conns.py`): - new read-only `BeanCore.in_scope(local_port, remote_ip, remote_port)` mirrors `decide` steps 1-2 - (process + destination gates) under the core lock, plus `BeanCore.targeting_active()`; both - delegated by `BeanEngine.in_scope_now` / `targeting_active`. The connections page recomputes a - row's scope from the CURRENT target (`_in_scope`, called for visible rows only via - `SortableTree.repaint`) for both the scope column and the "impaired" tag, instead of reading the - flow's stored `scoped` (which was the LAST packet's decision - an idle flow kept a stale flag, so - a firefox row stayed highlighted after the target was narrowed to chrome). `_build_model` now - returns `scope_active = engine.targeting_active()` (one lock) in place of the old O(n) - `any(scoped) and any(not scoped)` snapshot scan. - -- Tooltips can advertise a keyboard shortcut (`gui/tooltip.py`, wired in `gui/app.py`): new - `tooltip_text(key, shortcut)` appends `[F5]`-style bracket line (no translatable word -> no i18n - key); `add_tooltip(widget, key, shortcut=)` uses it and stores the `Tooltip` on the widget - (`_bnt_tooltip`) for tests. Wired to START/STOP (`F5`), Apply (`Ctrl+Enter`), Save (`Ctrl+S`), - Load (`Ctrl+O`). New convention 40 in PROJECT_NOTES. - -- Statistics -> Session panel now shows host identity: computer name + private IPv4/IPv6 - (`gui/pages/stats.py` `SESSION_ROWS` + `refresh_session`). Backed by a new pure helper - `utils.host_identity() -> (hostname, ipv4, ipv6)` built on `utils._route_source_ip(family, - probe)`, a connected-UDP-socket route lookup that puts NO packet on the wire (never - disturbs capture) and degrades to `"-"` when a family has no route. Deliberately NOT added - to the repro report or the NDJSON schema (privacy - the tool sends no data anywhere). New - i18n keys `session.host`, `session.private_ipv4`, `session.private_ipv6` in both langs. - New test `tests/test_core.py::test_host_identity` (asserts a 3-tuple of non-empty strings). -- Connections table split traffic into `down`/`up`/total (`gui/pages/conns.py`, - `views.py`). The engine already recorded `bytes_in`/`bytes_out` per flow in `_log_conn`; - this is presentation-only, nothing touches the capture thread. `views.DERIVED` gained - `down` (`bytes_in`/1024) and `up` (`bytes_out`/1024) and both are in the numeric sort set; - the existing `kb` column id is kept as the TOTAL column (unchanged semantics, only the - label/tooltip), so the persisted default sort (`ui_state.py` `conn_sort=kb`) and its guard - (`test_validators_settings.py`) keep working with no migration. New i18n keys `conns.down`, - `conns.up`, `tips.col_down`, `tips.col_up` in both langs; `conns.kb`/`tips.col_kb` reworded. -- Connection table gained the per-flow columns backed by the engine work above: - `pid`, `scoped` ("impaired?"), `dropped`, plus derived `avg` (`gui/pages/conns.py`, - `views.py`). `views.DERIVED` gained `avg` (`bytes`/`packets`) and `scoped` (1/0 so the - column sorts numerically); `dropped`/`pid` added to the numeric sort set. The search - predicate was extracted to `views._filter_connections`/`_connection_blob` (one source), so - the new `views.traffic_totals(conns, query, proc_map)` sums download/upload/total bytes - over the SAME filtered set - computed in the worker (`_build_model`) over the whole filtered - set, not the limited `shown`, and shown in a footer label (`conns.totals`). In-scope rows - carry a `tag_of` -> `theme.CONN_COLORS["impaired"]` highlight. The `_render` tuple grew to - 15 values to match COLUMNS (guarded so it cannot drift). New i18n keys `conns.pid`, - `conns.scoped`, `conns.dropped`, `conns.avg`, `conns.yes`, `conns.no`, `conns.totals`, - `tips.col_pid`/`col_scoped`/`col_dropped`/`col_avg` in both langs. -- Connection-table CSV export: `App.export_connections_csv` (`gui/app.py`), button on - `gui/pages/conns.py`. Reuses `views.filter_sort_connections` with the page's current - `conn_query`/`conn_sort` and `limit=0` (the display row-limit is a render cap, not part - of the export), so the file mirrors the visible order over the whole filtered set. Raw - byte columns (`download_bytes`/`upload_bytes`/`total_bytes` = `bytes_in`/`bytes_out`/`bytes`) - rather than the table's KB. Atomic overwrite (tmp + `os.replace`) to a snapshot file - `paths.CONNECTIONS_CSV_FILE` (also exported from the package `__init__`), not an append - log like the stats CSV. New i18n keys `buttons.export_conns`, `tips.export_conns`, - `log.conns_saved_to` in both langs. `CONN_CSV_HEADER` now MIRRORS the table's columns: - added `pid`, `impaired` ("yes"/"no", English like the headers - the CSV is - language-independent), `dropped` and `avg_bytes`, so the export no longer lagged the new - columns. - -### Engine / core - -- Blocking (firewall). New pipeline step 2c in `core.decide()` (documented in the module - docstring), placed AFTER the targeting gate next to LAN mode, so a process/destination - target scopes it. `BeanCore` gained `block_active`/`block_ip`/`block_port` + two matchers - and `set_block(active, ip, port)` (mirrors `set_dest`); the gate is `block_ip_matcher OR - block_port_matcher` where each side takes part only when the matcher is non-empty (an empty - `Matcher` is falsy and would otherwise match everything - that is the OR skip-empty rule, and - the reason blocking is NOT modelled as AND like destination targeting). Drops carry - `reason="block"`, `scoped=True`. Two registry fields `block_ip`/`block_port` (`fields.py`, - section `block`, kinds `KIND_IP`/`KIND_INT`+`PORT_BOUNDS`, flags `--block-ip`/`--block-port`) - - so `MATCH_FIELDS`, `build_matchers`, form, live validation and profile scope derive - themselves. `settings.py`: `DEFAULT_SETTINGS` gained `block_ip`/`block_port`, `apply_settings` - calls `engine.set_block` (tolerant `try/except` like destination), `setting_expression` - normalises `block_port`. `engine.py`: counter `drop_block` in `st`, `_bump` map entry, and a - `set_block` delegate. Surfaced additively (no contract break): CSV column `dropped_block` - (`gui/app.py::CSV_COLUMNS`), live stat row `stats.block_cut` (`gui/pages/stats.py`), one-line - `summary.block`, NDJSON `sample.drop_block` and the `[bean]` text line, repro metric - `blocked`, and the reproduce command (`repro.settings_to_cli` emits `--block-ip`/`--block-port` - - it is a hand-maintained emitter, not registry-driven, so a missing field silently drops from - the copy-paste repro command; guarded by the round-trip test below). New i18n keys `frames.block`, `tips.block`, `stats.block_cut`, `tips.stat_block`, - `summary.block` in both langs; field labels reuse `fields.ip`/`fields.port`. - -- Per-flow impairment bookkeeping behind the upcoming "impaired?"/"dropped" connection - columns. `core.Decision` gained `scoped: bool = True`; the three targeting early-returns - (process step 1, destination ip/port step 2) now pass `scoped=False`, so `scoped` marks - whether a packet cleared the targeting gate - i.e. the flow is in scope for impairment, not - merely observed. Every impairment path (loss, LAN, rst, nat, flap, mtu, syn, rate) keeps - `scoped=True`. Zero added cost: same early returns. In `engine.py`, `_log_conn` now records - `dropped` (per-flow drop count), `scoped` (latest packet's scope) and `pid` - (`_pid_for` -> `portmap.pid_for`, resolved once at flow creation like the process name); the - connection dict carries `dropped`/`scoped`/`pid`. The capture loop was REORDERED so - `core.decide()` runs before `_log_conn` (decide reads none of the connection log, so the - order is free) - one lock acquisition still records the packet plus its drop/scope. No new - O(n) work on the capture thread. These fields flow through `connections_snapshot` into the - repro report automatically (additive, backward-compatible). - -- Bounded link buffer for the speed limit: `buffer` field (`fields.py::FIELD_DEFS`, section - `speed_limit`, flag `--buffer`, default 2000 ms, 0 = unbounded). The token bucket can no - longer run unbounded ahead; offered load above the cap is dropped once the buffer fills, - which bounds the added latency to ~`buffer` ms and lets a mid-session cap increase take - effect within that window. New counter `drop_rate` (CSV `dropped_rate_limit`), kept - separate from `drop_flap`, from loss, and from the queue's own `drop_overflow`. A sub-byte/s - cap now floors at 1 B/s instead of rounding to 0 ("unlimited"). -- RST injection and SYN dropping are exercised off Windows: `--simulate` now carries a real - TCP/UDP/ICMP protocol mix and the RST packet is built through the traffic source, so - `rst_sent` moves in tests and simulation instead of only on Windows with WinDivert. - -### Tests - -- `tests/test_engine.py::test_packets_queued_at_stop_are_counted_as_drop_shutdown`: a 60 s latency - parks 200 packets in the delay queue; after `stop()` it asserts `drop_shutdown == 200`, nothing - delivered, and `seen == delivered + drop_shutdown` (the balance closes to the end of the session). - -- `tests/test_gui_layout.py`: `test_only_filter_expressions_get_the_syntax_cheat_sheet` renamed to - `test_fields_with_a_help_sheet_get_the_question_mark_button` and now asserts `buffer` is present - in `ControlForm.helps` alongside the five expression fields (guards the generalised "?" help - wiring from `Field.help_body`). - -- `tests/test_failsafe.py::test_start_and_stop_run_off_the_ui_thread`: a slow `engine.start`/ - `engine.stop` (0.4 s sleep) proves `_start()`/`_stop()` return in < 0.2 s (do not block the UI - thread), the button keeps showing START/STOP with no transitional label, and - `_settle_transition()` drives the async result to `running` True/False. - `test_the_gui_starts_the_session_with_its_duration` gained the matching `_settle_transition()` - call now that start is asynchronous. - -- Release-polish pass: - - `tests/test_crashlog.py`: `test_launch_creates_no_crash_folder_until_a_capture_arms_it` and - `test_arm_native_is_a_noop_when_native_was_not_requested` guard the lazy `arm_native`; plus - `test_cleanup_removes_the_empty_native_file_and_dir` and `test_cleanup_keeps_a_non_empty_native_file` - for `_cleanup_native` (empty file + empty dir removed on clean exit; a written report is preserved). - - `tests/test_virtual_tables.py::test_clicking_a_blank_slot_selects_nothing` guards the - `_on_select` blank-slot fix (clicking a blank slot clears it; a real+blank click keeps only - the real key). `tests/fake_tk.py` `Treeview` gained `selection_remove`. - - `tests/test_windows.py::test_a_window_is_dark_and_hidden_before_it_is_shown` gained a - `maxsize` assertion (every window is capped, not just given a minsize). - - `tests/test_gui_release_fixes.py::test_tooltip_is_suppressed_while_a_dropdown_is_open` guards - the tooltip grab-guard: `_grab_active` is false with no grab, true after `grab_set()`, and - `_show_bubble` returns `None` (short-circuits before rendering) while a grab is held. - - `tests/test_gui_release_fixes.py`: `test_short_dropdowns_do_not_spawn_a_popdown_scrollbar` - (filter/language comboboxes carry `height == len(values)`), - `test_profile_menu_has_no_indicator_gutter` (every profile `add_command` has `hidemargin`), - `test_scenario_dialog_defaults_to_the_bundled_scenarios_dir` (`paths.scenarios_dir()` exists and - is named `scenarios`), `test_shortcut_buttons_advertise_their_key` (START/Apply tooltips carry - `F5`/`Ctrl+Enter`), `test_profile_picker_uses_the_combobox_field_style` (Menubutton uses - `Profile.TMenubutton`). Combobox focus-drop (#7) verified live on real Tk, not in the fake - (fake `focus_set` is a no-op). - - `tests/test_conns_columns.py::test_connection_columns_tag_and_footer` rewritten for live scope: - it sets `engine.core.set_target(True, {5000})` and gives the out-of-scope svchost row a STALE - `scoped=True`, asserting the column and tag follow the current target (svchost -> "no", no tag) - rather than the stored flag. -- Blocking: `tests/test_core.py` gained six tests for pipeline step 2c - block by IP drops - matching only (reason `block`, `scoped=True`), block by port ignores an empty IP field (the - OR skip-empty rule), IP and port combine with OR (not AND like destination targeting), block - sits after the process-targeting gate (a non-target flow passes even when its destination is - on the block list), inactive block passes everything, and a malformed expression raises - `ValueError`. `block_ip`/`block_port` added to `IMPAIRMENT_OFF` and `drop_block` to - `DAMAGE_COUNTERS` in `tests/test_passthrough.py` (a new default-harm field would otherwise slip - through). Registry-guard sets updated for the two new expression fields: - `test_field_registry.py`, `test_settings_config_scenario.py` (MATCH_FIELDS view), - `test_gui_layout.py` (filter fields get the syntax cheat sheet), and the hand-written fake - engine in `test_cli_runtime.py` gained a `set_block` stub. - `test_summary_repro_views.py::test_settings_to_cli_roundtrip` extended with `block_ip`/ - `block_port` so the reproduce command is proven to round-trip them. -- Engine and settings coverage: `test_engine.py::test_block_integration` (block drops matching - IP and port through a real `BeanEngine`+`FakeDivert`, `drop_block == 2`, OR of the two - fields); `test_settings_config_scenario.py` gained `test_apply_settings_bad_expression_disables_blocking` - (tolerant path), `test_apply_settings_with_block_expressions`, `test_scenario_block_step_applies_and_clears`, - and extends `test_apply_settings_maps_engine`, `test_config_roundtrip_keeps_expressions` and - `test_validate_settings_rejects_bad_expressions` with block fields; `test_summary_repro_views.py::test_summary_shows_blocking`. -- `scenarios/blocked-endpoint.json` shipped (loops a backend/API outage via `block_ip` on a - TEST-NET-3 range, then clears it) - auto-validated by `test_shipped_scenarios.py`. -- `tests/test_conns_columns.py` - drives the real `ConnsPage` on fake-tk: the 15-column - render tuple lines up with COLUMNS, `pid`/`scoped`/`dropped` cells render, the in-scope row - gets the `impaired` tag and the observed-only row does not, and the footer sums the filtered - traffic. `tests/test_views.py` gained `test_traffic_totals_sum_filtered_bytes` (footer sum - honours the search), `test_derived_avg_and_scoped` and `test_sort_by_every_new_numeric_column` - (down/up/dropped/pid each actually order the table). `tests/test_conns_export.py` now also - covers the connection export's new columns (pid/impaired/dropped/avg_bytes) and adds - `test_export_csv_stats_appends_then_rotates_on_a_column_change` - the FIRST test of the stats - CSV export at all (append, and the roll-aside-on-header-change branch). -- `tests/test_core.py::test_decision_scoped_reflects_targeting` - `Decision.scoped` is False - only when process/destination targeting excludes the packet, True with no targeting, on a - matching target, and on an impairment drop (LAN). `tests/test_engine.py::test_connection_records_scope_and_dropped` - - with targeting on one port and 100% loss, only the targeted flow is `scoped` and counts - its `dropped`; the other is merely observed; every row carries a `pid` field. -- `tests/test_engine.py::test_lan_mode_integration` strengthened: now also feeds an INBOUND - packet from a public `src_addr`, so LAN mode is proven to cut the internet both ways - (`remote_ip` = `dst_addr` outbound / `src_addr` inbound), not only outbound. Expected - `drop_lan` 2 -> 3, with a LAN-bound packet still passing. -- `tests/test_conns_export.py` - guards `App.export_connections_csv`: the CSV header, the - raw `download_bytes`/`upload_bytes`/`total_bytes` split, sort order carried into the file, - the atomic overwrite leaving no `.tmp`, and that the current search narrows the export. - First test to exercise a CSV export at all (the stats export was never covered). -- `tests/test_bandwidth_buffer.py` - bounded queueing delay (added latency capped by - `buffer`), recovery after a mid-session rate increase, `drop_rate` counting on a full - buffer, and the legacy unbounded behaviour. Locks in the fix for the old token bucket that - ran seconds ahead at a low rate and swallowed every later high-rate step. -- Reminder (convention "new impairment"): every traffic-damaging field must also be added to - `IMPAIRMENT_OFF` in `tests/test_passthrough.py`, or a harmful default slips through unseen. - -### Build / packaging - -- `BeanNetworkTester.spec`: two size trims, neither touching startup or runtime (onedir does not - unpack at launch - PROJECT_NOTES "performance > size"). - (1) Drop Tcl's bundled IANA timezone database (`_tcl_data/tzdata`) and msgcat catalogs - (`_tcl_data/msgs`, `_tk_data/msgs`) from `a.datas` after Analysis - the tool uses Python's - `time` (never Tcl `[clock]`) and its own `lang/*.json` i18n, so ~750 files were dead weight. - (2) `excludes` now also drops `ssl`, `_ssl`, `_hashlib` -> OpenSSL (libcrypto ~6 MB + libssl - ~1.3 MB) is no longer collected. The app has no network TLS (convention 36: no telemetry) and - `import bean_network_tester` never pulls in `ssl`; its only hashing is crashlog's sha1 - fingerprint, and `hashlib` falls back to the built-in `_sha1` module when `_hashlib` is absent - (verified: `sys.modules['_hashlib']=None; hashlib.sha1(...)` still works on this CPython, and - `_sha1` is built into python314.dll, not a separate excluded .pyd). - Measured `dist/BeanNetworkTester/_internal`: 1020 -> 262 files, 27.9 -> 19.2 MB. Encodings are - KEPT (Tk needs them). Verified on the rebuilt exe: `--version`, `--simulate --loss 10 - --duration 2` (exit 0), and the GUI window opens; `libcrypto*`/`libssl*`/`_hashlib*`/`_ssl*` are - absent from `_internal`. - -### CI / tooling - -- `.github/workflows/ci.yml`: pinned actions moved to their Node 24 majors - (`checkout@v5`, `setup-python@v6`, `upload-artifact@v6`); fixed the coverage-artifact name - (`matrix.python` -> `matrix.python-version`, previously empty and colliding across the two - Python versions); added `concurrency` (cancel superseded runs), least-privilege - `permissions: contents: read`, per-job `timeout-minutes`, a weekly `schedule` run (catches - drift in unpinned `pydivert`/`psutil`), and `CHANGELOG.md` to the required-release-files - check. New headed GUI render check under Xvfb (`tools/ci_gui_render.py`) that builds the - real Tk `App` at 1366x768 in both languages and fails on truncated key widgets - catches - layout regressions the fake-tk smoke cannot see. -- `.github/dependabot.yml`: weekly updates for the `github-actions` and `pip` ecosystems. - -## [0.2.0] - 2026-07 - first public release - -First tagged release. See `CHANGELOG.md` for the user-facing summary. Internally this is the -package-refactor baseline: the pure decision core (`core.py`), the threaded engine -(`engine.py`), the field/filter/exit-code/preset/window/page registries as single sources of -truth, the virtualized tables, and the pytest suite (engine, i18n, CLI, fail-safe, property -based, concurrency chaos, GUI on the fake-tk harness). diff --git a/beantester/engine.py b/beantester/engine.py index ac5b7b0..d47c946 100644 --- a/beantester/engine.py +++ b/beantester/engine.py @@ -1825,7 +1825,7 @@ def _inject_loop(self): # will never see it either - without this it just left the # seen/delivered/dropped balance, silently, which is the one # thing keeping these numbers honest. Found while measuring the - # batched injector (rejected, see CHANGELOG-INTERNAL): the hole + # batched injector (built, measured and rejected): the hole # was one packet wide already, and batching would have made it # a whole batch wide. self._bump("drop_shutdown") diff --git a/tests/test_inject_batch.py b/tests/test_inject_batch.py index 1d46b82..2b86a0a 100644 --- a/tests/test_inject_batch.py +++ b/tests/test_inject_batch.py @@ -1,7 +1,7 @@ """A packet taken off the release heap must never leave the books. -Found while measuring the batched injector (measured and REJECTED - see -CHANGELOG-INTERNAL). The hole is older than that work and survives it: the inject +Found while measuring the batched injector, which was built, measured and then +rejected. The hole is older than that work and survives it: the inject loop pops a packet, then finds ``_divert`` gone because STOP cleared it in between. The packet is no longer in ``_heap``, so ``stop()``'s stranded sweep cannot see it either, and it used to vanish with no counter at all. diff --git a/tests/test_repo_conventions.py b/tests/test_repo_conventions.py index caffeff..7ff4a11 100644 --- a/tests/test_repo_conventions.py +++ b/tests/test_repo_conventions.py @@ -22,7 +22,8 @@ # exactly where they exist. SKIP_DIRS = {".git", "__pycache__", ".pytest_cache", "licenses", "build", "dist", ".hypothesis", "internal_tools", ".claude", "crashes"} -SKIP_FILES = {"PROJECT_NOTES.md", "HISTORY_NOTES.md", "CLAUDE.md"} +SKIP_FILES = {"PROJECT_NOTES.md", "HISTORY_NOTES.md", "CLAUDE.md", + "CHANGELOG-INTERNAL.md"} def repo_text_files(exts): @@ -289,7 +290,8 @@ def test_the_repository_scanners_stay_out_of_what_is_not_in_the_repository(): """ scanned = {os.path.relpath(p, ROOT).replace(os.sep, "/") for p in repo_text_files((".py", ".md", ".json", ".txt"))} - for stray in ("PROJECT_NOTES.md", "CLAUDE.md", "HISTORY_NOTES.md"): + for stray in ("PROJECT_NOTES.md", "CLAUDE.md", "HISTORY_NOTES.md", + "CHANGELOG-INTERNAL.md"): check(f"{stray} is not scanned (it is not in the repository)", stray not in scanned, f"({stray})") for prefix in ("internal_tools/", ".claude/", "crashes/"): diff --git a/tests/test_version_and_release.py b/tests/test_version_and_release.py index e3a01b7..3177573 100644 --- a/tests/test_version_and_release.py +++ b/tests/test_version_and_release.py @@ -114,7 +114,10 @@ def test_breaking_sections_come_first(): in both changelogs and nothing noticed - the em/en-dash guard reads changelog TEXT, never its structure. """ - for name in ("CHANGELOG.md", "CHANGELOG-INTERNAL.md"): + # CHANGELOG-INTERNAL.md is not in the repository (it is a maintainer file, kept + # in the private notes repo), so no test here can read it. Its structure is + # guarded by `.claude/hooks/check_notes.py`, which runs where the file exists. + for name in ("CHANGELOG.md",): lines = open(os.path.join(ROOT, name), encoding="utf-8").read().splitlines() version, sections = None, [] problems = [] @@ -145,7 +148,9 @@ def close(version, sections): # already been broken in practice. Prose is not a guard. -CHANGELOG_FILES = ("CHANGELOG.md", "CHANGELOG-INTERNAL.md") +# Only the shipped changelog. The internal one is a maintainer file kept out +# of the repository, so its guards live in `.claude/hooks/check_notes.py`. +CHANGELOG_FILES = ("CHANGELOG.md",) DATED_VERSION_RE = re.compile(r"^## \[(\d+\.\d+\.\d+)\]\s+-\s+\d{4}-\d{2}-\d{2}\b") @@ -164,8 +169,8 @@ def _versions_in(name): return blocks -def test_version_txt_has_a_dated_section_in_both_changelogs(): - """VERSION.txt must name a released, DATED section in both changelogs. +def test_version_txt_has_a_dated_section_in_the_changelog(): + """VERSION.txt must name a released, DATED section in the changelog. The failure this closes is silent by construction: `release.yml` checks the tag against VERSION.txt and nothing else, so a version bumped without its From 1f7092fddab63b8e4fab18959281de2419caf866 Mon Sep 17 00:00:00 2001 From: DonislawDev Date: Sun, 2 Aug 2026 18:27:55 +0200 Subject: [PATCH 2/2] docs(readme): say that what you make with the tool is not GPL A reader can reasonably wonder whether writing a scenario file with a GPLv3 program puts that scenario under the GPL. It does not, and both READMEs now say so in the licence section: scenarios you write, saved profiles and config files, reproduction reports, CSV exports, logs and screenshots are your own work, and using the tool obliges you to publish nothing. The distinction that would be misleading to leave out: the example scenarios shipped in scenarios/ ARE part of the project and stay GPLv3. One you wrote yourself is not, even if you started from one of them. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 8 ++++++++ README.md | 11 +++++++++++ README.pl.md | 12 ++++++++++++ 3 files changed, 31 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0f975d5..773bcab 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,14 @@ The format follows [Keep a Changelog](https://keepachangelog.com/); versions fol ## [Unreleased] +### Docs + +- Both READMEs now say in the licence section that **what you make with the tool is yours**. + Scenarios you write, saved profiles and config files, reproduction reports, CSV exports, logs + and screenshots are your own work: the GPL covers the program, not its output, and using the + tool does not oblige you to publish anything. The example scenarios shipped in `scenarios/` + are part of the project and stay GPLv3. + ### Fixed - In the Settings window, "Capture only the targeted traffic" stayed clickable after you pressed diff --git a/README.md b/README.md index 8e08377..ce1c5f3 100644 --- a/README.md +++ b/README.md @@ -1350,6 +1350,17 @@ change it, and to redistribute copies - including modified ones - provided you p under the same GPLv3 terms and make the corresponding source available. The program is provided "AS IS", with no warranty and no liability on the author's part. The author is **DonislawDev**. +### What you make with it is yours + +**The GPL covers the program, not your output.** Scenario files you write, saved profiles and +configuration files, reproduction reports, exported CSVs, log files and screenshots are your own +work. Using Bean Network Tester does not put them under the GPL and does not oblige you to publish +anything. Keep them private, ship them with a closed-source product, sell them - your call. + +The one thing to keep apart: the example scenarios that come **with** the program, in the +`scenarios/` directory, are part of the project and are GPLv3 like the rest of it. A scenario you +write yourself is not, even if you started from one of them and changed the numbers. + ## Third-party components The program uses libraries by other authors, under their own licenses - among them diff --git a/README.pl.md b/README.pl.md index 1bf314e..a8f6102 100644 --- a/README.pl.md +++ b/README.pl.md @@ -1209,6 +1209,18 @@ pod warunkiem przekazywania programu dalej na tych samych warunkach GPLv3 i udostępnienia odpowiadającego kodu źródłowego. Program dostarczany jest „AS IS”, bez gwarancji i bez odpowiedzialności autora. Autorem jest **DonislawDev**. +### To, co nim zrobisz, należy do Ciebie + +**GPL obejmuje program, a nie to, co nim wyprodukujesz.** Napisane przez Ciebie scenariusze, +zapisane profile i pliki konfiguracji, raporty reprodukcji, wyeksportowane pliki CSV, logi +i zrzuty ekranu są Twoją własną pracą. Użycie Bean Network Testera nie obejmuje ich licencją GPL +i nie zobowiązuje Cię do publikowania czegokolwiek. Możesz je trzymać u siebie, dołączyć do +zamkniętego produktu albo sprzedać - to Twoja decyzja. + +Jedna rzecz, którą trzeba oddzielić: przykładowe scenariusze dostarczane **razem** z programem, +w katalogu `scenarios/`, są częścią projektu i podlegają GPLv3 tak samo jak reszta. Scenariusz +napisany przez Ciebie już nie, nawet jeśli zacząłeś od jednego z nich i zmieniłeś liczby. + ## Komponenty firm trzecich Program korzysta z bibliotek innych autorów, na ich własnych licencjach - m.in.