From 7ea0f62977d6388dce216f4d000ede0d6ab46d86 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Arturo=20Filast=C3=B2?= Date: Fri, 7 Aug 2026 15:53:56 +0200 Subject: [PATCH] Add tunnel/circumvention-tool observation model to pipeline docs Extends ontology, requirements, architecture, implementation-plan, and label-corpus-design with a tunnel observation tier (obs_tunnel, phase axis, provider/protocol targets, cross-network control), an external-report evidence class for corroboration only, and new M8/C9 requirements. Converges with ooni/data#63; gated for rule weights until the corpus gains a screen_tunnel stratum (V2). produces through several iteration of AI assisted review and editing --- docs/architecture.md | 66 ++++++++++++++- docs/implementation-plan.md | 58 ++++++++++++- docs/label-corpus-design.md | 24 ++++++ docs/ontology.md | 164 ++++++++++++++++++++++++++++++++++++ docs/requirements.md | 13 ++- 5 files changed, 317 insertions(+), 8 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index ec9eda62..e391bba3 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -44,8 +44,9 @@ destroys the evidence needed to draw a better one later. | Tier | Table | Produced by | Grain | Status | |---|---|---|---|---| | 0 Raw | S3 JSONL | OONI collector (external) | measurement | built | -| 1 Observation | `obs_web`, `obs_web_ctrl`, `obs_http_middlebox` | `make_observations` | measurement × endpoint | built | -| 2 Evidence / judgment | `analysis_web_measurement` | `make_analysis` | measurement × target | built | +| 1 Observation | `obs_web`, `obs_web_ctrl`, `obs_http_middlebox`, `obs_tunnel` | `make_observations` | measurement × endpoint (× phase for tunnels) | built; `obs_tunnel` **[mvp]** | +| 1 External report | `external_reports` | collector (measurement type `vpnext`) | provider report (per-connection or aggregated) | **[later]** | +| 2 Evidence / judgment | `analysis_web_measurement`, `analysis_tunnel_measurement` | `make_analysis`, `make_tunnel_analysis` | measurement × target | built; tunnel **[mvp]** | | 3 Cell state | *(a view, not a table)* | `GROUP BY` over tier 2 | target × network × layer × hour | **[mvp]** | | 4 Changepoint | `event_detector_changepoints`, `event_detector_cusums` | `make_detector` | series × signal | built | | 5 Event | `events` | n/a | cc × target-set × time | **[mvp]** | @@ -55,6 +56,13 @@ computing its own hourly aggregate inline, which is why measurement counts are discarded and why a national event emits one alert per `(cc, asn, domain, layer)` rather than one alert per event. +`analysis_tunnel_measurement` deliberately mirrors `analysis_web_measurement`'s +shape (per measurement × target: blocked/down/ok plus rule id, keyed on phase +instead of layer) rather than inventing a parallel structure, so tier 3, the +detector and the alert feed consume both through one shape. `external_reports` +sits at tier 1 but outside this chain entirely: it joins no control set and +feeds only event grading at tier 5, never tiers 2 to 4 (ontology.md §5.2). + Tier 3 is deliberately **a view rather than a table** while its grain is still being learned. It holds per-layer histograms of rule firings, from which consensus, sample size and ambiguity all derive ([ontology.md](ontology.md) §9). @@ -85,6 +93,7 @@ graph TD OGEN --> OW[(obs_web)] OGEN --> OWC[(obs_web_ctrl)] OGEN --> OHM[(obs_http_middlebox)] + OGEN --> OT[(obs_tunnel)] OW --> AN[[make_analysis]] OWC --> AN @@ -92,13 +101,23 @@ graph TD RULES[analysis/rules.py
rule registry] --> AN AN --> AWM[(analysis_web_measurement)] + OT --> TAN[[make_tunnel_analysis]] + BASE[(tunnel baseline aggregate
cross-network rollup)] --> TAN + TAN --> ATM[(analysis_tunnel_measurement)] + AWM --> DET[[make_detector]] + ATM --> DET CLB[(citizenlab)] --> DET DET --> CP[(event_detector_changepoints)] DET --> CUS[(event_detector_cusums)] CP --> SLACK[Slack alerts] + EXT[External report collector
type vpnext] --> ER[(external_reports)] + ER -.corroboration only.-> EVT[[event grading]] + CP --> EVT + AWM --> API[[FastAPI]] + ATM --> API OW --> API API --> USERS[Explorer / researchers] @@ -107,7 +126,9 @@ graph TD ``` Note `fastpath` is the *v4* pipeline's table, not produced here. The two data -quality jobs read it; nothing else in v5 depends on it. +quality jobs read it; nothing else in v5 depends on it. The dotted edge +from `external_reports` marks that it feeds event grading and nothing +upstream of it. --- @@ -166,20 +187,35 @@ an oversight. **Only web_connectivity produces this.** - `obs_http_middlebox`: HIRL/HFM results. Different shape, hence a different table. +- `obs_tunnel` **[mvp]**: one row per (measurement, endpoint, phase) for + tunnel nettests (`openvpn` today). No control counterpart; see + [ontology.md](ontology.md) §2, §3.1. + +**External report tier** **[later]** + +- `external_reports`: provider-submitted connection reports, +- per-connection or provider-aggregated. Outside the tiered chain: + joins no control set. [ontology.md](ontology.md) §5.2. +- should consider aligning the semantics with [MANTA client metrics](https://0xacab.org/leap/manta/-/blob/no-masters/2026-06-09-client-metrics.md) **Judgment tier** - `analysis_web_measurement`: per measurement: `(blocked, down, ok)` per layer, the top failure per layer, and `top_{dns,tcp,tls}_rule_id`. +- `analysis_tunnel_measurement` **[mvp]**: per measurement: `(blocked, down, + ok)` per phase and the driving rule id, mirroring the web table's shape. **State tier** **[mvp]** - Cell state: a view over tier 2, not a table. §3.1 below; semantics in - [ontology.md](ontology.md) §9. + [ontology.md](ontology.md) §9. Extends to tunnel targets at the same grain, + keyed on phase instead of layer. **Detection tier** - `event_detector_cusums`: per `(cc, asn, domain)` CUSUM accumulator state. + Tunnel series extend the key to `(cc, asn, tunnel_target, phase)` + ([ontology.md](ontology.md) §11); no new table, same accumulator shape. - `event_detector_changepoints`: emitted transitions. **Reference data**: externally maintained, refreshed by updater DAGs: @@ -187,6 +223,11 @@ an oversight. `asnmeta`. The fingerprint tables are `EmbeddedRocksDB` and are swapped in atomically via `EXCHANGE TABLES`. +**Tunnel baseline aggregate** **[mvp]**: a per-(tunnel target, window) +success-by-network rollup, standing in for the control web targets get from +`obs_web_ctrl`. New infrastructure, not a byproduct of an existing job; +§3.2 below states the cost reasoning. + **Data quality**: `faulty_measurements`, written by the volume and time-inconsistency jobs. @@ -229,6 +270,23 @@ closed window into a plain table. **Not** an insert-time `AggregatingMergeTree` materialized view: MVs fire per insert and never observe `ReplacingMergeTree` replacement, so nightly re-analysis would double-count every re-scored row. +### 3.2 Tunnel baseline aggregate **[mvp]** + +Tunnel nettests have no test helper, so scoring needs a substitute control: +whether the same target succeeded elsewhere in the window +([ontology.md](ontology.md) §5.1). Unlike the web control, this cannot be +computed inline per run. §4.4 already gives the cost reason for why the web +control window stays narrow: scanning a trailing 24h or 7d of raw +observations on every hourly run is not affordable. The same arithmetic +applies here, with no narrow-window option to fall back on, since there is no +test helper to ask instead. + +The aggregate is therefore precomputed: per `(tunnel_target, window)`, a +success/failure rollup by network, rebuilt per closed window like tier 3. +Small by construction (the watchlist of registered targets, not the full +measurement volume), and this is the one piece of standing infrastructure +the tunnel work adds beyond extending existing tiers. + ## 4. Tradeoffs Each closes with the requirement conflict it embodies, in diff --git a/docs/implementation-plan.md b/docs/implementation-plan.md index d27f60f9..f86510e2 100644 --- a/docs/implementation-plan.md +++ b/docs/implementation-plan.md @@ -73,7 +73,7 @@ likelihood-ratio fits remain winner-censored. | Corpus thin, fits winner-censored | First labels are drawn (two adjudicators) and the LR fit runs, but volume is far below the targets that make per-rule LRs informative, and only the *winning* rule is persisted, so fits are conditional on cascade position. | V1, V2 | | Detector watchlist is narrow | Citizenlab global `GRP` plus `twitter.com`; country-specific news blocking is structurally invisible. | priority 5, D1 | | `resolver_transport` unused | `dns_engine IN ('getaddrinfo','system')` excludes DoT/DoH/UDP answers from the answer set, so `dnscheck`'s central comparison is dropped. | E1, M1 | -| ~28 spec nettests have no transformer | Notably `openvpn`, actively collected with no analysis path at all. | M1 | +| ~28 spec nettests have no transformer | `openvpn` is addressed by §3.14-3.16 below; the remaining ~27 stay deferred (§4). | M1, M8 | | Reference data is unversioned | Fingerprints and test lists join as of run time; no verdict names the corpus state that produced it, and a rebuild cannot reproduce old output for validation. | P3, A5 | | Alerts are not reconstructible | Changepoints reference no measurement set, tier-2 rows mutate under them, and the recovery procedure regenerates history under current code. | P5, P4 | | Late data never reaches detection | The daily DAG has no detector task; the CUSUM never revisits a closed hour. Late uploads correlate with censored networks (C7). | D2 | @@ -106,8 +106,11 @@ would re-derive (§6.1 there). | Provenance and reproducibility | 3.9 (versioned scoring inputs) | | Unreconstructible alerts, late data, backfill hazard | 3.10 (stateless detection) | | Silent degradation in production | 3.11 (health checks) | +| No circumvention-tool analysis or alerting | 3.14 (observation tier), 3.15 (analysis path), 3.16 (series and alerting) | -Roughly eight to ten weeks. Everything else is §4. +Roughly eight to ten weeks for 3.1-3.13; 3.14-3.16 add a further three to +five weeks, riding 3.6/3.7/3.10 rather than duplicating them. Everything else +is §4. --- @@ -488,6 +491,53 @@ controls the strictest admission standard in the system. **Effort:** days for the floor and sanity checks; revalidation waits on the certificate-storage decision. **Serves:** A4, A2. +### 3.14 Tunnel observation tier + +Land `obs_tunnel` ([ontology.md](ontology.md) §2), converging the OpenVPN +observation models already drafted in +[ooni/data#63](https://github.com/ooni/data/pull/63) with the shape that +entry specifies: one row per (measurement, endpoint, phase), the protocol +stack, and no control counterpart. Add the phase columns (§3.1) and wire +`make_observations` for `openvpn`. + +Additive: a new table and transformer, no change to existing rows. The PR +exists and is open for review; this item is convergence and test coverage, +not new design. **Effort:** the PR's scope is most of it; convergence with +the unsupported-nettests review (#48) and tests, days. **Serves:** M8. + +### 3.15 Tunnel analysis path + +Register tunnel targets (`provider/protocol+obfuscation`, +[ontology.md](ontology.md) §4.1), build the cross-network baseline aggregate +(architecture.md §3.2) that stands in for the missing test helper, and score +`analysis_tunnel_measurement` from it: a rule cascade keyed on +`(target, phase)`, mirroring the web judgment tier's shape. + +Rules ship **weightless** until 3.8 gains a tunnel stratum: this is the same +V2 discipline the web rules follow ("no unmeasured changes"), and skipping it +here would mean the alert feed's first tool-reachability alerts inherit +exactly the uncalibrated-constant problem the labelled corpus exists to +retire. **Effort:** the baseline aggregate is the real work; 1 to 2 weeks. +**Depends on:** 3.14. **Serves:** M8, C9 (aggregate-only publication is +enforced at this layer, not bolted on at presentation). + +### 3.16 Tunnel series and alerting + +Extend series to `(cc, asn, tunnel_target, phase)` +([ontology.md](ontology.md) §11), through the same cell state, detector and +alert feed as web series, not a parallel system. Events carry both the +provider ladder and the protocol ladder (§4.1) in their scope, so an alert +can say "RiseupVPN" or "openvpn+obfs4 generally" depending on what the +evidence supports. + +Rides 3.6 (cell state), 3.7 (event correlation) and 3.10 (stateless +detection) rather than duplicating any of them: once those land for web +series, extending them to a second series kind is the marginal cost. +**Effort:** days once 3.6/3.7/3.10 land. **Depends on:** 3.15, 3.6, 3.7, +3.10. **Serves:** M8, D1 (endpoint rotation must not enter the series key, +which matters more here than for web: a provider's own discovery system can +rotate a pool on a schedule unrelated to any network condition). + --- ## 4. Later @@ -513,7 +563,9 @@ blocked on it. | Full `locus` axis with inference (E7) | 3.5 stabilises series; naming *who* is responsible needs cross-network resolver checks that do not exist. Blockpage-fingerprint `scope` already attributes what it matches, and the planned middlebox and transit detectors widen that set. See [ontology.md](ontology.md) A.2. | | Dempster-Shafer verdicts / pipeline-side mechanism inference / target hierarchy / cert store | Ontology Appendix A, each with its trigger. (The mechanism *taxonomy* was promoted into ontology §12; what stays deferred is the pipeline emitting mechanism labels as output.) | | Widening the detector watchlist (priority 5) | Needs 3.7 first, and the stale hardcoded entry fixed (D1). | -| Coverage of ~28 unsupported nettests | `openvpn` first (actively collected, no analysis path); then the low-level probes, which map onto existing observation shapes. | +| Coverage of remaining ~27 unsupported nettests | `openvpn` moved into scope (3.14-3.16); the low-level probes remain, mapping onto existing observation shapes. | +| External reports tier (M8, C9) | `external_reports` table and `vpnext` transformer, corroboration join at event grading only ([ontology.md](ontology.md) §5.2, Appendix A.7). Trigger: a signed provider agreement under the published report format, with PR1's retention, access and aggregation-floor decisions settled first. This is where a companion project's provider-telemetry relationship (if one exists) would arrive pipeline-side: that project owns the collection and the relationship, this plan owns what the data may and may not mean once it does. | +| Tunnel mechanism taxonomy leaves (`tunnel.*`) | Reserved, not scored ([ontology.md](ontology.md) §12.3). Trigger: first adjudicated tunnel labels (3.8-style corpus extension) or alignment with an external taxonomy effort. | --- diff --git a/docs/label-corpus-design.md b/docs/label-corpus-design.md index 6aabff05..d0717f21 100644 --- a/docs/label-corpus-design.md +++ b/docs/label-corpus-design.md @@ -258,6 +258,20 @@ labels drawn either side are not one population. with a wrong predicate is worse than an absent one, because it silently deflates every LR denominator. +`screen_tunnel` is proposed, **not implemented**, and gated on +[implementation-plan.md](implementation-plan.md) §3.15 landing first: there is +nothing to draw from until `analysis_tunnel_measurement` exists. Once it does, +the same reasoning that motivates the layer strata applies with more force, +since tunnel evidence has no test-helper control at all to lean on: a uniform +draw would oversample whichever provider or protocol has the most traffic, so +draw explicitly per protocol stack, with `screen_negative`'s role (bounding +what the pipeline misses) played here by targets that tested reachable +everywhere. **This stratum must exist, populated, before any tunnel rule +weight ships** ([implementation-plan.md](implementation-plan.md) §3.15's V2 +gate): a tunnel rule cascade running on unlabelled data is the same mistake +web scoring made before this corpus existed, repeated on a target class OONI +has less experience judging. + --- ## Part 2: How an analyst should think about it @@ -342,6 +356,16 @@ build a corpus of any size. Controls: `B` blocked, `D` down, `O` ok, `U` can't call it, `X` unusable, then confidence, mechanism chips, and a rationale field. Commit advances. +**Tunnel adjudication: not implemented.** The queue and the Request/Observation/ +Control panel layout above assume a web measurement (DNS/TCP/TLS/HTTP, +control diffed against `obs_web_ctrl`). A tunnel row has no control panel to +diff against (ontology.md §5.1's cross-network comparison is not a +side-by-side view); the Observation panel would need to show phase reached +and the sibling-endpoint comparison instead. `tunnel.*` mechanism chips stay +greyed out (ontology.md §12.3 marks the nodes `[reserved]`) until this lands. +**Trigger:** [implementation-plan.md](implementation-plan.md) §3.15 landing, +which is also what `screen_tunnel` (§1.4 above) is gated on. + ### 3.2 The event editor: implemented [event-labeler.html](event-labeler.html). A form over the event schema, four diff --git a/docs/ontology.md b/docs/ontology.md index 59433418..9a3986cb 100644 --- a/docs/ontology.md +++ b/docs/ontology.md @@ -66,6 +66,16 @@ analysis does not extend to other nettests. verdict, so it is a separate table: shape, not subject matter, is the right reason to split one. +**`obs_tunnel`** **[mvp]** holds one row per (measurement, endpoint, phase) for +circumvention-tunnel nettests (`openvpn` today; `wireguard` and peers as they +land, X3). Columns record the protocol stack (`protocol`, `obfuscation`), the +endpoint attempted, the phase reached (§3.1) and the failure if any. No control +table exists for tunnels: the test-helper model does not apply, since there is +no third party positioned to fetch the same tunnel. Scoring instead leans on +cross-network comparison (§5). Shape, not subject matter, is again why this is +a separate table: a tunnel row has no hostname ladder and no HTTP transaction, +and a web row has no phase. + ## 3. Layer `dns | tcp | tls | http`. **Observed, not inferred**: the DNS query either @@ -84,6 +94,25 @@ A known defect: the analysis query restricts the DNS answer set to `dns_engine IN ('getaddrinfo','system')`, so DoT and DoH answers are silently excluded from scoring, which discards the entire point of `dnscheck`. +### 3.1 Phase, for tunnel observations **[mvp]** + +`bootstrap | dns | handshake | tunnel_check`. Phase is to `obs_tunnel` what +layer is to `obs_web`: observed, not inferred, and the one dimension that +never demands an attribution the evidence cannot make. A `handshake` failure +does not by itself say whether the endpoint address was targeted or the +protocol itself was: distinguishing those needs cross-endpoint comparison (an +address-blocked endpoint fails while a sibling on the same protocol succeeds +elsewhere; protocol-targeted DPI fails them all). That comparison is an +analysis conclusion (§4.1), never a per-observation field. + +`tunnel_check`, where the nettest verifies an established tunnel actually +passes traffic, is deliberately scored no more finely than reach / no-reach. +Anything about the quality of a passing tunnel is throughput and experience +measurement, which this pipeline does not do (M7's throttling deferral +applies here too, and a neighbouring project, LEAP's MANTA, covers +in-tunnel quality of experience from the client side; keeping `tunnel_check` +binary is what keeps the boundary clean). + ## 4. Target The subject being assessed. @@ -114,6 +143,37 @@ combine: `any_of` for redundant pools (WhatsApp's 16 hosts, Telegram's datacentres), `all_of` for independently required services (Signal's chat, directory, CDN, SFU, storage). Populated; **no scorer reads it yet**. +### 4.1 Tunnel targets **[mvp]** + +Tunnel endpoints rotate faster than CDN pools, so the naming rule above binds +harder: `target_id` keys on `provider/protocol+obfuscation` (`riseup/openvpn`, +`unknown/openvpn+obfs4`), never on the endpoint address. +`combination_rule` is `any_of` over the provider's gateway pool, the same +registry field IM targets already populate. +Endpoint addresses MUST be stored in such a way that they prevent enumeration +in the published dataset and MAY offer the ability for the provider to lookup +measurements for a particular addresses knowing the address and some shared +secret (eg. `public_address = hash(private_address, key)`). + +Two aggregation ladders rise from the same observations, and they answer +different questions: + +- The **provider ladder** groups by provider: "does RiseupVPN work in Iran?" +- The **protocol ladder** groups by protocol stack across providers: "does + `openvpn+obfs4` work on this network, for anyone?" + +The protocol ladder is the more censorship-relevant of the two, and it is +also what makes endpoint-grain evidence publishable in aggregate without +enumerating endpoints (C9): a reader learns that a protocol stack is blocked +on a network without learning which specific addresses were tested. + +A `target_id` here can also name a provider unknown to the registry +(`unknown/wireguard`), for endpoints reported without a registered owner +or when a provider wishes to not publicly disclose ownership of a particular +endpoint. +Unknown-provider rows aggregate only on the protocol ladder; there is no +provider identity to group them by. + ## 5. Control What the same target looks like from somewhere the censor is not. @@ -149,6 +209,63 @@ nettests: cross-network baselines ("this hostname resolves fine in 40 other ASNs"), per-resolver baselines, and historical baselines. All computable from `obs_web` alone. **[later]** +### 5.1 Control for tunnel targets **[mvp]** + +Tunnel nettests have no test helper, so the cross-network baseline that §5's +closing paragraph marks **[later]** for web is pulled forward here, in its +cheapest form, because there is no cheaper alternative: the control is **the +same target measured elsewhere**. A tunnel target succeeding from k +established credentialed probes (A3) in other networks inside the window is +the control; a target failing on one network while succeeding on twelve +others is evidence at the level `failure_ctrl_ok` occupies for web. A target +failing everywhere is `down`, the provider's problem rather than the +censor's. + +This needs a precomputed aggregate (a per-target, per-window success-by-network +rollup), for the same cost reason §5 already gives for widening the web +control window: scanning raw observations for every hourly run is not +affordable. [architecture.md](architecture.md) §3.3 records it as new +standing infrastructure. + +**Provider-side liveness data is not a control.** Where a registration +interface offers "this endpoint was up as of our own check", that is a +submitter-asserted signal and does not meet A4's admission standard on its +own: providers report on their own infrastructure, which is what A4 requires +independent corroboration for. It may however narrow which networks are worth +testing. + +### 5.2 External reports **[later]** + +A circumvention provider can submit connection failure or success reports +from its own users, following an externally-defined report format +(`vpn_ext`), rather than from an OONI probe. This is a distinct evidence class, +for these reasons that follow from the requirements: + +- **E1 (facts before conclusions).** The aggregated form of an external + report is a provider-computed failure ratio, a conclusion whose underlying + measurements OONI does not see. It cannot enter the observation tier as + though it were a fact, because there is no fact behind it to recompute + from. +- **A2/A3 (independence).** An external report, per-connection or + aggregated, is **one submitter** however large its claimed sample size. + It can never contribute to the k-established-probes corroboration floor + required by claims in §8.4. +- **E2/M5 (labelled provenance).** Any surface showing external-report data + labels it as such. No OONI-verdict surface should blend the two silently. + +An external report is stored in its own tier-1 table, joins no control set, +and works at event level corrobotation not as detection input (§11). +An OONI-detected tunnel event that an independent provider report also +names is stronger; a provider report with no OONI-side signal is a lead +for targeted testing, not a finding, as it cannot be independently +corroborated. + +**Trigger to promote from [later]:** a provider agreement under the +published report format, naming what is collected and at what aggregation +(PR1: this is new exposure and needs the same discuss-first treatment as any +other privacy-relevant decision, since per-connection reports carry +`client_asn`). See Appendix A.7. + ## 6. Verdict Per layer, a `(blocked, down, ok)` triple in `[0,1]`, plus the id of the rule @@ -536,6 +653,16 @@ the same way for probes in *other* networks, which is a cross-network check the pipeline does not yet compute. Until it does, series are stable and attribution stays with the probe's network (E7). +**Tunnel series [mvp]**: `(probe_cc, probe_asn, tunnel_target, phase)`, run +through the same cell state, detector and event machinery as web series +rather than a parallel system. Endpoint rotation inside a provider's pool +must never enter the key, for the same reason a CDN's rotating addresses +never enter a web series key (§4): a target-grain key is what makes rotation +invisible to the detector instead of reading as an event. This matters more +here than for web, since a provider's own discovery system (an external +system such as GTDB) can rotate a pool on a schedule unrelated to any +network condition. + **Signal**: one of `dns_isp_blocked`, `dns_other_blocked`, `tcp_blocked`, `tls_blocked`. Each is currently the hourly **median** LoNI across the series, a statistic of a statistic that discards both spread and count. §9 replaces it. @@ -646,6 +773,10 @@ http.throttle.host [unmeasurable today] ip (internal) ip.unreachable ICMP unreachable or no route ip.prefix_null_route [needs co-affected evidence] + +tunnel (internal) [reserved] +tunnel.protocol_dpi protocol-signature blocking, any destination [reserved] +tunnel.endpoint_list known-endpoint blocklisting [reserved] ``` Three deliberate choices, each of which resolves an ambiguity a labeller would @@ -672,6 +803,19 @@ measures reachability, not throughput, so nothing can currently produce them. Reserving the path is better than minting one under deadline later; treat their absence from a corpus as "never measured", not "never happened". +**Nodes marked `[reserved]`** are a related but distinct case: the manifestation +rule already covers most tunnel blocking with existing nodes (a tunnel reset +during the TCP handshake is `tcp.reset`; a reset after a TLS-camouflaged +ClientHello is `tls.reset.sni`). What has no home is protocol-targeted +interference, where the trigger is the tunnel protocol itself rather than +the destination. Distinguishing that from endpoint-address blocking needs the +cross-endpoint contrast §3.1 describes, and no adjudicated tunnel corpus +exists yet to confirm if the split is feasible. **Trigger to +lift the reservation:** the tunnel labelling work in +[label-corpus-design.md](label-corpus-design.md) producing its first +adjudicated rows, or alignment with a related taxonomy effort (GTDB's D4 +deliverable). Nodes stay append-only regardless (X2). + ### 12.4 Label the deepest node the evidence supports The central rule, and the reason for a hierarchy rather than a flat enum. @@ -952,3 +1096,23 @@ summarises (§9.4). **Trigger:** measurement showing the view is too slow for the detector or the API at production volume. + +### A.7 Ingesting external (non-probe) reports + +The requirements' non-requirements table excludes "ingesting other +observatories' data into the pipeline", with the trigger "a formal +data-sharing agreement with aligned schemas." + +**Status: the trigger is arriving, narrower than originally scoped.** A +draft external report format (`vpn_ext`) exists for circumvention providers +to submit connection failure and success reports, and a companion project +(GTDB) proposes collecting exactly this class of data. The exclusion was +written with peer observatories in mind; it was never meant to bar a +provider-scoped, OONI-defined format under an explicit agreement, and §5.2 +specifies the treatment: a distinct evidence class, one submitter +regardless of claimed volume, corroboration-only at event grading, never a +control or detection input. + +**Trigger to promote §5.2 from [later] to [mvp]:** a signed provider +agreement under the published format, with the retention, access and +aggregation-floor decisions PR1 requires settled first. diff --git a/docs/requirements.md b/docs/requirements.md index c29846a1..a4c9332a 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -56,6 +56,7 @@ assessment regardless of its other merits. | C6 | **Errors are asymmetric.** OONI output is cited in litigation, press and policy. A false blocking claim is ammunition against OONI's credibility everywhere and forever; a missed event is invisible. Both are failures, priced differently. | | C7 | **Data arrives late, correlated with censorship.** Probes on interfered networks upload late or never, often *because of* the event being measured. Designs should not assumes prompt arrival of data. | | C8 | **Coverage is uneven and uncontrolled.** The probe population is volunteer, with orders-of-magnitude differences between countries and networks. While it's possible to [prioritize the testing](https://docs.ooni.org/backend/ooniapi/services/#prioritization) of certain targets in specific geographies, if no volunteers exist in the region the coverage will remain scarse. Building additional coverage and capacity in a region is costly. | +| C9 | **Published tool data is an enumeration oracle.** Any surface reporting per-endpoint circumvention-tool results hands a censor a tested blocklist for free. This is not a general property of OONI's output (a web target's domain is already public), but it is a property of tunnel endpoints specifically, which providers entrust to OONI precisely because they are not public. Designs for tunnel evidence must publish at the aggregate grain (provider or protocol stack, by network and country) and never expose endpoint identity, unless they are already public. | --- @@ -109,6 +110,15 @@ than discovered later. A vocabulary that can only express reach or no-reach answers "no" to "is this throttled?" when the honest answer is "the schema cannot represent the question". +**M8. Tool reachability is a first-class question.** The system answers *"does +circumvention tool T, or protocol stack P, work in location X?"* at the same +ladder grains M1 establishes for web targets, from probe-measured evidence. +Tool targets key on stable names (provider, protocol stack), never on +endpoint addresses (§4 of ontology.md, generalised). Answers about a protocol +stack aggregate across providers, which is what lets a reader ask whether a +technique works on a network independent of who deployed it. The published +grain never exposes individual endpoints (C9). + --- ## 3. Evidence and honesty @@ -450,4 +460,5 @@ omitting these; it would be criticised for building them now. | Per-measurement belief fusion (Dempster–Shafer and kin) | The rule vocabulary already carries the distinctions categorically | A measurement needing partial belief that no rule split can express | | Learned models deciding published verdicts | Violates priority 2; permitted as firewalled triage feeding human curation | Never, for the published verdict itself. The firewalled-triage form has its own trigger: a blockpage family the corpus missed for days | | Crowdsourced labelling | Ground-truth quality over volume (V1) | The curated corpus demonstrably too small for the top rules, with adjudication capacity the binding constraint | -| Ingesting other observatories' data into the pipeline | Corroboration happens at analysis and reporting time rather than at ingestion | A formal data-sharing agreement with aligned schemas | +| Ingesting *peer observatories'* data into the pipeline | Corroboration happens at analysis and reporting time rather than at ingestion | A formal data-sharing agreement with aligned schemas | +| Treating externally-submitted (non-probe) evidence as ground truth or control | It is one submitter regardless of claimed volume (A2, A3); it cannot corroborate itself | Never for control or detection input; it corroborates at event grading once ontology.md §5.2's evidence class exists |