Record what the radio was doing, and turn PTT into a recording

CMakeLists.txt

@@ -38,7 +38,7 @@ FetchContent_Declare(nlohmann_json
   GIT_SHALLOW    TRUE)
 FetchContent_MakeAvailable(nlohmann_json)
 
-add_executable(hamdeck-host src/main.cpp src/log.cpp src/amp_tuner.cpp src/api.cpp src/transmit_routes.cpp src/cw_text.cpp src/http.cpp src/audio.cpp src/cat_sim.cpp src/serial_cat.cpp src/radio.cpp src/auth.cpp src/config.cpp src/tx_audio.cpp src/cat_proxy.cpp src/alsa_devices.cpp src/rig_cal.cpp src/alsa_audio.cpp src/tgxl.cpp src/recorder.cpp
+add_executable(hamdeck-host src/main.cpp src/log.cpp src/amp_tuner.cpp src/api.cpp src/transmit_routes.cpp src/cw_text.cpp src/http.cpp src/audio.cpp src/cat_sim.cpp src/serial_cat.cpp src/radio.cpp src/auth.cpp src/config.cpp src/tx_audio.cpp src/cat_proxy.cpp src/alsa_devices.cpp src/rig_cal.cpp src/alsa_audio.cpp src/tgxl.cpp src/recorder.cpp src/qso_record.cpp
     src/session_stats.cpp)
 target_link_libraries(hamdeck-host PRIVATE civetweb-c-library OpenSSL::Crypto nlohmann_json::nlohmann_json asound)
 target_compile_options(hamdeck-host PRIVATE -Wall -Wextra)
@@ -63,6 +63,9 @@ add_test(NAME audio_queue COMMAND test_audio_queue)
 add_executable(test_recorder tests/test_recorder.cpp src/recorder.cpp)
 add_test(NAME recorder COMMAND test_recorder)
 
+add_executable(test_qso_record tests/test_qso_record.cpp src/qso_record.cpp src/recorder.cpp)
+add_test(NAME qso_record COMMAND test_qso_record)
+
 add_executable(test_session_stats tests/test_session_stats.cpp src/session_stats.cpp)
 add_test(NAME session_stats COMMAND test_session_stats)
 

WIP.md

@@ -1511,3 +1511,126 @@ one thing most likely to regress — if `OUTPUT_NAME` stops applying the bundle
 
 **Not yet proven on hardware.** CI checks structure; nobody has opened the renamed bundle in
 Finder or keyed up on a Mac.
+
+---
+
+## 09/02/2026 — PTT auto-record, and provenance on every recording
+
+Started from the wrong premise ("port recording like the C# has") and measured before
+building: `src/recorder.cpp` already does WAV writing, continuous record AND the replay ring,
+wired at `main.cpp:213` and unit-tested. `AUDIT-CSHARP.md:42` had it as ✅ ported all along.
+
+**The real gap was that a recording carried no provenance.** Files were
+`hamdeck-rec-<local time>.wav` and nothing else — no frequency, no mode, and the only
+timestamp in local time while every log worth matching against is UTC. Nothing downstream
+could join on that, so "who was it" was unanswerable by construction.
+
+### 1. Every recording now writes a `.json` sidecar
+UTC start/end, rig-connected, frequency at start **and** end, mode, sample rate, and the
+operator's overs. Written for manual, replay and auto recordings alike.
+
+⚠️ **Provenance is PUSHED from the poll loop, not pulled.** The first cut had the recorder
+call back into `RadioPoller` for the current frequency — that takes the poller's lock while
+holding the recorder's, from the poll thread, waiting on the one API path that nests them the
+other way. `RadioPoller::OnPoll` hands the values over instead; `Recorder::UpdateProvenance`
+keeps them under its own small lock. One fan-out point, no second poller competing for CAT.
+
+⚠️ **A replay clip's `started_utc` is in the PAST**, derived from the sample count. The ring
+holds what happened *before* the press; stamping it "now" files the audio minutes after the
+exchange it contains and matches it to the wrong QSO.
+
+⚠️ **`"overs": null` ≠ `"overs": []`.** Null means not tracked (a replay clip). An empty array
+would claim the operator never transmitted. Different facts, and the second one is a lie.
+
+### 2. `src/qso_record.cpp` — PTT auto-record, ported from the C# behaviour
+`Views/MainWindow.xaml.cs:420`, not invented: start on the PTT rising edge, every later over
+pushes an idle deadline out, stop on idle (`ptt_record_seconds`, 60) or a QSY past
+`ptt_record_qsy_khz` (10) **from where the QSO started** — a reading-to-reading comparison
+never trips on someone tuning across the band in small steps.
+
+⚠️ **It must not start on a tune.** Keying the tuner is PTT to the rig; without the guard every
+band change litters the directory with two-second files no log will ever match. A tune also
+must not move the remembered PTT state, or its unkey closes an over that never opened.
+
+⚠️ **OFF by default** (`ptt_record_enabled`). It records whoever the operator is talking to,
+unasked. That is a decision made once in the config, not a default that arrives with an update.
+
+### The gates — `tests/test_qso_record.cpp`, PROVEN to fail
+| reintroduced | caught by |
+|---|---|
+| `NoteOver` before the file opens | `overs == 2` — the over that STARTS each QSO went unlisted while every later one was recorded |
+| `localtime_r` for the sidecar | the local-vs-UTC hour comparison (see below) |
+| replay stamped when saved | `age >= 19 && age < 60` |
+| auto-record starting on a tune | `!q.active()` |
+
+⚠️⚠️ **THE UTC TEST WAS BLIND AND PASSED ANYWAY.** The build box runs UTC, so `localtime_r`
+and `gmtime_r` return the same thing and the assertion held just as happily against a
+local-time stamp. The test now sets `TZ` to a **POSIX string** (`CST6CDT,M3.2.0/2,M11.1.0/2`,
+which glibc parses with no tzdata, so a bare container cannot silently drop it back to UTC),
+asserts the two clocks actually differ before testing anything, and compares the filename's
+local hour against the sidecar's UTC hour. Only then did swapping in `localtime_r` fail.
+
+### Proven on the built host, not only in tests
+Ran `hamdeck-host` against the simulated rig with `ptt_record_seconds: 3`, keyed via
+`/api/ptt/on`, unkeyed, waited out the idle timer:
+`hamdeck-qso-09-02-2026-135540.wav` — 3.0 s, 1ch/16bit/22050, **rms 5656 peak 7997** (real
+audio, not an empty header) — beside a sidecar carrying `trigger: idle`, both UTC times,
+14074000 Hz, USB, and one closed over.
+
+⚠️ **`pkill -f hamdeck-host` KILLS THE SHELL THAT RUNS IT** — the pattern matches the
+command line of the very shell issuing it. Cost a lost commit. Use `pkill -x hamdeck-host`.
+
+### Next, in order
+1. **Re-measure MONI.** `CARRYOVER.md:207` says it cannot be captured (120 s of `/ws`, band
+   noise only); the C# says the opposite at `WsAudioClient.cs:205` and mutes RX while keyed
+   *because* the operator hears themselves. `RadioController.cs:687` is the likely
+   explanation — MON needs **`ML0001;` to enable AND `ML1<level>;` to set the level**, and
+   enable-at-level-0 is on and silent, which reads exactly like "no transmission in it".
+   Gate: recorded RMS during a keyed window vs the same window unkeyed, into a dummy load.
+   ⚠️ It matters beyond convenience: recording the host's own `/ws/tx` PCM would have looked
+   perfect through every one of the six TX-chain bugs. MONI is the only source that proves
+   audio actually left the radio.
+2. **Identification, layered** — Wavelog QSO in the window and band ⇒ the callsign, stated as
+   fact; otherwise the NetLogger roster for whatever net was up ⇒ **candidates**, stated as
+   candidates. Never the same kind of claim, and never "nobody" as a finding.
+   ⚠️ **NetLogger has a hard 7-day wall and no bulk history endpoint.** Past net rosters
+   cannot be fetched retroactively — the only source is `netlogger_poll.py`'s
+   `netlogger_checkin` table, and only for the period it has actually been polling. Anything
+   older is unattributable from the net side, permanently.
+
+### Identification, layer 1 — `tools/identify_recording.py` (09/02/2026)
+
+Takes a recording's sidecar and answers "who was that" from the log. Two layers that are
+deliberately **not** the same kind of claim: **LOGGED** is a fact (the operator wrote the
+callsign down); **ON THE NET** is a list of **candidates** (a check-in says present, not that
+they were the voice on the tape).
+
+⚠️ **The net name is ALREADY in the log** and needs no API at all. NetLogger-sourced QSOs carry
+it in `COL_COMMENT`, in the two encodings qsl-queue's README measured over 29,573 rows —
+`OMISS 40m SSB Net` and `MT/HI [OMISS 40m SSB Net]`. So layer 1 works **retroactively over the
+whole log**, unlike anything that depends on the NetLogger API. Bracket text is not always a
+net (`[New call sign May 2025]` is in there), so a bracketed token is only taken as one when it
+is net-shaped — verified against that exact row.
+
+⚠️ **Matching is on TIME ALONE, and the band is shown so a wrong match is visible.** Filtering
+by band would silently drop true matches whenever the sidecar's frequency is unreliable (rig
+disconnected, or a QSY between the exchange and the log entry), and a dropped true match is
+invisible in a way a flagged odd one is not. A QSO on another band in the same window is marked
+`⚠️ DIFFERENT BAND`, never hidden. Proven by re-running a real window with the band claim
+changed and watching both rows flag.
+
+⚠️ **A logged QSO is an INSTANT, not a span** — `COL_TIME_OFF == COL_TIME_ON` on every row, so
+the timestamp is when it was *logged*, usually the end of the exchange. Hence `--pad` (120 s
+default) and hence each match prints its offset into the recording, so an edge match reads as
+one instead of a bullseye.
+
+⚠️ **"Nothing found" is never printed as "nobody".** An unlogged QSO and a station heard but
+not worked look identical to this tool. A false negative stated as a finding is worse than no
+answer.
+
+Measured against the real log, not a fixture: a 26-minute window over the 07/26 80m net
+returned **N4GTO / N4TTU / W4ETA**, each with its offset into the recording and the net name
+parsed from the comment.
+
+⚠️ **Connection details come from the environment** (`WAVELOG_DB_*`, or `WAVELOG_DB_DOCKER`),
+never from the file — same rule as the rest of this repo.

src/config.cpp

@@ -88,6 +88,9 @@ bool Config::Load(const std::string& path, Config& out, std::string& error) {
     Get(j, "record_buffer_seconds", cfg.record_buffer_seconds);
     Get(j, "record_max_seconds", cfg.record_max_seconds);
     Get(j, "record_warning_seconds", cfg.record_warning_seconds);
+    Get(j, "ptt_record_enabled", cfg.ptt_record_enabled);
+    Get(j, "ptt_record_seconds", cfg.ptt_record_seconds);
+    Get(j, "ptt_record_qsy_khz", cfg.ptt_record_qsy_khz);
     Get(j, "api_port", cfg.api_port);
     Get(j, "cat_proxy_port", cfg.cat_proxy_port);
     Get(j, "dashboard_port", cfg.dashboard_port);
@@ -173,6 +176,9 @@ bool Config::Save(const std::string& path, std::string& error) const {
   j["record_path"] = record_path;
   j["record_buffer_seconds"] = record_buffer_seconds;
   j["record_max_seconds"] = record_max_seconds;
+  j["ptt_record_enabled"] = ptt_record_enabled;
+  j["ptt_record_seconds"] = ptt_record_seconds;
+  j["ptt_record_qsy_khz"] = ptt_record_qsy_khz;
   j["alsa_capture_device"] = alsa_capture_device;
   j["alsa_playback_device"] = alsa_playback_device;
   j["api_port"] = api_port;

src/config.h

@@ -59,6 +59,13 @@ struct Config {
   int record_max_seconds = 10800;    // hard ceiling; 0 disables the ceiling
   int record_warning_seconds = 300;
 
+  // PTT auto-record. ⚠️ OFF unless the operator turns it on: it records whoever
+  // they are talking to, unasked, and that is their call to make once rather
+  // than a default that arrives with an update.
+  bool ptt_record_enabled = false;
+  int ptt_record_seconds = 60;        // idle time after the last over
+  int ptt_record_qsy_khz = 10;        // QSY from the start freq that ends it
+
   // API
   int  api_port = 5001;               // control, bound to loopback
 

src/main.cpp

@@ -22,6 +22,7 @@
 #include "auth.h"
 #include "alsa_audio.h"
 #include "cat_sim.h"
+#include "qso_record.h"
 #include "recorder.h"
 #include "session_stats.h"
 #include "tgxl.h"
@@ -219,6 +220,7 @@ int main(int argc, char** argv) {
     rx_audio.SetRecorder(&recorder);
   }
 
+
   rx_audio.Start();
 
   // TX audio. The null sink discards: the codec is on the reference host, so
@@ -274,6 +276,23 @@ int main(int argc, char** argv) {
   AmpTuner amp(tgxl_rig);
   if (tgxl.configured()) std::cout << "TGXL: " << tgxl.Describe() << '\n' << std::flush;
 
+  // ⚠️ FED FROM THE POLL LOOP, NOT ITS OWN TIMER. QsoRecorder is what turns PTT
+  // into a recording, and it also hands the recorder the frequency and mode
+  // that go in every sidecar - so it is wired up even when auto-record is off.
+  // The tuner check is here rather than inside it because the amp and the TGXL
+  // are what know a tune is running, and keying for a tune is PTT to the rig.
+  QsoRecorder qso_record(&recorder, QsoRecorder::Options{
+      config.ptt_record_enabled, config.ptt_record_seconds,
+      static_cast<long long>(config.ptt_record_qsy_khz) * 1000});
+  poller.OnPoll([&](bool connected, long long freq, const std::string& mode, bool tx) {
+    const bool tuning = amp.IsActive() || tgxl.IsActive();
+    qso_record.Observe(connected, freq, mode, tx, tuning);
+  });
+  if (config.ptt_record_enabled && recorder.available()) {
+    std::cout << "ptt auto-record: on (" << config.ptt_record_seconds
+              << "s idle, " << config.ptt_record_qsy_khz << " kHz QSY)\n" << std::flush;
+  }
+
   HostState host_state;
 
   ApiDeps deps;

src/qso_record.cpp

@@ -0,0 +1,69 @@
+#include "qso_record.h"
+
+#include "recorder.h"
+
+QsoRecorder::QsoRecorder(Recorder* rec, Options opts,
+                         std::function<Clock::time_point()> now)
+    : rec_(rec), opts_(opts), now_(std::move(now)) {}
+
+void QsoRecorder::Observe(bool connected, long long freq_hz,
+                          const std::string& mode, bool tx, bool tuning) {
+  if (!rec_) return;
+
+  // The poll loop is the only place that knows all of this at once, so it is
+  // also where the sidecar's frequency comes from - fed always, so a MANUAL
+  // recording gets provenance too.
+  rec_->UpdateProvenance(connected, freq_hz, mode);
+
+  const bool rising = tx && !last_tx_;
+  const bool falling = !tx && last_tx_;
+  // ⚠️ A tune does not move the remembered PTT state at all (the C# does the
+  // same at MainWindow.xaml.cs:438). Letting it would mean the tune's unkey
+  // registers as the end of an over the operator never started.
+  if (!tuning) last_tx_ = tx;
+
+  if (opts_.enabled && rising && !tuning) {
+    if (!active_) {
+      // ⚠️ active_ is set from whether the file OPENED, never from having
+      // decided to record - the same rule the Recorder itself follows. A full
+      // disk must not leave a state machine believing it is recording.
+      const auto r = rec_->Start("qso");
+      if (!r.ok) return;
+      active_ = true;
+      start_freq_ = freq_hz;
+    }
+    deadline_ = now_() + std::chrono::seconds(opts_.idle_seconds);
+  }
+
+  // ⚠️ AFTER the start, not before. Noting the over first drops the very over
+  // that began the recording: NoteOver does nothing when no file is open, so
+  // the first transmission of every QSO went unlisted while every later one
+  // was recorded. Caught by the test asserting two overs, not by reading it.
+  if (!tuning && (rising || falling)) rec_->NoteOver(tx);
+
+  if (!active_) return;
+
+  if (now_() > deadline_) {
+    Stop("idle");
+    return;
+  }
+  // ⚠️ QSY is measured from where the QSO STARTED, not from the last reading.
+  // Tuning across the band in small steps would never trip a
+  // reading-to-reading comparison, and the recording would run until the idle
+  // timer caught it - filed under a frequency the operator left long ago.
+  if (start_freq_ > 0 && connected) {
+    const long long moved = freq_hz > start_freq_ ? freq_hz - start_freq_
+                                                  : start_freq_ - freq_hz;
+    if (moved > opts_.qsy_threshold_hz) Stop("qsy");
+  }
+}
+
+void QsoRecorder::Stop(const std::string& reason) {
+  // Stop() writes the sidecar with this reason in it, so a recording says how
+  // it ended rather than only when.
+  rec_->Stop(reason);
+  active_ = false;
+  start_freq_ = 0;
+  last_stop_ = reason;
+  ++stopped_;
+}

src/qso_record.h

@@ -0,0 +1,65 @@
+#pragma once
+
+// PTT auto-record: a recording that brackets a QSO without being asked for.
+//
+// Ported from the C# panel (Views/MainWindow.xaml.cs:420) rather than invented,
+// because the behaviour is the operator's habit and not a design question:
+//   - the first time PTT goes down, start recording
+//   - every later press pushes an idle deadline out
+//   - stop when the operator has been quiet for idle_seconds, or has QSY'd
+//     further than qsy_threshold_hz from where the QSO started
+//
+// ⚠️ IT MUST NOT START ON A TUNE. Keying an antenna tuner is PTT as far as the
+// rig is concerned, and a tune at the top of every band change would litter the
+// directory with two-second files that no log will ever match.
+//
+// ⚠️ OFF BY DEFAULT. This writes audio of whoever the operator is talking to,
+// unasked. That is a decision for the operator to make once, in the config, not
+// something a version bump turns on for them.
+
+#include <chrono>
+#include <functional>
+#include <string>
+
+class Recorder;
+
+class QsoRecorder {
+ public:
+  using Clock = std::chrono::steady_clock;
+
+  struct Options {
+    bool enabled = false;
+    int idle_seconds = 60;         // C# PTTRecordSeconds default
+    long long qsy_threshold_hz = 10000;  // C# PTTQSYThresholdKHz default, 10 kHz
+  };
+
+  // The clock is injectable so the idle timeout can be tested without waiting
+  // a minute for it - a test that sleeps for the real timeout gets deleted or
+  // shortened until it no longer tests the thing.
+  QsoRecorder(Recorder* rec, Options opts,
+              std::function<Clock::time_point()> now = [] { return Clock::now(); });
+
+  // Fed once per poll cycle, from the same place SessionStats is fed.
+  void Observe(bool connected, long long freq_hz, const std::string& mode,
+               bool tx, bool tuning);
+
+  bool active() const { return active_; }
+  // Why the last automatic recording stopped: "idle", "qsy", or empty if none
+  // has. Reported so the operator can tell a finished QSO from a cut-off one.
+  std::string last_stop_reason() const { return last_stop_; }
+  int stopped_count() const { return stopped_; }
+
+ private:
+  void Stop(const std::string& reason);
+
+  Recorder* rec_;
+  Options opts_;
+  std::function<Clock::time_point()> now_;
+
+  bool active_ = false;
+  bool last_tx_ = false;
+  long long start_freq_ = 0;
+  Clock::time_point deadline_{};
+  std::string last_stop_;
+  int stopped_ = 0;
+};

src/radio.cpp

@@ -120,6 +120,7 @@ void RadioPoller::PollOnce() {
   auto id = cat_->Exchange("ID;");
   if (!id.has_value()) {
     if (stats_) stats_->Observe(false, 0, "", false);
+    if (poll_cb_) poll_cb_(false, 0, "", false);
     std::lock_guard<std::mutex> lock(mu_);
     snap_ = s;                                  // connected = false
     snap_.taken = std::chrono::steady_clock::now();
@@ -159,6 +160,7 @@ void RadioPoller::PollOnce() {
 
   s.taken = std::chrono::steady_clock::now();
   if (stats_) stats_->Observe(true, s.freq, s.mode, s.tx);
+  if (poll_cb_) poll_cb_(true, s.freq, s.mode, s.tx);
   CheckWatchdog(s.tx);
   std::lock_guard<std::mutex> lock(mu_);
   snap_ = s;

src/radio.h

@@ -105,6 +105,16 @@ class RadioPoller {
   // the counts follow the RADIO rather than any one client.
   void SetSessionStats(SessionStats* stats) { stats_ = stats; }
 
+  // ⚠️ ONE FAN-OUT POINT, NOT A SECOND POLLER. Everything that needs to react
+  // to what the rig is doing - session stats, PTT auto-record - is fed from the
+  // one cycle that already asked. A feature that polls the rig on its own
+  // schedule competes for the CAT port with the loop that keeps the panel live.
+  // Called on the poll thread with no lock held; keep the callback short.
+  void OnPoll(std::function<void(bool connected, long long freq_hz,
+                                 const std::string& mode, bool tx)> cb) {
+    poll_cb_ = std::move(cb);
+  }
+
   void SetPttTimeoutSeconds(int seconds) { ptt_timeout_s_.store(seconds); }
   int  PttTimeoutSeconds() const { return ptt_timeout_s_.load(); }
 
@@ -168,6 +178,7 @@ class RadioPoller {
 
   std::atomic<int> ptt_timeout_s_{kDefaultPttTimeoutSeconds};
   SessionStats* stats_ = nullptr;
+  std::function<void(bool, long long, const std::string&, bool)> poll_cb_;
   std::atomic<int> watchdog_trips_{0};
   std::function<void(double)> watchdog_cb_;
   std::chrono::steady_clock::time_point keyed_since_{};