bmp-in: record why a monitored router's BGP session went down

Changelog.md

@@ -115,6 +115,22 @@ Released yyyy-mm-dd.
   entry and so appeared nowhere. Peers observed through BMP are included in
   the same response, carrying the monitored router they were seen through.
 
+* Why a monitored router's BGP session went down. When a router reports a
+  session down with a BMP Peer Down Notification (RFC 7854 §4.9), netom now
+  records the reason instead of discarding it: the reason code, the BGP
+  NOTIFICATION the router sent or received (e.g. `Cease(MaximumPrefixesReached)`
+  or `Cease(AdministrativeShutdown)`, with its RFC 8203 shutdown
+  communication), or the FSM event that closed the session. BMP-monitored
+  peers in `/api/v1/bgp/neighbors` gain `lastError` and `lastDownTime`, and
+  their ingresses in `/api/v1/ingresses` gain a structured `last_down`. Both
+  are kept after the session comes back up. Previously a BMP peer's row only
+  said `Idle`. See `docs/bmp-tcp-in.md`.
+* `netom-cli show ip bgp neighbors` shows a BMP peer's last Peer Down as
+  `Last error` (why) and `Last down` (when, and how long ago).
+* New `bmp_state_num_peer_down_notifications` counter: Peer Down
+  Notifications per monitored router and reason (RFC 7854 §4.9), i.e. the
+  router's BGP sessions going down. See `docs/bmp-tcp-in.md`.
+
 * Native BGP sessions now record `session_up_time` in the ingress register.
   Besides giving those peers an uptime, this fixes the per-peer header of
   the Peer Up that `bmp-tcp-out` synthesizes for restreamed native

doc/netom-cli.1

@@ -230,6 +230,15 @@ The hold time reported by
 is the configured one, not the negotiated one, which the BGP library keeps
 private.
 
+For a peer observed through BMP,
+.B show ip bgp neighbors
+also shows why its session last went down and when
+.RB ( "Last error" ", " "Last down" ),
+from the monitored router's Peer Down Notification: the BGP NOTIFICATION the
+router sent (local) or received (remote), or the FSM event that closed the
+session. Routers report only sessions that came up, so a peer that never
+established is not shown.
+
 .SH PAGING
 Output is not paged. Piping a whole-table dump into a pager is inadvisable:
 the daemon aborts a dump whose reader stops draining, so a pager that stops

docs/bmp-tcp-in.md

@@ -71,6 +71,48 @@ Point a `clickhouse-out` target at the input unit names to record received
 observations before RIB mutation. See [ClickHouse export](clickhouse.md).
 Exporter-generated snapshots and EOR completeness tracking remain later work.
 
+## Why a peer's session went down
+
+When one of a monitored router's own BGP sessions goes down, the router sends
+a Peer Down Notification (RFC 7854 §4.9). netom records the reason on that
+peer and keeps it after the session comes back up, so you can still see why
+it last dropped:
+
+* `GET /api/v1/bgp/neighbors` gives the peer's row a `lastError` with a
+  one-line summary and a `lastDownTime`.
+* `GET /api/v1/ingresses` gives each view of the peer (pre-policy,
+  post-policy) a structured `last_down`.
+
+| Reason | `reason` | Carries |
+|---|---|---|
+| 1 | `localNotification` | the NOTIFICATION the router sent, e.g. `Cease(MaximumPrefixesReached)` |
+| 2 | `localFsm` | the FSM event code that made the router close the session |
+| 3 | `remoteNotification` | the NOTIFICATION the peer sent, e.g. `Cease(AdministrativeShutdown)` |
+| 4 | `remoteNoData` | nothing: the peer closed the session without a NOTIFICATION |
+| 5 | `peerDeconfigured` | nothing: the peer was removed from the router's configuration |
+| 6 | `localTlv` | (RFC 9069) the router closed the session; TLV data follows |
+
+For a Cease Administrative Shutdown or Administrative Reset NOTIFICATION, the
+shutdown communication (RFC 8203, RFC 9003) is decoded too, so a summary reads
+like `remote NOTIFICATION: Cease(AdministrativeShutdown) "maintenance"`. The
+time is the Peer Down's per-peer header timestamp, or the time netom received
+it when the router sends 0.
+
+Each Peer Down Notification is also counted, per router and reason, in the
+`bmp_state_num_peer_down_notifications` counter on `/metrics`, for example
+`{router="edge1",reason="localNotification"}`. Every reason has a series, so
+an alert on a rising `localNotification` rate catches a router tearing
+sessions down, typically for exceeded prefix limits.
+
+Two limits:
+
+* A router only reports sessions that reached Established. It sends Peer
+  Down only for a peer it sent Peer Up for, so a session that never comes up,
+  for example an OPEN rejected for a bad peer AS, never appears here. Look on
+  the router itself for those.
+* The record lives on the peer's ingress. If a peer stays down until the rib's
+  garbage collection reaps it, its record goes with it.
+
 ## Integration tests
 
 The ClickHouse test driver can act as a BMP exporter:

docs/cli.md

@@ -300,6 +300,30 @@ is waiting for a peer that has not connected. Only exactly-configured peers
 can be listed this way — a peer matched by a prefix has no single address to
 show until it connects.
 
+A peer seen through BMP is down when its router says so. The router's Peer
+Down Notification also says why, and netom keeps that after the session comes
+back:
+
+```
+netom> show ip bgp neighbors 192.0.2.8
+BGP neighbor is 192.0.2.8, remote AS 65101
+  BGP router identifier: 192.0.2.8
+  BGP state = Idle
+  Learned via: BMP feed
+  Monitored router: 10.99.0.1 (ingress 2)
+  RIB type: InPre
+  ...
+  Last error: remote NOTIFICATION: Cease(AdministrativeShutdown) "maintenance"
+  Last down: 2026-08-12T05:58:10Z (00:03:12 ago)
+```
+
+`remote` means the neighbor sent the NOTIFICATION and the router closed the
+session in response; `local` means the router sent it, e.g. `local
+NOTIFICATION: Cease(MaximumPrefixesReached)` when the neighbor exceeded a
+prefix limit. A router only reports sessions that came up, so a BMP peer that
+never established, for example over a peer AS mismatch, does not appear at
+all. See [BMP input](bmp-tcp-in.md#why-a-peers-session-went-down).
+
 ## Paging
 
 There is no built-in pager, and piping a whole-table dump into one is a bad

docs/rib-query-api.md

@@ -371,6 +371,11 @@ store walk, so it does not help a response fit under those caps.
 * `GET /api/v1/ingresses` — the peers and sessions the ids above refer to.
   Accepts `filter[type]`, `filter[state]`, `filter[ribType]`,
   `filter[peerAddress]`, `filter[peerAsn]` and `format`.
+  A BMP-monitored peer whose session has gone down carries `last_down`: why
+  and when, from the router's Peer Down Notification. See
+  [BMP input](bmp-tcp-in.md#why-a-peers-session-went-down).
 * `GET /api/v1/ingresses/{id}` — one ingress.
 * `GET /api/v1/bgp/neighbors[/{peer}]` — session state and per-peer counters,
-  merging natively terminated and BMP-monitored peers.
+  merging natively terminated and BMP-monitored peers. For BMP-monitored
+  peers, `lastError` and `lastDownTime` give the reason and time of the
+  session's last Peer Down.

scripts/e2e-addpath-bmp.py

@@ -150,9 +150,19 @@ def fs_withdraw(path_id, rule):
     return bmp_msg(0, pph() + bgp_update(pas, b""))
 
 
+SHUTDOWN_COMMUNICATION = b"e2e: maintenance"
+
+
 def peer_down():
-    # Reason 4: remote system closed without notification.
-    return bmp_msg(2, pph() + bytes([4]))
+    # Reason 3: the peer sent a NOTIFICATION, here Cease (6) / Administrative
+    # Shutdown (2) carrying an RFC 8203 shutdown communication.
+    data = bytes([len(SHUTDOWN_COMMUNICATION)]) + SHUTDOWN_COMMUNICATION
+    notification = (
+        b"\xff" * 16
+        + struct.pack("!HBBB", 21 + len(data), 3, 6, 2)
+        + data
+    )
+    return bmp_msg(2, pph() + bytes([3]) + notification)
 
 
 # --- BMP consumer-side parsing --------------------------------------------------
@@ -462,6 +472,46 @@ def main():
         )
         print(f"{context}: peer down emitted exactly once: OK")
 
+    # HTTP: why the router's session went down, from the Peer Down's
+    # NOTIFICATION, on the session ingress and in the neighbor row.
+    with urllib.request.urlopen(
+        f"http://{HTTP_ADDR}/api/v1/ingresses", timeout=10
+    ) as resp:
+        ingresses = json.load(resp)["data"]
+    sessions = [
+        e for e in ingresses if e.get("ingress_type") == "bgpViaBmp"
+    ]
+    assert len(sessions) == 1, sessions
+    last_down = sessions[0].get("last_down")
+    assert last_down, f"FAIL: no last_down on the session: {sessions[0]}"
+    assert last_down["reason"] == "remoteNotification", last_down
+    assert last_down["reason_code"] == 3, last_down
+    assert (
+        last_down["notification_code"],
+        last_down["notification_subcode"],
+    ) == (6, 2), last_down
+    assert (
+        last_down["shutdown_communication"]
+        == SHUTDOWN_COMMUNICATION.decode()
+    ), last_down
+    # The feeder's per-peer header timestamp is 0, so netom stamps it.
+    assert last_down["time"].startswith("20"), last_down
+    print("/ingresses records why the session went down: OK")
+
+    with urllib.request.urlopen(
+        f"http://{HTTP_ADDR}/api/v1/bgp/neighbors/{PEER_IP}", timeout=10
+    ) as resp:
+        neighbors = json.load(resp)["data"]
+    assert neighbors, f"FAIL: no neighbor row for {PEER_IP}"
+    row = neighbors[0]
+    assert row.get("state") == "Idle", row
+    assert row.get("lastError") == (
+        "remote NOTIFICATION: Cease(AdministrativeShutdown) "
+        f'"{SHUTDOWN_COMMUNICATION.decode()}"'
+    ), row
+    assert row.get("lastDownTime") == last_down["time"], row
+    print("/bgp/neighbors reports lastError and lastDownTime: OK")
+
     feeder.close()
     for _, reader in consumers:
         reader.sock.close()

scripts/e2e-addpath-bmp.sh

@@ -17,7 +17,10 @@
 #     path of each family (with its path id),
 #   * the /ingresses HTTP API shows the two bgpPath children with pathId and
 #     parentIngress,
-#   * PeerDown is emitted exactly once (for the session, not per child).
+#   * PeerDown is emitted exactly once (for the session, not per child),
+#   * the Peer Down's NOTIFICATION (Cease / Administrative Shutdown with a
+#     shutdown communication) is recorded as `last_down` in /ingresses and
+#     as lastError / lastDownTime in /bgp/neighbors.
 #
 # Requirements: cargo, python3. Set NETOM_BIN to skip the build.
 set -euo pipefail

src/bin/netom-cli/commands/bgp.rs

@@ -254,6 +254,15 @@ pub fn render_neighbors<W: Write>(
         if let Some(err) = n["lastError"].as_str() {
             writeln!(out, "  Last error: {err}")?;
         }
+        // BMP peers: when the router reported the session down (Peer Down
+        // Notification); lastError above says why.
+        if let Some(when) = n["lastDownTime"].as_str() {
+            writeln!(
+                out,
+                "  Last down: {when} ({} ago)",
+                super::bmp::uptime_from(&n["lastDownTime"]),
+            )?;
+        }
     }
     Ok(())
 }
@@ -1233,7 +1242,7 @@ mod tests {
         let out = render(NEIGHBORS, None);
         assert!(out.contains("10.1.0.1"));
         assert!(out.contains("192.0.2.7"));
-        assert!(out.contains("Total neighbors 4 (bgp 3, bmp 1)"), "{out}");
+        assert!(out.contains("Total neighbors 5 (bgp 3, bmp 2)"), "{out}");
     }
 
     /// The whole point of the FSM work: a configured peer that never came
@@ -1308,6 +1317,29 @@ mod tests {
         assert!(out.contains("Duplicates: 2,410,338"), "{out}");
     }
 
+    /// A BMP-monitored peer that went down says why and when, from the
+    /// router's Peer Down Notification.
+    #[test]
+    fn neighbor_detail_shows_why_a_bmp_peer_went_down() {
+        let mut buf = Vec::new();
+        render_neighbors(&mut buf, NEIGHBORS).unwrap();
+        let out = String::from_utf8(buf).unwrap();
+        let down = out
+            .split("\n\n")
+            .find(|block| block.contains("BGP neighbor is 192.0.2.8"))
+            .expect("the down BMP peer must be rendered");
+        assert!(down.contains("BGP state = Idle"), "{down}");
+        assert!(
+            down.contains(
+                "Last error: remote NOTIFICATION: \
+                 Cease(AdministrativeShutdown) \"maintenance\""
+            ),
+            "{down}"
+        );
+        assert!(down.contains("Last down: 2026-08-12T05:58:10Z ("), "{down}");
+        assert!(down.contains(" ago)"), "{down}");
+    }
+
     #[test]
     fn neighbor_detail_reports_no_match_clearly() {
         let mut buf = Vec::new();

src/bin/netom-cli/commands/bmp.rs

@@ -203,7 +203,7 @@ pub fn render_ingresses<W: Write>(
 //------------ helpers -------------------------------------------------------
 
 /// Render an RFC 3339 timestamp as an elapsed time.
-fn uptime_from(value: &serde_json::Value) -> String {
+pub(super) fn uptime_from(value: &serde_json::Value) -> String {
     let Some(text) = value.as_str() else {
         return "never".to_string();
     };