From 9ebae21ddf7a8a96c388d076a52a2d9b1827fa79 Mon Sep 17 00:00:00 2001 From: Alex Typaldos Date: Thu, 1 Oct 2026 15:56:27 -0400 Subject: [PATCH 1/4] bmp-in: record why a monitored router's BGP session went down MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A router sends a Peer Down Notification (RFC 7854 §4.9) when one of its own BGP sessions goes down. Besides the per-peer header, it carries a reason code and, depending on it, the BGP NOTIFICATION the router sent (reason 1) or received (reason 3), or the FSM event that closed the session (reason 2). peer_down() read only the per-peer header and threw the rest away, so a BMP-monitored peer could only ever be shown as Idle: a max-prefix teardown, an operator shutdown and a hold timer expiry all looked the same. Decode the message into a PeerDownInfo (time, reason, NOTIFICATION code and subcode, RFC 8203/9003 shutdown communication, FSM event, and a one-line description in the same Debug wording as bgp-tcp-in's last_error) and store it as `last_down` on every view of the peer while marking it Disconnected. The record survives the next PeerUp, because update_info only merges fields that are set; a synthesized view does not inherit it from the view it was copied from. /api/v1/ingresses shows the structured record; /api/v1/bgp/neighbors fills `lastError` and a new `lastDownTime` for BMP-monitored peers. The reason code is read from the message itself because routecore's PeerDownReason has no numeric value and folds RFC 9069's code 6 into Unknown, and NOTIFICATION data is bounded by the NOTIFICATION's length because routecore's data() runs to the end of the BMP message. Limits: routers only report sessions that reached Established, so a session that never comes up (e.g. bad peer AS) is not covered, and the record goes away with the ingress when the rib GC reaps a peer that never returns. Test: decoding of every reason, shutdown communication edge cases, the record on every view, kept across PeerUp, not inherited by synthesized views; full library suite 357 passed, 31 ignored. Signed-off-by: Alex Typaldos Co-Authored-By: Claude Opus 5.5 --- Changelog.md | 11 + docs/bmp-tcp-in.md | 36 +++ docs/rib-query-api.md | 7 +- src/ingress/register.rs | 192 ++++++++++++- src/tests/util.rs | 31 ++- src/units/bgp_tcp_in/http_ng.rs | 21 ++ src/units/bmp_tcp_in/state_machine/machine.rs | 28 +- src/units/bmp_tcp_in/state_machine/mod.rs | 1 + .../bmp_tcp_in/state_machine/peer_down.rs | 259 ++++++++++++++++++ src/units/bmp_tcp_in/state_machine/tests.rs | 203 ++++++++++++++ 10 files changed, 782 insertions(+), 7 deletions(-) create mode 100644 src/units/bmp_tcp_in/state_machine/peer_down.rs diff --git a/Changelog.md b/Changelog.md index 1e53192..b702a4e 100644 --- a/Changelog.md +++ b/Changelog.md @@ -115,6 +115,17 @@ Released yyyy-mm-dd. entry and so appeared nowhere. Peers observed through BMP are included in the same response, carrying the monitored router they were seen through. +* Why a monitored router's BGP session went down. When a router reports a + session down with a BMP Peer Down Notification (RFC 7854 §4.9), netom now + records the reason instead of discarding it: the reason code, the BGP + NOTIFICATION the router sent or received (e.g. `Cease(MaximumPrefixesReached)` + or `Cease(AdministrativeShutdown)`, with its RFC 8203 shutdown + communication), or the FSM event that closed the session. BMP-monitored + peers in `/api/v1/bgp/neighbors` gain `lastError` and `lastDownTime`, and + their ingresses in `/api/v1/ingresses` gain a structured `last_down`. Both + are kept after the session comes back up. Previously a BMP peer's row only + said `Idle`. See `docs/bmp-tcp-in.md`. + * Native BGP sessions now record `session_up_time` in the ingress register. Besides giving those peers an uptime, this fixes the per-peer header of the Peer Up that `bmp-tcp-out` synthesizes for restreamed native diff --git a/docs/bmp-tcp-in.md b/docs/bmp-tcp-in.md index b7d7607..0bbd948 100644 --- a/docs/bmp-tcp-in.md +++ b/docs/bmp-tcp-in.md @@ -71,6 +71,42 @@ Point a `clickhouse-out` target at the input unit names to record received observations before RIB mutation. See [ClickHouse export](clickhouse.md). Exporter-generated snapshots and EOR completeness tracking remain later work. +## Why a peer's session went down + +When one of a monitored router's own BGP sessions goes down, the router sends +a Peer Down Notification (RFC 7854 §4.9). netom records the reason on that +peer and keeps it after the session comes back up, so you can still see why +it last dropped: + +* `GET /api/v1/bgp/neighbors` gives the peer's row a `lastError` with a + one-line summary and a `lastDownTime`. +* `GET /api/v1/ingresses` gives each view of the peer (pre-policy, + post-policy) a structured `last_down`. + +| Reason | `reason` | Carries | +|---|---|---| +| 1 | `localNotification` | the NOTIFICATION the router sent, e.g. `Cease(MaximumPrefixesReached)` | +| 2 | `localFsm` | the FSM event code that made the router close the session | +| 3 | `remoteNotification` | the NOTIFICATION the peer sent, e.g. `Cease(AdministrativeShutdown)` | +| 4 | `remoteNoData` | nothing: the peer closed the session without a NOTIFICATION | +| 5 | `peerDeconfigured` | nothing: the peer was removed from the router's configuration | +| 6 | `localTlv` | (RFC 9069) the router closed the session; TLV data follows | + +For a Cease Administrative Shutdown or Administrative Reset NOTIFICATION, the +shutdown communication (RFC 8203, RFC 9003) is decoded too, so a summary reads +like `remote NOTIFICATION: Cease(AdministrativeShutdown) "maintenance"`. The +time is the Peer Down's per-peer header timestamp, or the time netom received +it when the router sends 0. + +Two limits: + +* A router only reports sessions that reached Established. It sends Peer + Down only for a peer it sent Peer Up for, so a session that never comes up, + for example an OPEN rejected for a bad peer AS, never appears here. Look on + the router itself for those. +* The record lives on the peer's ingress. If a peer stays down until the rib's + garbage collection reaps it, its record goes with it. + ## Integration tests The ClickHouse test driver can act as a BMP exporter: diff --git a/docs/rib-query-api.md b/docs/rib-query-api.md index 1904c30..ed2e22d 100644 --- a/docs/rib-query-api.md +++ b/docs/rib-query-api.md @@ -371,6 +371,11 @@ store walk, so it does not help a response fit under those caps. * `GET /api/v1/ingresses` — the peers and sessions the ids above refer to. Accepts `filter[type]`, `filter[state]`, `filter[ribType]`, `filter[peerAddress]`, `filter[peerAsn]` and `format`. + A BMP-monitored peer whose session has gone down carries `last_down`: why + and when, from the router's Peer Down Notification. See + [BMP input](bmp-tcp-in.md#why-a-peers-session-went-down). * `GET /api/v1/ingresses/{id}` — one ingress. * `GET /api/v1/bgp/neighbors[/{peer}]` — session state and per-peer counters, - merging natively terminated and BMP-monitored peers. + merging natively terminated and BMP-monitored peers. For BMP-monitored + peers, `lastError` and `lastDownTime` give the reason and time of the + session's last Peer Down. diff --git a/src/ingress/register.rs b/src/ingress/register.rs index edd0f0c..beb990d 100644 --- a/src/ingress/register.rs +++ b/src/ingress/register.rs @@ -647,11 +647,30 @@ impl Register { /// Mark an existing ingress down without resurrecting a GC'd child /// that remains in an input handler's cache until its next prune. pub fn mark_disconnected(&self, id: IngressId) -> bool { + self.mark_disconnected_with(id, None) + } + + /// Like [`Register::mark_disconnected`], also recording why the session + /// went down when it is known (a BMP Peer Down Notification). + /// + /// The record is set under the same lock as the state change, and bumps + /// `history_revision` once, so exporters keyed on the revision pick up + /// both together. It is deliberately left in place when the peer comes + /// back up: `update_info` only merges fields that are set, and a PeerUp + /// never sets `last_down`. + pub fn mark_disconnected_with( + &self, + id: IngressId, + last_down: Option, + ) -> bool { let mut lock = self.info.write().unwrap(); let Some(info) = lock.get_mut(&id) else { return false; }; info.state = Some(IngressState::Disconnected); + if last_down.is_some() { + info.last_down = last_down; + } info.history_revision = info.history_revision.saturating_add(1); true } @@ -983,9 +1002,115 @@ info_for_field!(IngressInfo{ // SessionConfig::enabled_addpaths() (the negotiated set), not from // remote_capabilities (which is one side's OPEN, not the intersection). #[serde(serialize_with = "serialize_addpath_families")] - addpath_families: Vec + addpath_families: Vec, + // Why this BGP session last went down, from the monitored router's BMP + // Peer Down Notification. Kept across reconnects; see PeerDownInfo. + last_down: PeerDownInfo }); +/// Why a monitored router's BGP session went down, as reported by the +/// router in a BMP Peer Down Notification (RFC 7854 §4.9). +/// +/// This is about the BGP session between the monitored router and its peer, +/// not the BMP session to netom. Only sessions that reached Established are +/// ever reported: a router sends Peer Down only for a peer it sent Peer Up +/// for, so a session that fails to come up (e.g. an OPEN rejected for a bad +/// peer AS) never shows up here. +#[serde_with::skip_serializing_none] +#[derive(Clone, Debug, Eq, PartialEq, Ord, PartialOrd, serde::Serialize)] +pub struct PeerDownInfo { + /// When the session went down: the Peer Down per-peer header timestamp, + /// or the time netom received the message if the router sent 0. + pub time: DateTime, + pub reason: PeerDownReason, + /// The raw Peer Down reason code, also for codes netom has no name for. + pub reason_code: u8, + /// Error code and subcode of the BGP NOTIFICATION the router sent + /// (reason 1) or received (reason 3). + pub notification_code: Option, + pub notification_subcode: Option, + /// The RFC 8203/9003 shutdown communication carried by a Cease + /// Administrative Shutdown or Administrative Reset NOTIFICATION. + pub shutdown_communication: Option, + /// The BGP FSM event code that closed the session (reason 2). + pub fsm_event: Option, + /// A one-line human-readable summary, e.g. + /// `remote NOTIFICATION: Cease(AdministrativeShutdown)`. + pub description: String, +} + +/// The reason code of a BMP Peer Down Notification (RFC 7854 §4.9, code 6 +/// from RFC 9069). +#[derive( + Clone, Copy, Debug, Eq, Hash, PartialEq, Ord, PartialOrd, serde::Serialize, +)] +#[serde(rename_all = "camelCase")] +pub enum PeerDownReason { + /// 0: reserved. + Reserved, + /// 1: the router closed the session and sent a NOTIFICATION. + LocalNotification, + /// 2: the router closed the session without a NOTIFICATION; an FSM + /// event code follows. + LocalFsm, + /// 3: the peer closed the session and sent a NOTIFICATION. + RemoteNotification, + /// 4: the peer closed the session without a NOTIFICATION. + RemoteNoData, + /// 5: the peer was de-configured; information for it stops. + PeerDeconfigured, + /// 6: the router closed the session, with TLV data (RFC 9069, Loc-RIB). + LocalTlv, + /// Any other code. + Unknown, +} + +impl PeerDownReason { + /// All variants, in reason code order. + pub const ALL: [PeerDownReason; 8] = [ + Self::Reserved, + Self::LocalNotification, + Self::LocalFsm, + Self::RemoteNotification, + Self::RemoteNoData, + Self::PeerDeconfigured, + Self::LocalTlv, + Self::Unknown, + ]; + + pub fn from_code(code: u8) -> Self { + match code { + 0 => Self::Reserved, + 1 => Self::LocalNotification, + 2 => Self::LocalFsm, + 3 => Self::RemoteNotification, + 4 => Self::RemoteNoData, + 5 => Self::PeerDeconfigured, + 6 => Self::LocalTlv, + _ => Self::Unknown, + } + } + + /// Position in [`PeerDownReason::ALL`], for per-reason counters. + pub fn index(self) -> usize { + self as usize + } + + /// The name used in JSON output and metric labels. + pub fn as_str(self) -> &'static str { + match self { + Self::Reserved => "reserved", + Self::LocalNotification => "localNotification", + Self::LocalFsm => "localFsm", + Self::RemoteNotification => "remoteNotification", + Self::RemoteNoData => "remoteNoData", + Self::PeerDeconfigured => "peerDeconfigured", + Self::LocalTlv => "localTlv", + Self::Unknown => "unknown", + } + } +} + /// Serialize a raw BGP capability blob (`capabilities_as_vec` wire format) as /// a human-readable list of capability names, e.g. /// `["MultiProtocol","FourOctetAsn","AddPath"]`, for the `/ingresses` dump. @@ -1124,6 +1249,71 @@ mod tests { assert_eq!(res.get(id).unwrap().rib_type, Some(RibType::LocRib)); } + #[test] + fn mark_disconnected_with_records_and_keeps_the_peer_down() { + let register = Register::new(); + let id = register.register(); + register.update_info( + id, + IngressInfo::new().with_state(IngressState::Connected), + ); + + let down = PeerDownInfo { + time: DateTime::from_timestamp(1_700_000_000, 0).unwrap(), + reason: PeerDownReason::RemoteNotification, + reason_code: 3, + notification_code: Some(6), + notification_subcode: Some(2), + shutdown_communication: Some("maintenance".into()), + fsm_event: None, + description: "remote NOTIFICATION: \ + Cease(AdministrativeShutdown) \"maintenance\"" + .into(), + }; + assert!(register.mark_disconnected_with(id, Some(down.clone()))); + let info = register.get(id).unwrap(); + assert_eq!(info.state, Some(IngressState::Disconnected)); + assert_eq!(info.last_down.as_ref(), Some(&down)); + + // A later disconnect without a reason (e.g. the whole BMP session + // closing) leaves the last known reason alone. + assert!(register.mark_disconnected(id)); + assert_eq!(register.get(id).unwrap().last_down, Some(down)); + + // /api/v1/ingresses: snake_case keys like the rest of IngressInfo, + // absent fields omitted. + let json = + serde_json::to_value(IdAndInfo::from((id, &info))).unwrap(); + assert_eq!( + json["last_down"], + serde_json::json!({ + "time": "2023-11-14T22:13:20Z", + "reason": "remoteNotification", + "reason_code": 3, + "notification_code": 6, + "notification_subcode": 2, + "shutdown_communication": "maintenance", + "description": "remote NOTIFICATION: \ + Cease(AdministrativeShutdown) \"maintenance\"", + }) + ); + } + + #[test] + fn peer_down_reason_names_match_their_codes() { + for (code, reason) in PeerDownReason::ALL.iter().enumerate() { + assert_eq!(reason.index(), code); + if code <= 6 { + assert_eq!(PeerDownReason::from_code(code as u8), *reason); + } + assert_eq!( + serde_json::to_value(reason).unwrap(), + serde_json::json!(reason.as_str()) + ); + } + assert_eq!(PeerDownReason::from_code(200), PeerDownReason::Unknown); + } + #[test] fn cloned_info_for_takes_the_ids_and_their_parents() { let register = Register::new(); diff --git a/src/tests/util.rs b/src/tests/util.rs index 7d1ab8e..93ea0d1 100644 --- a/src/tests/util.rs +++ b/src/tests/util.rs @@ -922,6 +922,18 @@ pub mod bgp { pub fn mk_peer_down_notification_msg( per_peer_header: &PerPeerHeader, + ) -> Bytes { + mk_peer_down_notification_msg_with(per_peer_header, 5, &[]) + } + + /// A Peer Down Notification with the given reason code and data: + /// a BGP NOTIFICATION for reasons 1 and 3 (see + /// [`mk_bgp_notification_msg`]), a 2-byte FSM event code for + /// reason 2, nothing for the others. + pub fn mk_peer_down_notification_msg_with( + per_peer_header: &PerPeerHeader, + reason: u8, + data: &[u8], ) -> Bytes { let mut buf = BytesMut::new(); push_bmp_common_header( @@ -943,12 +955,29 @@ pub mod bgp { // // From: https://www.rfc-editor.org/rfc/rfc7854.html#section-4.9 - buf.extend_from_slice(&5u8.to_be_bytes()); // reason code 5 + buf.extend_from_slice(&[reason]); + buf.extend_from_slice(data); finalize_bmp_msg_len(&mut buf); buf.freeze() } + /// A BGP NOTIFICATION message (RFC 4271 §4.5) with the given error + /// code, subcode and data. + pub fn mk_bgp_notification_msg( + code: u8, + subcode: u8, + data: &[u8], + ) -> Bytes { + let mut buf = BytesMut::new(); + buf.extend_from_slice(&[0xFFu8; 16]); // marker + buf.extend_from_slice(&[0, 0]); // length, finalized below + buf.extend_from_slice(&[3, code, subcode]); // type 3: NOTIFICATION + buf.extend_from_slice(data); + finalize_bgp_msg_len(&mut buf); + buf.freeze() + } + pub fn mk_statistics_report_msg( per_peer_header: &PerPeerHeader, ) -> Bytes { diff --git a/src/units/bgp_tcp_in/http_ng.rs b/src/units/bgp_tcp_in/http_ng.rs index f34fdd3..7ab219c 100644 --- a/src/units/bgp_tcp_in/http_ng.rs +++ b/src/units/bgp_tcp_in/http_ng.rs @@ -12,6 +12,8 @@ use std::net::IpAddr; +use chrono::{DateTime, Utc}; + use axum::{ extract::{Path, State}, response::IntoResponse, @@ -116,8 +118,22 @@ pub struct Neighbor { #[serde(skip_serializing_if = "Option::is_none")] pub peer_rib_type: Option, + /// Why the session last failed or went down. For native sessions, the + /// last NOTIFICATION or connection error. For BMP-monitored peers, the + /// reason from the router's last Peer Down Notification (RFC 7854 + /// §4.9), e.g. `remote NOTIFICATION: Cease(AdministrativeShutdown)`; + /// the structured form is `last_down` in `/api/v1/ingresses`. Kept + /// after the session comes back up. #[serde(skip_serializing_if = "Option::is_none")] pub last_error: Option, + + /// When a BMP-monitored peer's session last went down, from the same + /// Peer Down Notification as `lastError`. + /// + /// Like the rest of a BMP peer's row, it goes away when the rib's GC + /// reaps a peer that never comes back. + #[serde(skip_serializing_if = "Option::is_none")] + pub last_down_time: Option>, } /// Collect every neighbor netom knows about, from both sources. @@ -206,6 +222,11 @@ fn bmp_neighbors(state: &ApiState) -> Vec { via_router: via.as_ref().and_then(|v| v.remote_addr), via_ingress_id: info.parent_ingress, peer_rib_type: info.peer_rib_type.map(|t| format!("{t:?}")), + last_error: info + .last_down + .as_ref() + .map(|down| down.description.clone()), + last_down_time: info.last_down.as_ref().map(|down| down.time), ..Default::default() } }) diff --git a/src/units/bmp_tcp_in/state_machine/machine.rs b/src/units/bmp_tcp_in/state_machine/machine.rs index feab47f..4f23d9a 100644 --- a/src/units/bmp_tcp_in/state_machine/machine.rs +++ b/src/units/bmp_tcp_in/state_machine/machine.rs @@ -719,6 +719,11 @@ where let removed_peers = self.details.remove_peer_identity_siblings(&pph); if !removed_peers.is_empty() { + // Why the router's BGP session went down, recorded on each view + // of the peer so /bgp/neighbors and /ingresses can report it. + let last_down = + super::peer_down::peer_down_info(&msg, Utc::now()); + // Reap every PeerState that shares this peer's identity. The // rib_type/policy-flag workaround in route_monitoring() can // create entries keyed on a synthesized post-policy PPH alongside @@ -772,12 +777,21 @@ where )> = removed_peers .iter() .flat_map(|peer| { - std::iter::once(peer.ingress_id) - .chain(peer.path_children.values().copied()) + // The peer's own ingress (one per RIB view) carries the + // reason; its ADD-PATH path-children are only flipped. + std::iter::once((peer.ingress_id, Some(&last_down))) + .chain( + peer.path_children + .values() + .map(|&child| (child, None)), + ) }) - .filter_map(|ingress_id| { + .filter_map(|(ingress_id, last_down)| { self.ingress_register - .mark_disconnected(ingress_id) + .mark_disconnected_with( + ingress_id, + last_down.cloned(), + ) .then_some((ingress_id, None)) }) .collect(); @@ -983,6 +997,9 @@ where )); adapted_ingress_info.state = Some(ingress::register::IngressState::Connected); + // The source peer's last Peer Down is its own; don't + // copy it onto this view. + adapted_ingress_info.last_down = None; // Layer D reuse: if this synthesized (peer, policy) was // seen in a prior session and kept as Disconnected, rebind // its IngressId instead of minting a fresh one each session. @@ -1057,6 +1074,9 @@ where )); adapted_ingress_info.state = Some(ingress::register::IngressState::Connected); + // The source peer's last Peer Down is its own; don't + // copy it onto this view. + adapted_ingress_info.last_down = None; // Layer D reuse (see the nulled-flags arm above). let new_ingress_id = self .ingress_register diff --git a/src/units/bmp_tcp_in/state_machine/mod.rs b/src/units/bmp_tcp_in/state_machine/mod.rs index fcf3fa0..782a3bd 100644 --- a/src/units/bmp_tcp_in/state_machine/mod.rs +++ b/src/units/bmp_tcp_in/state_machine/mod.rs @@ -1,5 +1,6 @@ mod machine; mod metrics; +mod peer_down; mod processing; mod states; mod status_reporter; diff --git a/src/units/bmp_tcp_in/state_machine/peer_down.rs b/src/units/bmp_tcp_in/state_machine/peer_down.rs new file mode 100644 index 0000000..a20a0a4 --- /dev/null +++ b/src/units/bmp_tcp_in/state_machine/peer_down.rs @@ -0,0 +1,259 @@ +//! Why a monitored router's BGP session went down. +//! +//! A router sends a BMP Peer Down Notification (RFC 7854 §4.9) when one of +//! its own BGP sessions goes down. Besides the per-peer header identifying +//! the session, it carries a reason code and, depending on the reason, the +//! BGP NOTIFICATION the router sent or received, or the FSM event that +//! closed the session. This module turns that into a [`PeerDownInfo`] for +//! the ingress register. + +use bytes::Bytes; +use chrono::{DateTime, Utc}; +use routecore::bgp::message::notification::{CeaseSubcode, Details}; +use routecore::bmp::message::PeerDownNotification; + +use crate::ingress::register::{PeerDownInfo, PeerDownReason}; + +/// Offset of the reason code: common header (6) plus per-peer header (42). +const REASON_OFFSET: usize = 48; + +/// Offset of the NOTIFICATION data: header (19) plus code and subcode. +const NOTIFICATION_DATA_OFFSET: usize = 21; + +/// Decode a Peer Down Notification into a [`PeerDownInfo`]. +/// +/// `received` is used as the time when the router sent a zero per-peer +/// header timestamp, which some exporters do. +pub(super) fn peer_down_info( + msg: &PeerDownNotification, + received: DateTime, +) -> PeerDownInfo { + let pph_time = msg.per_peer_header().timestamp(); + let time = if pph_time.timestamp() > 0 { + pph_time + } else { + received + }; + + // routecore's PeerDownReason has no numeric value and folds RFC 9069's + // code 6 into Unknown, so read the code itself. The byte is always + // there: PeerDownNotification::check() parses it. + let reason_code = + msg.as_ref().get(REASON_OFFSET).copied().unwrap_or_default(); + let reason = PeerDownReason::from_code(reason_code); + + let mut info = PeerDownInfo { + time, + reason, + reason_code, + notification_code: None, + notification_subcode: None, + shutdown_communication: None, + fsm_event: None, + description: String::new(), + }; + + info.description = match reason { + PeerDownReason::LocalNotification + | PeerDownReason::RemoteNotification => { + let side = if reason == PeerDownReason::LocalNotification { + "local" + } else { + "remote" + }; + match msg.notification() { + Some(notification) + if notification.as_ref().len() + >= NOTIFICATION_DATA_OFFSET => + { + let details = notification.details(); + let [code, subcode] = details.raw(); + info.notification_code = Some(code); + info.notification_subcode = Some(subcode); + + // routecore's data() runs to the end of the buffer, + // which here is the end of the BMP message; bound it + // by the NOTIFICATION's own length. + let bytes = notification.as_ref(); + let end = + usize::from(notification.length()).min(bytes.len()); + let data = bytes + .get(NOTIFICATION_DATA_OFFSET..end) + .unwrap_or_default(); + info.shutdown_communication = + shutdown_communication(details, data); + + // Same wording as the NOTIFICATIONs of sessions netom + // terminates itself (bgp_tcp_in's last_error). + match &info.shutdown_communication { + Some(text) => format!( + "{side} NOTIFICATION: {details:?} \"{text}\"" + ), + None => format!("{side} NOTIFICATION: {details:?}"), + } + } + _ => format!("{side} NOTIFICATION (not included)"), + } + } + PeerDownReason::LocalFsm => { + info.fsm_event = msg.fsm(); + match info.fsm_event { + Some(event) => format!("local FSM event {event}"), + None => "local FSM event (not included)".to_string(), + } + } + PeerDownReason::RemoteNoData => { + "remote closed without NOTIFICATION".to_string() + } + PeerDownReason::PeerDeconfigured => "peer de-configured".to_string(), + PeerDownReason::LocalTlv => "local close with TLV data".to_string(), + PeerDownReason::Reserved | PeerDownReason::Unknown => { + format!("reason code {reason_code}") + } + }; + + info +} + +/// The shutdown communication of a Cease Administrative Shutdown or +/// Administrative Reset NOTIFICATION (RFC 8203, length limit raised to 255 +/// by RFC 9003): a length byte followed by that many bytes of UTF-8. +/// +/// A missing, empty or truncated communication yields `None`; invalid UTF-8 +/// is replaced rather than dropped, since this is for display only. +fn shutdown_communication(details: Details, data: &[u8]) -> Option { + if !matches!( + details, + Details::Cease( + CeaseSubcode::AdministrativeShutdown + | CeaseSubcode::AdministrativeReset + ) + ) { + return None; + } + let (&len, rest) = data.split_first()?; + let text = rest.get(..usize::from(len)).filter(|t| !t.is_empty())?; + Some(String::from_utf8_lossy(text).into_owned()) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::bgp::encode::{ + mk_bgp_notification_msg, mk_peer_down_notification_msg_with, + mk_per_peer_header, + }; + + fn decode(reason: u8, data: &[u8]) -> PeerDownInfo { + let pph = mk_per_peer_header("10.0.0.1", 65001); + let bytes = mk_peer_down_notification_msg_with(&pph, reason, data); + let msg = PeerDownNotification::from_octets(bytes).unwrap(); + peer_down_info(&msg, DateTime::::MIN_UTC) + } + + fn shutdown_data(text: &[u8]) -> Vec { + let mut data = vec![text.len() as u8]; + data.extend_from_slice(text); + data + } + + #[test] + fn remote_notification_with_shutdown_communication() { + let notification = + mk_bgp_notification_msg(6, 2, &shutdown_data(b"maintenance")); + let info = decode(3, ¬ification); + assert_eq!(info.reason, PeerDownReason::RemoteNotification); + assert_eq!(info.reason_code, 3); + assert_eq!(info.notification_code, Some(6)); + assert_eq!(info.notification_subcode, Some(2)); + assert_eq!( + info.shutdown_communication.as_deref(), + Some("maintenance") + ); + assert_eq!( + info.description, + "remote NOTIFICATION: Cease(AdministrativeShutdown) \ + \"maintenance\"" + ); + } + + #[test] + fn local_max_prefix_notification() { + let info = decode(1, &mk_bgp_notification_msg(6, 1, &[])); + assert_eq!(info.reason, PeerDownReason::LocalNotification); + assert_eq!(info.shutdown_communication, None); + assert_eq!( + info.description, + "local NOTIFICATION: Cease(MaximumPrefixesReached)" + ); + } + + #[test] + fn notification_data_is_bounded_by_its_length() { + // Bytes after the NOTIFICATION (here a stray trailer) must not be + // read as part of the shutdown communication. + let mut data = + mk_bgp_notification_msg(6, 2, &shutdown_data(b"bye")).to_vec(); + data.extend_from_slice(b"trailer"); + let info = decode(3, &data); + assert_eq!(info.shutdown_communication.as_deref(), Some("bye")); + } + + #[test] + fn truncated_or_invalid_shutdown_communication() { + // Claims 10 bytes, carries 3: not a valid communication. + let short = mk_bgp_notification_msg(6, 2, &[10, b'a', b'b', b'c']); + assert_eq!(decode(3, &short).shutdown_communication, None); + + let invalid = + mk_bgp_notification_msg(6, 4, &shutdown_data(&[b'o', 0xFF])); + assert_eq!( + decode(3, &invalid).shutdown_communication.as_deref(), + Some("o\u{FFFD}") + ); + + // Only Cease shutdown/reset carry a communication. + let other = mk_bgp_notification_msg(4, 0, &shutdown_data(b"x")); + let info = decode(1, &other); + assert_eq!(info.shutdown_communication, None); + assert_eq!(info.description, "local NOTIFICATION: HoldTimerExpired"); + } + + #[test] + fn fsm_event_and_reasons_without_data() { + let info = decode(2, &18u16.to_be_bytes()); + assert_eq!(info.reason, PeerDownReason::LocalFsm); + assert_eq!(info.fsm_event, Some(18)); + assert_eq!(info.description, "local FSM event 18"); + + for (code, reason, description) in [ + ( + 4, + PeerDownReason::RemoteNoData, + "remote closed without NOTIFICATION", + ), + (5, PeerDownReason::PeerDeconfigured, "peer de-configured"), + (6, PeerDownReason::LocalTlv, "local close with TLV data"), + (9, PeerDownReason::Unknown, "reason code 9"), + ] { + let info = decode(code, &[]); + assert_eq!(info.reason, reason); + assert_eq!(info.reason_code, code); + assert_eq!(info.description, description); + } + } + + #[test] + fn missing_notification_is_reported_as_such() { + let info = decode(3, &[]); + assert_eq!(info.notification_code, None); + assert_eq!(info.description, "remote NOTIFICATION (not included)"); + } + + #[test] + fn per_peer_header_timestamp_is_used_when_set() { + let info = decode(4, &[]); + // mk_per_peer_header stamps the current time, not 0. + assert!(info.time > DateTime::::MIN_UTC); + } +} diff --git a/src/units/bmp_tcp_in/state_machine/tests.rs b/src/units/bmp_tcp_in/state_machine/tests.rs index ffd62cb..cd625ad 100644 --- a/src/units/bmp_tcp_in/state_machine/tests.rs +++ b/src/units/bmp_tcp_in/state_machine/tests.rs @@ -2516,3 +2516,206 @@ fn assert_invalid_msg_starts_with( ); } } + +// --------------------------------------------------------------------------- +// Why a monitored router's BGP session went down (Peer Down reasons) +// --------------------------------------------------------------------------- + +/// A Peer Down for `pph` with reason 3: the peer sent a Cease +/// Administrative Shutdown NOTIFICATION with a shutdown communication. +fn mk_remote_shutdown_peer_down_msg( + pph: &crate::bgp::encode::PerPeerHeader, + text: &str, +) -> BmpMsg { + let mut data = vec![text.len() as u8]; + data.extend_from_slice(text.as_bytes()); + let notification = + crate::bgp::encode::mk_bgp_notification_msg(6, 2, &data); + BmpMsg::from_octets( + crate::bgp::encode::mk_peer_down_notification_msg_with( + pph, + 3, + ¬ification, + ), + ) + .unwrap() +} + +#[test] +fn peer_down_records_why_the_session_went_down() { + use crate::ingress::register::{IngressState, PeerDownReason}; + + let register: Arc = Arc::default(); + let (pph, peer_up_msg_buf, real_pph) = + mk_peer_up_notification_msg_without_rfc4724_support( + "127.0.0.1", + 12345, + ); + let processor = mk_test_processor_with_register(®ister) + .process_msg( + Instant::now(), + mk_initiation_msg(TEST_ROUTER_SYS_NAME, TEST_ROUTER_SYS_DESC), + None, + ) + .next_state + .process_msg(Instant::now(), peer_up_msg_buf, None) + .next_state; + let ingress_id = if let BmpState::Dumping(p) = &processor { + p.details + .peer_states + .get_peer_ingress_id(&real_pph) + .unwrap() + } else { + unreachable!("expected Dumping after PeerUp"); + }; + assert_eq!(register.get(ingress_id).unwrap().last_down, None); + + let processor = processor + .process_msg( + Instant::now(), + mk_remote_shutdown_peer_down_msg(&pph, "maintenance"), + None, + ) + .next_state; + + let info = register.get(ingress_id).unwrap(); + assert_eq!(info.state, Some(IngressState::Disconnected)); + let last_down = info.last_down.expect("Peer Down must record a reason"); + assert_eq!(last_down.reason, PeerDownReason::RemoteNotification); + assert_eq!(last_down.notification_code, Some(6)); + assert_eq!(last_down.notification_subcode, Some(2)); + assert_eq!( + last_down.description, + "remote NOTIFICATION: Cease(AdministrativeShutdown) \"maintenance\"" + ); + + // The record outlives the outage: a reconnect rebinds the same + // ingress without wiping why it last went down. + let (_, peer_up_msg_buf, _) = + mk_peer_up_notification_msg_without_rfc4724_support( + "127.0.0.1", + 12345, + ); + processor.process_msg(Instant::now(), peer_up_msg_buf, None); + let info = register.get(ingress_id).unwrap(); + assert_eq!(info.state, Some(IngressState::Connected)); + assert_eq!( + info.last_down.map(|down| down.reason), + Some(PeerDownReason::RemoteNotification) + ); +} + +#[test] +fn peer_down_reason_is_recorded_on_every_view_of_the_peer() { + let register: Arc = Arc::default(); + let (pph_pre, peer_up_msg_buf, _) = + mk_peer_up_notification_msg_without_rfc4724_support( + "127.0.0.1", + 12345, + ); + let mut pph_post = mk_per_peer_header("127.0.0.1", 12345); + pph_post.peer_flags = 0x40; + + let processor = mk_test_processor_with_register(®ister) + .process_msg( + Instant::now(), + mk_initiation_msg(TEST_ROUTER_SYS_NAME, TEST_ROUTER_SYS_DESC), + None, + ) + .next_state + .process_msg(Instant::now(), peer_up_msg_buf, None) + .next_state + // Post-policy Route Monitoring with no matching PeerUp: a + // synthesized post-policy view of the same peer. + .process_msg(Instant::now(), mk_route_monitoring_msg(&pph_post), None) + .next_state; + let views: Vec<_> = register + .cloned_info() + .into_iter() + .filter(|(_, info)| { + info.ingress_type + == Some(crate::ingress::register::IngressType::BgpViaBmp) + }) + .map(|(id, _)| id) + .collect(); + assert_eq!(views.len(), 2, "pre-policy and synthesized post-policy"); + + processor.process_msg( + Instant::now(), + mk_remote_shutdown_peer_down_msg(&pph_pre, "bye"), + None, + ); + for id in views { + let last_down = register.get(id).unwrap().last_down; + assert_eq!( + last_down.map(|down| down.shutdown_communication), + Some(Some("bye".to_string())), + "view {id} must carry the reason" + ); + } +} + +#[test] +fn synthesized_views_do_not_inherit_a_stale_peer_down() { + let register: Arc = Arc::default(); + let (pph_pre, peer_up_msg_buf, real_pph) = + mk_peer_up_notification_msg_without_rfc4724_support( + "127.0.0.1", + 12345, + ); + let processor = mk_test_processor_with_register(®ister) + .process_msg( + Instant::now(), + mk_initiation_msg(TEST_ROUTER_SYS_NAME, TEST_ROUTER_SYS_DESC), + None, + ) + .next_state + .process_msg(Instant::now(), peer_up_msg_buf, None) + .next_state + .process_msg( + Instant::now(), + mk_remote_shutdown_peer_down_msg(&pph_pre, "first outage"), + None, + ) + .next_state; + + // The pre-policy view comes back carrying its last Peer Down... + let (_, peer_up_msg_buf, _) = + mk_peer_up_notification_msg_without_rfc4724_support( + "127.0.0.1", + 12345, + ); + let processor = processor + .process_msg(Instant::now(), peer_up_msg_buf, None) + .next_state; + let pre_id = if let BmpState::Dumping(p) = &processor { + p.details + .peer_states + .get_peer_ingress_id(&real_pph) + .unwrap() + } else { + unreachable!("expected Dumping after PeerUp"); + }; + assert!(register.get(pre_id).unwrap().last_down.is_some()); + + // ...but a post-policy view synthesized from it now has never been + // down, so it must not copy that record. + let mut pph_post = mk_per_peer_header("127.0.0.1", 12345); + pph_post.peer_flags = 0x40; + processor.process_msg( + Instant::now(), + mk_route_monitoring_msg(&pph_post), + None, + ); + let synthesized: Vec<_> = register + .cloned_info() + .into_iter() + .filter(|(id, info)| { + *id != pre_id + && info.ingress_type + == Some(crate::ingress::register::IngressType::BgpViaBmp) + }) + .collect(); + assert_eq!(synthesized.len(), 1); + assert_eq!(synthesized[0].1.last_down, None); +} From 6f6d2dbc92560e0e9347485034f2b598d9c7aee8 Mon Sep 17 00:00:00 2001 From: Alex Typaldos Date: Thu, 1 Oct 2026 17:16:23 -0400 Subject: [PATCH 2/4] netom-cli: show why and when a BMP peer last went down `show ip bgp neighbors` already printed `Last error` for sessions netom terminates itself. BMP-monitored peers now carry the same field, filled from the router's Peer Down Notification, plus `lastDownTime`; show the latter as `Last down: