diff --git a/Changelog.md b/Changelog.md index 1e53192..3cda3b9 100644 --- a/Changelog.md +++ b/Changelog.md @@ -115,6 +115,22 @@ Released yyyy-mm-dd. entry and so appeared nowhere. Peers observed through BMP are included in the same response, carrying the monitored router they were seen through. +* Why a monitored router's BGP session went down. When a router reports a + session down with a BMP Peer Down Notification (RFC 7854 §4.9), netom now + records the reason instead of discarding it: the reason code, the BGP + NOTIFICATION the router sent or received (e.g. `Cease(MaximumPrefixesReached)` + or `Cease(AdministrativeShutdown)`, with its RFC 8203 shutdown + communication), or the FSM event that closed the session. BMP-monitored + peers in `/api/v1/bgp/neighbors` gain `lastError` and `lastDownTime`, and + their ingresses in `/api/v1/ingresses` gain a structured `last_down`. Both + are kept after the session comes back up. Previously a BMP peer's row only + said `Idle`. See `docs/bmp-tcp-in.md`. +* `netom-cli show ip bgp neighbors` shows a BMP peer's last Peer Down as + `Last error` (why) and `Last down` (when, and how long ago). +* New `bmp_state_num_peer_down_notifications` counter: Peer Down + Notifications per monitored router and reason (RFC 7854 §4.9), i.e. the + router's BGP sessions going down. See `docs/bmp-tcp-in.md`. + * Native BGP sessions now record `session_up_time` in the ingress register. Besides giving those peers an uptime, this fixes the per-peer header of the Peer Up that `bmp-tcp-out` synthesizes for restreamed native diff --git a/doc/netom-cli.1 b/doc/netom-cli.1 index 888bdb2..65e1428 100644 --- a/doc/netom-cli.1 +++ b/doc/netom-cli.1 @@ -230,6 +230,15 @@ The hold time reported by is the configured one, not the negotiated one, which the BGP library keeps private. +For a peer observed through BMP, +.B show ip bgp neighbors +also shows why its session last went down and when +.RB ( "Last error" ", " "Last down" ), +from the monitored router's Peer Down Notification: the BGP NOTIFICATION the +router sent (local) or received (remote), or the FSM event that closed the +session. Routers report only sessions that came up, so a peer that never +established is not shown. + .SH PAGING Output is not paged. Piping a whole-table dump into a pager is inadvisable: the daemon aborts a dump whose reader stops draining, so a pager that stops diff --git a/docs/bmp-tcp-in.md b/docs/bmp-tcp-in.md index b7d7607..e410ed6 100644 --- a/docs/bmp-tcp-in.md +++ b/docs/bmp-tcp-in.md @@ -71,6 +71,48 @@ Point a `clickhouse-out` target at the input unit names to record received observations before RIB mutation. See [ClickHouse export](clickhouse.md). Exporter-generated snapshots and EOR completeness tracking remain later work. +## Why a peer's session went down + +When one of a monitored router's own BGP sessions goes down, the router sends +a Peer Down Notification (RFC 7854 §4.9). netom records the reason on that +peer and keeps it after the session comes back up, so you can still see why +it last dropped: + +* `GET /api/v1/bgp/neighbors` gives the peer's row a `lastError` with a + one-line summary and a `lastDownTime`. +* `GET /api/v1/ingresses` gives each view of the peer (pre-policy, + post-policy) a structured `last_down`. + +| Reason | `reason` | Carries | +|---|---|---| +| 1 | `localNotification` | the NOTIFICATION the router sent, e.g. `Cease(MaximumPrefixesReached)` | +| 2 | `localFsm` | the FSM event code that made the router close the session | +| 3 | `remoteNotification` | the NOTIFICATION the peer sent, e.g. `Cease(AdministrativeShutdown)` | +| 4 | `remoteNoData` | nothing: the peer closed the session without a NOTIFICATION | +| 5 | `peerDeconfigured` | nothing: the peer was removed from the router's configuration | +| 6 | `localTlv` | (RFC 9069) the router closed the session; TLV data follows | + +For a Cease Administrative Shutdown or Administrative Reset NOTIFICATION, the +shutdown communication (RFC 8203, RFC 9003) is decoded too, so a summary reads +like `remote NOTIFICATION: Cease(AdministrativeShutdown) "maintenance"`. The +time is the Peer Down's per-peer header timestamp, or the time netom received +it when the router sends 0. + +Each Peer Down Notification is also counted, per router and reason, in the +`bmp_state_num_peer_down_notifications` counter on `/metrics`, for example +`{router="edge1",reason="localNotification"}`. Every reason has a series, so +an alert on a rising `localNotification` rate catches a router tearing +sessions down, typically for exceeded prefix limits. + +Two limits: + +* A router only reports sessions that reached Established. It sends Peer + Down only for a peer it sent Peer Up for, so a session that never comes up, + for example an OPEN rejected for a bad peer AS, never appears here. Look on + the router itself for those. +* The record lives on the peer's ingress. If a peer stays down until the rib's + garbage collection reaps it, its record goes with it. + ## Integration tests The ClickHouse test driver can act as a BMP exporter: diff --git a/docs/cli.md b/docs/cli.md index ef3f7f6..b0cbd28 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -300,6 +300,30 @@ is waiting for a peer that has not connected. Only exactly-configured peers can be listed this way — a peer matched by a prefix has no single address to show until it connects. +A peer seen through BMP is down when its router says so. The router's Peer +Down Notification also says why, and netom keeps that after the session comes +back: + +``` +netom> show ip bgp neighbors 192.0.2.8 +BGP neighbor is 192.0.2.8, remote AS 65101 + BGP router identifier: 192.0.2.8 + BGP state = Idle + Learned via: BMP feed + Monitored router: 10.99.0.1 (ingress 2) + RIB type: InPre + ... + Last error: remote NOTIFICATION: Cease(AdministrativeShutdown) "maintenance" + Last down: 2026-08-12T05:58:10Z (00:03:12 ago) +``` + +`remote` means the neighbor sent the NOTIFICATION and the router closed the +session in response; `local` means the router sent it, e.g. `local +NOTIFICATION: Cease(MaximumPrefixesReached)` when the neighbor exceeded a +prefix limit. A router only reports sessions that came up, so a BMP peer that +never established, for example over a peer AS mismatch, does not appear at +all. See [BMP input](bmp-tcp-in.md#why-a-peers-session-went-down). + ## Paging There is no built-in pager, and piping a whole-table dump into one is a bad diff --git a/docs/rib-query-api.md b/docs/rib-query-api.md index 1904c30..ed2e22d 100644 --- a/docs/rib-query-api.md +++ b/docs/rib-query-api.md @@ -371,6 +371,11 @@ store walk, so it does not help a response fit under those caps. * `GET /api/v1/ingresses` — the peers and sessions the ids above refer to. Accepts `filter[type]`, `filter[state]`, `filter[ribType]`, `filter[peerAddress]`, `filter[peerAsn]` and `format`. + A BMP-monitored peer whose session has gone down carries `last_down`: why + and when, from the router's Peer Down Notification. See + [BMP input](bmp-tcp-in.md#why-a-peers-session-went-down). * `GET /api/v1/ingresses/{id}` — one ingress. * `GET /api/v1/bgp/neighbors[/{peer}]` — session state and per-peer counters, - merging natively terminated and BMP-monitored peers. + merging natively terminated and BMP-monitored peers. For BMP-monitored + peers, `lastError` and `lastDownTime` give the reason and time of the + session's last Peer Down. diff --git a/scripts/e2e-addpath-bmp.py b/scripts/e2e-addpath-bmp.py index e2cf6e0..1805c86 100644 --- a/scripts/e2e-addpath-bmp.py +++ b/scripts/e2e-addpath-bmp.py @@ -150,9 +150,19 @@ def fs_withdraw(path_id, rule): return bmp_msg(0, pph() + bgp_update(pas, b"")) +SHUTDOWN_COMMUNICATION = b"e2e: maintenance" + + def peer_down(): - # Reason 4: remote system closed without notification. - return bmp_msg(2, pph() + bytes([4])) + # Reason 3: the peer sent a NOTIFICATION, here Cease (6) / Administrative + # Shutdown (2) carrying an RFC 8203 shutdown communication. + data = bytes([len(SHUTDOWN_COMMUNICATION)]) + SHUTDOWN_COMMUNICATION + notification = ( + b"\xff" * 16 + + struct.pack("!HBBB", 21 + len(data), 3, 6, 2) + + data + ) + return bmp_msg(2, pph() + bytes([3]) + notification) # --- BMP consumer-side parsing -------------------------------------------------- @@ -462,6 +472,46 @@ def main(): ) print(f"{context}: peer down emitted exactly once: OK") + # HTTP: why the router's session went down, from the Peer Down's + # NOTIFICATION, on the session ingress and in the neighbor row. + with urllib.request.urlopen( + f"http://{HTTP_ADDR}/api/v1/ingresses", timeout=10 + ) as resp: + ingresses = json.load(resp)["data"] + sessions = [ + e for e in ingresses if e.get("ingress_type") == "bgpViaBmp" + ] + assert len(sessions) == 1, sessions + last_down = sessions[0].get("last_down") + assert last_down, f"FAIL: no last_down on the session: {sessions[0]}" + assert last_down["reason"] == "remoteNotification", last_down + assert last_down["reason_code"] == 3, last_down + assert ( + last_down["notification_code"], + last_down["notification_subcode"], + ) == (6, 2), last_down + assert ( + last_down["shutdown_communication"] + == SHUTDOWN_COMMUNICATION.decode() + ), last_down + # The feeder's per-peer header timestamp is 0, so netom stamps it. + assert last_down["time"].startswith("20"), last_down + print("/ingresses records why the session went down: OK") + + with urllib.request.urlopen( + f"http://{HTTP_ADDR}/api/v1/bgp/neighbors/{PEER_IP}", timeout=10 + ) as resp: + neighbors = json.load(resp)["data"] + assert neighbors, f"FAIL: no neighbor row for {PEER_IP}" + row = neighbors[0] + assert row.get("state") == "Idle", row + assert row.get("lastError") == ( + "remote NOTIFICATION: Cease(AdministrativeShutdown) " + f'"{SHUTDOWN_COMMUNICATION.decode()}"' + ), row + assert row.get("lastDownTime") == last_down["time"], row + print("/bgp/neighbors reports lastError and lastDownTime: OK") + feeder.close() for _, reader in consumers: reader.sock.close() diff --git a/scripts/e2e-addpath-bmp.sh b/scripts/e2e-addpath-bmp.sh index 78c4946..2bced48 100755 --- a/scripts/e2e-addpath-bmp.sh +++ b/scripts/e2e-addpath-bmp.sh @@ -17,7 +17,10 @@ # path of each family (with its path id), # * the /ingresses HTTP API shows the two bgpPath children with pathId and # parentIngress, -# * PeerDown is emitted exactly once (for the session, not per child). +# * PeerDown is emitted exactly once (for the session, not per child), +# * the Peer Down's NOTIFICATION (Cease / Administrative Shutdown with a +# shutdown communication) is recorded as `last_down` in /ingresses and +# as lastError / lastDownTime in /bgp/neighbors. # # Requirements: cargo, python3. Set NETOM_BIN to skip the build. set -euo pipefail diff --git a/src/bin/netom-cli/commands/bgp.rs b/src/bin/netom-cli/commands/bgp.rs index ddccc10..5a032f7 100644 --- a/src/bin/netom-cli/commands/bgp.rs +++ b/src/bin/netom-cli/commands/bgp.rs @@ -254,6 +254,15 @@ pub fn render_neighbors( if let Some(err) = n["lastError"].as_str() { writeln!(out, " Last error: {err}")?; } + // BMP peers: when the router reported the session down (Peer Down + // Notification); lastError above says why. + if let Some(when) = n["lastDownTime"].as_str() { + writeln!( + out, + " Last down: {when} ({} ago)", + super::bmp::uptime_from(&n["lastDownTime"]), + )?; + } } Ok(()) } @@ -1233,7 +1242,7 @@ mod tests { let out = render(NEIGHBORS, None); assert!(out.contains("10.1.0.1")); assert!(out.contains("192.0.2.7")); - assert!(out.contains("Total neighbors 4 (bgp 3, bmp 1)"), "{out}"); + assert!(out.contains("Total neighbors 5 (bgp 3, bmp 2)"), "{out}"); } /// The whole point of the FSM work: a configured peer that never came @@ -1308,6 +1317,29 @@ mod tests { assert!(out.contains("Duplicates: 2,410,338"), "{out}"); } + /// A BMP-monitored peer that went down says why and when, from the + /// router's Peer Down Notification. + #[test] + fn neighbor_detail_shows_why_a_bmp_peer_went_down() { + let mut buf = Vec::new(); + render_neighbors(&mut buf, NEIGHBORS).unwrap(); + let out = String::from_utf8(buf).unwrap(); + let down = out + .split("\n\n") + .find(|block| block.contains("BGP neighbor is 192.0.2.8")) + .expect("the down BMP peer must be rendered"); + assert!(down.contains("BGP state = Idle"), "{down}"); + assert!( + down.contains( + "Last error: remote NOTIFICATION: \ + Cease(AdministrativeShutdown) \"maintenance\"" + ), + "{down}" + ); + assert!(down.contains("Last down: 2026-08-12T05:58:10Z ("), "{down}"); + assert!(down.contains(" ago)"), "{down}"); + } + #[test] fn neighbor_detail_reports_no_match_clearly() { let mut buf = Vec::new(); diff --git a/src/bin/netom-cli/commands/bmp.rs b/src/bin/netom-cli/commands/bmp.rs index 7843824..f2cde9c 100644 --- a/src/bin/netom-cli/commands/bmp.rs +++ b/src/bin/netom-cli/commands/bmp.rs @@ -203,7 +203,7 @@ pub fn render_ingresses( //------------ helpers ------------------------------------------------------- /// Render an RFC 3339 timestamp as an elapsed time. -fn uptime_from(value: &serde_json::Value) -> String { +pub(super) fn uptime_from(value: &serde_json::Value) -> String { let Some(text) = value.as_str() else { return "never".to_string(); }; diff --git a/src/ingress/register.rs b/src/ingress/register.rs index edd0f0c..beb990d 100644 --- a/src/ingress/register.rs +++ b/src/ingress/register.rs @@ -647,11 +647,30 @@ impl Register { /// Mark an existing ingress down without resurrecting a GC'd child /// that remains in an input handler's cache until its next prune. pub fn mark_disconnected(&self, id: IngressId) -> bool { + self.mark_disconnected_with(id, None) + } + + /// Like [`Register::mark_disconnected`], also recording why the session + /// went down when it is known (a BMP Peer Down Notification). + /// + /// The record is set under the same lock as the state change, and bumps + /// `history_revision` once, so exporters keyed on the revision pick up + /// both together. It is deliberately left in place when the peer comes + /// back up: `update_info` only merges fields that are set, and a PeerUp + /// never sets `last_down`. + pub fn mark_disconnected_with( + &self, + id: IngressId, + last_down: Option, + ) -> bool { let mut lock = self.info.write().unwrap(); let Some(info) = lock.get_mut(&id) else { return false; }; info.state = Some(IngressState::Disconnected); + if last_down.is_some() { + info.last_down = last_down; + } info.history_revision = info.history_revision.saturating_add(1); true } @@ -983,9 +1002,115 @@ info_for_field!(IngressInfo{ // SessionConfig::enabled_addpaths() (the negotiated set), not from // remote_capabilities (which is one side's OPEN, not the intersection). #[serde(serialize_with = "serialize_addpath_families")] - addpath_families: Vec + addpath_families: Vec, + // Why this BGP session last went down, from the monitored router's BMP + // Peer Down Notification. Kept across reconnects; see PeerDownInfo. + last_down: PeerDownInfo }); +/// Why a monitored router's BGP session went down, as reported by the +/// router in a BMP Peer Down Notification (RFC 7854 §4.9). +/// +/// This is about the BGP session between the monitored router and its peer, +/// not the BMP session to netom. Only sessions that reached Established are +/// ever reported: a router sends Peer Down only for a peer it sent Peer Up +/// for, so a session that fails to come up (e.g. an OPEN rejected for a bad +/// peer AS) never shows up here. +#[serde_with::skip_serializing_none] +#[derive(Clone, Debug, Eq, PartialEq, Ord, PartialOrd, serde::Serialize)] +pub struct PeerDownInfo { + /// When the session went down: the Peer Down per-peer header timestamp, + /// or the time netom received the message if the router sent 0. + pub time: DateTime, + pub reason: PeerDownReason, + /// The raw Peer Down reason code, also for codes netom has no name for. + pub reason_code: u8, + /// Error code and subcode of the BGP NOTIFICATION the router sent + /// (reason 1) or received (reason 3). + pub notification_code: Option, + pub notification_subcode: Option, + /// The RFC 8203/9003 shutdown communication carried by a Cease + /// Administrative Shutdown or Administrative Reset NOTIFICATION. + pub shutdown_communication: Option, + /// The BGP FSM event code that closed the session (reason 2). + pub fsm_event: Option, + /// A one-line human-readable summary, e.g. + /// `remote NOTIFICATION: Cease(AdministrativeShutdown)`. + pub description: String, +} + +/// The reason code of a BMP Peer Down Notification (RFC 7854 §4.9, code 6 +/// from RFC 9069). +#[derive( + Clone, Copy, Debug, Eq, Hash, PartialEq, Ord, PartialOrd, serde::Serialize, +)] +#[serde(rename_all = "camelCase")] +pub enum PeerDownReason { + /// 0: reserved. + Reserved, + /// 1: the router closed the session and sent a NOTIFICATION. + LocalNotification, + /// 2: the router closed the session without a NOTIFICATION; an FSM + /// event code follows. + LocalFsm, + /// 3: the peer closed the session and sent a NOTIFICATION. + RemoteNotification, + /// 4: the peer closed the session without a NOTIFICATION. + RemoteNoData, + /// 5: the peer was de-configured; information for it stops. + PeerDeconfigured, + /// 6: the router closed the session, with TLV data (RFC 9069, Loc-RIB). + LocalTlv, + /// Any other code. + Unknown, +} + +impl PeerDownReason { + /// All variants, in reason code order. + pub const ALL: [PeerDownReason; 8] = [ + Self::Reserved, + Self::LocalNotification, + Self::LocalFsm, + Self::RemoteNotification, + Self::RemoteNoData, + Self::PeerDeconfigured, + Self::LocalTlv, + Self::Unknown, + ]; + + pub fn from_code(code: u8) -> Self { + match code { + 0 => Self::Reserved, + 1 => Self::LocalNotification, + 2 => Self::LocalFsm, + 3 => Self::RemoteNotification, + 4 => Self::RemoteNoData, + 5 => Self::PeerDeconfigured, + 6 => Self::LocalTlv, + _ => Self::Unknown, + } + } + + /// Position in [`PeerDownReason::ALL`], for per-reason counters. + pub fn index(self) -> usize { + self as usize + } + + /// The name used in JSON output and metric labels. + pub fn as_str(self) -> &'static str { + match self { + Self::Reserved => "reserved", + Self::LocalNotification => "localNotification", + Self::LocalFsm => "localFsm", + Self::RemoteNotification => "remoteNotification", + Self::RemoteNoData => "remoteNoData", + Self::PeerDeconfigured => "peerDeconfigured", + Self::LocalTlv => "localTlv", + Self::Unknown => "unknown", + } + } +} + /// Serialize a raw BGP capability blob (`capabilities_as_vec` wire format) as /// a human-readable list of capability names, e.g. /// `["MultiProtocol","FourOctetAsn","AddPath"]`, for the `/ingresses` dump. @@ -1124,6 +1249,71 @@ mod tests { assert_eq!(res.get(id).unwrap().rib_type, Some(RibType::LocRib)); } + #[test] + fn mark_disconnected_with_records_and_keeps_the_peer_down() { + let register = Register::new(); + let id = register.register(); + register.update_info( + id, + IngressInfo::new().with_state(IngressState::Connected), + ); + + let down = PeerDownInfo { + time: DateTime::from_timestamp(1_700_000_000, 0).unwrap(), + reason: PeerDownReason::RemoteNotification, + reason_code: 3, + notification_code: Some(6), + notification_subcode: Some(2), + shutdown_communication: Some("maintenance".into()), + fsm_event: None, + description: "remote NOTIFICATION: \ + Cease(AdministrativeShutdown) \"maintenance\"" + .into(), + }; + assert!(register.mark_disconnected_with(id, Some(down.clone()))); + let info = register.get(id).unwrap(); + assert_eq!(info.state, Some(IngressState::Disconnected)); + assert_eq!(info.last_down.as_ref(), Some(&down)); + + // A later disconnect without a reason (e.g. the whole BMP session + // closing) leaves the last known reason alone. + assert!(register.mark_disconnected(id)); + assert_eq!(register.get(id).unwrap().last_down, Some(down)); + + // /api/v1/ingresses: snake_case keys like the rest of IngressInfo, + // absent fields omitted. + let json = + serde_json::to_value(IdAndInfo::from((id, &info))).unwrap(); + assert_eq!( + json["last_down"], + serde_json::json!({ + "time": "2023-11-14T22:13:20Z", + "reason": "remoteNotification", + "reason_code": 3, + "notification_code": 6, + "notification_subcode": 2, + "shutdown_communication": "maintenance", + "description": "remote NOTIFICATION: \ + Cease(AdministrativeShutdown) \"maintenance\"", + }) + ); + } + + #[test] + fn peer_down_reason_names_match_their_codes() { + for (code, reason) in PeerDownReason::ALL.iter().enumerate() { + assert_eq!(reason.index(), code); + if code <= 6 { + assert_eq!(PeerDownReason::from_code(code as u8), *reason); + } + assert_eq!( + serde_json::to_value(reason).unwrap(), + serde_json::json!(reason.as_str()) + ); + } + assert_eq!(PeerDownReason::from_code(200), PeerDownReason::Unknown); + } + #[test] fn cloned_info_for_takes_the_ids_and_their_parents() { let register = Register::new(); diff --git a/src/tests/util.rs b/src/tests/util.rs index 7d1ab8e..93ea0d1 100644 --- a/src/tests/util.rs +++ b/src/tests/util.rs @@ -922,6 +922,18 @@ pub mod bgp { pub fn mk_peer_down_notification_msg( per_peer_header: &PerPeerHeader, + ) -> Bytes { + mk_peer_down_notification_msg_with(per_peer_header, 5, &[]) + } + + /// A Peer Down Notification with the given reason code and data: + /// a BGP NOTIFICATION for reasons 1 and 3 (see + /// [`mk_bgp_notification_msg`]), a 2-byte FSM event code for + /// reason 2, nothing for the others. + pub fn mk_peer_down_notification_msg_with( + per_peer_header: &PerPeerHeader, + reason: u8, + data: &[u8], ) -> Bytes { let mut buf = BytesMut::new(); push_bmp_common_header( @@ -943,12 +955,29 @@ pub mod bgp { // // From: https://www.rfc-editor.org/rfc/rfc7854.html#section-4.9 - buf.extend_from_slice(&5u8.to_be_bytes()); // reason code 5 + buf.extend_from_slice(&[reason]); + buf.extend_from_slice(data); finalize_bmp_msg_len(&mut buf); buf.freeze() } + /// A BGP NOTIFICATION message (RFC 4271 §4.5) with the given error + /// code, subcode and data. + pub fn mk_bgp_notification_msg( + code: u8, + subcode: u8, + data: &[u8], + ) -> Bytes { + let mut buf = BytesMut::new(); + buf.extend_from_slice(&[0xFFu8; 16]); // marker + buf.extend_from_slice(&[0, 0]); // length, finalized below + buf.extend_from_slice(&[3, code, subcode]); // type 3: NOTIFICATION + buf.extend_from_slice(data); + finalize_bgp_msg_len(&mut buf); + buf.freeze() + } + pub fn mk_statistics_report_msg( per_peer_header: &PerPeerHeader, ) -> Bytes { diff --git a/src/units/bgp_tcp_in/http_ng.rs b/src/units/bgp_tcp_in/http_ng.rs index f34fdd3..7ab219c 100644 --- a/src/units/bgp_tcp_in/http_ng.rs +++ b/src/units/bgp_tcp_in/http_ng.rs @@ -12,6 +12,8 @@ use std::net::IpAddr; +use chrono::{DateTime, Utc}; + use axum::{ extract::{Path, State}, response::IntoResponse, @@ -116,8 +118,22 @@ pub struct Neighbor { #[serde(skip_serializing_if = "Option::is_none")] pub peer_rib_type: Option, + /// Why the session last failed or went down. For native sessions, the + /// last NOTIFICATION or connection error. For BMP-monitored peers, the + /// reason from the router's last Peer Down Notification (RFC 7854 + /// §4.9), e.g. `remote NOTIFICATION: Cease(AdministrativeShutdown)`; + /// the structured form is `last_down` in `/api/v1/ingresses`. Kept + /// after the session comes back up. #[serde(skip_serializing_if = "Option::is_none")] pub last_error: Option, + + /// When a BMP-monitored peer's session last went down, from the same + /// Peer Down Notification as `lastError`. + /// + /// Like the rest of a BMP peer's row, it goes away when the rib's GC + /// reaps a peer that never comes back. + #[serde(skip_serializing_if = "Option::is_none")] + pub last_down_time: Option>, } /// Collect every neighbor netom knows about, from both sources. @@ -206,6 +222,11 @@ fn bmp_neighbors(state: &ApiState) -> Vec { via_router: via.as_ref().and_then(|v| v.remote_addr), via_ingress_id: info.parent_ingress, peer_rib_type: info.peer_rib_type.map(|t| format!("{t:?}")), + last_error: info + .last_down + .as_ref() + .map(|down| down.description.clone()), + last_down_time: info.last_down.as_ref().map(|down| down.time), ..Default::default() } }) diff --git a/src/units/bmp_tcp_in/state_machine/machine.rs b/src/units/bmp_tcp_in/state_machine/machine.rs index feab47f..1cbd54e 100644 --- a/src/units/bmp_tcp_in/state_machine/machine.rs +++ b/src/units/bmp_tcp_in/state_machine/machine.rs @@ -719,6 +719,15 @@ where let removed_peers = self.details.remove_peer_identity_siblings(&pph); if !removed_peers.is_empty() { + // Why the router's BGP session went down, recorded on each view + // of the peer so /bgp/neighbors and /ingresses can report it. + let last_down = + super::peer_down::peer_down_info(&msg, Utc::now()); + self.status_reporter.peer_down_notification( + self.router_id.clone(), + last_down.reason, + ); + // Reap every PeerState that shares this peer's identity. The // rib_type/policy-flag workaround in route_monitoring() can // create entries keyed on a synthesized post-policy PPH alongside @@ -772,12 +781,21 @@ where )> = removed_peers .iter() .flat_map(|peer| { - std::iter::once(peer.ingress_id) - .chain(peer.path_children.values().copied()) + // The peer's own ingress (one per RIB view) carries the + // reason; its ADD-PATH path-children are only flipped. + std::iter::once((peer.ingress_id, Some(&last_down))) + .chain( + peer.path_children + .values() + .map(|&child| (child, None)), + ) }) - .filter_map(|ingress_id| { + .filter_map(|(ingress_id, last_down)| { self.ingress_register - .mark_disconnected(ingress_id) + .mark_disconnected_with( + ingress_id, + last_down.cloned(), + ) .then_some((ingress_id, None)) }) .collect(); @@ -983,6 +1001,9 @@ where )); adapted_ingress_info.state = Some(ingress::register::IngressState::Connected); + // The source peer's last Peer Down is its own; don't + // copy it onto this view. + adapted_ingress_info.last_down = None; // Layer D reuse: if this synthesized (peer, policy) was // seen in a prior session and kept as Disconnected, rebind // its IngressId instead of minting a fresh one each session. @@ -1057,6 +1078,9 @@ where )); adapted_ingress_info.state = Some(ingress::register::IngressState::Connected); + // The source peer's last Peer Down is its own; don't + // copy it onto this view. + adapted_ingress_info.last_down = None; // Layer D reuse (see the nulled-flags arm above). let new_ingress_id = self .ingress_register diff --git a/src/units/bmp_tcp_in/state_machine/metrics.rs b/src/units/bmp_tcp_in/state_machine/metrics.rs index 13df382..cf86b87 100644 --- a/src/units/bmp_tcp_in/state_machine/metrics.rs +++ b/src/units/bmp_tcp_in/state_machine/metrics.rs @@ -8,6 +8,7 @@ use chrono::{DateTime, Utc}; use crate::{ common::frim::FrimMap, + ingress::register::PeerDownReason, metrics::{ self, util::append_per_router_metric, Metric, MetricType, MetricUnit, }, @@ -148,6 +149,11 @@ pub struct RouterBmpMetrics { /// ADD-PATH withdrawals dropped because their path id was never seen /// announced on the session. pub num_addpath_unknown_path_id_withdrawals: Arc, + /// Peer Down Notifications received from this router, indexed by + /// [`PeerDownReason::index`]: one per BGP session of the monitored + /// router going down. + pub num_peer_down_notifications: + Arc<[AtomicUsize; PeerDownReason::ALL.len()]>, pub parse_errors: Arc, } @@ -218,6 +224,12 @@ impl BmpStateMachineMetrics { MetricType::Gauge, MetricUnit::Total, ); + const NUM_PEER_DOWN_NOTIFICATIONS_METRIC: Metric = Metric::new( + "bmp_state_num_peer_down_notifications", + "the number of BMP Peer Down Notifications from this router, i.e. its BGP sessions going down, by reason (RFC 7854 section 4.9)", + MetricType::Counter, + MetricUnit::Total, + ); const NUM_ADDPATH_PATH_CHILDREN_MINTED_METRIC: Metric = Metric::new( "bmp_state_num_addpath_path_children_minted", "the number of ADD-PATH path-child ingresses minted for this router's peers", @@ -336,6 +348,25 @@ impl metrics::Source for BmpStateMachineMetrics { Self::NUM_PEERS_UP_WITH_PENDING_EORS_METRIC, metrics.num_peers_up_dumping.load(SeqCst), ); + // One series per reason, so a jump in e.g. localNotification + // (the router tearing sessions down on max-prefix) stands out. + target.append( + &Self::NUM_PEER_DOWN_NOTIFICATIONS_METRIC, + Some(unit_name), + |records| { + for reason in PeerDownReason::ALL { + records.label_value( + &[ + ("router", router_id), + ("reason", reason.as_str()), + ], + metrics.num_peer_down_notifications + [reason.index()] + .load(SeqCst), + ); + } + }, + ); } } } diff --git a/src/units/bmp_tcp_in/state_machine/mod.rs b/src/units/bmp_tcp_in/state_machine/mod.rs index fcf3fa0..782a3bd 100644 --- a/src/units/bmp_tcp_in/state_machine/mod.rs +++ b/src/units/bmp_tcp_in/state_machine/mod.rs @@ -1,5 +1,6 @@ mod machine; mod metrics; +mod peer_down; mod processing; mod states; mod status_reporter; diff --git a/src/units/bmp_tcp_in/state_machine/peer_down.rs b/src/units/bmp_tcp_in/state_machine/peer_down.rs new file mode 100644 index 0000000..a20a0a4 --- /dev/null +++ b/src/units/bmp_tcp_in/state_machine/peer_down.rs @@ -0,0 +1,259 @@ +//! Why a monitored router's BGP session went down. +//! +//! A router sends a BMP Peer Down Notification (RFC 7854 §4.9) when one of +//! its own BGP sessions goes down. Besides the per-peer header identifying +//! the session, it carries a reason code and, depending on the reason, the +//! BGP NOTIFICATION the router sent or received, or the FSM event that +//! closed the session. This module turns that into a [`PeerDownInfo`] for +//! the ingress register. + +use bytes::Bytes; +use chrono::{DateTime, Utc}; +use routecore::bgp::message::notification::{CeaseSubcode, Details}; +use routecore::bmp::message::PeerDownNotification; + +use crate::ingress::register::{PeerDownInfo, PeerDownReason}; + +/// Offset of the reason code: common header (6) plus per-peer header (42). +const REASON_OFFSET: usize = 48; + +/// Offset of the NOTIFICATION data: header (19) plus code and subcode. +const NOTIFICATION_DATA_OFFSET: usize = 21; + +/// Decode a Peer Down Notification into a [`PeerDownInfo`]. +/// +/// `received` is used as the time when the router sent a zero per-peer +/// header timestamp, which some exporters do. +pub(super) fn peer_down_info( + msg: &PeerDownNotification, + received: DateTime, +) -> PeerDownInfo { + let pph_time = msg.per_peer_header().timestamp(); + let time = if pph_time.timestamp() > 0 { + pph_time + } else { + received + }; + + // routecore's PeerDownReason has no numeric value and folds RFC 9069's + // code 6 into Unknown, so read the code itself. The byte is always + // there: PeerDownNotification::check() parses it. + let reason_code = + msg.as_ref().get(REASON_OFFSET).copied().unwrap_or_default(); + let reason = PeerDownReason::from_code(reason_code); + + let mut info = PeerDownInfo { + time, + reason, + reason_code, + notification_code: None, + notification_subcode: None, + shutdown_communication: None, + fsm_event: None, + description: String::new(), + }; + + info.description = match reason { + PeerDownReason::LocalNotification + | PeerDownReason::RemoteNotification => { + let side = if reason == PeerDownReason::LocalNotification { + "local" + } else { + "remote" + }; + match msg.notification() { + Some(notification) + if notification.as_ref().len() + >= NOTIFICATION_DATA_OFFSET => + { + let details = notification.details(); + let [code, subcode] = details.raw(); + info.notification_code = Some(code); + info.notification_subcode = Some(subcode); + + // routecore's data() runs to the end of the buffer, + // which here is the end of the BMP message; bound it + // by the NOTIFICATION's own length. + let bytes = notification.as_ref(); + let end = + usize::from(notification.length()).min(bytes.len()); + let data = bytes + .get(NOTIFICATION_DATA_OFFSET..end) + .unwrap_or_default(); + info.shutdown_communication = + shutdown_communication(details, data); + + // Same wording as the NOTIFICATIONs of sessions netom + // terminates itself (bgp_tcp_in's last_error). + match &info.shutdown_communication { + Some(text) => format!( + "{side} NOTIFICATION: {details:?} \"{text}\"" + ), + None => format!("{side} NOTIFICATION: {details:?}"), + } + } + _ => format!("{side} NOTIFICATION (not included)"), + } + } + PeerDownReason::LocalFsm => { + info.fsm_event = msg.fsm(); + match info.fsm_event { + Some(event) => format!("local FSM event {event}"), + None => "local FSM event (not included)".to_string(), + } + } + PeerDownReason::RemoteNoData => { + "remote closed without NOTIFICATION".to_string() + } + PeerDownReason::PeerDeconfigured => "peer de-configured".to_string(), + PeerDownReason::LocalTlv => "local close with TLV data".to_string(), + PeerDownReason::Reserved | PeerDownReason::Unknown => { + format!("reason code {reason_code}") + } + }; + + info +} + +/// The shutdown communication of a Cease Administrative Shutdown or +/// Administrative Reset NOTIFICATION (RFC 8203, length limit raised to 255 +/// by RFC 9003): a length byte followed by that many bytes of UTF-8. +/// +/// A missing, empty or truncated communication yields `None`; invalid UTF-8 +/// is replaced rather than dropped, since this is for display only. +fn shutdown_communication(details: Details, data: &[u8]) -> Option { + if !matches!( + details, + Details::Cease( + CeaseSubcode::AdministrativeShutdown + | CeaseSubcode::AdministrativeReset + ) + ) { + return None; + } + let (&len, rest) = data.split_first()?; + let text = rest.get(..usize::from(len)).filter(|t| !t.is_empty())?; + Some(String::from_utf8_lossy(text).into_owned()) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::bgp::encode::{ + mk_bgp_notification_msg, mk_peer_down_notification_msg_with, + mk_per_peer_header, + }; + + fn decode(reason: u8, data: &[u8]) -> PeerDownInfo { + let pph = mk_per_peer_header("10.0.0.1", 65001); + let bytes = mk_peer_down_notification_msg_with(&pph, reason, data); + let msg = PeerDownNotification::from_octets(bytes).unwrap(); + peer_down_info(&msg, DateTime::::MIN_UTC) + } + + fn shutdown_data(text: &[u8]) -> Vec { + let mut data = vec![text.len() as u8]; + data.extend_from_slice(text); + data + } + + #[test] + fn remote_notification_with_shutdown_communication() { + let notification = + mk_bgp_notification_msg(6, 2, &shutdown_data(b"maintenance")); + let info = decode(3, ¬ification); + assert_eq!(info.reason, PeerDownReason::RemoteNotification); + assert_eq!(info.reason_code, 3); + assert_eq!(info.notification_code, Some(6)); + assert_eq!(info.notification_subcode, Some(2)); + assert_eq!( + info.shutdown_communication.as_deref(), + Some("maintenance") + ); + assert_eq!( + info.description, + "remote NOTIFICATION: Cease(AdministrativeShutdown) \ + \"maintenance\"" + ); + } + + #[test] + fn local_max_prefix_notification() { + let info = decode(1, &mk_bgp_notification_msg(6, 1, &[])); + assert_eq!(info.reason, PeerDownReason::LocalNotification); + assert_eq!(info.shutdown_communication, None); + assert_eq!( + info.description, + "local NOTIFICATION: Cease(MaximumPrefixesReached)" + ); + } + + #[test] + fn notification_data_is_bounded_by_its_length() { + // Bytes after the NOTIFICATION (here a stray trailer) must not be + // read as part of the shutdown communication. + let mut data = + mk_bgp_notification_msg(6, 2, &shutdown_data(b"bye")).to_vec(); + data.extend_from_slice(b"trailer"); + let info = decode(3, &data); + assert_eq!(info.shutdown_communication.as_deref(), Some("bye")); + } + + #[test] + fn truncated_or_invalid_shutdown_communication() { + // Claims 10 bytes, carries 3: not a valid communication. + let short = mk_bgp_notification_msg(6, 2, &[10, b'a', b'b', b'c']); + assert_eq!(decode(3, &short).shutdown_communication, None); + + let invalid = + mk_bgp_notification_msg(6, 4, &shutdown_data(&[b'o', 0xFF])); + assert_eq!( + decode(3, &invalid).shutdown_communication.as_deref(), + Some("o\u{FFFD}") + ); + + // Only Cease shutdown/reset carry a communication. + let other = mk_bgp_notification_msg(4, 0, &shutdown_data(b"x")); + let info = decode(1, &other); + assert_eq!(info.shutdown_communication, None); + assert_eq!(info.description, "local NOTIFICATION: HoldTimerExpired"); + } + + #[test] + fn fsm_event_and_reasons_without_data() { + let info = decode(2, &18u16.to_be_bytes()); + assert_eq!(info.reason, PeerDownReason::LocalFsm); + assert_eq!(info.fsm_event, Some(18)); + assert_eq!(info.description, "local FSM event 18"); + + for (code, reason, description) in [ + ( + 4, + PeerDownReason::RemoteNoData, + "remote closed without NOTIFICATION", + ), + (5, PeerDownReason::PeerDeconfigured, "peer de-configured"), + (6, PeerDownReason::LocalTlv, "local close with TLV data"), + (9, PeerDownReason::Unknown, "reason code 9"), + ] { + let info = decode(code, &[]); + assert_eq!(info.reason, reason); + assert_eq!(info.reason_code, code); + assert_eq!(info.description, description); + } + } + + #[test] + fn missing_notification_is_reported_as_such() { + let info = decode(3, &[]); + assert_eq!(info.notification_code, None); + assert_eq!(info.description, "remote NOTIFICATION (not included)"); + } + + #[test] + fn per_peer_header_timestamp_is_used_when_set() { + let info = decode(4, &[]); + // mk_per_peer_header stamps the current time, not 0. + assert!(info.time > DateTime::::MIN_UTC); + } +} diff --git a/src/units/bmp_tcp_in/state_machine/status_reporter.rs b/src/units/bmp_tcp_in/state_machine/status_reporter.rs index bc2ec18..34d14fd 100644 --- a/src/units/bmp_tcp_in/state_machine/status_reporter.rs +++ b/src/units/bmp_tcp_in/state_machine/status_reporter.rs @@ -10,6 +10,7 @@ use crate::{ common::status_reporter::{ AnyStatusReporter, Chainable, Named, UnitStatusReporter, }, + ingress::register::PeerDownReason, payload::RouterId, }; @@ -64,6 +65,20 @@ impl BmpStateMachineStatusReporter { } } + /// Count a Peer Down Notification: one of the router's BGP sessions + /// went down, for `reason`. Once per message, however many views of + /// the peer it takes down. + pub fn peer_down_notification( + &self, + router_id: Arc, + reason: PeerDownReason, + ) { + self.metrics + .router_metrics(router_id) + .num_peer_down_notifications[reason.index()] + .fetch_add(1, SeqCst); + } + pub fn peer_unknown(&self, router_id: Arc) { self.metrics .router_metrics(router_id) diff --git a/src/units/bmp_tcp_in/state_machine/tests.rs b/src/units/bmp_tcp_in/state_machine/tests.rs index ffd62cb..7ce0937 100644 --- a/src/units/bmp_tcp_in/state_machine/tests.rs +++ b/src/units/bmp_tcp_in/state_machine/tests.rs @@ -2516,3 +2516,261 @@ fn assert_invalid_msg_starts_with( ); } } + +// --------------------------------------------------------------------------- +// Why a monitored router's BGP session went down (Peer Down reasons) +// --------------------------------------------------------------------------- + +/// A Peer Down for `pph` with reason 3: the peer sent a Cease +/// Administrative Shutdown NOTIFICATION with a shutdown communication. +fn mk_remote_shutdown_peer_down_msg( + pph: &crate::bgp::encode::PerPeerHeader, + text: &str, +) -> BmpMsg { + let mut data = vec![text.len() as u8]; + data.extend_from_slice(text.as_bytes()); + let notification = + crate::bgp::encode::mk_bgp_notification_msg(6, 2, &data); + BmpMsg::from_octets( + crate::bgp::encode::mk_peer_down_notification_msg_with( + pph, + 3, + ¬ification, + ), + ) + .unwrap() +} + +#[test] +fn peer_down_records_why_the_session_went_down() { + use crate::ingress::register::{IngressState, PeerDownReason}; + + let register: Arc = Arc::default(); + let (pph, peer_up_msg_buf, real_pph) = + mk_peer_up_notification_msg_without_rfc4724_support( + "127.0.0.1", + 12345, + ); + let processor = mk_test_processor_with_register(®ister) + .process_msg( + Instant::now(), + mk_initiation_msg(TEST_ROUTER_SYS_NAME, TEST_ROUTER_SYS_DESC), + None, + ) + .next_state + .process_msg(Instant::now(), peer_up_msg_buf, None) + .next_state; + let ingress_id = if let BmpState::Dumping(p) = &processor { + p.details + .peer_states + .get_peer_ingress_id(&real_pph) + .unwrap() + } else { + unreachable!("expected Dumping after PeerUp"); + }; + assert_eq!(register.get(ingress_id).unwrap().last_down, None); + + let processor = processor + .process_msg( + Instant::now(), + mk_remote_shutdown_peer_down_msg(&pph, "maintenance"), + None, + ) + .next_state; + + let info = register.get(ingress_id).unwrap(); + assert_eq!(info.state, Some(IngressState::Disconnected)); + let last_down = info.last_down.expect("Peer Down must record a reason"); + assert_eq!(last_down.reason, PeerDownReason::RemoteNotification); + assert_eq!(last_down.notification_code, Some(6)); + assert_eq!(last_down.notification_subcode, Some(2)); + assert_eq!( + last_down.description, + "remote NOTIFICATION: Cease(AdministrativeShutdown) \"maintenance\"" + ); + + // The record outlives the outage: a reconnect rebinds the same + // ingress without wiping why it last went down. + let (_, peer_up_msg_buf, _) = + mk_peer_up_notification_msg_without_rfc4724_support( + "127.0.0.1", + 12345, + ); + processor.process_msg(Instant::now(), peer_up_msg_buf, None); + let info = register.get(ingress_id).unwrap(); + assert_eq!(info.state, Some(IngressState::Connected)); + assert_eq!( + info.last_down.map(|down| down.reason), + Some(PeerDownReason::RemoteNotification) + ); +} + +#[test] +fn peer_down_reason_is_recorded_on_every_view_of_the_peer() { + let register: Arc = Arc::default(); + let (pph_pre, peer_up_msg_buf, _) = + mk_peer_up_notification_msg_without_rfc4724_support( + "127.0.0.1", + 12345, + ); + let mut pph_post = mk_per_peer_header("127.0.0.1", 12345); + pph_post.peer_flags = 0x40; + + let processor = mk_test_processor_with_register(®ister) + .process_msg( + Instant::now(), + mk_initiation_msg(TEST_ROUTER_SYS_NAME, TEST_ROUTER_SYS_DESC), + None, + ) + .next_state + .process_msg(Instant::now(), peer_up_msg_buf, None) + .next_state + // Post-policy Route Monitoring with no matching PeerUp: a + // synthesized post-policy view of the same peer. + .process_msg(Instant::now(), mk_route_monitoring_msg(&pph_post), None) + .next_state; + let views: Vec<_> = register + .cloned_info() + .into_iter() + .filter(|(_, info)| { + info.ingress_type + == Some(crate::ingress::register::IngressType::BgpViaBmp) + }) + .map(|(id, _)| id) + .collect(); + assert_eq!(views.len(), 2, "pre-policy and synthesized post-policy"); + + processor.process_msg( + Instant::now(), + mk_remote_shutdown_peer_down_msg(&pph_pre, "bye"), + None, + ); + for id in views { + let last_down = register.get(id).unwrap().last_down; + assert_eq!( + last_down.map(|down| down.shutdown_communication), + Some(Some("bye".to_string())), + "view {id} must carry the reason" + ); + } +} + +#[test] +fn synthesized_views_do_not_inherit_a_stale_peer_down() { + let register: Arc = Arc::default(); + let (pph_pre, peer_up_msg_buf, real_pph) = + mk_peer_up_notification_msg_without_rfc4724_support( + "127.0.0.1", + 12345, + ); + let processor = mk_test_processor_with_register(®ister) + .process_msg( + Instant::now(), + mk_initiation_msg(TEST_ROUTER_SYS_NAME, TEST_ROUTER_SYS_DESC), + None, + ) + .next_state + .process_msg(Instant::now(), peer_up_msg_buf, None) + .next_state + .process_msg( + Instant::now(), + mk_remote_shutdown_peer_down_msg(&pph_pre, "first outage"), + None, + ) + .next_state; + + // The pre-policy view comes back carrying its last Peer Down... + let (_, peer_up_msg_buf, _) = + mk_peer_up_notification_msg_without_rfc4724_support( + "127.0.0.1", + 12345, + ); + let processor = processor + .process_msg(Instant::now(), peer_up_msg_buf, None) + .next_state; + let pre_id = if let BmpState::Dumping(p) = &processor { + p.details + .peer_states + .get_peer_ingress_id(&real_pph) + .unwrap() + } else { + unreachable!("expected Dumping after PeerUp"); + }; + assert!(register.get(pre_id).unwrap().last_down.is_some()); + + // ...but a post-policy view synthesized from it now has never been + // down, so it must not copy that record. + let mut pph_post = mk_per_peer_header("127.0.0.1", 12345); + pph_post.peer_flags = 0x40; + processor.process_msg( + Instant::now(), + mk_route_monitoring_msg(&pph_post), + None, + ); + let synthesized: Vec<_> = register + .cloned_info() + .into_iter() + .filter(|(id, info)| { + *id != pre_id + && info.ingress_type + == Some(crate::ingress::register::IngressType::BgpViaBmp) + }) + .collect(); + assert_eq!(synthesized.len(), 1); + assert_eq!(synthesized[0].1.last_down, None); +} + +#[test] +fn peer_down_notifications_are_counted_by_reason() { + let (pph, peer_up_msg_buf, _) = + mk_peer_up_notification_msg_without_rfc4724_support( + "127.0.0.1", + 12345, + ); + let processor = mk_test_processor() + .process_msg( + Instant::now(), + mk_initiation_msg(TEST_ROUTER_SYS_NAME, TEST_ROUTER_SYS_DESC), + None, + ) + .next_state + .process_msg(Instant::now(), peer_up_msg_buf, None) + .next_state + .process_msg( + Instant::now(), + mk_remote_shutdown_peer_down_msg(&pph, "maintenance"), + None, + ) + .next_state; + + let metrics = processor.status_reporter().unwrap().metrics().unwrap(); + let metrics = get_testable_metrics_snapshot(&metrics); + let count = |reason: &str| { + metrics.with_labels::( + "bmp_state_num_peer_down_notifications", + &[("router", "1"), ("reason", reason)], + ) + }; + assert_eq!(count("remoteNotification"), 1); + // Every reason has a series, so a rate() has something to start from. + assert_eq!(count("localNotification"), 0); + assert_eq!(count("peerDeconfigured"), 0); + + // A Peer Down for a peer that was never up is rejected, not counted. + let unknown = mk_per_peer_header("127.0.0.2", 12345); + let processor = processor + .process_msg( + Instant::now(), + mk_remote_shutdown_peer_down_msg(&unknown, "x"), + None, + ) + .next_state; + let metrics = processor.status_reporter().unwrap().metrics().unwrap(); + assert_eq!( + get_testable_metrics_snapshot(&metrics).with_labels::( + "bmp_state_num_peer_down_notifications", + &[("router", "1"), ("reason", "remoteNotification")], + ), + 1 + ); +} diff --git a/test-data/cli/bgp-neighbors.json b/test-data/cli/bgp-neighbors.json index 5a7753e..86d2b33 100644 --- a/test-data/cli/bgp-neighbors.json +++ b/test-data/cli/bgp-neighbors.json @@ -2,5 +2,6 @@ {"peerAddress":"10.1.0.1","peerAsn":null,"source":"bgp","state":"Idle","configured":true,"name":"PeerA","updatesReceived":0,"notificationsReceived":0}, {"peerAddress":"10.1.0.2","peerAsn":65002,"source":"bgp","state":"Established","configured":true,"name":"PeerB","ingressId":4,"upSeconds":8073,"holdTimeConfigured":90,"updatesReceived":13980,"notificationsReceived":1,"prefixesReceived":84211,"prefixesRejected":12,"dupPrefixAdvertisements":2410338}, {"peerAddress":"127.0.0.9","peerAsn":65003,"source":"bgp","state":"Active","configured":true,"connectMode":true,"name":"PeerDead","updatesReceived":0,"notificationsReceived":0,"lastError":"Connection refused (os error 111)"}, -{"peerAddress":"192.0.2.7","peerAsn":65100,"source":"bmp","state":"Established","configured":false,"routerId":"192.0.2.7","ingressId":3,"upSeconds":3731,"viaRouter":"10.99.0.1","viaIngressId":2,"peerRibType":"InPre"} +{"peerAddress":"192.0.2.7","peerAsn":65100,"source":"bmp","state":"Established","configured":false,"routerId":"192.0.2.7","ingressId":3,"upSeconds":3731,"viaRouter":"10.99.0.1","viaIngressId":2,"peerRibType":"InPre"}, +{"peerAddress":"192.0.2.8","peerAsn":65101,"source":"bmp","state":"Idle","configured":false,"routerId":"192.0.2.8","ingressId":6,"viaRouter":"10.99.0.1","viaIngressId":2,"peerRibType":"InPre","lastError":"remote NOTIFICATION: Cease(AdministrativeShutdown) \"maintenance\"","lastDownTime":"2026-08-12T05:58:10Z"} ]} diff --git a/test-data/cli/ingresses.json b/test-data/cli/ingresses.json index 60f38d6..2442155 100644 --- a/test-data/cli/ingresses.json +++ b/test-data/cli/ingresses.json @@ -3,5 +3,5 @@ {"id":3,"ingress_type":"bgpViaBmp","parent_ingress":2,"state":"Connected","remote_addr":"192.0.2.7","remote_asn":65100,"bgp_id":[10,99,0,1],"peer_rib_type":"inPre","session_up_time":"2026-08-12T06:00:00+00:00"}, {"id":4,"ingress_type":"bgpPath","parent_ingress":3,"state":"Connected","remote_addr":"192.0.2.7","remote_asn":65100,"path_id":1}, {"id":5,"ingress_type":"bgpPath","parent_ingress":3,"state":"Connected","remote_addr":"192.0.2.7","remote_asn":65100,"path_id":2}, -{"id":6,"ingress_type":"bgpViaBmp","parent_ingress":2,"state":"Disconnected","remote_addr":"192.0.2.8","remote_asn":65101,"peer_rib_type":"inPre"} +{"id":6,"ingress_type":"bgpViaBmp","parent_ingress":2,"state":"Disconnected","remote_addr":"192.0.2.8","remote_asn":65101,"peer_rib_type":"inPre","last_down":{"time":"2026-08-12T05:58:10Z","reason":"remoteNotification","reason_code":3,"notification_code":6,"notification_subcode":2,"shutdown_communication":"maintenance","description":"remote NOTIFICATION: Cease(AdministrativeShutdown) \"maintenance\""}} ]}