Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,21 @@

## [Unreleased]

- Relay TLS and lifecycle polish:
- `rds-server`/`rds-relay` gain native TLS on the relay listener:
`--tls-cert/--tls-key` for PEM files (rustls `ring` provider), or
in-process Let's Encrypt via `--tls-acme-domain/--tls-acme-contact/
--tls-acme-cache` (TLS-ALPN-01, needs :443). HTTPS binds
`--tls-https-addr` (default 3443); the HTTP port keeps only the
captive-portal probe and `/healthz` for monitoring.
- All binaries shut down gracefully: `rds` closes its endpoint on
exit (no more iroh "ungraceful abort" on `rds id`/`ticket`), and
`rds-agent`/`rds-server`/`rds-relay` handle SIGINT+SIGTERM —
peers get `CONNECTION_CLOSE` and relay websockets close cleanly
under `systemctl stop` instead of dying mid-accept.
- `rds id` no longer binds a socket: it prints the key's identity
directly, so it is instant, offline, and refuses clearly when no
key file can be resolved instead of printing a fresh ephemeral id.
- Stability/latency hardening across the workspace:
- `rds-net`: the uni demux hands each accepted stream its own
tag-read task under a 10s bound — a peer that opens a stream and
Expand Down
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,8 @@ noq-proto = "1"
postcard = "1"
proptest = "1"
rand = "0.9"
rustls = { version = "0.23", default-features = false, features = ["std", "logging", "tls12", "ring"] }
rustls-pki-types = { version = "1", features = ["alloc"] }
rds-agent = { path = "crates/rds-agent" }
rds-audio = { path = "crates/rds-audio" }
rds-bench = { path = "crates/rds-bench" }
Expand Down
21 changes: 19 additions & 2 deletions crates/rds-cli/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,16 @@ async fn main() -> anyhow::Result<()> {
.map(|p| load_or_create_key(&p))
.transpose()?,
};
// `id` is a pure function of the key: no socket, no relay contact.
// (An unresolvable key previously fell back to an ephemeral identity,
// printing a different id on every run.)
if let Command::Id = cli.command {
let key = key.ok_or_else(|| {
anyhow::anyhow!("no endpoint key: pass --key-file or set HOME/XDG_CONFIG_HOME")
})?;
println!("{}", key.public());
return Ok(());
}
let backend = match cli.backend.as_str() {
"iroh" => rds_net::Backend::Iroh,
#[cfg(feature = "transport-noq")]
Expand All @@ -130,9 +140,16 @@ async fn main() -> anyhow::Result<()> {
})
.transpose()?;
match cli.command {
Command::Id => println!("{}", endpoint.id()),
Command::Id => unreachable!("handled before endpoint bind"),
Command::Ticket => {
endpoint.online().await;
// Bound the wait: an unreachable relay must not hang the
// command; the ticket still carries any resolved addresses.
if tokio::time::timeout(std::time::Duration::from_secs(15), endpoint.online())
.await
.is_err()
{
eprintln!("warning: relay unreachable after 15s; ticket may lack a relay address");
}
println!("{}", Ticket::of(&endpoint));
}
Command::Ping { target, count } => {
Expand Down
2 changes: 2 additions & 0 deletions crates/rds-relay/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ iroh-relay = { workspace = true, features = ["server"] }
postcard = { workspace = true, features = ["alloc", "use-std"] }
rds-core.workspace = true
rds-net = { workspace = true, optional = true }
rustls.workspace = true
rustls-pki-types.workspace = true
tokio.workspace = true
tracing.workspace = true
tracing-subscriber.workspace = true
Expand Down
121 changes: 119 additions & 2 deletions crates/rds-relay/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,13 @@ pub mod server;

use std::collections::HashSet;
use std::net::SocketAddr;
use std::path::PathBuf;
use std::sync::Arc;

use iroh::EndpointId;
use iroh_relay::server::{
Access, AccessControl, ClientRequest, ConnectionId, RelayConfig, Server, ServerConfig,
Access, AccessControl, AcmeConfig, CertConfig, ClientRequest, ConnectionId, RelayConfig,
Server, ServerConfig, TlsConfig,
};

/// Admit only endpoint ids on the relay allowlist.
Expand All @@ -40,14 +42,129 @@ impl AccessControl for AllowList {
fn on_disconnect(&self, _endpoint_id: EndpointId, _connection_id: ConnectionId) {}
}

/// TLS mode for the relay's HTTPS listener.
///
/// When set, the relay protocol moves to `https_addr`; `addr` keeps
/// serving only the plaintext captive-portal probe. Endpoints then dial
/// `--relay https://<host>:<port>`.
#[derive(Debug)]
pub enum RelayTls {
/// PEM certificate chain and private key files (e.g. certbot or an
/// internal CA). Relayed payloads stay end-to-end encrypted either
/// way; TLS here hides relay *usage* metadata from on-path observers.
Manual {
https_addr: SocketAddr,
cert_pem: PathBuf,
key_pem: PathBuf,
},
/// Let's Encrypt via in-process ACME (TLS-ALPN-01). Requires the
/// HTTPS listener on port 443, reachable from the internet.
LetsEncrypt {
https_addr: SocketAddr,
domains: Vec<String>,
/// ACME contacts; emails need a `mailto:` prefix.
contact: Vec<String>,
/// Directory caching issued certificates across restarts.
cache_dir: PathBuf,
/// Use the LE staging directory (for tests; certs untrusted).
staging: bool,
},
}

/// Spawn the iroh relay on `addr`, restricted to `allow` when non-empty.
/// Returns the bound server; dropping it stops the relay.
pub async fn serve(addr: SocketAddr, allow: Vec<EndpointId>) -> anyhow::Result<Server> {
pub async fn serve(
addr: SocketAddr,
allow: Vec<EndpointId>,
tls: Option<RelayTls>,
) -> anyhow::Result<Server> {
let mut relay_config = RelayConfig::new(addr);
if !allow.is_empty() {
relay_config.access = Arc::new(AllowList(allow.into_iter().collect()));
}
if let Some(tls) = tls {
relay_config.tls = Some(tls_config(tls)?);
}
let mut config = ServerConfig::default();
config.relay = Some(relay_config);
Ok(Server::spawn(config).await?)
}

/// Resolve CLI-shaped TLS flags into a [`RelayTls`].
///
/// `None` when neither manual nor ACME flags are present. ACME requires a
/// cache directory: without one every restart re-issues certificates and
/// hits Let's Encrypt rate limits.
pub fn tls_from_flags(
https_addr: SocketAddr,
cert: Option<PathBuf>,
key: Option<PathBuf>,
acme_domains: Vec<String>,
acme_contact: Vec<String>,
acme_cache: Option<PathBuf>,
acme_staging: bool,
) -> anyhow::Result<Option<RelayTls>> {
match (cert, key, acme_domains.is_empty()) {
(Some(cert_pem), Some(key_pem), _) => Ok(Some(RelayTls::Manual {
https_addr,
cert_pem,
key_pem,
})),
(None, None, false) => {
let cache_dir = acme_cache
.ok_or_else(|| anyhow::anyhow!("--tls-acme-cache is required for ACME"))?;
Ok(Some(RelayTls::LetsEncrypt {
https_addr,
domains: acme_domains,
contact: acme_contact,
cache_dir,
staging: acme_staging,
}))
}
(None, None, true) => Ok(None),
// clap `requires`/`conflicts_with` guard the halves.
_ => anyhow::bail!("--tls-cert and --tls-key must be given together"),
}
}

fn tls_config(tls: RelayTls) -> anyhow::Result<TlsConfig> {
// ring: pure-Rust provider already in the tree; aws-lc-rs would add a
// C build dependency for no gain here.
let provider = Arc::new(rustls::crypto::ring::default_provider());
let builder = rustls::ServerConfig::builder_with_provider(provider)
.with_safe_default_protocol_versions()?;
Ok(match tls {
RelayTls::Manual {
https_addr,
cert_pem,
key_pem,
} => {
use rustls_pki_types::pem::PemObject;
let certs = rustls_pki_types::CertificateDer::pem_file_iter(&cert_pem)?
.collect::<Result<Vec<_>, _>>()?;
let key = rustls_pki_types::PrivateKeyDer::from_pem_file(&key_pem)?;
let server_config = builder.with_no_client_auth().with_single_cert(certs, key)?;
TlsConfig::new(https_addr, CertConfig::Manual { server_config })
}
RelayTls::LetsEncrypt {
https_addr,
domains,
contact,
cache_dir,
staging,
} => {
let acme = AcmeConfig::letsencrypt(!staging)
.domains(domains)
.contact(contact)
.cache_path(cache_dir);
TlsConfig::new(
https_addr,
CertConfig::LetsEncrypt {
acme_config: acme,
// iroh-relay injects the ACME cert resolver itself.
server_config_builder: builder.with_no_client_auth(),
},
)
}
})
}
37 changes: 36 additions & 1 deletion crates/rds-relay/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,29 @@ struct Cli {
/// Restrict relay use to these endpoint ids. Empty = open relay.
#[arg(long = "allow")]
allow: Vec<String>,
/// PEM certificate chain enabling HTTPS relaying. Requires --tls-key.
/// Unprivileged services cannot bind :443 — use --tls-https-addr ≥1024.
#[arg(long, requires = "tls_key")]
tls_cert: Option<std::path::PathBuf>,
/// PEM private key for --tls-cert.
#[arg(long, requires = "tls_cert")]
tls_key: Option<std::path::PathBuf>,
/// HTTPS bind address when TLS is enabled. ACME validation needs :443.
#[arg(long, default_value = "0.0.0.0:3443")]
tls_https_addr: SocketAddr,
/// Let's Encrypt domain via in-process ACME (TLS-ALPN-01, needs :443
/// reachable). Repeatable. Mutually exclusive with --tls-cert.
#[arg(long, conflicts_with = "tls_cert")]
tls_acme_domain: Vec<String>,
/// ACME contact (repeatable); emails need a `mailto:` prefix.
#[arg(long, requires = "tls_acme_domain")]
tls_acme_contact: Vec<String>,
/// Directory caching issued ACME certificates across restarts.
#[arg(long, requires = "tls_acme_domain")]
tls_acme_cache: Option<std::path::PathBuf>,
/// Use the Let's Encrypt staging directory (untrusted certs; testing).
#[arg(long)]
tls_acme_staging: bool,
}

#[tokio::main]
Expand All @@ -34,11 +57,23 @@ async fn main() -> anyhow::Result<()> {
.map(|s| s.parse())
.collect::<Result<_, _>>()?;

let server = rds_relay::serve(cli.addr, allow).await?;
let tls = rds_relay::tls_from_flags(
cli.tls_https_addr,
cli.tls_cert,
cli.tls_key,
cli.tls_acme_domain,
cli.tls_acme_contact,
cli.tls_acme_cache,
cli.tls_acme_staging,
)?;
let server = rds_relay::serve(cli.addr, allow, tls).await?;
println!(
"relay listening on http://{}",
server.http_addr().expect("relay config enabled")
);
if let Some(addr) = server.https_addr() {
println!("relay tls url: https://{addr}");
}
shutdown_signal().await;
// Graceful stop: close listener + client websockets instead of
// letting attached endpoints hit a silent RST.
Expand Down
37 changes: 36 additions & 1 deletion crates/rds-server/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,29 @@ struct Cli {
/// JSON file with the initial estate-signed registry snapshot.
#[arg(long)]
registry: Option<PathBuf>,
/// PEM certificate chain enabling HTTPS relaying. Requires --tls-key.
/// Unprivileged services cannot bind :443 — use --tls-https-addr ≥1024.
#[arg(long, requires = "tls_key")]
tls_cert: Option<PathBuf>,
/// PEM private key for --tls-cert.
#[arg(long, requires = "tls_cert")]
tls_key: Option<PathBuf>,
/// HTTPS bind address when relay TLS is enabled. ACME needs :443.
#[arg(long, default_value = "0.0.0.0:3443")]
tls_https_addr: SocketAddr,
/// Let's Encrypt domain via in-process ACME (TLS-ALPN-01, needs :443
/// reachable). Repeatable. Mutually exclusive with --tls-cert.
#[arg(long, conflicts_with = "tls_cert")]
tls_acme_domain: Vec<String>,
/// ACME contact (repeatable); emails need a `mailto:` prefix.
#[arg(long, requires = "tls_acme_domain")]
tls_acme_contact: Vec<String>,
/// Directory caching issued ACME certificates across restarts.
#[arg(long, requires = "tls_acme_domain")]
tls_acme_cache: Option<PathBuf>,
/// Use the Let's Encrypt staging directory (untrusted certs; testing).
#[arg(long)]
tls_acme_staging: bool,
}

#[tokio::main]
Expand Down Expand Up @@ -103,8 +126,20 @@ async fn main() -> anyhow::Result<()> {
.iter()
.map(|s| s.parse())
.collect::<Result<_, _>>()?;
let relay = rds_relay::serve(cli.relay_addr, allow).await?;
let tls = rds_relay::tls_from_flags(
cli.tls_https_addr,
cli.tls_cert,
cli.tls_key,
cli.tls_acme_domain,
cli.tls_acme_contact,
cli.tls_acme_cache,
cli.tls_acme_staging,
)?;
let relay = rds_relay::serve(cli.relay_addr, allow, tls).await?;
info!(addr = %relay.http_addr().expect("relay config enabled"), "relay listening");
if let Some(addr) = relay.https_addr() {
info!(%addr, "relay tls listening");
}

shutdown_signal().await;
// Graceful stop: close listener + client websockets instead of
Expand Down
5 changes: 5 additions & 0 deletions deploy/systemd/rds-server.service
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,11 @@ ExecStart=/usr/local/bin/rds-server \
# --registry-key <base32 verifying key> --registry /var/lib/rds/registry.json
# --allow <endpoint-id>… (empty = open relay — do NOT expose open
# relays on a public IP beyond short-lived bring-up)
# Optional native relay TLS (docs/deployment.md):
# --tls-cert /var/lib/rds/cert.pem --tls-key /var/lib/rds/key.pem
# or --tls-acme-domain … --tls-acme-cache /var/lib/rds/acme
# ACME's TLS-ALPN-01 needs :443 reachable — that also requires
# AmbientCapabilities=CAP_NET_BIND_SERVICE below (default :3443).

Environment=RUST_LOG=info
Restart=on-failure
Expand Down
30 changes: 24 additions & 6 deletions docs/deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,12 +68,30 @@ endpoint's discovered addresses).
(WebSocket over HTTP) on 3340 in plaintext: relayed payloads are
end-to-end-encrypted QUIC the relay cannot read, and relay admission is
keyed by `EndpointId` signature challenge, so a network MITM can only
disrupt, not decrypt. For defence-in-depth on a public IP, terminate TLS
in front of 3340 with any TCP-level TLS terminator (nginx `stream`,
haproxy) and give endpoints `--relay https://<host>:<tls-port>`; the
relay protocol rides WebSocket inside TLS unchanged. The embedded
iroh-relay also supports ACME natively — wiring `TlsConfig` through
`rds_relay::serve` is future work, not required for launch.
disrupt, not decrypt. For defence-in-depth on a public IP the relay
supports native TLS in two modes:

```bash
# manual PEM (certbot, internal CA, any provider)
rds-server --tls-cert /etc/rds/cert.pem --tls-key /etc/rds/key.pem

# or in-process Let's Encrypt (TLS-ALPN-01 — requires port 443
# reachable from the internet, so a privileged bind or a redirect)
rds-server --tls-acme-domain relay.example.com \
--tls-acme-contact mailto:ops@example.com \
--tls-acme-cache /var/lib/rds/acme
```

HTTPS binds `--tls-https-addr` (default `0.0.0.0:3443`; the
unprivileged unit cannot bind 443 — for 443 add
`AmbientCapabilities=CAP_NET_BIND_SERVICE` to the unit). The plaintext
HTTP port keeps serving only the captive-portal probe. Endpoints then
dial `--relay https://<host>:<port>`; the relay protocol rides WebSocket
inside TLS unchanged. A TCP-level TLS terminator (nginx `stream`,
haproxy) in front of 3340 remains a valid alternative.

The relay serves `GET /healthz` → `200` on its HTTP(S) listener for
load-balancer and monitoring probes.

**No tunnel/VPN dependency.** The design assumes only *outbound*
connectivity from endpoints: tcp/3340 (relay) + tcp/3341 (directory) +
Expand Down
Loading