Skip to content

bootstrap_keys() auto-bootstrap path does not write bootstrap-info.json #539

@dmvt

Description

@dmvt

When KMS is configured with auto_bootstrap_domain (non-empty string in kms.toml), the auto-bootstrap code path in onboard_service.rs:bootstrap_keys() generates keys and stores them via keys.store(cfg), but never writes bootstrap-info.json.

This causes GetMeta to return bootstrap_info: null, which blocks the kms:set-info hardhat task from registering the KMS on-chain.

Steps to Reproduce

  1. Deploy KMS CVM with auto_bootstrap_domain set in kms.toml:

    [onboard]
    auto_bootstrap_domain = "kms.example.com"
    quote_enabled = true
  2. KMS starts, auto-bootstrap runs successfully (keys generated, certs created)

  3. Query GetMeta:

    curl -sk https://localhost:9100/prpc/KMS.GetMeta?json | jq .bootstrap_info
  4. Result: null

Root Cause

Auto-bootstrap (onboard_service.rs:330-339):

pub(crate) async fn bootstrap_keys(cfg: &KmsConfig) -> Result<()> {
    let keys = Keys::generate(
        &cfg.onboard.auto_bootstrap_domain,
        cfg.onboard.quote_enabled,
    )
    .await
    .context("Failed to generate keys")?;
    keys.store(cfg)?;  // Stores 8 files (keys, certs, rpc-domain) — NOT bootstrap-info.json
    Ok(())
}

Manual bootstrap (onboard_service.rs:54-78):

async fn bootstrap(self, request: BootstrapRequest) -> Result<BootstrapResponse> {
    let quote_enabled = self.state.config.onboard.quote_enabled;
    let keys = Keys::generate(&request.domain, quote_enabled)
        .await
        .context("Failed to generate keys")?;

    let k256_pubkey = keys.k256_key.verifying_key().to_sec1_bytes().to_vec();
    let ca_pubkey = keys.ca_key.public_key_der();
    let attestation = if quote_enabled {
        Some(attest_keys(&ca_pubkey, &k256_pubkey).await?)
    } else {
        None
    };

    let cfg = &self.state.config;
    let response = BootstrapResponse {
        ca_pubkey,
        k256_pubkey,
        attestation: attestation.unwrap_or_default(),
    };
    // Store the bootstrap info
    safe_write(cfg.bootstrap_info(), serde_json::to_vec(&response)?)?;  // ← Writes the file
    keys.store(cfg)?;
    Ok(response)
}

The manual path extracts public keys, optionally generates an attestation quote via attest_keys(), constructs a BootstrapResponse, and writes it to {cert_dir}/bootstrap-info.json (defined in config.rs:88-90 as self.cert_dir.join("bootstrap-info.json")). The auto-bootstrap path does none of this.

GetMeta (main_service.rs:323-326) silently returns None when the file is missing:

let bootstrap_info = fs::read_to_string(self.state.config.bootstrap_info())
    .ok()
    .and_then(|s| serde_json::from_str(&s).ok());

Impact

  • GetMeta returns bootstrap_info: null
  • kms:set-info hardhat task fails (cannot read/parse null bootstrap info)
  • KMS public keys and TDX attestation quote are never registered on-chain
  • Applications cannot verify KMS attestation through the smart contract
  • The entire on-chain trust anchor is missing

No workaround exists — bootstrap-info.json is only written in one place in the codebase (the manual bootstrap() RPC handler at line 75). No other service or background task populates it.

History

  • ae75c86e (2025-01-12): Added bootstrap_keys() — no bootstrap-info.json write
  • 6900f54e (2025-01-15): Added bootstrap-info.json write to the manual bootstrap() path only

The auto path has never written this file since it was introduced.

Suggested Fix

Add public key extraction, attestation quote generation, and bootstrap-info.json writing to bootstrap_keys(), mirroring the manual bootstrap() path:

pub(crate) async fn bootstrap_keys(cfg: &KmsConfig) -> Result<()> {
    let keys = Keys::generate(
        &cfg.onboard.auto_bootstrap_domain,
        cfg.onboard.quote_enabled,
    )
    .await
    .context("Failed to generate keys")?;
    keys.store(cfg)?;

    // Write bootstrap-info.json (same as manual bootstrap path)
    let k256_pubkey = keys.k256_key.verifying_key().to_sec1_bytes().to_vec();
    let ca_pubkey = keys.ca_key.public_key_der();
    let attestation = if cfg.onboard.quote_enabled {
        Some(attest_keys(&ca_pubkey, &k256_pubkey).await?)
    } else {
        None
    };
    let response = BootstrapResponse {
        ca_pubkey,
        k256_pubkey,
        attestation: attestation.unwrap_or_default(),
    };
    safe_write(cfg.bootstrap_info(), serde_json::to_vec(&response)?)?;

    Ok(())
}

Note: The attest_keys() function (line 351) generates a TDX attestation quote binding both the P256 CA public key and the secp256k1 key. This is needed for on-chain registration to include a valid attestation.

Also affected: feat/auto-onboard-kms branch

The auto_onboard_keys() function on the unmerged feat/auto-onboard-kms feature branch has the same gap — it generates keys and stores them but does not write bootstrap-info.json.

Environment

  • dstack version: v0.5.7
  • Platform: TDX-enabled server (Intel Xeon with TDX support)
  • Ubuntu 24.04 LTS

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions