diff --git a/.agents/skills/README.md b/.agents/skills/README.md index b688d36..e07c74a 100644 --- a/.agents/skills/README.md +++ b/.agents/skills/README.md @@ -6,6 +6,6 @@ has the instructions in Markdown. | Skill | Use it to | | --- | --- | -| [`install-berth`](install-berth/SKILL.md) | Install the berth client, a berth server, an Incus and Docker host, or Ubuntu on a server from a rescue system | +| [`install-berth`](install-berth/SKILL.md) | Install the berth client, a berth server, a node for Incus and Docker machines, or Ubuntu on a server from a rescue system | `.claude/skills` is a link to this folder, so Claude Code finds the same skills. diff --git a/.agents/skills/install-berth/SKILL.md b/.agents/skills/install-berth/SKILL.md index bfeda7a..55399e6 100644 --- a/.agents/skills/install-berth/SKILL.md +++ b/.agents/skills/install-berth/SKILL.md @@ -1,6 +1,6 @@ --- name: install-berth -description: "Use when installing or setting up berth. That covers the client, a berth server and an Incus and Docker host for machines. It also covers a server installed from a provider's rescue system. Covers which recipe to pick, the exact commands, what is destructive, and how to hand the server over to the user." +description: "Use when installing or setting up berth. That covers the client, a berth server and a node, a server that hosts Incus and Docker machines. It also covers a server installed from a provider's rescue system. Covers which recipe to pick, the exact commands, what is destructive, and how to hand the server over to the user." --- # Install berth @@ -14,8 +14,9 @@ terminal. A set-up has these pieces: | --- | --- | | `berth` client | The program on the user's machine. It has a terminal UI and commands. | | berth server | Optional. `berth serve`, usually in a Docker container, reached over a tailnet with TLS. It keeps the hosts, profiles and secrets. | -| Host | A machine berth works with, at one of three levels. `reachable` means berth only logs in, `controller` adds berthd and box, and `workstation` is set up from a profile. A machine that exists is added with `berth host add NAME ADDRESS --user USER`. | -| Provisioner | Creates the machine for a host, such as on Docker or Incus. Plugins bring the types `docker/container`, `incus/vm` and `incus/container`, and berth runs each provisioner under a name of its own, registered with `berth provisioner add NAME TYPE`. The `incus-host` recipe registers `incus-vm`, `incus-container` and `docker`. `berth provisioner` lists them, and the Provisioners tab of the terminal UI (key `2`) shows each one's page. | +| Host | A machine berth logs in to over SSH. Its type is one of four: `ssh` means berth only logs in, `managed` adds berthd and box, `node` is a server berth applies without a desktop, and `workstation` is set up from a profile. A machine that exists is added with `berth host add NAME ADDRESS --user USER`. | +| Node | A server for machines. Its features `incus/server` and `docker/engine` install Incus and Docker, and after the apply berth registers their provisioners, such as `big-incus-vm`, `big-incus-container` and `big-docker-container`. | +| Provisioner | Creates the machine for a host, such as on Docker or Incus. Plugins bring the types `docker/container`, `incus/vm` and `incus/container`, and berth runs each provisioner under a name of its own, registered with `berth provisioner add NAME TYPE` or by a node. `berth provisioner list` lists them, and the Provisioners tab of the terminal UI (key `2`) shows each one's page. | ## Where to look things up @@ -24,21 +25,23 @@ terminal. A set-up has these pieces: | `README.md` | Installing the client | | `docs/content/docs/(guide)/start/quick-start.mdx` | A first host from the terminal UI | | `docs/content/docs/(guide)/use/server.mdx` | Running a server and connecting to it | -| `docs/content/docs/(guide)/use/levels.mdx` | Host levels, and adding a machine by its address | +| `docs/content/docs/(guide)/use/types.mdx` | Host types, and adding a machine by its address | +| `docs/content/docs/(guide)/use/nodes.mdx` | Nodes, which are servers for Incus and Docker machines, and taking in one that exists | | `docs/content/docs/(guide)/use/store.mdx` | Where a server keeps its state, and restoring it | | `docs/content/docs/(guide)/deploy/index.mdx` | The install methods | | `docs/content/docs/(guide)/deploy/provisioners.mdx` | Creating a host's machine with a provisioner | | `recipes/README.md` | The list of recipes | | `recipes/*/README.md` | One recipe, with what it needs, its command and its settings | -| `plugins/incus/guide.md`, `plugins/docker/guide.md` | The provisioners and their arguments | +| `plugins/incus/guide.md`, `plugins/docker/guide.md` | The provisioners and their arguments, and the settings of the features `incus/server` and `docker/engine` | | `AGENTS.md` | Working in this repository | ## Choosing a set-up | The user has or wants | Follow | | --- | --- | -| A rented dedicated server whose installer fails | `rescue-install`, then `incus-host` | -| An empty server to host many machines | `incus-host`. Add `berth_server=true` to run the server there too. | +| A rented dedicated server whose installer fails | `rescue-install`, then add it as a node | +| An empty server to host many machines | Add it as a node with the features `incus/server` and `docker/engine`. To run the server there too, `berth-server` with `berth_server_node=true` first. | +| A server the old `incus-host` recipe set up | Add it as a node with the same features. They keep its pools, bridge, trust and guests. | | A small VM just for the server | `berth-server` | | Only a client | The install instructions in `README.md` and the quick start | @@ -69,50 +72,37 @@ Find the disks with `lsblk` first. Afterwards the user sets the server to start from its disk in the provider's panel, restarts it and logs in over SSH. A second run would wipe the disks again, so there is no reason to repeat it. -### `incus-host` +### A node for machines -Turns an empty Ubuntu server into a host for machines. It installs Incus with QEMU and Docker, -and sets up a btrfs storage pool, the bridge `incusbr0` and a `ufw` firewall. berth creates -its hosts there as Incus guests. It needs Ubuntu 24.04 or later, run as root on -the server, from a fresh install. It does not wipe anything. +berth applies a node itself, with no recipe. It needs Ubuntu 24.04 or later on +x86_64 and a login as root or as an account with sudo. On the user's own +machine, or on a berth connected to their server, with the made-up name `big`: ```bash -curl -fsSL https://github.com/helightdev/berth/archive/refs/heads/main.tar.gz \ - | tar xz --strip-components=1 --wildcards '*/recipes' -cd recipes/incus-host -sudo ./install.sh -e tailscale_auth_key=tskey-auth-EXAMPLE -``` - -To run a berth server on the same machine: - -```bash -sudo ./install.sh -e berth_server=true -e tailscale_auth_key=tskey-auth-EXAMPLE +berth host add big 192.0.2.20 --user root --auth ssh --type node \ + --feature docker/engine --feature incus/server +berth apply big ``` -To also install Sysbox, so that berth's Docker provisioner can run containers -with `runtime=sysbox` instead of privileged, add `-e sysbox=true` on the first -run. It downloads a package and checks it against a sum in `vars.yml`. - -A second run changes nothing in place. It leaves a server alone that has a -storage pool, so it never touches existing guests. Settings are in `vars.yml`, -and the README lists them. - -### Add the server to berth - -A server from `rescue-install` and `incus-host` takes the user's public key for -`root`. On the user's own machine, with the made-up name `big`: - -```bash -berth host add big 192.0.2.20 --user root --auth ssh -berth host install big -``` - -`host add` saves a reachable host. The default `--auth berth` uses berth's own -key, which berth installs on first contact with a password or a private key that -logs in once. `--auth ssh` uses the user's own SSH configuration and agent, and -works with a local berth only. `host install` puts the controller alone on the -server, for status, updates, a reboot and commands. Sessions then run as `root`. -Such a server is never applied as a workstation. +`--auth ssh` uses the user's own SSH configuration and agent, and works with a +local berth only. The default `--auth berth` uses berth's own key, which berth +installs on first contact with a password or a private key that logs in once. +The apply gives the SSH user berth's key and allows only keys for SSH. Root +keeps a login with keys. Its `ufw` firewall keeps these open: SSH, the +tailnet, the guests' bridge `incusbr0` and the Incus API on port 8444. berth +then registers the provisioners. Check with `berth run big status` and +`berth provisioner list`. + +The features never erase a disk. They make an Incus storage pool in a folder, +or use one the user names in `devbox_incus_pool`. A server with Incus or Docker +already keeps its pools, networks, trusted clients, containers and guests, and +a provisioner registered for it by hand becomes the node's when it names the +node's address. The apply names any it left alone, with the command that makes +it the node's: `berth provisioner run NAME adopt node=big`. + +Virtual machines need KVM. On a rented virtual server without it, set +`devbox_incus_virtual_machines` to `false` in `ansible_vars`. For Sysbox beside +Docker, set `devbox_docker_sysbox` to `true` on the first apply. ### `berth-server` @@ -123,15 +113,29 @@ port 7777 on this machine only, Tailscale, `tailscale serve` with TLS and a ```bash curl -fsSL https://github.com/helightdev/berth/archive/refs/heads/main.tar.gz \ - | tar xz --strip-components=1 --wildcards '*/recipes' + | tar xz --strip-components=1 --wildcards '*/recipes' \ + '*/provision/ansible/roles/devbox/files/berth-server-upgrade.py' cd recipes/berth-server sudo ./install.sh -e tailscale_auth_key=tskey-auth-EXAMPLE ``` -A second run changes nothing in place. It leaves the container `berth` as it -is and starts it when it was stopped. To update, run `sudo docker pull -ghcr.io/helightdev/berth`, `sudo docker rm -f berth`, then `sudo ./install.sh`. -The data in `/srv/berth` stays. +To let the server apply its own machine as a node, add +`-e berth_server_node=true`. The server's own key then logs in to it as +`root`. The container finds the machine at the name `node` and has the Docker +socket. Add the +node from the server with `berth host add big node --user root --type node`, +then as in the node's section above. + +A second run changes nothing in place. The container `berth` keeps its image, +data and options. A stopped one is started. To update the +server, fetch the recipe of the new version and run +`sudo ./install.sh -e berth_server_update=true`. It makes the container again +from the new image with the options of the old one, and starts the old one +again if the new one fails. On a node, the node's action `upgrade-server` does +the same. Never update with `docker rm -f berth`: a new container loses the +options passed by hand, such as the name `incus-host` the provisioners of +berth 0.5 reach. A server of berth 0.5 is updated before its clients and hosts: +the page Upgrading from 0.5 in the docs has the order. ## Things that go wrong @@ -139,11 +143,13 @@ The data in `/srv/berth` stays. | --- | --- | | The tailnet address does not work | HTTPS certificates must be on for the tailnet. Or set `berth_server_tailscale_serve` to `false` to keep the server on the machine only. | | The image does not pull | A private image needs `sudo docker login ghcr.io` before the run. | -| Virtual machines fail on a rented virtual server | They need KVM. Set `incus_virtual_machines` to `false` and use containers. | -| The bridge clashes with the server's network | Change `incus_bridge_address` if `10.77.0.0/24` is taken. | -| Locked out of SSH | With `ssh_from` set to one address, SSH from anywhere else is refused. The provider's console is the way back in. | +| Virtual machines fail on a rented virtual server | They need KVM. Set `devbox_incus_virtual_machines` to `false` and use containers. | +| The bridge clashes with the server's network | Change `devbox_incus_bridge_address` if `10.77.0.0/24` is taken. | +| Locked out of SSH | With `ssh_from` in `berth-server`, or `devbox_ssh_from` on a node, set to one address, SSH from anywhere else is refused. The provider's console is the way back in. | +| A storage pool on its own disk | The features never erase a disk. Create it with `incus storage create` and name it in `devbox_incus_pool`. | | Docker ports are open to the world | Docker publishes past `ufw`. Publish on `127.0.0.1` or the tailnet address. | | The enrollment key leaks | Pass `tailscale_auth_key` on the command line. Never save it in a file you commit. | +| Hosts berth sets up do not join the tailnet | berth makes their keys itself, with the secret `tailnet_api_key`: a Tailscale OAuth client as `CLIENT_ID:CLIENT_SECRET`, a Tailscale API access token or a Headscale API key. Ask the user to set it with `berth secret set tailnet_api_key`. Only the recipes' own machine takes a key by hand, because the recipe runs before berth does. | | A rescue install refuses to run | It refuses a system that did not start with UEFI, and a disk the running system is on. | ## Hand the server over to the user @@ -154,16 +160,16 @@ The server starts with a setup key. On the server machine this prints a one-time key that is valid for 24 hours: ```bash -sudo docker exec berth berth setup-key +sudo docker exec berth berth server setup-key ``` The recipes make a key on the first run and print it at the end. Give the user the key and the address, which `tailscale serve status` shows: -1. On their own machine they run `berth connect https://THIS-MACHINE.YOUR-TAILNET.ts.net` and enter the key. -2. Or they open `https://THIS-MACHINE.YOUR-TAILNET.ts.net/#setup-key=KEY` in a browser. +1. On their own machine they run `berth login https://THIS-MACHINE.YOUR-TAILNET.ts.net` and enter the key. It works on Windows and macOS too. +2. Then `berth ui` opens the screens, and the other commands use that server. -The key ends when they choose Done, or when `berth connect` has saved the token. +The key ends when they choose Done, or when `berth login` has saved the token. **Return the setup key and the address to the user. Never read, store or print the permanent token in `serve.token`.** diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index db0cf93..eda4ac5 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -109,9 +109,9 @@ A pull request is ready when: vale --minAlertLevel=warning "docs/content/docs/(guide)" README.md AGENTS.md ``` -- The terminal client and the page in a browser do the same things. A feature - added to one is added to the other, with the shared part in - `crates/berth-tui`. +- The screens do everything through the API of a server, whichever it is. A + feature of theirs is an operation or an action of the server first, and + then their part in `crates/berth-page` and `crates/berth-tui`. - Commits follow [Conventional Commits](https://www.conventionalcommits.org/), with a short subject that says what changed. - The pull request links its issue and stays within it. Leave unrelated diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index adaea1f..d38fad9 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -11,5 +11,5 @@ Issue: # - [ ] `ci/run.sh all` passes - [ ] Tried on a real host as well as in the containers (say which kind above) - [ ] Documentation is updated and Vale doesn't report warnings -- [ ] The terminal client and the page in a browser both have it, where it applies +- [ ] The screens do it through the API, and the client build for macOS and Windows still checks - [ ] I read every line of this change myself diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 51196b3..f13437d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -81,7 +81,7 @@ jobs: key: e2e-${{ hashFiles('Cargo.lock', 'ci/dev.Dockerfile') }}-${{ github.sha }} restore-keys: e2e-${{ hashFiles('Cargo.lock', 'ci/dev.Dockerfile') }}- - name: Build berth - run: ci/run.sh run cargo build --locked -p berth --features web + run: ci/run.sh run cargo build --locked -p berth # berth, and the berthd and box it puts into the role's files. A tar # keeps their modes, which an artifact does not. - name: Pack what the shards test diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index dfddb30..5dfbd9b 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -24,16 +24,10 @@ jobs: - uses: actions/setup-node@v4 with: node-version: 24 - # The site has the page of berth serve under /web, which building berth makes. - - uses: dtolnay/rust-toolchain@stable - with: - targets: x86_64-unknown-linux-musl, wasm32-unknown-unknown - - uses: Swatinem/rust-cache@v2 - run: npm install --no-audit --no-fund && npm run build working-directory: docs env: DOCS_BASE_PATH: ${{ steps.pages.outputs.base_path }} - - run: test -f docs/out/web/berth-web_bg.wasm - uses: actions/upload-pages-artifact@v5 with: path: docs/out diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 23e0059..fa65a69 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -8,97 +8,30 @@ on: permissions: contents: read jobs: + # berth for each system, built on that system so that its version can be + # run there. On Linux it carries berthd and box for the hosts, which are + # built for x86_64 wherever berth is; on macOS and Windows it is a client + # of a berth serve elsewhere. binary: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v7 - # A tag vX.Y.Z builds version X.Y.Z. Any other build says the version in - # Cargo.toml, which is that of the latest release. - - name: The version - run: | - if [ "$GITHUB_REF_TYPE" = tag ]; then - version=${GITHUB_REF_NAME#v} - if ! echo "$version" | grep -Eq '^[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$'; then - echo "::error::$GITHUB_REF_NAME is not a version tag such as v1.2.0 or v1.2.0-rc.1" - exit 1 - fi - echo "BERTH_BUILD_VERSION=$version" >> "$GITHUB_ENV" - fi - - uses: dtolnay/rust-toolchain@stable - with: - targets: x86_64-unknown-linux-musl, wasm32-unknown-unknown - - name: musl compiler for the bundled SQLite - run: sudo apt-get update && sudo apt-get install -y --no-install-recommends musl-tools - - uses: Swatinem/rust-cache@v2 - - name: Build berth, with berthd and box inside - run: cargo build --release --locked --target x86_64-unknown-linux-musl -p berth - - name: Pack - run: | - mkdir dist - cp target/x86_64-unknown-linux-musl/release/berth LICENSE-MIT LICENSE-APACHE dist/ - tar -czf berth-x86_64-linux.tar.gz -C dist berth LICENSE-MIT LICENSE-APACHE - sha256sum berth-x86_64-linux.tar.gz > berth-x86_64-linux.tar.gz.sha256 - cp target/host/x86_64-unknown-linux-musl/release/berthd target/host/x86_64-unknown-linux-musl/release/box dist/ - - name: Build berth with the page for a browser - run: cargo build --release --locked --target x86_64-unknown-linux-musl -p berth --features web - - name: Pack it - run: | - cp target/x86_64-unknown-linux-musl/release/berth dist/berth - tar -czf berth-web-x86_64-linux.tar.gz -C dist berth LICENSE-MIT LICENSE-APACHE - sha256sum berth-web-x86_64-linux.tar.gz > berth-web-x86_64-linux.tar.gz.sha256 - - name: Check the versions - # berth in both archives, and the berthd and box it carries, all say the same. - run: | - want=${BERTH_BUILD_VERSION:-$(sed -n '/^\[workspace.package\]/,/^\[/s/^version = "\(.*\)"/\1/p' Cargo.toml)} - mkdir check check-web - tar -xzf berth-x86_64-linux.tar.gz -C check berth - tar -xzf berth-web-x86_64-linux.tar.gz -C check-web berth - for program in check/berth check-web/berth dist/berthd dist/box; do - said=$("$program" --version) - echo "$program: $said" - case "$said" in - "${program##*/} $want" | "${program##*/} $want "*) ;; - *) echo "::error::$program says $said, not $want"; exit 1 ;; - esac - done - rm -r check check-web - - uses: actions/upload-artifact@v4 - with: - name: berth-x86_64-linux - path: | - berth-x86_64-linux.tar.gz - berth-x86_64-linux.tar.gz.sha256 - berth-web-x86_64-linux.tar.gz - berth-web-x86_64-linux.tar.gz.sha256 - - uses: actions/upload-artifact@v4 - with: - name: host-programs-x86_64-linux - path: | - dist/berthd - dist/box - # berth-remote, the page of berth serve in a terminal, to use a server's - # hosts from another machine. Each is built on the system it is for, so - # that its version can be run there. - remote: strategy: fail-fast: false matrix: include: - - runner: windows-latest - target: x86_64-pc-windows-msvc - asset: berth-remote-x86_64-windows.zip - - runner: macos-15 - target: aarch64-apple-darwin - asset: berth-remote-aarch64-macos.tar.gz - - runner: macos-15-intel - target: x86_64-apple-darwin - asset: berth-remote-x86_64-macos.tar.gz - runner: ubuntu-latest target: x86_64-unknown-linux-musl - asset: berth-remote-x86_64-linux.tar.gz + asset: berth-x86_64-linux.tar.gz - runner: ubuntu-24.04-arm target: aarch64-unknown-linux-musl - asset: berth-remote-aarch64-linux.tar.gz + asset: berth-aarch64-linux.tar.gz + - runner: macos-15 + target: aarch64-apple-darwin + asset: berth-aarch64-macos.tar.gz + - runner: macos-15-intel + target: x86_64-apple-darwin + asset: berth-x86_64-macos.tar.gz + - runner: windows-latest + target: x86_64-pc-windows-msvc + asset: berth-x86_64-windows.zip runs-on: ${{ matrix.runner }} defaults: run: @@ -106,11 +39,14 @@ jobs: env: # The C runtime goes into the program, so that Windows needs nothing installed. CARGO_TARGET_X86_64_PC_WINDOWS_MSVC_RUSTFLAGS: -C target-feature=+crt-static - # ring's C is built against musl too, so that the program runs on any Linux. + # The bundled SQLite and ring's C are built against musl too, so that + # the program runs on any Linux. CC_x86_64_unknown_linux_musl: musl-gcc CC_aarch64_unknown_linux_musl: musl-gcc steps: - uses: actions/checkout@v7 + # A tag vX.Y.Z builds version X.Y.Z. Any other build says the version in + # Cargo.toml, which is that of the latest release. - name: The version run: | if [ "$GITHUB_REF_TYPE" = tag ]; then @@ -123,20 +59,26 @@ jobs: fi - uses: dtolnay/rust-toolchain@stable with: - targets: ${{ matrix.target }} - - name: musl compiler for ring + # On Linux the host programs are built for x86_64 as well. + targets: ${{ matrix.target }}${{ runner.os == 'Linux' && ',x86_64-unknown-linux-musl' || '' }} + - name: musl compiler if: runner.os == 'Linux' run: sudo apt-get update && sudo apt-get install -y --no-install-recommends musl-tools + # berthd and box have no C in them: on an arm runner they are linked + # for x86_64 by Rust's own linker. + - name: A linker for the host programs + if: matrix.target == 'aarch64-unknown-linux-musl' + run: echo "CARGO_TARGET_X86_64_UNKNOWN_LINUX_MUSL_LINKER=rust-lld" >> "$GITHUB_ENV" - uses: Swatinem/rust-cache@v2 with: key: ${{ matrix.target }} - - name: Build berth-remote - run: cargo build --release --locked --target ${{ matrix.target }} -p berth-remote + - name: Build berth + run: cargo build --release --locked --target ${{ matrix.target }} -p berth - name: Pack run: | mkdir dist - program=berth-remote - [ "$RUNNER_OS" = Windows ] && program=berth-remote.exe + program=berth + [ "$RUNNER_OS" = Windows ] && program=berth.exe cp "target/${{ matrix.target }}/release/$program" LICENSE-MIT LICENSE-APACHE dist/ case "${{ matrix.asset }}" in *.zip) (cd dist && 7z a -tzip -bso0 "../${{ matrix.asset }}" "$program" LICENSE-MIT LICENSE-APACHE) ;; @@ -147,21 +89,39 @@ jobs: else shasum -a 256 "${{ matrix.asset }}" > "${{ matrix.asset }}.sha256" fi - - name: Check the version + - name: Check the versions + # berth, and on Linux the berthd and box it carries, all say the same. run: | want=${BERTH_BUILD_VERSION:-$(sed -n '/^\[workspace.package\]/,/^\[/s/^version = "\(.*\)"/\1/p' Cargo.toml)} - said=$(dist/berth-remote --version) - echo "berth-remote: $said" - [ "$said" = "berth-remote $want" ] || { echo "::error::berth-remote says $said, not $want"; exit 1; } + programs=dist/berth + if [ "$RUNNER_OS" = Linux ]; then + cp target/host/x86_64-unknown-linux-musl/release/berthd target/host/x86_64-unknown-linux-musl/release/box dist/ + [ "${{ matrix.target }}" = x86_64-unknown-linux-musl ] && programs="$programs dist/berthd dist/box" + fi + for program in $programs; do + said=$("$program" --version) + echo "$program: $said" + case "$said" in + "${program##*/} $want" | "${program##*/} $want "*) ;; + *) echo "::error::$program says $said, not $want"; exit 1 ;; + esac + done - uses: actions/upload-artifact@v4 with: - name: berth-remote-${{ matrix.target }} + name: berth-${{ matrix.target }} path: | ${{ matrix.asset }} ${{ matrix.asset }}.sha256 + - uses: actions/upload-artifact@v4 + if: matrix.target == 'x86_64-unknown-linux-musl' + with: + name: host-programs-x86_64-linux + path: | + dist/berthd + dist/box release: if: startsWith(github.ref, 'refs/tags/v') - needs: [binary, remote] + needs: [binary] runs-on: ubuntu-latest permissions: contents: write @@ -169,10 +129,7 @@ jobs: - uses: actions/checkout@v7 - uses: actions/download-artifact@v4 with: - name: berth-x86_64-linux - - uses: actions/download-artifact@v4 - with: - pattern: berth-remote-* + pattern: berth-* merge-multiple: true - name: Publish the release env: @@ -186,9 +143,8 @@ jobs: # A release made on GitHub's page exists already, and only lacks the files. gh release view "$GITHUB_REF_NAME" >/dev/null 2>&1 \ || gh release create "$GITHUB_REF_NAME" --verify-tag --generate-notes $prerelease - gh release upload "$GITHUB_REF_NAME" --clobber berth-x86_64-linux.tar.gz berth-x86_64-linux.tar.gz.sha256 berth-web-x86_64-linux.tar.gz berth-web-x86_64-linux.tar.gz.sha256 \ - berth-remote-*.zip berth-remote-*.zip.sha256 berth-remote-*.tar.gz berth-remote-*.tar.gz.sha256 \ - install.sh install-remote.sh install.ps1 VERSION + gh release upload "$GITHUB_REF_NAME" --clobber berth-*.tar.gz berth-*.tar.gz.sha256 \ + berth-*.zip berth-*.zip.sha256 install.sh install.ps1 VERSION image: runs-on: ubuntu-latest permissions: diff --git a/.vale.ini b/.vale.ini index 0e5e382..0c691a0 100644 --- a/.vale.ini +++ b/.vale.ini @@ -17,6 +17,10 @@ Vale.Spelling = NO [{**/LICENSE*,crates/vendor/**,docs/node_modules/**,docs/.next/**,docs/out/**,docs/.source/**,build/**,.vale/**}] BasedOnStyles = +# The command reference is made from berth's --help texts, which keep to the terminal's short style. +[docs/content/docs/(guide)/reference/cli.mdx] +BasedOnStyles = + # Tables put a group next to a task of the same name. [README.md] Vale.Repetition = NO diff --git a/AGENTS.md b/AGENTS.md index 202ce0e..9e495b8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -6,15 +6,13 @@ berth sets up and looks after agent workstations. These are Ubuntu machines with | Path | What it is | | --- | --- | -| `crates/berth/` | The `berth` program: terminal UI, commands and `berth serve` (`crates/berth/src/serve/`). In `serve/`, `actions.rs` has the actions of hosts and of the installation, `keys.rs` the key ring's routes, `provisioners.rs` and `resources.rs` those of provisioners and resources | +| `crates/berth/` | The `berth` program, for Linux, macOS and Windows. `main.rs` opens the screens, and `src/cli/` is the command line, the same on every system. In `cli/`, `args.rs` is the grammar in clap and `docs.rs` writes the reference page from it. `client.rs` reaches the server. `ask.rs` asks, `follow.rs` shows progress, and `out.rs` prints. On Linux, `src/local/` adds the commands of the server's own machine and `berth serve` (`src/local/serve/`), behind one `#[cfg(target_os = "linux")]`, and `client_only.rs` stands in for it elsewhere. In `serve/`, `here.rs` is the server in berth's own process that the screens and commands use when no other is set, `actions.rs` has the actions of hosts and of the installation, `keys.rs` the key ring's routes, `provisioners.rs` and `resources.rs` those of provisioners and resources | | `crates/berth-api/` | Types and the operations table shared by every program (`ops.rs` names each operation, its HTTP route and its role) | -| `crates/berth-core/` | Hosts, profiles, links to hosts, registries and secrets, and plugins. The store is in `store/`. `kept.rs` and `plugin_state.rs` write files from it, and `import.rs` takes in the files of older versions. `kept_profile.rs` is the profile a host keeps of itself and `berth adopt`. `runner.rs` runs every action as a job. `keys/` is the key ring and how berth logs in to hosts. `level.rs` says what a host can do at each level. `provisioner.rs` and `resource.rs` are the provisioners of plugins and what they made | -| `crates/berth-tui/` | The parts of the UI that run in a terminal and in a browser | +| `crates/berth-core/` | Hosts, profiles, links to hosts, registries and secrets, and plugins. The store is in `store/`. `kept.rs` and `plugin_state.rs` write files from it, and `import.rs` takes in the files of older versions. `kept_profile.rs` is the profile a host keeps of itself and `berth host adopt`. `runner.rs` runs every action as a job. `keys/` is the key ring and how berth logs in to hosts. `host_type.rs` says what a host of each type can do. `provisioner.rs` and `resource.rs` are the provisioners of plugins and what they made | +| `crates/berth-tui/` | The parts of the screens with no terminal in them. `fields.rs`, `setup.rs` and `flow.rs` describe a host's set-up once, for the screens' tabs and the command line's prompts and flags alike, and `theme.rs` has the colours both draw with | | `crates/berth-pty/` | Terminals for sessions | | `crates/berthd/`, `crates/box/` | The controller on each host and the agent's own tool there | -| `crates/berth-page/` | The screens of the page of `berth serve`, shared by `berth-web` and `berth-remote`. `device.rs` is what a browser and a terminal each give them, and `portal.rs` the server's portal over it | -| `crates/berth-web/` | The page of `berth serve --web`, built for WebAssembly: the browser's side of `berth-page`'s device | -| `crates/berth-remote/` | `berth-remote`, the same screens in a terminal on Linux, macOS and Windows, for a `berth serve` on another machine | +| `crates/berth-page/` | berth's screens in a terminal. They do everything through the API of a server: one elsewhere, a `berth serve` on its socket, or the one in the process. `net.rs` reaches it, `portal.rs` is its portal, `settings.rs` keeps the server and its token, and `device.rs` is what the program starts them with | | `crates/vendor/` | The vendored `tui-input` crate | | `provision/` | The installer side of the provisioning source: the role in `ansible/`, first-boot provisioning in `bootstrap/`, the builder of seeds and releases in `bin/devbox.py` and the example profile in `config/`. berth names these paths in `layout` in `crates/berth-core/src/paths.rs` | | `plugins/` | The plugins that come with berth, each an npm-format package with a `guide.md`. With `provision/` they make the provisioning source, which berth carries and unpacks with the same layout | @@ -40,7 +38,7 @@ A new skill gets its own folder and a row here and in `.agents/skills/README.md` - Follow the `install-berth` skill. It says which recipe in `recipes/` fits and gives its command. - A recipe that erases disks needs the user's word first, for that server and those disks. - Do not write an enrollment key or a password into a file in the repository. -- When a berth server is installed, give the user its address and a setup key from `berth setup-key`. Do not read or print the token in `serve.token`. +- When a berth server is installed, give the user its address and a setup key from `berth server setup-key`. Do not read or print the token in `serve.token`. ## Build and check @@ -48,16 +46,15 @@ Use cargo for everything Rust. These are what CI runs: ```bash cargo test --locked -cargo test --locked -p berth --features web -cargo fmt --check -p berth-api -p berth-pty -p berth-core -p berth-tui -p berth -p berthd -p box -p berth-page -p berth-web -p berth-remote +cargo fmt --check -p berth-api -p berth-pty -p berth-core -p berth-tui -p berth -p berthd -p box -p berth-page cargo clippy --locked --all-targets -- -D warnings -cargo clippy --locked --all-targets -p berth --features web -- -D warnings -cargo clippy --locked -p berth-web --target wasm32-unknown-unknown -- -D warnings python3 -m unittest discover -s tests -v -vale --minAlertLevel=warning README.md docs/content/docs plugins/*/guide.md +vale --minAlertLevel=warning README.md AGENTS.md docs/content/docs plugins/*/guide.md ``` -Development and test builds embed quick builds of berthd, box and the page (the `host-dev` and `web-dev` profiles). Only `--release` embeds the release builds. The `web` feature needs the `wasm32-unknown-unknown` target. The Python tests skip their YAML checks without PyYAML, so install `provision/requirements.txt` in a virtual environment to run them all. +On macOS and Windows berth is the client alone. Check that it still builds there with `cargo check -p berth --target x86_64-pc-windows-msvc` or `--target aarch64-apple-darwin`; the release builds it on those systems. + +Development and test builds embed quick builds of berthd and box (the `host-dev` profile). Only `--release` embeds the release builds. The Python tests skip their YAML checks without PyYAML, so install `provision/requirements.txt` in a virtual environment to run them all. ### In containers @@ -79,6 +76,7 @@ The end-to-end tests in `tests/e2e/` use throw-away containers over SSH, with be | `e2e_controller.Controller` | The same host and berth's controller alone, with one playbook run | | `e2e_provisioners` | A provisioner of a made-up plugin, and no test host. With Docker, `ci/run.sh` starts an engine on the test network for the Docker plugin's cases | | `e2e_restore` | The host `studio` as it is, and a server of its own with an encrypted store. It leaves the host as it is | +| `e2e_node` | The bare host `dock` applied as a node, with a made-up plugin's feature that offers a provisioner. It runs last there and leaves the node | | The rest | A full apply of the profile | That engine starts with Docker when the run is complete, names `e2e_provisioners` or is the shard that holds it. `BERTH_CI_DOCKER` (`1` or `0`) forces it on or off, and `BERTH_CI_DIND_IMAGES` lists images of your own engine to load into it, as it cannot pull. The header of `ci/run.sh` has the rest. @@ -92,21 +90,22 @@ CI builds berth once and runs the tests of each test host as a shard, in a job o ## API changes - Add the operation to `crates/berth-api/src/ops.rs` with the role it needs. A test fails when a route has no operation or the other way round. -- The API documents in `docs/openapi/` are committed. Regenerate them with `BERTH_UPDATE_DOCS=1 cargo test -p berth -p berthd`. -- The pages under `docs/content/docs/api/host` and `api/serve` are generated from those documents. Do not edit them by hand. +- The API documents in `docs/openapi/` are committed. Regenerate them with `BERTH_UPDATE_DOCS=1 cargo test -p berth -p berthd`. The same writes the command line's reference page, `docs/content/docs/(guide)/reference/cli.mdx`, from the grammar in `crates/berth/src/cli/args.rs`. A test fails while either is stale. +- The pages under `docs/content/docs/api/host` and `api/serve` are generated from those documents when the docs are built (`docs/scripts/reference.mjs`), and are not committed. Do not edit them by hand. +- The screenshots of the docs are taken offscreen by `docs/scripts/screens.py`, from made-up hosts. ### Actions -Give a new thing a host, the installation, a provisioner or a resource can do an action in the matching registry. It does not get a route of its own. The routes `hosts.act`, `actions.act`, `provisioners.act` and `resources.act` run it, and each action says whether it can run now and what role it needs. +Give a new thing a host, the installation, a provisioner or a resource can do an action in the matching registry. It does not get a route of its own. The routes `hosts.act`, `installation.act`, `provisioners.act` and `resources.act` run it, and each action says whether it can run now and what role it needs. | Subject | Where its actions are | | --- | --- | -| A host | `HOST` in `crates/berth/src/serve/actions.rs`, besides the tasks its controller lists | +| A host | `HOST` in `crates/berth/src/local/serve/actions.rs`, besides the actions its controller lists | | The installation | `INSTALLATION` in the same file | | A provisioner | `provisioner::actions` in `crates/berth-core/src/provisioner.rs`: its plugin's actions and berth's own | | A resource | `resource::actions` in `crates/berth-core/src/resource.rs`: those of its kind and its plugin's | -An action that changes something runs as a job through `runner.rs`. +An action that changes something runs as a job through `runner.rs`. The screens do everything through the API. What works only with the server's own machine, such as writing a USB drive, is an action that is available to a caller there alone (`Seen::local`: the socket or the process), and says why not to one over the network. ## Versions and releases diff --git a/Cargo.lock b/Cargo.lock index c8145e5..06f0ea7 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2,12 +2,6 @@ # It is not intended for manual editing. version = 4 -[[package]] -name = "adler2" -version = "2.0.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "320119579fcad9c21884f5c4861d16174d0e06250625266f50fe6898340abefa" - [[package]] name = "aead" version = "0.5.2" @@ -33,6 +27,56 @@ version = "0.2.21" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923" +[[package]] +name = "anstream" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "824a212faf96e9acacdbd09febd34438f8f711fb84e09a8916013cd7815ca28d" +dependencies = [ + "anstyle", + "anstyle-parse", + "anstyle-query", + "anstyle-wincon", + "colorchoice", + "is_terminal_polyfill", + "utf8parse", +] + +[[package]] +name = "anstyle" +version = "1.0.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "940b3a0ca603d1eade50a4846a2afffd5ef57a9feac2c0e2ec2e14f9ead76000" + +[[package]] +name = "anstyle-parse" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "52ce7f38b242319f7cabaa6813055467063ecdc9d355bbb4ce0c68908cd8130e" +dependencies = [ + "utf8parse", +] + +[[package]] +name = "anstyle-query" +version = "1.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "40c48f72fd53cd289104fc64099abca73db4166ad86ea0b4341abe65af83dadc" +dependencies = [ + "windows-sys 0.61.2", +] + +[[package]] +name = "anstyle-wincon" +version = "3.0.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d" +dependencies = [ + "anstyle", + "once_cell_polyfill", + "windows-sys 0.61.2", +] + [[package]] name = "anyhow" version = "1.0.104" @@ -63,6 +107,45 @@ version = "0.7.8" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d3fb67a6e08acf24fdeccbac2cb6ac4305825bd1f117462e0e6f2f193345ad56" +[[package]] +name = "asn1-rs" +version = "0.7.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b7f43a50ac4fdca5df8e885c21b835997f0a1cdee65494a6847694a98652d9d8" +dependencies = [ + "asn1-rs-derive", + "asn1-rs-impl", + "displaydoc", + "nom", + "num-traits", + "rusticata-macros", + "thiserror 2.0.21", + "time", +] + +[[package]] +name = "asn1-rs-derive" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3109e49b1e4909e9db6515a30c633684d68cdeaa252f215214cb4fa1a5bfee2c" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", + "synstructure", +] + +[[package]] +name = "asn1-rs-impl" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7b18050c2cd6fe86c3a76584ef5e0baf286d038cda203eb6223df2cc413565f7" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + [[package]] name = "async-trait" version = "0.1.92" @@ -102,7 +185,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "31b698c5f9a010f6573133b09e0de5408834d0c82f8d7475a89fc1867a71cd90" dependencies = [ "axum-core", - "base64", + "base64 0.22.1", "bytes", "form_urlencoded", "futures-util", @@ -161,67 +244,16 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6" [[package]] -name = "base64ct" -version = "1.8.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06" - -[[package]] -name = "beamterm-core" -version = "1.0.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0375f7f3df163a52a06807f2150bac9a5da1933864ed40d52dda7dcd7cfa193b" -dependencies = [ - "beamterm-data", - "beamterm-unicode", - "bitflags 2.13.2", - "compact_str", - "glow", - "lru 0.16.4", - "rustc-hash", - "thiserror 2.0.21", - "unicode-width", -] - -[[package]] -name = "beamterm-data" -version = "1.0.0" +name = "base64" +version = "0.23.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f21ff1fc630079352bae9b024f85519bf1f641cf7f326623f4c0b59f7ea834fd" -dependencies = [ - "compact_str", - "miniz_oxide", - "thiserror 2.0.21", -] +checksum = "ac07cdecf99051d9a5238b80f35af32cdeba5b336e55d957b318b50137e18da5" [[package]] -name = "beamterm-renderer" -version = "1.0.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d8db76d04d575a0534ad5dba63acc05ccbe69597f977f11c59a612cbea199112" -dependencies = [ - "beamterm-core", - "beamterm-data", - "bitflags 2.13.2", - "compact_str", - "console_error_panic_hook", - "glow", - "js-sys", - "thiserror 2.0.21", - "unicode-width", - "wasm-bindgen", - "wasm-bindgen-futures", - "web-sys", -] - -[[package]] -name = "beamterm-unicode" -version = "1.0.0" +name = "base64ct" +version = "1.8.3" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1f44155933cdd0625a699c554c504aab30a571e4df68f3532fb95bf9cf53b935" -dependencies = [ - "unicode-width", -] +checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06" [[package]] name = "berth" @@ -231,15 +263,14 @@ dependencies = [ "berth-api", "berth-core", "berth-page", - "berth-pty", - "berth-remote", "berth-tui", "berthd", + "clap", + "clap_complete", "crossterm", "futures-util", "jsonwebtoken", "libc", - "pulldown-cmark", "ratatui", "ring", "serde", @@ -249,8 +280,6 @@ dependencies = [ "tui-input", "utoipa", "utoipa-axum", - "vt100", - "wasm-bindgen-cli-support", ] [[package]] @@ -279,6 +308,7 @@ dependencies = [ "hmac 0.12.1", "libc", "postgres", + "rcgen", "rusqlite", "rustls", "serde", @@ -298,26 +328,6 @@ version = "0.5.2" dependencies = [ "berth-api", "berth-tui", - "ratatui", - "serde_json", - "tui-input", - "web-time", -] - -[[package]] -name = "berth-pty" -version = "0.5.2" -dependencies = [ - "libc", -] - -[[package]] -name = "berth-remote" -version = "0.5.2" -dependencies = [ - "berth-api", - "berth-page", - "berth-tui", "crossterm", "futures-util", "ratatui", @@ -327,9 +337,17 @@ dependencies = [ "tempfile", "tokio", "tokio-rustls", + "tui-input", "webpki-roots", ] +[[package]] +name = "berth-pty" +version = "0.5.2" +dependencies = [ + "libc", +] + [[package]] name = "berth-tui" version = "0.5.2" @@ -342,21 +360,6 @@ dependencies = [ "serde_json", "tui-input", "vt100", - "web-time", -] - -[[package]] -name = "berth-web" -version = "0.5.2" -dependencies = [ - "berth-page", - "berth-tui", - "console_error_panic_hook", - "critical-section", - "js-sys", - "ratzilla", - "wasm-bindgen", - "web-sys", ] [[package]] @@ -387,7 +390,7 @@ version = "0.5.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0700ddab506f33b20a03b13996eccd309a48e5ff77d0d95926aa0210fb4e95f1" dependencies = [ - "bit-vec", + "bit-vec 0.6.3", ] [[package]] @@ -396,6 +399,15 @@ version = "0.6.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "349f9b6a179ed607305526ca489b34ad0a41aed5f7980fa90eb03160b69598fb" +[[package]] +name = "bit-vec" +version = "0.9.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b71798fca2c1fe1086445a7258a4bc81e6e49dcd24c8d0dd9a1e57395b603f51" +dependencies = [ + "serde", +] + [[package]] name = "bitflags" version = "1.3.2" @@ -408,18 +420,6 @@ version = "2.13.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "3ded4057c258ba199e2d26386d3af3780957ecaee6c4ef4041c6b4b8b97c0b06" -[[package]] -name = "bitvec" -version = "1.1.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ddcec3d12c579d40898fe0a9a358a803c23e9c52ca3c425707f81c9436211837" -dependencies = [ - "funty", - "radium", - "tap", - "wyz", -] - [[package]] name = "block-buffer" version = "0.10.4" @@ -558,12 +558,68 @@ dependencies = [ "zeroize", ] +[[package]] +name = "clap" +version = "4.6.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "aa8876b300ab35ba921adea3dfd70157a46249b33f95c9084ae5709785478946" +dependencies = [ + "clap_builder", + "clap_derive", +] + +[[package]] +name = "clap_builder" +version = "4.6.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ec0797fb7aeb1406c84efac526901f7ec3ead2124f946b494e72879d4b54704d" +dependencies = [ + "anstream", + "anstyle", + "clap_lex", + "strsim", + "terminal_size", +] + +[[package]] +name = "clap_complete" +version = "4.6.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "037e2a1a92236d0aff7e845093f64661d6df4c02c9fcc61a60e9e1d736fa392f" +dependencies = [ + "clap", +] + +[[package]] +name = "clap_derive" +version = "4.6.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f9c751b79415d4e559e3d1fcf128e09e720eb673a06d26cf6f392d37d75b66e0" +dependencies = [ + "heck", + "proc-macro2", + "quote", + "syn 3.0.9", +] + +[[package]] +name = "clap_lex" +version = "1.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1c133bc6a41be0d194c306b5506d15e6feeea7b1d6604bd3f8310dfb2ca96486" + [[package]] name = "cmov" version = "0.5.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0c9ea0ac24bc397ab3c98583a3c9ba74fa56b09a4449bbe172b9b1ddb016027a" +[[package]] +name = "colorchoice" +version = "1.0.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570" + [[package]] name = "compact_str" version = "0.9.1" @@ -578,16 +634,6 @@ dependencies = [ "static_assertions", ] -[[package]] -name = "console_error_panic_hook" -version = "0.1.7" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a06aeb73f470f66dcdbf7223caeebb85984942f22f1adb2a088cf9668146bbbc" -dependencies = [ - "cfg-if", - "wasm-bindgen", -] - [[package]] name = "const-oid" version = "0.9.6" @@ -649,31 +695,6 @@ version = "1.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "790eea4361631c5e7d22598ecd5723ff611904e3344ce8720784c93e3d83d40b" -[[package]] -name = "crossbeam-deque" -version = "0.8.8" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "622f3fc73690be383c7214310406f28a90e6edeadc3cea882f9d71e495b9711a" -dependencies = [ - "crossbeam-epoch", - "crossbeam-utils", -] - -[[package]] -name = "crossbeam-epoch" -version = "0.9.21" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "dc74980687109a3b14c72fd458107bf0baa1da1a1a805e178d15501ba9b86d9d" -dependencies = [ - "crossbeam-utils", -] - -[[package]] -name = "crossbeam-utils" -version = "0.8.23" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a31eee39dddec8330830986fcd7625edb5a24ec90ea038215273bbc3adb08ac6" - [[package]] name = "crossterm" version = "0.29.0" @@ -837,6 +858,20 @@ dependencies = [ "zeroize", ] +[[package]] +name = "der-parser" +version = "10.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "07da5016415d5a3c4dd39b11ed26f915f52fc4e0dc197d87908bc916e51bc1a6" +dependencies = [ + "asn1-rs", + "displaydoc", + "nom", + "num-bigint", + "num-traits", + "rusticata-macros", +] + [[package]] name = "der_derive" version = "0.7.3" @@ -900,6 +935,17 @@ dependencies = [ "ctutils", ] +[[package]] +name = "displaydoc" +version = "0.2.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c6232dd377dcc64799954cbd3a9bb882e9cdc1308ccd87b1c098f1fb2eaf82a8" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.9", +] + [[package]] name = "document-features" version = "0.2.12" @@ -1105,12 +1151,6 @@ dependencies = [ "percent-encoding", ] -[[package]] -name = "funty" -version = "2.0.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e6d5a32815ae3f33302d95fdcb2ce17862f8c65363dcfd29360480ba1001fc9c" - [[package]] name = "futures-channel" version = "0.3.34" @@ -1198,29 +1238,6 @@ dependencies = [ "rand_core 0.10.1", ] -[[package]] -name = "gimli" -version = "0.32.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e629b9b98ef3dd8afe6ca2bd0f89306cec16d43d907889945bc5d6687f2f13c7" -dependencies = [ - "fallible-iterator 0.3.0", - "indexmap", - "stable_deref_trait", -] - -[[package]] -name = "glow" -version = "0.17.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "29038e1c483364cc6bb3cf78feee1816002e127c331a1eec55a4d202b9e1adb5" -dependencies = [ - "js-sys", - "slotmap", - "wasm-bindgen", - "web-sys", -] - [[package]] name = "group" version = "0.13.0" @@ -1241,8 +1258,6 @@ dependencies = [ "allocator-api2", "equivalent", "foldhash", - "serde", - "serde_core", ] [[package]] @@ -1393,15 +1408,6 @@ dependencies = [ "tower-service", ] -[[package]] -name = "id-arena" -version = "2.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954" -dependencies = [ - "rayon", -] - [[package]] name = "ident_case" version = "1.0.1" @@ -1451,6 +1457,12 @@ dependencies = [ "syn 3.0.9", ] +[[package]] +name = "is_terminal_polyfill" +version = "1.70.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a6cb138bb79a146c1bd460005623e142ef0181e3d0219cb493e02f7d08a35695" + [[package]] name = "itertools" version = "0.14.0" @@ -1483,7 +1495,7 @@ version = "11.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e75fe14a82d81e5f5af639997db37d8b96045938a7ac6ab18cdbe1c7467e05e1" dependencies = [ - "base64", + "base64 0.22.1", "ed25519-dalek", "getrandom 0.2.17", "hmac 0.12.1", @@ -1525,18 +1537,6 @@ dependencies = [ "spin", ] -[[package]] -name = "leb128" -version = "0.2.7" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c83bff1d572d6b9aeef67ddfc8448e4a3737909cb28e81f97c791b9018703e52" - -[[package]] -name = "leb128fmt" -version = "0.1.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "09edd9e8b54e49e587e4f6295a7d29c3ea94d469cb40ab8ca70b288248a81db2" - [[package]] name = "libc" version = "0.2.191" @@ -1605,15 +1605,6 @@ version = "0.4.34" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f9f8bd3e56ce4dfc153cf470fffbfa98c7620958b312ca5c3a4b8d5181fd13c6" -[[package]] -name = "lru" -version = "0.16.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7f66e8d5d03f609abc3a39e6f08e4164ebf1447a732906d39eb9b99b7919ef39" -dependencies = [ - "hashbrown 0.16.1", -] - [[package]] name = "lru" version = "0.18.5" @@ -1682,15 +1673,6 @@ version = "0.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a" -[[package]] -name = "miniz_oxide" -version = "0.9.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b63fbc4a50860e98e7b2aa7804ded1db5cbc3aff9193adaff57a6931bf7c4b4c" -dependencies = [ - "adler2", -] - [[package]] name = "mio" version = "1.2.4" @@ -1738,6 +1720,16 @@ dependencies = [ "minimal-lexical", ] +[[package]] +name = "num-bigint" +version = "0.4.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c89e69e7e0f03bea5ef08013795c25018e101932225a656383bd384495ecc367" +dependencies = [ + "num-integer", + "num-traits", +] + [[package]] name = "num-bigint-dig" version = "0.8.6" @@ -1842,12 +1834,27 @@ dependencies = [ "memchr", ] +[[package]] +name = "oid-registry" +version = "0.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "12f40cff3dde1b6087cc5d5f5d4d65712f34016a03ed60e9c08dcc392736b5b7" +dependencies = [ + "asn1-rs", +] + [[package]] name = "once_cell" version = "1.21.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" +[[package]] +name = "once_cell_polyfill" +version = "1.70.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe" + [[package]] name = "opaque-debug" version = "0.3.1" @@ -1955,6 +1962,16 @@ version = "0.2.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "2ee67f1008b1ba2321834326597b8e186293b049a023cdef258527550b9935b4" +[[package]] +name = "pem" +version = "4.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d354a98a3d1251555de99e8fdd8afda05573c31b82f59063a7b0a29b5527f120" +dependencies = [ + "base64 0.23.1", + "serde_core", +] + [[package]] name = "pem-rfc7468" version = "0.7.0" @@ -2164,7 +2181,7 @@ version = "0.6.12" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "08808e3c483c46e999108051c78334f473d5adb59d78bb80a1268c7e6aa6c514" dependencies = [ - "base64", + "base64 0.22.1", "byteorder", "bytes", "fallible-iterator 0.2.0", @@ -2268,12 +2285,6 @@ version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf" -[[package]] -name = "radium" -version = "0.7.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "dc33ff2d4973d518d823d61aa239014831e521c75da58e3df4840d3f47749d09" - [[package]] name = "rand" version = "0.8.8" @@ -2379,7 +2390,7 @@ dependencies = [ "hashbrown 0.17.1", "itertools", "kasuari", - "lru 0.18.5", + "lru", "palette", "serde", "strum", @@ -2464,39 +2475,17 @@ dependencies = [ ] [[package]] -name = "ratzilla" -version = "0.3.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d53309a50b2cfb9828d1ea653b77fc6d2fe08137b05e65aa070ab87630e35b49" -dependencies = [ - "beamterm-renderer", - "bitvec", - "compact_str", - "console_error_panic_hook", - "ratatui", - "thiserror 2.0.21", - "unicode-width", - "web-sys", -] - -[[package]] -name = "rayon" -version = "1.12.0" +name = "rcgen" +version = "0.14.10" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "fb39b166781f92d482534ef4b4b1b2568f42613b53e5b6c160e24cfbfa30926d" +checksum = "8774e05a7d0de114588e6a28fe7e71694b82614ed569d86d8b389dfbc98b8ad8" dependencies = [ - "either", - "rayon-core", -] - -[[package]] -name = "rayon-core" -version = "1.13.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "22e18b0f0062d30d4230b2e85ff77fdfe4326feb054b9783a3460d8435c8ab91" -dependencies = [ - "crossbeam-deque", - "crossbeam-utils", + "pem", + "ring", + "rustls-pki-types", + "time", + "x509-parser", + "yasna", ] [[package]] @@ -2606,24 +2595,6 @@ dependencies = [ "sqlite-wasm-rs", ] -[[package]] -name = "rustc-demangle" -version = "0.1.28" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b74b56ffa8bb2830709a538c2cbcae9aa062db0d2a42563bfb09bdaae44020eb" - -[[package]] -name = "rustc-hash" -version = "2.1.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6b1e7f9a428571be2dc5bc0505c13fb6bf936822b894ec87abf8a08a4e51742d" - -[[package]] -name = "rustc-std-workspace-core" -version = "1.0.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "aa9c45b374136f52f2d6311062c7146bff20fec063c3f5d46a410bd937746955" - [[package]] name = "rustc_version" version = "0.4.1" @@ -2633,6 +2604,15 @@ dependencies = [ "semver", ] +[[package]] +name = "rusticata-macros" +version = "4.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "faf0c4a6ece9950b9abdb62b1cfcf2a68b3b67a10ba445b3bb85be2a293d0632" +dependencies = [ + "nom", +] + [[package]] name = "rustix" version = "1.1.6" @@ -2931,15 +2911,6 @@ version = "0.4.12" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5" -[[package]] -name = "slotmap" -version = "1.1.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "bdd58c3c93c3d278ca835519292445cb4b0d4dc59ccfdf7ceadaab3f8aeb4038" -dependencies = [ - "version_check", -] - [[package]] name = "smallvec" version = "1.16.4" @@ -2984,12 +2955,6 @@ dependencies = [ "wasm-bindgen", ] -[[package]] -name = "stable_deref_trait" -version = "1.2.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596" - [[package]] name = "stacker" version = "0.1.25" @@ -3093,10 +3058,15 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0bf256ce5efdfa370213c1dabab5935a12e49f2c58d15e9eac2870d3b4f27263" [[package]] -name = "tap" -version = "1.0.1" +name = "synstructure" +version = "0.13.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "55937e1799185b12863d447f42597ed69d9928686b8d88a1df17376a097d8369" +checksum = "728a70f3dbaf5bab7f0c4b1ac8d7ae5ea60a4b5549c8a5914361c99147a709d2" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] [[package]] name = "tempfile" @@ -3124,6 +3094,16 @@ dependencies = [ "windows-sys 0.61.2", ] +[[package]] +name = "terminal_size" +version = "0.4.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "230a1b821ccbd75b185820a1f1ff7b14d21da1e442e22c0863ea5f08771a8874" +dependencies = [ + "rustix", + "windows-sys 0.61.2", +] + [[package]] name = "terminfo" version = "0.9.0" @@ -3162,7 +3142,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "4676b37242ccbd1aabf56edb093a4827dc49086c0ffd764a5705899e0f35f8f7" dependencies = [ "anyhow", - "base64", + "base64 0.22.1", "bitflags 2.13.2", "fancy-regex", "filedescriptor", @@ -3250,6 +3230,7 @@ dependencies = [ "powerfmt", "serde_core", "time-core", + "time-macros", ] [[package]] @@ -3258,6 +3239,16 @@ version = "0.1.9" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9e1c906769ad99c88eaa54e728060edef082f8e358ff32030cb7c7d315e81109" +[[package]] +name = "time-macros" +version = "0.2.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7e689342a48d2ea927c87ea50cabf8594854bf940e9310208848d680d668ed85" +dependencies = [ + "num-conv", + "time-core", +] + [[package]] name = "tinyvec" version = "1.13.3" @@ -3545,9 +3536,6 @@ name = "unicode-width" version = "0.2.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b4ac048d71ede7ee76d585517add45da530660ef4390e49b098733c6e897f254" -dependencies = [ - "rustc-std-workspace-core", -] [[package]] name = "universal-hash" @@ -3662,35 +3650,6 @@ dependencies = [ "utf8parse", ] -[[package]] -name = "walrus" -version = "0.27.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5691aeb3546554822fe2c13d63149c26a027cd07d900a72c7c7f37f4321bfaea" -dependencies = [ - "anyhow", - "gimli", - "id-arena", - "leb128", - "log", - "rayon", - "walrus-macro", - "wasm-encoder", - "wasmparser", -] - -[[package]] -name = "walrus-macro" -version = "0.26.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1a9b0525d7ea6e5f906aca581a172e5c91b4c595290dfa8ad4a2bc9ffef33b44" -dependencies = [ - "heck", - "proc-macro2", - "quote", - "syn 2.0.119", -] - [[package]] name = "want" version = "0.3.2" @@ -3746,35 +3705,6 @@ dependencies = [ "wasm-bindgen-shared", ] -[[package]] -name = "wasm-bindgen-cli-support" -version = "0.2.129" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a0724352b96c7779dcfd3f580b9e7e29beac8f4229c9a27cd2ea53e04f1cc65a" -dependencies = [ - "anyhow", - "base64", - "leb128", - "log", - "rustc-demangle", - "serde", - "serde_json", - "walrus", - "wasm-bindgen-shared", - "wasmparser", -] - -[[package]] -name = "wasm-bindgen-futures" -version = "0.4.79" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3cbab34de2d982e9b48e18d216d04c4a6f641066ff19ffb699980f591ee3610e" -dependencies = [ - "js-sys", - "tokio", - "wasm-bindgen", -] - [[package]] name = "wasm-bindgen-macro" version = "0.2.129" @@ -3807,29 +3737,6 @@ dependencies = [ "unicode-ident", ] -[[package]] -name = "wasm-encoder" -version = "0.245.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3f9dca005e69bf015e45577e415b9af8c67e8ee3c0e38b5b0add5aa92581ed5c" -dependencies = [ - "leb128fmt", - "wasmparser", -] - -[[package]] -name = "wasmparser" -version = "0.245.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "4f08c9adee0428b7bddf3890fc27e015ac4b761cc608c822667102b8bfd6995e" -dependencies = [ - "bitflags 2.13.2", - "hashbrown 0.16.1", - "indexmap", - "semver", - "serde", -] - [[package]] name = "web-sys" version = "0.3.106" @@ -3840,16 +3747,6 @@ dependencies = [ "wasm-bindgen", ] -[[package]] -name = "web-time" -version = "1.1.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5a6580f308b1fad9207618087a65c04e7a10bc77e02c8e84e9b00dd4b12fa0bb" -dependencies = [ - "js-sys", - "wasm-bindgen", -] - [[package]] name = "webpki-roots" version = "1.0.12" @@ -4060,15 +3957,6 @@ version = "0.57.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1ebf944e87a7c253233ad6766e082e3cd714b5d03812acc24c318f549614536e" -[[package]] -name = "wyz" -version = "0.5.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "05f360fc0b24296329c78fda852a1e9ae82de9cf7b27dae4b7f62f118f77b9ed" -dependencies = [ - "tap", -] - [[package]] name = "x509-cert" version = "0.2.5" @@ -4081,6 +3969,34 @@ dependencies = [ "tls_codec", ] +[[package]] +name = "x509-parser" +version = "0.18.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d43b0f71ce057da06bc0851b23ee24f3f86190b07203dd8f567d0b706a185202" +dependencies = [ + "asn1-rs", + "data-encoding", + "der-parser", + "lazy_static", + "nom", + "oid-registry", + "ring", + "rusticata-macros", + "thiserror 2.0.21", + "time", +] + +[[package]] +name = "yasna" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b5f6765e852b9b4dc8e2a76843e4d64d1cea8e79bcde0b6901aea8e7c7f08282" +dependencies = [ + "bit-vec 0.9.1", + "time", +] + [[package]] name = "zerocopy" version = "0.8.63" diff --git a/Cargo.toml b/Cargo.toml index a2fd53a..1fd468f 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,9 +1,9 @@ [workspace] resolver = "3" -members = ["crates/berth-api", "crates/berth-pty", "crates/berth-core", "crates/berth-tui", "crates/berth", "crates/berthd", "crates/box", "crates/berth-page", "crates/berth-web", "crates/berth-remote"] +members = ["crates/berth-api", "crates/berth-pty", "crates/berth-core", "crates/berth-tui", "crates/berth", "crates/berthd", "crates/box", "crates/berth-page"] # The vendored crate is a member because it lives inside; it is left out of what # is built, tested and linted by default. -default-members = ["crates/berth-api", "crates/berth-pty", "crates/berth-core", "crates/berth-tui", "crates/berth", "crates/berthd", "crates/box", "crates/berth-page", "crates/berth-remote"] +default-members = ["crates/berth-api", "crates/berth-pty", "crates/berth-core", "crates/berth-tui", "crates/berth", "crates/berthd", "crates/box", "crates/berth-page"] [workspace.package] version = "0.5.2" @@ -12,7 +12,7 @@ rust-version = "1.88" license = "MIT OR Apache-2.0" [workspace.dependencies] -# Members that talk to a server ask for the client; the browser build does not. +# Members that talk to a server ask for the client; the screens alone do not. berth-api = { path = "crates/berth-api", default-features = false } berth-pty = { path = "crates/berth-pty" } berth-core = { path = "crates/berth-core" } @@ -51,25 +51,10 @@ debug = false [profile.release] strip = true -# The page of berth serve: small rather than fast, since it is carried in berth. -[profile.web] -inherits = "release" -opt-level = "s" -lto = "fat" -codegen-units = 1 - # What a development or test build of berth carries. berthd and box are built -# incrementally, and small, as berth packs them. The page keeps its -# optimisation without the whole-program link that takes most of its build -# time after a change. +# incrementally, and small, as berth packs them. [profile.host-dev] inherits = "dev" opt-level = "s" debug = false strip = true - -[profile.web-dev] -inherits = "web" -lto = false -codegen-units = 16 -incremental = true diff --git a/README.md b/README.md index 847c71c..01a587c 100644 --- a/README.md +++ b/README.md @@ -12,14 +12,14 @@ installs and can apply again. Once the host runs, its page in berth has the status, sign-ins, updates and secrets. A machine you only log in to can be a host too: `berth host add dock 192.0.2.4 --user ops`. -berth's terminal UI showing the overview of a host with its tasks +berth's terminal UI showing the overview of a host with its actions **Status:** early. Try a profile in the container or a VM before booting the installer on real hardware: **the installer erases the disk it selects.** ## Install -On Linux x86_64: +On Linux or macOS: ```bash curl -fsSL https://raw.githubusercontent.com/helightdev/berth/main/install.sh | sh @@ -37,21 +37,21 @@ Setting up hosts also needs `ssh`, `python3` and `ansible-playbook` on your machine. berth is also an image, `ghcr.io/helightdev/berth`, which runs its local API server by default. -`berth-remote` uses the hosts of a berth server from a terminal on Windows, -macOS or Linux: +On macOS and Windows berth uses the hosts of a berth server elsewhere, with the +same screens. `install.sh` installs it on macOS too, and on Windows: ```powershell -irm https://raw.githubusercontent.com/helightdev/berth/main/install.ps1 | iex # Windows +irm https://raw.githubusercontent.com/helightdev/berth/main/install.ps1 | iex ``` -```bash -curl -fsSL https://raw.githubusercontent.com/helightdev/berth/main/install-remote.sh | sh # macOS, Linux -``` +Coming from berth 0.5? Update the server first, then the clients and the +hosts, as [Upgrading from 0.5](docs/content/docs/%28guide%29/use/upgrading.mdx) +says. ## Quick start ```bash -berth studio # a new profile: choose how to install, features and settings +berth studio # a new host: choose how to install, features and settings berth # the home screen with all hosts berth run studio status ``` @@ -69,6 +69,7 @@ The documentation site is built from [`docs/`](docs) (`cd docs && npm install && - [Getting started](docs/content/docs/%28guide%29/start/install.mdx): installing berth and a first host. - [Deploying](docs/content/docs/%28guide%29/deploy/index.mdx): the install methods, and testing in QEMU. - [Using berth](docs/content/docs/%28guide%29/use/index.mdx): the terminal UI and commands, sign-ins, secrets and updates. +- [Upgrading from 0.5](docs/content/docs/%28guide%29/use/upgrading.mdx): the order, what is converted, and going back. - [Extending](docs/content/docs/%28guide%29/extend/index.mdx): plugins, services and `berth serve`. - [Reference](docs/content/docs/%28guide%29/reference/index.mdx): tools, `box`, profile fields, building and releasing. - The API reference of berthd and `berth serve`, generated from [`docs/openapi/`](docs/openapi). diff --git a/ci/boot.compose.yml b/ci/boot.compose.yml index eeefa0c..86360a4 100644 --- a/ci/boot.compose.yml +++ b/ci/boot.compose.yml @@ -40,6 +40,8 @@ services: BERTH_BOOT_SSH_PORT: ${BERTH_CI_BOOT_SSH_PORT:-22722} BERTH_BOOT_CONTROL_URL: ${BERTH_CI_BOOT_CONTROL_URL:-} BERTH_BOOT_CONTROL_KEY: ${BERTH_CI_BOOT_CONTROL_KEY:-} + BERTH_BOOT_CONTROL_USER: ${BERTH_CI_BOOT_CONTROL_USER:-} + BERTH_BOOT_CONTROL_TAGS: ${BERTH_CI_BOOT_CONTROL_TAGS:-} CI: ${CI:-} GITHUB_ACTIONS: ${GITHUB_ACTIONS:-} TMPDIR: /work/tmp diff --git a/ci/boot.sh b/ci/boot.sh index 8ebf7c8..fee4b3a 100644 --- a/ci/boot.sh +++ b/ci/boot.sh @@ -19,8 +19,12 @@ # BERTH_CI_BOOT_CPUS its vCPUs (default: 4, at most what is there) # BERTH_CI_BOOT_SSH_PORT the loopback port of its SSH with --keep (default: 22722) # BERTH_CI_BOOT_CONTROL_URL, BERTH_CI_BOOT_CONTROL_KEY -# a tailnet control server and a key to enrol the -# guest with on its first boot +# a tailnet control server, a Headscale address or +# tailscale for Tailscale's own, and its API key, with +# which berth makes the key the guest joins with on +# its first boot +# BERTH_CI_BOOT_CONTROL_USER, BERTH_CI_BOOT_CONTROL_TAGS +# berth's tailnet_user and tailnet_tags for it boot_compose() { "$engine" compose -f ci/boot.compose.yml "$@"; } @@ -63,7 +67,7 @@ boot() { boot_compose build --quiet fi # The guest gets berth's host programs from the role's files, which this build puts there. - dev cargo build --locked -p berth --features web + dev cargo build --locked -p berth local kind status=0 run for kind in "${kinds[@]}"; do diff --git a/ci/checks.sh b/ci/checks.sh index 6c2bd00..1217a4a 100755 --- a/ci/checks.sh +++ b/ci/checks.sh @@ -4,18 +4,15 @@ set -euo pipefail cd /src -crates=(-p berth-api -p berth-pty -p berth-core -p berth-tui -p berth -p berthd -p box -p berth-page -p berth-web -p berth-remote) +crates=(-p berth-api -p berth-pty -p berth-core -p berth-tui -p berth -p berthd -p box -p berth-page) # Each runs as the condition of an if, where set -e does not stop it: the # steps are joined with && instead. -rust-tests() { - cargo test --locked && - cargo test --locked -p berth --features web -} +rust-tests() { cargo test --locked; } python-tests() { python3 -m unittest discover -s tests -v; } shell-syntax() { local script bad=0 - for script in install.sh install-remote.sh provision/bin/*.sh provision/bootstrap/*.sh container/*.sh ci/*.sh \ + for script in install.sh provision/bin/*.sh provision/bootstrap/*.sh container/*.sh ci/*.sh \ provision/ansible/roles/devbox/files/*.sh provision/ansible/roles/devbox/files/tasks/*.sh plugins/*/files/*.sh \ recipes/*/*.sh; do bash -n "$script" || bad=1 @@ -25,15 +22,10 @@ shell-syntax() { ansible-syntax() { (cd provision && ansible-playbook -i localhost, ansible/site.yml --syntax-check) && (cd provision && ansible-playbook -i localhost, ansible/controller.yml --syntax-check) && - (cd recipes/incus-host && ansible-playbook -i localhost, site.yml --syntax-check) && (cd recipes/berth-server && ansible-playbook -i localhost, site.yml --syntax-check) } fmt() { cargo fmt --check "${crates[@]}"; } -clippy() { - cargo clippy --locked --all-targets -- -D warnings && - cargo clippy --locked --all-targets -p berth --features web -- -D warnings && - cargo clippy --locked -p berth-web --target wasm32-unknown-unknown -- -D warnings -} +clippy() { cargo clippy --locked --all-targets -- -D warnings; } vale() { command vale --minAlertLevel=warning README.md AGENTS.md docs/content/docs plugins/*/guide.md; } all=(fmt shell-syntax ansible-syntax vale python-tests rust-tests clippy) diff --git a/ci/dev.Dockerfile b/ci/dev.Dockerfile index a48d1d2..13da9fc 100644 --- a/ci/dev.Dockerfile +++ b/ci/dev.Dockerfile @@ -11,7 +11,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ procps python3 python3-venv tini xz-utils \ && rm -rf /var/lib/apt/lists/* \ && rustup component add clippy rustfmt \ - && rustup target add x86_64-unknown-linux-musl wasm32-unknown-unknown + && rustup target add x86_64-unknown-linux-musl RUN curl -fsSL "https://github.com/errata-ai/vale/releases/download/v${VALE_VERSION}/vale_${VALE_VERSION}_Linux_64-bit.tar.gz" \ | tar -xz -C /usr/local/bin vale COPY provision/requirements.txt /tmp/requirements.txt diff --git a/ci/run.sh b/ci/run.sh index c9fb3bd..c59d924 100755 --- a/ci/run.sh +++ b/ci/run.sh @@ -144,7 +144,7 @@ docker_up() { e2e() { # The test host and the runner have no way out, so berth is built first. if [[ ${BERTH_CI_PREBUILT:-} != 1 ]]; then - dev cargo build --locked -p berth --features web + dev cargo build --locked -p berth fi e2e_down trap e2e_down EXIT diff --git a/container/berth.Dockerfile b/container/berth.Dockerfile index e2bac4c..e7cef49 100644 --- a/container/berth.Dockerfile +++ b/container/berth.Dockerfile @@ -12,12 +12,11 @@ FROM rust:1-bookworm AS build RUN apt-get update && apt-get install -y --no-install-recommends musl-tools python3 \ && rm -rf /var/lib/apt/lists/* \ - && rustup target add x86_64-unknown-linux-musl wasm32-unknown-unknown + && rustup target add x86_64-unknown-linux-musl WORKDIR /src COPY . . ARG BERTH_BUILD_VERSION= -# With the page for a browser, which the default command serves. -RUN cargo build --release --locked --target x86_64-unknown-linux-musl -p berth --features web \ +RUN cargo build --release --locked --target x86_64-unknown-linux-musl -p berth \ && said=$(target/x86_64-unknown-linux-musl/release/berth --version) && echo "$said" \ && { [ -z "$BERTH_BUILD_VERSION" ] || [ "$said" = "berth $BERTH_BUILD_VERSION" ]; } @@ -37,4 +36,4 @@ ENV XDG_CONFIG_HOME=/config XDG_DATA_HOME=/data TERM=xterm-256color VOLUME ["/config", "/data"] EXPOSE 7777 ENTRYPOINT ["tini", "--", "berth"] -CMD ["serve", "--listen", "0.0.0.0:7777", "--expose", "--web"] +CMD ["serve", "--listen", "0.0.0.0:7777", "--expose"] diff --git a/crates/berth-api/Cargo.toml b/crates/berth-api/Cargo.toml index 62521b1..7a4cb24 100644 --- a/crates/berth-api/Cargo.toml +++ b/crates/berth-api/Cargo.toml @@ -20,5 +20,5 @@ futures-util = { workspace = true, optional = true } [features] default = ["client"] # The client over a byte stream; without it only the types and tables remain, -# which is what a browser build takes. +# which is all the shared screens take. client = ["dep:tokio", "dep:hyper", "dep:hyper-util", "dep:http-body-util", "dep:tokio-tungstenite", "dep:futures-util"] diff --git a/crates/berth-api/src/client.rs b/crates/berth-api/src/client.rs index 910fd1f..a71c9be 100644 --- a/crates/berth-api/src/client.rs +++ b/crates/berth-api/src/client.rs @@ -68,6 +68,20 @@ fn transport(error: impl std::fmt::Display) -> Error { pub type Result = std::result::Result; +/// A connection to a server, whatever carries it: TCP, TLS, a unix socket, +/// or a pipe to a server in the same process. +pub trait Io: AsyncRead + AsyncWrite + Unpin + Send {} +impl Io for T {} +pub type Stream = Box; + +/// How a client reaches its server: each call opens a new connection, for a +/// request or for a WebSocket. +pub type Dial = std::sync::Arc< + dyn Fn() -> std::pin::Pin> + Send>> + + Send + + Sync, +>; + /// A raw answer: status, headers and body. pub struct Response { pub status: u16, @@ -238,12 +252,27 @@ where Ok((stream, _)) => Ok(stream), Err(tungstenite::Error::Http(response)) => { let body = response.body().clone().unwrap_or_default(); - Err(Response { - status: response.status().as_u16(), - headers: Default::default(), - body: body.into(), + let status = response.status().as_u16(); + // The body may still be on its way when the head is read, and + // then only the headers say which error it was. + let header = |name: &str| response.headers().get(name)?.to_str().ok(); + let named = header(crate::ERROR_HEADER); + let said = header(crate::ERROR_MESSAGE_HEADER) + .map(decode) + .filter(|said| !said.is_empty()); + match (serde_json::from_slice::(&body), named) { + (Err(_), Some(code)) => Err(Error::Api { + status, + code: code.to_owned(), + message: said.unwrap_or_else(|| format!("HTTP {status}: {code}")), + }), + _ => Err(Response { + status, + headers: Default::default(), + body: body.into(), + } + .error()), } - .error()) } Err(error) => Err(transport(error)), } @@ -261,3 +290,31 @@ pub async fn websocket_unix( } pub use crate::escape::{decode, encode, encode_path}; + +#[cfg(test)] +mod tests { + use super::*; + use tokio::io::{AsyncReadExt, AsyncWriteExt}; + + #[tokio::test] + async fn a_refused_stream_says_why_from_the_head_alone() { + let (ours, mut theirs) = tokio::io::duplex(4096); + let server = tokio::spawn(async move { + let mut asked = [0; 1024]; + let _ = theirs.read(&mut asked).await; + let said = crate::error_message_header("Nothing answers at db: né"); + // The head alone: the body it promises never comes. + let head = format!( + "HTTP/1.1 503 Service Unavailable\r\ncontent-length: 80\r\n{}: unreachable\r\n{}: {said}\r\n\r\n", + crate::ERROR_HEADER, + crate::ERROR_MESSAGE_HEADER, + ); + theirs.write_all(head.as_bytes()).await.unwrap(); + theirs + }); + let refused = websocket(ours, "/v1/sessions/1", "").await.err().unwrap(); + assert_eq!(refused.status(), 503); + assert_eq!(refused.to_string(), "Nothing answers at db: né"); + drop(server.await); + } +} diff --git a/crates/berth-api/src/compat.rs b/crates/berth-api/src/compat.rs new file mode 100644 index 0000000..604148b --- /dev/null +++ b/crates/berth-api/src/compat.rs @@ -0,0 +1,119 @@ +//! What berth says when two of its programs are of different versions: a +//! client and its server, or the server and a host's controller. Each is one +//! sentence that names both versions and what to update, the same for the +//! command line and the screens. + +use crate::{API, HostType, VERSION}; + +/// The lowest `api` of a host's controller this berth can work with. +pub const NEEDS_HOST_API: u32 = 2; + +/// Where updating from an older berth is explained. +pub const UPGRADE_DOCS: &str = + "https://github.com/helightdev/berth/blob/main/docs/content/docs/(guide)/use/upgrading.mdx"; + +/// The operations of berth 0.5 that this server does not have, which only a +/// client of that version still calls. +pub const OLDER_OPS: &[&str] = &["tasks.list", "tasks.run", "tasks.choices", "actions.act"]; + +/// Why a client refuses to work with the server at `url`, which says it is +/// berth `version` of `api`; `None` when the server is new enough. +pub fn older_server(url: &str, version: &str, api: u32) -> Option { + (api < API).then(|| { + format!( + "The berth server at {url} is berth {version} (API {api}), older than this berth {VERSION} (API {API}), and would not understand what it is sent. Update the server first, then the clients and the hosts: {UPGRADE_DOCS}" + ) + }) +} + +/// What updates the controller on a host of this type. +pub fn controller_update(host: &str, host_type: HostType) -> String { + match host_type { + HostType::Workstation | HostType::Node => format!("berth apply {host}"), + HostType::Managed | HostType::Ssh => format!("berth run {host} install-controller"), + } +} + +/// What a host says when its controller is berth `version` of `api`, too +/// old for this berth; `None` when it is new enough. +pub fn older_controller( + host: &str, + version: &str, + api: u32, + host_type: HostType, +) -> Option { + (api < NEEDS_HOST_API).then(|| controller_is_older(host, version, host_type)) +} + +/// That the controller on `host`, berth `version`, is too old for this berth, +/// and what updates it. +pub fn controller_is_older(host: &str, version: &str, host_type: HostType) -> String { + format!( + "The controller on {host} is berth {version}, older than this berth {VERSION}. Update it with {}", + controller_update(host, host_type) + ) +} + +/// What the server answers a client that calls an operation it does not +/// have: for one of an older berth, that the client is older. +pub fn no_operation(name: &str) -> String { + match OLDER_OPS.contains(&name) { + true => format!( + "No operation {name}: it is a call of berth 0.5, and this server is berth {VERSION}. Update berth where this call comes from: {UPGRADE_DOCS}" + ), + false => format!("No operation {name}"), + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn an_older_server_is_named_with_both_versions() { + assert_eq!(older_server("https://pier.example", VERSION, API), None); + let said = older_server("https://pier.example", "0.5.2", 1).unwrap(); + assert!(said.contains("is berth 0.5.2 (API 1)"), "{said}"); + assert!( + said.contains(&format!("this berth {VERSION} (API {API})")), + "{said}" + ); + assert!(said.contains("Update the server first"), "{said}"); + assert!(said.ends_with(UPGRADE_DOCS), "{said}"); + } + + #[test] + fn an_older_controller_says_what_updates_it() { + assert_eq!( + older_controller("studio", VERSION, NEEDS_HOST_API, HostType::Workstation), + None + ); + let said = older_controller("studio", "0.5.2", 1, HostType::Workstation).unwrap(); + assert!(said.starts_with("The controller on studio is berth 0.5.2, older than this berth")); + assert!( + said.ends_with("Update it with berth apply studio"), + "{said}" + ); + let said = older_controller("big", "0.5.2", 1, HostType::Node).unwrap(); + assert!(said.ends_with("berth apply big"), "{said}"); + let said = older_controller("pier", "0.5.2", 1, HostType::Managed).unwrap(); + assert!( + said.ends_with("berth run pier install-controller"), + "{said}" + ); + } + + #[test] + fn a_call_of_an_older_client_says_so() { + assert_eq!(no_operation("hosts.nothing"), "No operation hosts.nothing"); + assert!(no_operation("tasks.list").contains("a call of berth 0.5")); + // None of them is an operation now. + for name in OLDER_OPS { + assert!( + crate::ops::find(crate::ops::SERVE, name).is_none(), + "{name}" + ); + assert!(crate::ops::find(crate::ops::HOST, name).is_none(), "{name}"); + } + } +} diff --git a/crates/berth-api/src/lib.rs b/crates/berth-api/src/lib.rs index c9182a6..b365743 100644 --- a/crates/berth-api/src/lib.rs +++ b/crates/berth-api/src/lib.rs @@ -3,12 +3,15 @@ #[cfg(feature = "client")] pub mod client; +pub mod compat; pub mod escape; pub mod ops; #[cfg(feature = "client")] pub mod portal; pub mod profile; pub mod streams; +#[cfg(feature = "client")] +pub mod tunnel; mod types; pub use types::*; @@ -26,7 +29,7 @@ const _: () = assert!( "BERTH_BUILD_VERSION is a version such as 0.3.0, without the v of its tag" ); /// Rises when something is added to the API. -pub const API: u32 = 1; +pub const API: u32 = 2; /// Where berthd listens in system mode. pub const RUN_DIR: &str = "/run/berth"; diff --git a/crates/berth-api/src/ops.rs b/crates/berth-api/src/ops.rs index c7fe239..a6def3a 100644 --- a/crates/berth-api/src/ops.rs +++ b/crates/berth-api/src/ops.rs @@ -29,6 +29,16 @@ //! ↔ {"sub": 7, "end": true} the last frame; from the server it may carry "error" //! ``` //! +//! A request to open a stream may add `"window": BYTES`. A server that keeps +//! to it says so in its answer, `{"open": true, "window": BYTES}`, with the +//! window it took. Each side then sends at most that many bytes of binary +//! frames of the stream that the other side has not yet said it took, and +//! says what it took as it goes: +//! +//! ```text +//! ↔ {"sub": 7, "credit": 131072} so many more bytes may come +//! ``` +//! //! Bytes in a result or as input are `{"base64": "…"}` (a result adds //! `content_type`), up to [`PORTAL_BYTES`]; larger transfers are HTTP's. @@ -72,7 +82,7 @@ pub enum Body { pub enum Role { /// Look: hosts, sessions and their output, logs, the inbox, the store. View, - /// Use the hosts: run tasks and commands, write files, control units. + /// Use the hosts: run actions and commands, write files, control units. Operate, /// Set up: profiles, secrets, registries, plugins, and applying them to hosts. Admin, @@ -111,6 +121,10 @@ pub struct Op { /// What the caller shows instead of a token with a role, for an /// operation that checks a key of its own; empty for the others. pub key: &'static str, + /// Why only `berth serve` calls it on a host, for itself; empty for the + /// others. It passes none of these through for a caller, and no portal + /// carries them. + pub server_only: &'static str, } const fn call(name: &'static str, method: &'static str, path: &'static str) -> Op { @@ -128,6 +142,7 @@ const fn call(name: &'static str, method: &'static str, path: &'static str) -> O _ => Role::Operate, }, key: "", + server_only: "", } } @@ -142,6 +157,7 @@ const fn stream(name: &'static str, path: &'static str) -> Op { http_only: "", role: Role::Operate, key: "", + server_only: "", } } @@ -167,6 +183,14 @@ const fn keyed(mut op: Op, key: &'static str) -> Op { op } +/// An operation of berthd that `berth serve` calls itself after checks of +/// its own, and that no caller reaches through it. +const fn server_only(mut op: Op, why: &'static str) -> Op { + op.server_only = why; + op.http_only = why; + op +} + const fn headers(mut op: Op, headers: &'static [(&'static str, &'static str)]) -> Op { op.headers = headers; op @@ -188,12 +212,12 @@ pub static HOST: &[Op] = &[ call("sessions.input", "POST", "/v1/sessions/{id}/input"), call("sessions.resize", "POST", "/v1/sessions/{id}/resize"), call("sessions.signal", "POST", "/v1/sessions/{id}/signal"), - call("tasks.list", "GET", "/v1/tasks"), - call("tasks.run", "POST", "/v1/tasks/{name}"), + call("actions.list", "GET", "/v1/actions"), + call("actions.run", "POST", "/v1/actions/{action}"), call( - "tasks.choices", + "actions.choices", "GET", - "/v1/tasks/{name}/choices/{argument}", + "/v1/actions/{action}/choices/{argument}", ), call("system.stats", "GET", "/v1/stats"), call("system.power", "POST", "/v1/power/{verb}"), @@ -218,6 +242,11 @@ pub static HOST: &[Op] = &[ "/v1/services/{plugin}/{name}/{method}", ), stream("services.serve", "/v1/services/{plugin}/{name}/serve"), + stream("ports.connect", "/v1/ports/{port}"), + server_only( + stream("endpoints.connect", "/v1/endpoints/{address}/{port}"), + "berth serve dials it itself, only for an endpoint a provisioner reported: open the endpoint with resources.connect", + ), ]; const IF_MATCH: (&str, &str) = ("if_match", "if-match"); @@ -248,9 +277,13 @@ pub static SERVE: &[Op] = &[ call("hosts.act", "POST", "/v1/hosts/{name}/actions/{action}"), Role::View, ), - call("actions.list", "GET", "/v1/actions"), + call("installation.actions", "GET", "/v1/installation/actions"), role( - call("actions.act", "POST", "/v1/actions/{action}"), + call( + "installation.act", + "POST", + "/v1/installation/actions/{action}", + ), Role::View, ), admin(call("secrets.list", "GET", "/v1/secrets")), @@ -321,10 +354,16 @@ pub static SERVE: &[Op] = &[ Role::Operate, ), stream("resources.follow", "/v1/resources/{id}/logs"), + stream( + "resources.connect", + "/v1/resources/{id}/endpoints/{endpoint}", + ), call("jobs.list", "GET", "/v1/jobs"), call("jobs.get", "GET", "/v1/jobs/{id}"), role(stream("jobs.follow", "/v1/jobs/{id}"), Role::View), call("jobs.output", "GET", "/v1/jobs/{id}/output"), + // Anyone let in may ask; the role that starts a job of its kind is checked. + role(call("jobs.stop", "POST", "/v1/jobs/{id}/stop"), Role::View), call("inbox.list", "GET", "/v1/inbox"), role(stream("inbox.follow", "/v1/inbox"), Role::View), call("inbox.done", "POST", "/v1/inbox/{host}/{id}/done"), @@ -445,6 +484,7 @@ mod tests { assert_eq!(role(HOST, "files.read"), Role::Operate); assert_eq!(role(HOST, "applied.profile"), Role::Admin); assert_eq!(role(HOST, "sessions.attach"), Role::Operate); + assert_eq!(role(HOST, "ports.connect"), Role::Operate); assert_eq!(role(SERVE, "hosts.save"), Role::Admin); assert_eq!(role(SERVE, "hosts.act"), Role::View); assert_eq!(role(SERVE, "provisioners.get"), Role::View); @@ -456,6 +496,7 @@ mod tests { assert_eq!(role(SERVE, "resources.act"), Role::View); assert_eq!(role(SERVE, "resources.logs"), Role::Operate); assert_eq!(role(SERVE, "resources.follow"), Role::Operate); + assert_eq!(role(SERVE, "resources.connect"), Role::Operate); let (op, args) = recognise( SERVE, Kind::Call, diff --git a/crates/berth-api/src/portal.rs b/crates/berth-api/src/portal.rs index 5f97bf8..fc0fa83 100644 --- a/crates/berth-api/src/portal.rs +++ b/crates/berth-api/src/portal.rs @@ -2,14 +2,14 @@ //! WebSocket. The frames are described in [`crate::ops`]. use crate::client::{Error, Result}; -use futures_util::{Sink, SinkExt, Stream, StreamExt}; +use futures_util::{Sink, SinkExt, Stream, StreamExt, task::AtomicWaker}; use serde_json::{Value, json}; use std::{ collections::HashMap, pin::Pin, sync::{ Arc, Mutex, - atomic::{AtomicU32, Ordering}, + atomic::{AtomicBool, AtomicI64, AtomicU32, AtomicUsize, Ordering}, }, task::{Context, Poll}, }; @@ -27,15 +27,152 @@ pub enum Piece { } enum Event { - Piece(Piece), + /// A frame, with the bytes it counts as unread. + Piece(Piece, usize), End(Option), } +/// The window a client asks for on each stream it opens: bytes of binary +/// frames on their way each way. Its frames wait in the other side's queue +/// for its reader, so this is what one stream can hold there. +pub const WINDOW: u32 = 256 * 1024; + +/// The least and the most window a server takes. +pub const WINDOWS: (u32, u32) = (16 * 1024, 4 << 20); + +/// The most bytes of one message on the portal: an operation's bytes as +/// base64, and room for the rest of the frame. +pub const FRAME_BYTES: usize = crate::ops::PORTAL_BYTES / 3 * 4 + (1 << 20); + +/// The most bytes a server holds for one portal's calls in flight, their +/// answers until they are written, and input waiting for its stream. +pub const HELD_BYTES: usize = 64 << 20; + +/// The most calls in flight on one portal. +pub const CALLS: usize = 256; + +/// The most streams open on one portal. +pub const STREAMS: usize = 256; + +/// The most bytes of a stream a client holds that it has not read yet. A +/// server within the window sends far less; one that sends more ends the +/// stream. +pub const UNREAD_BYTES: usize = 16 << 20; + +/// The longest name of an operation a server takes. +pub const NAME_BYTES: usize = 128; + +/// The longest message of an error a server sends back. +pub const MESSAGE_BYTES: usize = 2048; + +/// A message cut to [`MESSAGE_BYTES`], at a character. +pub fn shorten(message: &str) -> String { + if message.len() <= MESSAGE_BYTES { + return message.to_owned(); + } + let mut end = MESSAGE_BYTES - 3; + while !message.is_char_boundary(end) { + end -= 1; + } + format!("{}…", &message[..end]) +} + +/// What one side of a stream may still send, by what the other side said it +/// took. Without a window it may always send. +#[derive(Default)] +pub struct Credit { + on: AtomicBool, + room: AtomicI64, + waiting: AtomicWaker, +} + +impl Credit { + pub fn with(window: Option) -> Credit { + let credit = Credit::default(); + if let Some(window) = window { + credit.start(window); + } + credit + } + + pub fn start(&self, window: u32) { + self.room.store(window as i64, Ordering::SeqCst); + self.on.store(true, Ordering::SeqCst); + } + + /// The other side took so many more bytes. + pub fn grant(&self, bytes: u64) { + self.room + .fetch_add(bytes.min(i64::MAX as u64) as i64, Ordering::SeqCst); + self.waiting.wake(); + } + + /// So many bytes were sent. One frame may go beyond the window, so a + /// frame larger than it is not held forever. + pub fn spend(&self, bytes: usize) { + if self.on.load(Ordering::SeqCst) { + self.room.fetch_sub(bytes as i64, Ordering::SeqCst); + } + } + + /// Ready once a frame may be sent. One task waits on it at a time. + pub fn poll_room(&self, cx: &mut Context<'_>) -> Poll<()> { + let open = || !self.on.load(Ordering::SeqCst) || self.room.load(Ordering::SeqCst) > 0; + if open() { + return Poll::Ready(()); + } + self.waiting.register(cx.waker()); + // A grant between the look and the registration would be missed otherwise. + if open() { + Poll::Ready(()) + } else { + Poll::Pending + } + } + + pub async fn room(&self) { + std::future::poll_fn(|cx| self.poll_room(cx)).await + } +} + +/// What the receiving side of a stream took and has not yet said: the credit +/// to send, once it is half the window, so not every frame costs one. +pub struct Taken { + window: Option, + bytes: u64, +} + +impl Taken { + pub fn new(window: Option) -> Taken { + Taken { window, bytes: 0 } + } + + /// The frame that says what was taken, when it is time for one. + pub fn took(&mut self, sub: u32, bytes: usize) -> Option { + let window = self.window?; + self.bytes += bytes as u64; + if self.bytes < u64::from(window / 2) { + return None; + } + let credit = std::mem::take(&mut self.bytes); + Some(json!({"sub": sub, "credit": credit})) + } +} + +struct Open { + frames: mpsc::UnboundedSender, + credit: Arc, + /// Bytes of its frames that are not read yet. + unread: Arc, +} + #[derive(Default)] struct Shared { next: AtomicU32, calls: Mutex>>>, - streams: Mutex>>, + // Unbounded so the reader never waits on one stream: what the server may + // send a stream with a window is bounded by it. + streams: Mutex>, /// Why the connection ended, once it has. closed: Mutex>, } @@ -48,7 +185,9 @@ impl Shared { let _ = call.send(Err(Error::Transport(reason.to_owned()))); } for (_, stream) in self.streams.lock().unwrap().drain() { - let _ = stream.send(Event::End(Some(Error::Transport(reason.to_owned())))); + let _ = stream + .frames + .send(Event::End(Some(Error::Transport(reason.to_owned())))); } } } @@ -99,19 +238,20 @@ impl Portal { let _ = sink.close().await; }); let reader = shared.clone(); + let answer = out.clone(); tokio::spawn(async move { let reason = loop { match source.next().await { Some(Ok(Message::Text(text))) => { + let size = text.len(); if let Ok(frame) = serde_json::from_str::(text.as_str()) { - reader.take(frame); + reader.take(frame, size, &answer); } } Some(Ok(Message::Binary(bytes))) => { - if let Some((id, bytes)) = untagged(&bytes) - && let Some(stream) = reader.streams.lock().unwrap().get(&id) - { - let _ = stream.send(Event::Piece(Piece::Binary(bytes.to_vec()))); + if let Some((id, bytes)) = untagged(&bytes) { + let piece = Piece::Binary(bytes.to_vec()); + reader.give(id, piece, bytes.len(), &answer); } } Some(Ok(Message::Close(_))) | None => break "The connection closed".to_owned(), @@ -135,6 +275,7 @@ impl Portal { host: Option<&str>, args: Value, data: Option, + window: Option, ) -> Result<(u32, oneshot::Receiver>)> { if let Some(reason) = self.shared.closed.lock().unwrap().clone() { return Err(Error::Transport(reason)); @@ -156,6 +297,9 @@ impl Portal { if let Some(data) = data { frame["data"] = data; } + if let Some(window) = window { + frame["window"] = json!(window); + } if self .out .send(Message::Text(frame.to_string().into())) @@ -175,28 +319,46 @@ impl Portal { args: Value, data: Option, ) -> Result { - let (_, waiting) = self.request(op, host, args, data)?; + let (_, waiting) = self.request(op, host, args, data, None)?; waiting .await .unwrap_or_else(|_| Err(Error::Transport("The connection closed".into()))) } - /// Open a streaming operation. + /// Open a streaming operation, with a window on its bytes when the + /// server takes one. pub async fn open(&self, op: &str, host: Option<&str>, args: Value) -> Result { let (frames, incoming) = mpsc::unbounded_channel(); + let credit = Arc::new(Credit::default()); + let unread = Arc::new(AtomicUsize::new(0)); let (id, waiting) = { // Registered before the request leaves, so no frame can come too early. let mut streams = self.shared.streams.lock().unwrap(); - let (id, waiting) = self.request(op, host, args, None)?; - streams.insert(id, frames); + let (id, waiting) = self.request(op, host, args, None, Some(WINDOW))?; + let open = Open { + frames, + credit: credit.clone(), + unread: unread.clone(), + }; + streams.insert(id, open); (id, waiting) }; let opened = waiting .await .unwrap_or_else(|_| Err(Error::Transport("The connection closed".into()))); - if let Err(error) = opened { - self.shared.streams.lock().unwrap().remove(&id); - return Err(error); + let opened = match opened { + Ok(opened) => opened, + Err(error) => { + self.shared.streams.lock().unwrap().remove(&id); + return Err(error); + } + }; + // A server that does not say a window keeps to none. + let window = opened["window"] + .as_u64() + .and_then(|window| u32::try_from(window).ok()); + if let Some(window) = window { + credit.start(window); } Ok(PortalStream { id, @@ -204,12 +366,65 @@ impl Portal { incoming, shared: self.shared.clone(), ended: false, + credit, + taken: Taken::new(window), + unread, }) } + + /// A TCP connection to a port on a host's loopback, through `ports.connect`. + pub async fn connect_port( + &self, + host: Option<&str>, + port: u16, + ) -> Result> { + let stream = self + .open("ports.connect", host, json!({"port": port})) + .await?; + Ok(crate::tunnel::Tunnel::new(stream)) + } + + /// A TCP connection to an endpoint a resource's provisioner reported, by + /// its name or its port, through `resources.connect`. + pub async fn connect_endpoint( + &self, + id: &str, + endpoint: &str, + ) -> Result> { + let stream = self + .open( + "resources.connect", + None, + json!({"id": id, "endpoint": endpoint}), + ) + .await?; + Ok(crate::tunnel::Tunnel::new(stream)) + } } impl Shared { - fn take(&self, frame: Value) { + /// A frame of the stream `id`, unless the server sent it more than the + /// client holds unread, which ends the stream. + fn give(&self, id: u32, piece: Piece, size: usize, out: &mpsc::UnboundedSender) { + let mut streams = self.streams.lock().unwrap(); + let Some(stream) = streams.get(&id) else { + return; + }; + let size = size.max(1); + if stream.unread.fetch_add(size, Ordering::SeqCst) + size <= UNREAD_BYTES { + let _ = stream.frames.send(Event::Piece(piece, size)); + return; + } + if let Some(stream) = streams.remove(&id) { + let _ = stream.frames.send(Event::End(Some(Error::Transport(format!( + "The server sent more than {UNREAD_BYTES} bytes of the stream that were not read yet" + ))))); + let end = json!({"sub": id, "end": true}).to_string(); + let _ = out.send(Message::Text(end.into())); + } + } + + fn take(&self, frame: Value, size: usize, out: &mpsc::UnboundedSender) { if let Some(id) = frame["id"].as_u64() { if let Some(call) = self.calls.lock().unwrap().remove(&(id as u32)) { let _ = call.send(match frame.get("error") { @@ -222,10 +437,19 @@ impl Shared { let mut streams = self.streams.lock().unwrap(); if frame["end"].as_bool() == Some(true) { if let Some(stream) = streams.remove(&id) { - let _ = stream.send(Event::End(frame.get("error").map(error_of))); + let _ = stream + .frames + .send(Event::End(frame.get("error").map(error_of))); } } else if let Some(stream) = streams.get(&id) { - let _ = stream.send(Event::Piece(Piece::Text(frame["data"].clone()))); + match frame["credit"].as_u64() { + Some(bytes) => stream.credit.grant(bytes), + None => { + let piece = Piece::Text(frame["data"].clone()); + drop(streams); + self.give(id, piece, size, out); + } + } } } } @@ -238,6 +462,12 @@ pub struct PortalStream { incoming: mpsc::UnboundedReceiver, shared: Arc, ended: bool, + /// What this side may still send. + credit: Arc, + /// What this side took and has not said yet. + taken: Taken, + /// Bytes of its frames that came and are not read yet. + unread: Arc, } impl PortalStream { @@ -251,7 +481,15 @@ impl PortalStream { return Poll::Ready(None); } match self.incoming.poll_recv(cx) { - Poll::Ready(Some(Event::Piece(piece))) => Poll::Ready(Some(Ok(piece))), + Poll::Ready(Some(Event::Piece(piece, size))) => { + self.unread.fetch_sub(size, Ordering::SeqCst); + if let Piece::Binary(bytes) = &piece + && let Some(credit) = self.taken.took(self.id, bytes.len()) + { + let _ = self.out.send(Message::Text(credit.to_string().into())); + } + Poll::Ready(Some(Ok(piece))) + } Poll::Ready(Some(Event::End(error))) => { self.ended = true; Poll::Ready(error.map(Err)) @@ -264,12 +502,16 @@ impl PortalStream { } } + /// Send a frame now. It does not wait for the window: the [`Sink`] does. pub fn push(&self, piece: Piece) -> Result<()> { let message = match piece { Piece::Text(data) => { Message::Text(json!({"sub": self.id, "data": data}).to_string().into()) } - Piece::Binary(bytes) => Message::Binary(tagged(self.id, &bytes).into()), + Piece::Binary(bytes) => { + self.credit.spend(bytes.len()); + Message::Binary(tagged(self.id, &bytes).into()) + } }; self.out .send(message) @@ -318,11 +560,12 @@ impl Stream for PortalStream { impl Sink for PortalStream { type Error = WsError; + /// Ready once the window has room. fn poll_ready( self: Pin<&mut Self>, - _: &mut Context<'_>, + cx: &mut Context<'_>, ) -> Poll> { - Poll::Ready(Ok(())) + self.credit.poll_room(cx).map(Ok) } fn start_send(self: Pin<&mut Self>, message: Message) -> std::result::Result<(), WsError> { @@ -350,3 +593,70 @@ impl Sink for PortalStream { Poll::Ready(Ok(())) } } + +#[cfg(test)] +mod tests { + use super::*; + use tokio_tungstenite::tungstenite::protocol::Role; + + #[tokio::test(flavor = "multi_thread")] + async fn a_server_that_floods_a_stream_ends_it() { + let (ours, theirs) = tokio::io::duplex(1 << 20); + let client = WebSocketStream::from_raw_socket(ours, Role::Client, None).await; + let mut server = WebSocketStream::from_raw_socket(theirs, Role::Server, None).await; + let portal = Portal::over(client); + let serving = tokio::spawn(async move { + let asked = loop { + if let Some(Ok(Message::Text(text))) = server.next().await { + break serde_json::from_str::(text.as_str()).unwrap(); + } + }; + let id = asked["id"].as_u64().unwrap() as u32; + let opened = json!({"id": id, "result": {"open": true, "window": WINDOWS.0}}); + server + .send(Message::Text(opened.to_string().into())) + .await + .unwrap(); + // Far past its window, and past what the client holds unread. + let chunk = vec![1u8; 64 * 1024]; + for _ in 0..(UNREAD_BYTES / chunk.len() + 8) { + if server + .send(Message::Binary(tagged(id, &chunk).into())) + .await + .is_err() + { + break; + } + } + // The client says it ended the stream. + loop { + match server.next().await { + Some(Ok(Message::Text(text))) => { + let frame: Value = serde_json::from_str(text.as_str()).unwrap(); + if frame["end"] == json!(true) { + return (frame, server); + } + } + Some(Ok(_)) => {} + other => panic!("{other:?}"), + } + } + }); + let mut stream = portal.open("flood", None, Value::Null).await.unwrap(); + let (ended, _server) = serving.await.unwrap(); + assert_eq!(ended["sub"], json!(stream.id)); + let mut read = 0; + let error = loop { + match stream.recv().await { + Some(Ok(Piece::Binary(bytes))) => read += bytes.len(), + Some(Ok(Piece::Text(_))) => {} + Some(Err(error)) => break error, + None => panic!("it ended without an error"), + } + }; + assert!(read <= UNREAD_BYTES, "{read}"); + assert!(error.to_string().contains("not read yet"), "{error}"); + // The portal itself goes on. + assert!(portal.usable()); + } +} diff --git a/crates/berth-api/src/profile.rs b/crates/berth-api/src/profile.rs index e365eef..6a3f2e5 100644 --- a/crates/berth-api/src/profile.rs +++ b/crates/berth-api/src/profile.rs @@ -60,8 +60,17 @@ pub fn operator(config: &Value) -> &str { /// keep what they say. Says whether it changed the profile. pub fn settle_after_apply(config: &mut Value) -> bool { let mut access = Access::of(config); - if config["profile"] == "container" - || level(config) != crate::HostLevel::Workstation + // A node's role gave the SSH user berth's key and left sudo as it was. + if host_type(config) == crate::HostType::Node { + if access.link.is_some() || access.user.is_empty() || access.auth != Auth::Password { + return false; + } + access.auth = Auth::Berth; + access.write(config); + return true; + } + if config["target"] == "container" + || host_type(config) != crate::HostType::Workstation || access.link.is_some() || access.user.is_empty() || access.user != operator(config) @@ -295,70 +304,264 @@ impl Access { } } -/// The level a profile means its host to have: `level`, a workstation when -/// it does not say, so profiles from before levels keep their meaning. -pub fn level(config: &Value) -> crate::HostLevel { - config["level"] +/// The type a profile means its host to be: `type`, a workstation when it +/// does not say, so profiles from before types keep their meaning. +pub fn host_type(config: &Value) -> crate::HostType { + config["type"] .as_str() - .and_then(crate::HostLevel::parse) + .and_then(crate::HostType::parse) .unwrap_or_default() } -/// Set the level a profile means its host to have. A workstation is left -/// unsaid, as profiles from before levels are. -pub fn set_level(config: &mut Value, level: crate::HostLevel) { +/// Set the type a profile means its host to be. A workstation is left +/// unsaid, as profiles from before types are. +pub fn set_host_type(config: &mut Value, host_type: crate::HostType) { if !config.is_object() { *config = json!({}); } - match level { - crate::HostLevel::Workstation => { + match host_type { + crate::HostType::Workstation => { if let Some(profile) = config.as_object_mut() { - profile.remove("level"); + profile.remove("type"); } } - other => config["level"] = json!(other.name()), + // A node has none of a workstation's accounts, so what described them goes. + crate::HostType::Node => { + if let Some(profile) = config.as_object_mut() { + for field in ACCOUNT_FIELDS { + profile.remove(*field); + } + profile.insert("type".into(), json!("node")); + } + if config["target"] == "laptop" { + config["target"] = json!("cloud"); + } + } + other => config["type"] = json!(other.name()), + } +} + +/// The fields of a profile that describe a workstation's agent and operator +/// accounts. A node has neither: sessions run as the SSH user. +pub const ACCOUNT_FIELDS: &[&str] = &[ + "agent_username", + "agent_password_hash", + "agent_passwordless_sudo", + "agent_privileges", + "agent_ssh_public_keys", + "agent_ssh_public_key", + "operator_username", + "operator_password_hash", + "operator_ssh_public_keys", + "operator_ssh_public_key", +]; + +/// Whether a field of a profile says something: not absent, empty or false. +fn said(value: &Value) -> bool { + match value { + Value::Null | Value::Bool(false) => false, + Value::String(text) => !text.trim().is_empty(), + Value::Array(items) => !items.is_empty(), + Value::Object(fields) => !fields.is_empty(), + _ => true, } } -/// The profile of a host berth only reaches or only controls: its level and -/// how it is reached, nothing of a workstation. -pub fn minimal(level: crate::HostLevel, access: &Access) -> Value { +/// What a target is installed on, as a node's role takes it: a server +/// unless the profile says a container. +pub fn node_target(config: &Value) -> &str { + match config["target"].as_str() { + Some("container") => "container", + _ => "cloud", + } +} + +/// The profile of a host berth only logs in to or only manages: its type +/// and how it is reached, nothing of a workstation. +pub fn minimal(host_type: crate::HostType, access: &Access) -> Value { let mut config = json!({}); - set_level(&mut config, level); + set_host_type(&mut config, host_type); access.write(&mut config); config } -/// What is wrong with a profile as a whole, in sentences: its level, its -/// `access`, and for a workstation the `builder` its set-up keeps. A host -/// of a lower level needs only to be reachable. +/// Give a profile of berth 0.5 or earlier the names fields have now: +/// `level` (`reachable`, `controller`, `workstation`) becomes `type` +/// (`ssh`, `managed`, a workstation left unsaid), and `profile`, what the +/// host is installed on, becomes `target`. What a later berth wrote stays. +/// The ways of joining the tailnet berth has no more go too +/// ([`take_old_tailnet`]), so a pasted key is never read again. Says whether +/// it changed the profile. +pub fn take_old_names(config: &mut Value) -> bool { + let mut changed = take_old_tailnet(config).is_some(); + let Some(profile) = config.as_object_mut() else { + return false; + }; + if let Some(level) = profile.remove("level") { + changed = true; + if !profile.contains_key("type") { + match level + .as_str() + .map(|text| (text, crate::HostType::from_level(text))) + { + Some((_, Some(crate::HostType::Workstation))) => {} + Some((_, Some(host_type))) => { + profile.insert("type".into(), json!(host_type.name())); + } + // One berth never wrote is kept, so that problems() names it. + _ => { + profile.insert("type".into(), level); + } + } + } + } + if profile.get("profile").is_some_and(Value::is_string) { + let target = profile.remove("profile").unwrap_or_default(); + changed = true; + profile.entry("target").or_insert(target); + } + changed +} + +/// Whether `name` can be a setting of the role in a profile's +/// `ansible_vars`: `devbox_` and a word. Ansible's own variables, such as +/// `ansible_connection` or `ansible_ssh_common_args`, say how and where +/// commands run, which berth sets itself from `access`. +pub fn role_variable(name: &str) -> bool { + name.strip_prefix("devbox_").is_some_and(|rest| { + !rest.is_empty() + && rest + .chars() + .all(|ch| ch.is_ascii_lowercase() || ch.is_ascii_digit() || ch == '_') + }) +} + +/// Whether `name` can be a plugin's feature, as `PLUGIN/FEATURE`. +pub fn feature_name(name: &str) -> bool { + let part = |part: &str| { + part.starts_with(|ch: char| ch.is_ascii_lowercase()) + && part + .chars() + .all(|ch| ch.is_ascii_lowercase() || ch.is_ascii_digit() || ch == '-' || ch == '_') + }; + name.split_once('/') + .is_some_and(|(plugin, feature)| part(plugin) && part(feature)) +} + +/// What of a profile's `ansible_vars` an apply leaves out: names that are +/// not settings of the role, and features that are not `PLUGIN/FEATURE`. +pub fn left_out(config: &Value) -> Vec { + let Some(vars) = config["ansible_vars"].as_object() else { + return vec![]; + }; + let mut out: Vec = vars + .keys() + .filter(|name| !role_variable(name)) + .cloned() + .collect(); + out.extend( + vars.get("devbox_plugin_features") + .and_then(Value::as_array) + .into_iter() + .flatten() + .filter(|feature| !feature.as_str().is_some_and(feature_name)) + .map(|feature| format!("the feature {feature}")), + ); + out +} + +/// What is wrong with a profile's `ansible_vars`, in sentences. +fn vars_problems(config: &Value) -> Vec { + let vars = &config["ansible_vars"]; + if vars.is_null() { + return vec![]; + } + let Some(vars) = vars.as_object() else { + return vec!["ansible_vars must be an object".into()]; + }; + let mut problems = vec![]; + let foreign: Vec<&str> = vars + .keys() + .filter(|name| !role_variable(name)) + .map(String::as_str) + .collect(); + if !foreign.is_empty() { + problems.push(format!( + "ansible_vars holds settings of the role, whose names start with devbox_, not {}. berth sets how it reaches the host from access", + foreign.join(", ") + )); + } + match vars.get("devbox_plugin_features") { + None => {} + Some(Value::Array(features)) => { + let bad: Vec = features + .iter() + .filter(|feature| !feature.as_str().is_some_and(feature_name)) + .map(Value::to_string) + .collect(); + if !bad.is_empty() { + problems.push(format!( + "devbox_plugin_features names features as PLUGIN/FEATURE, not {}", + bad.join(", ") + )); + } + } + Some(_) => problems.push("devbox_plugin_features must be a list".into()), + } + problems +} + +/// What is wrong with a profile as a whole, in sentences: its type, its +/// `access` and `ansible_vars`, and for a workstation the `builder` its set-up keeps. A host +/// of another type needs only to be reachable. pub fn problems(config: &Value) -> Vec { if !config.is_object() { return vec!["A profile is an object".into()]; } let mut problems = vec![]; - let level = match &config["level"] { - Value::Null => Some(crate::HostLevel::Workstation), - Value::String(text) => crate::HostLevel::parse(text), + let host_type = match &config["type"] { + Value::Null => Some(crate::HostType::Workstation), + Value::String(text) => crate::HostType::parse(text), _ => None, }; - let Some(level) = level else { + let Some(host_type) = host_type else { return vec![format!( - "level is reachable, controller or workstation, not {}", - config["level"] + "type is {}, not {}", + crate::HostType::choices(), + config["type"] )]; }; problems.extend(Access::problems(config)); - match level { - crate::HostLevel::Workstation if !config["builder"].is_object() => { + problems.extend(vars_problems(config)); + match host_type { + crate::HostType::Workstation if !config["builder"].is_object() => { problems.push("A workstation's profile has a builder object".into()) } - crate::HostLevel::Workstation => {} + crate::HostType::Workstation => {} _ if !Access::of(config).reachable() => problems.push(format!( - "A {level} host needs access.address, where berth reaches it" + "A host of the type {host_type} needs access.address, where berth reaches it" )), _ => {} } + if host_type == crate::HostType::Node { + let described: Vec<&str> = ACCOUNT_FIELDS + .iter() + .copied() + .filter(|field| said(&config[*field])) + .collect(); + if !described.is_empty() { + problems.push(format!( + "A node has no agent or operator accounts, so {} {} for a workstation: sessions on a node run as the SSH user. Remove {}, or make the host a workstation", + described.join(", "), + if described.len() == 1 { "is" } else { "are" }, + if described.len() == 1 { "it" } else { "them" } + )); + } + if config["target"] == "laptop" { + problems + .push("A node is a server: its target is cloud or container, not laptop".into()); + } + } problems } @@ -451,35 +654,70 @@ pub fn take_builder_access(config: &mut Value) -> Option { /// uses it itself, to create enrollment keys, and never gives it to a host. pub const TAILNET_API_KEY: &str = "tailnet_api_key"; -/// Whether the host gets Tailscale, which it does unless the profile says not. -pub fn installs_tailscale(config: &Value) -> bool { +/// Whether the host gets Tailscale and joins the tailnet, with a key berth +/// makes, which it does unless the profile says not. +pub fn joins_tailnet(config: &Value) -> bool { config["ansible_vars"]["devbox_install_tailscale"] .as_bool() .unwrap_or(true) } -/// Whether berth creates the host's enrollment key itself when one is needed: -/// switched on, and no key pasted into the profile. -pub fn joins_automatically(config: &Value) -> bool { - installs_tailscale(config) - && config["tailscale_auto_key"].as_bool().unwrap_or(false) - && config["tailscale_auth_key"] - .as_str() - .unwrap_or("") - .is_empty() +/// What a profile of an older berth said of joining the tailnet in ways +/// berth has no more. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct OldTailnet { + /// It held an enrollment key pasted into it. + pub pasted: bool, + /// It was to be joined by hand: berth was not to make its key. + pub by_hand: bool, +} + +/// Take out of a profile what older berths kept of joining the tailnet: a +/// pasted enrollment key (`tailscale_auth_key`, also as the role's variable), +/// the switch for keys berth makes (`tailscale_auto_key`), which is always on +/// now, and a login server of the host's own (`tailscale_login_server`), as +/// berth makes keys only at the one of its settings. Says what there was, or +/// `None` when there was nothing. +pub fn take_old_tailnet(config: &mut Value) -> Option { + let joins = joins_tailnet(config); + let profile = config.as_object_mut()?; + let filled = |value: &Option| { + value + .as_ref() + .and_then(Value::as_str) + .is_some_and(|key| !key.is_empty()) + }; + let key = profile.remove("tailscale_auth_key"); + let switch = profile.remove("tailscale_auto_key"); + let server = profile.remove("tailscale_login_server"); + let variable = profile + .get_mut("ansible_vars") + .and_then(Value::as_object_mut) + .and_then(|vars| vars.remove("devbox_tailscale_auth_key")); + if key.is_none() && switch.is_none() && server.is_none() && variable.is_none() { + return None; + } + let pasted = filled(&key) || filled(&variable); + // Older profiles stored the switch as a string too. + let automatic = matches!(&switch, Some(Value::Bool(true))) + || matches!(&switch, Some(Value::String(text)) if text == "true"); + Some(OldTailnet { + pasted, + by_hand: joins && !automatic, + }) } /// Where a QEMU guest on the user-mode network sees the machine it runs on. pub const QEMU_HOST: &str = "10.0.2.2/32"; -/// Make a profile of the `qemu` kind there once was a `cloud` one. That kind -/// let SSH in only from the QEMU host when no recovery network was named, so +/// Make a profile of the `qemu` target there once was a `cloud` one. That +/// target let SSH in only from the QEMU host when no recovery network was named, so /// it gets that network rather than SSH open to every address. pub fn retire_qemu(config: &mut Value) { - if config["profile"] != "qemu" { + if config["target"] != "qemu" { return; } - config["profile"] = json!("cloud"); + config["target"] = json!("cloud"); if !config["ansible_vars"].is_object() { config["ansible_vars"] = json!({}); } @@ -505,7 +743,7 @@ mod tests { for bad in ["-oProxyCommand=touch /tmp/x;#", "a b", "a@b", "a\tb"] { assert!(!valid_user(bad), "{bad}"); } - let problems = |access: Value| problems(&json!({"level": "reachable", "access": access})); + let problems = |access: Value| problems(&json!({"type": "ssh", "access": access})); assert!(problems(json!({"address": "192.0.2.10", "user": "ops"})).is_empty()); assert_eq!(problems(json!({"address": "-oProxyCommand=x"})).len(), 1); assert_eq!( @@ -515,16 +753,70 @@ mod tests { } #[test] - fn a_pasted_key_wins_over_the_automatic_one() { - assert!(!joins_automatically(&json!({}))); - assert!(joins_automatically(&json!({"tailscale_auto_key": true}))); - assert!(!joins_automatically( - &json!({"tailscale_auto_key": true, "tailscale_auth_key": "tskey-1"}) + fn a_host_joins_the_tailnet_unless_its_profile_says_not() { + assert!(joins_tailnet(&json!({}))); + assert!(!joins_tailnet( + &json!({"ansible_vars": {"devbox_install_tailscale": false}}) )); - assert!(!joins_automatically(&json!({ - "tailscale_auto_key": true, - "ansible_vars": {"devbox_install_tailscale": false}, - }))); + } + + #[test] + fn a_pasted_key_and_the_switch_of_older_profiles_are_dropped_on_reading() { + let mut pasted = json!({ + "hostname": "studio", + "tailscale_auth_key": "tskey-auth-kFAKE-0000", + "tailscale_auto_key": false, + "tailscale_login_server": "https://hs.example.org", + "ansible_vars": {"devbox_geometry": "1x1", "devbox_tailscale_auth_key": "tskey-auth-kFAKE-1111"}, + }); + assert!(take_old_names(&mut pasted)); + assert_eq!( + pasted, + json!({"hostname": "studio", "ansible_vars": {"devbox_geometry": "1x1"}}) + ); + assert!(!take_old_names(&mut pasted)); + let said = |mut config: Value| take_old_tailnet(&mut config); + assert_eq!( + said(json!({"tailscale_auth_key": "tskey-auth-kFAKE-0000"})), + Some(OldTailnet { + pasted: true, + by_hand: true + }) + ); + // A host that was to be joined by hand joins with berth's key from now on. + assert_eq!( + said(json!({"tailscale_auto_key": false, "tailscale_auth_key": ""})), + Some(OldTailnet { + pasted: false, + by_hand: true + }) + ); + assert_eq!( + said(json!({"tailscale_auto_key": "true"})), + Some(OldTailnet { + pasted: false, + by_hand: false + }) + ); + // Without Tailscale nobody was joining it by hand. + assert_eq!( + said( + json!({"tailscale_auto_key": false, "ansible_vars": {"devbox_install_tailscale": false}}) + ), + Some(OldTailnet { + pasted: false, + by_hand: false + }) + ); + assert_eq!( + said(json!({"tailscale_login_server": "https://hs.example.org"})), + Some(OldTailnet { + pasted: false, + by_hand: true + }) + ); + assert_eq!(said(json!({"hostname": "studio"})), None); + assert_eq!(said(json!([])), None); } #[test] @@ -580,12 +872,12 @@ mod tests { let mut named = json!({"operator_username": "admin", "builder": {}, "access": {"user": "admin", "sudo": "password"}}); assert!(settle_after_apply(&mut named)); assert!(!asks_become_pass(&named)); - // A container has no operator, and a host below a workstation was not set up by berth. + // A container has no operator, and a host of another type was not set up by berth. let mut container = - json!({"profile": "container", "access": {"user": "ops", "sudo": "password"}}); + json!({"target": "container", "access": {"user": "ops", "sudo": "password"}}); assert!(!settle_after_apply(&mut container)); - let mut controller = json!({"level": "controller", "access": {"address": "192.0.2.4", "user": "ops", "sudo": "password"}}); - assert!(!settle_after_apply(&mut controller)); + let mut managed = json!({"type": "managed", "access": {"address": "192.0.2.4", "user": "ops", "sudo": "password"}}); + assert!(!settle_after_apply(&mut managed)); } #[test] @@ -652,58 +944,175 @@ mod tests { } #[test] - fn a_profile_without_a_level_is_a_workstation() { - use crate::HostLevel; - assert_eq!(level(&json!({"builder": {}})), HostLevel::Workstation); + fn a_profile_without_a_type_is_a_workstation() { + use crate::HostType; + assert_eq!(host_type(&json!({"builder": {}})), HostType::Workstation); + assert_eq!(host_type(&json!({"type": "managed"})), HostType::Managed); + // One berth does not know reads as the default, and is named a problem. + assert_eq!(host_type(&json!({"type": "server"})), HostType::Workstation); assert_eq!( - level(&json!({"level": "controller"})), - HostLevel::Controller + problems(&json!({"type": "server"})), + ["type is ssh, managed, node or workstation, not \"server\""] ); - // One berth does not know reads as the default, and is named a problem. - assert_eq!(level(&json!({"level": "server"})), HostLevel::Workstation); - assert!(problems(&json!({"level": "server"}))[0].contains("not \"server\"")); let access = Access { address: "192.0.2.10".into(), user: "ops".into(), ..Access::default() }; - let reachable = minimal(HostLevel::Reachable, &access); + let ssh = minimal(HostType::Ssh, &access); assert_eq!( - reachable, - json!({"level": "reachable", "access": {"address": "192.0.2.10", "user": "ops", "auth": "berth"}}) + ssh, + json!({"type": "ssh", "access": {"address": "192.0.2.10", "user": "ops", "auth": "berth"}}) ); - assert!(problems(&reachable).is_empty()); - // Only a workstation needs the set-up's part; a lower level needs an address. + assert!(problems(&ssh).is_empty()); + // Only a workstation needs the set-up's part; another type needs an address. assert_eq!( problems(&json!({"access": {"address": "studio"}})), ["A workstation's profile has a builder object"] ); - assert!(problems(&json!({"level": "controller"}))[0].contains("access.address")); - let mut changed = reachable.clone(); - set_level(&mut changed, HostLevel::Workstation); - assert!(changed.get("level").is_none()); + assert!(problems(&json!({"type": "managed"}))[0].contains("access.address")); + let mut changed = ssh.clone(); + set_host_type(&mut changed, HostType::Workstation); + assert!(changed.get("type").is_none()); + } + + #[test] + fn a_node_has_no_accounts_of_a_workstation() { + use crate::HostType; + let node = json!({"type": "node", "target": "cloud", + "access": {"address": "192.0.2.20", "user": "root"}, + "ansible_vars": {"devbox_plugin_features": ["incus/server"]}}); + assert_eq!(host_type(&node), HostType::Node); + assert!(problems(&node).is_empty(), "{:?}", problems(&node)); + // Empty fields, as a profile from the example has them, say nothing. + let mut quiet = node.clone(); + quiet["agent_password_hash"] = json!(""); + quiet["operator_ssh_public_keys"] = json!([]); + assert!(problems(&quiet).is_empty()); + let mut desktop = node.clone(); + desktop["agent_username"] = json!("agent"); + desktop["operator_ssh_public_keys"] = json!(["ssh-ed25519 KEY"]); + desktop["target"] = json!("laptop"); + let found = problems(&desktop); + assert_eq!(found.len(), 2, "{found:?}"); + assert!(found[0].starts_with( + "A node has no agent or operator accounts, so agent_username, operator_ssh_public_keys are for a workstation" + )); + assert!(found[1].contains("not laptop")); + // Making a workstation a node takes its accounts out of the profile. + let mut workstation = desktop.clone(); + set_host_type(&mut workstation, HostType::Workstation); + set_host_type(&mut workstation, HostType::Node); + assert!( + problems(&workstation).is_empty(), + "{:?}", + problems(&workstation) + ); + assert_eq!(workstation["ansible_vars"], node["ansible_vars"]); + assert_eq!(node_target(&workstation), "cloud"); + assert_eq!(node_target(&json!({"target": "container"})), "container"); + } + + #[test] + fn a_node_applied_with_a_password_is_logged_in_to_with_berths_key_after() { + let mut node = json!({"type": "node", + "access": {"address": "192.0.2.20", "user": "root", "auth": "password"}}); + assert!(settle_after_apply(&mut node)); + assert_eq!(Access::of(&node).auth, Auth::Berth); + assert!(!settle_after_apply(&mut node)); + let mut keyed = json!({"type": "node", + "access": {"address": "192.0.2.20", "user": "root", "auth": "ssh", "sudo": "password"}}); + assert!(!settle_after_apply(&mut keyed)); + } + + #[test] + fn a_profile_of_berth_05_takes_the_names_of_now() { + for (level, written) in [ + ("reachable", json!("ssh")), + ("controller", json!("managed")), + ("workstation", Value::Null), + ("server", json!("server")), + ] { + let mut old = + json!({"level": level, "profile": "cloud", "access": {"address": "192.0.2.4"}}); + assert!(take_old_names(&mut old), "{level}"); + assert_eq!(old["type"], written, "{level}"); + assert_eq!(old["target"], "cloud"); + assert!(old.get("level").is_none() && old.get("profile").is_none()); + assert!(!take_old_names(&mut old), "{level}"); + } + // What a later berth wrote wins, and the old field goes. + let mut both = json!({"type": "managed", "level": "reachable", "target": "laptop", "profile": "cloud"}); + assert!(take_old_names(&mut both)); + assert_eq!(both, json!({"type": "managed", "target": "laptop"})); + // A profile of now is left as it is. + let mut now = json!({"type": "ssh", "target": "cloud", "builder": {}}); + assert!(!take_old_names(&mut now)); + assert_eq!( + now, + json!({"type": "ssh", "target": "cloud", "builder": {}}) + ); + assert!(!take_old_names(&mut json!([]))); } #[test] fn a_qemu_profile_is_a_cloud_one_reached_from_the_qemu_host() { - let mut old = json!({"profile": "qemu"}); + let mut old = json!({"target": "qemu"}); retire_qemu(&mut old); assert_eq!( old, - json!({"profile": "cloud", "ansible_vars": {"devbox_bootstrap_ssh_cidr": "10.0.2.2/32"}}) + json!({"target": "cloud", "ansible_vars": {"devbox_bootstrap_ssh_cidr": "10.0.2.2/32"}}) ); // A network it names stays, and other kinds are left as they are. - let mut named = json!({"profile": "qemu", "ansible_vars": {"devbox_bootstrap_ssh_cidr": "192.0.2.0/24"}}); + let mut named = json!({"target": "qemu", "ansible_vars": {"devbox_bootstrap_ssh_cidr": "192.0.2.0/24"}}); retire_qemu(&mut named); - assert_eq!(named["profile"], "cloud"); + assert_eq!(named["target"], "cloud"); assert_eq!( named["ansible_vars"]["devbox_bootstrap_ssh_cidr"], "192.0.2.0/24" ); - for kept in [json!({"profile": "laptop"}), json!({"profile": "cloud"})] { + for kept in [json!({"target": "laptop"}), json!({"target": "cloud"})] { let mut profile = kept.clone(); retire_qemu(&mut profile); assert_eq!(profile, kept); } } + + #[test] + fn ansible_vars_hold_settings_of_the_role_alone() { + let with = |vars: Value| problems(&json!({"builder": {}, "ansible_vars": vars})); + assert!( + with(json!({"devbox_geometry": "1x1", "devbox_plugin_features": ["claude/cli"]})) + .is_empty() + ); + assert_eq!( + with( + json!({"ansible_become_exe": "/tmp/x", "ansible_python_interpreter": "/tmp/y", "devbox_ok": 1}) + ), + [ + "ansible_vars holds settings of the role, whose names start with devbox_, not ansible_become_exe, ansible_python_interpreter. berth sets how it reaches the host from access" + ] + ); + for name in [ + "devbox_", + "devbox_X", + "devbox_a-b", + "other", + "ansible_shell_executable", + ] { + assert!(!role_variable(name), "{name}"); + } + assert_eq!(with(json!([])), ["ansible_vars must be an object"]); + assert_eq!( + with(json!({"devbox_plugin_features": ["claude/cli", "../x/y", "a/../b", "plain"]})), + [ + r#"devbox_plugin_features names features as PLUGIN/FEATURE, not "../x/y", "a/../b", "plain""# + ] + ); + assert_eq!( + with(json!({"devbox_plugin_features": "claude/cli"})), + ["devbox_plugin_features must be a list"] + ); + assert!(feature_name("t3/server") && feature_name("my_plugin/a-b")); + } } diff --git a/crates/berth-api/src/streams.rs b/crates/berth-api/src/streams.rs index 91a3131..b77e8e7 100644 --- a/crates/berth-api/src/streams.rs +++ b/crates/berth-api/src/streams.rs @@ -161,8 +161,59 @@ pub const HOST: &[Stream] = &[ ), ], }, + Stream { + id: "port", + path: "/v1/ports/{port}", + summary: "A TCP connection to a port on the host's loopback, 127.0.0.1. The port is dialled first, so one that nothing listens on is an error and not a stream.", + level: "host", + parameters: &[], + resumable: false, + server: &[ + binary("bytes", "What the program on the port sent."), + text( + "eof", + "The program on the port closed its side. Bytes sent towards it still arrive. The stream ends once both sides have sent this.", + EOF, + ), + ], + client: &[ + binary("bytes", "Bytes for the program on the port."), + text( + "eof", + "The client sends no more, and the program on the port reads the end. What it sends still comes back.", + EOF, + ), + ], + }, + Stream { + id: "endpoint", + path: "/v1/endpoints/{address}/{port}", + summary: "For berth serve alone: a TCP connection to an address on a network of the host's own interfaces, such as a container on a bridge. The address is resolved once, every address it has is checked, and the one checked is dialled first, so one that nothing answers on is an error and not a stream.", + level: "host", + parameters: &[], + resumable: false, + server: &[ + binary("bytes", "What the endpoint sent."), + text( + "eof", + "The endpoint closed its side. Bytes sent towards it still arrive. The stream ends once both sides have sent this.", + EOF, + ), + ], + client: &[ + binary("bytes", "Bytes for the endpoint."), + text( + "eof", + "The client sends no more, and the endpoint reads the end. What it sends still comes back.", + EOF, + ), + ], + }, ]; +/// The frame by which one side of a stream of a port or an endpoint says it sends no more. +pub const EOF: &str = r#"{"eof":true}"#; + /// The streams `berth serve` adds to each host's own. pub const SERVE: &[Stream] = &[ Stream { @@ -202,6 +253,30 @@ pub const SERVE: &[Stream] = &[ ], client: &[], }, + Stream { + id: "resource-connect", + path: "/v1/resources/{id}/endpoints/{endpoint}", + summary: "A TCP connection to an endpoint its provisioner reported for a resource, named by the endpoint's name or its port. The server dials it first, so one that nothing answers on, or an address berth refuses, is an error and not a stream.", + level: "host", + parameters: &[], + resumable: false, + server: &[ + binary("bytes", "What the endpoint sent."), + text( + "eof", + "The endpoint closed its side. Bytes sent towards it still arrive. The stream ends once both sides have sent this.", + EOF, + ), + ], + client: &[ + binary("bytes", "Bytes for the endpoint."), + text( + "eof", + "The client sends no more, and the endpoint reads the end. What it sends still comes back.", + EOF, + ), + ], + }, Stream { id: "inbox", path: "/v1/inbox", @@ -212,11 +287,18 @@ pub const SERVE: &[Stream] = &[ ("topic", "Only topics starting with this."), ], resumable: false, - server: &[text( - "message", - "A stored message with its host.", - r#"{"host":"server","id":4812,"topic":"myplugin/progress","body":{},"keep":"log","from":{"level":"agent","uid":1001},"time":"2026-11-12T10:00:00Z","done":false}"#, - )], + server: &[ + text( + "message", + "A stored message with its host.", + r#"{"host":"server","id":4812,"topic":"myplugin/progress","body":{},"keep":"log","from":{"level":"agent","uid":1001},"time":"2026-11-12T10:00:00Z","done":false}"#, + ), + text( + "dropped", + "This follower was too slow. `events` counts the server's events it missed, of every kind. Read the inbox again.", + r#"{"dropped":{"events":12}}"#, + ), + ], client: &[], }, Stream { @@ -237,13 +319,18 @@ pub const SERVE: &[Stream] = &[ "A key was removed.", r#"{"key":"layout","removed":true}"#, ), + text( + "dropped", + "This follower was too slow. `events` counts the server's events it missed, of every kind. List the namespace again.", + r#"{"dropped":{"events":12}}"#, + ), ], client: &[], }, Stream { id: "events", path: "/v1/events", - summary: "Channel messages of chosen hosts, jobs and key-value changes together. The first frame sent names what to follow; each frame received says where it came from.", + summary: "Channel messages of chosen hosts, jobs, key-value changes and changes to hosts and resources together. The first frame sent names what to follow; each frame received says where it came from.", level: "host", parameters: &[], resumable: false, @@ -263,11 +350,26 @@ pub const SERVE: &[Stream] = &[ "A key changed.", r#"{"source":"kv","namespace":"ui","key":"layout","value":{},"revision":4}"#, ), + text( + "host", + "A saved host was `added`, `removed`, `renamed` (with the name it had `from`) or its profile `changed`. Read the host itself with hosts.get.", + r#"{"source":"host","host":"studio","change":"renamed","from":"desk"}"#, + ), + text( + "resource", + "A resource was `created`, `removed` or its record `changed`. Read it with resources.get.", + r#"{"source":"resource","resource":"r-k3v9qz2m","provisioner":"docker","kind":"deployment","change":"created"}"#, + ), + text( + "dropped", + "This follower was too slow. `events` counts the server's events it missed, of every kind. Read what it follows again.", + r#"{"dropped":{"events":12}}"#, + ), ], client: &[text( "follow", "What to follow. Sent once, first.", - r#"{"hosts":["server"],"topics":["myplugin/"],"jobs":true,"kv":["ui"]}"#, + r#"{"hosts":["server"],"topics":["myplugin/"],"jobs":true,"kv":["ui"],"changes":["hosts","resources"]}"#, )], }, ]; diff --git a/crates/berth-api/src/tunnel.rs b/crates/berth-api/src/tunnel.rs new file mode 100644 index 0000000..d0cd654 --- /dev/null +++ b/crates/berth-api/src/tunnel.rs @@ -0,0 +1,179 @@ +//! A TCP connection to a port of a host, as `ports.connect` gives it, or to +//! an endpoint of a resource, as `resources.connect` does: read and written +//! like a socket, over the stream's own WebSocket or the portal. + +use crate::client::{Frame, Result, WebSocketStream, WsError, websocket}; +use crate::streams::EOF; +use futures_util::{Sink, Stream}; +use std::{ + io, + pin::Pin, + task::{Context, Poll, ready}, +}; +use tokio::io::{AsyncRead, AsyncWrite, ReadBuf}; + +/// The most bytes one frame carries, so a large write does not become one +/// frame that the other side has to hold whole. +const FRAME: usize = 64 * 1024; + +/// The bytes of a stream of a port. Shutting down the writing side sends +/// the end, and reading goes on until the other side has sent its own. +/// Dropping it ends the connection. +pub struct Tunnel { + inner: S, + /// What was received and not read yet. + buffer: Vec, + at: usize, + /// The other side said it sends no more. + read_end: bool, + /// This side said it sends no more. + write_end: bool, +} + +/// Open a port of the host at the other end of `io`, a connection to its berthd. +pub async fn tunnel(io: S, port: u16) -> Result>> +where + S: tokio::io::AsyncRead + tokio::io::AsyncWrite + Unpin, +{ + let stream = websocket(io, &format!("/v1/ports/{port}"), "").await?; + Ok(Tunnel::new(stream)) +} + +/// Open `address` and `port` on a network of the host at the other end of +/// `io`, a connection to its berthd, as `berth serve` does for an endpoint. +pub async fn address_tunnel( + io: S, + address: &str, + port: u16, +) -> Result>> +where + S: tokio::io::AsyncRead + tokio::io::AsyncWrite + Unpin, +{ + let path = format!("/v1/endpoints/{}/{port}", crate::client::encode(address)); + let stream = websocket(io, &path, "").await?; + Ok(Tunnel::new(stream)) +} + +/// Open an endpoint of a resource, by its name or its port, through the +/// `berth serve` at the other end of `io`. `protocols` carries a bearer for +/// a server over the network, as the portal's handshake does. +pub async fn endpoint_tunnel( + io: S, + id: &str, + endpoint: &str, + protocols: &str, +) -> Result>> +where + S: tokio::io::AsyncRead + tokio::io::AsyncWrite + Unpin, +{ + let path = format!( + "/v1/resources/{}/endpoints/{}", + crate::client::encode(id), + crate::client::encode(endpoint) + ); + let stream = websocket(io, &path, protocols).await?; + Ok(Tunnel::new(stream)) +} + +impl Tunnel { + /// The stream of a port that is already open. + pub fn new(inner: S) -> Tunnel { + Tunnel { + inner, + buffer: vec![], + at: 0, + read_end: false, + write_end: false, + } + } + + pub fn into_inner(self) -> S { + self.inner + } +} + +fn broken(error: impl std::fmt::Display) -> io::Error { + io::Error::other(error.to_string()) +} + +fn said_end(text: &str) -> bool { + serde_json::from_str::(text).is_ok_and(|frame| frame["eof"] == true) +} + +impl AsyncRead for Tunnel +where + S: Stream> + Unpin, +{ + fn poll_read( + mut self: Pin<&mut Self>, + cx: &mut Context<'_>, + out: &mut ReadBuf<'_>, + ) -> Poll> { + let this = &mut *self; + loop { + if this.at < this.buffer.len() { + let count = out.remaining().min(this.buffer.len() - this.at); + out.put_slice(&this.buffer[this.at..this.at + count]); + this.at += count; + return Poll::Ready(Ok(())); + } + if this.read_end { + return Poll::Ready(Ok(())); + } + match ready!(Pin::new(&mut this.inner).poll_next(cx)) { + Some(Ok(Frame::Binary(bytes))) => { + this.buffer = bytes.to_vec(); + this.at = 0; + } + Some(Ok(Frame::Text(text))) if said_end(text.as_str()) => this.read_end = true, + Some(Ok(_)) => {} + Some(Err(error)) => return Poll::Ready(Err(broken(error))), + // Ended without saying so first: the connection was lost, not closed. + None => { + return Poll::Ready(Err(io::Error::new( + io::ErrorKind::UnexpectedEof, + "The connection to the port ended before its end", + ))); + } + } + } + } +} + +impl AsyncWrite for Tunnel +where + S: Sink + Unpin, +{ + fn poll_write( + mut self: Pin<&mut Self>, + cx: &mut Context<'_>, + data: &[u8], + ) -> Poll> { + if self.write_end { + return Poll::Ready(Err(io::ErrorKind::BrokenPipe.into())); + } + ready!(Pin::new(&mut self.inner).poll_ready(cx)).map_err(broken)?; + let count = data.len().min(FRAME); + let frame = Frame::Binary(data[..count].to_vec().into()); + Pin::new(&mut self.inner) + .start_send(frame) + .map_err(broken)?; + Poll::Ready(Ok(count)) + } + + fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> { + Pin::new(&mut self.inner).poll_flush(cx).map_err(broken) + } + + /// Say the end and leave the other direction open. + fn poll_shutdown(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> { + if !self.write_end { + ready!(Pin::new(&mut self.inner).poll_ready(cx)).map_err(broken)?; + Pin::new(&mut self.inner) + .start_send(Frame::Text(EOF.into())) + .map_err(broken)?; + self.write_end = true; + } + Pin::new(&mut self.inner).poll_flush(cx).map_err(broken) + } +} diff --git a/crates/berth-api/src/types.rs b/crates/berth-api/src/types.rs index 27b615c..40ac58e 100644 --- a/crates/berth-api/src/types.rs +++ b/crates/berth-api/src/types.rs @@ -28,6 +28,23 @@ pub struct ErrorDetail { pub message: String, } +/// The header that repeats an error's code, for a caller that is not given +/// the body: a WebSocket handshake that was refused shows only its head. +pub const ERROR_HEADER: &str = "berth-error"; + +/// The header that repeats an error's message for the same caller, escaped +/// as a part of a path is and cut short. +pub const ERROR_MESSAGE_HEADER: &str = "berth-error-message"; + +/// What [`ERROR_MESSAGE_HEADER`] says of `message`. +pub fn error_message_header(message: &str) -> String { + let end = (0..=message.len().min(512)) + .rev() + .find(|at| message.is_char_boundary(*at)) + .unwrap_or(0); + crate::escape::encode(&message[..end]) +} + #[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] pub struct ErrorBody { pub error: ErrorDetail, @@ -44,15 +61,33 @@ pub struct Version { /// recorded it. An older berthd does not say. #[serde(default, skip_serializing_if = "Option::is_none")] pub applied: Option, - /// What was installed on the host, by that record: `controller` when + /// What was installed on the host, by that record: `managed` when /// only the controller was, `workstation` for a whole profile. Absent /// when the host keeps no record. - #[serde(default, skip_serializing_if = "Option::is_none")] - pub level: Option, + #[serde(rename = "type", default, skip_serializing_if = "Option::is_none")] + pub host_type: Option, +} + +impl Version { + /// A berthd's answer to `version`. A berthd of API 1 named the type + /// `level`, with the values `reachable`, `controller` and + /// `workstation`; its answer is read as this API's. + pub fn from_reply(mut reply: serde_json::Value) -> serde_json::Result { + if let Some(object) = reply.as_object_mut() + && !object.contains_key("type") + && let Some(level) = object.remove("level") + { + let converted = level.as_str().and_then(HostType::from_level); + if let Some(host_type) = converted { + object.insert("type".into(), serde_json::json!(host_type.name())); + } + } + serde_json::from_value(reply) + } } -/// How much of berth a host has: what its profile means it to have, or -/// what it was seen to have. Each level can do what the ones before it can. +/// What berth does with a host: what its profile means it to be, or what it +/// was seen to be. They are in the order of how much of berth each has. #[derive( Debug, Clone, @@ -68,49 +103,78 @@ pub struct Version { ToSchema, )] #[serde(rename_all = "lowercase")] -pub enum HostLevel { +pub enum HostType { /// berth logs in over SSH. Nothing of berth is installed. - Reachable, - /// berthd and box are installed and answer: sessions, tasks and files. - Controller, + Ssh, + /// berthd and box are installed and answer: sessions, actions and files. + Managed, + /// A server berth applies without a desktop: berthd, updates, the + /// firewall and plugin features, such as the engines that host machines. + Node, /// The whole profile is applied: desktop, agents and plugins. #[default] Workstation, } -impl HostLevel { - pub const ALL: [HostLevel; 3] = [ - HostLevel::Reachable, - HostLevel::Controller, - HostLevel::Workstation, +impl HostType { + pub const ALL: [HostType; 4] = [ + HostType::Ssh, + HostType::Managed, + HostType::Node, + HostType::Workstation, ]; pub fn name(self) -> &'static str { match self { - HostLevel::Reachable => "reachable", - HostLevel::Controller => "controller", - HostLevel::Workstation => "workstation", + HostType::Ssh => "ssh", + HostType::Managed => "managed", + HostType::Node => "node", + HostType::Workstation => "workstation", } } - pub fn parse(text: &str) -> Option { - HostLevel::ALL + pub fn parse(text: &str) -> Option { + HostType::ALL .into_iter() - .find(|level| level.name() == text.trim()) + .find(|host_type| host_type.name() == text.trim()) } - /// What a host of this level has, in a few words that fit a line - /// beside the level on a narrow page. + /// The names of the types, as a sentence says them: `a, b or c`. + pub fn choices() -> String { + let names: Vec<&str> = HostType::ALL.iter().map(|t| t.name()).collect(); + match names.split_last() { + Some((last, [])) => (*last).to_owned(), + Some((last, rest)) => format!("{} or {last}", rest.join(", ")), + None => String::new(), + } + } + + /// The type a value of the `level` that profiles and berthd had before + /// types stands for. The role still records the type so, and names a + /// node `node`. + pub fn from_level(level: &str) -> Option { + match level.trim() { + "reachable" => Some(HostType::Ssh), + "controller" => Some(HostType::Managed), + "node" => Some(HostType::Node), + "workstation" => Some(HostType::Workstation), + _ => None, + } + } + + /// What a host of this type has, in a few words that fit a line + /// beside the type on a narrow page. pub fn describe(self) -> &'static str { match self { - HostLevel::Reachable => "SSH only, nothing of berth installed", - HostLevel::Controller => "berthd alone: sessions, tasks, files", - HostLevel::Workstation => "the whole profile", + HostType::Ssh => "SSH only, nothing of berth installed", + HostType::Managed => "berthd alone: sessions, actions, files", + HostType::Node => "a server: berthd, firewall, plugin features", + HostType::Workstation => "the whole profile", } } } -impl std::fmt::Display for HostLevel { +impl std::fmt::Display for HostType { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { f.write_str(self.name()) } @@ -118,18 +182,15 @@ impl std::fmt::Display for HostLevel { /// What to say when a host is seen to be more than its profile here says, /// with how to adopt it: as when a workstation is added by its address. -pub fn level_mismatch( - name: &str, - intended: HostLevel, - observed: Option, -) -> Option { +pub fn type_mismatch(name: &str, intended: HostType, observed: Option) -> Option { let observed = observed.filter(|observed| *observed > intended)?; Some(match observed { - HostLevel::Workstation => format!( - "{name} is a workstation; its profile here says {intended}. To adopt it from the profile it keeps: berth adopt {name}" + // Both keep the profile they were applied from. + HostType::Workstation | HostType::Node => format!( + "{name} is a {observed}; its profile here says {intended}. To adopt it from the profile it keeps: berth host adopt {name}" ), _ => format!( - "{name} has berth's controller; its profile here says {intended}. To adopt it: berth host level {name} controller" + "{name} has berth's controller; its profile here says {intended}. To adopt it: berth host set {name} type=managed" ), }) } @@ -234,9 +295,10 @@ pub struct Session { pub command: Vec, pub user: String, pub started_by: From, - /// The task it was started for, if any. - #[serde(default, skip_serializing_if = "Option::is_none")] - pub task: Option, + /// The action it was started for, if any. The sessions an older + /// berthd kept on the host say `task`. + #[serde(default, alias = "task", skip_serializing_if = "Option::is_none")] + pub action: Option, pub started: String, #[serde(default, skip_serializing_if = "Option::is_none")] pub ended: Option, @@ -331,7 +393,7 @@ pub struct Signal { pub signal: String, } -// ── Tasks ─────────────────────────────────────────────────────────────────── +// ── A host's actions ─────────────────────────────────────────────────────────────── #[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize, ToSchema)] pub struct Argument { @@ -374,11 +436,30 @@ impl Argument { pub fn is_resource(&self) -> bool { self.kind == Self::RESOURCE } + + /// The arguments of `given` a job may show as `NAME=VALUE`, in the order + /// they are declared: never a secret one, nor one of several lines, such + /// as what a command reads on its standard input. + pub fn shown( + declared: &[Argument], + given: &std::collections::BTreeMap, + ) -> Vec { + declared + .iter() + .filter(|argument| !argument.secret && argument.kind != Self::TEXT) + .filter_map(|argument| { + let value = given.get(&argument.name)?; + (!value.is_empty() && !value.contains('\n')) + .then(|| format!("{}={value}", argument.name)) + }) + .collect() + } } -/// An entry of `/etc/berth/tasks.json`. Listed without `run`. +/// An action of the host's controller: an entry of `/etc/berth/actions.json`. +/// Listed without `run`. #[derive(Debug, Clone, Default, Serialize, Deserialize, ToSchema)] -pub struct Task { +pub struct HostAction { pub name: String, #[serde(default)] pub label: String, @@ -393,7 +474,7 @@ pub struct Task { /// The agent may run it. #[serde(default)] pub agent: bool, - /// Tasks it is a step of; it then has no row of its own. + /// Actions it is a step of; it then has no row of its own. #[serde(default)] pub extends: Vec, #[serde(default)] @@ -408,7 +489,7 @@ pub struct Task { } #[derive(Debug, Clone, Default, Serialize, Deserialize, ToSchema)] -pub struct RunTask { +pub struct RunAction { #[serde(default)] pub arguments: BTreeMap, #[serde(default, skip_serializing_if = "Option::is_none")] @@ -513,26 +594,26 @@ impl Default for Action { } impl Action { - /// The actions of a host's tasks, in the order the host lists them: each - /// task that is no step of another, with the steps plugins add to it. The - /// host keeps the rest of a task (its user, its command, its input), and - /// runs it by name. - pub fn of_tasks(tasks: &[Task]) -> Vec { - tasks + /// The actions of a host's controller as actions to describe, in the + /// order the host lists them: each that is no step of another, with the + /// steps plugins add to it. The host keeps the rest of one (its user, its + /// command, its input), and runs it by name. + pub fn of_host(listed: &[HostAction]) -> Vec { + listed .iter() - .filter(|task| task.extends.is_empty()) - .map(|task| Action { - name: task.name.clone(), - label: task.label.clone(), - help: task.help.clone(), - group: task.group.clone(), - arguments: task.arguments.clone(), - confirm: task.confirm.clone(), + .filter(|action| action.extends.is_empty()) + .map(|action| Action { + name: action.name.clone(), + label: action.label.clone(), + help: action.help.clone(), + group: action.group.clone(), + arguments: action.arguments.clone(), + confirm: action.confirm.clone(), runs: Runs::Host, - terminal: task.terminal, - steps: tasks + terminal: action.terminal, + steps: listed .iter() - .filter(|step| step.extends.contains(&task.name)) + .filter(|step| step.extends.contains(&action.name)) .map(|step| ActionStep { label: step.label.clone(), help: step.help.clone(), @@ -606,6 +687,14 @@ pub struct JobInfo { /// How a roll-out went, host by host, once it has ended. #[serde(default, skip_serializing_if = "Option::is_none")] pub report: Option, + /// Who stopped it before it ended by itself. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub stopped_by: Option, + /// What it was started with, as people may see it: the command of a + /// resource's `exec`, or an action's arguments as `NAME=VALUE`. Never a + /// secret argument, a password or standard input. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub arguments: Vec, } // ── System ────────────────────────────────────────────────────────────────── @@ -889,10 +978,10 @@ pub enum PluginOrigin { Folder, } -/// A task a feature adds to a host's overview. +/// An action a feature adds to a host. #[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize, ToSchema)] -pub struct PluginTask { - /// The task is `PLUGIN:NAME` on the host. +pub struct PluginAction { + /// The action is `PLUGIN:NAME` on the host. pub name: String, pub label: String, /// `agent`, `operator` or `root`. @@ -900,7 +989,7 @@ pub struct PluginTask { /// The agent may run it with `box run`. #[serde(default)] pub agent: bool, - /// Tasks of berth it runs after, with no row of its own. + /// Actions of berth it runs after, with no row of its own. #[serde(default)] pub extends: Vec, /// The names of the arguments it asks for. @@ -929,7 +1018,7 @@ pub struct PluginFeature { #[serde(default)] pub help: String, #[serde(default)] - pub tasks: Vec, + pub actions: Vec, #[serde(default)] pub services: Vec, /// Channel topics, without the plugin's id. @@ -1041,6 +1130,9 @@ pub struct ProvisionerInfo { /// The values of its own arguments that are kept, by name. #[serde(default, skip_serializing_if = "BTreeMap::is_empty")] pub settings: BTreeMap, + /// The node it lives on, when a feature of that host offered it. + #[serde(default, skip_serializing_if = "String::is_empty")] + pub node: String, /// What it can make: `machine`, and kinds its plugin names. #[serde(default, skip_serializing_if = "Vec::is_empty")] pub kinds: Vec, @@ -1164,6 +1256,81 @@ impl ResourceSecret { } } +/// Where a resource can be reached: an address and a port, as its +/// provisioner reports it. The address is as seen from the machine the +/// provisioner's engine runs on, such as a Docker engine or an Incus server. +#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize, ToSchema)] +pub struct Endpoint { + /// Unique among the resource's endpoints: lowercase letters, digits, `-`, `_` and `.`. + pub name: String, + /// An IP address or a DNS name. + pub address: String, + pub port: u16, + /// `tcp` or `udp`. + pub protocol: String, + /// What speaks there, for people, such as `http` or `postgres`. + #[serde(default, skip_serializing_if = "String::is_empty")] + pub hint: String, + /// The engine runs on the server's own machine, so the address is one + /// the server reaches as it is, a loopback address among them. + #[serde(default, skip_serializing_if = "std::ops::Not::not")] + pub local: bool, +} + +impl Endpoint { + pub const TCP: &'static str = "tcp"; + pub const UDP: &'static str = "udp"; + + /// Whether a text can be an endpoint's name. + pub fn valid_name(name: &str) -> bool { + !name.is_empty() + && name.len() <= 63 + && name.starts_with(|ch: char| ch.is_ascii_alphanumeric()) + && name + .chars() + .all(|ch| ch.is_ascii_lowercase() || ch.is_ascii_digit() || "-_.".contains(ch)) + } + + /// One endpoint of a provisioner's answer, or why it is none. + pub fn from_answer(value: &Value) -> Result { + let text = |key: &str| value[key].as_str().unwrap_or("").trim().to_owned(); + let name = text("name"); + if !Endpoint::valid_name(&name) { + return Err(format!("{name:?} cannot name an endpoint")); + } + let address = text("address"); + let usable = !address.is_empty() + && address.len() <= 253 + && (address.parse::().is_ok() + || address + .chars() + .all(|ch| ch.is_ascii_alphanumeric() || "-.".contains(ch))); + if !usable { + return Err(format!("the endpoint {name} has no address berth can use")); + } + let port = value["port"] + .as_u64() + .or_else(|| value["port"].as_str().and_then(|port| port.parse().ok())) + .and_then(|port| u16::try_from(port).ok()) + .filter(|port| *port > 0) + .ok_or_else(|| format!("the endpoint {name} has no port from 1 to 65535"))?; + let protocol = match text("protocol").to_ascii_lowercase().as_str() { + "" | Endpoint::TCP => Endpoint::TCP.to_owned(), + Endpoint::UDP => Endpoint::UDP.to_owned(), + other => return Err(format!("the endpoint {name} has the protocol {other:?}")), + }; + let hint: String = text("hint").chars().take(32).collect(); + Ok(Endpoint { + name, + address, + port, + protocol, + hint, + local: value["local"] == true, + }) + } +} + /// Something a provisioner made for berth: a machine, which may be a /// host's, or a kind its plugin names, such as a deployment. #[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize, ToSchema)] @@ -1186,6 +1353,10 @@ pub struct Resource { pub host: Option, #[serde(default, skip_serializing_if = "String::is_empty")] pub address: String, + /// Where it can be reached, as its provisioner last said: when it was + /// made, changed or checked. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub endpoints: Vec, #[serde(default)] pub arguments: BTreeMap, /// The berth secrets it is given, by name; never their values. @@ -1201,7 +1372,7 @@ pub struct Resource { #[serde(default, skip_serializing_if = "String::is_empty")] pub checked: String, /// What can be done to it besides being removed: `start`, `stop`, - /// `change`, `logs`, as its kind declares. + /// `change`, `logs`, `exec`, as its kind declares. #[serde(default)] pub verbs: Vec, /// Why its state is `unknown`, or what is wrong with it. @@ -1272,12 +1443,45 @@ pub struct AppliedPlugin { } /// What berth recorded on a host when it applied the configuration: its own -/// version and the plugins whose features the host has. +/// version, the plugins whose features the host has, and the provisioners +/// those features offer. #[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize, ToSchema)] pub struct Applied { pub berth: String, #[serde(default)] pub plugins: Vec, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub offers: Vec, +} + +/// Provisioners a feature of a host offers, as its tasks found them: berth +/// registers each under a name of its own that names the host. The host +/// serves this to anyone who may read it, so it holds public values only, +/// such as a server's certificate, never a key. +#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize, ToSchema)] +pub struct Offer { + /// The feature, as `PLUGIN/FEATURE`. + pub feature: String, + /// The plugin's provisioners it offers, by their ids in the plugin. + pub provisioners: Vec, + /// Values of the provisioners' own arguments. `{address}`, `{user}` and + /// `{port}` stand for where berth reaches the host and as whom. + #[serde(default)] + pub settings: BTreeMap, + /// Values of the provisioners' secrets, such as the certificate of the + /// server to pin, kept as the provisioners' own. + #[serde(default)] + pub secrets: BTreeMap, + /// Other values of a setting that reach the same server, such as the + /// name a berth server on the host gives it: a provisioner registered by + /// hand with one of them is named to its owner, who can make it the + /// host's. + #[serde(default, skip_serializing_if = "BTreeMap::is_empty")] + pub aliases: BTreeMap>, + /// What the host says identifies the server, such as the fingerprint of + /// its certificate, for the person who reads the report. + #[serde(default, skip_serializing_if = "String::is_empty")] + pub fingerprint: String, } /// The profile a host keeps of itself, at `/etc/berth/profile.json`: the @@ -1389,10 +1593,13 @@ pub struct Rollout { /// These hosts, in this order; all saved hosts by name when empty. #[serde(default, skip_serializing_if = "Vec::is_empty")] pub hosts: Vec, + /// Hosts left out. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub except: Vec, /// Also the hosts that are current. #[serde(default)] pub all: bool, - /// Run each host's `update` task after applying. + /// Run each host's `update` action after applying. #[serde(default)] pub update: bool, /// Reboot a host that says it needs it, and wait for it to come back. @@ -1416,6 +1623,9 @@ impl Rollout { if !self.hosts.is_empty() { arguments.insert("hosts".into(), self.hosts.join(",")); } + if !self.except.is_empty() { + arguments.insert("except".into(), self.except.join(",")); + } arguments } @@ -1426,16 +1636,19 @@ impl Rollout { Some("yes" | "true") => Ok(true), Some(other) => Err(format!("{name} is yes or no, not {other}")), }; - let hosts = arguments.get("hosts").map_or(vec![], |hosts| { - hosts - .split(',') - .map(str::trim) - .filter(|host| !host.is_empty()) - .map(str::to_owned) - .collect() - }); + let names = |key: &str| { + arguments.get(key).map_or(vec![], |hosts| { + hosts + .split(',') + .map(str::trim) + .filter(|host| !host.is_empty()) + .map(str::to_owned) + .collect() + }) + }; let rollout = Rollout { - hosts, + hosts: names("hosts"), + except: names("except"), all: flag("all")?, update: flag("update")?, reboot: flag("reboot")?, @@ -1484,7 +1697,7 @@ pub struct RolloutHost { } /// Whether a line of a roll-out's output is one of its own, rather than -/// what Ansible or a task wrote: those start with ▸, ✓, ! or ✗. +/// what Ansible or an action wrote: those start with ▸, ✓, ! or ✗. pub fn rollout_line(line: &str) -> bool { ["▸ ", "✓ ", "! ", "✗ "] .iter() @@ -1498,3 +1711,112 @@ pub struct RolloutReport { /// Every host is done or current. pub ok: bool, } + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn a_job_shows_no_secret_argument_and_no_text() { + let declared = vec![ + Argument { + name: "size".into(), + ..Argument::default() + }, + Argument { + name: "password".into(), + secret: true, + ..Argument::default() + }, + Argument { + name: "stdin".into(), + kind: Argument::TEXT.into(), + ..Argument::default() + }, + Argument { + name: "empty".into(), + ..Argument::default() + }, + ]; + let given: std::collections::BTreeMap = [ + ("password", "hunter2"), + ("size", "2"), + ("stdin", "select 1;"), + ("empty", ""), + ("undeclared", "x"), + ] + .into_iter() + .map(|(name, value)| (name.into(), value.into())) + .collect(); + assert_eq!(Argument::shown(&declared, &given), ["size=2"]); + } + use serde_json::json; + + #[test] + fn a_host_type_is_named_and_parsed() { + for host_type in HostType::ALL { + assert_eq!(HostType::parse(host_type.name()), Some(host_type)); + assert_eq!(json!(host_type), json!(host_type.name())); + } + assert_eq!(HostType::choices(), "ssh, managed, node or workstation"); + // In the order of how much of berth each has. + assert!(HostType::Managed < HostType::Node && HostType::Node < HostType::Workstation); + assert_eq!(HostType::from_level("node"), Some(HostType::Node)); + assert_eq!( + type_mismatch("big", HostType::Managed, Some(HostType::Node)).unwrap(), + "big is a node; its profile here says managed. To adopt it from the profile it keeps: berth host adopt big" + ); + assert_eq!(HostType::parse("controller"), None); + assert_eq!(HostType::from_level("reachable"), Some(HostType::Ssh)); + assert_eq!(HostType::from_level("controller"), Some(HostType::Managed)); + assert_eq!(HostType::from_level("ssh"), None); + } + + #[test] + fn a_session_an_older_berthd_kept_names_its_action() { + let kept = json!({"id": "s1", "kind": "process", "title": "", "command": [], "user": "agent", + "started_by": {"level": "host", "uid": 0}, "task": "status", "started": "2026-11-12T10:00:00Z"}); + let session: Session = serde_json::from_value(kept).unwrap(); + assert_eq!(session.action.as_deref(), Some("status")); + let written = json!(session); + assert_eq!(written["action"], "status"); + assert!(written.get("task").is_none()); + } + + #[test] + fn an_apply_record_carries_what_a_node_offers() { + let record = json!({"berth": "0.6.0", "plugins": [], "level": "node", "offers": [{ + "feature": "incus/server", "provisioners": ["vm", "container"], + "settings": {"server": "https://{address}:8444"}, + "secrets": {"incus_server_cert": "-----BEGIN CERTIFICATE-----"}}]}); + let applied: Applied = serde_json::from_value(record).unwrap(); + assert_eq!(applied.offers[0].provisioners, ["vm", "container"]); + assert!(applied.offers[0].aliases.is_empty()); + // A record without offers is written as before. + let plain = json!(Applied { + berth: "0.6.0".into(), + ..Applied::default() + }); + assert!(plain.get("offers").is_none()); + } + + #[test] + fn the_version_of_an_older_berthd_is_read_with_its_type() { + let old = json!({"version": "0.5.1", "api": 1, "mode": "system", "level": "controller"}); + let version = Version::from_reply(old).unwrap(); + assert_eq!(version.host_type, Some(HostType::Managed)); + let new = json!({"version": "0.6.0", "api": 2, "type": "ssh"}); + assert_eq!( + Version::from_reply(new).unwrap().host_type, + Some(HostType::Ssh) + ); + let without = json!({"version": "0.5.1", "api": 1}); + assert_eq!(Version::from_reply(without).unwrap().host_type, None); + // What a berthd says now is written with the name of this API. + let written = json!( + Version::from_reply(json!({"version": "1", "api": 2, "level": "workstation"})).unwrap() + ); + assert_eq!(written["type"], "workstation"); + assert!(written.get("level").is_none()); + } +} diff --git a/crates/berth-core/Cargo.toml b/crates/berth-core/Cargo.toml index d903f29..ab151ac 100644 --- a/crates/berth-core/Cargo.toml +++ b/crates/berth-core/Cargo.toml @@ -24,4 +24,5 @@ chacha20poly1305.workspace = true ed25519-dalek = { version = "2", default-features = false } hmac = "0.12" sha1 = "0.10" +rcgen = { version = "0.14", default-features = false, features = ["ring", "pem"] } diff --git a/crates/berth-core/src/apply.rs b/crates/berth-core/src/apply.rs index 1ba056b..4e3607a 100644 --- a/crates/berth-core/src/apply.rs +++ b/crates/berth-core/src/apply.rs @@ -16,6 +16,9 @@ use std::{io::Write, process::Command}; pub struct Prepared { pub command: Command, pub files: Vec, + /// Lines for the output before the command's own, such as what of the + /// profile was left out. + pub notes: Vec, } /// Where each plugin is on this machine, by id. @@ -39,6 +42,24 @@ pub fn ansible_vars(config: &Value, plugin_dirs: &Value, own: &[String]) -> Valu if !vars.is_object() { vars = json!({}); } + // How the host joins the tailnet is berth's to say, not the profile's: + // the key command runs on this machine. + if let Some(vars) = vars.as_object_mut() { + for berths in crate::tailnet::ROLE_VARS { + vars.remove(*berths); + } + // Ansible's own variables would say how and where commands run, which + // berth says from `access`; a profile saved before this was checked + // may still hold them. + vars.retain(|name, _| berth_api::profile::role_variable(name)); + if let Some(Value::Array(features)) = vars.get_mut("devbox_plugin_features") { + features.retain(|feature| { + feature + .as_str() + .is_some_and(berth_api::profile::feature_name) + }); + } + } for (source, dest) in [ ("agent_username", "devbox_user"), ("agent_password_hash", "devbox_agent_password_hash"), @@ -62,25 +83,33 @@ pub fn ansible_vars(config: &Value, plugin_dirs: &Value, own: &[String]) -> Valu .filter(|key| !key.is_empty()) .collect(); vars["devbox_berth_ssh_public_key"] = json!(own.join("\n")); - vars["devbox_profile"] = config["profile"].clone(); + // The role names the target by its older name. + vars["devbox_profile"] = config["target"].clone(); vars["devbox_plugin_dirs"] = plugin_dirs.clone(); - if let Some(key) = config["tailscale_auth_key"] - .as_str() - .filter(|key| !key.is_empty()) - { - vars["devbox_tailscale_auth_key"] = json!(key); - } - if let Some(server) = config["tailscale_login_server"] - .as_str() - .filter(|server| !server.is_empty()) - { - vars["devbox_tailscale_login_server"] = json!(server); + // A node takes the role's path without the desktop, and only the features its profile names. + if berth_api::profile::host_type(config) == berth_api::HostType::Node { + vars["devbox_level"] = json!("node"); + vars["devbox_profile"] = json!(berth_api::profile::node_target(config)); + if !vars["devbox_plugin_features"].is_array() { + vars["devbox_plugin_features"] = json!([]); + } + for account in ["devbox_user", "devbox_operator_user"] { + if let Some(vars) = vars.as_object_mut() { + vars.remove(account); + } + } + if let Some(user) = config["access"]["user"] + .as_str() + .filter(|user| !user.is_empty()) + { + vars["devbox_node_user"] = json!(user); + } } vars } /// Add what joining the tailnet needs from this machine: berth's login -/// server, and for a host that joins automatically the way to a key, which +/// server, and the way to a key or why there is none, which /// names the host as `berth tailnet-key` finds it again: by its name, or by /// the path of its file. That command runs as a child of the apply, with its /// environment, so it looks in the same store. @@ -110,12 +139,14 @@ pub fn playbook(source: &std::path::Path) -> Command { /// The Ansible run that applies `host`'s profile to it over SSH. With a /// password it becomes root with it, taken from a private file; without one a -/// host that asks for a password is asked for on the terminal. Only a -/// workstation is applied: every way to apply comes through here. +/// host that asks for a password is asked for on the terminal. Only a node +/// or a workstation is applied: every way to apply comes through here. pub fn prepare(paths: &Paths, host: &Host, become_password: Option<&str>) -> Result { - crate::level::guard(host, crate::level::Capability::Apply)?; + crate::host_type::guard(host, crate::host_type::Capability::Apply)?; let source = paths.source()?; let plugins = plugins::load(&[source.join("plugins"), paths.config_dir.join("plugins")]); + crate::node::check_features(&plugins, &host.name, &host.config)?; + crate::node::check_removal(paths, host)?; let dirs = plugin_dirs(&plugins); // Without its own key in the operator's authorized_keys, berth could be // locked out of the host it sets up, so it does not go on without one. @@ -130,21 +161,36 @@ pub fn prepare(paths: &Paths, host: &Host, become_password: Option<&str>) -> Res }); // The role keeps the profile on the host too, for a berth that lacks it. vars["devbox_berth_profile"] = crate::kept_profile::for_host(paths, host); + // Only the certificates go to the host; their keys stay in berth's store. + let store = crate::registry::Store::own(crate::store::shared(paths)?); + vars["devbox_client_certificates"] = + crate::node::client_certificates(&plugins, &store, &host.config)?; let mut prepared = run(paths, host, layout::PLAYBOOK, vars, become_password)?; + let left = berth_api::profile::left_out(&host.config); + if !left.is_empty() { + prepared.notes.push(format!( + "! {}'s ansible_vars hold {}, which the role does not take: it is applied without {}. Remove {} from the profile", + host.name, + left.join(", "), + if left.len() == 1 { "it" } else { "them" }, + if left.len() == 1 { "it" } else { "them" }, + )); + } prepared.command.env("BERTH_PLUGINS", dirs.to_string()); prepared.command.env("BERTH_SSH_PUBLIC_KEY", own.join("\n")); Ok(prepared) } -/// The Ansible run that puts berth's controller alone on a host below a -/// workstation: berthd and box, what they read, and the record of what was -/// installed. The SSH user is the account sessions run as. +/// The Ansible run that puts berth's controller alone on a host that is not +/// a workstation: berthd and box, what they read, and the record of what +/// was installed. The SSH user is the account sessions run as. pub fn prepare_controller( paths: &Paths, host: &Host, become_password: Option<&str>, ) -> Result { - crate::level::guard(host, crate::level::Capability::InstallController)?; + crate::host_type::guard(host, crate::host_type::Capability::InstallController)?; + // The role calls the type managed by its older name. let mut vars = json!({ "devbox_level": "controller", "devbox_berth_applied": {"berth": berth_api::VERSION, "plugins": []}, @@ -179,7 +225,7 @@ fn run( // an askpass helper of its own, and takes it from this private file. let login = crate::keys::login_of(paths, host)?; let ssh_password = match login.auth { - berth_api::profile::Auth::Password if host.served(paths).is_none() => { + berth_api::profile::Auth::Password => { crate::registry::Store::own(crate::store::shared(paths)?) .secret(&crate::keys::password_secret(&host.place))? } @@ -190,10 +236,10 @@ fn run( } // Secrets go to private files rather than argv. let mut extra = tempfile::NamedTempFile::new()?; - serde_json::to_writer(&mut extra, &vars)?; + extra.write_all(untemplated(&vars).as_bytes())?; extra.flush()?; let mut inventory = tempfile::NamedTempFile::new()?; - serde_json::to_writer(&mut inventory, &login.inventory()?)?; + inventory.write_all(untemplated(&login.inventory()?).as_bytes())?; inventory.flush()?; let mut command = playbook(&source); command @@ -211,21 +257,108 @@ fn run( // A password is Ansible's to hand to ssh, through its own helper; no // other program, such as the desktop's dialog, is ever asked for one. command.env("SSH_ASKPASS_REQUIRE", "never"); + // A job's output is a log, not a terminal; its colours are kept for + // whoever shows it on one, and taken out for whoever does not. + command.env("ANSIBLE_FORCE_COLOR", "1"); command.env("DEVBOX_BUILD_DIR", paths.build_dir()); Ok(Prepared { command, files: vec![extra, inventory], + notes: vec![], }) } +/// `value` as YAML that Ansible reads with every text marked `!unsafe`, so +/// that none of it is a template: Ansible renders `{{ … }}` in variables +/// and inventories, and a lookup in one would run on this machine. +pub fn untemplated(value: &Value) -> String { + let mut out = String::new(); + write_untemplated(value, &mut out); + out +} + +fn write_untemplated(value: &Value, out: &mut String) { + match value { + // Ansible reads an empty text marked unsafe as null, and an empty one is no template. + Value::String(text) if text.is_empty() => out.push_str("\"\""), + Value::String(text) => { + out.push_str("!unsafe "); + quoted(text, out); + } + // YAML 1.1 reads a number such as 1e5 as text unless it is told otherwise. + Value::Number(number) if number.is_f64() => { + out.push_str("!!float "); + out.push_str(&number.to_string()); + } + Value::Number(_) | Value::Bool(_) | Value::Null => out.push_str(&value.to_string()), + Value::Array(items) => { + out.push('['); + for (index, item) in items.iter().enumerate() { + if index > 0 { + out.push_str(", "); + } + write_untemplated(item, out); + } + out.push(']'); + } + Value::Object(fields) => { + out.push('{'); + for (index, (key, item)) in fields.iter().enumerate() { + if index > 0 { + out.push_str(", "); + } + quoted(key, out); + out.push_str(": "); + write_untemplated(item, out); + } + out.push('}'); + } + } +} + +/// A text in YAML's double quotes, with everything but printable ASCII +/// escaped, as YAML refuses some characters even in quotes. +fn quoted(text: &str, out: &mut String) { + use std::fmt::Write as _; + out.push('"'); + for ch in text.chars() { + match ch { + '"' => out.push_str("\\\""), + '\\' => out.push_str("\\\\"), + ' '..='~' => out.push(ch), + ch if (ch as u32) <= 0xffff => { + let _ = write!(out, "\\u{:04x}", ch as u32); + } + ch => { + let _ = write!(out, "\\U{:08x}", ch as u32); + } + } + } + out.push('"'); +} + /// After a successful apply of a workstation that berth reaches as its /// operator: the role gave the operator sudo without a password, SSH with /// keys only and berth's keys, so the saved profile says so now. A password -/// kept for the first apply is not needed again and is removed. A host of -/// a berth server is settled by the server, which applies it. +/// kept for the first apply is not needed again and is removed. pub fn settle(paths: &Paths, host: &Host) -> Result<()> { - if host.served(paths).is_some() { - return Ok(()); + settle_with(paths, host, &mut std::io::sink()) +} + +/// [`settle`], saying on `out` what it did for a node: the provisioners its +/// features offer are registered, and those they no longer offer forgotten. +/// A node that does not say is named on `out`, and its profile is settled +/// all the same. +pub fn settle_with(paths: &Paths, host: &Host, out: &mut dyn Write) -> Result<()> { + match settle_node(paths, host) { + Ok(lines) => { + for line in lines { + let _ = writeln!(out, "{line}"); + } + } + Err(error) => { + let _ = writeln!(out, "! {error}"); + } } let Some(mut profile) = host.place.load(paths)? else { return Ok(()); @@ -237,7 +370,10 @@ pub fn settle(paths: &Paths, host: &Host) -> Result<()> { host.place.save(paths, &profile)?; let store = crate::store::shared(paths)?; let secrets = crate::registry::Store::own(store.clone()); - secrets.set_secret(&crate::keys::sudo_secret(&host.place), None)?; + // A node's sudo is as it was; only a workstation's operator was given one that never asks. + if crate::host_type::of(&profile) != crate::host_type::HostType::Node { + secrets.set_secret(&crate::keys::sudo_secret(&host.place), None)?; + } if before.auth == berth_api::profile::Auth::Password { secrets.set_secret(&crate::keys::password_secret(&host.place), None)?; } @@ -249,12 +385,25 @@ pub fn settle(paths: &Paths, host: &Host) -> Result<()> { Ok(()) } +/// After a successful apply of a node: register the provisioners its +/// features offer, and forget those they no longer do. Says what it did, a +/// line each. +pub fn settle_node(paths: &Paths, host: &Host) -> Result> { + if crate::host_type::of(&host.config) != crate::host_type::HostType::Node { + return Ok(vec![]); + } + let offers = crate::node::reported(paths, host) + .map_err(|error| format!("{} did not say what its features offer: {error}", host.name))?; + let store = crate::registry::Store::own(crate::store::shared(paths)?); + crate::node::register(paths, &store, host, &offers) +} + /// How long a controller just installed has to answer. const CONTROLLER_PATIENCE: std::time::Duration = std::time::Duration::from_secs(60); /// Install berth's controller alone on a host from where berth runs, its /// output to `out`, and wait for it to answer: whether all went well. A -/// reachable host becomes a controller in its profile once the controller +/// host of the type ssh becomes managed in its profile once the controller /// is there. pub fn install_controller( paths: &Paths, @@ -268,9 +417,9 @@ pub fn install_controller( }; // The profile here may say less than the host is, as for a workstation // added by its address; the role refuses it too. - let intended = crate::level::of(&host.config); + let intended = crate::host_type::of(&host.config); let asked = crate::keys::refusal::now(); - let observed = crate::level::observe(paths, host, become_password); + let observed = crate::host_type::observe(paths, host, become_password); // A host that refused the login just now would refuse Ansible's too. if observed.is_none() && let Ok(login) = crate::keys::login_of(paths, host) @@ -279,7 +428,7 @@ pub fn install_controller( { return fail(out, &refused.sentence(&host.name, refused.at)); } - if let Some(why) = crate::level::refuse_install(&host.name, intended, observed) { + if let Some(why) = crate::host_type::refuse_install(&host.name, intended, observed) { return fail(out, &why); } let prepared = match prepare_controller(paths, host, become_password) { @@ -301,16 +450,16 @@ pub fn install_controller( } None => return fail(out, &"Installing the controller was ended by a signal"), } - if crate::level::of(&host.config) != crate::level::Level::Controller { + if crate::host_type::of(&host.config) != crate::host_type::HostType::Managed { let mut config = host.config.clone(); - berth_api::profile::set_level(&mut config, crate::level::Level::Controller); + berth_api::profile::set_host_type(&mut config, crate::host_type::HostType::Managed); if let Err(error) = host.place.save(paths, &config) { return fail( out, &format!("The controller is installed, but the profile was not saved: {error}"), ); } - let _ = writeln!(out, "✓ {} is a controller host now", host.name); + let _ = writeln!(out, "✓ {} is a managed host now", host.name); } match answers(paths, host, become_password) { Ok(version) => { @@ -339,11 +488,7 @@ fn answers(paths: &Paths, host: &Host, password: Option<&str>) -> Result let started = std::time::Instant::now(); loop { let asked = runtime.block_on(async { - tokio::time::timeout( - std::time::Duration::from_secs(15), - link.get::("/v1/version"), - ) - .await + tokio::time::timeout(std::time::Duration::from_secs(15), link.version()).await }); match asked { Ok(Ok(version)) => return Ok(version.version), @@ -365,8 +510,8 @@ mod tests { #[test] fn variables_come_from_the_profile() { let config = json!({ - "profile": "cloud", "agent_username": "bot", "agent_privileges": "limited", - "tailscale_auth_key": "tskey", "ansible_vars": {"devbox_geometry": "1x1"}, + "target": "cloud", "agent_username": "bot", "agent_privileges": "limited", + "ansible_vars": {"devbox_geometry": "1x1", "devbox_tailscale_key_command": ["sh", "-c", "id"]}, "operator_ssh_public_key": "ssh-ed25519 ME", }); let vars = ansible_vars( @@ -390,21 +535,36 @@ mod tests { assert_eq!(vars["devbox_profile"], "cloud"); assert_eq!(vars["devbox_geometry"], "1x1"); assert_eq!(vars["devbox_plugin_dirs"]["demo"], "/p/demo"); - assert_eq!(vars["devbox_tailscale_auth_key"], "tskey"); - // A pasted key without a login server goes to Tailscale, as always. - assert!(vars.get("devbox_tailscale_login_server").is_none()); - // A profile of the qemu kind there once was is applied as cloud, reached from the QEMU host. - let qemu = ansible_vars(&json!({"profile": "qemu"}), &json!({}), &[]); + // A profile of the qemu target there once was is applied as cloud, reached from the QEMU host. + let qemu = ansible_vars(&json!({"target": "qemu"}), &json!({}), &[]); assert_eq!(qemu["devbox_profile"], "cloud"); assert_eq!(qemu["devbox_bootstrap_ssh_cidr"], "10.0.2.2/32"); - let headscale = json!({"tailscale_login_server": "https://hs.example.org"}); - assert_eq!( - ansible_vars(&headscale, &json!({}), &[])["devbox_tailscale_login_server"], - "https://hs.example.org" - ); + // The tailnet's variables are berth's, not the profile's. + assert!(vars.get("devbox_tailscale_key_command").is_none()); + assert!(vars.get("devbox_tailscale_auth_key").is_none()); + assert!(vars.get("devbox_tailscale_login_server").is_none()); + } + + #[test] + fn a_node_takes_the_path_without_the_desktop() { + let node = json!({"type": "node", "access": {"address": "192.0.2.20", "user": "root"}, + "agent_username": "bot", "ansible_vars": {"devbox_user": "x"}}); + let vars = ansible_vars(&node, &json!({}), &["ssh-ed25519 OWN".to_owned()]); + assert_eq!(vars["devbox_level"], "node"); + assert_eq!(vars["devbox_profile"], "cloud"); + assert_eq!(vars["devbox_node_user"], "root"); + // A node has only the features its profile names, and no accounts of a workstation. + assert_eq!(vars["devbox_plugin_features"], json!([])); + assert!(vars.get("devbox_user").is_none() && vars.get("devbox_operator_user").is_none()); + assert_eq!(vars["devbox_berth_ssh_public_key"], "ssh-ed25519 OWN"); + let contained = json!({"type": "node", "target": "container", + "ansible_vars": {"devbox_plugin_features": ["docker/engine"]}}); + let vars = ansible_vars(&contained, &json!({}), &[]); + assert_eq!(vars["devbox_profile"], "container"); + assert_eq!(vars["devbox_plugin_features"], json!(["docker/engine"])); assert!( - ansible_vars(&json!({}), &json!({}), &[]) - .get("devbox_tailscale_auth_key") + ansible_vars(&json!({"builder": {}}), &json!({}), &[]) + .get("devbox_level") .is_none() ); } @@ -425,7 +585,7 @@ mod tests { let host = Host { name: "box".into(), place: Place::own("box").unwrap(), - config: json!({"profile": "cloud", "access": {"address": "box", "user": "ops", "sudo": "password"}}), + config: json!({"target": "cloud", "access": {"address": "box", "user": "ops", "sudo": "password"}}), }; let prepared = prepare(&paths, &host, Some("hunter2")).unwrap(); let args: Vec = prepared @@ -446,11 +606,9 @@ mod tests { .and_then(|(_, value)| value); let wanted = paths.source().unwrap().join(layout::ANSIBLE_CONFIG); assert_eq!(config, Some(wanted.as_os_str())); - let extra: Value = - serde_json::from_slice(&std::fs::read(prepared.files[0].path()).unwrap()).unwrap(); + let extra = read_back(prepared.files[0].path()); assert_eq!(extra["ansible_become_password"], "hunter2"); - let inventory: Value = - serde_json::from_slice(&std::fs::read(prepared.files[1].path()).unwrap()).unwrap(); + let inventory = read_back(prepared.files[1].path()); assert_eq!( inventory["all"]["hosts"]["devbox_target"]["ansible_user"], "ops" @@ -470,7 +628,7 @@ mod tests { paths.own_public_key().unwrap() ); assert_eq!(extra["devbox_berth_profile"]["host"], "box"); - assert_eq!(extra["devbox_berth_profile"]["profile"]["profile"], "cloud"); + assert_eq!(extra["devbox_berth_profile"]["profile"]["target"], "cloud"); assert!( extra["devbox_berth_profile"]["profile"] .get("access") @@ -526,4 +684,57 @@ mod tests { Some("hunter2") ); } + + #[test] + fn a_profile_says_nothing_of_how_ansible_runs() { + let config = json!({"builder": {}, "ansible_vars": { + "devbox_geometry": "1x1", + "ansible_connection": "local", + "ansible_ssh_common_args": "-o ProxyCommand=\"sh -c id\"", + "ansible_python_interpreter": "/tmp/x", + "devbox_plugin_features": ["claude/cli", "../../tmp/x", 7]}}); + let vars = ansible_vars(&config, &json!({}), &[]); + assert_eq!(vars["devbox_geometry"], "1x1"); + for gone in [ + "ansible_connection", + "ansible_ssh_common_args", + "ansible_python_interpreter", + ] { + assert!(vars.get(gone).is_none(), "{gone}"); + } + assert_eq!(vars["devbox_plugin_features"], json!(["claude/cli"])); + assert_eq!( + berth_api::profile::left_out(&config), + [ + "ansible_connection", + "ansible_python_interpreter", + "ansible_ssh_common_args", + "the feature \"../../tmp/x\"", + "the feature 7" + ] + ); + } + + /// A file [`untemplated`] wrote, without its tags, of texts without any. + fn read_back(path: &std::path::Path) -> Value { + let text = std::fs::read_to_string(path).unwrap(); + serde_json::from_str(&text.replace("!unsafe ", "").replace("!!float ", "")).unwrap() + } + + #[test] + fn what_goes_to_ansible_is_never_a_template() { + let value = json!({"devbox_x": "{{ lookup('pipe', 'id') }}", "n": 3, "f": 1e20, "b": true, + "z": null, "e": "", "l": ["a\"b\\c", "é\u{7f}\u{1F600}"], "d": {"{{ k }}": "v"}}); + assert_eq!( + untemplated(&value), + r#"{"b": true, "d": {"{{ k }}": !unsafe "v"}, "devbox_x": !unsafe "{{ lookup('pipe', 'id') }}", "e": "", "f": !!float 1e+20, "l": [!unsafe "a\"b\\c", !unsafe "\u00e9\u007f\U0001f600"], "n": 3, "z": null}"# + ); + if let Ok(dir) = std::env::var("BERTH_UNTEMPLATED_TO") { + std::fs::write(format!("{dir}/vars.yml"), untemplated(&value)).unwrap(); + let inventory = json!({"all": {"hosts": {"devbox_target": { + "ansible_host": "{{ lookup('pipe', 'echo RAN') }}", "ansible_connection": "local", + "ansible_ssh_common_args": "-o ControlMaster=auto"}}}}); + std::fs::write(format!("{dir}/inventory"), untemplated(&inventory)).unwrap(); + } + } } diff --git a/crates/berth-core/src/credentials.rs b/crates/berth-core/src/credentials.rs index 0db88af..4126e6e 100644 --- a/crates/berth-core/src/credentials.rs +++ b/crates/berth-core/src/credentials.rs @@ -83,7 +83,7 @@ pub fn hash(password: &str) -> Result { } /// Keep a generated password of a host's account in berth's store, beside -/// the host's profile, for `berth credentials` to show. +/// the host's profile, for `berth host credentials` to show. pub fn save(paths: &Paths, place: &Place, user: &str, password: &str) -> Result<()> { let (scope, name) = place.side(); let store = store::shared(paths)?; diff --git a/crates/berth-core/src/host.rs b/crates/berth-core/src/host.rs index 50f4410..173a6dd 100644 --- a/crates/berth-core/src/host.rs +++ b/crates/berth-core/src/host.rs @@ -1,6 +1,6 @@ //! A host as its profile describes it: where it is and how it is reached. -use crate::{Result, link::LinkSpec, paths::Paths, profiles::Place, server::Server}; +use crate::{Result, link::LinkSpec, paths::Paths, profiles::Place}; use berth_api::profile::Access; use serde_json::Value; use std::process::Command; @@ -62,18 +62,6 @@ impl Host { berth_api::profile::asks_become_pass(&self.config) } - /// The `berth serve` this host is reached through, for a profile written - /// from a server's list of hosts. The token is a setting of `paths`, not - /// part of a profile. - pub fn served(&self, paths: &Paths) -> Option { - let url = self.config["builder"]["served"].as_str()?; - let configured = Server::configured(paths).filter(|server| server.url == url); - Some(configured.unwrap_or(Server { - url: url.to_owned(), - token: std::env::var("BERTH_TOKEN").ok(), - })) - } - pub fn agent(&self) -> &str { match self.config["agent_username"].as_str() { None | Some("") => "agent", @@ -82,8 +70,8 @@ impl Host { } /// The SSH user's sudo password, for a host that asks for one: the - /// secret `sudo/HOST` when it is kept, `BERTH_SUDO_PASSWORD`, or asked - /// for on a terminal. The UI has its own dialog. + /// secret `sudo/HOST` when it is kept, or asked for on a terminal. The + /// screens and the commands ask for it themselves. pub fn sudo_password(&self, paths: &Paths) -> Result> { if !self.asks_password() { return Ok(None); @@ -91,9 +79,6 @@ impl Host { if let Some(kept) = self.kept_sudo_password(paths) { return Ok(Some(kept)); } - if let Ok(password) = std::env::var("BERTH_SUDO_PASSWORD") { - return Ok(Some(password)); - } use std::io::IsTerminal; if !std::io::stdin().is_terminal() { // Tried without one: sudo says so itself when it does want it. @@ -105,9 +90,6 @@ impl Host { /// The sudo password kept as the secret `sudo/HOST`, for a host whose /// user chose to keep it. pub fn kept_sudo_password(&self, paths: &Paths) -> Option { - if self.served(paths).is_some() { - return None; - } let store = crate::store::shared(paths).ok()?; crate::registry::Store::own(store) .secret(&crate::keys::sudo_secret(&self.place)) @@ -134,13 +116,6 @@ impl Host { /// How to open a connection to the host's berthd. `password` is the SSH /// user's sudo password, for a host that asks for one. pub fn link(&self, paths: &Paths, password: Option<&str>) -> Result { - if let Some(server) = self.served(paths) { - return Ok(LinkSpec { - via: Some((server, self.name.clone())), - name: self.name.clone(), - ..LinkSpec::default() - }); - } // A profile can name the command itself, as for a container: // ["docker", "exec", "-i", "NAME", "berthd", "connect"]. if let Some(command) = self.access().link { diff --git a/crates/berth-core/src/host_type.rs b/crates/berth-core/src/host_type.rs new file mode 100644 index 0000000..67f2b62 --- /dev/null +++ b/crates/berth-core/src/host_type.rs @@ -0,0 +1,469 @@ +//! What a host of each type can do, and the one check that every client +//! asks before it offers or runs something: the UIs, the commands and the +//! server's API. A host's type is what its profile says it is meant to be; +//! what it is seen to be comes from asking it. + +use crate::{Result, host::Host}; +use berth_api::{Action, Version, ops::Role}; + +pub use berth_api::HostType; + +/// Something berth can do with a host. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Capability { + /// A terminal and commands over SSH, the inventory, whether it answers. + Ssh, + /// Install or update berth's controller alone. + InstallController, + /// Sessions, the channel, files, units, logs and processes. + Sessions, + /// The controller's actions: status, update and reboot. + Actions, + /// Secrets and registry logins sent to the host. + Secrets, + /// Plugin features, their actions and services. + Plugins, + /// Applying the profile over SSH, and a roll-out over it. + Apply, + /// A cloud seed or an installer image, which install a whole workstation. + Seed, + /// The desktop: screenshots and access links. + Desktop, +} + +impl Capability { + pub const ALL: [Capability; 9] = [ + Capability::Ssh, + Capability::InstallController, + Capability::Sessions, + Capability::Actions, + Capability::Secrets, + Capability::Plugins, + Capability::Apply, + Capability::Seed, + Capability::Desktop, + ]; + + pub fn name(self) -> &'static str { + match self { + Capability::Ssh => "ssh", + Capability::InstallController => "install-controller", + Capability::Sessions => "sessions", + Capability::Actions => "actions", + Capability::Secrets => "secrets", + Capability::Plugins => "plugins", + Capability::Apply => "apply", + Capability::Seed => "seed", + Capability::Desktop => "desktop", + } + } + + /// What it is, as the subject of a sentence. + pub fn label(self) -> &'static str { + match self { + Capability::Ssh => "a terminal over SSH", + Capability::InstallController => "installing the controller alone", + Capability::Sessions => "working with sessions, the channel and files", + Capability::Actions => "running the controller's actions", + Capability::Secrets => "sending secrets and registry logins", + Capability::Plugins => "using plugin features", + Capability::Apply => "applying a profile or a roll-out", + Capability::Seed => "a cloud seed or an installer image", + Capability::Desktop => "using the desktop", + } + } + + /// Whether a host of `host_type` has it. The controller alone is + /// installed on a host that is not applied; the controller of a node or + /// a workstation comes with applying. + pub fn allowed(self, host_type: HostType) -> bool { + use HostType::*; + match self { + Capability::Ssh => true, + Capability::InstallController => matches!(host_type, Ssh | Managed), + Capability::Sessions | Capability::Actions | Capability::Secrets => { + matches!(host_type, Managed | Node | Workstation) + } + Capability::Plugins | Capability::Apply => matches!(host_type, Node | Workstation), + Capability::Seed | Capability::Desktop => host_type == Workstation, + } + } +} + +/// The capabilities and the types that have each, in the order of +/// [`HostType::ALL`]: for a page that shows the table. +pub fn table() -> Vec<(Capability, [bool; HostType::ALL.len()])> { + Capability::ALL + .into_iter() + .map(|capability| { + ( + capability, + HostType::ALL.map(|host_type| capability.allowed(host_type)), + ) + }) + .collect() +} + +/// Why something cannot be done with a host now, and the action that +/// changes its type so it can. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Refusal { + pub why: String, + /// `install-controller`, `apply`, or `setup` for the set-up flow that + /// makes the host a workstation. + pub raise: &'static str, +} + +impl std::fmt::Display for Refusal { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(&self.why) + } +} + +impl std::error::Error for Refusal {} + +/// Whether the host `name` of `host_type` may do this, or why not. +pub fn check( + name: &str, + host_type: HostType, + capability: Capability, +) -> std::result::Result<(), Refusal> { + if capability.allowed(host_type) { + return Ok(()); + } + let what = capability.label(); + Err(match (capability, host_type) { + (Capability::InstallController, _) => Refusal { + why: format!( + "{name} is a {host_type}: applying its configuration installs and updates its controller. Apply it with: berth apply {name}" + ), + raise: "apply", + }, + (_, HostType::Ssh) if capability.allowed(HostType::Managed) => Refusal { + why: format!( + "{name} is a host of the type ssh: berth's controller is not installed there, and {what} needs it. Install it with: berth run {name} install-controller" + ), + raise: "install-controller", + }, + (Capability::Desktop, HostType::Node) => Refusal { + why: format!( + "{name} is a node, a server without a desktop: {what} needs a workstation. Set its type to workstation in its set-up, then apply it" + ), + raise: "setup", + }, + (_, _) if capability.allowed(HostType::Node) => Refusal { + why: format!( + "{name} is a host of the type {host_type}, neither a node nor a workstation: {what} needs berth to apply its profile. Make it a node (berth host set {name} type=node) or a workstation in its set-up, then apply it" + ), + raise: "setup", + }, + _ => Refusal { + why: format!( + "{name} is a host of the type {host_type}, not a workstation: {what} needs its whole profile. Set its type to workstation in its set-up, then apply it" + ), + raise: "setup", + }, + }) +} + +/// The same for a host by its profile, as an error to stop with. +pub fn guard(host: &Host, capability: Capability) -> Result<()> { + check(&host.name, of(&host.config), capability)?; + Ok(()) +} + +/// The type a profile means its host to be. +pub fn of(config: &serde_json::Value) -> HostType { + berth_api::profile::host_type(config) +} + +/// An action as a host of `host_type` may run it: unavailable, with the +/// reason, when the type rules it out. +pub fn fit(mut action: Action, name: &str, host_type: HostType, capability: Capability) -> Action { + if let Err(refusal) = check(name, host_type, capability) { + action.available = false; + action.why = refusal.why; + } + action +} + +/// What a host was seen to be by asking its controller: what its record +/// says when it answers, ssh when berth logged in and found no +/// controller, nothing when it could not be asked. +pub fn seen(answer: std::result::Result<&Version, bool>) -> Option { + match answer { + // A controller without a record was installed outside berth. + Ok(version) => Some(version.host_type.unwrap_or(HostType::Managed)), + Err(true) => Some(HostType::Ssh), + Err(false) => None, + } +} + +pub use berth_api::type_mismatch as mismatch; + +/// Why installing the controller alone must not run on a host seen to be a +/// workstation, whatever its profile says: it would take the workstation's +/// controller apart. +pub fn refuse_install( + name: &str, + intended: HostType, + observed: Option, +) -> Option { + match (intended, observed) { + (HostType::Workstation | HostType::Node, _) | (_, None) => None, + (_, Some(HostType::Workstation | HostType::Node)) => Some(format!( + "{} Installing the controller alone would take its controller apart, so it is not offered", + mismatch(name, intended, observed).unwrap_or_default() + )), + _ => None, + } +} + +/// What the host's controller says the host is, asked through the link +/// once and briefly: `None` when no controller answers. +pub fn observe( + paths: &crate::paths::Paths, + host: &Host, + password: Option<&str>, +) -> Option { + let link = crate::link::Link::new(host.link(paths, password).ok()?); + let runtime = crate::link::runtime().ok()?; + let asked = runtime.block_on(async { + tokio::time::timeout(std::time::Duration::from_secs(15), link.version()).await + }); + runtime.shutdown_background(); + match asked { + Ok(Ok(version)) => seen(Ok(&version)), + _ => None, + } +} + +/// berth's own action that puts the controller on a host, as a host of +/// `host_type` sees it. `observed` is what its controller says it is, when +/// it answered. +pub fn install_controller_action( + name: &str, + host_type: HostType, + observed: Option, +) -> Action { + let (label, help) = match host_type { + HostType::Managed => ( + "Update the controller", + "Install berthd and box of this berth on the host again, and restart the controller: for example after berth was updated. Nothing else of the profile is applied.", + ), + _ => ( + "Install the controller", + "Put berthd and box on the host and start the controller, so that sessions, actions and files work there. Nothing else of a workstation is installed: no desktop, agents or plugins. The SSH user is the account sessions run as. The host's type becomes managed.", + ), + }; + let action = Action { + name: Capability::InstallController.name().into(), + label: label.into(), + help: help.into(), + group: "Set-Up".into(), + role: Role::Admin, + confirm: Some(format!("Install berth's controller on {name} now?")), + ..Action::default() + }; + let mut action = fit(action, name, host_type, Capability::InstallController); + if let Some(why) = refuse_install(name, host_type, observed) { + action.available = false; + action.why = why; + } + action +} + +/// berth's own action that gives a host another name, of any type: the +/// machine stays as it is. +pub fn rename_action() -> Action { + Action { + name: "rename".into(), + label: "Rename".into(), + help: "Give the host another name in berth. Its profile, passwords, host keys, machine record, messages and jobs go with it. The machine itself stays as it is and keeps running, and so does its hostname unless hostname says otherwise.".into(), + group: "Set-Up".into(), + role: Role::Admin, + arguments: vec![ + berth_api::Argument { + name: "name".into(), + label: "New name".into(), + help: "The host's name in berth from now on: letters, digits, `-`, `_` and `.`." + .into(), + required: true, + ..Default::default() + }, + berth_api::Argument { + name: "hostname".into(), + label: "Hostname too".into(), + help: "Make the new name the profile's hostname as well. The machine takes it at its next apply.".into(), + default: "no".into(), + values: vec!["no".into(), "yes".into()], + ..Default::default() + }, + ], + ..Action::default() + } +} + +/// berth's own actions on a host of this machine, which need no server: +/// those that change the host's type, and renaming it. +pub fn local_actions(name: &str, host_type: HostType, observed: Option) -> Vec { + vec![ + install_controller_action(name, host_type, observed), + rename_action(), + ] +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn each_type_can_do_what_the_ones_before_it_can() { + use Capability::*; + let rows: Vec<(&str, [bool; 4])> = table() + .into_iter() + .map(|(capability, types)| (capability.name(), types)) + .collect(); + // ssh, managed, node, workstation. + assert_eq!( + rows, + [ + (Ssh.name(), [true, true, true, true]), + (InstallController.name(), [true, true, false, false]), + (Sessions.name(), [false, true, true, true]), + (Actions.name(), [false, true, true, true]), + (Secrets.name(), [false, true, true, true]), + (Plugins.name(), [false, false, true, true]), + (Apply.name(), [false, false, true, true]), + (Seed.name(), [false, false, false, true]), + (Desktop.name(), [false, false, false, true]), + ] + ); + } + + #[test] + fn a_refusal_names_the_action_that_changes_the_type() { + let refused = check("studio", HostType::Ssh, Capability::Sessions).unwrap_err(); + assert_eq!(refused.raise, "install-controller"); + assert!(refused.why.contains("berth run studio install-controller")); + let refused = check("studio", HostType::Ssh, Capability::Apply).unwrap_err(); + assert_eq!(refused.raise, "setup"); + assert!( + refused + .why + .contains("of the type ssh, neither a node nor a workstation") + ); + let refused = check("studio", HostType::Managed, Capability::Desktop).unwrap_err(); + assert_eq!(refused.raise, "setup"); + let refused = check( + "studio", + HostType::Workstation, + Capability::InstallController, + ) + .unwrap_err(); + assert_eq!(refused.raise, "apply"); + assert!(check("studio", HostType::Workstation, Capability::Apply).is_ok()); + } + + #[test] + fn a_node_is_applied_and_has_no_desktop() { + assert!(check("big", HostType::Node, Capability::Apply).is_ok()); + assert!(check("big", HostType::Node, Capability::Plugins).is_ok()); + assert!(check("big", HostType::Node, Capability::Sessions).is_ok()); + let refused = check("big", HostType::Node, Capability::Desktop).unwrap_err(); + assert!( + refused + .why + .starts_with("big is a node, a server without a desktop") + ); + let refused = check("big", HostType::Node, Capability::InstallController).unwrap_err(); + assert_eq!(refused.raise, "apply"); + assert!(refused.why.starts_with("big is a node: applying")); + let refused = check("big", HostType::Managed, Capability::Apply).unwrap_err(); + assert!(refused.why.contains("neither a node nor a workstation")); + assert!(refused.why.contains("berth host set big type=node")); + // The controller alone is never put over a node's. + let action = install_controller_action("big", HostType::Managed, Some(HostType::Node)); + assert!(!action.available); + assert!( + action + .why + .starts_with("big is a node; its profile here says managed") + ); + assert!(!install_controller_action("big", HostType::Node, None).available); + let refused = check("big", HostType::Node, Capability::Seed).unwrap_err(); + assert!(refused.why.contains("not a workstation")); + } + + #[test] + fn the_apply_guard_reads_the_profile() { + let host = |config| Host { + name: "dock".into(), + place: crate::profiles::Place::File("/dock.json".into()), + config, + }; + let ssh = host(serde_json::json!({"type": "ssh", "access": {"address": "192.0.2.4"}})); + let refused = guard(&ssh, Capability::Apply).unwrap_err(); + assert!( + refused + .to_string() + .contains("neither a node nor a workstation") + ); + assert!(guard(&host(serde_json::json!({"builder": {}})), Capability::Apply).is_ok()); + } + + #[test] + fn what_a_host_is_seen_to_be_comes_from_its_controller() { + let version = |host_type| Version { + version: "1".into(), + api: 2, + mode: "system".into(), + applied: None, + host_type, + }; + assert_eq!(seen(Ok(&version(None))), Some(HostType::Managed)); + assert_eq!( + seen(Ok(&version(Some(HostType::Workstation)))), + Some(HostType::Workstation) + ); + assert_eq!(seen(Err(true)), Some(HostType::Ssh)); + assert_eq!(seen(Err(false)), None); + } + + #[test] + fn installing_the_controller_is_offered_to_a_host_that_is_not_a_workstation() { + let action = install_controller_action("dock", HostType::Ssh, None); + assert!(action.available && action.label == "Install the controller"); + assert_eq!( + install_controller_action("dock", HostType::Managed, None).label, + "Update the controller" + ); + let action = install_controller_action("dock", HostType::Workstation, None); + assert!(!action.available && action.why.contains("berth apply dock")); + // A controller seen to be one is updated as its profile says. + let action = install_controller_action("dock", HostType::Ssh, Some(HostType::Managed)); + assert!(action.available); + } + + #[test] + fn a_host_seen_to_be_a_workstation_never_gets_the_controller_alone() { + for intended in [HostType::Ssh, HostType::Managed] { + let action = install_controller_action("dock", intended, Some(HostType::Workstation)); + assert!(!action.available); + assert!( + action.why.starts_with(&format!( + "dock is a workstation; its profile here says {intended}" + )), + "{}", + action.why + ); + assert!(action.why.contains("berth host adopt dock")); + } + assert_eq!( + mismatch("dock", HostType::Ssh, Some(HostType::Managed)).unwrap(), + "dock has berth's controller; its profile here says ssh. To adopt it: berth host set dock type=managed" + ); + assert!(mismatch("dock", HostType::Workstation, Some(HostType::Managed)).is_none()); + assert!(mismatch("dock", HostType::Managed, None).is_none()); + } +} diff --git a/crates/berth-core/src/import.rs b/crates/berth-core/src/import.rs index b5241bc..cc0cef5 100644 --- a/crates/berth-core/src/import.rs +++ b/crates/berth-core/src/import.rs @@ -10,8 +10,10 @@ //! the token and the plugin directory stay where they are, as they are //! used as files; only `serve.setup-key` goes. //! -//! Two more change rows the store has already: how hosts are reached moves -//! into `access`, and the records of hosts' machines become resources. +//! More change rows the store has already: how hosts are reached moves into +//! `access`, the records of hosts' machines become resources, provisioners +//! get names of their own, and profiles take the names their fields have +//! now ([`take_in_names`]). //! //! The rows of an import go into the store in one change with its marker, so //! they are all there or none of them are. Only then are the old files put @@ -44,6 +46,24 @@ pub const RESOURCES_MARKER: &str = "import:resources"; /// The marker of the sixth, which registers each provisioner that has /// settings or made something under a name of its own. pub const PROVISIONERS_MARKER: &str = "import:provisioners"; +/// The marker of the seventh, which gives every stored profile the names its +/// fields have now: `type` for `level` and `target` for `profile`. +pub const NAMES_MARKER: &str = "import:names"; +/// The marker of the eighth, which takes out of every stored profile what +/// older berths kept of joining the tailnet without a key berth makes. +pub const TAILNET_MARKER: &str = "import:tailnet"; + +/// Every import's marker: a store that lacks one is taken in by this berth. +pub const MARKERS: &[&str] = &[ + MARKER, + PROFILES_MARKER, + KEYS_MARKER, + ACCESS_MARKER, + RESOURCES_MARKER, + PROVISIONERS_MARKER, + NAMES_MARKER, + TAILNET_MARKER, +]; /// What a row of the import is. #[derive(Debug, Default, PartialEq, Eq)] @@ -74,6 +94,8 @@ pub fn run(paths: &Paths, installation: &Installation, store: &Store) -> Result< && marked(store, ACCESS_MARKER)? && marked(store, RESOURCES_MARKER)? && marked(store, PROVISIONERS_MARKER)? + && marked(store, NAMES_MARKER)? + && marked(store, TAILNET_MARKER)? { return Ok(()); } @@ -146,14 +168,185 @@ pub fn run(paths: &Paths, installation: &Installation, store: &Store) -> Result< let registered = take_in_provisioners(store)?; if !registered.is_empty() { eprintln!( - "berth: the provisioners in use are registered as {}: see berth provisioner", + "berth: the provisioners in use are registered as {}: see berth provisioner list", registered.join(", ") ); } } + // Before the names, which drop the same fields without a word. + if !marked(store, TAILNET_MARKER)? { + let taken = take_in_tailnet(store)?; + if let Some(said) = tailnet_message(&taken, store)? { + eprintln!("berth: {said}"); + } + } + if !marked(store, NAMES_MARKER)? { + let renamed = take_in_names(store)?; + if renamed > 0 { + let (noun, verb) = match renamed { + 1 => ("profile", "says"), + _ => ("profiles", "say"), + }; + eprintln!("berth: {renamed} {noun} {verb} type for level and target for profile now"); + } + } Ok(()) } +/// The seventh import, in one change with its marker: every stored profile, +/// of every scope, takes the names of now ([`berth_api::profile::take_old_names`]). +/// The values may be sealed, so this is done here rather than by a migration +/// of the database. Returns how many profiles changed. +pub(crate) fn take_in_names(store: &Store) -> Result { + let mut scopes = std::collections::BTreeSet::new(); + if let crate::store::Rows::State(rows) = store.export(crate::store::Table::State)? { + for row in rows.into_iter().filter(|row| row.kind == kind::PROFILE) { + scopes.insert(row.scope); + } + } + let mut writes: Vec<(String, String, Vec)> = vec![]; + for scope in scopes { + for (name, value) in store.entries(kind::PROFILE, &scope)? { + let Ok(mut config) = serde_json::from_slice::(&value) else { + continue; + }; + if berth_api::profile::take_old_names(&mut config) { + writes.push((scope.clone(), name, config.to_string().into_bytes())); + } + } + } + let renamed = writes.len(); + let marker = + json!({"version": berth_api::VERSION, "at": berth_api::rfc3339(berth_api::unix_now())}); + let marker = marker.to_string().into_bytes(); + let mut rows: Vec<(&str, &str, &str, Vec)> = writes + .iter() + .map(|(scope, name, value)| (kind::PROFILE, scope.as_str(), name.as_str(), value.clone())) + .collect(); + rows.push((kind::META, OWN, NAMES_MARKER, marker)); + store.change(&rows, &[])?; + Ok(renamed) +} + +/// The hosts whose profiles the tailnet's import changed: those that held a +/// pasted key, and those berth 0.5 never joined by itself. +#[derive(Debug, Default, PartialEq)] +pub(crate) struct TakenTailnet { + pub pasted: Vec, + pub optional: Vec, +} + +/// The role's variable that lets an apply go on, without the tailnet, when +/// berth cannot make the host's key. +pub const TAILNET_OPTIONAL: &str = "devbox_tailnet_optional"; + +/// The eighth import, in one change with its marker: every stored profile, +/// of every scope, loses its pasted enrollment key and the switch of keys +/// berth makes ([`berth_api::profile::take_old_tailnet`]), so no key lingers +/// in the store. berth 0.5 joined a host only with a pasted key or with that +/// switch on, which was off unless set, and an apply without either left the +/// host off the tailnet. A profile of such a host keeps that: it joins once +/// berth can make its key, and until then an apply goes on without it. Says +/// which hosts of berth's own scope it changed. +pub(crate) fn take_in_tailnet(store: &Store) -> Result { + let mut scopes = std::collections::BTreeSet::new(); + if let crate::store::Rows::State(rows) = store.export(crate::store::Table::State)? { + for row in rows.into_iter().filter(|row| row.kind == kind::PROFILE) { + scopes.insert(row.scope); + } + } + let mut taken = TakenTailnet::default(); + let mut writes: Vec<(String, String, Vec)> = vec![]; + for scope in scopes { + for (name, value) in store.entries(kind::PROFILE, &scope)? { + let Ok(mut config) = serde_json::from_slice::(&value) else { + continue; + }; + if !config.is_object() { + continue; + } + let joins = berth_api::profile::joins_tailnet(&config); + let old = berth_api::profile::take_old_tailnet(&mut config); + let optional = joins + && old.is_none_or(|old| old.by_hand && !old.pasted) + && config["ansible_vars"].get(TAILNET_OPTIONAL).is_none(); + if optional { + if !config["ansible_vars"].is_object() { + config["ansible_vars"] = json!({}); + } + config["ansible_vars"][TAILNET_OPTIONAL] = json!(true); + } + if old.is_none() && !optional { + continue; + } + if scope == OWN { + match (old.is_some_and(|old| old.pasted), optional) { + (true, _) => taken.pasted.push(name.clone()), + (false, true) => taken.optional.push(name.clone()), + _ => {} + } + } + writes.push((scope.clone(), name, config.to_string().into_bytes())); + } + } + let marker = + json!({"version": berth_api::VERSION, "at": berth_api::rfc3339(berth_api::unix_now())}); + let marker = marker.to_string().into_bytes(); + let mut rows: Vec<(&str, &str, &str, Vec)> = writes + .iter() + .map(|(scope, name, value)| (kind::PROFILE, scope.as_str(), name.as_str(), value.clone())) + .collect(); + rows.push((kind::META, OWN, TAILNET_MARKER, marker)); + store.change(&rows, &[])?; + Ok(taken) +} + +/// What the tailnet's import tells the user, once, or `None` when it +/// changed nothing they need to know. +pub(crate) fn tailnet_message(taken: &TakenTailnet, store: &Store) -> Result> { + if taken.pasted.is_empty() && taken.optional.is_empty() { + return Ok(None); + } + let list = |names: &[String]| names.join(", "); + let joins = |names: &[String]| if names.len() == 1 { "joins" } else { "join" }; + let has_key = store + .names(kind::SECRET, OWN)? + .iter() + .any(|name| name == berth_api::profile::TAILNET_API_KEY); + let secret = berth_api::profile::TAILNET_API_KEY; + let mut said = String::from("berth makes the keys hosts join the tailnet with itself now."); + if !taken.pasted.is_empty() { + said += &format!( + " The keys pasted into the profiles of {0} are removed. {0} {1} the tailnet on the next apply, unless already on it", + list(&taken.pasted), + joins(&taken.pasted) + ); + said += &match has_key { + true => ".".to_owned(), + false => format!(", once the secret {secret} is set: berth secret set {secret}."), + }; + } + if !taken.optional.is_empty() { + let names = list(&taken.optional); + said += &match has_key { + true => format!( + " {names} {} the tailnet on the next apply, unless already on it.", + joins(&taken.optional) + ), + false => format!( + " {names} {} the tailnet once the secret {secret} is set: berth secret set {secret}. Until then an apply leaves {} off the tailnet, as berth 0.5 did.", + joins(&taken.optional), + if taken.optional.len() == 1 { + "it" + } else { + "them" + } + ), + }; + } + Ok(Some(said)) +} + /// The sixth import, in one change with its marker: each type of provisioner /// that has settings, or that made a resource of any scope, is registered /// under a name of its own, such as `incus-vm` for `incus/vm`, with those @@ -166,7 +359,14 @@ pub(crate) fn take_in_provisioners(store: &Store) -> Result> { for (kind, value) in store.entries(kind::PROVISIONER, OWN)? { removals.push((kind::PROVISIONER, OWN.into(), kind.clone())); let settings = serde_json::from_slice(&value).unwrap_or_default(); - types.insert(kind.clone(), Entry { kind, settings }); + types.insert( + kind.clone(), + Entry { + kind, + settings, + ..Entry::default() + }, + ); } let mut scopes = std::collections::BTreeSet::new(); if let crate::store::Rows::State(rows) = store.export(crate::store::Table::State)? { @@ -397,7 +597,7 @@ pub(crate) fn convert_access( } Err(error) => { converted.notices.push(format!( - "{title} named the key file {}, which berth could not take into its key ring ({error}). It logs in with your SSH configuration now. Add the key with: berth key add NAME FILE, and choose it as the host's login", + "{title} named the key file {}, which berth could not take into its key ring ({error}). It logs in with your SSH configuration now. Add the key with: berth key import NAME FILE, and choose it as the host's login", file.display() )); Auth::Ssh @@ -545,10 +745,17 @@ pub(crate) fn take_in_access( Ok(converted.notices) } -/// A profile of an older layout as it is read or given: of the access -/// layout, its key file taken into the ring. A file is written only when the -/// user saves it. +/// A profile of an older layout as it is read or given: with the names its +/// fields have now, and of the access layout, its key file taken into the +/// ring. A file is written only when the user saves it. pub fn convert_profile(paths: &Paths, title: &str, config: &mut Value) { + berth_api::profile::take_old_names(config); + convert_layout(paths, title, config); +} + +/// A profile of the layout before `access`, in the access layout, its key +/// file taken into the ring. +pub fn convert_layout(paths: &Paths, title: &str, config: &mut Value) { use crate::keys::ring::{Halves, blob}; if config["builder"].as_object().is_none_or(|builder| { ![ @@ -1208,6 +1415,8 @@ mod tests { json!({"address": "192.0.2.10", "user": "admin", "auth": "ssh", "sudo": "password"}) ); assert!(berth_api::profile::asks_become_pass(&studio)); + assert_eq!(studio["target"], "cloud"); + assert!(studio.get("profile").is_none()); assert_eq!(machine_of(&store, "studio").unwrap().external_id, "vm-1"); let mut applied = studio.clone(); applied["access"]["user"] = json!("ops"); @@ -1288,18 +1497,199 @@ mod tests { assert_eq!(read(&store, kind::SECRET, "registry/nexus").unwrap(), "pw"); } + #[test] + fn an_installation_of_berth_05_gives_its_profiles_the_names_of_now() { + let (_dir, paths) = scratch(); + // berth 0.5 had run every import but the last, and saved hosts of each level. + let store = open(&paths); + store + .change(&[], &[(kind::META, OWN, NAMES_MARKER)]) + .unwrap(); + for (name, level) in [("dock", "reachable"), ("pier", "controller")] { + let old = + json!({"level": level, "profile": "cloud", "access": {"address": "192.0.2.4"}}); + store + .write(kind::PROFILE, OWN, name, old.to_string().as_bytes()) + .unwrap(); + } + let studio = json!({"hostname": "studio", "profile": "laptop", "builder": {}}); + store + .write(kind::PROFILE, OWN, "studio", studio.to_string().as_bytes()) + .unwrap(); + let found = installation(&paths); + run(&paths, &found, &store).unwrap(); + assert!(read(&store, kind::META, NAMES_MARKER).is_some()); + let dock = json_row(&store, kind::PROFILE, "dock").unwrap(); + assert_eq!( + (&dock["type"], &dock["target"]), + (&json!("ssh"), &json!("cloud")) + ); + assert_eq!( + json_row(&store, kind::PROFILE, "pier").unwrap()["type"], + "managed" + ); + assert_eq!( + json_row(&store, kind::PROFILE, "studio").unwrap(), + json!({"hostname": "studio", "target": "laptop", "builder": {}}) + ); + // A row an older berth writes later is read with the names of now. + let late = json!({"level": "controller", "access": {"address": "192.0.2.5"}}); + store + .write(kind::PROFILE, OWN, "late", late.to_string().as_bytes()) + .unwrap(); + let place = crate::profiles::Place::own("late").unwrap(); + assert_eq!(place.load(&paths).unwrap().unwrap()["type"], "managed"); + let listed = crate::profiles::list(&paths, OWN).unwrap(); + let (_, listed) = listed.iter().find(|(name, _)| name == "late").unwrap(); + assert!(listed.get("level").is_none()); + } + + #[test] + fn stored_profiles_lose_their_pasted_keys_and_say_so_once() { + let (_dir, paths) = scratch(); + let store = open(&paths); + store + .change(&[], &[(kind::META, OWN, TAILNET_MARKER)]) + .unwrap(); + let rows = [ + ( + OWN, + "studio", + json!({"hostname": "studio", "tailscale_auth_key": "tskey-auth-kFAKE-0000", "tailscale_auto_key": false}), + ), + ( + OWN, + "dock", + json!({"hostname": "dock", "tailscale_auto_key": false}), + ), + ( + OWN, + "pier", + json!({"hostname": "pier", "tailscale_auto_key": true}), + ), + (OWN, "attic", json!({"hostname": "attic", "builder": {}})), + ( + "ffffffff", + "theirs", + json!({"ansible_vars": {"devbox_tailscale_auth_key": "tskey-auth-kFAKE-1111"}}), + ), + ]; + for (scope, name, profile) in &rows { + store + .write(kind::PROFILE, scope, name, profile.to_string().as_bytes()) + .unwrap(); + } + let taken = take_in_tailnet(&store).unwrap(); + assert_eq!( + taken, + TakenTailnet { + pasted: vec!["studio".into()], + optional: vec!["attic".into(), "dock".into()], + } + ); + assert!(read(&store, kind::META, TAILNET_MARKER).is_some()); + assert_eq!( + json_row(&store, kind::PROFILE, "studio").unwrap(), + json!({"hostname": "studio"}) + ); + assert_eq!( + json_row(&store, kind::PROFILE, "pier").unwrap(), + json!({"hostname": "pier"}) + ); + // berth 0.5 joined neither of these by itself, and an apply goes on without the tailnet still. + assert_eq!( + json_row(&store, kind::PROFILE, "attic").unwrap(), + json!({"hostname": "attic", "builder": {}, "ansible_vars": {(TAILNET_OPTIONAL): true}}) + ); + assert_eq!( + json_row(&store, kind::PROFILE, "dock").unwrap(), + json!({"hostname": "dock", "ansible_vars": {(TAILNET_OPTIONAL): true}}) + ); + // No key is left in any row of the store, of any scope. + let crate::store::Rows::State(all) = store.export(crate::store::Table::State).unwrap() + else { + panic!("no rows"); + }; + assert!( + all.iter() + .all(|row| !String::from_utf8_lossy(&row.value).contains("tskey-auth")) + ); + let said = tailnet_message(&taken, &store).unwrap().unwrap(); + assert_eq!( + said, + "berth makes the keys hosts join the tailnet with itself now. The keys pasted into the profiles of studio are removed. studio joins the tailnet on the next apply, unless already on it, once the secret tailnet_api_key is set: berth secret set tailnet_api_key. attic, dock join the tailnet once the secret tailnet_api_key is set: berth secret set tailnet_api_key. Until then an apply leaves them off the tailnet, as berth 0.5 did." + ); + store + .write( + kind::SECRET, + OWN, + berth_api::profile::TAILNET_API_KEY, + b"tskey-api-kFAKE-0000", + ) + .unwrap(); + assert!( + tailnet_message(&taken, &store) + .unwrap() + .unwrap() + .ends_with("unless already on it.") + ); + assert_eq!( + tailnet_message(&TakenTailnet::default(), &store).unwrap(), + None + ); + // One host joins, several join. + let one = TakenTailnet { + pasted: vec![], + optional: vec!["dock".into()], + }; + assert!( + tailnet_message(&one, &store) + .unwrap() + .unwrap() + .contains(" dock joins the tailnet on the next apply") + ); + // Once is enough: the next start finds the marker and changes nothing. + let before = store.export(crate::store::Table::State).unwrap(); + let found = installation(&paths); + run(&paths, &found, &store).unwrap(); + assert_eq!(store.export(crate::store::Table::State).unwrap(), before); + } + + #[test] + fn a_profile_given_in_a_file_is_read_with_the_names_of_now() { + let (dir, paths) = scratch(); + let file = dir.path().join("dock.json"); + fs::write( + &file, + r#"{"level": "reachable", "profile": "cloud", "access": {"address": "192.0.2.4"}}"#, + ) + .unwrap(); + let place = crate::profiles::Place::File(file.clone()); + let read = place.load(&paths).unwrap().unwrap(); + assert_eq!( + read, + json!({"type": "ssh", "target": "cloud", "access": {"address": "192.0.2.4"}}) + ); + // The file keeps what it said until it is saved. + assert!(fs::read_to_string(&file).unwrap().contains("reachable")); + place.save(&paths, &read).unwrap(); + assert!(!fs::read_to_string(&file).unwrap().contains("level")); + } + #[test] fn a_new_installation_gets_only_the_marker() { let (_dir, paths) = scratch(); let store = open(&paths); let report = store.seal_report().unwrap(); - assert_eq!((report.sealed, report.plain), (0, 6)); + assert_eq!((report.sealed, report.plain), (0, 8)); assert!(read(&store, kind::META, MARKER).is_some()); assert!(read(&store, kind::META, PROVISIONERS_MARKER).is_some()); assert!(read(&store, kind::META, PROFILES_MARKER).is_some()); assert!(read(&store, kind::META, KEYS_MARKER).is_some()); assert!(read(&store, kind::META, ACCESS_MARKER).is_some()); assert!(read(&store, kind::META, RESOURCES_MARKER).is_some()); + assert!(read(&store, kind::META, NAMES_MARKER).is_some()); + assert!(read(&store, kind::META, TAILNET_MARKER).is_some()); assert!(!paths.config_dir.join("migrated").exists()); assert!(!paths.settings_file().exists()); } @@ -1313,8 +1703,8 @@ mod tests { let store = open(&paths); assert!(store.encrypted()); let report = store.seal_report().unwrap(); - // Two secrets, a registry, two settings and the six markers. - assert_eq!((report.sealed, report.plain), (11, 0)); + // Two secrets, a registry, two settings and the eight markers. + assert_eq!((report.sealed, report.plain), (13, 0)); assert_eq!(read(&store, kind::SECRET, "github").unwrap(), "ghp-made-up"); } @@ -1433,9 +1823,11 @@ mod tests { old_hosts(&paths); let store = open(&paths); // In the layout of now: an empty key file was the user's keys and berth's. + // berth 0.5 did not join it to the tailnet by itself, and an apply still goes on without. assert_eq!( json_row(&store, kind::PROFILE, "studio").unwrap(), - json!({"hostname": "studio", "builder": {}, "access": {"address": "192.0.2.10", "auth": "ssh"}}) + json!({"hostname": "studio", "builder": {}, "access": {"address": "192.0.2.10", "auth": "ssh"}, + "ansible_vars": {(TAILNET_OPTIONAL): true}}) ); assert!(json_row(&store, kind::PROFILE, "attic").is_some()); assert_eq!( diff --git a/crates/berth-core/src/kept_profile.rs b/crates/berth-core/src/kept_profile.rs index 8b685d9..116fef1 100644 --- a/crates/berth-core/src/kept_profile.rs +++ b/crates/berth-core/src/kept_profile.rs @@ -1,6 +1,6 @@ //! The profile a host keeps of itself. Every apply and every first boot leaves //! it on the host, so that a berth that does not have the profile, or has lost -//! it, can take the host over from the host itself: `berth adopt`. +//! it, can take the host over from the host itself: `berth host adopt`. use crate::{ Result, @@ -16,15 +16,17 @@ use std::{io::Write, path::PathBuf, process::Stdio}; /// Where the role writes it, readable by root only. pub const PATH: &str = "/etc/berth/profile.json"; -/// The format of the document: 2 has the profile in the layout with -/// `access`. Format 1, of berth 0.4.1, had the layout before it; it is -/// converted when it is read. A berth refuses one newer than it knows. -pub const FORMAT: u64 = 2; +/// The format of the document: 3 has the profile with the fields `type` +/// and `target`. Format 2 named them `level` and `profile`, and format 1, of +/// berth 0.4.1, had the layout before `access`; both are converted when they +/// are read. A berth refuses one newer than it knows, so a berth of format 2 +/// asks to be updated rather than read `type` as a workstation. +pub const FORMAT: u64 = 3; /// What a profile's `builder` held of how one berth reached the host, or of /// that berth's own machine, in the layout before `access`, and what still /// belongs to this machine alone: a served host names this berth's server, -/// and a USB drive and the last task were never meant to be kept. +/// and a USB drive and the last action were never meant to be kept. const LOCAL_BUILDER: &[&str] = &[ "ssh_host", "ssh_user", @@ -46,22 +48,16 @@ const LOCAL_BUILDER: &[&str] = &[ /// a program of this machine. pub fn stored(config: &Value) -> Value { let mut config = config.clone(); + // berth makes a host's key to join the tailnet when the host needs one, + // so a key a profile of an older berth held never travels. + berth_api::profile::take_old_tailnet(&mut config); let Some(profile) = config.as_object_mut() else { return json!({}); }; - // An enrollment key is used up when the host joins, and a new one is made - // when it has to join again. - profile.remove("tailscale_auth_key"); profile.remove("access"); // The operator always has sudo without a password and berth's key. profile.remove("operator_passwordless_sudo"); profile.remove("berth_ssh_key"); - if let Some(vars) = profile - .get_mut("ansible_vars") - .and_then(Value::as_object_mut) - { - vars.remove("devbox_tailscale_auth_key"); - } if let Some(builder) = profile.get_mut("builder").and_then(Value::as_object_mut) { for key in LOCAL_BUILDER { builder.remove(*key); @@ -243,8 +239,8 @@ pub fn adopted(paths: &Paths, title: &str, kept: &Value, access: &Access) -> Res if !profile["builder"].is_object() { profile["builder"] = json!({}); } - // berth 0.4.1 kept a profile of the layout before `access`; what is left - // of that becomes the new layout, and then the adopting berth's way in. + // An older berth kept a profile of older names or of the layout before + // `access`; it takes those of now, and then the adopting berth's way in. crate::import::convert_profile(paths, title, &mut profile); profile = stored(&profile); if !profile["builder"].is_object() { @@ -436,7 +432,7 @@ mod tests { fn profile() -> Value { json!({ - "hostname": "studio", "profile": "cloud", + "hostname": "studio", "target": "cloud", "operator_username": "ops", "operator_password_hash": "$6$salt$hash", "operator_ssh_public_keys": ["ssh-ed25519 AAAA user"], "agent_username": "agent", "agent_password_hash": "$6$salt$agent", @@ -475,7 +471,13 @@ mod tests { #[test] fn what_is_used_once_or_is_this_berths_way_in_stays_out() { let kept = stored(&profile()); - assert!(kept.get("tailscale_auth_key").is_none()); + for old in [ + "tailscale_auth_key", + "tailscale_auto_key", + "tailscale_login_server", + ] { + assert!(kept.get(old).is_none(), "{old}"); + } assert!( kept["ansible_vars"] .get("devbox_tailscale_auth_key") @@ -527,12 +529,7 @@ mod tests { let back = adopted(&paths, "studio", &kept, &reached()).unwrap(); let dirs = json!({"demo": "/p/demo"}); let own = ["ssh-ed25519 OWN berth".to_owned()]; - let mut wanted = ansible_vars(&original, &dirs, &own); - // What is used once is not applied again. - wanted - .as_object_mut() - .unwrap() - .remove("devbox_tailscale_auth_key"); + let wanted = ansible_vars(&original, &dirs, &own); assert_eq!(ansible_vars(&back.profile, &dirs, &own), wanted); // Another berth applies the same keys of the profile and gives its own // apart, in a block of its own, so the first berth's block stays. @@ -580,6 +577,8 @@ mod tests { }); let back = adopted(&paths, "studio", &kept, &reached()).unwrap(); assert_eq!(back.profile["builder"], json!({"shipped_plugins": true})); + assert_eq!(back.profile["target"], "cloud"); + assert!(back.profile.get("profile").is_none()); assert_eq!(back.profile["access"]["address"], "192.0.2.10"); assert_eq!(back.profile["access"]["auth"], "berth"); // sudo is how the adopting berth found it, not what the old switch said. @@ -597,6 +596,30 @@ mod tests { assert_eq!(machine.arguments["server"], "https://192.0.2.20:8443"); } + #[test] + fn a_profile_kept_by_berth_05_takes_the_names_of_now() { + let (_home, paths) = scratch(); + let old = |level: &str| { + json!({ + "format": 2, "berth": "0.5.2", "host": "studio", "written": "2026-11-13T10:00:00Z", + "profile": {"hostname": "studio", "profile": "laptop", "level": level, "builder": {}}, + }) + }; + let back = adopted(&paths, "studio", &old("workstation"), &reached()).unwrap(); + assert_eq!(back.profile["target"], "laptop"); + assert!(back.profile.get("profile").is_none() && back.profile.get("type").is_none()); + assert!(back.profile.get("level").is_none()); + let back = adopted(&paths, "studio", &old("controller"), &reached()).unwrap(); + assert_eq!(back.profile["type"], "managed"); + // Kept again, it is written in the format of now. + let again = document("studio", &back.profile, None); + assert_eq!(again["format"], 3); + assert_eq!( + (&again["profile"]["type"], &again["profile"]["target"]), + (&json!("managed"), &json!("laptop")) + ); + } + #[test] fn a_newer_format_or_no_profile_is_refused() { let (_home, paths) = scratch(); diff --git a/crates/berth-core/src/keys/contact.rs b/crates/berth-core/src/keys/contact.rs index c83cac3..20d7f61 100644 --- a/crates/berth-core/src/keys/contact.rs +++ b/crates/berth-core/src/keys/contact.rs @@ -110,13 +110,6 @@ pub fn trust_key( once: Option, keep_password: bool, ) -> Result { - if host.served(paths).is_some() { - return Err(format!( - "{} is a host of a berth server, which logs in to it: ask there", - host.name - ) - .into()); - } let access = Access::of(&host.config); if access.link.is_some() { return Err(format!( @@ -198,7 +191,7 @@ pub fn trust_key( }) } -/// What `berth key host HOST` shows, and with `accept` does: the recorded +/// What `berth host verify HOST` shows, and with `accept` does: the recorded /// host keys, or, accepting a changed key, the ones the host has now. pub fn host_key(paths: &Paths, host: &Host, accept: bool) -> Result> { let store = crate::store::shared(paths)?; diff --git a/crates/berth-core/src/keys/known.rs b/crates/berth-core/src/keys/known.rs index 21b878c..28e031d 100644 --- a/crates/berth-core/src/keys/known.rs +++ b/crates/berth-core/src/keys/known.rs @@ -175,6 +175,33 @@ pub fn seed_from( Ok(false) } +/// Record the host keys that `known_hosts`, the known-hosts file of the user +/// berth runs as, has for a host berth has none of yet, under its address +/// or `[ADDRESS]:PORT`, so that a first login asks no question that user +/// has answered already. Returns the keys taken. +pub fn take_from_user( + paths: &Paths, + store: &Store, + place: &Place, + config: &serde_json::Value, + known_hosts: &std::path::Path, +) -> Result> { + let access = berth_api::profile::Access::of(config); + if access.address.is_empty() { + return Ok(vec![]); + } + let names = match access.port.as_str() { + "" | "22" => vec![access.address.clone()], + port => vec![format!("[{}]:{port}", access.address)], + }; + if !seed_from(store, place, known_hosts, &names)? { + return Ok(vec![]); + } + let record = Record::load(store, place)?; + write(paths, place, &record)?; + Ok(record.host_keys) +} + /// The host keys, `TYPE BASE64` each, that the text of a known-hosts file /// such as `~/.ssh/known_hosts` has for `name`, an address or /// `[ADDRESS]:PORT`, as `ssh-keygen -F` finds them: in plain and hashed @@ -240,7 +267,7 @@ pub fn changed_key(name: &str, said: &str) -> Option { || said.contains("Host key verification failed"); changed.then(|| { format!( - "The SSH host key of {name} is not the one berth recorded: the machine was set up again or replaced, or something is in between. If you know which, accept the new key with: berth key host {name} --accept" + "The SSH host key of {name} is not the one berth recorded: the machine was set up again or replaced, or something is in between. If you know which, accept the new key with: berth host verify {name} --accept" ) }) } @@ -249,6 +276,42 @@ pub fn changed_key(name: &str, said: &str) -> Option { mod tests { use super::*; + #[test] + fn a_first_login_takes_the_key_the_users_known_hosts_has() { + let dir = tempfile::tempdir().unwrap(); + let paths = Paths { + config_dir: dir.path().join("config"), + data_dir: dir.path().join("data"), + }; + let store = Store::open(&dir.path().join("berth.db")).unwrap(); + let place = Place::own("studio").unwrap(); + let known_hosts = dir.path().join("known_hosts"); + std::fs::write( + &known_hosts, + "192.0.2.10 ssh-ed25519 AAAAstudio\n[192.0.2.11]:2222 ssh-ed25519 AAAAdock\n", + ) + .unwrap(); + let config = serde_json::json!({"access": {"address": "192.0.2.10"}}); + let taken = take_from_user(&paths, &store, &place, &config, &known_hosts).unwrap(); + assert_eq!(taken, ["ssh-ed25519 AAAAstudio"]); + assert_eq!(Record::load(&store, &place).unwrap().host_keys, taken); + let written = std::fs::read_to_string(file(&paths, &place)).unwrap(); + assert!(written.contains("ssh-ed25519 AAAAstudio"), "{written}"); + // Once berth has a key, the file is not asked again. + std::fs::write(&known_hosts, "192.0.2.10 ssh-ed25519 AAAAother\n").unwrap(); + assert!( + take_from_user(&paths, &store, &place, &config, &known_hosts) + .unwrap() + .is_empty() + ); + // A port of its own is looked up as ssh writes it. + let dock = Place::own("dock").unwrap(); + let config = serde_json::json!({"access": {"address": "192.0.2.11", "port": "2222"}}); + std::fs::write(&known_hosts, "[192.0.2.11]:2222 ssh-ed25519 AAAAdock\n").unwrap(); + let taken = take_from_user(&paths, &store, &dock, &config, &known_hosts).unwrap(); + assert_eq!(taken, ["ssh-ed25519 AAAAdock"]); + } + #[test] fn a_known_hosts_file_is_searched_as_ssh_keygen_does() { // What `ssh-keygen -H` makes of a line for 192.0.2.10. @@ -318,7 +381,7 @@ mod tests { assert!( changed_key("studio", "Host key verification failed.") .unwrap() - .contains("berth key host studio --accept") + .contains("berth host verify studio --accept") ); } } diff --git a/crates/berth-core/src/keys/mod.rs b/crates/berth-core/src/keys/mod.rs index 95166ea..92dec1e 100644 --- a/crates/berth-core/src/keys/mod.rs +++ b/crates/berth-core/src/keys/mod.rs @@ -414,7 +414,6 @@ impl Login { program: "ssh".into(), args, first_line: password.map(str::to_owned), - via: None, env: self.env(), name: self.name.clone(), guard: self.guard.clone(), diff --git a/crates/berth-core/src/keys/ring.rs b/crates/berth-core/src/keys/ring.rs index cc77793..35902e0 100644 --- a/crates/berth-core/src/keys/ring.rs +++ b/crates/berth-core/src/keys/ring.rs @@ -158,7 +158,7 @@ pub fn public(paths: &Paths, store: &Store, name: &str) -> Result { } fn no_key(name: &str) -> Box { - format!("No key {name} in berth's key ring. See: berth key").into() + format!("No key {name} in berth's key ring. See: berth key list").into() } /// The key `berth`, made the first time it is asked for. Under a lock, so diff --git a/crates/berth-core/src/level.rs b/crates/berth-core/src/level.rs deleted file mode 100644 index 829b457..0000000 --- a/crates/berth-core/src/level.rs +++ /dev/null @@ -1,396 +0,0 @@ -//! What a host can do at each of its levels, and the one check that every -//! client asks before it offers or runs something: the UIs, the commands -//! and the server's API. A host's level is what its profile says it is -//! meant to be; what it is seen to be comes from asking it. - -use crate::{Result, host::Host}; -use berth_api::{Action, Version, ops::Role}; - -pub use berth_api::HostLevel as Level; - -/// Something berth can do with a host. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub enum Capability { - /// A terminal and commands over SSH, the inventory, whether it answers. - Ssh, - /// Install or update berth's controller alone. - InstallController, - /// Sessions, the channel, files, units, logs and processes. - Sessions, - /// The controller's tasks: status, update and reboot. - Tasks, - /// Secrets and registry logins sent to the host. - Secrets, - /// Plugin features, their tasks and services. - Plugins, - /// Applying the profile, a roll-out over it, a seed or an installer image. - Apply, - /// The desktop: screenshots and access links. - Desktop, -} - -impl Capability { - pub const ALL: [Capability; 8] = [ - Capability::Ssh, - Capability::InstallController, - Capability::Sessions, - Capability::Tasks, - Capability::Secrets, - Capability::Plugins, - Capability::Apply, - Capability::Desktop, - ]; - - pub fn name(self) -> &'static str { - match self { - Capability::Ssh => "ssh", - Capability::InstallController => "install-controller", - Capability::Sessions => "sessions", - Capability::Tasks => "tasks", - Capability::Secrets => "secrets", - Capability::Plugins => "plugins", - Capability::Apply => "apply", - Capability::Desktop => "desktop", - } - } - - /// What it is, as the subject of a sentence. - pub fn label(self) -> &'static str { - match self { - Capability::Ssh => "a terminal over SSH", - Capability::InstallController => "installing the controller alone", - Capability::Sessions => "working with sessions, the channel and files", - Capability::Tasks => "running the controller's tasks", - Capability::Secrets => "sending secrets and registry logins", - Capability::Plugins => "using plugin features", - Capability::Apply => "applying a profile, a roll-out, a seed or an installer", - Capability::Desktop => "using the desktop", - } - } - - /// Whether a host of `level` has it. The controller alone is installed - /// on a host below a workstation; a workstation's comes with applying. - pub fn allowed(self, level: Level) -> bool { - match self { - Capability::Ssh => true, - Capability::InstallController => level != Level::Workstation, - Capability::Sessions | Capability::Tasks | Capability::Secrets => { - level >= Level::Controller - } - Capability::Plugins | Capability::Apply | Capability::Desktop => { - level == Level::Workstation - } - } - } -} - -/// The capabilities and the levels that have each, in the order of -/// [`Level::ALL`]: for a page that shows the table. -pub fn table() -> Vec<(Capability, [bool; 3])> { - Capability::ALL - .into_iter() - .map(|capability| { - ( - capability, - Level::ALL.map(|level| capability.allowed(level)), - ) - }) - .collect() -} - -/// Why something cannot be done with a host now, and the action that -/// raises its level so it can. -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct Refusal { - pub why: String, - /// `install-controller`, `apply`, or `setup` for the set-up flow that - /// makes the host a workstation. - pub raise: &'static str, -} - -impl std::fmt::Display for Refusal { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - f.write_str(&self.why) - } -} - -impl std::error::Error for Refusal {} - -/// Whether the host `name` of `level` may do this, or why not. -pub fn check(name: &str, level: Level, capability: Capability) -> std::result::Result<(), Refusal> { - if capability.allowed(level) { - return Ok(()); - } - let what = capability.label(); - Err(match (capability, level) { - (Capability::InstallController, _) => Refusal { - why: format!( - "{name} is a workstation: applying its configuration installs and updates its controller. Apply it with: berth apply {name}" - ), - raise: "apply", - }, - (_, Level::Reachable) if capability.allowed(Level::Controller) => Refusal { - why: format!( - "{name} is reachable only: berth's controller is not installed there, and {what} needs it. Install it with: berth host install {name}" - ), - raise: "install-controller", - }, - _ => Refusal { - why: format!( - "{name} is a {level} host, not a workstation: {what} needs its whole profile. Set its level to workstation in its set-up, then apply it" - ), - raise: "setup", - }, - }) -} - -/// The same for a host by its profile, as an error to stop with. -pub fn guard(host: &Host, capability: Capability) -> Result<()> { - check(&host.name, of(&host.config), capability)?; - Ok(()) -} - -/// The level a profile means its host to have. -pub fn of(config: &serde_json::Value) -> Level { - berth_api::profile::level(config) -} - -/// An action as a host of `level` may run it: unavailable, with the reason, -/// when the level rules it out. -pub fn fit(mut action: Action, name: &str, level: Level, capability: Capability) -> Action { - if let Err(refusal) = check(name, level, capability) { - action.available = false; - action.why = refusal.why; - } - action -} - -/// What a host was seen to be by asking its controller: what its record -/// says when it answers, reachable when berth logged in and found no -/// controller, nothing when it could not be asked. -pub fn seen(answer: std::result::Result<&Version, bool>) -> Option { - match answer { - // A controller without a record was installed outside berth. - Ok(version) => Some(version.level.unwrap_or(Level::Controller)), - Err(true) => Some(Level::Reachable), - Err(false) => None, - } -} - -pub use berth_api::level_mismatch as mismatch; - -/// Why installing the controller alone must not run on a host seen to be a -/// workstation, whatever its profile says: it would take the workstation's -/// controller apart. -pub fn refuse_install(name: &str, intended: Level, observed: Option) -> Option { - match (intended, observed) { - (Level::Workstation, _) | (_, None) => None, - (_, Some(Level::Workstation)) => Some(format!( - "{} Installing the controller alone would take its controller apart, so it is not offered", - mismatch(name, intended, observed).unwrap_or_default() - )), - _ => None, - } -} - -/// What the host's controller says the host is, asked through the link -/// once and briefly: `None` when no controller answers. -pub fn observe(paths: &crate::paths::Paths, host: &Host, password: Option<&str>) -> Option { - let link = crate::link::Link::new(host.link(paths, password).ok()?); - let runtime = crate::link::runtime().ok()?; - let asked = runtime.block_on(async { - tokio::time::timeout( - std::time::Duration::from_secs(15), - link.get::("/v1/version"), - ) - .await - }); - runtime.shutdown_background(); - match asked { - Ok(Ok(version)) => seen(Ok(&version)), - _ => None, - } -} - -/// berth's own action that puts the controller on a host, as a host of -/// `level` sees it. `observed` is what its controller says it is, when it -/// answered. -pub fn install_controller_action(name: &str, level: Level, observed: Option) -> Action { - let (label, help) = match level { - Level::Controller => ( - "Update the controller", - "Install berthd and box of this berth on the host again, and restart the controller: for example after berth was updated. Nothing else of the profile is applied.", - ), - _ => ( - "Install the controller", - "Put berthd and box on the host and start the controller, so that sessions, tasks and files work there. Nothing else of a workstation is installed: no desktop, agents or plugins. The SSH user is the account sessions run as. The host's level becomes controller.", - ), - }; - let action = Action { - name: Capability::InstallController.name().into(), - label: label.into(), - help: help.into(), - group: "Set-Up".into(), - role: Role::Admin, - confirm: Some(format!("Install berth's controller on {name} now?")), - ..Action::default() - }; - let mut action = fit(action, name, level, Capability::InstallController); - if let Some(why) = refuse_install(name, level, observed) { - action.available = false; - action.why = why; - } - action -} - -/// berth's own action that gives a host another name, at any level: the -/// machine stays as it is. -pub fn rename_action() -> Action { - Action { - name: "rename".into(), - label: "Rename".into(), - help: "Give the host another name in berth. Its profile, passwords, host keys, machine record, messages and jobs go with it. The machine itself stays as it is and keeps running, and so does its hostname unless hostname says otherwise.".into(), - group: "Set-Up".into(), - role: Role::Admin, - arguments: vec![ - berth_api::Argument { - name: "name".into(), - label: "New name".into(), - help: "The host's name in berth from now on: letters, digits, `-`, `_` and `.`." - .into(), - required: true, - ..Default::default() - }, - berth_api::Argument { - name: "hostname".into(), - label: "Hostname too".into(), - help: "Make the new name the profile's hostname as well. The machine takes it at its next apply.".into(), - default: "no".into(), - values: vec!["no".into(), "yes".into()], - ..Default::default() - }, - ], - ..Action::default() - } -} - -/// berth's own actions on a host of this machine, which need no server: -/// those that change the host's level, and renaming it. -pub fn local_actions(name: &str, level: Level, observed: Option) -> Vec { - vec![ - install_controller_action(name, level, observed), - rename_action(), - ] -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn each_level_can_do_what_the_ones_below_it_can() { - use Capability::*; - let rows: Vec<(&str, [bool; 3])> = table() - .into_iter() - .map(|(capability, levels)| (capability.name(), levels)) - .collect(); - assert_eq!( - rows, - [ - (Ssh.name(), [true, true, true]), - (InstallController.name(), [true, true, false]), - (Sessions.name(), [false, true, true]), - (Tasks.name(), [false, true, true]), - (Secrets.name(), [false, true, true]), - (Plugins.name(), [false, false, true]), - (Apply.name(), [false, false, true]), - (Desktop.name(), [false, false, true]), - ] - ); - } - - #[test] - fn a_refusal_names_the_action_that_raises_the_level() { - let refused = check("studio", Level::Reachable, Capability::Sessions).unwrap_err(); - assert_eq!(refused.raise, "install-controller"); - assert!(refused.why.contains("berth host install studio")); - let refused = check("studio", Level::Reachable, Capability::Apply).unwrap_err(); - assert_eq!(refused.raise, "setup"); - assert!(refused.why.contains("a reachable host, not a workstation")); - let refused = check("studio", Level::Controller, Capability::Desktop).unwrap_err(); - assert_eq!(refused.raise, "setup"); - let refused = - check("studio", Level::Workstation, Capability::InstallController).unwrap_err(); - assert_eq!(refused.raise, "apply"); - assert!(check("studio", Level::Workstation, Capability::Apply).is_ok()); - } - - #[test] - fn the_apply_guard_reads_the_profile() { - let host = |config| Host { - name: "dock".into(), - place: crate::profiles::Place::File("/dock.json".into()), - config, - }; - let reachable = - host(serde_json::json!({"level": "reachable", "access": {"address": "192.0.2.4"}})); - let refused = guard(&reachable, Capability::Apply).unwrap_err(); - assert!(refused.to_string().contains("not a workstation")); - assert!(guard(&host(serde_json::json!({"builder": {}})), Capability::Apply).is_ok()); - } - - #[test] - fn what_a_host_is_seen_to_be_comes_from_its_controller() { - let version = |level| Version { - version: "1".into(), - api: 1, - mode: "system".into(), - applied: None, - level, - }; - assert_eq!(seen(Ok(&version(None))), Some(Level::Controller)); - assert_eq!( - seen(Ok(&version(Some(Level::Workstation)))), - Some(Level::Workstation) - ); - assert_eq!(seen(Err(true)), Some(Level::Reachable)); - assert_eq!(seen(Err(false)), None); - } - - #[test] - fn installing_the_controller_is_offered_below_a_workstation() { - let action = install_controller_action("dock", Level::Reachable, None); - assert!(action.available && action.label == "Install the controller"); - assert_eq!( - install_controller_action("dock", Level::Controller, None).label, - "Update the controller" - ); - let action = install_controller_action("dock", Level::Workstation, None); - assert!(!action.available && action.why.contains("berth apply dock")); - // A controller seen to be one is updated as its profile says. - let action = install_controller_action("dock", Level::Reachable, Some(Level::Controller)); - assert!(action.available); - } - - #[test] - fn a_host_seen_to_be_a_workstation_never_gets_the_controller_alone() { - for intended in [Level::Reachable, Level::Controller] { - let action = install_controller_action("dock", intended, Some(Level::Workstation)); - assert!(!action.available); - assert!( - action.why.starts_with(&format!( - "dock is a workstation; its profile here says {intended}" - )), - "{}", - action.why - ); - assert!(action.why.contains("berth adopt dock")); - } - assert_eq!( - mismatch("dock", Level::Reachable, Some(Level::Controller)).unwrap(), - "dock has berth's controller; its profile here says reachable. To adopt it: berth host level dock controller" - ); - assert!(mismatch("dock", Level::Workstation, Some(Level::Controller)).is_none()); - assert!(mismatch("dock", Level::Controller, None).is_none()); - } -} diff --git a/crates/berth-core/src/lib.rs b/crates/berth-core/src/lib.rs index 1f84c7d..44923d9 100644 --- a/crates/berth-core/src/lib.rs +++ b/crates/berth-core/src/lib.rs @@ -5,13 +5,14 @@ pub mod apply; pub mod credentials; pub mod discovery; pub mod host; +pub mod host_type; pub mod import; pub mod job; pub mod kept; pub mod kept_profile; pub mod keys; -pub mod level; pub mod link; +pub mod node; pub mod paths; pub mod plugin_state; pub mod plugins; @@ -30,3 +31,23 @@ pub mod store; pub mod tailnet; pub type Result = std::result::Result>; + +/// A reason found before anything began not to begin: the host or this +/// server cannot do it now, as when a secret it needs is not set. A caller +/// says it as a refusal rather than as a request that was wrong. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Refused(pub String); + +impl std::fmt::Display for Refused { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(&self.0) + } +} + +impl std::error::Error for Refused {} + +/// Whether `error` is a refusal before anything began: [`Refused`], or a +/// host of a type that cannot do it. +pub fn is_refused(error: &(dyn std::error::Error + 'static)) -> bool { + error.is::() || error.is::() +} diff --git a/crates/berth-core/src/link.rs b/crates/berth-core/src/link.rs index e1b104a..13c1d79 100644 --- a/crates/berth-core/src/link.rs +++ b/crates/berth-core/src/link.rs @@ -3,10 +3,7 @@ //! connection whole, so each concurrent request or stream is its own process; //! idle ones are kept and used again. -use crate::server::Server; use berth_api::client::{self, Client, Error, Frame, Response}; -use berth_api::ops::{self, Kind}; -use berth_api::portal::Portal; use futures_util::{Sink, Stream}; use serde::de::DeserializeOwned; use serde_json::Value; @@ -29,8 +26,6 @@ pub struct LinkSpec { pub args: Vec, /// Written before anything else: a sudo password. pub first_line: Option, - /// Instead of a command: the host of this name behind a `berth serve`. - pub via: Option<(Server, String)>, /// Set in the command's environment, as ssh's askpass helper needs. pub env: Vec<(String, String)>, /// The host's name, for what is said when it fails. @@ -102,8 +97,6 @@ pub type LinkStream = Pin>; pub struct Link { spec: LinkSpec, - /// The portal of the server the host is behind, once it is open. - portal: tokio::sync::Mutex>, idle: Mutex>, /// What the link commands last wrote to standard error. complaints: Arc>, @@ -116,7 +109,6 @@ impl Link { pub fn new(spec: LinkSpec) -> Link { Link { spec, - portal: Default::default(), idle: Mutex::new(vec![]), complaints: Default::default(), worked: Default::default(), @@ -209,7 +201,7 @@ impl Link { } else if let Some(changed) = crate::keys::known::changed_key(&self.spec.name, &said) { changed } else if said.contains("a password is required") { - format!("{said}: set BERTH_SUDO_PASSWORD") + format!("{said}: keep the sudo password with berth secret set sudo/HOST") } else { said }) @@ -231,9 +223,6 @@ impl Link { headers: &[(&str, &str)], body: &[u8], ) -> client::Result { - if let Some((server, host)) = &self.spec.via { - return self.send_via(server, host, method, path, body).await; - } let kept = self.idle.lock().unwrap().pop().filter(Client::usable); let mut attempt = kept.is_some(); let mut client = match kept { @@ -284,42 +273,27 @@ impl Link { self.call("GET", path, None).await } + /// What the host's controller says of itself, read as this API says it + /// also from a berthd of an older API. + pub async fn version(&self) -> client::Result { + let reply: Value = self.get("/v1/version").await?; + berth_api::Version::from_reply(reply) + .map_err(|error| Error::Transport(format!("version: {error}"))) + } + pub async fn post(&self, path: &str, body: &Value) -> client::Result { self.call("POST", path, Some(body)).await } - /// Run a task of the host, as `tasks.run` takes it, and the session it - /// runs as. Through a berth server it is the host's action there, which - /// the server keeps a job of and gives the server's secrets to. - pub async fn run_task(&self, name: &str, body: &Value) -> client::Result { - let Some((server, host)) = &self.spec.via else { - let path = format!("/v1/tasks/{}", client::encode(name)); - return self.post(&path, body).await; - }; - let portal = self.portal(server).await?; - let args = serde_json::json!({"name": host, "action": name}); - let acted = portal - .call("hosts.act", None, args, Some(body.clone())) - .await?; - serde_json::from_value(acted["session"].clone()).map_err(|_| { - Error::Transport(format!( - "{name} on {host} is not a task of the host: run it with berth run {host} {name}" - )) - }) + /// Run an action of the host, as `actions.run` takes it, and the session it + /// runs as. + pub async fn run_action(&self, name: &str, body: &Value) -> client::Result { + let path = format!("/v1/actions/{}", client::encode(name)); + self.post(&path, body).await } /// Open a stream. It holds its connection for as long as it lives. pub async fn stream(&self, path: &str) -> client::Result { - if let Some((server, host)) = &self.spec.via { - let (path, query) = path.split_once('?').unwrap_or((path, "")); - let (op, mut args) = recognised(Kind::Stream, "GET", path)?; - args.extend(parameters(query)); - let portal = self.portal(server).await?; - let stream = portal - .open(op.name, Some(host), Value::Object(args)) - .await?; - return Ok(Box::pin(stream)); - } let conn = self.connect().await?; match client::websocket(conn, path, "").await { Ok(stream) => { @@ -330,56 +304,6 @@ impl Link { } } - /// The server's portal: the one that is open, or a new one. - async fn portal(&self, server: &Server) -> client::Result { - let mut portal = self.portal.lock().await; - match portal.as_ref().filter(|portal| portal.usable()) { - Some(portal) => Ok(portal.clone()), - None => { - let opened = server.portal().await?; - *portal = Some(opened.clone()); - Ok(opened) - } - } - } - - /// A request as the operation it is, through the server's portal. - async fn send_via( - &self, - server: &Server, - host: &str, - method: &str, - path: &str, - body: &[u8], - ) -> client::Result { - let (path, query) = path.split_once('?').unwrap_or((path, "")); - let (op, mut args) = recognised(Kind::Call, method, path)?; - args.extend(parameters(query)); - let data = match (body.is_empty(), op.body) { - (true, _) => None, - (false, ops::Body::Bytes) => Some(serde_json::json!({"base64": berth_api::b64(body)})), - (false, ops::Body::Json) => Some( - serde_json::from_slice(body) - .map_err(|error| Error::Transport(error.to_string()))?, - ), - }; - let portal = self.portal(server).await?; - let result = portal - .call(op.name, Some(host), Value::Object(args), data) - .await?; - // Bytes come back wrapped; everything else is the JSON the request answers with. - let body = match result["base64"].as_str().and_then(berth_api::unb64) { - Some(bytes) if result.get("content_type").is_some() => bytes, - _ if result.is_null() => vec![], - _ => result.to_string().into_bytes(), - }; - Ok(Response { - status: 200, - headers: Default::default(), - body: body.into(), - }) - } - /// Whether the failure means the host has no berthd to talk to yet. pub fn missing(error: &Error) -> bool { let Error::Transport(message) = error else { @@ -392,33 +316,6 @@ impl Link { } } -type Arguments = serde_json::Map; - -fn recognised( - kind: Kind, - method: &str, - path: &str, -) -> client::Result<(&'static ops::Op, Arguments)> { - let (op, found) = ops::recognise(ops::HOST, kind, method, path) - .ok_or_else(|| Error::Transport(format!("{method} {path} is no operation of a host")))?; - let args = found - .into_iter() - .map(|(name, value)| (name.to_owned(), Value::String(value))); - Ok((op, args.collect())) -} - -/// The parameters of a query string. -fn parameters(query: &str) -> Arguments { - query - .split('&') - .filter(|pair| !pair.is_empty()) - .map(|pair| { - let (name, value) = pair.split_once('=').unwrap_or((pair, "")); - (client::decode(name), Value::String(client::decode(value))) - }) - .collect() -} - /// A runtime for a program that is not asynchronous itself. pub fn runtime() -> std::io::Result { tokio::runtime::Builder::new_multi_thread() diff --git a/crates/berth-core/src/node.rs b/crates/berth-core/src/node.rs new file mode 100644 index 0000000..91b915f --- /dev/null +++ b/crates/berth-core/src/node.rs @@ -0,0 +1,1409 @@ +//! Nodes: servers berth applies without a desktop, whose plugin features +//! set up what hosts machines, such as Incus or a Docker engine. A feature +//! may offer provisioners: its tasks report where the server is and what +//! pins it, the role keeps the report in the host's record of the apply, and +//! berth registers a provisioner of its own for each, naming the node it +//! lives on. A feature may also ask for berth's client certificate, which +//! berth makes once and the node then trusts. + +use crate::{ + Result, + host::Host, + paths::Paths, + plugins::{self, Plugin}, + provisioner::{self, Entry}, + registry::Store, +}; +use berth_api::{HostType, Offer}; +use serde_json::{Value, json}; +use sha2::Digest; +use std::{ + collections::BTreeMap, + net::{IpAddr, ToSocketAddrs}, +}; + +/// The plugin features a profile names, as `PLUGIN/FEATURE`. A node names +/// none unless its profile does; a workstation's role has defaults of its own. +pub fn features(config: &Value) -> Vec { + config["ansible_vars"]["devbox_plugin_features"] + .as_array() + .into_iter() + .flatten() + .filter_map(Value::as_str) + .map(str::to_owned) + .collect() +} + +/// What the feature named `PLUGIN/FEATURE` says of the hosts it is for. +pub fn feature(plugins: &[Plugin], name: &str) -> Option { + let (plugin, id) = name.split_once('/')?; + let plugin = plugins.iter().find(|known| known.id == plugin)?; + plugins::hosting(plugin) + .into_iter() + .find(|known| known.feature == id) +} + +/// Why the profile of `name` names a feature that is not for a host of its +/// type, such as a server's engine on a workstation or an agent on a node. +pub fn check_features(plugins: &[Plugin], name: &str, config: &Value) -> Result<()> { + let host_type = crate::host_type::of(config); + let unfit: Vec = features(config) + .into_iter() + .filter(|wanted| { + feature(plugins, wanted).is_some_and(|found| !found.types.contains(&host_type)) + }) + .collect(); + if unfit.is_empty() { + return Ok(()); + } + let other = match host_type { + HostType::Node => "a workstation", + _ => "a node", + }; + Err(format!( + "{name} is a {host_type}, and {} {} for {other}: take {} out of its features", + unfit.join(", "), + if unfit.len() == 1 { "is" } else { "are" }, + if unfit.len() == 1 { "it" } else { "them" }, + ) + .into()) +} + +// ── berth's client certificates ───────────────────────────────────────────── + +/// The SHA-256 of a PEM certificate's DER, in lowercase hex, as Incus +/// names a certificate it trusts. +pub fn fingerprint(pem: &str) -> Option { + let body: String = pem + .lines() + .skip_while(|line| !line.starts_with("-----BEGIN CERTIFICATE")) + .skip(1) + .take_while(|line| !line.starts_with("-----END")) + .collect(); + let der = berth_api::unb64(body.trim())?; + if der.is_empty() { + return None; + } + Some( + sha2::Sha256::digest(&der) + .iter() + .map(|byte| format!("{byte:02x}")) + .collect(), + ) +} + +/// A key and a self-signed certificate for berth as a client, as PEM. +fn make_client_certificate() -> Result<(String, String)> { + let key = rcgen::KeyPair::generate_for(&rcgen::PKCS_ECDSA_P384_SHA384) + .map_err(|error| format!("Making a client key: {error}"))?; + let mut params = rcgen::CertificateParams::new(Vec::::new()) + .map_err(|error| format!("Making a client certificate: {error}"))?; + params + .distinguished_name + .push(rcgen::DnType::CommonName, "berth"); + params.extended_key_usages = vec![rcgen::ExtendedKeyUsagePurpose::ClientAuth]; + let certificate = params + .self_signed(&key) + .map_err(|error| format!("Making a client certificate: {error}"))?; + Ok((certificate.pem(), key.serialize_pem())) +} + +/// The client certificates the features of a profile ask for, by feature: +/// the certificate, its fingerprint and the name the node trusts it by. A +/// secret pair berth does not have yet is made now and kept in the store; +/// only the certificate goes to the node. +pub fn client_certificates(plugins: &[Plugin], store: &Store, config: &Value) -> Result { + let mut found = serde_json::Map::new(); + for wanted in features(config) { + let Some(feature) = feature(plugins, &wanted) else { + continue; + }; + let Some((cert_name, key_name)) = &feature.trusts else { + continue; + }; + let certificate = match (store.secret(cert_name)?, store.secret(key_name)?) { + (Some(certificate), Some(key)) + if !certificate.trim().is_empty() && !key.trim().is_empty() => + { + certificate + } + _ => { + let (certificate, key) = make_client_certificate()?; + store.set_secret(key_name, Some(&key))?; + store.set_secret(cert_name, Some(&certificate))?; + certificate + } + }; + let print = fingerprint(&certificate) + .ok_or_else(|| format!("The berth secret {cert_name} is not a PEM certificate"))?; + found.insert( + wanted, + json!({ + "certificate": certificate, + "fingerprint": print, + "name": format!("berth-{}", &print[..12]), + }), + ); + } + Ok(Value::Object(found)) +} + +// ── Provisioners a node offers ────────────────────────────────────────────── + +/// An address as the host part of a URL: an IPv6 one in brackets. +fn url_host(address: &str) -> String { + match address.contains(':') && !address.starts_with('[') { + true => format!("[{address}]"), + false => address.to_owned(), + } +} + +/// A value of an offer with `{address}`, `{user}` and `{port}` filled in +/// from where berth reaches the node and as whom. +fn fill(value: &str, host: &Host) -> String { + let access = host.access(); + let user = match access.user.is_empty() { + true => "root".to_owned(), + false => access.user.clone(), + }; + let port = match access.port.is_empty() { + true => "22".to_owned(), + false => access.port.clone(), + }; + value + .replace("{address}", &url_host(&access.address)) + .replace("{user}", &user) + .replace("{port}", &port) +} + +/// The name berth registers a node's provisioner under: `NODE-PLUGIN-ID`, +/// made a valid name, with a number when it is taken by another. +fn free_name(registered: &[(String, Entry)], node: &str, kind: &str) -> String { + let mut stem: String = format!("{node}-{}", provisioner::default_name(kind)) + .to_lowercase() + .chars() + .map( + |ch| match ch.is_ascii_alphanumeric() || ch == '-' || ch == '_' { + true => ch, + false => '-', + }, + ) + .collect(); + if !stem.starts_with(|ch: char| ch.is_ascii_lowercase()) { + stem = format!("n-{stem}"); + } + let mut free = stem.clone(); + let mut number = 2; + while registered.iter().any(|(known, _)| *known == free) { + free = format!("{stem}-{number}"); + number += 1; + } + free +} + +/// A URL or an address as what identifies a server: its host and port, +/// in lowercase, without the scheme, the user or a path. +fn server_of(value: &str) -> String { + let rest = value.split_once("://").map_or(value, |(_, rest)| rest); + let rest = rest.split('/').next().unwrap_or(rest); + let rest = rest.rsplit_once('@').map_or(rest, |(_, rest)| rest); + rest.trim().to_lowercase() +} + +/// The host and the port of what [`server_of`] gives, without brackets. +fn host_port(server: &str) -> (&str, &str) { + if let Some(rest) = server.strip_prefix('[') { + let (host, rest) = rest.split_once(']').unwrap_or((rest, "")); + return (host, rest.strip_prefix(':').unwrap_or("")); + } + match server.matches(':').count() { + 1 => server.split_once(':').unwrap_or((server, "")), + _ => (server, ""), + } +} + +/// How berth finds the addresses of a name: as it would connect to it. +type Resolve<'a> = &'a dyn Fn(&str) -> Vec; + +/// The addresses a name has where berth runs. +fn resolve(name: &str) -> Vec { + if let Ok(address) = name.parse::() { + return vec![address]; + } + (name, 0) + .to_socket_addrs() + .map(|found| found.map(|address| address.ip()).collect()) + .unwrap_or_default() +} + +/// Whether the hosts `one` and `other` are the same machine to berth: the +/// same name, or names that berth's own lookup gives an address in common, +/// as `node` and `incus-host` both name the gateway of a server's container. +fn same_machine(one: &str, other: &str, resolve: Resolve) -> bool { + let (one, other) = (one.trim().to_lowercase(), other.trim().to_lowercase()); + if one.is_empty() || other.is_empty() { + return false; + } + if one == other { + return true; + } + let theirs = resolve(&other); + resolve(&one).iter().any(|address| theirs.contains(address)) +} + +/// Whether berth reaches the node `host` at the URL or address `value`. +fn on_node(value: &str, host: &Host, resolve: Resolve) -> bool { + let server = server_of(value); + same_machine(host_port(&server).0, &host.access().address, resolve) +} + +/// Whether berth itself can tell that the provisioner `entry`, registered by +/// hand, reaches the server the node offers: a setting of it names the same +/// port on the same machine as the offer does, and that machine is where +/// berth reaches the node. Nothing else the node reports counts, as it could +/// name any server. +fn reaches_node( + entry: &Entry, + settings: &BTreeMap, + host: &Host, + resolve: Resolve, +) -> bool { + settings.iter().any(|(argument, offered)| { + let kept = entry.settings.get(argument).map_or("", String::as_str); + let (offered, kept) = (server_of(offered), server_of(kept)); + let ((offered_host, offered_port), (kept_host, kept_port)) = + (host_port(&offered), host_port(&kept)); + offered_port == kept_port + && same_machine(offered_host, &host.access().address, resolve) + && same_machine(kept_host, &host.access().address, resolve) + }) +} + +/// Why the node says the provisioner `entry` reaches the server it offers, +/// which berth cannot check: it pins the certificate the node reports, which +/// is public, or names a server the node gives as its own. +fn said_same( + store: &Store, + name: &str, + entry: &Entry, + offer: &Offer, + host: &Host, +) -> Option<&'static str> { + let pins = |secret: &str, value: &str| { + store + .secret(secret) + .ok() + .flatten() + .is_some_and(|kept| !kept.trim().is_empty() && kept.trim() == value.trim()) + }; + let pinned = offer.secrets.iter().any(|(secret, value)| { + pins(&provisioner::secret_name(name, secret), value) + || (store + .secret(&provisioner::secret_name(name, secret)) + .ok() + .flatten() + .is_none() + && pins(secret, value)) + }); + if pinned { + return Some("it pins the certificate the node reports"); + } + let named = offer.settings.iter().any(|(argument, wanted)| { + let kept = entry.settings.get(argument).map_or("", String::as_str); + let mut reaches = vec![fill(wanted, host)]; + reaches.extend( + offer + .aliases + .get(argument) + .into_iter() + .flatten() + .map(|alias| fill(alias, host)), + ); + // An empty value names the engine or server where berth runs. + reaches + .iter() + .any(|reach| match (reach.is_empty(), kept.is_empty()) { + (true, true) => true, + (false, false) => server_of(reach) == server_of(kept), + _ => false, + }) + }); + named.then_some("it names a server the node gives as its own") +} + +/// Register the provisioners the node `host` offers, as its last apply +/// reported them: one berth registered for the node before is brought up to +/// date, one registered by hand that berth can tell reaches the node is taken +/// over, and any other is registered under a new name. Those of the node no +/// feature offers any more are forgotten, unless they still hold resources. +/// Says what it did, a line each. +/// +/// What a node reports comes from a machine berth manages, so it touches +/// only the node's own provisioners: an offer is taken only from a feature +/// its profile names, of the provisioners that feature's manifest offers, +/// with that provisioner's own arguments and secrets, and with URLs that +/// name the node. +pub fn register( + paths: &Paths, + store: &Store, + host: &Host, + offers: &[Offer], +) -> Result> { + register_by(paths, store, host, offers, &resolve) +} + +fn register_by( + paths: &Paths, + store: &Store, + host: &Host, + offers: &[Offer], + resolve: Resolve, +) -> Result> { + let plugins = provision_plugins(paths); + let wanted = features(&host.config); + let mut said = vec![]; + let mut kept: Vec = vec![]; + for offer in offers { + let Some((plugin_id, _)) = offer.feature.split_once('/') else { + continue; + }; + let hosting = feature(&plugins, &offer.feature); + let Some(hosting) = hosting.filter(|_| wanted.contains(&offer.feature)) else { + said.push(format!( + "! {} reports what {} offers, which is not among its features; it is left out", + host.name, offer.feature + )); + continue; + }; + for id in &offer.provisioners { + let kind = format!("{plugin_id}/{id}"); + if !hosting.offers.contains(id) { + said.push(format!( + "! {} offers {kind}, which {} does not offer; it is left out", + host.name, offer.feature + )); + continue; + } + let Ok((_, declared)) = crate::provision::find(&plugins, &kind) else { + said.push(format!( + "! {} offers {kind}, which no plugin here has", + host.name + )); + continue; + }; + let settings: BTreeMap = offer + .settings + .iter() + .filter(|(argument, _)| declared.own.contains(argument)) + .map(|(argument, value)| (argument.clone(), fill(value, host))) + // An empty value is the argument's default, as berth keeps no empty settings. + .filter(|(_, value)| !value.is_empty()) + .collect(); + if let Some((argument, value)) = settings + .iter() + .find(|(_, value)| value.contains("://") && !on_node(value, host, resolve)) + { + said.push(format!( + "! {} offers {kind} with {argument}={value}, which is not where berth reaches {}; it is left out", + host.name, host.name + )); + continue; + } + let registered = provisioner::entries(paths)?; + let ours = registered + .iter() + .find(|(_, entry)| entry.node == host.name && entry.kind == kind); + let by_hand = || { + registered.iter().find(|(_, entry)| { + entry.node.is_empty() + && entry.kind == kind + && reaches_node(entry, &settings, host, resolve) + }) + }; + let found = ours.or_else(by_hand); + if found.is_none() { + // One the node only says is the same is left to its owner, and no second one is made beside it. + let doubt = registered.iter().find_map(|(name, entry)| { + (entry.kind == kind && entry.node != host.name) + .then(|| { + said_same(store, name, entry, offer, host).map(|why| (name, entry, why)) + }) + .flatten() + }); + match doubt { + Some((name, entry, _)) if !entry.node.is_empty() => { + said.push(format!( + "! {} offers {kind}, and says it is the server of {name}, which is {}'s; both are left as they are", + host.name, entry.node + )); + continue; + } + Some((name, _, why)) => { + said.push(format!( + "! {} offers {kind}, and {name} may reach the same server ({why}), which berth cannot check; both are left as they are. To make {name} {}'s: berth provisioner run {name} adopt node={}", + host.name, host.name, host.name + )); + continue; + } + None => {} + } + } + let (name, how) = match found { + Some((name, entry)) if entry.node.is_empty() => (name.clone(), "took over"), + Some((name, _)) => (name.clone(), "kept"), + None => (free_name(®istered, &host.name, &kind), "registered"), + }; + let before = provisioner::entry(paths, &name)?; + // One taken over reaches the server already, as it was set up by hand. + let mut values = before + .as_ref() + .map(|entry| entry.settings.clone()) + .unwrap_or_default(); + if how != "took over" { + values.extend(settings.clone()); + } + let entry = Entry { + kind: kind.clone(), + settings: values, + node: host.name.clone(), + feature: offer.feature.clone(), + }; + provisioner::save_entry(paths, &name, &entry)?; + for (secret, value) in &offer.secrets { + if !declared.secrets.iter().any(|known| &known.name == secret) { + continue; + } + let own = provisioner::secret_name(&name, secret); + if store.secret(&own)?.as_deref() != Some(value.as_str()) { + store.set_secret(&own, Some(value))?; + } + } + match (how, before.as_ref() == Some(&entry)) { + ("kept", true) => {} + ("kept", false) => { + said.push(format!("✓ Updated the provisioner {name} of {}", host.name)) + } + ("took over", _) => said.push(format!( + "✓ The provisioner {name} reaches the same server, and is {}'s now", + host.name + )), + _ => said.push(format!( + "✓ Registered the provisioner {name} ({kind}) of {}{}", + host.name, + match offer.fingerprint.is_empty() { + true => String::new(), + false => format!( + ", which pins the server's certificate {}", + offer.fingerprint + ), + } + )), + } + kept.push(name); + } + } + for (name, entry) in provisioner::entries(paths)? { + if entry.node != host.name || kept.contains(&name) { + continue; + } + match provisioner::remove_entry(paths, store, &name) { + Ok(()) => said.push(format!( + "✓ Forgot the provisioner {name}: no feature of {} offers it any more", + host.name + )), + Err(error) => said.push(format!( + "! {} no longer offers {name}, which is kept: {error}", + host.name + )), + } + } + Ok(said) +} + +/// Make the provisioner `name`, registered by hand, the node `node`'s, on +/// the word of the person who knows it reaches the node's server: the next +/// apply of the node brings its settings and pinned certificate up to date. +pub fn adopt(paths: &Paths, name: &str, node: &str) -> Result { + let mut entry = provisioner::entry(paths, name)? + .ok_or_else(|| format!("No provisioner {name} is registered"))?; + if !entry.node.is_empty() { + return Err(format!("{name} is the node {}'s already", entry.node).into()); + } + let host = Host::all(paths) + .into_iter() + .find(|host| host.name == node) + .ok_or_else(|| format!("No host {node}"))?; + if crate::host_type::of(&host.config) != HostType::Node { + return Err(format!("{node} is not a node").into()); + } + let plugins = provision_plugins(paths); + let (plugin_id, id) = entry + .kind + .split_once('/') + .unwrap_or((entry.kind.as_str(), "")); + let offering = features(&host.config).into_iter().find(|wanted| { + wanted + .split_once('/') + .is_some_and(|(plugin, _)| plugin == plugin_id) + && feature(&plugins, wanted) + .is_some_and(|found| found.offers.iter().any(|offered| offered == id)) + }); + let Some(offering) = offering else { + return Err(format!( + "No feature of {node} offers {}: add the one that does to its features first", + entry.kind + ) + .into()); + }; + if let Some((other, _)) = provisioner::entries(paths)? + .into_iter() + .find(|(_, known)| known.node == node && known.kind == entry.kind) + { + return Err(format!( + "{node} has the provisioner {other} of the type {} already. Remove it first: berth provisioner run {other} remove", + entry.kind + ) + .into()); + } + entry.node = node.to_owned(); + entry.feature = offering; + provisioner::save_entry(paths, name, &entry)?; + Ok(format!( + "{name} is {node}'s now. The next apply of {node} brings its settings and the certificate it pins up to date: berth apply {node}" + )) +} + +/// The plugins whose provisioners berth runs here. +fn provision_plugins(paths: &Paths) -> Vec { + crate::provision::found(paths) +} + +/// Why applying `host` now must not go on: its profile no longer names the +/// feature that offers a provisioner of the node that still holds resources. +pub fn check_removal(paths: &Paths, host: &Host) -> Result<()> { + let wanted = features(&host.config); + let applied = crate::host_type::Capability::Plugins.allowed(crate::host_type::of(&host.config)); + for (name, entry) in provisioner::entries(paths)? { + if entry.node != host.name || (applied && wanted.contains(&entry.feature)) { + continue; + } + let held = holds(paths, &name)?; + if !held.is_empty() { + return Err(format!( + "{} offers the provisioner {name} through {}, which still holds {}. Remove {} first (berth resource ID remove), or keep {} among the features of {}", + host.name, + entry.feature, + held.join(", "), + if held.len() == 1 { "it" } else { "them" }, + entry.feature, + host.name + ) + .into()); + } + } + Ok(()) +} + +/// The resources a registered provisioner holds, by id. +fn holds(paths: &Paths, name: &str) -> Result> { + Ok(crate::resource::list(paths)? + .into_iter() + .filter(|kept| kept.record.provisioner == name) + .map(|kept| kept.record.id) + .collect()) +} + +/// Forget the provisioners of the node `name` before the host goes: refused +/// while one still holds resources, as these would have no provisioner. +pub fn forget(paths: &Paths, store: &Store, name: &str) -> Result<()> { + let ours: Vec = provisioner::entries(paths)? + .into_iter() + .filter(|(_, entry)| entry.node == name) + .map(|(provisioner, _)| provisioner) + .collect(); + for provisioner in &ours { + let held = holds(paths, provisioner)?; + if !held.is_empty() { + return Err(format!( + "{name} is the node of the provisioner {provisioner}, which still holds {}. Remove {} first: berth resource ID remove", + held.join(", "), + if held.len() == 1 { "it" } else { "them" } + ) + .into()); + } + } + for provisioner in ours { + provisioner::remove_entry(paths, store, &provisioner)?; + } + Ok(()) +} + +/// Give the provisioners of the node `old` its new name. +pub fn rename(paths: &Paths, old: &str, new: &str) -> Result<()> { + for (name, mut entry) in provisioner::entries(paths)? { + if entry.node == old { + entry.node = new.to_owned(); + provisioner::save_entry(paths, &name, &entry)?; + } + } + Ok(()) +} + +/// What the node `host` reported at the end of its last apply, asked of its +/// controller until it answers: the provisioners its features offer. +pub fn reported(paths: &Paths, host: &Host) -> Result> { + let password = host.kept_sudo_password(paths); + let link = crate::link::Link::new(host.link(paths, password.as_deref())?); + let runtime = crate::link::runtime()?; + let started = std::time::Instant::now(); + let patience = std::time::Duration::from_secs(60); + loop { + let asked = runtime.block_on(async { + tokio::time::timeout(std::time::Duration::from_secs(15), link.version()).await + }); + match asked { + Ok(Ok(version)) => { + return Ok(version + .applied + .map(|applied| applied.offers) + .unwrap_or_default()); + } + Ok(Err(error)) if started.elapsed() >= patience => { + return Err(error.to_string().into()); + } + Err(_) if started.elapsed() >= patience => return Err("no answer in time".into()), + _ => std::thread::sleep(std::time::Duration::from_secs(1)), + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::profiles::Place; + use std::fs; + + /// A berth with a source whose plugin has a node's feature that offers + /// its provisioner, a workstation's feature, and the node `big`. + struct World { + _home: tempfile::TempDir, + paths: Paths, + store: Store, + } + + impl World { + fn new() -> World { + let home = tempfile::tempdir().unwrap(); + let paths = Paths { + config_dir: home.path().join("config"), + data_dir: home.path().join("data"), + }; + paths.prepare().unwrap(); + let source = home.path().join("source"); + fs::create_dir_all(source.join("provision/bin")).unwrap(); + fs::write(source.join(crate::paths::layout::BUILDER), "").unwrap(); + paths.set("source", source.to_str().unwrap()).unwrap(); + let plugin = source.join("plugins/fake"); + fs::create_dir_all(plugin.join("bin")).unwrap(); + fs::write(plugin.join("bin/provision"), "#!/bin/sh\n").unwrap(); + fs::write( + plugin.join("package.json"), + r#"{"name": "berth-plugin-fake", "berth": { + "features": [ + {"id": "engine", "types": ["node"], "offers": ["vm"], + "trusts": {"certificate": "fake_client_cert", "key": "fake_client_key"}}, + {"id": "desk"}], + "provisioners": [ + {"id": "vm", "label": "Fake VM", "command": "bin/provision", "protocol": 2, + "secrets": [{"name": "server_cert", "required": false}], + "arguments": [{"name": "server", "required": true}], + "kinds": [{"id": "machine", "setup": "apply"}]}]}}"#, + ) + .unwrap(); + let store = Store::own(crate::store::shared(&paths).unwrap()); + World { + _home: home, + paths, + store, + } + } + + fn host(&self, features: &[&str]) -> Host { + let config = json!({"type": "node", + "access": {"address": "192.0.2.20", "user": "root"}, + "ansible_vars": {"devbox_plugin_features": features}}); + let place = Place::own("big").unwrap(); + place.save(&self.paths, &config).unwrap(); + Host { + name: "big".into(), + place, + config, + } + } + + fn plugins(&self) -> Vec { + crate::provision::found(&self.paths) + } + + fn holding(&self, provisioner: &str) { + crate::resource::Kept { + scope: crate::store::OWN.into(), + record: crate::resource::Record { + id: "r1".into(), + provisioner: provisioner.into(), + kind: "machine".into(), + name: "studio-2".into(), + ..Default::default() + }, + } + .save(&self.paths) + .unwrap(); + } + } + + fn offer() -> Offer { + Offer { + feature: "fake/engine".into(), + provisioners: vec!["vm".into(), "gone".into()], + settings: [("server".into(), "https://{address}:8444".into())].into(), + secrets: [("server_cert".into(), "CERT".into())].into(), + fingerprint: "ab12".into(), + ..Offer::default() + } + } + + #[test] + fn a_node_registers_what_its_features_offer_once() { + let world = World::new(); + let host = world.host(&["fake/engine"]); + let said = register(&world.paths, &world.store, &host, &[offer()]).unwrap(); + assert_eq!(said.len(), 2, "{said:?}"); + assert!(said[0].starts_with("✓ Registered the provisioner big-fake-vm (fake/vm) of big, which pins the server's certificate ab12")); + assert!( + said[1] == "! big offers fake/gone, which fake/engine does not offer; it is left out", + "{said:?}" + ); + let entry = provisioner::entry(&world.paths, "big-fake-vm") + .unwrap() + .unwrap(); + assert_eq!( + (entry.node.as_str(), entry.feature.as_str()), + ("big", "fake/engine") + ); + assert_eq!(entry.settings["server"], "https://192.0.2.20:8444"); + assert_eq!( + world + .store + .secret("provisioner/big-fake-vm/server_cert") + .unwrap() + .as_deref(), + Some("CERT") + ); + let info = provisioner::info(&world.paths, &world.store, "big-fake-vm").unwrap(); + assert_eq!(info.node, "big"); + // The same report again changes nothing and says nothing. + let again = register(&world.paths, &world.store, &host, &[offer()]).unwrap(); + assert_eq!(again.len(), 1, "{again:?}"); + assert_eq!(provisioner::entries(&world.paths).unwrap().len(), 1); + // A new certificate is pinned in its place. + let mut renewed = offer(); + renewed.secrets.insert("server_cert".into(), "NEW".into()); + register(&world.paths, &world.store, &host, &[renewed]).unwrap(); + assert_eq!( + world + .store + .secret("provisioner/big-fake-vm/server_cert") + .unwrap() + .as_deref(), + Some("NEW") + ); + } + + /// The lookup of a server's container on the node, where `node` and + /// `incus-host` both name the node's address. + fn lookup(name: &str) -> Vec { + match name { + "node" | "incus-host" => vec!["192.0.2.20".parse().unwrap()], + "other.example" => vec!["192.0.2.30".parse().unwrap()], + other => other.parse().into_iter().collect(), + } + } + + fn hand(world: &World, name: &str, server: &str) { + let entry = Entry { + kind: "fake/vm".into(), + settings: [("server".into(), server.into())].into(), + ..Entry::default() + }; + provisioner::save_entry(&world.paths, name, &entry).unwrap(); + } + + fn names(world: &World) -> Vec { + provisioner::entries(&world.paths) + .unwrap() + .into_iter() + .map(|(name, _)| name) + .collect() + } + + #[test] + fn one_registered_by_hand_for_the_node_is_taken_over() { + let world = World::new(); + let host = world.host(&["fake/engine"]); + // As the recipe registered it: its own name and setting, the shared certificate. + hand(&world, "incus-vm", "https://incus-host:8444"); + world.store.set_secret("server_cert", Some("CERT")).unwrap(); + hand(&world, "elsewhere", "https://192.0.2.99:8444"); + let said = register_by(&world.paths, &world.store, &host, &[offer()], &lookup).unwrap(); + assert!( + said.iter() + .any(|line| line.contains("incus-vm reaches the same server, and is big's now")), + "{said:?}" + ); + assert_eq!(names(&world), ["elsewhere", "incus-vm"]); + let taken = provisioner::entry(&world.paths, "incus-vm") + .unwrap() + .unwrap(); + assert_eq!( + (taken.node.as_str(), taken.feature.as_str()), + ("big", "fake/engine") + ); + // Its setting reaches the server already, and stays. + assert_eq!(taken.settings["server"], "https://incus-host:8444"); + assert!( + provisioner::entry(&world.paths, "elsewhere") + .unwrap() + .unwrap() + .node + .is_empty() + ); + + // The address berth reaches the node at is enough, with or without a certificate. + let world = World::new(); + let host = world.host(&["fake/engine"]); + hand(&world, "by-address", "https://192.0.2.20:8444/"); + let mut unpinned = offer(); + unpinned.secrets.clear(); + register_by(&world.paths, &world.store, &host, &[unpinned], &lookup).unwrap(); + assert_eq!(names(&world), ["by-address"]); + assert_eq!( + provisioner::entry(&world.paths, "by-address") + .unwrap() + .unwrap() + .node, + "big" + ); + + // Another port of the same machine is another server. + let world = World::new(); + let host = world.host(&["fake/engine"]); + hand(&world, "other-port", "https://192.0.2.20:9000"); + register_by(&world.paths, &world.store, &host, &[offer()], &lookup).unwrap(); + assert_eq!(names(&world), ["big-fake-vm", "other-port"]); + } + + #[test] + fn one_the_node_alone_says_is_the_same_waits_for_its_owner() { + let world = World::new(); + // Reached by its tailnet name, which berth does not find at incus-host. + let mut host = world.host(&["fake/engine"]); + host.config["access"]["address"] = json!("big.example.org"); + hand(&world, "incus-vm", "https://incus-host:8444"); + world.store.set_secret("server_cert", Some("CERT")).unwrap(); + let mut aliased = offer(); + aliased + .aliases + .insert("server".into(), vec!["https://incus-host:8444".into()]); + let said = register_by( + &world.paths, + &world.store, + &host, + &[aliased.clone()], + &lookup, + ) + .unwrap(); + assert!( + said.iter().any(|line| line + == "! big offers fake/vm, and incus-vm may reach the same server (it pins the certificate the node reports), which berth cannot check; both are left as they are. To make incus-vm big's: berth provisioner run incus-vm adopt node=big"), + "{said:?}" + ); + // Nothing changed, and there is no second one beside it. + assert_eq!(names(&world), ["incus-vm"]); + let left = provisioner::entry(&world.paths, "incus-vm") + .unwrap() + .unwrap(); + assert!(left.node.is_empty()); + assert_eq!( + world + .store + .secret("provisioner/incus-vm/server_cert") + .unwrap(), + None + ); + + // The owner's word makes it the node's, and the next apply brings it up to date. + let message = adopt(&world.paths, "incus-vm", "big").unwrap(); + assert!(message.starts_with("incus-vm is big's now."), "{message}"); + let adopted = provisioner::entry(&world.paths, "incus-vm") + .unwrap() + .unwrap(); + assert_eq!( + (adopted.node.as_str(), adopted.feature.as_str()), + ("big", "fake/engine") + ); + let said = register_by(&world.paths, &world.store, &host, &[aliased], &lookup).unwrap(); + assert!( + said.iter() + .any(|line| line == "✓ Updated the provisioner incus-vm of big"), + "{said:?}" + ); + let updated = provisioner::entry(&world.paths, "incus-vm") + .unwrap() + .unwrap(); + assert_eq!(updated.settings["server"], "https://big.example.org:8444"); + assert_eq!( + world + .store + .secret("provisioner/incus-vm/server_cert") + .unwrap() + .as_deref(), + Some("CERT") + ); + + // A local engine is the node's only on the owner's word, too. + let world = World::new(); + let host = world.host(&["fake/engine"]); + provisioner::save_entry( + &world.paths, + "local", + &Entry { + kind: "fake/vm".into(), + ..Entry::default() + }, + ) + .unwrap(); + let mut local = offer(); + local.secrets.clear(); + local.aliases.insert("server".into(), vec![String::new()]); + let said = register_by(&world.paths, &world.store, &host, &[local], &lookup).unwrap(); + assert!( + said.iter().any(|line| line.contains( + "local may reach the same server (it names a server the node gives as its own)" + )), + "{said:?}" + ); + assert_eq!(names(&world), ["local"]); + assert!( + provisioner::entry(&world.paths, "local") + .unwrap() + .unwrap() + .node + .is_empty() + ); + } + + #[test] + fn a_node_does_not_reach_a_provisioner_of_another_server() { + let world = World::new(); + let host = world.host(&["fake/engine"]); + hand(&world, "other", "https://other.example:8444"); + world + .store + .set_secret("provisioner/other/server_cert", Some("REAL-CERT")) + .unwrap(); + let untouched = |world: &World| { + let other = provisioner::entry(&world.paths, "other").unwrap().unwrap(); + assert!(other.node.is_empty()); + assert_eq!(other.settings["server"], "https://other.example:8444"); + assert_eq!( + world + .store + .secret("provisioner/other/server_cert") + .unwrap() + .as_deref(), + Some("REAL-CERT") + ); + }; + // It names the other server among its aliases, with a certificate of its own. + let mut hostile = offer(); + hostile + .aliases + .insert("server".into(), vec!["https://other.example:8444".into()]); + hostile.secrets = [("server_cert".into(), "FORGED-CERT".into())].into(); + let said = register_by( + &world.paths, + &world.store, + &host, + &[hostile.clone()], + &lookup, + ) + .unwrap(); + assert!( + said.iter().any(|line| line + .starts_with("! big offers fake/vm, and other may reach the same server")), + "{said:?}" + ); + untouched(&world); + // It reports the other server's certificate, which is public. + hostile.secrets = [("server_cert".into(), "REAL-CERT".into())].into(); + register_by( + &world.paths, + &world.store, + &host, + &[hostile.clone()], + &lookup, + ) + .unwrap(); + untouched(&world); + // It names the other server as its own. + hostile.aliases.clear(); + hostile.settings = [("server".into(), "https://other.example:8444".into())].into(); + let said = register_by(&world.paths, &world.store, &host, &[hostile], &lookup).unwrap(); + assert!( + said.iter().any(|line| line + == "! big offers fake/vm with server=https://other.example:8444, which is not where berth reaches big; it is left out"), + "{said:?}" + ); + untouched(&world); + // Dropping the feature removes only what is the node's. + let said = register_by(&world.paths, &world.store, &host, &[], &lookup).unwrap(); + assert!(said.is_empty(), "{said:?}"); + untouched(&world); + assert_eq!(names(&world), ["other"]); + + // One of another node is that node's. + let world = World::new(); + let host = world.host(&["fake/engine"]); + provisioner::save_entry( + &world.paths, + "dock-fake-vm", + &Entry { + kind: "fake/vm".into(), + settings: [("server".into(), "https://192.0.2.40:8444".into())].into(), + node: "dock".into(), + feature: "fake/engine".into(), + }, + ) + .unwrap(); + let mut claims = offer(); + claims + .aliases + .insert("server".into(), vec!["https://192.0.2.40:8444".into()]); + let said = register_by(&world.paths, &world.store, &host, &[claims], &lookup).unwrap(); + assert!( + said.iter().any(|line| line + == "! big offers fake/vm, and says it is the server of dock-fake-vm, which is dock's; both are left as they are"), + "{said:?}" + ); + assert_eq!(names(&world), ["dock-fake-vm"]); + let theirs = provisioner::entry(&world.paths, "dock-fake-vm") + .unwrap() + .unwrap(); + assert_eq!( + (theirs.node.as_str(), theirs.settings["server"].as_str()), + ("dock", "https://192.0.2.40:8444") + ); + } + + #[test] + fn an_offer_holds_what_the_features_manifest_offers() { + let world = World::new(); + let host = world.host(&["fake/engine"]); + // A feature the profile does not name. + let mut unnamed = offer(); + unnamed.feature = "fake/desk".into(); + let said = register_by(&world.paths, &world.store, &host, &[unnamed], &lookup).unwrap(); + assert_eq!( + said, + [ + "! big reports what fake/desk offers, which is not among its features; it is left out" + ] + ); + // Settings that are not the provisioner's own arguments. + let mut extra = offer(); + extra.provisioners = vec!["vm".into()]; + extra.settings.insert("command".into(), "/bin/sh".into()); + extra.secrets.insert("other_secret".into(), "x".into()); + register_by(&world.paths, &world.store, &host, &[extra], &lookup).unwrap(); + let entry = provisioner::entry(&world.paths, "big-fake-vm") + .unwrap() + .unwrap(); + assert_eq!(entry.settings.keys().collect::>(), ["server"]); + assert_eq!( + world + .store + .secret("provisioner/big-fake-vm/other_secret") + .unwrap(), + None + ); + assert_eq!(names(&world), ["big-fake-vm"]); + } + + #[test] + fn a_provisioner_is_made_a_nodes_only_where_that_fits() { + let world = World::new(); + world.host(&["fake/engine"]); + hand(&world, "incus-vm", "https://incus.example.net:8444"); + assert_eq!( + adopt(&world.paths, "nothing", "big") + .unwrap_err() + .to_string(), + "No provisioner nothing is registered" + ); + assert_eq!( + adopt(&world.paths, "incus-vm", "dock") + .unwrap_err() + .to_string(), + "No host dock" + ); + world.host(&[]); + assert!( + adopt(&world.paths, "incus-vm", "big") + .unwrap_err() + .to_string() + .starts_with("No feature of big offers fake/vm") + ); + let host = world.host(&["fake/engine"]); + register_by(&world.paths, &world.store, &host, &[offer()], &lookup).unwrap(); + assert_eq!( + adopt(&world.paths, "incus-vm", "big") + .unwrap_err() + .to_string(), + "big has the provisioner big-fake-vm of the type fake/vm already. Remove it first: berth provisioner run big-fake-vm remove" + ); + assert_eq!( + adopt(&world.paths, "big-fake-vm", "big") + .unwrap_err() + .to_string(), + "big-fake-vm is the node big's already" + ); + let mut station = world.host(&["fake/engine"]); + station.config["type"] = json!("workstation"); + station.place.save(&world.paths, &station.config).unwrap(); + assert_eq!( + adopt(&world.paths, "incus-vm", "big") + .unwrap_err() + .to_string(), + "big is not a node" + ); + } + + #[test] + fn one_registered_by_hand_has_the_action_that_makes_it_a_nodes() { + let world = World::new(); + world.host(&["fake/engine"]); + hand(&world, "incus-vm", "https://incus.example.net:8444"); + let listed = provisioner::actions(&world.paths, &world.store, "incus-vm").unwrap(); + let found = listed + .iter() + .find(|action| action.name == provisioner::ADOPT) + .unwrap(); + assert_eq!(found.role, berth_api::ops::Role::Admin); + let given = [("node".to_owned(), "big".to_owned())].into(); + let acting = provisioner::prepare( + &world.paths, + &world.store, + "incus-vm", + provisioner::ADOPT, + &given, + ) + .unwrap(); + let answered = acting + .run( + &world.paths, + &crate::provision::Options::default(), + &mut std::io::sink(), + ) + .unwrap(); + assert!(answered.message.starts_with("incus-vm is big's now")); + // A node's provisioner has no such action. + let listed = provisioner::actions(&world.paths, &world.store, "incus-vm").unwrap(); + assert!( + listed + .iter() + .all(|action| action.name != provisioner::ADOPT) + ); + } + + #[test] + fn what_a_node_no_longer_offers_goes_unless_it_holds_resources() { + let world = World::new(); + let host = world.host(&["fake/engine"]); + register(&world.paths, &world.store, &host, &[offer()]).unwrap(); + world.holding("big-fake-vm"); + // Applying without the feature is refused while the machine is there. + let without = world.host(&[]); + let refused = check_removal(&world.paths, &without) + .unwrap_err() + .to_string(); + assert!( + refused.contains( + "big offers the provisioner big-fake-vm through fake/engine, which still holds r1" + ), + "{refused}" + ); + assert!(check_removal(&world.paths, &host).is_ok()); + // Made a managed host, it would no longer be applied with it either. + let mut managed = world.host(&["fake/engine"]); + managed.config["type"] = json!("managed"); + assert!(check_removal(&world.paths, &managed).is_err()); + // A report without it keeps it, and says why. + let said = register(&world.paths, &world.store, &host, &[]).unwrap(); + assert!( + said[0].starts_with("! big no longer offers big-fake-vm, which is kept"), + "{said:?}" + ); + assert!( + provisioner::entry(&world.paths, "big-fake-vm") + .unwrap() + .is_some() + ); + // The node is not forgotten while it holds them, nor renamed apart from them. + let refused = crate::profiles::remove(&world.paths, &host.place) + .unwrap_err() + .to_string(); + assert!( + refused + .contains("big is the node of the provisioner big-fake-vm, which still holds r1"), + "{refused}" + ); + let renamed = crate::profiles::rename(&world.paths, &host.place, "huge", false).unwrap(); + assert_eq!( + provisioner::entry(&world.paths, "big-fake-vm") + .unwrap() + .unwrap() + .node, + "huge" + ); + crate::profiles::rename(&world.paths, &renamed, "big", false).unwrap(); + // Once the machine is gone, so is the provisioner. + crate::store::shared(&world.paths) + .unwrap() + .forget(crate::store::kind::RESOURCE, crate::store::OWN, "r1") + .unwrap(); + let said = register(&world.paths, &world.store, &host, &[]).unwrap(); + assert!( + said[0].starts_with("✓ Forgot the provisioner big-fake-vm"), + "{said:?}" + ); + assert!( + provisioner::entry(&world.paths, "big-fake-vm") + .unwrap() + .is_none() + ); + assert_eq!( + world + .store + .secret("provisioner/big-fake-vm/server_cert") + .unwrap(), + None + ); + // Forgetting the node forgets its provisioners. + register(&world.paths, &world.store, &host, &[offer()]).unwrap(); + crate::profiles::remove(&world.paths, &host.place).unwrap(); + assert!(provisioner::entries(&world.paths).unwrap().is_empty()); + } + + #[test] + fn a_feature_is_for_the_types_its_manifest_names() { + let world = World::new(); + let plugins = world.plugins(); + let node = world.host(&["fake/engine", "fake/desk"]); + let refused = check_features(&plugins, "big", &node.config) + .unwrap_err() + .to_string(); + assert_eq!( + refused, + "big is a node, and fake/desk is for a workstation: take it out of its features" + ); + let workstation = json!({"builder": {}, "ansible_vars": {"devbox_plugin_features": ["fake/engine", "fake/desk"]}}); + let refused = check_features(&plugins, "studio", &workstation) + .unwrap_err() + .to_string(); + assert_eq!( + refused, + "studio is a workstation, and fake/engine is for a node: take it out of its features" + ); + assert!( + check_features( + &plugins, + "big", + &world.host(&["fake/engine", "other/x"]).config + ) + .is_ok() + ); + } + + #[test] + fn berth_makes_the_client_certificate_a_feature_trusts_once() { + let world = World::new(); + let plugins = world.plugins(); + let config = world.host(&["fake/engine"]).config; + let first = client_certificates(&plugins, &world.store, &config).unwrap(); + let made = &first["fake/engine"]; + let print = made["fingerprint"].as_str().unwrap(); + assert_eq!(made["name"], format!("berth-{}", &print[..12])); + assert!(made.get("key").is_none()); + assert!( + world + .store + .secret("fake_client_key") + .unwrap() + .unwrap() + .contains("PRIVATE KEY") + ); + let again = client_certificates(&plugins, &world.store, &config).unwrap(); + assert_eq!(again, first); + assert_eq!( + client_certificates(&plugins, &world.store, &world.host(&[]).config).unwrap(), + json!({}) + ); + } + + #[test] + fn a_fingerprint_is_of_the_certificate_itself() { + let (certificate, key) = make_client_certificate().unwrap(); + assert!(certificate.starts_with("-----BEGIN CERTIFICATE-----")); + assert!(key.contains("PRIVATE KEY")); + let print = fingerprint(&certificate).unwrap(); + assert_eq!(print.len(), 64); + assert_eq!(fingerprint(&format!("\n{certificate}\n")).unwrap(), print); + assert!(fingerprint("not a certificate").is_none()); + } + + #[test] + fn offered_values_name_where_berth_reaches_the_node() { + let host = |access: Value| Host { + name: "big".into(), + place: Place::own("big").unwrap(), + config: json!({"type": "node", "access": access}), + }; + let root = host(json!({"address": "192.0.2.20", "user": "root"})); + assert_eq!( + fill("https://{address}:8444", &root), + "https://192.0.2.20:8444" + ); + assert_eq!( + fill("ssh://{user}@{address}:{port}", &root), + "ssh://root@192.0.2.20:22" + ); + let six = host(json!({"address": "2001:db8::20", "user": "ops", "port": "2222"})); + assert_eq!( + fill("https://{address}:8444", &six), + "https://[2001:db8::20]:8444" + ); + assert_eq!( + fill("ssh://{user}@{address}:{port}", &six), + "ssh://ops@[2001:db8::20]:2222" + ); + assert_eq!(server_of("https://Big.example:8444/x"), "big.example:8444"); + assert_eq!(server_of("ssh://root@192.0.2.20"), "192.0.2.20"); + assert_eq!(free_name(&[], "big", "incus/vm"), "big-incus-vm"); + let taken = vec![("big-incus-vm".to_owned(), Entry::default())]; + assert_eq!(free_name(&taken, "big", "incus/vm"), "big-incus-vm-2"); + assert_eq!( + free_name(&[], "7.node", "docker/container"), + "n-7-node-docker-container" + ); + } +} diff --git a/crates/berth-core/src/paths.rs b/crates/berth-core/src/paths.rs index 406f0be..4783da8 100644 --- a/crates/berth-core/src/paths.rs +++ b/crates/berth-core/src/paths.rs @@ -55,7 +55,7 @@ pub mod layout { pub(crate) const OLD_BUILDER: &str = "bin/devbox.py"; } -/// Settings that `berth config KEY VALUE` can change. Those [`is_local`] +/// Settings that `berth config set KEY VALUE` can change. Those [`is_local`] /// names are in `config.json`, the others in the store. pub const SETTINGS: &[(&str, &str)] = &[ ( @@ -113,7 +113,7 @@ pub const SETTINGS: &[(&str, &str)] = &[ ), ( "tailnet_tags", - "tags new hosts get, comma separated, such as tag:berth (for Tailscale also DEVBOX_TAILSCALE_TAGS)", + "tags new hosts get, comma separated, such as tag:berth; an OAuth client of Tailscale needs them", ), ( "store", @@ -121,7 +121,7 @@ pub const SETTINGS: &[(&str, &str)] = &[ ), ( "store_key", - "the key file that seals the store's values, made by berth store encrypt (default: store.key in the configuration directory)", + "the key file that seals the store's values, made by berth server store encrypt (default: store.key in the configuration directory)", ), ( "store_ca", @@ -469,7 +469,7 @@ fn is_source(dir: &Path) -> bool { fn older_source(dir: &Path) -> Result<()> { if dir.join(layout::OLD_BUILDER).is_file() && !is_source(dir) { return Err(format!( - "{} is a provisioning source of an older berth, with {} where this berth looks for {}. Update that checkout, or stop using it with: berth config source \"\" (and unset BERTH_SOURCE)", + "{} is a provisioning source of an older berth, with {} where this berth looks for {}. Update that checkout, or stop using it with: berth config unset source (and unset BERTH_SOURCE)", dir.display(), layout::OLD_BUILDER, layout::BUILDER @@ -618,11 +618,6 @@ mod tests { paths.setting_with("server", None).as_deref(), Some("http://192.0.2.7:7777") ); - assert!(crate::server::Server::configured(&paths).is_some()); - assert_eq!( - crate::profiles::scope(&paths), - "served:http://192.0.2.7:7777" - ); assert_eq!(paths.setting_with("default_host", None), None); assert!(paths.settings().is_err()); assert!(paths.set("default_host", "studio").is_err()); diff --git a/crates/berth-core/src/plugin_state.rs b/crates/berth-core/src/plugin_state.rs index cd2f770..49c0863 100644 --- a/crates/berth-core/src/plugin_state.rs +++ b/crates/berth-core/src/plugin_state.rs @@ -399,7 +399,7 @@ pub fn carry_out(own: &Path, plan: &Plan) -> Result> { .map(|(name, _)| name.as_str()) .collect(); notices.push(format!( - "berth: {} came from a file or folder of the machine they were added on, and are not installed here; add them again with berth plugins add", + "berth: {} came from a file or folder of the machine they were added on, and are not installed here; add them again with berth plugin add", names.join(", ") )); } diff --git a/crates/berth-core/src/plugins.rs b/crates/berth-core/src/plugins.rs index 1014257..e2a884f 100644 --- a/crates/berth-core/src/plugins.rs +++ b/crates/berth-core/src/plugins.rs @@ -1,13 +1,13 @@ //! Plugins: npm-format packages whose `package.json` has a `berth` section. -//! Each contributes features to the groups of the Features page, and tasks to -//! the overview of a host. The role does the installing from the same manifest. +//! Each contributes features to the groups of the Features page, and actions +//! to the overview of a host. The role does the installing from the same manifest. //! A plugin can also bring provisioners, programs that create the machine a //! profile describes; berth runs those itself (see [`crate::provision`]). -use crate::{Result, paths::Paths, server::Server}; +use crate::{Result, paths::Paths}; use berth_api::{ - AppliedPlugin, PluginFeature, PluginInfo, PluginList as Listing, PluginOrigin, - PluginReport as Report, PluginService, PluginTask, PluginUpdate, PluginUpdates, + AppliedPlugin, PluginAction as ListedAction, PluginFeature, PluginInfo, PluginList as Listing, + PluginOrigin, PluginReport as Report, PluginService, PluginUpdate, PluginUpdates, ProvisionerInfo, }; use serde_json::{Value, json}; @@ -19,7 +19,7 @@ pub struct PluginAction { pub label: String, pub help: &'static str, /// Shell command the host runs. berth does not run it from here: the - /// role writes it into the host's task list. + /// role writes it into the host's list of actions. pub run: String, /// `agent`, `operator` or `root`. pub user: String, @@ -27,7 +27,7 @@ pub struct PluginAction { pub terminal: bool, /// The group of the overview it is listed in; the plugin's own when empty. pub group: String, - /// Tasks of berth it is part of, such as `status`. It runs after them and + /// Actions of berth it is part of, such as `status`. It runs after them and /// has no row of its own. pub extends: Vec, /// The agent may run it with `box run`. @@ -53,6 +53,74 @@ pub struct Feature { pub channels: Vec, } +/// What a feature says of the hosts it is for and what it gives them, +/// beside what [`Feature`] holds. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Hosting { + pub feature: String, + /// The types of host it is for: a workstation unless the manifest names + /// others, such as `node` for what a server needs. + pub types: Vec, + /// The ids of the plugin's provisioners it may offer once it is applied. + pub offers: Vec, + /// The berth secrets of a client certificate and its key that the host + /// is to trust, which berth makes when it has none. + pub trusts: Option<(String, String)>, +} + +/// What each feature of a plugin says of the hosts it is for, read from its +/// manifest. +pub fn hosting(plugin: &Plugin) -> Vec { + let Some(data) = fs::read(plugin.dir.join("package.json")) + .ok() + .and_then(|data| serde_json::from_slice::(&data).ok()) + else { + return vec![]; + }; + data["berth"]["features"] + .as_array() + .into_iter() + .flatten() + .filter_map(|feature| { + let id = feature["id"].as_str().filter(|id| valid_name(id))?; + let strings = |value: &Value| -> Vec { + value + .as_array() + .into_iter() + .flatten() + .filter_map(Value::as_str) + .map(str::to_owned) + .collect() + }; + Some(Hosting { + feature: id.to_owned(), + types: match feature["types"].as_array() { + Some(_) => strings(&feature["types"]) + .iter() + .filter_map(|name| berth_api::HostType::parse(name)) + .collect(), + None => vec![berth_api::HostType::Workstation], + }, + offers: strings(&feature["offers"]) + .into_iter() + .filter(|id| valid_name(id)) + .collect(), + trusts: match ( + feature["trusts"]["certificate"].as_str(), + feature["trusts"]["key"].as_str(), + ) { + (Some(certificate), Some(key)) + if valid_name(certificate) && valid_name(key) => + { + Some((certificate.to_owned(), key.to_owned())) + } + _ => None, + }, + }) + }) + .collect() +} + /// How a machine a provisioner created becomes the host. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum Setup { @@ -117,7 +185,7 @@ pub struct Provisioner { pub const OPTIONAL_VERBS: &[&str] = &["start", "stop", "list"]; /// The verbs a kind may answer besides `create`, `status` and `destroy`. -pub const KIND_VERBS: &[&str] = &["start", "stop", "change", "logs"]; +pub const KIND_VERBS: &[&str] = &["start", "stop", "change", "logs", "exec"]; /// The kind of the machines a provisioner makes for hosts. pub const MACHINE: &str = "machine"; @@ -142,6 +210,7 @@ impl Provisioner { arguments: self.arguments.clone(), own: self.own.clone(), settings: Default::default(), + node: String::new(), kinds: self.kinds.clone(), actions: self.actions.clone(), problem: self.refused.clone().unwrap_or_default(), @@ -272,10 +341,10 @@ fn read(dir: &Path) -> Option { terminal: action["terminal"].as_bool().unwrap_or(false), group: text(action, "group"), extends: match &action["extends"] { - Value::String(task) => vec![task.clone()], - tasks => list(tasks) + Value::String(name) => vec![name.clone()], + names => list(names) .iter() - .filter_map(|task| task.as_str().map(str::to_owned)) + .filter_map(|name| name.as_str().map(str::to_owned)) .collect(), }, agent: action["agent"].as_bool().unwrap_or(false), @@ -286,7 +355,7 @@ fn read(dir: &Path) -> Option { }) .filter(|argument| valid_name(&argument.name)) // Only a provisioner's arguments size a machine or are a - // list of values; a task's offers them with `choices`. + // list of values; another action offers them with `choices`. .map(|argument| berth_api::Argument { kind: String::new(), values: vec![], @@ -570,10 +639,10 @@ impl Plugin { label: feature.label.clone(), group: feature.group.clone(), help: feature.help.to_owned(), - tasks: feature + actions: feature .actions .iter() - .map(|action| PluginTask { + .map(|action| ListedAction { name: action.name.clone(), label: action.label.clone(), user: action.user.clone(), @@ -615,115 +684,6 @@ impl Plugin { } } -/// The listing of `berth plugins`: each plugin with what its features add. -pub fn describe(plugins: &[PluginInfo]) -> String { - let mut text = String::new(); - for plugin in plugins { - text += &format!( - "{} {} {} {}\n", - plugin.id, plugin.package, plugin.version, plugin.dir - ); - for feature in &plugin.features { - text += &format!( - " {}/{} {} ({})\n", - plugin.id, feature.id, feature.label, feature.group - ); - for task in &feature.tasks { - let mut notes = vec![format!("as {}", task.user)]; - if task.agent { - notes.push("the agent may run it".into()); - } - if !task.extends.is_empty() { - notes.push(format!("part of {}", task.extends.join(", "))); - } - for argument in &task.arguments { - notes.push(format!("asks {argument}")); - } - text += &format!( - " task {}:{} {} ({})\n", - plugin.id, - task.name, - task.label, - notes.join("; ") - ); - } - for service in &feature.services { - text += &format!( - " service {}/{} as {}: {}\n", - plugin.id, - service.name, - service.user, - service.methods.join(", ") - ); - } - for channel in &feature.channels { - text += &format!(" channel {}/{channel}\n", plugin.id); - } - } - for provisioner in &plugin.provisioners { - text += &format!( - " provisioner {} {} ({})\n", - provisioner.id, - provisioner.label, - provisioner_notes(provisioner).join("; ") - ); - } - } - text -} - -/// What a provisioner does and asks for, in a few words each. -pub fn provisioner_notes(provisioner: &ProvisionerInfo) -> Vec { - if !provisioner.problem.is_empty() && provisioner.kinds.is_empty() { - return vec![provisioner.problem.clone()]; - } - let makes = |kind: &berth_api::ProvisionerKind| match (kind.id.as_str(), kind.setup.as_str()) { - (MACHINE, "seed") => "machines that set themselves up from the cloud seed".to_owned(), - (MACHINE, _) => "machines berth applies the profile to over SSH".to_owned(), - _ => kind.label.clone(), - }; - let kinds: Vec = provisioner.kinds.iter().map(makes).collect(); - let mut notes = vec![]; - if !kinds.is_empty() { - notes.push(format!("makes {}", kinds.join(" and "))); - } - if !provisioner.actions.is_empty() { - let names: Vec<&str> = provisioner - .actions - .iter() - .map(|action| action.name.as_str()) - .collect(); - notes.push(format!("actions {}", names.join(", "))); - } - if !provisioner.verbs.is_empty() { - notes.push(format!("also {}", provisioner.verbs.join(", "))); - } - for argument in &provisioner.arguments { - let kept = provisioner.settings.get(&argument.name); - let mut note = match (argument.required, argument.default.as_str(), kept) { - (_, _, Some(value)) => format!("asks {} (set to {value})", argument.name), - (true, _, None) => format!("asks {}", argument.name), - (false, "", None) => format!("may ask {}", argument.name), - (false, default, None) => format!("asks {} (default {default})", argument.name), - }; - if !argument.values.is_empty() { - note += &format!(": {}", argument.values.join(" or ")); - } - notes.push(note); - } - for secret in &provisioner.secrets { - let missing = provisioner.missing_secrets.contains(secret); - notes.push(match missing { - true => format!("secret {secret}, not set"), - false => format!("secret {secret}"), - }); - } - if !provisioner.problem.is_empty() { - notes.push(format!("not usable: {}", provisioner.problem)); - } - notes -} - /// SHA-256 over a plugin's files, their paths and contents, so a plugin /// that changed without a new version is told apart. What npm installed /// beside it (`node_modules`) and a `.git` folder are not part of it. @@ -1013,7 +973,7 @@ pub fn update(paths: &Paths, plugins: &[Plugin], name: Option<&str>) -> Result { @@ -1111,13 +1071,11 @@ pub fn local_spec(spec: &str) -> bool { .any(|end| spec.ends_with(end)) } -/// Where plugins are managed: this machine's plugin directory, or that of -/// the berth server berth is set to use. +/// Where plugins are managed: this machine's plugin directory. pub struct Manager { paths: Paths, source: Option, own: PathBuf, - server: Option, } impl Manager { @@ -1126,14 +1084,9 @@ impl Manager { paths: paths.clone(), source: paths.source().ok(), own: own_dir(&paths.config_dir), - server: Server::configured(paths), } } - pub fn server(&self) -> Option<&Server> { - self.server.as_ref() - } - fn found(&self) -> Vec { match &self.source { Some(source) => found(source, &self.own), @@ -1143,13 +1096,6 @@ impl Manager { /// The plugins there are, and where the user's own are kept. pub fn list(&self) -> Result { - if let Some(server) = &self.server { - return Ok(serde_json::from_value(server.call( - "plugins.list", - Value::Null, - None, - )?)?); - } Ok(Listing { plugins: self .found() @@ -1160,86 +1106,36 @@ impl Manager { }) } - /// Install a plugin. A server installs a package of a registry itself, - /// and is sent a .tgz or a folder of this machine as a .tgz. + /// Install a plugin: a package of a registry, or a file or folder of + /// this machine. pub fn add(&self, spec: &str) -> Result> { - let local = local_spec(spec); - let path = match local { - true => fs::canonicalize(crate::discovery::expand_path( - spec.strip_prefix("file:").unwrap_or(spec), - )) - .map_err(|error| format!("{spec}: {error}"))?, - false => PathBuf::new(), - }; - let Some(server) = &self.server else { - return install( - &self.paths, - match local { - true => path.to_str().unwrap_or(spec), - false => spec, - }, - ); - }; - if !local { - let report = server.call("plugins.add", Value::Null, Some(json!({"spec": spec})))?; - return Ok(serde_json::from_value::(report)?.lines); + if !local_spec(spec) { + return install(&self.paths, spec); } - let package = match path.is_dir() { - true => pack(&path)?, - false => fs::read(&path)?, - }; - if package.len() > berth_api::ops::PORTAL_BYTES { - return Err(format!("{spec} is larger than the 8 MiB a server is sent").into()); - } - let data = json!({"base64": berth_api::b64(&package)}); - let report = server.call("plugins.upload", Value::Null, Some(data))?; - Ok(serde_json::from_value::(report)?.lines) + let path = fs::canonicalize(crate::discovery::expand_path( + spec.strip_prefix("file:").unwrap_or(spec), + )) + .map_err(|error| format!("{spec}: {error}"))?; + install(&self.paths, path.to_str().unwrap_or(spec)) } /// Remove a plugin by its id or its npm name. pub fn remove(&self, name: &str) -> Result> { - let listing = self.list()?; - let plugin = listing - .plugins + let found = self.found(); + let plugin = found .iter() - .find(|plugin| plugin.id == name || plugin.package == name); - match (&self.server, plugin) { - (_, None) => Err(format!("No plugin {name}. See: berth plugins").into()), - (Some(server), Some(plugin)) => { - let report = server.call("plugins.remove", json!({"id": plugin.id}), None)?; - Ok(serde_json::from_value::(report)?.lines) - } - (None, Some(plugin)) => { - let found = self.found(); - let plugin = found - .iter() - .find(|known| known.id == plugin.id) - .ok_or("The plugin is gone")?; - remove(&self.paths, plugin) - } - } + .find(|plugin| plugin.id == name || plugin.package == name) + .ok_or_else(|| format!("No plugin {name}. See: berth plugin list"))?; + remove(&self.paths, plugin) } /// Which of the user's plugins have a newer version. pub fn check(&self) -> Result { - if let Some(server) = &self.server { - return Ok(serde_json::from_value(server.call( - "plugins.check", - Value::Null, - None, - )?)?); - } check(&self.own, &self.found()) } - /// Update one plugin, by its id or npm name, or all. A server says which - /// of its hosts are behind afterwards. + /// Update one plugin, by its id or npm name, or all. pub fn update(&self, name: Option<&str>) -> Result { - if let Some(server) = &self.server { - let data = json!({"id": name}); - let report = server.call("plugins.update", Value::Null, Some(data))?; - return Ok(serde_json::from_value(report)?); - } Ok(Report { lines: update(&self.paths, &self.found(), name)?, behind: vec![], @@ -1247,33 +1143,6 @@ impl Manager { } } -/// A folder as the .tgz `npm pack` makes of it, without running its scripts. -fn pack(folder: &Path) -> Result> { - let into = tempfile::tempdir()?; - let output = std::process::Command::new("npm") - .args(["pack", "--ignore-scripts", "--silent", "--pack-destination"]) - .arg(into.path()) - .arg("--") - .arg(folder) - .stdin(std::process::Stdio::null()) - .output() - .map_err(|error| format!("npm could not be started: {error}"))?; - if !output.status.success() { - return Err(format!( - "npm pack {}: {}", - folder.display(), - String::from_utf8_lossy(&output.stderr).trim() - ) - .into()); - } - let packed = fs::read_dir(into.path())? - .flatten() - .map(|entry| entry.path()) - .find(|path| path.extension().is_some_and(|end| end == "tgz")) - .ok_or("npm pack made no .tgz")?; - Ok(fs::read(packed)?) -} - #[cfg(test)] mod tests { use super::*; @@ -1514,11 +1383,13 @@ mod tests { let back: berth_api::ProvisionerInfo = serde_json::from_value(sent).unwrap(); assert!(back.arguments[1].is_resource() && !back.arguments[0].is_resource()); assert_eq!(info.provisioners[0].secrets, ["key", "cert"]); - let listing = describe(&[info]); - assert!( - listing.contains(" provisioner cloud/vm A VM (makes machines that set themselves up from the cloud seed and Compose project; actions prune, images, grow; also stop, list; asks server; asks cpus (default 4); may ask image: a or b; secret key; secret cert)"), - "{listing}" - ); + let kinds: Vec<&str> = info.provisioners[0] + .kinds + .iter() + .map(|kind| kind.id.as_str()) + .collect(); + assert_eq!(kinds.len(), 2); + assert_eq!(info.provisioners[0].verbs, ["stop", "list"]); } #[test] @@ -1585,8 +1456,7 @@ mod tests { assert_eq!(linked.len(), 1); assert_eq!(linked[0].origin(&own), PluginOrigin::Npm); let info = linked[0].info(&own); - assert_eq!(info.features[0].tasks[0].name, "go"); - assert!(describe(&[info]).contains(" task x:go")); + assert_eq!(info.features[0].actions[0].name, "go"); assert!(remove(&paths, &shipped[0]).is_err()); fs::remove_file(own.join("node_modules/berth-plugin-x")).unwrap(); write(&own.join("x")); @@ -1701,6 +1571,7 @@ mod tests { let before = berth_api::Applied { berth: berth_api::VERSION.into(), plugins: records(&plugins), + ..Default::default() }; assert_eq!(behind(&before, &plugins), None); // What npm put beside it does not count; a task does. diff --git a/crates/berth-core/src/profiles.rs b/crates/berth-core/src/profiles.rs index ab6a504..7551417 100644 --- a/crates/berth-core/src/profiles.rs +++ b/crates/berth-core/src/profiles.rs @@ -8,7 +8,6 @@ use crate::{ Result, paths::Paths, - server::Server, store::{self, OWN, kind}, }; use serde_json::Value; @@ -26,13 +25,10 @@ pub enum Place { File(PathBuf), } -/// The scope of the saved hosts: the berth server's when one is set, and -/// this installation's own otherwise. -pub fn scope(paths: &Paths) -> String { - match Server::configured(paths) { - Some(server) => server.scope(), - None => OWN.to_owned(), - } +/// The scope of the saved hosts: this installation's own. A client of a +/// server elsewhere keeps none. +pub fn scope(_: &Paths) -> String { + OWN.to_owned() } /// Whether a name can be a saved host's: it is a file name of its own on @@ -135,14 +131,21 @@ impl Place { pub fn load(&self, paths: &Paths) -> Result> { match self { Place::Stored { scope, name } => { - Ok(store::shared(paths)?.read_json(kind::PROFILE, scope, name)?) + let mut profile: Option = + store::shared(paths)?.read_json(kind::PROFILE, scope, name)?; + // A row an older berth or berth server wrote after the store's + // names were converted is read with the names of now. + if let Some(profile) = &mut profile { + berth_api::profile::take_old_names(profile); + } + Ok(profile) } Place::File(path) => match fs::read(path) { Ok(data) => { let mut profile: Value = serde_json::from_slice(&data) .map_err(|error| format!("{}: {error}", path.display()))?; - // A profile of berth 0.4 is read in the access layout; the - // file keeps the old one until the user saves. + // A profile of an older berth is read in the layout of now; + // the file keeps the old one until the user saves. crate::import::convert_profile(paths, &self.title(), &mut profile); Ok(Some(profile)) } @@ -184,7 +187,11 @@ pub fn list(paths: &Paths, scope: &str) -> Result> { let entries = store::shared(paths)?.entries(kind::PROFILE, scope)?; Ok(entries .into_iter() - .filter_map(|(name, value)| Some((name, serde_json::from_slice(&value).ok()?))) + .filter_map(|(name, value)| { + let mut profile = serde_json::from_slice(&value).ok()?; + berth_api::profile::take_old_names(&mut profile); + Some((name, profile)) + }) .collect()) } @@ -193,6 +200,11 @@ pub fn list(paths: &Paths, scope: &str) -> Result> { /// logging in, in one change. A profile in a file is removed with the file. pub fn remove(paths: &Paths, place: &Place) -> Result<()> { let (scope, name) = place.side(); + // The provisioners a node offers go with it, unless they still hold resources. + if scope == OWN { + let secrets = crate::registry::Store::own(store::shared(paths)?); + crate::node::forget(paths, &secrets, &name)?; + } let password = crate::keys::password_secret(place); let sudo = crate::keys::sudo_secret(place); // The record of a machine that was never made goes with the host; a @@ -232,10 +244,10 @@ pub fn remove(paths: &Paths, place: &Place) -> Result<()> { /// Whether a rename may give the host's profile the new name as its /// hostname: only a workstation is applied, which is what sets it. pub fn hostname_allowed(name: &str, profile: &Value) -> Result<()> { - match crate::level::of(profile) { - crate::level::Level::Workstation => Ok(()), - level => Err(format!( - "{name} is a {level} host, and only a workstation is given its hostname by berth" + match crate::host_type::of(profile) { + crate::host_type::HostType::Workstation => Ok(()), + host_type => Err(format!( + "{name} is a host of the type {host_type}, and only a workstation is given its hostname by berth" ) .into()), } @@ -310,6 +322,22 @@ pub fn rename(paths: &Paths, from: &Place, to: &str, hostname: bool) -> Result

(&value) else { + continue; + }; + if entry.node == *old { + entry.node = to.to_owned(); + offered.push((name, serde_json::to_vec(&entry)?)); + } + } + } + for (name, value) in &offered { + writes.push((kind::PROVISIONER_ENTRY, OWN, name.as_str(), value.clone())); + } match scope == OWN { true => store.rename_host(old, to, &writes, &removals)?, false => store.change(&writes, &removals)?, @@ -351,14 +379,10 @@ mod tests { for bad in ["-x", ".hidden", "a b", ""] { assert!(Place::resolve(&paths, bad).is_err(), "{bad}"); } - // With a berth server set, a name is one of the server's hosts. + // A berth server set for the commands leaves the saved hosts its own. paths.set("server", "http://192.0.2.7:7777").unwrap(); - let served = Place::resolve(&paths, "studio").unwrap(); - assert_eq!(served.server_url(), Some("http://192.0.2.7:7777")); - assert_eq!( - served.describe(), - "studio on the berth server at http://192.0.2.7:7777" - ); + let own = Place::resolve(&paths, "studio").unwrap(); + assert_eq!(own.server_url(), None); let path = home.path().join("x.json"); assert_eq!( Place::resolve(&paths, path.to_str().unwrap()).unwrap(), @@ -543,12 +567,12 @@ mod tests { let dock = Place::own("dock").unwrap(); dock.save( &paths, - &json!({"level": "reachable", "access": {"address": "192.0.2.30"}}), + &json!({"type": "ssh", "access": {"address": "192.0.2.30"}}), ) .unwrap(); assert_eq!( error(rename(&paths, &dock, "pier", true)), - "dock is a reachable host, and only a workstation is given its hostname by berth" + "dock is a host of the type ssh, and only a workstation is given its hostname by berth" ); assert!(dock.load(&paths).unwrap().is_some()); let file = Place::File(home.path().join("attic.json")); diff --git a/crates/berth-core/src/provision.rs b/crates/berth-core/src/provision.rs index b6d9536..9202f61 100644 --- a/crates/berth-core/src/provision.rs +++ b/crates/berth-core/src/provision.rs @@ -18,9 +18,7 @@ use crate::{ plugins::{self, Plugin, Provisioner, Setup}, profiles::Place, registry::Store, - relay, seed, - server::KEPT, - store, + relay, seed, store, }; use berth_api::{NewResource, ProvisionerInfo}; use serde_json::{Map, Value, json}; @@ -138,7 +136,7 @@ pub(crate) fn find<'a>(plugins: &'a [Plugin], name: &str) -> Result<(&'a Plugin, let provisioner = known.provisioners.iter().find(|known| known.id == id)?; Some((known, provisioner)) }) - .ok_or_else(|| format!("No provisioner {name}. See: berth provisioners").into()) + .ok_or_else(|| format!("No provisioner {name}. See: berth provisioner list").into()) } /// The provisioner a name stands for: a registered entry's type, or the @@ -159,7 +157,7 @@ fn secret(store: &Store, name: &str) -> Option { .secret(name) .ok() .flatten() - .filter(|value| !value.is_empty() && value != KEPT) + .filter(|value| !value.is_empty()) } /// The secrets of the provisioner registered as `name`, under the names its @@ -188,11 +186,11 @@ pub(crate) fn secrets( pub(crate) fn needs_secrets(name: &str, missing: &[String]) -> Result<()> { match missing { [] => Ok(()), - missing => Err(format!( + missing => Err(crate::Refused(format!( "{name} needs the secret{} {}: berth secret set NAME", if missing.len() == 1 { "" } else { "s" }, missing.join(", ") - ) + )) .into()), } } @@ -363,6 +361,7 @@ pub(crate) fn call( .spawn() .map_err(|error| format!("{}: {error}", provisioner.command.display()))?; let group = child.id() as i32; + let _stopped = crate::runner::watch(&child); let mut input = serde_json::to_vec(request)?; input.push(b'\n'); let mut stdin = child.stdin.take().ok_or("no standard input")?; @@ -484,7 +483,7 @@ pub fn prepare(paths: &Paths, store: &Store, host: &str, asked: &NewResource) -> .filter(|kept| !kept.record.external_id.is_empty()) { return Err(format!( - "{host} has a machine already, {} of {}. See it with: berth resource {}; delete it with: berth resource {} remove", + "{host} has a machine already, {} of {}. See it with: berth resource show {}; delete it with: berth resource remove {}", kept.record.external_id, kept.record.provisioner, kept.record.id, kept.record.id ) .into()); @@ -494,7 +493,7 @@ pub fn prepare(paths: &Paths, store: &Store, host: &str, asked: &NewResource) -> .filter(|kept| kept.record.provisioner != asked.provisioner) { return Err(format!( - "A machine for {host} was asked of {}, which did not say it made one. Forget it with: berth resource {} remove", + "A machine for {host} was asked of {}, which did not say it made one. Forget it with: berth resource remove {}", kept.record.provisioner, kept.record.id ) .into()); @@ -527,6 +526,11 @@ pub fn prepare(paths: &Paths, store: &Store, host: &str, asked: &NewResource) -> ) .into()); } + // A new machine is on no tailnet, so a host that joins one needs a key + // berth can make, before anything is created. + if let Some(problem) = crate::tailnet::problem(paths, &profile.config, host) { + return Err(crate::Refused(problem).into()); + } let (scope, side) = profile.place.side(); // Asked again after a failure, the provisioner finds the machine by its token. let kept = match kept { @@ -565,16 +569,16 @@ pub fn prepare(paths: &Paths, store: &Store, host: &str, asked: &NewResource) -> }) } -/// Make the host's profile a `cloud` one before the machine is made from it: +/// Make the host's target `cloud` before the machine is made from it: /// a created machine has no laptop's physical login or power settings, and /// the seed is rendered from, or the role applied with, what is saved. -fn cloud_profile(paths: &Paths, host: &str) -> Result<()> { +fn cloud_target(paths: &Paths, host: &str) -> Result<()> { let place = Place::resolve(paths, host)?; let mut profile = place .load(paths)? .ok_or_else(|| format!("No host {host}"))?; - if profile["profile"] != "cloud" && profile.is_object() { - profile["profile"] = json!("cloud"); + if profile["target"] != "cloud" && profile.is_object() { + profile["target"] = json!("cloud"); place.save(paths, &profile)?; } Ok(()) @@ -630,7 +634,7 @@ impl Prepared { ) -> Result<()> { let (host, name) = (self.host.clone(), self.name.clone()); say(out, &format!("▸ {host}: creating its machine with {name}")); - cloud_profile(paths, &host)?; + cloud_target(paths, &host)?; let profile = Host::load(paths, &host)?; let mut request = resource::request( "create", @@ -638,6 +642,7 @@ impl Prepared { &self.kept.record, &self.kept.record.arguments, &self.secrets, + &crate::provisioner::node_of(paths, &self.name), ); request["token"] = json!(self.kept.record.token); match self.provisioner.setup { @@ -687,7 +692,7 @@ impl Prepared { let id = answer["id"].as_str().unwrap_or("").to_owned(); if id.is_empty() { return Err(format!( - "{name} create said no id. Asking again finds the same machine: berth provision {host} {name}" + "{name} create said no id. Asking again finds the same machine: berth host create {host} --provisioner {name}" ) .into()); } @@ -702,7 +707,7 @@ impl Prepared { let resource = self.kept.record.id.clone(); let exists = |error: Box| -> Box { format!( - "{error}. The machine {id} exists and stays: berth resource {resource} remove deletes it" + "{error}. The machine {id} exists and stays: berth resource remove {resource} deletes it" ) .into() }; @@ -759,6 +764,7 @@ impl Prepared { &self.kept.record, &self.kept.record.arguments, &self.secrets, + &crate::provisioner::node_of(paths, &self.name), ); let mut log = resource::request( "logs", @@ -766,6 +772,7 @@ impl Prepared { &self.kept.record, &self.kept.record.arguments, &self.secrets, + &crate::provisioner::node_of(paths, &self.name), ); (log["tail"], log["follow"]) = (json!(CONSOLE_LINES), json!(false)); let logs = self @@ -1054,7 +1061,6 @@ fn password(paths: &Paths, host: &Host) -> Option { return None; } host.kept_sudo_password(paths) - .or_else(|| std::env::var("BERTH_SUDO_PASSWORD").ok()) } /// Whether SSH lets berth in. A machine just made has a key nobody has seen; @@ -1083,13 +1089,8 @@ fn answers(paths: &Paths, host: &Host) -> bool { }; let link = Link::new(spec); // The timer is made inside the runtime, which is what it runs on. - let asked = runtime.block_on(async { - tokio::time::timeout( - Duration::from_secs(20), - link.get::("/v1/version"), - ) - .await - }); + let asked = runtime + .block_on(async { tokio::time::timeout(Duration::from_secs(20), link.version()).await }); runtime.shutdown_background(); matches!(asked, Ok(Ok(_))) } @@ -1195,15 +1196,19 @@ fn finish(paths: &Paths, host: &Host, setup: Setup, out: &mut dyn Write) -> Resu let password = password(paths, host); if host.asks_password() && password.is_none() { return Err(format!( - "{}'s SSH user's sudo asks for a password: set BERTH_SUDO_PASSWORD, or apply it with: berth apply {}", - host.name, host.name + "{name}'s SSH user's sudo asks for a password: keep it with berth secret set sudo/{name}, or apply it with berth host apply {name}", + name = host.name ) .into()); } say(out, &format!("▸ {}: applying", host.name)); let mut prepared = apply::prepare(paths, host, password.as_deref())?; + for note in &prepared.notes { + say(out, note); + } let (reader, writer) = std::io::pipe()?; let copy = writer.try_clone()?; + crate::runner::own_group(&mut prepared.command); let mut child = prepared .command .stdin(Stdio::null()) @@ -1211,6 +1216,7 @@ fn finish(paths: &Paths, host: &Host, setup: Setup, out: &mut dyn Write) -> Resu .stderr(copy) .spawn() .map_err(|error| format!("ansible-playbook: {error}"))?; + let _group = crate::runner::watch(&child); // The command holds the pipe's other ends; they close when it is dropped. drop(prepared.command); for line in std::io::BufReader::new(reader).lines() { @@ -1262,7 +1268,7 @@ out = build / config['hostname'] / 'cloud' out.mkdir(parents=True, exist_ok=True) (build / 'devbox-release.tar.gz').write_bytes(b'release') (out / 'user-data').write_text('#cloud-config\n# delivered=' + os.environ.get('BERTH_RELEASE_DELIVERED', '') + '\n' - + '# profile=' + config.get('profile', '') + '\n') + + '# target=' + config.get('target', '') + '\n') print(build / 'devbox-release.tar.gz') print(out) "#; @@ -1420,7 +1426,7 @@ print(out) &format!( "finished {} as {}", host.address(), - host.config["profile"].as_str().unwrap_or("") + host.config["target"].as_str().unwrap_or("") ), ); Ok(()) @@ -1477,7 +1483,7 @@ print(out) .unwrap_err() .to_string(); assert!( - said.contains(&format!("berth resource {} remove", kept.id)), + said.contains(&format!("berth resource remove {}", kept.id)), "{said}" ); @@ -1640,6 +1646,34 @@ print(out) assert!(!alive, "the provisioner's child {child} is still running"); } + #[test] + fn a_host_that_joins_the_tailnet_gets_no_machine_without_berths_api_key() { + let world = World::new(); + world + .store() + .set_secret("api_token", Some("tok-made-up")) + .unwrap(); + world.file("answer.create", r#"{"id": "vm-1"}"#); + world.profile( + "studio", + json!({"hostname": "studio", "access": {"user": "ops"}}), + ); + let said = world + .provision("fake/vm", &[("server", "x")]) + .0 + .unwrap_err() + .to_string(); + assert!(said.contains("berth secret set tailnet_api_key"), "{said}"); + // Nothing was asked of the provisioner, and no record was kept. + assert!(world.requests().iter().all(|(verb, _)| verb != "create")); + let place = crate::profiles::Place::own("studio").unwrap(); + assert!( + crate::resource::of_place(&world.paths, &place) + .unwrap() + .is_none() + ); + } + #[test] fn secrets_go_on_standard_input_and_never_into_the_output() { let world = World::new(); @@ -1656,7 +1690,7 @@ print(out) world.file("answer.create", r#"{"id": "vm-1"}"#); world.profile( "studio", - json!({"access": {"user": "ops", "address": "192.0.2.30"}}), + json!({"access": {"user": "ops", "address": "192.0.2.30"}, "ansible_vars": {"devbox_install_tailscale": false}}), ); let (done, out) = world.provision("fake/vm", &[("server", "x")]); done.unwrap(); @@ -1704,7 +1738,7 @@ print(out) world.file("answer.create", r#"{"id": "vm-2"}"#); world.profile( "studio", - json!({"access": {"user": "ops", "address": "192.0.2.20"}}), + json!({"access": {"user": "ops", "address": "192.0.2.20"}, "ansible_vars": {"devbox_install_tailscale": false}}), ); world.provision("fake/vm", &[("server", "x")]).0.unwrap(); world.file("answer.status", r#"{"state": "running"}"#); @@ -1819,7 +1853,7 @@ print(out) .unwrap_err() .to_string(); assert!( - said.contains("gone") && said.contains("berth resource r-") && said.contains(" remove"), + said.contains("gone") && said.contains("berth resource remove r-"), "{said}" ); assert_eq!(world.record().unwrap().external_id, "seeded-1"); @@ -1828,10 +1862,10 @@ print(out) // A created machine is a cloud host, whatever the profile said before. assert_eq!( create["seed"]["user_data"], - "#cloud-config\n# delivered=1\n# profile=cloud\n" + "#cloud-config\n# delivered=1\n# target=cloud\n" ); let host = Host::load(&world.paths, "studio").unwrap(); - assert_eq!(host.config["profile"], "cloud"); + assert_eq!(host.config["target"], "cloud"); assert_eq!( create["seed"]["meta_data"], format!("instance-id: {token}\nlocal-hostname: studio\n") @@ -1903,7 +1937,7 @@ print(out) Ok(match asked("first boot") { 1 => "state provisioning\n==> Installing Ansible\n-- cursor: s=1;i=1\n", 2 => "state provisioning\n==> Applying the role\nTASK [devbox : Check the profile] *\nTASK [devbox : Install Docker] *\n-- cursor: s=1;i=2\n", - _ => "state configured-needs-enrollment\n==> The role is applied\n-- cursor: s=1;i=3\n", + _ => "state configured\n==> The role is applied\n-- cursor: s=1;i=3\n", } .to_owned()) }, @@ -1927,7 +1961,7 @@ print(out) " │ Installing Ansible", " │ Applying the role", " │ Ansible task 2: Install Docker", - " │ first boot: configured-needs-enrollment", + " │ first boot: configured", " │ The role is applied", "✓ studio: berthd answers", "finished 192.0.2.40 as cloud", @@ -1975,10 +2009,7 @@ print(out) let said = done.unwrap_err().to_string(); assert!(started.elapsed() < Duration::from_secs(20)); assert!(said.contains("studio's first boot failed"), "{said}"); - assert!( - said.contains("berth resource r-") && said.contains(" remove"), - "{said}" - ); + assert!(said.contains("berth resource remove r-"), "{said}"); let end = out .split_once("! studio: its first boot failed. The end of its log:\n") .unwrap() @@ -2032,7 +2063,7 @@ print(out) #[test] fn the_secrets_a_seed_carries_are_found_in_it() { - let deployment = json!({"profile": "cloud", "ansible_vars": { + let deployment = json!({"target": "cloud", "ansible_vars": { "devbox_tailscale_auth_key": "tskey-auth-kAbc123-xyz", "devbox_operator_password_hash": "$6$salt$hashhashhash", "devbox_operator_ssh_public_key": "ssh-ed25519 AAAAexample", @@ -2231,7 +2262,7 @@ print(out) assert!( listed[0] .problem - .contains("berth provisioner lab run settings server=VALUE"), + .contains("berth provisioner set lab server=VALUE"), "{}", listed[0].problem ); diff --git a/crates/berth-core/src/provisioner.rs b/crates/berth-core/src/provisioner.rs index d23a2e7..ac7504e 100644 --- a/crates/berth-core/src/provisioner.rs +++ b/crates/berth-core/src/provisioner.rs @@ -32,6 +32,8 @@ pub const SETTINGS: &str = "settings"; pub const SECRETS: &str = "secrets"; /// berth's own action that forgets a registered provisioner. pub const REMOVE: &str = "remove"; +/// berth's own action that makes a provisioner registered by hand a node's. +pub const ADOPT: &str = "adopt"; /// The start of berth's own action that makes a resource of a kind, such as /// `new-deployment`. pub const NEW: &str = "new-"; @@ -48,6 +50,13 @@ pub struct Entry { pub kind: String, #[serde(default)] pub settings: BTreeMap, + /// The node it lives on, when a feature of that host offered it: its + /// name among this berth's hosts. + #[serde(default, skip_serializing_if = "String::is_empty")] + pub node: String, + /// The node's feature that offered it, as `PLUGIN/FEATURE`. + #[serde(default, skip_serializing_if = "String::is_empty")] + pub feature: String, } /// The entry registered under a provisioner's name, if there is one. @@ -57,6 +66,15 @@ pub fn entry(paths: &Paths, name: &str) -> Result> { .and_then(|entry| serde_json::from_value(entry).ok())) } +/// The node a registered provisioner lives on, or nothing. +pub fn node_of(paths: &Paths, name: &str) -> String { + entry(paths, name) + .ok() + .flatten() + .map(|entry| entry.node) + .unwrap_or_default() +} + /// Register a provisioner under `name`, of the type `entry.kind`. pub fn save_entry(paths: &Paths, name: &str, entry: &Entry) -> Result<()> { if !plugins::valid_name(name) { @@ -131,10 +149,15 @@ pub fn register(paths: &Paths, name: &str, saved: &berth_api::SavedProvisioner) let settings = match &saved.settings { Some(given) => kept_values(name, provisioner, given)?, None => existing - .map(|existing| existing.settings) + .as_ref() + .map(|existing| existing.settings.clone()) .unwrap_or_default(), }; - let entry = Entry { kind, settings }; + let entry = Entry { + kind, + settings, + ..existing.unwrap_or_default() + }; save_entry(paths, name, &entry)?; Ok(entry) } @@ -202,7 +225,7 @@ pub fn adopted_name(paths: &Paths, name: &str, kind: &str) -> Result { /// holds resources is kept: they would have no provisioner. pub fn remove_entry(paths: &Paths, store: &Store, name: &str) -> Result<()> { if entry(paths, name)?.is_none() { - return Err(format!("No provisioner {name}. See: berth provisioner").into()); + return Err(format!("No provisioner {name}. See: berth provisioner list").into()); } let held: Vec = crate::resource::list(paths)? .into_iter() @@ -211,7 +234,7 @@ pub fn remove_entry(paths: &Paths, store: &Store, name: &str) -> Result<()> { .collect(); if !held.is_empty() { return Err(format!( - "{name} still holds {}. Remove {} first: berth resource ID remove", + "{name} still holds {}. Remove {} first: berth resource remove ID", held.join(", "), if held.len() == 1 { "it" } else { "them" } ) @@ -286,6 +309,7 @@ pub fn describe( let mut info = provisioner.info(plugin, missing.clone()); info.id = name.to_owned(); info.settings = settings(paths, name).unwrap_or_default(); + info.node = node_of(paths, name); if info.problem.is_empty() { info.problem = problem(&info.id, provisioner, &missing, &info.settings); } @@ -313,7 +337,7 @@ fn problem( match unset.is_empty() { true => String::new(), false => format!( - "{name} needs {}: berth provisioner {name} run {SETTINGS} {}", + "{name} needs {}: berth provisioner set {name} {}", unset.join(" and "), unset .iter() @@ -524,6 +548,9 @@ pub fn actions(paths: &Paths, store: &Store, name: &str) -> Result> if !provisioner.secrets.is_empty() { actions.push(secrets_action(name, &provisioner)); } + if entry(paths, name)?.is_some_and(|kept| kept.node.is_empty()) { + actions.push(adopt_action(name)); + } actions.push(remove_action(paths, name)); } Ok(actions) @@ -597,7 +624,7 @@ fn new_action( }, match kind.id == "deployment" { true => - " From the command line also: berth deploy NAME --provisioner PROVISIONER --spec FILE", + " From the command line also: berth resource create NAME --provisioner PROVISIONER --spec FILE", false => "", } ), @@ -634,6 +661,29 @@ fn secrets_action(name: &str, provisioner: &Provisioner) -> Action { } } +/// berth's own action that makes a provisioner registered by hand the one a +/// node offers, for a server the node's report alone cannot vouch for. +fn adopt_action(name: &str) -> Action { + Action { + name: ADOPT.into(), + label: "Make it a node's".into(), + help: format!( + "Make {name} the provisioner a node offers, when you know it reaches that node's server, such as one registered for a server that is now applied as a node. The next apply of the node brings its settings and the certificate it pins up to date." + ), + group: "Set-Up".into(), + arguments: vec![Argument { + name: "node".into(), + label: "Node".into(), + required: true, + help: "The node, by its name among berth's hosts".into(), + ..Argument::default() + }], + confirm: Some(format!("Make {name} the node's?")), + role: Role::Admin, + ..Action::default() + } +} + /// berth's own action that forgets a registered provisioner, which it does /// only when it holds nothing. fn remove_action(paths: &Paths, name: &str) -> Action { @@ -692,6 +742,7 @@ enum Doing { Settings(BTreeMap), Secrets(BTreeMap, Store), Remove(Store), + Adopt(String), Create(Box, Store), Program { dir: PathBuf, @@ -705,6 +756,8 @@ enum Doing { pub struct Answered { pub message: String, pub details: Vec, + /// The exit status of a command it ran, which its job ends with. + pub exit_code: Option, } impl Answered { @@ -717,7 +770,11 @@ impl Answered { pub fn lines(&self) -> Vec { let mut lines = vec![]; if !self.message.is_empty() { - lines.push(format!("✓ {}", self.message)); + let mark = match self.exit_code { + Some(code) if code != 0 => "✗", + _ => "✓", + }; + lines.push(format!("{mark} {}", self.message)); } let width = self .details @@ -795,6 +852,15 @@ pub fn prepare( work: Doing::Remove(store.clone()), }); } + if action == ADOPT { + let values = values_of(name, &found.arguments, given, &BTreeMap::new())?; + return Ok(Acting { + name: name.to_owned(), + hidden: vec![], + action: found, + work: Doing::Adopt(values.get("node").cloned().unwrap_or_default()), + }); + } if let Some(kind) = action .strip_prefix(NEW) .filter(|kind| provisioner.kinds.iter().any(|known| known.id == *kind)) @@ -924,6 +990,7 @@ impl Acting { false => format!("{} uses {}", self.name, shown.join(" ")), }, details: vec![], + exit_code: None, }) } Doing::Secrets(values, store) => { @@ -952,6 +1019,7 @@ impl Acting { false => format!("{}'s secrets", self.name), }, details, + exit_code: None, }) } Doing::Remove(store) => { @@ -959,15 +1027,22 @@ impl Acting { Ok(Answered { message: format!("{} is removed", self.name), details: vec![], + exit_code: None, }) } + Doing::Adopt(node) => Ok(Answered { + message: crate::node::adopt(paths, &self.name, &node)?, + details: vec![], + exit_code: None, + }), Doing::Create(creating, store) => { let id = creating.kept().record.id.clone(); let title = creating.kept().title(); creating.run(paths, &store, options, out)?; Ok(Answered { - message: format!("{title} is made. Its page: berth resource {id}"), + message: format!("{title} is made. Its page: berth resource show {id}"), details: vec![], + exit_code: None, }) } Doing::Program { @@ -983,6 +1058,7 @@ impl Acting { Ok(Answered { message: answer["message"].as_str().unwrap_or("").to_owned(), details: details(&answer), + exit_code: None, }) } } @@ -1009,6 +1085,7 @@ pub(crate) fn stream( .spawn() .map_err(|error| format!("{}: {error}", provisioner.command.display()))?; let group = child.id() as i32; + let _stopped = crate::runner::watch(&child); let mut input = serde_json::to_vec(asked)?; input.push(b'\n'); let mut stdin = child.stdin.take().ok_or("no standard input")?; diff --git a/crates/berth-core/src/registry.rs b/crates/berth-core/src/registry.rs index 9e6676f..d92bfb0 100644 --- a/crates/berth-core/src/registry.rs +++ b/crates/berth-core/src/registry.rs @@ -4,15 +4,11 @@ //! Registries are settings of the tool, because a server is the same for //! every host. A Sonatype Nexus server is one registry with its repositories //! under it. Both are kept in berth's store: registries as their JSON, -//! secrets as their text, sealed when the store is encrypted. What is known -//! of a berth server's is kept under the server's scope, with its secret -//! values standing in as [`KEPT`]. +//! secrets as their text, sealed when the store is encrypted. use crate::{ Result, paths::Paths, - plugins::valid_name, - server::{KEPT, Server}, store::{self, kind}, }; use serde_json::{Map, Value, json}; @@ -36,16 +32,12 @@ pub fn type_label(kind: &str) -> &'static str { } pub const FORMATS: &[&str] = &["npm", "raw", "docker"]; -/// The registries and secrets of this installation, or what is known of a -/// berth server's. +/// The registries and secrets of this installation. #[derive(Clone)] pub struct Store { store: Arc, - /// [`store::OWN`], or the scope of the server. + /// [`store::OWN`]. scope: String, - /// The berth serve these are the registries and secrets of. Changes go - /// there, and the values of its secrets stay there. - server: Option, } pub fn base64(data: &[u8]) -> String { @@ -103,52 +95,25 @@ pub fn repository_url(registry: &Value, name: &str) -> String { } impl Store { - /// Those berth uses: the server's when one is configured, else its own. + /// This installation's. pub fn new(paths: &Paths) -> Result { - Ok(Store::of(store::shared(paths)?, Server::configured(paths))) + Ok(Store::own(store::shared(paths)?)) } - /// This installation's own, whatever server its settings name: what - /// `berth serve` keeps and sends to its hosts. + /// This installation's, in a store that is open: what `berth serve` + /// keeps and sends to its hosts. pub fn own(store: Arc) -> Self { - Store::of(store, None) - } - - /// What is known of `server`'s, or this installation's own without one. - pub fn of(store: Arc, server: Option) -> Self { - let scope = match &server { - Some(server) => server.scope(), - None => store::OWN.to_owned(), - }; Store { store, - scope, - server, + scope: store::OWN.to_owned(), } } - /// The server whose store this is. - pub fn server(&self) -> Option<&Server> { - self.server.as_ref() - } - /// Whether the values are sealed in the store. pub fn encrypted(&self) -> bool { self.store.encrypted() } - /// Fails for a server's store, whose secrets have their values there. - fn own_values(&self, what: &str) -> Result<()> { - match &self.server { - Some(server) => Err(format!( - "{what} needs the secrets, which stay on the berth server at {}", - server.url - ) - .into()), - None => Ok(()), - } - } - pub fn registries(&self) -> Result> { self.store .entries(kind::REGISTRY, &self.scope)? @@ -175,37 +140,14 @@ impl Store { .collect() } - /// Send the changes of the registries to the server, when there is one. - fn send_registries(&self, registries: &Map) -> Result<()> { - if let Some(server) = &self.server { - let known = self.registries()?; - for (name, registry) in registries { - if known.get(name) != Some(registry) { - server.call( - "registries.save", - json!({"name": name}), - Some(registry.clone()), - )?; - } - } - for name in known.keys().filter(|name| !registries.contains_key(*name)) { - server.call("registries.remove", json!({"name": name}), None)?; - } - } - Ok(()) - } - /// Make these the registries, in one change of the store. pub fn save_registries(&self, registries: Map) -> Result<()> { - self.send_registries(®istries)?; let rows = Self::registry_rows(®istries); Ok(self.store.replace(&self.scope, &[kind::REGISTRY], &rows)?) } - /// Give a registry another name, with its stored login. Here it is one - /// change of the store; a server keeps logins by the registry's name and - /// drops the old one with it, so for a server's registry the login has - /// to be given again. Returns whether the login went with the name. + /// Give a registry another name, with its stored login, in one change of + /// the store. Returns whether there was a login to go with the name. pub fn rename_registry(&self, old: &str, new: &str) -> Result { let mut registries = self.registries()?; let registry = registries @@ -216,11 +158,7 @@ impl Store { } registries.insert(new.to_owned(), registry.clone()); let (old_login, new_login) = (format!("registry/{old}"), format!("registry/{new}")); - let login = match &self.server { - Some(_) => None, - None => self.store.read(kind::SECRET, &self.scope, &old_login)?, - }; - self.send_registries(®istries)?; + let login = self.store.read(kind::SECRET, &self.scope, &old_login)?; let mut writes = vec![( kind::REGISTRY, self.scope.as_str(), @@ -237,7 +175,7 @@ impl Store { (kind::SECRET, &self.scope, &old_login), ], )?; - Ok(login.is_some() || self.server.is_none()) + Ok(login.is_some()) } fn registry(&self, name: &str) -> Result { @@ -266,7 +204,7 @@ impl Store { .collect() } - /// Whether a secret has a value, here or on the server. Nothing is unsealed. + /// Whether a secret has a value. Nothing is unsealed. pub fn has_secret(&self, name: &str) -> Result { Ok(self.all_secret_names()?.iter().any(|known| known == name)) } @@ -281,20 +219,6 @@ impl Store { } pub fn set_secret(&self, name: &str, value: Option<&str>) -> Result<()> { - let value = match (&self.server, value) { - (Some(server), Some(value)) => { - let data = Some(json!({"value": value})); - server.call("secrets.set", json!({"name": name}), data)?; - Some(KEPT) - } - (Some(server), None) => { - if self.has_secret(name)? { - server.call("secrets.remove", json!({"name": name}), None)?; - } - None - } - (None, value) => value, - }; match value { Some(value) => self .store @@ -349,7 +273,6 @@ impl Store { /// What `devbox-secrets` on a host installs for a profile: the profile's /// secrets as environment variables and files, and the logins of its registries. pub fn host_payload(&self, profile: &Value) -> Result { - self.own_values("Sending secrets to a host")?; let list = |value: &Value| value.as_array().cloned().unwrap_or_default(); let mut env = Map::new(); let mut files = vec![]; @@ -483,7 +406,6 @@ fn http_status(url: &str, login: &Option) -> std::result::Result Result> { - self.own_values("Logging in to a registry")?; let registry = self.registry(name)?; let quote = |text: &str| text.replace('\\', "\\\\").replace('"', "\\\""); Ok( @@ -509,16 +431,6 @@ impl Store { /// the account may also upload is not tried. One line per thing checked, /// starting with ✓, ! or ✗. pub fn check(store: &Store, name: &str) -> Result> { - // A server's registry is checked from the server, which has its login. - if let Some(server) = &store.server { - let checked = server.call("registries.check", json!({"name": name}), None)?; - return Ok(checked["lines"] - .as_array() - .into_iter() - .flatten() - .filter_map(|line| line.as_str().map(str::to_owned)) - .collect()); - } let registry = store.registry(name)?; let login = store.curl_login(name)?; let with_login = if login.is_some() { @@ -671,118 +583,6 @@ pub fn describe(store: &Store) -> Result> { Ok(lines) } -/// `berth registry ...` -pub fn registry_command(paths: &Paths, args: &[&str]) -> Result<()> { - let store = Store::new(paths)?; - let mut registries = store.registries()?; - match *args { - [] => { - let lines = describe(&store)?; - if lines.is_empty() { - println!( - "No registries. Add one: berth registry add NAME {} URL [USERNAME]", - TYPES.join("|") - ); - } - lines.iter().for_each(|line| println!("{line}")); - return Ok(()); - } - ["add", name, kind, url] | ["add", name, kind, url, _] => { - if !valid_name(name) || !TYPES.contains(&kind) { - return Err(format!("Usage: berth registry add NAME {} URL [USERNAME] (without a user name the login is a token)", TYPES.join("|")).into()); - } - if kind != "docker" && !url.starts_with("https://") && !url.starts_with("http://") { - return Err("This kind of registry needs its address with https://".into()); - } - let mut registry = json!({"type": kind, "url": url.trim_end_matches('/')}); - if let Some(user) = args.get(4) { - registry["username"] = json!(user); - } - // Adding again changes the address or user and keeps the repositories. - if let Some(known) = registries.get(name) { - registry["repositories"] = known["repositories"].clone(); - } - registries.insert(name.to_owned(), registry); - } - ["repo", name, format, repository] | ["repo", name, format, repository, _] => { - let registry = registries - .get_mut(name) - .filter(|registry| registry["type"] == "nexus") - .ok_or_else(|| format!("{name} is not a Nexus registry"))?; - if !FORMATS.contains(&format) { - return Err(format!("Formats: {}", FORMATS.join(", ")).into()); - } - let mut entry = json!({"format": format, "name": repository}); - if let Some(address) = args.get(4) { - entry["address"] = json!(bare(address)); - } - let mut repositories = registry["repositories"] - .as_array() - .cloned() - .unwrap_or_default(); - repositories.retain(|known| known["name"] != repository); - repositories.push(entry); - registry["repositories"] = json!(repositories); - } - ["check"] => { - for name in registries.keys() { - println!("{name}"); - check(&store, name)? - .iter() - .for_each(|line| println!(" {line}")); - } - return Ok(()); - } - ["check", name] => { - check(&store, name)? - .iter() - .for_each(|line| println!("{line}")); - return Ok(()); - } - ["login", name] => { - store.registry(name)?; - let password = read_secret(&format!("Password or token for {name}"))?; - return store.set_secret(&format!("registry/{name}"), Some(&password)); - } - ["remove", name] => { - registries - .remove(name) - .ok_or_else(|| format!("No registry {name}"))?; - store.set_secret(&format!("registry/{name}"), None)?; - } - _ => { - return Err("Usage: berth registry [add NAME TYPE URL [USERNAME] | repo NAME FORMAT REPOSITORY [ADDRESS] | login NAME | check [NAME] | remove NAME]".into()); - } - } - store.save_registries(registries) -} - -/// `berth secret ...` -pub fn secret_command(paths: &Paths, args: &[&str]) -> Result<()> { - let store = Store::new(paths)?; - match *args { - [] => { - for name in store.all_secret_names()? { - println!("{name}"); - } - Ok(()) - } - // A host's SSH or sudo password is kept under the host's name. - ["set", name] - if valid_name(name) - || crate::provisioner::is_secret_name(name) - || name - .strip_prefix("ssh/") - .or_else(|| name.strip_prefix("sudo/")) - .is_some_and(crate::profiles::valid_name) => - { - store.set_secret(name, Some(&read_secret(&format!("Value of {name}"))?)) - } - ["remove", name] => store.set_secret(name, None), - _ => Err("Usage: berth secret [set NAME | remove NAME] (names: lower-case letters, digits, - and _; ssh/HOST and sudo/HOST are a host's SSH and sudo passwords, provisioner/NAME/SECRET a provisioner's own)".into()), - } -} - /// Split `REGISTRY` or `REGISTRY/REPOSITORY`, and find the registry. fn target(store: &Store, target: &str) -> Result<(String, Value, String)> { let (name, repository) = target.split_once('/').unwrap_or((target, "")); @@ -793,7 +593,7 @@ fn target(store: &Store, target: &str) -> Result<(String, Value, String)> { )) } -/// `berth publish REGISTRY/REPOSITORY`: put the release archive in a Nexus raw +/// `berth registry publish REGISTRY/REPOSITORY`: put the release archive in a Nexus raw /// repository. Returns the address it is served at. pub fn publish(paths: &Paths, destination: &str) -> Result { let dirs: Map = @@ -870,7 +670,7 @@ pub fn publish_from( Ok(url) } -/// `berth image push REGISTRY[/REPOSITORY] [TAG]`: push the container image +/// `berth registry push-image REGISTRY[/REPOSITORY] [TAG]`: push the container image /// built by the Docker action. pub fn push_image(paths: &Paths, destination: &str, tag: &str) -> Result { let store = Store::new(paths)?; @@ -1051,44 +851,4 @@ mod tests { let names: Vec<_> = store.registries().unwrap().keys().cloned().collect(); assert_eq!(names, ["only"]); } - - #[test] - fn a_servers_scope_is_apart_and_holds_no_values() { - let (_home, opened) = opened(); - let own = Store::own(opened.clone()); - own.set_secret("github", Some("ghp-made-up")).unwrap(); - // A server that does not answer: nothing is written for it. - let server = Server { - url: "unix:/nonexistent/serve.sock".into(), - token: None, - }; - let served = Store::of(opened.clone(), Some(server.clone())); - assert!(served.set_secret("deploy", Some("value")).is_err()); - assert!(served.all_secret_names().unwrap().is_empty()); - // What a mirror wrote is seen there, and not here. - opened - .replace( - &server.scope(), - &[kind::SECRET, kind::REGISTRY], - &[ - (kind::SECRET, "deploy", KEPT.as_bytes().to_vec()), - ( - kind::REGISTRY, - "hub", - br#"{"type":"docker","url":"ghcr.io"}"#.to_vec(), - ), - ], - ) - .unwrap(); - assert_eq!(served.secret_names(), ["deploy"]); - assert_eq!(served.secret("deploy").unwrap().as_deref(), Some(KEPT)); - assert!(served.registries().unwrap().contains_key("hub")); - assert!(served.host_payload(&json!({})).is_err()); - assert_eq!(own.secret_names(), ["github"]); - assert!(own.registries().unwrap().is_empty()); - assert_eq!( - own.secret("github").unwrap().as_deref(), - Some("ghp-made-up") - ); - } } diff --git a/crates/berth-core/src/resource.rs b/crates/berth-core/src/resource.rs index 2de91d4..a8f5019 100644 --- a/crates/berth-core/src/resource.rs +++ b/crates/berth-core/src/resource.rs @@ -22,7 +22,7 @@ use crate::{ runner::{Lock, Subject}, store::{self, OWN, kind}, }; -use berth_api::{Action, Argument, Resource, ResourceSecret, ops::Role}; +use berth_api::{Action, Argument, Endpoint, Resource, ResourceSecret, ops::Role}; use serde::{Deserialize, Serialize}; use serde_json::{Map, Value, json}; use std::{collections::BTreeMap, io::Write, path::PathBuf}; @@ -63,6 +63,9 @@ pub struct Record { pub checked: String, #[serde(default, skip_serializing_if = "String::is_empty")] pub address: String, + /// Where it can be reached, as its provisioner last said. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub endpoints: Vec, } /// A record with the scope it is kept in. @@ -143,7 +146,7 @@ pub fn find_by(paths: &Paths, given: &str) -> Result { .collect(); match named.as_slice() { [one] => Ok((*one).clone()), - [] => Err(format!("No resource {given}. See: berth resource").into()), + [] => Err(format!("No resource {given}. See: berth resource list").into()), _ => Err(format!( "Several resources are called {given}: name one by its id ({})", named @@ -161,7 +164,7 @@ pub fn find(paths: &Paths, id: &str) -> Result { list(paths)? .into_iter() .find(|kept| kept.record.id == id) - .ok_or_else(|| format!("No resource {id}. See: berth resource").into()) + .ok_or_else(|| format!("No resource {id}. See: berth resource list").into()) } /// The machine of the host whose profile is at `place`. @@ -332,7 +335,7 @@ pub fn info(paths: &Paths, kept: &Kept, plugins: &[Plugin], busy: bool) -> Resou (true, false) => ( "unknown".to_owned(), format!( - "Creating it did not end. Asking again finds the same one; removing it forgets it: berth resource {} remove", + "Creating it did not end. Asking again finds the same one; removing it forgets it: berth resource remove {}", record.id ), ), @@ -348,6 +351,7 @@ pub fn info(paths: &Paths, kept: &Kept, plugins: &[Plugin], busy: bool) -> Resou external_id: record.external_id.clone(), host: (!record.host.is_empty()).then(|| record.host.clone()), address: record.address.clone(), + endpoints: record.endpoints.clone(), arguments: record.arguments.clone(), secrets: record.secrets.clone(), spec_sha256: record.spec_sha256.clone(), @@ -387,6 +391,8 @@ pub(crate) struct Called { pub dir: PathBuf, pub provisioner: Provisioner, pub secrets: Map, + /// The node its provisioner lives on, or nothing. + pub node: String, } impl Called { @@ -407,6 +413,7 @@ impl Called { dir: plugin.dir.clone(), provisioner: provisioner.clone(), secrets, + node: crate::provisioner::node_of(paths, &kept.record.provisioner), kept, }) } @@ -431,6 +438,7 @@ impl Called { &self.kept.record, arguments, &self.secrets, + &self.node, ) } @@ -451,9 +459,14 @@ pub(crate) fn request( record: &Record, arguments: &BTreeMap, secrets: &Map, + node: &str, ) -> Value { let mut request = provision::request(verb, provisioner, arguments, secrets); request["kind"] = json!(record.kind); + // Its endpoints are dialled from the node, so the plugin reports them as the node sees them. + if !node.is_empty() { + request["node"] = json!(node); + } request["name"] = json!(record.name); if record.kind == plugins::MACHINE { // A machine's labels are those of berth before resources, which @@ -477,7 +490,7 @@ pub(crate) fn environment(store: &Store, wanted: &[ResourceSecret]) -> Result { given.insert(secret.variable().to_owned(), json!(value)); @@ -527,6 +540,40 @@ fn state_of(answer: &Value) -> Option { } } +/// Keep the endpoints an answer reports, when it reports them: what it does +/// not name it no longer has. One that is gone has none. An endpoint that +/// does not fit is left out, and said to `out`. +fn take_endpoints(record: &mut Record, answer: &Value, out: &mut dyn Write) { + if record.state == "missing" { + record.endpoints.clear(); + return; + } + let Some(listed) = answer.get("endpoints").and_then(Value::as_array) else { + return; + }; + let mut endpoints: Vec = vec![]; + for entry in listed { + match Endpoint::from_answer(entry) { + Ok(endpoint) if endpoints.iter().any(|known| known.name == endpoint.name) => say( + out, + &format!( + "! {} reported the endpoint {} twice; the first stands", + record.provisioner, endpoint.name + ), + ), + Ok(endpoint) => endpoints.push(endpoint), + Err(why) => say( + out, + &format!( + "! {} reported an endpoint berth leaves out: {why}", + record.provisioner + ), + ), + } + } + record.endpoints = endpoints; +} + /// Ask the provisioner how a resource is, and keep what it says. One it no /// longer finds is marked missing, and its record stays. pub fn status(paths: &Paths, store: &Store, id: &str, options: &Options) -> Result { @@ -564,6 +611,7 @@ pub fn status(paths: &Paths, store: &Store, id: &str, options: &Options) -> Resu { kept.record.address = address.into(); } + take_endpoints(&mut kept.record, &answer, &mut std::io::sink()); kept.record.checked = berth_api::now(); // Seen by someone else meanwhile, such as removed: what it said stands. if find(paths, id).is_ok() { @@ -705,7 +753,7 @@ pub fn prepare(paths: &Paths, store: &Store, asked: &berth_api::NewResource) -> .filter(|kept| !kept.record.external_id.is_empty()) { return Err(format!( - "There is a {kind} {name} of {} already, {}. Change it with: berth resource {} change, or remove it first", + "There is a {kind} {name} of {} already, {}. Change it with: berth resource set {}, or remove it first", asked.provisioner, existing.record.id, existing.record.id ) .into()); @@ -751,6 +799,7 @@ pub fn prepare(paths: &Paths, store: &Store, asked: &berth_api::NewResource) -> &kept.record, &kept.record.arguments, &secrets, + &crate::provisioner::node_of(paths, &asked.provisioner), ); request["token"] = json!(kept.record.token); if let Some(spec) = spec { @@ -843,6 +892,7 @@ impl Creating { if let Some(address) = answer["address"].as_str() { kept.record.address = address.to_owned(); } + take_endpoints(&mut kept.record, &answer, out); kept.save(paths)?; say( out, @@ -869,6 +919,14 @@ pub const REMOVE: &str = "remove"; /// berth's own action that changes a resource's arguments and specification. pub const CHANGE: &str = "change"; +/// berth's own action that runs a command in a resource, for a kind whose +/// provisioner answers `exec`. +pub const EXEC: &str = "exec"; + +/// The arguments of `exec`. +pub const COMMAND_FIELD: &str = "command"; +pub const STDIN_FIELD: &str = "stdin"; + /// The argument of the change form that holds the specification, when the /// kind has one. pub const SPEC_FIELD: &str = "spec"; @@ -956,7 +1014,7 @@ pub fn actions(paths: &Paths, store: &Store, kept: &Kept) -> Vec { .collect(); let help = match kind.spec.is_some() { true => { - "Give it other values of its arguments, and its specification, which berth keeps and opens in the editor. From the command line a new one is given with: berth resource ID change --spec FILE" + "Give it other values of its arguments, and its specification, which berth keeps and opens in the editor. From the command line a new one is given with: berth resource set ID --spec FILE" } false => "Give it other values of its arguments, as its provisioner can.", }; @@ -974,6 +1032,33 @@ pub fn actions(paths: &Paths, store: &Store, kept: &Kept) -> Vec { } actions.push(limit(own(CHANGE, "Change", help).with_arguments(arguments))); } + if can(EXEC) { + let arguments = vec![ + Argument { + name: COMMAND_FIELD.into(), + label: "Command".into(), + required: true, + help: "The command and its arguments, split at spaces, or as a JSON list such as [\"sh\", \"-c\", \"echo hi\"] for an argument with spaces in it.".into(), + ..Argument::default() + }, + Argument { + name: STDIN_FIELD.into(), + label: "Standard input".into(), + kind: Argument::TEXT.into(), + help: "What the command reads on its standard input. Empty gives it none.".into(), + ..Argument::default() + }, + ]; + actions.push(limit(Action { + role: Role::Operate, + ..own( + EXEC, + "Run a command", + "Run a command in it, as its provisioner does, and wait for it to end. Its output and its exit status are the job's, and so is the command line; standard input is not recorded.", + ) + .with_arguments(arguments) + })); + } for action in kind.map(|kind| kind.actions.clone()).unwrap_or_default() { actions.push(limit(action)); } @@ -1047,6 +1132,9 @@ pub struct Acting { work: Doing, lock: Option, hidden: Vec, + /// What its job's record shows: the command of `exec`, or the + /// arguments that are no secret. + shown: Vec, } enum Doing { @@ -1060,6 +1148,12 @@ enum Doing { /// What it has after a change: its arguments, secrets and specification. after: Option, }, + /// A command, run with `exec`. + Exec { + called: Box, + request: Value, + command: Vec, + }, /// One of its plugin's actions, run with `act`. Act { called: Box, @@ -1129,7 +1223,8 @@ pub fn prepare_action( if (given.spec.is_some() || given.secrets.is_some()) && action != CHANGE { return Err(format!("Only {CHANGE} takes a specification or secrets").into()); } - let lock = match found.changes { + // Commands change what is in it, not berth's record, so several may run at once. + let lock = match found.changes && action != EXEC { true => Some(kept.lock(paths)?), false => None, }; @@ -1140,6 +1235,7 @@ pub fn prepare_action( work: Doing::Forget, lock, hidden: vec![], + shown: vec![], }); } let called = Box::new(Called::load(paths, store, kept.clone())?); @@ -1218,6 +1314,25 @@ pub fn prepare_action( after: Some((arguments, wanted, new_spec)), } } + EXEC => { + let theirs = values_of( + &format!("{EXEC} of {}", kept.record.id), + &found.arguments, + &given.arguments, + &BTreeMap::new(), + )?; + let command = argv(theirs.get(COMMAND_FIELD).map_or("", String::as_str))?; + let mut request = called.request(EXEC, &called.arguments()); + request["command"] = json!(command); + if let Some(stdin) = theirs.get(STDIN_FIELD).filter(|stdin| !stdin.is_empty()) { + request["stdin"] = json!(stdin); + } + Doing::Exec { + called, + request, + command, + } + } _ => { let declared = found.arguments.clone(); let theirs = values_of( @@ -1247,12 +1362,17 @@ pub fn prepare_action( } } }; + let shown = match &work { + Doing::Exec { command, .. } => command.clone(), + _ => Argument::shown(&found.arguments, &given.arguments), + }; Ok(Acting { kept, action: found, work, lock, hidden, + shown, }) } @@ -1261,6 +1381,12 @@ impl Acting { &self.action } + /// What its job's record shows: the command of `exec`, or the + /// arguments that are no secret. + pub fn shown(&self) -> Vec { + self.shown.clone() + } + pub fn kept(&self) -> &Kept { &self.kept } @@ -1292,7 +1418,7 @@ impl Acting { let _lock = lock; let done = |message: String| crate::provisioner::Answered { message, - details: vec![], + ..crate::provisioner::Answered::default() }; match work { Doing::Forget => { @@ -1373,6 +1499,7 @@ impl Acting { } else if verb == "start" { kept.record.state = "running".into(); } + take_endpoints(&mut kept.record, &answer, out); match after { Some((arguments, secrets, spec)) => { kept.record.arguments = arguments; @@ -1387,6 +1514,32 @@ impl Acting { _ => format!("Changed {}: {}", kept.title(), kept.record.state), })) } + Doing::Exec { + called, + request, + command, + } => { + // The output is the command's own, so berth adds no line of its own to it. + let answer = call( + &called.dir, + &called.provisioner, + EXEC, + &request, + options.create, + out, + )?; + let code = answer["exit_code"] + .as_i64() + .and_then(|code| i32::try_from(code).ok()) + .ok_or_else(|| { + format!("{} exec said no exit status", kept.record.provisioner) + })?; + Ok(crate::provisioner::Answered { + message: format!("{} exited with {code}", command[0]), + exit_code: Some(code), + ..crate::provisioner::Answered::default() + }) + } Doing::Act { called, request, @@ -1404,15 +1557,37 @@ impl Acting { limit, out, )?; + // An action that made it anew, such as pulling new images, may say where it is now. + if changes && answer.get("endpoints").is_some() { + let mut kept = find(paths, &kept.record.id).unwrap_or(kept); + take_endpoints(&mut kept.record, &answer, out); + kept.save(paths)?; + } Ok(crate::provisioner::Answered { message: answer["message"].as_str().unwrap_or("").to_owned(), details: crate::provisioner::details(&answer), + exit_code: None, }) } } } } +/// A command as `exec` is given it: a JSON list of words, or words split +/// at spaces. +pub fn argv(text: &str) -> Result> { + let text = text.trim(); + let words: Vec = match text.starts_with('[') { + true => serde_json::from_str(text) + .map_err(|_| "The command is not a JSON list of texts, such as [\"ls\", \"-l\"]")?, + false => text.split_whitespace().map(str::to_owned).collect(), + }; + match words.first() { + Some(first) if !first.is_empty() => Ok(words), + _ => Err("Give the command to run".into()), + } +} + // ── Its log ───────────────────────────────────────────────────────────────── /// What a resource wrote, as its provisioner's `logs` verb gives it: the diff --git a/crates/berth-core/src/rollout.rs b/crates/berth-core/src/rollout.rs index 5ffa715..9f3d5e3 100644 --- a/crates/berth-core/src/rollout.rs +++ b/crates/berth-core/src/rollout.rs @@ -22,8 +22,8 @@ pub trait Target { fn version(&mut self) -> std::result::Result, String>; /// Apply its configuration, its output going to `out`. fn apply(&mut self, out: &mut dyn Write) -> std::result::Result<(), String>; - /// Run one of its tasks to the end, with its output to `out`: the exit code. - fn task(&mut self, name: &str, out: &mut dyn Write) -> std::result::Result; + /// Run one of its actions to the end, with its output to `out`: the exit code. + fn run_action(&mut self, name: &str, out: &mut dyn Write) -> std::result::Result; /// Seconds it has been up, and whether it says it needs a reboot. fn uptime(&mut self) -> std::result::Result<(u64, bool), String>; /// How many sessions run there. @@ -87,7 +87,9 @@ pub fn run( name: target.name().to_owned(), ..Default::default() }; - if stopped { + // A roll-out that was stopped starts no other host. + if stopped || crate::runner::stopped().is_some() { + stopped = true; hosts.push(host); continue; } @@ -159,7 +161,7 @@ fn one( target.apply(out)?; if options.update { say(out, &format!("▸ {name}: update")); - match target.task("update", out) { + match target.run_action("update", out) { Ok(0) => {} Ok(code) => host.warnings.push(format!("update ended with {code}")), Err(error) => host.warnings.push(format!("update did not run: {error}")), @@ -200,9 +202,9 @@ fn one( return Err(format!("still behind after applying: {}", behind.text())); } } - match target.task("status", out)? { + match target.run_action("status", out)? { 0 => {} - code => return Err(format!("its status task ended with {code}")), + code => return Err(format!("its status action ended with {code}")), } host.state = RolloutState::Done; say(out, &format!("✓ {name} is up to date")); @@ -244,11 +246,11 @@ pub fn chosen(all: Vec, asked: &[String]) -> Result> { } /// The hosts a roll-out applies, and the names of those it leaves because -/// they are not workstations: only a workstation is applied. +/// berth does not apply them: only a node or a workstation is applied. pub fn workstations(hosts: Vec) -> (Vec, Vec) { - let (taken, left): (Vec, Vec) = hosts - .into_iter() - .partition(|host| crate::level::of(&host.config) == crate::level::Level::Workstation); + let (taken, left): (Vec, Vec) = hosts.into_iter().partition(|host| { + crate::host_type::Capability::Apply.allowed(crate::host_type::of(&host.config)) + }); (taken, left.into_iter().map(|host| host.name).collect()) } @@ -257,11 +259,11 @@ pub fn skipped_note(skipped: &[String]) -> Option { match skipped.len() { 0 => None, 1 => Some(format!( - "! Skipped 1 host that is not a workstation: {}", + "! Skipped 1 host that is neither a node nor a workstation: {}", skipped[0] )), count => Some(format!( - "! Skipped {count} hosts that are not workstations: {}", + "! Skipped {count} hosts that are neither nodes nor workstations: {}", skipped.join(", ") )), } @@ -325,10 +327,7 @@ impl Target for Live<'_> { } fn version(&mut self) -> std::result::Result, String> { - match self - .runtime - .block_on(self.link.get::("/v1/version")) - { + match self.runtime.block_on(self.link.version()) { Ok(version) => Ok(Some(version)), Err(error) if Link::missing(&error) => Ok(None), Err(error) => Err(error.to_string()), @@ -337,15 +336,19 @@ impl Target for Live<'_> { fn apply(&mut self, out: &mut dyn Write) -> std::result::Result<(), String> { if self.host.asks_password() && self.password.is_none() { - return Err( - "its SSH user's sudo asks for a password: set BERTH_SUDO_PASSWORD, or apply it by itself" - .into(), - ); + return Err(format!( + "its SSH user's sudo asks for a password: keep it with berth secret set sudo/{name}, or apply it by itself with berth host apply {name}", + name = self.host.name + )); } let mut prepared = apply::prepare(self.paths, &self.host, self.password.as_deref()) .map_err(|error| error.to_string())?; + for note in &prepared.notes { + let _ = writeln!(out, "{note}"); + } let (reader, writer) = std::io::pipe().map_err(|error| error.to_string())?; let copy = writer.try_clone().map_err(|error| error.to_string())?; + crate::runner::own_group(&mut prepared.command); let mut child = prepared .command .stdin(std::process::Stdio::null()) @@ -353,6 +356,7 @@ impl Target for Live<'_> { .stderr(copy) .spawn() .map_err(|error| format!("ansible-playbook: {error}"))?; + let _group = crate::runner::watch(&child); // The command holds the pipe's other ends; they close when it is dropped. drop(prepared.command); for line in std::io::BufReader::new(reader).lines() { @@ -362,7 +366,9 @@ impl Target for Live<'_> { let status = child.wait().map_err(|error| error.to_string())?; drop(prepared.files); match status.success() { - true => apply::settle(self.paths, &self.host).map_err(|error| error.to_string()), + true => { + apply::settle_with(self.paths, &self.host, out).map_err(|error| error.to_string()) + } false => Err(format!( "applying failed (ansible-playbook ended with {})", status.code().unwrap_or(-1) @@ -370,12 +376,12 @@ impl Target for Live<'_> { } } - fn task(&mut self, name: &str, out: &mut dyn Write) -> std::result::Result { - let path = format!("/v1/tasks/{}?wait=10m", api::client::encode(name)); + fn run_action(&mut self, name: &str, out: &mut dyn Write) -> std::result::Result { + let path = format!("/v1/actions/{}?wait=10m", api::client::encode(name)); let started: api::Started = self.post(&path, &json!({}))?; let id = started.session.id.clone(); let mut session = started.session; - // A task that takes longer than the wait is asked after again. + // An action that takes longer than the wait is asked after again. while session.running() { self.pause(REBOOT_EVERY); session = self.get(&format!("/v1/sessions/{id}"))?; @@ -464,20 +470,17 @@ pub fn survey(paths: &Paths) -> Vec { let hosts = Host::all(paths); let asked = runtime.block_on(futures_util::future::join_all(hosts.iter().map( |host| async move { - let password = std::env::var("BERTH_SUDO_PASSWORD").ok(); + let password = host.kept_sudo_password(paths); let link = Link::new(host.link(paths, password.as_deref()).ok()?); // A host that refused the login a short while ago is left out, // as one that does not answer is. if link.refused().is_some() { return None; } - let version = tokio::time::timeout( - std::time::Duration::from_secs(20), - link.get::("/v1/version"), - ) - .await - .ok()? - .ok()?; + let version = tokio::time::timeout(std::time::Duration::from_secs(20), link.version()) + .await + .ok()? + .ok()?; Some((host.name.clone(), version)) }, ))); @@ -529,7 +532,7 @@ mod tests { api: api::API, mode: "system".into(), applied: self.applied.clone(), - level: None, + host_type: None, })) } fn apply(&mut self, _out: &mut dyn Write) -> std::result::Result<(), String> { @@ -542,7 +545,11 @@ mod tests { } } } - fn task(&mut self, name: &str, _out: &mut dyn Write) -> std::result::Result { + fn run_action( + &mut self, + name: &str, + _out: &mut dyn Write, + ) -> std::result::Result { self.log.borrow_mut().push(format!("{} {name}", self.name)); Ok(match name { "update" => self.update_code, @@ -596,6 +603,7 @@ mod tests { version: version.into(), digest: version.into(), }], + ..Applied::default() } } @@ -815,7 +823,7 @@ mod tests { let log = Default::default(); let mut targets: Vec> = vec![Box::new(make(&log, 0, true, 3))]; let report = run(&mut targets, &reboot, &now, &mut vec![]); - assert_eq!(report.hosts[0].error, "its status task ended with 3"); + assert_eq!(report.hosts[0].error, "its status action ended with 3"); } #[test] @@ -835,14 +843,14 @@ mod tests { }; let (taken, skipped) = workstations(vec![ host("studio", json!({"builder": {}})), - host("dock", json!({"level": "controller"})), - host("pier", json!({"level": "reachable"})), + host("dock", json!({"type": "managed"})), + host("pier", json!({"type": "ssh"})), ]); assert_eq!(taken.len(), 1); assert_eq!(taken[0].name, "studio"); assert_eq!( skipped_note(&skipped).unwrap(), - "! Skipped 2 hosts that are not workstations: dock, pier" + "! Skipped 2 hosts that are neither nodes nor workstations: dock, pier" ); assert!(skipped_note(&[]).is_none()); } diff --git a/crates/berth-core/src/runner.rs b/crates/berth-core/src/runner.rs index 18f424d..5e4fdd0 100644 --- a/crates/berth-core/src/runner.rs +++ b/crates/berth-core/src/runner.rs @@ -6,23 +6,137 @@ //! //! What a job runs is one of three executors: a function of berth (applying //! a host's profile, a roll-out, a provisioning), a program (ansible, or a -//! plugin's provisioner), or a task of a host's controller. A task runs as +//! plugin's provisioner), or an action of a host's controller. An action runs as //! a session on the host, and its job copies the session's output while it //! runs, so the job stays readable when the host is not and after the host //! has forgotten the session. use crate::{Result, paths::Paths, profiles::Place, store}; +use berth_api::ops::Role; use serde_json::Value; use std::{ + cell::RefCell, + collections::HashMap, fs, io::{Read, Write}, - os::unix::fs::OpenOptionsExt, + os::unix::{fs::OpenOptionsExt, process::CommandExt}, path::{Path, PathBuf}, - process::{Command, Stdio}, - sync::Arc, + process::{Child, Command, Stdio}, + sync::{Arc, Mutex}, time::{Duration, Instant}, }; +// ── Stopping a job ────────────────────────────────────────────────────────── + +/// How long a stopped job's programs have to end after SIGTERM before +/// they are killed. +const GRACE: Duration = Duration::from_secs(10); + +/// What stops a running job: the process groups its programs run in, and +/// who asked it to stop. +#[derive(Default)] +pub struct Stop { + by: Mutex>, + groups: Mutex>, +} + +impl Stop { + /// Who asked the job to stop, once someone has. + pub fn asked(&self) -> Option { + self.by.lock().unwrap().clone() + } + + fn signal(&self, signal: i32) { + for group in self.groups.lock().unwrap().iter() { + // SAFETY: a signal to a process group of this job's, by number. + unsafe { + libc::kill(-(*group as i32), signal); + } + } + } + + /// Ask for the stop: its programs get SIGTERM, and SIGKILL when they + /// outlast the grace. Says whether this was the first to ask. + fn request(self: &Arc, by: &str) -> bool { + { + let mut asked = self.by.lock().unwrap(); + if asked.is_some() { + return false; + } + *asked = Some(by.to_owned()); + } + self.signal(libc::SIGTERM); + let stop = self.clone(); + std::thread::spawn(move || { + std::thread::sleep(GRACE); + stop.signal(libc::SIGKILL); + }); + true + } +} + +thread_local! { + /// The stop of the job this thread runs, when it runs one. + static CURRENT: RefCell>> = const { RefCell::new(None) }; +} + +/// Who asked the job this thread runs to stop, for work that looks +/// between its steps. +pub fn stopped() -> Option { + CURRENT.with(|current| current.borrow().as_ref().and_then(|stop| stop.asked())) +} + +/// Give a program a process group of its own when this thread runs a job, +/// so that stopping the job ends it with what it started. A program of a +/// berth that keeps no jobs stays in the terminal's group. +pub fn own_group(command: &mut Command) { + if CURRENT.with(|current| current.borrow().is_some()) { + command.process_group(0); + } +} + +/// A program of a job, ended with the job when it is stopped, for as long +/// as this is held. +pub struct Group(Option<(Arc, u32)>); + +impl Drop for Group { + fn drop(&mut self) { + if let Some((stop, pid)) = self.0.take() { + stop.groups.lock().unwrap().retain(|known| *known != pid); + } + } +} + +/// Have a program started with [`own_group`] stopped with the job this +/// thread runs. One started after the stop was asked for is ended at once. +pub fn watch(child: &Child) -> Group { + let Some(stop) = CURRENT.with(|current| current.borrow().clone()) else { + return Group(None); + }; + let pid = child.id(); + stop.groups.lock().unwrap().push(pid); + if stop.asked().is_some() { + // SAFETY: a signal to the process group of the program just started. + unsafe { + libc::kill(-(pid as i32), libc::SIGTERM); + } + } + Group(Some((stop, pid))) +} + +/// What came of asking a job to stop. +#[derive(Debug, PartialEq, Eq)] +pub enum Stopping { + /// It is being stopped now. + Asked, + /// Someone asked before; it is ending. + AskedBefore(String), + /// It has ended already. + Ended, + /// No job of that id runs in this process: it ran elsewhere or never was. + NotHere, +} + /// What a job is done to. #[derive(Debug, Clone, PartialEq, Eq)] pub enum Subject { @@ -275,6 +389,10 @@ impl Drop for Output<'_> { /// server's link and a test's double are each one. pub trait Controller: Send { fn get(&mut self, path: &str) -> std::result::Result; + /// End a session on the host, for a job that is stopped. + fn stop(&mut self, _id: &str) -> std::result::Result<(), String> { + Err("this controller cannot stop sessions".into()) + } } /// A program and what it is given: its standard output and error are the @@ -291,8 +409,11 @@ pub enum Work { /// A function of berth: its output goes to the writer, and it says /// whether it went well. Function(Box bool + Send>), + /// A function of berth that ends with an exit code of its own, such as + /// that of a command it ran. + Coded(Box i32 + Send>), Program(Box), - /// A session a task of a host's controller runs, by its id there, + /// A session an action of a host's controller runs, by its id there, /// followed until it ends. Session { controller: Box, @@ -313,6 +434,12 @@ pub struct Request { pub held: Option, /// Texts that never reach the output, such as a password the work is given. pub secrets: Vec, + /// The role that may start it, which may also stop it. + pub role: Role, + /// What it was started with that its record shows, such as the command + /// of `exec`. Made with [`berth_api::Argument::shown`] for an action's + /// arguments, so that a secret one never gets here. + pub arguments: Vec, } impl Request { @@ -329,6 +456,8 @@ impl Request { changes: true, held: None, secrets: vec![], + role: Role::Admin, + arguments: vec![], } } @@ -352,6 +481,7 @@ const PATIENCE: Duration = Duration::from_secs(300); fn execute(work: Work, out: &mut Output) -> Option { match work { Work::Function(function) => Some(if function(out) { 0 } else { 1 }), + Work::Coded(function) => Some(function(out)), Work::Program(program) => run_program(*program, out), Work::Session { controller, id } => follow(controller, &id, out), } @@ -384,11 +514,13 @@ pub fn run_program(program: Program, out: &mut Output) -> Option { }) .stdout(writer) .stderr(copy); + own_group(&mut command); let program_name = command.get_program().to_string_lossy().into_owned(); let mut child = match command.spawn() { Ok(child) => child, Err(error) => return fail(out, format!("{program_name}: {error}")), }; + let _group = watch(&child); // The command holds the pipe's other ends; they close when it is dropped. drop(command); if let (Some(input), Some(mut stdin)) = (input, child.stdin.take()) { @@ -412,7 +544,19 @@ pub fn run_program(program: Program, out: &mut Output) -> Option { /// Copy a session's output as it comes, until it has ended: its exit code. fn follow(mut controller: Box, id: &str, out: &mut Output) -> Option { let (mut since, mut answered) = (0u64, Instant::now()); + let mut ending = false; loop { + // A stopped job ends its session on the host, and follows it to its end. + if !ending && let Some(by) = stopped() { + ending = true; + if let Err(error) = controller.stop(id) { + let _ = writeln!( + out, + "\n✗ Stopped by {by}, but the host did not end the session {id}: {error}" + ); + return None; + } + } let asked = controller.get(&format!("/v1/sessions/{id}/output?since={since}")); let output: berth_api::Output = match asked .and_then(|value| serde_json::from_value(value).map_err(|error| error.to_string())) @@ -450,6 +594,9 @@ fn follow(mut controller: Box, id: &str, out: &mut Output) -> Op // ── The runner ────────────────────────────────────────────────────────────── +/// A running job's stop, with the role that may use it. +type Stoppable = (Role, Arc); + /// Told of a job when it starts and when it ends. pub type Notify = Arc; @@ -460,6 +607,10 @@ pub struct Runner { store: Arc, jobs_dir: PathBuf, notify: Notify, + /// The jobs this process runs now, which end with it. + running: Arc>>, + /// What stops each of them, with the role that may. + stops: Arc>>, } /// Where an installation keeps the output of the jobs it runs. @@ -510,7 +661,47 @@ impl Runner { store, jobs_dir, notify, + running: Arc::default(), + stops: Arc::default(), + } + } + + /// The role that may stop a job this process runs. + pub fn stop_role(&self, id: &str) -> Option { + self.stops.lock().unwrap().get(id).map(|(role, _)| *role) + } + + /// Stop a job this process runs, as `by` asks: its programs end, a + /// session of a host ends there, and its record says who stopped it. + pub fn stop(&self, id: &str, by: &str) -> Result { + // The record is asked first: a job that has just ended is still + // known here for a moment. + if self.store.job(id)?.is_some_and(|job| job.ended.is_some()) { + return Ok(Stopping::Ended); + } + let found = self.stops.lock().unwrap().get(id).cloned(); + let Some((_, stop)) = found else { + return Ok(Stopping::NotHere); + }; + // A job ends in the store a moment before it leaves the stops. + if self.store.job(id)?.is_some_and(|job| job.ended.is_some()) { + return Ok(Stopping::Ended); + } + if let Some(before) = stop.asked() { + return Ok(Stopping::AskedBefore(before)); } + // The record comes first: once its programs are signalled the job + // may end, and one that has ended takes no such note. + self.store.stop_job(id, by)?; + if !stop.request(by) { + return Ok(Stopping::AskedBefore(stop.asked().unwrap_or_default())); + } + Ok(Stopping::Asked) + } + + /// The jobs that run in this process now, as they started. + pub fn running(&self) -> Vec { + self.running.lock().unwrap().clone() } /// Start the work as a job, in a thread of its own, and say which job it @@ -549,6 +740,18 @@ impl Runner { subject, action, token: request.token.clone(), + stopped_by: None, + arguments: request + .arguments + .iter() + .filter(|argument| { + !request + .secrets + .iter() + .any(|secret| !secret.is_empty() && argument.contains(secret.as_str())) + }) + .cloned() + .collect(), }; let job = self.store.add_job(&asked)?; // Another caller with the same token was first. @@ -575,6 +778,7 @@ impl Runner { file, path, hidden: hidden(request.secrets.iter().map(String::as_str)), + role: request.role, })) } } @@ -595,6 +799,7 @@ pub struct Ticket { file: fs::File, path: PathBuf, hidden: Vec, + role: Role, } impl Ticket { @@ -605,6 +810,13 @@ impl Ticket { /// Run the work in a thread of its own: the job as it started. pub fn run(self, work: Work) -> store::Job { let job = self.job.clone(); + self.runner.running.lock().unwrap().push(job.clone()); + let stop = Arc::new(Stop::default()); + self.runner + .stops + .lock() + .unwrap() + .insert(job.id.clone(), (self.role, stop.clone())); std::thread::spawn(move || { let Ticket { runner, @@ -613,14 +825,32 @@ impl Ticket { mut file, path, hidden, + .. } = self; - let exit_code = { + CURRENT.with(|current| *current.borrow_mut() = Some(stop.clone())); + let mut exit_code = { let mut out = Output::new(&mut file, hidden); out.log = Some(path); - execute(work, &mut out) + let code = execute(work, &mut out); + // A stopped job says so last, and ends as one that was interrupted. + if let Some(by) = stop.asked() { + let _ = writeln!(out, "✗ Stopped by {by}"); + } + code }; + CURRENT.with(|current| *current.borrow_mut() = None); + if stop.asked().is_some() { + exit_code = Some(130); + } drop(lock); + // Ended in the store before it leaves the stops, so that a stop asked between finds one or the other. runner.end(&job.id, exit_code); + runner.stops.lock().unwrap().remove(&job.id); + runner + .running + .lock() + .unwrap() + .retain(|running| running.id != job.id); }); job } @@ -806,6 +1036,8 @@ mod tests { let mut request = Request::new(Subject::Installation, "rollout"); request.host = "studio, lab".into(); request.secrets = vec!["s3cret-value".into()]; + // An argument that holds a secret never reaches the record. + request.arguments = vec!["only=studio".into(), "key=s3cret-value".into()]; let work = Work::Function(Box::new(|out: &mut Output| { writeln!(out, "using s3cret-value").unwrap(); assert!( @@ -823,6 +1055,7 @@ mod tests { assert_eq!(job.exit_code, Some(1)); assert_eq!(log(&job), "using [secret]\n"); assert_eq!(job.host, "studio, lab"); + assert_eq!(job.arguments, ["only=studio"]); // Told when it started and when it ended. let told = world.told.lock().unwrap(); assert_eq!(told.len(), 2); @@ -868,6 +1101,52 @@ mod tests { assert!(log(&job).contains("/nonexistent/program")); } + #[test] + fn a_stopped_job_ends_its_programs_and_says_who_stopped_it() { + let world = world(); + // A program that starts another, which the stop ends as well. + let mut command = Command::new("sh"); + // It takes a moment to end once stopped, as a program that cleans up does. + command.args([ + "-c", + "trap 'sleep 0.5; exit 1' TERM; echo started; sleep 30 & wait", + ]); + let work = Work::Program(Box::new(Program { + command, + input: None, + files: vec![], + })); + let mut request = Request::new(Subject::Host("studio".into()), "apply"); + request.role = Role::Operate; + let job = world.runner.start(request, work).unwrap(); + assert_eq!(world.runner.stop_role(&job.id), Some(Role::Operate)); + for _ in 0..200 { + if fs::read_to_string(&job.output_path) + .unwrap_or_default() + .contains("started") + { + break; + } + std::thread::sleep(Duration::from_millis(10)); + } + let begun = Instant::now(); + assert_eq!(world.runner.stop(&job.id, "ops").unwrap(), Stopping::Asked); + assert_eq!( + world.runner.stop(&job.id, "olga").unwrap(), + Stopping::AskedBefore("ops".into()) + ); + let ended = ended(&world, &job.id); + assert!(begun.elapsed() < GRACE, "it waited for the grace"); + assert_eq!(ended.exit_code, Some(130)); + assert_eq!(ended.stopped_by.as_deref(), Some("ops")); + assert!(log(&ended).contains("✗ Stopped by ops"), "{}", log(&ended)); + assert_eq!(world.runner.stop(&job.id, "ops").unwrap(), Stopping::Ended); + assert_eq!( + world.runner.stop("jnothing", "ops").unwrap(), + Stopping::NotHere + ); + } + /// A host's controller that has one session, whose output comes in pieces. struct Fake { pieces: Vec<&'static str>, @@ -904,7 +1183,7 @@ mod tests { } #[test] - fn a_task_of_a_host_is_a_job_that_copies_its_session() { + fn an_action_of_a_host_is_a_job_that_copies_its_session() { let world = world(); let asked = Arc::new(Mutex::new(vec![])); let controller = Box::new(Fake { @@ -913,7 +1192,7 @@ mod tests { failures: 1, }); let mut request = Request::new(Subject::Host("studio".into()), "status"); - // A task runs beside the host's other sessions, without the lock. + // An action runs beside the host's other sessions, without the lock. request.changes = false; let work = Work::Session { controller, diff --git a/crates/berth-core/src/seed.rs b/crates/berth-core/src/seed.rs index 01d7dc8..97c9ff4 100644 --- a/crates/berth-core/src/seed.rs +++ b/crates/berth-core/src/seed.rs @@ -19,6 +19,8 @@ pub struct Seed { /// The release archive the machine is given instead of downloading /// one, when it was asked for. pub archive: Option, + /// Until when the seed's key joins the tailnet, for a host that joins it. + pub tailnet: Option, } pub struct Archive { @@ -34,7 +36,11 @@ pub struct Archive { pub fn cloud(paths: &Paths, store: &Store, name: &str, delivered: bool) -> Result { let host = Host::load(paths, name)?; // A seed installs a whole workstation. - crate::level::guard(&host, crate::level::Capability::Apply)?; + crate::host_type::guard(&host, crate::host_type::Capability::Seed)?; + // Nothing is published or made for a host that could not join. + if let Some(problem) = tailnet::problem(paths, &host.config, name) { + return Err(crate::Refused(problem).into()); + } let source = paths.source()?; let dirs = apply::plugin_dirs(&plugins::found( &source, @@ -51,13 +57,14 @@ pub fn cloud(paths: &Paths, store: &Store, name: &str, delivered: bool) -> Resul profile["release_url"] = json!(url); host.place.save(paths, &profile)?; } - // The seed is rendered from a private copy of the profile, which holds a - // new enrollment key for a host that joins the tailnet automatically. + // The seed is rendered from a private copy of the profile, and a host + // that joins the tailnet gets a new key in a private file of its own. let saved: Value = host .place .load(paths)? .ok_or_else(|| format!("No host {name}"))?; - let mut config = named(tailnet::seed_profile(paths, &saved, name)?, name); + let key = tailnet::seed_key(paths, &saved, name)?; + let mut config = named(saved.clone(), name); crate::kept_profile::for_seed(&mut config, &host.place.title(), &saved); let seeded = tailnet::profile_file(&config)?; // The machine's operator takes these keys, so that berth can always reach it. @@ -75,6 +82,9 @@ pub fn cloud(paths: &Paths, store: &Store, name: &str, delivered: bool) -> Resul if delivered { command.env("BERTH_RELEASE_DELIVERED", "1"); } + if let Some(key) = &key { + key.pass(&mut command); + } // A release that is not public: the seed carries the login of the registry it is in. let private = host.config["release_private"].as_bool().unwrap_or(false); let login = match host.config["release_login"].as_str() { @@ -109,7 +119,11 @@ pub fn cloud(paths: &Paths, store: &Store, name: &str, delivered: bool) -> Resul true => Some(archive(&build, name)?), false => None, }; - Ok(Seed { user_data, archive }) + Ok(Seed { + user_data, + archive, + tailnet: key.as_ref().map(tailnet::SeedKey::note), + }) } /// A copy of the release archive that was just built, for this host alone, @@ -157,3 +171,105 @@ pub fn named(mut config: Value, name: &str) -> Value { } config } + +/// The unattended installer of a saved host: an ISO built from Ubuntu's +/// installer with the host's seed in it, ready to write to a USB drive. +pub struct Installer { + /// The builder, run in the provisioning source. + pub command: Command, + /// The private copy of the profile it reads, removed once it has run. + pub profile: tempfile::NamedTempFile, + /// What it joins the tailnet with, read by the command while it runs. + pub key: Option, + /// The ISO it writes. + pub iso: PathBuf, +} + +/// The builder of `name`'s installer from Ubuntu's ISO at `ubuntu`, which +/// is checked against the profile's `iso_sha256` when it has one. +pub fn installer(paths: &Paths, name: &str, ubuntu: &std::path::Path) -> Result { + let host = Host::load(paths, name)?; + crate::host_type::guard(&host, crate::host_type::Capability::Apply)?; + let source = paths.source()?; + let dirs = apply::plugin_dirs(&plugins::found( + &source, + &plugins::own_dir(&paths.config_dir), + )); + let saved = host.config.clone(); + let key = tailnet::seed_key(paths, &saved, name)?; + let mut config = named(saved.clone(), name); + crate::kept_profile::for_seed(&mut config, &host.place.title(), &saved); + let hostname = config["hostname"].as_str().unwrap_or(name).to_owned(); + let profile = tailnet::profile_file(&config)?; + let own = crate::keys::operator_keys(paths, &saved)?; + let build = paths.build_dir(); + let mut command = Command::new("python3"); + command + .current_dir(&source) + .args([layout::BUILDER, "iso", "--config"]) + .arg(profile.path()) + .arg("--iso") + .arg(ubuntu) + .arg("--overwrite") + .env("DEVBOX_BUILD_DIR", &build) + .env("BERTH_PLUGINS", dirs.to_string()) + .env("BERTH_SSH_PUBLIC_KEY", own.join("\n")) + .stdin(std::process::Stdio::null()); + if let Some(key) = &key { + key.pass(&mut command); + } + if let Some(sha) = saved["builder"]["iso_sha256"] + .as_str() + .filter(|sha| !sha.is_empty()) + { + command.args(["--sha256", sha]); + } + Ok(Installer { + command, + profile, + key, + iso: build.join(format!("{hostname}-autoinstall.iso")), + }) +} + +/// Ubuntu's installer ISO a host's installer is built from: the one its +/// profile names, or none yet. +pub fn ubuntu_iso(config: &Value) -> Option { + config["builder"]["iso"] + .as_str() + .filter(|iso| !iso.is_empty()) + .map(crate::discovery::expand_path) +} + +/// The builder that downloads Ubuntu's current installer ISO into the ISO +/// folder and checks it, printing its path last. +pub fn fetch_iso(paths: &Paths) -> Result { + let mut command = Command::new("python3"); + command + .current_dir(paths.source()?) + .args([layout::BUILDER, "fetch-iso", "--dir"]) + .arg(paths.iso_dir()) + .stdin(std::process::Stdio::null()); + Ok(command) +} + +/// `sudo` writing the ISO to a whole USB drive, which it erases. With a +/// password it is read from standard input, and else sudo must need none. +pub fn write_usb( + paths: &Paths, + iso: &std::path::Path, + device: &str, + password: bool, +) -> Result { + let mut command = Command::new("sudo"); + match password { + true => command.args(["-S", "-p", ""]), + false => command.arg("-n"), + }; + command + .current_dir(paths.source()?) + .args(["bash", layout::WRITE_USB]) + .arg(iso) + .arg(device); + Ok(command) +} diff --git a/crates/berth-core/src/server.rs b/crates/berth-core/src/server.rs index bca5412..09eefab 100644 --- a/crates/berth-core/src/server.rs +++ b/crates/berth-core/src/server.rs @@ -1,25 +1,9 @@ -//! A `berth serve` somewhere else, as this berth's way to its hosts. With the -//! `server` setting (or `BERTH_SERVER`) berth keeps no hosts of its own: it -//! shows the server's and reaches them through the server's portal. -//! -//! - `http://HOST:PORT`, `https://HOST[:PORT]`: over TCP, with the token -//! - `unix:/path/to/serve.sock`: the socket of a `berth serve` on this machine +//! Plain HTTP and HTTPS requests of berth's own: an OAuth issuer's keys, a +//! resource's page, a release. -use crate::{Result, paths::Paths, store::kind}; -use berth_api::{ - client::{self, Error}, - ops, - portal::Portal, -}; -use serde_json::{Value, json}; +use berth_api::client::{self, Error}; use std::sync::Arc; -use tokio::net::{TcpStream, UnixStream}; - -#[derive(Debug, Clone, PartialEq)] -pub struct Server { - pub url: String, - pub token: Option, -} +use tokio::net::TcpStream; fn transport(error: impl std::fmt::Display) -> Error { Error::Transport(error.to_string()) @@ -133,344 +117,6 @@ pub async fn fetch(url: &str) -> client::Result> { } } -impl Server { - /// The server this berth is set to use, if any. The environment wins. - pub fn configured(paths: &Paths) -> Option { - // Local settings only, so that finding the server never opens the store. - let settings = paths.local_settings().unwrap_or_default(); - let setting = |key: &str| settings.get(key).and_then(Value::as_str).map(str::to_owned); - let variable = |name: &str| std::env::var(name).ok().filter(|value| !value.is_empty()); - let url = variable("BERTH_SERVER").or_else(|| setting("server"))?; - Some(Server { - url: url.trim_end_matches('/').to_owned(), - token: variable("BERTH_TOKEN").or_else(|| setting("server_token")), - }) - } - - /// Open the server's portal, with the configured token if there is one. - pub async fn portal(&self) -> client::Result { - let token = self.token.as_deref().unwrap_or(""); - if let Some(path) = self.url.strip_prefix("unix:") { - let stream = UnixStream::connect(path) - .await - .map_err(|error| Error::Transport(format!("{path}: {error}")))?; - let socket = client::websocket(stream, ops::PORTAL, "").await?; - return Ok(Portal::over(socket)); - } - let (host, port, secure) = parts(&self.url)?; - let name = host.trim_matches(['[', ']']).to_owned(); - let stream = TcpStream::connect((name.as_str(), port)) - .await - .map_err(|error| Error::Transport(format!("{}: {error}", self.url)))?; - let authority = format!("{host}:{port}"); - if !secure { - let socket = client::websocket_at(stream, &authority, ops::PORTAL, token).await?; - return Ok(Portal::over(socket)); - } - let stream = secure_stream(&name, stream) - .await - .map_err(|error| Error::Transport(format!("{}: {error}", self.url)))?; - let socket = client::websocket_at(stream, &authority, ops::PORTAL, token).await?; - Ok(Portal::over(socket)) - } - - /// One operation of the server itself, waited for. - pub fn call(&self, op: &str, args: Value, data: Option) -> Result { - crate::link::runtime()?.block_on(async { - let portal = self.portal().await?; - self.ask(&portal, op, args, data).await - }) - } - - /// One operation on an open portal. A server that lacks the operation is - /// an older one, which is said in words, with its version when it tells. - pub async fn ask( - &self, - portal: &Portal, - op: &str, - args: Value, - data: Option, - ) -> Result { - match portal.call(op, None, args, data).await { - Err(error) if missing_operation(&error).is_some() => { - let version = portal - .call("version", None, Value::Null, None) - .await - .ok() - .and_then(|version| version["version"].as_str().map(str::to_owned)); - Err(unsupported(&self.url, version.as_deref(), &error).into()) - } - other => Ok(other?), - } - } - - /// The server's hosts: name, address and whether each answers. - pub async fn hosts(&self) -> client::Result> { - let hosts = self - .portal() - .await? - .call("hosts.list", None, Value::Null, None) - .await?; - Ok(hosts.as_array().cloned().unwrap_or_default()) - } - - /// The scope of the store that holds what is known of this server: its - /// registries, the names of its secrets, and what the last mirror found. - pub fn scope(&self) -> String { - format!("served:{}", self.url) - } - - /// Keep what the server has in the store, under the server's scope: a - /// profile for each of its hosts, its registries and the names of its - /// secrets. Whoever may set the - /// server up gets the profiles as they are there, and saving one sends it - /// back; anyone else gets profiles that only say where the hosts are. - pub fn mirror(&self, paths: &Paths) -> Result<()> { - let fetched = crate::link::runtime()?.block_on(self.fetch())?; - self.keep(paths, fetched) - } - - fn keep(&self, paths: &Paths, fetched: Fetched) -> Result<()> { - let mut state = serde_json::Map::new(); - for (host, _) in &fetched.hosts { - if let Some(name) = host["name"].as_str().filter(|name| safe(name)) { - state.insert( - name.to_owned(), - json!({ - "reachable": host["reachable"] == true, - "error": host["error"], - "version": host["version"], - "rotating": host["rotating"] == true, - }), - ); - } - } - // One transaction, so a screen that reads meanwhile sees the old - // mirror or the new one, not half of one. - let mut items: Vec<(&str, String, Vec)> = vec![]; - // A whole profile, so every screen has what it expects; only where the - // host is and that it is the server's are its own. - let mut example: Option = None; - for (host, profile) in fetched.hosts { - let Some(name) = host["name"].as_str().filter(|name| safe(name)) else { - continue; - }; - let mut profile = match profile { - Some(profile) if profile.is_object() => profile, - _ => { - if example.is_none() { - let file = paths.source()?.join(crate::paths::layout::EXAMPLE); - example = Some(serde_json::from_slice(&std::fs::read(file)?)?); - } - let mut profile = example.clone().unwrap_or_default(); - profile["access"] = json!({"address": host["address"], "sudo": "none"}); - profile["builder"]["served_only"] = json!(true); - profile - } - }; - profile["builder"]["served"] = json!(self.url); - items.push(( - kind::PROFILE, - name.to_owned(), - profile.to_string().into_bytes(), - )); - } - for (name, registry) in fetched.registries.as_object().into_iter().flatten() { - items.push(( - kind::REGISTRY, - name.clone(), - registry.to_string().into_bytes(), - )); - } - for name in fetched.secrets { - items.push((kind::SECRET, name, KEPT.as_bytes().to_vec())); - } - let state = json!({"version": fetched.version, "hosts": state}); - items.push((kind::MIRROR, "state".into(), state.to_string().into_bytes())); - items.push(( - kind::MIRROR, - "access".into(), - fetched.access.to_string().into_bytes(), - )); - let items: Vec<(&str, &str, Vec)> = items - .iter() - .map(|(kind, name, value)| (*kind, name.as_str(), value.clone())) - .collect(); - crate::store::shared(paths)?.replace( - &self.scope(), - &[kind::PROFILE, kind::REGISTRY, kind::SECRET, kind::MIRROR], - &items, - )?; - Ok(()) - } - - /// Keep one host's profile as the server has it now, for whoever may set - /// the server up: a job there, such as creating the host's machine, may - /// have changed it since the last mirror. - pub fn mirror_host(&self, paths: &Paths, name: &str) -> Result<()> { - let mut profile = self.call("hosts.profile", json!({"name": name}), None)?; - if !profile.is_object() || !safe(name) { - return Ok(()); - } - profile["builder"]["served"] = json!(self.url); - crate::store::shared(paths)? - .write_json(kind::PROFILE, &self.scope(), name, &profile) - .map_err(transport)?; - Ok(()) - } - - /// A row the last mirror wrote, as JSON; null when there is none. - fn mirrored(&self, paths: &Paths, name: &str) -> Value { - crate::store::shared(paths) - .ok() - .and_then(|store| store.read_json(kind::MIRROR, &self.scope(), name).ok()) - .flatten() - .unwrap_or(Value::Null) - } - - /// What the last mirror found out about the server and its hosts. - pub fn known(&self, paths: &Paths) -> Known { - let state = self.mirrored(paths, "state"); - let hosts = state["hosts"] - .as_object() - .into_iter() - .flatten() - .map(|(name, host)| { - let text = |key: &str| host[key].as_str().map(str::to_owned); - ( - name.clone(), - Reach { - reachable: host["reachable"] == true, - error: text("error"), - version: text("version"), - rotating: host["rotating"] == true, - }, - ) - }) - .collect(); - Known { - version: state["version"].as_str().map(str::to_owned), - hosts, - } - } - - async fn fetch(&self) -> Result { - let portal = self.portal().await?; - let call = |op: &'static str, args: Value| self.ask(&portal, op, args, None); - let listed = call("hosts.list", Value::Null).await?; - // A server too old to tell its version is shown as not knowing it. - let version = call("version", Value::Null) - .await - .ok() - .and_then(|version| version["version"].as_str().map(str::to_owned)); - // A server from before roles were there lets its token do everything - // it has, which is not set-up. - let access = call("access", Value::Null) - .await - .unwrap_or_else(|_| json!({"role": "operate", "setup": false})); - let admin = access["role"] == "admin" && access["setup"] == true; - let mut hosts = vec![]; - for host in listed.as_array().cloned().unwrap_or_default() { - let profile = match (admin, host["name"].as_str()) { - (true, Some(name)) => call("hosts.profile", json!({"name": name})).await.ok(), - _ => None, - }; - hosts.push((host, profile)); - } - let (mut registries, mut secrets) = (json!({}), vec![]); - if admin { - registries = call("registries.list", Value::Null).await?["registries"].take(); - secrets = call("secrets.list", Value::Null).await?["names"] - .as_array() - .into_iter() - .flatten() - .filter_map(|name| name.as_str().map(str::to_owned)) - .collect(); - } - Ok(Fetched { - hosts, - registries, - secrets, - access, - version, - }) - } - - /// Whether this berth may set the server up, as the last mirror found. - pub fn admin(&self, paths: &Paths) -> bool { - let access = self.mirrored(paths, "access"); - access["role"] == "admin" && access["setup"] == true - } - - /// Give the server a profile to keep under a name. What marks it as the - /// server's stays here. - pub fn save_host(&self, name: &str, profile: &Value) -> Result<()> { - let mut profile = profile.clone(); - if let Some(builder) = profile["builder"].as_object_mut() { - builder.remove("served"); - builder.remove("served_only"); - } - self.call("hosts.save", json!({"name": name}), Some(profile))?; - Ok(()) - } -} - -/// What stands for a secret's value here: the server keeps it. -pub const KEPT: &str = "(kept on the berth server)"; - -struct Fetched { - /// What `hosts.list` says of each host, with its profile for an admin. - hosts: Vec<(Value, Option)>, - registries: Value, - secrets: Vec, - access: Value, - /// The server's own version, when it tells. - version: Option, -} - -/// Whether one host answered the server when the mirror was made. -#[derive(Debug, Clone, PartialEq, Default)] -pub struct Reach { - pub reachable: bool, - /// Why it did not. - pub error: Option, - /// The version of its berthd. - pub version: Option, - /// A rotation of a key it logs in with has not reached it yet. - pub rotating: bool, -} - -/// What a mirror kept of the server beside the profiles. -#[derive(Debug, Clone, PartialEq, Default)] -pub struct Known { - /// The version of the server, when it said. - pub version: Option, - pub hosts: std::collections::BTreeMap, -} - -/// The operation a server does not have, when that is what the error says. -pub fn missing_operation(error: &Error) -> Option<&str> { - match error { - Error::Api { code, message, .. } if code == "not_found" => { - message.strip_prefix("No operation ") - } - _ => None, - } -} - -/// What to tell of an operation the server does not have. -pub fn unsupported(url: &str, version: Option<&str>, error: &Error) -> String { - let op = missing_operation(error).unwrap_or("this"); - match version { - Some(version) => format!( - "The berth server at {url} runs {version}, which does not have {op} yet. Update the berth server." - ), - None => format!( - "The berth server at {url} does not have {op} yet, so it is older than this berth. Update the berth server." - ), - } -} - /// A name that is a file name and nothing else. pub fn safe(name: &str) -> bool { !name.is_empty() @@ -503,122 +149,4 @@ mod tests { assert!(parts("http://").is_err()); assert!(safe("studio-1") && !safe("../x") && !safe(".x") && !safe("a/b")); } - - #[test] - fn a_missing_operation_is_said_in_words() { - let error = Error::api(berth_api::code::NOT_FOUND, "No operation instances.get"); - assert_eq!(missing_operation(&error), Some("instances.get")); - assert_eq!( - unsupported("http://192.0.2.7:7777", Some("0.3.0"), &error), - "The berth server at http://192.0.2.7:7777 runs 0.3.0, which does not have instances.get yet. Update the berth server." - ); - assert!( - unsupported("http://192.0.2.7:7777", None, &error).contains("older than this berth") - ); - // Another not-found is not this. - let host = Error::api(berth_api::code::NOT_FOUND, "No host studio"); - assert_eq!(missing_operation(&host), None); - assert_eq!( - missing_operation(&Error::Transport("No operation x".into())), - None - ); - } - - fn scratch() -> (tempfile::TempDir, Paths, Server) { - let directory = tempfile::tempdir().unwrap(); - let paths = Paths { - config_dir: directory.path().join("config"), - data_dir: directory.path().join("data"), - }; - let server = Server { - url: "http://192.0.2.7:7777".into(), - token: None, - }; - (directory, paths, server) - } - - fn fetched(registry: &str, secret: &str, admin: bool) -> Fetched { - Fetched { - hosts: vec![ - ( - json!({"name": "studio", "reachable": true, "version": "0.3.0"}), - Some( - json!({"hostname": "studio", "builder": {}, "access": {"address": "192.0.2.20"}}), - ), - ), - ( - json!({"name": "attic", "reachable": false, "error": "timed out"}), - Some(json!({"hostname": "attic", "builder": {}})), - ), - ], - registries: json!({registry: {"type": "npm", "url": "https://npm.example"}}), - secrets: vec![secret.to_owned(), format!("registry/{registry}")], - access: json!({"role": if admin { "admin" } else { "operate" }, "setup": true}), - version: Some("0.3.0".into()), - } - } - - #[test] - fn a_mirror_keeps_who_answers() { - let (_directory, paths, server) = scratch(); - assert_eq!(server.known(&paths), Known::default()); - assert!(!server.admin(&paths)); - server.keep(&paths, fetched("npm", "github", true)).unwrap(); - let known = server.known(&paths); - assert_eq!(known.version.as_deref(), Some("0.3.0")); - assert!(known.hosts["studio"].reachable); - assert_eq!(known.hosts["attic"].error.as_deref(), Some("timed out")); - assert!(server.admin(&paths)); - // The profiles are in the store, under the server's scope, and no file is written. - let studio = crate::profiles::Place::stored(&server.scope(), "studio") - .unwrap() - .load(&paths) - .unwrap() - .unwrap(); - assert_eq!(studio["builder"]["served"], json!(server.url)); - assert_eq!(studio["access"]["address"], "192.0.2.20"); - assert!(!paths.data_dir.join("served").exists()); - } - - #[test] - fn a_mirror_replaces_its_scope_and_leaves_the_own_alone() { - let (_directory, paths, server) = scratch(); - let store = crate::store::shared(&paths).unwrap(); - let own = crate::store::OWN; - store - .write(kind::SECRET, own, "github", b"ghp-made-up") - .unwrap(); - store.write(kind::REGISTRY, own, "npm", b"{}").unwrap(); - store - .write(kind::PROFILE, own, "studio", br#"{"builder": {}}"#) - .unwrap(); - server.keep(&paths, fetched("npm", "github", true)).unwrap(); - let profiles = |scope: &str| store.names(kind::PROFILE, scope).unwrap(); - assert_eq!(profiles(&server.scope()), ["attic", "studio"]); - server - .keep(&paths, fetched("hub", "deploy", false)) - .unwrap(); - let scope = server.scope(); - assert_eq!(scope, "served:http://192.0.2.7:7777"); - assert_eq!(store.names(kind::REGISTRY, &scope).unwrap(), ["hub"]); - assert_eq!( - store.names(kind::SECRET, &scope).unwrap(), - ["deploy", "registry/hub"] - ); - for (_, value) in store.entries(kind::SECRET, &scope).unwrap() { - assert_eq!(value, KEPT.as_bytes()); - } - assert!(!server.admin(&paths)); - assert_eq!( - store.read(kind::SECRET, own, "github").unwrap().unwrap(), - b"ghp-made-up" - ); - assert_eq!(store.names(kind::REGISTRY, own).unwrap(), ["npm"]); - // The host of this installation's own stays as it was. - assert_eq!(profiles(own), ["studio"]); - assert_eq!( - store.read(kind::PROFILE, own, "studio").unwrap().unwrap(), - br#"{"builder": {}}"# - ); - } } diff --git a/crates/berth-core/src/store/installation.rs b/crates/berth-core/src/store/installation.rs index 2242d79..7935528 100644 --- a/crates/berth-core/src/store/installation.rs +++ b/crates/berth-core/src/store/installation.rs @@ -75,8 +75,31 @@ impl Installation { } /// Open the store with its keys, finishing or undoing a key change that - /// was interrupted. + /// was interrupted. A SQLite store an older berth wrote is copied first + /// ([`keep_before_converting`]). pub fn open(&self) -> crate::Result { + let _held = self.hold_converting()?; + self.open_held() + } + + /// Held while a store is copied and converted, so that two berths copy + /// it once, before either converts it. + fn hold_converting(&self) -> crate::Result> { + match &self.target { + Target::Sqlite(file) => Ok(Some(wait_lock( + &beside(file, "convert.lock"), + "converting the store", + )?)), + Target::Postgres(_) => Ok(None), + } + } + + fn open_held(&self) -> crate::Result { + if let Target::Sqlite(file) = &self.target + && let Some(said) = keep_before_converting(file)? + { + eprintln!("berth: {said}"); + } let store = Store::connect_trusting(&self.target, self.ca.as_deref())?; let keys = self.keys(&store)?; Ok(store.with_keys(keys, Some(self.key_file.clone()))) @@ -169,13 +192,192 @@ pub fn shared(paths: &Paths) -> crate::Result> { if let Some(store) = opened.get(&key) { return Ok(store.clone()); } + let store = Arc::new(open_converting(paths)?); + opened.insert(key, store.clone()); + Ok(store) +} + +/// Open the installation's store and take in what older versions kept. A +/// store an older berth wrote is copied first, as it was, for that berth to +/// go back to: once converted it reads it no more. +pub fn open_converting(paths: &Paths) -> crate::Result { let installation = Installation::of(paths); - let store = Arc::new(installation.open()?); + // Held until the imports are done too, which convert it as well. + let _held = installation.hold_converting()?; + let store = installation.open_held()?; + if let Target::Postgres(_) = installation.target + && let Some(old) = older_writer(&store)? + { + eprintln!( + "berth: berth {} converts the store in PostgreSQL that berth {old} wrote, which berth {old} cannot read afterwards. Going back needs a dump made before, with pg_dump: {}", + berth_api::VERSION, + berth_api::compat::UPGRADE_DOCS + ); + } crate::import::run(paths, &installation, &store)?; - opened.insert(key, store.clone()); Ok(store) } +/// The version of berth that last took the store in, when it is older than +/// this one: a marker of this berth's imports is missing. +fn older_writer(store: &Store) -> crate::Result> { + let names = store.names(super::kind::META, super::OWN)?; + // A store no berth took in yet is a new one. + if !names.iter().any(|name| name == crate::import::MARKER) { + return Ok(None); + } + if crate::import::MARKERS + .iter() + .all(|marker| names.iter().any(|name| name == marker)) + { + return Ok(None); + } + let mut versions: Vec = vec![]; + for name in &names { + if let Some(value) = store.read_json(super::kind::META, super::OWN, name)? + && let Some(version) = value["version"].as_str() + { + versions.push(version.to_owned()); + } + } + versions.sort_by_key(|version| version_key(version)); + Ok(Some(versions.pop().unwrap_or_else(|| "0.5".into()))) +} + +fn version_key(version: &str) -> Vec { + version + .split(|c: char| !c.is_ascii_digit()) + .filter_map(|part| part.parse().ok()) + .collect() +} + +/// Before this berth converts a SQLite store an older one wrote, for the +/// first time, a consistent copy of it goes beside it, named for the schema +/// and the berth it had, in place of any copy made before. The key file is +/// not copied: the copy opens with the same one. Says what it did, or why +/// not, and nothing when the store needs no conversion. +pub fn keep_before_converting(file: &Path) -> crate::Result> { + use rusqlite::{Connection, OpenFlags}; + if !file.is_file() { + return Ok(None); + } + let db = Connection::open_with_flags( + file, + OpenFlags::SQLITE_OPEN_READ_WRITE | OpenFlags::SQLITE_OPEN_NO_MUTEX, + )?; + db.busy_timeout(std::time::Duration::from_secs(5))?; + let schema: i64 = db.query_row("PRAGMA user_version", [], |row| row.get(0))?; + // A file no berth wrote yet, or one of a newer berth, which opening refuses. + if schema == 0 || schema as usize > super::sqlite::MIGRATIONS.len() { + return Ok(None); + } + // A sealed store's values are not text, and then say no version. + let markers: Vec<(String, Vec)> = match db.prepare( + "SELECT name, value FROM state WHERE kind = 'meta' AND scope = '' AND name LIKE 'import:%'", + ) { + Ok(mut statement) => statement + .query_map([], |row| Ok((row.get(0)?, row.get(1)?)))? + .collect::>()?, + // A schema older than the table of state. + Err(_) => vec![], + }; + // An older schema, or a store an older berth took in: one that has the + // first import's marker and lacks one of this berth's. A store with none + // was made by this berth, as a copy or before its first import. + let marked = |marker: &str| markers.iter().any(|(name, _)| name == marker); + let older = (schema as usize) < super::sqlite::MIGRATIONS.len() + || (marked(crate::import::MARKER) && !crate::import::MARKERS.iter().all(|m| marked(m))); + if !older { + return Ok(None); + } + let mut versions: Vec = markers + .iter() + .filter_map(|(_, value)| { + serde_json::from_slice::(value).ok()?["version"] + .as_str() + .map(str::to_owned) + }) + .collect(); + versions.sort_by_key(|version| version_key(version)); + let old = versions.pop(); + let suffix = match &old { + Some(version) => format!("berth-{version}-schema-{schema}"), + None => format!("schema-{schema}"), + }; + let copy = beside(file, &suffix); + let dir = file.parent().unwrap_or(Path::new(".")); + let wal = PathBuf::from(format!("{}-wal", file.display())); + let size = fs::metadata(file)?.len() + fs::metadata(wal).map_or(0, |wal| wal.len()); + let named = match &old { + Some(version) => format!("berth {version}"), + None => "the older berth".to_owned(), + }; + if free_space(dir).is_some_and(|free| free < size + size / 10) { + return Ok(Some(format!( + "{} has no room for a copy of the store ({size} bytes), so none was made before berth {} converts it. {named} cannot read it afterwards: {}", + dir.display(), + berth_api::VERSION, + berth_api::compat::UPGRADE_DOCS + ))); + } + // Made empty and private first, as the copy holds what the store does. + let partial = beside(file, &format!("{suffix}.partial")); + let _ = fs::remove_file(&partial); + { + use std::os::unix::fs::OpenOptionsExt; + fs::OpenOptions::new() + .write(true) + .create_new(true) + .mode(0o600) + .open(&partial)?; + } + if let Err(error) = db.execute("VACUUM INTO ?1", [partial.to_string_lossy()]) { + let _ = fs::remove_file(&partial); + return Ok(Some(format!( + "No copy of the store was made before berth {} converts it ({error}). {named} cannot read it afterwards: {}", + berth_api::VERSION, + berth_api::compat::UPGRADE_DOCS + ))); + } + drop(db); + fs::rename(&partial, ©)?; + // Only the latest copy is kept. + let prefix = format!( + "{}.", + file.file_name().unwrap_or_default().to_string_lossy() + ); + for entry in fs::read_dir(dir)?.flatten() { + let name = entry.file_name().to_string_lossy().into_owned(); + let Some(rest) = name.strip_prefix(&prefix) else { + continue; + }; + let earlier = (rest.starts_with("berth-") || rest.starts_with("schema-")) + && rest.contains("schema-") + && !rest.ends_with(".partial"); + if earlier && entry.path() != copy { + let _ = fs::remove_file(entry.path()); + } + } + Ok(Some(format!( + "before berth {} converts the store, a copy of it as {named} left it is in {}. {named} needs that copy to go back: {}", + berth_api::VERSION, + copy.display(), + berth_api::compat::UPGRADE_DOCS + ))) +} + +/// The bytes free for this user in `dir`, when the system says. +fn free_space(dir: &Path) -> Option { + use std::os::unix::ffi::OsStrExt; + let path = std::ffi::CString::new(dir.as_os_str().as_bytes()).ok()?; + let mut stat: libc::statvfs = unsafe { std::mem::zeroed() }; + // SAFETY: the path is a C string and the struct is written by the call. + if unsafe { libc::statvfs(path.as_ptr(), &mut stat) } != 0 { + return None; + } + Some(stat.f_bavail as u64 * stat.f_frsize as u64) +} + #[cfg(test)] mod tests { use super::*; diff --git a/crates/berth-core/src/store/mod.rs b/crates/berth-core/src/store/mod.rs index f64c718..79a7799 100644 --- a/crates/berth-core/src/store/mod.rs +++ b/crates/berth-core/src/store/mod.rs @@ -9,7 +9,10 @@ mod sqlite; #[cfg(test)] mod suite; -pub use installation::{Installation, default_file, hold_lock, shared, wait_lock}; +pub use installation::{ + Installation, default_file, hold_lock, keep_before_converting, open_converting, shared, + wait_lock, +}; pub use postgres::PostgresBackend; pub use sqlite::SqliteBackend; @@ -168,6 +171,10 @@ pub struct Job { /// The caller's token: the same one again for the same subject and /// action finds this job. pub token: Option, + /// Who stopped it, when it was stopped before it ended by itself. + pub stopped_by: Option, + /// What it was started with that people may see; never a secret. + pub arguments: Vec, } impl Job { @@ -180,10 +187,22 @@ impl Job { json!({ "id": self.id, "subject": self.subject, "action": action, "host": self.host, "started": self.started, "ended": self.ended, "exit_code": self.exit_code, + "stopped_by": self.stopped_by, "arguments": self.arguments, }) } } +/// A job's arguments as its column keeps them: a JSON list, or nothing. +fn arguments_text(arguments: &[String]) -> Option { + (!arguments.is_empty()).then(|| json!(arguments).to_string()) +} + +/// A job's arguments from its column; none in rows from before it was kept. +fn arguments_from(text: Option) -> Vec { + text.and_then(|text| serde_json::from_str(&text).ok()) + .unwrap_or_default() +} + /// A row of the `state` table: a value of some kind, for a scope, by name. #[derive(Debug, Clone, PartialEq, Eq)] pub struct StateRow { @@ -370,6 +389,8 @@ pub trait Backend: Send { fn add_job(&mut self, job: &Job) -> Result; fn job_by_token(&mut self, subject: &str, action: &str, token: &str) -> Result>; fn end_job(&mut self, id: &str, exit_code: Option, ended: &str) -> Result<()>; + /// Say who stopped a job that has not ended. + fn stop_job(&mut self, id: &str, by: &str) -> Result<()>; fn end_lost_jobs(&mut self, ended: &str) -> Result; fn job(&mut self, id: &str) -> Result>; fn jobs(&mut self, limit: usize) -> Result>; @@ -606,6 +627,11 @@ impl Store { self.backend().end_job(id, exit_code, &now()) } + /// Say who stopped a job that has not ended. + pub fn stop_job(&self, id: &str, by: &str) -> Result<()> { + self.backend().stop_job(id, by) + } + /// Jobs that were running when the program that ran them went away. pub fn end_lost_jobs(&self) -> Result { self.backend().end_lost_jobs(&now()) diff --git a/crates/berth-core/src/store/postgres.rs b/crates/berth-core/src/store/postgres.rs index 4058e04..da41209 100644 --- a/crates/berth-core/src/store/postgres.rs +++ b/crates/berth-core/src/store/postgres.rs @@ -49,6 +49,8 @@ pub(super) const MIGRATIONS: &[&str] = &[ ALTER TABLE jobs ADD COLUMN action TEXT NOT NULL DEFAULT ''; ALTER TABLE jobs ADD COLUMN token TEXT; CREATE UNIQUE INDEX jobs_token ON jobs(subject, action, token) WHERE token IS NOT NULL;", + "ALTER TABLE jobs ADD COLUMN stopped_by TEXT;", + "ALTER TABLE jobs ADD COLUMN arguments TEXT;", ]; /// The advisory lock that keeps two berths from migrating at once. @@ -218,8 +220,7 @@ fn entry(row: &Row) -> Entry { } } -const JOB_COLUMNS: &str = - "id, host, what, started, ended, exit_code, output_path, subject, action, token"; +const JOB_COLUMNS: &str = "id, host, what, started, ended, exit_code, output_path, subject, action, token, stopped_by, arguments"; fn job(row: &Row) -> Job { Job { @@ -233,6 +234,8 @@ fn job(row: &Row) -> Job { subject: row.get(7), action: row.get(8), token: row.get(9), + stopped_by: row.get(10), + arguments: super::arguments_from(row.get(11)), } } @@ -240,7 +243,7 @@ fn job(row: &Row) -> Job { fn insert_job(client: &mut impl postgres::GenericClient, job: &Job) -> PgResult { client.execute( &format!( - "INSERT INTO jobs({JOB_COLUMNS}) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10) + "INSERT INTO jobs({JOB_COLUMNS}) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12) ON CONFLICT DO NOTHING" ), &[ @@ -254,6 +257,8 @@ fn insert_job(client: &mut impl postgres::GenericClient, job: &Job) -> PgResult< &job.subject, &job.action, &job.token, + &job.stopped_by, + &super::arguments_text(&job.arguments), ], ) } @@ -707,6 +712,17 @@ impl Backend for PostgresBackend { }) } + fn stop_job(&mut self, id: &str, by: &str) -> Result<()> { + let (id, by) = (id.to_owned(), by.to_owned()); + self.call(move |client| { + client.execute( + "UPDATE jobs SET stopped_by = $2 WHERE id = $1 AND ended IS NULL", + &[&id, &by], + )?; + Ok(()) + }) + } + fn end_lost_jobs(&mut self, ended: &str) -> Result { let ended = ended.to_owned(); self.call(move |client| { diff --git a/crates/berth-core/src/store/sqlite.rs b/crates/berth-core/src/store/sqlite.rs index d0e11b5..c03d030 100644 --- a/crates/berth-core/src/store/sqlite.rs +++ b/crates/berth-core/src/store/sqlite.rs @@ -33,6 +33,8 @@ pub(super) const MIGRATIONS: &[&str] = &[ ALTER TABLE jobs ADD COLUMN action TEXT NOT NULL DEFAULT ''; ALTER TABLE jobs ADD COLUMN token TEXT; CREATE UNIQUE INDEX jobs_token ON jobs(subject, action, token) WHERE token IS NOT NULL;", + "ALTER TABLE jobs ADD COLUMN stopped_by TEXT;", + "ALTER TABLE jobs ADD COLUMN arguments TEXT;", ]; impl std::convert::From for Error { @@ -92,8 +94,7 @@ fn stored(row: &rusqlite::Row<'_>) -> rusqlite::Result { const MESSAGE_COLUMNS: &str = "seq, host, id, topic, body, keep, from_level, from_uid, session, time, done"; -const JOB_COLUMNS: &str = - "id, host, what, started, ended, exit_code, output_path, subject, action, token"; +const JOB_COLUMNS: &str = "id, host, what, started, ended, exit_code, output_path, subject, action, token, stopped_by, arguments"; fn job(row: &rusqlite::Row<'_>) -> rusqlite::Result { Ok(Job { @@ -107,6 +108,8 @@ fn job(row: &rusqlite::Row<'_>) -> rusqlite::Result { subject: row.get(7)?, action: row.get(8)?, token: row.get(9)?, + stopped_by: row.get(10)?, + arguments: super::arguments_from(row.get::<_, Option>(11)?), }) } @@ -114,7 +117,7 @@ fn job(row: &rusqlite::Row<'_>) -> rusqlite::Result { fn insert_job(db: &Connection, job: &Job) -> rusqlite::Result { db.execute( &format!( - "INSERT INTO jobs({JOB_COLUMNS}) VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10) + "INSERT INTO jobs({JOB_COLUMNS}) VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12) ON CONFLICT DO NOTHING" ), params![ @@ -127,7 +130,9 @@ fn insert_job(db: &Connection, job: &Job) -> rusqlite::Result { job.output_path, job.subject, job.action, - job.token + job.token, + job.stopped_by, + super::arguments_text(&job.arguments) ], ) } @@ -446,6 +451,14 @@ impl Backend for SqliteBackend { Ok(()) } + fn stop_job(&mut self, id: &str, by: &str) -> Result<()> { + self.db.execute( + "UPDATE jobs SET stopped_by = ?2 WHERE id = ?1 AND ended IS NULL", + params![id, by], + )?; + Ok(()) + } + fn end_lost_jobs(&mut self, ended: &str) -> Result { Ok(self .db diff --git a/crates/berth-core/src/store/suite.rs b/crates/berth-core/src/store/suite.rs index 1d4d310..4d359f7 100644 --- a/crates/berth-core/src/store/suite.rs +++ b/crates/berth-core/src/store/suite.rs @@ -86,6 +86,7 @@ on_each_backend!( two_writers_on_one_key_and_one_loses, old_messages_and_jobs_are_pruned_and_values_stay, a_repeated_token_finds_the_job_it_started, + a_job_keeps_what_it_was_started_with, a_store_is_up_to_date_and_says_what_it_is, state_rows_are_kept_listed_and_removed, replacing_states_leaves_other_scopes_and_kinds_alone, @@ -97,6 +98,7 @@ on_each_backend!( resealing_keeps_the_times, resealing_refuses_a_table_that_changed, the_old_files_go_into_an_empty_store, + sealed_profiles_of_an_older_berth_take_the_names_of_now, a_copy_takes_every_table_as_it_is_and_back, a_copy_into_a_store_that_has_rows_keeps_them, ); @@ -486,6 +488,31 @@ fn a_repeated_token_finds_the_job_it_started(place: &Place) { assert_eq!(ids.len(), 1); } +fn a_job_keeps_what_it_was_started_with(place: &Place) { + let store = place.open(); + let job = |id: &str, arguments: &[&str]| Job { + id: id.into(), + host: "studio".into(), + what: "exec".into(), + started: rfc3339(unix_now()), + output_path: format!("/tmp/{id}.log"), + subject: "resource:r-1".into(), + action: "exec".into(), + arguments: arguments.iter().map(|word| word.to_string()).collect(), + ..Job::default() + }; + store + .add_job(&job("j1", &["sh", "-c", "echo 'a b'"])) + .unwrap(); + store.add_job(&job("j2", &[])).unwrap(); + let kept = store.job("j1").unwrap().unwrap(); + assert_eq!(kept.arguments, ["sh", "-c", "echo 'a b'"]); + assert_eq!(kept.json()["arguments"], json!(["sh", "-c", "echo 'a b'"])); + assert!(store.job("j2").unwrap().unwrap().arguments.is_empty()); + let listed = store.jobs(10).unwrap(); + assert!(listed.iter().any(|job| job.arguments.len() == 3)); +} + fn a_store_is_up_to_date_and_says_what_it_is(place: &Place) { let store = place.open(); let (name, steps) = match place { @@ -1222,6 +1249,60 @@ fn the_old_files_go_into_an_empty_store(place: &Place) { ); } +/// Profiles of berth 0.5, sealed and in any scope, are given the names of now +/// once, with nothing else changed. +fn sealed_profiles_of_an_older_berth_take_the_names_of_now(place: &Place) { + let store = place.open().with_keys(Some(keys()), None); + let old = |level: &str| { + json!({"level": level, "profile": "cloud", "access": {"address": "192.0.2.4"}}).to_string() + }; + store + .write(kind::PROFILE, OWN, "dock", old("reachable").as_bytes()) + .unwrap(); + store + .write(kind::PROFILE, OWN, "studio", old("workstation").as_bytes()) + .unwrap(); + store + .write( + kind::PROFILE, + "served:http://192.0.2.7:7777", + "yard", + old("controller").as_bytes(), + ) + .unwrap(); + let now = br#"{"type":"ssh","target":"laptop"}"#; + store.write(kind::PROFILE, OWN, "pier", now).unwrap(); + let before = stored_row(&store, kind::PROFILE, OWN, "pier"); + assert_eq!(crate::import::take_in_names(&store).unwrap(), 3); + let read = |scope: &str, name: &str| { + store + .read_json(kind::PROFILE, scope, name) + .unwrap() + .unwrap() + }; + assert_eq!( + read(OWN, "dock"), + json!({"type": "ssh", "target": "cloud", "access": {"address": "192.0.2.4"}}) + ); + assert_eq!( + read(OWN, "studio"), + json!({"target": "cloud", "access": {"address": "192.0.2.4"}}) + ); + assert_eq!( + read("served:http://192.0.2.7:7777", "yard")["type"], + "managed" + ); + assert!(stored_row(&store, kind::PROFILE, OWN, "dock").sealed); + assert_eq!(stored_row(&store, kind::PROFILE, OWN, "pier"), before); + assert!( + store + .names(kind::META, OWN) + .unwrap() + .contains(&crate::import::NAMES_MARKER.to_owned()) + ); + assert_eq!(crate::import::take_in_names(&store).unwrap(), 0); +} + /// A store with something in every table: messages, a cursor, values at a /// later revision, jobs, sealed and plain state, and a berth server's scope. fn filled(store: &Store) { @@ -1254,6 +1335,8 @@ fn filled(store: &Store) { subject: "host:studio".into(), action: "apply".into(), token: Some("t-1".into()), + stopped_by: Some("ops".into()), + arguments: vec!["psql".into(), "-U".into(), "app".into()], }) .unwrap(); store diff --git a/crates/berth-core/src/tailnet.rs b/crates/berth-core/src/tailnet.rs index 3b19513..5605dd5 100644 --- a/crates/berth-core/src/tailnet.rs +++ b/crates/berth-core/src/tailnet.rs @@ -1,15 +1,16 @@ //! The tailnet hosts join: Tailscale's own service, or a Headscale server. //! //! Both take the same `tailscale` client; a Headscale host is pointed at its -//! server with `--login-server`. With an API key for the control server berth -//! creates the one-use enrollment keys itself, at the moment a host needs one. +//! server with `--login-server`. A host joins with a one-use key berth makes +//! through the control server's API, at the moment the host needs one: when +//! an apply finds it off the tailnet, or when a seed or installer is written. //! //! - `tailnet_login_server`: empty for Tailscale, the address of a Headscale -//! server otherwise. A profile's `tailscale_login_server` wins for its host +//! server otherwise //! - `tailnet_user`: the Headscale user a new host belongs to //! - `tailnet_tags`: tags for new hosts, comma separated -//! - the secret `tailnet_api_key`: Tailscale API access token or Headscale API -//! key. `TS_API_KEY` in the environment works for Tailscale too +//! - the secret `tailnet_api_key`: a Tailscale API access token, a Tailscale +//! OAuth client as `CLIENT_ID:CLIENT_SECRET`, or a Headscale API key use crate::{Result, paths::Paths, registry::Store}; use berth_api::profile::{self, TAILNET_API_KEY}; @@ -18,7 +19,6 @@ use std::{ io::Write, path::Path, process::{Command, Stdio}, - time::{SystemTime, UNIX_EPOCH}, }; /// How long a key lives that goes into a seed or an installer: long enough to @@ -29,9 +29,54 @@ pub const APPLY_LIFETIME: u64 = 60 * 60; const TAILSCALE_API: &str = "https://api.tailscale.com"; +/// The role's variables of joining the tailnet, which berth sets and a +/// profile cannot. +pub const ROLE_VARS: &[&str] = &[ + "devbox_tailscale_auth_key", + "devbox_tailscale_login_server", + "devbox_tailscale_key_command", + "devbox_tailnet_problem", +]; + /// The settings of the tailnet, in the order the Settings tab lists them. pub const SETTINGS: &[&str] = &["tailnet_login_server", "tailnet_user", "tailnet_tags"]; +/// How to set the secret, for messages that ask for it. +fn set_the_secret() -> String { + format!( + "set the secret {TAILNET_API_KEY} with berth secret set {TAILNET_API_KEY}, or on the Secrets tab of the home screen" + ) +} + +/// What berth logs in to the control server's API with. +#[derive(Debug, Clone, PartialEq)] +enum Credential { + /// A Tailscale API access token or a Headscale API key. + Token(String), + /// A Tailscale OAuth client, which is exchanged for an access token of an + /// hour whenever a key is made, as its own does not expire. + Client { id: String, secret: String }, +} + +impl Credential { + /// The secret as it is stored: an OAuth client is its id and secret with + /// a colon between, which neither API keys nor client ids contain. + fn parse(text: &str) -> Option { + let text = text.trim(); + if text.is_empty() || text.contains(['"', '\\', '\n', '\r', ' ']) { + return None; + } + Some(match text.split_once(':') { + Some((id, secret)) if !id.is_empty() && !secret.is_empty() => Credential::Client { + id: id.to_owned(), + secret: secret.to_owned(), + }, + Some(_) => return None, + None => Credential::Token(text.to_owned()), + }) + } +} + /// The control server berth creates keys at, and what it needs for that. #[derive(Debug, Clone, PartialEq)] pub struct Control { @@ -39,7 +84,8 @@ pub struct Control { pub headscale: Option, pub user: String, pub tags: Vec, - token: Option, + /// The secret, or what is wrong with it. + credential: std::result::Result, /// Where Tailscale's API is, other than in tests. tailscale_api: String, } @@ -65,25 +111,25 @@ impl Control { /// The control server of berth's settings, with its credential. pub fn configured(paths: &Paths) -> Control { let setting = |key: &str| paths.setting(key).unwrap_or_default(); - let headscale = server_address(&setting("tailnet_login_server")); - let mut tags = tags(&setting("tailnet_tags")); - let mut token = Store::new(paths) + let secret = Store::new(paths) .and_then(|store| store.secret(TAILNET_API_KEY)) .ok() .flatten() - .filter(|token| !token.is_empty() && token != crate::server::KEPT); - if headscale.is_none() { - let variable = |name: &str| std::env::var(name).ok().filter(|value| !value.is_empty()); - token = token.or_else(|| variable("TS_API_KEY")); - if tags.is_empty() { - tags = self::tags(&variable("DEVBOX_TAILSCALE_TAGS").unwrap_or_default()); - } - } + .filter(|secret| !secret.is_empty()); + let credential = match secret { + None => Err(format!("berth has no API key of the tailnet: {}", set_the_secret())), + Some(secret) => Credential::parse(&secret).ok_or_else(|| { + format!( + "The secret {TAILNET_API_KEY} is neither an API key nor an OAuth client as CLIENT_ID:CLIENT_SECRET: {}", + set_the_secret() + ) + }), + }; Control { - headscale, + headscale: server_address(&setting("tailnet_login_server")), user: setting("tailnet_user"), - tags, - token, + tags: tags(&setting("tailnet_tags")), + credential, tailscale_api: TAILSCALE_API.into(), } } @@ -96,31 +142,86 @@ impl Control { } } + /// Why berth cannot create keys at this control server, if it cannot. + /// Only what berth knows without asking the server. + pub fn problem(&self) -> Option { + let credential = match &self.credential { + Ok(credential) => credential, + Err(why) => return Some(why.clone()), + }; + match (&self.headscale, credential) { + (Some(_), Credential::Client { .. }) => Some(format!( + "Headscale has no OAuth clients: the secret {TAILNET_API_KEY} is one of its API keys (headscale apikeys create)" + )), + (Some(_), _) if self.user.is_empty() && self.tags.is_empty() => Some( + "Headscale needs a user or tags for a new host's key: berth config set tailnet_user NAME, or berth config set tailnet_tags TAG".into(), + ), + (None, Credential::Client { .. }) if self.tags.is_empty() => Some( + "Keys made by a Tailscale OAuth client carry tags: berth config set tailnet_tags with the client's tags, such as tag:berth".into(), + ), + _ => None, + } + } + /// Whether berth has what it needs to create keys. pub fn can_create(&self) -> bool { - self.token.is_some() + self.problem().is_none() } /// A new pre-approved key that joins one host, once, within `lifetime` seconds. pub fn create_key(&self, description: &str, lifetime: u64) -> Result { - let token = self.token.as_deref().ok_or_else(|| match &self.headscale { - Some(_) => format!( - "berth needs an API key of Headscale to create enrollment keys: berth secret set {TAILNET_API_KEY} (make one with: headscale apikeys create)" - ), - None => format!( - "berth needs a Tailscale API access token to create enrollment keys: berth secret set {TAILNET_API_KEY}, or TS_API_KEY in the environment" - ), - })?; - if token.contains(['"', '\\', '\n', '\r']) { - return Err(format!("The secret {TAILNET_API_KEY} is not a usable API key").into()); + if let Some(problem) = self.problem() { + return Err(problem.into()); } - match &self.headscale { - Some(url) => self.headscale_key(url, token, lifetime), - None => self.tailscale_key(token, description, lifetime), + let credential = self.credential.as_ref().map_err(String::clone)?; + match (&self.headscale, credential) { + (Some(url), Credential::Token(token)) => self.headscale_key(url, token, lifetime), + (None, Credential::Token(token)) => { + self.tailscale_key(&format!("user = \"{token}:\""), description, lifetime) + } + (None, Credential::Client { id, secret }) => { + let token = self.exchange(id, secret)?; + let login = format!("header = \"Authorization: Bearer {token}\""); + self.tailscale_key(&login, description, lifetime) + } + (Some(_), Credential::Client { .. }) => Err("Headscale has no OAuth clients".into()), } } - fn tailscale_key(&self, token: &str, description: &str, lifetime: u64) -> Result { + /// An access token for an OAuth client, by the client credentials grant. + fn exchange(&self, id: &str, secret: &str) -> Result { + let form = format!( + "grant_type=client_credentials&client_id={}&client_secret={}", + query(id), + query(secret) + ); + let url = format!("{}/api/v2/oauth/token", self.tailscale_api); + let (status, reply) = call(&url, "", Some(Body::Form(form)))?; + match reply["access_token"].as_str() { + Some(token) + if status == 200 + && !token.is_empty() + && !token.contains(['"', '\\', '\n', '\r']) => + { + Ok(token.to_owned()) + } + _ => { + let reason = reply["error_description"] + .as_str() + .or(reply["error"].as_str()) + .map(str::to_owned); + Err(match reason { + Some(reason) => format!( + "Tailscale refused the OAuth client of {TAILNET_API_KEY} (HTTP {status}): {reason}" + ) + .into(), + None => refused("Tailscale", status, &reply), + }) + } + } + } + + fn tailscale_key(&self, login: &str, description: &str, lifetime: u64) -> Result { let mut create = json!({"reusable": false, "ephemeral": false, "preauthorized": true}); if !self.tags.is_empty() { create["tags"] = json!(self.tags); @@ -140,7 +241,7 @@ impl Control { "description": description, }); let url = format!("{}/api/v2/tailnet/-/keys", self.tailscale_api); - let (status, reply) = call(&url, &format!("user = \"{token}:\""), Some(&request))?; + let (status, reply) = call(&url, login, Some(Body::Json(request)))?; match reply["key"].as_str() { Some(key) if status == 200 && !key.is_empty() => Ok(key.to_owned()), _ => Err(refused("Tailscale", status, &reply)), @@ -152,21 +253,18 @@ impl Control { let mut request = json!({ "reusable": false, "ephemeral": false, - "expiration": rfc3339(now() + lifetime), + "expiration": berth_api::rfc3339(berth_api::unix_now() + lifetime), }); if !self.tags.is_empty() { request["aclTags"] = json!(self.tags); } - match (self.user.as_str(), self.tags.is_empty()) { - ("", true) => { - return Err("Headscale needs a user or tags for a new host's key: berth config tailnet_user NAME, or berth config tailnet_tags TAG".into()); - } - ("", false) => {} + match self.user.as_str() { + "" => {} // Headscale's API names the user by its number. - (user, _) if user.bytes().all(|byte| byte.is_ascii_digit()) => { + user if user.bytes().all(|byte| byte.is_ascii_digit()) => { request["user"] = json!(user); } - (user, _) => { + user => { let lookup = format!("{url}/api/v1/user?name={}", query(user)); let (status, reply) = call(&lookup, &login, None)?; if status != 200 { @@ -180,7 +278,11 @@ impl Control { request["user"] = json!(id); } } - let (status, reply) = call(&format!("{url}/api/v1/preauthkey"), &login, Some(&request))?; + let (status, reply) = call( + &format!("{url}/api/v1/preauthkey"), + &login, + Some(Body::Json(request)), + )?; match reply["preAuthKey"]["key"].as_str() { Some(key) if status == 200 && !key.is_empty() => Ok(key.to_owned()), _ => Err(refused("Headscale", status, &reply)), @@ -208,11 +310,17 @@ fn query(text: &str) -> String { .collect() } +/// What a request sends. +enum Body { + Json(Value), + Form(String), +} + /// One request with curl: the status, and the answer as JSON or as text. The -/// login is a line of curl's configuration, given on its standard input so it -/// never appears in the process list. -fn call(url: &str, login: &str, body: Option<&Value>) -> Result<(u16, Value)> { - let file = tempfile::NamedTempFile::new()?; +/// login is a line of curl's configuration, given on its standard input, and +/// the body is a private file, so neither appears in the process list. +fn call(url: &str, login: &str, body: Option) -> Result<(u16, Value)> { + let mut file = tempfile::NamedTempFile::new()?; let mut command = Command::new("curl"); command.args([ "--silent", @@ -225,14 +333,20 @@ fn call(url: &str, login: &str, body: Option<&Value>) -> Result<(u16, Value)> { "\n%{http_code}", ]); if let Some(body) = body { - serde_json::to_writer(&file, body)?; + let kind = match body { + Body::Json(value) => { + serde_json::to_writer(&mut file, &value)?; + "application/json" + } + Body::Form(text) => { + file.write_all(text.as_bytes())?; + "application/x-www-form-urlencoded" + } + }; + file.flush()?; command - .args([ - "--request", - "POST", - "--header", - "Content-Type: application/json", - ]) + .args(["--request", "POST", "--header"]) + .arg(format!("Content-Type: {kind}")) .arg("--data-binary") .arg(format!("@{}", file.path().display())); } @@ -259,40 +373,10 @@ fn call(url: &str, login: &str, body: Option<&Value>) -> Result<(u16, Value)> { Ok((status, answer)) } -fn now() -> u64 { - SystemTime::now() - .duration_since(UNIX_EPOCH) - .map(|since| since.as_secs()) - .unwrap_or(0) -} - -/// Seconds since 1970 as an RFC 3339 time in UTC. -fn rfc3339(seconds: u64) -> String { - let (days, rest) = ((seconds / 86400) as i64, seconds % 86400); - // Howard Hinnant's days_from_civil, the other way round. - let z = days + 719_468; - let era = z.div_euclid(146_097); - let doe = z.rem_euclid(146_097); - let yoe = (doe - doe / 1460 + doe / 36524 - doe / 146_096) / 365; - let doy = doe - (365 * yoe + yoe / 4 - yoe / 100); - let mp = (5 * doy + 2) / 153; - let day = doy - (153 * mp + 2) / 5 + 1; - let month = if mp < 10 { mp + 3 } else { mp - 9 }; - let year = yoe + era * 400 + i64::from(month <= 2); - format!( - "{year:04}-{month:02}-{day:02}T{:02}:{:02}:{:02}Z", - rest / 3600, - rest / 60 % 60, - rest % 60 - ) -} - -/// The login server a host's client is pointed at: the profile's own, or -/// berth's. Empty for Tailscale's service. -pub fn login_server(paths: &Paths, config: &Value) -> String { - server_address(config["tailscale_login_server"].as_str().unwrap_or("")) - .or_else(|| server_address(&paths.setting("tailnet_login_server").unwrap_or_default())) - .unwrap_or_default() +/// The login server hosts are pointed at: berth's setting. Empty for +/// Tailscale's service. +pub fn login_server(paths: &Paths) -> String { + server_address(&paths.setting("tailnet_login_server").unwrap_or_default()).unwrap_or_default() } /// What a key is described as at the control server. @@ -303,34 +387,28 @@ fn description(host: &str) -> String { } } -/// A key for a host to join with, from the control server of berth's -/// settings. A host whose profile names another login server gets none, as -/// that server would not take it. -pub fn create_key(paths: &Paths, config: &Value, host: &str, lifetime: u64) -> Result { - let control = Control::configured(paths); - let wanted = login_server(paths, config); - if control.headscale.as_deref().unwrap_or("") != wanted { - return Err(format!( - "{host} joins the tailnet at {}, and berth creates keys at {}: set tailnet_login_server to the same server, or paste a key", - match wanted.as_str() { - "" => "Tailscale", - other => other, - }, - control.name() - ) - .into()); +/// Why a host of this profile cannot join the tailnet, in words that say +/// what to do, or `None` when it can or does not join at all. It is asked +/// before anything runs, so a host is not set up halfway. +pub fn problem(paths: &Paths, config: &Value, host: &str) -> Option { + if !profile::joins_tailnet(config) { + return None; } - control.create_key(&description(host), lifetime) + let control = Control::configured(paths); + let why = control.problem()?; + let host = match host { + "" => "The host".to_owned(), + host => host.to_owned(), + }; + Some(format!( + "{host} joins the tailnet at {}, and berth cannot make its key. {why}. Or switch Tailscale off in its profile, and berth reaches it over SSH at its address", + control.name() + )) } -/// A key for the profile's key field, as F4 makes one, and what to say of it. -pub fn generate(paths: &Paths, config: &Value, host: &str) -> Result<(String, String)> { - let key = create_key(paths, config, host, SEED_LIFETIME)?; - let note = format!( - "One-use key created at {} · valid for 24 hours", - Control::configured(paths).name() - ); - Ok((key, note)) +/// A key for a host to join with, from the control server of berth's settings. +pub fn create_key(paths: &Paths, host: &str, lifetime: u64) -> Result { + Control::configured(paths).create_key(&description(host), lifetime) } /// The key `berth tailnet-key WHAT` prints. An apply runs that on this @@ -338,25 +416,59 @@ pub fn generate(paths: &Paths, config: &Value, host: &str) -> Result<(String, St /// path [`apply_vars`] was given, so it finds the host in the same place. pub fn key_for(paths: &Paths, what: &str) -> Result { let host = crate::host::Host::load(paths, what)?; - create_key(paths, &host.config, &host.place.title(), APPLY_LIFETIME) + create_key(paths, &host.place.title(), APPLY_LIFETIME) } -/// The profile as a seed carries it: the login server filled in, and for a -/// host that joins automatically a new key. Keys are created only here, when -/// a seed is written, and live for [`SEED_LIFETIME`]. -pub fn seed_profile(paths: &Paths, config: &Value, host: &str) -> Result { - let mut config = config.clone(); - if !profile::installs_tailscale(&config) { - return Ok(config); +/// What a seed or an installer is given to join the tailnet with: a private +/// file for the seed builder, which names it in `BERTH_TAILNET_FILE`, and +/// until when its key works. +pub struct SeedKey { + pub file: tempfile::NamedTempFile, + /// Seconds since 1970. + pub until: u64, +} + +impl SeedKey { + /// What to tell whoever writes the seed or installer. + pub fn note(&self) -> String { + until_note(self.until) } - if profile::joins_automatically(&config) { - config["tailscale_auth_key"] = json!(create_key(paths, &config, host, SEED_LIFETIME)?); + + /// Give the seed builder the key. + pub fn pass(&self, command: &mut Command) { + command.env("BERTH_TAILNET_FILE", self.file.path()); } - let server = login_server(paths, &config); - if !server.is_empty() { - config["tailscale_login_server"] = json!(server); +} + +/// Until when a seed or installer can join the tailnet, and what happens after. +pub fn until_note(until: u64) -> String { + let time = berth_api::rfc3339(until); + let shown = time + .get(..16) + .map(|minutes| minutes.replace('T', " ")) + .unwrap_or(time); + format!( + "Its key joins the tailnet until {shown} UTC, 24 hours from now. A machine first started later is set up without the tailnet; applying it from berth then joins it" + ) +} + +/// A new key for a seed or an installer of a host that joins the tailnet, +/// living for [`SEED_LIFETIME`], with the login server. Keys are made only +/// here, when a seed is written, and never kept in the profile. +pub fn seed_key(paths: &Paths, config: &Value, host: &str) -> Result> { + if !profile::joins_tailnet(config) { + return Ok(None); } - Ok(config) + if let Some(problem) = problem(paths, config, host) { + return Err(problem.into()); + } + let key = create_key(paths, host, SEED_LIFETIME)?; + let until = berth_api::unix_now() + SEED_LIFETIME; + let mut file = tempfile::NamedTempFile::new()?; + let given = json!({"key": key, "login_server": login_server(paths)}); + serde_json::to_writer(&mut file, &given)?; + file.flush()?; + Ok(Some(SeedKey { file, until })) } /// A private copy of a profile for the seed builder, which reads a file. @@ -367,21 +479,26 @@ pub fn profile_file(config: &Value) -> Result { Ok(file) } -/// What an apply adds to the role's variables: the login server, and for a -/// host that joins automatically the command the role runs on this machine -/// for a key, only when the host is not on the tailnet yet. `berth` is this -/// program. +/// What an apply adds to the role's variables for a host that joins the +/// tailnet: the login server, and the command the role runs on this machine +/// for a key, only when the host is not on the tailnet yet. When berth cannot +/// make keys the role is given why instead, and stops before it changes +/// anything on a host that is not on the tailnet. `berth` is this program. pub fn apply_vars(paths: &Paths, config: &Value, host: &str, berth: &Path) -> Value { let mut vars = json!({}); - if !profile::installs_tailscale(config) { + if !profile::joins_tailnet(config) { return vars; } - let server = login_server(paths, config); + let server = login_server(paths); if !server.is_empty() { vars["devbox_tailscale_login_server"] = json!(server); } - if profile::joins_automatically(config) { - vars["devbox_tailscale_key_command"] = json!([berth, "tailnet-key", host]); + let name = crate::profiles::Place::resolve(paths, host) + .map(|place| place.title()) + .unwrap_or_else(|_| host.to_owned()); + match problem(paths, config, &name) { + Some(problem) => vars["devbox_tailnet_problem"] = json!(problem), + None => vars["devbox_tailscale_key_command"] = json!([berth, "tailnet-key", host]), } vars } @@ -400,6 +517,8 @@ mod tests { struct Seen { line: String, authorization: String, + kind: String, + text: String, body: Value, } @@ -415,7 +534,7 @@ mod tests { let mut reader = BufReader::new(stream.try_clone().unwrap()); let mut line = String::new(); reader.read_line(&mut line).unwrap(); - let (mut length, mut authorization) = (0, String::new()); + let (mut length, mut authorization, mut kind) = (0, String::new(), String::new()); loop { let mut header = String::new(); reader.read_line(&mut header).unwrap(); @@ -427,6 +546,7 @@ mod tests { match name.to_ascii_lowercase().as_str() { "content-length" => length = value.trim().parse().unwrap(), "authorization" => authorization = value.trim().to_owned(), + "content-type" => kind = value.trim().to_owned(), _ => {} } } @@ -436,6 +556,8 @@ mod tests { .send(Seen { line: line.trim_end().to_owned(), authorization, + kind, + text: String::from_utf8_lossy(&body).into_owned(), body: serde_json::from_slice(&body).unwrap_or(Value::Null), }) .unwrap(); @@ -457,7 +579,7 @@ mod tests { headscale: headscale.map(str::to_owned), user: String::new(), tags: vec![], - token: Some("secret-token".into()), + credential: Ok(Credential::Token("secret-token".into())), tailscale_api: tailscale_api.into(), } } @@ -496,6 +618,81 @@ mod tests { assert!(said.to_string().contains("enough permissions"), "{said}"); } + #[test] + fn an_oauth_client_is_exchanged_for_a_token_of_its_own_each_time() { + let (url, seen) = fake(vec![ + ( + 200, + json!({"access_token": "tskey-access-FAKE", "token_type": "Bearer", "expires_in": 3600}), + ), + (200, json!({"key": "tskey-auth-by-client"})), + ]); + let mut control = control(None, &url); + control.credential = + Ok(Credential::parse("kFAKE1CNTRL:tskey-client-kFAKE1CNTRL-0000").unwrap()); + // Its keys carry tags, and berth says so before it asks. + let said = control.create_key("berth studio", 60).unwrap_err(); + assert!(said.to_string().contains("tailnet_tags"), "{said}"); + control.tags = tags("berth"); + assert_eq!( + control.create_key("berth studio", 60).unwrap(), + "tskey-auth-by-client" + ); + let exchange = seen.recv().unwrap(); + assert_eq!(exchange.line, "POST /api/v2/oauth/token HTTP/1.1"); + assert_eq!(exchange.kind, "application/x-www-form-urlencoded"); + assert_eq!( + exchange.text, + "grant_type=client_credentials&client_id=kFAKE1CNTRL&client_secret=tskey-client-kFAKE1CNTRL-0000" + ); + assert_eq!(exchange.authorization, ""); + let create = seen.recv().unwrap(); + assert_eq!(create.line, "POST /api/v2/tailnet/-/keys HTTP/1.1"); + assert_eq!(create.authorization, "Bearer tskey-access-FAKE"); + assert_eq!( + create.body["capabilities"]["devices"]["create"]["tags"], + json!(["tag:berth"]) + ); + } + + #[test] + fn a_refused_oauth_client_says_why() { + let (url, _seen) = fake(vec![( + 401, + json!({"error": "invalid_client", "error_description": "client secret is invalid"}), + )]); + let mut control = control(None, &url); + control.credential = Ok(Credential::parse("kFAKE:tskey-client-kFAKE-0000").unwrap()); + control.tags = tags("berth"); + let said = control.create_key("berth", 60).unwrap_err().to_string(); + assert!( + said.contains("HTTP 401") && said.contains("client secret is invalid"), + "{said}" + ); + } + + #[test] + fn the_secret_is_a_token_or_a_client_and_nothing_else() { + assert_eq!( + Credential::parse(" tskey-api-kFAKE-0000\n"), + Some(Credential::Token("tskey-api-kFAKE-0000".into())) + ); + assert_eq!( + Credential::parse("hsFAKE.0000"), + Some(Credential::Token("hsFAKE.0000".into())) + ); + assert_eq!( + Credential::parse("kFAKE:tskey-client-kFAKE-0000"), + Some(Credential::Client { + id: "kFAKE".into(), + secret: "tskey-client-kFAKE-0000".into() + }) + ); + for bad in ["", ":secret", "id:", "two words", "a\"b"] { + assert_eq!(Credential::parse(bad), None, "{bad}"); + } + } + #[test] fn headscale_finds_the_user_and_makes_a_one_use_key() { let (url, seen) = fake(vec![ @@ -508,7 +705,7 @@ mod tests { let mut control = control(Some(&url), "http://127.0.0.1:9"); control.user = "agents".into(); control.tags = tags("berth"); - let before = now(); + let before = berth_api::unix_now(); assert_eq!( control.create_key("berth studio", 3600).unwrap(), "hskey-auth-new" @@ -525,7 +722,11 @@ mod tests { assert_eq!(create.body["aclTags"], json!(["tag:berth"])); let expiry = create.body["expiration"].as_str().unwrap(); assert!( - [rfc3339(before + 3600), rfc3339(before + 3601)].contains(&expiry.to_owned()), + [ + berth_api::rfc3339(before + 3600), + berth_api::rfc3339(before + 3601) + ] + .contains(&expiry.to_owned()), "{expiry}" ); } @@ -546,6 +747,14 @@ mod tests { ); let create = seen.recv().unwrap(); assert!(create.body.get("user").is_none(), "{:?}", create.body); + // An OAuth client is Tailscale's alone. + control.credential = Ok(Credential::parse("kFAKE:tskey-client-kFAKE-0000").unwrap()); + assert!( + control + .problem() + .unwrap() + .contains("Headscale has no OAuth clients") + ); } #[test] @@ -564,13 +773,6 @@ mod tests { assert!(said.to_string().contains("Unauthorized"), "{said}"); } - #[test] - fn times_are_written_as_rfc3339() { - assert_eq!(rfc3339(0), "1970-01-01T00:00:00Z"); - assert_eq!(rfc3339(951_782_400), "2000-02-29T00:00:00Z"); - assert_eq!(rfc3339(1_798_761_599), "2026-12-31T23:59:59Z"); - } - fn paths() -> (tempfile::TempDir, Paths) { let home = tempfile::tempdir().unwrap(); let paths = Paths { @@ -581,6 +783,13 @@ mod tests { (home, paths) } + fn set_secret(paths: &Paths, value: Option<&str>) { + Store::new(paths) + .unwrap() + .set_secret(TAILNET_API_KEY, value) + .unwrap(); + } + #[test] fn the_settings_and_the_secret_make_the_control_server() { let (_home, paths) = paths(); @@ -602,60 +811,82 @@ mod tests { .contains("berth secret set tailnet_api_key"), "{said}" ); - Store::new(&paths) - .unwrap() - .set_secret(TAILNET_API_KEY, Some("hskey-api-x")) - .unwrap(); + set_secret(&paths, Some("hskey-api-x")); assert!(Control::configured(&paths).can_create()); assert_eq!( - Control::configured(&paths).token.as_deref(), - Some("hskey-api-x") + Control::configured(&paths).credential, + Ok(Credential::Token("hskey-api-x".into())) + ); + set_secret(&paths, Some("not one")); + assert!( + Control::configured(&paths) + .problem() + .unwrap() + .contains("CLIENT_ID:CLIENT_SECRET") ); } #[test] - fn hosts_join_the_login_server_of_their_profile_or_of_berth() { + fn a_missing_api_key_is_a_problem_before_anything_runs() { let (_home, paths) = paths(); - let auto = json!({"tailscale_auto_key": true}); - assert_eq!(login_server(&paths, &json!({})), ""); + let joins = json!({"hostname": "studio"}); + let said = problem(&paths, &joins, "studio").unwrap(); + assert!( + said.starts_with("studio joins the tailnet at Tailscale"), + "{said}" + ); + assert!(said.contains("berth secret set tailnet_api_key"), "{said}"); + assert!(said.contains("Secrets tab"), "{said}"); + assert!(said.contains("over SSH at its address"), "{said}"); + // A host that does not join needs nothing. + let off = json!({"ansible_vars": {"devbox_install_tailscale": false}}); + assert_eq!(problem(&paths, &off, "studio"), None); + // An apply gives the role the problem in place of the way to a key. + let vars = apply_vars(&paths, &joins, "studio", Path::new("/b")); + assert_eq!(vars["devbox_tailnet_problem"], json!(said)); + assert!(vars.get("devbox_tailscale_key_command").is_none()); + // A seed is not written at all. + let said = seed_key(&paths, &joins, "studio") + .err() + .unwrap() + .to_string(); + assert!(said.contains("berth secret set tailnet_api_key"), "{said}"); + assert!(seed_key(&paths, &off, "studio").unwrap().is_none()); + set_secret(&paths, Some("tskey-api-kFAKE-0000")); + assert_eq!(problem(&paths, &joins, "studio"), None); + } + + #[test] + fn hosts_join_the_login_server_of_berth() { + let (_home, paths) = paths(); + set_secret(&paths, Some("hskey-api-x")); + assert_eq!(login_server(&paths), ""); assert_eq!( - apply_vars( - &paths, - &json!({"tailscale_auth_key": "k"}), - "studio", - Path::new("/b") - ), - json!({}) + apply_vars(&paths, &json!({}), "studio", Path::new("/usr/bin/berth")), + json!({"devbox_tailscale_key_command": ["/usr/bin/berth", "tailnet-key", "studio"]}) ); paths .set("tailnet_login_server", "http://hs.test:8080") .unwrap(); - assert_eq!(login_server(&paths, &json!({})), "http://hs.test:8080"); - let own = json!({"tailscale_login_server": "https://other.test/"}); - assert_eq!(login_server(&paths, &own), "https://other.test"); + paths.set("tailnet_user", "1").unwrap(); + assert_eq!(login_server(&paths), "http://hs.test:8080"); assert_eq!( - apply_vars(&paths, &auto, "studio", Path::new("/usr/bin/berth")), + apply_vars(&paths, &json!({}), "studio", Path::new("/usr/bin/berth")), json!({ "devbox_tailscale_login_server": "http://hs.test:8080", "devbox_tailscale_key_command": ["/usr/bin/berth", "tailnet-key", "studio"], }) ); - // berth makes no key a host's own login server would not take. - let mut other = own.clone(); - other["tailscale_auto_key"] = json!(true); - let said = create_key(&paths, &other, "studio", 60).unwrap_err(); - assert!(said.to_string().contains("https://other.test"), "{said}"); // Without Tailscale there is nothing to join. - let off = json!({"tailscale_auto_key": true, "ansible_vars": {"devbox_install_tailscale": false}}); + let off = json!({"ansible_vars": {"devbox_install_tailscale": false}}); assert_eq!( apply_vars(&paths, &off, "studio", Path::new("/b")), json!({}) ); - assert_eq!(seed_profile(&paths, &off, "studio").unwrap(), off); } #[test] - fn a_seed_carries_a_new_key_and_the_login_server() { + fn a_seed_gets_a_new_key_in_a_private_file_and_says_until_when() { let (_home, paths) = paths(); let (url, seen) = fake(vec![( 200, @@ -663,25 +894,53 @@ mod tests { )]); paths.set("tailnet_login_server", &url).unwrap(); paths.set("tailnet_user", "5").unwrap(); - Store::new(&paths) + set_secret(&paths, Some("hskey-api-x")); + let before = berth_api::unix_now(); + let given = seed_key(&paths, &json!({"hostname": "studio"}), "studio") .unwrap() - .set_secret(TAILNET_API_KEY, Some("hskey-api-x")) .unwrap(); - let config = json!({"hostname": "studio", "tailscale_auto_key": true}); - let seeded = seed_profile(&paths, &config, "studio").unwrap(); - assert_eq!(seeded["tailscale_auth_key"], "hskey-auth-seed"); - assert_eq!(seeded["tailscale_login_server"], json!(url)); let asked = seen.recv().unwrap(); assert_eq!(asked.body["user"], "5"); - // A pasted key is used as it is, and no key is created for it. - let pasted = json!({"tailscale_auto_key": true, "tailscale_auth_key": "hskey-auth-pasted"}); - let seeded = seed_profile(&paths, &pasted, "studio").unwrap(); - assert_eq!(seeded["tailscale_auth_key"], "hskey-auth-pasted"); - let file = profile_file(&seeded).unwrap(); + let expiry = asked.body["expiration"].as_str().unwrap().to_owned(); + assert!( + [ + berth_api::rfc3339(before + SEED_LIFETIME), + berth_api::rfc3339(before + SEED_LIFETIME + 1) + ] + .contains(&expiry), + "{expiry}" + ); + assert!((before + SEED_LIFETIME..=before + SEED_LIFETIME + 1).contains(&given.until)); + let file: Value = + serde_json::from_slice(&std::fs::read(given.file.path()).unwrap()).unwrap(); + assert_eq!(file, json!({"key": "hskey-auth-seed", "login_server": url})); let mode = std::os::unix::fs::PermissionsExt::mode( - &file.as_file().metadata().unwrap().permissions(), + &given.file.as_file().metadata().unwrap().permissions(), ); assert_eq!(mode & 0o077, 0); + let mut command = Command::new("true"); + given.pass(&mut command); + assert!( + command + .get_envs() + .any(|(name, value)| name == "BERTH_TAILNET_FILE" + && value == Some(given.file.path().as_os_str())) + ); + } + + #[test] + fn the_note_of_an_installer_says_until_when_it_can_join() { + // 2026-11-16 10:30:00 UTC. + let note = until_note(1_794_825_000); + assert!( + note.starts_with("Its key joins the tailnet until 2026-11-16 10:30 UTC"), + "{note}" + ); + assert!(note.contains("24 hours"), "{note}"); + assert!( + note.contains("applying it from berth then joins it"), + "{note}" + ); } #[test] @@ -693,11 +952,8 @@ mod tests { ]); paths.set("tailnet_login_server", &url).unwrap(); paths.set("tailnet_user", "5").unwrap(); - Store::new(&paths) - .unwrap() - .set_secret(TAILNET_API_KEY, Some("hskey-api-x")) - .unwrap(); - let config = json!({"hostname": "studio", "tailscale_auto_key": true}); + set_secret(&paths, Some("hskey-api-x")); + let config = json!({"hostname": "studio"}); let stored = crate::profiles::Place::own("studio").unwrap(); let file = crate::profiles::Place::File(home.path().join("attic.json")); let mut keys = vec![]; @@ -714,11 +970,6 @@ mod tests { assert!(seen.recv().unwrap().line.starts_with("POST")); } assert_eq!(keys, ["hskey-auth-named", "hskey-auth-file"]); - // A berth that uses a berth server has none of its own hosts by name; - // berth serve refuses to start like that, so its children never do. - paths.set("server", "http://192.0.2.7:7777").unwrap(); - let said = key_for(&paths, "studio").unwrap_err().to_string(); - assert_eq!(said, "No host studio"); let by_path = file.argument(); assert_eq!( crate::profiles::Place::resolve(&paths, &by_path).unwrap(), diff --git a/crates/berth-core/tests/fixtures/store-0.5.2.db b/crates/berth-core/tests/fixtures/store-0.5.2.db new file mode 100644 index 0000000..403f5c1 Binary files /dev/null and b/crates/berth-core/tests/fixtures/store-0.5.2.db differ diff --git a/crates/berth-core/tests/link.rs b/crates/berth-core/tests/link.rs index 6390fe5..8f7b94c 100644 --- a/crates/berth-core/tests/link.rs +++ b/crates/berth-core/tests/link.rs @@ -50,7 +50,6 @@ async fn connections_are_kept_replaced_and_can_start_with_a_password() { program: "sh".into(), args: vec!["-c".into(), script.clone()], first_line: None, - via: None, ..LinkSpec::default() }); let made = || { @@ -117,7 +116,6 @@ async fn connections_are_kept_replaced_and_can_start_with_a_password() { program: "sh".into(), args: vec!["-c".into(), guarded.clone()], first_line: Some(line.into()), - via: None, ..LinkSpec::default() }; let version: Value = Link::new(spec("secret")).get("/v1/version").await.unwrap(); diff --git a/crates/berth-core/tests/sshd.rs b/crates/berth-core/tests/sshd.rs index fe9ffd5..5a3ae47 100644 --- a/crates/berth-core/tests/sshd.rs +++ b/crates/berth-core/tests/sshd.rs @@ -221,7 +221,7 @@ fn first_contact_host_keys_and_rekeying_against_sshd() { let said = contact::works(&keys::login_of(&paths, &studio).unwrap()) .unwrap_err() .to_string(); - assert!(said.contains("berth key host studio --accept"), "{said}"); + assert!(said.contains("berth host verify studio --accept"), "{said}"); // First contact says so too, without a credential and with one, and the // credential is not sent to a machine that may not be the host. let said = contact::trust_key(&paths, &studio, None, false) @@ -234,7 +234,7 @@ fn first_contact_host_keys_and_rekeying_against_sshd() { .unwrap_err() .to_string(); assert!(said.starts_with("The SSH host key of studio"), "{said}"); - assert!(said.contains("berth key host studio --accept"), "{said}"); + assert!(said.contains("berth host verify studio --accept"), "{said}"); assert_eq!(server.authorized(), authorized); let before = contact::host_key(&paths, &studio, false).unwrap(); let after = contact::host_key(&paths, &studio, true).unwrap(); diff --git a/crates/berth-core/tests/upgrade.rs b/crates/berth-core/tests/upgrade.rs new file mode 100644 index 0000000..feebfc5 --- /dev/null +++ b/crates/berth-core/tests/upgrade.rs @@ -0,0 +1,138 @@ +//! A store that berth 0.5.2 wrote, with made-up hosts and fake keys, opened by +//! this berth: it is copied as it was, converted once, and the next start +//! changes nothing. + +use berth_core::paths::Paths; +use berth_core::store::{self, kind}; +use std::os::unix::fs::PermissionsExt; + +const FIXTURE: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/tests/fixtures/store-0.5.2.db"); + +fn installation() -> (tempfile::TempDir, Paths) { + let dir = tempfile::tempdir().unwrap(); + let paths = Paths { + config_dir: dir.path().join("config"), + data_dir: dir.path().join("data"), + }; + std::fs::create_dir_all(&paths.data_dir).unwrap(); + let file = paths.data_dir.join("berth.db"); + std::fs::copy(FIXTURE, &file).unwrap(); + std::fs::set_permissions(&file, std::fs::Permissions::from_mode(0o600)).unwrap(); + (dir, paths) +} + +fn schema(file: &std::path::Path) -> i64 { + let db = rusqlite::Connection::open(file).unwrap(); + db.query_row("PRAGMA user_version", [], |row| row.get(0)) + .unwrap() +} + +fn rows(store: &store::Store) -> store::Rows { + store.export(store::Table::State).unwrap() +} + +#[test] +fn a_store_of_berth_0_5_2_is_copied_then_converted_once() { + let (_dir, paths) = installation(); + let file = paths.data_dir.join("berth.db"); + assert_eq!(schema(&file), 4); + + let store = store::open_converting(&paths).unwrap(); + let copy = paths.data_dir.join("berth.db.berth-0.5.2-schema-4"); + assert!(copy.is_file()); + assert_eq!( + std::fs::metadata(©).unwrap().permissions().mode() & 0o777, + 0o600 + ); + // The copy is the store as berth 0.5.2 left it, which that berth opens. + assert_eq!(schema(©), 4); + let old = rusqlite::Connection::open(©).unwrap(); + let level: String = old + .query_row( + "SELECT CAST(value AS TEXT) FROM state WHERE kind = 'profile' AND scope = '' AND name = 'studio'", + [], + |row| row.get(0), + ) + .unwrap(); + assert!(level.contains("\"level\":\"workstation\""), "{level}"); + assert!(level.contains("tskey-auth-kFAKE"), "{level}"); + + // The store itself is converted: names of now, no pasted key, every marker. + assert!(store.schema_version().unwrap() > 4); + let studio = store + .read_json(kind::PROFILE, store::OWN, "studio") + .unwrap() + .unwrap(); + assert_eq!( + berth_api::profile::host_type(&studio), + berth_api::HostType::Workstation + ); + assert!(studio.get("level").is_none() && studio.get("tailscale_auth_key").is_none()); + let pier = store + .read_json(kind::PROFILE, store::OWN, "pier") + .unwrap() + .unwrap(); + assert_eq!( + berth_api::profile::host_type(&pier), + berth_api::HostType::Managed + ); + // dock had berth 0.5 make its key, so it joins as before; pier never joined. + let dock = store + .read_json(kind::PROFILE, store::OWN, "dock") + .unwrap() + .unwrap(); + assert!( + dock["ansible_vars"] + .get("devbox_tailnet_optional") + .is_none(), + "{dock}" + ); + assert_eq!(pier["ansible_vars"]["devbox_tailnet_optional"], true); + let names = store.names(kind::META, store::OWN).unwrap(); + for marker in berth_core::import::MARKERS { + assert!(names.iter().any(|name| name == marker), "{marker}"); + } + let secrets = store.names(kind::SECRET, store::OWN).unwrap(); + assert!(secrets.iter().any(|name| name == "tailnet_api_key")); + let converted = rows(&store); + drop(store); + + // The next start finds nothing to do, and makes no copy. + std::fs::remove_file(©).unwrap(); + let store = store::open_converting(&paths).unwrap(); + assert_eq!(rows(&store), converted); + assert!(!copy.exists()); + let left: Vec = std::fs::read_dir(&paths.data_dir) + .unwrap() + .map(|entry| entry.unwrap().file_name().to_string_lossy().into_owned()) + .filter(|name| name.contains("schema-")) + .collect(); + assert!(left.is_empty(), "{left:?}"); +} + +#[test] +fn only_the_latest_copy_is_kept() { + let (_dir, paths) = installation(); + let earlier = paths.data_dir.join("berth.db.berth-0.4.1-schema-3"); + std::fs::write(&earlier, b"an earlier copy").unwrap(); + let said = store::keep_before_converting(&paths.data_dir.join("berth.db")) + .unwrap() + .unwrap(); + assert!( + said.contains("berth 0.5.2 needs that copy to go back"), + "{said}" + ); + assert!(!earlier.exists()); + assert!( + paths + .data_dir + .join("berth.db.berth-0.5.2-schema-4") + .is_file() + ); + // A store of this berth needs no copy. + store::open_converting(&paths).unwrap(); + assert_eq!( + store::keep_before_converting(&paths.data_dir.join("berth.db")).unwrap(), + None + ); +} diff --git a/crates/berth-page/Cargo.toml b/crates/berth-page/Cargo.toml index f329c54..fc7fc51 100644 --- a/crates/berth-page/Cargo.toml +++ b/crates/berth-page/Cargo.toml @@ -4,12 +4,21 @@ version.workspace = true edition.workspace = true rust-version.workspace = true license.workspace = true -description = "The page of berth serve, its screens and its portal, for a browser (berth-web) and a terminal (berth-remote)" +description = "The screens of berth in a terminal, over the API of a berth serve, on Linux, macOS and Windows" [dependencies] -berth-api.workspace = true +berth-api = { workspace = true, features = ["client"] } berth-tui.workspace = true serde_json.workspace = true tui-input.workspace = true -web-time = "1" -ratatui = { version = "=0.30.2", default-features = false } +ratatui = { version = "=0.30.2", default-features = false, features = ["crossterm"] } +crossterm = "=0.29.0" +tokio.workspace = true +futures-util.workspace = true +rustls = { version = "0.23", default-features = false, features = ["ring", "std", "tls12"] } +tokio-rustls = { version = "0.26", default-features = false, features = ["ring", "tls12"] } +webpki-roots = "1" +rustls-native-certs = "0.8" + +[dev-dependencies] +tempfile.workspace = true diff --git a/crates/berth-page/src/admin.rs b/crates/berth-page/src/admin.rs index 292d590..f2a4a7a 100644 --- a/crates/berth-page/src/admin.rs +++ b/crates/berth-page/src/admin.rs @@ -171,6 +171,7 @@ impl Admin { label: text("label"), group: text("group"), help: text("help"), + types: berth_tui::setup::feature_types(feature), } }) .collect(); @@ -348,7 +349,7 @@ impl Admin { let body = json!({"arguments": options.arguments()}); let asked = json!({"action": "rollout"}); self.portal - .call("actions.act", None, asked, body, move |result| { + .call("installation.act", None, asked, body, move |result| { let job = match result { Ok(acted) => acted["job"].clone(), Err(error) => { @@ -556,7 +557,8 @@ impl Admin { json!({"name": name, "action": "seed"}), Value::Null, |admin, acted| { - admin.say(""); + // Until when the seed can join the tailnet, for a host that joins it. + admin.say(acted["result"]["tailnet"].as_str().unwrap_or("")); admin.arrive(Arrived::Seed( acted["result"]["user_data"] .as_str() @@ -577,7 +579,7 @@ impl Admin { self.job("apply", name, body, title, size, |_| {}); } - /// What the server lists as the host's actions: its tasks and berth's own. + /// What the server lists as the host's actions: its controller's and berth's own. pub fn actions(&self, name: &str) { let admin = self.clone(); let host = name.to_owned(); @@ -648,7 +650,7 @@ impl Admin { Ok(acted) => { let job = &acted["job"]; admin.say(""); - admin.data.borrow_mut().job = Some(View::plain(&title, size.0, size.1)); + admin.data.borrow_mut().job = Some(View::of_job(&title, size.0, size.1, job)); let follower = admin.clone(); admin.portal.stream( "jobs.follow", @@ -748,7 +750,7 @@ impl Admin { let mut asked = asked.clone(); asked.host = Some(name.to_owned()); asked.kind = "machine".into(); - let title = format!("Create machine · {name}"); + let title = format!("Create · {name}"); let (admin, host) = (self.clone(), name.to_owned()); self.say("Starting…"); self.portal.call( @@ -760,7 +762,7 @@ impl Admin { Ok(created) => { admin.say(""); let again = host.clone(); - admin.follow(created["job"]["id"].clone(), &title, size, move |admin| { + admin.follow(&created["job"], &title, size, move |admin| { admin.machine(&again) }); } @@ -772,7 +774,7 @@ impl Admin { /// Delete the host's machine, the resource `id`, as a job of the server, /// and follow it. pub fn unprovision(&self, name: &str, id: &str, size: (u16, u16)) { - let title = format!("Delete machine · {name}"); + let title = format!("Destroy · {name}"); let (admin, host) = (self.clone(), name.to_owned()); self.portal.call( "resources.remove", @@ -782,7 +784,7 @@ impl Admin { move |result| match result { Ok(acted) => { let again = host.clone(); - admin.follow(acted["job"]["id"].clone(), &title, size, move |admin| { + admin.follow(&acted["job"], &title, size, move |admin| { admin.machine(&again) }); } @@ -795,7 +797,7 @@ impl Admin { /// server, and follow it. pub fn power(&self, name: &str, id: &str, verb: &str, size: (u16, u16)) { let title = format!( - "{} machine · {name}", + "{} · {name}", if verb == "start" { "Start" } else { "Stop" } ); let (admin, host) = (self.clone(), name.to_owned()); @@ -808,7 +810,7 @@ impl Admin { Ok(acted) if acted["job"].is_null() => admin.machine(&host), Ok(acted) => { let again = host.clone(); - admin.follow(acted["job"]["id"].clone(), &title, size, move |admin| { + admin.follow(&acted["job"], &title, size, move |admin| { admin.machine(&again) }); } @@ -882,7 +884,7 @@ impl Admin { Ok(acted) => { admin.say(""); let again = id.clone(); - admin.follow(acted["job"]["id"].clone(), &title, size, move |admin| { + admin.follow(&acted["job"], &title, size, move |admin| { admin.resource(&again) }); } @@ -1067,7 +1069,7 @@ impl Admin { Ok(acted) => { admin.say(""); let again = name.clone(); - admin.follow(acted["job"]["id"].clone(), &title, size, move |admin| { + admin.follow(&acted["job"], &title, size, move |admin| { admin.provisioner(&again) }); } @@ -1088,8 +1090,9 @@ impl Admin { } /// Follow a job of the server in a panel; `ended` is run when it has ended. - fn follow(&self, id: Value, title: &str, size: (u16, u16), ended: impl Fn(&Admin) + 'static) { - self.data.borrow_mut().job = Some(View::plain(title, size.0, size.1)); + fn follow(&self, job: &Value, title: &str, size: (u16, u16), ended: impl Fn(&Admin) + 'static) { + let id = job["id"].clone(); + self.data.borrow_mut().job = Some(View::of_job(title, size.0, size.1, job)); let follower = self.clone(); self.portal .stream("jobs.follow", None, json!({"id": id}), move |frame| { diff --git a/crates/berth-page/src/app.rs b/crates/berth-page/src/app.rs index fa4080b..f259848 100644 --- a/crates/berth-page/src/app.rs +++ b/crates/berth-page/src/app.rs @@ -24,8 +24,8 @@ use serde_json::{Value, json}; use std::cell::RefCell; use std::collections::BTreeMap; use std::rc::Rc; +use std::time::{Duration, Instant}; use tui_input::Input; -use web_time::{Duration, Instant}; /// How often the list of hosts is asked for again. const HOSTS_EVERY: Duration = Duration::from_secs(10); @@ -35,13 +35,13 @@ const RETRY: Duration = Duration::from_secs(3); const SAID_FOR: Duration = Duration::from_secs(5); /// What the status line says while the server is away. const AWAY: &str = "The berth server does not answer; trying again…"; -/// The key that leaves a running session: a browser keeps Ctrl+Q to itself. -const LEAVE: &str = "Alt+Q"; +/// The key that leaves a running session, and elsewhere quits. +const LEAVE: &str = "Ctrl+Q"; #[derive(PartialEq)] enum Screen { Token, - /// What a setup key in the link opens. + /// What a setup key opens. Installed, Hosts, Host, @@ -91,10 +91,8 @@ pub struct App { /// The portal has been open with this token, so it is the right one. proven: bool, retry: Option, - /// The server: where the page came from, or the one that was named. + /// The server that was named, or the one that was kept. server: String, - /// `berth serve` gave this page, so the server is not asked for. - embedded: bool, token: String, typing: Input, typing_server: Input, @@ -146,34 +144,41 @@ pub struct App { qr: Option, /// The token collected with a setup key, until Done. installed: Option, + /// A host to open once the server has listed its hosts. + wanted: Option, + /// The screens are left. + quit: bool, + /// An action of the open host to begin once the server has listed them. + pending: Option<&'static str>, + /// Why the server is too old for these screens, once it said its version. + older: Rc>>, } impl App { /// Not a `Default`: making the screens signs in and opens the portal. #[allow(clippy::new_without_default)] pub fn new() -> App { - let embedded = device::embedded(); let linked = device::linked(); let (linked_server, linked_token) = (linked.server, linked.token); let kept = device::stored(Kept::Server).unwrap_or_default(); - let server = match embedded { - true => device::origin(), - false => linked_server - .map(|server| device::address(&server)) - .unwrap_or_else(|| kept.clone()), - }; + let server = linked_server + .map(|server| device::address(&server)) + .unwrap_or_else(|| kept.clone()); // The token that was kept is the one of the server that was kept. let token = linked_token .or_else(|| { - (embedded || server == kept) + (server == kept) .then(|| device::stored(Kept::Token)) .flatten() }) .unwrap_or_default(); - if !embedded { + // A server on this machine is never kept: it is found again each run. + // Another server named for this run, without its token, leaves the + // kept server and token alone until it signs in. + if !device::local(&server) && !token.is_empty() { device::store(Kept::Server, &server); + device::store(Kept::Token, &token); } - device::store(Kept::Token, &token); let mut app = App { screen: Screen::Token, portal: None, @@ -182,7 +187,6 @@ impl App { on_server: server.is_empty(), typing_server: Input::new(server.clone()), server, - embedded, token, typing: Input::default(), refused: String::new(), @@ -216,6 +220,10 @@ impl App { over_links: vec![], qr: None, installed: None, + wanted: None, + quit: false, + pending: None, + older: Rc::default(), }; // A setup key signs in with the token it collects, whatever was kept. if let (Some(key), false) = (linked.setup_key, app.server.is_empty()) { @@ -223,19 +231,70 @@ impl App { app.screen = Screen::Installed; return app; } - if !app.token.is_empty() && !app.server.is_empty() { + if (!app.token.is_empty() && !app.server.is_empty()) || device::local(&app.server) { app.connect(); } app } + /// The server is on this machine, and the screens need no token for it. + fn local(&self) -> bool { + device::local(&self.server) + } + + /// Whether the screens were left, with Ctrl+Q or q on the list of hosts. + pub fn quit(&self) -> bool { + self.quit + } + + /// Open the host `name` once the server has listed it. + pub fn open_when_listed(&mut self, name: &str) { + self.wanted = Some(name.to_owned()); + } + + /// Open the host that was asked for, once the list has come. + fn open_wanted(&mut self) { + if !self.hosts.borrow().answered || self.screen != Screen::Hosts { + return; + } + let Some(name) = self.wanted.take() else { + return; + }; + let address = self + .hosts + .borrow() + .list + .iter() + .find(|host| host["name"] == name.as_str()) + .map(|host| host["address"].as_str().unwrap_or("").to_owned()); + match address { + Some(address) => self.open_host(&name, &address), + // A name the server does not know starts a new host, once it has + // said what a new host starts from. + None if self.admin.is_some() && !self.data.borrow().template.is_object() => { + self.wanted = Some(name); + } + None if self.admin.is_some() => self.name_host(&name), + None => self.status = format!("There is no host {name}"), + } + } + /// Open the portal with the token there is. fn connect(&mut self) { self.host.disconnect(); self.asked = None; self.retry = None; - match Portal::open(&device::portal_url(&self.server), &self.token) { + match Portal::open(&self.server, &self.token) { Ok(portal) => { + // A server older than these screens would not understand what they send. + let (older, url) = (self.older.clone(), self.server.clone()); + portal.call("version", None, json!({}), Value::Null, move |result| { + if let Ok(said) = result { + let version = said["version"].as_str().unwrap_or(""); + let api = said["api"].as_u64().unwrap_or(0) as u32; + *older.borrow_mut() = berth_api::compat::older_server(&url, version, api); + } + }); // Who the token is here decides what the page offers. let admin = Admin::new(portal.clone(), self.data.clone()); admin.load(); @@ -255,6 +314,13 @@ impl App { /// Back to the token dialog, forgetting the token. fn sign_out(&mut self, why: &str) { + self.ask_token(why); + device::store(Kept::Token, ""); + } + + /// Back to the token dialog. The kept token stays until the one typed + /// next replaces it. + fn ask_token(&mut self, why: &str) { self.portal = None; self.admin = None; *self.data.borrow_mut() = Data::default(); @@ -266,7 +332,6 @@ impl App { self.host.disconnect(); *self.hosts.borrow_mut() = Hosts::default(); self.token.clear(); - device::store(Kept::Token, ""); self.typing = Input::default(); self.refused = why.to_owned(); self.screen = Screen::Token; @@ -297,6 +362,7 @@ impl App { match installed.tick() { Outcome::SignIn(token) => { self.token = token; + device::store(Kept::Server, &self.server); device::store(Kept::Token, &self.token); self.connect(); } @@ -320,6 +386,16 @@ impl App { } else if !self.status.is_empty() && self.said.1.elapsed() >= SAID_FOR { self.status.clear(); } + if let Some(older) = self.older.borrow_mut().take() { + self.portal = None; + self.admin = None; + self.host.disconnect(); + *self.hosts.borrow_mut() = Hosts::default(); + self.retry = None; + self.refused = older; + self.screen = Screen::Token; + return; + } match self.portal.as_ref().map(Portal::link) { Some(Link::Open) => { self.proven = true; @@ -327,13 +403,14 @@ impl App { self.status.clear(); } } - // A token that never opened the portal is asked for again; one that - // has is the right one, and the server is only away. - Some(Link::Refused) if !self.proven => { - self.sign_out("The server did not open its portal for that token."); + // A token the server refused is asked for again. The kept one stays + // until another is given or the user signs out: a server that does + // not answer, or not yet, never costs the token. + Some(Link::Refused) if !self.local() => { + self.ask_token("The server did not take that token. Enter another, or sign out."); return; } - Some(Link::Refused | Link::Lost) => { + Some(Link::Refused | Link::Unreached | Link::Lost) => { self.portal = None; self.host.disconnect(); self.retry = Some(Instant::now()); @@ -348,6 +425,7 @@ impl App { return; } self.take_in(); + self.open_wanted(); if self .asked .is_none_or(|asked| asked.elapsed() >= HOSTS_EVERY) @@ -396,15 +474,6 @@ impl App { ) } - /// The server, and the link that opens this page signed in with its - /// token: at this page's own address, which is the server's when the - /// server gave it, and told which server otherwise. - fn connect_link(&self) -> Option<(String, String)> { - let server = (!self.embedded).then_some(self.server.as_str()); - let link = qr::connect_link(&device::page(), server, &self.token)?; - Some((self.server.clone(), link)) - } - fn is_admin(&self) -> bool { self.data.borrow().admin() } @@ -467,7 +536,7 @@ impl App { setup.machine.found(found); } } - // berth's own actions are beside the host's tasks; applying + // berth's own actions are beside the controller's; applying // and the seed have rows of their own. Arrived::Actions(name, listed) if name == self.name => { self.host.own = listed @@ -475,6 +544,7 @@ impl App { .filter(|action| action.runs == berth_api::Runs::Berth) .filter(|action| !["apply", "seed"].contains(&action.name.as_str())) .collect(); + self.begin_pending(); } Arrived::Actions(..) => {} // One that was removed takes its page back to the list. @@ -558,18 +628,24 @@ impl App { rows.push(Row::info("")); } rows.push(heading("Connection")); - rows.push(Row::info(format!( - " {} · through the berth server at {}", - self.address, - device::shown(&self.server) - ))); - for line in host::level_lines(&self.name, &self.host) { + let address = match self.address.as_str() { + "" => "no SSH address yet", + address => address, + }; + rows.push(Row::info(match self.local() { + true => format!(" {address}"), + false => format!( + " {address} · through the berth server at {}", + device::shown(&self.server) + ), + })); + for line in host::type_lines(&self.name, &self.host) { rows.push(Row::info(Line::styled( format!(" {line}"), Style::new().fg(theme::MUTED), ))); } - let workstation = self.host.level == berth_api::HostLevel::Workstation; + let workstation = self.host.host_type == berth_api::HostType::Workstation; let said = match &self.host.state { State::Ready(_) => { if let Some(text) = host::behind_note(&self.host) { @@ -605,10 +681,10 @@ impl App { State::Idle | State::Connecting | State::Password => { vec![note("Asking the host what it can do…".into(), theme::MUTED)] } - // Below a workstation the controller is installed alone, by berth's own action. + // On a host that is not a workstation the controller is installed alone, by berth's own action. State::Missing if !workstation => vec![ note( - "berth's controller is not on this host. Install it for sessions, tasks and files here.".into(), + "berth's controller is not on this host. Install it for sessions, actions and files here.".into(), theme::MUTED, ), again(), @@ -622,8 +698,10 @@ impl App { ], State::Old(version) => vec![ note( - format!( - "The host's tools ({version}) are older than this page needs. Applying its configuration, where berth serve runs, brings them up to date." + berth_api::compat::controller_is_older( + &self.name, + version, + self.host.host_type, ), theme::WARN, ), @@ -643,14 +721,14 @@ impl App { // A host that cannot be worked with yet still has berth's own // actions, such as first contact and installing the controller. let actions = self.host.actions(); - let mut groups: Vec<(String, Vec)> = berth_tui::page::groups(&actions, Pick::Task) + let mut groups: Vec<(String, Vec)> = berth_tui::page::groups(&actions, Pick::Action) .into_iter() .map(|group| { let picks = group .rows .into_iter() .filter( - |(_, pick)| matches!(pick, Pick::Task(index) if self.host.is_own(*index)), + |(_, pick)| matches!(pick, Pick::Action(index) if self.host.is_own(*index)), ) .map(|(line, pick)| Row { line: line.into(), @@ -722,7 +800,7 @@ impl App { fn help(&self) -> String { match self.selected() { - Some(Entry::Pick(Pick::Task(index))) => host::task_help(&self.host, index), + Some(Entry::Pick(Pick::Action(index))) => host::action_help(&self.host, index), Some(Entry::Pick(Pick::Session(index))) => host::session_help(&self.host, index), Some(Entry::Pick(Pick::Terminal)) => format!( "A shell on the host, as one of its sessions. Space switches between the agent and the SSH user.\n\nLeaving the shell closes it; {LEAVE} leaves it running, to be opened again from Sessions." @@ -740,12 +818,10 @@ impl App { self.host.disconnect(); let listed = self.hosts.borrow(); let found = listed.list.iter().find(|host| host["name"] == name); - let level = |key: &str| { - found - .and_then(|host| host[key].as_str()) - .and_then(berth_api::HostLevel::parse) - }; - self.host.level = level("level").unwrap_or_default(); + self.host.host_type = found + .and_then(|host| host["type"].as_str()) + .and_then(berth_api::HostType::parse) + .unwrap_or_default(); drop(listed); self.host.own.clear(); if let Some(admin) = &self.admin { @@ -769,21 +845,43 @@ impl App { } return; } - let body = self.host.task_body(&action, &values); + let body = self.host.action_body(&action, &values); self.status = format!("Starting {}…", action.title()); self.host.run(&action.name, body); } + /// Begin the action the set-up left the overview for, now that the + /// server has listed the host's actions. + fn begin_pending(&mut self) { + let Some(name) = self.pending else { + return; + }; + // A host saved just now may not be listed yet: its actions come again. + if self.host.own.is_empty() { + return; + } + self.pending = None; + let index = self + .host + .actions() + .iter() + .position(|action| action.name == name); + match index { + Some(index) => self.choose(Entry::Pick(Pick::Action(index))), + None => self.status = format!("The server offers no action {name} for {}", self.name), + } + } + fn choose(&mut self, entry: Entry) { match entry { - Entry::Pick(Pick::Task(index)) => { + Entry::Pick(Pick::Action(index)) => { if let Some(action) = self.host.actions().get(index) && !action.available { self.status = format!("{} cannot run now: {}", action.title(), action.why); return; } - if let Some((index, values)) = self.host.begin_task(index, false) { + if let Some((index, values)) = self.host.begin_action(index, false) { self.launch(index, values); } } @@ -808,7 +906,7 @@ impl App { } Entry::Setup => { if let Some(admin) = &self.admin { - self.setup = Some(Setup::open(&self.name, admin)); + self.setup = Some(Setup::open(&self.name, admin, self.local())); self.screen = Screen::Setup; } } @@ -893,7 +991,7 @@ impl App { match key.code { Code::Char('y' | 'Y') => match self.host.question.take() { Some((_, Pending::Action(index))) => { - if let Some((index, values)) = self.host.begin_task(index, true) { + if let Some((index, values)) = self.host.begin_action(index, true) { self.launch(index, values); } } @@ -969,7 +1067,16 @@ impl App { right, ); // The cursor only stops on rows that do something. - let rows = self.rows(); + let rows: Vec = berth_tui::setup::wrapped_rows( + self.rows() + .into_iter() + .map(|row| (row.line, row.entry)) + .collect(), + left.width, + ) + .into_iter() + .map(|(line, entry)| Row { line, entry }) + .collect(); let positions: Vec = rows .iter() .enumerate() @@ -1010,7 +1117,7 @@ impl App { }; draw_footer(frame, footer, keys, &self.status, theme::ACCENT); (self.links, self.words) = match &self.host.view { - Some(view) => draw_view(frame, view, &self.status, LEAVE), + Some(view) => draw_view(frame, view, &self.status), None => (vec![], vec![]), }; if let Some((question, _)) = &self.host.question { @@ -1088,7 +1195,7 @@ impl App { self.address.clear(); self.status = format!("{name} is kept on the server once it is saved"); if let Some(admin) = &self.admin { - self.setup = Some(Setup::new(name, template, admin)); + self.setup = Some(Setup::new(name, template, admin, self.local())); self.screen = Screen::Setup; } } @@ -1145,7 +1252,8 @@ impl App { } _ => {} }, - Code::Char('o') => self.sign_out(""), + Code::Char('o') if !self.local() => self.sign_out(""), + Code::Char('q') => self.quit = true, _ => {} } } @@ -1240,9 +1348,6 @@ impl App { if let Some(said) = self.home.key(tab - 1, key, &admin) { self.status = said; } - if let Some(shown) = self.home.qr.take() { - self.qr = Some(shown); - } if std::mem::take(&mut self.home.to_hosts) { self.open_tab(0); } @@ -1264,7 +1369,6 @@ impl App { if !admin { self.tab = 0; } - self.home.connect = self.connect_link(); self.home.server = device::shown(&self.server).to_owned(); let tabs: &[&str] = if admin { &TABS } else { &[] }; self.tab_hits = draw_header(frame, header, tabs, self.tab, device::shown(&self.server)); @@ -1350,7 +1454,15 @@ impl App { let mut lines: Vec = vec![]; for host in hosts.list.iter() { let name = host["name"].as_str().unwrap_or(""); + let older = host["api"] + .as_u64() + .is_some_and(|api| api < u64::from(berth_api::compat::NEEDS_HOST_API)); let state = match (host["reachable"].as_bool(), host["version"].as_str()) { + // A controller too old for this berth is updated before it is worked with. + (Some(true), Some(version)) if older => Span::styled( + format!("● {version} old controller"), + Style::new().fg(theme::WARN), + ), (Some(true), Some(version)) => { Span::styled(format!("● {version}"), Style::new().fg(theme::GOOD)) } @@ -1378,11 +1490,11 @@ impl App { if host["behind"].is_object() { line.push(Span::styled(" behind", Style::new().fg(theme::WARN))); } - if let Some(level) = host["level"] + if let Some(host_type) = host["type"] .as_str() - .filter(|level| *level != "workstation") + .filter(|host_type| *host_type != "workstation") { - line.push(Span::styled(format!(" {level}"), muted)); + line.push(Span::styled(format!(" {host_type}"), muted)); } if host["rotating"] == true { line.push(Span::styled( @@ -1419,6 +1531,7 @@ impl App { let text = match (&hosts.error, hosts.answered) { (Some(error), _) => format!("✗ {error}"), (None, false) => "Asking the server for its hosts…".into(), + (None, true) if self.local() => "No hosts yet.".into(), (None, true) => { "The server has no hosts yet. They are set up with berth where it runs, or from here with the role admin.".into() } @@ -1438,12 +1551,15 @@ impl App { if let Some(input) = &self.naming { draw_editor(frame, edit, " New host name ", input, false); } - let keys = match (admin, self.naming.is_some()) { - (_, true) => "Enter create · Esc cancel · Ctrl+O copy", - (true, false) => { - "↑↓ select · Enter open · n new host · d default · x remove · r refresh · o sign out" + let keys = match (admin, self.naming.is_some(), self.local()) { + (_, true, _) => "Enter create · Esc cancel · Ctrl+O copy", + (true, false, true) => { + "↑↓ select · Enter open · n new host · d default · x remove · r refresh · q quit" } - (false, false) => "↑↓ select · Enter open · r refresh · o sign out", + (true, false, false) => { + "↑↓ select · Enter open · n new host · d default · x remove · r refresh · o sign out · q quit" + } + (false, false, _) => "↑↓ select · Enter open · r refresh · o sign out · q quit", }; draw_footer(frame, footer, keys, &self.status, theme::ACCENT); } @@ -1596,27 +1712,21 @@ impl App { fn token_event(&mut self, event: Event) { match event { Event::Key(key) - if !self.embedded - && matches!(key.code, Code::Tab | Code::BackTab | Code::Up | Code::Down) => + if matches!(key.code, Code::Tab | Code::BackTab | Code::Up | Code::Down) => { self.on_server = !self.on_server; } Event::Key(key) if key.code == Code::Enter && self.on_server => self.on_server = false, Event::Key(key) if key.code == Code::Enter => { let token = self.typing.value().trim().to_owned(); - let server = match self.embedded { - true => self.server.clone(), - false => device::address(self.typing_server.value()), - }; + let server = device::address(self.typing_server.value()); if server.is_empty() { self.on_server = true; } else if !token.is_empty() { self.token = token; self.server = server; - if !self.embedded { - device::store(Kept::Server, &self.server); - self.typing_server = Input::new(self.server.clone()); - } + device::store(Kept::Server, &self.server); + self.typing_server = Input::new(self.server.clone()); device::store(Kept::Token, &self.token); self.typing = Input::default(); self.refused.clear(); @@ -1683,16 +1793,10 @@ impl App { self.draw_backdrop(frame, area); let masked = ellipsize(&"•".repeat(self.typing.value().chars().count()), 44); let named = ellipsize(self.typing_server.value(), 44); - let mut lines = match self.embedded { - true => vec![ - Line::raw("The token of this berth server. It is in serve.token,"), - Line::raw("in the data directory of the berth that serves."), - ], - false => vec![ - Line::raw("The address of a berth serve, and its token. The token"), - Line::raw("is in serve.token, in that berth's data directory."), - ], - }; + let mut lines = vec![ + Line::raw("The address of a berth serve, and its token. The token"), + Line::raw("is in serve.token, in that berth's data directory."), + ]; if !self.refused.is_empty() { for line in screen::wrap(&self.refused, 56) { lines.push(Line::styled(line, Style::new().fg(theme::WARN))); @@ -1709,16 +1813,11 @@ impl App { Span::raw(value.to_owned()), ]) }; - if !self.embedded { - lines.push(field("Server ", &named, self.on_server)); - } + lines.push(field("Server ", &named, self.on_server)); let server_row = lines.len() as u16; lines.push(field("Token ", &masked, !self.on_server)); let rows = lines.len() as u16; - let keys = match self.embedded { - true => " Enter connect · Ctrl+O copy ", - false => " Tab switch · Enter connect · Ctrl+O copy ", - }; + let keys = " Tab switch · Enter connect · Ctrl+O copy "; let area = dialog(frame, " Connect ", keys, lines); if area.width > 14 && area.height > rows { let (typed, row) = match self.on_server { @@ -1734,6 +1833,15 @@ impl App { pub fn event(&mut self, event: Event) { let event = hint_click(event); + // Over a session, Ctrl+Q leaves the session; everywhere else it quits. + let in_session = self.screen == Screen::Host && self.host.view.is_some(); + if let Event::Key(key) = &event + && key.is_ctrl('q') + && !in_session + { + self.quit = true; + return; + } // Alt+R over a panel shows its links as QR codes. let over = self.seed.is_some() || self.data.borrow().job.is_some(); let links = match ( @@ -1766,11 +1874,16 @@ impl App { self.screen = Screen::Hosts; return; }; - if let Flow::Leave = setup.event(event, admin, &mut self.status) { + let flow = setup.event(event, admin, &mut self.status); + if let Flow::Leave | Flow::Act(_) = flow { // A host the server never got has no overview to go back to. - let kept = setup.kept(); + let acting = matches!(flow, Flow::Act(_)); + let kept = setup.kept() || acting; self.setup = None; self.asked = None; + if let Flow::Act(action) = flow { + self.pending = Some(action); + } match kept { true => { let (name, address) = (self.name.clone(), self.address.clone()); diff --git a/crates/berth-page/src/device.rs b/crates/berth-page/src/device.rs index e34848e..0032dca 100644 --- a/crates/berth-page/src/device.rs +++ b/crates/berth-page/src/device.rs @@ -1,8 +1,9 @@ -//! What the screens need from the device they run on, which a browser and a -//! terminal give in their own ways: the clipboard, links, files, what is kept -//! between runs, plain HTTP and the portal's socket. The program installs its -//! [`Device`] before it makes the [`App`](crate::App). +//! What the screens take from the program and the system they run on: what +//! the program was started with, what is kept between runs, the clipboard, +//! links, files, and plain HTTP. +use crate::settings::{self, Settings}; +use crate::{net, terminal}; use berth_tui::screen::Platform; use std::cell::RefCell; use std::rc::Rc; @@ -16,7 +17,7 @@ pub enum Kept { Token, } -/// What the program was started with, such as from a link to the page. +/// What the program was started with. #[derive(Clone, Default)] pub struct Linked { pub server: Option, @@ -25,135 +26,137 @@ pub struct Linked { pub setup_key: Option, } -/// What the portal's socket says, handed over on the thread that draws. -pub enum Said { - Open, - Text(String), - Bytes(Vec), - /// The socket is closed, or never opened. - Closed, -} - -/// An open socket; dropping it closes it. -pub trait Socket { - fn send_text(&self, text: &str) -> bool; - fn send_bytes(&self, bytes: &[u8]) -> bool; -} - -pub type Answer = Box)>; - -pub trait Device { - fn copy(&self, text: &str) -> bool; - /// What is on the clipboard, where it can be read right away. - fn paste(&self) -> Option; - fn open(&self, link: &str) -> Result<(), String>; - /// Let the user choose a file whose name ends with one of `accept` - /// (comma-separated), and hand over its name and bytes. Nothing happens - /// when none is chosen; an error says why this device cannot ask. - fn choose_file( - &self, - accept: &str, - then: Box)>, - ) -> Result<(), String>; - fn stored(&self, key: Kept) -> Option; - /// Keep `value`, or forget what was kept when it is empty. - fn store(&self, key: Kept, value: &str); - /// Whether the server that gave these screens is the one to use. - fn embedded(&self) -> bool; - fn linked(&self) -> Linked; - /// The address of the server that gave these screens, when it did. - fn origin(&self) -> String; - /// The page a link for a phone opens. - fn page(&self) -> String; - /// One plain HTTP request with a bearer; `then` gets its status and body. - fn request(&self, method: &str, url: &str, bearer: &str, then: Answer); - /// Open a WebSocket to `url` with `token` as its protocol. `said` gets - /// what it says, on the thread that draws, from a later turn on. - fn connect( - &self, - url: &str, - token: &str, - said: Box, - ) -> Result, String>; +impl Linked { + /// What the environment `var` gives names: `BERTH_SERVER` and + /// `BERTH_TOKEN` this once. Secrets come from the environment or a file, + /// not the command line, where other users could read them. + pub fn from(var: impl Fn(&str) -> Option) -> Linked { + let variable = |name: &str| { + var(name) + .map(|value| value.trim().to_owned()) + .filter(|value| !value.is_empty()) + }; + Linked { + server: variable("BERTH_SERVER"), + token: variable("BERTH_TOKEN"), + setup_key: None, + } + } + + /// Whether no server is named, here or in `settings`, so that the + /// screens use one on this machine. + pub fn local(&self, settings: &Settings) -> bool { + self.server.is_none() && settings.get(settings::SERVER).is_none() + } +} + +/// What the screens keep between runs, and what the program was started +/// with. The program installs it before it makes the [`App`](crate::App). +struct Device { + linked: Linked, + settings: Settings, } thread_local! { - static DEVICE: RefCell>> = const { RefCell::new(None) }; + static DEVICE: RefCell>> = const { RefCell::new(None) }; } -/// Make `device` the one the screens use on this thread. -pub fn install(device: impl Device + 'static) { - DEVICE.set(Some(Rc::new(device))); +/// Start the screens of this thread with `linked`, keeping the server and +/// its token in `settings`. +pub fn install(linked: Linked, settings: Settings) { + DEVICE.set(Some(Rc::new(Device { linked, settings }))); } -pub(crate) fn device() -> Rc { +fn device() -> Rc { DEVICE .with_borrow(Clone::clone) .expect("the program installs its device before the screens start") } -/// The device as the shared screens of berth-tui take it. +/// The system as the shared screens of berth-tui take it. pub(crate) struct Here; impl Platform for Here { fn copy(&mut self, text: &str) -> bool { - device().copy(text) + terminal::copy_text(text) } fn paste(&mut self) -> Option { - device().paste() + terminal::paste_text() } fn open(&mut self, link: &str) -> Result<(), String> { - device().open(link) + terminal::open(link) } } -pub(crate) fn embedded() -> bool { - device().embedded() -} - pub(crate) fn linked() -> Linked { - device().linked() -} - -pub(crate) fn origin() -> String { - device().origin() + device().linked.clone() } -pub(crate) fn page() -> String { - device().page() +fn setting(key: Kept) -> &'static str { + match key { + Kept::Server => settings::SERVER, + Kept::Token => settings::TOKEN, + } } pub(crate) fn stored(key: Kept) -> Option { - device().stored(key).filter(|value| !value.is_empty()) + device().settings.get(setting(key)) } +/// Keep `value`, or forget what was kept when it is empty. The screens go +/// on without it when it cannot be written: the next run asks again. pub(crate) fn store(key: Kept, value: &str) { - device().store(key, value) + let _ = device().settings.set(&[(setting(key), value)]); } pub(crate) fn choose_file( accept: &str, then: impl FnOnce(String, Vec) + 'static, ) -> Result<(), String> { - device().choose_file(accept, Box::new(then)) + terminal::choose_file(accept, then) } +/// One plain HTTP request to `path` of `server` with a bearer; `then` gets +/// its status and body at a later turn. pub(crate) fn request( method: &str, - url: &str, + server: &str, + path: &str, bearer: &str, then: impl FnOnce(Result<(u16, String), String>) + 'static, ) { - device().request(method, url, bearer, Box::new(then)) + let (method, server, path, bearer) = ( + method.to_owned(), + server.to_owned(), + path.to_owned(), + bearer.to_owned(), + ); + net::later( + async move { net::http(&method, &server, &path, &bearer).await }, + then, + ); +} + +/// The server that runs in the program itself, as the screens name it. +pub const HERE: &str = "here:"; + +/// Whether the server is on this machine, reached without a token: the one +/// in the program, or the socket of a `berth serve` of this user. +pub fn local(server: &str) -> bool { + server == HERE || server.starts_with("unix:") } /// A server's address as typed, made whole: without a scheme it is https, /// or http for this machine itself. pub fn address(typed: &str) -> String { let typed = typed.trim().trim_end_matches('/'); - if typed.is_empty() || typed.starts_with("http://") || typed.starts_with("https://") { + if typed.is_empty() + || typed.starts_with("http://") + || typed.starts_with("https://") + || local(typed) + { return typed.to_owned(); } let local = ["localhost", "127.", "[::1]"] @@ -162,26 +165,49 @@ pub fn address(typed: &str) -> String { format!("{}://{typed}", if local { "http" } else { "https" }) } -/// A server as it is shown: `host:port`. +/// A server as it is shown: `host:port`, or where on this machine it is. pub fn shown(server: &str) -> &str { + if server == HERE { + return "this machine"; + } + if server.starts_with("unix:") { + return "berth serve on this machine"; + } server .trim_start_matches("https://") .trim_start_matches("http://") } -/// The address of the portal of a server. -pub fn portal_url(server: &str) -> String { - let scheme = match server.starts_with("https://") { - true => "wss", - false => "ws", - }; - format!("{scheme}://{}{}", shown(server), berth_api::ops::PORTAL) -} - #[cfg(test)] mod tests { use super::*; + #[test] + fn a_server_named_or_kept_is_used_and_else_one_here() { + let scratch = tempfile::tempdir().unwrap(); + let settings = Settings::new(Some(scratch.path().to_owned())); + let env = |pairs: &'static [(&'static str, &'static str)]| { + move |name: &str| { + pairs + .iter() + .find(|(key, _)| *key == name) + .map(|(_, value)| value.to_string()) + } + }; + assert!(Linked::from(env(&[])).local(&settings)); + assert!(Linked::from(env(&[("BERTH_SERVER", " ")])).local(&settings)); + let named = Linked::from(env(&[ + ("BERTH_SERVER", "https://studio.example"), + ("BERTH_TOKEN", "s3cret\n"), + ])); + assert!(!named.local(&settings)); + assert_eq!(named.token.as_deref(), Some("s3cret")); + settings + .set(&[(settings::SERVER, "http://192.0.2.10:7878")]) + .unwrap(); + assert!(!Linked::from(env(&[])).local(&settings)); + } + #[test] fn addresses() { assert_eq!( @@ -191,13 +217,12 @@ mod tests { assert_eq!(address("127.0.0.1:7878/"), "http://127.0.0.1:7878"); assert_eq!(address(" https://192.0.2.10/ "), "https://192.0.2.10"); assert_eq!(shown("https://studio.example:8080"), "studio.example:8080"); + assert_eq!(address("here:"), "here:"); assert_eq!( - portal_url("https://studio.example"), - "wss://studio.example/v1/portal" - ); - assert_eq!( - portal_url("http://192.0.2.10:7878"), - "ws://192.0.2.10:7878/v1/portal" + address("unix:/run/berth/serve.sock"), + "unix:/run/berth/serve.sock" ); + assert!(local(HERE) && !local("https://studio.example")); + assert_eq!(shown(HERE), "this machine"); } } diff --git a/crates/berth-page/src/home.rs b/crates/berth-page/src/home.rs index 34d301f..44d4b6b 100644 --- a/crates/berth-page/src/home.rs +++ b/crates/berth-page/src/home.rs @@ -6,7 +6,7 @@ use crate::admin::{Admin, Data}; use crate::device::Here; use berth_tui::berth_key; use berth_tui::fields::Kind; -use berth_tui::plugins::{self, Asked, Place}; +use berth_tui::plugins::{self, Asked}; use berth_tui::provisioners; use berth_tui::screen; use berth_tui::setup::{valid_name, value_line}; @@ -230,13 +230,8 @@ pub struct Home { pub provisioners: provisioners::Tab, /// Set when the hosts' list is to be shown again, for the page to take. pub to_hosts: bool, - /// The server, and the link that opens this page signed in, when there - /// is one to give: a row of the Settings tab. - pub connect: Option<(String, String)>, /// The address of the server, as the header shows it. pub server: String, - /// A QR code to show, for the page to take. - pub qr: Option, } /// The rows of the Settings tab after the server's own settings. @@ -244,7 +239,6 @@ pub struct Home { enum SettingsRow { /// A key of the server's ring, by its place in the list. Key(usize), - Connect, } /// The tabs after Hosts, in their order. @@ -382,9 +376,6 @@ impl Home { return extras; } extras.extend((0..data.keys.len()).map(SettingsRow::Key)); - if self.connect.is_some() { - extras.push(SettingsRow::Connect); - } extras } @@ -418,38 +409,27 @@ impl Home { SettingsRow::Key(index) => { berth_key::row(&data.keys[*index].0, &data.keys[*index].1) } - SettingsRow::Connect => Line::from(vec![ - Span::styled("Open on another device ", Style::new().fg(theme::TEXT)), - Span::styled("QR code and link, with the token", muted), - ]), }); } let extra = self.selected[SETTINGS] .checked_sub(data.settings.len()) .and_then(|index| extras.get(index).copied()); - let help = match ( - data.settings.get(self.selected[SETTINGS]), - extra, - &self.connect, - ) { + let help = match (data.settings.get(self.selected[SETTINGS]), extra) { (Some(setting), ..) => { let meaning = berth_tui::screen::describe_setting(setting["meaning"].as_str().unwrap_or("")); match setting["open"] == true { true => format!("{meaning}.\n\nLeave it empty to use the default."), false => format!( - "{meaning}.\n\nIt says who gets in or where the server's files are, so it is set on the server itself: `berth config {} VALUE`. A token is never shown.", + "{meaning}.\n\nIt says who gets in or where the server's files are, so it is set on the server itself: `berth config set {} VALUE`. A token is never shown.", setting["key"].as_str().unwrap_or("") ), } } - (None, Some(SettingsRow::Key(index)), _) => { + (None, Some(SettingsRow::Key(index))) => { let (name, public) = &data.keys[index]; berth_key::help(name, public, Some(&self.server)) } - (None, Some(SettingsRow::Connect), Some((server, _))) => { - berth_tui::qr::connect_help(server, false) - } _ => "Asking the server for its settings…".into(), }; Shown { @@ -460,9 +440,6 @@ impl Home { keys: match (&self.editor, extra) { (Some(_), _) => "Enter accept · Esc cancel · Ctrl+O copy", (None, Some(SettingsRow::Key(_))) => "↑↓ select · Enter copy the key · Esc hosts", - (None, Some(SettingsRow::Connect)) => { - "↑↓ select · Enter show the QR code · Esc hosts" - } (None, None) => "↑↓ select · Enter change · Esc hosts", }, } @@ -502,14 +479,6 @@ impl Home { let public = keys.get(index).map(|(_, public)| public.as_str()); return Some(berth_key::copy(&mut Here, public.unwrap_or(""))); } - // The last row shows the link that opens this page. - if extra == Some(SettingsRow::Connect) - && (key.code == Code::Enter || berth_tui::qr::is_key(&key)) - && let Some((_, link)) = &self.connect - { - self.qr = Some(berth_tui::qr::Overlay::connect(link.clone())); - return None; - } match key.code { Code::Up => self.selected[SETTINGS] = selected.saturating_sub(1), Code::Down => self.selected[SETTINGS] = (selected + 1).min(count.saturating_sub(1)), @@ -924,14 +893,11 @@ impl Home { let listed = plugins::rows(&data.plugins, &data.updates); let selected = self.plugins.selected(&data.plugins); let plugin = listed[selected].0.map(|index| &data.plugins[index]); - let help = plugins::help(plugin, &data.report, &Place::Page.adding(&data.plugin_dir)); + let help = plugins::help(plugin, &data.report, &plugins::adding(&data.plugin_dir)); Shown { rows: listed.into_iter().map(|(_, line)| line).collect(), selected, - editor: self - .editor - .as_ref() - .map(|_| (Place::Page.editor_title(), false)), + editor: self.editor.as_ref().map(|_| (plugins::EDITOR_TITLE, false)), help, keys: match self.editor { Some(_) => "Enter install · Esc cancel · Ctrl+O copy", @@ -957,7 +923,7 @@ impl Home { } let asked = self .plugins - .key(key, &mut self.editor, &listed, &mut report, &Place::Page); + .key(key, &mut self.editor, &listed, &mut report); admin.data.borrow_mut().report = report; match asked { Asked::Nothing(said) => return said, diff --git a/crates/berth-page/src/installed.rs b/crates/berth-page/src/installed.rs index 820d480..d041d7d 100644 --- a/crates/berth-page/src/installed.rs +++ b/crates/berth-page/src/installed.rs @@ -1,6 +1,6 @@ -//! The screen a setup key opens, after an install: berth is installed, this -//! browser is signed in, and the token for a terminal. The key works until -//! Done, so opening the link again before then shows the screen again. +//! The screen a setup key opens, after an install: berth is installed, these +//! screens are signed in, and the token for berth. The key works until Done, +//! so starting again with it before then shows the screen again. use crate::device::{self, Here}; use berth_tui::screen::{Platform, note_hints, wrap}; @@ -49,12 +49,8 @@ impl Installed { pub fn claim(server: &str, key: String) -> Installed { let claim = Rc::new(RefCell::new(Claim::Asking)); let answered = claim.clone(); - device::request( - "POST", - &format!("{server}/v1/setup-key"), - &key, - move |answer| { - *answered.borrow_mut() = match answer { + device::request("POST", server, "/v1/setup-key", &key, move |answer| { + *answered.borrow_mut() = match answer { Ok((200, body)) => { let said: Value = serde_json::from_str(&body).unwrap_or(Value::Null); let text = |name: &str| said[name].as_str().unwrap_or("").to_owned(); @@ -67,7 +63,7 @@ impl Installed { } } Ok((401, _)) => Claim::Failed( - "The setup key in the link was not taken: it is wrong, used up or expired. Make a new one where the server runs (berth setup-key), or give the token." + "The setup key was not taken: it is wrong, used up or expired. Make a new one where the server runs (berth server setup-key), or give the token." .into(), ), Ok((status, body)) => { @@ -79,8 +75,7 @@ impl Installed { } Err(error) => Claim::Failed(format!("The server did not answer: {error}")), }; - }, - ); + }); Installed { server: server.to_owned(), key, @@ -108,11 +103,10 @@ impl Installed { } } - fn commands(&self, token: &str) -> [String; 2] { - [ - format!("berth config server {}", self.server), - format!("berth config server_token {token}"), - ] + /// What uses this server from berth on another machine: the command, + /// which reads the token from standard input, so it is in no argv. + fn commands(&self, _token: &str) -> [String; 1] { + [format!("berth login {} --token-stdin", self.server)] } /// A key on the screen, and what the status line then says. @@ -140,13 +134,14 @@ impl Installed { pub fn done(&self, then: impl FnOnce(Option) + 'static) { device::request( "DELETE", - &format!("{}/v1/setup-key", self.server), + &self.server, + "/v1/setup-key", &self.key, move |answer| { then(match answer { Ok((200, _)) => None, _ => Some( - "The setup key was not ended. It stops by itself within a day, or with berth setup-key --remove where the server runs." + "The setup key was not ended. It stops by itself within a day, or with berth server setup-key --revoke where the server runs." .into(), ), }) diff --git a/crates/berth-remote/src/keys.rs b/crates/berth-page/src/keys.rs similarity index 100% rename from crates/berth-remote/src/keys.rs rename to crates/berth-page/src/keys.rs diff --git a/crates/berth-page/src/lib.rs b/crates/berth-page/src/lib.rs index 00b4ca3..e8ef721 100644 --- a/crates/berth-page/src/lib.rs +++ b/crates/berth-page/src/lib.rs @@ -1,14 +1,26 @@ -//! The page of `berth serve`: the hosts of a server, its plugins, settings -//! and setup, through its portal. A browser draws it (berth-web) and so does -//! a terminal (berth-remote); each gives it a [`device::Device`]. +//! The screens of berth: the hosts of a `berth serve`, its plugins, settings +//! and setup, through its portal, in a terminal. The server is elsewhere, +//! the socket of a `berth serve` on this machine, or one in the program. mod admin; mod app; pub mod device; mod home; mod installed; +pub mod keys; +pub mod net; pub mod portal; mod remote; +pub mod screens; +pub mod settings; mod setup; +pub mod socket; +pub mod terminal; pub use app::App; + +/// Take in what the network said since the last turn, on the thread that draws. +pub fn pump() { + net::pump(); + socket::pump(); +} diff --git a/crates/berth-remote/src/net.rs b/crates/berth-page/src/net.rs similarity index 56% rename from crates/berth-remote/src/net.rs rename to crates/berth-page/src/net.rs index 7d35485..aea92e6 100644 --- a/crates/berth-remote/src/net.rs +++ b/crates/berth-page/src/net.rs @@ -5,13 +5,9 @@ use berth_api::client::{self, Error}; use std::cell::RefCell; use std::future::Future; use std::sync::{Arc, OnceLock, mpsc}; -use tokio::io::{AsyncRead, AsyncWrite}; use tokio::net::TcpStream; -/// A connection, over TLS or not. -pub trait Io: AsyncRead + AsyncWrite + Unpin + Send {} -impl Io for T {} -pub type Stream = Box; +pub use berth_api::client::Stream; /// The runtime the connections run on; the screens stay on the main thread. pub fn runtime() -> &'static tokio::runtime::Runtime { @@ -25,34 +21,65 @@ pub fn runtime() -> &'static tokio::runtime::Runtime { }) } -/// Where an address goes: the name and port to connect to, the authority to -/// name in requests, the path after it, and whether over TLS. +/// How a server is reached. +pub enum Way { + /// Over TCP, with TLS when `secure`. + Network { + name: String, + port: u16, + secure: bool, + }, + /// The socket of a `berth serve` on this machine. + Socket(std::path::PathBuf), + /// The server in this process. + Here, +} + +/// Where a request goes: how the server is reached, the authority to name in +/// requests, and the path. pub struct Target { - pub name: String, - pub port: u16, + pub way: Way, pub authority: String, pub path: String, - pub secure: bool, } -/// Take an `http(s)://` or `ws(s)://` address apart. -pub fn target(url: &str) -> Result { - let (secure, rest) = if let Some(rest) = url - .strip_prefix("https://") - .or_else(|| url.strip_prefix("wss://")) - { +/// The server in this process, when the program runs one. +static HERE: std::sync::RwLock> = std::sync::RwLock::new(None); + +/// Reach `here:` through `dial`, until another is given. +pub fn serve_here(dial: client::Dial) { + *HERE + .write() + .unwrap_or_else(|poisoned| poisoned.into_inner()) = Some(dial); +} + +/// Where `path` of `server` is: `http(s)://HOST[:PORT]`, `unix:/path` or `here:`. +pub fn target(server: &str, path: &str) -> Result { + if server == crate::device::HERE { + return Ok(Target { + way: Way::Here, + authority: "berth".into(), + path: path.to_owned(), + }); + } + if let Some(socket) = server.strip_prefix("unix:") { + return Ok(Target { + way: Way::Socket(socket.into()), + authority: "berth".into(), + path: path.to_owned(), + }); + } + let (secure, rest) = if let Some(rest) = server.strip_prefix("https://") { (true, rest) - } else if let Some(rest) = url - .strip_prefix("http://") - .or_else(|| url.strip_prefix("ws://")) - { + } else if let Some(rest) = server.strip_prefix("http://") { (false, rest) } else { - return Err(format!("{url} is no http:// or https:// address")); + return Err(format!("{server} is no http:// or https:// address")); }; - let (authority, path) = match rest.find('/') { - Some(at) => (&rest[..at], &rest[at..]), - None => (rest, "/"), + // What follows the authority is the server's own prefix, before the path. + let (authority, prefix) = match rest.find('/') { + Some(at) => (&rest[..at], rest[at..].trim_end_matches('/')), + None => (rest, ""), }; let default = if secure { 443 } else { 80 }; let (host, port) = match authority.rsplit_once(':') { @@ -64,14 +91,16 @@ pub fn target(url: &str) -> Result { _ => (authority, default), }; if host.is_empty() { - return Err(format!("{url} names no host")); + return Err(format!("{server} names no host")); } Ok(Target { - name: host.trim_matches(['[', ']']).to_owned(), - port, + way: Way::Network { + name: host.trim_matches(['[', ']']).to_owned(), + port, + secure, + }, authority: format!("{host}:{port}"), - path: path.to_owned(), - secure, + path: format!("{prefix}{path}"), }) } @@ -98,15 +127,40 @@ fn tls() -> Arc { /// Connect to where `target` points, with TLS when it says so. pub async fn connect(target: &Target) -> Result { - let stream = TcpStream::connect((target.name.as_str(), target.port)) + let (name, port, secure) = match &target.way { + Way::Here => { + let dial = HERE + .read() + .unwrap_or_else(|poisoned| poisoned.into_inner()) + .clone() + .ok_or("There is no server in this program")?; + return dial().await.map_err(|error| describe(&error)); + } + #[cfg(unix)] + Way::Socket(path) => { + let stream = tokio::net::UnixStream::connect(path) + .await + .map_err(|error| format!("{}: {error}", path.display()))?; + return Ok(Box::new(stream)); + } + #[cfg(not(unix))] + Way::Socket(path) => { + return Err(format!( + "{} is a socket, which this system has none of", + path.display() + )); + } + Way::Network { name, port, secure } => (name, *port, *secure), + }; + let stream = TcpStream::connect((name.as_str(), port)) .await .map_err(|error| format!("{}: {error}", target.authority))?; let _ = stream.set_nodelay(true); - if !target.secure { + if !secure { return Ok(Box::new(stream)); } - let name = rustls::pki_types::ServerName::try_from(target.name.clone()) - .map_err(|error| error.to_string())?; + let name = + rustls::pki_types::ServerName::try_from(name.clone()).map_err(|error| error.to_string())?; let stream = tokio_rustls::TlsConnector::from(tls()) .connect(name, stream) .await @@ -115,8 +169,13 @@ pub async fn connect(target: &Target) -> Result { } /// One request with a bearer: the status and the body of the answer. -pub async fn http(method: &str, url: &str, bearer: &str) -> Result<(u16, String), String> { - let target = target(url)?; +pub async fn http( + method: &str, + server: &str, + path: &str, + bearer: &str, +) -> Result<(u16, String), String> { + let target = target(server, path)?; let stream = connect(&target).await?; let mut client = client::Client::over(stream) .await @@ -208,28 +267,41 @@ pub fn pump() { mod tests { use super::*; + fn network(target: &Target) -> (&str, u16, bool) { + match &target.way { + Way::Network { name, port, secure } => (name, *port, *secure), + _ => panic!("not over the network"), + } + } + #[test] fn addresses_are_taken_apart() { - let at = target("wss://studio.example/v1/portal").unwrap(); + let at = target("https://studio.example", "/v1/portal").unwrap(); + assert_eq!(network(&at), ("studio.example", 443, true)); assert_eq!( - (at.name.as_str(), at.port, at.authority.as_str()), - ("studio.example", 443, "studio.example:443") + (at.authority.as_str(), at.path.as_str()), + ("studio.example:443", "/v1/portal") ); - assert_eq!((at.path.as_str(), at.secure), ("/v1/portal", true)); - let at = target("http://192.0.2.10:7878").unwrap(); - assert_eq!((at.name.as_str(), at.port), ("192.0.2.10", 7878)); - assert_eq!((at.path.as_str(), at.secure), ("/", false)); - let at = target("ws://[2001:db8::1]:7878/v1/portal").unwrap(); - assert_eq!((at.name.as_str(), at.port), ("2001:db8::1", 7878)); - let at = target("https://[2001:db8::1]/").unwrap(); - assert_eq!((at.name.as_str(), at.port), ("2001:db8::1", 443)); + let at = target("http://192.0.2.10:7878", "/v1/access").unwrap(); + assert_eq!(network(&at), ("192.0.2.10", 7878, false)); + let at = target("http://[2001:db8::1]:7878/berth/", "/v1/portal").unwrap(); + assert_eq!(network(&at), ("2001:db8::1", 7878, false)); + assert_eq!(at.path, "/berth/v1/portal"); + let at = target("https://[2001:db8::1]", "/").unwrap(); + assert_eq!(network(&at).1, 443); + let at = target("unix:/run/user/1000/berth/serve.sock", "/v1/portal").unwrap(); + assert!(matches!(&at.way, Way::Socket(path) if path.ends_with("serve.sock"))); + assert!(matches!( + target("here:", "/v1/portal").unwrap().way, + Way::Here + )); for wrong in [ "studio.example", "ftp://studio.example", "https://", "http://studio:port", ] { - assert!(target(wrong).is_err(), "{wrong}"); + assert!(target(wrong, "/").is_err(), "{wrong}"); } } } diff --git a/crates/berth-page/src/portal.rs b/crates/berth-page/src/portal.rs index 7fb8848..e70be8c 100644 --- a/crates/berth-page/src/portal.rs +++ b/crates/berth-page/src/portal.rs @@ -1,7 +1,7 @@ //! The server's portal over the device's WebSocket: calls that are answered //! once, and streams that send frames until either side ends them. -use crate::device::{self, Said, Socket}; +use crate::socket::{self, Said, Socket}; use serde_json::{Value, json}; use std::cell::{Cell, RefCell}; use std::collections::HashMap; @@ -12,8 +12,11 @@ use std::rc::{Rc, Weak}; pub enum Link { Connecting, Open, - /// It closed before it was open: a wrong token, or nobody there. + /// The server refused the token. Refused, + /// It closed before it was open, and not by the server's refusal: nobody + /// answered there, or not in time. + Unreached, /// It was open and is not any more. Lost, } @@ -89,6 +92,10 @@ impl Inner { } } Said::Closed => self.closed(), + Said::Refused => { + self.link.set(Link::Refused); + self.closed(); + } } } @@ -115,7 +122,8 @@ impl Inner { fn closed(&self) { self.link.set(match self.link.get() { Link::Open | Link::Lost => Link::Lost, - _ => Link::Refused, + Link::Refused => Link::Refused, + Link::Connecting | Link::Unreached => Link::Unreached, }); let calls: Vec = self .calls @@ -172,9 +180,20 @@ fn request(id: u32, op: &str, host: Option<&str>, args: Value, data: Value) -> V } impl Portal { - /// Open the portal at `url`, showing the token the way a browser can: as - /// the WebSocket's protocol. - pub fn open(url: &str, token: &str) -> Result { + /// Open the portal of `server`, showing the token as the WebSocket's + /// protocol. + pub fn open(server: &str, token: &str) -> Result { + Portal::attach(|said| { + let socket = socket::open(server, berth_api::ops::PORTAL, token, said)?; + Ok(Box::new(socket) as Box) + }) + } + + /// The portal over the socket `connect` opens, which tells `said` what + /// it hears. + pub fn attach( + connect: impl FnOnce(Box) -> Result, String>, + ) -> Result { let inner = Rc::new(Inner { socket: RefCell::default(), link: Cell::new(Link::Connecting), @@ -184,15 +203,11 @@ impl Portal { streams: RefCell::default(), }); let weak: Weak = Rc::downgrade(&inner); - let socket = device::device().connect( - url, - token, - Box::new(move |said| { - if let Some(inner) = weak.upgrade() { - inner.said(said); - } - }), - )?; + let socket = connect(Box::new(move |said| { + if let Some(inner) = weak.upgrade() { + inner.said(said); + } + }))?; *inner.socket.borrow_mut() = Some(socket); Ok(Portal(inner)) } @@ -268,11 +283,10 @@ impl Portal { #[cfg(test)] mod tests { use super::*; - use crate::device::{Answer, Device, Kept, Linked}; type Hears = Box; - /// A device whose socket the test speaks for. + /// A socket the test speaks for. #[derive(Clone, Default)] struct Wire { sent: Rc>>, @@ -306,57 +320,14 @@ mod tests { } } - impl Device for Wire { - fn copy(&self, _: &str) -> bool { - false - } - fn paste(&self) -> Option { - None - } - fn open(&self, _: &str) -> Result<(), String> { - Err("no".into()) - } - fn choose_file(&self, _: &str, _: Box)>) -> Result<(), String> { - Err("no".into()) - } - fn stored(&self, _: Kept) -> Option { - None - } - fn store(&self, _: Kept, _: &str) {} - fn embedded(&self) -> bool { - false - } - fn linked(&self) -> Linked { - Linked::default() - } - fn origin(&self) -> String { - String::new() - } - fn page(&self) -> String { - String::new() - } - fn request(&self, _: &str, _: &str, _: &str, then: Answer) { - then(Err("no".into())) - } - fn connect( - &self, - url: &str, - token: &str, - said: Box, - ) -> Result, String> { - assert_eq!( - (url, token), - ("ws://192.0.2.10:7878/v1/portal", "the-token") - ); - *self.said.borrow_mut() = Some(said); - Ok(Box::new(WireSocket(self.sent.clone()))) - } - } - fn portal() -> (Wire, Portal) { let wire = Wire::default(); - device::install(wire.clone()); - let portal = Portal::open("ws://192.0.2.10:7878/v1/portal", "the-token").unwrap(); + let speaking = wire.clone(); + let portal = Portal::attach(move |said| { + *speaking.said.borrow_mut() = Some(said); + Ok(Box::new(WireSocket(speaking.sent.clone()))) + }) + .unwrap(); (wire, portal) } @@ -409,9 +380,12 @@ mod tests { } #[test] - fn a_socket_that_never_opens_is_refused() { + fn a_socket_that_never_opens_is_unreached_unless_the_server_refused_it() { let (wire, portal) = portal(); wire.say(Said::Closed); + assert_eq!(portal.link(), Link::Unreached); + let (wire, portal) = self::portal(); + wire.say(Said::Refused); assert_eq!(portal.link(), Link::Refused); } } diff --git a/crates/berth-page/src/remote.rs b/crates/berth-page/src/remote.rs index 0229636..6da0e0a 100644 --- a/crates/berth-page/src/remote.rs +++ b/crates/berth-page/src/remote.rs @@ -175,11 +175,11 @@ impl Transport for Remote { events: events.clone(), stream: Cell::new(None), }; - again.call("tasks.list", json!({}), Value::Null, |result, events| { - if let Ok(tasks) = result { + again.call("actions.list", json!({}), Value::Null, |result, events| { + if let Ok(listed) = result { events .borrow_mut() - .push_back(Event::Tasks(list(tasks, "tasks"))); + .push_back(Event::Actions(list(listed, "actions"))); } }); again.poll(); @@ -192,7 +192,7 @@ impl Transport for Remote { }); } Request::Poll => self.poll(), - // A task is the host's action on the server, which keeps a job of it. + // An action is the host's action on the server, which keeps a job of it. Request::Run { name, body } => { let events = self.events.clone(); let args = json!({"name": self.host, "action": name}); @@ -245,17 +245,22 @@ impl Transport for Remote { }, ); } - Request::Choices { task, argument } => { - let args = json!({"name": task, "argument": argument}); - self.call("tasks.choices", args, Value::Null, move |result, events| { - events.borrow_mut().push_back(Event::Choices { - task, - argument, - choices: result - .map(|value| list(value, "choices")) - .unwrap_or_default(), - }); - }); + Request::Choices { action, argument } => { + let args = json!({"action": action, "argument": argument}); + self.call( + "actions.choices", + args, + Value::Null, + move |result, events| { + events.borrow_mut().push_back(Event::Choices { + action, + argument, + choices: result + .map(|value| list(value, "choices")) + .unwrap_or_default(), + }); + }, + ); } } } diff --git a/crates/berth-page/src/screens.rs b/crates/berth-page/src/screens.rs new file mode 100644 index 0000000..dc01505 --- /dev/null +++ b/crates/berth-page/src/screens.rs @@ -0,0 +1,63 @@ +//! The screens in a terminal: drawing, and the keys and mouse taken in. + +use crate::keys; +use crossterm::event; +use std::time::Duration; + +/// Stop the mouse and paste reports, which ratatui's own restore leaves on. +fn stop_reports() { + let _ = crossterm::execute!( + std::io::stdout(), + event::DisableBracketedPaste, + event::DisableMouseCapture + ); +} + +/// The screens in this terminal until they are left, opening the host `open` once +/// the server has listed it. +pub fn run(open: Option) -> std::io::Result<()> { + let mut terminal = ratatui::init(); + let restore = std::panic::take_hook(); + std::panic::set_hook(Box::new(move |info| { + stop_reports(); + restore(info); + })); + // Not every terminal of Windows knows bracketed paste: there a paste + // arrives as keys. + let _ = crossterm::execute!( + std::io::stdout(), + event::EnableMouseCapture, + event::EnableBracketedPaste + ); + let outcome = screens(&mut terminal, open); + stop_reports(); + ratatui::restore(); + outcome +} + +fn screens(terminal: &mut ratatui::DefaultTerminal, open: Option) -> std::io::Result<()> { + let mut app = crate::App::new(); + if let Some(host) = open { + app.open_when_listed(&host); + } + loop { + crate::pump(); + app.tick(); + terminal.draw(|frame| app.draw(frame))?; + // Wait a little for input; what the network says is taken in on the next turn. + if !keys::poll(Duration::from_millis(30))? { + continue; + } + loop { + if let Some(event) = keys::event(&keys::read()?) { + app.event(event); + } + if app.quit() { + return Ok(()); + } + if !keys::poll(Duration::ZERO)? { + break; + } + } + } +} diff --git a/crates/berth-page/src/settings.rs b/crates/berth-page/src/settings.rs new file mode 100644 index 0000000..e98649d --- /dev/null +++ b/crates/berth-page/src/settings.rs @@ -0,0 +1,425 @@ +//! The server the screens use and its token, kept between runs as berth's +//! own `server` and `server_token` settings: in `config.json` of berth's +//! folder, a file only its user can read, in a folder only its user can open. +//! The other settings of that file are left as they are. + +use serde_json::{Map, Value}; +use std::io::Write; +use std::path::{Path, PathBuf}; + +/// The settings that say which server berth uses, and with what token. +pub const SERVER: &str = "server"; +pub const TOKEN: &str = "server_token"; + +/// Where the settings of a program are for the environment `var` gives: +/// under `%LOCALAPPDATA%` on Windows, `$XDG_CONFIG_HOME` where it is set, +/// and else `~/Library/Application Support` on macOS and `~/.config` +/// elsewhere. +fn base(var: &impl Fn(&str) -> Option) -> Option { + let var = |name: &str| var(name).filter(|value| !value.is_empty()); + if cfg!(windows) { + // Local rather than roaming, so that the token is not copied to + // other machines with the profile. + return Some(PathBuf::from(var("LOCALAPPDATA")?)); + } + // Only an absolute one, as berth's directories take it. + if let Some(dir) = var("XDG_CONFIG_HOME").filter(|dir| Path::new(dir).is_absolute()) { + return Some(PathBuf::from(dir)); + } + let home = PathBuf::from(var("HOME")?); + match cfg!(target_os = "macos") { + true => Some(home.join("Library/Application Support")), + false => Some(home.join(".config")), + } +} + +/// berth's folder of settings, which on Linux holds the rest of its settings too. +pub fn folder(var: impl Fn(&str) -> Option) -> Option { + Some(base(&var)?.join("berth")) +} + +/// Where berth-remote kept its server and token, before berth took it in. +pub fn old_folder(var: impl Fn(&str) -> Option) -> Option { + Some(base(&var)?.join("berth-remote")) +} + +pub struct Settings { + dir: Option, +} + +impl Settings { + /// The settings in `dir`; with `None`, nothing is read or written. + pub fn new(dir: Option) -> Settings { + Settings { dir } + } + + fn file(&self) -> Option { + Some(self.dir.as_ref()?.join("config.json")) + } + + /// Everything the file holds; nothing when it is missing or unreadable. + fn read(&self) -> Map { + self.file() + .and_then(|file| std::fs::read(file).ok()) + .and_then(|bytes| serde_json::from_slice::(&bytes).ok()) + .and_then(|value| match value { + Value::Object(map) => Some(map), + _ => None, + }) + .unwrap_or_default() + } + + /// A setting's value, when it is set. + pub fn get(&self, key: &str) -> Option { + self.read() + .get(key) + .and_then(Value::as_str) + .map(str::to_owned) + .filter(|value| !value.is_empty()) + } + + /// Set the settings `changes` names, removing those whose value is + /// empty. A file that is there but is not settings is not written over. + pub fn set(&self, changes: &[(&str, &str)]) -> std::io::Result<()> { + let (Some(dir), Some(file)) = (&self.dir, self.file()) else { + return Ok(()); + }; + let mut kept = match std::fs::read(&file) { + Ok(bytes) => match serde_json::from_slice::(&bytes) { + Ok(Value::Object(map)) => map, + _ => { + return Err(std::io::Error::other(format!( + "{} holds no settings, and is left as it is", + file.display() + ))); + } + }, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => Map::new(), + Err(error) => return Err(error), + }; + let before = kept.clone(); + for (key, value) in changes { + match value.is_empty() { + true => kept.remove(*key), + false => kept.insert((*key).to_owned(), Value::String((*value).to_owned())), + }; + } + if kept == before { + return Ok(()); + } + private_dir(dir)?; + let text = serde_json::to_string_pretty(&Value::Object(kept))? + "\n"; + private_write(&file, text.as_bytes()) + } + + /// Take in the server and token berth-remote kept in `old`, once. When + /// berth has no server, or the same one without a token, they become + /// berth's and the old file goes. When berth uses another server, or the + /// same with another token, the file stays, and that is said once. + pub fn take_in(&self, old: Option<&Path>) -> std::io::Result { + let Some(old) = old else { + return Ok(TakenIn::Nothing); + }; + let file = old.join("settings.json"); + let Ok(bytes) = std::fs::read(&file) else { + return Ok(TakenIn::Nothing); + }; + let kept: Value = serde_json::from_slice(&bytes).unwrap_or(Value::Null); + let text = |key: &str| kept[key].as_str().unwrap_or("").trim().to_owned(); + let (server, token) = (text("server"), text("token")); + // A file that names no server holds nothing to take in, and is left as it is. + if server.is_empty() { + return Ok(TakenIn::Nothing); + } + let same = + |one: &str, other: &str| one.trim_end_matches('/') == other.trim_end_matches('/'); + let taken = match (self.get(SERVER), self.get(TOKEN)) { + (None, _) => { + self.set(&[(SERVER, &server), (TOKEN, &token)])?; + TakenIn::Taken(server) + } + (Some(ours), None) if same(&ours, &server) && !token.is_empty() => { + self.set(&[(TOKEN, &token)])?; + TakenIn::Taken(server) + } + // Its contents are berth's already. + (Some(ours), ours_token) + if same(&ours, &server) + && (token.is_empty() || ours_token.as_deref() == Some(&token)) => + { + TakenIn::Same + } + (Some(ours), _) => { + if self.get(NOTED).as_deref() == Some(server.as_str()) { + return Ok(TakenIn::Nothing); + } + self.set(&[(NOTED, &server)])?; + return Ok(TakenIn::Left { + theirs: server, + ours, + file, + }); + } + }; + for gone in [file.clone(), file.with_extension("json.partial")] { + match std::fs::remove_file(&gone) { + Err(error) if error.kind() != std::io::ErrorKind::NotFound => return Err(error), + _ => {} + } + } + let _ = std::fs::remove_dir(old); + Ok(taken) + } +} + +/// The setting that remembers which server of berth-remote's settings was +/// said to be left, so it is said once. +const NOTED: &str = "berth_remote_noted"; + +/// What became of berth-remote's settings. +#[derive(Debug, PartialEq, Eq)] +pub enum TakenIn { + /// There were none, or they were said to be left before. + Nothing, + /// berth uses their server and token now; the old file is gone. + Taken(String), + /// berth had them already; the old file is gone. + Same, + /// berth uses another server, or the same with another token: the file + /// stays where it is. + Left { + theirs: String, + ours: String, + file: PathBuf, + }, +} + +/// Make the folder of the settings, which only its owner can open. +fn private_dir(dir: &Path) -> std::io::Result<()> { + let mut builder = std::fs::DirBuilder::new(); + builder.recursive(true); + #[cfg(unix)] + std::os::unix::fs::DirBuilderExt::mode(&mut builder, 0o700); + builder.create(dir)?; + // A folder that was there already keeps its mode otherwise. + #[cfg(unix)] + std::fs::set_permissions(dir, std::os::unix::fs::PermissionsExt::from_mode(0o700))?; + Ok(()) +} + +/// Write a file only its owner can read, since it holds the token. On +/// Windows the file takes the access list of the user's profile, which lets +/// in the user, SYSTEM and the administrators. +fn private_write(file: &Path, bytes: &[u8]) -> std::io::Result<()> { + let partial = file.with_extension("json.partial"); + // A file left over would keep its own mode. + match std::fs::remove_file(&partial) { + Err(error) if error.kind() != std::io::ErrorKind::NotFound => return Err(error), + _ => {} + } + let mut options = std::fs::OpenOptions::new(); + options.write(true).create_new(true); + #[cfg(unix)] + std::os::unix::fs::OpenOptionsExt::mode(&mut options, 0o600); + let mut out = options.open(&partial)?; + out.write_all(bytes)?; + out.sync_all()?; + drop(out); + std::fs::rename(partial, file) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn keeps_the_server_beside_the_other_settings() { + let scratch = tempfile::tempdir().unwrap(); + let dir = scratch.path().join("config/berth"); + std::fs::create_dir_all(&dir).unwrap(); + std::fs::write(dir.join("config.json"), r#"{"store": "/srv/berth.db"}"#).unwrap(); + let settings = Settings::new(Some(dir.clone())); + assert_eq!(settings.get(SERVER), None); + settings + .set(&[(SERVER, "https://studio.example"), (TOKEN, "secret")]) + .unwrap(); + assert_eq!( + settings.get(SERVER).as_deref(), + Some("https://studio.example") + ); + assert_eq!(settings.get(TOKEN).as_deref(), Some("secret")); + settings.set(&[(TOKEN, "")]).unwrap(); + assert_eq!(settings.get(TOKEN), None); + let all: Value = + serde_json::from_slice(&std::fs::read(dir.join("config.json")).unwrap()).unwrap(); + assert_eq!(all["store"], "/srv/berth.db"); + assert!(!dir.join("config.json.partial").exists()); + } + + #[test] + fn a_file_that_is_not_settings_is_left_alone() { + let scratch = tempfile::tempdir().unwrap(); + let settings = Settings::new(Some(scratch.path().to_owned())); + std::fs::write(scratch.path().join("config.json"), "{not json").unwrap(); + assert_eq!(settings.get(SERVER), None); + assert!(settings.set(&[(SERVER, "https://dock.example")]).is_err()); + let left = std::fs::read_to_string(scratch.path().join("config.json")).unwrap(); + assert_eq!(left, "{not json"); + } + + fn remote(scratch: &Path, text: &str) -> PathBuf { + let old = scratch.join("berth-remote"); + std::fs::create_dir_all(&old).unwrap(); + std::fs::write(old.join("settings.json"), text).unwrap(); + old + } + + #[test] + fn berth_remotes_server_is_taken_in_once() { + let scratch = tempfile::tempdir().unwrap(); + let old = remote( + scratch.path(), + r#"{"server": "https://studio.example", "token": "s3cret"}"#, + ); + let settings = Settings::new(Some(scratch.path().join("berth"))); + assert_eq!( + settings.take_in(Some(&old)).unwrap(), + TakenIn::Taken("https://studio.example".into()) + ); + assert_eq!(settings.get(TOKEN).as_deref(), Some("s3cret")); + assert!(!old.exists()); + assert_eq!(settings.take_in(Some(&old)).unwrap(), TakenIn::Nothing); + } + + #[test] + fn berth_remotes_token_for_berths_own_server_is_taken_in() { + let scratch = tempfile::tempdir().unwrap(); + let settings = Settings::new(Some(scratch.path().join("berth"))); + settings.set(&[(SERVER, "https://studio.example")]).unwrap(); + let old = remote( + scratch.path(), + r#"{"server": "https://studio.example/", "token": "s3cret"}"#, + ); + assert_eq!( + settings.take_in(Some(&old)).unwrap(), + TakenIn::Taken("https://studio.example/".into()) + ); + assert_eq!(settings.get(TOKEN).as_deref(), Some("s3cret")); + assert!(!old.exists()); + + // The same server and token are berth's already: the file goes. + let old = remote( + scratch.path(), + r#"{"server": "https://studio.example", "token": "s3cret"}"#, + ); + assert_eq!(settings.take_in(Some(&old)).unwrap(), TakenIn::Same); + assert!(!old.exists()); + } + + #[test] + fn berth_remotes_other_server_is_left_and_said_once() { + let scratch = tempfile::tempdir().unwrap(); + let settings = Settings::new(Some(scratch.path().join("berth"))); + settings + .set(&[(SERVER, "https://studio.example"), (TOKEN, "mine")]) + .unwrap(); + for other in [ + r#"{"server": "https://dock.example", "token": "other"}"#, + r#"{"server": "https://studio.example", "token": "other"}"#, + ] { + let old = remote(scratch.path(), other); + let file = old.join("settings.json"); + let taken = settings.take_in(Some(&old)).unwrap(); + assert!( + matches!(&taken, TakenIn::Left { file: left, ours, .. } + if *left == file && ours == "https://studio.example"), + "{taken:?}" + ); + // The file stays, and is not spoken of again. + assert!(file.exists()); + assert_eq!(settings.take_in(Some(&old)).unwrap(), TakenIn::Nothing); + assert_eq!(settings.get(TOKEN).as_deref(), Some("mine")); + assert_eq!( + settings.get(SERVER).as_deref(), + Some("https://studio.example") + ); + std::fs::remove_dir_all(&old).unwrap(); + } + } + + #[test] + fn a_file_of_berth_remote_with_no_server_is_left_as_it_is() { + let scratch = tempfile::tempdir().unwrap(); + let settings = Settings::new(Some(scratch.path().join("berth"))); + for text in ["{not json", r#"{"token": "s3cret"}"#] { + let old = remote(scratch.path(), text); + assert_eq!(settings.take_in(Some(&old)).unwrap(), TakenIn::Nothing); + assert!(old.join("settings.json").exists()); + } + assert_eq!(settings.get(TOKEN), None); + } + + #[cfg(unix)] + #[test] + fn only_the_owner_can_read_them() { + use std::os::unix::fs::PermissionsExt; + let scratch = tempfile::tempdir().unwrap(); + let dir = scratch.path().join("berth"); + // A folder made earlier with a looser mode is made private too. + std::fs::create_dir(&dir).unwrap(); + std::fs::set_permissions(&dir, std::fs::Permissions::from_mode(0o755)).unwrap(); + let settings = Settings::new(Some(dir.clone())); + settings.set(&[(TOKEN, "secret")]).unwrap(); + let mode = |path: &Path| std::fs::metadata(path).unwrap().permissions().mode() & 0o777; + assert_eq!(mode(&dir), 0o700); + assert_eq!(mode(&dir.join("config.json")), 0o600); + } + + #[test] + fn the_folder_follows_the_system() { + let env = |pairs: &'static [(&'static str, &'static str)]| { + move |name: &str| { + pairs + .iter() + .find(|(key, _)| *key == name) + .map(|(_, value)| value.to_string()) + } + }; + if cfg!(windows) { + assert_eq!( + folder(env(&[("LOCALAPPDATA", r"C:\Users\ada\AppData\Local")])), + Some(PathBuf::from(r"C:\Users\ada\AppData\Local").join("berth")) + ); + assert_eq!(folder(env(&[])), None); + return; + } + assert_eq!( + folder(env(&[ + ("XDG_CONFIG_HOME", "/scratch/c"), + ("HOME", "/home/ada") + ])), + Some(PathBuf::from("/scratch/c/berth")) + ); + assert_eq!( + old_folder(env(&[ + ("XDG_CONFIG_HOME", "/scratch/c"), + ("HOME", "/home/ada") + ])), + Some(PathBuf::from("/scratch/c/berth-remote")) + ); + let home = match cfg!(target_os = "macos") { + true => "/home/ada/Library/Application Support/berth", + false => "/home/ada/.config/berth", + }; + for relative in ["", "scratch/c"] { + assert_eq!( + folder(env(match relative { + "" => &[("XDG_CONFIG_HOME", ""), ("HOME", "/home/ada")], + _ => &[("XDG_CONFIG_HOME", "scratch/c"), ("HOME", "/home/ada")], + })), + Some(PathBuf::from(home)) + ); + } + assert_eq!(folder(env(&[])), None); + } +} diff --git a/crates/berth-page/src/setup.rs b/crates/berth-page/src/setup.rs index 1d20ff0..6bd2430 100644 --- a/crates/berth-page/src/setup.rs +++ b/crates/berth-page/src/setup.rs @@ -21,18 +21,18 @@ use serde_json::{Value, json}; use tui_input::Input; /// The ways the server installs a host itself. -const METHODS: [Action; 4] = [ - Action::Machine, - Action::CloudSeed, - Action::Ssh, - Action::Reach, -]; +const METHODS: [Action; 5] = Action::SERVED; + +/// The action of the host that builds its USB installer. +pub const INSTALLER: &str = "installer"; /// What a key on the set-up pages leads to. pub enum Flow { Stay, /// Back to the host's overview, or to the hosts for one never saved. Leave, + /// To the host's overview, to run its action of this name there. + Act(&'static str), } pub struct Setup { @@ -48,10 +48,15 @@ pub struct Setup { features: Vec, registries: Vec, secrets: Vec, + /// The accounts of a workstation, kept while the type says node, which + /// has none, so that choosing workstation again brings them back. + accounts: serde_json::Map, /// Where a release can be published. stores: Vec, /// The server's own SSH public key. own_key: String, + /// Whether the server has the API key of the tailnet. + tailnet_key: bool, /// The names of the keys of the server's key ring. ring_keys: Vec, list: ListState, @@ -66,14 +71,19 @@ pub struct Setup { pub panel: (u16, u16), /// The machine the server creates for the host, on the set-up tabs. pub machine: Machine, + /// The server runs on this machine, which builds what is made of its drives. + local: bool, + /// An action of the host to run once the profile is saved. + then: Option<&'static str>, } impl Setup { /// The set-up of a host the server has: its profile is on its way. - pub fn open(name: &str, admin: &Admin) -> Setup { + pub fn open(name: &str, admin: &Admin, local: bool) -> Setup { admin.profile(name); let mut setup = Setup { loading: true, + local, ..Setup::with(name, Value::Null, Value::Null) }; setup.ask_machine(admin); @@ -81,9 +91,12 @@ impl Setup { } /// The set-up of a new host, from the server's starting profile. - pub fn new(name: &str, template: Value, admin: &Admin) -> Setup { - let mut setup = Setup::with(name, template, Value::Null); - setup::fit_profile(&mut setup.config, setup.action); + pub fn new(name: &str, template: Value, admin: &Admin, local: bool) -> Setup { + let mut setup = Setup { + local, + ..Setup::with(name, template, Value::Null) + }; + setup::fit_target(&mut setup.config, setup.action); setup.ask_machine(admin); setup } @@ -109,7 +122,9 @@ impl Setup { secrets: vec![], stores: vec![], own_key: String::new(), + tailnet_key: false, ring_keys: vec![], + accounts: serde_json::Map::new(), list: ListState::default().with_selected(Some(0)), display: ListState::default(), editing: None, @@ -119,6 +134,8 @@ impl Setup { tabs: vec![], panel: (24, 80), machine: Machine::default(), + local: false, + then: None, } } @@ -139,12 +156,13 @@ impl Setup { berth_api::profile::retire_qemu(&mut profile); self.config = profile.clone(); self.saved = profile; - // A host with an address is one to install over SSH. - if !self.view().text(Field::SshHost).is_empty() { + // A host with an address, or of a type only SSH installs, is one to install over SSH. + let host_type = berth_api::profile::host_type(&self.config); + if !self.view().text(Field::SshHost).is_empty() || !self.action.offered(host_type) { self.action = Action::Ssh; } // A profile saved with another kind opens as the machine's. - setup::fit_profile(&mut self.config, self.action); + setup::fit_target(&mut self.config, self.action); } Err(error) => self.failed = Some(error), } @@ -167,6 +185,9 @@ impl Setup { if !self.loading { setup::default_enrollment(&mut self.config, &names); } + self.tailnet_key = names + .iter() + .any(|name| name == berth_api::profile::TAILNET_API_KEY); self.secrets = setup::host_secrets(names); self.stores = setup::release_stores(&data.registries); self.own_key = data.own_key.clone(); @@ -193,28 +214,18 @@ impl Setup { secrets: &self.secrets, own_key: &self.own_key, named: setup::valid_hostname(&self.name), + tailnet_key: self.tailnet_key, } } /// Every problem, each on the tab of what it is about. fn problems(&self) -> Vec { let view = self.view(); - // The keys that make these up are other ones here. - let mut problems: Vec = view - .problems_with(&self.machine) - .into_iter() - .map(|problem| Problem { - text: problem - .text - .replace("(F4 generates)", "(g generates)") - .replace("(F4 or F5)", "(g or p)"), - ..problem - }) - .collect(); - if self.action == Action::Ssh && view.text(Field::SshHost).is_empty() { + let mut problems: Vec = view.problems_with(&self.machine); + if self.action == Action::Iso && !self.local { problems.push(Problem::new( Step::Install, - "The host's address is required: its name, IP address or tailnet name", + "The USB installer is built where berth serve runs, with berth ui there", )); } problems @@ -253,8 +264,7 @@ impl Setup { fn help(&self) -> String { match self.selected() { - // F5 is the browser's own, so the letters do what berth's F4 and F5 do. - Some(Entry::Field(field)) => field.spec().help.replace("F4", "g").replace("F5", "p"), + Some(Entry::Field(field)) => field.spec().help.into(), Some(Entry::Key(Account::Operator, Some(_))) => Field::OperatorKey.spec().help.into(), Some(Entry::Key(Account::Agent, Some(_))) => Field::AgentKey.spec().help.into(), Some(Entry::Key(_, None)) => setup::ADD_KEY_HELP.into(), @@ -265,10 +275,10 @@ impl Setup { Some(Entry::Feature(index)) => self.features[index].help.clone(), Some(Entry::Registry(_)) => setup::REGISTRY_HELP.into(), Some(Entry::Secret(_)) => setup::SECRET_HELP.into(), - Some(Entry::Run) => match (self.action, berth_api::profile::level(&self.config)) { + Some(Entry::Run) => match (self.action, berth_api::profile::host_type(&self.config)) { (Action::CloudSeed, _) => "Saves the profile on the server and renders the seed. Paste it into the provider's user data field when creating the machine.".into(), - (Action::Ssh, berth_api::HostLevel::Reachable) => "Saves the host on the server. Nothing is installed: the server logs in to it over SSH, with a key of its own that the host takes.".into(), - (Action::Ssh, berth_api::HostLevel::Controller) => "Saves the profile on the server and installs berth's controller alone on the host from there.".into(), + (Action::Ssh, berth_api::HostType::Ssh) => "Saves the host on the server. Nothing is installed: the server logs in to it over SSH, with a key of its own that the host takes.".into(), + (Action::Ssh, berth_api::HostType::Managed) => "Saves the profile on the server and installs berth's controller alone on the host from there.".into(), _ => "Saves the profile on the server and applies it to the host from there.".into(), }, Some(Entry::Machine(pick)) => self.machine.help(pick), @@ -328,6 +338,26 @@ impl Setup { setup::set_pointer(&mut self.config, field.spec().pointer, value); } + /// Give the profile another type, as `setup::commit` does, keeping a + /// workstation's accounts aside while it is a node. + fn choose_type(&mut self, name: &str) { + if name == berth_api::HostType::Node.name() { + for field in berth_api::profile::ACCOUNT_FIELDS { + if let Some(value) = self.config.get(*field) { + self.accounts.insert((*field).to_owned(), value.clone()); + } + } + } + let _ = setup::commit(&mut self.config, Field::Type, name); + if name == berth_api::HostType::Workstation.name() + && let Some(profile) = self.config.as_object_mut() + { + for (field, value) in std::mem::take(&mut self.accounts) { + profile.entry(field).or_insert(value); + } + } + } + /// Toggle or cycle the field, or open the editor for free text. fn activate(&mut self, field: Field) -> Option { if field == Field::Login { @@ -347,6 +377,16 @@ impl Setup { "No file server or raw repository yet. Add one on the Registries tab of the home screen.".into() }); } + if field == Field::Type { + let types = berth_tui::fields::TYPES; + let current = self.view().text(field); + let next = types + .iter() + .position(|known| *known == current) + .map_or(0, |index| (index + 1) % types.len()); + self.choose_type(types[next]); + return None; + } match field.spec().kind { Kind::Toggle => { let on = !self.view().enabled(field); @@ -402,9 +442,6 @@ impl Setup { let view = self.view(); let asked = match field { Field::Hostname => json!({"what": "hostname"}), - Field::TailscaleKey => { - json!({"what": "tailscale_key", "host": view.text(Field::Hostname)}) - } _ if field.is_password_hash() => { let user = match field { Field::OperatorHash => Field::OperatorUser, @@ -429,23 +466,29 @@ impl Setup { return Some("Fix the listed problems first".into()); } let (name, action, size) = (self.name.clone(), self.action, self.panel); - // Over SSH a host below a workstation gets the controller alone, or nothing. - let level = berth_api::profile::level(&self.config); + // Over SSH a host that is not a workstation gets the controller alone, or nothing. + let host_type = berth_api::profile::host_type(&self.config); let install = berth_api::Action { name: "install-controller".into(), label: "Install the controller".into(), ..berth_api::Action::default() }; admin.save(&self.name, &self.config, move |admin| { - match (action, level) { + match (action, host_type) { (Action::CloudSeed, _) => admin.seed(&name), - (Action::Ssh, berth_api::HostLevel::Controller) => { + // Its drive and sudo password are asked for on the host's + // overview, once the server lists the host's actions. + (Action::Iso, _) => admin.actions(&name), + (Action::Ssh, berth_api::HostType::Managed) => { admin.act(&name, &install, &Default::default(), size) } - (Action::Ssh, berth_api::HostLevel::Reachable) => {} + (Action::Ssh, berth_api::HostType::Ssh) => {} _ => admin.apply(&name, None, size), } }); + if action == Action::Iso { + self.then = Some(INSTALLER); + } Some("Saving…".into()) } @@ -453,7 +496,7 @@ impl Setup { match entry { Entry::Action(action) => { self.action = action.choose(&mut self.config); - setup::fit_profile(&mut self.config, self.action); + setup::fit_target(&mut self.config, self.action); } Entry::Field(field) => return self.activate(field), Entry::Registry(index) => { @@ -472,7 +515,7 @@ impl Setup { // The machine is made from the profile, which has to be whole // first, and of the kind a created machine is. if pick == machine::Pick::Create { - setup::fit_profile(&mut self.config, Action::Machine); + setup::fit_target(&mut self.config, Action::Machine); } if pick == machine::Pick::Create && !self.problems().is_empty() { return Some("Fix the listed problems first".into()); @@ -557,14 +600,11 @@ impl Setup { match key.code { Code::Up => self.move_selection(true), Code::Down => self.move_selection(false), - // F5 is the browser's own, so the letters do what berth's F4 and F5 do. - Code::Char('g') | Code::F(4) - if self.selected_field().is_some_and(Field::can_generate) => - { + Code::F(4) if self.selected_field().is_some_and(Field::can_generate) => { let field = self.selected_field().unwrap(); say(self.generate(field, admin)); } - Code::Char('p') if self.selected_field().is_some_and(Field::is_password_hash) => { + Code::F(5) if self.selected_field().is_some_and(Field::is_password_hash) => { self.password_entry = true; self.editing = Some(Input::default()); } @@ -617,6 +657,14 @@ impl Setup { } pub fn event(&mut self, event: Event, admin: &Admin, status: &mut String) -> Flow { + let flow = self.take(event, admin, status); + match self.then.take() { + Some(action) => Flow::Act(action), + None => flow, + } + } + + fn take(&mut self, event: Event, admin: &Admin, status: &mut String) -> Flow { match event { Event::Key(key) => { status.clear(); @@ -721,7 +769,7 @@ impl Setup { ]) .areas(left); // The cursor only stops on rows that do something. - let rows = self.rows(); + let rows = berth_tui::setup::wrapped(self.rows(), body.width); let positions: Vec = rows .iter() .enumerate() @@ -770,10 +818,10 @@ impl Setup { let keys = match (&self.editing, field) { (Some(_), _) => "Enter accept · Esc cancel · Ctrl+O copy · Ctrl+S save".to_owned(), (None, Some(field)) if field.is_password_hash() => { - "↑↓ select · g generate · p type a password · x clear · 1-4 tab · Ctrl+S save · Esc back".to_owned() + "↑↓ select · F4 generate · F5 type a password · x clear · 1-4 tab · Ctrl+S save · Esc back".to_owned() } (None, Some(field)) if field.can_generate() => { - "↑↓ select · Enter change · g generate · x clear · 1-4 tab · Ctrl+S save · Esc back".to_owned() + "↑↓ select · Enter change · F4 generate · x clear · 1-4 tab · Ctrl+S save · Esc back".to_owned() } _ => "↑↓ select · Enter change · Space switch · x clear · 1-4 tab · Ctrl+S save · Esc back".to_owned(), }; diff --git a/crates/berth-remote/src/socket.rs b/crates/berth-page/src/socket.rs similarity index 73% rename from crates/berth-remote/src/socket.rs rename to crates/berth-page/src/socket.rs index 6a6bc7d..122b00f 100644 --- a/crates/berth-remote/src/socket.rs +++ b/crates/berth-page/src/socket.rs @@ -1,15 +1,32 @@ -//! The WebSocket under the page's portal: it runs on the runtime, and what it +//! The WebSocket under the screens' portal: it runs on the runtime, and what it //! says is handed to the main thread at [`pump`]. use crate::net; use berth_api::client::{self, Frame as Message}; -use berth_page::device::{Said, Socket}; use futures_util::{SinkExt, StreamExt}; use std::cell::RefCell; use std::sync::mpsc; use std::time::Duration; use tokio::sync::mpsc::{UnboundedReceiver, UnboundedSender, unbounded_channel}; +/// What the portal's socket says, handed over on the thread that draws. +pub enum Said { + Open, + Text(String), + Bytes(Vec), + /// The socket is closed, or never opened. + Closed, + /// The server answered the handshake and refused what was shown, as a + /// wrong token: 401 or 403. + Refused, +} + +/// An open socket; dropping it closes it. +pub trait Socket { + fn send_text(&self, text: &str) -> bool; + fn send_bytes(&self, bytes: &[u8]) -> bool; +} + /// What a socket says goes to its listener here, a turn at a time. struct Heard { said: mpsc::Receiver, @@ -42,10 +59,14 @@ impl Socket for NativeSocket { } } -/// Open a WebSocket to `url`, with `token` as its protocol, the way the page -/// sends it. -pub fn open(url: &str, token: &str, to: Box) -> Result { - let target = net::target(url)?; +/// Open a WebSocket to `path` of `server`, with `token` as its protocol. +pub fn open( + server: &str, + path: &str, + token: &str, + to: Box, +) -> Result { + let target = net::target(server, path)?; let (out, outgoing) = unbounded_channel(); let (said, heard) = mpsc::channel(); net::runtime().spawn(run(target, token.to_owned(), outgoing, said)); @@ -80,11 +101,14 @@ async fn run( mut out: UnboundedReceiver, said: mpsc::Sender, ) { + // Whether the server itself refused the handshake, beside why it failed. let opened = async { - let stream = net::connect(&target).await?; + let stream = net::connect(&target) + .await + .map_err(|error| (false, error))?; client::websocket_at(stream, &target.authority, &target.path, &token) .await - .map_err(|error| net::describe(&error)) + .map_err(|error| (matches!(error.status(), 401 | 403), net::describe(&error))) }; let opened = tokio::select! { opened = tokio::time::timeout(CONNECT, opened) => opened.ok(), @@ -95,9 +119,16 @@ async fn run( } } => return, }; - let Some(Ok(socket)) = opened else { - let _ = said.send(Said::Closed); - return; + let socket = match opened { + Some(Ok(socket)) => socket, + Some(Err((true, _))) => { + let _ = said.send(Said::Refused); + return; + } + _ => { + let _ = said.send(Said::Closed); + return; + } }; let _ = said.send(Said::Open); let (mut write, mut read) = socket.split(); diff --git a/crates/berth-remote/src/device.rs b/crates/berth-page/src/terminal.rs similarity index 54% rename from crates/berth-remote/src/device.rs rename to crates/berth-page/src/terminal.rs index 215a90f..42daf18 100644 --- a/crates/berth-remote/src/device.rs +++ b/crates/berth-page/src/terminal.rs @@ -1,12 +1,6 @@ -//! What a browser does for the page, done by this program instead: the -//! clipboard, opening links, choosing a file, plain HTTP, the portal's socket, -//! and the server and token it keeps between runs. +//! What the screens do with the system they run on: the clipboard, opening +//! links in the system's browser, and its dialog for choosing a file. -use crate::settings::Settings; -use crate::{net, socket}; -use berth_page::device::{Answer, Device, Kept, Linked, Said, Socket}; -use serde_json::{Map, Value}; -use std::cell::RefCell; use std::io::Write; use std::path::PathBuf; use std::process::{Command, Stdio}; @@ -14,183 +8,82 @@ use std::process::{Command, Stdio}; /// The variables that carry secrets, which no helper program is handed. pub const SECRETS: [&str; 2] = ["BERTH_TOKEN", "BERTH_SETUP_KEY"]; -/// The device of a terminal. -pub struct Terminal { - linked: Linked, - settings: Settings, - /// What is kept, as the screens last set it. - kept: RefCell>, - /// Keep nothing on disk. - forget: bool, -} - -fn name(key: Kept) -> &'static str { - match key { - Kept::Server => "server", - Kept::Token => "token", - } -} - -impl Terminal { - /// With `forget`, what earlier runs kept is removed and nothing new is. - pub fn new(linked: Linked, settings: Settings, forget: bool) -> std::io::Result { - let kept = match forget { - true => { - settings.forget()?; - Map::new() - } - false => settings.read(), - }; - Ok(Terminal { - linked, - settings, - kept: RefCell::new(kept), - forget, - }) - } -} - -impl Device for Terminal { - fn copy(&self, text: &str) -> bool { - copy_text(text) - } - - fn paste(&self) -> Option { - paste_text() - } - - fn open(&self, link: &str) -> Result<(), String> { - let link = openable(link)?; - let mut command = if cfg!(windows) { - // Not through cmd, which would take the & of an address for its own. - let mut command = helper("rundll32"); - command.args(["url.dll,FileProtocolHandler", link]); - command - } else if cfg!(target_os = "macos") { - let mut command = helper("open"); - command.arg(link); - command - } else { - // Without a desktop xdg-open falls back to a browser in this - // very terminal, under the screens. - if !has_display() { - return Err("there is no desktop here to open it on".into()); - } - let mut command = helper("xdg-open"); - command.arg(link); - command - }; +/// Open a web address in the system's browser. +pub fn open(link: &str) -> Result<(), String> { + let link = openable(link)?; + let mut command = if cfg!(windows) { + // Not through cmd, which would take the & of an address for its own. + let mut command = helper("rundll32"); + command.args(["url.dll,FileProtocolHandler", link]); command - .stdin(Stdio::null()) - .stdout(Stdio::null()) - .stderr(Stdio::null()) - .spawn() - .map(drop) - .map_err(|error| error.to_string()) - } + } else if cfg!(target_os = "macos") { + let mut command = helper("open"); + command.arg(link); + command + } else { + // Without a desktop xdg-open falls back to a browser in this + // very terminal, under the screens. + if !has_display() { + return Err("there is no desktop here to open it on".into()); + } + let mut command = helper("xdg-open"); + command.arg(link); + command + }; + command + .stdin(Stdio::null()) + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .spawn() + .map(drop) + .map_err(|error| error.to_string()) +} - fn choose_file( - &self, - accept: &str, - then: Box)>, - ) -> Result<(), String> { - let endings = endings(accept); - let given = std::env::var_os("BERTH_REMOTE_FILE").map(PathBuf::from); - let dialog = match given { - Some(_) => None, - None => Some(file_dialog(&endings).ok_or( - "There is no file dialog here: set BERTH_REMOTE_FILE to the package's path", - )?), - }; - net::later_blocking( - move || { - let path = match (given, dialog) { - (Some(path), _) => path, - (None, Some(mut dialog)) => { - let output = dialog - .stdin(Stdio::null()) - .stderr(Stdio::null()) - .output() - .ok()?; - let chosen = String::from_utf8_lossy(&output.stdout).trim().to_owned(); - if !output.status.success() || chosen.is_empty() { - return None; - } - PathBuf::from(chosen) +/// Let the user choose a file whose name ends with one of `accept` +/// (comma-separated), and hand over its name and bytes at a later turn. +/// Nothing happens when none is chosen; an error says why none can be asked for. +pub fn choose_file( + accept: &str, + then: impl FnOnce(String, Vec) + 'static, +) -> Result<(), String> { + let endings = endings(accept); + let given = std::env::var_os("BERTH_FILE").map(PathBuf::from); + let dialog = match given { + Some(_) => None, + None => Some( + file_dialog(&endings) + .ok_or("There is no file dialog here: set BERTH_FILE to the package's path")?, + ), + }; + crate::net::later_blocking( + move || { + let path = match (given, dialog) { + (Some(path), _) => path, + (None, Some(mut dialog)) => { + let output = dialog + .stdin(Stdio::null()) + .stderr(Stdio::null()) + .output() + .ok()?; + let chosen = String::from_utf8_lossy(&output.stdout).trim().to_owned(); + if !output.status.success() || chosen.is_empty() { + return None; } - (None, None) => return None, - }; - let bytes = std::fs::read(&path).ok()?; - let name = path.file_name()?.to_string_lossy().into_owned(); - Some((name, bytes)) - }, - move |chosen| { - if let Some((name, bytes)) = chosen { - then(name, bytes) + PathBuf::from(chosen) } - }, - ); - Ok(()) - } - - fn stored(&self, key: Kept) -> Option { - self.kept - .borrow() - .get(name(key)) - .and_then(Value::as_str) - .map(str::to_owned) - } - - fn store(&self, key: Kept, value: &str) { - let mut kept = self.kept.borrow_mut(); - let had = kept.get(name(key)).and_then(Value::as_str).unwrap_or(""); - if had == value { - return; - } - match value.is_empty() { - true => kept.remove(name(key)), - false => kept.insert(name(key).to_owned(), Value::String(value.to_owned())), - }; - if !self.forget { - // The screens go on without it; the next run asks again. - let _ = self.settings.write(&kept); - } - } - - /// This program is never given by a server: it always asks which one. - fn embedded(&self) -> bool { - false - } - - fn linked(&self) -> Linked { - self.linked.clone() - } - - fn origin(&self) -> String { - String::new() - } - - /// The page `berth serve --web` gives at the server in use, for a phone. - fn page(&self) -> String { - self.stored(Kept::Server) - .map(|server| format!("{}/", server.trim_end_matches('/'))) - .unwrap_or_default() - } - - fn request(&self, method: &str, url: &str, bearer: &str, then: Answer) { - let (method, url, bearer) = (method.to_owned(), url.to_owned(), bearer.to_owned()); - net::later(async move { net::http(&method, &url, &bearer).await }, then); - } - - fn connect( - &self, - url: &str, - token: &str, - said: Box, - ) -> Result, String> { - let socket = socket::open(url, token, said)?; - Ok(Box::new(socket)) - } + (None, None) => return None, + }; + let bytes = std::fs::read(&path).ok()?; + let name = path.file_name()?.to_string_lossy().into_owned(); + Some((name, bytes)) + }, + move |chosen| { + if let Some((name, bytes)) = chosen { + then(name, bytes) + } + }, + ); + Ok(()) } /// A program this one starts, such as a clipboard or a browser: without @@ -289,7 +182,7 @@ fn copiers() -> Vec<(Helper, bool)> { } } -fn copy_text(text: &str) -> bool { +pub fn copy_text(text: &str) -> bool { if copiers() .into_iter() .any(|((program, args), wait)| feed(program, args, text, wait)) @@ -319,7 +212,7 @@ fn base64(bytes: &[u8]) -> String { out } -fn paste_text() -> Option { +pub fn paste_text() -> Option { let pasters: Vec = if cfg!(windows) { vec![( "powershell", @@ -362,7 +255,7 @@ fn paste_text() -> Option { /// environment variable, so that nothing is spliced into the script. const WINDOWS_DIALOG: &str = "Add-Type -AssemblyName System.Windows.Forms; \ $dialog = New-Object System.Windows.Forms.OpenFileDialog; \ - $dialog.Filter = $env:BERTH_REMOTE_FILTER; \ + $dialog.Filter = $env:BERTH_FILTER; \ if ($dialog.ShowDialog() -eq 'OK') { [Console]::OutputEncoding = [Text.Encoding]::UTF8; $dialog.FileName }"; /// The file dialog of this system, given what the accepted names end with; @@ -387,7 +280,7 @@ fn file_dialog(endings: &[String]) -> Option { WINDOWS_DIALOG, ]) .env( - "BERTH_REMOTE_FILTER", + "BERTH_FILTER", format!("Packages ({filter})|{filter}|All files (*.*)|*.*"), ); return Some(command); @@ -461,42 +354,4 @@ mod tests { assert_eq!(endings(".tgz, .tar.gz,"), [".tgz", ".tar.gz"]); assert!(endings("").is_empty()); } - - #[test] - fn forget_removes_what_was_kept() { - let scratch = tempfile::tempdir().unwrap(); - let dir = scratch.path().join("berth-remote"); - let kept = - Terminal::new(Linked::default(), Settings::new(Some(dir.clone())), false).unwrap(); - kept.store(Kept::Server, "https://studio.example"); - kept.store(Kept::Token, "secret"); - assert!(dir.join("settings.json").exists()); - let again = - Terminal::new(Linked::default(), Settings::new(Some(dir.clone())), false).unwrap(); - assert_eq!(again.stored(Kept::Token).as_deref(), Some("secret")); - assert_eq!(again.page(), "https://studio.example/"); - - let forgetting = - Terminal::new(Linked::default(), Settings::new(Some(dir.clone())), true).unwrap(); - assert!(!dir.exists()); - assert_eq!(forgetting.stored(Kept::Token), None); - // What this run is given is used, and still not written. - forgetting.store(Kept::Server, "https://dock.example"); - forgetting.store(Kept::Token, "other"); - assert_eq!(forgetting.stored(Kept::Token).as_deref(), Some("other")); - assert_eq!(forgetting.page(), "https://dock.example/"); - assert!(!dir.exists()); - } - - #[test] - fn an_empty_value_forgets_it() { - let scratch = tempfile::tempdir().unwrap(); - let dir = scratch.path().join("berth-remote"); - let device = - Terminal::new(Linked::default(), Settings::new(Some(dir.clone())), false).unwrap(); - device.store(Kept::Token, "secret"); - device.store(Kept::Token, ""); - assert_eq!(device.stored(Kept::Token), None); - assert!(!dir.exists()); - } } diff --git a/crates/berth-remote/Cargo.toml b/crates/berth-remote/Cargo.toml deleted file mode 100644 index 2f46df5..0000000 --- a/crates/berth-remote/Cargo.toml +++ /dev/null @@ -1,28 +0,0 @@ -[package] -name = "berth-remote" -version.workspace = true -edition.workspace = true -rust-version.workspace = true -license.workspace = true -description = "The page of berth serve in a terminal: the hosts of a berth serve elsewhere, on Linux, macOS or Windows" - -[[bin]] -name = "berth-remote" -path = "src/main.rs" - -[dependencies] -berth-api = { workspace = true, features = ["client"] } -berth-page.workspace = true -berth-tui.workspace = true -serde_json.workspace = true -ratatui = { version = "=0.30.2", default-features = false, features = ["crossterm"] } -crossterm = "=0.29.0" -tokio.workspace = true -futures-util.workspace = true -rustls = { version = "0.23", default-features = false, features = ["ring", "std", "tls12"] } -tokio-rustls = { version = "0.26", default-features = false, features = ["ring", "tls12"] } -webpki-roots = "1" -rustls-native-certs = "0.8" - -[dev-dependencies] -tempfile.workspace = true diff --git a/crates/berth-remote/README.md b/crates/berth-remote/README.md deleted file mode 100644 index 773ea52..0000000 --- a/crates/berth-remote/README.md +++ /dev/null @@ -1,66 +0,0 @@ -# berth-remote - -berth-remote shows the hosts of a `berth serve` on another machine, in a terminal. It has the screens of the page that `berth serve --web` gives, and it runs on Windows, macOS and Linux. - -It is only a client. Setting up hosts, Ansible and SSH stay on the server. - -## Install - -On Windows, in PowerShell: - -```powershell -irm https://raw.githubusercontent.com/helightdev/berth/main/install.ps1 | iex -``` - -On macOS or Linux: - -```bash -curl -fsSL https://raw.githubusercontent.com/helightdev/berth/main/install-remote.sh | sh -``` - -Each script checks the download against its SHA-256. It installs to `%LOCALAPPDATA%\Programs\berth-remote` on Windows and to `~/.local/bin` elsewhere. `BERTH_INSTALL_DIR` and `BERTH_VERSION` change where and which release. - -## Use - -Start the server on an address of your network: - -```bash -berth serve --listen 0.0.0.0:7878 -``` - -Then on your own machine: - -```bash -berth-remote --server http://192.0.2.10:7878 -``` - -berth-remote asks for the token and keeps it for the next run. The token is in `serve.token` in the server's data directory. A setup key from `berth setup-key` works too. - -| Option or variable | What it does | -| --- | --- | -| `--server ADDRESS`, `BERTH_SERVER` | The server. An address without a scheme is https, or http for `localhost` | -| `BERTH_TOKEN` | The token, so that it is not asked for | -| `BERTH_SETUP_KEY` | A setup key, which collects the token once | -| `--forget` | Remove the kept server and token, and keep nothing from this run | -| `BERTH_REMOTE_FILE` | The plugin package to send, on a system without a file dialog | -| `--version` | Say the version | - -A token or a setup key never goes on the command line, where other users of the machine could read it, and berth-remote does not pass either on to the programs it starts. - -The settings are in `settings.json` in `%LOCALAPPDATA%\berth-remote` on Windows, `~/Library/Application Support/berth-remote` on macOS and `~/.config/berth-remote` elsewhere (`$XDG_CONFIG_HOME` where it is set). On macOS and Linux the folder is mode 0700 and the file 0600. On Windows the file takes the access list of your profile, which lets in you, SYSTEM and the administrators. - -A server over https needs a certificate that the system or the Mozilla roots trust. There is no way to turn the check off. Only `http` and `https` links are opened, in the system's browser. - -Ctrl+Q quits. - -## How it is built - -The screens are in `crates/berth-page`, which the page for a browser (`crates/berth-web`) uses too. This crate gives them a terminal in place of a browser: `device.rs` has the clipboard, links, the file dialog and the kept settings, `net.rs` and `socket.rs` the HTTP requests and the portal's WebSocket over TCP or TLS. - -On the system it is for: - -```bash -cargo build --release --locked -p berth-remote -``` - -On Windows, that needs Rust from rustup and the Visual Studio build tools. Use Windows Terminal or the console of Windows 10 or later, which draw its colours. diff --git a/crates/berth-remote/src/lib.rs b/crates/berth-remote/src/lib.rs deleted file mode 100644 index 259c488..0000000 --- a/crates/berth-remote/src/lib.rs +++ /dev/null @@ -1,19 +0,0 @@ -//! berth-remote: the page of `berth serve --web` in a terminal, for a -//! machine that only looks after the hosts of a berth serve elsewhere. It -//! runs on Linux, macOS and Windows: the screens are berth-page's, and this -//! crate gives them a terminal and a network in place of a browser. It is a -//! library as well as the program so that berth's tests can sign it in to a -//! server of their own. - -pub mod device; -pub mod keys; -pub mod net; -pub mod options; -pub mod settings; -pub mod socket; - -/// Take in what the network said since the last turn, on the thread that draws. -pub fn pump() { - net::pump(); - socket::pump(); -} diff --git a/crates/berth-remote/src/main.rs b/crates/berth-remote/src/main.rs deleted file mode 100644 index 782eac4..0000000 --- a/crates/berth-remote/src/main.rs +++ /dev/null @@ -1,97 +0,0 @@ -//! berth-remote: the hosts of a berth serve, in this terminal. - -use berth_remote::options::{self, Parsed}; -use berth_remote::{device, keys, settings}; -use crossterm::event::{self, Event as Native, KeyCode, KeyEventKind, KeyModifiers}; -use std::time::Duration; - -fn main() { - let options = match options::parse(std::env::args().skip(1), |name| std::env::var(name).ok()) { - Ok(Parsed::Run(options)) => options, - Ok(Parsed::Version) => { - println!("berth-remote {}", berth_api::VERSION); - return; - } - Ok(Parsed::Help) => { - println!("{}", options::USAGE); - return; - } - Err(error) => { - eprintln!("berth-remote: {error}\n\n{}", options::USAGE); - std::process::exit(2); - } - }; - let settings = settings::Settings::new(settings::folder(|name| std::env::var(name).ok())); - match device::Terminal::new(options.linked, settings, options.forget) { - Ok(terminal) => berth_page::device::install(terminal), - Err(error) => { - eprintln!("berth-remote: could not forget the kept sign-in: {error}"); - std::process::exit(1); - } - } - if let Err(error) = run() { - eprintln!("berth-remote: {error}"); - std::process::exit(1); - } -} - -/// Stop the mouse and paste reports, which ratatui's own restore leaves on. -fn stop_reports() { - let _ = crossterm::execute!( - std::io::stdout(), - event::DisableBracketedPaste, - event::DisableMouseCapture - ); -} - -fn run() -> std::io::Result<()> { - let mut terminal = ratatui::init(); - let restore = std::panic::take_hook(); - std::panic::set_hook(Box::new(move |info| { - stop_reports(); - restore(info); - })); - // Not every terminal of Windows knows bracketed paste: there a paste - // arrives as keys. - let _ = crossterm::execute!( - std::io::stdout(), - event::EnableMouseCapture, - event::EnableBracketedPaste - ); - let outcome = screens(&mut terminal); - stop_reports(); - ratatui::restore(); - outcome -} - -fn screens(terminal: &mut ratatui::DefaultTerminal) -> std::io::Result<()> { - let mut app = berth_page::App::new(); - loop { - berth_remote::pump(); - app.tick(); - terminal.draw(|frame| app.draw(frame))?; - // Wait a little for input; what the network says is taken in on the next turn. - if !keys::poll(Duration::from_millis(30))? { - continue; - } - loop { - match keys::read()? { - Native::Key(key) - if key.kind != KeyEventKind::Release - && key.modifiers.contains(KeyModifiers::CONTROL) - && matches!(key.code, KeyCode::Char('q' | 'Q')) => - { - return Ok(()); - } - native => { - if let Some(event) = keys::event(&native) { - app.event(event); - } - } - } - if !keys::poll(Duration::ZERO)? { - break; - } - } - } -} diff --git a/crates/berth-remote/src/options.rs b/crates/berth-remote/src/options.rs deleted file mode 100644 index 6fbff85..0000000 --- a/crates/berth-remote/src/options.rs +++ /dev/null @@ -1,165 +0,0 @@ -//! What berth-remote is started with: its options, and from the environment -//! a token or a setup key, which never go on a command line. - -use berth_page::device::Linked; - -pub const USAGE: &str = "\ -berth-remote: the hosts of a berth serve, in this terminal - -Usage: berth-remote [--server ADDRESS] [--forget] - - --server ADDRESS the berth serve to use, such as https://berth.example - or http://192.0.2.10:7878 (also BERTH_SERVER) - --forget remove the kept server and token, and keep nothing - from this run - --version say the version - -The token comes from BERTH_TOKEN, or is asked for. A setup key from -`berth setup-key` comes from BERTH_SETUP_KEY. Neither goes on the command -line, where other users of the machine could read it. - -A plugin package is chosen in the file dialog of the system: on Linux that -of zenity or kdialog. Without one, BERTH_REMOTE_FILE names the file. - -The server and the token are kept for this user: in -%LOCALAPPDATA%\\berth-remote on Windows, ~/Library/Application Support/berth-remote -on macOS and ~/.config/berth-remote elsewhere ($XDG_CONFIG_HOME where it is set). - -Ctrl+Q quits."; - -pub struct Options { - pub linked: Linked, - /// Remove what was kept, and keep nothing. - pub forget: bool, -} - -pub enum Parsed { - Run(Options), - Version, - Help, -} - -/// Read the arguments, and the variables `var` gives. An argument that is -/// not an option is not repeated in the error, since it could be a token. -pub fn parse( - args: impl IntoIterator, - var: impl Fn(&str) -> Option, -) -> Result { - let variable = |name: &str| { - var(name) - .map(|value| value.trim().to_owned()) - .filter(|value| !value.is_empty()) - }; - let mut options = Options { - linked: Linked { - server: variable("BERTH_SERVER"), - token: variable("BERTH_TOKEN"), - setup_key: variable("BERTH_SETUP_KEY"), - }, - forget: false, - }; - let mut args = args.into_iter(); - while let Some(arg) = args.next() { - let (name, value) = match arg.split_once('=') { - Some((name, value)) if name.starts_with("--") => (name, Some(value.to_owned())), - _ => (arg.as_str(), None), - }; - match name { - "--server" | "-s" => { - let server = value - .or_else(|| args.next()) - .filter(|server| !server.trim().is_empty()) - .ok_or("--server needs an address")?; - options.linked.server = Some(server.trim().to_owned()); - } - "--forget" if value.is_none() => options.forget = true, - "--version" | "-V" if value.is_none() => return Ok(Parsed::Version), - "--help" | "-h" if value.is_none() => return Ok(Parsed::Help), - "--forget" | "--version" | "--help" => return Err(format!("{name} takes no value")), - other if other.starts_with('-') => { - return Err(format!("{other} is no option of berth-remote")); - } - _ => { - return Err( - "berth-remote takes no arguments but its options: a token comes from BERTH_TOKEN" - .into(), - ); - } - } - } - Ok(Parsed::Run(options)) -} - -#[cfg(test)] -mod tests { - use super::*; - - fn run(args: &[&str], vars: &[(&str, &str)]) -> Result { - let vars: Vec<(String, String)> = vars - .iter() - .map(|(name, value)| (name.to_string(), value.to_string())) - .collect(); - let parsed = parse(args.iter().map(|arg| arg.to_string()), |name| { - vars.iter() - .find(|(key, _)| key == name) - .map(|(_, value)| value.clone()) - })?; - match parsed { - Parsed::Run(options) => Ok(options), - _ => Err("not a run".into()), - } - } - - #[test] - fn the_server_comes_from_an_option_or_the_environment() { - let options = run(&[], &[("BERTH_SERVER", " https://studio.example ")]).unwrap(); - assert_eq!( - options.linked.server.as_deref(), - Some("https://studio.example") - ); - assert!(!options.forget); - let options = run( - &["--server", "http://192.0.2.10:7878"], - &[("BERTH_SERVER", "https://studio.example")], - ) - .unwrap(); - assert_eq!( - options.linked.server.as_deref(), - Some("http://192.0.2.10:7878") - ); - let options = run(&["--server=https://dock.example", "--forget"], &[]).unwrap(); - assert_eq!( - options.linked.server.as_deref(), - Some("https://dock.example") - ); - assert!(options.forget); - assert!(run(&["--server"], &[]).is_err()); - assert!(run(&["--server="], &[]).is_err()); - } - - #[test] - fn secrets_come_from_the_environment_only() { - let options = run( - &[], - &[("BERTH_TOKEN", "the-token\n"), ("BERTH_SETUP_KEY", "")], - ) - .unwrap(); - assert_eq!(options.linked.token.as_deref(), Some("the-token")); - assert_eq!(options.linked.setup_key, None); - // A token given as an argument is refused, and not repeated. - let error = run(&["s3cr3t-token"], &[]).err().unwrap(); - assert!(!error.contains("s3cr3t"), "{error}"); - let error = run(&["--token=s3cr3t-token"], &[]).err().unwrap(); - assert!(!error.contains("s3cr3t"), "{error}"); - assert!(error.contains("--token"), "{error}"); - } - - #[test] - fn version_and_help() { - let parsed = |args: &[&str]| parse(args.iter().map(|arg| arg.to_string()), |_| None); - assert!(matches!(parsed(&["--version"]), Ok(Parsed::Version))); - assert!(matches!(parsed(&["-V"]), Ok(Parsed::Version))); - assert!(matches!(parsed(&["--help"]), Ok(Parsed::Help))); - assert!(parsed(&["--forget=yes"]).is_err()); - } -} diff --git a/crates/berth-remote/src/settings.rs b/crates/berth-remote/src/settings.rs deleted file mode 100644 index 231456b..0000000 --- a/crates/berth-remote/src/settings.rs +++ /dev/null @@ -1,224 +0,0 @@ -//! The server and token berth-remote keeps between runs: a JSON file that -//! only its user can read, in a folder that only its user can open. - -use serde_json::{Map, Value}; -use std::io::Write; -use std::path::{Path, PathBuf}; - -/// The folder of the settings for the environment `var` gives: under -/// `%LOCALAPPDATA%` on Windows, `$XDG_CONFIG_HOME` where it is set, and else -/// `~/Library/Application Support` on macOS and `~/.config` elsewhere. -pub fn folder(var: impl Fn(&str) -> Option) -> Option { - let var = |name: &str| var(name).filter(|value| !value.is_empty()); - let base = if cfg!(windows) { - // Local rather than roaming, so that the token is not copied to - // other machines with the profile. - PathBuf::from(var("LOCALAPPDATA")?) - } else if let Some(dir) = var("XDG_CONFIG_HOME") { - PathBuf::from(dir) - } else if cfg!(target_os = "macos") { - PathBuf::from(var("HOME")?).join("Library/Application Support") - } else { - PathBuf::from(var("HOME")?).join(".config") - }; - Some(base.join("berth-remote")) -} - -pub struct Settings { - dir: Option, -} - -impl Settings { - /// The settings in `dir`; with `None`, nothing is read or written. - pub fn new(dir: Option) -> Settings { - Settings { dir } - } - - fn file(&self) -> Option { - Some(self.dir.as_ref()?.join("settings.json")) - } - - /// What is kept; nothing when the file is missing or unreadable. - pub fn read(&self) -> Map { - self.file() - .and_then(|file| std::fs::read(file).ok()) - .and_then(|bytes| serde_json::from_slice::(&bytes).ok()) - .and_then(|value| match value { - Value::Object(map) => Some(map), - _ => None, - }) - .unwrap_or_default() - } - - /// Keep `kept`, replacing what was. An empty map removes the file. - pub fn write(&self, kept: &Map) -> std::io::Result<()> { - let (Some(dir), Some(file)) = (&self.dir, self.file()) else { - return Ok(()); - }; - if kept.is_empty() { - return self.forget(); - } - private_dir(dir)?; - private_write(&file, Value::Object(kept.clone()).to_string().as_bytes()) - } - - /// Remove what was kept: the file, and its folder when nothing else is in it. - pub fn forget(&self) -> std::io::Result<()> { - let (Some(dir), Some(file)) = (&self.dir, self.file()) else { - return Ok(()); - }; - for gone in [file.clone(), file.with_extension("json.partial")] { - match std::fs::remove_file(&gone) { - Err(error) if error.kind() != std::io::ErrorKind::NotFound => return Err(error), - _ => {} - } - } - // Not an error: a folder with other files in it stays. - let _ = std::fs::remove_dir(dir); - Ok(()) - } -} - -/// Make the folder of the settings, which only its owner can open. -fn private_dir(dir: &Path) -> std::io::Result<()> { - let mut builder = std::fs::DirBuilder::new(); - builder.recursive(true); - #[cfg(unix)] - std::os::unix::fs::DirBuilderExt::mode(&mut builder, 0o700); - builder.create(dir)?; - // A folder that was there already keeps its mode otherwise. - #[cfg(unix)] - std::fs::set_permissions(dir, std::os::unix::fs::PermissionsExt::from_mode(0o700))?; - Ok(()) -} - -/// Write a file only its owner can read, since it holds the token. On -/// Windows the file takes the access list of the user's profile, which lets -/// in the user, SYSTEM and the administrators. -fn private_write(file: &Path, bytes: &[u8]) -> std::io::Result<()> { - let partial = file.with_extension("json.partial"); - // A file left over would keep its own mode. - match std::fs::remove_file(&partial) { - Err(error) if error.kind() != std::io::ErrorKind::NotFound => return Err(error), - _ => {} - } - let mut options = std::fs::OpenOptions::new(); - options.write(true).create_new(true); - #[cfg(unix)] - std::os::unix::fs::OpenOptionsExt::mode(&mut options, 0o600); - let mut out = options.open(&partial)?; - out.write_all(bytes)?; - out.sync_all()?; - drop(out); - std::fs::rename(partial, file) -} - -#[cfg(test)] -mod tests { - use super::*; - use serde_json::json; - - fn kept(value: Value) -> Map { - value.as_object().unwrap().clone() - } - - #[test] - fn keeps_and_forgets() { - let scratch = tempfile::tempdir().unwrap(); - let dir = scratch.path().join("config/berth-remote"); - let settings = Settings::new(Some(dir.clone())); - assert!(settings.read().is_empty()); - let both = kept(json!({"server": "https://studio.example", "token": "secret"})); - settings.write(&both).unwrap(); - assert_eq!(settings.read(), both); - settings - .write(&kept(json!({"server": "https://dock.example"}))) - .unwrap(); - assert_eq!(settings.read()["server"], "https://dock.example"); - assert!(!settings.read().contains_key("token")); - assert!(!dir.join("settings.json.partial").exists()); - settings.forget().unwrap(); - assert!(!dir.exists()); - assert!(settings.read().is_empty()); - // Forgetting twice, or what was never kept, is no error. - settings.forget().unwrap(); - } - - #[test] - fn an_empty_map_removes_the_file() { - let scratch = tempfile::tempdir().unwrap(); - let settings = Settings::new(Some(scratch.path().join("berth-remote"))); - settings.write(&kept(json!({"token": "secret"}))).unwrap(); - settings.write(&Map::new()).unwrap(); - assert!(!scratch.path().join("berth-remote").exists()); - } - - #[test] - fn a_file_that_is_not_settings_reads_as_nothing() { - let scratch = tempfile::tempdir().unwrap(); - let settings = Settings::new(Some(scratch.path().to_owned())); - std::fs::write(scratch.path().join("settings.json"), "[1, 2]").unwrap(); - assert!(settings.read().is_empty()); - std::fs::write(scratch.path().join("settings.json"), "{not json").unwrap(); - assert!(settings.read().is_empty()); - } - - #[cfg(unix)] - #[test] - fn only_the_owner_can_read_them() { - use std::os::unix::fs::PermissionsExt; - let scratch = tempfile::tempdir().unwrap(); - let dir = scratch.path().join("berth-remote"); - // A folder made earlier with a looser mode is made private too. - std::fs::create_dir(&dir).unwrap(); - std::fs::set_permissions(&dir, std::fs::Permissions::from_mode(0o755)).unwrap(); - // So is a file left over from a write that did not finish. - std::fs::write(dir.join("settings.json.partial"), "{}").unwrap(); - std::fs::set_permissions( - dir.join("settings.json.partial"), - std::fs::Permissions::from_mode(0o644), - ) - .unwrap(); - let settings = Settings::new(Some(dir.clone())); - settings.write(&kept(json!({"token": "secret"}))).unwrap(); - let mode = |path: &Path| std::fs::metadata(path).unwrap().permissions().mode() & 0o777; - assert_eq!(mode(&dir), 0o700); - assert_eq!(mode(&dir.join("settings.json")), 0o600); - } - - #[test] - fn the_folder_follows_the_system() { - let env = |pairs: &'static [(&'static str, &'static str)]| { - move |name: &str| { - pairs - .iter() - .find(|(key, _)| *key == name) - .map(|(_, value)| value.to_string()) - } - }; - if cfg!(windows) { - assert_eq!( - folder(env(&[("LOCALAPPDATA", r"C:\Users\ada\AppData\Local")])), - Some(PathBuf::from(r"C:\Users\ada\AppData\Local").join("berth-remote")) - ); - assert_eq!(folder(env(&[])), None); - return; - } - assert_eq!( - folder(env(&[ - ("XDG_CONFIG_HOME", "/scratch/c"), - ("HOME", "/home/ada") - ])), - Some(PathBuf::from("/scratch/c/berth-remote")) - ); - let home = match cfg!(target_os = "macos") { - true => "/home/ada/Library/Application Support/berth-remote", - false => "/home/ada/.config/berth-remote", - }; - assert_eq!( - folder(env(&[("XDG_CONFIG_HOME", ""), ("HOME", "/home/ada")])), - Some(PathBuf::from(home)) - ); - assert_eq!(folder(env(&[])), None); - } -} diff --git a/crates/berth-tui/Cargo.toml b/crates/berth-tui/Cargo.toml index 5cb0ecb..041985e 100644 --- a/crates/berth-tui/Cargo.toml +++ b/crates/berth-tui/Cargo.toml @@ -4,7 +4,7 @@ version.workspace = true edition.workspace = true rust-version.workspace = true license.workspace = true -description = "The screens of a host with no terminal in them, for berth and for its page in a browser" +description = "The parts of berth's screens with no terminal in them" [dependencies] berth-api.workspace = true @@ -16,4 +16,3 @@ ratatui = { version = "=0.30.2", default-features = false } qrcode = { version = "0.14", default-features = false } pulldown-cmark = { version = "0.13", default-features = false } tui-input.workspace = true -web-time = "1" diff --git a/crates/berth-tui/src/berth_key.rs b/crates/berth-tui/src/berth_key.rs index c6b3ffa..3c34180 100644 --- a/crates/berth-tui/src/berth_key.rs +++ b/crates/berth-tui/src/berth_key.rs @@ -54,7 +54,7 @@ pub fn help(name: &str, public: &str, server: Option<&str>) -> String { _ => "", }; format!( - "{whose}\n\nAdd it as an authorized key at your cloud provider, or to `~/.ssh/authorized_keys` on a host, and berth gets in without a key of yours; `berth key trust HOST` does that with a password used once.{more} The private half stays with berth: `berth key` lists the keys.\n\nEnter copies it.\n\n`{public}`" + "{whose}\n\nAdd it as an authorized key at your cloud provider, or to `~/.ssh/authorized_keys` on a host, and berth gets in without a key of yours; `berth key trust HOST` does that with a password used once.{more} The private half stays with berth: `berth key list` lists the keys.\n\nEnter copies it.\n\n`{public}`" ) } @@ -92,7 +92,7 @@ mod tests { } fn open(&mut self, _: &str) -> Result<(), String> { - Err("no browser".into()) + Err("nothing to open it with".into()) } } diff --git a/crates/berth-tui/src/event.rs b/crates/berth-tui/src/event.rs index 91bb5c1..0a9acff 100644 --- a/crates/berth-tui/src/event.rs +++ b/crates/berth-tui/src/event.rs @@ -1,5 +1,5 @@ -//! Keys and the mouse as the screens take them. A terminal's and a browser's -//! events are both turned into these. +//! Keys and the mouse as the screens take them. The events of a terminal are +//! turned into these. #[derive(Clone, Copy, Debug, PartialEq, Eq)] pub enum Code { diff --git a/crates/berth-tui/src/fields.rs b/crates/berth-tui/src/fields.rs index 7a7e82e..5399d53 100644 --- a/crates/berth-tui/src/fields.rs +++ b/crates/berth-tui/src/fields.rs @@ -1,6 +1,7 @@ //! The settings the UI can edit, and the wizard steps and actions around them. -pub const PROFILES: &[&str] = &["laptop", "cloud"]; +/// What a host can be installed on, as its profile's `target` says. +pub const TARGETS: &[&str] = &["laptop", "cloud"]; pub const DISK_MODES: &[&str] = &[ "auto", "serial", "model", "path", "largest", "smallest", "ssd", "hdd", ]; @@ -8,8 +9,8 @@ pub const DISK_TYPES: &[&str] = &["any", "ssd", "hdd"]; /// The ways to log in that are not a key of the ring; `key:NAME` is offered /// for each of those. pub const LOGINS: &[&str] = &["berth", "password", "ssh"]; -/// The levels a host can be meant to have, the whole profile first. -pub const LEVELS: &[&str] = &["workstation", "controller", "reachable"]; +/// The types a host can be meant to be, the whole profile first. +pub const TYPES: &[&str] = &["workstation", "node", "managed", "ssh"]; /// The pages of a host: its overview (`Manage`) and the tabs of the set-up /// flow. Each tab depends only on the ones before it. @@ -50,12 +51,12 @@ impl Step { } /// The rows of a host's overview that are berth's own: name, label and help. -/// Everything else there is a task the host lists itself. -pub const TASKS: &[(&str, &str, &str)] = &[ +/// Everything else there is an action the host lists itself. +pub const OFFERS: &[(&str, &str, &str)] = &[ ( "terminal", "Terminal", - "A shell on the host, in a panel here. Space switches between the agent account, with its tools on the PATH and the virtual desktop as its display, and the SSH user. It is a session of the host: Ctrl+Q leaves it running and it can be opened again from Sessions; exit or Ctrl+D ends it. The mouse wheel scrolls back. From the command line: `berth attach HOST`.", + "A shell on the host, in a panel here. Space switches between the agent account, with its tools on the PATH and the virtual desktop as its display, and the SSH user. It is a session of the host: Ctrl+Q leaves it running and it can be opened again from Sessions; exit or Ctrl+D ends it. The mouse wheel scrolls back. From the command line: `berth session attach HOST`.", ), ( "apply", @@ -75,7 +76,7 @@ pub const TASKS: &[(&str, &str, &str)] = &[ ( "install-controller", "Install the controller", - "Put berthd and box on the host and start berth's controller, so that sessions, tasks and files work there, without the rest of a workstation. The host's level becomes controller. Continues in the terminal. From the command line: `berth host install HOST`.", + "Put berthd and box on the host and start berth's controller, so that sessions, actions and files work there, without the rest of a workstation. The host's type becomes managed. Continues in the terminal. From the command line: `berth run HOST install-controller`.", ), ( "ssh", @@ -98,7 +99,7 @@ pub enum Action { Docker, Ssh, /// An existing host berth only logs in to: a way offered on the Install - /// tab that chooses [`Action::Ssh`] at the level reachable. + /// tab that chooses [`Action::Ssh`] for a host of the type ssh. Reach, Manage, } @@ -115,20 +116,79 @@ impl Action { Action::Docker, ]; - /// The action a way of the Install tab sets, with the level it gives the - /// profile when it gives one. + /// The ways a berth server installs a host itself, as the screens and + /// `berth host create` offer them. The USB installer is built on the + /// machine the server runs on. + pub const SERVED: [Action; 5] = [ + Action::Machine, + Action::Iso, + Action::CloudSeed, + Action::Ssh, + Action::Reach, + ]; + + /// The way's name on the command line, as `--method` takes it. + pub fn key(self) -> &'static str { + match self { + Action::Save => "save", + Action::Release => "release", + Action::UsbSeed => "usb-seed", + Action::Iso => "iso", + Action::CloudSeed => "cloud-seed", + Action::Machine => "provisioner", + Action::Apply => "inventory", + Action::Docker => "container", + Action::Ssh => "ssh", + Action::Reach => "ssh-only", + Action::Manage => "manage", + } + } + + /// The way of [`Action::SERVED`] a name on the command line means. + pub fn by_key(key: &str) -> Option { + Action::SERVED + .into_iter() + .find(|action| action.key() == key) + } + + /// The types a host installed this way can have. Only over SSH can berth + /// install less than a workstation: an installer, a seed or a provisioner + /// installs the whole profile from a blank machine. + pub fn types(self) -> &'static [berth_api::HostType] { + use berth_api::HostType::*; + match self { + Action::Reach => &[Ssh], + Action::Ssh | Action::Manage => &[Managed, Node, Workstation], + _ => &[Workstation], + } + } + + /// Whether the Install tab offers this way to a host of `host_type`: + /// the ways its type allows, and the two over SSH, which move a host + /// between the type ssh and the types berth installs. + pub fn offered(self, host_type: berth_api::HostType) -> bool { + matches!(self, Action::Ssh | Action::Reach) || self.types().contains(&host_type) + } + + /// The action a way of the Install tab sets, with the type it gives the + /// profile when it implies one. pub fn choose(self, config: &mut serde_json::Value) -> Action { - let level = berth_api::profile::level(config); + use berth_api::HostType; + let host_type = berth_api::profile::host_type(config); match self { Action::Reach => { - berth_api::profile::set_level(config, berth_api::HostLevel::Reachable); + berth_api::profile::set_host_type(config, HostType::Ssh); Action::Ssh } // Installing over SSH installs something: the whole profile, unless the controller is chosen. - Action::Ssh if level == berth_api::HostLevel::Reachable => { - berth_api::profile::set_level(config, berth_api::HostLevel::Workstation); + Action::Ssh if host_type == HostType::Ssh => { + berth_api::profile::set_host_type(config, HostType::Workstation); Action::Ssh } + action if !action.types().contains(&host_type) => { + berth_api::profile::set_host_type(config, HostType::Workstation); + action + } action => action, } } @@ -140,7 +200,7 @@ impl Action { Action::UsbSeed => "Render USB seed", Action::Iso => "USB installer", Action::CloudSeed => "Cloud-init seed", - Action::Machine => "Create a machine", + Action::Machine => "From a provisioner", Action::Apply => "This machine or an inventory", Action::Docker => "Container image", Action::Ssh => "Existing host over SSH", @@ -157,13 +217,13 @@ impl Action { "Generate the NoCloud files and provisioning bundle for an unattended installer." } Action::Iso => { - "For a blank machine: build an installer image from the Ubuntu Server ISO that installs and sets up the host without questions, and write it to a USB drive. F4 on the ISO field downloads the ISO." + "For a blank machine: build an installer image from the Ubuntu Server ISO that installs and sets up the host without questions, and write it to a USB drive. Without an ISO the current one is downloaded and checked first. It is built on the machine berth serve runs on." } Action::CloudSeed => { "Generate cloud-init user data. Upload the generated release archive to the configured URL." } Action::Machine => { - "Have a provisioner create the machine, such as a container or a virtual machine on a server, and set it up as this host. Choose the provisioner below; plugins bring them. Its size is on the Settings tab, and Create is on the Run tab." + "Have a provisioner make the host, such as a container or a virtual machine on a server, and set it up. Choose the provisioner below; plugins bring them. Its size is on the Settings tab, and Create is on the Run tab." } Action::Apply => { "Provision a dedicated Ubuntu host using Ansible. Blank inventory applies to this machine." @@ -172,13 +232,13 @@ impl Action { "Build the desktop image using Docker Compose. Start it afterwards with docker compose." } Action::Manage => { - "Look after a host that is already set up, over SSH: show its status and access links, publish the desktop on the tailnet, run the smoke test, install updates or reboot. Pick the task on the overview." + "Look after a host that is already set up, over SSH: show its status and access links, publish the desktop on the tailnet, run the smoke test, install updates or reboot. Pick the action on the overview." } Action::Ssh => { - "Install on an existing Ubuntu host over SSH. Enter a hostname, IP address or SSH config alias, and how berth logs in: its own key by default. A host that does not take that key yet gets it with berth key trust HOST and a password used once. Sudo prompts use the normal terminal." + "Install on an existing Ubuntu host over SSH. Enter a hostname, IP address or SSH config alias, and how berth logs in: its own key by default. A host that does not take that key yet gets it with berth key trust HOST and a password used once." } Action::Reach => { - "Add a host berth only logs in to over SSH, such as a server that stays as it is: nothing is installed on it. Enter a hostname, IP address or SSH config alias, and how berth logs in. Run saves it; a host that does not take berth's key yet gets it with berth key trust HOST. Its page can install the controller later, for sessions, tasks and files." + "Add a host berth only logs in to over SSH, such as a server that stays as it is: nothing is installed on it. Enter a hostname, IP address or SSH config alias, and how berth logs in. Run saves it; a host that does not take berth's key yet gets it with berth key trust HOST. Its page can install the controller later, for sessions, actions and files." } } } @@ -219,7 +279,7 @@ pub struct Spec { #[derive(Clone, Copy, PartialEq, Eq, Debug)] pub enum Field { - Profile, + Target, Hostname, Keyboard, Geometry, @@ -227,9 +287,6 @@ pub enum Field { RecoveryCidr, ReleaseStore, ReleasePrivate, - TailscaleKey, - TailscaleAuto, - TailscaleLoginServer, DiskMode, DiskSerial, DiskModel, @@ -269,25 +326,114 @@ pub enum Field { SshUser, SshPort, Login, - Level, + Type, } impl Field { + /// Every field, in the order of the set-up tabs. + pub const ALL: [Field; 48] = { + use Field::*; + [ + Type, + Target, + SshHost, + SshUser, + SshPort, + Login, + AskBecomePass, + Hostname, + Tailscale, + ToolsWeb, + ToolsNative, + ToolsJvm, + ToolsDotnet, + ToolsScripting, + ToolsScientific, + ToolsChrome, + Playwright, + SoftwareRendering, + Cockpit, + Docker, + OperatorUser, + OperatorKey, + OperatorHash, + AgentUser, + AgentHash, + AgentKey, + AgentPrivileges, + AgentSudo, + HardenSsh, + RecoveryCidr, + Geometry, + DiskMode, + DiskSerial, + DiskModel, + DiskPath, + DiskType, + Keyboard, + Wifi, + WifiSsid, + WifiPassword, + WifiInterface, + ReleaseStore, + ReleaseUrl, + ReleasePrivate, + Iso, + IsoSha, + UsbDevice, + Inventory, + ] + }; + + /// The field's name on the command line, as `--set KEY=VALUE` takes it: + /// its place in the profile, without the parts every field of a kind + /// shares. + pub fn key(self) -> String { + let pointer = self.spec().pointer.trim_start_matches('/'); + for (part, instead) in [ + ("access/", ""), + ("builder/", ""), + ("ansible_vars/devbox_", ""), + ("wifi/", "wifi_"), + ] { + if let Some(rest) = pointer.strip_prefix(part) { + return format!("{instead}{rest}"); + } + } + pointer.to_owned() + } + + /// The field a name on the command line means. + pub fn by_key(key: &str) -> Option { + Field::ALL.into_iter().find(|field| field.key() == key) + } + + /// Whether `berth host create` asks for the field when it is shown and + /// not given: a choice most hosts make. Others are asked for only when + /// a problem names them. + pub fn asked(self) -> bool { + use Field::*; + matches!( + self, + Target | Type | SshHost | SshUser | OperatorUser | AgentUser + ) + } + pub fn spec(self) -> Spec { use Field::*; use Kind::*; let (label, pointer, kind, help) = match self { - Profile => ( - "Host profile", - "/profile", - Choice(PROFILES), - "Choose laptop or cloud provisioning defaults. Only laptop adds a physical login and power settings. Cloud leaves SSH open to any address when no recovery SSH CIDR is set; for a QEMU guest, set that to 10.0.2.2/32.", + Target => ( + "Target", + "/target", + Choice(TARGETS), + "What the host is installed on: laptop or cloud, with their provisioning defaults. Only laptop adds a physical login and power settings. Cloud leaves SSH open to any address when no recovery SSH CIDR is set; for a QEMU guest, set that to 10.0.2.2/32.", ), Hostname => ( "Hostname", "/hostname", Text, - "Hostname assigned to the installed target. Lowercase letters, numbers and hyphens, starting with a letter. F4 generates one.", + "Hostname assigned to the installed target. Lowercase letters, numbers and hyphens, starting with a letter. Left empty, the host's name is used when it can be one.", ), Keyboard => ( "Keyboard layout", @@ -325,24 +471,6 @@ impl Field { Text, "Network allowed to use SSH from outside the tailnet, such as your admin IP with /32. Left empty, a cloud host accepts SSH from any address, and other hosts only from the tailnet and the session used for installing.", ), - TailscaleKey => ( - "Tailscale enrollment key", - "/tailscale_auth_key", - Secret, - "Optional enrollment key from Tailscale or Headscale. Prefer a short-lived, one-use key. With the API key of the control server in the secret tailnet_api_key (or a Tailscale token in TS_API_KEY), F4 creates a one-use key valid for 24 hours.", - ), - TailscaleAuto => ( - "Automatic enrollment key", - "/tailscale_auto_key", - Toggle, - "berth creates a one-use key whenever this host is about to join the tailnet: when a seed or installer is made (valid for 24 hours) and when an apply finds the host not on the tailnet yet (valid for an hour). It needs the control server's API key in the secret tailnet_api_key and, for Headscale, the tailnet_ settings. A host on the tailnet already needs no key.", - ), - TailscaleLoginServer => ( - "Login server", - "/tailscale_login_server", - Text, - "The Headscale server this host joins, such as https://hs.example.org. Leave it empty for berth's tailnet_login_server setting, or Tailscale's own service when that is empty too. It takes effect when the host joins.", - ), DiskMode => ( "Disk selection", "/target_disk_mode", @@ -389,7 +517,7 @@ impl Field { "Wi-Fi SSID", "/wifi/ssid", Text, - "Wireless network name. Networks saved on this machine are listed in Suggestions. F5 unlocks their saved passwords, so choosing one also fills in its password.", + "Wireless network name. Networks saved on this machine are listed in Suggestions, and choosing one with its saved password fills that in too.", ), WifiPassword => ( "Wi-Fi password", @@ -413,7 +541,7 @@ impl Field { "Operator password hash", "/operator_password_hash", Secret, - "SHA-512 crypt or yescrypt hash of the operator password. F4 generates a password and hash, and berth credentials HOST shows the password later. F5 hashes a password you enter.", + "SHA-512 crypt or yescrypt hash of the operator password. Generate a password and hash, and berth host credentials HOST shows the password later, or enter a password to hash.", ), AgentUser => ( "Agent username", @@ -431,7 +559,7 @@ impl Field { "Agent password hash", "/agent_password_hash", Secret, - "Optional local password for the agent. F4 generates a password and hash, and berth credentials HOST shows the password later. F5 hashes a password you enter. Empty locks password login.", + "Optional local password for the agent. Generate a password and hash, and berth host credentials HOST shows the password later, or enter a password to hash. Empty locks password login.", ), AgentPrivileges => ( "Agent privileges", @@ -521,19 +649,19 @@ impl Field { "Tailscale", "/ansible_vars/devbox_install_tailscale", Toggle, - "Install Tailscale on the target host and enable optional enrollment.", + "Install Tailscale and join the tailnet with a one-use key berth makes through the control server's API, from the secret tailnet_api_key and the tailnet_ settings. A host on the tailnet already is left on it. Off, berth reaches the host over SSH at its address, and the desktop and T3 through an SSH tunnel.", ), Iso => ( "Source ISO", "/builder/iso", Path, - "Ubuntu Server 26.04 amd64 installer ISO to build from. F4 downloads the current one and checks it against Ubuntu's published SHA256SUMS.", + "Ubuntu Server 26.04 amd64 installer ISO to build from, a path on the machine berth serve runs on. Left empty, the build downloads the current one and checks it against Ubuntu's published SHA256SUMS.", ), IsoSha => ( "ISO SHA256", "/builder/iso_sha256", Text, - "SHA-256 the source ISO must have. Filled in from the file when you choose the ISO, and F4 recomputes it. Compare it with Ubuntu's published SHA256SUMS before trusting the image.", + "SHA-256 the source ISO must have, which the build checks. Take it from Ubuntu's published SHA256SUMS.", ), Inventory => ( "Ansible inventory", @@ -571,17 +699,17 @@ impl Field { Choice(LOGINS), "How berth logs in. berth: the installation's own key, which hosts it sets up take. key:NAME: another key of berth's key ring (the Settings tab of the home screen lists them). password: a password, kept as the secret ssh/HOST or asked for. ssh: your own SSH configuration and agent, with berth's key offered after them. A host that does not take its key yet gets it with: berth key trust HOST.", ), - Level => ( - "Level", - "/level", - Choice(LEVELS), - "How much of berth the host gets. workstation: the whole profile, with the desktop, the agents and plugins. controller: berthd and box alone, for sessions, tasks and files on a machine that stays as it is. reachable: nothing is installed; berth logs in over SSH. A lower level can be raised later: install the controller, or set workstation and apply.", + Type => ( + "Type", + "/type", + Choice(TYPES), + "What berth does with the host. workstation: the whole profile, with the desktop, the agents and plugins. node: a server for machines, applied without a desktop or accounts, with the plugin features that install Incus or Docker and register their provisioners. managed: berthd and box alone, for sessions, actions and files on a machine that stays as it is. ssh: nothing is installed; berth logs in over SSH. The type can change later: install the controller, or set workstation and apply.", ), UsbDevice => ( "Then write to USB drive", "/builder/usb_device", Text, - "Enter switches between not writing and each connected USB drive. The chosen drive is erased and gets the finished ISO; that asks for confirmation and your sudo password. You can also press w after the build instead.", + "The USB drive the finished ISO is written to, such as /dev/sdb, on the machine berth serve runs on. It is erased. The host's overview asks for it again, with your sudo password, when the installer is built.", ), }; Spec { @@ -604,7 +732,6 @@ impl Field { | ToolsChrome => Group::Heading("Development Tools"), DiskSerial | DiskModel | DiskPath | DiskType => Group::Under(DiskMode), WifiInterface | WifiSsid | WifiPassword => Group::Under(Wifi), - TailscaleKey | TailscaleAuto | TailscaleLoginServer => Group::Under(Tailscale), ReleaseUrl | ReleasePrivate => Group::Under(ReleaseStore), IsoSha | UsbDevice => Group::Under(Iso), SshUser | SshPort | Login => Group::Under(SshHost), @@ -626,11 +753,8 @@ impl Field { matches!(self.spec().kind, Kind::Text | Kind::Secret | Kind::Path) } - /// Fields with a Generate (F4) action. + /// Fields with a Generate action. pub fn can_generate(self) -> bool { - matches!( - self, - Field::Hostname | Field::Iso | Field::IsoSha | Field::TailscaleKey - ) || self.is_password_hash() + matches!(self, Field::Hostname) || self.is_password_hash() } } diff --git a/crates/berth-tui/src/flow.rs b/crates/berth-tui/src/flow.rs new file mode 100644 index 0000000..3827232 --- /dev/null +++ b/crates/berth-tui/src/flow.rs @@ -0,0 +1,1188 @@ +//! The set-up of a new host as questions. It walks the tabs of the screens +//! in their order, Install, Features, Settings and Run, with the same rows: +//! what [`Profile::fields`] shows on each, written with [`setup::commit`] +//! and [`setup::set_feature`], and stopped by the same [`Problem`]s. What +//! the command line was given is taken first; whoever runs the flow is asked +//! for the rest, one [`Question`] at a time, or names the flag that gives it. + +use crate::fields::{Action, Field, Kind, Step}; +use crate::machine::Machine; +use crate::setup::{self, Feature, Problem, Profile}; +use serde_json::{Value, json}; + +/// Something to choose from. +#[derive(Clone, Debug, PartialEq)] +pub struct Choice { + /// What it is called on the command line. + pub key: String, + pub label: String, + pub help: String, +} + +impl Choice { + fn new(key: &str, label: &str, help: &str) -> Choice { + Choice { + key: key.to_owned(), + label: label.to_owned(), + help: help.to_owned(), + } + } +} + +#[derive(Clone, Debug, PartialEq)] +pub enum Asked { + Text { + default: String, + secret: bool, + }, + Choice { + options: Vec, + default: usize, + }, + Many { + options: Vec, + chosen: Vec, + }, + Confirm { + default: bool, + }, +} + +#[derive(Clone, Debug, PartialEq)] +pub enum Answer { + Text(String), + Choice(usize), + Many(Vec), + Confirm(bool), +} + +/// One thing the flow wants to know. +#[derive(Clone, Debug, PartialEq)] +pub struct Question { + /// The tab it belongs to, as the screens show it. + pub step: Step, + pub label: String, + pub help: String, + /// How to give it without being asked, such as `--set hostname=NAME`. + pub flag: String, + pub asked: Asked, + /// Whether nothing can go on without it. A question that is not + /// required keeps its default when nobody can be asked. + pub required: bool, +} + +/// Whoever answers: a person at a terminal, a list of answers in a test, +/// or nobody. +pub trait Asker { + /// The answer to `question`, or `None` when nobody can be asked. + fn ask(&mut self, question: &Question) -> Result, String>; +} + +/// An asker for when nobody is there: every question keeps its default. +pub struct Nobody; + +impl Asker for Nobody { + fn ask(&mut self, _: &Question) -> Result, String> { + Ok(None) + } +} + +/// What the server has for the set-up of a new host. +#[derive(Clone, Debug, Default)] +pub struct Catalogue { + /// What a new host starts from. + pub template: Value, + pub features: Vec, + /// The names of the secrets the server keeps. + pub secrets: Vec, + /// The server's own SSH public key. + pub own_key: String, + /// The provisioners, for a host one of them makes. + pub provisioners: Vec, +} + +/// What the command line gave, before anyone is asked. +#[derive(Clone, Debug, Default)] +pub struct Given { + pub method: Option, + /// Fields by their keys, in the order given. + pub values: Vec<(Field, String)>, + /// Features switched on or off, by `PLUGIN/FEATURE`. + pub features: Vec<(String, bool)>, + /// Secrets and registries the host is given, as `secret.NAME` and + /// `registry.NAME` say. + pub secrets: Vec<(String, String)>, + pub registries: Vec<(String, bool)>, + pub generate_passwords: bool, + pub provisioner: Option, + /// The provisioner's arguments. + pub arguments: Vec<(String, String)>, +} + +/// What a word `KEY=VALUE` of the command line sets. +#[derive(Clone, Debug, PartialEq)] +pub enum Setting { + Field(Field, String), + Feature(String, bool), + Secret(String, String), + Registry(String, bool), +} + +/// The keys `--set` and `berth host set` take, with what each is. +pub fn keys() -> Vec<(String, &'static str, &'static str)> { + Field::ALL + .iter() + .map(|field| { + let spec = field.spec(); + (field.key(), spec.label, spec.help) + }) + .collect() +} + +fn switch(value: &str) -> Option { + match value { + "on" | "true" | "yes" => Some(true), + "off" | "false" | "no" => Some(false), + _ => None, + } +} + +/// Read one `KEY=VALUE`. +pub fn setting(word: &str) -> Result { + let (key, value) = word + .split_once('=') + .ok_or_else(|| format!("{word} is no KEY=VALUE"))?; + let on = || switch(value).ok_or_else(|| format!("{key} is on or off, not {value}")); + if let Some(feature) = key.strip_prefix("feature.") { + return Ok(Setting::Feature(feature.to_owned(), on()?)); + } + if let Some(secret) = key.strip_prefix("secret.") { + return Ok(Setting::Secret(secret.to_owned(), value.to_owned())); + } + if let Some(registry) = key.strip_prefix("registry.") { + return Ok(Setting::Registry(registry.to_owned(), on()?)); + } + let field = Field::by_key(key).ok_or_else(|| { + format!("{key} is no field of a profile. berth host create --help lists them") + })?; + // A switch is given as on or off, as the screens show it. + let value = match field.spec().kind { + Kind::Toggle => on()?.to_string(), + _ => value.to_owned(), + }; + Ok(Setting::Field(field, value)) +} + +impl Given { + /// Take in `KEY=VALUE` words. + pub fn set(&mut self, words: &[String]) -> Result<(), String> { + for word in words { + match setting(word)? { + Setting::Field(field, value) => self.values.push((field, value)), + Setting::Feature(key, on) => self.features.push((key, on)), + Setting::Secret(name, how) => self.secrets.push((name, how)), + Setting::Registry(name, on) => self.registries.push((name, on)), + } + } + Ok(()) + } + + fn has(&self, field: Field) -> bool { + self.values.iter().any(|(given, _)| *given == field) + } +} + +/// Write `value` into a field of the profile, as the screens' row does. +pub fn write(config: &mut Value, field: Field, value: &str) -> Result<(), String> { + match field.spec().kind { + Kind::Toggle => { + let on = switch(value).ok_or_else(|| format!("{} is on or off", field.key()))?; + setup::set_pointer(config, field.spec().pointer, json!(on)); + Ok(()) + } + _ => setup::commit(config, field, value), + } +} + +/// Apply one setting to a profile, with the features there are. +pub fn apply(config: &mut Value, features: &[Feature], setting: &Setting) -> Result<(), String> { + match setting { + Setting::Field(field, value) => write(config, *field, value), + Setting::Feature(key, on) => { + let index = features + .iter() + .position(|feature| &feature.key == key) + .ok_or_else(|| format!("No plugin has the feature {key}"))?; + let host_type = berth_api::profile::host_type(config); + let fits = match host_type { + berth_api::HostType::Node => features[index].fits(host_type), + _ => features[index].fits(berth_api::HostType::Workstation), + }; + if *on && !fits { + return Err(format!( + "The feature {key} is not for a host of the type {host_type}" + )); + } + setup::set_feature(config, features, index, *on); + Ok(()) + } + Setting::Secret(name, how) => { + // A file may say its mode after it, as PATH:0640. + let (how, mode) = match how.rsplit_once(':') { + Some((path, mode)) + if (3..=4).contains(&mode.len()) + && mode.chars().all(|ch| ('0'..='7').contains(&ch)) => + { + (path, Some(mode)) + } + _ => (how.as_str(), None), + }; + setup::set_secret(config, name, how).map_err(|why| format!("secret.{name}: {why}"))?; + if let Some(mode) = mode + && let Some(entry) = config["secrets"] + .as_array_mut() + .and_then(|secrets| { + secrets + .iter_mut() + .find(|entry| entry["name"] == name.as_str()) + }) + .filter(|entry| entry["file"].is_string()) + { + entry["mode"] = json!(mode); + } + Ok(()) + } + Setting::Registry(name, on) => { + let has = config["registries"] + .as_array() + .is_some_and(|names| names.iter().any(|known| known == name.as_str())); + if has != *on { + setup::toggle_registry(config, name); + } + Ok(()) + } + } +} + +/// The flag that gives a field. +pub fn flag_of(field: Field) -> String { + let value = match field.spec().kind { + Kind::Toggle => "on|off".to_owned(), + Kind::Choice(choices) => choices.join("|"), + _ => field.spec().label.to_uppercase().replace(' ', "_"), + }; + format!("--set {}={value}", field.key()) +} + +/// What makes a password hash: a password made up, or one typed, and the +/// account it is for. +pub enum Hash<'a> { + Generate { user: &'a str }, + Of(&'a str), +} + +/// The server's part of the flow: what only it can make. +pub trait Maker { + /// A password hash for an account of the host. + fn hash(&mut self, host: &str, hash: Hash) -> Result; +} + +/// The set-up of one new host. +pub struct Flow { + pub name: String, + pub config: Value, + pub action: Action, + pub catalogue: Catalogue, + pub machine: Machine, +} + +/// Why the flow stopped. +#[derive(Debug, PartialEq)] +pub enum Stop { + /// A question that has to be answered, and nobody could be asked: its flag. + Missing(Box), + /// What still stops it, each on its tab. + Problems(Vec), + /// A value was refused, or the asker gave up. + Refused(String), +} + +/// The ways offered for a host of `host_type`, as the Install tab offers +/// them; with the type given, only the ways that keep it. +fn methods_for(host_type: berth_api::HostType, given: bool) -> Vec { + Action::SERVED + .into_iter() + .filter(|action| match given { + true => action.types().contains(&host_type), + false => action.offered(host_type), + }) + .collect() +} + +impl Flow { + /// A new host from the server's starting profile, as the screens begin one. + pub fn new(name: &str, catalogue: Catalogue) -> Flow { + let mut config = catalogue.template.clone(); + if !config.is_object() { + config = json!({}); + } + let mut machine = Machine::default(); + machine.asking(); + machine.listed(Ok(catalogue.provisioners.clone())); + machine.found(Ok(None)); + let mut flow = Flow { + name: name.to_owned(), + config, + action: Action::Machine, + catalogue, + machine, + }; + setup::fit_target(&mut flow.config, flow.action); + setup::default_enrollment(&mut flow.config, &flow.catalogue.secrets.clone()); + flow + } + + /// The profile as the screens' tab `step` sees it. + pub fn view(&self, step: Step) -> Profile<'_> { + Profile { + config: &self.config, + step, + action: self.action, + features: &self.catalogue.features, + registries: &[], + secrets: &[], + own_key: &self.catalogue.own_key, + named: setup::valid_hostname(&self.name), + tailnet_key: self + .catalogue + .secrets + .iter() + .any(|name| name == berth_api::profile::TAILNET_API_KEY), + } + } + + /// Choose the way it is installed, as a row of the Install tab does. + pub fn choose(&mut self, action: Action) { + self.action = action.choose(&mut self.config); + setup::fit_target(&mut self.config, self.action); + } + + /// Everything that stops it, as the Run tab lists it. + pub fn problems(&self) -> Vec { + let mut problems = self.view(Step::Run).problems_with(&self.machine); + if self.action == Action::Machine { + problems.retain(|problem| { + // An argument given later is no problem of the flow. + problem.field.is_some() || problem.step != Step::Settings + }); + } + problems + } + + fn question(&self, step: Step, field: Field, required: bool) -> Question { + let spec = field.spec(); + let view = self.view(step); + let asked = match spec.kind { + Kind::Toggle => Asked::Confirm { + default: view.enabled(field), + }, + Kind::Choice(choices) => { + let current = view.text(field); + Asked::Choice { + options: choices + .iter() + .map(|choice| Choice::new(choice, choice, "")) + .collect(), + default: choices + .iter() + .position(|choice| *choice == current) + .unwrap_or(0), + } + } + Kind::Secret => Asked::Text { + default: String::new(), + secret: true, + }, + Kind::Text | Kind::Path => Asked::Text { + default: view.text(field).to_owned(), + secret: false, + }, + }; + Question { + step, + label: spec.label.to_owned(), + help: spec.help.to_owned(), + flag: flag_of(field), + asked, + required, + } + } + + /// Ask a question: the answer, or `None` when its default stays. + fn put(asker: &mut dyn Asker, question: Question) -> Result, Stop> { + match asker.ask(&question).map_err(Stop::Refused)? { + Some(answer) => Ok(Some(answer)), + None if question.required => Err(Stop::Missing(Box::new(question))), + None => Ok(None), + } + } + + /// Ask for a field and write the answer into the profile, asking again + /// while it is refused. + fn ask_field( + &mut self, + asker: &mut dyn Asker, + step: Step, + field: Field, + required: bool, + ) -> Result<(), Stop> { + for _ in 0..3 { + let question = self.question(step, field, required); + let options = match &question.asked { + Asked::Choice { options, .. } => options.clone(), + _ => vec![], + }; + let value = match Flow::put(asker, question)? { + None => return Ok(()), + Some(Answer::Text(text)) => text, + Some(Answer::Confirm(on)) => if on { "on" } else { "off" }.to_owned(), + Some(Answer::Choice(index)) => options + .get(index) + .map(|choice| choice.key.clone()) + .unwrap_or_default(), + Some(Answer::Many(_)) => return Err(Stop::Refused("Not a list".into())), + }; + match write(&mut self.config, field, &value) { + Ok(()) => return Ok(()), + Err(why) if required => { + let _ = why; + } + Err(why) => return Err(Stop::Refused(why)), + } + } + Err(Stop::Refused(format!( + "{} was not taken", + field.spec().label + ))) + } + + /// Run the flow: take what is given, then ask the tabs' questions in + /// their order, and what is still a problem at the end. + pub fn run( + &mut self, + given: &Given, + asker: &mut dyn Asker, + maker: &mut dyn Maker, + ) -> Result<(), Stop> { + // Install: the way first, which decides what the tabs hold. A type + // given narrows the ways to those that can install it. + let typed = given + .values + .iter() + .find(|(field, _)| *field == Field::Type) + .and_then(|(_, value)| berth_api::HostType::parse(value)); + let host_type = typed.unwrap_or(berth_api::profile::host_type(&self.config)); + let methods = methods_for(host_type, typed.is_some()); + let method = match (given.method, &given.provisioner) { + (Some(method), _) => Some(method), + (None, Some(_)) => Some(Action::Machine), + (None, None) if methods.len() == 1 => Some(methods[0]), + (None, None) if given.has(Field::SshHost) => Some(Action::Ssh), + _ => None, + }; + match method { + Some(method) => self.choose(method), + None => { + let question = Question { + step: Step::Install, + label: "How it is installed".into(), + help: "The ways the server installs a host.".into(), + flag: "--method METHOD".into(), + asked: Asked::Choice { + options: methods + .iter() + .map(|action| Choice::new(action.key(), action.label(), action.help())) + .collect(), + default: 0, + }, + required: true, + }; + let chosen = match Flow::put(asker, question)? { + Some(Answer::Choice(index)) => methods.get(index).copied(), + _ => None, + }; + self.choose(chosen.unwrap_or(methods[0])); + } + } + // What the command line says comes before anything is asked. + for (field, value) in &given.values { + write(&mut self.config, *field, value) + .map_err(|why| Stop::Refused(format!("{}: {why}", field.key())))?; + } + for (key, on) in &given.features { + apply( + &mut self.config, + &self.catalogue.features.clone(), + &Setting::Feature(key.clone(), *on), + ) + .map_err(Stop::Refused)?; + } + for (name, how) in &given.secrets { + setup::set_secret(&mut self.config, name, how) + .map_err(|why| Stop::Refused(format!("secret.{name}: {why}")))?; + } + for (name, on) in &given.registries { + apply(&mut self.config, &[], &Setting::Registry(name.clone(), *on)) + .map_err(Stop::Refused)?; + } + if self.action == Action::Machine { + self.machine_questions(given, asker)?; + } + for step in [Step::Install, Step::Settings] { + for field in self.view(step).fields() { + if field.asked() && !given.has(field) { + self.ask_field(asker, step, field, false)?; + } + } + if step == Step::Install && given.features.is_empty() { + self.feature_question(asker)?; + } + } + self.passwords(given, asker, maker)?; + // What is still in the way is asked for, field by field. + for _ in 0..3 { + let open: Vec = self + .problems() + .into_iter() + .filter(|problem| problem.field.is_some_and(|field| !given.has(field))) + .collect(); + let Some(problem) = open.first() else { + break; + }; + let field = problem.field.expect("only problems of fields are left"); + match field { + _ if field.is_password_hash() => self.password(field, true, asker, maker)?, + // An account's keys are added one by one, with berth host set. + Field::OperatorKey | Field::AgentKey => break, + _ => self.ask_field(asker, problem.step, field, true)?, + } + if self.problems().first() == Some(problem) { + break; + } + } + let problems = self.problems(); + match problems.is_empty() { + true => Ok(()), + false => Err(Stop::Problems(problems)), + } + } + + /// The provisioner and the arguments it requires, as the Install tab + /// asks for them when a provisioner makes the host. + pub fn machine_questions(&mut self, given: &Given, asker: &mut dyn Asker) -> Result<(), Stop> { + match &given.provisioner { + Some(name) => { + if !self + .catalogue + .provisioners + .iter() + .any(|known| &known.id == name) + { + return Err(Stop::Refused(format!( + "No provisioner {name}. berth provisioner list names them" + ))); + } + self.machine.choose_id(name); + } + None if self.catalogue.provisioners.len() > 1 => { + let options = self + .machine + .choices() + .into_iter() + .map(|(label, id)| Choice::new(&id, &label, "")) + .collect(); + let question = Question { + step: Step::Install, + label: "Provisioner".into(), + help: crate::fields::Action::Machine.help().into(), + flag: "--provisioner NAME".into(), + asked: Asked::Choice { + options, + default: 0, + }, + required: true, + }; + if let Some(Answer::Choice(index)) = Flow::put(asker, question)? { + self.machine.choose(index); + } + } + None => {} + } + for (name, value) in &given.arguments { + let known = self + .machine + .provisioner() + .is_some_and(|chosen| chosen.arguments.iter().any(|known| &known.name == name)); + if !known { + return Err(Stop::Refused(format!( + "The provisioner takes no argument {name}" + ))); + } + self.machine.values.insert(name.clone(), value.clone()); + } + let arguments = self + .machine + .provisioner() + .map(|chosen| chosen.arguments.clone()) + .unwrap_or_default(); + for argument in arguments { + let value = self + .machine + .values + .get(&argument.name) + .cloned() + .unwrap_or_default(); + if !argument.required || !value.trim().is_empty() { + continue; + } + let label = match argument.label.as_str() { + "" => argument.name.clone(), + label => label.to_owned(), + }; + let asked = match argument.values.is_empty() { + true => Asked::Text { + default: argument.default.clone(), + secret: argument.secret, + }, + false => Asked::Choice { + options: argument + .values + .iter() + .map(|value| Choice::new(value, value, "")) + .collect(), + default: 0, + }, + }; + let question = Question { + step: Step::Install, + label, + help: argument.help.clone(), + flag: format!("{}=VALUE", argument.name), + asked, + required: true, + }; + let value = match Flow::put(asker, question)? { + Some(Answer::Text(text)) => text, + Some(Answer::Choice(index)) => { + argument.values.get(index).cloned().unwrap_or_default() + } + _ => continue, + }; + self.machine.values.insert(argument.name.clone(), value); + } + Ok(()) + } + + /// The Features tab as one list to tick: the plugins' features and + /// berth's own things to install. + fn feature_question(&mut self, asker: &mut dyn Asker) -> Result<(), Stop> { + let view = self.view(Step::Features); + let fields = view.fields(); + let chosen_features = setup::chosen_features(&self.config); + let offered = view.offered_features(); + let mut options = vec![]; + let mut chosen = vec![]; + for index in &offered { + let feature = &self.catalogue.features[*index]; + options.push(Choice::new(&feature.key, &feature.label, &feature.help)); + chosen.push(chosen_features.contains(&feature.key)); + } + for field in &fields { + let spec = field.spec(); + options.push(Choice::new(&field.key(), spec.label, spec.help)); + chosen.push(view.enabled(*field)); + } + if options.is_empty() { + return Ok(()); + } + let question = Question { + step: Step::Features, + label: "Features".into(), + help: setup::GROUP_HELP.into(), + flag: "--feature PLUGIN/FEATURE".into(), + asked: Asked::Many { + options, + chosen: chosen.clone(), + }, + required: false, + }; + let Some(Answer::Many(ticked)) = Flow::put(asker, question)? else { + return Ok(()); + }; + let features = self.catalogue.features.clone(); + for (index, on) in ticked.iter().enumerate() { + if chosen.get(index) == Some(on) { + continue; + } + if let Some(feature) = offered.get(index) { + setup::set_feature(&mut self.config, &features, *feature, *on); + } else if let Some(field) = fields.get(index - offered.len()) { + setup::set_pointer(&mut self.config, field.spec().pointer, json!(on)); + } + } + Ok(()) + } + + /// The accounts' passwords: generated, typed, or left empty. + fn passwords( + &mut self, + given: &Given, + asker: &mut dyn Asker, + maker: &mut dyn Maker, + ) -> Result<(), Stop> { + let fields: Vec = self + .view(Step::Settings) + .fields() + .into_iter() + .filter(|field| field.is_password_hash() && !given.has(*field)) + .filter(|field| self.view(Step::Settings).text(*field).is_empty()) + .collect(); + if fields.is_empty() { + return Ok(()); + } + if given.generate_passwords { + for field in fields { + self.generate(field, maker)?; + } + return Ok(()); + } + let question = Question { + step: Step::Settings, + label: "Passwords".into(), + help: Field::OperatorHash.spec().help.into(), + flag: "--generate-passwords".into(), + asked: Asked::Choice { + options: vec![ + Choice::new( + "generate", + "generate them", + "berth host credentials NAME shows them later.", + ), + Choice::new("enter", "enter them", "Typed here, and kept as hashes."), + Choice::new("none", "none", "An empty password locks password login."), + ], + default: 0, + }, + required: false, + }; + match Flow::put(asker, question)? { + Some(Answer::Choice(0)) => { + for field in fields { + self.generate(field, maker)?; + } + } + Some(Answer::Choice(1)) => { + for field in fields { + self.password(field, false, asker, maker)?; + } + } + _ => {} + } + Ok(()) + } + + fn user_of(&self, field: Field) -> String { + let user = match field { + Field::OperatorHash => Field::OperatorUser, + _ => Field::AgentUser, + }; + self.view(Step::Settings).text(user).to_owned() + } + + fn generate(&mut self, field: Field, maker: &mut dyn Maker) -> Result<(), Stop> { + let user = self.user_of(field); + let hash = maker + .hash(&self.name, Hash::Generate { user: &user }) + .map_err(Stop::Refused)?; + setup::set_pointer(&mut self.config, field.spec().pointer, json!(hash)); + Ok(()) + } + + /// A typed password for one account, hashed by the server. + fn password( + &mut self, + field: Field, + required: bool, + asker: &mut dyn Asker, + maker: &mut dyn Maker, + ) -> Result<(), Stop> { + let user = self.user_of(field); + let question = Question { + step: Step::Settings, + label: format!("Password of {user}"), + help: field.spec().help.into(), + flag: "--generate-passwords".into(), + asked: Asked::Text { + default: String::new(), + secret: true, + }, + required, + }; + if let Some(Answer::Text(password)) = Flow::put(asker, question)? + && !password.is_empty() + { + let hash = maker + .hash(&self.name, Hash::Of(&password)) + .map_err(Stop::Refused)?; + setup::set_pointer(&mut self.config, field.spec().pointer, json!(hash)); + } + Ok(()) + } + + /// What creating the host with its provisioner asks the server for. + pub fn request(&self) -> Result { + let mut request = self.machine.request()?; + request.host = Some(self.name.clone()); + Ok(request) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + /// Answers from a list, in the order the questions come, and the + /// questions as they were asked. + struct Script { + answers: Vec, + asked: Vec, + } + + impl Asker for Script { + fn ask(&mut self, question: &Question) -> Result, String> { + self.asked.push(question.label.clone()); + Ok(match self.answers.is_empty() { + true => None, + false => Some(self.answers.remove(0)), + }) + } + } + + /// Hashes that say what they were made of. + struct Hashes; + + impl Maker for Hashes { + fn hash(&mut self, host: &str, hash: Hash) -> Result { + Ok(match hash { + Hash::Generate { user } => { + format!("$6$made${host}{user}{}", "x".repeat(40)) + } + Hash::Of(password) => format!("$6$typed${password}{}", "x".repeat(40)), + }) + } + } + + fn catalogue() -> Catalogue { + Catalogue { + template: json!({ + "target": "laptop", + "operator_username": "ops", + "agent_username": "agent", + "builder": {}, + "ansible_vars": {} + }), + features: vec![ + Feature { + key: "claude/cli".into(), + label: "Claude Code".into(), + group: "Agents".into(), + help: String::new(), + types: vec![], + }, + Feature { + key: "codex/cli".into(), + label: "Codex".into(), + group: "Agents".into(), + help: String::new(), + types: vec![], + }, + ], + secrets: vec!["tailnet_api_key".into()], + own_key: "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOwn berth".into(), + provisioners: vec![], + } + } + + /// The same host set up as the screens do it: a way chosen on the + /// Install tab, features ticked, rows of the Settings tab written. + fn on_the_screens() -> Value { + let catalogue = catalogue(); + let mut config = catalogue.template.clone(); + let mut action = Action::Machine; + setup::fit_target(&mut config, action); + action = Action::Iso.choose(&mut config); + setup::fit_target(&mut config, action); + // A new host starts as one a provisioner makes, which is a cloud + // host; the Install tab's Target is chosen again for the installer. + setup::commit(&mut config, Field::Target, "laptop").unwrap(); + setup::set_feature(&mut config, &catalogue.features, 0, true); + setup::commit(&mut config, Field::Hostname, "studio").unwrap(); + setup::commit(&mut config, Field::OperatorUser, "olga").unwrap(); + setup::set_pointer( + &mut config, + Field::OperatorHash.spec().pointer, + json!(format!("$6$made$studioolga{}", "x".repeat(40))), + ); + setup::set_pointer( + &mut config, + Field::AgentHash.spec().pointer, + json!(format!("$6$made$studioagent{}", "x".repeat(40))), + ); + setup::commit(&mut config, Field::DiskMode, "largest").unwrap(); + config + } + + #[test] + fn answers_flags_and_the_screens_make_the_same_profile() { + // With flags, nothing is asked. + let mut given = Given { + method: Action::by_key("iso"), + generate_passwords: true, + ..Given::default() + }; + given + .set(&[ + "target=laptop".into(), + "hostname=studio".into(), + "operator_username=olga".into(), + "feature.claude/cli=on".into(), + "target_disk_mode=largest".into(), + ]) + .unwrap(); + let mut flagged = Flow::new("studio", catalogue()); + let mut nobody = Script { + answers: vec![], + asked: vec![], + }; + flagged.run(&given, &mut nobody, &mut Hashes).unwrap(); + + // With a terminal, the same answers to the questions in their order. + let mut asked = Flow::new("studio", catalogue()); + let mut script = Script { + answers: vec![ + // How it is installed: the USB installer. + Answer::Choice(1), + // Target: a laptop. + Answer::Choice(0), + // Features: Claude Code, and berth's own as they are. + Answer::Many({ + let view = asked.view(Step::Features); + let mut ticked = vec![true, false]; + ticked.extend(view.fields().iter().map(|field| view.enabled(*field))); + ticked + }), + // The operator and the agent. + Answer::Text("olga".into()), + Answer::Text("agent".into()), + // Passwords: generated. + Answer::Choice(0), + ], + asked: vec![], + }; + let given = Given { + values: vec![ + (Field::Hostname, "studio".into()), + (Field::DiskMode, "largest".into()), + ], + ..Given::default() + }; + asked.run(&given, &mut script, &mut Hashes).unwrap(); + assert_eq!( + script.asked, + [ + "How it is installed", + "Target", + "Features", + "Operator username", + "Agent username", + "Passwords" + ] + ); + assert!(script.answers.is_empty()); + + let screens = on_the_screens(); + assert_eq!(flagged.config, screens); + assert_eq!(asked.config, screens); + assert_eq!((flagged.action, asked.action), (Action::Iso, Action::Iso)); + } + + #[test] + fn without_a_terminal_what_is_missing_names_its_flag() { + let mut flow = Flow::new("studio", catalogue()); + let stopped = flow.run(&Given::default(), &mut Nobody, &mut Hashes); + match stopped { + Err(Stop::Missing(question)) => assert_eq!(question.flag, "--method METHOD"), + other => panic!("{other:?}"), + } + let mut flow = Flow::new("studio", catalogue()); + let given = Given { + method: Some(Action::Iso), + ..Given::default() + }; + // The operator's password is required for an installer. + match flow.run(&given, &mut Nobody, &mut Hashes) { + Err(Stop::Missing(question)) => { + assert_eq!(question.flag, "--generate-passwords"); + } + other => panic!("{other:?}"), + } + } + + #[test] + fn a_node_is_applied_over_ssh_with_a_servers_features_and_no_accounts() { + let mut catalogue = catalogue(); + catalogue.features.push(Feature { + key: "incus/server".into(), + label: "Incus".into(), + group: "Servers".into(), + help: String::new(), + types: vec![berth_api::HostType::Node], + }); + let mut given = Given::default(); + given + .set(&[ + "type=node".into(), + "address=192.0.2.20".into(), + "user=root".into(), + ]) + .unwrap(); + let mut flow = Flow::new("big", catalogue.clone()); + let mut script = Script { + answers: vec![Answer::Many(vec![true])], + asked: vec![], + }; + flow.run(&given, &mut script, &mut Hashes).unwrap(); + // The way is SSH, and of the features only the server's is offered. + assert_eq!(flow.action, Action::Ssh); + assert!( + script + .asked + .iter() + .all(|asked| asked != "How it is installed") + ); + assert!( + script.asked.contains(&"Features".to_owned()), + "{:?}", + script.asked + ); + assert_eq!( + flow.config["ansible_vars"]["devbox_plugin_features"], + json!(["incus/server"]) + ); + assert_eq!(flow.config["type"], "node"); + assert!(flow.config.get("operator_username").is_none()); + assert!(flow.config.get("agent_username").is_none()); + // A workstation's feature is refused for a node. + let mut given = given.clone(); + given.features.push(("claude/cli".into(), true)); + let mut flow = Flow::new("big", catalogue); + match flow.run(&given, &mut Nobody, &mut Hashes) { + Err(Stop::Refused(why)) => { + assert!(why.contains("not for a host of the type node"), "{why}") + } + other => panic!("{other:?}"), + } + } + + #[test] + fn the_ways_offered_follow_the_type() { + use berth_api::HostType::*; + let keys = |host_type, given| -> Vec<&str> { + methods_for(host_type, given) + .into_iter() + .map(Action::key) + .collect() + }; + assert_eq!( + keys(Workstation, false), + ["provisioner", "iso", "cloud-seed", "ssh", "ssh-only"] + ); + for host_type in [Node, Managed, Ssh] { + assert_eq!(keys(host_type, false), ["ssh", "ssh-only"], "{host_type}"); + } + // A type given keeps only the ways that install it. + assert_eq!(keys(Node, true), ["ssh"]); + assert_eq!(keys(Managed, true), ["ssh"]); + assert_eq!(keys(Ssh, true), ["ssh-only"]); + // A way that installs a whole workstation makes the host one. + let mut config = json!({"type": "node"}); + assert_eq!(Action::Iso.choose(&mut config), Action::Iso); + assert_eq!(berth_api::profile::host_type(&config), Workstation); + } + + #[test] + fn a_node_cannot_be_installed_by_an_installer() { + let mut given = Given { + method: Action::by_key("iso"), + generate_passwords: true, + ..Given::default() + }; + given + .set(&["type=node".into(), "hostname=big".into()]) + .unwrap(); + let mut flow = Flow::new("big", catalogue()); + match flow.run(&given, &mut Nobody, &mut Hashes) { + Err(Stop::Problems(problems)) => assert!( + problems + .iter() + .any(|problem| problem.text.contains("installs a workstation")), + "{problems:?}" + ), + other => panic!("{other:?}"), + } + } + + #[test] + fn a_host_over_ssh_needs_its_address() { + let mut flow = Flow::new("dock", catalogue()); + let given = Given { + method: Action::by_key("ssh-only"), + ..Given::default() + }; + match flow.run(&given, &mut Nobody, &mut Hashes) { + Err(Stop::Missing(question)) => assert_eq!(question.flag, "--set address=SSH_ADDRESS"), + other => panic!("{other:?}"), + } + let mut given = given; + given.set(&["address=192.0.2.11".into()]).unwrap(); + let mut flow = Flow::new("dock", catalogue()); + flow.run(&given, &mut Nobody, &mut Hashes).unwrap(); + assert_eq!(flow.config["access"]["address"], "192.0.2.11"); + assert_eq!(flow.config["type"], "ssh"); + } + + #[test] + fn every_field_has_a_key_of_its_own() { + let keys: Vec = Field::ALL.iter().map(|field| field.key()).collect(); + let mut unique = keys.clone(); + unique.sort(); + unique.dedup(); + assert_eq!(unique.len(), keys.len(), "{keys:?}"); + for key in [ + "hostname", + "operator_username", + "address", + "user", + "wifi_ssid", + "iso", + "tools_web", + "type", + ] { + assert!(keys.iter().any(|known| known == key), "{key}"); + } + assert!( + matches!(setting("install_docker=off"), Ok(Setting::Field(Field::Docker, value)) if value == "false") + ); + assert!(setting("nothing=1").is_err()); + } +} diff --git a/crates/berth-tui/src/host.rs b/crates/berth-tui/src/host.rs index 23ac5a3..84af691 100644 --- a/crates/berth-tui/src/host.rs +++ b/crates/berth-tui/src/host.rs @@ -8,10 +8,10 @@ pub use crate::page::{Arguments, Launch, argument_label}; use berth_api as api; use serde_json::{Value, json}; use std::collections::BTreeMap; -use web_time::{Duration, Instant}; +use std::time::{Duration, Instant}; /// The lowest `api` of a host this berth can work with. -pub const NEEDS_API: u32 = 1; +pub use berth_api::compat::NEEDS_HOST_API as NEEDS_API; /// How often the overview asks for stats and sessions again. const POLL: Duration = Duration::from_secs(10); /// Lines of a process session's transcript kept for scrolling back. @@ -25,11 +25,11 @@ pub trait Transport { } pub enum Request { - /// Version, tasks, sessions and stats. + /// Version, actions, sessions and stats. Refresh, /// Sessions and stats. Poll, - /// Run a task; the answer is `Opened`. + /// Run an action; the answer is `Opened`. Run { name: String, body: Value, @@ -45,7 +45,7 @@ pub enum Request { Close, Delete(String), Choices { - task: String, + action: String, argument: String, }, } @@ -60,7 +60,7 @@ pub enum Event { message: String, missing: bool, }, - Tasks(Vec), + Actions(Vec), Sessions(Vec), Stats(Box), Opened(Box), @@ -70,7 +70,7 @@ pub enum Event { /// The stream of a session ended without it having exited. Closed(String), Choices { - task: String, + action: String, argument: String, choices: Vec, }, @@ -123,6 +123,16 @@ impl View { } } + /// The panel of a job of the server, which begins with what the job + /// was started with, such as the command of `exec`, as its record says. + pub fn of_job(title: &str, rows: u16, cols: u16, job: &Value) -> View { + let mut view = View::plain(title, rows, cols); + if let Some(line) = started_with(job) { + view.feed(format!("\x1b[2m{line}\x1b[0m\r\n").as_bytes()); + } + view + } + pub fn screen(&self) -> &vt100::Screen { self.parser.screen() } @@ -230,16 +240,16 @@ pub enum Pending { pub struct HostView { remote: Option>, pub state: State, - pub tasks: Vec, + pub host_actions: Vec, pub sessions: Vec, pub stats: Option, /// What changed since the host was applied, when berth knows. pub behind: Option, - /// The level the host's profile means it to have. - pub level: api::HostLevel, - /// The level the host was seen to have, once it was asked. - pub seen: Option, - /// berth's own actions on the host, beside its tasks: what a berth + /// The type the host's profile means it to be. + pub host_type: api::HostType, + /// The type the host was seen to be, once it was asked. + pub seen: Option, + /// berth's own actions on the host, beside its controller's: what a berth /// server lists for its host, or what this berth does itself. pub own: Vec, pub view: Option, @@ -259,11 +269,11 @@ impl Default for HostView { HostView { remote: None, state: State::Idle, - tasks: vec![], + host_actions: vec![], sessions: vec![], stats: None, behind: None, - level: api::HostLevel::Workstation, + host_type: api::HostType::Workstation, seen: None, own: vec![], view: None, @@ -300,7 +310,7 @@ impl HostView { pub fn disconnect(&mut self) { self.remote = None; self.view = None; - self.tasks.clear(); + self.host_actions.clear(); self.sessions.clear(); self.stats = None; self.behind = None; @@ -314,7 +324,7 @@ impl HostView { } } - /// Run a task; its session opens in the panel when the host has started it. + /// Run an action; its session opens in the panel when the host has started it. pub fn run(&mut self, name: &str, body: Value) { self.starting = Some(false); self.send(Request::Run { @@ -323,15 +333,15 @@ impl HostView { }); } - /// What can be done on the host: its tasks as actions, each task that + /// What can be done on the host: its controller's actions, each that /// is no step of another, with the steps plugins add to it, and then - /// berth's own that are no task of the host. + /// berth's own that are no action of the host. pub fn actions(&self) -> Vec { - let mut actions = api::Action::of_tasks(&self.tasks); + let mut actions = api::Action::of_host(&self.host_actions); let own = self .own .iter() - .filter(|action| !actions.iter().any(|task| task.name == action.name)) + .filter(|action| !actions.iter().any(|listed| listed.name == action.name)) .cloned() .collect::>(); actions.extend(own); @@ -343,12 +353,12 @@ impl HostView { pub fn is_own(&self, index: usize) -> bool { self.actions() .get(index) - .is_some_and(|action| self.task(action).is_none()) + .is_some_and(|action| self.host_action(action).is_none()) } /// Begin an action: its question first, then its arguments. Returns the /// action to run now when it needs neither. - pub fn begin_task(&mut self, index: usize, confirmed: bool) -> Option { + pub fn begin_action(&mut self, index: usize, confirmed: bool) -> Option { match page::begin(&self.actions(), index, confirmed) { Begin::Ask(question) => self.question = Some((question, Pending::Action(index))), Begin::Arguments(dialog) => { @@ -362,12 +372,12 @@ impl HostView { } fn ask_choices(&self) { - if let Some((task, argument)) = self + if let Some((action, argument)) = self .arguments .as_ref() .and_then(|dialog| dialog.wants_choices(&self.actions())) { - self.send(Request::Choices { task, argument }); + self.send(Request::Choices { action, argument }); } } @@ -391,8 +401,8 @@ impl HostView { } } - /// What starting a task sends: its arguments, and the panel's size for a terminal. - pub fn task_body(&self, action: &api::Action, values: &BTreeMap) -> Value { + /// What starting an action sends: its arguments, and the panel's size for a terminal. + pub fn action_body(&self, action: &api::Action, values: &BTreeMap) -> Value { let mut body = json!({"arguments": values}); if action.terminal { (body["rows"], body["cols"]) = (json!(self.size.0), json!(self.size.1)); @@ -400,9 +410,11 @@ impl HostView { body } - /// The task an action of the host runs, with what the host keeps of it. - pub fn task(&self, action: &api::Action) -> Option<&api::Task> { - self.tasks.iter().find(|task| task.name == action.name) + /// What the host keeps of one of its controller's actions. + pub fn host_action(&self, action: &api::Action) -> Option<&api::HostAction> { + self.host_actions + .iter() + .find(|kept| kept.name == action.name) } /// Start a shell as `user`. @@ -468,7 +480,7 @@ impl HostView { } Event::Version(version) => { // A controller without a record was installed outside berth. - self.seen = Some(version.level.unwrap_or(api::HostLevel::Controller)); + self.seen = Some(version.host_type.unwrap_or(api::HostType::Managed)); self.state = State::Ready(version.version); } Event::Failed { message, missing } => { @@ -480,7 +492,7 @@ impl HostView { self.state = State::Password; note = Some("sudo on the host did not take that password".to_owned()); } else if missing { - self.seen = Some(api::HostLevel::Reachable); + self.seen = Some(api::HostType::Ssh); self.state = State::Missing; } else if message.contains("a password is required") { self.state = State::Password; @@ -489,7 +501,7 @@ impl HostView { } } Event::Behind(behind) => self.behind = behind, - Event::Tasks(tasks) => self.tasks = tasks, + Event::Actions(listed) => self.host_actions = listed, Event::Sessions(sessions) => self.sessions = sessions, Event::Stats(stats) => self.stats = Some(*stats), Event::Opened(session) => { @@ -526,13 +538,13 @@ impl HostView { } } Event::Choices { - task, + action, argument, choices, } => { let actions = self.actions(); if let Some(dialog) = &mut self.arguments { - dialog.offered(&actions, &task, &argument, choices); + dialog.offered(&actions, &action, &argument, choices); } } Event::Error(message) => { @@ -610,20 +622,21 @@ pub fn session_state(session: &api::Session) -> String { #[derive(Clone, Copy, Debug, PartialEq, Eq)] pub enum Pick { /// An action, by index into [`HostView::actions`]. - Task(usize), + Action(usize), Terminal, /// A session, by index. Session(usize), } -/// The groups of the overview of a host that answers, as it lists its tasks: +/// The groups of the overview of a host that answers, as it lists its actions: /// each in the group it names, with the terminal among the first and the /// host's sessions after them. pub fn groups(host: &HostView, terminal_user: &str) -> Vec<(String, Vec<(String, Pick)>)> { - let mut groups: Vec<(String, Vec<(String, Pick)>)> = page::groups(&host.actions(), Pick::Task) - .into_iter() - .map(|group| (group.title, group.rows)) - .collect(); + let mut groups: Vec<(String, Vec<(String, Pick)>)> = + page::groups(&host.actions(), Pick::Action) + .into_iter() + .map(|group| (group.title, group.rows)) + .collect(); let terminal = (format!(" ▸ Terminal · as {terminal_user}"), Pick::Terminal); match groups.iter_mut().find(|(title, _)| title == "System") { Some((_, rows)) => rows.insert(1.min(rows.len()), terminal), @@ -646,13 +659,13 @@ pub fn groups(host: &HostView, terminal_user: &str) -> Vec<(String, Vec<(String, groups } -/// The line the overview shows of the host's level, with what the host +/// The line the overview shows of the host's type, with what the host /// itself was seen to be when that is different. -pub fn level_line(name: &str, host: &HostView) -> String { - let mut line = format!("Level · {} ({})", host.level, host.level.describe()); - if let Some(said) = api::level_mismatch(name, host.level, host.seen) { +pub fn type_line(name: &str, host: &HostView) -> String { + let mut line = format!("Type · {} ({})", host.host_type, host.host_type.describe()); + if let Some(said) = api::type_mismatch(name, host.host_type, host.seen) { line += &format!(". {said}"); - } else if let Some(seen) = host.seen.filter(|seen| *seen != host.level) { + } else if let Some(seen) = host.seen.filter(|seen| *seen != host.host_type) { line += &format!(". The host itself is {seen} now"); } line @@ -662,9 +675,9 @@ pub fn level_line(name: &str, host: &HostView) -> String { /// the page beside the information box. pub const NOTE_WIDTH: usize = 64; -/// [`level_line`], wrapped to the overview's width. -pub fn level_lines(name: &str, host: &HostView) -> Vec { - crate::screen::wrap(&level_line(name, host), NOTE_WIDTH) +/// [`type_line`], wrapped to the overview's width. +pub fn type_lines(name: &str, host: &HostView) -> Vec { + crate::screen::wrap(&type_line(name, host), NOTE_WIDTH) } /// What the overview says of a host whose configuration is behind. @@ -692,6 +705,42 @@ pub fn list(value: Value, key: &str) -> Vec { .collect() } +/// What a job was started with, as a line for people: `$ COMMAND` for a +/// resource's `exec`, else its arguments; `None` for a job with none. +pub fn started_with(job: &Value) -> Option { + let words = job_arguments(job)?; + Some(match job["action"].as_str() { + Some("exec") => format!("$ {words}"), + _ => format!("with {words}"), + }) +} + +/// A job's arguments as words a shell reads back; `None` when it has none. +pub fn job_arguments(job: &Value) -> Option { + let arguments: Vec = job["arguments"] + .as_array()? + .iter() + .filter_map(|word| word.as_str().map(str::to_owned)) + .collect(); + (!arguments.is_empty()).then(|| shell_words(&arguments)) +} + +/// Words as a shell reads them back: a word with a space or a character the +/// shell gives a meaning is quoted. +pub fn shell_words(words: &[String]) -> String { + let quote = |word: &String| { + let plain = !word.is_empty() + && word + .chars() + .all(|ch| ch.is_ascii_alphanumeric() || "-_./=:,+@%^".contains(ch)); + match plain { + true => word.clone(), + false => format!("'{}'", word.replace('\'', r"'\''")), + } + }; + words.iter().map(quote).collect::>().join(" ") +} + /// The exit code an `exited` frame of a session carries, when the frame is one. pub fn exited(frame: &Value) -> Option> { let exited = frame.get("exited")?; @@ -700,7 +749,7 @@ pub fn exited(frame: &Value) -> Option> { } /// What the information box says of an action of the host. -pub fn task_help(host: &HostView, index: usize) -> String { +pub fn action_help(host: &HostView, index: usize) -> String { match host.actions().get(index) { Some(action) => page::help(action, &command_of(&action.name)), None => String::new(), @@ -711,11 +760,11 @@ pub fn task_help(host: &HostView, index: usize) -> String { /// of their own, the others are run by name. pub fn command_of(action: &str) -> String { match action { - "install-controller" => "berth host install HOST".into(), + "install-controller" => "berth run HOST install-controller".into(), "trust-key" => "berth key trust HOST".into(), - "host-key" => "berth key host HOST [--accept]".into(), - "apply" => "berth apply HOST".into(), - "seed" => "berth seed HOST".into(), + "host-key" => "berth host verify HOST [--accept]".into(), + "apply" => "berth host apply HOST".into(), + "seed" => "berth host seed HOST".into(), "rename" => "berth host rename HOST NAME [--hostname]".into(), name => format!("berth run HOST {name}"), } @@ -732,7 +781,7 @@ pub fn session_help(host: &HostView, index: usize) -> String { (Some(ended), None) => format!("ended {ended}"), }; format!( - "`{}` as {}, started {}, {state}. It lives on the host and goes on while nobody watches.\n\nEnter opens it here. x forgets one that has ended, with its output. From the command line: `berth attach HOST {id}` or `berth output HOST {id}`.", + "`{}` as {}, started {}, {state}. It lives on the host and goes on while nobody watches.\n\nEnter opens it here. x forgets one that has ended, with its output. From the command line: `berth session attach HOST {id}` or `berth session logs HOST {id}`.", session.command.join(" "), session.user, session.started, @@ -745,13 +794,23 @@ mod tests { use super::*; #[test] - fn berths_own_actions_follow_the_tasks_and_say_when_they_cannot_run() { + fn words_are_quoted_as_a_shell_reads_them() { + let words = |list: &[&str]| list.iter().map(|word| word.to_string()).collect::>(); + assert_eq!(shell_words(&words(&["psql", "-U", "app"])), "psql -U app"); + assert_eq!( + shell_words(&words(&["sh", "-c", "echo 'a b'", ""])), + r"sh -c 'echo '\''a b'\''' ''" + ); + } + + #[test] + fn berths_own_actions_follow_the_controllers_and_say_when_they_cannot_run() { let mut host = HostView { - tasks: vec![api::Task { + host_actions: vec![api::HostAction { name: "status".into(), label: "Status".into(), group: "System".into(), - ..api::Task::default() + ..api::HostAction::default() }], ..HostView::default() }; @@ -771,42 +830,42 @@ mod tests { ]; let actions = host.actions(); let names: Vec<&str> = actions.iter().map(|action| action.name.as_str()).collect(); - // A task of the host wins over one of berth's of the same name. + // An action of the host wins over one of berth's of the same name. assert_eq!(names, ["status", "install-controller"]); assert!(!host.is_own(0) && host.is_own(1)); let groups = groups(&host, "agent"); let setup = groups.iter().find(|(title, _)| title == "Set-Up").unwrap(); assert_eq!(setup.1[0].0, " ▸ Update the controller · not now"); - assert!(task_help(&host, 1).contains("It cannot run now: studio is a workstation")); + assert!(action_help(&host, 1).contains("It cannot run now: studio is a workstation")); } #[test] - fn the_level_line_says_what_the_host_was_seen_to_be() { + fn the_type_line_says_what_the_host_was_seen_to_be() { let mut host = HostView { - level: api::HostLevel::Reachable, + host_type: api::HostType::Ssh, ..HostView::default() }; assert_eq!( - level_line("studio", &host), - "Level · reachable (SSH only, nothing of berth installed)" + type_line("studio", &host), + "Type · ssh (SSH only, nothing of berth installed)" ); - host.seen = Some(api::HostLevel::Reachable); - assert!(!level_line("studio", &host).contains("now")); + host.seen = Some(api::HostType::Ssh); + assert!(!type_line("studio", &host).contains("now")); // One seen to be more than its profile says names how to adopt it. - host.seen = Some(api::HostLevel::Workstation); + host.seen = Some(api::HostType::Workstation); assert!( - level_line("studio", &host) - .contains("studio is a workstation; its profile here says reachable. To adopt it"), + type_line("studio", &host) + .contains("studio is a workstation; its profile here says ssh. To adopt it"), "{}", - level_line("studio", &host) + type_line("studio", &host) ); - host.level = api::HostLevel::Workstation; - host.seen = Some(api::HostLevel::Controller); - assert!(level_line("studio", &host).ends_with("The host itself is controller now")); + host.host_type = api::HostType::Workstation; + host.seen = Some(api::HostType::Managed); + assert!(type_line("studio", &host).ends_with("The host itself is managed now")); // Wrapped, every line fits the overview beside the information box. - host.level = api::HostLevel::Reachable; - host.seen = Some(api::HostLevel::Workstation); - let lines = level_lines("studio", &host); + host.host_type = api::HostType::Ssh; + host.seen = Some(api::HostType::Workstation); + let lines = type_lines("studio", &host); assert!(lines.len() > 1); assert!( lines.iter().all(|line| line.chars().count() <= NOTE_WIDTH), diff --git a/crates/berth-tui/src/lib.rs b/crates/berth-tui/src/lib.rs index 0a78bc9..aa82061 100644 --- a/crates/berth-tui/src/lib.rs +++ b/crates/berth-tui/src/lib.rs @@ -1,10 +1,11 @@ -//! The screens of a host with no terminal in them: they draw into ratatui -//! frames and take keys of their own type, which berth turns a terminal's -//! events into and its page in a browser the browser's. +//! The parts of berth's screens with no terminal in them: they draw into +//! ratatui frames and take keys of their own type, which berth-page turns a +//! terminal's events into. pub mod berth_key; pub mod event; pub mod fields; +pub mod flow; pub mod form; pub mod host; pub mod links; diff --git a/crates/berth-tui/src/machine.rs b/crates/berth-tui/src/machine.rs index c07e193..fce6b09 100644 --- a/crates/berth-tui/src/machine.rs +++ b/crates/berth-tui/src/machine.rs @@ -4,8 +4,7 @@ //! rest of its arguments on the Settings tab, and creating it, or the machine //! once there is one, on the Run tab. The machine is the host's resource of //! the machine kind, and starting, stopping and deleting it are its actions. -//! berth and its page in a browser ask for these and run the jobs each their -//! own way. +//! The screens ask the server for these, which runs the jobs. use crate::fields::{Kind, Step}; use crate::setup::{Entry, Problem, Row, value_line}; @@ -90,7 +89,7 @@ fn label(argument: &berth_api::Argument) -> &str { /// What the editor is called while it holds what is typed for `pick`. pub fn editor_title(pick: Pick, host: &str) -> String { match pick { - Pick::Delete => format!(" Type {host} to delete its machine "), + Pick::Delete => format!(" Type {host} to destroy what its provisioner made "), _ => " Edit ".into(), } } @@ -442,7 +441,7 @@ impl Machine { } if let Some(instance) = self.machine() { return vec![note(&format!( - "{} created it · the Run tab has the machine", + "{} created it · the Run tab has it", instance.provisioner ))]; } @@ -468,7 +467,7 @@ impl Machine { let mut rows = vec![]; let resources = self.argument_rows(Step::Settings, |argument| argument.is_resource()); if !resources.is_empty() { - rows.push(Row::heading("Machine")); + rows.push(Row::heading("Size")); rows.extend(resources); } let others = self.argument_rows(Step::Settings, |argument| !argument.is_resource()); @@ -492,16 +491,16 @@ impl Machine { let Some(instance) = self.machine() else { let label = self .provisioner() - .map_or(String::new(), |known| format!(" · {}", known.label)); + .map_or(String::new(), |known| format!(" with {}", known.label)); return vec![Row::entry( - format!("[Create machine{label}]"), + format!("[Create{label}]"), Entry::Machine(Pick::Create), )]; }; - let mut rows = vec![Row::heading("Machine")]; + let mut rows = vec![Row::heading("Provisioner")]; let line = |label: &str, value: &str| Row::info(value_line(label, value, Kind::Text)); rows.push(line(" Created with", &instance.provisioner)); - rows.push(line(" Machine", &instance.external_id)); + rows.push(line(" Its name there", &instance.external_id)); rows.push(line(" Resource", &instance.id)); let colour = match instance.state.as_str() { "running" => theme::GOOD, @@ -524,9 +523,9 @@ impl Machine { } for pick in Machine::actions(instance) { let text = match pick { - Pick::Start => " ▸ Start machine", - Pick::Stop => " ▸ Stop machine", - _ => " ▸ Delete machine", + Pick::Start => " ▸ Start", + Pick::Stop => " ▸ Stop", + _ => " ▸ Destroy", }; rows.push(Row::entry(text, Entry::Machine(pick))); } @@ -587,10 +586,10 @@ impl Machine { false => text + " Space or Enter switches to the next value.", } } - Pick::Create => "Creates the machine and sets it up as this host, with the profile saved first. The output is shown as it runs. When setting it up fails, the machine stays: Delete machine removes it.".into(), - Pick::Start => "Starts the machine.".into(), - Pick::Stop => "Stops the machine.".into(), - Pick::Delete => "Deletes the machine berth created for this host. The profile stays, and so does its node on the tailnet. Enter, then type the host's name to say so.".into(), + Pick::Create => "Creates the machine and sets it up as this host, with the profile saved first. The output is shown as it runs. When setting it up fails, the machine stays: Destroy removes it.".into(), + Pick::Start => "Starts the host's machine.".into(), + Pick::Stop => "Stops the host's machine.".into(), + Pick::Delete => "Destroys the machine the provisioner made for this host. The profile stays, and so does its node on the tailnet. Enter, then type the host's name to say so. From the command line: berth host remove HOST --destroy, which forgets the host too.".into(), } } @@ -622,7 +621,7 @@ impl Machine { Pick::Create => match self.request() { Ok(_) => { let label = self.provisioner().map_or("", |known| known.label.as_str()); - self.question = Some(format!("Create a machine for {host} with {label}?")); + self.question = Some(format!("Create {host} with {label}?")); Next::Confirm } Err(problem) => Next::Say(problem), @@ -653,7 +652,7 @@ impl Machine { } Pick::Delete if value.trim() == host => Next::Destroy, Pick::Delete if value.trim().is_empty() => Next::Nothing, - Pick::Delete => Next::Say(format!("Not deleted: type {host} to delete its machine")), + Pick::Delete => Next::Say(format!("Not destroyed: type {host} to say so")), _ => Next::Nothing, } } @@ -830,10 +829,7 @@ mod tests { Next::Edit("4".into()) ); assert_eq!(picks(&machine.run_rows()), [Pick::Create]); - assert_eq!( - text(&machine.run_rows()), - ["[Create machine · Lab machine]"] - ); + assert_eq!(text(&machine.run_rows()), ["[Create with Lab machine]"]); } #[test] @@ -849,7 +845,7 @@ mod tests { assert_eq!( text(&machine.settings_rows()), [ - "Machine", + "Size", " ├─ cpus 1", " ├─ memory 1", " └─ disk 1", @@ -864,7 +860,7 @@ mod tests { let machine_rows = text(&machine.settings_rows()); assert_eq!( machine_rows[..3], - ["Machine", " ├─ cpus 1", " └─ memory 1"] + ["Size", " ├─ cpus 1", " └─ memory 1"] ); // The Suggestions panel lists them; the help says what the keys do. assert_eq!( @@ -938,7 +934,7 @@ mod tests { assert_eq!(machine.enter(Pick::Create, "studio"), Next::Confirm); assert_eq!( machine.question.as_deref(), - Some("Create a machine for studio with Lab machine?") + Some("Create studio with Lab machine?") ); let Next::Create(asked) = machine.answer(true) else { panic!("not created"); @@ -999,7 +995,7 @@ mod tests { assert!(machine.settings_rows().is_empty() && machine.problems().is_empty()); let rows = machine.run_rows(); let shown = text(&rows); - assert_eq!(shown[0], "Machine"); + assert_eq!(shown[0], "Provisioner"); assert!(shown.contains(&" State unknown".to_owned())); assert!(shown.contains(&" Address 192.0.2.20".to_owned())); assert!(shown.contains(&" ! lab/vm status said no state".to_owned())); diff --git a/crates/berth-tui/src/page.rs b/crates/berth-tui/src/page.rs index e1929a4..3beeec6 100644 --- a/crates/berth-tui/src/page.rs +++ b/crates/berth-tui/src/page.rs @@ -1,8 +1,8 @@ //! The page of a subject: lines that say how it is, what can be done to it //! in groups, and the dialogs before an action runs, its question and then //! its arguments one after another. A host's page is one. A page of another -//! subject is its state and its actions given to the same functions; the -//! terminal and the browser draw the rows each their own way. +//! subject is its state and its actions given to the same functions, which +//! the screens draw the rows of. use berth_api::{Action, Argument}; use std::collections::BTreeMap; @@ -338,11 +338,11 @@ mod tests { items.last(), Some(&Item::Pick(" ▸ Resize a pool · not now".into(), 2)) ); - let said = help(&actions[2], "berth provisioner incus/vm resize"); + let said = help(&actions[2], "berth provisioner run incus/vm resize"); assert!(said.starts_with("Grows the pool.") && said.contains("Asks for: pool, New size.")); assert!(said.contains("- **Check**: Says how full it is.")); assert!(said.contains("It cannot run now: The pool is busy")); - assert!(said.ends_with("`berth provisioner incus/vm resize`")); + assert!(said.ends_with("`berth provisioner run incus/vm resize`")); } #[test] diff --git a/crates/berth-tui/src/plugins.rs b/crates/berth-tui/src/plugins.rs index 72221ce..3b4667f 100644 --- a/crates/berth-tui/src/plugins.rs +++ b/crates/berth-tui/src/plugins.rs @@ -1,8 +1,7 @@ //! The Plugins tab of the home screen: the plugins there are with their //! features, installing, updating and removing one, and bringing the hosts -//! that are behind up to date afterwards. berth shows those of this -//! machine or of its berth server, the page in a browser those of the server -//! it came from. What the tab asks for, each of them does its own way. +//! that are behind up to date afterwards: those of the server the screens +//! use. What the tab asks for goes to the server. use crate::event::{self, Code, Key}; use crate::theme; @@ -13,54 +12,20 @@ use ratatui::{ }; use tui_input::Input; -/// Where the plugins are, for what the tab says about adding one. -pub enum Place<'a> { - /// This machine's. - Here, - /// The server's at this address, for berth in a terminal. - Server(&'a str), - /// The server's, for the page in a browser. - Page, -} - -impl Place<'_> { - /// Where a removed plugin is gone from. - fn name(&self) -> &'static str { - match self { - Place::Here => "this machine", - _ => "the server", - } - } +/// The title of the editor for what to install. +pub const EDITOR_TITLE: &str = " Package or address of a .tgz "; - /// The title of the editor for what to install. - pub fn editor_title(&self) -> &'static str { - match self { - Place::Page => " Package or address of a .tgz ", - _ => " Package, .tgz or folder ", - } - } - - /// What the row that adds a plugin means. `dir` is where the own ones are. - pub fn adding(&self, dir: &str) -> String { - match self { - Place::Here => format!( - "Install a plugin with npm: a package name from your registry, a .tgz from npm pack, or a folder. Registries and logins come from your npm configuration. Nothing of a plugin runs on this machine.\n\nA folder copied to {dir} works as well." - ), - Place::Server(url) => format!( - "The berth server at {url} installs it with npm: a package name from its registry, or the https:// address of a .tgz. A .tgz or a folder on this machine is packed and sent to the server.\n\n{}", - server_note(dir) - ), - Place::Page => format!( - "The server installs it with npm: a package name from its registry, or the https:// address of a .tgz. u sends a .tgz from this device instead.\n\n{}", - server_note(dir) - ), - } - } +/// What the row that adds a plugin means. `dir` is where the own ones are. +pub fn adding(dir: &str) -> String { + format!( + "The server installs it with npm: a package name from its registry, or the https:// address of a .tgz. u sends a .tgz from this device instead.\n\n{}", + server_note(dir) + ) } fn server_note(dir: &str) -> String { format!( - "Registries and logins come from the server's npm configuration. npm runs none of the plugin's scripts; its tasks run when a host is applied.\n\nA folder copied to {dir} on the server works as well." + "Registries and logins come from the server's npm configuration. npm runs none of the plugin's scripts; its Ansible tasks run when a host is applied.\n\nA folder copied to {dir} on the server works as well." ) } @@ -266,7 +231,6 @@ impl Tab { editor: &mut Option, plugins: &[PluginInfo], report: &mut Vec, - place: &Place, ) -> Asked { let count = rows(plugins, &[]).len(); let selected = self.plugin(plugins).cloned(); @@ -340,11 +304,7 @@ impl Tab { } Some(plugin) => { self.armed = Some(Armed::Remove); - Some(format!( - "x again removes {} from {}", - plugin.label, - place.name() - )) + Some(format!("x again removes {} from the server", plugin.label)) } None => None, }); @@ -455,10 +415,8 @@ mod tests { let mut tab = Tab::default(); let mut editor = None; let mut report = vec!["✓ npm install x".to_owned()]; - let place = Place::Server("https://berth.example"); - let mut press = |tab: &mut Tab, code| { - tab.key(Key::plain(code), &mut editor, &plugins, &mut report, &place) - }; + let mut press = + |tab: &mut Tab, code| tab.key(Key::plain(code), &mut editor, &plugins, &mut report); // The one that comes with berth stays; another goes after x twice. assert_eq!( press(&mut tab, Code::Char('x')), @@ -484,7 +442,7 @@ mod tests { Asked::Install("@acme/foo".into()) ); assert!(report.is_empty() && editor.is_none()); - assert!(help(None, &[], &place.adding("/srv/plugins")).contains("https://berth.example")); + assert!(help(None, &[], &adding("/srv/plugins")).contains("/srv/plugins on the server")); } #[test] @@ -496,9 +454,8 @@ mod tests { let mut tab = Tab::default(); let mut editor = None; let mut report = vec![]; - let place = Place::Here; let mut press = |tab: &mut Tab, report: &mut Vec, code| { - tab.key(Key::plain(code), &mut editor, &plugins, report, &place) + tab.key(Key::plain(code), &mut editor, &plugins, report) }; assert_eq!(press(&mut tab, &mut report, Code::Char('c')), Asked::Check); let said = tab.checked(vec![PluginUpdate { diff --git a/crates/berth-tui/src/provisioners.rs b/crates/berth-tui/src/provisioners.rs index 4001a86..ea5c4b8 100644 --- a/crates/berth-tui/src/provisioners.rs +++ b/crates/berth-tui/src/provisioners.rs @@ -2,9 +2,8 @@ //! with whether it can be used here, and the page of one, built with //! [`crate::page`]: how it is now, what it says of itself, what it holds by //! kind, and its actions. A resource berth made with it has a page of its own -//! with its state, its actions and its log. berth shows those of this machine or of its -//! berth server, the page in a browser those of the server it came from. What -//! the tab asks for, each of them does its own way, and tells the tab what +//! with its state, its actions and its log: those of the server the screens +//! use. What the tab asks for goes to the server, which tells the tab what //! came back. use crate::event::{Code, Event, Key}; @@ -114,7 +113,10 @@ pub struct Tab { } /// What a host's list offers to make besides hosts, as (kind, label). -pub const MAKES: [(&str, &str); 2] = [("instance", "instance"), ("deployment", "compose project")]; +pub const MAKES: [(&str, &str); 2] = [ + ("instance", "resource from an image"), + ("deployment", "compose project"), +]; /// The action of the dialog that registers a provisioner: its name, then /// its type, of `types`. @@ -258,6 +260,12 @@ pub fn help(provisioner: Option<&ProvisionerInfo>, place: &str) -> String { if !provisioner.problem.is_empty() { text += &format!("It cannot be used here: {}\n\n", provisioner.problem); } + if !provisioner.node.is_empty() { + text += &format!( + "It lives on the node {}, whose feature offered it when the node was applied.\n\n", + provisioner.node + ); + } let kinds: Vec = provisioner .kinds .iter() @@ -285,7 +293,7 @@ pub fn help(provisioner: Option<&ProvisionerInfo>, place: &str) -> String { text += &format!("Its settings: {}.\n\n", set.join(", ")); } text + &format!( - "Enter opens its page, which asks it how it is.\n\nThe provisioners of {place}. From the command line: `berth provisioner {}`", + "Enter opens its page, which asks it how it is.\n\nThe provisioners of {place}. From the command line: `berth provisioner show {}`", provisioner.id ) } @@ -366,6 +374,32 @@ impl Page { .map(|(key, value)| format!("{key}={value}")) .collect(); row("Arguments", &arguments.join(" ")); + if !seen.endpoints.is_empty() { + lines.push((String::new(), Tone::Muted)); + lines.push(( + format!("Endpoints · berth port forward {} ENDPOINT", seen.id), + Tone::Plain, + )); + let width = seen + .endpoints + .iter() + .map(|endpoint| endpoint.name.chars().count()) + .max() + .unwrap_or(0); + for endpoint in &seen.endpoints { + let hint = match endpoint.hint.is_empty() { + true => String::new(), + false => format!(" {}", endpoint.hint), + }; + lines.push(( + format!( + " {: = info .own @@ -660,7 +703,7 @@ impl Tab { .collect(); let Some((first, _)) = makers.first().cloned() else { return Asked::Nothing(Some(format!( - "No provisioner makes {label}s. Add one on the Provisioners tab" + "No provisioner makes a {label}. Add one on the Provisioners tab" ))); }; let form = Form::making(&format!("new {label}"), &format!("new-{kind}"), makers); @@ -905,11 +948,11 @@ impl Tab { &page.actions[index], &match &page.resource { Some(_) => format!( - "berth resource {} run {}", + "berth resource run {} {}", page.id, page.actions[index].name ), None => format!( - "berth provisioner {} run {}", + "berth provisioner run {} {}", page.id, page.actions[index].name ), }, @@ -918,11 +961,11 @@ impl Tab { let held = &page.held()[index]; match (held.resource.is_empty(), held.kind.as_str()) { (false, "machine") if !held.host.is_empty() => format!( - "The machine of {}, {} as the provisioner calls it. Enter opens its page, with its state, its actions and its log. It is looked after as that host too, on the host's page. From the command line: `berth resource {}`", + "The machine of {}, {} as the provisioner calls it. Enter opens its page, with its state, its actions and its log. It is looked after as that host too, on the host's page. From the command line: `berth resource show {}`", held.host, held.id, held.resource ), (false, _) => format!( - "The {} {}, {} as the provisioner calls it. Enter opens its page, with its state, its actions and its log. From the command line: `berth resource {}`", + "The {} {}, {} as the provisioner calls it. Enter opens its page, with its state, its actions and its log. From the command line: `berth resource show {}`", held.kind, held.name, held.id, held.resource ), (true, _) => format!( @@ -932,15 +975,15 @@ impl Tab { } } Some(Entry::Logs) => format!( - "Its log as its provisioner gives it, the last lines and then what follows, with secrets masked. From the command line: `berth resource {} logs --follow`", + "Its log as its provisioner gives it, the last lines and then what follows, with secrets masked. From the command line: `berth resource logs {} -f`", page.id ), None if page.resource.is_some() => format!( - "Asking about the resource. From the command line: `berth resource {}`", + "Asking about the resource. From the command line: `berth resource show {}`", page.id ), None => format!( - "Asking the provisioner how it is. From the command line: `berth provisioner {}`", + "Asking the provisioner how it is. From the command line: `berth provisioner show {}`", page.id ), }; @@ -1311,7 +1354,10 @@ mod tests { Ok(vec![Action::default(), made.clone()]), ); let form = tab.form().unwrap(); - assert_eq!(form.context(), format!("{first} › new instance")); + assert_eq!( + form.context(), + format!("{first} › new resource from an image") + ); assert!(!form.waiting()); // Enter on its first row goes on to the next provisioner, which is asked. assert_eq!( @@ -1491,7 +1537,7 @@ mod tests { assert!( shown .help - .contains("berth provisioner docker/container run images") + .contains("berth provisioner run docker/container images") ); let Asked::Run { action, values, .. } = tab.key(key(Code::Enter), &list, &mut Nowhere) else { @@ -1635,7 +1681,7 @@ mod tests { let rows = text(&shown); assert_eq!(rows[0], "wiki · deployment of docker/container"); assert!(rows.contains(&" ▸ Show its log".to_owned()), "{rows:?}"); - assert!(shown.help.contains("berth resource r-wiki run remove")); + assert!(shown.help.contains("berth resource run r-wiki remove")); // Removing asks first, and then runs as the resource's. tab.key(key(Code::Enter), &list, &mut Nowhere); assert_eq!( diff --git a/crates/berth-tui/src/qr.rs b/crates/berth-tui/src/qr.rs index 82ca3c3..0190f85 100644 --- a/crates/berth-tui/src/qr.rs +++ b/crates/berth-tui/src/qr.rs @@ -19,7 +19,7 @@ use ratatui::{ }; /// The key that shows a link as a QR code. Alt keeps it from the program in a -/// terminal panel, and a browser leaves it to the page. +/// terminal panel. pub const KEY: &str = "Alt+R"; /// Its hint in a footer. pub const HINT: &str = "Alt+R QR code"; @@ -176,8 +176,6 @@ pub struct Overlay { pub title: String, pub links: Vec, pub index: usize, - /// A few words in the bottom border: what the code gives whoever scans it. - pub note: Option, } impl Overlay { @@ -193,20 +191,9 @@ impl Overlay { title: "QR code".into(), links, index: 0, - note: None, }) } - /// The link that opens a berth server's page, signed in. - pub fn connect(link: String) -> Overlay { - Overlay { - title: "Open berth on another device".into(), - links: vec![link], - index: 0, - note: Some("holds the server's token".into()), - } - } - pub fn link(&self) -> &str { &self.links[self.index.min(self.links.len() - 1)] } @@ -293,12 +280,7 @@ impl Overlay { _ => "←→ other links · Esc close", }; note_hints(keys, area.x + 1, area.bottom().saturating_sub(1), true); - let mut block = theme::popup(&title, theme::ACCENT).title_bottom(theme::hints(keys)); - if let Some(note) = &self.note { - block = block.title_bottom( - Line::styled(format!(" {note} "), Style::new().fg(theme::WARN)).right_aligned(), - ); - } + let block = theme::popup(&title, theme::ACCENT).title_bottom(theme::hints(keys)); let inner = block.inner(area); frame.render_widget(Clear, area); frame.render_widget(block, area); @@ -361,80 +343,6 @@ pub fn any_link<'a>(lines: impl IntoIterator) -> bool { }) } -/// A value as it may stand in an address: letters, digits and `-._~` as they -/// are, and with `keep` also those characters; everything else -/// percent-encoded. -fn encoded(value: &str, keep: &str) -> String { - let mut out = String::new(); - for byte in value.bytes() { - let ch = byte as char; - if ch.is_ascii_alphanumeric() || "-._~".contains(ch) || keep.contains(ch) { - out.push(ch); - } else { - out.push_str(&format!("%{byte:02X}")); - } - } - out -} - -/// An access token of an OAuth issuer is a JWT, of three parts in base64. -/// It runs out, and belongs to whoever it was issued to, so it is never handed on. -fn issued(token: &str) -> bool { - token.starts_with("eyJ") && token.matches('.').count() == 2 -} - -/// The link that opens the browser page of a berth server, signed in with -/// `token`: `page` is where the page is. A page that is not the server's own, -/// as on the documentation site, is told the `server` as well. `None` when -/// there is no such link: no token, an access token of an issuer, or no address over the -/// network. -pub fn connect_link(page: &str, server: Option<&str>, token: &str) -> Option { - let token = token.trim(); - let web = |address: &str| address.starts_with("https://") || address.starts_with("http://"); - if token.is_empty() || issued(token) || !web(page) || server.is_some_and(|server| !web(server)) - { - return None; - } - let query = match server { - Some(server) => format!("?server={}", encoded(server.trim_end_matches('/'), ":/")), - None => String::new(), - }; - Some(format!("{page}{query}#token={}", encoded(token, ""))) -} - -/// The link to the page that `berth serve --web` gives at `server`. -pub fn served_link(server: &str, token: &str) -> Option { - connect_link(&format!("{}/", server.trim_end_matches('/')), None, token) -} - -/// What the information box says of the row that shows the link to the -/// server's page. `served` when the link is to the page `berth serve --web` -/// gives, which a server without `--web` does not have. -pub fn connect_help(server: &str, served: bool) -> String { - let mut help = format!( - "A QR code and a link that open this berth server, {server}, in a browser on a phone or another computer, signed in.\n\n**Both hold the server's token**, which gives full control of its hosts. Show them to nobody who should not have it." - ); - if served { - help += &format!( - "\n\nThe link is to the page of `berth serve --web`. Without `--web`, the page of the documentation site takes `/web/?server={server}#token=TOKEN`." - ); - } - if loopback(server) { - help += "\n\nThat address is this machine's own, which a phone cannot reach. Use one of the server's addresses on your network or tailnet."; - } - help -} - -/// Whether an address is this machine's own, which a phone cannot reach. -pub fn loopback(address: &str) -> bool { - let rest = address - .trim_start_matches("https://") - .trim_start_matches("http://"); - ["localhost", "127.", "[::1]", "0.0.0.0"] - .iter() - .any(|name| rest.starts_with(name)) -} - #[cfg(test)] mod tests { use super::*; @@ -587,44 +495,18 @@ mod tests { #[test] fn the_overlay_falls_back_to_text_in_a_small_window() { - let overlay = Overlay::connect("https://berth.example.ts.net/#token=secret".into()); + let overlay = Overlay { + title: "QR code".into(), + links: vec!["https://berth.example.ts.net/?query=a-long-enough-link".into()], + index: 0, + }; let drawn = text(&screen(60, 12, |frame| overlay.draw(frame))); assert!( drawn.contains("too small") && !drawn.contains('▀'), "{drawn}" ); - assert!(drawn.contains("#token=secret") && drawn.contains("token")); + assert!(drawn.contains("a-long-enough-link")); let drawn = text(&screen(100, 40, |frame| overlay.draw(frame))); - assert!( - drawn.contains('▀') && drawn.contains("holds the server's token"), - "{drawn}" - ); - } - - #[test] - fn connect_links_are_what_the_page_takes() { - assert_eq!( - served_link("https://berth.example.ts.net/", "a1b2").as_deref(), - Some("https://berth.example.ts.net/#token=a1b2") - ); - assert_eq!( - connect_link( - "https://docs.example.org/web/", - Some("https://berth.example.ts.net"), - "a+b/c" - ) - .as_deref(), - Some( - "https://docs.example.org/web/?server=https://berth.example.ts.net#token=a%2Bb%2Fc" - ) - ); - // No token, a login's token, or a socket: nothing to hand on. - assert_eq!(served_link("https://berth.example.ts.net", ""), None); - assert_eq!( - served_link("https://berth.example.ts.net", "eyJhbGciOi.eyJzdWIi.c2ln"), - None - ); - assert_eq!(served_link("unix:/run/berth/serve.sock", "a1b2"), None); - assert!(loopback("http://127.0.0.1:7777") && !loopback("https://berth.example.ts.net")); + assert!(drawn.contains('▀'), "{drawn}"); } } diff --git a/crates/berth-tui/src/screen.rs b/crates/berth-tui/src/screen.rs index 12255b9..27dd1f7 100644 --- a/crates/berth-tui/src/screen.rs +++ b/crates/berth-tui/src/screen.rs @@ -199,10 +199,14 @@ pub fn hint_click(event: Event) -> Event { /// The bottom line: key hints on the left, the last message on the right. pub fn draw_footer(frame: &mut Frame, area: Rect, keys: &str, status: &str, colour: Color) { - let width = if status.is_empty() { + let long = status.chars().count() as u16 + 2 > area.width / 2; + if long { + draw_said(frame, area, status, colour); + } + let width = if status.is_empty() || long { 0 } else { - (status.chars().count() as u16 + 2).min(area.width / 2) + status.chars().count() as u16 + 2 }; let [hints, note] = Layout::horizontal([Constraint::Min(0), Constraint::Length(width)]).areas(area); @@ -217,6 +221,26 @@ pub fn draw_footer(frame: &mut Frame, area: Rect, keys: &str, status: &str, colo ); } +/// What the status line says when it is too long to sit beside the keys: +/// in full, wrapped, on the rows just above the footer, so that a refusal +/// or an error is never cut off. +fn draw_said(frame: &mut Frame, footer: Rect, status: &str, colour: Color) { + let lines = crate::setup::wrap_line( + &Line::styled(status.to_owned(), Style::new().fg(colour)), + usize::from(footer.width), + ); + let height = (lines.len() as u16).min(footer.y.saturating_sub(frame.area().y)); + if height == 0 { + return; + } + let area = Rect::new(footer.x, footer.y - height, footer.width, height); + frame.render_widget(Clear, area); + frame.render_widget( + Paragraph::new(lines).style(Style::new().bg(theme::BASE)), + area, + ); +} + /// A small centred box over the rest of the screen. Returns its area. pub fn dialog(frame: &mut Frame, title: &str, keys: &str, lines: Vec) -> Rect { let screen = frame.area(); @@ -317,19 +341,18 @@ pub fn words(line: &str) -> Vec<(u16, String)> { } /// Draw a session of the host: the same panel, with what leaving it means -/// there. `leave` is the key that leaves a running one: `Ctrl+Q`, or `Alt+Q` -/// where Ctrl+Q is not the page's to take. -pub fn draw_view(frame: &mut Frame, view: &View, note: &str, leave: &str) -> (Spots, Spots) { +/// there. Ctrl+Q leaves a running one. +pub fn draw_view(frame: &mut Frame, view: &View, note: &str) -> (Spots, Spots) { let running = match view.terminal { - true => format!( - " Ctrl+V paste · {leave} leave it running · Click opens a link · Right-click copies · Wheel scrolls " - ), - false => format!( - " {leave} leave it running · Click opens a link · Right-click copies · Wheel scrolls " - ), + true => { + " Ctrl+V paste · Ctrl+Q leave it running · Click opens a link · Right-click copies · Wheel scrolls " + } + false => { + " Ctrl+Q leave it running · Click opens a link · Right-click copies · Wheel scrolls " + } }; let keys = match (view.outcome(), view.ours) { - (None, _) => running.as_str(), + (None, _) => running, (Some(true), true) => " ✓ Done · Enter close · k keep it in Sessions ", (Some(false), true) => " ✗ It did not finish · Enter close · k keep it in Sessions ", (Some(true), false) => " ✓ Done · Enter close · d forget it ", @@ -491,10 +514,10 @@ pub fn draw_arguments(frame: &mut Frame, host: &HostView) { /// The dialog that asks for the arguments of one of a subject's actions. pub fn draw_action_arguments(frame: &mut Frame, actions: &[berth_api::Action], asked: &Arguments) { - let Some(task) = actions.get(asked.action) else { + let Some(action) = actions.get(asked.action) else { return; }; - let Some(argument) = task.arguments.get(asked.index) else { + let Some(argument) = action.arguments.get(asked.index) else { return; }; let mut lines = vec![Line::raw(ellipsize(argument_label(argument), 56))]; @@ -522,9 +545,9 @@ pub fn draw_action_arguments(frame: &mut Frame, actions: &[berth_api::Action], a let title = ellipsize( &format!( " {} · {} of {} ", - task.title(), + action.title(), asked.index + 1, - task.arguments.len() + action.arguments.len() ), 58, ); @@ -660,7 +683,7 @@ pub fn view_event( return Some(pasted(text)); } Event::Key(key) => { - if key.is_ctrl('q') || (key.alt && key.code == Code::Char('q')) { + if key.is_ctrl('q') { let running = view.running(); host.close(false); return running.then(|| "It runs on; open it again from Sessions".into()); @@ -699,7 +722,7 @@ pub fn view_event( None } -/// Keys of the dialog that asks for a task's arguments. Returns whether the +/// Keys of the dialog that asks for an action's arguments. Returns whether the /// value was accepted, for the caller to go on with. pub fn arguments_event( host: &mut HostView, @@ -752,3 +775,46 @@ pub fn action_arguments_event( } false } + +#[cfg(test)] +mod tests { + use super::*; + use ratatui::{Terminal, backend::TestBackend}; + + fn screen(width: u16, height: u16, status: &str) -> Vec { + let mut terminal = Terminal::new(TestBackend::new(width, height)).unwrap(); + terminal + .draw(|frame| { + let area = frame.area(); + let footer = Rect::new(area.x, area.bottom() - 1, area.width, 1); + draw_footer(frame, footer, "↑↓ select · q quit", status, theme::BAD); + }) + .unwrap(); + let buffer = terminal.backend().buffer().clone(); + (0..height) + .map(|y| { + (0..width) + .map(|x| buffer[(x, y)].symbol().to_owned()) + .collect::() + .trim_end() + .to_owned() + }) + .collect() + } + + #[test] + fn a_long_message_is_shown_whole_at_a_narrow_width() { + let said = "✗ The USB installer is built where berth serve runs, with berth ui there"; + let rows = screen(40, 8, said); + let words: Vec<&str> = rows[..7] + .iter() + .flat_map(|row| row.split_whitespace()) + .collect(); + assert_eq!(words.join(" "), said, "{rows:#?}"); + assert!(rows[7].starts_with("↑↓ select"), "{rows:#?}"); + // A short one stays beside the keys. + let rows = screen(80, 4, "Saved"); + assert!(rows[3].ends_with("Saved"), "{rows:#?}"); + assert!(rows[..3].iter().all(String::is_empty), "{rows:#?}"); + } +} diff --git a/crates/berth-tui/src/setup.rs b/crates/berth-tui/src/setup.rs index f04c100..e75798d 100644 --- a/crates/berth-tui/src/setup.rs +++ b/crates/berth-tui/src/setup.rs @@ -1,6 +1,6 @@ //! The set-up pages of a host: what its profile holds, the rows each page //! shows of it, and what changing a row does to it. Nothing here reads a file -//! or runs a command, so berth and its page in a browser share it. +//! or runs a command: the server keeps the profile and does what it says. use crate::fields::{Action, Field, Group, Kind, Step}; use crate::theme; @@ -19,6 +19,29 @@ pub struct Feature { /// The group of the Features page it is listed in. pub group: String, pub help: String, + /// The types of host it is for; none says a workstation. + pub types: Vec, +} + +impl Feature { + /// Whether a host of this type can have it. + pub fn fits(&self, host_type: berth_api::HostType) -> bool { + match self.types.is_empty() { + true => host_type == berth_api::HostType::Workstation, + false => self.types.contains(&host_type), + } + } +} + +/// The types of host a feature of the catalogue is for, as the API says them. +pub fn feature_types(feature: &Value) -> Vec { + feature["types"] + .as_array() + .into_iter() + .flatten() + .filter_map(Value::as_str) + .filter_map(berth_api::HostType::parse) + .collect() } /// What a selectable row of a set-up page does when activated. @@ -127,6 +150,124 @@ pub struct Row { pub entry: Option, } +/// The rows as a list draws them in `width` cells, its cursor's mark taken +/// off: a row the cursor skips, such as a problem, a refusal or a note, +/// wraps under itself instead of running off the panel. A row the cursor +/// stops on stays one line, and the Information panel has its whole text. +pub fn wrapped(rows: Vec, width: u16) -> Vec { + wrapped_rows( + rows.into_iter().map(|row| (row.line, row.entry)).collect(), + width, + ) + .into_iter() + .map(|(line, entry)| Row { line, entry }) + .collect() +} + +/// [`wrapped`] for the rows of any list: each a line and what choosing it does. +pub fn wrapped_rows( + rows: Vec<(Line<'static>, Option)>, + width: u16, +) -> Vec<(Line<'static>, Option)> { + let width = usize::from(width.saturating_sub(theme::CURSOR.chars().count() as u16)); + let mut drawn = Vec::with_capacity(rows.len()); + for (line, entry) in rows { + match entry { + Some(entry) => drawn.push((line, Some(entry))), + None => drawn.extend(wrap_line(&line, width).into_iter().map(|line| (line, None))), + } + } + drawn +} + +/// Characters of a line, each with its style. +type Cells = Vec<(String, Style)>; + +/// A line broken at spaces into lines `width` cells wide. Its lines after +/// the first are indented as far as its text begins, past a mark such as +/// `✗ ` or `! `, so they read as one item. +pub fn wrap_line(line: &Line<'static>, width: usize) -> Vec> { + let cells: Vec<(String, Style)> = line + .spans + .iter() + .flat_map(|span| { + let style = line.style.patch(span.style); + span.content.chars().map(move |ch| (ch.to_string(), style)) + }) + .collect(); + let wide = |text: &str| Span::raw(text).width(); + let total: usize = cells.iter().map(|(text, _)| wide(text)).sum(); + if width < 8 || total <= width { + return vec![line.clone()]; + } + let mut indent = cells.iter().take_while(|(text, _)| text == " ").count(); + let marked = cells.get(indent + 1).is_some_and(|(text, _)| text == " ") + && cells + .get(indent) + .is_some_and(|(text, _)| !text.chars().all(char::is_alphanumeric)); + if marked { + indent += 2; + } + let indent = indent.min(width / 2); + // Words with the spaces before them. + let mut words: Vec<(Cells, Cells)> = vec![]; + let mut spaces = vec![]; + let mut word = vec![]; + for cell in cells { + if cell.0 == " " { + if !word.is_empty() { + words.push((std::mem::take(&mut spaces), std::mem::take(&mut word))); + } + spaces.push(cell); + } else { + word.push(cell); + } + } + if !word.is_empty() || !spaces.is_empty() { + words.push((spaces, word)); + } + let to_line = |cells: &[(String, Style)]| { + let mut spans: Vec> = vec![]; + for (text, style) in cells { + match spans.last_mut() { + Some(last) if last.style == *style => last.content.to_mut().push_str(text), + _ => spans.push(Span::styled(text.clone(), *style)), + } + } + Line::from(spans) + }; + let mut lines = vec![]; + let mut current: Vec<(String, Style)> = vec![]; + let mut used = 0; + for (spaces, word) in words { + let word_wide: usize = word.iter().map(|(text, _)| wide(text)).sum(); + let starting = lines.is_empty() && current.iter().all(|(text, _)| text == " "); + if !starting && used + spaces.len() + word_wide > width && used > indent { + lines.push(to_line(¤t)); + current = vec![(" ".to_owned(), Style::new()); indent]; + used = indent; + } else { + used += spaces.len(); + current.extend(spaces); + } + for cell in word { + let cell_wide = wide(&cell.0); + // A word longer than a whole line is broken where the line ends. + if used + cell_wide > width && used > indent { + lines.push(to_line(¤t)); + current = vec![(" ".to_owned(), Style::new()); indent]; + used = indent; + } + used += cell_wide; + current.push(cell); + } + } + if !current.is_empty() { + lines.push(to_line(¤t)); + } + lines +} + impl Row { pub fn info(line: impl Into>) -> Self { Row { @@ -161,35 +302,39 @@ pub fn host_secrets(mut names: Vec) -> Vec { names } -/// Give the profile the host profile the way of installing it takes. A -/// seed would give a laptop's physical login and power settings to a server, -/// and a created machine has none, so both make the profile `cloud`. -pub fn fit_profile(config: &mut Value, action: Action) { - if action.seeds() && config["profile"] != "cloud" && config.is_object() { - config["profile"] = json!("cloud"); +/// Give the profile the target the way of installing it takes. A seed would +/// give a laptop's physical login and power settings to a server, and a +/// created machine has none, so both make the target `cloud`. +pub fn fit_target(config: &mut Value, action: Action) { + if action.seeds() && config["target"] != "cloud" && config.is_object() { + config["target"] = json!("cloud"); } } -/// Let a host that is not set up yet join the tailnet with keys berth makes, -/// when berth has the control server's API key to make them with. +/// A host that is not set up yet joins the tailnet when berth has the +/// control server's API key to make its key with, and is reached over SSH +/// alone when it has not. A host that was set up keeps its choice. pub fn default_enrollment(config: &mut Value, names: &[String]) { - let unset = |key: &str| config[key].as_str().unwrap_or("").is_empty(); - if names + let has_key = names .iter() - .any(|name| name == berth_api::profile::TAILNET_API_KEY) + .any(|name| name == berth_api::profile::TAILNET_API_KEY); + if !has_key && config.is_object() - && config.get("tailscale_auto_key").is_none() - && unset("hostname") - && unset("tailscale_auth_key") + && config["hostname"].as_str().unwrap_or("").is_empty() + && config["ansible_vars"]["devbox_install_tailscale"].is_null() { - config["tailscale_auto_key"] = json!(true); + set_pointer( + config, + "/ansible_vars/devbox_install_tailscale", + json!(false), + ); } } /// What the home screen's Secrets tab says of a secret berth uses itself, if it is one. pub fn own_secret_help(name: &str) -> Option<&'static str> { (name == berth_api::profile::TAILNET_API_KEY).then_some( - "The API key of the tailnet's control server: a Tailscale API access token, or a Headscale API key (headscale apikeys create). berth creates one-use enrollment keys with it, for hosts that join the tailnet automatically and for F4 on the key field. It is never sent to a host. Enter replaces it, x twice removes it.", + "The API key of the tailnet's control server: a Tailscale OAuth client with the auth_keys scope as CLIENT_ID:CLIENT_SECRET, a Tailscale API access token, or a Headscale API key (headscale apikeys create). berth makes a one-use key with it for each host that joins the tailnet. An access token expires after at most 90 days, and an OAuth client does not. It is never sent to a host. Enter replaces it, x twice removes it.", ) } @@ -199,13 +344,6 @@ pub fn set_pointer(config: &mut Value, pointer: &str, value: Value) { ("/access/sudo", Value::Bool(asks)) => json!(if *asks { "password" } else { "none" }), _ => value, }; - // A pasted key would win over the automatic one and is hidden from then on. - if pointer == "/tailscale_auto_key" - && value == json!(true) - && let Some(profile) = config.as_object_mut() - { - profile.remove("tailscale_auth_key"); - } let mut node = config; let mut parts = pointer.trim_start_matches('/').split('/').peekable(); while let Some(part) = parts.next() { @@ -420,6 +558,13 @@ pub fn commit(config: &mut Value, field: Field, value: &str) -> Result<(), Strin { return Err(format!("Choose one of: {}", choices.join(", "))); } + // A type changes more than its field, such as a node's accounts going. + if field == Field::Type { + let host_type = berth_api::HostType::parse(value) + .ok_or_else(|| format!("Choose one of: {}", crate::fields::TYPES.join(", ")))?; + berth_api::profile::set_host_type(config, host_type); + return Ok(()); + } if let Some(name) = spec.pointer.strip_prefix("/ansible_vars/") && value.is_empty() { @@ -521,6 +666,9 @@ pub struct Profile<'a> { /// Whether the host's name can be the machine's: a profile without a /// hostname takes it. pub named: bool, + /// Whether berth has the API key of the tailnet's control server, which + /// a host that joins the tailnet needs. + pub tailnet_key: bool, } impl<'a> Profile<'a> { @@ -528,6 +676,29 @@ impl<'a> Profile<'a> { account_keys(self.config, account.name()) } + /// Whether the profile is a node's: a server applied without a desktop + /// or accounts of its own. + pub fn is_node(&self) -> bool { + berth_api::profile::host_type(self.config) == berth_api::HostType::Node + } + + /// The type of host whose features the Features tab offers: a node's, + /// or else a workstation's, as only those two are given features. + fn feature_host_type(&self) -> berth_api::HostType { + match self.is_node() { + true => berth_api::HostType::Node, + false => berth_api::HostType::Workstation, + } + } + + /// The features, by index, a host of this profile can have. + pub fn offered_features(&self) -> Vec { + let host_type = self.feature_host_type(); + (0..self.features.len()) + .filter(|index| self.features[*index].fits(host_type)) + .collect() + } + pub fn text(&self, field: Field) -> &'a str { let text = self .config @@ -536,7 +707,7 @@ impl<'a> Profile<'a> { .unwrap_or(""); match (field, text) { (Field::Login, "") => "berth", - (Field::Level, "") => "workstation", + (Field::Type, "") => "workstation", _ => text, } } @@ -662,6 +833,7 @@ impl<'a> Profile<'a> { true => &[Cockpit, Docker], false => &[], }; + let host_type = self.feature_host_type(); let builtin: [(&str, &[Field]); 6] = [ ("Coding Agents", &[]), ("Agent Orchestrator", &[]), @@ -684,10 +856,17 @@ impl<'a> Profile<'a> { .iter() .map(|(title, fields)| FeatureGroup { title: (*title).to_owned(), - items: fields.iter().map(|field| Item::Field(*field)).collect(), + // berth's own things to install are a workstation's. + items: match self.is_node() { + true => vec![], + false => fields.iter().map(|field| Item::Field(*field)).collect(), + }, }) .collect(); for (index, feature) in self.features.iter().enumerate() { + if !feature.fits(host_type) { + continue; + } let item = Item::Feature(index); let known = groups .iter_mut() @@ -769,6 +948,7 @@ impl<'a> Profile<'a> { pub fn fields(&self) -> Vec { use Field::*; match self.step { + Step::Features if self.is_node() => vec![], Step::Features => { let mut fields = vec![ ToolsWeb, @@ -820,7 +1000,7 @@ impl<'a> Profile<'a> { if self.locked() { return vec![]; } - vec![Level, SshHost, SshUser, SshPort, Login, AskBecomePass] + vec![Type, SshHost, SshUser, SshPort, Login, AskBecomePass] } /// The fields of the Install tab: those before the provisioner's rows, @@ -832,34 +1012,43 @@ impl<'a> Profile<'a> { return (vec![], vec![]); } // A host below a workstation needs only to be reached. - let level = berth_api::profile::level(self.config); - if action == Action::Ssh && level != berth_api::HostLevel::Workstation { - let mut after = vec![Level, SshHost, SshUser, SshPort, Login]; - // Nothing is installed on a reachable host, so nothing asks for sudo. - if level == berth_api::HostLevel::Controller { + let host_type = berth_api::profile::host_type(self.config); + // A node is applied over SSH, without a desktop, and may join the tailnet. + if action == Action::Ssh && host_type == berth_api::HostType::Node { + return ( + vec![], + vec![ + Type, + SshHost, + SshUser, + SshPort, + Login, + AskBecomePass, + Tailscale, + ], + ); + } + if action == Action::Ssh && host_type != berth_api::HostType::Workstation { + let mut after = vec![Type, SshHost, SshUser, SshPort, Login]; + // Nothing is installed on a host of the type ssh, so nothing asks for sudo. + if host_type == berth_api::HostType::Managed { after.push(AskBecomePass); } return (vec![], after); } - // A created machine is always a cloud host, so there is no profile to choose. + // A created machine is always a cloud host, so there is no target to choose. let before = match action { Action::Machine => vec![], - _ => vec![Profile], + _ => vec![Target], }; let mut after = vec![]; if action == Action::Ssh { - after.extend([Level, SshHost, SshUser, SshPort, Login, AskBecomePass]); + after.extend([Type, SshHost, SshUser, SshPort, Login, AskBecomePass]); } if action.installer() || action.seeds() { after.push(Hostname); } after.push(Tailscale); - if self.enabled(Tailscale) { - after.extend([TailscaleLoginServer, TailscaleAuto]); - if !self.enabled(TailscaleAuto) { - after.push(TailscaleKey); - } - } (before, after) } @@ -873,6 +1062,8 @@ impl<'a> Profile<'a> { Action::Save | Action::Release | Action::Manage => return (vec![], vec![], vec![]), // The container only has the agent account. Action::Docker => return (vec![AgentUser], vec![], vec![]), + // A node has no accounts of its own: sessions run as the SSH user. + _ if self.is_node() => return (vec![], vec![("Access", vec![HardenSsh])], vec![]), _ => {} } let accounts = vec![ @@ -964,19 +1155,7 @@ impl<'a> Profile<'a> { } else { self.text(field).to_owned() }; - let (value, kind) = match field { - // Enough of the key to recognise it, not enough to use it. - Field::TailscaleKey if value.len() > 24 && value.is_ascii() => ( - format!( - "{}…{}", - &value[..value.len().min(16)], - &value[value.len() - 4..] - ), - Kind::Text, - ), - _ => (value, spec.kind), - }; - let mut line = value_line(spec.label, &value, kind); + let mut line = value_line(spec.label, &value, spec.kind); line.spans.insert(0, Span::styled(prefix.to_owned(), muted)); rows.push(Row::entry(line, Entry::Field(field))); } @@ -1046,11 +1225,11 @@ impl<'a> Profile<'a> { rows } - /// The way of the Install tab the action and the profile's level are: - /// over SSH at the level reachable, the host is only logged in to. + /// The way of the Install tab the action and the profile's type are: + /// over SSH to a host of the type ssh, the host is only logged in to. pub fn way(&self) -> Action { - match (self.action, berth_api::profile::level(self.config)) { - (Action::Ssh, berth_api::HostLevel::Reachable) => Action::Reach, + match (self.action, berth_api::profile::host_type(self.config)) { + (Action::Ssh, berth_api::HostType::Ssh) => Action::Reach, (action, _) => action, } } @@ -1059,7 +1238,11 @@ impl<'a> Profile<'a> { pub fn method_rows(&self, methods: &[Action]) -> Vec { let mut rows = vec![Row::heading("How To Install")]; let chosen = self.way(); + let host_type = berth_api::profile::host_type(self.config); for action in methods.iter().copied() { + if !action.offered(host_type) && action != chosen { + continue; + } let mark = if action == chosen { "●" } else { "○" }; rows.push(Row::entry( format!(" {mark} {}", action.label()), @@ -1116,7 +1299,7 @@ impl<'a> Profile<'a> { } for problem in own { rows.push(Row::info(Line::styled( - format!("✗ {}", problem.text), + format!("✗ {}", problem.shown()), Style::new().fg(theme::BAD), ))); } @@ -1142,7 +1325,7 @@ impl<'a> Profile<'a> { } for problem in placed { rows.push(Row::info(Line::styled( - format!(" ✗ {}", problem.text), + format!(" ✗ {}", problem.shown()), bad, ))); } @@ -1185,7 +1368,7 @@ impl<'a> Profile<'a> { if matches!(self.action, Action::Machine | Action::Docker) { return None; } - let cloud = self.action == Action::CloudSeed || self.text(Field::Profile) == "cloud"; + let cloud = self.action == Action::CloudSeed || self.text(Field::Target) == "cloud"; (cloud && self.text(Field::RecoveryCidr).is_empty()).then(|| { if self.enabled(Field::HardenSsh) { "! No recovery SSH CIDR: SSH is reachable from any address (keys only)" @@ -1200,43 +1383,92 @@ impl<'a> Profile<'a> { pub fn problems(&self) -> Vec { self.placed_problems() .into_iter() - .map(|problem| problem.text) + .map(|problem| problem.shown()) .collect() } /// Every problem the profile and the machine show, each on its tab. pub fn problems_with(&self, machine: &crate::machine::Machine) -> Vec { + let mut problems = vec![]; + if self.action == Action::Ssh && self.text(Field::SshHost).is_empty() { + problems.push(Problem::of( + Step::Install, + Field::SshHost, + "The host's address is required: its name, IP address or tailnet name", + )); + } + if let Some(problem) = self.way_problem() { + problems.push(problem); + } + if self.is_node() { + problems.extend(self.placed_problems()); + return problems; + } // A host below a workstation is only reached: the workstation's settings do not apply. if self.action == Action::Ssh - && berth_api::profile::level(self.config) != berth_api::HostLevel::Workstation + && berth_api::profile::host_type(self.config) != berth_api::HostType::Workstation { - return vec![]; + return problems; } - let mut problems = self.placed_problems(); + problems.extend(self.placed_problems()); if self.action == Action::Machine && !machine.exists() { problems.extend(machine.problems()); } problems } + /// That the way chosen cannot install a host of the profile's type. + fn way_problem(&self) -> Option { + let host_type = berth_api::profile::host_type(self.config); + let way = self.way(); + if !self.serves_host() || way.types().contains(&host_type) { + return None; + } + Some(Problem::new( + Step::Install, + format!( + "{} installs a workstation, and this host's type is {host_type}: install it over SSH (method ssh), or make its type workstation", + way.label() + ), + )) + } + /// [`Profile::problems`], each on the tab of the field it is about. pub fn placed_problems(&self) -> Vec { use Field::*; let action = self.action; let mut problems = Vec::new(); - let mut check = |ok: bool, step: Step, message: &str| { + let mut check = |ok: bool, step: Step, field: Field, message: &str| { if !ok { - problems.push(Problem::new(step, message)); + problems.push(Problem::of(step, field, message)); } }; if matches!(action, Action::Save | Action::Release | Action::Manage) { return problems; } + if self.is_node() { + if action != Action::Ssh { + problems.push(Problem::of( + Step::Install, + Type, + "A node is applied over SSH to a server that is installed already: choose Existing host over SSH", + )); + } + // What the profile says that a node has not got, such as accounts. + // Its address is the SSH address's own problem. + for problem in berth_api::profile::problems(self.config) { + if !problem.contains("needs access.address") { + problems.push(Problem::new(Step::Install, problem)); + } + } + return problems; + } let settings = Step::Settings; let agent = self.text(AgentUser); check( valid_username(agent), settings, + AgentUser, "Agent username is invalid or reserved by Ubuntu", ); if action == Action::Docker { @@ -1247,18 +1479,21 @@ impl<'a> Profile<'a> { check( valid_hostname(hostname) || hostname.is_empty() && self.named, Step::Install, - "Hostname is missing or invalid (F4 generates)", + Hostname, + "Hostname is missing or invalid", ); } let operator = self.text(OperatorUser); check( valid_username(operator), settings, + OperatorUser, "Operator username is invalid or reserved by Ubuntu (operator and admin are)", ); check( operator != agent, settings, + AgentUser, "Operator and agent need different usernames", ); for (field, name) in [(OperatorHash, "Operator"), (AgentHash, "Agent")] { @@ -1266,6 +1501,7 @@ impl<'a> Profile<'a> { check( hash.is_empty() || valid_password_hash(hash), settings, + field, &format!("{name} password hash is not SHA-512 crypt or yescrypt"), ); } @@ -1273,6 +1509,7 @@ impl<'a> Profile<'a> { check( self.keys(account).iter().all(|key| is_public_key(key)), settings, + account.field(), &format!("{name} has an SSH public key that is not a supported OpenSSH key"), ); } @@ -1281,6 +1518,7 @@ impl<'a> Profile<'a> { || !self.text(AgentPrivileges).is_empty() || !self.text(AgentHash).is_empty(), settings, + AgentHash, "Agent needs a password when passwordless sudo is off", ); if action.installer() || action.seeds() { @@ -1288,6 +1526,7 @@ impl<'a> Profile<'a> { check( !self.keys(Account::Operator).is_empty() || !self.own_key.is_empty(), settings, + OperatorKey, "Operator needs an SSH key: add one, or make this berth's own", ); // The USB installer asks for a password for the account it makes. @@ -1296,7 +1535,8 @@ impl<'a> Profile<'a> { check( !self.text(OperatorHash).is_empty() || action.seeds(), settings, - "Operator password is required (F4 or F5)", + OperatorHash, + "Operator password is required", ); } let geometry = self.text(Geometry); @@ -1308,6 +1548,7 @@ impl<'a> Profile<'a> { }) }), settings, + Geometry, "Desktop size must look like 1600x900", ); if action.installer() { @@ -1315,16 +1556,19 @@ impl<'a> Profile<'a> { "serial" => check( !self.text(DiskSerial).is_empty(), settings, + DiskSerial, "Serial disk selection needs the target disk serial", ), "model" => check( !self.text(DiskModel).is_empty(), settings, + DiskModel, "Model disk selection needs a model pattern", ), "path" => check( self.text(DiskPath).starts_with("/dev/"), settings, + DiskPath, "Path disk selection needs a /dev/ path or pattern", ), _ => {} @@ -1335,15 +1579,27 @@ impl<'a> Profile<'a> { .iter() .all(|field| !self.text(*field).is_empty()), settings, + WifiSsid, "Wi-Fi needs an SSID and password", ); } } + // A new machine always needs a key. A host applied over SSH may be on + // the tailnet already, which only the host can say, so the apply asks it. + if action.installer() || action.seeds() { + check( + !self.enabled(Tailscale) || self.tailnet_key, + Step::Install, + Tailscale, + "Tailscale is on, and berth has no API key of the tailnet to make the host's key: set the secret tailnet_api_key on the Secrets tab of the home screen, or switch Tailscale off", + ); + } if action == Action::CloudSeed { check( !self.text(ReleaseStore).is_empty() || self.text(ReleaseUrl).starts_with("https://"), settings, + ReleaseStore, "Choose where to publish the release, or enter its HTTPS address", ); } @@ -1367,6 +1623,9 @@ pub struct Page<'a> { pub struct Problem { pub step: Step, pub text: String, + /// The field it is about, when it is about one: where the screens + /// point, and what the command line asks for or names as the flag. + pub field: Option, } impl Problem { @@ -1374,6 +1633,24 @@ impl Problem { Problem { step, text: text.into(), + field: None, + } + } + + /// A problem of one field. + pub fn of(step: Step, field: Field, text: impl Into) -> Self { + Problem { + field: Some(field), + ..Problem::new(step, text) + } + } + + /// What the screens show: the text, and the key that fixes it there. + pub fn shown(&self) -> String { + match self.field { + Some(field) if field.is_password_hash() => format!("{} (F4 or F5)", self.text), + Some(field) if field.can_generate() => format!("{} (F4 generates)", self.text), + _ => self.text.clone(), } } } @@ -1393,6 +1670,7 @@ mod tests { label: key.into(), group: group.into(), help: String::new(), + types: vec![], }; vec![ feature("demo/agent", "coding agents"), @@ -1416,6 +1694,7 @@ mod tests { secrets: &[], own_key: "", named: false, + tailnet_key: true, }; let groups = profile.feature_groups(); assert_eq!(groups[0].title, "Coding Agents"); @@ -1425,28 +1704,37 @@ mod tests { } #[test] - fn a_host_joins_automatically_or_with_a_pasted_key() { - let mut config = json!({"tailscale_auth_key": "tskey-old"}); - let fields = |config: &Value| { + fn the_tailnet_is_one_switch_and_needs_berths_api_key() { + let config = json!({"hostname": "studio", "builder": {}}); + let profile = |tailnet_key: bool| Profile { + config: &config, + step: Step::Install, + action: Action::CloudSeed, + features: &[], + registries: &[], + secrets: &[], + own_key: "", + named: false, + tailnet_key, + }; + let shown = profile(true).fields(); + assert!(shown.contains(&Field::Tailscale)); + let lacks = |problems: Vec| { + problems + .iter() + .any(|problem| problem.contains("set the secret tailnet_api_key")) + }; + assert!(!lacks(profile(true).problems())); + assert!(lacks(profile(false).problems())); + // Without the tailnet there is nothing to make a key for. + let off = json!({"builder": {}, "ansible_vars": {"devbox_install_tailscale": false}}); + assert!(!lacks( Profile { - config, - step: Step::Install, - action: Action::Ssh, - features: &[], - registries: &[], - secrets: &[], - own_key: "", - named: false, + config: &off, + ..profile(false) } - .fields() - }; - let shown = fields(&config); - assert!(shown.contains(&Field::TailscaleKey)); - assert!(shown.contains(&Field::TailscaleLoginServer)); - set_pointer(&mut config, "/tailscale_auto_key", json!(true)); - assert!(config.get("tailscale_auth_key").is_none()); - let shown = fields(&config); - assert!(shown.contains(&Field::TailscaleAuto) && !shown.contains(&Field::TailscaleKey)); + .problems() + )); // berth's own API key is no secret a host can be given. let names = vec!["github".to_owned(), "tailnet_api_key".to_owned()]; assert_eq!(host_secrets(names), ["github"]); @@ -1479,6 +1767,7 @@ mod tests { secrets: &[], own_key: "", named: false, + tailnet_key: true, }; let problems = profile.problems(); assert!(problems.iter().any(|problem| problem.contains("Hostname"))); @@ -1511,23 +1800,22 @@ mod tests { } #[test] - fn a_new_host_enrolls_itself_when_berth_has_the_api_key() { + fn a_new_host_joins_the_tailnet_when_berth_has_the_api_key() { let names = vec!["tailnet_api_key".to_owned()]; let mut new = json!({}); default_enrollment(&mut new, &names); - assert_eq!(new["tailscale_auto_key"], true); - // Without the key there is nothing to make enrollment keys with. + assert_eq!(new, json!({})); + // Without the key there is nothing to make its key with: it is reached over SSH. let mut new = json!({}); default_enrollment(&mut new, &["github".to_owned()]); - assert!(new.get("tailscale_auto_key").is_none()); - // A host that was set up, or has a key of its own, keeps its choice. + assert_eq!(new["ansible_vars"]["devbox_install_tailscale"], false); + // A host that was set up, or chose already, keeps its choice. for kept in [ json!({"hostname": "studio"}), - json!({"tailscale_auth_key": "tskey-auth-example"}), - json!({"tailscale_auto_key": false}), + json!({"ansible_vars": {"devbox_install_tailscale": true}}), ] { let mut profile = kept.clone(); - default_enrollment(&mut profile, &names); + default_enrollment(&mut profile, &["github".to_owned()]); assert_eq!(profile, kept); } } @@ -1548,6 +1836,7 @@ mod tests { secrets: &[], own_key: "ssh-ed25519 AAAA berth@studio", named: true, + tailnet_key: true, }; let problems = profile.problems_with(machine); profile @@ -1613,7 +1902,7 @@ mod tests { let install = tab(&config, Action::Machine, Step::Install, &machine); // What has to be chosen, in order, and none of what has a default. let order = [ - "● Create a machine", + "● From a provisioner", "Provisioner Incus virtual machine", "└─ server * Not set", "Hostname", @@ -1625,9 +1914,9 @@ mod tests { "{pair:?}" ); } - // A created machine is always a cloud host: there is no profile to choose. + // A created machine is always a cloud host: there is no target to choose. for elsewhere in [ - "Host profile", + "Target", "cpus", "Operator username", "Desktop size", @@ -1644,7 +1933,7 @@ mod tests { let settings = tab(&config, Action::Machine, Step::Settings, &machine); let order = [ - "Machine", + "Size", "├─ cpus 1", "└─ memory 1", "Provisioner Options", @@ -1676,9 +1965,9 @@ mod tests { assert!(place(&run, "Install") < place(&run, " ✗ server is required")); assert!( place(&run, "✗ server is required") - < place(&run, "[Create machine · Incus virtual machine]") + < place(&run, "[Create with Incus virtual machine]") ); - assert!(place(&run, "[Create machine") < place(&run, "[Save · Ctrl+S]")); + assert!(place(&run, "[Create with") < place(&run, "[Save · Ctrl+S]")); assert!( !run.iter() .any(|row| row.contains("✓ Ready") || row.contains("[Run ·")) @@ -1692,7 +1981,7 @@ mod tests { "agent_passwordless_sudo": true}); let machine = crate::machine::Machine::default(); let install = tab(&config, Action::CloudSeed, Step::Install, &machine); - assert!(place(&install, "Host profile") < place(&install, "Hostname studio")); + assert!(place(&install, "Target") < place(&install, "Hostname studio")); assert!( !install .iter() @@ -1707,7 +1996,7 @@ mod tests { assert!( !settings .iter() - .any(|row| row.contains("Disk selection") || row.contains("Machine")) + .any(|row| row.contains("Disk selection") || row.contains("Size")) ); assert_eq!( settings.last().unwrap(), @@ -1765,13 +2054,14 @@ mod tests { secrets: &[], own_key: "ssh-ed25519 AAAA berth@studio", named: true, + tailnet_key: true, }; let placed = profile.placed_problems(); let on = |step: Step| -> Vec { placed .iter() .filter(|problem| problem.step == step) - .map(|problem| format!("✗ {}", problem.text)) + .map(|problem| format!("✗ {}", problem.shown())) .collect() }; assert_eq!( @@ -1821,6 +2111,7 @@ mod tests { secrets: &["github".to_owned()], own_key: "", named: true, + tailnet_key: true, }; let problems = profile.problems_with(&machine); for row in profile.page_rows(Page { @@ -1838,11 +2129,83 @@ mod tests { } } + #[test] + fn a_long_problem_wraps_under_itself_and_rows_to_choose_stay_whole() { + let problem = "✗ The USB installer is built where berth serve runs, with berth ui there"; + let rows = vec![ + Row::entry( + " ○ USB installer, a way that has a long label too", + Entry::Action(Action::Iso), + ), + Row::info(Line::styled(problem, Style::new().fg(theme::BAD))), + ]; + let drawn = wrapped(rows, 40); + let text: Vec = drawn.iter().map(|row| row.line.to_string()).collect(); + assert_eq!( + text, + [ + " ○ USB installer, a way that has a long label too", + "✗ The USB installer is built where", + " berth serve runs, with berth ui", + " there", + ] + ); + assert!(drawn[0].entry.is_some() && drawn[1..].iter().all(|row| row.entry.is_none())); + // Every word is kept, in its colour, and nothing is wider than the panel. + let joined: Vec<&str> = text[1..] + .iter() + .flat_map(|line| line.split_whitespace()) + .collect(); + assert_eq!(joined.join(" "), problem); + assert!(text[1..].iter().all(|line| line.chars().count() <= 38)); + assert!( + drawn[2] + .line + .spans + .iter() + .any(|span| span.style.fg == Some(theme::BAD)) + ); + // A word longer than a line is broken at its end. + let long = wrap_line(&Line::raw(format!(" ! {}", "x".repeat(30))), 20); + assert!(long.iter().all(|line| line.width() <= 20), "{long:?}"); + assert_eq!(long.len(), 2); + } + + #[test] + fn a_nodes_install_tab_offers_only_the_ways_over_ssh() { + for host_type in ["node", "managed", "ssh"] { + let config = json!({"target": "laptop", "type": host_type}); + let rows = tab(&config, Action::Ssh, Step::Install, &incus()); + let ways: Vec<&String> = rows.iter().filter(|row| row.starts_with(" ")).collect(); + for way in ["From a provisioner", "USB installer", "Cloud-init seed"] { + assert!( + !ways.iter().any(|row| row.ends_with(way)), + "{host_type}: {rows:?}" + ); + } + assert!( + ways.iter() + .any(|row| row.ends_with("Existing host over SSH")), + "{host_type}: {rows:?}" + ); + } + let rows = tab( + &json!({"target": "laptop"}), + Action::Ssh, + Step::Install, + &incus(), + ); + assert!( + rows.iter().any(|row| row.ends_with("USB installer")), + "{rows:?}" + ); + } + #[test] fn an_existing_host_can_be_added_to_be_reached_over_ssh_alone() { - let mut config = json!({"profile": "laptop"}); + let mut config = json!({"target": "laptop"}); assert_eq!(Action::Reach.choose(&mut config), Action::Ssh); - assert_eq!(config["level"], "reachable"); + assert_eq!(config["type"], "ssh"); let machine = incus(); let profile = Profile { config: &config, @@ -1853,6 +2216,7 @@ mod tests { secrets: &[], own_key: "", named: true, + tailnet_key: true, }; assert_eq!(profile.way(), Action::Reach); let rows: Vec = profile @@ -1879,16 +2243,13 @@ mod tests { "{rows:?}" ); assert!(!rows.iter().any(|row| row.contains("sudo")), "{rows:?}"); - assert!( - !rows.iter().any(|row| row.contains("Host profile")), - "{rows:?}" - ); + assert!(!rows.iter().any(|row| row.contains("Target")), "{rows:?}"); // Installing over SSH again means installing something. assert_eq!(Action::Ssh.choose(&mut config), Action::Ssh); - assert!(config.get("level").is_none()); - let mut controller = json!({"level": "controller"}); - Action::Ssh.choose(&mut controller); - assert_eq!(controller["level"], "controller"); + assert!(config.get("type").is_none()); + let mut managed = json!({"type": "managed"}); + Action::Ssh.choose(&mut managed); + assert_eq!(managed["type"], "managed"); } #[test] @@ -1899,9 +2260,9 @@ mod tests { (Action::Iso, "laptop", "laptop"), (Action::Ssh, "laptop", "laptop"), ] { - let mut config = json!({"profile": before}); - fit_profile(&mut config, action); - assert_eq!(config["profile"], after, "{action:?}"); + let mut config = json!({"target": before}); + fit_target(&mut config, action); + assert_eq!(config["target"], after, "{action:?}"); } } } diff --git a/crates/berth-tui/src/theme.rs b/crates/berth-tui/src/theme.rs index 47b8e0d..9a802b4 100644 --- a/crates/berth-tui/src/theme.rs +++ b/crates/berth-tui/src/theme.rs @@ -11,26 +11,178 @@ use ratatui::{ widgets::{Block, BorderType, Padding}, }; +/// What a colour is for. The screens, the command line's prompts, its +/// progress, its tables and its help all take their colours from here, so +/// that the two look like one program. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Role { + /// Body text. + Text, + /// Secondary text: hints, help, what was already said. + Muted, + /// Values, keys, commands and what is selected. + Accent, + Good, + Warn, + Bad, + /// The background of the selected row. + Selection, +} + +impl Role { + /// The colour in a terminal that shows any colour. + pub const fn rgb(self) -> (u8, u8, u8) { + match self { + Role::Text => (205, 214, 244), + Role::Muted => (124, 137, 176), + Role::Accent => (116, 178, 255), + Role::Good => (158, 216, 140), + Role::Warn => (240, 198, 116), + Role::Bad => (242, 128, 150), + Role::Selection => (24, 31, 48), + } + } + + const fn colour(self) -> Color { + let (r, g, b) = self.rgb(); + Color::Rgb(r, g, b) + } + + /// The nearest of the sixteen colours every terminal has; `None` where + /// the terminal's own text colour or dim text serves better. + const fn basic(self) -> Option { + match self { + Role::Text | Role::Muted | Role::Selection => None, + Role::Accent => Some(Color::LightBlue), + Role::Good => Some(Color::Green), + Role::Warn => Some(Color::Yellow), + Role::Bad => Some(Color::LightRed), + } + } +} + +/// How many colours the terminal shows. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Default)] +pub enum Depth { + /// None: `--color never`, `NO_COLOR` or a dumb terminal. + None, + /// The sixteen colours of any terminal. + #[default] + Basic, + /// The 256 colours of xterm. + Indexed, + /// Any colour. + True, +} + +impl Depth { + /// The depth the terminal says it has, from `TERM` and `COLORTERM` + /// (and Windows Terminal's `WT_SESSION`), given that colours are on. + pub fn of_terminal(var: impl Fn(&str) -> Option) -> Depth { + let colorterm = var("COLORTERM").unwrap_or_default().to_lowercase(); + let term = var("TERM").unwrap_or_default().to_lowercase(); + if colorterm == "truecolor" || colorterm == "24bit" || var("WT_SESSION").is_some() { + Depth::True + } else if term == "dumb" { + Depth::None + } else if term.contains("256color") || term.contains("direct") { + Depth::Indexed + } else { + Depth::Basic + } + } +} + +/// The colour of `role` at `depth`, if it has one there. +pub fn colour(role: Role, depth: Depth) -> Option { + match depth { + Depth::None => None, + Depth::Basic => role.basic(), + Depth::Indexed => { + let (r, g, b) = role.rgb(); + Some(Color::Indexed(indexed(r, g, b))) + } + Depth::True => Some(role.colour()), + } +} + +/// Text in the colour of `role` at `depth`. Secondary text is dim where +/// there are only a few colours, as a grey of sixteen is often unreadable. +pub fn style(role: Role, depth: Depth) -> Style { + let style = match colour(role, depth) { + Some(colour) if role == Role::Selection => Style::new().bg(colour), + Some(colour) => Style::new().fg(colour), + None => Style::new(), + }; + match (role, depth) { + (Role::Muted, Depth::Basic) => style.dim(), + _ => style, + } +} + +/// The parameters of an SGR escape that writes text in `role`'s colour, +/// such as `38;2;116;178;255`; empty without a colour. +pub fn sgr(role: Role, depth: Depth) -> String { + let ground = if role == Role::Selection { 48 } else { 38 }; + match (colour(role, depth), role, depth) { + (_, Role::Muted, Depth::Basic) => "2".into(), + (Some(Color::Rgb(r, g, b)), ..) => format!("{ground};2;{r};{g};{b}"), + (Some(Color::Indexed(index)), ..) => format!("{ground};5;{index}"), + (Some(Color::LightBlue), ..) => format!("{}", ground + 56), + (Some(Color::Green), ..) => format!("{}", ground - 6), + (Some(Color::Yellow), ..) => format!("{}", ground - 5), + (Some(Color::LightRed), ..) => format!("{}", ground + 53), + _ => String::new(), + } +} + +/// The nearest colour of xterm's 6×6×6 cube or its grey ramp. +fn indexed(r: u8, g: u8, b: u8) -> u8 { + const STEPS: [u8; 6] = [0, 95, 135, 175, 215, 255]; + let near = |value: u8| { + (0..6) + .min_by_key(|&step| (i32::from(STEPS[step]) - i32::from(value)).abs()) + .unwrap_or(0) + }; + let (cr, cg, cb) = (near(r), near(g), near(b)); + let cube = 16 + 36 * cr + 6 * cg + cb; + let distance = |x: (u8, u8, u8)| { + let d = |a: u8, b: u8| (i32::from(a) - i32::from(b)).pow(2); + d(x.0, r) + d(x.1, g) + d(x.2, b) + }; + let average = ((u32::from(r) + u32::from(g) + u32::from(b)) / 3) as u8; + let grey_step = (u32::from(average.saturating_sub(8)) / 10).min(23) as u8; + let grey = 8 + 10 * grey_step; + if distance((grey, grey, grey)) < distance((STEPS[cr], STEPS[cg], STEPS[cb])) { + 232 + grey_step + } else { + cube as u8 + } +} + /// The screen. pub const BASE: Color = Color::Rgb(13, 17, 28); /// Pop-ups and the selected row. -pub const SURFACE: Color = Color::Rgb(24, 31, 48); +pub const SURFACE: Color = Role::Selection.colour(); pub const RAISED: Color = Color::Rgb(33, 44, 70); /// Panel borders. pub const LINE: Color = Color::Rgb(52, 66, 100); /// Body text. -pub const TEXT: Color = Color::Rgb(205, 214, 244); +pub const TEXT: Color = Role::Text.colour(); /// Secondary text. -pub const MUTED: Color = Color::Rgb(124, 137, 176); +pub const MUTED: Color = Role::Muted.colour(); /// Selection, values and keys. -pub const ACCENT: Color = Color::Rgb(116, 178, 255); +pub const ACCENT: Color = Role::Accent.colour(); /// The quay of the logo. pub const STEEL: Color = Color::Rgb(92, 118, 170); /// Text on an accent background. pub const INK: Color = Color::Rgb(13, 17, 28); -pub const GOOD: Color = Color::Rgb(158, 216, 140); -pub const WARN: Color = Color::Rgb(240, 198, 116); -pub const BAD: Color = Color::Rgb(242, 128, 150); +pub const GOOD: Color = Role::Good.colour(); +pub const WARN: Color = Role::Warn.colour(); +pub const BAD: Color = Role::Bad.colour(); + +/// The mark in front of the selected row, in the screens and in a prompt. +pub const CURSOR: &str = "▌ "; /// The current tab. pub fn plate() -> Style { @@ -69,7 +221,7 @@ pub fn popup(title: &str, colour: Color) -> Block<'static> { /// The mark in front of the selected row. pub fn cursor() -> Line<'static> { - Line::styled("▌ ", Style::new().fg(ACCENT)) + Line::styled(CURSOR, Style::new().fg(ACCENT)) } /// Key hints such as `↑↓ select · Enter open`: the key stands out, the rest recedes. @@ -107,7 +259,10 @@ pub fn logo() -> Vec> { let mut name = row("█ ", "▄▄", " █", 1.0); name.push(Span::styled("berth", Style::new().fg(ACCENT).bold())); let mut about = row("█ ", "██", " █", 0.6); - about.push(Span::styled("set up and look after agent machines", muted)); + about.push(Span::styled( + "set up and look after agent workstations", + muted, + )); let mut version = row("▀▀", "▀▀", "▀▀", 0.0); version.push(Span::styled(format!("v{}", berth_api::VERSION), muted)); vec![Line::from(name), Line::from(about), Line::from(version)] @@ -145,3 +300,63 @@ pub fn swell(width: u16, seconds: f32) -> Vec> { } vec![Line::from(crests), Line::from(body)] } + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn every_depth_has_a_colour_for_the_accent_and_the_states() { + for role in [Role::Accent, Role::Good, Role::Warn, Role::Bad] { + for depth in [Depth::Basic, Depth::Indexed, Depth::True] { + assert!(colour(role, depth).is_some(), "{role:?} at {depth:?}"); + assert!(!sgr(role, depth).is_empty(), "{role:?} at {depth:?}"); + } + assert_eq!(colour(role, Depth::None), None); + assert_eq!(sgr(role, Depth::None), ""); + } + } + + #[test] + fn the_screens_and_the_command_line_share_the_accent() { + assert_eq!(colour(Role::Accent, Depth::True), Some(ACCENT)); + assert_eq!(sgr(Role::Accent, Depth::True), "38;2;116;178;255"); + assert_eq!(sgr(Role::Accent, Depth::Basic), "94"); + assert_eq!(sgr(Role::Bad, Depth::Basic), "91"); + assert_eq!(sgr(Role::Good, Depth::Basic), "32"); + assert_eq!(sgr(Role::Warn, Depth::Basic), "33"); + assert_eq!(sgr(Role::Muted, Depth::Basic), "2"); + assert_eq!(sgr(Role::Selection, Depth::True), "48;2;24;31;48"); + } + + #[test] + fn colours_come_near_in_256() { + assert_eq!(indexed(0, 0, 0), 16); + assert_eq!(indexed(255, 255, 255), 231); + // The accent is a light blue of the cube. + assert_eq!(indexed(116, 178, 255), 111); + assert_eq!(indexed(128, 128, 128), 244); + } + + #[test] + fn the_depth_follows_the_terminal() { + let env = |pairs: &'static [(&'static str, &'static str)]| { + move |name: &str| { + pairs + .iter() + .find(|(key, _)| *key == name) + .map(|(_, value)| value.to_string()) + } + }; + assert_eq!( + Depth::of_terminal(env(&[("COLORTERM", "truecolor")])), + Depth::True + ); + assert_eq!( + Depth::of_terminal(env(&[("TERM", "xterm-256color")])), + Depth::Indexed + ); + assert_eq!(Depth::of_terminal(env(&[("TERM", "linux")])), Depth::Basic); + assert_eq!(Depth::of_terminal(env(&[("TERM", "dumb")])), Depth::None); + } +} diff --git a/crates/berth-web/Cargo.toml b/crates/berth-web/Cargo.toml deleted file mode 100644 index 7ffe444..0000000 --- a/crates/berth-web/Cargo.toml +++ /dev/null @@ -1,29 +0,0 @@ -[package] -name = "berth-web" -version.workspace = true -edition.workspace = true -rust-version.workspace = true -license.workspace = true -description = "berth in a browser: the hosts of a berth serve, drawn by ratzilla" - -# Only a browser runs it; berth's own build makes the wasm and carries it. -[[bin]] -name = "berth-web" -path = "src/main.rs" - -[target.'cfg(target_arch = "wasm32")'.dependencies] -berth-page.workspace = true -berth-tui.workspace = true -ratzilla = "0.3.1" -wasm-bindgen = "=0.2.129" -js-sys = "0.3" -console_error_panic_hook = "0.1" -# ratatui locks through critical-section; a page has one thread, and std's lock does. -critical-section = { version = "1", features = ["std"] } -web-sys = { version = "0.3", features = [ - "BinaryType", "Blob", "Clipboard", "ClipboardEvent", "CloseEvent", "DataTransfer", "Document", "DomRect", - "Element", "Event", "EventTarget", "File", "FileList", "Headers", "History", "HtmlElement", "HtmlInputElement", - "KeyboardEvent", "Location", - "MessageEvent", "MouseEvent", "Navigator", "RequestInit", "ResizeObserver", "Response", "Storage", "WebSocket", "WheelEvent", - "Window", -] } diff --git a/crates/berth-web/index.html b/crates/berth-web/index.html deleted file mode 100644 index ed88a6b..0000000 --- a/crates/berth-web/index.html +++ /dev/null @@ -1,35 +0,0 @@ - - - - - - - -berth - - - - -

- - - - diff --git a/crates/berth-web/src/browser.rs b/crates/berth-web/src/browser.rs deleted file mode 100644 index 63fa8b0..0000000 --- a/crates/berth-web/src/browser.rs +++ /dev/null @@ -1,511 +0,0 @@ -//! What the browser does for the page: its keys and mouse as the screens -//! take them, the clipboard, the token it keeps, and the canvas. - -use berth_page::App; -use berth_page::device::{Answer, Device, Kept, Linked, Said, Socket}; -use berth_tui::{Click, Code, Event, Key, Mouse}; -use ratzilla::backend::webgl2::WebGl2BackendOptions; -use ratzilla::ratatui::{Terminal, style::Color}; -use ratzilla::{CellSized, FontAtlasConfig, WebGl2Backend, WebRenderer}; -use std::cell::RefCell; -use std::rc::Rc; -use wasm_bindgen::{JsCast, prelude::*}; -use web_sys::{KeyboardEvent, window}; - -/// Where the browser keeps what the page keeps between visits. -fn item(key: Kept) -> &'static str { - match key { - Kept::Token => "berth.token", - Kept::Server => "berth.server", - } -} - -/// The fonts the page draws with, the first the browser has. -const FONTS: &[&str] = &[ - "JetBrains Mono", - "JetBrainsMono Nerd Font Mono", - "Cascadia Mono", - "DejaVu Sans Mono", - "Menlo", - "Consolas", - "monospace", -]; - -pub struct Browser; - -/// Let the user choose a file of this device, and hand over its name and -/// bytes once the browser has read it. Nothing happens when none is chosen. -fn choose_file(accept: &str, then: Box)>) { - let Some(element) = window() - .and_then(|window| window.document()) - .and_then(|document| document.create_element("input").ok()) - else { - return; - }; - let input: web_sys::HtmlInputElement = element.unchecked_into(); - input.set_type("file"); - input.set_accept(accept); - let chosen = input.clone(); - let changed = Closure::once_into_js(move || { - let Some(file) = chosen.files().and_then(|files| files.get(0)) else { - return; - }; - let name = file.name(); - let read = Closure::once_into_js(move |buffer: JsValue| { - then(name, js_sys::Uint8Array::new(&buffer).to_vec()); - }); - let reading: JsValue = file.array_buffer().into(); - if let Ok(then) = js_sys::Reflect::get(&reading, &"then".into()) { - let _ = then - .unchecked_into::() - .call1(&reading, &read); - } - }); - input.set_onchange(Some(changed.unchecked_ref())); - input.click(); -} - -fn storage() -> Option { - window()?.local_storage().ok()? -} - -/// Whether `berth serve` gave this page itself, so that it is the server. -/// A copy hosted elsewhere, as on the documentation site, asks which server. -fn embedded() -> bool { - window() - .and_then(|window| window.document()) - .and_then(|document| { - document - .query_selector("meta[name=berth-server]") - .ok() - .flatten() - }) - .and_then(|meta| meta.get_attribute("content")) - .is_some_and(|content| content == "self") -} - -/// Undo the percent-encoding of a value in an address. -fn decoded(value: &str) -> String { - js_sys::decode_uri_component(value) - .ok() - .and_then(|text| text.as_string()) - .unwrap_or_else(|| value.to_owned()) -} - -/// What the link to this page carried. A token or a setup key is taken out -/// of the address so that it is not left in the bar or the history. -fn linked() -> Linked { - let Some(window) = window() else { - return Linked::default(); - }; - let location = window.location(); - let (search, hash) = ( - location.search().unwrap_or_default(), - location.hash().unwrap_or_default(), - ); - let parts = |text: &str| -> Vec { - text.trim_start_matches(['?', '#']) - .split('&') - .filter(|part| !part.is_empty()) - .map(str::to_owned) - .collect() - }; - let (query, fragment) = (parts(&search), parts(&hash)); - let value = |name: &str| { - query.iter().chain(&fragment).find_map(|part| { - let (key, value) = part.split_once('=')?; - (key == name && !value.is_empty()).then(|| decoded(value)) - }) - }; - let linked = Linked { - server: value("server"), - token: value("token"), - setup_key: value("setup-key"), - }; - if linked.token.is_some() || linked.setup_key.is_some() { - let secret = |part: &str| part.starts_with("token=") || part.starts_with("setup-key="); - let keep = |parts: &[String], mark: &str| { - let kept: Vec<&str> = parts - .iter() - .filter(|part| !secret(part)) - .map(String::as_str) - .collect(); - match kept.is_empty() { - true => String::new(), - false => format!("{mark}{}", kept.join("&")), - } - }; - let bare = format!( - "{}{}{}", - location.pathname().unwrap_or_default(), - keep(&query, "?"), - keep(&fragment, "#") - ); - if let Ok(history) = window.history() { - let _ = history.replace_state_with_url(&JsValue::NULL, "", Some(&bare)); - } - } - linked -} - -/// Call `then` on the outcome of a promise: what it gave, or why it failed. -fn settle(promise: &JsValue, then: impl FnOnce(Result) + 'static) { - let then = Rc::new(RefCell::new(Some(then))); - let failed = then.clone(); - let given = Closure::once_into_js(move |value: JsValue| { - if let Some(then) = then.borrow_mut().take() { - then(Ok(value)); - } - }); - let refused = Closure::once_into_js(move |error: JsValue| { - if let Some(then) = failed.borrow_mut().take() { - let said = js_sys::Reflect::get(&error, &"message".into()) - .ok() - .and_then(|message| message.as_string()) - .unwrap_or_else(|| "the browser did not send the request".into()); - then(Err(said)); - } - }); - if let Ok(method) = js_sys::Reflect::get(promise, &"then".into()) { - let _ = method - .unchecked_into::() - .call2(promise, &given, &refused); - } -} - -/// One plain HTTP request with a bearer, and then its status and body. For -/// what the portal cannot carry because the caller has no token yet. -fn request(method: &str, url: &str, bearer: &str, then: Answer) { - let Some(window) = window() else { - return then(Err("there is no window".into())); - }; - let init = web_sys::RequestInit::new(); - init.set_method(method); - if let Ok(headers) = web_sys::Headers::new() { - let _ = headers.set("Authorization", &format!("Bearer {bearer}")); - init.set_headers(&headers); - } - let sent: JsValue = window.fetch_with_str_and_init(url, &init).into(); - settle(&sent, move |answer| { - let response: web_sys::Response = match answer { - Ok(response) => response.unchecked_into(), - Err(error) => return then(Err(error)), - }; - let status = response.status(); - match response.text() { - Ok(text) => settle(&text.into(), move |body| { - then(body.map(|body| (status, body.as_string().unwrap_or_default()))) - }), - Err(_) => then(Ok((status, String::new()))), - } - }); -} - -/// The address this page came from: `http://host:port`. -fn origin() -> String { - window() - .and_then(|window| window.location().origin().ok()) - .unwrap_or_default() -} - -/// Where this page is: its origin and path, without what follows them. -fn page() -> String { - window() - .map(|window| { - let location = window.location(); - format!( - "{}{}", - location.origin().unwrap_or_default(), - location.pathname().unwrap_or_default() - ) - }) - .unwrap_or_default() -} - -impl Device for Browser { - fn copy(&self, text: &str) -> bool { - match window() { - Some(window) => { - let _ = window.navigator().clipboard().write_text(text); - true - } - None => false, - } - } - - /// Pasting is the browser's: its `paste` event carries the text. - fn paste(&self) -> Option { - None - } - - fn open(&self, link: &str) -> Result<(), String> { - let window = window().ok_or("there is no window")?; - match window.open_with_url_and_target(link, "_blank") { - Ok(Some(_)) => Ok(()), - _ => Err("the browser kept the tab from opening".into()), - } - } - - fn choose_file( - &self, - accept: &str, - then: Box)>, - ) -> Result<(), String> { - choose_file(accept, then); - Ok(()) - } - - fn stored(&self, key: Kept) -> Option { - storage()?.get_item(item(key)).ok()? - } - - fn store(&self, key: Kept, value: &str) { - if let Some(storage) = storage() { - let _ = match value.is_empty() { - true => storage.remove_item(item(key)), - false => storage.set_item(item(key), value), - }; - } - } - - fn embedded(&self) -> bool { - embedded() - } - - fn linked(&self) -> Linked { - linked() - } - - fn origin(&self) -> String { - origin() - } - - fn page(&self) -> String { - page() - } - - fn request(&self, method: &str, url: &str, bearer: &str, then: Answer) { - request(method, url, bearer, then) - } - - fn connect( - &self, - url: &str, - token: &str, - said: Box, - ) -> Result, String> { - let socket = crate::socket::open(url, token, said)?; - Ok(Box::new(socket)) - } -} - -/// The key a `keydown` was; `None` for keys the screens have no use for. -fn key_of(event: &KeyboardEvent) -> Option { - let mut name = event.key(); - // Option on a Mac types a character of its own, such as ® for R, where the - // screens want the letter of the key. AltGr, which adds Ctrl, types text. - if event.alt_key() - && !event.ctrl_key() - && let Some(letter) = event.code().strip_prefix("Key") - && letter.len() == 1 - && !(name.len() == 1 && name.chars().all(|ch| ch.is_ascii_alphabetic())) - { - name = letter.to_lowercase(); - } - let mut chars = name.chars(); - let code = match (chars.next(), chars.next()) { - (Some(ch), None) => Code::Char(ch), - _ => match name.as_str() { - "Enter" => Code::Enter, - "Escape" => Code::Esc, - "Backspace" => Code::Backspace, - "Delete" => Code::Delete, - "Insert" => Code::Insert, - "Tab" if event.shift_key() => Code::BackTab, - "Tab" => Code::Tab, - "ArrowUp" => Code::Up, - "ArrowDown" => Code::Down, - "ArrowLeft" => Code::Left, - "ArrowRight" => Code::Right, - "Home" => Code::Home, - "End" => Code::End, - "PageUp" => Code::PageUp, - "PageDown" => Code::PageDown, - other => Code::F(other.strip_prefix('F')?.parse().ok()?), - }, - }; - Some(Key { - code, - ctrl: event.ctrl_key(), - alt: event.alt_key(), - shift: event.shift_key(), - }) -} - -/// Keys whose answer is the browser's own: its developer tools, a reload, -/// full screen, and anything with the Meta key. -fn leaves_to_browser(event: &KeyboardEvent) -> bool { - let key = event.key().to_lowercase(); - event.meta_key() - || matches!(key.as_str(), "f5" | "f11" | "f12") - || (event.ctrl_key() && event.shift_key() && matches!(key.as_str(), "i" | "j" | "c" | "r")) - || (event.ctrl_key() && matches!(key.as_str(), "r" | "+" | "-" | "0")) -} - -fn canvas() -> Option { - window()? - .document()? - .query_selector("#terminal canvas") - .ok()? -} - -/// Size and place the canvas inside `#terminal` for cells of `cell` CSS -/// pixels: a whole number of cells, centred. ratzilla follows on its next frame. -fn fit(cell: (f32, f32)) { - let Some(container) = window() - .and_then(|window| window.document()) - .and_then(|document| document.get_element_by_id("terminal")) - else { - return; - }; - let Some(canvas) = canvas() else { - return; - }; - let (width, height) = ( - container.client_width() as f32, - container.client_height() as f32, - ); - let whole = |space: f32, cell: f32| match cell > 0.0 { - true => (space / cell).floor().max(1.0) * cell, - false => space, - }; - let (w, h) = (whole(width, cell.0), whole(height, cell.1)); - let _ = canvas.set_attribute( - "style", - &format!( - "position:absolute;display:block;left:{}px;top:{}px;width:{w}px;height:{h}px", - ((width - w) / 2.0).max(0.0).floor(), - ((height - h) / 2.0).max(0.0).floor() - ), - ); -} - -fn listen(target: &web_sys::EventTarget, name: &str, handler: impl FnMut(E) + 'static) -where - E: wasm_bindgen::convert::FromWasmAbi + 'static, -{ - let closure = Closure::::new(handler); - let _ = target.add_event_listener_with_callback(name, closure.as_ref().unchecked_ref()); - // The page keeps its listeners for as long as it is open. - closure.forget(); -} - -/// The cell of the canvas a mouse event is over. -fn cell_of(event: &web_sys::MouseEvent, cell: (f32, f32)) -> Option<(u16, u16)> { - let area = canvas()?.get_bounding_client_rect(); - let (x, y) = ( - event.client_x() as f64 - area.left(), - event.client_y() as f64 - area.top(), - ); - let inside = x >= 0.0 && y >= 0.0 && x < area.width() && y < area.height(); - (inside && cell.0 > 0.0 && cell.1 > 0.0) - .then(|| ((x / cell.0 as f64) as u16, (y / cell.1 as f64) as u16)) -} - -pub fn start() { - console_error_panic_hook::set_once(); - berth_page::device::install(Browser); - let Some(window) = window() else { - return; - }; - let app = Rc::new(RefCell::new(App::new())); - - let keys = app.clone(); - listen(&window, "keydown", move |event: KeyboardEvent| { - // Pasting is the browser's: a prevented key would not give its text. - if leaves_to_browser(&event) || (event.ctrl_key() && event.key().eq_ignore_ascii_case("v")) - { - return; - } - let Some(key) = key_of(&event) else { - return; - }; - event.prevent_default(); - keys.borrow_mut().event(Event::Key(key)); - }); - let pasted = app.clone(); - listen(&window, "paste", move |event: web_sys::ClipboardEvent| { - if let Some(text) = event - .clipboard_data() - .and_then(|data| data.get_data("text").ok()) - { - event.prevent_default(); - pasted.borrow_mut().event(Event::Paste(text)); - } - }); - - let container = window - .document() - .and_then(|document| document.get_element_by_id("terminal")); - let (width, height) = container.as_ref().map_or((960, 600), |container| { - ( - container.client_width().max(320) as u32, - container.client_height().max(200) as u32, - ) - }); - // The browser draws the glyphs, so box drawing and arrows come out of - // whatever monospace font it has. - let backend = WebGl2Backend::new_with_options( - WebGl2BackendOptions::new() - .grid_id("terminal") - .size((width, height)) - .disable_auto_css_resize() - .canvas_padding_color(Color::Rgb(13, 17, 28)) - .font_atlas_config(FontAtlasConfig::dynamic(FONTS, 15.0)), - ) - .expect("the page has a #terminal element and WebGL2"); - let cell = backend.cell_size_css_px(); - fit(cell); - - if let Some(container) = &container { - for name in ["mousedown", "wheel"] { - let app = app.clone(); - listen(container, name, move |event: web_sys::MouseEvent| { - let Some((column, row)) = cell_of(&event, cell) else { - return; - }; - let click = match event.dyn_ref::() { - Some(wheel) if wheel.delta_y() < 0.0 => Click::ScrollUp, - Some(wheel) if wheel.delta_y() > 0.0 => Click::ScrollDown, - Some(_) => return, - None => match event.button() { - 0 => Click::Left, - 2 => Click::Right, - _ => return, - }, - }; - // The page's own wheel and selection are not wanted over the screen. - event.prevent_default(); - app.borrow_mut() - .event(Event::Mouse(Mouse { click, column, row })); - }); - } - // A right click copies a value here; it is not the browser's menu. - listen(container, "contextmenu", |event: web_sys::Event| { - event.prevent_default() - }); - // The container's size is watched itself: a zoom or a font that loads - // late changes it without a `resize`. - let resized = Closure::::new(move || fit(cell)).into_js_value(); - if let Ok(observer) = web_sys::ResizeObserver::new(resized.unchecked_ref()) { - observer.observe(container); - } - } - - let terminal = Terminal::new(backend).expect("the terminal starts"); - terminal.draw_web(move |frame| { - let mut app = app.borrow_mut(); - app.tick(); - app.draw(frame); - }); -} diff --git a/crates/berth-web/src/main.rs b/crates/berth-web/src/main.rs deleted file mode 100644 index 4be3e9b..0000000 --- a/crates/berth-web/src/main.rs +++ /dev/null @@ -1,19 +0,0 @@ -//! berth in a browser: the hosts of the `berth serve` that served this page, -//! through its portal. The screens are berth-page's; this program gives them -//! the browser's canvas, keys, clipboard, storage and WebSocket. The page -//! keeps nothing but the token; every host is the server's. - -#[cfg(target_arch = "wasm32")] -mod browser; -#[cfg(target_arch = "wasm32")] -mod socket; - -#[cfg(target_arch = "wasm32")] -fn main() { - browser::start(); -} - -#[cfg(not(target_arch = "wasm32"))] -fn main() { - eprintln!("berth-web is the page berth serve shows: building berth builds it for wasm32"); -} diff --git a/crates/berth-web/src/socket.rs b/crates/berth-web/src/socket.rs deleted file mode 100644 index 7ec680d..0000000 --- a/crates/berth-web/src/socket.rs +++ /dev/null @@ -1,72 +0,0 @@ -//! The browser's WebSocket under the page's portal. - -use berth_page::device::{Said, Socket}; -use std::rc::Rc; -use wasm_bindgen::{JsCast, prelude::*}; -use web_sys::{BinaryType, MessageEvent, WebSocket}; - -pub struct BrowserSocket(WebSocket); - -impl Drop for BrowserSocket { - fn drop(&mut self) { - for name in ["onopen", "onmessage", "onclose"] { - let _ = js_sys::Reflect::set(&self.0, &name.into(), &JsValue::NULL); - } - let _ = self.0.close(); - } -} - -impl Socket for BrowserSocket { - fn send_text(&self, text: &str) -> bool { - self.0.send_with_str(text).is_ok() - } - - fn send_bytes(&self, bytes: &[u8]) -> bool { - self.0.send_with_u8_array(bytes).is_ok() - } -} - -/// Open a WebSocket to `url`, showing the token the way a browser can: as -/// its protocol. -pub fn open(url: &str, token: &str, said: Box) -> Result { - let socket = match token.is_empty() { - true => WebSocket::new(url), - false => WebSocket::new_with_str(url, token), - } - .map_err(|_| match url.starts_with("ws://") { - // A browser does not let a page over https open plain ws. - true => { - "The browser did not open it: from a page over https, use an https address.".to_owned() - } - false => "The browser did not open that address.".to_owned(), - })?; - socket.set_binary_type(BinaryType::Arraybuffer); - let said: Rc = Rc::from(said); - let listen = |name: &str, handler: Box Option>| { - let said = said.clone(); - let closure = Closure::::new(move |event: JsValue| { - if let Some(what) = handler(event) { - said(what); - } - }); - let _ = js_sys::Reflect::set(&socket, &name.into(), closure.as_ref()); - // The socket keeps it for as long as the page has it. - closure.forget(); - }; - listen("onopen", Box::new(|_| Some(Said::Open))); - listen("onclose", Box::new(|_| Some(Said::Closed))); - listen( - "onmessage", - Box::new(|event| { - let data = event.dyn_into::().ok()?.data(); - if let Some(text) = data.as_string() { - Some(Said::Text(text)) - } else if data.is_instance_of::() { - Some(Said::Bytes(js_sys::Uint8Array::new(&data).to_vec())) - } else { - None - } - }), - ); - Ok(BrowserSocket(socket)) -} diff --git a/crates/berth/Cargo.toml b/crates/berth/Cargo.toml index bbbf1a1..80cd42f 100644 --- a/crates/berth/Cargo.toml +++ b/crates/berth/Cargo.toml @@ -5,42 +5,37 @@ edition.workspace = true rust-version.workspace = true license.workspace = true +# The screens and the client of a berth serve, on every system. [dependencies] -berth-core.workspace = true berth-api = { workspace = true, features = ["client"] } -berth-pty.workspace = true +berth-page.workspace = true berth-tui.workspace = true -# For its API document, which berth serve's includes. -berthd = { path = "../berthd" } tokio.workspace = true -futures-util.workspace = true +serde_json = "1" serde.workspace = true +tempfile = "3" +crossterm = "=0.29.0" +# The command line: its grammar, help, errors and shell completions. +clap = { version = "4.5", features = ["derive", "wrap_help"] } +clap_complete = "4.5" +# The questions and progress drawn inline, below the command. +ratatui = { version = "=0.30.2", default-features = false, features = ["crossterm"] } +tui-input.workspace = true +futures-util.workspace = true + +# The installation, berth serve and the commands, on Linux, where hosts are +# set up from. +[target.'cfg(target_os = "linux")'.dependencies] +berth-core.workspace = true +# For its API document, which berth serve's includes. +berthd = { path = "../berthd" } axum.workspace = true utoipa.workspace = true utoipa-axum.workspace = true -ratatui = { version = "=0.30.2", default-features = false, features = ["crossterm"] } -crossterm = "=0.29.0" -serde_json = "1" -tempfile = "3" -vt100 = "=0.16.2" libc = "0.2" -pulldown-cmark = { version = "0.13", default-features = false } jsonwebtoken = { version = "11", default-features = false, features = ["rust_crypto"] } -tui-input = { workspace = true, features = ["crossterm"] } - -[features] -# The page for a browser, carried in the program for `berth serve --web`. -# Building it needs the wasm32-unknown-unknown target. -web = ["dep:wasm-bindgen-cli-support"] - -[build-dependencies] -# The same version as the wasm-bindgen the page is built with. -wasm-bindgen-cli-support = { version = "=0.2.129", optional = true } - -[dev-dependencies] +[target.'cfg(target_os = "linux")'.dev-dependencies] # Signs the access tokens of the tests. ring = "0.17" -# berth-remote signs in to a berth serve of the tests. -berth-page.workspace = true -berth-remote = { path = "../berth-remote" } + diff --git a/crates/berth/build.rs b/crates/berth/build.rs deleted file mode 100644 index cc58ce1..0000000 --- a/crates/berth/build.rs +++ /dev/null @@ -1,127 +0,0 @@ -//! With the `web` feature, builds the page `berth serve --web` shows in a -//! browser: berth-web for wasm32, and the JavaScript that loads it, into -//! OUT_DIR for the program to carry. It needs the target once: -//! `rustup target add wasm32-unknown-unknown`. -//! -//! A release build of berth carries the page of the `web` profile. Any other -//! build carries that of `web-dev`, which is built in seconds rather than -//! linked whole again after each change. BERTH_WEB_PROFILE names another. - -#[cfg(not(feature = "web"))] -fn main() {} - -#[cfg(feature = "web")] -fn main() { - page::build(); -} - -#[cfg(feature = "web")] -mod page { - - use std::{ - env, fs, - hash::{DefaultHasher, Hash, Hasher}, - path::Path, - process::Command, - }; - - const TARGET: &str = "wasm32-unknown-unknown"; - - /// A cargo that knows rustup's targets: a system cargo found first on the path does not. - fn cargo() -> Command { - let rustup = |tool: &str| { - let output = Command::new("rustup").args(["which", tool]).output().ok()?; - let path = String::from_utf8(output.stdout).ok()?.trim().to_owned(); - (output.status.success() && !path.is_empty()).then_some(path) - }; - match (rustup("cargo"), rustup("rustc")) { - (Some(cargo), Some(rustc)) => { - let mut command = Command::new(cargo); - command.env("RUSTC", rustc); - command - } - _ => Command::new(env::var("CARGO").unwrap_or_else(|_| "cargo".into())), - } - } - - pub fn build() { - let out = env::var("OUT_DIR").expect("OUT_DIR"); - let manifest = env::var("CARGO_MANIFEST_DIR").expect("CARGO_MANIFEST_DIR"); - let root = Path::new(&manifest) - .ancestors() - .nth(2) - .expect("the checkout"); - for entry in [ - "crates/berth-web", - "crates/berth-page", - "crates/berth-tui", - "crates/berth-api", - "crates/vendor/tui-input", - "Cargo.lock", - ] { - println!("cargo:rerun-if-changed=../../{entry}"); - } - println!("cargo:rerun-if-env-changed=BERTH_WEB_PROFILE"); - println!("cargo:rerun-if-env-changed=BERTH_BUILD_VERSION"); - let profile = env::var("BERTH_WEB_PROFILE").unwrap_or_else(|_| { - match env::var("PROFILE").as_deref() { - Ok("release") => "web", - _ => "web-dev", - } - .into() - }); - // Its own target directory: the outer build holds the lock on the usual one. - let target = root.join("target/web"); - let mut build = cargo(); - build - .current_dir(root) - .args(["build", "--locked", "--target", TARGET]) - .args(["--profile", &profile]) - .args(["-p", "berth-web", "--target-dir"]) - .arg(&target); - // Flags and wrappers of the outer build (clippy's among them) are for another target. - for variable in [ - "CARGO_ENCODED_RUSTFLAGS", - "RUSTFLAGS", - "CARGO_BUILD_TARGET", - "TARGET", - "RUSTC_WRAPPER", - "RUSTC_WORKSPACE_WRAPPER", - ] { - build.env_remove(variable); - } - let status = build.status().expect("cargo is needed to build the page"); - assert!( - status.success(), - "building the page failed; the target is added with: rustup target add {TARGET}" - ); - // Cargo puts what the dev and test profiles build in debug/. - let folder = match profile.as_str() { - "dev" | "test" => "debug", - other => other, - }; - let wasm = target.join(TARGET).join(folder).join("berth-web.wasm"); - let page = Path::new(&out).join("web"); - let _ = fs::remove_dir_all(&page); - wasm_bindgen_cli_support::Bindgen::new() - .input_path(&wasm) - .web(true) - .expect("a page for the web") - .typescript(false) - .generate(&page) - .expect("the JavaScript that loads the page"); - // A copy at a place that does not move, for the documentation site. - let dist = root.join("target/web-dist"); - fs::create_dir_all(&dist).expect("target/web-dist"); - for name in ["berth-web.js", "berth-web_bg.wasm"] { - fs::copy(page.join(name), dist.join(name)).expect("copying the page"); - } - // Named in the page's addresses, so a browser never runs the script of one - // build with the wasm of another. - let mut hasher = DefaultHasher::new(); - fs::read(page.join("berth-web_bg.wasm")) - .expect("the page's wasm") - .hash(&mut hasher); - println!("cargo:rustc-env=BERTH_WEB_BUILD={:016x}", hasher.finish()); - } -} diff --git a/crates/berth/src/app.rs b/crates/berth/src/app.rs deleted file mode 100644 index 7d7849c..0000000 --- a/crates/berth/src/app.rs +++ /dev/null @@ -1,2799 +0,0 @@ -//! Wizard state: the profile being edited, what each step shows, and running the chosen action. - -use crate::{ - Result, credentials, - discovery::{self, Choices, expand_path}, - fields::{Action, Field, Kind, Step, TASKS}, - job::{Job, Stage}, - picker::Picker, - plugins::{self, Plugin}, - remote::{HostView, State}, - theme, -}; -use berth_core::{ - apply, - host::Host, - level, - paths::{Paths, layout}, - profiles::Place, -}; -use berth_tui::host::Pick; -use berth_tui::machine::{self, Machine, Next}; -use berth_tui::setup::{self, set_pointer}; -pub use berth_tui::setup::{FeatureGroup, Item, value_line}; -use ratatui::{style::Style, text::Line, widgets::ListState}; -use serde_json::{Value, json}; -use std::{io::Write, path::PathBuf, process::Command, sync::mpsc}; -use tui_input::Input; - -/// What a selectable row does when activated. -#[derive(Clone, Copy, PartialEq, Eq, Debug)] -pub enum Entry { - Action(Action), - Field(Field), - /// Something the overview of a host offers, by name. - Task(&'static str), - /// The heading of a group on the Features page. - Group(usize), - /// A plugin's feature, switched on or off for this host: its index among them. - Feature(usize), - /// A task of the host, by index into what it listed. - HostTask(usize), - /// A session on the host, by index into what it listed. - HostSession(usize), - /// A registry this host can be given, by index into `App::registry_names`. - HostRegistry(usize), - /// A secret this host can be given, by index into `App::secret_names`. - HostSecret(usize), - /// One of an account's SSH keys, or the row that adds one. - Key(setup::Account, Option), - /// A row of the machine berth creates for the host. - Machine(machine::Pick), - /// Open the set-up flow. - Setup, - Run, - /// Save the profile, as Ctrl+S does. - Save, -} - -pub struct Row { - pub line: Line<'static>, - /// `None` for headings and other rows the cursor skips. - pub entry: Option, -} - -impl Row { - fn info(line: impl Into>) -> Self { - Row { - line: line.into(), - entry: None, - } - } - fn entry(line: impl Into>, entry: Entry) -> Self { - Row { - line: line.into(), - entry: Some(entry), - } - } -} - -pub struct App { - pub config: Value, - /// The profile as it was last saved, for detecting unsaved changes. - saved: Value, - /// Where the profile is kept. - pub place: Place, - /// The installation this berth is. - pub paths: Paths, - pub step: Step, - /// What Run does: the install method, or `Manage` while on the Manage tab. - pub action: Action, - /// Action for the next `execute`, when it differs from the tab's own. - pub terminal_action: Option, - /// A command of this machine in a terminal inside the UI: `ssh` to a host - /// that is not set up yet. - pub session: Option, - /// What the host itself says it can do, its sessions, and the panel onto one. - pub host: HostView, - /// The password dialog for the host was shown; it is not opened by itself again. - password_asked: bool, - /// The next connection was asked for, by Try again or a changed - /// address: it logs in even after the host refused the login. - force_login: bool, - /// The Terminal row opens the agent's shell and not the SSH user's. - pub as_agent: bool, - /// Plugins found in the source and in the user's plugin directory. - pub plugins: Vec, - /// The public half of this berth's own SSH key, which hosts take. For a - /// server's host it is the server's. - pub own_public: String, - /// The names of the keys of the key ring that logs in to this host: this - /// berth's, or the server's for a server's host. - pub ring_keys: Vec, - /// What those plugins can install, for the Features page. - catalogue: Vec, - /// The registries and secrets the host can be given: of this berth, or - /// what is known of its server's. Without berth's store, as in tests or - /// when it does not open, why there are none; the host is still reached. - pub access: std::result::Result, - /// The registries and secrets that exist, for the Settings tab. - pub registry_names: Vec, - pub secret_names: Vec, - /// Web addresses in the output on screen, and where they are, for clicking. - pub links: Vec<(ratatui::layout::Rect, String)>, - /// The words of that output, for copying. - pub words: Vec<(ratatui::layout::Rect, String)>, - /// The install method to return to when leaving the Manage tab. - method: Action, - /// Index into the selectable rows of the current step. - pub list: ListState, - pub editing: Option, - /// The editor holds a plain password to hash rather than a field value. - pub password_entry: bool, - pub suggestions: Choices, - pub suggestion_list: ListState, - pub suggestion_focus: bool, - pub public_keys: Choices, - pub private_keys: Choices, - pub ssh_hosts: Choices, - /// Machines on this machine's tailnet, as (description, address). - pub tailnet_hosts: Choices, - /// Wi-Fi networks saved on this machine, as (SSID, password). - pub wifi_networks: Choices, - pub status: String, - /// The SSH address is being changed to reach a host again: saving it - /// connects once more. - pub retry_address: bool, - pub picker: Option, - /// sudo is authorised, so the saved Wi-Fi passwords can be read. - pub privileged: bool, - /// The administrator password dialog, while it is open. - pub unlock: Option, - pub unlock_failed: bool, - /// What the administrator password is being asked for. - pub unlock_reason: &'static str, - /// Run the action once the password dialog succeeds. - pub pending_run: bool, - /// The password dialog is for sudo on the managed host, not this machine. - pub unlock_remote: bool, - /// Where the builder writes archives, seeds and installer images. - pub build_dir: PathBuf, - /// Where downloaded installer ISOs are kept. - pub iso_dir: PathBuf, - /// The running job downloads the installer ISO; its last line is the path. - fetching_iso: bool, - /// Only write the ISO that is already built, without building it again. - pub write_only: bool, - /// The dialog for picking one of several USB drives is open. - pub choosing_drive: bool, - /// Question that needs a yes before the action runs. - pub confirm: Option, - /// The action running inside the UI, and its output once it has ended. - pub job: Option, - /// Connected USB drives, as (description, device path). - pub usb_drives: Choices, - /// A link of the panel as a QR code, over everything else. - pub qr: Option, - /// The machine berth creates for the host, on the set-up tabs. - pub machine: Machine, - /// What is being asked of it and of the provisioners, until it answers. - machine_answers: Option>, - /// The running job is one of the machine's. - pub machine_job: bool, - /// What a berth server lists of its own actions on the host, until it answers. - own_answers: Option>>, - /// The berth command one of berth's own actions on the host runs, in - /// the normal terminal, the next time the screen is left. - pub own_command: Option>, - /// The host's new name after one of berth's own actions renamed it, so - /// its screen is opened again under that name. - pub renamed: Option, - /// What the host's controller says the host is, asked in the background - /// once its address is given on the set-up's Install tab. - found_level: Option>>, - /// The level was chosen on the Level row, so what the host says is only told. - level_chosen: bool, -} - -/// What the provisioners there are, and the host's machine, came back as. -type Answers = ( - std::result::Result, String>, - std::result::Result, String>, -); - -/// Ask where provisioners run, as `berth provision` does: the berth server -/// when one is set, otherwise here. The host's machine is its resource of the -/// machine kind, as its provisioner says it is now. -fn machine_answers(paths: &Paths, name: &str) -> Answers { - let text = |error: &dyn std::fmt::Display| error.to_string(); - let source = crate::provisioner_command::Source::of(paths); - let listed = source.list().map_err(|error| text(&error)); - let found = source - .resources(None, Some("machine"), Some(name)) - .map_err(|error| text(&error)) - .and_then(|found| match found.first() { - None => Ok(None), - Some(made) => source - .resource(&made.id) - .map(Some) - .map_err(|error| text(&error)), - }); - let found = match found { - Err(error) if machine::absent(&error) => Ok(None), - found => found, - }; - (listed, found) -} - -/// The rows of a set-up page, as rows of this screen. -fn lift(rows: Vec) -> Vec { - rows.into_iter() - .map(|row| Row { - line: row.line, - entry: row.entry.map(|entry| match entry { - setup::Entry::Action(action) => Entry::Action(action), - setup::Entry::Field(field) => Entry::Field(field), - setup::Entry::Group(index) => Entry::Group(index), - setup::Entry::Feature(index) => Entry::Feature(index), - setup::Entry::Registry(index) => Entry::HostRegistry(index), - setup::Entry::Secret(index) => Entry::HostSecret(index), - setup::Entry::Key(account, index) => Entry::Key(account, index), - setup::Entry::Machine(pick) => Entry::Machine(pick), - setup::Entry::Run => Entry::Run, - setup::Entry::Save => Entry::Save, - }), - }) - .collect() -} - -/// The title of the job that creates a host's machine. -const CREATE_MACHINE: &str = "Create machine"; - -impl App { - pub fn new(paths: Paths, place: Place, config: Value) -> Self { - let mut app = App { - saved: Value::Null, - config, - place, - paths, - step: Step::Install, - action: Action::Iso, - method: Action::Iso, - terminal_action: None, - session: None, - host: HostView::default(), - password_asked: false, - force_login: false, - as_agent: true, - links: vec![], - words: vec![], - list: ListState::default().with_selected(Some(0)), - editing: None, - password_entry: false, - suggestions: vec![], - suggestion_list: ListState::default(), - suggestion_focus: false, - public_keys: vec![], - private_keys: vec![], - ssh_hosts: vec![], - tailnet_hosts: vec![], - wifi_networks: vec![], - plugins: vec![], - own_public: String::new(), - ring_keys: vec![], - catalogue: vec![], - access: Err("berth's store is not open, so there are no secrets or registries".into()), - registry_names: vec![], - secret_names: vec![], - status: String::new(), - retry_address: false, - picker: None, - privileged: false, - unlock: None, - unlock_failed: false, - unlock_reason: "", - pending_run: false, - unlock_remote: false, - build_dir: "build".into(), - iso_dir: "build/iso".into(), - fetching_iso: false, - write_only: false, - choosing_drive: false, - confirm: None, - job: None, - usb_drives: vec![], - qr: None, - machine: Machine::default(), - machine_answers: None, - machine_job: false, - own_answers: None, - own_command: None, - renamed: None, - found_level: None, - level_chosen: false, - }; - app.saved = app.config.clone(); - app - } - - /// The profile as its set-up pages see it. - fn view(&self) -> setup::Profile<'_> { - setup::Profile { - config: &self.config, - step: self.step, - action: self.action, - features: &self.catalogue, - registries: &self.registry_names, - secrets: &self.secret_names, - own_key: &self.own_public, - named: setup::valid_hostname(&self.name()), - } - } - - pub fn text(&self, field: Field) -> &str { - self.view().text(field) - } - - /// State of a toggle. Unset feature switches are on, like the role defaults. - pub fn enabled(&self, field: Field) -> bool { - self.view().enabled(field) - } - - /// Read which registries and secrets exist. They are kept outside the profile. - pub fn load_access(&mut self) { - let Ok(store) = &self.access else { - return; - }; - self.registry_names = store - .registries() - .unwrap_or_default() - .keys() - .cloned() - .collect(); - let names = store.secret_names(); - setup::default_enrollment(&mut self.config, &names); - self.secret_names = berth_tui::setup::host_secrets(names); - } - - /// The registries and secrets, or why there are none. - pub fn access(&self) -> Result<&crate::registry::Store> { - self.access.as_ref().map_err(|error| error.as_str().into()) - } - - pub fn toggle_host_registry(&mut self, index: usize) { - setup::toggle_registry(&mut self.config, &self.registry_names[index]); - } - - /// How this host gets a secret: `GH_TOKEN` for a variable, a path for a - /// file, nothing when it does not get it. - pub fn host_secret(&self, index: usize) -> String { - self.view().secret_how(index) - } - - pub fn set_host_secret(&mut self, index: usize, how: &str) { - if let Err(why) = setup::set_secret(&mut self.config, &self.secret_names[index], how) { - self.status = why.into(); - } - } - - /// Read the plugins of the source (the working directory) and the user's own. - pub fn load_plugins(&mut self, user_dir: PathBuf) { - self.set_plugins(plugins::load(&["plugins".into(), user_dir])); - } - - pub fn set_plugins(&mut self, plugins: Vec) { - self.catalogue = plugins - .iter() - .flat_map(|plugin| { - plugin.features.iter().map(|feature| setup::Feature { - key: format!("{}/{}", plugin.id, feature.id), - label: feature.label.clone(), - group: feature.group.clone(), - help: feature.help.to_owned(), - }) - }) - .collect(); - self.plugins = plugins; - } - - /// What a server's host can have installed: what the server's plugins - /// offer, as `setup.catalogue` lists it. - pub fn set_features(&mut self, features: Vec) { - self.catalogue = features; - } - - /// The groups of the Features page: the built-in ones, with what plugins add - /// to them, then groups only plugins have. - pub fn feature_groups(&self) -> Vec { - self.view().feature_groups() - } - - pub fn item_enabled(&self, item: Item) -> bool { - self.view().item_enabled(item) - } - - pub fn set_item(&mut self, item: Item, on: bool) { - match item { - Item::Field(field) => self.set_toggle(field, on), - Item::Feature(index) => { - setup::set_feature(&mut self.config, &self.catalogue, index, on) - } - } - } - - /// The group whose heading is selected. - pub fn selected_group(&self) -> Option { - match self.selected_entry() { - Some(Entry::Group(index)) => Some(index), - _ => None, - } - } - - /// Whether everything in a group is on, which is when switching it turns it off. - pub fn group_all_on(&self, index: usize) -> bool { - self.view().group_all_on(index) - } - - /// Switch everything in a group on, or off when all of it is on already. - pub fn toggle_group(&mut self, index: usize) { - let on = !self.group_all_on(index); - let group = self.feature_groups().swap_remove(index); - for item in group.items { - self.set_item(item, on); - } - self.status = format!("{}: all {}", group.title, if on { "on" } else { "off" }); - } - - /// Set a toggle. The coding agents share the older switch for all of them, - /// which the role still reads: it is on while any agent is. - fn set_toggle(&mut self, field: Field, on: bool) { - self.set(field, json!(on)); - } - - fn set(&mut self, field: Field, value: Value) { - set_pointer(&mut self.config, field.spec().pointer, value); - // Another address is another host to ask. - use Field::*; - if matches!(field, SshHost | SshUser | SshPort | Login | AskBecomePass) { - self.reconnect_host(); - } - } - - /// The profile as it is kept now, filled in as the screen expects it. - fn stored(&self) -> Option { - let profile = self.place.load(&self.paths).ok()??; - parse_config(profile.to_string().as_bytes()).ok() - } - - /// Take the profile in as it is kept now, when it changed. Without unsaved - /// changes it replaces the one on screen, and the host is asked at its - /// address again if that changed. Returns whether it was taken in, or - /// `None` when nothing changed. - fn take_stored(&mut self) -> Option<(bool, Value)> { - let profile = self.stored()?; - if profile == self.saved { - return None; - } - if self.unsaved() { - return Some((false, profile)); - } - let moved = profile["access"] != self.config["access"]; - self.config = profile.clone(); - self.saved = profile; - if moved { - self.reconnect_host(); - } - Some((true, Value::Null)) - } - - /// Read the profile again, after a mirror rewrote it. Returns what to say - /// when something is worth it. - pub fn take_in_mirror(&mut self) -> Option { - match self.take_stored()? { - (true, _) => Some("Took in the berth server's copy of this host".into()), - (false, _) => { - Some("The berth server's copy of this host changed. Saving sends this one".into()) - } - } - } - - /// Read the profile again after one of berth's own commands ran from the - /// screen, which may have saved what it found: the machine it created - /// gives the host its address. With unsaved changes those fields are - /// taken into the changes, which stay unsaved. - pub fn take_in_command(&mut self) -> Option { - let (taken, profile) = self.take_stored()?; - if taken { - return Some("Took in what the command saved of this host".into()); - } - let mut changed = false; - for pointer in [ - "/access/address", - "/access/port", - "/access/user", - "/access/auth", - "/profile", - ] { - let kept = profile.pointer(pointer).cloned().unwrap_or(Value::Null); - if self.config.pointer(pointer) != Some(&kept) && !kept.is_null() { - set_pointer(&mut self.config, pointer, kept); - changed = true; - } - } - // What is saved now is the command's; the rest of the changes are still to save. - self.saved = profile; - if changed { - self.reconnect_host(); - } - Some(match changed { - true => { - "Took in the address the command saved; the other changes are not saved yet".into() - } - false => "The command saved this host too; saving keeps the changes made here".into(), - }) - } - - pub fn unsaved(&self) -> bool { - self.config != self.saved - } - - /// Fields shown by the current step for the chosen action. - #[cfg(test)] - pub fn fields(&self) -> Vec { - self.view().fields() - } - - fn field_rows(&self) -> Vec { - lift(self.view().field_rows()) - } - - /// What the Run tab says besides the shared rows: whether the Wi-Fi - /// password is the one this machine knows. - fn run_notes(&self) -> Vec { - if !(self.action.installer() && self.enabled(Field::Wifi)) { - return vec![]; - } - let (text, color) = match self.wifi_password_matches() { - Some(true) => ( - "✓ Wi-Fi password matches the one saved on this machine", - theme::GOOD, - ), - Some(false) => ( - "! Wi-Fi password differs from the one saved on this machine", - theme::WARN, - ), - None => ( - "· Wi-Fi login not checked: this machine has not saved that network", - theme::MUTED, - ), - }; - vec![setup::Row::info(Line::styled(text, Style::new().fg(color)))] - } - - pub fn rows(&self) -> Vec { - let heading = |text: &'static str| { - Row::info(Line::styled(text, Style::new().fg(theme::ACCENT).bold())) - }; - let mut rows = vec![]; - match self.step { - Step::Install | Step::Features | Step::Settings | Step::Run => { - let problems = self.placed_problems(); - let page = setup::Page { - methods: self.methods(), - machine: &self.machine, - problems: &problems, - notes: self.run_notes(), - }; - rows = lift(self.view().page_rows(page)); - // The drive to write to is named as this machine knows it. - for row in &mut rows { - if row.entry == Some(Entry::Field(Field::UsbDevice)) { - let device = self.text(Field::UsbDevice); - let drive = self.usb_drives.iter().find(|(_, path)| path == device); - let value = match drive { - Some((label, _)) => label.as_str(), - None if device.is_empty() => "no", - None => device, - }; - let mut line = value_line("Then write to USB drive", value, Kind::Text); - line.spans.insert(0, row.line.spans[0].clone()); - row.line = line; - } - } - } - Step::Manage => { - if let Some(line) = self.host.stats_line() { - rows.push(Row::info(Line::styled(line, Style::new().fg(theme::MUTED)))); - rows.push(Row::info("")); - } - // berthd and box come with berth, so a newer berth has newer ones to give. - if let State::Ready(version) = &self.host.state - && version != berth_api::VERSION - && self.served().is_none() - { - rows.push(Row::info(Line::styled( - format!( - "The host's tools are {version}, this berth is {}. Apply this configuration updates them.", - berth_api::VERSION - ), - Style::new().fg(theme::WARN), - ))); - rows.push(Row::info("")); - } - rows.push(heading("Connection")); - for line in berth_tui::host::level_lines(&self.name(), &self.host) { - rows.push(Row::info(Line::styled( - format!(" {line}"), - Style::new().fg(theme::MUTED), - ))); - } - if let Some(server) = self.served() { - rows.push(Row::info(format!( - " {} · through the berth server at {server}", - self.text(Field::SshHost) - ))); - } - if !self.locked() { - rows.extend(self.field_rows()); - } - for problem in self.problems() { - rows.push(Row::info(Line::styled( - format!("✗ {problem}"), - Style::new().fg(theme::BAD), - ))); - } - let mut groups = self.overview_groups(); - // Set-up and applying it are for whoever may set the server up, - // in the group berth's own actions of set-up are in. - if !self.locked() { - let mut setup = vec![Row::entry( - " ▸ Change what is installed, accounts or install method ›", - Entry::Setup, - )]; - // A host without its controller offers Apply above already; - // only a workstation is applied. - if !matches!(self.host.state, State::Missing | State::Old(_)) - && self.host.level == level::Level::Workstation - { - setup.push(Row::entry( - " ▸ Apply this configuration", - Entry::Task("apply"), - )); - } - match groups.iter_mut().find(|(title, _)| title == "Set-Up") { - Some((_, rows)) => { - setup.append(rows); - *rows = setup; - } - None => groups.push(("Set-Up".to_owned(), setup)), - } - } - for (title, group) in groups { - rows.push(Row::info("")); - rows.push(Row::info(Line::styled( - title, - Style::new().fg(theme::ACCENT).bold(), - ))); - rows.extend(group); - } - } - } - rows - } - - /// The groups of the overview under the connection, as the host lists its - /// tasks: each in the group it names, with the terminal among the first and - /// the host's sessions after them. A host that cannot be asked says why. - fn overview_groups(&self) -> Vec<(String, Vec)> { - let note = |text: String, colour| { - Row::info(Line::styled(format!(" {text}"), Style::new().fg(colour))) - }; - let fixed = |name: &'static str| { - let label = TASKS - .iter() - .find(|(task, ..)| *task == name) - .map_or(name, |(_, label, _)| label); - Row::entry(format!(" ▸ {label}"), Entry::Task(name)) - }; - // A host that cannot be worked with yet still has berth's own - // actions, such as first contact and installing the controller. - let host = |rows| { - let mut groups = vec![("Host".to_owned(), rows)]; - groups.extend(self.own_groups()); - groups - }; - match &self.host.state { - State::Ready(_) => {} - State::Idle if !self.problems().is_empty() => return vec![], - State::Idle | State::Connecting => { - return host(vec![note( - "Asking the host what it can do…".into(), - theme::MUTED, - )]); - } - State::Password => { - return host(vec![ - note( - "The SSH user's sudo password is needed to reach the host.".into(), - theme::WARN, - ), - fixed("connect"), - ]); - } - // Below a workstation the controller is installed alone. - State::Missing if self.host.level != level::Level::Workstation => { - let rows = vec![ - note("berth's controller is not on this host. Install it for sessions, tasks and files here; berth reaches the host over SSH meanwhile.".into(), theme::MUTED), - fixed("ssh"), - ]; - return host(rows); - } - State::Missing => { - return host(vec![ - note("berth's controller is not on this host yet. Applying the configuration installs it.".into(), theme::WARN), - fixed("apply"), - ]); - } - State::Old(version) => { - let raise = match self.host.level { - level::Level::Workstation => "apply", - _ => "install-controller", - }; - let mut rows = vec![note( - format!( - "The host's tools ({version}) are older than this berth needs. {} brings them up to date.", - TASKS - .iter() - .find(|(name, ..)| *name == raise) - .map_or(raise, |(_, label, _)| label) - ), - theme::WARN, - )]; - // berth's own actions list the install already. - if !self.host.own.iter().any(|action| action.name == raise) { - rows.push(fixed(raise)); - } - return host(rows); - } - State::Unreachable(message) => { - // The error whole, as the lines of the list are not wrapped. - let who = match self.served() { - Some(server) => { - format!("The berth server at {server} could not reach the host:") - } - None => "This machine could not reach the host:".to_owned(), - }; - let mut rows = vec![note(who, theme::WARN)]; - for line in message - .lines() - .flat_map(|line| berth_tui::screen::wrap(line, 70)) - { - rows.push(note(format!("✗ {line}"), theme::BAD)); - } - rows.push(fixed("connect")); - // A host that the server keeps is changed by whoever sets the server up. - if !self.locked() { - rows.push(fixed("address")); - } - rows.push(fixed("ssh")); - return host(rows); - } - } - let user = match self.as_agent { - true => self.agent_label().to_owned(), - false => self.operator_label(), - }; - let raise = match self.host.level { - level::Level::Workstation => "apply", - _ => "install-controller", - }; - let behind = berth_tui::host::behind_note(&self.host).map(|text| { - ( - "Configuration".to_owned(), - vec![note(text, theme::WARN), fixed(raise)], - ) - }); - behind - .into_iter() - .chain( - berth_tui::host::groups(&self.host, &user) - .into_iter() - .map(|(title, rows)| { - let rows = rows - .into_iter() - .map(|(line, pick)| { - let entry = match pick { - Pick::Task(index) => Entry::HostTask(index), - Pick::Terminal => Entry::Task("terminal"), - Pick::Session(index) => Entry::HostSession(index), - }; - Row::entry(line, entry) - }) - .collect(); - (title, rows) - }), - ) - .collect() - } - - /// berth's own actions on the host in their groups, without its tasks. - fn own_groups(&self) -> Vec<(String, Vec)> { - berth_tui::page::groups(&self.host.actions(), |index| index) - .into_iter() - .map(|group| { - let rows: Vec = group - .rows - .into_iter() - .filter(|(_, index)| self.host.is_own(*index)) - .map(|(line, index)| Row::entry(line, Entry::HostTask(index))) - .collect(); - (group.title, rows) - }) - .filter(|(_, rows)| !rows.is_empty()) - .collect() - } - - fn agent_label(&self) -> &str { - match self.text(Field::AgentUser) { - "" => "agent", - user => user, - } - } - - fn operator_label(&self) -> String { - match self.text(Field::SshUser) { - "" => "the SSH user".into(), - user => format!("{user}, the SSH user"), - } - } - - pub fn entries(&self) -> Vec { - self.rows() - .into_iter() - .filter_map(|row| row.entry) - .collect() - } - - pub fn selected_entry(&self) -> Option { - self.entries() - .get(self.list.selected().unwrap_or(0)) - .copied() - } - - pub fn selected_field(&self) -> Option { - match self.selected_entry() { - Some(Entry::Field(field)) => Some(field), - _ => None, - } - } - - /// The information box for the selected row, as Markdown. A task that has - /// steps from plugins lists what each of them adds. - pub fn help(&self) -> String { - match self.selected_entry() { - Some(Entry::Feature(index)) => self.catalogue[index].help.clone(), - Some(Entry::HostTask(index)) => berth_tui::host::task_help(&self.host, index), - Some(Entry::HostSession(index)) => berth_tui::host::session_help(&self.host, index), - Some(Entry::Machine(pick)) => self.machine.help(pick), - _ => self.entry_help().to_owned(), - } - } - - fn entry_help(&self) -> &'static str { - match self.selected_entry() { - Some(Entry::Action(action)) => action.help(), - Some(Entry::Field(field)) => field.spec().help, - Some(Entry::Task(name)) => TASKS - .iter() - .find(|(task, ..)| *task == name) - .map_or("", |(.., help)| help), - Some(Entry::Group(_)) => setup::GROUP_HELP, - Some(Entry::Feature(_)) => "", - Some(Entry::HostTask(_) | Entry::HostSession(_) | Entry::Machine(_)) => "", - Some(Entry::HostRegistry(_)) => { - "Give this host the login of this registry. Its npm repositories become the agent's npm registry with the login in ~/.npmrc, and its Docker addresses get the login in ~/.docker/config.json. Registries are set up on the Registries tab of the home screen." - } - Some(Entry::Key(account, Some(_))) => match account { - setup::Account::Operator => Field::OperatorKey.spec().help, - setup::Account::Agent => Field::AgentKey.spec().help, - }, - Some(Entry::Key(_, None)) => setup::ADD_KEY_HELP, - Some(Entry::HostSecret(_)) => { - "How this host gets the secret. Enter, then type a variable name such as GH_TOKEN to put it in the agent's environment, or a path such as ~/.config/app/token to write it to a file. Empty, or x, means the host does not get it. A secret given as GH_TOKEN also logs git in to GitHub through gh. Values are set on the Secrets tab of the home screen." - } - Some(Entry::Setup) => { - "Opens the set-up tabs of this host: how to install it, what is installed, its settings, and running it. Esc comes back here. Changes reach a running host with Apply this configuration." - } - Some(Entry::Save) => "Saves the profile, as Ctrl+S does, without running anything.", - Some(Entry::Run) | None => self.action.help(), - } - } - - /// The selected row takes a file path, so Browse applies. - pub fn path_selected(&self) -> bool { - match self.selected_entry() { - Some(Entry::Field(field)) => field.spec().kind == Kind::Path, - _ => false, - } - } - - /// The `berth serve` this host belongs to, when it is not a host of this machine. - pub fn served(&self) -> Option<&str> { - self.config["builder"]["served"].as_str() - } - - /// A server's host that this berth may use and not set up. - pub fn locked(&self) -> bool { - self.config["builder"]["served_only"] == true - } - - /// The server a host is set up on through this berth. - fn server(&self) -> Option { - self.served()?; - (!self.locked()) - .then(|| self.host_profile().served(&self.paths)) - .flatten() - } - - fn name(&self) -> String { - self.host_profile().name - } - - /// The ways to install this host: for a server's host those the server - /// does itself. - pub fn methods(&self) -> &'static [Action] { - match self.served() { - Some(_) => &[ - Action::Machine, - Action::CloudSeed, - Action::Ssh, - Action::Reach, - ], - None => &Action::METHODS, - } - } - - pub fn set_step(&mut self, step: Step) { - if let Some(server) = self.served() - && self.locked() - && step != Step::Manage - { - self.status = format!( - "This host is set up on the berth server at {server}, by whoever has the role admin there" - ); - return; - } - self.step = step; - // A berth server offers fewer ways to install than the one a new host starts with. - if step == Step::Install - && self.action != Action::Manage - && !self.methods().contains(&self.action) - { - self.action = self.methods()[0]; - setup::fit_profile(&mut self.config, self.action); - } - // The Manage tab acts on the running host; the others on the install method. - if step == Step::Manage { - if self.action != Action::Manage { - self.method = self.action; - } - self.action = Action::Manage; - } else if self.action == Action::Manage { - self.action = self.method; - } - // A profile saved with another kind opens as the machine's. - if step != Step::Manage && self.action == Action::Machine { - setup::fit_profile(&mut self.config, self.action); - } - // A host with an address opens on its tasks rather than the address. - let task = |entry: &Entry| matches!(entry, Entry::Task(_) | Entry::HostTask(_)); - let first = match self.text(Field::SshHost).is_empty() { - false if step == Step::Manage => self.entries().iter().position(task), - _ => None, - }; - if step == Step::Settings { - self.load_access(); - } - if step == Step::Install { - self.load_machine(); - } - self.list = ListState::default().with_selected(Some(first.unwrap_or(0))); - self.status.clear(); - self.suggestion_focus = false; - } - - /// Open the first tab of the set-up flow, for a host that is not on its - /// overview, and keep what was said on opening it. - pub fn start(&mut self) { - if self.step == Step::Manage || self.locked() { - return; - } - let said = std::mem::take(&mut self.status); - self.set_step(Step::Install); - self.status = said; - } - - pub fn move_selection(&mut self, up: bool) { - let index = self.list.selected().unwrap_or(0); - let last = self.entries().len().saturating_sub(1); - self.list.select(Some(if up { - index.saturating_sub(1) - } else { - (index + 1).min(last) - })); - } - - pub fn open_picker(&mut self) { - if !self.path_selected() { - return; - } - let entry = self.selected_entry(); - if self.editing.is_none() { - match entry { - Some(Entry::Field(field)) => self.activate(field), - _ => self.editing = Some(Input::default()), - } - } - let value = self.editing.as_ref().map_or("", |input| input.value()); - let start = match value.is_empty() { - false => value.to_owned(), - true => ".".into(), - }; - match Picker::new(&start) { - Ok(picker) => self.picker = Some(picker), - Err(error) => self.status = error.to_string(), - } - } - - /// Generate a hostname, or a password whose hash goes into the selected field. - pub fn generate(&mut self) -> Result<()> { - let Some(field) = self.selected_field() else { - return Ok(()); - }; - if field == Field::Iso { - let mut command = Command::new("python3"); - command - .args([layout::BUILDER, "fetch-iso", "--dir"]) - .arg(&self.iso_dir); - let stage = Stage { - title: "Download Ubuntu Server ISO".into(), - command, - source: None, - input: None, - }; - let next = "The ISO path and checksum are filled in."; - self.job = Some(Job::start(vec![stage], next, vec![])?); - self.fetching_iso = true; - } else if field == Field::Hostname { - self.set(field, json!(credentials::hostname()?)); - } else if field == Field::TailscaleKey { - let host = self.text(Field::Hostname).to_owned(); - // A server's host gets its key from the server's control server. - let (key, note) = match self.host_profile().served(&self.paths) { - Some(server) => { - let made = server.call( - "setup.generate", - json!({}), - Some(json!({"what": "tailscale_key", "host": self.name()})), - )?; - let text = |key: &str| made[key].as_str().unwrap_or("").to_owned(); - (text("value"), text("note")) - } - None => berth_core::tailnet::generate(&self.paths, &self.config, &host)?, - }; - self.set(field, json!(key)); - self.status = note; - } else if field == Field::IsoSha { - self.hash_iso()?; - } else if field.is_password_hash() { - let user = self.text(if field == Field::OperatorHash { - Field::OperatorUser - } else { - Field::AgentUser - }); - let user = if user.is_empty() { "account" } else { user }.to_owned(); - // A server's host has its password made and kept on the server, - // which the next mirror would not keep here. - if let Some(server) = self.host_profile().served(&self.paths) { - let made = server.call( - "setup.generate", - json!({}), - Some(json!({"what": "password", "host": self.name(), "user": user})), - )?; - let hash = made["value"].as_str().unwrap_or(""); - if !hash.starts_with("$6$") { - return Err("The berth server made no password".into()); - } - self.set(field, json!(hash)); - self.status = format!( - "Generated password kept on the berth server at {}; see it with: berth credentials {}", - server.url, - self.name() - ); - return Ok(()); - } - let password = credentials::password()?; - let hash = credentials::hash(&password)?; - credentials::save(&self.paths, &self.place, &user, &password)?; - self.set(field, json!(hash)); - self.status = format!( - "Generated password kept in berth's store; see it with: berth credentials {}", - self.place.argument() - ); - } - Ok(()) - } - - pub fn refresh_discovery(&mut self) { - if let Some(home) = std::env::var_os("HOME") { - let ssh = PathBuf::from(home).join(".ssh"); - (self.public_keys, self.private_keys) = discovery::keys(&ssh); - self.ssh_hosts = discovery::hosts(&ssh.join("config")); - } - self.wifi_networks = discovery::wifi_networks(self.privileged); - self.usb_drives = discovery::usb_drives(); - self.tailnet_hosts = discovery::tailnet_hosts(); - for entry in discovery::agent_keys() { - let same_key = |(_, key): &(String, String)| { - key.split_whitespace() - .take(2) - .eq(entry.1.split_whitespace().take(2)) - }; - if !self.public_keys.iter().any(same_key) { - self.public_keys.push(entry); - } - } - } - - pub fn update_suggestions(&mut self) { - let own = |values: &[&str]| -> Choices { - values - .iter() - .map(|value| (value.to_string(), value.to_string())) - .collect() - }; - let choices = match self.selected_entry() { - Some(Entry::Key(..)) => self.public_keys.clone(), - // The provisioner is a choice like a field's, of what there is. - Some(Entry::Machine(machine::Pick::Provisioner)) => self.machine.choices(), - Some(Entry::Machine(machine::Pick::Argument(index))) => { - self.machine.argument_choices(index) - } - Some(Entry::Field(field)) => match (field, field.spec().kind) { - (Field::Login, _) => self - .login_choices() - .into_iter() - .map(|choice| (choice.clone(), choice)) - .collect(), - (_, Kind::Choice(values)) => own(values), - (Field::SshHost, _) => self - .tailnet_hosts - .iter() - .chain(&self.ssh_hosts) - .cloned() - .collect(), - (Field::WifiSsid, _) => self - .wifi_networks - .iter() - .map(|(ssid, _)| (ssid.clone(), ssid.clone())) - .collect(), - _ => vec![], - }, - _ => vec![], - }; - if self.suggestions != choices { - self.suggestions = choices; - self.suggestion_list = ListState::default().with_selected(Some(0)); - self.suggestion_focus = false; - } - // A choice's list starts at what is chosen, and follows it as it changes. - if !self.suggestion_focus - && let Some(current) = self.current_choice() - && let Some(at) = self - .suggestions - .iter() - .position(|(_, value)| *value == current) - { - self.suggestion_list.select(Some(at)); - } - } - - /// What is chosen on the selected row, when it is a choice. - fn current_choice(&self) -> Option { - match self.selected_entry()? { - Entry::Field(field) if matches!(field.spec().kind, Kind::Choice(_)) => { - Some(self.text(field).to_owned()) - } - Entry::Machine(machine::Pick::Provisioner) => self.machine.chosen.clone(), - Entry::Machine(machine::Pick::Argument(index)) => { - self.machine.value(index).map(str::to_owned) - } - _ => None, - } - } - - pub fn select_suggestion(&mut self) { - if let Some((_, value)) = self - .suggestions - .get(self.suggestion_list.selected().unwrap_or(0)) - { - self.editing = Some(Input::new(value.clone())); - self.password_entry = false; - self.accept_edit(); - } - self.suggestion_focus = false; - } - - pub fn cancel_edit(&mut self) { - self.editing = None; - self.password_entry = false; - } - - pub fn accept_edit(&mut self) { - let Some(input) = self.editing.take() else { - return; - }; - let hashing = std::mem::take(&mut self.password_entry); - match self.selected_entry() { - Some(Entry::Field(field)) if hashing => match credentials::hash(input.value()) { - Ok(hash) => self.set(field, json!(hash)), - Err(error) => self.status = error.to_string(), - }, - Some(Entry::Field(field)) => self.commit(field, input.value()), - Some(Entry::HostSecret(index)) => self.set_host_secret(index, input.value()), - Some(Entry::Machine(pick)) => { - let name = self.name(); - let next = self.machine.typed(pick, input.value(), &name); - self.follow(next); - } - Some(Entry::Key(account, index)) => { - // Nothing typed changes nothing: x is what removes a key. - if !input.value().trim().is_empty() - && let Err(why) = - setup::set_key(&mut self.config, account, index, input.value()) - { - self.status = why; - } - } - _ => {} - } - } - - /// Toggle or cycle the field, or open the editor for free text. - pub fn activate(&mut self, field: Field) { - if field == Field::ReleaseStore { - // Cycle through entering an address and the configured stores. - let stores = self - .access - .as_ref() - .map(|store| store.release_stores()) - .unwrap_or_default(); - let next = match stores.iter().position(|store| store == self.text(field)) { - None if self.text(field).is_empty() => stores.first(), - None => None, - Some(index) => stores.get(index + 1), - }; - if stores.is_empty() { - self.status = "No file server or raw repository yet. Add one on the Registries tab of the home screen.".into(); - } - let value = next.cloned().unwrap_or_default(); - self.set(field, json!(value)); - return; - } - if field == Field::UsbDevice { - // Cycle through "no" and the drives connected right now. - self.usb_drives = discovery::usb_drives(); - let current = self - .usb_drives - .iter() - .position(|(_, path)| path == self.text(field)); - let next = match current { - None if self.text(field).is_empty() => self.usb_drives.first(), - None => None, - Some(index) => self.usb_drives.get(index + 1), - }; - let device = next.map_or(String::new(), |(_, path)| path.clone()); - self.status = match device.is_empty() { - true if self.usb_drives.is_empty() => { - "No usable USB drive. Replug one that was ejected.".into() - } - _ => String::new(), - }; - self.set(field, json!(device)); - return; - } - if field == Field::Login { - let choices = self.login_choices(); - let current = choices.iter().position(|choice| choice == self.text(field)); - let next = current.map_or(0, |index| (index + 1) % choices.len()); - self.set(field, json!(choices[next])); - return; - } - match field.spec().kind { - Kind::Toggle => self.set_toggle(field, !self.enabled(field)), - Kind::Choice(choices) => { - let current = choices - .iter() - .position(|choice| *choice == self.text(field)); - let next = current.map_or(0, |index| (index + 1) % choices.len()); - self.set(field, json!(choices[next])); - } - _ => self.editing = Some(Input::new(self.text(field).to_owned())), - } - } - - pub fn commit(&mut self, field: Field, value: &str) { - let spec = field.spec(); - let value = if spec.kind == Kind::Secret { - value - } else { - value.trim() - }; - if field == Field::Login { - if !self.login_choices().iter().any(|choice| choice == value) { - self.status = format!("Choose one of: {}", self.login_choices().join(", ")); - return; - } - } else if let Kind::Choice(choices) = spec.kind - && !choices.contains(&value) - { - self.status = format!("Choose one of: {}", choices.join(", ")); - return; - } - if let Some(name) = spec.pointer.strip_prefix("/ansible_vars/") - && value.is_empty() - { - // An empty override would replace the role default with nothing. - if let Some(vars) = self.config["ansible_vars"].as_object_mut() { - vars.remove(name); - } - return; - } - self.set(field, json!(value)); - if spec.pointer.starts_with("/wifi/") - && self.config["wifi"] - .as_object() - .is_some_and(|wifi| wifi.values().all(|v| v.as_str() == Some(""))) - { - self.config["wifi"] = Value::Null; - } - if field == Field::SshHost && !value.is_empty() { - self.fill_from_ssh_config(value); - } - if field == Field::Level { - self.level_chosen = true; - } - // An existing host over SSH is asked what it is, as berth host add does. - if matches!( - field, - Field::SshHost | Field::SshUser | Field::SshPort | Field::Login - ) && self.action == Action::Ssh - && self.step != Step::Manage - && !self.text(Field::SshHost).is_empty() - { - self.ask_found_level(); - } - if field == Field::WifiSsid - && let Some(password) = self.saved_wifi_password().map(str::to_owned) - { - self.set(Field::WifiPassword, json!(password)); - self.status = "Wi-Fi password filled from this machine".into(); - } - if field == Field::WifiPassword { - match self.wifi_password_matches() { - Some(true) => { - self.status = "Matches the Wi-Fi password saved on this machine".into() - } - Some(false) => { - self.status = "Differs from the Wi-Fi password saved on this machine".into() - } - None => {} - } - } - if field == Field::Iso - && !value.is_empty() - && let Err(error) = self.hash_iso() - { - self.status = error.to_string(); - } - } - - /// Ask the host's controller, in the background, what the host is. - fn ask_found_level(&mut self) { - let (paths, host) = (self.paths.clone(), self.host_profile()); - let (sender, receiver) = mpsc::channel(); - std::thread::spawn(move || { - let _ = sender.send(level::observe(&paths, &host, None)); - }); - self.found_level = Some(receiver); - } - - /// Take in what the host said it is, when it has. Called between events. - pub fn watch_found_level(&mut self) { - let Some(answer) = &self.found_level else { - return; - }; - if let Ok(found) = answer.try_recv() { - self.found_level = None; - self.offer_level(found); - } - } - - /// Offer the level the host was found to have: it is taken unless a - /// level was chosen on the Level row, which is only told of it. - pub fn offer_level(&mut self, found: Option) { - let current = level::of(&self.config); - let Some(found) = found.filter(|found| *found != current) else { - return; - }; - let name = self.name(); - if self.level_chosen { - self.status = - format!("{name} answers as a {found} host; its Level here says {current}"); - return; - } - berth_api::profile::set_level(&mut self.config, found); - self.status = format!("{name} answers as a {found} host, so its Level is {found} now"); - } - - /// Record the checksum of the chosen ISO so the build can detect a changed file. - fn hash_iso(&mut self) -> Result<()> { - let path = expand_path(self.text(Field::Iso)); - if !path.is_file() { - return Err("Choose the source ISO first".into()); - } - let digest = credentials::sha256(&path)?; - self.set(Field::IsoSha, json!(digest)); - self.status = "ISO SHA256 computed · compare with Ubuntu's SHA256SUMS".into(); - Ok(()) - } - - /// Get administrator rights for reading saved Wi-Fi passwords, asking for the - /// password in a dialog unless sudo needs none. - pub fn request_unlock(&mut self, reason: &'static str) { - self.unlock_reason = reason; - if self.privileged || credentials::sudo_unlock(None) { - self.unlocked(); - } else { - self.unlock_failed = false; - self.unlock = Some(Input::default()); - } - } - - pub fn submit_unlock(&mut self) { - let Some(input) = self.unlock.take() else { - return; - }; - if std::mem::take(&mut self.unlock_remote) { - self.host.password = Some(input.value().to_owned()); - self.connect_host(); - return; - } - if credentials::sudo_unlock(Some(input.value())) { - self.unlocked(); - if self.pending_run { - self.run(true); - } - } else { - self.unlock_failed = true; - self.unlock = Some(Input::default()); - } - } - - fn unlocked(&mut self) { - self.privileged = true; - self.wifi_networks = discovery::wifi_networks(true); - let known = self - .wifi_networks - .iter() - .filter(|(_, password)| !password.is_empty()) - .count(); - self.status = format!("Unlocked {known} saved Wi-Fi passwords"); - if self.text(Field::WifiPassword).is_empty() - && let Some(password) = self.saved_wifi_password().map(str::to_owned) - { - self.set(Field::WifiPassword, json!(password)); - self.status = "Wi-Fi password filled from this machine".into(); - } - } - - /// Password this machine has saved for the profile's SSID, if it knows the network. - fn saved_wifi_password(&self) -> Option<&str> { - let ssid = self.text(Field::WifiSsid); - self.wifi_networks - .iter() - .find(|(known, password)| known == ssid && !password.is_empty()) - .map(|(_, password)| password.as_str()) - } - - /// Whether the profile's Wi-Fi password is the one this machine uses for that - /// network. `None` when this machine has nothing to compare against. - pub fn wifi_password_matches(&self) -> Option { - self.saved_wifi_password() - .map(|saved| saved == self.text(Field::WifiPassword)) - } - - /// Fill blank connection fields with what OpenSSH would use for this host. - fn fill_from_ssh_config(&mut self, host: &str) { - let Ok(output) = Command::new("ssh").args(["-G", "--", host]).output() else { - return; - }; - for line in String::from_utf8_lossy(&output.stdout).lines() { - let Some((name, value)) = line.split_once(' ') else { - continue; - }; - let field = match name { - "user" => Field::SshUser, - "port" => Field::SshPort, - _ => continue, - }; - if self.text(field).is_empty() { - self.set(field, json!(value)); - } - } - } - - /// Fill blank fields from local SSH keys and DEVBOX_* environment variables. - pub fn detect(&mut self) { - self.refresh_discovery(); - let only_key = match self.public_keys.as_slice() { - [(_, key)] => Some(key.clone()), - _ => None, - }; - let env = |name: &str| std::env::var(name).ok(); - let mut filled = Vec::new(); - for (label, field, detected) in [ - ("hostname", Field::Hostname, env("DEVBOX_HOSTNAME")), - ( - "disk serial", - Field::DiskSerial, - env("DEVBOX_TARGET_DISK_SERIAL"), - ), - ("release URL", Field::ReleaseUrl, env("DEVBOX_RELEASE_URL")), - ("ISO path", Field::Iso, env("DEVBOX_ISO")), - ("ISO checksum", Field::IsoSha, env("DEVBOX_ISO_SHA256")), - ("inventory", Field::Inventory, env("DEVBOX_INVENTORY")), - ] { - let Some(value) = detected - .map(|value| value.trim().to_owned()) - .filter(|value| !value.is_empty()) - else { - continue; - }; - if self.text(field).is_empty() { - self.set(field, json!(value)); - filled.push(label); - } - } - // Without this berth's own key the operator needs one of the user's. - if let Some(key) = only_key - && self.view().keys(setup::Account::Operator).is_empty() - && self.own_public.is_empty() - && setup::set_key(&mut self.config, setup::Account::Operator, None, &key).is_ok() - { - filled.push("public key"); - } - self.status = if filled.is_empty() { - format!( - "Found {} public keys, {} private keys and {} SSH hosts. Select a field to choose.", - self.public_keys.len(), - self.private_keys.len(), - self.ssh_hosts.len() - ) - } else { - format!("Review detected values: {}", filled.join(", ")) - }; - } - - /// Reasons the chosen action would fail, checked before leaving the UI. - pub fn problems(&self) -> Vec { - self.placed_problems() - .into_iter() - .map(|problem| problem.text) - .collect() - } - - /// [`App::problems`], each on the tab of what it is about. - pub fn placed_problems(&self) -> Vec { - use Field::*; - use setup::Problem; - let action = self.action; - if action == Action::Manage { - let access = self.host_profile().access(); - return berth_core::keys::check(&access.address, &access.port) - .err() - .map(|error| Problem::new(Step::Manage, error.to_string())) - .into_iter() - .collect(); - } - // A host below a workstation needs only to be reached. - if action == Action::Ssh && level::of(&self.config) != level::Level::Workstation { - let access = self.host_profile().access(); - return berth_core::keys::check(&access.address, &access.port) - .err() - .map(|error| Problem::new(Step::Install, error.to_string())) - .into_iter() - .collect(); - } - let mut problems = self.view().problems_with(&self.machine); - let mut check = |ok: bool, step: Step, message: &str| { - if !ok { - problems.push(Problem::new(step, message)); - } - }; - match action { - Action::Iso => { - check( - expand_path(self.text(Iso)).is_file(), - Step::Run, - "Source ISO file does not exist", - ); - let usb = self.text(UsbDevice); - check( - usb.is_empty() || self.usb_drives.iter().any(|(_, path)| path == usb), - Step::Run, - "The USB drive to write to is not connected", - ); - let sha = self.text(IsoSha); - check( - sha.is_empty() - || sha.len() == 64 - && sha - .chars() - .all(|c| c.is_ascii_digit() || ('a'..='f').contains(&c)), - Step::Run, - "ISO SHA256 must be 64 lowercase hex characters", - ); - } - Action::Apply => { - let inventory = self.text(Inventory); - check( - inventory.is_empty() || expand_path(inventory).exists(), - Step::Run, - "Ansible inventory file does not exist", - ); - } - Action::Ssh => { - let access = self.host_profile().access(); - if let Err(error) = berth_core::keys::check(&access.address, &access.port) { - problems.push(Problem::new(Step::Install, error.to_string())); - } - } - _ => {} - } - problems - } - - /// The ways this host can be logged in to, with the keys of the ring. - pub fn login_choices(&self) -> Vec { - setup::login_choices( - &self.ring_keys, - self.config["access"]["auth"].as_str().unwrap_or(""), - self.served().is_some(), - ) - } - - /// The public keys a host set up from here takes for berth: this - /// berth's, with the one replacing it while it rotates, or the server's. - fn own_keys(&self) -> Vec { - match self.served() { - Some(_) => vec![self.own_public.clone()] - .into_iter() - .filter(|key| !key.is_empty()) - .collect(), - None => berth_core::keys::operator_keys(&self.paths, &self.config).unwrap_or_default(), - } - } - - /// This host as its profile describes it, for reaching and applying it. - pub fn host_profile(&self) -> Host { - Host { - name: self.place.title(), - place: self.place.clone(), - config: self.config.clone(), - } - } - - /// Start talking to the host's controller. A host whose sudo asks for a - /// password gets the dialog first, once. - pub fn connect_host(&mut self) { - let host = self.host_profile(); - self.host.level = level::of(&self.config); - self.ask_own_actions(); - let own_link = host.access().link.is_some() || host.served(&self.paths).is_some(); - if host.asks_password() && !own_link && self.host.password.is_none() { - self.host.state = State::Password; - if !std::mem::replace(&mut self.password_asked, true) { - self.ask_host_password(false); - } - return; - } - let asked = std::mem::take(&mut self.force_login); - match host.link(&self.paths, self.host.password.as_deref()) { - // Opening the overview is no reason to repeat a login the host - // refused a moment ago; Try again is. - Ok(spec) - if !asked - && let Some(refused) = - spec.guard.as_ref().and_then(|guard| guard.refused()) => - { - self.host.state = State::Unreachable(refused); - } - Ok(spec) => { - crate::remote::connect_with(&mut self.host, spec, compare(&self.paths, &host)); - } - Err(error) => self.host.state = State::Unreachable(error.to_string()), - } - } - - /// berth's own actions on the host: those of this berth, or what the - /// server lists, asked for in the background, as the server asks the host. - fn ask_own_actions(&mut self) { - let host = self.host_profile(); - let Some(server) = host.served(&self.paths) else { - self.host.own = level::local_actions(&host.name, self.host.level, self.host.seen); - return; - }; - let (sender, receiver) = mpsc::channel(); - let name = host.name.clone(); - std::thread::spawn(move || { - let listed = server - .call("hosts.actions", json!({"name": name}), None) - .ok() - .and_then(|listed| serde_json::from_value::>(listed).ok()); - let _ = sender.send(listed.unwrap_or_default()); - }); - self.own_answers = Some(receiver); - } - - /// Of what a server lists, berth's own that the overview has no row of: - /// applying and the seed are on the set-up, sending secrets is a task. - fn take_own_actions(&mut self) { - let Some(answers) = &self.own_answers else { - return; - }; - if let Ok(listed) = answers.try_recv() { - self.own_answers = None; - // A secret is asked for by the command that runs it, not typed here. - self.host.own = listed - .into_iter() - .filter(|action| action.runs == berth_api::Runs::Berth) - .filter(|action| !["apply", "seed"].contains(&action.name.as_str())) - .map(|mut action| { - action.arguments.retain(|argument| !argument.secret); - action - }) - .collect(); - } - } - - fn ask_host_password(&mut self, failed: bool) { - self.unlock_remote = true; - self.unlock_failed = failed; - self.unlock_reason = "Your sudo password on the host, to reach its controller."; - self.unlock = Some(Input::default()); - } - - /// Go to the SSH address of the host to change it. Saving it tries to - /// reach the host again. - pub fn edit_address(&mut self) { - let position = |app: &App| { - app.entries() - .iter() - .position(|entry| *entry == Entry::Field(Field::SshHost)) - }; - // The overview lists the connection's fields itself; if it does not, - // the tab that has them does. - if position(self).is_none() { - self.set_step(Step::Install); - } - match position(self) { - Some(at) => { - self.list.select(Some(at)); - self.activate(Field::SshHost); - self.retry_address = self.editing.is_some(); - } - None => self.status = "This host has no SSH address to change".into(), - } - } - - /// Connect again from nothing, asking for the password again if needed. - pub fn reconnect_host(&mut self) { - self.password_asked = false; - self.force_login = true; - self.host.disconnect(); - } - - /// Keep the overview in step with the host: connect when there is an - /// address, take in what it said. Called between events. - pub fn watch_host(&mut self) { - if self.step != Step::Manage { - return; - } - if self.host.state == State::Idle && self.unlock.is_none() && self.problems().is_empty() { - self.connect_host(); - } - self.take_own_actions(); - let had_password = self.host.password.is_some(); - if let Some(note) = self.host.pump() { - self.status = note; - } - // sudo did not take the password: ask again. - if had_password && self.host.password.is_none() && self.unlock.is_none() { - self.ask_host_password(true); - } - let last = self.entries().len().saturating_sub(1); - if self.list.selected().unwrap_or(0) > last { - self.list.select(Some(last)); - } - } - - /// Run a task of the host: after its question and its arguments, if it has any. - pub fn begin_task(&mut self, index: usize, confirmed: bool) { - if let Some(action) = self.host.actions().get(index) - && !action.available - { - self.status = format!("{} cannot run now: {}", action.title(), action.why); - return; - } - if let Some((index, values)) = self.host.begin_task(index, confirmed) { - self.launch_task(index, values); - } - } - - /// Take the value typed for the argument being asked and go on to the - /// next, or run the task after the last. - pub fn accept_argument(&mut self) { - match self.host.accept_argument() { - Ok(Some((index, values))) => self.launch_task(index, values), - Ok(None) => {} - Err(missing) => self.status = missing, - } - } - - fn launch_task(&mut self, index: usize, values: std::collections::BTreeMap) { - let Some(action) = self.host.actions().get(index).cloned() else { - return; - }; - if self.host.is_own(index) { - let _ = self.save(); - self.own_command = Some(own_command(&self.place.argument(), &action, &values)); - return; - } - let mut body = self.host.task_body(&action, &values); - // Secrets are built here and travel on the task's standard input. A - // server's host is given the server's own, which its action sends. - let secrets = self - .host - .task(&action) - .and_then(|task| task.input.as_deref()); - if secrets == Some("secrets") && self.served().is_none() { - let payload = self - .access() - .and_then(|store| store.host_payload(&self.config)); - match payload { - Ok(payload) => { - body["input_b64"] = json!(berth_api::b64(format!("{payload}\n").as_bytes())) - } - Err(error) => { - self.status = error.to_string(); - return; - } - } - } - let _ = self.save(); - self.status = format!("Starting {}…", action.title()); - self.host.run(&action.name, body); - } - - /// A shell on the host as one of its sessions: the agent's, or the SSH user's. - pub fn start_host_shell(&mut self) { - let (user, title) = match self.as_agent { - true => ( - "agent", - format!("{} on {}", self.agent_label(), self.text(Field::SshHost)), - ), - false => ("operator", self.ssh_target()), - }; - self.host.shell(user, &title); - self.status = "Starting a shell…".into(); - } - - pub fn save(&mut self) -> Result<()> { - self.place.save(&self.paths, &self.config)?; - // A server's host is the server's to keep. - if let Some(server) = self.server() { - server - .save_host(&self.name(), &self.config) - .map_err(|error| format!("The berth server did not take the profile: {error}"))?; - } - self.saved = self.config.clone(); - Ok(()) - } - - /// What to say once the profile is saved: the host and where it is kept, - /// or the file. - pub fn saved_note(&self) -> String { - match (&self.place, self.server()) { - (Place::Stored { name, .. }, Some(server)) => { - format!("Saved {name} on the berth server at {}", server.url) - } - (Place::Stored { name, .. }, None) => format!("Saved {name} in berth's store"), - (Place::File(path), _) => format!("Saved {}", path.display()), - } - } - - /// Where each plugin is on this machine, by id. - fn plugin_dirs(&self) -> Value { - apply::plugin_dirs(&self.plugins) - } - - /// The command for the chosen action, and the private files it reads. - fn command(&self) -> Result<(Command, Vec)> { - let action = self.action; - // A server's host is installed by the server: berth itself asks it to. - if let Some(server) = self.served() { - let verb = match action { - Action::Ssh => "apply", - Action::CloudSeed => "seed", - _ => { - return Err(format!( - "{} is done on a machine that keeps the host itself, not through the berth server at {server}", - action.label() - ) - .into()); - } - }; - let mut command = Command::new(std::env::current_exe()?); - command.arg(verb).arg(self.place.argument()); - return Ok((command, vec![])); - } - // Secrets go to private files rather than argv. They live until the command ends. - let mut extra = tempfile::NamedTempFile::new()?; - let inventory = tempfile::NamedTempFile::new()?; - // The profile a seed is rendered from, with a new enrollment key in it - // for a host that joins the tailnet automatically. - let seeded = match action { - Action::UsbSeed | Action::CloudSeed | Action::Iso => { - let saved = self - .place - .load(&self.paths)? - .ok_or("The profile is not saved")?; - let config = berth_core::tailnet::seed_profile(&self.paths, &saved, &self.name())?; - let mut config = berth_core::seed::named(config, &self.name()); - berth_core::kept_profile::for_seed(&mut config, &self.name(), &saved); - Some(berth_core::tailnet::profile_file(&config)?) - } - _ => None, - }; - let profile = seeded.as_ref().map(|file| file.path().to_owned()); - let mut command; - match action { - // Reach is only a way to choose, which sets Ssh at the level reachable. - Action::Save | Action::Manage | Action::Machine | Action::Reach => { - return Err("Nothing to run".into()); - } - Action::Release => { - command = Command::new("python3"); - command.args([layout::BUILDER, "release"]); - } - Action::UsbSeed | Action::CloudSeed => { - command = Command::new("python3"); - command - .args([layout::BUILDER, "render", "--kind"]) - .arg(if action == Action::UsbSeed { - "usb" - } else { - "cloud" - }) - .arg("--config") - .args(&profile); - // A release that is not public: the seed carries the login of - // the registry named for it. - let store = self.text(Field::ReleaseStore); - let login = match self.config["release_login"].as_str() { - Some(name) => Some(name), - None if self.enabled(Field::ReleasePrivate) && !store.is_empty() => { - store.split('/').next() - } - None => None, - }; - if let (Action::CloudSeed, Some(name)) = (action, login) { - let login = self.access()?.curl_login(name)?.ok_or_else(|| { - format!("Registry {name} has no stored login for downloading the release") - })?; - command.env("BERTH_RELEASE_LOGIN", login); - } - } - Action::Iso => { - command = Command::new("python3"); - command - .args([layout::BUILDER, "iso", "--config"]) - .args(&profile) - .arg("--iso") - .arg(expand_path(self.text(Field::Iso))) - .arg("--overwrite"); - if !self.text(Field::IsoSha).is_empty() { - command.args(["--sha256", self.text(Field::IsoSha)]); - } - } - // Applying to a host over SSH is berth-core's; the password the - // overview was given saves Ansible asking for it again. - Action::Ssh => { - let prepared = apply::prepare( - &self.paths, - &self.host_profile(), - self.host.password.as_deref(), - )?; - return Ok((prepared.command, prepared.files)); - } - Action::Apply => { - command = apply::playbook(&std::env::current_dir()?); - let mut vars = - apply::ansible_vars(&self.config, &self.plugin_dirs(), &self.own_keys()); - vars["devbox_berth_profile"] = - berth_core::kept_profile::document(&self.name(), &self.config, None); - apply::add_tailnet(&mut vars, &self.paths, &self.config, &self.place)?; - serde_json::to_writer(&mut extra, &vars)?; - extra.flush()?; - command - .args([layout::PLAYBOOK, "--become", "-e"]) - .arg(format!("@{}", extra.path().display())) - .arg("-i"); - if self.text(Field::Inventory).is_empty() { - command.args(["localhost,", "-c", "local"]); - } else { - command.arg(expand_path(self.text(Field::Inventory))); - } - if self.enabled(Field::AskBecomePass) { - command.arg("--ask-become-pass"); - } - } - Action::Docker => { - command = Command::new("docker"); - command.args(["compose", "-f", "container/compose.yml", "build"]); - command.env("OPERATOR_USER", self.text(Field::OperatorUser)); - command.env("AGENT_USER", self.text(Field::AgentUser)); - for (env, field) in [ - ("INSTALL_PLAYWRIGHT", Field::Playwright), - ("INSTALL_WEB_TOOLS", Field::ToolsWeb), - ("INSTALL_NATIVE_TOOLS", Field::ToolsNative), - ("INSTALL_JVM_TOOLS", Field::ToolsJvm), - ("INSTALL_DOTNET_TOOLS", Field::ToolsDotnet), - ("INSTALL_SCRIPTING_TOOLS", Field::ToolsScripting), - ("INSTALL_SCIENTIFIC_TOOLS", Field::ToolsScientific), - ("INSTALL_CHROME_TOOLS", Field::ToolsChrome), - ("SOFTWARE_RENDERING", Field::SoftwareRendering), - ] { - command.env(env, self.enabled(field).to_string()); - } - command.env( - "PLUGIN_FEATURES", - json!(setup::chosen_features(&self.config)).to_string(), - ); - } - } - command.env("DEVBOX_BUILD_DIR", &self.build_dir); - // The release archive packs the user's plugins next to the source's. - command.env("BERTH_PLUGINS", self.plugin_dirs().to_string()); - command.env("BERTH_SSH_PUBLIC_KEY", self.own_keys().join("\n")); - let mut files = vec![extra, inventory]; - files.extend(seeded); - Ok((command, files)) - } - - fn next_hint(&self) -> &'static str { - match self.action { - Action::UsbSeed => { - "Next: build the ISO from this seed with the Build unattended ISO action." - } - Action::Iso => "Booting the installer erases the selected disk.", - Action::CloudSeed => { - "Next: upload the devbox-release.tar.gz shown above unchanged to the release URL, then pass the user-data file when creating the server." - } - Action::Docker => { - "Next: docker compose -f container/compose.yml up -d in the source directory, then open http://127.0.0.1:16080/vnc.html" - } - Action::Apply | Action::Ssh => { - "Next: log in as the agent and run `agent-env devbox-smoke`, then sign in to the coding clients." - } - _ => "", - } - } - - /// Actions that prompt for passwords, so they run in the normal terminal - /// after the UI has closed. Everything else runs inside the UI. - pub fn leaves_ui(&self) -> bool { - matches!(self.action, Action::Save | Action::Apply | Action::Ssh) - } - - /// Save the profile and run the chosen action in the normal terminal. - pub fn execute(&mut self) -> Result<()> { - if let Some(words) = self.own_command.take() { - return self.run_own(&words); - } - let tab_action = self.action; - if let Some(action) = self.terminal_action.take() { - self.action = action; - } - let outcome = self.execute_action(); - self.action = tab_action; - outcome - } - - /// For a cloud seed with a store chosen: upload the release there and take - /// its address, so the seed points at what was just published. - fn publish_release(&mut self) -> Result<()> { - let store = self.text(Field::ReleaseStore).to_owned(); - // A server publishes the release itself when it renders the seed. - if self.action != Action::CloudSeed || store.is_empty() || self.served().is_some() { - return Ok(()); - } - let url = crate::registry::publish_from( - self.access()?, - &self.build_dir, - &self.plugin_dirs(), - &store, - ) - .map_err(|error| format!("Publishing to {store} failed: {error}"))?; - self.set(Field::ReleaseUrl, json!(url)); - Ok(()) - } - - /// Run berth with these words in the normal terminal, as one of berth's - /// own actions on the host does. - fn run_own(&mut self, words: &[String]) -> Result<()> { - println!("berth {}", words.join(" ")); - let status = Command::new(std::env::current_exe()?) - .args(words) - .status()?; - // What the host is and can do may be different afterwards. - if let Ok(Some(saved)) = self.place.load(&self.paths) { - self.config = saved.clone(); - self.saved = saved; - } - self.host.disconnect(); - if let ["host", "rename", _, new, ..] = words - .iter() - .map(String::as_str) - .collect::>() - .as_slice() - && status.success() - { - self.renamed = Some((*new).to_owned()); - } - match status.success() { - true => Ok(()), - false => Err(format!("berth {} failed ({status})", words.join(" ")).into()), - } - } - - fn execute_action(&mut self) -> Result<()> { - self.publish_release()?; - self.save()?; - println!("{}", self.saved_note()); - if self.action == Action::Save { - return Ok(()); - } - // Over SSH, a host below a workstation gets the controller alone, or nothing. - if self.action == Action::Ssh { - match level::of(&self.config) { - level::Level::Workstation => {} - level::Level::Controller => { - let words = ["host", "install", &self.place.argument()].map(str::to_owned); - return self.run_own(&words); - } - level::Level::Reachable => { - println!( - "Nothing is installed on a reachable host. If it does not take berth's key yet: berth key trust {}", - self.place.argument() - ); - return Ok(()); - } - } - } - let (command, files) = self.command()?; - self.run_command(command, files)?; - // A successful apply settles how the host is logged in to in the saved profile. - if self.action == Action::Ssh - && let Ok(Some(saved)) = self.place.load(&self.paths) - { - self.config["access"] = saved["access"].clone(); - } - println!("{}", self.next_hint()); - Ok(()) - } - - /// Run the action's command in the normal terminal. Applying a host of - /// this machine holds the host's lock, as berth apply does, so that - /// nothing else changes the host meanwhile. - pub fn run_command( - &self, - mut command: Command, - files: Vec, - ) -> Result<()> { - let failed = |code: String| format!("{} failed ({code})", self.action.label()); - if self.action == Action::Ssh && self.served().is_none() { - use berth_core::runner; - let subject = runner::Subject::Host(self.place.argument()); - let request = runner::Request::new(subject, "apply"); - let work = runner::Work::Program(Box::new(runner::Program { - command, - input: None, - files, - })); - let code = runner::here(&self.paths, request, work, &mut std::io::stdout())?; - return match code { - Some(0) => apply::settle(&self.paths, &self.host_profile()), - Some(code) => Err(failed(format!("exit status: {code}")).into()), - None => Err(failed("ended by a signal".into()).into()), - }; - } - let _files = files; - let status = command.status().map_err(|error| { - format!( - "Could not start {}: {error}", - command.get_program().to_string_lossy() - ) - })?; - match status.success() { - true => Ok(()), - false => Err(failed(status.to_string()).into()), - } - } - - /// Run the chosen action inside the UI. Writing a USB drive first asks for - /// confirmation and for administrator rights. - pub fn run(&mut self, confirmed: bool) { - // The overview has no action of its own: its rows are the host's tasks. - if self.action == Action::Manage { - return; - } - let mut usb = String::new(); - if self.action == Action::Iso { - usb = self.text(Field::UsbDevice).to_owned(); - } - if !usb.is_empty() { - self.usb_drives = discovery::usb_drives(); - let Some(drive) = self.usb_drives.iter().find(|(_, path)| *path == usb) else { - self.status = format!("{usb} is not a connected USB drive"); - return; - }; - if !confirmed { - self.confirm = Some(format!("Erase {}?", drive.0)); - return; - } - if !self.privileged { - self.pending_run = true; - self.request_unlock("Write the installer to the USB drive."); - if !self.privileged { - return; - } - } - } - self.pending_run = false; - if let Err(error) = self.start_job(&usb) { - self.status = error.to_string(); - } - } - - /// Close the finished job's screen and take over what it produced. - pub fn close_job(&mut self) { - let Some(job) = self.job.take() else { - return; - }; - let succeeded = job.outcome == Some(Ok(())); - // The machine is another one now, or none, and the command may have - // given the profile the machine's address. - let mut taken = None; - let machine_job = std::mem::take(&mut self.machine_job); - if machine_job { - self.load_machine(); - // A berth server ran the command and saved the profile there: without - // its copy, the next save from here would undo what it saved. - let refreshed = match self.server() { - Some(server) => server.mirror_host(&self.paths, &self.name()), - None => Ok(()), - }; - taken = self.take_in_command(); - if let Err(error) = refreshed { - taken = Some(format!( - "The berth server did not give the profile again: {error}" - )); - } - } - if std::mem::take(&mut self.fetching_iso) { - if let (true, Some(path)) = (succeeded, job.lines.last()) { - self.commit(Field::Iso, path); - } - return; - } - self.status = if succeeded { - format!("{} finished", job.title) - } else { - format!("{} failed", job.title) - }; - if let Some(said) = taken { - self.status = format!("{} · {said}", self.status); - } - // The machine is the host now, so setting it up is over. - if machine_job && succeeded && job.title == CREATE_MACHINE { - let status = std::mem::take(&mut self.status); - self.set_step(Step::Manage); - self.reconnect_host(); - self.status = status; - } - } - - /// A shell over plain SSH in a terminal inside the UI, as the SSH user: for - /// a host whose controller cannot be reached. - pub fn start_shell(&mut self, rows: u16, cols: u16) { - let title = self.ssh_target(); - let started = self.terminal(&[]).and_then(|command| { - let _ = self.save(); - crate::session::Session::start(&title, command, rows, cols) - }); - match started { - Ok(mut session) => { - session.closes = true; - self.session = Some(session); - } - Err(error) => self.status = error.to_string(), - } - } - - /// `ssh` with this host's options. - fn ssh(&self) -> Result { - self.host_profile().ssh(&self.paths) - } - - fn ssh_target(&self) -> String { - self.host_profile().target() - } - - /// An interactive shell on the host, or `remote` run there. - pub fn terminal(&self, remote: &[&str]) -> Result { - if self.text(Field::SshHost).is_empty() { - return Err("SSH host is required".into()); - } - let mut command = self.ssh()?; - command.arg(self.ssh_target()).args(remote); - Ok(command) - } - - /// An interactive shell as the agent. Root login and the agent's own key are - /// not a given, so it goes through the SSH user and sudo. `agent-env` adds the - /// display and tool paths; a host without it still gets a login shell. - pub fn agent_terminal(&self) -> Result { - let agent = self.agent_name()?; - let mut command = self.terminal(&[])?; - command.arg("-t").arg(format!( - "sudo -iu {agent} sh -c 'command -v agent-env >/dev/null && exec agent-env; exec bash -l'" - )); - Ok(command) - } - - /// The agent's account name, checked because it ends up in a remote shell command. - fn agent_name(&self) -> Result<&str> { - let agent = match self.text(Field::AgentUser) { - "" => "agent", - user => user, - }; - if !agent - .chars() - .all(|ch| ch.is_ascii_alphanumeric() || ch == '_' || ch == '-') - || agent.starts_with('-') - { - return Err("Agent username is not a valid account name".into()); - } - Ok(agent) - } - - /// Remove a field's value. - pub fn clear(&mut self, field: Field) { - if field.can_clear() { - self.commit(field, ""); - if field == Field::Iso { - self.commit(Field::IsoSha, ""); - } - self.status = format!("{} unset", field.spec().label); - } - } - - /// Write the ISO that was just built to a USB drive: the only one connected, - /// the given one, or ask which. - pub fn write_built_iso(&mut self, device: Option) { - self.usb_drives = discovery::usb_drives(); - let device = match (device, self.usb_drives.as_slice()) { - (Some(device), _) => device, - (None, [(_, only)]) => only.clone(), - (None, []) => { - if let Some(job) = &mut self.job { - let hint = "No usable USB drive. Plug one in (replug it if it was ejected) and press w."; - job.lines.push(hint.into()); - } - return; - } - (None, _) => { - self.choosing_drive = true; - return; - } - }; - self.choosing_drive = false; - self.job = None; - self.set(Field::UsbDevice, json!(device)); - self.write_only = true; - self.run(false); - } - - /// Ask for the provisioners and the host's machine again, without - /// waiting for the answer. - pub fn load_machine(&mut self) { - // Without berth's store, as in tests, there is nobody to ask. - if self.access.is_err() { - return; - } - self.machine.asking(); - let name = self.place.argument(); - let paths = self.paths.clone(); - let (send, answers) = mpsc::channel(); - std::thread::spawn(move || { - let _ = send.send(machine_answers(&paths, &name)); - }); - self.machine_answers = Some(answers); - } - - /// Whether the machine's answers are still awaited. - pub fn machine_awaited(&self) -> bool { - self.machine_answers.is_some() - } - - /// Take in the machine's answers once they are there. Called between events. - pub fn watch_machine(&mut self) { - let Some(answers) = &self.machine_answers else { - return; - }; - let (listed, found) = match answers.try_recv() { - Ok(answers) => answers, - Err(mpsc::TryRecvError::Empty) => return, - Err(mpsc::TryRecvError::Disconnected) => ( - Err("Nothing answered".into()), - Err("Nothing answered".into()), - ), - }; - self.machine_answers = None; - self.machine.listed(listed); - self.machine.found(found); - } - - /// Enter on a row of the machine. - pub fn machine_enter(&mut self, pick: machine::Pick) { - // The machine is made from the profile, which has to be whole first, - // and of the kind a created machine is. - if pick == machine::Pick::Create { - setup::fit_profile(&mut self.config, Action::Machine); - } - if pick == machine::Pick::Create && !self.problems().is_empty() { - self.status = "Fix the listed problems first".into(); - return; - } - let name = self.name(); - let next = self.machine.enter(pick, &name); - self.follow(next); - } - - /// The answer to the question before creating the machine. - pub fn machine_answer(&mut self, yes: bool) { - let next = self.machine.answer(yes); - self.follow(next); - } - - /// Do what a row of the machine led to. Its jobs are berth's own - /// commands, which go to the berth server when one is set. - fn follow(&mut self, next: Next) { - let name = self.place.argument(); - let (title, args, after) = match next { - Next::Nothing | Next::Confirm => return, - Next::Say(text) => { - self.status = text; - return; - } - Next::Edit(value) => { - self.editing = Some(Input::new(value)); - return; - } - Next::Create(asked) => { - let mut args = vec!["provision".to_owned(), name, asked.provisioner]; - args.extend( - asked - .arguments - .iter() - .map(|(key, value)| format!("{key}={value}")), - ); - args.push("--yes".into()); - ( - CREATE_MACHINE, - args, - "The machine is this host now: its overview reaches it.", - ) - } - Next::Power(verb) => ( - if verb == "start" { - "Start machine" - } else { - "Stop machine" - }, - vec!["instance".to_owned(), name, verb.to_owned()], - "", - ), - Next::Destroy => ( - "Delete machine", - vec![ - "instance".to_owned(), - name, - "destroy".to_owned(), - "--yes".to_owned(), - ], - "", - ), - }; - if let Err(error) = self.machine_job(title, &args, after) { - self.status = error.to_string(); - } - } - - fn machine_job(&mut self, title: &str, args: &[String], next: &str) -> Result<()> { - // The provisioner works from the profile as it is kept. - self.save()?; - let mut command = Command::new(std::env::current_exe()?); - command.args(args); - let stage = Stage { - title: title.into(), - command, - source: None, - input: None, - }; - self.job = Some(Job::start(vec![stage], next, vec![])?); - self.machine_job = true; - Ok(()) - } - - fn start_job(&mut self, usb: &str) -> Result<()> { - self.publish_release()?; - if self.action != Action::Release { - self.save()?; - } - let mut stages = vec![]; - let mut files = vec![]; - if !std::mem::take(&mut self.write_only) { - let command; - (command, files) = self.command()?; - stages.push(Stage { - title: self.action.label().into(), - command, - source: None, - input: None, - }); - } - let mut next = self.next_hint(); - if !usb.is_empty() { - let iso = self - .build_dir - .join(format!("{}-autoinstall.iso", self.text(Field::Hostname))); - let mut write = Command::new("sudo"); - write - .args(["-n", "bash", layout::WRITE_USB]) - .arg(&iso) - .arg(usb); - stages.push(Stage { - title: format!("Write to {usb}"), - command: write, - source: Some(iso), - input: None, - }); - next = "Next: boot the target from the USB drive. Booting it erases the selected disk."; - } - self.job = Some(Job::start(stages, next, files)?); - Ok(()) - } -} - -/// Load a profile and fill in everything the UI expects to be present. -pub fn parse_config(data: &[u8]) -> Result { - let mut config: Value = serde_json::from_slice(data)?; - if !config.is_object() { - return Err("Configuration must be a JSON object".into()); - } - berth_api::profile::retire_qemu(&mut config); - for field in ["wifi", "ansible_vars", "builder"] { - if !config[field].is_null() && !config[field].is_object() { - return Err(format!("{field} must be a JSON object or null").into()); - } - } - for key in ["ansible_vars", "builder"] { - if config[key].is_null() { - config[key] = json!({}); - } - } - for (key, value) in [ - ("profile", json!("laptop")), - ("keyboard", json!("de")), - ("operator_username", json!("ops")), - ("agent_username", json!("agent")), - ("agent_passwordless_sudo", json!(true)), - ("target_disk_ssd", json!("any")), - ] { - if config[key].is_null() { - config[key] = value; - } - } - // Placeholders from older example profiles count as unset. - for key in [ - "operator_password_hash", - "target_disk_serial", - "release_url", - ] { - if config[key].as_str().is_some_and(|v| v.contains("REPLACE")) { - config[key] = json!(""); - } - } - if config["target_disk_mode"].is_null() { - let has_serial = config["target_disk_serial"] - .as_str() - .is_some_and(|serial| !serial.is_empty()); - config["target_disk_mode"] = json!(if has_serial { "serial" } else { "auto" }); - } - if config["wifi_enabled"].is_null() { - let configured = config["wifi"].as_object().is_some_and(|w| !w.is_empty()); - config["wifi_enabled"] = json!(configured); - } - // The coding agents, T3 Code and GitHub's CLI were switches once and are - // features of plugins that come with berth now. They go first: later - // plugins, such as Unity's, set themselves up for the agents they find. - // Only a profile of that layout is converted: one with the old switches, - // or one with no list of features at all, which had every switch on. A - // profile that lists its features is left as it is, marked or not. - const OLD_AGENT_SWITCHES: [&str; 5] = [ - "devbox_install_agents", - "devbox_agent_claude", - "devbox_agent_codex", - "devbox_agent_pi", - "devbox_install_t3", - ]; - let old_layout = OLD_AGENT_SWITCHES - .iter() - .any(|key| !config["ansible_vars"][*key].is_null()) - || config["ansible_vars"]["devbox_plugin_features"].is_null(); - if config["builder"]["shipped_plugins"] != true && old_layout { - let vars = &config["ansible_vars"]; - let on = |key: &str, default: bool| vars[key].as_bool().unwrap_or(default); - let agents = on("devbox_install_agents", true); - let mut features: Vec = [ - ("claude/cli", agents && on("devbox_agent_claude", true)), - ("codex/cli", agents && on("devbox_agent_codex", true)), - ("github/cli", true), - ("pi/cli", agents && on("devbox_agent_pi", true)), - ("t3/server", on("devbox_install_t3", agents)), - ] - .into_iter() - .filter(|(_, chosen)| *chosen) - .map(|(feature, _)| json!(feature)) - .collect(); - for feature in vars["devbox_plugin_features"] - .as_array() - .into_iter() - .flatten() - { - if !features.contains(feature) { - features.push(feature.clone()); - } - } - if !config["ansible_vars"].is_object() { - config["ansible_vars"] = json!({}); - } - let vars = config["ansible_vars"].as_object_mut().unwrap(); - for old in OLD_AGENT_SWITCHES { - vars.remove(old); - } - vars.insert("devbox_plugin_features".into(), json!(features)); - config["builder"]["shipped_plugins"] = json!(true); - } - // Profiles from before the privilege levels keep what they had: sudo for - // everything without a password is `full`. One whose agent types a - // password stays as it is until a level is chosen. - if config["agent_privileges"].is_null() && config["agent_passwordless_sudo"] == true { - config["agent_privileges"] = json!("full"); - } - // The task last run from the overview was kept once; the host lists its own now. - if let Some(builder) = config["builder"].as_object_mut() { - builder.remove("manage_task"); - } - // Device names change between sessions, so the drive to write is chosen each time. - if let Some(builder) = config["builder"].as_object_mut() { - builder.remove("usb_device"); - } - // The operator always has sudo without a password and always takes the key - // of berth; profiles once had switches for both. - if let Some(profile) = config.as_object_mut() { - profile.remove("operator_passwordless_sudo"); - profile.remove("berth_ssh_key"); - } - let problems = berth_api::profile::Access::problems(&config); - if !problems.is_empty() { - return Err(problems.join("; ").into()); - } - Ok(config) -} - -/// How the overview tells whether a host is behind: a server's host the way -/// the server sees it, another with this machine's plugins. -fn compare(paths: &Paths, host: &Host) -> Option { - if let Some(server) = host.served(paths) { - let name = host.name.clone(); - return Some(std::sync::Arc::new(move |_: &berth_api::Version| { - let found = server - .call("hosts.get", serde_json::json!({"name": name}), None) - .ok()?; - serde_json::from_value(found.get("behind")?.clone()).ok() - })); - } - let now = berth_core::rollout::comparer(paths); - Some(std::sync::Arc::new(move |version: &berth_api::Version| { - now(version.applied.as_ref()?) - })) -} - -/// The berth command that runs one of berth's own actions on a host, with -/// the values given. A secret is asked for by the command itself. -fn own_command( - host: &str, - action: &berth_api::Action, - values: &std::collections::BTreeMap, -) -> Vec { - let given = |name: &str| values.get(name).map(String::as_str).unwrap_or("").trim(); - let mut words: Vec = match action.name.as_str() { - "install-controller" => vec!["host".into(), "install".into(), host.into()], - "trust-key" => { - let mut words = vec!["key".into(), "trust".into(), host.into()]; - if !given("key").is_empty() { - words.extend(["--key".into(), given("key").into()]); - } - if given("keep_password") == "yes" { - words.push("--keep-password".into()); - } - return words; - } - "rename" => { - let mut words = vec![ - "host".into(), - "rename".into(), - host.into(), - given("name").into(), - ]; - if given("hostname") == "yes" { - words.push("--hostname".into()); - } - return words; - } - "host-key" => { - let mut words = vec!["key".into(), "host".into(), host.into()]; - if given("accept") == "yes" { - words.push("--accept".into()); - } - return words; - } - name => vec!["run".into(), host.into(), name.into()], - }; - if words[0] == "run" { - words.extend( - action - .arguments - .iter() - .filter(|argument| !argument.secret && !given(&argument.name).is_empty()) - .map(|argument| format!("{}={}", argument.name, given(&argument.name))), - ); - // The question was answered on the screen. - words.push("--yes".into()); - } - words -} diff --git a/crates/berth/src/cli/args.rs b/crates/berth/src/cli/args.rs new file mode 100644 index 0000000..7edcf76 --- /dev/null +++ b/crates/berth/src/cli/args.rs @@ -0,0 +1,981 @@ +//! The grammar of the command line, `berth THING ACTION [NAME] [FLAGS]`, as +//! one definition that the help, the errors, the shell completions and the +//! reference page of the docs are all made from. + +use clap::{Args, Parser, Subcommand, ValueEnum}; +use std::path::PathBuf; + +/// What the exit statuses mean, for the help and the reference page. +pub const EXIT_STATUSES: &[(i32, &str)] = &[ + (0, "done"), + (1, "the work failed: a job failed, or a host did not answer"), + ( + 2, + "the command line was wrong: an unknown word, a missing value, a bad flag", + ), + ( + 3, + "refused: the role is too low, the host or this server cannot do it now, or a confirmation was declined", + ), + (4, "the server could not be reached"), +]; + +#[derive(Parser, Debug)] +#[command( + name = "berth", + version = berth_api::VERSION, + about = "berth sets up and looks after agent workstations.", + disable_help_subcommand = false, + propagate_version = true, + max_term_width = 100 +)] +pub struct Cli { + #[command(flatten)] + pub global: Global, + #[command(subcommand)] + pub command: Option, +} + +/// The flags every command takes. +#[derive(Args, Debug, Clone, Default)] +pub struct Global { + /// Use this berth server for this command: https://HOST, http://HOST:PORT or unix:/path. berth login keeps one. + #[arg(long, global = true, value_name = "URL", help_heading = "Global")] + pub server: Option, + /// Read the server's token from this file. + #[arg(long, global = true, value_name = "FILE", help_heading = "Global")] + pub token_file: Option, + /// Read the server's token from standard input. + #[arg(long, global = true, help_heading = "Global")] + pub token_stdin: bool, + /// What is printed on standard output: a table for people, or the API's own JSON. + #[arg(short = 'o', long, global = true, value_enum, default_value_t = Output::Table, help_heading = "Global")] + pub output: Output, + /// Never ask. What is missing is an error that names the flag to pass. + #[arg(long, global = true, help_heading = "Global")] + pub no_input: bool, + /// Answer yes to a confirmation. + #[arg(short = 'y', long, global = true, help_heading = "Global")] + pub yes: bool, + /// Print only the result. + #[arg(short = 'q', long, global = true, help_heading = "Global")] + pub quiet: bool, + /// Read the SSH user's sudo password from standard input, for a host whose sudo asks for one. + #[arg(long, global = true, help_heading = "Global")] + pub sudo_password_stdin: bool, + /// Colours: on a terminal, always or never. NO_COLOR turns them off as well. + #[arg(long, global = true, value_enum, default_value_t = Colour::Auto, value_name = "WHEN", help_heading = "Global")] + pub color: Colour, +} + +#[derive(ValueEnum, Clone, Copy, Debug, Default, PartialEq, Eq)] +pub enum Output { + #[default] + Table, + Json, +} + +#[derive(ValueEnum, Clone, Copy, Debug, Default, PartialEq, Eq)] +pub enum Colour { + #[default] + Auto, + Always, + Never, +} + +/// The flag of a command that starts a job. +#[derive(Args, Debug, Clone, Default)] +pub struct Detach { + /// Print the job's id and return; berth job wait ID waits for it. Needs a berth serve. + #[arg(long)] + pub detach: bool, +} + +#[derive(Subcommand, Debug)] +pub enum Command { + /// Short for berth host list. + Ls, + /// Short for berth host apply. + Apply(HostApply), + /// Run a command on a host and pass its exit status through. + Exec(Exec), + /// A shell on a host over plain SSH, or a command run there. Needs the installation on this machine. + Ssh(Ssh), + /// Run one of a host's actions: berth action list HOST names them. + Run(RunAction), + /// Open berth's screens: the home screen, or the page of one host. + Ui { + /// The host to open. + host: Option, + }, + /// The hosts: list, create, add, change, install and remove them. + #[command(subcommand)] + Host(Host), + /// Bring the hosts up to date, one at a time. + Rollout(Rollout), + /// What a host can do. + #[command(subcommand)] + Action(ActionCommand), + /// The terminal and process sessions of a host. + #[command(subcommand)] + Session(Session), + /// What can make hosts and resources, such as Incus or Docker. + #[command(subcommand)] + Provisioner(Provisioner), + /// What provisioners made that is not a host, such as a deployment. + #[command(subcommand)] + Resource(Resource), + /// The work the server does: applies, installers, roll-outs. + #[command(subcommand)] + Job(Job), + /// The SSH keys berth logs in to hosts with. + #[command(subcommand)] + Key(Key), + /// Secret values the server keeps and gives to hosts. + #[command(subcommand)] + Secret(Secret), + /// Registries: Nexus servers, npm, file and Docker registries. + #[command(subcommand)] + Registry(Registry), + /// Plugins and what they install. + #[command(subcommand)] + Plugin(Plugin), + /// The settings: the server's, and this berth's own, such as the server it uses. + #[command(subcommand)] + Config(Config), + /// Reach a port of a host, or an endpoint of a resource, from this machine. + #[command(subcommand)] + Port(Port), + /// Messages taken in from the hosts' channels. + #[command(subcommand)] + Inbox(Inbox), + /// The server's key-value store. + #[command(subcommand)] + Kv(Kv), + /// A host's channel. + #[command(subcommand)] + Channel(Channel), + /// Services that plugins run on a host. + #[command(subcommand)] + Service(Service), + /// The berth server of this machine: running it, its setup key, its store and its API documents. + #[cfg(target_os = "linux")] + #[command(subcommand)] + Server(Server), + /// Short for berth server run. + #[cfg(target_os = "linux")] + Serve(ServerRun), + /// Use the berth server at URL, with a setup key made there. + Login(Login), + /// Stop using the berth server berth login kept. + Logout, + /// Install the latest release in this program's place. + Update(Update), + /// The version of this berth, and of the server it uses. + Version, + /// The completions of a shell, to source from its start-up file. + Completion { + /// bash, zsh, fish, elvish or powershell. + shell: clap_complete::Shell, + }, + /// ssh's askpass helper. + #[command(hide = true)] + Askpass { prompt: Vec }, + /// A one-use enrollment key for a host, which an apply asks for. + #[cfg(target_os = "linux")] + #[command(hide = true)] + TailnetKey { host: String }, +} + +// ── host ──────────────────────────────────────────────────────────────────── + +#[derive(Subcommand, Debug)] +pub enum Host { + /// The hosts, with their address and whether each answers. + #[command(visible_alias = "ls")] + List, + /// One host: its type, address, state and actions. + Show { name: String }, + /// Describe a new host, then install it. Asks what is missing on a terminal. + #[command(after_long_help = super::setup::keys_help())] + Create(HostCreate), + /// Take in a machine that exists, by its address, and log in to it once. + Add(HostAdd), + /// Change settings of a host's profile: KEY=VALUE, as berth host create --help lists the keys. + #[command(after_long_help = super::setup::keys_help())] + Set(HostSet), + /// Change a host's profile in $EDITOR. + Edit { name: String }, + /// A host's profile as JSON on standard output. + Export { name: String }, + /// Save the profile in FILE (- for standard input) as the host's, new or changed. + Import { + name: String, + /// The profile as JSON, such as berth host export wrote; - for standard input. + #[arg(value_name = "FILE")] + file: String, + }, + /// Apply the host's profile to it over SSH, or install its controller alone when it is managed. + Apply(HostApply), + /// The cloud-init user data that installs the host on first boot, on standard output. + Seed { + name: String, + /// Write it to FILE instead. + #[arg(long, value_name = "FILE")] + file: Option, + }, + /// Build the host's unattended USB installer where the server runs, and write it to a drive. + Iso(HostIso), + /// Give a host another name in berth. The machine is left as it is. + Rename { + name: String, + new: String, + /// Make NEW its profile's hostname as well, for the next apply. + #[arg(long)] + hostname: bool, + }, + /// Forget a host. The machine is left as it is, unless --destroy. + #[command(visible_alias = "rm")] + Remove { + name: String, + /// Delete the machine a provisioner made for it as well. + #[arg(long)] + destroy: bool, + }, + /// Take in a host berth set up before, from the profile it keeps. + Adopt(HostAdopt), + /// The passwords berth generated for a host's accounts. + Credentials { + name: String, + /// Print them even when standard output is no terminal. + #[arg(long)] + show: bool, + }, + /// Check the host's SSH host key, and accept a new one. + Verify { + name: String, + /// Accept the key the host shows now. + #[arg(long)] + accept: bool, + }, + /// Start the machine a provisioner made for the host. + Start(HostPower), + /// Stop the machine a provisioner made for the host. + Stop(HostPower), + /// The hosts as an Ansible inventory. Needs the installation on this machine. + #[cfg(target_os = "linux")] + Inventory { + /// The hosts; all with an address when none is named. + names: Vec, + }, +} + +#[derive(Args, Debug, Clone, Default)] +pub struct HostCreate { + pub name: String, + /// How it is installed: provisioner, iso, cloud-seed, ssh or ssh-only. + #[arg(long, value_name = "METHOD")] + pub method: Option, + /// Start from a profile: a host's name, a JSON file, or - for standard input. + #[arg(long, value_name = "PROFILE")] + pub from: Option, + /// What berth does with it: workstation, or node for a server of machines, applied over SSH. + #[arg(long = "type", value_name = "TYPE")] + pub host_type: Option, + /// Switch on a plugin's feature, PLUGIN/FEATURE. Again for more. + #[arg(long, value_name = "FEATURE")] + pub feature: Vec, + /// Switch off a plugin's feature. + #[arg(long, value_name = "FEATURE")] + pub no_feature: Vec, + /// Set a field of the profile. Again for more; the keys are listed below. + #[arg(long = "set", value_name = "KEY=VALUE")] + pub set: Vec, + /// Generate the accounts' passwords; berth host credentials NAME shows them. + #[arg(long)] + pub generate_passwords: bool, + /// Make the machine with this provisioner, which also sets it up. + #[arg(long, value_name = "PROVISIONER")] + pub provisioner: Option, + /// The provisioner's arguments, such as cpus=2. + #[arg(value_name = "NAME=VALUE")] + pub arguments: Vec, + /// Install it once it is saved: apply, render the seed or build the installer. + #[arg(long)] + pub run: bool, + #[command(flatten)] + pub detach: Detach, +} + +#[derive(Args, Debug, Clone)] +pub struct HostAdd { + pub name: String, + /// Its name or IP address, or a Host alias of the server's SSH configuration. + pub address: String, + /// The account berth logs in as. + #[arg(long)] + pub user: Option, + /// Its SSH port, when not 22. + #[arg(long)] + pub port: Option, + /// How berth logs in: berth, key:NAME, password or ssh. + #[arg(long, value_name = "AUTH")] + pub auth: Option, + /// What berth does with it: ssh, managed, node or workstation. + #[arg(long = "type", value_name = "TYPE", default_value = "ssh")] + pub host_type: String, + /// Switch on a plugin's feature of a node or a workstation, PLUGIN/FEATURE, such as incus/server. Again for more. + #[arg(long, value_name = "FEATURE")] + pub feature: Vec, + /// Save it without logging in to it. + #[arg(long)] + pub no_contact: bool, + /// Read a password from standard input that installs berth's key on it, once. + #[arg(long)] + pub password_stdin: bool, +} + +#[derive(Args, Debug, Clone)] +pub struct HostSet { + pub name: String, + /// KEY=VALUE, where KEY is a field (berth host create --help), feature.PLUGIN/FEATURE, secret.NAME or registry.NAME. + #[arg(value_name = "KEY=VALUE", required = true)] + pub values: Vec, +} + +#[derive(Args, Debug, Clone, Default)] +pub struct HostApply { + pub name: String, + #[command(flatten)] + pub detach: Detach, +} + +#[derive(Args, Debug, Clone)] +pub struct HostIso { + pub name: String, + /// Write it to this USB drive, which is erased, such as /dev/sdb. + #[arg(long, value_name = "DEVICE")] + pub write: Option, + /// Read your sudo password for writing the drive from standard input. + #[arg(long)] + pub password_stdin: bool, + #[command(flatten)] + pub detach: Detach, +} + +#[derive(Args, Debug, Clone)] +pub struct HostAdopt { + pub name: String, + /// [USER@]ADDRESS; without it, the saved host of that name. + pub address: Option, + /// Its SSH port, when not 22. + #[arg(long)] + pub port: Option, + /// A private key file where the server runs, offered as well for reading the profile only. + #[arg(long, value_name = "FILE")] + pub key: Option, + /// Replace a host of that name. + #[arg(long)] + pub force: bool, + #[command(flatten)] + pub detach: Detach, +} + +#[derive(Args, Debug, Clone)] +pub struct HostPower { + pub name: String, + #[command(flatten)] + pub detach: Detach, +} + +#[derive(Args, Debug, Clone)] +pub struct Exec { + pub host: String, + /// The account it runs as: agent, operator or root. + #[arg(long, default_value = "agent")] + pub user: String, + /// Start it and return; berth session logs HOST ID shows what it writes. + #[arg(long)] + pub detach: bool, + /// Keep the session once it has ended, for berth session logs. + #[arg(long)] + pub keep: bool, + /// The command and its arguments, after --. + #[arg(last = true, required = true, value_name = "COMMAND")] + pub command: Vec, +} + +#[derive(Args, Debug, Clone)] +pub struct Ssh { + pub host: String, + /// Log in as the agent account. + #[arg(long)] + pub agent: bool, + /// A command to run there, after --. + #[arg(last = true, value_name = "COMMAND")] + pub command: Vec, +} + +#[derive(Args, Debug, Clone)] +pub struct RunAction { + pub host: String, + pub action: String, + /// The action's arguments. + #[arg(value_name = "NAME=VALUE")] + pub arguments: Vec, + #[command(flatten)] + pub detach: Detach, +} + +#[derive(Args, Debug, Clone, Default)] +pub struct Rollout { + /// Only these hosts. Again for more. + #[arg(long, value_name = "HOST")] + pub only: Vec, + /// Leave these hosts out. Again for more. + #[arg(long, value_name = "HOST")] + pub except: Vec, + /// Take hosts that are up to date as well. + #[arg(long)] + pub all: bool, + /// Run each host's update action as well. + #[arg(long)] + pub update: bool, + /// Reboot the hosts that need it, unless sessions run there. + #[arg(long)] + pub reboot: bool, + /// With --reboot: even when sessions run. + #[arg(long, requires = "reboot")] + pub force: bool, + #[command(flatten)] + pub detach: Detach, +} + +#[derive(Subcommand, Debug)] +pub enum ActionCommand { + /// What a host can do, or the installation without a host. + #[command(visible_alias = "ls")] + List { host: Option }, +} + +#[derive(Subcommand, Debug)] +pub enum Session { + /// A host's sessions, running and ended. + #[command(visible_alias = "ls")] + List { host: String }, + /// This terminal onto a terminal session, or onto a new shell as the agent. Ctrl+Q leaves it running. + Attach { host: String, id: Option }, + /// What a session wrote. + Logs { + host: String, + id: String, + /// Go on as it writes more. + #[arg(short = 'f', long)] + follow: bool, + /// Only the last N lines. + #[arg(long, value_name = "N")] + tail: Option, + }, + /// End a session. + Stop { host: String, id: String }, +} + +// ── provisioners and resources ────────────────────────────────────────────── + +#[derive(Subcommand, Debug)] +pub enum Provisioner { + /// The provisioners, and whether each can be used here. + #[command(visible_alias = "ls")] + List, + /// The types of provisioner the plugins bring. + Types, + /// Register a provisioner of a type, such as incus/vm, under a name of its own. + Add { + name: String, + /// Its type, as berth provisioner types lists them. + #[arg(value_name = "TYPE")] + kind: String, + /// Its settings. + #[arg(value_name = "NAME=VALUE")] + settings: Vec, + }, + /// How it is now and what it holds. NAME=VALUE gives one of its settings another value for this once. + Show { + name: String, + /// A setting with another value for this once. + #[arg(value_name = "NAME=VALUE")] + settings: Vec, + }, + /// Change its settings. + Set { + name: String, + /// A setting and its new value. + #[arg(value_name = "NAME=VALUE", required = true)] + settings: Vec, + }, + /// Forget a provisioner that holds nothing. + #[command(visible_alias = "rm")] + Remove { name: String }, + /// What it can do. + Actions { name: String }, + /// Run one of its actions. + Run { + name: String, + action: String, + /// The action's arguments; a required one that is missing is asked for. + #[arg(value_name = "NAME=VALUE")] + arguments: Vec, + #[command(flatten)] + detach: Detach, + }, +} + +#[derive(Subcommand, Debug)] +pub enum Resource { + /// What provisioners made. + #[command(visible_alias = "ls")] + List { + /// Only those of this kind, such as deployment or machine. + #[arg(long)] + kind: Option, + /// Only those this provisioner made. + #[arg(long)] + provisioner: Option, + /// Only the machine of this host. + #[arg(long)] + host: Option, + }, + /// One of them as its provisioner says it is now. + Show { id: String }, + /// Make a resource, such as a deployment from a Compose file. + Create(ResourceCreate), + /// Change a resource: other arguments, a new specification, other secrets. + Set(ResourceSet), + /// Remove a resource with its provisioner. + #[command(visible_alias = "rm")] + Remove { + id: String, + #[command(flatten)] + detach: Detach, + }, + /// What it wrote. + Logs { + id: String, + /// Go on as it writes more. + #[arg(short = 'f', long)] + follow: bool, + /// Only the last N lines. + #[arg(long, value_name = "N")] + tail: Option, + }, + /// Run a command in it, for a kind that has exec, and exit with its exit status. + Exec { + id: String, + /// Give the command what berth reads on standard input, to its end. + #[arg(long)] + stdin: bool, + /// The command and its arguments, after --. + #[arg(last = true, required = true, value_name = "COMMAND")] + command: Vec, + #[command(flatten)] + detach: Detach, + }, + /// What it can do. + Actions { id: String }, + /// Run one of its actions. + Run { + id: String, + action: String, + /// The action's arguments; a required one that is missing is asked for. + #[arg(value_name = "NAME=VALUE")] + arguments: Vec, + #[command(flatten)] + detach: Detach, + }, +} + +#[derive(Args, Debug, Clone)] +pub struct ResourceCreate { + pub name: String, + /// The provisioner that makes it. + #[arg(long, required = true)] + pub provisioner: String, + /// Its kind, where the provisioner makes several. + #[arg(long)] + pub kind: Option, + /// The specification, such as a Compose file; - for standard input. + #[arg(long, value_name = "FILE")] + pub spec: Option, + /// A secret it gets, NAME or NAME:VARIABLE. Again for more. + #[arg(long, value_name = "NAME[:VARIABLE]")] + pub secret: Vec, + /// The arguments of its kind. + #[arg(value_name = "NAME=VALUE")] + pub arguments: Vec, + #[command(flatten)] + pub detach: Detach, +} + +#[derive(Args, Debug, Clone)] +pub struct ResourceSet { + pub id: String, + /// A new specification; - for standard input. + #[arg(long, value_name = "FILE")] + pub spec: Option, + /// The secrets it gets from now on, NAME or NAME:VARIABLE. Again for more. + #[arg(long, value_name = "NAME[:VARIABLE]")] + pub secret: Vec, + /// Arguments with new values. + #[arg(value_name = "NAME=VALUE")] + pub arguments: Vec, + #[command(flatten)] + pub detach: Detach, +} + +// ── jobs ──────────────────────────────────────────────────────────────────── + +#[derive(Subcommand, Debug)] +pub enum Job { + /// The jobs, newest first. + #[command(visible_alias = "ls")] + List { + #[arg(long, default_value_t = 20)] + limit: u32, + }, + /// One job's state. + Show { id: String }, + /// What a job wrote. + Logs { + id: String, + /// Go on as it writes more, until it ends. + #[arg(short = 'f', long)] + follow: bool, + }, + /// Wait for a job to end and exit with its result. + Wait { id: String }, + /// Stop a running job: its programs end, and its record says who stopped it. + Stop { id: String }, +} + +// ── keys, secrets, registries, plugins, settings ──────────────────────────── + +#[derive(Subcommand, Debug)] +pub enum Key { + /// The keys of berth's key ring, by their public halves. + #[command(visible_alias = "ls")] + List, + /// One key's public half, which a host's operator takes. + Show { + #[arg(default_value = "berth")] + name: String, + }, + /// Make a new ed25519 key. + Create { name: String }, + /// Add an OpenSSH private key without a passphrase from FILE (- for standard input). + Import { + name: String, + /// The private key file, in OpenSSH format. + #[arg(value_name = "FILE")] + file: String, + }, + /// Remove a key no host logs in with. + #[command(visible_alias = "rm")] + Remove { name: String }, + /// Replace a key on every host that takes it. + Rotate { + name: String, + #[command(flatten)] + detach: Detach, + }, + /// Install a key on a host that does not take it yet, with a password used once. + Trust { + host: String, + /// Log in once with this key of the ring, to install the one the host logs in with. + #[arg(long, value_name = "NAME")] + key: Option, + /// Read the password from standard input. + #[arg(long)] + password_stdin: bool, + }, +} + +#[derive(Subcommand, Debug)] +pub enum Secret { + /// The names of the secrets. + #[command(visible_alias = "ls")] + List, + /// Set a secret's value: asked for, or read from a file or standard input. + Set { + name: String, + /// Read the value from FILE. + #[arg(long, value_name = "FILE")] + value_file: Option, + }, + /// Remove a secret. + #[command(visible_alias = "rm")] + Remove { name: String }, +} + +#[derive(Subcommand, Debug)] +pub enum Registry { + /// The registries. + #[command(visible_alias = "ls")] + List, + /// Add a registry, or change its address or user. + Add { + name: String, + /// nexus, npm, raw or docker. + #[arg(value_name = "TYPE")] + kind: String, + url: String, + /// The user of its login; without one the login is a token. + #[arg(long)] + user: Option, + }, + /// Add a repository of a Nexus registry. + AddRepo { + name: String, + /// npm, raw or docker. + format: String, + /// The repository's name on the registry. + repository: String, + /// Its own address, HOST[:PORT], for a Docker repository. + #[arg(long)] + address: Option, + }, + /// Keep the password or token of a registry. + Login { + name: String, + /// Read it from standard input. + #[arg(long)] + password_stdin: bool, + }, + /// Whether a registry answers and takes its login. + Check { name: Option }, + /// Remove a registry with its login. + #[command(visible_alias = "rm")] + Remove { name: String }, + /// Upload the release archive to a raw repository or file server, and use it as HOST's release address. + #[cfg(target_os = "linux")] + Publish { + /// REGISTRY or REGISTRY/REPOSITORY. + destination: String, + #[arg(long)] + host: Option, + /// The cloud seed carries this registry's login; the destination's without a name. + #[arg(long, value_name = "REGISTRY", num_args = 0..=1, default_missing_value = "")] + login: Option, + }, + /// Push berth's container image. + #[cfg(target_os = "linux")] + PushImage { + /// REGISTRY or REGISTRY/REPOSITORY. + destination: String, + #[arg(default_value = "latest")] + tag: String, + }, +} + +#[derive(Subcommand, Debug)] +pub enum Plugin { + /// The plugins and where they come from. + #[command(visible_alias = "ls")] + List, + /// What a plugin brings: its features with their actions, services and channels, and its provisioners. + Show { id: String }, + /// Install a plugin with npm: a name, or the address of a tarball. + Add { spec: String }, + /// Remove one of the server's own plugins. + #[command(visible_alias = "rm")] + Remove { id: String }, + /// Update one plugin or all to their registry's latest version. + Update { + id: Option, + /// Only say which have a newer version. + #[arg(long)] + check: bool, + }, +} + +#[derive(Subcommand, Debug)] +pub enum Config { + /// The settings, with what each means and where it is kept. + #[command(visible_alias = "ls")] + List, + /// One setting's value. + Get { key: String }, + /// Change a setting. + Set { key: String, value: String }, + /// Set a setting back to its default. + Unset { key: String }, +} + +#[derive(Subcommand, Debug)] +pub enum Port { + /// Listen on this machine and pass each connection to a port of a host, or to an endpoint of a resource. + /// + /// TARGET is a host's name. A resource's id or name is taken when no host has that name, or always with --resource. + Forward { + /// A host's name, or a resource's id or name. + #[arg(value_name = "TARGET")] + target: String, + /// [LOCAL:]PORT for a host's port. For a resource, [LOCAL:]ENDPOINT, an endpoint's name or port; LOCAL is the endpoint's port unless given. + #[arg(value_name = "[LOCAL:]PORT|ENDPOINT", required = true)] + ports: Vec, + /// TARGET is a resource, even where a host has the same name. + #[arg(long)] + resource: bool, + /// The address to listen on. + #[arg(long, default_value = "127.0.0.1")] + address: String, + }, +} + +#[derive(Subcommand, Debug)] +pub enum Inbox { + /// The messages, oldest first. + #[command(visible_alias = "ls")] + List { + host: Option, + /// Only topics that start with PREFIX. + #[arg(long, value_name = "PREFIX")] + topic: Option, + /// Done ones as well. + #[arg(long)] + all: bool, + }, +} + +#[derive(Subcommand, Debug)] +pub enum Kv { + /// The keys of a namespace with their values. + #[command(visible_alias = "ls")] + List { namespace: String }, + /// One key's value. + Get { namespace: String, key: String }, + /// Write a key: JSON, or else text. + Set { + namespace: String, + key: String, + value: String, + }, + /// Remove a key. + #[command(visible_alias = "rm")] + Remove { namespace: String, key: String }, +} + +#[derive(Subcommand, Debug)] +pub enum Channel { + /// Messages on a host's channel as they come, one JSON object per line. + Listen { + host: String, + /// Only topics that start with PREFIX. + #[arg(long, value_name = "PREFIX")] + topic: Option, + /// From after this message on. + #[arg(long, value_name = "ID")] + since: Option, + }, +} + +#[derive(Subcommand, Debug)] +pub enum Service { + /// Call a method of a service a plugin runs on the host. + Call { + host: String, + service: String, + method: String, + /// The body, as JSON. + body: Option, + }, +} + +// ── the server of this machine ────────────────────────────────────────────── + +#[cfg(target_os = "linux")] +#[derive(Subcommand, Debug)] +pub enum Server { + /// Serve the API over all hosts on a socket of this user, and over HTTP with --listen. + Run(ServerRun), + /// Make a key, valid for a day, that lets the first user collect the server's token. + SetupKey { + /// End the key that is open. + #[arg(long)] + revoke: bool, + }, + /// Where the server keeps its state, and sealing it. + #[command(subcommand)] + Store(Store), + /// The server's API documents. + #[command(subcommand)] + Api(Api), +} + +#[cfg(target_os = "linux")] +#[derive(Args, Debug, Clone, Default)] +pub struct ServerRun { + /// Also listen over HTTP, behind a token. + #[arg(long, value_name = "ADDRESS:PORT")] + pub listen: Option, + /// Allow an address other than this machine's own. + #[arg(long, requires = "listen")] + pub expose: bool, + /// Taken and left unused: the page in the browser of berth 0.5 is gone, + /// and containers and units of that version still name it. + #[arg(long, hide = true)] + pub web: bool, +} + +#[cfg(target_os = "linux")] +#[derive(Subcommand, Debug)] +pub enum Store { + /// Where the store is, and whether it is sealed. + Status, + /// Seal the store's values with a new key file. + Encrypt, + /// Seal them with another key. + Rekey, + /// Write them plain again. + Decrypt, + /// Copy the store into another SQLite file or PostgreSQL database, and switch to it. + Copy { + /// A file, a PostgreSQL URL, or - to read it from standard input. + target: Option, + /// Read the target from FILE. + #[arg(long, value_name = "FILE")] + target_file: Option, + /// Copy into a database that holds a store already. + #[arg(long)] + into_existing: bool, + }, +} + +#[cfg(target_os = "linux")] +#[derive(Subcommand, Debug)] +pub enum Api { + /// The OpenAPI document. + Openapi, + /// The AsyncAPI document of the streams. + Asyncapi, + /// The operations, with their routes and roles. + Operations, +} + +#[derive(Args, Debug, Clone)] +pub struct Login { + /// https://HOST or http://HOST:PORT. + pub url: String, + /// Read the setup key from standard input. + #[arg(long)] + pub setup_key_stdin: bool, +} + +#[derive(Args, Debug, Clone)] +pub struct Update { + /// Only say whether there is a newer release. + #[arg(long)] + pub check: bool, + /// Where releases are, instead of berth's own. + #[arg(long, value_name = "URL")] + pub from: Option, +} diff --git a/crates/berth/src/cli/ask.rs b/crates/berth/src/cli/ask.rs new file mode 100644 index 0000000..40da609 --- /dev/null +++ b/crates/berth/src/cli/ask.rs @@ -0,0 +1,460 @@ +//! Questions asked inline, below the command: a line of text, a hidden +//! password, a choice, a list to tick, a confirmation. Each is drawn with +//! ratatui in a few lines of the terminal and leaves one line behind with +//! its answer. Nobody is asked without a terminal or with `--no-input`. + +use super::fail::{Fail, Result}; +use super::out; +use berth_tui::flow::{Answer, Asked, Asker, Choice, Question}; +use berth_tui::theme::{self, Role}; +use crossterm::event::{self, Event, KeyCode, KeyEvent, KeyEventKind, KeyModifiers}; +use ratatui::{ + Terminal, TerminalOptions, Viewport, + backend::CrosstermBackend, + style::{Modifier, Style}, + text::{Line, Span}, + widgets::{Paragraph, Wrap}, +}; +use tui_input::Input; + +/// The most rows of choices shown at once. +const ROWS: usize = 8; + +/// What a question has been answered so far. +struct State { + input: Input, + cursor: usize, + ticked: Vec, + /// Why the last answer was not taken. + said: Option, +} + +fn accent() -> Style { + out::look(Role::Accent) +} + +fn dim() -> Style { + out::look(Role::Muted) +} + +/// The first sentence of a help text, which is what fits under a question. +fn first_sentence(help: &str) -> &str { + match help.find(". ") { + Some(at) => &help[..=at], + None => help, + } +} + +fn options_of(question: &Question) -> &[Choice] { + match &question.asked { + Asked::Choice { options, .. } | Asked::Many { options, .. } => options, + _ => &[], + } +} + +/// The lines of a question while it is asked. +fn lines(question: &Question, state: &State) -> Vec> { + let mark = Span::styled("? ", accent().add_modifier(Modifier::BOLD)); + let label = Span::styled( + question.label.clone(), + Style::new().add_modifier(Modifier::BOLD), + ); + let mut lines = vec![]; + match &question.asked { + Asked::Text { secret, .. } => { + let shown = match secret { + true => "•".repeat(state.input.value().chars().count()), + false => state.input.value().to_owned(), + }; + lines.push(Line::from(vec![ + mark, + label, + Span::raw(" › "), + Span::raw(shown), + ])); + lines.push(Line::styled( + first_sentence(&question.help).to_owned(), + dim(), + )); + } + Asked::Confirm { default } => { + let hint = if *default { + " (Y/n) › " + } else { + " (y/N) › " + }; + lines.push(Line::from(vec![mark, label, Span::styled(hint, dim())])); + } + Asked::Choice { options, .. } | Asked::Many { options, .. } => { + let many = matches!(question.asked, Asked::Many { .. }); + let keys = match many { + true => " ↑↓ move · space tick · enter done", + false => " ↑↓ move · enter choose", + }; + lines.push(Line::from(vec![mark, label, Span::styled(keys, dim())])); + let first = state.cursor.saturating_sub(ROWS - 1); + for (index, option) in options.iter().enumerate().skip(first).take(ROWS) { + let here = index == state.cursor; + let pointer = if here { theme::CURSOR } else { " " }; + let tick = match (many, state.ticked.get(index)) { + (true, Some(true)) => "◉ ", + (true, _) => "○ ", + (false, _) => "", + }; + let style = if here { accent() } else { Style::new() }; + lines.push(Line::from(vec![ + Span::styled(format!(" {pointer}{tick}"), style), + Span::styled(option.label.clone(), style), + ])); + } + let help = options + .get(state.cursor) + .map(|option| option.help.as_str()) + .filter(|help| !help.is_empty()) + .unwrap_or(first_sentence(&question.help)); + lines.push(Line::styled(first_sentence(help).to_owned(), dim())); + } + } + if let Some(said) = &state.said { + lines.push(Line::styled(said.clone(), out::look(Role::Bad))); + } + lines +} + +/// What a question was answered, as the line it leaves behind shows it. +fn shown(question: &Question, answer: &Answer) -> String { + let options = options_of(question); + match answer { + Answer::Text(_) if matches!(question.asked, Asked::Text { secret: true, .. }) => { + "••••••".into() + } + Answer::Text(text) if text.is_empty() => "none".into(), + Answer::Text(text) => text.clone(), + Answer::Confirm(yes) => if *yes { "yes" } else { "no" }.into(), + Answer::Choice(index) => options + .get(*index) + .map(|option| option.label.clone()) + .unwrap_or_default(), + Answer::Many(ticked) => { + let chosen: Vec<&str> = options + .iter() + .zip(ticked) + .filter(|(_, on)| **on) + .map(|(option, _)| option.key.as_str()) + .collect(); + match chosen.is_empty() { + true => "none".into(), + false => chosen.join(", "), + } + } + } +} + +/// What a key does to a question: its answer when it ends it. +fn key(question: &Question, state: &mut State, key: KeyEvent) -> Result> { + let count = options_of(question).len(); + if key.modifiers.contains(KeyModifiers::CONTROL) + && matches!(key.code, KeyCode::Char('c') | KeyCode::Char('d')) + { + return Err(Fail::exit(130)); + } + if key.code == KeyCode::Esc { + return Err(Fail::exit(130)); + } + state.said = None; + match &question.asked { + Asked::Text { .. } => match key.code { + KeyCode::Enter => return Ok(Some(Answer::Text(state.input.value().to_owned()))), + _ => { + if let Some(key) = berth_page::keys::key(&key) { + berth_tui::event::edit(&mut state.input, &key); + } + } + }, + Asked::Confirm { default } => match key.code { + KeyCode::Enter => return Ok(Some(Answer::Confirm(*default))), + KeyCode::Char('y' | 'Y') => return Ok(Some(Answer::Confirm(true))), + KeyCode::Char('n' | 'N') => return Ok(Some(Answer::Confirm(false))), + _ => {} + }, + Asked::Choice { .. } | Asked::Many { .. } => match key.code { + KeyCode::Up | KeyCode::Char('k') => state.cursor = state.cursor.saturating_sub(1), + KeyCode::Down | KeyCode::Char('j') => { + state.cursor = (state.cursor + 1).min(count.saturating_sub(1)) + } + KeyCode::Char(' ') if matches!(question.asked, Asked::Many { .. }) => { + if let Some(on) = state.ticked.get_mut(state.cursor) { + *on = !*on; + } + } + KeyCode::Enter => { + return Ok(Some(match question.asked { + Asked::Many { .. } => Answer::Many(state.ticked.clone()), + _ => Answer::Choice(state.cursor), + })); + } + _ => {} + }, + } + Ok(None) +} + +fn start(question: &Question) -> State { + let (input, cursor, ticked) = match &question.asked { + Asked::Text { default, .. } => (Input::new(default.clone()), 0, vec![]), + Asked::Choice { default, .. } => (Input::default(), *default, vec![]), + Asked::Many { chosen, .. } => (Input::default(), 0, chosen.clone()), + Asked::Confirm { .. } => (Input::default(), 0, vec![]), + }; + State { + input, + cursor, + ticked, + said: None, + } +} + +/// Ask one question on the terminal, drawn on standard error. +fn on_terminal(question: &Question) -> Result { + let mut state = start(question); + let height = (lines(question, &state).len() + 1) as u16; + crossterm::terminal::enable_raw_mode()?; + let backend = CrosstermBackend::new(std::io::stderr()); + let mut terminal = Terminal::with_options( + backend, + TerminalOptions { + viewport: Viewport::Inline(height), + }, + )?; + let answered = (|| -> Result { + loop { + terminal.draw(|frame| { + let text = lines(question, &state); + frame.render_widget( + Paragraph::new(text).wrap(Wrap { trim: false }), + frame.area(), + ); + if let Asked::Text { .. } = question.asked { + let at = question.label.chars().count() + 5 + state.input.visual_cursor(); + frame.set_cursor_position((at as u16, frame.area().y)); + } + })?; + if let Event::Key(pressed) = event::read()? + && pressed.kind != KeyEventKind::Release + && let Some(answer) = key(question, &mut state, pressed)? + { + return Ok(answer); + } + } + })(); + let _ = terminal.clear(); + out::to_line_of(&mut terminal); + let _ = crossterm::terminal::disable_raw_mode(); + drop(terminal); + let answer = answered?; + eprintln!( + "{} {} › {}", + out::styled_err("?", out::Style::Accent), + question.label, + out::styled_err(&shown(question, &answer), out::Style::Accent) + ); + Ok(answer) +} + +/// Asks on the terminal when there is one and `--no-input` was not given. +pub struct Person; + +impl Asker for Person { + fn ask(&mut self, question: &Question) -> std::result::Result, String> { + if !out::get().ask { + return Ok(None); + } + match on_terminal(question) { + Ok(answer) => Ok(Some(answer)), + Err(fail) if fail.status == 130 => Err("Stopped".into()), + Err(fail) => Err(fail.message), + } + } +} + +/// The asker of this run. +pub fn asker() -> Person { + Person +} + +/// Ask for a line of text, or say which flag gives it. +pub fn text(label: &str, help: &str, flag: &str) -> Result { + let question = Question { + step: berth_tui::fields::Step::Install, + label: label.into(), + help: help.into(), + flag: flag.into(), + asked: Asked::Text { + default: String::new(), + secret: false, + }, + required: true, + }; + match ask(&question)? { + Answer::Text(text) => Ok(text), + _ => Err(Fail::usage(format!("{label} is missing: pass {flag}"))), + } +} + +/// Ask for a secret without showing it, or say which flag gives it. +pub fn secret(label: &str, flag: &str) -> Result { + let question = Question { + step: berth_tui::fields::Step::Install, + label: label.into(), + help: String::new(), + flag: flag.into(), + asked: Asked::Text { + default: String::new(), + secret: true, + }, + required: true, + }; + match ask(&question)? { + Answer::Text(text) => Ok(text), + _ => Err(Fail::usage(format!("{label} is missing: pass {flag}"))), + } +} + +/// Ask a question, or fail with its flag when nobody can be asked. +pub fn ask(question: &Question) -> Result { + if !out::get().ask { + return Err(Fail::usage(format!( + "{} is missing: pass {}", + question.label, question.flag + ))); + } + on_terminal(question).map_err(|fail| match fail.status { + 130 => Fail::exit(130), + _ => fail, + }) +} + +/// Confirm something that cannot be undone: `--yes` does, and without a +/// terminal it is refused. +pub fn confirm(question: &str) -> Result<()> { + let out = out::get(); + if out.yes { + return Ok(()); + } + if !out.ask { + return Err(Fail::refused(format!( + "{question} Confirm with --yes, as nobody can be asked here" + ))); + } + let asked = Question { + step: berth_tui::fields::Step::Run, + label: question.into(), + help: String::new(), + flag: "--yes".into(), + asked: Asked::Confirm { default: false }, + required: true, + }; + match on_terminal(&asked)? { + Answer::Confirm(true) => Ok(()), + _ => Err(Fail::refused("Not confirmed, so nothing was done")), + } +} + +/// A secret from a file, from standard input, or asked for: never from the +/// command line. +pub fn secret_from( + label: &str, + file: Option<&std::path::Path>, + stdin: bool, + flag: &str, +) -> Result { + if let Some(file) = file { + let text = std::fs::read_to_string(file) + .map_err(|error| Fail::usage(format!("{}: {error}", file.display())))?; + return Ok(text.trim_end_matches(['\n', '\r']).to_owned()); + } + let piped = !std::io::IsTerminal::is_terminal(&std::io::stdin()); + if stdin || piped { + let mut text = String::new(); + std::io::Read::read_to_string(&mut std::io::stdin(), &mut text)?; + return Ok(text.trim_end_matches(['\n', '\r']).to_owned()); + } + secret(label, flag) +} + +#[cfg(test)] +mod tests { + use super::*; + use berth_tui::fields::Step; + + fn press(code: KeyCode) -> KeyEvent { + KeyEvent::new(code, KeyModifiers::NONE) + } + + fn choice() -> Question { + Question { + step: Step::Install, + label: "How it is installed".into(), + help: "The ways.".into(), + flag: "--method METHOD".into(), + asked: Asked::Choice { + options: vec![ + Choice { + key: "provisioner".into(), + label: "From a provisioner".into(), + help: String::new(), + }, + Choice { + key: "iso".into(), + label: "USB installer".into(), + help: String::new(), + }, + ], + default: 0, + }, + required: true, + } + } + + #[test] + fn keys_move_through_a_choice_and_enter_takes_it() { + let question = choice(); + let mut state = start(&question); + assert_eq!( + key(&question, &mut state, press(KeyCode::Down)).unwrap(), + None + ); + let shown = lines(&question, &state); + assert!( + shown[2].to_string().contains("▌ USB installer"), + "{shown:?}" + ); + let answer = key(&question, &mut state, press(KeyCode::Enter)).unwrap(); + assert_eq!(answer, Some(Answer::Choice(1))); + assert_eq!(shown_answer(&question, 1), "USB installer"); + assert!(key(&question, &mut state, press(KeyCode::Esc)).is_err()); + } + + fn shown_answer(question: &Question, index: usize) -> String { + shown(question, &Answer::Choice(index)) + } + + #[test] + fn typed_text_is_the_answer_and_a_secret_is_not_shown() { + let question = Question { + asked: Asked::Text { + default: String::new(), + secret: true, + }, + ..choice() + }; + let mut state = start(&question); + for ch in "s3cret".chars() { + key(&question, &mut state, press(KeyCode::Char(ch))).unwrap(); + } + assert!(lines(&question, &state)[0].to_string().ends_with("••••••")); + let answer = key(&question, &mut state, press(KeyCode::Enter)).unwrap(); + assert_eq!(answer, Some(Answer::Text("s3cret".into()))); + assert_eq!(shown(&question, &Answer::Text("s3cret".into())), "••••••"); + } +} diff --git a/crates/berth/src/cli/client.rs b/crates/berth/src/cli/client.rs new file mode 100644 index 0000000..3206d75 --- /dev/null +++ b/crates/berth/src/cli/client.rs @@ -0,0 +1,304 @@ +//! The server a command talks to, the same way the screens do: the one +//! named by `--server` or `BERTH_SERVER`, the one `berth login` kept, a +//! `berth serve` of this user on this machine, or one in this process for +//! as long as the command runs. Every command is a call of its API. + +use super::args::Global; +use super::fail::{Fail, Result}; +use berth_api::portal::{Portal, PortalStream}; +use berth_page::net; +use berth_page::settings::{self, Settings}; +use serde_json::{Value, json}; +use std::future::Future; + +/// What is done when the command ends: the server in this process waits for +/// the jobs it runs. +type Finish = Box; + +pub struct Client { + /// The server's address, as `--server` takes it. + pub url: String, + portal: Portal, + finish: Option, + /// The hosts whose sudo password the server in this process was given + /// or found it needs none, in this run. + sudo_ready: std::cell::RefCell>, +} + +/// The settings of this user that `berth login` keeps. +pub fn settings() -> Settings { + Settings::new(settings::folder(|name| std::env::var(name).ok())) +} + +/// Take in what berth-remote kept, once, before a server is looked for, and +/// say what became of it. Nothing is lost: its file goes only once berth has +/// what was in it. +pub fn take_in_remote(kept: &Settings, old: Option<&std::path::Path>) { + use super::out; + match kept.take_in(old) { + Ok(settings::TakenIn::Taken(server)) => { + out::note(format!("berth-remote's server is berth's now: {server}")) + } + Ok(settings::TakenIn::Left { theirs, ours, file }) => out::note(format!( + "berth-remote's settings in {} name the server {theirs}, and berth uses {ours}: the file is left as it is. To use {theirs}: berth login {theirs}", + file.display() + )), + Ok(settings::TakenIn::Nothing | settings::TakenIn::Same) => {} + Err(error) => out::warn(format!( + "berth-remote's settings were not taken in, and are left as they are: {error}" + )), + } +} + +/// berth-remote's folder of settings for this user. +pub fn remote_folder() -> Option { + settings::old_folder(|name| std::env::var(name).ok()) +} + +fn variable(name: &str) -> Option { + std::env::var(name) + .ok() + .map(|value| value.trim().to_owned()) + .filter(|value| !value.is_empty()) +} + +/// The token of this run: from a file, standard input, `BERTH_TOKEN`, or +/// the one `berth login` kept for that server. +fn token(global: &Global, url: &str, kept: &Settings) -> Result { + if let Some(file) = &global.token_file { + let text = std::fs::read_to_string(file) + .map_err(|error| Fail::usage(format!("--token-file {}: {error}", file.display())))?; + return Ok(text.trim().to_owned()); + } + if global.token_stdin { + let mut text = String::new(); + std::io::Read::read_to_string(&mut std::io::stdin(), &mut text)?; + return Ok(text.trim().to_owned()); + } + if let Some(token) = variable("BERTH_TOKEN") { + return Ok(token); + } + let same = kept + .get(settings::SERVER) + .is_some_and(|server| server.trim_end_matches('/') == url); + Ok(match same { + true => kept.get(settings::TOKEN).unwrap_or_default(), + false => String::new(), + }) +} + +/// The server this run uses, if one is named: by a flag, the environment, +/// or the settings. +pub fn named(global: &Global) -> Option { + named_in(global, &settings()) +} + +fn named_in(global: &Global, kept: &Settings) -> Option { + global + .server + .clone() + .filter(|server| !server.trim().is_empty()) + .or_else(|| variable("BERTH_SERVER")) + .or_else(|| kept.get(settings::SERVER)) + .map(|server| berth_page::device::address(&server)) +} + +impl Client { + /// Reach the server this run uses. + pub fn open(global: &Global) -> Result { + let kept = settings(); + take_in_remote(&kept, remote_folder().as_deref()); + Client::open_in(global, &kept) + } + + /// Reach the server this run uses, with these settings. + pub fn open_in(global: &Global, kept: &Settings) -> Result { + let (url, token, finish) = match named_in(global, kept) { + Some(url) => { + let token = token(global, &url, kept)?; + (url, token, None) + } + None => match crate::local::server(false)? { + Some((url, finish)) => (url, String::new(), Some(finish)), + None => { + return Err(Fail::unreachable( + "No berth server is set. Name one with berth login URL, or with --server URL for one command", + )); + } + }, + }; + let in_process = finish.is_some(); + let client = Client { + portal: connect(&url, &token)?, + url, + finish, + sudo_ready: Default::default(), + }; + // One in this process is this berth; any other may be older. + if !in_process { + let (version, api) = client.server_version()?; + if let Some(older) = berth_api::compat::older_server(&client.url, &version, api) { + return Err(Fail::refused(older)); + } + } + Ok(client) + } + + /// Reach the server this run uses, whatever its version, only to ask it. + pub fn open_any(global: &Global, kept: &Settings) -> Result { + let url = + named_in(global, kept).ok_or_else(|| Fail::unreachable("No berth server is set"))?; + let token = token(global, &url, kept)?; + Ok(Client { + portal: connect(&url, &token)?, + url, + finish: None, + sudo_ready: Default::default(), + }) + } + + /// The server's version and the number of its API. + pub fn server_version(&self) -> Result<(String, u32)> { + let said = self.call("version", json!({}), None)?; + let version = said["version"].as_str().unwrap_or("").to_owned(); + let api = said["api"].as_u64().unwrap_or(0) as u32; + Ok((version, api)) + } + + /// Whether the server runs in this process, and ends with it. + pub fn in_process(&self) -> bool { + self.url == berth_page::device::HERE + } + + pub fn wait(&self, future: F) -> F::Output { + net::runtime().block_on(future) + } + + /// One operation of the server, waited for. + pub fn call(&self, op: &str, args: Value, data: Option) -> Result { + self.wait(self.portal.call(op, None, args, data)) + .map_err(Fail::from) + } + + /// Before the server in this process reaches `host`, give it the host's + /// sudo password when its sudo asks for one and no secret `sudo/HOST` + /// keeps it: from `--sudo-password-stdin`, or asked once on the terminal. + /// A server elsewhere has its own. + fn sudo_for(&self, host: &str) -> Result<()> { + if !self.in_process() || !self.sudo_ready.borrow_mut().insert(host.to_owned()) { + return Ok(()); + } + if let Some(password) = super::host::sudo_password(self, host)? { + crate::local::give_sudo_password(host, &password); + } + Ok(()) + } + + /// One operation of a host's controller, through the server. + pub fn on(&self, host: &str, op: &str, args: Value, data: Option) -> Result { + self.sudo_for(host)?; + self.wait(self.portal.call(op, Some(host), args, data)) + .map_err(Fail::from) + } + + /// Open a stream of the server, or of a host's controller. + pub fn stream(&self, op: &str, host: Option<&str>, args: Value) -> Result { + if let Some(host) = host { + self.sudo_for(host)?; + } + self.wait(self.portal.open(op, host, args)) + .map_err(Fail::from) + } + + pub fn portal(&self) -> &Portal { + &self.portal + } + + /// Run an action of a subject: a host, a provisioner, a resource, or + /// the installation. + pub fn act( + &self, + subject: &Subject, + action: &str, + act: &berth_api::Act, + ) -> Result { + if let Subject::Host(name) = subject { + self.sudo_for(name)?; + } + let (op, args) = match subject { + Subject::Host(name) => ("hosts.act", json!({"name": name, "action": action})), + Subject::Provisioner(name) => { + ("provisioners.act", json!({"name": name, "action": action})) + } + Subject::Resource(id) => ("resources.act", json!({"id": id, "action": action})), + Subject::Installation => ("installation.act", json!({"action": action})), + }; + let acted = self.call(op, args, Some(serde_json::to_value(act)?))?; + Ok(serde_json::from_value(acted)?) + } + + /// The actions of a subject. + pub fn actions(&self, subject: &Subject) -> Result> { + if let Subject::Host(name) = subject { + self.sudo_for(name)?; + } + let (op, args) = match subject { + Subject::Host(name) => ("hosts.actions", json!({"name": name})), + Subject::Provisioner(name) => ("provisioners.actions", json!({"name": name})), + Subject::Resource(id) => ("resources.actions", json!({"id": id})), + Subject::Installation => ("installation.actions", json!({})), + }; + Ok(serde_json::from_value(self.call(op, args, None)?)?) + } +} + +impl Drop for Client { + fn drop(&mut self) { + if let Some(finish) = self.finish.take() { + finish(); + } + } +} + +/// What an action is done to. +#[derive(Debug, Clone, PartialEq)] +pub enum Subject { + Host(String), + Provisioner(String), + Resource(String), + Installation, +} + +impl Subject { + /// How a job of it is named in its record, as `host:studio`. + pub fn shown(&self) -> String { + match self { + Subject::Host(name) => name.clone(), + Subject::Provisioner(name) => format!("provisioner {name}"), + Subject::Resource(id) => format!("resource {id}"), + Subject::Installation => "the installation".into(), + } + } +} + +/// Open the portal of the server at `url` with `token`. +pub fn connect(url: &str, token: &str) -> Result { + let target = net::target(url, berth_api::ops::PORTAL).map_err(Fail::usage)?; + net::runtime().block_on(async { + let stream = net::connect(&target).await.map_err(|error| { + Fail::unreachable(format!("The berth server at {url} cannot be reached: {error}")) + })?; + let socket = berth_api::client::websocket_at(stream, &target.authority, &target.path, token) + .await + .map_err(|error| match error.status() { + 401 | 403 => Fail::refused(format!( + "The berth server at {url} did not take the token. Log in again with: berth login {url}" + )), + _ => Fail::unreachable(format!( + "The berth server at {url} cannot be reached: {}", + net::describe(&error) + )), + })?; + Ok(Portal::over(socket)) + }) +} diff --git a/crates/berth/src/cli/data.rs b/crates/berth/src/cli/data.rs new file mode 100644 index 0000000..3ccac85 --- /dev/null +++ b/crates/berth/src/cli/data.rs @@ -0,0 +1,362 @@ +//! What flows between hosts and the server: a port of a host reached from +//! here, the inbox, the key-value store, a host's channel and its services. + +use super::args::{Channel, Inbox, Kv, Service}; +use super::client::Client; +use super::fail::{Fail, Result}; +use super::out::{self, Table}; +use berth_api::portal::Piece; +use serde_json::{Value, json}; +use std::io::Write; + +/// A port number of a word, which is more than 0. +fn port_of(text: &str, word: &str) -> Result { + text.parse::() + .ok() + .filter(|port| *port > 0) + .ok_or_else(|| Fail::usage(format!("{text} is no port, in {word}"))) +} + +/// One `[LOCAL:]PORT`: the port to listen on here, and the host's. +fn pair(word: &str) -> Result<(u16, String)> { + match word.split_once(':') { + Some((local, remote)) => Ok((port_of(local, word)?, port_of(remote, word)?.to_string())), + None => { + let remote = port_of(word, word)?; + Ok((remote, remote.to_string())) + } + } +} + +/// One `[LOCAL:]ENDPOINT` of a resource: the port to listen on here, and +/// the endpoint by its name or port. Without LOCAL it is the endpoint's port. +fn endpoint_pair(word: &str, endpoints: &[berth_api::Endpoint]) -> Result<(u16, String)> { + let (local, wanted) = match word.split_once(':') { + Some((local, wanted)) => (Some(port_of(local, word)?), wanted), + None => (None, word), + }; + let found = endpoints + .iter() + .find(|endpoint| endpoint.name == wanted || endpoint.port.to_string() == wanted) + .ok_or_else(|| { + let names: Vec = endpoints + .iter() + .map(|endpoint| format!("{} ({})", endpoint.name, endpoint.port)) + .collect(); + Fail::usage(match names.is_empty() { + true => format!("The resource reports no endpoints, so there is no {wanted}"), + false => format!( + "The resource has no endpoint {wanted}. It has {}", + names.join(", ") + ), + }) + })?; + Ok((local.unwrap_or(found.port), found.name.clone())) +} + +/// What a forwarded connection goes to. +#[derive(Clone)] +enum Target { + Host(String), + Resource(String), +} + +/// `berth port forward TARGET [LOCAL:]PORT…`: listen here, and pass each +/// connection through a tunnel of its own to the host's port or the +/// resource's endpoint, until Ctrl+C. +pub fn forward( + client: &Client, + target: &str, + ports: &[String], + resource: bool, + address: &str, +) -> Result<()> { + // A host's name comes first; what no host is called may be a resource. + let host = match resource { + true => false, + false => match client.call("hosts.get", json!({"name": target}), None) { + Ok(_) => true, + Err(fail) if fail.code == "not_found" => false, + Err(fail) => return Err(fail), + }, + }; + let (target, shown, pairs) = match host { + true => { + let pairs: Vec<(u16, String)> = + ports.iter().map(|word| pair(word)).collect::>()?; + (Target::Host(target.to_owned()), target.to_owned(), pairs) + } + false => { + let id = super::things::resource_id(client, target).map_err(|fail| match resource { + true => fail, + false => Fail::failed(format!( + "There is no host or resource {target}. berth ls and berth resource list name them" + )), + })?; + let found = client.call("resources.get", json!({"id": id}), None)?; + let made: berth_api::Resource = serde_json::from_value(found)?; + let pairs: Vec<(u16, String)> = ports + .iter() + .map(|word| endpoint_pair(word, &made.endpoints)) + .collect::>()?; + let shown = match made.name.is_empty() { + true => id.clone(), + false => made.name.clone(), + }; + (Target::Resource(id), shown, pairs) + } + }; + let portal = client.portal().clone(); + client.wait(async { + let mut listening = vec![]; + for (local, remote) in &pairs { + let listener = tokio::net::TcpListener::bind((address, *local)) + .await + .map_err(|error| Fail::failed(format!("{address}:{local}: {error}")))?; + let bound = listener.local_addr()?; + out::line(format!("{bound} → {shown}:{remote}")); + if out::get().json { + let to = match &target { + Target::Host(host) => json!({"host": host, "port": remote}), + Target::Resource(id) => json!({"resource": id, "endpoint": remote}), + }; + let mut said = json!({"listen": bound.to_string()}); + for (key, value) in to.as_object().into_iter().flatten() { + said[key] = value.clone(); + } + out::json(&said); + } + listening.push((listener, remote.clone())); + } + out::hint("Ctrl+C ends it"); + let mut tasks = vec![]; + for (listener, remote) in listening { + let (portal, target, shown) = (portal.clone(), target.clone(), shown.clone()); + tasks.push(tokio::spawn(async move { + loop { + let Ok((mut socket, from)) = listener.accept().await else { + return; + }; + let (portal, target, shown, remote) = ( + portal.clone(), + target.clone(), + shown.clone(), + remote.clone(), + ); + tokio::spawn(async move { + let opened = match &target { + Target::Host(host) => { + let port = remote.parse().unwrap_or(0); + portal.connect_port(Some(host), port).await + } + Target::Resource(id) => portal.connect_endpoint(id, &remote).await, + }; + match opened { + Ok(mut tunnel) => { + let _ = + tokio::io::copy_bidirectional(&mut socket, &mut tunnel).await; + } + Err(error) => out::warn(format!( + "{from}: {shown}:{remote} did not open: {}", + berth_page::net::describe(&error) + )), + } + }); + } + })); + } + let _ = tokio::signal::ctrl_c().await; + for task in tasks { + task.abort(); + } + Ok(()) + }) +} + +pub fn inbox(client: &Client, command: Inbox) -> Result<()> { + let Inbox::List { host, topic, all } = command; + let mut args = json!({"limit": 200}); + if let Some(host) = host { + args["host"] = json!(host); + } + if let Some(topic) = topic { + args["topic"] = json!(topic); + } + if !all { + args["done"] = json!(false); + } + let found = client.call("inbox.list", args, None)?; + out::data(&found, || { + let mut table = Table::new(&["", "time", "host", "id", "topic", "body"]); + let messages = found["messages"].as_array().cloned().unwrap_or_default(); + for message in messages.iter().rev() { + let text = |key: &str| message[key].as_str().unwrap_or("").to_owned(); + table.row(vec![ + if message["done"] == true { + "✓".into() + } else { + String::new() + }, + text("time"), + text("host"), + message["id"].to_string(), + text("topic"), + message["body"].to_string(), + ]); + } + table.print("No messages"); + if !found["next"].is_null() { + out::hint("Older messages are not shown"); + } + }); + Ok(()) +} + +pub fn kv(client: &Client, command: Kv) -> Result<()> { + match command { + Kv::List { namespace } => { + let found = client.call("kv.list", json!({"namespace": namespace}), None)?; + out::data(&found, || { + let mut table = Table::new(&["key", "revision", "value"]); + let entries = match &found { + Value::Array(entries) => entries.clone(), + found => found["entries"].as_array().cloned().unwrap_or_default(), + }; + for entry in entries { + table.row(vec![ + entry["key"].as_str().unwrap_or("").to_owned(), + entry["revision"].to_string(), + entry["value"].to_string(), + ]); + } + table.print(&format!("Nothing in {namespace}")); + }); + Ok(()) + } + Kv::Get { namespace, key } => { + let found = client.call("kv.get", json!({"namespace": namespace, "key": key}), None)?; + out::data(&found, || println!("{}", found["value"])); + Ok(()) + } + Kv::Set { + namespace, + key, + value, + } => { + // Anything that is not JSON is taken as text. + let value: Value = serde_json::from_str(&value).unwrap_or_else(|_| json!(value)); + let found = client.call( + "kv.set", + json!({"namespace": namespace, "key": key}), + Some(json!({"value": value})), + )?; + out::data(&found, || { + out::done(format!("{key} is at revision {}", found["revision"])); + }); + Ok(()) + } + Kv::Remove { namespace, key } => { + client.call( + "kv.delete", + json!({"namespace": namespace, "key": key}), + None, + )?; + out::done(format!("Removed {key} from {namespace}")); + Ok(()) + } + } +} + +pub fn channel(client: &Client, command: Channel) -> Result<()> { + let Channel::Listen { host, topic, since } = command; + let mut args = json!({}); + if let Some(since) = since { + args["since"] = json!(since); + } + let mut stream = client.stream("channel.listen", Some(&host), args)?; + client.wait(async { + let mut stdout = std::io::stdout(); + loop { + tokio::select! { + piece = stream.recv() => match piece { + Some(Ok(Piece::Text(message))) => { + let fits = topic.as_ref().is_none_or(|prefix| { + message["topic"].as_str().is_some_and(|topic| topic.starts_with(prefix.as_str())) + }); + if fits { + writeln!(stdout, "{message}")?; + stdout.flush()?; + } + } + Some(Ok(Piece::Binary(_))) => {} + Some(Err(error)) => return Err(Fail::from(error)), + None => return Ok(()), + }, + _ = tokio::signal::ctrl_c() => return Ok(()), + } + } + }) +} + +pub fn service(client: &Client, command: Service) -> Result<()> { + let Service::Call { + host, + service, + method, + body, + } = command; + let (plugin, name) = service + .split_once('/') + .ok_or_else(|| Fail::usage(format!("A service is PLUGIN/NAME, not {service}")))?; + let body: Value = match body { + Some(body) => serde_json::from_str(&body) + .map_err(|error| Fail::usage(format!("The body is no JSON: {error}")))?, + None => json!({}), + }; + let answer = client.on( + &host, + "services.call", + json!({"plugin": plugin, "name": name, "method": method}), + Some(body), + )?; + out::json(&answer); + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::{endpoint_pair, pair}; + + #[test] + fn ports_are_the_hosts_and_maybe_a_local_one() { + assert_eq!(pair("8080").unwrap(), (8080, "8080".into())); + assert_eq!(pair("9000:80").unwrap(), (9000, "80".into())); + assert!(pair("0").is_err() && pair("a:80").is_err() && pair("70000").is_err()); + } + + #[test] + fn an_endpoint_is_named_or_given_by_its_port() { + let endpoints = vec![berth_api::Endpoint { + name: "db".into(), + address: "192.0.2.30".into(), + port: 5432, + protocol: "tcp".into(), + hint: "postgres".into(), + local: false, + }]; + assert_eq!( + endpoint_pair("db", &endpoints).unwrap(), + (5432, "db".into()) + ); + assert_eq!( + endpoint_pair("5432", &endpoints).unwrap(), + (5432, "db".into()) + ); + assert_eq!( + endpoint_pair("15432:db", &endpoints).unwrap(), + (15432, "db".into()) + ); + let missing = endpoint_pair("web", &endpoints).unwrap_err().message; + assert!(missing.contains("It has db (5432)"), "{missing}"); + } +} diff --git a/crates/berth/src/cli/docs.rs b/crates/berth/src/cli/docs.rs new file mode 100644 index 0000000..9628134 --- /dev/null +++ b/crates/berth/src/cli/docs.rs @@ -0,0 +1,301 @@ +//! The reference page of the commands, written from the same definition as +//! the help. A test writes it with `BERTH_UPDATE_DOCS=1`, and fails when +//! the committed page is stale. + +use super::args::EXIT_STATUSES; +use clap::{Arg, ArgAction, Command}; + +/// Where the page is, from the repository's root. +pub const PAGE: &str = "docs/content/docs/(guide)/reference/cli.mdx"; + +/// Text made safe for MDX: no tags or expressions, and no pipes that end a +/// table's cell. +fn safe(text: &str) -> String { + text.replace('|', "\\|") + .replace('<', "<") + .replace('>', ">") + .replace('{', "\\{") + .replace('}', "\\}") +} + +fn about(command: &Command) -> String { + command + .get_long_about() + .or(command.get_about()) + .map(|text| text.to_string()) + .unwrap_or_default() +} + +/// How an argument is written: `--flag VALUE`, `-o, --output FORMAT`, `NAME`. +fn shown(arg: &Arg) -> String { + let value = arg + .get_value_names() + .map(|names| { + names + .iter() + .map(|name| name.to_string()) + .collect::>() + .join(" ") + }) + .unwrap_or_else(|| arg.get_id().to_string().to_uppercase()); + let takes = matches!(arg.get_action(), ArgAction::Set | ArgAction::Append); + if arg.is_positional() { + return match arg + .get_num_args() + .is_some_and(|range| range.max_values() > 1) + || matches!(arg.get_action(), ArgAction::Append) + { + true => format!("{value}…"), + false => value, + }; + } + let mut names = vec![]; + if let Some(short) = arg.get_short() { + names.push(format!("-{short}")); + } + if let Some(long) = arg.get_long() { + names.push(format!("--{long}")); + } + let mut text = names.join(", "); + if takes { + text.push(' '); + text.push_str(&value); + } + text +} + +/// What a name on the command line is, for one without help of its own. +fn named(id: &str) -> &'static str { + match id { + "name" => "Its name.", + "host" => "A host's name.", + "id" => "Its id.", + "new" => "Its new name.", + "action" => "The action's name.", + "key" => "The setting or key.", + "value" => "The value.", + "namespace" => "The namespace.", + "spec" => "A plugin's npm name, or the address of its .tgz.", + "url" => "The address.", + "service" => "PLUGIN/NAME of the service.", + "method" => "The service's method.", + "prompt" => "The prompt.", + _ => "", + } +} + +/// A sentence ends with a full stop, which clap leaves out of a short help. +fn sentence(text: &str) -> String { + let text = text.trim(); + match text.ends_with(['.', '?', '!', ':']) || text.is_empty() { + true => text.to_owned(), + false => format!("{text}."), + } +} + +fn help(arg: &Arg) -> String { + let mut text = arg + .get_long_help() + .or(arg.get_help()) + .map(|text| sentence(&text.to_string())) + .unwrap_or_else(|| named(arg.get_id().as_str()).to_owned()); + let values: Vec = arg + .get_possible_values() + .iter() + .filter(|value| !value.is_hide_set()) + .map(|value| format!("`{}`", value.get_name())) + .collect(); + if !values.is_empty() && !arg.is_positional() && matches!(arg.get_action(), ArgAction::Set) { + text = format!("{} One of {}.", text.trim_end(), values.join(", ")); + } + if !values.is_empty() && arg.is_positional() { + text = format!("{} One of {}.", text.trim_end(), values.join(", ")); + } + let defaults: Vec = arg + .get_default_values() + .iter() + .map(|value| value.to_string_lossy().into_owned()) + .collect(); + if !defaults.is_empty() + && !defaults.iter().all(|value| value.is_empty()) + && !matches!(arg.get_action(), ArgAction::SetTrue) + { + text = format!("{} By default `{}`.", text.trim_end(), defaults.join(" ")); + } + text.trim().to_owned() +} + +/// The usage line of a command, without the global flags. +fn usage(path: &str, command: &Command) -> String { + let mut words = vec![path.to_owned()]; + if command.has_subcommands() { + words.push("ACTION".into()); + } + for arg in command.get_arguments() { + if arg.is_global_set() || arg.is_hide_set() { + continue; + } + let text = shown(arg); + let text = match (arg.is_positional(), arg.is_last_set()) { + (true, true) => format!("-- {text}"), + _ => text, + }; + words.push(match arg.is_required_set() { + true => text, + false => format!("[{text}]"), + }); + } + words.join(" ") +} + +fn table(rows: &[(String, String)], heading: (&str, &str)) -> String { + let mut text = format!("| {} | {} |\n| --- | --- |\n", heading.0, heading.1); + for (what, does) in rows { + text.push_str(&format!("| `{}` | {} |\n", what, safe(does))); + } + text +} + +fn section(out: &mut String, path: &str, command: &Command, depth: usize) { + if command.is_hide_set() { + return; + } + let hashes = "#".repeat(depth.min(4)); + out.push_str(&format!("{hashes} {path}\n\n")); + let said = about(command); + if !said.is_empty() { + out.push_str(&format!("{}\n\n", safe(&sentence(&said)))); + } + let before = path.rsplit_once(' ').map_or("", |(before, _)| before); + for alias in command.get_visible_aliases() { + out.push_str(&format!("`{before} {alias}` is the same.\n\n")); + } + if !command.has_subcommands() { + out.push_str(&format!("```text\n{}\n```\n\n", usage(path, command))); + let rows: Vec<(String, String)> = command + .get_arguments() + .filter(|arg| !arg.is_global_set() && !arg.is_hide_set()) + .filter(|arg| !["help", "version"].contains(&arg.get_id().as_str())) + .map(|arg| (shown(arg), help(arg))) + .collect(); + if !rows.is_empty() { + out.push_str(&table(&rows, ("Argument", "What it is"))); + out.push('\n'); + } + if path == "berth host create" { + out.push_str("The ways to install, for `--method`:\n\n"); + let methods: Vec<(String, String)> = berth_tui::fields::Action::SERVED + .iter() + .map(|action| (action.key().to_owned(), action.label().to_owned())) + .collect(); + out.push_str(&table(&methods, ("Method", "What it is"))); + out.push_str("\nThe fields of a profile, for `--set KEY=VALUE` here and `berth host set`. A switch takes `on` or `off`. `feature.PLUGIN/FEATURE=on`, `secret.NAME=VARIABLE` and `registry.NAME=on` give a host those as well.\n\n"); + let keys: Vec<(String, String)> = berth_tui::flow::keys() + .into_iter() + .map(|(key, label, _)| (key, label.to_owned())) + .collect(); + out.push_str(&table(&keys, ("Key", "Field"))); + out.push('\n'); + } + return; + } + for sub in command.get_subcommands() { + if sub.get_name() == "help" { + continue; + } + section(out, &format!("{path} {}", sub.get_name()), sub, depth + 1); + } +} + +/// The whole page. +pub fn page() -> String { + let command = super::command(); + let mut out = String::from( + "---\ntitle: Command reference\ndescription: Every command of berth with its arguments, made from the same definition as berth --help.\nicon: SquareTerminal\n---\n\n", + ); + out.push_str("This page is generated from the commands themselves. [The command line](/docs/use/cli) explains them by what they do.\n\n"); + out.push_str("## Flags of every command\n\n"); + let global: Vec<(String, String)> = command + .get_arguments() + .filter(|arg| arg.is_global_set()) + .map(|arg| (shown(arg), help(arg))) + .collect(); + out.push_str(&table(&global, ("Flag", "What it does"))); + out.push_str("\nFlags go before or after the names, as `--tail 50` or `--tail=50`. `--` ends the flags; what follows it is passed on as it is.\n\n"); + out.push_str("## Exit statuses\n\n| Status | Means |\n| --- | --- |\n"); + for (status, means) in EXIT_STATUSES { + out.push_str(&format!("| {status} | {} |\n", safe(means))); + } + out.push_str( + "\nA command run on a host, such as `berth exec`, ends with that command's own status.\n\n", + ); + out.push_str("## Environment\n\n| Variable | What it does |\n| --- | --- |\n"); + for (name, does) in ENVIRONMENT { + out.push_str(&format!("| `{name}` | {} |\n", safe(does))); + } + out.push('\n'); + for sub in command.get_subcommands() { + if sub.get_name() == "help" { + continue; + } + section(&mut out, &format!("berth {}", sub.get_name()), sub, 2); + } + out.trim_end().to_owned() + "\n" +} + +/// The variables berth reads, besides its flags and settings. +pub const ENVIRONMENT: &[(&str, &str)] = &[ + ( + "BERTH_SERVER", + "The server, when no --server is given. It wins over the one berth login kept.", + ), + ( + "BERTH_TOKEN", + "Its token, when no --token-file or --token-stdin is given. For containers and CI.", + ), + ("NO_COLOR", "No colours, as --color never."), + ( + "BERTH_ and a setting's name", + "A setting of the server, in capitals, such as BERTH_RETENTION_DAYS, for a server in a container.", + ), + ( + "XDG_CONFIG_HOME, XDG_DATA_HOME, XDG_RUNTIME_DIR", + "Where berth keeps its settings, its data and its socket.", + ), + ("VISUAL, EDITOR", "The editor of berth host edit."), +]; + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn the_reference_page_is_the_commands_as_they_are() { + let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../.."); + let file = root.join(PAGE); + let page = page(); + if std::env::var_os("BERTH_UPDATE_DOCS").is_some() { + std::fs::write(&file, &page).unwrap(); + return; + } + let committed = std::fs::read_to_string(&file).unwrap_or_default(); + assert!( + committed == page, + "{PAGE} is stale: write it again with BERTH_UPDATE_DOCS=1 cargo test -p berth" + ); + } + + #[test] + fn every_command_is_on_the_page() { + let page = page(); + for line in [ + "## berth host", + "### berth host create", + "`--set KEY=VALUE`", + "| 4 |", + ] { + assert!(page.contains(line), "{line}"); + } + assert!(!page.contains("berth askpass")); + } +} diff --git a/crates/berth/src/cli/fail.rs b/crates/berth/src/cli/fail.rs new file mode 100644 index 0000000..61f7c1c --- /dev/null +++ b/crates/berth/src/cli/fail.rs @@ -0,0 +1,146 @@ +//! Why a command did not do what it was asked, as one sentence and the exit +//! status that says what kind of failure it was. + +use berth_api::client::Error; + +/// The work failed: a job failed, a host did not answer. +pub const FAILED: i32 = 1; +/// The command line was wrong: an unknown word, a missing value, a bad flag. +pub const USAGE: i32 = 2; +/// Refused: the role is too low, it is not available with this server, or +/// a confirmation was declined. +pub const REFUSED: i32 = 3; +/// The server could not be reached. +pub const UNREACHABLE: i32 = 4; + +#[derive(Debug, Clone, PartialEq)] +pub struct Fail { + pub status: i32, + /// The API's error code, or berth's own word for the kind of failure. + pub code: String, + /// One sentence with what to do next; empty when there is nothing to + /// say, as for a command on a host that exits with its own status. + pub message: String, +} + +pub type Result = std::result::Result; + +impl Fail { + fn with(status: i32, code: &str, message: impl Into) -> Fail { + Fail { + status, + code: code.to_owned(), + message: message.into(), + } + } + + pub fn failed(message: impl Into) -> Fail { + Fail::with(FAILED, "failed", message) + } + + pub fn usage(message: impl Into) -> Fail { + Fail::with(USAGE, "usage", message) + } + + pub fn refused(message: impl Into) -> Fail { + Fail::with(REFUSED, "refused", message) + } + + pub fn unreachable(message: impl Into) -> Fail { + Fail::with(UNREACHABLE, "unreachable", message) + } + + /// An exit status passed through with nothing to say, such as that of a + /// command run on a host. + pub fn exit(status: i32) -> Fail { + Fail::with(status, "exit", "") + } + + /// The same failure, said with more in front. + pub fn context(mut self, before: &str) -> Fail { + if !self.message.is_empty() { + self.message = format!("{before}: {}", self.message); + } + self + } +} + +/// The exit status an error of the API stands for. +pub fn status_of(status: u16) -> i32 { + match status { + 400 | 413 => USAGE, + 401 | 403 | 412 | 501 => REFUSED, + 503 => UNREACHABLE, + _ => FAILED, + } +} + +impl From for Fail { + fn from(error: Error) -> Fail { + match error { + Error::Api { + status, + code, + message, + } => Fail { + status: status_of(status), + code, + message, + }, + Error::Transport(message) => Fail::unreachable(message), + } + } +} + +impl From for Fail { + fn from(error: std::io::Error) -> Fail { + Fail::failed(error.to_string()) + } +} + +impl From for Fail { + fn from(error: serde_json::Error) -> Fail { + Fail::failed(error.to_string()) + } +} + +impl From for Fail { + fn from(message: String) -> Fail { + Fail::failed(message) + } +} + +impl From<&str> for Fail { + fn from(message: &str) -> Fail { + Fail::failed(message) + } +} + +impl From> for Fail { + fn from(error: Box) -> Fail { + Fail::failed(error.to_string()) + } +} + +impl From for Fail { + fn from(error: clap::Error) -> Fail { + Fail::usage(error.to_string().trim().trim_start_matches("error: ")) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn errors_of_the_api_have_the_status_of_their_kind() { + let api = |status, code: &str| Fail::from(Error::api((status, code), "said")); + assert_eq!(api(400, "bad_request").status, USAGE); + assert_eq!(api(403, "forbidden").status, REFUSED); + assert_eq!(api(501, "not_available").status, REFUSED); + assert_eq!(api(502, "unreachable").status, FAILED); + assert_eq!(api(404, "not_found").code, "not_found"); + let lost = Fail::from(Error::Transport("refused".into())); + assert_eq!(lost.status, UNREACHABLE); + } +} diff --git a/crates/berth/src/cli/follow.rs b/crates/berth/src/cli/follow.rs new file mode 100644 index 0000000..d167fa7 --- /dev/null +++ b/crates/berth/src/cli/follow.rs @@ -0,0 +1,381 @@ +//! Following a job of the server to its end. On a terminal it is a line +//! with a spinner and the last lines of the job's output beneath it, which +//! give way to one line with the result. Elsewhere the output goes to +//! standard error as it comes, and the result to standard output. + +use super::client::Client; +use super::fail::{Fail, Result}; +use super::out::{self, Style}; +use berth_api::portal::Piece; +use berth_tui::theme::Role; +use ratatui::{ + Terminal, TerminalOptions, Viewport, + backend::CrosstermBackend, + style::{Modifier, Style as Look}, + text::{Line, Span}, + widgets::Paragraph, +}; +use serde_json::{Value, json}; +use std::io::Write; +use std::time::{Duration, Instant}; + +/// How many lines of output are shown under the spinner. +const TAIL: usize = 5; +/// How many lines of a failed job are printed when it ends. +const SHOWN_ON_FAILURE: usize = 25; +const SPINNER: [&str; 10] = ["⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "⠧", "⠇", "⠏"]; + +/// How a job ended. +pub struct Ended { + pub code: i32, + /// What the server says of it now. + pub job: Value, + /// The last lines of its output, without their colours. + pub tail: Vec, +} + +/// The lines of output seen, the last of which may not be finished. +#[derive(Default)] +struct Lines { + done: Vec, + open: String, +} + +impl Lines { + fn take(&mut self, bytes: &[u8]) { + let text = String::from_utf8_lossy(bytes); + for ch in text.chars() { + match ch { + '\n' => self.done.push(std::mem::take(&mut self.open)), + // A line written over: what follows replaces it. + '\r' => self.open.clear(), + ch => self.open.push(ch), + } + } + // Enough to show the end of a failed job. + let over = self.done.len().saturating_sub(SHOWN_ON_FAILURE * 4); + self.done.drain(..over); + } + + fn last(&self, count: usize) -> Vec { + let mut all: Vec = self.done.iter().map(|line| out::strip_ansi(line)).collect(); + let open = out::strip_ansi(&self.open); + if !open.trim().is_empty() { + all.push(open); + } + all.retain(|line| !line.trim().is_empty()); + let from = all.len().saturating_sub(count); + all[from..].to_vec() + } +} + +fn elapsed(since: Instant) -> String { + let seconds = since.elapsed().as_secs(); + match seconds { + 0..60 => format!("{seconds}s"), + _ => format!("{}m {:02}s", seconds / 60, seconds % 60), + } +} + +/// The progress drawn on the terminal. +struct Inline { + terminal: Terminal>, +} + +impl Inline { + fn start() -> Option { + let terminal = Terminal::with_options( + CrosstermBackend::new(std::io::stderr()), + TerminalOptions { + viewport: Viewport::Inline(TAIL as u16 + 1), + }, + ) + .ok()?; + Some(Inline { terminal }) + } + + fn draw(&mut self, title: &str, since: Instant, lines: &Lines) { + let frame = (since.elapsed().as_millis() / 100) as usize % SPINNER.len(); + let tail = lines.last(TAIL); + let _ = self.terminal.draw(|f| { + let width = f.area().width as usize; + let mut shown = vec![Line::from(vec![ + Span::styled(SPINNER[frame], out::look(Role::Accent)), + Span::raw(" "), + Span::styled(title.to_owned(), Look::new().add_modifier(Modifier::BOLD)), + Span::styled(format!(" {}", elapsed(since)), out::look(Role::Muted)), + ])]; + for line in &tail { + let cut: String = line.chars().take(width.saturating_sub(4)).collect(); + shown.push(Line::styled(format!(" {cut}"), out::look(Role::Muted))); + } + f.render_widget(Paragraph::new(shown), f.area()); + }); + } + + fn end(mut self) { + let _ = self.terminal.clear(); + out::to_line_of(&mut self.terminal); + } +} + +/// What a frame of `jobs.follow` says when it is the end. +fn exit_of(said: &Value) -> Option { + said.get("exited") + .map(|exited| exited["exit_code"].as_i64().unwrap_or(1) as i32) +} + +/// What following a job came to before it ended. +enum Watched { + Ended, + /// Ctrl+C: stop watching. + Interrupted, +} + +/// Ask whether Ctrl+C stops a job the server keeps, or only the watching. +fn stop_or_keep(id: &str) -> bool { + let question = berth_tui::flow::Question { + step: berth_tui::fields::Step::Run, + label: "Stop the job, or keep it running?".into(), + help: format!("It goes on where the server runs; berth job stop {id} stops it later."), + flag: String::new(), + asked: berth_tui::flow::Asked::Choice { + options: vec![ + berth_tui::flow::Choice { + key: "keep".into(), + label: "keep it running".into(), + help: format!("Stop watching. berth job logs {id} -f follows it again."), + }, + berth_tui::flow::Choice { + key: "stop".into(), + label: "stop the job".into(), + help: "Its programs end, and its record says who stopped it.".into(), + }, + ], + default: 0, + }, + required: false, + }; + matches!( + super::ask::ask(&question), + Ok(berth_tui::flow::Answer::Choice(1)) + ) +} + +/// Follow the job `id` to its end, shown as `title`. With `own` the job's +/// output is what the command is for, such as an action of a host, and goes +/// to standard output whole. Ctrl+C asks whether to stop a job that a +/// server keeps, at a terminal; elsewhere it stops watching. +pub fn job(client: &Client, id: &str, title: &str, own: bool) -> Result { + let shown = out::get(); + let colour = shown.colour.1; + let mut inline = match shown.terminal.1 && !shown.quiet && !own { + true => Inline::start(), + false => None, + }; + let since = Instant::now(); + let mut lines = Lines::default(); + let mut code = None; + let mut warned = false; + let mut offset = 0u64; + let mut asked_to_stop = false; + let in_process = client.in_process(); + let stopped = loop { + let mut stream = client.stream("jobs.follow", None, json!({"id": id, "since": offset}))?; + let watched = client.wait(async { + let mut tick = tokio::time::interval(Duration::from_millis(100)); + loop { + tokio::select! { + piece = stream.recv() => match piece { + Some(Ok(Piece::Binary(bytes))) => { + offset += bytes.len() as u64; + lines.take(&bytes); + if own { + let mut stdout = std::io::stdout().lock(); + let _ = stdout.write_all(&bytes); + let _ = stdout.flush(); + } else if inline.is_none() && !shown.quiet { + let text = String::from_utf8_lossy(&bytes); + let text = match colour { true => text.into_owned(), false => out::strip_ansi(&text) }; + let mut stderr = std::io::stderr().lock(); + let _ = stderr.write_all(text.as_bytes()); + let _ = stderr.flush(); + } + } + Some(Ok(Piece::Text(said))) => { + if let Some(exit) = exit_of(&said) { + code = Some(exit); + } + } + Some(Err(error)) => return Err(Fail::from(error)), + None => return Ok(Watched::Ended), + }, + _ = tick.tick() => { + if let Some(inline) = &mut inline { + inline.draw(title, since, &lines); + } + } + _ = tokio::signal::ctrl_c() => { + if in_process && !warned { + warned = true; + let say = format!("{title} runs in this berth and stops when it ends. Ctrl+C again stops it; berth serve keeps jobs running after berth ends"); + match &mut inline { + Some(inline) => { + let said = say.clone(); + let _ = inline.terminal.insert_before(1, |buffer| { + Paragraph::new(Line::styled(said, out::look(Role::Warn))).render(buffer.area, buffer); + }); + } + None => out::warn(say), + } + continue; + } + return Ok(Watched::Interrupted); + } + } + } + })?; + match watched { + Watched::Ended => break false, + Watched::Interrupted if in_process || asked_to_stop || !shown.ask => break true, + Watched::Interrupted => { + // The question needs the terminal's lines to itself. + if let Some(drawn) = inline.take() { + drawn.end(); + } + if !stop_or_keep(id) { + break true; + } + asked_to_stop = true; + client.call("jobs.stop", json!({"id": id}), None)?; + out::note(format!("Asked {id} to stop. Ctrl+C again stops watching")); + if !shown.quiet && !own { + inline = Inline::start(); + } + } + } + }; + let drew = inline.is_some(); + if let Some(inline) = inline { + inline.end(); + } + if stopped { + let message = match in_process { + true => format!("Stopped: {title} ended with berth"), + false => format!( + "Stopped watching. The job goes on where the server runs: berth job logs {id} -f" + ), + }; + return Err(Fail { + status: 130, + code: "stopped".into(), + message, + }); + } + let job = client + .call("jobs.get", json!({"id": id}), None) + .unwrap_or(Value::Null); + let code = code + .or_else(|| job["exit_code"].as_i64().map(|code| code as i32)) + .unwrap_or(1); + let took = elapsed(since); + if !shown.quiet { + match code { + 0 => eprintln!( + "{} {title} {}", + out::styled_err("✓", Style::Good), + out::styled_err(&format!("done in {took}"), Style::Dim) + ), + code => { + // Output drawn inline is gone from the screen; its end is shown again. + if drew { + let mut stderr = std::io::stderr().lock(); + for line in lines.done.iter().rev().take(SHOWN_ON_FAILURE).rev() { + let line = match colour { + true => line.clone(), + false => out::strip_ansi(line), + }; + let _ = writeln!(stderr, " {line}"); + } + } + let said = match job["stopped_by"].as_str() { + Some(by) => { + format!("stopped by {by} after {took}. Its log: berth job logs {id}") + } + None => format!( + "failed with exit status {code} after {took}. Its log: berth job logs {id}" + ), + }; + eprintln!( + "{} {title} {}", + out::styled_err("✗", Style::Bad), + out::styled_err(&said, Style::Dim) + ); + } + } + } + Ok(Ended { + code, + job, + tail: lines.last(SHOWN_ON_FAILURE), + }) +} + +use ratatui::widgets::Widget; + +/// What a command that started a job does with it: print its id and leave +/// with `--detach`, or follow it to its end. The exit status is the job's: +/// 1 when it failed, or its own when `own` says so, as for a command on a host. +pub fn acted( + client: &Client, + acted: berth_api::Acted, + title: &str, + detach: bool, + own: bool, +) -> Result { + let Some(job) = acted.job else { + if let Some(result) = &acted.result { + return Ok(result.clone()); + } + if let Some(session) = &acted.session { + return Ok(serde_json::to_value(session)?); + } + return Ok(Value::Null); + }; + if detach { + if client.in_process() { + return Err(Fail::refused( + "--detach needs a berth serve to keep the job running after berth ends. Start one with berth serve, or leave out --detach", + )); + } + out::data(&serde_json::to_value(&job)?, || println!("{}", job.id)); + return Ok(Value::Null); + } + let ended = self::job(client, &job.id, title, own)?; + if out::get().json { + out::json(&ended.job); + } + match (ended.code, own) { + (0, _) => Ok(ended.job), + (code, true) => Err(Fail::exit(code)), + (_, false) => Err(Fail::exit(1)), + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn the_tail_is_the_last_lines_without_their_colours() { + let mut lines = Lines::default(); + lines.take(b"TASK [packages]\n\x1b[32mok: [studio]\x1b[0m\n"); + lines.take(b"progress 10%\rprogress 60%"); + assert_eq!(lines.last(2), ["ok: [studio]", "progress 60%"]); + lines.take(b"\n\n"); + assert_eq!( + lines.last(5), + ["TASK [packages]", "ok: [studio]", "progress 60%"] + ); + } +} diff --git a/crates/berth/src/cli/host.rs b/crates/berth/src/cli/host.rs new file mode 100644 index 0000000..0132af9 --- /dev/null +++ b/crates/berth/src/cli/host.rs @@ -0,0 +1,715 @@ +//! `berth host …`: the hosts of the server, as the screens' home list and +//! a host's overview and set-up show them. + +use super::args::{self, Global, Host}; +use super::ask; +use super::client::{Client, Subject}; +use super::fail::{Fail, Result}; +use super::follow; +use super::out::{self, Style, Table, styled}; +use berth_api::{Act, HostType, profile}; +use serde_json::{Value, json}; +use std::collections::BTreeMap; + +pub fn command(global: &Global, command: Host) -> Result<()> { + let open = || Client::open(global); + match command { + Host::List => list(&open()?), + Host::Show { name } => show(&open()?, &name), + Host::Create(create) => super::setup::create(&open()?, &create), + Host::Add(add) => self::add(&open()?, &add), + Host::Set(set) => self::set(&open()?, &set.name, &set.values), + Host::Edit { name } => edit(&open()?, &name), + Host::Export { name } => { + let profile = profile_of(&open()?, &name)?; + out::json(&profile); + Ok(()) + } + Host::Import { name, file } => import(&open()?, &name, &file), + Host::Apply(apply) => self::apply(&open()?, &apply), + Host::Seed { name, file } => seed(&open()?, &name, file.as_deref()), + Host::Iso(iso) => installer(&open()?, &iso), + Host::Rename { + name, + new, + hostname, + } => rename(&open()?, &name, &new, hostname), + Host::Remove { name, destroy } => remove(&open()?, &name, destroy), + Host::Adopt(adopt) => self::adopt(&open()?, &adopt), + Host::Credentials { name, show } => credentials(&open()?, &name, show), + Host::Verify { name, accept } => verify(&open()?, &name, accept), + Host::Start(power) => self::power(&open()?, &power, "start"), + Host::Stop(power) => self::power(&open()?, &power, "stop"), + #[cfg(target_os = "linux")] + Host::Inventory { names } => crate::local::inventory(global, &names), + } +} + +/// The state of a host in a word or two, as the home list shows it. +fn state_of(host: &Value) -> String { + if host["reachable"] == true { + let api = host["api"].as_u64().unwrap_or(u64::MAX); + if api < u64::from(berth_api::compat::NEEDS_HOST_API) { + return styled("old controller", Style::Warn); + } + return match &host["behind"] { + Value::Null => styled("answers", Style::Good), + _ => styled("behind", Style::Warn), + }; + } + match host["address"].as_str() { + Some("") | None => styled("no address", Style::Dim), + _ => styled("unreachable", Style::Bad), + } +} + +pub fn list(client: &Client) -> Result<()> { + let hosts = client.call("hosts.list", json!({}), None)?; + out::data(&hosts, || { + let mut table = Table::new(&["name", "type", "address", "state", "berthd"]); + for host in hosts.as_array().into_iter().flatten() { + let text = |key: &str| host[key].as_str().unwrap_or("").to_owned(); + table.row(vec![ + text("name"), + text("type"), + text("address"), + state_of(host), + text("version"), + ]); + } + table.print( + "No hosts yet. Add one with berth host create NAME, or berth host add NAME ADDRESS", + ); + }); + Ok(()) +} + +pub fn show(client: &Client, name: &str) -> Result<()> { + let host = client.call("hosts.get", json!({"name": name}), None)?; + let machine = machine_of(client, name).ok().flatten(); + // The profile needs the role admin, and the actions a host that answers: each is left out without. + let given = profile_of(client, name).ok(); + let actions = client.actions(&Subject::Host(name.to_owned())).ok(); + out::data(&host, || { + let text = |key: &str| host[key].as_str().unwrap_or("").to_owned(); + let behind = match &host["behind"] { + Value::Null => String::new(), + behind => behind["text"] + .as_str() + .map(str::to_owned) + .unwrap_or_else(|| behind.to_string()), + }; + let mut pairs = vec![ + ("Name", text("name")), + ("Type", text("type")), + ("Address", text("address")), + ("State", state_of(&host)), + ("Problem", text("error")), + ("berthd", text("version")), + ("Applied", text("applied")), + ("Behind", behind), + ]; + if host["seen"].is_string() && host["seen"] != host["type"] { + pairs.push(("Seen as", text("seen"))); + } + if let Some(machine) = &machine { + let made = format!( + "{} of {}, {}", + machine["external_id"].as_str().unwrap_or(""), + machine["provisioner"].as_str().unwrap_or(""), + machine["state"].as_str().unwrap_or("") + ); + pairs.push(("Made by", made)); + } + if let Some(profile) = &given { + let secrets: Vec = profile["secrets"] + .as_array() + .into_iter() + .flatten() + .map(|secret| { + let named = secret["name"].as_str().unwrap_or(""); + match (secret["env"].as_str(), secret["file"].as_str()) { + (Some(env), _) => format!("{named} as {env}"), + (None, Some(file)) => match secret["mode"].as_str() { + Some(mode) => format!("{named} in {file} ({mode})"), + None => format!("{named} in {file}"), + }, + _ => named.to_owned(), + } + }) + .collect(); + pairs.push(("Secrets", secrets.join(", "))); + let registries: Vec<&str> = profile["registries"] + .as_array() + .into_iter() + .flatten() + .filter_map(Value::as_str) + .collect(); + pairs.push(("Registries", registries.join(", "))); + } + out::fields(&pairs); + if let Some(actions) = &actions { + println!(); + super::things::print_actions(actions); + } + out::hint(format!( + "Run an action with berth run {name} ACTION. Its profile: berth host export {name}" + )); + }); + Ok(()) +} + +/// The profile the server keeps of a host. +pub fn profile_of(client: &Client, name: &str) -> Result { + client.call("hosts.profile", json!({"name": name}), None) +} + +/// Save a profile as a host's. +pub fn save(client: &Client, name: &str, profile: &Value) -> Result<()> { + client.call("hosts.save", json!({"name": name}), Some(profile.clone()))?; + Ok(()) +} + +/// The machine a provisioner made for a host, as its resource. +pub fn machine_of(client: &Client, name: &str) -> Result> { + let found = client.call( + "resources.list", + json!({"kind": "machine", "host": name}), + None, + )?; + Ok(found + .as_array() + .and_then(|all| all.iter().find(|made| made["host"] == name).cloned())) +} + +fn exists(client: &Client, name: &str) -> Result { + match client.call("hosts.get", json!({"name": name}), None) { + Ok(_) => Ok(true), + Err(fail) if fail.code == "not_found" => Ok(false), + Err(fail) => Err(fail), + } +} + +fn add(client: &Client, add: &args::HostAdd) -> Result<()> { + let host_type = HostType::parse(&add.host_type).ok_or_else(|| { + Fail::usage(format!( + "--type is ssh, managed, node or workstation, not {}", + add.host_type + )) + })?; + let mut access = profile::Access { + address: add.address.clone(), + user: add.user.clone().unwrap_or_default(), + port: add.port.clone().unwrap_or_default(), + ..profile::Access::default() + }; + if let Some(auth) = &add.auth { + access.auth = profile::Auth::parse(auth).ok_or_else(|| { + Fail::usage(format!( + "--auth is berth, key:NAME, password or ssh, not {auth}" + )) + })?; + } + let mut config = profile::minimal(host_type, &access); + if host_type == HostType::Workstation { + config["builder"] = json!({}); + } + if !add.feature.is_empty() { + let features = super::setup::catalogue(client)?.features; + for key in &add.feature { + let setting = berth_tui::flow::Setting::Feature(key.clone(), true); + berth_tui::flow::apply(&mut config, &features, &setting) + .map_err(|why| Fail::usage(format!("--feature {key}: {why}")))?; + } + } + let problems = profile::problems(&config); + if !problems.is_empty() { + return Err(Fail::usage(problems.join("; "))); + } + if exists(client, &add.name)? { + return Err(Fail::failed(format!( + "There is a host {} already. Change it with berth host set {} KEY=VALUE", + add.name, add.name + ))); + } + save(client, &add.name, &config)?; + out::done(format!( + "Added {} as a host of the type {host_type}", + add.name + )); + if add.no_contact { + return Ok(()); + } + // First contact: the key is tried, and a host that refuses it is + // logged in to once with a password to install it. + let password = match add.password_stdin { + true => Some(ask::secret_from( + "Password", + None, + true, + "--password-stdin", + )?), + false => None, + }; + trust(client, &add.name, None, password).map_err(|fail| { + fail.context(&format!( + "{} is saved, but logging in to it did not work yet. Try again with berth key trust {}", + add.name, add.name + )) + })?; + let host = client.call("hosts.get", json!({"name": add.name}), None)?; + let seen = host["seen"].as_str().and_then(HostType::parse); + if let Some(said) = berth_api::type_mismatch(&add.name, host_type, seen) { + out::warn(said); + } + if host_type == HostType::Node { + out::hint(format!( + "Its features install with: berth host apply {}", + add.name + )); + } + Ok(()) +} + +/// Have a host take a key of the server's: berth's own is tried, and a +/// password or another key logs in once when the host refuses it. The +/// password is asked for when it is needed and not given. +pub fn trust( + client: &Client, + name: &str, + key: Option<&str>, + password: Option, +) -> Result<()> { + let mut arguments = BTreeMap::new(); + if let Some(key) = key { + arguments.insert("key".to_owned(), key.to_owned()); + } + if let Some(password) = &password { + arguments.insert("password".to_owned(), password.clone()); + } + let act = Act { + arguments: arguments.clone(), + ..Act::default() + }; + let title = format!("Logging in to {name}"); + let tried = client.act(&Subject::Host(name.to_owned()), "trust-key", &act)?; + match follow::acted(client, tried, &title, false, false) { + Ok(_) => Ok(()), + // Refused without a password: ask for one, once. + Err(_) if password.is_none() && key.is_none() && out::get().ask => { + let password = ask::secret( + &format!("Password of the SSH user of {name}, used once"), + "--password-stdin", + )?; + arguments.insert("password".into(), password); + let act = Act { + arguments, + ..Act::default() + }; + let tried = client.act(&Subject::Host(name.to_owned()), "trust-key", &act)?; + follow::acted(client, tried, &title, false, false).map(|_| ()) + } + Err(fail) => Err(fail), + } +} + +/// Write `KEY=VALUE` settings into a host's profile and save it. +pub fn set(client: &Client, name: &str, values: &[String]) -> Result<()> { + let mut profile = profile_of(client, name)?; + let features = super::setup::catalogue(client)?.features; + for word in values { + let setting = berth_tui::flow::setting(word).map_err(Fail::usage)?; + berth_tui::flow::apply(&mut profile, &features, &setting) + .map_err(|why| Fail::usage(format!("{word}: {why}")))?; + } + save(client, name, &profile)?; + out::done(format!("Saved {name}")); + if matches!( + profile::host_type(&profile), + HostType::Workstation | HostType::Node + ) { + out::hint(format!( + "It takes effect when it is applied: berth host apply {name}" + )); + } + Ok(()) +} + +fn edit(client: &Client, name: &str) -> Result<()> { + let profile = profile_of(client, name)?; + let dir = tempfile::Builder::new().prefix("berth-edit").tempdir()?; + let file = dir.path().join(format!("{name}.json")); + std::fs::write(&file, serde_json::to_string_pretty(&profile)? + "\n")?; + let editor = std::env::var("VISUAL") + .or_else(|_| std::env::var("EDITOR")) + .unwrap_or_else(|_| "vi".into()); + let mut words = editor.split_whitespace(); + let program = words.next().unwrap_or("vi"); + let status = std::process::Command::new(program) + .args(words) + .arg(&file) + .status() + .map_err(|error| Fail::failed(format!("{editor}: {error}")))?; + if !status.success() { + return Err(Fail::failed(format!( + "{editor} ended with {status}; nothing saved" + ))); + } + let edited: Value = serde_json::from_slice(&std::fs::read(&file)?).map_err(|error| { + Fail::usage(format!( + "The profile is no JSON now: {error}. Nothing saved" + )) + })?; + if edited == profile { + out::hint("Nothing changed"); + return Ok(()); + } + save(client, name, &edited)?; + out::done(format!("Saved {name}")); + Ok(()) +} + +/// What a file or standard input holds. +pub fn read_file(file: &str) -> Result> { + match file { + "-" => { + let mut bytes = vec![]; + std::io::Read::read_to_end(&mut std::io::stdin(), &mut bytes)?; + Ok(bytes) + } + file => std::fs::read(file).map_err(|error| Fail::usage(format!("{file}: {error}"))), + } +} + +fn import(client: &Client, name: &str, file: &str) -> Result<()> { + let profile: Value = serde_json::from_slice(&read_file(file)?) + .map_err(|error| Fail::usage(format!("{file} holds no JSON profile: {error}")))?; + save(client, name, &profile)?; + out::done(format!("Saved {name}")); + Ok(()) +} + +/// The sudo password `--sudo-password-stdin` gave, read once for the run. +pub fn sudo_from_stdin() -> Result { + static READ: std::sync::OnceLock = std::sync::OnceLock::new(); + if let Some(read) = READ.get() { + return Ok(read.clone()); + } + let read = ask::secret_from("Sudo password", None, true, "--sudo-password-stdin")?; + Ok(READ.get_or_init(|| read).clone()) +} + +/// The sudo password an action of a host needs, when its SSH user's sudo +/// asks for one and the server keeps none: from standard input, or asked +/// on the terminal once in a run. +pub fn sudo_password(client: &Client, name: &str) -> Result> { + static ASKED: std::sync::Mutex> = + std::sync::Mutex::new(BTreeMap::new()); + if out::get().sudo_stdin { + return sudo_from_stdin().map(Some); + } + if let Some(asked) = ASKED.lock().unwrap().get(name) { + return Ok(Some(asked.clone())); + } + let profile = profile_of(client, name)?; + if !profile::asks_become_pass(&profile) || profile["access"]["link"].is_array() { + return Ok(None); + } + let kept = client.call("secrets.list", json!({}), None)?; + let secret = format!("sudo/{name}"); + if kept["names"] + .as_array() + .is_some_and(|names| names.iter().any(|known| known == secret.as_str())) + { + return Ok(None); + } + match out::get().ask { + true => { + let asked = ask::secret( + &format!("Sudo password of the SSH user of {name}"), + "--sudo-password-stdin", + )?; + ASKED.lock().unwrap().insert(name.to_owned(), asked.clone()); + Ok(Some(asked)) + } + // Without one sudo says so itself, when it does want it. + false => Ok(None), + } +} + +pub fn apply(client: &Client, apply: &args::HostApply) -> Result<()> { + let name = &apply.name; + let profile = profile_of(client, name)?; + let action = match profile::host_type(&profile) { + HostType::Ssh => { + return Err(Fail::refused(format!( + "{name} is a host of the type ssh, neither a node nor a workstation, and berth installs nothing on it. Make it managed for the controller alone (berth host set {name} type=managed), or a node or a workstation, and apply it" + ))); + } + HostType::Managed => "install-controller", + _ => "apply", + }; + let act = Act { + sudo_password: sudo_password(client, name)?, + rows: Some(24), + cols: Some(100), + ..Act::default() + }; + let acted = client.act(&Subject::Host(name.clone()), action, &act)?; + let title = match action { + "apply" => format!("Applying {name}"), + _ => format!("Installing the controller on {name}"), + }; + follow::acted(client, acted, &title, apply.detach.detach, false)?; + Ok(()) +} + +fn seed(client: &Client, name: &str, file: Option<&std::path::Path>) -> Result<()> { + let acted = client.act(&Subject::Host(name.to_owned()), "seed", &Act::default())?; + let seed = acted.result.unwrap_or_default(); + let data = seed["user_data"].as_str().unwrap_or(""); + if out::get().json { + out::json(&seed); + return Ok(()); + } + // Until when its key joins the tailnet, for whoever starts the machine. + if let Some(note) = seed["tailnet"].as_str().filter(|note| !note.is_empty()) { + out::note(note); + } + match file { + Some(file) => { + write_private(file, data.as_bytes())?; + out::done(format!("Wrote the seed of {name} to {}", file.display())); + out::hint("It holds secrets of the host; keep it to yourself"); + } + None => print!("{data}"), + } + Ok(()) +} + +/// Write a file only its owner can read. +fn write_private(file: &std::path::Path, data: &[u8]) -> Result<()> { + let mut options = std::fs::OpenOptions::new(); + options.write(true).create(true).truncate(true); + #[cfg(unix)] + { + use std::os::unix::fs::OpenOptionsExt; + options.mode(0o600); + } + std::io::Write::write_all(&mut options.open(file)?, data)?; + Ok(()) +} + +fn installer(client: &Client, iso: &args::HostIso) -> Result<()> { + let mut arguments = BTreeMap::new(); + if let Some(device) = &iso.write { + ask::confirm(&format!( + "This erases {device} and writes the installer of {} to it.", + iso.name + ))?; + let password = match iso.password_stdin { + true => ask::secret_from("Sudo password", None, true, "--password-stdin")?, + false if out::get().ask => ask::secret( + "Your sudo password here, to write the drive (empty when sudo needs none)", + "--password-stdin", + )?, + false => String::new(), + }; + arguments.insert("device".to_owned(), device.clone()); + arguments.insert("password".to_owned(), password); + } + let act = Act { + arguments, + ..Act::default() + }; + let acted = client.act(&Subject::Host(iso.name.clone()), "installer", &act)?; + let title = match &iso.write { + Some(device) => format!( + "Building the installer of {} and writing it to {device}", + iso.name + ), + None => format!("Building the installer of {}", iso.name), + }; + let Some(job) = acted.job.as_ref().filter(|_| !iso.detach.detach) else { + follow::acted(client, acted, &title, iso.detach.detach, false)?; + return Ok(()); + }; + let ended = follow::job(client, &job.id, &title, false)?; + // Until when its key joins the tailnet, which the job's output says among its lines. + for line in &ended.tail { + if line.contains("joins the tailnet until") { + out::note(line); + } + } + if out::get().json { + out::json(&ended.job); + } + match ended.code { + 0 => Ok(()), + _ => Err(Fail::exit(1)), + } +} + +fn rename(client: &Client, name: &str, new: &str, hostname: bool) -> Result<()> { + let arguments = BTreeMap::from([ + ("name".to_owned(), new.to_owned()), + ( + "hostname".to_owned(), + if hostname { "yes" } else { "no" }.to_owned(), + ), + ]); + let act = Act { + arguments, + ..Act::default() + }; + let acted = client.act(&Subject::Host(name.to_owned()), "rename", &act)?; + follow::acted( + client, + acted, + &format!("Renaming {name} to {new}"), + false, + false, + )?; + Ok(()) +} + +fn remove(client: &Client, name: &str, destroy: bool) -> Result<()> { + let machine = machine_of(client, name)?; + match (&machine, destroy) { + (Some(machine), true) => { + let id = machine["id"].as_str().unwrap_or(""); + ask::confirm(&format!( + "This destroys {} of {}, the machine of {name}, and forgets {name}.", + machine["external_id"].as_str().unwrap_or(id), + machine["provisioner"].as_str().unwrap_or("its provisioner") + ))?; + let acted = client.call("resources.remove", json!({"id": id}), None)?; + let acted: berth_api::Acted = serde_json::from_value(acted["acted"].clone()) + .or_else(|_| serde_json::from_value(acted.clone()))?; + follow::acted( + client, + acted, + &format!("Destroying the machine of {name}"), + false, + false, + )?; + } + (Some(machine), false) => { + return Err(Fail::refused(format!( + "{name} has a machine {} made, {}. Destroy it with the host: berth host remove {name} --destroy", + machine["provisioner"].as_str().unwrap_or("a provisioner"), + machine["external_id"].as_str().unwrap_or("") + ))); + } + (None, true) => { + return Err(Fail::usage(format!( + "No provisioner made {name}, so there is nothing to destroy. berth host remove {name} forgets it" + ))); + } + (None, false) => ask::confirm(&format!( + "This forgets {name}, with its profile and passwords. The machine is left as it is." + ))?, + } + client.call("hosts.remove", json!({"name": name}), None)?; + out::done(format!("Removed {name}")); + Ok(()) +} + +fn adopt(client: &Client, adopt: &args::HostAdopt) -> Result<()> { + let mut arguments = BTreeMap::from([("name".to_owned(), adopt.name.clone())]); + if let Some(address) = &adopt.address { + arguments.insert("address".into(), address.clone()); + } + if let Some(port) = &adopt.port { + arguments.insert("port".into(), port.clone()); + } + if let Some(key) = &adopt.key { + arguments.insert("key".into(), key.clone()); + } + if adopt.force { + arguments.insert("force".into(), "yes".into()); + } + let sudo_password = match out::get().sudo_stdin { + true => Some(sudo_from_stdin()?), + false => None, + }; + let act = Act { + arguments, + sudo_password, + ..Act::default() + }; + let acted = client.act(&Subject::Installation, "adopt", &act)?; + follow::acted( + client, + acted, + &format!("Adopting {}", adopt.name), + adopt.detach.detach, + false, + )?; + Ok(()) +} + +fn credentials(client: &Client, name: &str, show: bool) -> Result<()> { + let found = client.call("hosts.credentials", json!({"name": name}), None)?; + if !out::get().terminal.0 && !show && !out::get().json { + return Err(Fail::refused( + "The passwords go to a terminal only. Add --show to print them here", + )); + } + out::data(&found, || { + let mut table = Table::new(&["account", "password"]); + for (account, password) in found["credentials"].as_object().into_iter().flatten() { + table.row(vec![ + account.clone(), + password.as_str().unwrap_or("").to_owned(), + ]); + } + table.print(&format!("berth generated no passwords for {name}")); + }); + Ok(()) +} + +fn verify(client: &Client, name: &str, accept: bool) -> Result<()> { + if accept { + ask::confirm(&format!( + "This forgets the host key recorded for {name} and takes the one it shows now." + ))?; + } + let act = Act { + arguments: BTreeMap::from([( + "accept".to_owned(), + if accept { "yes" } else { "no" }.to_owned(), + )]), + ..Act::default() + }; + let acted = client.act(&Subject::Host(name.to_owned()), "host-key", &act)?; + let result = acted.result.unwrap_or_default(); + out::data(&result, || { + let mut table = Table::new(&["type", "fingerprint"]); + for key in result["host_keys"].as_array().into_iter().flatten() { + table.row(vec![ + key["type"].as_str().unwrap_or("").to_owned(), + key["fingerprint"].as_str().unwrap_or("").to_owned(), + ]); + } + table.print(&format!("No host key of {name} is recorded yet")); + }); + Ok(()) +} + +fn power(client: &Client, power: &args::HostPower, verb: &str) -> Result<()> { + let name = &power.name; + let machine = machine_of(client, name)?.ok_or_else(|| { + Fail::usage(format!( + "No provisioner made {name}, so berth cannot {verb} it. berth host create NAME --provisioner P makes one that can" + )) + })?; + let id = machine["id"].as_str().unwrap_or("").to_owned(); + let acted = client.act(&Subject::Resource(id), verb, &Act::default())?; + let title = match verb { + "start" => format!("Starting {name}"), + _ => format!("Stopping {name}"), + }; + follow::acted(client, acted, &title, power.detach.detach, false)?; + Ok(()) +} diff --git a/crates/berth/src/cli/jobs.rs b/crates/berth/src/cli/jobs.rs new file mode 100644 index 0000000..7406428 --- /dev/null +++ b/crates/berth/src/cli/jobs.rs @@ -0,0 +1,200 @@ +//! `berth job …` and `berth rollout`: the work the server does, followed +//! again or started over all hosts. + +use super::args::{self, Job}; +use super::client::{Client, Subject}; +use super::fail::{Fail, Result}; +use super::follow; +use super::out::{self, Style, Table, styled}; +use berth_api::Act; +use berth_api::portal::Piece; +use serde_json::{Value, json}; +use std::io::Write; + +fn state_of(job: &Value) -> String { + if job["ended"].is_null() { + return styled("running", Style::Accent); + } + match job["exit_code"].as_i64() { + Some(0) => styled("done", Style::Good), + Some(code) => styled(&format!("failed ({code})"), Style::Bad), + None => styled("failed", Style::Bad), + } +} + +pub fn command(client: &Client, command: Job) -> Result<()> { + match command { + Job::List { limit } => { + let found = client.call("jobs.list", json!({"limit": limit}), None)?; + out::data(&found, || { + let mut table = Table::new(&["id", "action", "subject", "state", "started"]); + for job in found.as_array().into_iter().flatten() { + let text = |key: &str| job[key].as_str().unwrap_or("").to_owned(); + let subject = match text("host").as_str() { + "" => text("subject"), + host => host.to_owned(), + }; + table.row(vec![ + text("id"), + text("action"), + subject, + state_of(job), + text("started"), + ]); + } + table.print("No jobs yet"); + }); + Ok(()) + } + Job::Show { id } => { + let job = client.call("jobs.get", json!({"id": id}), None)?; + out::data(&job, || { + let text = |key: &str| job[key].as_str().unwrap_or("").to_owned(); + out::fields(&[ + ("Id", text("id")), + ("Action", text("action")), + ("Subject", text("subject")), + ("Host", text("host")), + ( + match text("action").as_str() { + "exec" => "Command", + _ => "Arguments", + }, + berth_tui::host::job_arguments(&job).unwrap_or_default(), + ), + ("State", state_of(&job)), + ("Started", text("started")), + ("Ended", text("ended")), + ]); + if let Some(report) = job["report"].as_object() { + let report: berth_api::RolloutReport = + serde_json::from_value(Value::Object(report.clone())).unwrap_or_default(); + for line in berth_tui::plugins::summary(&report) { + println!("{line}"); + } + } + }); + Ok(()) + } + Job::Logs { id, follow } => logs(client, &id, follow), + Job::Stop { id } => { + let job = client.call("jobs.stop", json!({"id": id}), None)?; + out::data(&job, || { + out::done(format!( + "Asked {id} to stop. berth job wait {id} waits for its end" + )) + }); + Ok(()) + } + Job::Wait { id } => { + let job = client.call("jobs.get", json!({"id": id}), None)?; + let title = format!( + "{} {}", + job["action"].as_str().unwrap_or("job"), + job["host"].as_str().unwrap_or("") + ); + let ended = follow::job(client, &id, title.trim(), false)?; + if out::get().json { + out::json(&ended.job); + } + match ended.code { + 0 => Ok(()), + _ => Err(Fail::exit(1)), + } + } + } +} + +/// What a job wrote, with its colours where they are shown, and with +/// `follow` what it writes until it ends. +fn logs(client: &Client, id: &str, follow: bool) -> Result<()> { + let colour = out::get().colour.0; + let write = |bytes: &[u8]| -> std::io::Result<()> { + let mut stdout = std::io::stdout().lock(); + match colour { + true => stdout.write_all(bytes)?, + false => { + stdout.write_all(out::strip_ansi(&String::from_utf8_lossy(bytes)).as_bytes())? + } + } + stdout.flush() + }; + if !follow { + let found = client.call("jobs.output", json!({"id": id}), None)?; + let bytes = match &found["data_b64"] { + Value::String(data) => berth_api::unb64(data).unwrap_or_default(), + _ => match &found["lines"] { + Value::Array(lines) => lines + .iter() + .map(|line| format!("{}\n", line.as_str().unwrap_or(""))) + .collect::() + .into_bytes(), + _ => found["base64"] + .as_str() + .and_then(berth_api::unb64) + .unwrap_or_default(), + }, + }; + write(&bytes)?; + return Ok(()); + } + let mut stream = client.stream("jobs.follow", None, json!({"id": id}))?; + let code = client.wait(async { + let mut code = None; + loop { + tokio::select! { + piece = stream.recv() => match piece { + Some(Ok(Piece::Binary(bytes))) => write(&bytes)?, + Some(Ok(Piece::Text(said))) => { + if let Some(exited) = said.get("exited") { + code = Some(exited["exit_code"].as_i64().unwrap_or(1) as i32); + } + } + Some(Err(error)) => return Err(Fail::from(error)), + None => return Ok(code), + }, + _ = tokio::signal::ctrl_c() => return Ok(Some(0)), + } + } + })?; + match code.unwrap_or(0) { + 0 => Ok(()), + _ => Err(Fail::exit(1)), + } +} + +/// `berth rollout`: the installation's roll-out, as a job of the server. +pub fn rollout(client: &Client, rollout: &args::Rollout) -> Result<()> { + let options = berth_api::Rollout { + hosts: rollout.only.clone(), + except: rollout.except.clone(), + all: rollout.all, + update: rollout.update, + reboot: rollout.reboot, + force: rollout.force, + }; + let act = Act { + arguments: options.arguments(), + ..Act::default() + }; + let acted = client.act(&Subject::Installation, "rollout", &act)?; + let (Some(job), false) = (&acted.job, rollout.detach.detach) else { + follow::acted(client, acted, "Rolling out", rollout.detach.detach, false)?; + return Ok(()); + }; + // The line of each host is said whether the roll-out ended well or not. + let ended = follow::job(client, &job.id, "Rolling out", false)?; + if out::get().json { + out::json(&ended.job); + } else if ended.job["report"].is_object() { + let report: berth_api::RolloutReport = + serde_json::from_value(ended.job["report"].clone()).unwrap_or_default(); + for line in berth_tui::plugins::summary(&report) { + println!("{line}"); + } + } + match ended.code { + 0 => Ok(()), + _ => Err(Fail::exit(1)), + } +} diff --git a/crates/berth/src/cli/login.rs b/crates/berth/src/cli/login.rs new file mode 100644 index 0000000..13e23c9 --- /dev/null +++ b/crates/berth/src/cli/login.rs @@ -0,0 +1,127 @@ +//! `berth login URL`: use the berth server at URL, with the token a setup +//! key collects, or a token given with `--token-file` or `--token-stdin`. +//! The key comes from a prompt or standard input, never from the command +//! line, and the server and token are kept as this user's settings. + +use super::args::{Global, Login}; +use super::ask; +use super::client; +use super::fail::{Fail, Result}; +use super::out; +use berth_page::net; +use berth_page::settings::{self, Settings}; +use serde_json::Value; + +pub fn login(global: &Global, login: &Login) -> Result<()> { + let url = berth_page::device::address(&login.url); + if !(url.starts_with("https://") || url.starts_with("http://")) { + return Err(Fail::usage(format!( + "{} is no address of a berth server: https://HOST or http://HOST:PORT", + login.url + ))); + } + let settings = client::settings(); + // A token given as such is checked and kept; else a setup key collects one. + if global.token_file.is_some() || global.token_stdin { + let token = read_token(global)?; + let portal = client::connect(&url, &token)?; + net::runtime() + .block_on(portal.call("access", None, Value::Null, None)) + .map_err(Fail::from)?; + settings + .set(&[(settings::SERVER, &url), (settings::TOKEN, &token)]) + .map_err(|error| Fail::failed(error.to_string()))?; + out::done(format!("berth uses the berth server at {url} now")); + return Ok(()); + } + let key = ask::secret_from( + "Setup key", + None, + login.setup_key_stdin, + "--setup-key-stdin", + )?; + let (url, version) = claim(&settings, &url, key.trim())?; + out::done(format!( + "berth uses the berth server at {url} now, berth {version}" + )); + if std::env::var_os("BERTH_SERVER").is_some() { + out::warn("BERTH_SERVER is set in this shell and is used before the setting"); + } + Ok(()) +} + +fn read_token(global: &Global) -> Result { + if let Some(file) = &global.token_file { + return Ok(std::fs::read_to_string(file)?.trim().to_owned()); + } + let mut text = String::new(); + std::io::Read::read_to_string(&mut std::io::stdin(), &mut text)?; + Ok(text.trim().to_owned()) +} + +pub fn logout() -> Result<()> { + let settings = client::settings(); + let server = settings.get(settings::SERVER); + settings + .set(&[(settings::SERVER, ""), (settings::TOKEN, "")]) + .map_err(|error| Fail::failed(error.to_string()))?; + match server { + Some(server) => out::done(format!("berth no longer uses the berth server at {server}")), + None => out::hint("berth used no server of its own"), + } + Ok(()) +} + +/// Claim a token with `key` at `url`: the server's address and version. +/// Nothing is kept unless all of it works. +pub fn claim(settings: &Settings, url: &str, key: &str) -> Result<(String, String)> { + let url = url.trim().trim_end_matches('/').to_owned(); + let key = key.to_owned(); + let ask = |method: &'static str| { + let (url, key) = (url.clone(), key.clone()); + async move { + let (status, body) = net::http(method, &url, "/v1/setup-key", &key) + .await + .map_err(Fail::unreachable)?; + let said: Value = serde_json::from_str(&body).unwrap_or(Value::Null); + match status { + 200 => Ok(said), + 401 => Err(Fail::refused( + "The server did not take the setup key: it is wrong, used up or expired. Make a new one where the server runs: berth server setup-key", + )), + 404 | 405 => Err(Fail::refused( + "The server knows no setup keys: it is an older berth", + )), + status => Err(Fail::failed(match said["error"]["message"].as_str() { + Some(message) => format!("HTTP {status}: {message}"), + None => format!("HTTP {status}"), + })), + } + } + }; + let (version, token) = net::runtime().block_on(async { + let checked = ask("GET").await?; + let claimed = ask("POST").await?; + let token = claimed["token"].as_str().unwrap_or("").to_owned(); + if token.is_empty() { + return Err(Fail::failed("The server sent no token")); + } + let version = checked["version"].as_str().unwrap_or("").to_owned(); + Ok::<_, Fail>((version, token)) + })?; + // Kept only once the token is seen to open the server. + let portal = client::connect(&url, &token)?; + net::runtime() + .block_on(portal.call("access", None, Value::Null, None)) + .map_err(Fail::from)?; + settings + .set(&[(settings::SERVER, &url), (settings::TOKEN, &token)]) + .map_err(|error| Fail::failed(error.to_string()))?; + if let Err(error) = net::runtime().block_on(ask("DELETE")) { + out::warn(format!( + "The setup key could not be ended ({}). It stops by itself within a day, or with berth server setup-key --revoke where the server runs", + error.message + )); + } + Ok((url, version)) +} diff --git a/crates/berth/src/cli/mod.rs b/crates/berth/src/cli/mod.rs new file mode 100644 index 0000000..05f5c24 --- /dev/null +++ b/crates/berth/src/cli/mod.rs @@ -0,0 +1,460 @@ +//! berth's command line: one grammar, `berth THING ACTION [NAME] [FLAGS]`, +//! whose commands are calls of the same API the screens use. + +pub mod args; +pub mod ask; +pub mod client; +mod data; +#[cfg(test)] +mod docs; +pub mod fail; +pub mod follow; +mod host; +mod jobs; +pub mod login; +mod old; +pub mod out; +mod pages; +mod setup; +mod things; + +#[cfg(target_os = "linux")] +pub use host::{profile_of as host_profile, save as save_host}; + +use args::{Cli, Command}; +use clap::{CommandFactory, FromArgMatches}; +use client::Client; +use fail::{Fail, Result}; + +/// What bare `berth` prints: what it is, the commands most people need +/// first, and how to open the screens. +pub const SHORT_HELP: &[(&str, &[(&str, &str)])] = &[ + ( + "Start here", + &[ + ("berth ui", "the screens: hosts, set-up, plugins"), + ( + "berth host create NAME", + "describe a new host, then install it", + ), + ( + "berth host add NAME ADDRESS", + "take in a machine that exists", + ), + ("berth login URL", "use the hosts of a berth server"), + ], + ), + ( + "Every day", + &[ + ("berth ls", "the hosts and whether each answers"), + ("berth exec HOST -- COMMAND", "run a command on a host"), + ("berth ssh HOST", "a shell on a host"), + ("berth apply HOST", "apply a host's profile to it"), + ], + ), +]; + +fn short_help() { + use out::{Style, styled}; + println!("berth sets up and looks after agent workstations."); + let wide = SHORT_HELP + .iter() + .flat_map(|(_, lines)| lines.iter().map(|(command, _)| command.len())) + .max() + .unwrap_or(0); + for (heading, lines) in SHORT_HELP { + println!("\n{}", styled(heading, Style::Bold)); + for (command, what) in *lines { + println!( + " {}{} {}", + styled(command, Style::Accent), + " ".repeat(wide - command.len()), + styled(what, Style::Dim) + ); + } + } + println!("\nberth help lists every command. berth THING --help explains one."); +} + +/// The command tree, as clap and the reference page take it. +pub fn command() -> clap::Command { + let statuses: Vec = args::EXIT_STATUSES + .iter() + .map(|(status, means)| format!("{status} {means}")) + .collect(); + Cli::command().styles(help_styles()).after_help(format!( + "berth THING --help explains a thing and its actions.\nExit statuses: {}. A command run on a host passes its own through.", + statuses.join("; ") + )) +} + +/// The colours of the help and of clap's errors, from the screens' palette. +/// clap leaves them out by itself without a terminal or with `NO_COLOR`. +fn help_styles() -> clap::builder::Styles { + use berth_tui::theme::{Depth, Role, colour}; + use clap::builder::styling::{Ansi256Color, AnsiColor, Color, RgbColor, Style, Styles}; + let depth = Depth::of_terminal(|name| std::env::var(name).ok()); + let paint = |role: Role| -> Option { + use ratatui::style::Color as Tui; + match colour(role, depth)? { + Tui::Rgb(r, g, b) => Some(RgbColor(r, g, b).into()), + Tui::Indexed(index) => Some(Ansi256Color(index).into()), + Tui::LightBlue => Some(AnsiColor::BrightBlue.into()), + Tui::Green => Some(AnsiColor::Green.into()), + Tui::Yellow => Some(AnsiColor::Yellow.into()), + Tui::LightRed => Some(AnsiColor::BrightRed.into()), + _ => None, + } + }; + let style = |role: Role| Style::new().fg_color(paint(role)); + Styles::styled() + .header(Style::new().bold()) + .usage(Style::new().bold()) + .literal(style(Role::Accent)) + .placeholder(match depth { + Depth::Basic => Style::new().dimmed(), + _ => style(Role::Muted), + }) + .valid(style(Role::Good)) + .invalid(style(Role::Warn)) + .error(style(Role::Bad).bold()) + .context(style(Role::Muted)) +} + +/// Run berth with these arguments: the exit status. +pub fn main(args: Vec) -> i32 { + if let Some(fail) = old::refuse(&args) { + out::fail(&fail); + return fail.status; + } + let matches = match command() + .color(colour_asked(&args)) + .try_get_matches_from(&args) + { + Ok(matches) => matches, + Err(error) => return clap_error(error, &args), + }; + let cli = match Cli::from_arg_matches(&matches) { + Ok(cli) => cli, + Err(error) => return clap_error(error, &args), + }; + out::init(&cli.global); + match run(cli) { + Ok(()) => 0, + Err(fail) => { + out::fail(&fail); + fail.status + } + } +} + +/// `--color` as given, for the help clap prints before berth reads its +/// flags. +fn colour_asked(args: &[String]) -> clap::ColorChoice { + let mut words = args.iter().skip(1).take_while(|arg| *arg != "--"); + while let Some(word) = words.next() { + let value = match word.strip_prefix("--color") { + Some("") => words.next().map(String::as_str), + Some(rest) => rest.strip_prefix('='), + None => None, + }; + match value { + Some("always") => return clap::ColorChoice::Always, + Some("never") => return clap::ColorChoice::Never, + Some(_) => return clap::ColorChoice::Auto, + None => {} + } + } + clap::ColorChoice::Auto +} + +/// Help and versions as clap prints them, and usage errors as berth says +/// them: one sentence, with what was meant when that is clear. +fn clap_error(error: clap::Error, args: &[String]) -> i32 { + use clap::error::{ContextKind, ContextValue, ErrorKind}; + match error.kind() { + ErrorKind::DisplayHelp | ErrorKind::DisplayVersion => { + let _ = error.print(); + return 0; + } + ErrorKind::DisplayHelpOnMissingArgumentOrSubcommand => { + let _ = error.print(); + return fail::USAGE; + } + _ => {} + } + let json = args + .windows(2) + .any(|pair| (pair[0] == "-o" || pair[0] == "--output") && pair[1] == "json") + || args + .iter() + .any(|arg| arg == "--output=json" || arg == "-ojson"); + let value = |kind| match error.get(kind) { + Some(ContextValue::String(text)) => Some(text.clone()), + Some(ContextValue::Strings(texts)) => texts.first().cloned(), + _ => None, + }; + let message = match error.kind() { + ErrorKind::InvalidSubcommand => { + let word = value(ContextKind::InvalidSubcommand).unwrap_or_default(); + let before = before_word(args, &word); + match value(ContextKind::SuggestedSubcommand) { + Some(meant) => { + let after = after_word(args, &word); + format!( + "no command \"{word}\". Did you mean: {}", + [before.as_str(), meant.as_str(), after.as_str()] + .iter() + .filter(|part| !part.is_empty()) + .copied() + .collect::>() + .join(" ") + ) + } + None => format!("no command \"{word}\". {before} --help lists the commands"), + } + } + _ => { + let text = error.render().to_string(); + let mut lines = text.lines().filter(|line| !line.trim().is_empty()); + let first = lines + .next() + .unwrap_or("") + .trim_start_matches("error: ") + .to_owned(); + let tip = lines + .find(|line| line.trim_start().starts_with("tip:")) + .map(|tip| format!(". {}", tip.trim().trim_start_matches("tip: "))) + .unwrap_or_default(); + let usage = text + .lines() + .find(|line| line.starts_with("Usage:")) + .map(|usage| format!(". {}", usage.trim())) + .unwrap_or_default(); + format!("{first}{tip}{usage}") + } + }; + let fail = Fail::usage(message); + if json { + let body = serde_json::json!({"error": {"code": fail.code, "message": fail.message, "exit": fail.status}}); + eprintln!("{body}"); + } else { + eprintln!("berth: {}", fail.message); + } + fail.status +} + +/// The command up to an unknown word, as `berth host`. +fn before_word(args: &[String], word: &str) -> String { + let at = args + .iter() + .position(|arg| arg == word) + .unwrap_or(args.len()); + let mut words: Vec<&str> = vec!["berth"]; + words.extend( + args[1..at.max(1)] + .iter() + .filter(|arg| !arg.starts_with('-')) + .map(String::as_str), + ); + words.join(" ") +} + +/// What came after an unknown word. +fn after_word(args: &[String], word: &str) -> String { + let at = args + .iter() + .position(|arg| arg == word) + .map_or(args.len(), |at| at + 1); + args[at..].join(" ") +} + +fn run(cli: Cli) -> Result<()> { + let global = cli.global; + let Some(command) = cli.command else { + short_help(); + return Ok(()); + }; + let open = || Client::open(&global); + match command { + Command::Ls => host::list(&open()?), + Command::Apply(apply) => host::apply(&open()?, &apply), + Command::Exec(exec) => things::exec(&open()?, &exec), + Command::Ssh(ssh) => crate::local::ssh(&global, &ssh), + Command::Run(run) => things::run_action(&open()?, &run), + Command::Ui { host } => crate::screens(&global, host.as_deref()), + Command::Host(command) => host::command(&global, command), + Command::Rollout(rollout) => jobs::rollout(&open()?, &rollout), + Command::Action(args::ActionCommand::List { host }) => { + things::actions(&open()?, host.as_deref()) + } + Command::Session(command) => things::session(&open()?, command), + Command::Provisioner(command) => things::provisioner(&open()?, command), + Command::Resource(command) => things::resource(&open()?, command), + Command::Job(command) => jobs::command(&open()?, command), + Command::Key(command) => setup::key(&open()?, command), + Command::Secret(command) => setup::secret(&open()?, command), + Command::Registry(command) => setup::registry(&global, command), + Command::Plugin(command) => setup::plugin(&open()?, command), + Command::Config(command) => setup::config(&open()?, command), + Command::Port(args::Port::Forward { + target, + ports, + resource, + address, + }) => data::forward(&open()?, &target, &ports, resource, &address), + Command::Inbox(command) => data::inbox(&open()?, command), + Command::Kv(command) => data::kv(&open()?, command), + Command::Channel(command) => data::channel(&open()?, command), + Command::Service(command) => data::service(&open()?, command), + #[cfg(target_os = "linux")] + Command::Server(command) => crate::local::server_command(command), + #[cfg(target_os = "linux")] + Command::Serve(serve) => crate::local::server_command(args::Server::Run(serve)), + Command::Login(login) => login::login(&global, &login), + Command::Logout => login::logout(), + Command::Update(update) => crate::update::command(&update), + Command::Version => version(&global), + Command::Completion { shell } => { + let mut tree = self::command(); + clap_complete::generate(shell, &mut tree, "berth", &mut std::io::stdout()); + Ok(()) + } + Command::Askpass { .. } => Err(Fail::usage( + "berth askpass is run by ssh, with what it needs in its environment", + )), + #[cfg(target_os = "linux")] + Command::TailnetKey { host } => crate::local::tailnet_key(&host), + } +} + +/// `berth version`: this berth's, and that of the server it is set to use, +/// which an older server still answers. +fn version(global: &args::Global) -> Result<()> { + use serde_json::json; + let server = client::named(global).map(|url| { + match Client::open_any(global, &client::settings()) + .and_then(|client| client.server_version()) + { + Ok((version, api)) => json!({"url": url, "version": version, "api": api}), + Err(fail) => json!({"url": url, "error": fail.message}), + } + }); + let mut said = json!({"version": berth_api::VERSION, "api": berth_api::API}); + if let Some(server) = &server { + said["server"] = server.clone(); + } + out::data(&said, || { + println!("berth {} (API {})", berth_api::VERSION, berth_api::API); + if let Some(server) = &server { + let url = server["url"].as_str().unwrap_or(""); + match server["version"].as_str() { + Some(version) => { + println!("server {url}: berth {version} (API {})", server["api"]); + let api = server["api"].as_u64().unwrap_or(0) as u32; + if let Some(older) = berth_api::compat::older_server(url, version, api) { + out::warn(older); + } + } + None => println!( + "server {url}: not reached ({})", + server["error"].as_str().unwrap_or("") + ), + } + } + }); + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn parse(line: &str) -> std::result::Result { + let args: Vec<&str> = std::iter::once("berth") + .chain(line.split_whitespace()) + .collect(); + let matches = command().try_get_matches_from(args)?; + Cli::from_arg_matches(&matches) + } + + #[test] + fn the_definition_holds_together() { + command().debug_assert(); + } + + #[test] + fn the_short_help_names_commands_there_are() { + for (_, lines) in SHORT_HELP { + for (command, _) in *lines { + let line = command + .trim_start_matches("berth") + .replace("NAME", "studio") + .replace("HOST", "studio") + .replace("ADDRESS", "192.0.2.10") + .replace("URL", "https://berth.example") + .replace("COMMAND", "true"); + assert!(parse(&line).is_ok(), "{command}"); + } + } + } + + #[test] + fn flags_go_before_or_after_the_names_in_both_forms() { + for line in [ + "host show studio -o json", + "-o json host show studio", + "host show --output=json studio", + "--no-input host apply studio --detach", + ] { + assert!(parse(line).is_ok(), "{line}"); + } + let cli = parse("session logs studio s1 --tail 50").unwrap(); + let cli2 = parse("session logs --tail=50 studio s1").unwrap(); + assert_eq!(format!("{:?}", cli.command), format!("{:?}", cli2.command)); + } + + #[cfg(target_os = "linux")] + #[test] + fn the_command_line_of_a_server_of_0_5_still_starts_it() { + // Containers and units of berth 0.5 run the server with --web. + let cli = parse("serve --listen 0.0.0.0:7777 --expose --web").unwrap(); + assert!(matches!(cli.command, Some(Command::Serve(serve)) if serve.web && serve.expose)); + let help = command() + .find_subcommand_mut("serve") + .unwrap() + .render_help() + .to_string(); + assert!(!help.contains("--web"), "{help}"); + } + + #[test] + fn a_host_may_have_the_name_of_an_action() { + let cli = parse("host show list").unwrap(); + assert!(matches!( + cli.command, + Some(Command::Host(args::Host::Show { name })) if name == "list" + )); + let cli = parse("exec add -- uname -a").unwrap(); + assert!( + matches!(cli.command, Some(Command::Exec(exec)) if exec.host == "add" && exec.command == ["uname", "-a"]) + ); + } + + #[test] + fn an_unknown_word_is_a_usage_error_with_what_was_meant() { + let args: Vec = ["berth", "aply", "studio"].map(String::from).to_vec(); + assert_eq!(main(args), fail::USAGE); + let error = parse("aply studio").unwrap_err(); + assert_eq!(error.kind(), clap::error::ErrorKind::InvalidSubcommand); + assert_eq!( + after_word(&["berth".into(), "aply".into(), "studio".into()], "aply"), + "studio" + ); + assert_eq!( + before_word(&["berth".into(), "host".into(), "shwo".into()], "shwo"), + "berth host" + ); + } +} diff --git a/crates/berth/src/cli/old.rs b/crates/berth/src/cli/old.rs new file mode 100644 index 0000000..bb90335 --- /dev/null +++ b/crates/berth/src/cli/old.rs @@ -0,0 +1,226 @@ +//! The spellings of berth 0.5 that mean something else now, or nothing. +//! Each fails with the spelling to use instead, so that a script or a +//! habit does not do something it did not mean. + +use super::fail::Fail; + +/// The flags of every command that take a value, which come before the +/// first word of the command. +const VALUED: &[&str] = &["--server", "--token-file", "-o", "--output", "--color"]; + +/// Old command words and what to type now. +const OLD: &[(&[&str], &str)] = &[ + (&["list"], "berth host list, or berth ls"), + (&["remove"], "berth host remove NAME"), + ( + &["profile"], + "berth host export NAME, berth host import NAME FILE or berth host set NAME KEY=VALUE", + ), + (&["credentials"], "berth host credentials NAME"), + (&["adopt"], "berth host adopt NAME [ADDRESS]"), + (&["iso"], "berth host iso NAME"), + (&["seed"], "berth host seed NAME"), + (&["inventory"], "berth host inventory [NAME…]"), + (&["provisioners"], "berth provisioner list"), + ( + &["provision"], + "berth host create NAME --provisioner PROVISIONER", + ), + ( + &["instance"], + "berth host start NAME, berth host stop NAME or berth host remove NAME --destroy", + ), + ( + &["deploy"], + "berth resource create NAME --provisioner PROVISIONER --spec FILE", + ), + (&["actions"], "berth action list HOST"), + (&["manage"], "berth run HOST ACTION"), + (&["attach"], "berth session attach HOST [ID]"), + (&["sessions"], "berth session list HOST"), + (&["output"], "berth session logs HOST ID"), + (&["call"], "berth service call HOST SERVICE METHOD"), + (&["plugins"], "berth plugin list, add, remove or update"), + (&["connect"], "berth login URL"), + (&["setup-key"], "berth server setup-key"), + (&["store"], "berth server store"), + (&["openapi"], "berth server api openapi"), + (&["asyncapi"], "berth server api asyncapi"), + (&["operations"], "berth server api operations"), + ( + &["publish"], + "berth registry publish DESTINATION --host HOST", + ), + (&["image"], "berth registry push-image DESTINATION"), + ( + &["host", "install"], + "berth host apply NAME for a managed host, or berth run HOST install-controller", + ), + (&["host", "type"], "berth host set NAME type=TYPE"), + (&["key", "new"], "berth key create NAME"), + (&["key", "add"], "berth key import NAME FILE"), + (&["key", "host"], "berth host verify NAME [--accept]"), + ( + &["registry", "repo"], + "berth registry add-repo NAME FORMAT REPOSITORY", + ), + (&["apply", "--behind"], "berth rollout"), + (&["apply", "--all"], "berth rollout --all"), + (&["channel"], "berth channel listen HOST"), +]; + +/// The words of the command, without its flags. +fn words(args: &[String]) -> Vec<&str> { + let mut words = vec![]; + let mut rest = args.iter().skip(1); + while let Some(arg) = rest.next() { + if VALUED.contains(&arg.as_str()) { + rest.next(); + continue; + } + // What comes after `--` is a command for a host, not berth's words. + if arg == "--" { + break; + } + // The flags the old spellings name are kept; the others are left out. + if arg.starts_with('-') && !matches!(arg.as_str(), "--behind" | "--all") { + continue; + } + words.push(arg.as_str()); + } + words +} + +/// Why these arguments are refused, when they are a spelling of before. +pub fn refuse(args: &[String]) -> Option { + if args + .iter() + .skip(1) + .any(|arg| arg == "--token" || arg.starts_with("--token=")) + { + return Some(Fail::usage( + "--token puts the token where other users can read it. Use --token-file FILE or --token-stdin, or keep it with berth login URL", + )); + } + let words = words(args); + let first = *words.first()?; + for (old, new) in OLD { + let matches = old + .iter() + .enumerate() + .all(|(index, word)| words.get(index) == Some(word)); + if !matches { + continue; + } + // `berth channel listen` is the new spelling of the old `channel`. + if old == &["channel"] && words.get(1) == Some(&"listen") { + continue; + } + // A word that is a command now as well, with its new grammar. + if old.len() == 1 && first == "channel" && words.len() < 2 { + continue; + } + return Some(Fail::usage(format!("berth {} is {new} now", old.join(" ")))); + } + // `berth resource ID remove` and the like: the action goes first now. + if let [ + noun @ ("resource" | "provisioner" | "host"), + name, + action, + .., + ] = words[..] + && !name.starts_with('-') + && !is_action(noun, name) + && is_action(noun, action) + { + return Some(Fail::usage(format!( + "The action comes before the name now: berth {noun} {action} {name}" + ))); + } + None +} + +fn is_action(noun: &str, word: &str) -> bool { + let command = super::command(); + command + .find_subcommand(noun) + .is_some_and(|thing| thing.find_subcommand(word).is_some()) +} + +/// The table of the old spellings on the page about upgrading, between its +/// markers, as a test writes it. +#[cfg(test)] +fn table() -> String { + let mut out = String::from("| berth 0.5 | Now |\n| --- | --- |\n"); + for (old, new) in OLD { + out += &format!("| `berth {}` | `{new}` |\n", old.join(" ")); + } + out +} + +#[cfg(test)] +mod tests { + use super::*; + + /// The page about upgrading from 0.5, from the repository's root. + const PAGE: &str = "docs/content/docs/(guide)/use/upgrading.mdx"; + const START: &str = "{/* old-spellings:start */}\n"; + const END: &str = "{/* old-spellings:end */}"; + + #[test] + fn the_upgrade_page_lists_the_old_spellings() { + let file = std::path::Path::new(env!("CARGO_MANIFEST_DIR")) + .join("../..") + .join(PAGE); + let text = std::fs::read_to_string(&file).unwrap(); + let (before, rest) = text.split_once(START).expect("the start marker"); + let (_, after) = rest.split_once(END).expect("the end marker"); + // Blank lines around it, so that MDX reads it as a table. + let page = format!("{before}{START}\n{}\n{END}{after}", table()); + if std::env::var_os("BERTH_UPDATE_DOCS").is_some() { + std::fs::write(&file, &page).unwrap(); + return; + } + assert!( + text == page, + "{PAGE} is stale: write it again with BERTH_UPDATE_DOCS=1 cargo test -p berth" + ); + } + + fn said(line: &str) -> Option { + let args: Vec = std::iter::once("berth") + .chain(line.split_whitespace()) + .map(String::from) + .collect(); + refuse(&args).map(|fail| fail.message) + } + + #[test] + fn old_spellings_name_the_new_ones() { + assert_eq!( + said("remove studio").as_deref(), + Some("berth remove is berth host remove NAME now") + ); + assert_eq!( + said("--server https://berth.example apply --behind").as_deref(), + Some("berth apply --behind is berth rollout now") + ); + assert_eq!( + said("resource r1 remove").as_deref(), + Some("The action comes before the name now: berth resource remove r1") + ); + assert!(said("channel studio").is_some()); + assert!(said("--token s3cret ls").unwrap().contains("--token-file")); + for fine in [ + "host remove studio", + "apply studio", + "channel listen studio", + "ls", + "resource show r1", + "channel --help", + "rollout --all", + ] { + assert_eq!(said(fine), None, "{fine}"); + } + } +} diff --git a/crates/berth/src/cli/out.rs b/crates/berth/src/cli/out.rs new file mode 100644 index 0000000..cecbdcf --- /dev/null +++ b/crates/berth/src/cli/out.rs @@ -0,0 +1,341 @@ +//! What a command prints: results on standard output, as an aligned table +//! or sentence for people and the API's own JSON with `-o json`, and what +//! goes on, hints and warnings on standard error. + +use super::args::{Colour, Global, Output}; +use super::fail::Fail; +use berth_tui::theme::{self, Depth, Role}; +use serde_json::Value; +use std::io::{IsTerminal, Write}; +use std::sync::OnceLock; + +#[derive(Debug, Clone, Copy)] +pub struct Out { + pub json: bool, + pub quiet: bool, + /// Colours on standard output and on standard error. + pub colour: (bool, bool), + /// How many colours the terminal shows, when there are any. + pub depth: Depth, + /// Whether each of the two is a terminal. + pub terminal: (bool, bool), + /// Whether someone can be asked: a terminal, and no --no-input. + pub ask: bool, + pub yes: bool, + /// `--sudo-password-stdin`: a host's sudo password is read from standard input. + pub sudo_stdin: bool, +} + +static OUT: OnceLock = OnceLock::new(); + +/// Set how this run prints, from its flags and its terminal. +pub fn init(global: &Global) { + let terminal = ( + std::io::stdout().is_terminal(), + std::io::stderr().is_terminal(), + ); + let no_colour = std::env::var_os("NO_COLOR").is_some_and(|value| !value.is_empty()); + let colour = match global.color { + Colour::Always => (true, true), + Colour::Never => (false, false), + Colour::Auto if no_colour => (false, false), + Colour::Auto => terminal, + }; + let depth = Depth::of_terminal(|name| std::env::var(name).ok()); + let colour = match depth { + // A dumb terminal shows the escapes as text, unless asked for. + Depth::None if global.color == Colour::Auto => (false, false), + _ => colour, + }; + let out = Out { + json: global.output == Output::Json, + quiet: global.quiet, + colour, + depth: match depth { + Depth::None => Depth::Basic, + depth => depth, + }, + terminal, + ask: !global.no_input && std::io::stdin().is_terminal() && terminal.1, + yes: global.yes, + sudo_stdin: global.sudo_password_stdin, + }; + let _ = OUT.set(out); +} + +pub fn get() -> Out { + *OUT.get_or_init(|| Out { + json: false, + quiet: false, + colour: (false, false), + depth: Depth::Basic, + terminal: (false, false), + ask: false, + yes: false, + sudo_stdin: false, + }) +} + +/// A piece of text in a style, when the stream it goes to takes colours. +#[derive(Clone, Copy)] +pub enum Style { + Good, + Bad, + Warn, + Dim, + Bold, + Accent, +} + +fn paint(text: &str, style: Style, on: bool) -> String { + if !on || text.is_empty() { + return text.to_owned(); + } + let code = match style { + Style::Bold => "1".into(), + Style::Good => theme::sgr(Role::Good, get().depth), + Style::Bad => theme::sgr(Role::Bad, get().depth), + Style::Warn => theme::sgr(Role::Warn, get().depth), + Style::Dim => theme::sgr(Role::Muted, get().depth), + Style::Accent => theme::sgr(Role::Accent, get().depth), + }; + format!("\x1b[{code}m{text}\x1b[0m") +} + +/// A style of ratatui in `role`'s colour, for what the command line draws +/// with it on standard error: prompts and progress. Plain without colours. +pub fn look(role: Role) -> ratatui::style::Style { + let out = get(); + match out.colour.1 { + true => theme::style(role, out.depth), + false if role == Role::Muted => ratatui::style::Style::new().dim(), + false => ratatui::style::Style::new(), + } +} + +/// `text` styled for standard output. +pub fn styled(text: &str, style: Style) -> String { + paint(text, style, get().colour.0) +} + +/// `text` styled for standard error. +pub fn styled_err(text: &str, style: Style) -> String { + paint(text, style, get().colour.1) +} + +/// A result for people, unless only JSON is wanted. +pub fn line(text: impl AsRef) { + if !get().json { + println!("{}", text.as_ref()); + } +} + +/// A document of the API as the result. +pub fn json(value: &Value) { + let text = serde_json::to_string_pretty(value).unwrap_or_default(); + println!("{text}"); +} + +/// The result of a command: its JSON with `-o json`, else what `people` +/// prints. +pub fn data(value: &Value, people: impl FnOnce()) { + match get().json { + true => json(value), + false => people(), + } +} + +/// That something was done: a sentence on standard output, which `-q` and +/// `-o json` leave out. +pub fn done(text: impl AsRef) { + let out = get(); + if !out.quiet && !out.json { + println!("{} {}", styled("✓", Style::Good), text.as_ref()); + } +} + +/// What to do next, on standard error. +pub fn hint(text: impl AsRef) { + let out = get(); + if !out.quiet && !out.json { + eprintln!("{}", styled_err(text.as_ref(), Style::Dim)); + } +} + +/// What goes on, on standard error. +pub fn note(text: impl AsRef) { + if !get().quiet { + eprintln!("{}", text.as_ref()); + } +} + +/// Something to look out for, on standard error even with `-q`. +pub fn warn(text: impl AsRef) { + eprintln!("{} {}", styled_err("!", Style::Warn), text.as_ref()); +} + +/// Why the command failed, on standard error: a sentence, or with `-o json` +/// an object with the API's error code. +pub fn fail(fail: &Fail) { + if fail.message.is_empty() { + return; + } + if get().json { + let body = serde_json::json!({"error": {"code": fail.code, "message": fail.message, "exit": fail.status}}); + eprintln!("{body}"); + return; + } + eprintln!("{} {}", styled_err("berth:", Style::Bad), fail.message); +} + +/// The width of text as it is shown, without its colours. +pub fn width(text: &str) -> usize { + strip_ansi(text).chars().count() +} + +/// Text without the escape sequences of colours and cursor movements. +pub fn strip_ansi(text: &str) -> String { + let mut plain = String::with_capacity(text.len()); + let mut chars = text.chars().peekable(); + while let Some(ch) = chars.next() { + if ch != '\x1b' { + plain.push(ch); + continue; + } + match chars.peek() { + Some('[') => { + chars.next(); + // Up to the final byte of the sequence. + for next in chars.by_ref() { + if ('@'..='~').contains(&next) { + break; + } + } + } + Some(']') => { + // An operating system command ends with BEL or ESC \. + while let Some(next) = chars.next() { + if next == '\x07' || (next == '\x1b' && chars.next_if_eq(&'\\').is_some()) { + break; + } + } + } + _ => { + chars.next(); + } + } + } + plain +} + +/// A table for people: a header and rows, in columns as wide as their +/// widest cell. +pub struct Table { + header: Vec, + rows: Vec>, +} + +impl Table { + pub fn new(header: &[&str]) -> Table { + Table { + header: header.iter().map(|cell| cell.to_uppercase()).collect(), + rows: vec![], + } + } + + pub fn row(&mut self, cells: Vec) { + self.rows.push(cells); + } + + /// The lines of the table, without a newline at the end of the last. + pub fn render(&self, colour: bool) -> Vec { + let columns = self.header.len(); + let mut widths = vec![0; columns]; + for row in std::iter::once(&self.header).chain(&self.rows) { + for (index, cell) in row.iter().enumerate().take(columns) { + widths[index] = widths[index].max(width(cell)); + } + } + let line = |row: &[String]| { + let mut text = String::new(); + for (index, cell) in row.iter().enumerate().take(columns) { + text.push_str(cell); + if index + 1 < columns { + let pad = widths[index] - width(cell) + 3; + text.push_str(&" ".repeat(pad)); + } + } + text.trim_end().to_owned() + }; + let mut lines = vec![paint(&line(&self.header), Style::Dim, colour)]; + lines.extend(self.rows.iter().map(|row| line(row))); + lines + } + + /// Print it, or `empty` when it has no rows. + pub fn print(&self, empty: &str) { + if self.rows.is_empty() { + hint(empty); + return; + } + let mut stdout = std::io::stdout().lock(); + for line in self.render(get().colour.0) { + let _ = writeln!(stdout, "{line}"); + } + } +} + +/// Name and value pairs, as `berth host show` prints them. +pub fn fields(pairs: &[(&str, String)]) { + let wide = pairs.iter().map(|(name, _)| name.len()).max().unwrap_or(0); + for (name, value) in pairs { + if value.is_empty() { + continue; + } + println!( + "{}{} {value}", + styled(name, Style::Dim), + " ".repeat(wide - name.len()) + ); + } +} + +/// Put the cursor at the start of an inline viewport that was cleared, so +/// that what is printed next takes its place. +pub fn to_line_of(terminal: &mut ratatui::Terminal) { + let top = terminal.get_frame().area().y; + let _ = terminal.set_cursor_position((0, top)); + let _ = terminal.show_cursor(); +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn a_table_lines_up_its_columns_whatever_their_colours() { + let mut table = Table::new(&["name", "address", "state"]); + table.row(vec![ + "studio".into(), + "192.0.2.10".into(), + "\x1b[32manswers\x1b[0m".into(), + ]); + table.row(vec!["dock".into(), "".into(), "no address".into()]); + let lines = table.render(false); + assert_eq!( + lines, + [ + "NAME ADDRESS STATE", + "studio 192.0.2.10 \x1b[32manswers\x1b[0m", + "dock no address", + ] + ); + } + + #[test] + fn colours_and_titles_are_taken_out() { + assert_eq!(strip_ansi("\x1b[1;32mok\x1b[0m done"), "ok done"); + assert_eq!(strip_ansi("\x1b]0;title\x07text"), "text"); + assert_eq!(width("\x1b[31m✗\x1b[0m failed"), 8); + } +} diff --git a/crates/berth/src/cli/pages.rs b/crates/berth/src/cli/pages.rs new file mode 100644 index 0000000..e4854cc --- /dev/null +++ b/crates/berth/src/cli/pages.rs @@ -0,0 +1,502 @@ +//! The pages `berth plugin show`, `berth provisioner show` and `berth resource +//! show` print, and the lines of what an action answered. + +use berth_api::{Action, PluginInfo, ProvisionerInfo, ProvisionerKind, ProvisionerState, Resource}; +use std::collections::BTreeMap; + +pub use berth_tui::provisioners::answer_lines; + +/// What `berth plugin show` prints: each plugin with what its features add. +pub fn plugins(plugins: &[PluginInfo]) -> String { + let mut text = String::new(); + for plugin in plugins { + text += &format!( + "{} {} {} {}\n", + plugin.id, plugin.package, plugin.version, plugin.dir + ); + for feature in &plugin.features { + text += &format!( + " {}/{} {} ({})\n", + plugin.id, feature.id, feature.label, feature.group + ); + for action in &feature.actions { + let mut notes = vec![format!("as {}", action.user)]; + if action.agent { + notes.push("the agent may run it".into()); + } + if !action.extends.is_empty() { + notes.push(format!("part of {}", action.extends.join(", "))); + } + for argument in &action.arguments { + notes.push(format!("asks {argument}")); + } + text += &format!( + " action {}:{} {} ({})\n", + plugin.id, + action.name, + action.label, + notes.join("; ") + ); + } + for service in &feature.services { + text += &format!( + " service {}/{} as {}: {}\n", + plugin.id, + service.name, + service.user, + service.methods.join(", ") + ); + } + for channel in &feature.channels { + text += &format!(" channel {}/{channel}\n", plugin.id); + } + } + for provisioner in &plugin.provisioners { + text += &format!( + " provisioner {} {} ({})\n", + provisioner.id, + provisioner.label, + provisioner_notes(provisioner).join("; ") + ); + } + } + text +} + +/// What a provisioner does and asks for, in a few words each. +pub fn provisioner_notes(provisioner: &ProvisionerInfo) -> Vec { + if !provisioner.problem.is_empty() && provisioner.kinds.is_empty() { + return vec![provisioner.problem.clone()]; + } + let makes = |kind: &ProvisionerKind| match (kind.id.as_str(), kind.setup.as_str()) { + ("machine", "seed") => "machines that set themselves up from the cloud seed".to_owned(), + ("machine", _) => "machines berth applies the profile to over SSH".to_owned(), + _ => kind.label.clone(), + }; + let kinds: Vec = provisioner.kinds.iter().map(makes).collect(); + let mut notes = vec![]; + if !kinds.is_empty() { + notes.push(format!("makes {}", kinds.join(" and "))); + } + if !provisioner.actions.is_empty() { + let names: Vec<&str> = provisioner + .actions + .iter() + .map(|action| action.name.as_str()) + .collect(); + notes.push(format!("actions {}", names.join(", "))); + } + if !provisioner.verbs.is_empty() { + notes.push(format!("also {}", provisioner.verbs.join(", "))); + } + for argument in &provisioner.arguments { + let kept = provisioner.settings.get(&argument.name); + let mut note = match (argument.required, argument.default.as_str(), kept) { + (_, _, Some(value)) => format!("asks {} (set to {value})", argument.name), + (true, _, None) => format!("asks {}", argument.name), + (false, "", None) => format!("may ask {}", argument.name), + (false, default, None) => format!("asks {} (default {default})", argument.name), + }; + if !argument.values.is_empty() { + note += &format!(": {}", argument.values.join(" or ")); + } + notes.push(note); + } + for secret in &provisioner.secrets { + let missing = provisioner.missing_secrets.contains(secret); + notes.push(match missing { + true => format!("secret {secret}, not set"), + false => format!("secret {secret}"), + }); + } + if !provisioner.problem.is_empty() { + notes.push(format!("not usable: {}", provisioner.problem)); + } + notes +} + +/// What `berth provisioner show NAME` says of it. +pub fn provisioner( + state: &ProvisionerState, + actions: &[Action], + given: &BTreeMap, +) -> String { + let info = &state.provisioner; + let mut text = format!("{} {}\n", info.id, info.label); + if !info.description.is_empty() { + text += &format!("{}\n", info.description); + } + // A reason that is long, or has several lines, goes below in full. + let pieces: Vec<&str> = state + .message + .lines() + .flat_map(|line| line.split(" | ")) + .map(str::trim) + .filter(|piece| !piece.is_empty()) + .collect(); + let long = pieces.len() > 1 || state.message.chars().count() > 60; + match (state.message.is_empty(), long) { + (true, _) => text += &format!("\nState {}\n", state.state), + (false, false) => text += &format!("\nState {} · {}\n", state.state, state.message), + (false, true) => { + text += &format!("\nState {}\n", state.state); + for piece in pieces { + text += &format!(" {piece}\n"); + } + } + } + let width = state + .details + .iter() + .map(|row| row.label.chars().count()) + .max() + .unwrap_or(0) + .max(8); + for row in &state.details { + text += &format!("{: = info + .own + .iter() + .map(|name| match (given.get(name), info.settings.get(name)) { + (Some(value), _) => format!("{name}={value} (given for this call, not saved)"), + (None, Some(value)) => format!("{name}={value}"), + (None, None) => format!("{name} not set"), + }) + .collect(); + text += &format!("Settings {}\n", set.join(" · ")); + if info.own.iter().any(|name| given.contains_key(name)) { + text += &format!( + " To keep them: berth provisioner set {} NAME=VALUE\n", + info.id + ); + } + } + let mut kinds: Vec<&str> = vec![]; + for held in &state.resources { + if !kinds.contains(&held.kind.as_str()) { + kinds.push(&held.kind); + } + } + if kinds.is_empty() { + text += "\nWhat berth made with it\n none\n"; + } + for kind in kinds { + let label = info + .kinds + .iter() + .find(|known| known.id == kind) + .map(|known| known.label.as_str()) + .filter(|label| !label.is_empty() && kind != "machine"); + text += &format!( + "\n{} berth made with it\n", + match (kind, label) { + ("machine", _) => "Machines", + (_, Some(label)) => label, + (kind, None) => kind, + } + ); + for held in state.resources.iter().filter(|held| held.kind == kind) { + let mut notes = vec![]; + if !held.state.is_empty() { + notes.push(held.state.clone()); + } + match held.resource.is_empty() { + true => notes.push("berth has no record of it".into()), + false => notes.push(held.resource.clone()), + } + let id = match held.id.as_str() { + "" => "(being made)", + id => id, + }; + let name = match held.host.is_empty() { + true => &held.name, + false => &held.host, + }; + text += &format!(" {name:<16} {id} {}\n", notes.join(" · ")); + } + } + let mut groups: Vec<(&str, Vec<&Action>)> = vec![]; + for action in actions { + let title = berth_tui::page::group_of(action); + match groups + .iter_mut() + .find(|(known, _)| known.eq_ignore_ascii_case(title)) + { + Some((_, list)) => list.push(action), + None => groups.push((title, vec![action])), + } + } + if !groups.is_empty() { + text += &format!("\nActions (berth provisioner run {} ACTION)\n", info.id); + } + for (title, list) in groups { + text += &format!(" {title}\n"); + for action in list { + let why = match action.available { + true => String::new(), + false => format!(" ({})", action.why), + }; + text += &format!(" {:<18} {}{why}\n", action.name, action.title()); + } + } + text +} + +/// What `berth resource show ID` says of it. +pub fn resource(found: &Resource, actions: &[Action]) -> String { + let mut text = format!( + "{} {} {} of {}\n", + found.id, found.kind, found.name, found.provisioner + ); + let mut row = |label: &str, value: &str| { + if !value.is_empty() { + text += &format!("{label:<11}{value}\n"); + } + }; + row("State", &found.state); + row("Problem", &found.problem); + row("Made as", &found.external_id); + row("Host", found.host.as_deref().unwrap_or("")); + row("Address", &found.address); + row("Created", &found.created); + row("Asked", &found.checked); + row("Spec", &found.spec_sha256); + let secrets: Vec = found + .secrets + .iter() + .map(|secret| format!("{} as {}", secret.name, secret.variable())) + .collect(); + row("Secrets", &secrets.join(", ")); + let arguments: Vec = found + .arguments + .iter() + .filter(|(_, value)| !value.is_empty()) + .map(|(key, value)| format!("{key}={value}")) + .collect(); + row("Arguments", &arguments.join(" ")); + if !found.endpoints.is_empty() { + text += &format!("\nEndpoints (berth port forward {} ENDPOINT)\n", found.id); + let width = found + .endpoints + .iter() + .map(|endpoint| endpoint.name.chars().count()) + .max() + .unwrap_or(0); + for endpoint in &found.endpoints { + let mut notes = vec![]; + if !endpoint.hint.is_empty() { + notes.push(endpoint.hint.clone()); + } + if endpoint.local { + notes.push("on the server's machine".to_owned()); + } + let notes = match notes.is_empty() { + true => String::new(), + false => format!(" {}", notes.join(" · ")), + }; + text += &format!( + " {: String::new(), + false => format!(" ({})", action.why), + }; + text += &format!(" {:<18} {}{why}\n", action.name, action.title()); + } + if found.verbs.iter().any(|verb| verb == "logs") { + text += &format!("\nIts log: berth resource logs {} [--follow]\n", found.id); + } + text +} + +#[cfg(test)] +mod tests { + use super::*; + use berth_api::{Detail, ListedResource, ResourceSecret}; + use serde_json::json; + + #[test] + fn a_provisioners_page_says_how_it_is_what_it_made_and_what_it_can_do() { + let state = ProvisionerState { + provisioner: ProvisionerInfo { + id: "incus/vm".into(), + label: "Incus virtual machine".into(), + own: vec!["server".into(), "project".into()], + settings: [("server".to_owned(), "https://192.0.2.10:8443".to_owned())].into(), + ..Default::default() + }, + state: "ready".into(), + message: "Incus 6.18 answers".into(), + details: vec![Detail { + label: "Kernel".into(), + value: "Linux 6.17".into(), + }], + resources: vec![ + ListedResource { + id: "berth-studio".into(), + kind: "machine".into(), + name: "studio".into(), + host: "studio".into(), + state: "running".into(), + resource: "r-studio".into(), + }, + ListedResource { + id: "berth-attic".into(), + kind: "machine".into(), + name: "attic".into(), + host: "attic".into(), + ..ListedResource::default() + }, + ], + ..Default::default() + }; + let actions = [ + Action { + name: "pools".into(), + label: "Storage pools".into(), + group: "Storage".into(), + ..Action::default() + }, + Action { + name: "settings".into(), + label: "Settings".into(), + group: "Set-Up".into(), + available: false, + why: "no".into(), + ..Action::default() + }, + ]; + let text = provisioner(&state, &actions, &BTreeMap::new()); + for line in [ + "State ready · Incus 6.18 answers\n", + "Kernel Linux 6.17\n", + "Settings server=https://192.0.2.10:8443 · project not set\n", + " studio berth-studio running · r-studio\n", + " attic berth-attic berth has no record of it\n", + "Actions (berth provisioner run incus/vm ACTION)\n", + " pools Storage pools\n", + " settings Settings (no)\n", + ] { + assert!(text.contains(line), "{line:?} in\n{text}"); + } + let given: BTreeMap = + [("server".to_owned(), "https://192.0.2.11:8443".to_owned())].into(); + let long = ProvisionerState { + state: "unreachable".into(), + message: "docker version failed: Cannot connect to the Docker daemon | error during connect: ssh: handshake failed".into(), + ..state.clone() + }; + let text = provisioner(&long, &[], &given); + for line in [ + "State unreachable\n", + " docker version failed: Cannot connect to the Docker daemon\n", + " error during connect: ssh: handshake failed\n", + "server=https://192.0.2.11:8443 (given for this call, not saved) · project not set\n", + "berth provisioner set incus/vm NAME=VALUE\n", + ] { + assert!(text.contains(line), "{line:?} in\n{text}"); + } + let lines = answer_lines(&json!({"message": "2 pools", "details": [ + {"label": "default", "value": "zfs"}, {"label": "fast-ssd", "value": "dir"}]})); + assert_eq!(lines, ["2 pools", " default zfs", " fast-ssd dir"]); + } + + #[test] + fn a_resource_is_described_with_what_can_be_done_to_it() { + let found = Resource { + id: "r-wiki".into(), + provisioner: "docker/container".into(), + kind: "deployment".into(), + name: "wiki".into(), + state: "running".into(), + external_id: "wiki".into(), + secrets: vec![ResourceSecret { + name: "wiki_db".into(), + env: "DB_PASSWORD".into(), + }], + spec_sha256: "ab12".into(), + verbs: vec!["logs".into()], + ..Resource::default() + }; + let actions = [Action { + name: "remove".into(), + label: "Remove the deployment".into(), + ..Action::default() + }]; + let text = resource(&found, &actions); + for line in [ + "r-wiki deployment wiki of docker/container\n", + "State running\n", + "Secrets wiki_db as DB_PASSWORD\n", + " remove Remove the deployment\n", + "Its log: berth resource logs r-wiki [--follow]\n", + ] { + assert!(text.contains(line), "{line:?} in\n{text}"); + } + } + + #[test] + fn a_plugin_is_shown_with_its_actions_and_its_provisioners() { + let plugin = PluginInfo { + id: "cloud".into(), + package: "berth-plugin-cloud".into(), + version: "1.0.0".into(), + dir: "/srv/plugins/cloud".into(), + features: vec![berth_api::PluginFeature { + id: "cli".into(), + label: "CLI".into(), + group: "Tools".into(), + actions: vec![berth_api::PluginAction { + name: "go".into(), + label: "Go".into(), + user: "agent".into(), + agent: true, + arguments: vec!["where".into()], + ..Default::default() + }], + channels: vec!["notes".into()], + ..Default::default() + }], + provisioners: vec![ProvisionerInfo { + id: "cloud/vm".into(), + label: "A VM".into(), + kinds: vec![ProvisionerKind { + id: "machine".into(), + setup: "seed".into(), + ..Default::default() + }], + verbs: vec!["stop".into()], + secrets: vec!["key".into()], + arguments: vec![berth_api::Argument { + name: "cpus".into(), + default: "4".into(), + ..Default::default() + }], + ..Default::default() + }], + ..Default::default() + }; + let text = plugins(&[plugin]); + for line in [ + "cloud berth-plugin-cloud 1.0.0 /srv/plugins/cloud\n", + " cloud/cli CLI (Tools)\n", + " action cloud:go Go (as agent; the agent may run it; asks where)\n", + " channel cloud/notes\n", + " provisioner cloud/vm A VM (makes machines that set themselves up from the cloud seed; also stop; asks cpus (default 4); secret key)\n", + ] { + assert!(text.contains(line), "{line:?} in\n{text}"); + } + } +} diff --git a/crates/berth/src/cli/setup.rs b/crates/berth/src/cli/setup.rs new file mode 100644 index 0000000..428a5cc --- /dev/null +++ b/crates/berth/src/cli/setup.rs @@ -0,0 +1,990 @@ +//! What the set-up tabs of the screens do, as commands: a new host through +//! the same flow the screens walk, and the keys, secrets, registries, +//! plugins and settings of the home screen. + +use super::args::{self, Global, Key, Plugin, Registry, Secret}; +use super::ask; +use super::client::{Client, Subject}; +use super::fail::{Fail, Result}; +use super::follow; +use super::host; +use super::out::{self, Style, Table, styled}; +use berth_api::Act; +use berth_tui::fields::{Action, Step}; +use berth_tui::flow::{self, Catalogue, Flow, Given, Hash, Maker, Stop}; +use berth_tui::setup::Feature; +use serde_json::{Value, json}; + +/// The keys of `--set`, for the help of `berth host create` and `berth host set`. +pub fn keys_help() -> String { + let mut text = String::from("How it is installed, with --method:\n"); + for action in Action::SERVED { + text.push_str(&format!(" {:<12} {}\n", action.key(), action.label())); + } + text.push_str( + "\nThe fields of a profile, with --set KEY=VALUE. A switch takes on or off. feature.PLUGIN/FEATURE=on|off, secret.NAME=VARIABLE|PATH and registry.NAME=on|off give a host those:\n", + ); + let wide = flow::keys() + .iter() + .map(|(key, _, _)| key.len()) + .max() + .unwrap_or(0); + for (key, label, help) in flow::keys() { + let first = help + .split(". ") + .next() + .unwrap_or(help) + .trim_end_matches('.'); + let _ = label; + text.push_str(&format!(" {key: Result { + let found = client.call("setup.catalogue", json!({}), None)?; + let features = found["features"] + .as_array() + .into_iter() + .flatten() + .map(|feature| { + let text = |key: &str| feature[key].as_str().unwrap_or("").to_owned(); + Feature { + key: text("key"), + label: text("label"), + group: text("group"), + help: text("help"), + types: berth_tui::setup::feature_types(feature), + } + }) + .collect(); + let secrets = client + .call("secrets.list", json!({}), None) + .map(|found| names(&found["names"])) + .unwrap_or_default(); + let own_key = client + .call("keys.list", json!({}), None) + .ok() + .and_then(|found| { + found["keys"] + .as_array()? + .iter() + .find(|key| key["name"] == "berth") + .map(|key| key["public"].as_str().unwrap_or("").to_owned()) + }) + .unwrap_or_default(); + Ok(Catalogue { + template: found["template"].clone(), + features, + secrets, + own_key, + provisioners: vec![], + }) +} + +fn names(list: &Value) -> Vec { + list.as_array() + .into_iter() + .flatten() + .filter_map(|name| name.as_str().map(str::to_owned)) + .collect() +} + +/// The server makes the hashes of the accounts' passwords, and keeps the +/// passwords it makes up. +struct Server<'a>(&'a Client, bool); + +impl Maker for Server<'_> { + fn hash(&mut self, host: &str, hash: Hash) -> std::result::Result { + let asked = match hash { + Hash::Generate { user } => { + self.1 = true; + json!({"what": "password", "host": host, "user": user}) + } + Hash::Of(password) => json!({"what": "password", "password": password}), + }; + let made = self + .0 + .call("setup.generate", json!({}), Some(asked)) + .map_err(|fail| fail.message)?; + Ok(made["value"].as_str().unwrap_or("").to_owned()) + } +} + +/// The profile `--from` names: a host's, a file, or standard input. +fn starting_profile(client: &Client, from: &str) -> Result { + if from == "-" || std::path::Path::new(from).is_file() { + let bytes = host::read_file(from)?; + return serde_json::from_slice(&bytes) + .map_err(|error| Fail::usage(format!("{from} holds no JSON profile: {error}"))); + } + let mut profile = host::profile_of(client, from) + .map_err(|fail| fail.context(&format!("--from {from} is no file and no host")))?; + // What belongs to the other host itself stays with it. + if let Some(profile) = profile.as_object_mut() { + for key in ["hostname", "operator_password_hash", "agent_password_hash"] { + profile.remove(key); + } + } + if let Some(access) = profile.get_mut("access").and_then(Value::as_object_mut) { + access.remove("address"); + } + Ok(profile) +} + +/// `berth host create`: the flow of the screens' set-up tabs, asked inline +/// or taken from the flags, then saved, then run as its way says. +pub fn create(client: &Client, create: &args::HostCreate) -> Result<()> { + let name = &create.name; + if client + .call("hosts.get", json!({"name": name}), None) + .is_ok() + { + return match &create.provisioner { + Some(_) => make_saved(client, create), + None => Err(Fail::failed(format!( + "There is a host {name} already. Change it with berth host set {name} KEY=VALUE, or berth ui {name}" + ))), + }; + } + let mut catalogue = catalogue(client)?; + if let Some(from) = &create.from { + catalogue.template = starting_profile(client, from)?; + } + let method = match &create.method { + Some(key) => Some(Action::by_key(key).ok_or_else(|| { + let known: Vec<&str> = Action::SERVED.iter().map(|action| action.key()).collect(); + Fail::usage(format!("--method is {}, not {key}", known.join(", "))) + })?), + None => None, + }; + if method.is_none_or(|method| method == Action::Machine) { + catalogue.provisioners = provisioners(client)?; + } + let mut given = Given { + method, + generate_passwords: create.generate_passwords, + provisioner: create.provisioner.clone(), + ..Given::default() + }; + if let Some(host_type) = &create.host_type { + given + .set(&[format!("type={host_type}")]) + .map_err(Fail::usage)?; + } + given.set(&create.set).map_err(Fail::usage)?; + given + .features + .extend(create.feature.iter().map(|key| (key.clone(), true))); + given + .features + .extend(create.no_feature.iter().map(|key| (key.clone(), false))); + for word in &create.arguments { + let (key, value) = word.split_once('=').ok_or_else(|| { + Fail::usage(format!( + "The provisioner's arguments are NAME=VALUE, not {word}" + )) + })?; + given.arguments.push((key.to_owned(), value.to_owned())); + } + if !given.arguments.is_empty() && given.provisioner.is_none() && method != Some(Action::Machine) + { + return Err(Fail::usage( + "NAME=VALUE arguments go to a provisioner: add --provisioner NAME", + )); + } + let mut flow = Flow::new(name, catalogue); + let mut maker = Server(client, false); + flow.run(&given, &mut ask::asker(), &mut maker) + .map_err(stopped)?; + host::save(client, name, &flow.config)?; + out::done(format!("Saved {name}")); + if maker.1 { + out::hint(format!("Its passwords: berth host credentials {name}")); + } + run_way(client, &flow, create) +} + +/// The provisioners that can make a host, as the Install tab lists them. +fn provisioners(client: &Client) -> Result> { + let listed = client.call("provisioners.list", json!({}), None)?; + Ok(serde_json::from_value(listed)?) +} + +/// Make the machine of a host that is saved and has none, with its profile +/// as it is, as the Run tab of its set-up does. +fn make_saved(client: &Client, create: &args::HostCreate) -> Result<()> { + let name = &create.name; + if let Some(machine) = host::machine_of(client, name)? { + return Err(Fail::failed(format!( + "{name} has a machine already, {} of {}. berth host show {name} says how it is", + machine["external_id"].as_str().unwrap_or(""), + machine["provisioner"].as_str().unwrap_or("") + ))); + } + let catalogue = Catalogue { + provisioners: provisioners(client)?, + ..Catalogue::default() + }; + let mut flow = Flow::new(name, catalogue); + flow.config = host::profile_of(client, name)?; + let mut given = Given { + provisioner: create.provisioner.clone(), + ..Given::default() + }; + for word in &create.arguments { + let (key, value) = word.split_once('=').ok_or_else(|| { + Fail::usage(format!( + "The provisioner's arguments are NAME=VALUE, not {word}" + )) + })?; + given.arguments.push((key.to_owned(), value.to_owned())); + } + flow.machine_questions(&given, &mut ask::asker()) + .map_err(stopped)?; + run_way(client, &flow, create) +} + +/// Why the flow stopped, as the command line says it: with the flags that +/// give what is missing. +fn stopped(stop: Stop) -> Fail { + match stop { + Stop::Missing(question) => Fail::usage(format!( + "{} is missing: pass {}", + question.label, question.flag + )), + Stop::Problems(problems) => { + // Only a secret the server lacks stops it: the command line was right. + let refused = problems + .iter() + .all(|problem| problem.field == Some(berth_tui::fields::Field::Tailscale)); + let said: Vec = problems + .iter() + .map(|problem| match problem.field { + Some(field) if field.is_password_hash() => { + format!( + "{} (--generate-passwords, or {})", + problem.text, + flow::flag_of(field) + ) + } + Some(field) => format!("{} ({})", problem.text, flow::flag_of(field)), + None => problem.text.clone(), + }) + .collect(); + let said = format!("Not saved: {}", said.join("; ")); + match refused { + true => Fail::refused(said), + false => Fail::usage(said), + } + } + Stop::Refused(why) if why == "Stopped" => Fail::exit(130), + Stop::Refused(why) => Fail::usage(why), + } +} + +/// The Run tab: create the host with its provisioner, or install it the +/// way chosen, when that was asked for. +fn run_way(client: &Client, flow: &Flow, create: &args::HostCreate) -> Result<()> { + let name = &flow.name; + let host_type = berth_api::profile::host_type(&flow.config); + if flow.action == Action::Machine { + let request = flow.request().map_err(Fail::usage)?; + let made = client.call( + "resources.create", + json!({}), + Some(serde_json::to_value(&request)?), + )?; + let made: berth_api::Created = serde_json::from_value(made)?; + let acted = berth_api::Acted { + job: Some(made.job), + ..berth_api::Acted::default() + }; + let title = format!("Creating {name} with {}", request.provisioner); + follow::acted(client, acted, &title, create.detach.detach, false)?; + return Ok(()); + } + let (question, then) = match (flow.action, host_type) { + (Action::Iso, _) => ( + "Build the USB installer now?", + format!("berth host iso {name} --write DEVICE"), + ), + (Action::CloudSeed, _) => { + return { + out::hint(format!( + "Its cloud seed: berth host seed {name} > user-data" + )); + Ok(()) + }; + } + (Action::Reach, _) | (Action::Ssh, berth_api::HostType::Ssh) => { + out::hint(format!("Log in to it once: berth key trust {name}")); + return Ok(()); + } + (_, berth_api::HostType::Managed) => ( + "Install the controller on it now?", + format!("berth host apply {name}"), + ), + _ => ("Apply it now?", format!("berth host apply {name}")), + }; + let run = create.run + || (out::get().ask + && matches!( + ask::ask(&berth_tui::flow::Question { + step: Step::Run, + label: question.into(), + help: String::new(), + flag: "--run".into(), + asked: berth_tui::flow::Asked::Confirm { default: false }, + required: false, + })?, + berth_tui::flow::Answer::Confirm(true) + )); + if !run { + out::hint(format!("Install it with: {then}")); + return Ok(()); + } + match flow.action { + Action::Iso => { + let acted = client.act(&Subject::Host(name.clone()), "installer", &Act::default())?; + follow::acted( + client, + acted, + &format!("Building the installer of {name}"), + create.detach.detach, + false, + )?; + Ok(()) + } + _ => host::apply( + client, + &args::HostApply { + name: name.clone(), + detach: create.detach.clone(), + }, + ), + } +} + +// ── keys ──────────────────────────────────────────────────────────────────── + +pub fn key(client: &Client, command: Key) -> Result<()> { + match command { + Key::List => { + let found = client.call("keys.list", json!({}), None)?; + out::data(&found, || { + let mut table = Table::new(&["name", "fingerprint", "hosts"]); + for key in found["keys"].as_array().into_iter().flatten() { + let mut name = key["name"].as_str().unwrap_or("").to_owned(); + if key["pending"].is_object() { + name.push_str(&styled(" (rotating)", Style::Warn)); + } + table.row(vec![ + name, + key["fingerprint"].as_str().unwrap_or("").to_owned(), + names(&key["hosts"]).join(", "), + ]); + } + table.print("No keys"); + }); + Ok(()) + } + Key::Show { name } => { + let found = client.call("keys.list", json!({}), None)?; + let key = found["keys"] + .as_array() + .into_iter() + .flatten() + .find(|key| key["name"] == name.as_str()) + .cloned() + .ok_or_else(|| Fail::failed(format!("No key {name}. berth key list names them")))?; + out::data(&key, || { + println!("{}", key["public"].as_str().unwrap_or("")) + }); + Ok(()) + } + Key::Create { name } => { + let made = client.call("keys.add", json!({"name": name}), None)?; + out::data(&made, || { + out::done(format!("Made the key {name}")); + println!("{}", made["public"].as_str().unwrap_or("")); + }); + Ok(()) + } + Key::Import { name, file } => { + let private = host::read_file(&file)?; + let data = json!({"base64": berth_api::b64(&private)}); + let made = client.call("keys.add", json!({"name": name}), Some(data))?; + out::data(&made, || { + out::done(format!( + "Added the key {name}: {}", + made["fingerprint"].as_str().unwrap_or("") + )) + }); + Ok(()) + } + Key::Remove { name } => { + ask::confirm(&format!( + "This removes the key {name} from berth's key ring." + ))?; + client.call("keys.remove", json!({"name": name}), None)?; + out::done(format!("Removed the key {name}")); + Ok(()) + } + Key::Rotate { name, detach } => { + ask::confirm(&format!( + "This replaces the key {name} on every host that takes it." + ))?; + let acted: berth_api::Acted = + serde_json::from_value(client.call("keys.rotate", json!({"name": name}), None)?)?; + follow::acted( + client, + acted, + &format!("Rotating the key {name}"), + detach.detach, + false, + )?; + Ok(()) + } + Key::Trust { + host, + key, + password_stdin, + } => { + let password = match password_stdin { + true => Some(ask::secret_from( + "Password", + None, + true, + "--password-stdin", + )?), + false => None, + }; + host::trust(client, &host, key.as_deref(), password) + } + } +} + +// ── secrets ───────────────────────────────────────────────────────────────── + +pub fn secret(client: &Client, command: Secret) -> Result<()> { + match command { + Secret::List => { + let found = client.call("secrets.list", json!({}), None)?; + out::data(&found, || { + let all = names(&found["names"]); + if all.is_empty() { + out::hint("No secrets. Set one with berth secret set NAME"); + } + all.iter().for_each(|name| println!("{name}")); + }); + Ok(()) + } + Secret::Set { name, value_file } => { + let value = ask::secret_from( + &format!("Value of {name}"), + value_file.as_deref(), + false, + "--value-file FILE", + )?; + client.call( + "secrets.set", + json!({"name": name}), + Some(json!({"value": value})), + )?; + out::done(format!("Set the secret {name}")); + Ok(()) + } + Secret::Remove { name } => { + ask::confirm(&format!("This removes the secret {name}."))?; + client.call("secrets.remove", json!({"name": name}), None)?; + out::done(format!("Removed the secret {name}")); + Ok(()) + } + } +} + +// ── registries ────────────────────────────────────────────────────────────── + +pub fn registry(global: &Global, command: Registry) -> Result<()> { + #[cfg(target_os = "linux")] + match command { + Registry::Publish { + destination, + host, + login, + } => return crate::local::publish(global, &destination, host.as_deref(), login.as_deref()), + Registry::PushImage { destination, tag } => { + return crate::local::push_image(global, &destination, &tag); + } + _ => {} + } + let client = Client::open(global)?; + let registries = || -> Result> { + let found = client.call("registries.list", json!({}), None)?; + Ok(found["registries"].as_object().cloned().unwrap_or_default()) + }; + match command { + Registry::List => { + let found = client.call("registries.list", json!({}), None)?; + out::data(&found, || { + let mut table = Table::new(&["name", "type", "url", "user", "repositories"]); + for (name, registry) in found["registries"].as_object().into_iter().flatten() { + let repositories: Vec = registry["repositories"] + .as_array() + .into_iter() + .flatten() + .map(|repo| { + format!( + "{} {}", + repo["format"].as_str().unwrap_or(""), + repo["name"].as_str().unwrap_or("") + ) + }) + .collect(); + table.row(vec![ + name.clone(), + registry["type"].as_str().unwrap_or("").to_owned(), + registry["url"].as_str().unwrap_or("").to_owned(), + registry["username"].as_str().unwrap_or("").to_owned(), + repositories.join(", "), + ]); + } + table.print("No registries. Add one with berth registry add NAME TYPE URL"); + }); + Ok(()) + } + Registry::Add { + name, + kind, + url, + user, + } => { + let mut registry = json!({"type": kind, "url": url.trim_end_matches('/')}); + if let Some(user) = user { + registry["username"] = json!(user); + } + // Adding again changes the address or user, and keeps the repositories. + if let Some(known) = registries()?.get(&name) { + registry["repositories"] = known["repositories"].clone(); + } + client.call("registries.save", json!({"name": name}), Some(registry))?; + out::done(format!("Saved the registry {name}")); + out::hint(format!("Keep its login with: berth registry login {name}")); + Ok(()) + } + Registry::AddRepo { + name, + format, + repository, + address, + } => { + let mut registry = registries()? + .get(&name) + .filter(|registry| registry["type"] == "nexus") + .cloned() + .ok_or_else(|| Fail::usage(format!("{name} is no Nexus registry")))?; + let mut entry = json!({"format": format, "name": repository}); + if let Some(address) = address { + entry["address"] = json!(address); + } + let mut repositories = registry["repositories"] + .as_array() + .cloned() + .unwrap_or_default(); + repositories.retain(|known| known["name"] != repository.as_str()); + repositories.push(entry); + registry["repositories"] = json!(repositories); + client.call("registries.save", json!({"name": name}), Some(registry))?; + out::done(format!("Added {repository} to {name}")); + Ok(()) + } + Registry::Login { + name, + password_stdin, + } => { + if !registries()?.contains_key(&name) { + return Err(Fail::failed(format!( + "No registry {name}. berth registry list names them" + ))); + } + let value = ask::secret_from( + &format!("Password or token of {name}"), + None, + password_stdin, + "--password-stdin", + )?; + client.call( + "secrets.set", + json!({"name": format!("registry/{name}")}), + Some(json!({"value": value})), + )?; + out::done(format!("Kept the login of {name}")); + Ok(()) + } + Registry::Check { name } => { + let names: Vec = match name { + Some(name) => vec![name], + None => registries()?.keys().cloned().collect(), + }; + let mut all = serde_json::Map::new(); + let mut failed = false; + for name in names { + let checked = client.call("registries.check", json!({"name": name}), None)?; + failed |= checked["lines"] + .as_array() + .into_iter() + .flatten() + .any(|line| line.as_str().is_some_and(|line| line.starts_with('✗'))); + all.insert(name, checked); + } + out::data(&Value::Object(all.clone()), || { + for (name, checked) in &all { + println!("{}", styled(name, Style::Bold)); + for line in names_of(&checked["lines"]) { + println!(" {line}"); + } + } + }); + match failed { + true => Err(Fail::exit(1)), + false => Ok(()), + } + } + Registry::Remove { name } => { + ask::confirm(&format!("This removes the registry {name} with its login."))?; + client.call("registries.remove", json!({"name": name}), None)?; + out::done(format!("Removed the registry {name}")); + Ok(()) + } + #[cfg(target_os = "linux")] + Registry::Publish { .. } | Registry::PushImage { .. } => unreachable!("handled above"), + } +} + +fn names_of(list: &Value) -> Vec { + names(list) +} + +// ── plugins ───────────────────────────────────────────────────────────────── + +fn report(found: &Value) { + out::data(found, || { + for line in names(&found["lines"]) { + println!("{line}"); + } + let behind: Vec = + serde_json::from_value(found["behind"].clone()).unwrap_or_default(); + for host in &behind { + out::warn(format!( + "{} needs applying: {}", + host.name, + host.behind.text() + )); + } + if found["behind"] + .as_array() + .is_some_and(|behind| !behind.is_empty()) + { + out::hint("berth rollout brings them up to date"); + } + }); +} + +pub fn plugin(client: &Client, command: Plugin) -> Result<()> { + match command { + Plugin::List => { + let found = client.call("plugins.list", json!({}), None)?; + out::data(&found, || { + let mut table = Table::new(&["id", "version", "from", "features"]); + for plugin in found["plugins"].as_array().into_iter().flatten() { + let features: Vec = plugin["features"] + .as_array() + .into_iter() + .flatten() + .filter_map(|feature| feature["id"].as_str().map(str::to_owned)) + .collect(); + let origin = match &plugin["origin"] { + Value::String(origin) => origin.clone(), + other => other + .as_object() + .and_then(|origin| origin.keys().next().cloned()) + .unwrap_or_default(), + }; + table.row(vec![ + plugin["id"].as_str().unwrap_or("").to_owned(), + plugin["version"].as_str().unwrap_or("").to_owned(), + origin, + features.join(", "), + ]); + } + table.print("No plugins"); + }); + Ok(()) + } + Plugin::Show { id } => { + let found = client.call("plugins.list", json!({}), None)?; + let plugins: Vec = + serde_json::from_value(found["plugins"].clone())?; + let Some(plugin) = plugins.into_iter().find(|plugin| plugin.id == id) else { + return Err(Fail::failed(format!( + "There is no plugin {id}. The plugins are listed by berth plugin list" + ))); + }; + out::data(&serde_json::to_value(&plugin)?, || { + print!("{}", super::pages::plugins(std::slice::from_ref(&plugin))); + }); + Ok(()) + } + Plugin::Add { spec } => { + let local = std::path::Path::new(spec.strip_prefix("file:").unwrap_or(&spec)); + if !local.exists() { + report(&client.call("plugins.add", json!({}), Some(json!({"spec": spec})))?); + return Ok(()); + } + // A file or folder of this machine is sent as the .tgz npm packs. + let package = match local.is_dir() { + true => pack(local)?, + false => std::fs::read(local)?, + }; + if package.len() > berth_api::ops::PORTAL_BYTES { + return Err(Fail::usage(format!( + "{spec} is larger than the 8 MiB a server is sent" + ))); + } + let data = json!({"base64": berth_api::b64(&package)}); + report(&client.call("plugins.upload", json!({}), Some(data))?); + Ok(()) + } + Plugin::Remove { id } => { + // A plugin is named by its id or by its npm name. + let found = client.call("plugins.list", json!({}), None)?; + let id = found["plugins"] + .as_array() + .into_iter() + .flatten() + .find(|plugin| plugin["package"] == id.as_str()) + .and_then(|plugin| plugin["id"].as_str()) + .map(str::to_owned) + .unwrap_or(id); + ask::confirm(&format!("This removes the plugin {id}."))?; + report(&client.call("plugins.remove", json!({"id": id}), None)?); + Ok(()) + } + Plugin::Update { id, check: true } => { + let found = client.call("plugins.check", json!({}), None)?; + out::data(&found, || { + let mut table = Table::new(&["id", "version", "latest"]); + for plugin in found["plugins"].as_array().into_iter().flatten() { + if id.as_ref().is_some_and(|id| plugin["id"] != id.as_str()) { + continue; + } + let latest = match ( + plugin["latest"].as_str(), + plugin["note"].as_str().unwrap_or(""), + ) { + (Some(latest), _) => styled(latest, Style::Warn), + (None, "") => "the latest".to_owned(), + (None, note) => note.to_owned(), + }; + table.row(vec![ + plugin["id"].as_str().unwrap_or("").to_owned(), + plugin["version"].as_str().unwrap_or("").to_owned(), + latest, + ]); + } + table.print("None of the server's own plugins came from a registry"); + }); + Ok(()) + } + Plugin::Update { id, check: false } => { + let body = match id { + Some(id) => json!({"id": id}), + None => json!({}), + }; + report(&client.call("plugins.update", json!({}), Some(body))?); + Ok(()) + } + } +} + +/// A folder as the .tgz `npm pack` makes of it, without running its scripts. +fn pack(folder: &std::path::Path) -> Result> { + let into = tempfile::tempdir()?; + let output = std::process::Command::new("npm") + .args(["pack", "--ignore-scripts", "--silent", "--pack-destination"]) + .arg(into.path()) + .arg("--") + .arg(folder) + .stdin(std::process::Stdio::null()) + .output() + .map_err(|error| Fail::failed(format!("npm could not be started: {error}")))?; + if !output.status.success() { + return Err(Fail::failed(format!( + "npm pack {}: {}", + folder.display(), + String::from_utf8_lossy(&output.stderr).trim() + ))); + } + let packed = std::fs::read_dir(into.path())? + .flatten() + .map(|entry| entry.path()) + .find(|path| path.extension().is_some_and(|end| end == "tgz")) + .ok_or_else(|| Fail::failed("npm pack made no .tgz"))?; + Ok(std::fs::read(packed)?) +} + +// ── settings ──────────────────────────────────────────────────────────────── + +/// The settings of berth itself, on this machine, beside those of the +/// server: where the server is, and its token. +const CLIENT_SETTINGS: &[(&str, &str)] = &[ + ( + berth_page::settings::SERVER, + "the berth server this berth uses, kept by berth login; empty for the one in berth itself", + ), + ( + berth_page::settings::TOKEN, + "the token of that server, kept by berth login and never shown", + ), +]; + +/// The client's own settings, as the server's are listed. +fn client_settings() -> Vec { + let kept = super::client::settings(); + CLIENT_SETTINGS + .iter() + .map(|(key, meaning)| { + let value = match (kept.get(key), *key == berth_page::settings::TOKEN) { + (Some(_), true) => "(set)".to_owned(), + (Some(value), false) => value, + (None, _) => String::new(), + }; + json!({"key": key, "value": value, "meaning": meaning, "where": "client"}) + }) + .collect() +} + +fn is_client_setting(key: &str) -> bool { + CLIENT_SETTINGS.iter().any(|(known, _)| *known == key) +} + +/// Change one of the client's own settings. +fn set_client(key: &str, value: &str) -> Result<()> { + if key == berth_page::settings::TOKEN && !value.is_empty() { + return Err(Fail::usage( + "The token is not given on the command line, where others can read it. berth login URL keeps one", + )); + } + let mut changes = vec![(key, value)]; + // Another server takes another token. + if key == berth_page::settings::SERVER { + changes.push((berth_page::settings::TOKEN, "")); + } + super::client::settings() + .set(&changes) + .map_err(|error| Fail::failed(error.to_string())) +} + +pub fn config(client: &Client, command: args::Config) -> Result<()> { + let settings = || -> Result> { + let found = client.call("settings.list", json!({}), None)?; + // A server's own server and token are not this berth's: those are the client's. + let mut all: Vec = found + .as_array() + .cloned() + .unwrap_or_default() + .into_iter() + .filter(|setting| !is_client_setting(setting["key"].as_str().unwrap_or(""))) + .map(|mut setting| { + setting["where"] = json!("server"); + setting + }) + .collect(); + all.extend(client_settings()); + Ok(all) + }; + let known = |key: &str| -> Result { + settings()? + .into_iter() + .find(|setting| setting["key"] == key) + .ok_or_else(|| { + Fail::usage(format!("{key} is no setting. berth config list names them")) + }) + }; + let set = |key: &str, value: &str| -> Result<()> { + if is_client_setting(key) { + return set_client(key, value); + } + known(key)?; + client.call( + "settings.set", + json!({"key": key}), + Some(json!({"value": value})), + )?; + Ok(()) + }; + match command { + args::Config::List => { + let all = settings()?; + out::data(&Value::Array(all.clone()), || { + let width = all + .iter() + .map(|setting| setting["key"].as_str().unwrap_or("").chars().count()) + .max() + .unwrap_or(0); + let mut place = ""; + for setting in &all { + let here = setting["where"].as_str().unwrap_or(""); + if here != place { + place = here; + let heading = match here { + "client" => "This berth, on this machine", + _ => "The server", + }; + println!("{}", styled(heading, Style::Accent)); + } + let value = setting["value"].as_str().unwrap_or(""); + let shown = match value { + "" => styled("(not set)", Style::Dim), + value => value.to_owned(), + }; + let key = setting["key"].as_str().unwrap_or(""); + println!(" {key: { + if key == berth_page::settings::TOKEN { + return Err(Fail::usage( + "The token is never shown. berth config list says whether there is one", + )); + } + let setting = known(&key)?; + out::data(&setting, || { + println!("{}", setting["value"].as_str().unwrap_or("")) + }); + Ok(()) + } + args::Config::Set { key, value } => { + set(&key, &value)?; + out::done(format!("{key} = {value}")); + if key == berth_page::settings::SERVER { + out::hint(format!("Claim its token with: berth login {value}")); + } + Ok(()) + } + args::Config::Unset { key } => { + set(&key, "")?; + out::done(format!("{key} is back to its default")); + Ok(()) + } + } +} diff --git a/crates/berth/src/cli/things.rs b/crates/berth/src/cli/things.rs new file mode 100644 index 0000000..6170a6f --- /dev/null +++ b/crates/berth/src/cli/things.rs @@ -0,0 +1,978 @@ +//! What a host does and runs, and what provisioners make: sessions, exec, +//! actions, provisioners and resources, each through the server's portal, +//! with a host's own operations sent to it by name. + +use super::args::{self, Provisioner, Resource, Session}; +use super::ask; +use super::client::{Client, Subject}; +use super::fail::{Fail, Result}; +use super::follow; +use super::host; +use super::out::{self, Style, Table, styled}; +use super::pages; +use berth_api::portal::{Piece, PortalStream}; +use berth_api::{Act, SessionKind}; +use serde_json::{Value, json}; +use std::collections::BTreeMap; +use std::io::{IsTerminal, Write}; +use tokio::io::AsyncReadExt; + +/// Ctrl+Q, which leaves a terminal session without ending it. +const DETACH: u8 = 0x11; + +/// The items of a list the API answers with, bare or under `key`. +fn items(value: Value, key: &str) -> Vec { + match value { + Value::Array(items) => items, + mut value => value[key].take().as_array().cloned().unwrap_or_default(), + } +} + +/// `NAME=VALUE` words as a map. +pub fn arguments(words: &[String]) -> Result> { + words + .iter() + .map(|word| match word.split_once('=') { + Some((name, value)) if !name.is_empty() && !name.starts_with('-') => { + Ok((name.to_owned(), value.to_owned())) + } + _ => Err(Fail::usage(format!( + "Arguments are given as NAME=VALUE, not {word}" + ))), + }) + .collect() +} + +/// How following a session ended. +enum Ended { + Exited(Option), + /// Left with Ctrl+Q, or Ctrl+C when nothing is typed into it; it runs on. + Detached, + Lost, +} + +fn exited(said: &Value) -> Option> { + berth_tui::host::exited(said) +} + +/// Follow a process session: its two streams to ours, apart. With `stdin`, +/// ours is its input and Ctrl+C interrupts it; without, Ctrl+C only leaves. +async fn follow_process(mut stream: PortalStream, stdin: bool) -> Result { + let (sender, mut typed) = tokio::sync::mpsc::unbounded_channel::(); + if stdin && !std::io::stdin().is_terminal() { + tokio::spawn(async move { + let mut input = tokio::io::stdin(); + let mut buffer = vec![0; 16384]; + while let Ok(count) = input.read(&mut buffer).await + && count > 0 + { + let frame = [&[0u8][..], &buffer[..count]].concat(); + if sender.send(Piece::Binary(frame)).is_err() { + return; + } + } + let _ = sender.send(Piece::Text(json!({"eof": true}))); + }); + } else if stdin { + // Nothing will be typed into it. + let _ = sender.send(Piece::Text(json!({"eof": true}))); + } + let (mut out, mut err) = (std::io::stdout(), std::io::stderr()); + loop { + tokio::select! { + piece = stream.recv() => match piece { + Some(Ok(Piece::Binary(bytes))) => match bytes.split_first() { + Some((2, rest)) => { + err.write_all(rest)?; + err.flush()?; + } + Some((_, rest)) => { + out.write_all(rest)?; + out.flush()?; + } + None => {} + }, + Some(Ok(Piece::Text(said))) => { + if let Some(code) = exited(&said) { + return Ok(Ended::Exited(code)); + } + } + _ => return Ok(Ended::Lost), + }, + Some(piece) = typed.recv() => { + if stream.push(piece).is_err() { + return Ok(Ended::Lost); + } + } + _ = tokio::signal::ctrl_c() => { + if !stdin { + return Ok(Ended::Detached); + } + let _ = stream.push(Piece::Text(json!({"signal": "INT"}))); + } + } + } +} + +/// Puts the terminal back as it was when dropped. +struct Raw(bool); + +impl Raw { + fn enter() -> Raw { + let terminal = std::io::stdin().is_terminal() && std::io::stdout().is_terminal(); + Raw(terminal && crossterm::terminal::enable_raw_mode().is_ok()) + } +} + +impl Drop for Raw { + fn drop(&mut self) { + if self.0 { + let _ = crossterm::terminal::disable_raw_mode(); + } + } +} + +fn terminal_size() -> (u16, u16) { + // A terminal that does not know its size says 0 by 0. + match crossterm::terminal::size() { + Ok((cols, rows)) if cols > 0 && rows > 0 => (rows, cols), + _ => (24, 80), + } +} + +/// This terminal onto a terminal session. Ctrl+Q leaves it running. +async fn follow_terminal(mut stream: PortalStream) -> Result { + let raw = Raw::enter(); + let resize = + |(rows, cols): (u16, u16)| Piece::Text(json!({"resize": {"rows": rows, "cols": cols}})); + let mut size = terminal_size(); + if raw.0 { + let _ = stream.push(resize(size)); + } + let (sender, mut typed) = tokio::sync::mpsc::unbounded_channel::>(); + tokio::spawn(async move { + let mut input = tokio::io::stdin(); + let mut buffer = vec![0; 4096]; + while let Ok(count) = input.read(&mut buffer).await + && count > 0 + && sender.send(buffer[..count].to_vec()).is_ok() + {} + }); + let mut tick = tokio::time::interval(std::time::Duration::from_millis(250)); + let mut out = std::io::stdout(); + loop { + tokio::select! { + piece = stream.recv() => match piece { + Some(Ok(Piece::Binary(bytes))) => { + out.write_all(&bytes)?; + out.flush()?; + } + Some(Ok(Piece::Text(said))) => { + if let Some(code) = exited(&said) { + return Ok(Ended::Exited(code)); + } + } + _ => return Ok(Ended::Lost), + }, + Some(bytes) = typed.recv() => { + if raw.0 && let Some(at) = bytes.iter().position(|byte| *byte == DETACH) { + if at > 0 { + let _ = stream.push(Piece::Binary(bytes[..at].to_vec())); + } + return Ok(Ended::Detached); + } + if stream.push(Piece::Binary(bytes)).is_err() { + return Ok(Ended::Lost); + } + } + _ = tick.tick() => { + let now = terminal_size(); + if raw.0 && now != size { + size = now; + let _ = stream.push(resize(size)); + } + } + } + } +} + +/// What to say and which status to leave with once following has ended. A +/// session that was only for this command is forgotten when it has ended. +fn finish(client: &Client, host: &str, id: &str, ended: Ended, forget: bool) -> Result<()> { + match ended { + Ended::Exited(code) => { + if forget { + let _ = client.on(host, "sessions.delete", json!({"id": id}), None); + } + match code.unwrap_or(1) { + 0 => Ok(()), + code => Err(Fail::exit(code)), + } + } + Ended::Detached => { + eprintln!( + "\r\nLeft {id} running on {host}. Back to it: berth session attach {host} {id}, or berth session logs {host} {id} -f" + ); + Ok(()) + } + Ended::Lost => Err(Fail::failed(format!( + "The connection was lost; {id} may still run on {host}: berth session logs {host} {id} -f" + ))), + } +} + +fn attach_stream(client: &Client, host: &str, id: &str, since: u64) -> Result { + client.stream( + "sessions.attach", + Some(host), + json!({"id": id, "since": since}), + ) +} + +/// `berth exec HOST -- COMMAND…`: a process session, followed to its end, +/// with its exit status passed through. +pub fn exec(client: &Client, exec: &args::Exec) -> Result<()> { + if !["agent", "operator", "root"].contains(&exec.user.as_str()) { + return Err(Fail::usage("--user is agent, operator or root")); + } + let title: String = exec.command.join(" ").chars().take(200).collect(); + let body = + json!({"kind": "process", "command": exec.command, "user": exec.user, "title": title}); + let session: berth_api::Session = + serde_json::from_value(client.on(&exec.host, "sessions.start", json!({}), Some(body))?) + .map_err(Fail::from)?; + if exec.detach { + out::data(&serde_json::to_value(&session)?, || { + println!("{}", session.id) + }); + return Ok(()); + } + let stream = attach_stream(client, &exec.host, &session.id, 0)?; + let ended = client.wait(follow_process(stream, true))?; + finish(client, &exec.host, &session.id, ended, !exec.keep) +} + +fn attach(client: &Client, host: &str, id: Option<&str>) -> Result<()> { + let (rows, cols) = terminal_size(); + let session: berth_api::Session = match id { + Some(id) => { + serde_json::from_value(client.on(host, "sessions.get", json!({"id": id}), None)?)? + } + None => { + let body = json!({ + "kind": "terminal", "command": ["bash", "-l"], "user": "agent", + "title": format!("a shell on {host}"), "rows": rows, "cols": cols, + }); + serde_json::from_value(client.on(host, "sessions.start", json!({}), Some(body))?)? + } + }; + let stream = attach_stream(client, host, &session.id, 0)?; + let ended = match session.kind { + SessionKind::Terminal => client.wait(follow_terminal(stream))?, + SessionKind::Process => client.wait(follow_process(stream, false))?, + }; + // A shell started here and left with exit is done with. + finish(client, host, &session.id, ended, id.is_none()) +} + +fn session_state(session: &Value) -> String { + if session["ended"].is_null() { + return styled("running", Style::Good); + } + if session["lost"] == true { + return styled("lost", Style::Warn); + } + match session["exit_code"].as_i64() { + Some(0) => "exit 0".into(), + Some(code) => styled(&format!("exit {code}"), Style::Bad), + None => "ended".into(), + } +} + +pub fn session(client: &Client, command: Session) -> Result<()> { + match command { + Session::List { host } => { + let found = client.on(&host, "sessions.list", json!({}), None)?; + out::data(&found, || { + let mut table = Table::new(&["id", "kind", "state", "user", "started", "title"]); + for session in items(found.clone(), "sessions") { + let text = |key: &str| session[key].as_str().unwrap_or("").to_owned(); + table.row(vec![ + text("id"), + text("kind"), + session_state(&session), + text("user"), + text("started"), + text("title"), + ]); + } + table.print(&format!("No sessions on {host}")); + }); + Ok(()) + } + Session::Attach { host, id } => attach(client, &host, id.as_deref()), + Session::Logs { + host, + id, + follow, + tail, + } => logs(client, &host, &id, follow, tail), + Session::Stop { host, id } => { + client.on(&host, "sessions.delete", json!({"id": id}), None)?; + out::done(format!("Stopped {id} on {host}")); + Ok(()) + } + } +} + +fn logs(client: &Client, host: &str, id: &str, follow: bool, tail: Option) -> Result<()> { + let session: berth_api::Session = + serde_json::from_value(client.on(host, "sessions.get", json!({"id": id}), None)?)?; + let mut out = std::io::stdout(); + if session.kind == SessionKind::Terminal { + // A terminal's bytes are cursor movement; its screen is what reads. + let mut args = json!({"id": id}); + if let Some(lines) = tail { + args["tail"] = json!(lines); + } + let screen = client.on(host, "sessions.screen", args, None)?; + println!("{}", screen["text"].as_str().unwrap_or("").trim_end()); + if follow && session.running() { + out::hint(format!( + "A terminal is followed with: berth session attach {host} {id}" + )); + } + return Ok(()); + } + let mut since = 0; + loop { + let args = match tail { + Some(lines) => json!({"id": id, "tail": lines}), + None => json!({"id": id, "since": since}), + }; + let part: berth_api::Output = + serde_json::from_value(client.on(host, "sessions.output", args, None)?)?; + let bytes = berth_api::unb64(&part.data_b64).unwrap_or_default(); + match &part.lines { + Some(lines) => lines.iter().try_for_each(|line| writeln!(out, "{line}"))?, + None => out.write_all(&bytes)?, + } + let more = part.next_offset > since && !bytes.is_empty() && tail.is_none(); + since = part.next_offset; + if !more { + break; + } + } + out.flush()?; + if !follow || !session.running() { + return Ok(()); + } + let stream = attach_stream(client, host, id, since)?; + let ended = client.wait(follow_process(stream, false))?; + finish(client, host, id, ended, false) +} + +// ── actions ───────────────────────────────────────────────────────────────── + +/// A subject's actions as a table, and under it why each that cannot run +/// now cannot, which is too long for a column. +pub fn print_actions(actions: &[berth_api::Action]) { + let mut table = Table::new(&["action", "group", "label", "available"]); + for action in actions { + let available = match action.available { + true => styled("yes", Style::Good), + false => styled("no", Style::Warn), + }; + table.row(vec![ + action.name.clone(), + action.group.clone(), + action.label.clone(), + available, + ]); + } + table.print("Nothing to do"); + let refused: Vec<&berth_api::Action> = actions + .iter() + .filter(|action| !action.available && !action.why.is_empty()) + .collect(); + if !refused.is_empty() { + println!(); + } + for action in refused { + println!( + "{} {}", + styled(&format!("{}:", action.name), Style::Bold), + styled(&action.why, Style::Dim) + ); + } +} + +pub fn actions(client: &Client, host: Option<&str>) -> Result<()> { + let subject = match host { + Some(host) => Subject::Host(host.to_owned()), + None => Subject::Installation, + }; + let actions = client.actions(&subject)?; + out::data(&serde_json::to_value(&actions)?, || { + print_actions(&actions); + if let Some(host) = host { + out::hint(format!( + "Run one with: berth run {host} ACTION [NAME=VALUE…]" + )); + } + }); + Ok(()) +} + +/// Ask for the arguments an action requires and was not given, and confirm +/// it when it asks to be. +fn complete( + action: &berth_api::Action, + given: &mut BTreeMap, + subject: &str, +) -> Result<()> { + for argument in &action.arguments { + if !argument.required + || given + .get(&argument.name) + .is_some_and(|value| !value.is_empty()) + { + continue; + } + let label = match argument.label.as_str() { + "" => argument.name.clone(), + label => label.to_owned(), + }; + let flag = format!("{}=VALUE", argument.name); + let value = match argument.secret { + true => ask::secret(&label, &flag)?, + false => ask::text(&label, &argument.help, &flag)?, + }; + given.insert(argument.name.clone(), value); + } + if !action.available { + return Err(Fail::refused(format!( + "{} of {subject} cannot run now: {}", + action.name, action.why + ))); + } + if let Some(question) = &action.confirm { + ask::confirm(question)?; + } + Ok(()) +} + +fn find_action( + actions: Vec, + name: &str, + subject: &str, + list: &str, +) -> Result { + actions + .into_iter() + .find(|action| action.name == name) + .ok_or_else(|| Fail::usage(format!("{subject} has no action {name}. {list} names them"))) +} + +/// Run an action of a subject with its arguments, and follow what it started. +fn run_on( + client: &Client, + subject: &Subject, + name: &str, + words: &[String], + detach: bool, + sudo: Option, + own: bool, +) -> Result> { + let shown = subject.shown(); + let list = match subject { + Subject::Host(host) => format!("berth action list {host}"), + Subject::Provisioner(provisioner) => format!("berth provisioner actions {provisioner}"), + Subject::Resource(id) => format!("berth resource actions {id}"), + Subject::Installation => "berth action list".into(), + }; + let action = find_action(client.actions(subject)?, name, &shown, &list)?; + let mut given = arguments(words)?; + complete(&action, &mut given, &shown)?; + let (rows, cols) = terminal_size(); + let act = Act { + arguments: given, + sudo_password: sudo, + rows: Some(rows), + cols: Some(cols), + ..Act::default() + }; + let acted = client.act(subject, name, &act)?; + // A terminal of the host's is attached to here. + if acted.job.is_none() + && let (Some(session), Subject::Host(host)) = (&acted.session, subject) + { + let stream = attach_stream(client, host, &session.id, 0)?; + let ended = match session.kind { + SessionKind::Terminal => client.wait(follow_terminal(stream))?, + SessionKind::Process => client.wait(follow_process(stream, false))?, + }; + finish(client, host, &session.id, ended, true)?; + return Ok(None); + } + let label = match action.label.as_str() { + "" => action.name.clone(), + label => label.to_owned(), + }; + let result = follow::acted(client, acted, &format!("{label} · {shown}"), detach, own)?; + if !result.is_null() && result.get("id").is_none() { + out::data(&result, || { + for line in pages::answer_lines(&result) { + println!("{line}"); + } + }); + } + Ok(None) +} + +pub fn run_action(client: &Client, run: &args::RunAction) -> Result<()> { + let sudo = match out::get().sudo_stdin { + true => host::sudo_password(client, &run.host)?, + false => None, + }; + run_on( + client, + &Subject::Host(run.host.clone()), + &run.action, + &run.arguments, + run.detach.detach, + sudo, + true, + )?; + Ok(()) +} + +// ── provisioners ──────────────────────────────────────────────────────────── + +pub fn provisioner(client: &Client, command: Provisioner) -> Result<()> { + match command { + Provisioner::List => { + let found = client.call("provisioners.list", json!({}), None)?; + out::data(&found, || { + let mut table = Table::new(&["name", "type", "node", "usable", "label"]); + for provisioner in items(found.clone(), "provisioners") { + let text = |key: &str| provisioner[key].as_str().unwrap_or("").to_owned(); + let usable = match text("problem") { + problem if problem.is_empty() => styled("yes", Style::Good), + problem => styled(&format!("no: {problem}"), Style::Dim), + }; + table.row(vec![ + text("id"), + text("type"), + text("node"), + usable, + text("label"), + ]); + } + table.print("No provisioners. berth provisioner types lists what can be added"); + }); + Ok(()) + } + Provisioner::Types => { + let found = client.call("provisioners.types", json!({}), None)?; + out::data(&found, || { + let mut table = Table::new(&["type", "label", "makes"]); + for kind in items(found.clone(), "types") { + let text = |key: &str| kind[key].as_str().unwrap_or("").to_owned(); + let makes: Vec<&str> = kind["kinds"] + .as_array() + .into_iter() + .flatten() + .filter_map(|made| made["id"].as_str()) + .collect(); + table.row(vec![text("id"), text("label"), makes.join(", ")]); + } + table.print("No plugin brings a provisioner"); + }); + Ok(()) + } + Provisioner::Add { + name, + kind, + settings, + } => { + let body = json!({"type": kind, "settings": arguments(&settings)?}); + let saved = client.call("provisioners.save", json!({"name": name}), Some(body))?; + out::data(&saved, || { + out::done(format!("Added the provisioner {name}, of the type {kind}")) + }); + Ok(()) + } + Provisioner::Show { name, settings } => { + let given = arguments(&settings)?; + let mut args = json!({"name": name}); + for (key, value) in &given { + args[key] = json!(value); + } + let found = client.call("provisioners.get", args, None)?; + let state: berth_api::ProvisionerState = serde_json::from_value(found.clone())?; + let actions = client + .actions(&Subject::Provisioner(name.clone())) + .unwrap_or_default(); + out::data(&found, || { + print!("{}", pages::provisioner(&state, &actions, &given)) + }); + Ok(()) + } + Provisioner::Set { name, settings } => { + let run = settings_action(client, &name)?; + let words: Vec = settings; + run_on( + client, + &Subject::Provisioner(name.clone()), + &run, + &words, + false, + None, + false, + )?; + Ok(()) + } + Provisioner::Remove { name } => { + ask::confirm(&format!( + "This forgets the provisioner {name} with its settings." + ))?; + client.call("provisioners.remove", json!({"name": name}), None)?; + out::done(format!("Removed the provisioner {name}")); + Ok(()) + } + Provisioner::Actions { name } => { + let actions = client.actions(&Subject::Provisioner(name.clone()))?; + out::data(&serde_json::to_value(&actions)?, || { + print_actions(&actions); + out::hint(format!( + "Run one with: berth provisioner run {name} ACTION [NAME=VALUE…]" + )); + }); + Ok(()) + } + Provisioner::Run { + name, + action, + arguments, + detach, + } => { + run_on( + client, + &Subject::Provisioner(name), + &action, + &arguments, + detach.detach, + None, + false, + )?; + Ok(()) + } + } +} + +/// The action that keeps a provisioner's settings. +fn settings_action(client: &Client, name: &str) -> Result { + let actions = client.actions(&Subject::Provisioner(name.to_owned()))?; + actions + .iter() + .find(|action| action.name == "settings") + .map(|action| action.name.clone()) + .ok_or_else(|| Fail::failed(format!("{name} keeps no settings"))) +} + +// ── resources ─────────────────────────────────────────────────────────────── + +fn secrets_of(words: &[String]) -> Vec { + words + .iter() + .map(|word| match word.split_once(':') { + Some((name, variable)) => berth_api::ResourceSecret { + name: name.to_owned(), + env: variable.to_owned(), + }, + None => berth_api::ResourceSecret { + name: word.clone(), + ..Default::default() + }, + }) + .collect() +} + +fn spec_of(spec: &Option) -> Result> { + spec.as_deref() + .map(|file| { + let bytes = host::read_file(file)?; + String::from_utf8(bytes).map_err(|_| Fail::usage(format!("{file} is no text"))) + }) + .transpose() +} + +/// The id of a resource given by its id, or by a name only it has. +pub(super) fn resource_id(client: &Client, given: &str) -> Result { + match client.call("resources.get", json!({"id": given}), None) { + Ok(_) => return Ok(given.to_owned()), + Err(fail) if fail.code == "not_found" => {} + Err(fail) => return Err(fail), + } + let found = client.call("resources.list", json!({}), None)?; + let named: Vec = items(found, "resources") + .into_iter() + .filter(|made| made["name"] == given) + .filter_map(|made| made["id"].as_str().map(str::to_owned)) + .collect(); + match named.as_slice() { + [id] => Ok(id.clone()), + [] => Err(Fail::failed(format!( + "There is no resource {given}. berth resource list lists them" + ))), + _ => Err(Fail::usage(format!( + "Several resources are named {given}: give one of their ids, {}", + named.join(", ") + ))), + } +} + +pub fn resource(client: &Client, command: Resource) -> Result<()> { + match command { + Resource::List { + kind, + provisioner, + host, + } => { + let mut args = json!({}); + for (key, value) in [("kind", kind), ("provisioner", provisioner), ("host", host)] { + if let Some(value) = value { + args[key] = json!(value); + } + } + let found = client.call("resources.list", args, None)?; + out::data(&found, || { + let mut table = Table::new(&["id", "kind", "name", "provisioner", "state", "host"]); + for made in items(found.clone(), "resources") { + let text = |key: &str| made[key].as_str().unwrap_or("").to_owned(); + table.row(vec![ + text("id"), + text("kind"), + text("name"), + text("provisioner"), + text("state"), + text("host"), + ]); + } + table.print("Nothing made yet"); + }); + Ok(()) + } + Resource::Show { id } => { + let id = resource_id(client, &id)?; + let found = client.call("resources.get", json!({"id": id}), None)?; + let made: berth_api::Resource = serde_json::from_value(found.clone())?; + let actions = client + .actions(&Subject::Resource(id.clone())) + .unwrap_or_default(); + out::data(&found, || print!("{}", pages::resource(&made, &actions))); + Ok(()) + } + Resource::Create(create) => { + // A specification is what a deployment is made from; without one it is a host's machine. + let kind = match (&create.kind, &create.spec) { + (Some(kind), _) => kind.clone(), + (None, Some(_)) => "deployment".to_owned(), + (None, None) => String::new(), + }; + let found = client.call( + "resources.list", + json!({"provisioner": create.provisioner, "kind": kind}), + None, + )?; + let existing = items(found, "resources").into_iter().find(|made| { + made["name"] == create.name.as_str() + && made["external_id"] + .as_str() + .is_some_and(|id| !id.is_empty()) + }); + if let Some(existing) = existing + && !kind.is_empty() + { + let id = existing["id"].as_str().unwrap_or("").to_owned(); + out::note(format!( + "There is a {kind} {} already, {id}: giving it this specification", + create.name + )); + let set = args::ResourceSet { + id, + spec: create.spec.clone(), + secret: create.secret.clone(), + arguments: create.arguments.clone(), + detach: create.detach.clone(), + }; + return resource(client, Resource::Set(set)); + } + let request = berth_api::NewResource { + provisioner: create.provisioner.clone(), + kind, + name: create.name.clone(), + arguments: arguments(&create.arguments)?, + spec: spec_of(&create.spec)?, + secrets: secrets_of(&create.secret), + ..Default::default() + }; + let made = client.call( + "resources.create", + json!({}), + Some(serde_json::to_value(&request)?), + )?; + let made: berth_api::Created = serde_json::from_value(made)?; + let id = made.resource.id.clone(); + let acted = berth_api::Acted { + job: Some(made.job), + ..Default::default() + }; + follow::acted( + client, + acted, + &format!("Creating {} with {}", create.name, create.provisioner), + create.detach.detach, + false, + )?; + out::hint(format!("{} is {id}: berth resource show {id}", create.name)); + Ok(()) + } + Resource::Set(mut set) => { + set.id = resource_id(client, &set.id)?; + let change = berth_api::ResourceChange { + arguments: arguments(&set.arguments)?, + spec: spec_of(&set.spec)?, + secrets: (!set.secret.is_empty()).then(|| secrets_of(&set.secret)), + ..Default::default() + }; + let acted: berth_api::Acted = serde_json::from_value(client.call( + "resources.change", + json!({"id": set.id}), + Some(serde_json::to_value(&change)?), + )?)?; + follow::acted( + client, + acted, + &format!("Changing {}", set.id), + set.detach.detach, + false, + )?; + Ok(()) + } + Resource::Remove { id, detach } => { + let id = resource_id(client, &id)?; + ask::confirm(&format!("This removes {id} with its provisioner."))?; + let acted: berth_api::Acted = serde_json::from_value(client.call( + "resources.remove", + json!({"id": id}), + None, + )?)?; + follow::acted( + client, + acted, + &format!("Removing {id}"), + detach.detach, + false, + )?; + Ok(()) + } + Resource::Logs { id, follow, tail } => { + let id = resource_id(client, &id)?; + let mut args = json!({"id": id}); + if let Some(tail) = tail { + args["tail"] = json!(tail); + } + if !follow { + // A text answer comes as a string, or as its bytes in base64. + let found = client.call("resources.logs", args, None)?; + let text = match &found { + Value::String(text) => text.clone(), + other => other["base64"] + .as_str() + .and_then(berth_api::unb64) + .map(|bytes| String::from_utf8_lossy(&bytes).into_owned()) + .unwrap_or_default(), + }; + print!("{text}"); + return Ok(()); + } + let mut stream = client.stream("resources.follow", None, args)?; + client.wait(async { + let mut out = std::io::stdout(); + loop { + tokio::select! { + piece = stream.recv() => match piece { + Some(Ok(Piece::Binary(bytes))) => { out.write_all(&bytes)?; out.flush()?; } + Some(Ok(Piece::Text(said))) => { + if let Some(line) = said["line"].as_str() { writeln!(out, "{line}")?; } + } + Some(Err(error)) => return Err(Fail::from(error)), + None => return Ok(()), + }, + _ = tokio::signal::ctrl_c() => return Ok(()), + } + } + }) + } + Resource::Exec { + id, + stdin, + command, + detach, + } => { + let id = resource_id(client, &id)?; + let mut arguments = std::collections::BTreeMap::new(); + arguments.insert("command".to_owned(), serde_json::to_string(&command)?); + if stdin { + let bytes = super::host::read_file("-")?; + let text = String::from_utf8(bytes) + .map_err(|_| Fail::usage("--stdin takes text, and this is not UTF-8"))?; + arguments.insert("stdin".to_owned(), text); + } + let act = Act { + arguments, + ..Act::default() + }; + let acted = client.act(&Subject::Resource(id.clone()), "exec", &act)?; + follow::acted( + client, + acted, + &format!("{} in {id}", command[0]), + detach.detach, + true, + )?; + Ok(()) + } + Resource::Actions { id } => { + let id = resource_id(client, &id)?; + let actions = client.actions(&Subject::Resource(id.clone()))?; + out::data(&serde_json::to_value(&actions)?, || { + print_actions(&actions); + out::hint(format!( + "Run one with: berth resource run {id} ACTION [NAME=VALUE…]" + )); + }); + Ok(()) + } + Resource::Run { + id, + action, + arguments, + detach, + } => { + let id = resource_id(client, &id)?; + run_on( + client, + &Subject::Resource(id), + &action, + &arguments, + detach.detach, + None, + false, + )?; + Ok(()) + } + } +} diff --git a/crates/berth/src/client_only.rs b/crates/berth/src/client_only.rs new file mode 100644 index 0000000..266f419 --- /dev/null +++ b/crates/berth/src/client_only.rs @@ -0,0 +1,29 @@ +//! berth on a system other than Linux, where it is a client of a berth +//! serve elsewhere: every command but those of the server's own machine. + +use crate::Result; +use crate::cli::args::{Global, Ssh}; +use crate::cli::fail::{Fail, Result as Done}; + +/// Only berth on Linux answers for ssh. +pub fn askpass(_: &[String]) -> Result { + Ok(false) +} + +/// No server runs here: a command needs one named, and the screens ask for one. +pub fn server(_ask: bool) -> Result> { + Ok(None) +} + +/// No server runs in this process to give a sudo password to. +pub fn give_sudo_password(_: &str, _: &str) -> bool { + false +} + +/// Plain SSH needs the server's key, which is on the machine it runs on. +pub fn ssh(_: &Global, ssh: &Ssh) -> Done<()> { + Err(Fail::refused(format!( + "berth ssh logs in with the server's own key, on the Linux machine it runs on. berth session attach {} reaches the host through the server", + ssh.host + ))) +} diff --git a/crates/berth/src/commands.rs b/crates/berth/src/commands.rs deleted file mode 100644 index d53060c..0000000 --- a/crates/berth/src/commands.rs +++ /dev/null @@ -1,1999 +0,0 @@ -//! The commands that talk to a host's controller: sessions, tasks, the -//! channel and services. Each is a plain program that waits on a runtime. - -use crate::{ - Result, - remote::{exited, list}, -}; -use berth_api::{ - self as api, - client::{Error, Frame, encode, encode_path}, -}; -use berth_core::{ - host::Host, - level, - link::{self, Link}, - paths::Paths, - profiles::Place, - provision, registry, runner, -}; -use futures_util::{SinkExt, StreamExt}; -use serde_json::{Value, json}; -use std::collections::BTreeMap; -use std::io::{IsTerminal, Write}; -use tokio::io::AsyncReadExt; - -/// Ctrl+Q, which leaves a terminal session without ending it. -const DETACH: u8 = 0x11; - -struct Reach { - host: Host, - link: Link, - /// One runtime for the whole command: the link's kept connections live on it. - runtime: Option, -} - -impl Reach { - fn wait(&self, future: F) -> F::Output { - self.runtime - .as_ref() - .expect("until dropped") - .block_on(future) - } -} - -impl Drop for Reach { - /// A read of the terminal that is still waiting must not keep berth from leaving. - fn drop(&mut self) { - if let Some(runtime) = self.runtime.take() { - runtime.shutdown_background(); - } - } -} - -fn reach(paths: &Paths, name: &str) -> Result { - let host = Host::load(paths, name)?; - let password = host.sudo_password(paths)?; - let link = Link::new(host.link(paths, password.as_deref())?); - Ok(Reach { - host, - link, - runtime: Some(link::runtime()?), - }) -} - -/// The way to a host's controller, for what needs one: a host meant to be -/// only reachable is refused with the action that installs it. -fn controlled(paths: &Paths, name: &str) -> Result { - let reach = reach(paths, name)?; - level::guard(&reach.host, level::Capability::Sessions)?; - Ok(reach) -} - -/// An error of the link in words that say what to do about it. -fn explain(error: Error, host: &str) -> Box { - if Link::missing(&error) { - return format!( - "berth's controller is not on {host} yet. Install it alone with: berth host install {host}. For a workstation, apply its configuration: berth apply {host}" - ) - .into(); - } - error.into() -} - -/// `HOST [--user USER] [--keep] [--] COMMAND…` -pub struct Exec { - pub user: String, - pub keep: bool, - pub command: Vec, -} - -pub fn parse_exec(args: &[&str]) -> Result { - let mut exec = Exec { - user: "agent".into(), - keep: false, - command: vec![], - }; - let mut rest = args.iter(); - while let Some(arg) = rest.next() { - match *arg { - "--user" => { - let user = rest.next().ok_or("--user needs agent, operator or root")?; - if !["agent", "operator", "root"].contains(user) { - return Err("--user is agent, operator or root".into()); - } - exec.user = (*user).to_owned(); - } - "--keep" => exec.keep = true, - "--" => { - exec.command = rest.map(|word| (*word).to_owned()).collect(); - break; - } - word => { - exec.command = std::iter::once(word) - .chain(rest.copied()) - .map(str::to_owned) - .collect(); - break; - } - } - } - if exec.command.is_empty() { - return Err( - "Usage: berth exec HOST [--user agent|operator|root] [--keep] -- COMMAND…".into(), - ); - } - Ok(exec) -} - -/// `NAME=VALUE…` and `--yes`. -pub fn parse_arguments(args: &[&str]) -> Result<(BTreeMap, bool)> { - let mut values = BTreeMap::new(); - let mut yes = false; - for arg in args { - match (*arg, arg.split_once('=')) { - ("--yes" | "-y", _) => yes = true, - (_, Some((name, value))) if !name.is_empty() && !name.starts_with('-') => { - values.insert(name.to_owned(), value.to_owned()); - } - _ => return Err(format!("Arguments are given as NAME=VALUE: {arg}").into()), - } - } - Ok((values, yes)) -} - -/// Several hosts' inventories as one. -pub fn merge_inventory(hosts: &[(String, Value)]) -> Value { - let mut all = serde_json::Map::new(); - for (name, inventory) in hosts { - all.insert( - name.clone(), - inventory["all"]["hosts"]["devbox_target"].clone(), - ); - } - json!({"all": {"hosts": all}}) -} - -/// `berth inventory [HOST…]`: the hosts as an Ansible inventory. -pub fn inventory(paths: &Paths, names: &[&str]) -> Result<()> { - let hosts = match names { - [] => Host::all(paths) - .into_iter() - .filter(|host| !host.address().is_empty()) - .collect(), - names => names - .iter() - .map(|name| Host::load(paths, name)) - .collect::>>()?, - }; - let mut entries = vec![]; - for host in hosts { - let inventory = host - .inventory(paths) - .map_err(|error| format!("{}: {error}", host.name))?; - entries.push((host.name, inventory)); - } - println!( - "{}", - serde_json::to_string_pretty(&merge_inventory(&entries))? - ); - Ok(()) -} - -/// How following a session ended. -enum Ended { - Exited(Option), - /// Left with Ctrl+Q or Ctrl+C; it runs on. - Detached, - Lost, -} - -/// Follow a process session: its two streams to ours, apart. With `stdin`, -/// ours is its input and Ctrl+C interrupts it; without, Ctrl+C only leaves. -async fn follow_process(link: &Link, id: &str, since: u64, stdin: bool) -> Result { - let stream = link - .stream(&format!("/v1/sessions/{id}?since={since}")) - .await?; - let (mut sink, mut source) = stream.split(); - let (sender, mut outgoing) = tokio::sync::mpsc::unbounded_channel::(); - if stdin && !std::io::stdin().is_terminal() { - let sender = sender.clone(); - tokio::spawn(async move { - let mut input = tokio::io::stdin(); - let mut buffer = vec![0; 16384]; - while let Ok(count) = input.read(&mut buffer).await - && count > 0 - { - let frame = [&[0u8][..], &buffer[..count]].concat(); - if sender.send(Frame::Binary(frame.into())).is_err() { - return; - } - } - let _ = sender.send(Frame::Text(json!({"eof": true}).to_string().into())); - }); - } else if stdin { - // Nothing will be typed into it. - let _ = sender.send(Frame::Text(json!({"eof": true}).to_string().into())); - } - let (mut out, mut err) = (std::io::stdout(), std::io::stderr()); - loop { - tokio::select! { - frame = source.next() => match frame { - Some(Ok(Frame::Binary(bytes))) => match bytes.split_first() { - Some((2, rest)) => { - err.write_all(rest)?; - err.flush()?; - } - Some((_, rest)) => { - out.write_all(rest)?; - out.flush()?; - } - None => {} - }, - Some(Ok(Frame::Text(text))) => { - if let Some(code) = exited(text.as_str()) { - return Ok(Ended::Exited(code)); - } - } - Some(Ok(_)) => {} - _ => return Ok(Ended::Lost), - }, - Some(frame) = outgoing.recv() => { - if sink.send(frame).await.is_err() { - return Ok(Ended::Lost); - } - } - _ = tokio::signal::ctrl_c() => { - if !stdin { - return Ok(Ended::Detached); - } - let _ = sink.send(Frame::Text(json!({"signal": "INT"}).to_string().into())).await; - } - } - } -} - -/// Puts the terminal back as it was when dropped. -struct Raw(bool); - -impl Raw { - fn enter() -> Raw { - let terminal = std::io::stdin().is_terminal() && std::io::stdout().is_terminal(); - Raw(terminal && crossterm::terminal::enable_raw_mode().is_ok()) - } -} - -impl Drop for Raw { - fn drop(&mut self) { - if self.0 { - let _ = crossterm::terminal::disable_raw_mode(); - } - } -} - -fn terminal_size() -> (u16, u16) { - // A terminal that does not know its size says 0 by 0. - match crossterm::terminal::size() { - Ok((cols, rows)) if cols > 0 && rows > 0 => (rows, cols), - _ => (24, 80), - } -} - -/// This terminal onto a terminal session. Ctrl+Q leaves it running. -async fn follow_terminal(link: &Link, id: &str) -> Result { - let stream = link.stream(&format!("/v1/sessions/{id}")).await?; - let (mut sink, mut source) = stream.split(); - let raw = Raw::enter(); - let resize = |(rows, cols): (u16, u16)| { - Frame::Text( - json!({"resize": {"rows": rows, "cols": cols}}) - .to_string() - .into(), - ) - }; - let mut size = terminal_size(); - if raw.0 { - sink.send(resize(size)) - .await - .map_err(|error| error.to_string())?; - } - let (sender, mut typed) = tokio::sync::mpsc::unbounded_channel::>(); - tokio::spawn(async move { - let mut input = tokio::io::stdin(); - let mut buffer = vec![0; 4096]; - while let Ok(count) = input.read(&mut buffer).await - && count > 0 - && sender.send(buffer[..count].to_vec()).is_ok() - {} - }); - let mut tick = tokio::time::interval(std::time::Duration::from_millis(250)); - let mut out = std::io::stdout(); - loop { - tokio::select! { - frame = source.next() => match frame { - Some(Ok(Frame::Binary(bytes))) => { - out.write_all(&bytes)?; - out.flush()?; - } - Some(Ok(Frame::Text(text))) => { - if let Some(code) = exited(text.as_str()) { - return Ok(Ended::Exited(code)); - } - } - Some(Ok(_)) => {} - _ => return Ok(Ended::Lost), - }, - Some(bytes) = typed.recv() => { - if raw.0 && let Some(at) = bytes.iter().position(|byte| *byte == DETACH) { - if at > 0 { - let _ = sink.send(Frame::Binary(bytes[..at].to_vec().into())).await; - } - return Ok(Ended::Detached); - } - if sink.send(Frame::Binary(bytes.into())).await.is_err() { - return Ok(Ended::Lost); - } - } - _ = tick.tick() => { - let now = terminal_size(); - if raw.0 && now != size { - size = now; - let _ = sink.send(resize(size)).await; - } - } - } - } -} - -/// What to say and which code to leave with once following has ended. A -/// session that was only for this call is forgotten when it has ended. -async fn finish(reach: &Reach, id: &str, ended: Ended, forget: bool) -> i32 { - let name = &reach.host.name; - match ended { - Ended::Exited(code) => { - if forget { - let _ = reach - .link - .call::("DELETE", &format!("/v1/sessions/{id}"), None) - .await; - } - code.unwrap_or(1) - } - Ended::Detached => { - eprintln!( - "\r\nLeft {id} running on {name}. Back to it: berth attach {name} {id}, or berth output {name} {id} --follow" - ); - 0 - } - Ended::Lost => { - eprintln!( - "\r\nThe connection was lost; {id} may still run on {name}: berth output {name} {id} --follow" - ); - 1 - } - } -} - -/// `berth exec HOST [--user …] [--keep] -- COMMAND…`: run a command on the -/// host as a process session and leave with its exit code. -pub fn exec(paths: &Paths, name: &str, args: &[&str]) -> Result<()> { - let exec = parse_exec(args)?; - let reach = controlled(paths, name)?; - let code = reach.wait(async { - let body = json!({ - "kind": "process", "command": exec.command, "user": exec.user, - "title": exec.command.join(" ").chars().take(200).collect::(), - }); - let session: api::Session = reach - .link - .post("/v1/sessions", &body) - .await - .map_err(|error| explain(error, name))?; - let ended = follow_process(&reach.link, &session.id, 0, true).await?; - Ok::>(finish(&reach, &session.id, ended, !exec.keep).await) - })?; - std::process::exit(code); -} - -/// `berth attach HOST [ID]`: this terminal onto a terminal session; without -/// an id, onto a new shell as the agent. -pub fn attach(paths: &Paths, name: &str, id: Option<&str>) -> Result<()> { - let reach = controlled(paths, name)?; - let code = reach.wait(async { - let (rows, cols) = terminal_size(); - let session: api::Session = match id { - Some(id) => reach.link.get(&format!("/v1/sessions/{}", encode(id))).await, - None => { - let body = json!({ - "kind": "terminal", "command": ["bash", "-l"], "user": "agent", - "title": format!("{} on {name}", reach.host.agent()), "rows": rows, "cols": cols, - }); - reach.link.post("/v1/sessions", &body).await - } - } - .map_err(|error| explain(error, name))?; - let ended = match session.kind { - api::SessionKind::Terminal => follow_terminal(&reach.link, &session.id).await?, - api::SessionKind::Process => follow_process(&reach.link, &session.id, 0, false).await?, - }; - // A shell started here and left with exit is done with. - Ok::>(finish(&reach, &session.id, ended, id.is_none()).await) - })?; - std::process::exit(code); -} - -/// `berth sessions HOST` -pub fn sessions(paths: &Paths, name: &str) -> Result<()> { - let reach = controlled(paths, name)?; - let sessions: Vec = reach - .wait(reach.link.get::("/v1/sessions")) - .map(|value| list(value, "sessions")) - .map_err(|error| explain(error, name))?; - for session in sessions { - let state = match (session.running(), session.lost, session.exit_code) { - (true, ..) => "running".to_owned(), - (_, true, _) => "lost".to_owned(), - (_, _, Some(code)) => format!("exit {code}"), - _ => "ended".to_owned(), - }; - let kind = match session.kind { - api::SessionKind::Terminal => "terminal", - api::SessionKind::Process => "process", - }; - println!( - "{:<14} {:<9} {:<9} {:<9} {} {}", - session.id, kind, state, session.user, session.started, session.title - ); - } - Ok(()) -} - -/// `berth output HOST ID [--tail N] [--follow]` -pub fn output(paths: &Paths, name: &str, id: &str, args: &[&str]) -> Result<()> { - let (mut tail, mut follow) = (None, false); - let mut rest = args.iter(); - while let Some(arg) = rest.next() { - match *arg { - "--follow" | "-f" => follow = true, - "--tail" => { - let count = rest.next().and_then(|count| count.parse::().ok()); - tail = Some(count.ok_or("--tail needs a number of lines")?); - } - _ => return Err("Usage: berth output HOST ID [--tail N] [--follow]".into()), - } - } - let reach = controlled(paths, name)?; - let id = encode(id); - let code = reach.wait(async { - let link = &reach.link; - let session: api::Session = link - .get(&format!("/v1/sessions/{id}")) - .await - .map_err(|error| explain(error, name))?; - let mut out = std::io::stdout(); - if session.kind == api::SessionKind::Terminal { - // A terminal's bytes are cursor movement; its screen is what reads. - let path = match tail { - Some(lines) => format!("/v1/sessions/{id}/screen?tail={lines}"), - None => format!("/v1/sessions/{id}/screen"), - }; - let screen: api::Screen = link.get(&path).await?; - println!("{}", screen.text.trim_end()); - if follow && session.running() { - eprintln!("A terminal is followed with: berth attach {name} {id}"); - } - return Ok::>(0); - } - let mut since = 0; - loop { - let path = match tail { - Some(lines) => format!("/v1/sessions/{id}/output?tail={lines}"), - None => format!("/v1/sessions/{id}/output?since={since}"), - }; - let part: api::Output = link.get(&path).await?; - let bytes = api::unb64(&part.data_b64).unwrap_or_default(); - match &part.lines { - Some(lines) => lines.iter().try_for_each(|line| writeln!(out, "{line}"))?, - None => out.write_all(&bytes)?, - } - let more = part.next_offset > since && !bytes.is_empty() && tail.is_none(); - since = part.next_offset; - if !more { - break; - } - } - out.flush()?; - if !follow || !session.running() { - return Ok(0); - } - let ended = follow_process(link, &session.id, since, false).await?; - Ok(finish(&reach, &session.id, ended, false).await) - })?; - std::process::exit(code); -} - -fn host_tasks(reach: &Reach) -> Result> { - reach - .wait(reach.link.get::("/v1/tasks")) - .map(|value| list(value, "tasks")) - .map_err(|error| explain(error, &reach.host.name)) -} - -/// What a host can do, with the tasks its controller lists for a host of -/// this machine. A berth server's host has the server's list of its -/// actions, which has berth's own beside its tasks. -fn host_actions(paths: &Paths, reach: &Reach) -> Result<(Vec, Vec)> { - match reach.host.served(paths) { - Some(server) => { - let listed = server.call("hosts.actions", json!({"name": reach.host.name}), None)?; - Ok((serde_json::from_value(listed)?, vec![])) - } - None => { - // A host that is only reachable has no controller to ask. - let host_level = level::of(&reach.host.config); - let tasks = match host_level { - level::Level::Reachable => vec![], - _ => host_tasks(reach)?, - }; - let mut actions = api::Action::of_tasks(&tasks); - let own = level::local_actions(&reach.host.name, host_level, observed(reach)); - actions.retain(|action| !own.iter().any(|known| known.name == action.name)); - actions.extend(own); - Ok((actions, tasks)) - } - } -} - -/// What the host's controller says the host is, when one answers in time. -/// Only a host whose profile says less than a workstation is asked. -fn observed(reach: &Reach) -> Option { - if level::of(&reach.host.config) == level::Level::Workstation { - return None; - } - let asked = reach.wait(async { - match tokio::time::timeout( - std::time::Duration::from_secs(15), - reach.link.get::("/v1/version"), - ) - .await - { - Ok(Ok(version)) => Ok(version), - _ => Err(()), - } - }); - asked - .ok() - .as_ref() - .and_then(|version| level::seen(Ok(version))) -} - -/// `berth tasks HOST`: what the host can do, by group. -pub fn tasks(paths: &Paths, name: &str) -> Result<()> { - let reach = reach(paths, name)?; - let (actions, _) = host_actions(paths, &reach)?; - if let Some(said) = level::mismatch(name, level::of(&reach.host.config), observed(&reach)) { - println!("! {said}\n"); - } - let mut groups: Vec<(&str, Vec<&api::Action>)> = vec![]; - for action in &actions { - let title = if action.group.is_empty() { - "System" - } else { - action.group.as_str() - }; - match groups - .iter_mut() - .find(|(known, _)| known.eq_ignore_ascii_case(title)) - { - Some((_, list)) => list.push(action), - None => groups.push((title, vec![action])), - } - } - for (title, list) in groups { - println!("{title}"); - for action in list { - let arguments: Vec = action - .arguments - .iter() - .map(|argument| match argument.required { - true => format!("{}=…", argument.name), - false => format!("[{}=…]", argument.name), - }) - .collect(); - let why = match action.available { - true => String::new(), - false => format!(" ({})", action.why), - }; - println!( - " {:<22} {} {}{why}", - action.name, - action.label, - arguments.join(" ") - ); - for step in &action.steps { - println!(" {:<22} + {}", "", step.label); - } - } - } - Ok(()) -} - -fn ask(question: &str) -> Result { - print!("{question} "); - std::io::stdout().flush()?; - let mut answer = String::new(); - std::io::stdin().read_line(&mut answer)?; - Ok(answer.trim().to_owned()) -} - -/// The action of that name, with the values of its arguments: those given, -/// and the required ones asked for on a terminal. Its question is asked -/// unless `yes`. -pub(crate) fn ready( - subject: &str, - actions: &[api::Action], - name: &str, - mut values: BTreeMap, - yes: bool, -) -> Result<(api::Action, BTreeMap)> { - let Some(found) = actions.iter().find(|known| known.name == name) else { - let names: Vec<&str> = actions.iter().map(|known| known.name.as_str()).collect(); - return Err(format!("{subject} has no task {name}. Tasks: {}", names.join(", ")).into()); - }; - if !found.available { - return Err(format!("{name} cannot run now: {}", found.why).into()); - } - for argument in &found.arguments { - if values.contains_key(&argument.name) { - continue; - } - if argument.required && std::io::stdin().is_terminal() { - let label = if argument.label.is_empty() { - &argument.name - } else { - &argument.label - }; - let value = match argument.secret { - true => registry::read_secret(label)?, - false => ask(&format!("{label}:"))?, - }; - values.insert(argument.name.clone(), value); - } else if argument.required { - return Err(format!("{name} needs {}=VALUE", argument.name).into()); - } - } - if let (Some(question), false) = (&found.confirm, yes) - && !ask(&format!("{question} [y/N]"))?.eq_ignore_ascii_case("y") - { - return Err("Not run".into()); - } - Ok((found.clone(), values)) -} - -/// `berth run HOST TASK [NAME=VALUE…] [--yes]`: run one of the host's -/// actions and leave with its exit code. -pub fn run(paths: &Paths, name: &str, task: &str, args: &[&str]) -> Result<()> { - let (values, yes) = parse_arguments(args)?; - let reach = reach(paths, name)?; - let (actions, tasks) = host_actions(paths, &reach)?; - let (action, values) = ready(name, &actions, task, values, yes)?; - let mut act = api::Act { - arguments: values, - ..api::Act::default() - }; - if action.terminal { - let (rows, cols) = terminal_size(); - (act.rows, act.cols) = (Some(rows), Some(cols)); - } - let code = match reach.host.served(paths) { - Some(server) => { - // The server has no terminal to ask on: a password it needs is asked for here. - act.sudo_password = reach.host.sudo_password(paths)?; - let acted = act_on(&server, Some(name), &action.name, &act)?; - follow_acted(&server, Some(&reach), acted, &mut std::io::stdout())? - } - None if action.runs == api::Runs::Berth => run_own(paths, &reach.host, &action.name)?, - None => run_here(paths, &reach, &tasks, &action, act)?, - }; - std::process::exit(code); -} - -/// Run one of berth's own actions on a host of this machine: the exit code. -fn run_own(paths: &Paths, host: &Host, action: &str) -> Result { - match action { - "install-controller" => install_controller_here(paths, host), - other => Err(format!("{other} is not an action of berth on {}", host.name).into()), - } -} - -/// `berth host install HOST`: put berth's controller alone on the host. On -/// a berth server that is a job of the server, whose output is shown. -pub fn install_controller(paths: &Paths, name: &str) -> Result<()> { - let host = Host::load(paths, name)?; - let Some(server) = host.served(paths) else { - let code = install_controller_here(paths, &host)?; - std::process::exit(code); - }; - let act = api::Act { - sudo_password: host.sudo_password(paths)?, - ..api::Act::default() - }; - let acted = act_on(&server, Some(name), "install-controller", &act)?; - let code = follow_acted(&server, None, acted, &mut std::io::stdout())?; - std::process::exit(code); -} - -/// Install the controller from here, holding the host's lock as an apply -/// does: the exit code. -fn install_controller_here(paths: &Paths, host: &Host) -> Result { - level::guard(host, level::Capability::InstallController)?; - let password = host.sudo_password(paths)?; - let request = runner::Request { - secrets: password.iter().cloned().collect(), - ..runner::Request::new( - runner::Subject::Host(host.name.clone()), - "install-controller", - ) - }; - let (owned, place) = (paths.clone(), host.place.clone()); - let host = Host { - name: host.name.clone(), - place, - config: host.config.clone(), - }; - let work = runner::Work::Function(Box::new(move |out| { - berth_core::apply::install_controller(&owned, &host, password.as_deref(), out) - })); - let code = runner::here(paths, request, work, &mut std::io::stdout())?; - Ok(code.unwrap_or(1)) -} - -/// `berth host level HOST LEVEL`: save the level the host's profile says, -/// such as for a workstation added by its address. Nothing on the host -/// changes; a workstation's set-up then fills in the rest before an apply. -pub fn set_level(paths: &Paths, name: &str, given: &str) -> Result<()> { - let wanted = level::Level::parse(given) - .ok_or_else(|| format!("A level is reachable, controller or workstation, not {given}"))?; - let host = Host::load(paths, name)?; - let mut profile = host.config.clone(); - api::profile::set_level(&mut profile, wanted); - if wanted == level::Level::Workstation && !profile["builder"].is_object() { - profile["builder"] = json!({}); - } - match host.served(paths) { - Some(server) => { - server.save_host(&host.name, &profile)?; - server.mirror(paths)?; - } - None => host.place.save(paths, &profile)?, - } - println!("{} is a {wanted} host now in its profile", host.name); - if wanted == level::Level::Workstation { - println!("Check its set-up, then apply it: berth apply {}", host.name); - } - Ok(()) -} - -/// Offer to save the level a host just added was found to have, when it is -/// more than its profile says: a workstation added by its address is not -/// to be treated as a bare machine. -fn offer_found_level(paths: &Paths, name: &str, intended: level::Level) -> Result<()> { - let Ok(reach) = reach(paths, name) else { - return Ok(()); - }; - let found = { - let asked = reach.wait(async { - tokio::time::timeout( - std::time::Duration::from_secs(15), - reach.link.get::("/v1/version"), - ) - .await - }); - match asked { - Ok(Ok(version)) => level::seen(Ok(&version)), - _ => None, - } - }; - drop(reach); - let (Some(found), Some(said)) = (found, level::mismatch(name, intended, found)) else { - return Ok(()); - }; - println!("! {said}"); - // A workstation keeps its profile, which says more than a level would. - if found == level::Level::Workstation { - if std::io::stdin().is_terminal() - && !ask(&format!("Adopt {name} from the profile it keeps? [Y/n]"))? - .eq_ignore_ascii_case("n") - { - return adopt(paths, name, &[]); - } - println!("Adopt it with: berth adopt {name}"); - return Ok(()); - } - if std::io::stdin().is_terminal() - && !ask(&format!("Save {name} as a {found} host? [Y/n]"))?.eq_ignore_ascii_case("n") - { - return set_level(paths, name, found.name()); - } - println!("Save it with: berth host level {name} {found}"); - Ok(()) -} - -/// `berth host add NAME ADDRESS [--user USER] [--port PORT] [--auth AUTH] -/// [--level LEVEL] [--no-contact]`: save a host that exists already, by -/// where it is and how berth logs in, with nothing of a workstation. Then -/// first contact, unless told not to: berth's key is tried, and a host that -/// refuses it is logged in to once with a password or key to install it. -pub fn add_host(paths: &Paths, name: &str, address: &str, args: &[&str]) -> Result<()> { - const USAGE: &str = "Usage: berth host add NAME ADDRESS [--user USER] [--port PORT] [--auth berth|key:NAME|password|ssh] [--level reachable|controller|workstation] [--no-contact]"; - let mut access = api::profile::Access { - address: address.to_owned(), - ..api::profile::Access::default() - }; - let (mut host_level, mut contact) = (level::Level::Reachable, true); - let mut rest = args.iter(); - while let Some(arg) = rest.next() { - let mut value = || rest.next().copied().ok_or(USAGE); - match *arg { - "--user" => access.user = value()?.to_owned(), - "--port" => access.port = value()?.to_owned(), - "--auth" => { - let given = value()?; - access.auth = api::profile::Auth::parse(given).ok_or_else(|| { - format!("--auth is berth, key:NAME, password or ssh, not {given}") - })?; - } - "--level" => { - let given = value()?; - host_level = level::Level::parse(given).ok_or_else(|| { - format!("--level is reachable, controller or workstation, not {given}") - })?; - } - "--no-contact" => contact = false, - _ => return Err(USAGE.into()), - } - } - if !berth_core::profiles::valid_name(name) { - return Err( - format!("{name} cannot be a host's name: letters, digits, -, _ and . only").into(), - ); - } - let mut profile = api::profile::minimal(host_level, &access); - // A workstation keeps what its set-up fills in; it starts empty. - if host_level == level::Level::Workstation { - profile["builder"] = json!({}); - } - let problems = api::profile::problems(&profile); - if !problems.is_empty() { - return Err(problems.join("; ").into()); - } - let place = Place::resolve(paths, name)?; - if place.load(paths)?.is_some() { - return Err(format!( - "There is a host {name} already. Change it with: berth profile {name} --set FILE" - ) - .into()); - } - match berth_core::server::Server::configured(paths) { - Some(server) => { - server.save_host(name, &profile)?; - server.mirror(paths)?; - println!("Added {name} on the berth server at {}", server.url); - } - None => { - if let Some(key) = access.auth.key() { - let store = berth_core::store::shared(paths)?; - if key != api::profile::BERTH_KEY - && !berth_core::keys::ring::names(&store)? - .iter() - .any(|known| known == key) - { - return Err(format!( - "There is no key {key}. Add it with: berth key add {key} FILE" - ) - .into()); - } - } - place.save(paths, &profile)?; - println!("Added {name} as a {host_level} host: {}", place.describe()); - seed_host_key(paths, &place, &access)?; - } - } - if !contact { - return Ok(()); - } - crate::key_command::trust(paths, name, &[]).map_err(|error| -> Box { - format!("{name} is saved, but logging in to it did not work yet: {error}. Try again with: berth key trust {name}").into() - })?; - offer_found_level(paths, name, host_level) -} - -/// Take the host key of a host just added from the user's own -/// `~/.ssh/known_hosts`, when that knows the address, and say which. -fn seed_host_key(paths: &Paths, place: &Place, access: &api::profile::Access) -> Result<()> { - let Some(home) = std::env::var_os("HOME") else { - return Ok(()); - }; - let known_hosts = std::path::PathBuf::from(home).join(".ssh/known_hosts"); - let names = match access.port.as_str() { - "" | "22" => vec![access.address.clone()], - port => vec![format!("[{}]:{port}", access.address)], - }; - let store = berth_core::store::shared(paths)?; - if berth_core::keys::known::seed_from(&store, place, &known_hosts, &names)? { - let record = berth_core::keys::known::Record::load(&store, place)?; - for key in &record.host_keys { - println!( - "Recorded its host key from {}: {}", - known_hosts.display(), - berth_core::keys::ring::fingerprint(key) - ); - } - } - Ok(()) -} - -/// Run a task of a host of this machine as a session there and follow it. -fn run_here( - paths: &Paths, - reach: &Reach, - tasks: &[api::Task], - action: &api::Action, - act: api::Act, -) -> Result { - let mut body = json!({"arguments": act.arguments}); - if let (Some(rows), Some(cols)) = (act.rows, act.cols) { - (body["rows"], body["cols"]) = (json!(rows), json!(cols)); - } - // Secrets are built here and travel on the task's standard input. - let task = tasks.iter().find(|task| task.name == action.name); - if task.and_then(|task| task.input.as_deref()) == Some("secrets") { - let payload = registry::Store::new(paths)?.host_payload(&reach.host.config)?; - body["input_b64"] = json!(api::b64(format!("{payload}\n").as_bytes())); - } - reach.wait(async { - let path = format!("/v1/tasks/{}", encode(&action.name)); - let session: api::Session = reach.link.post(&path, &body).await?; - let ended = follow_session(&reach.link, &session).await?; - Ok::>(finish(reach, &session.id, ended, true).await) - }) -} - -async fn follow_session(link: &Link, session: &api::Session) -> Result { - match session.kind { - api::SessionKind::Terminal => follow_terminal(link, &session.id).await, - api::SessionKind::Process => follow_process(link, &session.id, 0, false).await, - } -} - -/// `berth channel HOST [TOPIC-PREFIX] [--since ID]`: messages as they come, -/// one JSON object per line. -pub fn channel(paths: &Paths, name: &str, args: &[&str]) -> Result<()> { - let (mut prefix, mut since) = ("", None); - let mut rest = args.iter(); - while let Some(arg) = rest.next() { - match *arg { - "--since" => { - since = Some( - rest.next() - .ok_or("--since needs a message id")? - .parse::()?, - ) - } - topic if !topic.starts_with('-') && prefix.is_empty() => prefix = topic, - _ => return Err("Usage: berth channel HOST [TOPIC-PREFIX] [--since ID]".into()), - } - } - let reach = controlled(paths, name)?; - let mut path = format!("/v1/channel?topic={}", encode(prefix)); - if let Some(since) = since { - path += &format!("&since={since}"); - } - reach.wait(async { - let mut stream = reach - .link - .stream(&path) - .await - .map_err(|error| explain(error, name))?; - let mut out = std::io::stdout(); - while let Some(Ok(frame)) = stream.next().await { - if let Frame::Text(text) = frame { - writeln!(out, "{}", text.as_str())?; - out.flush()?; - } - } - Err("The connection was lost".into()) - }) -} - -/// `berth call HOST SERVICE METHOD [JSON]`: call a plugin's service. -pub fn call( - paths: &Paths, - name: &str, - service: &str, - method: &str, - body: Option<&str>, -) -> Result<()> { - let body: Value = match body { - Some(text) => { - serde_json::from_str(text).map_err(|error| format!("The body is not JSON: {error}"))? - } - None => json!({}), - }; - let reach = controlled(paths, name)?; - let path = format!("/v1/services/{}/{}", encode_path(service), encode(method)); - let answer: Value = reach - .wait(reach.link.post(&path, &body)) - .map_err(|error| explain(error, name))?; - println!("{}", serde_json::to_string_pretty(&answer)?); - Ok(()) -} - -/// `berth apply HOST`: apply the host's profile to it. On a berth server that -/// is a job of the server, whose output is shown as it comes. -pub fn apply(paths: &Paths, name: &str) -> Result<()> { - let host = Host::load(paths, name)?; - let Some(server) = host.served(paths) else { - let password = host.sudo_password(paths)?; - let prepared = berth_core::apply::prepare(paths, &host, password.as_deref())?; - let work = runner::Work::Program(Box::new(runner::Program { - command: prepared.command, - input: None, - files: prepared.files, - })); - let request = runner::Request::new(runner::Subject::Host(name.to_owned()), "apply"); - let code = runner::here(paths, request, work, &mut std::io::stdout())?; - if code == Some(0) { - berth_core::apply::settle(paths, &host)?; - } - std::process::exit(code.unwrap_or(1)); - }; - // The server has no terminal to ask on: a password it needs is asked for here. - let act = api::Act { - sudo_password: host.sudo_password(paths)?, - ..api::Act::default() - }; - let acted = act_on(&server, Some(name), "apply", &act)?; - let code = follow_acted(&server, None, acted, &mut std::io::stdout())?; - std::process::exit(code); -} - -/// Run an action of a berth server's: of the host `host`, or of the -/// installation without one. -pub(crate) fn act_on( - server: &berth_core::server::Server, - host: Option<&str>, - action: &str, - act: &api::Act, -) -> Result { - let (op, args) = match host { - Some(name) => ("hosts.act", json!({"name": name, "action": action})), - None => ("actions.act", json!({"action": action})), - }; - let acted = server.call(op, args, Some(serde_json::to_value(act)?))?; - Ok(serde_json::from_value(acted)?) -} - -/// Follow the job of an action a berth server started, such as a key's rotation. -pub(crate) fn follow_server_job( - server: &berth_core::server::Server, - acted: api::Acted, - out: &mut dyn Write, -) -> Result { - follow_acted(server, None, acted, out) -} - -/// Follow what running an action answered to its end: a session on the -/// host through `reach`, or the job, whose output goes to `out`. A result -/// is written out as it is. The exit code. -fn follow_acted( - server: &berth_core::server::Server, - reach: Option<&Reach>, - acted: api::Acted, - out: &mut dyn Write, -) -> Result { - if let (Some(session), Some(reach)) = (&acted.session, reach) { - return reach.wait(async { - let ended = follow_session(&reach.link, session).await?; - // The server's job copies the session's output; it is forgotten - // on the host once the job has all of it. - let copied = match (&ended, &acted.job) { - (Ended::Exited(_), Some(job)) => { - let portal = server.portal().await?; - job_ended(&portal, &job.id).await - } - (Ended::Exited(_), None) => true, - _ => false, - }; - Ok::>(finish(reach, &session.id, ended, copied).await) - }); - } - if let Some(job) = &acted.job { - return link::runtime()?.block_on(async { - let portal = server.portal().await?; - eprintln!("Job {} on the berth server at {}", job.id, server.url); - follow_job(&portal, &job.id, out).await - }); - } - if let Some(result) = &acted.result { - writeln!(out, "{}", serde_json::to_string_pretty(result)?)?; - } - Ok(0) -} - -/// Wait a little for a job to end: whether it did. -async fn job_ended(portal: &api::portal::Portal, id: &str) -> bool { - for _ in 0..50 { - let job = portal.call("jobs.get", None, json!({"id": id}), None).await; - if job.is_ok_and(|job| !job["ended"].is_null()) { - return true; - } - tokio::time::sleep(std::time::Duration::from_millis(200)).await; - } - false -} - -/// A job's output to `out` as it comes, until it ends, and its exit code. -pub(crate) async fn follow_job( - portal: &api::portal::Portal, - id: &str, - out: &mut dyn Write, -) -> Result { - let mut output = portal.open("jobs.follow", None, json!({"id": id})).await?; - let mut code = None; - while let Some(piece) = output.recv().await { - match piece? { - api::portal::Piece::Binary(data) => { - out.write_all(&data)?; - out.flush()?; - } - api::portal::Piece::Text(said) => { - if let Some(exited) = said.get("exited") { - code = Some(exited["exit_code"].as_i64().unwrap_or(1) as i32); - } - } - } - } - Ok(code.unwrap_or(1)) -} - -/// A roll-out, from here or as a job of the berth server berth is set to -/// use. Its output goes to `out` as it comes. With `ask`, a host whose sudo -/// asks for a password has it asked for on the terminal. -pub fn rollout( - paths: &Paths, - options: &api::Rollout, - ask: bool, - out: &mut dyn Write, -) -> Result { - let Some(server) = berth_core::server::Server::configured(paths) else { - let password = |host: &Host| match ask { - true => host.sudo_password(paths).ok().flatten(), - false => std::env::var("BERTH_SUDO_PASSWORD") - .ok() - .filter(|_| host.asks_password()), - }; - // One roll-out at a time, as a job of a server holds the lock too. - let _lock = runner::lock(paths, &runner::Subject::Installation, "rollout")?; - return berth_core::rollout::local(paths, options, &password, out); - }; - let act = api::Act { - arguments: options.arguments(), - ..api::Act::default() - }; - let acted = act_on(&server, None, "rollout", &act)?; - let id = acted.job.map(|job| job.id).unwrap_or_default(); - link::runtime()?.block_on(async { - let portal = server.portal().await?; - let _ = writeln!(out, "Job {id} on the berth server at {}", server.url); - follow_job(&portal, &id, out).await?; - let ended = portal - .call("jobs.get", None, json!({"id": id}), None) - .await?; - Ok(serde_json::from_value(ended["report"].clone()).unwrap_or_default()) - }) -} - -/// `[--all] [--update] [--reboot] [--force]` of a roll-out. -pub fn parse_rollout(args: &[&str]) -> Result { - let mut options = api::Rollout::default(); - for arg in args { - match *arg { - "--behind" => {} - "--all" => options.all = true, - "--update" => options.update = true, - "--reboot" => options.reboot = true, - "--force" => options.force = true, - other => return Err(format!("Not an option of a roll-out: {other}").into()), - } - } - if options.force && !options.reboot { - return Err("--force only goes with --reboot".into()); - } - Ok(options) -} - -/// `berth apply --behind|--all [--update] [--reboot] [--force]`: bring the -/// hosts up to date, one at a time. -pub fn apply_hosts(paths: &Paths, args: &[&str]) -> Result<()> { - let options = parse_rollout(args)?; - let mut out = std::io::stdout(); - let report = rollout(paths, &options, true, &mut out)?; - println!("{}", rollout_summary(&report)); - if !report.ok { - std::process::exit(1); - } - Ok(()) -} - -/// A line per host of a roll-out's report. -pub fn rollout_summary(report: &api::RolloutReport) -> String { - berth_tui::plugins::summary(report).join("\n") -} - -/// `berth plugins update [ID] [--check] [--apply [--reboot [--force]]]`. -pub fn update_plugins(paths: &Paths, args: &[&str]) -> Result<()> { - let (mut name, mut check, mut rollout_args) = (None, false, vec![]); - let mut apply = false; - for arg in args { - match *arg { - "--check" => check = true, - "--apply" => apply = true, - "--reboot" | "--force" | "--update" => rollout_args.push(*arg), - word if !word.starts_with('-') && name.is_none() => name = Some(word), - other => return Err(format!("Not an option of berth plugins update: {other}").into()), - } - } - if !rollout_args.is_empty() && !apply { - return Err(format!("{} goes with --apply", rollout_args[0]).into()); - } - let manager = berth_core::plugins::Manager::new(paths); - if check { - let checked = manager.check()?; - for plugin in checked.plugins { - let state = match (&plugin.latest, plugin.note.as_str()) { - (Some(latest), _) => format!("{} → {latest}", plugin.version), - (None, "") => format!("{} is the latest", plugin.version), - (None, note) => format!("{}: {note}", plugin.version), - }; - println!("{:<16} {state}", plugin.id); - } - return Ok(()); - } - let mut report = manager.update(name)?; - for line in &report.lines { - println!("{line}"); - } - if manager.server().is_none() { - report.behind = berth_core::rollout::survey(paths); - } - for host in &report.behind { - println!("{} needs applying: {}", host.name, host.behind.text()); - } - if !apply { - if !report.behind.is_empty() { - println!("berth apply --behind brings them up to date."); - } - return Ok(()); - } - apply_hosts(paths, &rollout_args) -} - -/// `berth seed HOST` with a berth server: the server renders the seed, and it -/// is written here. -pub fn seed(paths: &Paths, name: &str) -> Result<()> { - let host = Host::load(paths, name)?; - let server = host.served(paths).ok_or("Not a host of a berth server")?; - let seed = act_on(&server, Some(name), "seed", &api::Act::default())? - .result - .unwrap_or_default(); - let dir = paths.build_dir().join(name).join("cloud"); - { - use std::os::unix::fs::DirBuilderExt; - std::fs::DirBuilder::new() - .recursive(true) - .mode(0o700) - .create(&dir)?; - } - let file = dir.join("user-data"); - { - use std::os::unix::fs::OpenOptionsExt; - let mut out = std::fs::OpenOptions::new() - .write(true) - .create(true) - .truncate(true) - .mode(0o600) - .open(&file)?; - out.write_all(seed["user_data"].as_str().unwrap_or("").as_bytes())?; - } - println!("{}", file.display()); - Ok(()) -} - -/// `berth remove HOST`: forget a host. The machine itself is left as it is, -/// and a host with a machine berth created keeps it until that is deleted. -pub fn remove(paths: &Paths, name: &str) -> Result<()> { - let host = Host::load(paths, name).map_err(|_| format!("No host {name}"))?; - if let Some(kept) = berth_core::resource::of_host(paths, name)? - && !kept.record.external_id.is_empty() - { - return Err(format!( - "{name} has a machine berth created, {} of {}. Delete it first: berth resource {} remove", - kept.record.external_id, kept.record.provisioner, kept.record.id - ) - .into()); - } - if provision::busy(paths, name) { - return Err(format!( - "A machine is being created for {name}. Remove it when that has ended" - ) - .into()); - } - if let Some(server) = host.served(paths) { - server.call("hosts.remove", json!({"name": name}), None)?; - } - berth_core::profiles::remove(paths, &host.place)?; - println!("Removed {}", host.place.describe()); - Ok(()) -} - -/// `berth host rename HOST NAME [--hostname]`: give a host another name in -/// berth, with everything berth keeps under its name. The machine stays as it -/// is; with `--hostname` its profile's hostname becomes the new name too, -/// which it takes at its next apply. -pub fn rename(paths: &Paths, name: &str, new: &str, hostname: bool) -> Result<()> { - let host = Host::load(paths, name).map_err(|_| format!("No host {name}"))?; - if hostname && let Some(problem) = berth_tui::setup::hostname_problem(new) { - return Err(problem.into()); - } - let Some(server) = host.served(paths) else { - let place = berth_core::profiles::rename(paths, &host.place, new, hostname)?; - println!("Renamed {name} to {}", place.describe()); - if hostname { - println!("Its hostname is {new} in its profile. Apply it: berth apply {new}"); - } - return Ok(()); - }; - let act = api::Act { - arguments: [ - ("name".to_owned(), new.to_owned()), - ( - "hostname".to_owned(), - if hostname { "yes" } else { "no" }.to_owned(), - ), - ] - .into(), - ..api::Act::default() - }; - let acted = act_on(&server, Some(name), "rename", &act)?; - let code = follow_acted(&server, None, acted, &mut std::io::stdout())?; - if code != 0 { - std::process::exit(code); - } - // What this berth keeps of the server's host, such as its host keys, goes along. - if let Err(error) = berth_core::profiles::rename(paths, &host.place, new, false) { - eprintln!("! {new} is renamed on the server, but not what is kept of it here: {error}"); - } - server.mirror(paths)?; - if hostname { - println!("Apply it to give the machine its new hostname: berth apply {new}"); - } - Ok(()) -} - -/// `berth profile HOST`: the profile as it is kept, as JSON. -pub fn show_profile(paths: &Paths, name: &str) -> Result<()> { - let place = Place::resolve(paths, name)?; - let profile = place - .load(paths)? - .ok_or_else(|| format!("No host {name}"))?; - println!("{}", serde_json::to_string_pretty(&profile)?); - Ok(()) -} - -/// `berth profile HOST --set FILE`: save a profile as the host's, checked as -/// the screen reads one. A host of a berth server is saved on the server, and -/// its copy here marks it as the server's, as a mirror does. -pub fn set_profile(paths: &Paths, name: &str, file: &str) -> Result<()> { - let data = match file { - "-" => { - let mut data = vec![]; - std::io::Read::read_to_end(&mut std::io::stdin(), &mut data)?; - data - } - file => std::fs::read(file).map_err(|error| format!("{file}: {error}"))?, - }; - crate::app::parse_config(&data).map_err(|error| format!("Not a profile: {error}"))?; - let mut profile: Value = serde_json::from_slice(&data)?; - let place = Place::resolve(paths, name)?; - // A profile of an older layout is kept in the one of now. - berth_core::import::convert_profile(paths, &place.title(), &mut profile); - if let Some(url) = place.server_url() { - let server = berth_core::server::Server::configured(paths) - .filter(|server| server.url == url) - .ok_or("The berth server of this host is not set any more")?; - server.save_host(&place.title(), &profile)?; - profile["builder"]["served"] = json!(url); - } - place.save(paths, &profile)?; - println!("Saved {}", place.describe()); - Ok(()) -} - -/// `berth credentials HOST [--show]`: the passwords berth generated for the -/// host's accounts, an account and its password to a line. They go to a -/// terminal, or elsewhere only when asked to with `--show`. -pub fn credentials(paths: &Paths, name: &str, show: bool) -> Result<()> { - let shown = show || std::io::stdout().is_terminal(); - write_credentials(paths, name, shown, &mut std::io::stdout()) -} - -/// The passwords of [`credentials`] to `out`, when they may be `shown` there. -fn write_credentials(paths: &Paths, name: &str, shown: bool, out: &mut dyn Write) -> Result<()> { - if !shown { - return Err( - "Standard output is not a terminal, and the passwords are shown only on one. Add --show to write them there anyway" - .into(), - ); - } - let place = Place::resolve(paths, name)?; - let kept = match place.server_url() { - Some(url) => { - let server = berth_core::server::Server::configured(paths) - .filter(|server| server.url == url) - .ok_or("The berth server of this host is not set any more")?; - eprintln!("{name}'s passwords are kept on the berth server at {url}"); - let answer = server.call("hosts.credentials", json!({"name": name}), None)?; - answer["credentials"] - .as_object() - .cloned() - .unwrap_or_default() - } - None => { - if place.load(paths)?.is_none() { - return Err(format!("No host {name}").into()); - } - berth_core::credentials::load(paths, &place)? - } - }; - if kept.is_empty() { - eprintln!("berth generated no passwords for {name}"); - } - for (user, password) in kept { - writeln!(out, "{user} {}", password.as_str().unwrap_or(""))?; - } - Ok(()) -} - -// ── Adopting a host ───────────────────────────────────────────────────────── - -/// What `berth adopt` is given after the name: `[USER@]ADDRESS`, and -/// `--port PORT`, `--key FILE` and `--force` in any order. Without an -/// address the host is a saved one, reached as its profile says. -#[derive(Debug, Default, PartialEq)] -pub struct Adopt { - pub address: String, - pub port: String, - pub key: Option, - pub force: bool, -} - -pub fn parse_adopt(args: &[&str]) -> Result { - let mut adopt = Adopt::default(); - let mut words = args.iter(); - while let Some(word) = words.next() { - match *word { - "--force" => adopt.force = true, - "--port" | "-p" => adopt.port = words.next().ok_or("--port needs a port")?.to_string(), - "--key" | "-i" => { - adopt.key = Some(words.next().ok_or("--key needs a file")?.to_string()) - } - word if word.starts_with('-') => { - return Err(format!("berth adopt has no option {word}").into()); - } - word if adopt.address.is_empty() => adopt.address = word.to_owned(), - word => return Err(format!("berth adopt takes one address, not also {word}").into()), - } - } - if adopt.address.is_empty() && !adopt.port.is_empty() { - return Err( - "Usage: berth adopt NAME [[USER@]ADDRESS] [--port PORT] [--key FILE] [--force]".into(), - ); - } - Ok(adopt) -} - -/// `berth adopt NAME [[USER@]ADDRESS]`: take a host over from the profile it -/// keeps, read over SSH with sudo so that it works while berthd is down. -/// Without an address it is the saved host NAME, as when berth host add -/// found that a host saved below a workstation is one. -pub fn adopt(paths: &Paths, name: &str, args: &[&str]) -> Result<()> { - use berth_core::kept_profile as kept; - let asked = parse_adopt(args)?; - if !berth_core::profiles::valid_name(name) { - return Err( - format!("{name} cannot be a host's name: letters, digits, -, _ and . only").into(), - ); - } - let server = berth_core::server::Server::configured(paths); - let place = Place::resolve(paths, name)?; - // Refused before anything is read, so the host is not asked for nothing. - let saved = place.load(paths)?; - let reach = match (&saved, asked.address.as_str()) { - (None, "") => { - return Err(format!( - "There is no host {name} to adopt by its own address. Say where it is: berth adopt {name} [USER@]ADDRESS" - ) - .into()); - } - // A host saved below a workstation says less than the host keeps; - // adopting it by its own address is what replacing it means. - (Some(profile), "") => { - if level::of(profile) == level::Level::Workstation && !asked.force { - return Err(format!( - "{name} is saved as a workstation already. Replace its profile with the one it keeps with --force" - ) - .into()); - } - let access = api::profile::Access::of(profile); - if access.link.is_some() || access.address.is_empty() { - return Err(format!( - "{name} has no SSH address to read its profile at: berth adopt {name} [USER@]ADDRESS" - ) - .into()); - } - let given = kept::Reach::parse(&access.address, &access.port, asked.key.as_deref())?; - kept::Reach { - key: given.key, - ..kept::Reach::of(&access) - } - } - (Some(_), _) if !asked.force => { - return Err(format!( - "There is a host {name} already. Choose another name, or replace it with --force" - ) - .into()); - } - (_, address) => kept::Reach::parse(address, &asked.port, asked.key.as_deref())?, - }; - let target = reach.target(); - let password = || -> berth_core::Result { - if let Ok(password) = std::env::var("BERTH_SUDO_PASSWORD") { - return Ok(password); - } - if !std::io::stdin().is_terminal() { - return Err(format!( - "sudo asks for a password on {target}: run on a terminal, or set BERTH_SUDO_PASSWORD" - ) - .into()); - } - registry::read_secret(&format!("Your sudo password on {target}")) - }; - // The login is this machine's, also for a host the berth server keeps. - let read = kept::read(paths, &place, &reach, &password)?; - let access = api::profile::Access { - address: reach.address.clone(), - user: reach.user.clone(), - port: reach.port.clone(), - auth: read.auth.clone(), - // The operator's sudo takes none; another account's may. - sudo: read.sudo_asked.then_some(true), - ..api::profile::Access::default() - }; - let adopted = kept::adopted(paths, &place.title(), &read.kept, &access)?; - let mut left = vec![]; - match &server { - Some(server) => { - server.save_host(name, &adopted.profile)?; - server.mirror(paths)?; - if let Some(record) = &adopted.machine { - left.push(format!( - "the record of its machine, {} of {}: the berth server takes no record, so berth resource does not know the machine", - record.external_id, record.provisioner - )); - } - } - None => kept::save(paths, &place, &adopted)?, - } - let written = match (adopted.berth.as_str(), adopted.written.as_str()) { - ("", _) => String::new(), - (berth, "") => format!(" by berth {berth}"), - (berth, when) => format!(" by berth {berth} at {when}"), - }; - let was = match adopted.host.as_str() { - "" => String::new(), - old if old == name => String::new(), - old => format!(", where it was {old}"), - }; - println!("Adopted {name} from {target}: its profile was written{written}{was}."); - let store = registry::Store::new(paths)?; - let known = store.secret_names(); - for secret in adopted.secrets() { - if !known.contains(&secret) { - left.push(format!( - "the value of secret {secret}: berth secret set {secret}" - )); - } - } - let registries = store.registries().unwrap_or_default(); - for registry in adopted.registries() { - if !registries.contains_key(®istry) { - left.push(format!( - "registry {registry}, with its login: berth registry add {registry} …" - )); - } - } - if berth_api::profile::joins_automatically(&adopted.profile) - && !known - .iter() - .any(|known| known == berth_api::profile::TAILNET_API_KEY) - { - left.push(format!( - "secret {}, which a host that has left the tailnet needs to join again", - berth_api::profile::TAILNET_API_KEY - )); - } - if !left.is_empty() { - println!("What did not come with it, and this berth has not got:"); - for line in &left { - println!(" {line}"); - } - } - let way = match read.auth { - api::profile::Auth::Berth => "with this berth's key".to_owned(), - _ => "with your SSH configuration and agent, until an apply has given the operator this berth's key".to_owned(), - }; - println!("berth logs in to it {way}."); - println!( - "Next: berth apply {name}. It gives this berth's own key a block of its own in the operator's authorized_keys." - ); - println!( - "The berth that set the host up still has access: its key keeps its BERTH block until you delete that block by hand." - ); - if asked.key.is_some() && read.auth != api::profile::Auth::Berth { - println!( - "The key given with --key is not saved. If only it lets you in, give the host this berth's key with it first: berth key trust {name} --key FILE" - ); - } - Ok(()) -} - -// ── Provisioners ──────────────────────────────────────────────────────────── - -/// Provisioners run where the hosts are kept: with a berth server that is -/// the server, which is asked for each of them. -fn provisioning_server(paths: &Paths) -> Option { - berth_core::server::Server::configured(paths) -} - -/// The provisioners there are, here or on the berth server. -fn known_provisioners( - paths: &Paths, - server: Option<&berth_core::server::Server>, -) -> Result> { - match server { - Some(server) => Ok(serde_json::from_value(server.call( - "provisioners.list", - Value::Null, - None, - )?)?), - None => Ok(provision::list(paths, ®istry::Store::new(paths)?)), - } -} - -/// `berth provisioners`: what can create machines, and what each asks for. -pub fn provisioners(paths: &Paths) -> Result<()> { - let found = known_provisioners(paths, provisioning_server(paths).as_ref())?; - if found.is_empty() { - println!( - "No provisioner is registered. Add one: berth provisioner add NAME TYPE, with a type of: berth provisioner types" - ); - } - for provisioner in found { - println!("{:<24} {}", provisioner.id, provisioner.label); - if !provisioner.description.is_empty() { - println!(" {}", provisioner.description); - } - for note in berth_core::plugins::provisioner_notes(&provisioner) { - println!(" · {note}"); - } - } - Ok(()) -} - -/// `berth connect URL`: use the berth server at URL, with the token a setup -/// key collects. The key comes on standard input, so it is in no argv. -pub fn connect(paths: &Paths, url: &str) -> Result<()> { - if !(url.starts_with("https://") || url.starts_with("http://")) { - return Err(format!( - "{url} is no address of a berth server: https://HOST or http://HOST:PORT" - ) - .into()); - } - let key = registry::read_secret("Setup key")?; - connect_with(paths, url, key.trim()) -} - -/// [`connect`] with the key in hand. Nothing is saved unless all of it works. -pub fn connect_with(paths: &Paths, url: &str, key: &str) -> Result<()> { - use berth_core::server::{Server, http_as}; - let url = url.trim().trim_end_matches('/').to_owned(); - let key = key.to_owned(); - let at = format!("{url}/v1/setup-key"); - let ask = |method: &'static str| { - let (at, key) = (at.clone(), key.clone()); - async move { - let (status, body) = http_as(method, &at, Some(&key), None, vec![]).await?; - match status { - 200 => Ok(serde_json::from_slice::(&body)?), - 401 => Err( - "The server did not take the setup key: it is wrong, used up or expired. Make a new one where the server runs: berth setup-key".into(), - ), - 404 | 405 => Err("The server knows no setup keys: it is an older berth".into()), - _ => Err(answer_error(status, &body)), - } - } - }; - let (version, token) = link::runtime()?.block_on(async { - let checked: api::SetupKey = serde_json::from_value(ask("GET").await?)?; - let claimed: api::SetupClaim = serde_json::from_value(ask("POST").await?)?; - // Saved only once the token is seen to open the server. - Server { - url: url.clone(), - token: Some(claimed.token.clone()), - } - .portal() - .await? - .call("access", None, Value::Null, None) - .await?; - Ok::<_, Box>((checked.version, claimed.token)) - })?; - let before = paths.setting("server"); - paths.set("server", &url)?; - if let Err(error) = paths.set("server_token", &token) { - paths.set("server", before.as_deref().unwrap_or(""))?; - return Err(error); - } - if let Err(error) = link::runtime()?.block_on(ask("DELETE")) { - eprintln!( - "The setup key could not be ended ({error}). It stops by itself within a day, or with berth setup-key --remove where the server runs." - ); - } - println!("This berth now uses the berth server at {url}, berth {version}"); - if std::env::var_os("BERTH_SERVER").is_some() { - eprintln!("BERTH_SERVER is set in this shell and is used before the setting"); - } - Ok(()) -} - -/// What an answer that is not a 200 says. -fn answer_error(status: u16, body: &[u8]) -> Box { - let said: Value = serde_json::from_slice(body).unwrap_or(Value::Null); - match said["error"]["message"].as_str() { - Some(message) => format!("HTTP {status}: {message}").into(), - None => format!("HTTP {status}").into(), - } -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn adopt_takes_an_address_and_its_options_in_any_order() { - let adopt = parse_adopt(&["--force", "ops@192.0.2.10", "--port", "2222"]).unwrap(); - assert_eq!( - adopt, - Adopt { - address: "ops@192.0.2.10".into(), - port: "2222".into(), - key: None, - force: true - } - ); - let keyed = parse_adopt(&["192.0.2.10", "--key", "~/.ssh/old"]).unwrap(); - assert_eq!(keyed.key.as_deref(), Some("~/.ssh/old")); - // Without an address it is a saved host, reached as its profile says. - assert_eq!(parse_adopt(&[]).unwrap(), Adopt::default()); - assert!(parse_adopt(&["--port", "2222"]).is_err()); - assert!(parse_adopt(&["192.0.2.10", "192.0.2.11"]).is_err()); - assert!(parse_adopt(&["192.0.2.10", "--port"]).is_err()); - assert!(parse_adopt(&["192.0.2.10", "--user"]).is_err()); - } - - #[test] - fn exec_takes_its_flags_before_the_command() { - let exec = parse_exec(&["--user", "root", "--", "ls", "--keep"]).unwrap(); - assert_eq!((exec.user.as_str(), exec.keep), ("root", false)); - assert_eq!(exec.command, ["ls", "--keep"]); - let exec = parse_exec(&["--keep", "uname", "-a"]).unwrap(); - assert!(exec.keep && exec.user == "agent"); - assert_eq!(exec.command, ["uname", "-a"]); - assert!(parse_exec(&["--user", "nobody", "true"]).is_err()); - assert!(parse_exec(&["--keep"]).is_err()); - } - - #[test] - fn task_arguments_are_names_with_values() { - let (values, yes) = parse_arguments(&["folder=~/my project", "--yes", "empty="]).unwrap(); - assert!(yes); - assert_eq!(values["folder"], "~/my project"); - assert_eq!(values["empty"], ""); - assert!(parse_arguments(&["folder"]).is_err()); - assert!(parse_arguments(&["=x"]).is_err()); - } - - #[test] - fn a_machine_is_added_by_its_address_with_nothing_of_a_workstation() { - let home = tempfile::tempdir().unwrap(); - let paths = Paths { - config_dir: home.path().join("config"), - data_dir: home.path().join("data"), - }; - paths.prepare().unwrap(); - let add = |name: &str, args: &[&str]| add_host(&paths, name, "192.0.2.4", args); - add("dock", &["--user", "ops", "--port", "2222", "--no-contact"]).unwrap(); - let saved = Place::resolve(&paths, "dock") - .unwrap() - .load(&paths) - .unwrap() - .unwrap(); - assert_eq!( - saved, - json!({"level": "reachable", "access": {"address": "192.0.2.4", "user": "ops", "port": "2222", "auth": "berth"}}) - ); - let said = |result: Result<()>| result.unwrap_err().to_string(); - assert!(said(add("dock", &["--no-contact"])).contains("already")); - assert!(said(add("pier", &["--level", "server", "--no-contact"])).contains("--level")); - assert!(said(add("pier", &["--auth", "key:none", "--no-contact"])).contains("no key none")); - assert!(said(add("../pier", &["--no-contact"])).contains("name")); - // A workstation starts with the part its set-up fills in. - add("studio", &["--level", "workstation", "--no-contact"]).unwrap(); - let studio = Host::load(&paths, "studio").unwrap(); - assert!(studio.config["builder"].is_object()); - // Applying is refused below a workstation, before anything runs. - let dock = Host::load(&paths, "dock").unwrap(); - let refused = berth_core::apply::prepare(&paths, &dock, None); - assert!(said(refused.map(|_| ())).contains("not a workstation")); - let refused = install_controller_here(&paths, &studio); - assert!(said(refused.map(|_| ())).contains("berth apply studio")); - } - - #[test] - fn a_host_with_a_machine_berth_created_stays_until_it_is_deleted() { - let home = tempfile::tempdir().unwrap(); - let paths = Paths { - config_dir: home.path().join("config"), - data_dir: home.path().join("data"), - }; - paths.prepare().unwrap(); - let place = Place::own("studio").unwrap(); - place - .save(&paths, &json!({"access": {"address": "192.0.2.20"}})) - .unwrap(); - berth_core::credentials::save(&paths, &place, "ops", "made-up").unwrap(); - let mut machine = berth_core::resource::Kept { - scope: berth_core::store::OWN.into(), - record: berth_core::resource::Record { - id: "r-studio".into(), - provisioner: "cloud/vm".into(), - kind: "machine".into(), - name: "studio".into(), - external_id: "vm-1".into(), - host: "studio".into(), - token: "t".into(), - ..Default::default() - }, - }; - machine.save(&paths).unwrap(); - let said = remove(&paths, "studio").unwrap_err().to_string(); - assert!(said.contains("berth resource r-studio remove"), "{said}"); - assert!(place.load(&paths).unwrap().is_some()); - // One whose creation never ended goes with the host, and its passwords with it. - machine.record.external_id.clear(); - machine.save(&paths).unwrap(); - remove(&paths, "studio").unwrap(); - assert!(place.load(&paths).unwrap().is_none()); - assert!( - berth_core::resource::of_host(&paths, "studio") - .unwrap() - .is_none() - ); - assert!( - berth_core::credentials::load(&paths, &place) - .unwrap() - .is_empty() - ); - } - - #[test] - fn a_profile_that_lists_its_features_keeps_them_without_the_marker() { - let home = tempfile::tempdir().unwrap(); - let paths = Paths { - config_dir: home.path().join("config"), - data_dir: home.path().join("data"), - }; - let file = home.path().join("studio.json"); - std::fs::write( - &file, - r#"{"hostname": "studio", "ansible_vars": {"devbox_plugin_features": ["codex/cli"]}}"#, - ) - .unwrap(); - set_profile(&paths, "studio", file.to_str().unwrap()).unwrap(); - let place = Place::own("studio").unwrap(); - let kept = place.load(&paths).unwrap().unwrap(); - let config = crate::app::parse_config(kept.to_string().as_bytes()).unwrap(); - let mut app = crate::app::App::new(paths.clone(), place.clone(), config); - app.save().unwrap(); - let saved = place.load(&paths).unwrap().unwrap(); - assert_eq!( - saved["ansible_vars"]["devbox_plugin_features"], - json!(["codex/cli"]) - ); - // One of the old layout is still converted. - let old = crate::app::parse_config(br#"{"ansible_vars": {"devbox_agent_codex": false}}"#) - .unwrap(); - assert!( - !old["ansible_vars"]["devbox_plugin_features"] - .as_array() - .unwrap() - .contains(&json!("codex/cli")) - ); - } - - #[test] - fn a_profile_goes_out_and_in_and_passwords_only_to_a_terminal() { - let home = tempfile::tempdir().unwrap(); - let paths = Paths { - config_dir: home.path().join("config"), - data_dir: home.path().join("data"), - }; - let file = home.path().join("studio.json"); - std::fs::write( - &file, - r#"{"hostname": "studio", "builder": {}, "access": {"address": "192.0.2.20"}}"#, - ) - .unwrap(); - set_profile(&paths, "studio", file.to_str().unwrap()).unwrap(); - let place = Place::own("studio").unwrap(); - let kept = place.load(&paths).unwrap().unwrap(); - assert_eq!(kept["access"]["address"], "192.0.2.20"); - // What is not a profile is refused, and the one kept stays. - std::fs::write(&file, r#"{"builder": []}"#).unwrap(); - assert!(set_profile(&paths, "studio", file.to_str().unwrap()).is_err()); - assert_eq!(place.load(&paths).unwrap().unwrap(), kept); - assert!(show_profile(&paths, "attic").is_err()); - show_profile(&paths, "studio").unwrap(); - berth_core::credentials::save(&paths, &place, "ops", "made-up").unwrap(); - let mut out = vec![]; - let said = write_credentials(&paths, "studio", false, &mut out) - .unwrap_err() - .to_string(); - assert!(said.contains("--show") && out.is_empty(), "{said}"); - write_credentials(&paths, "studio", true, &mut out).unwrap(); - assert_eq!(String::from_utf8(out).unwrap(), "ops made-up\n"); - assert!(write_credentials(&paths, "attic", true, &mut vec![]).is_err()); - } - - #[test] - fn hosts_share_one_inventory() { - let home = tempfile::tempdir().unwrap(); - let paths = Paths { - config_dir: home.path().join("config"), - data_dir: home.path().join("data"), - }; - let inventory = |name: &str, access: Value| { - let host = Host { - name: name.into(), - place: Place::own(name).unwrap(), - config: json!({"access": access}), - }; - host.inventory(&paths).unwrap() - }; - let one = inventory("server", json!({"address": "192.0.2.1", "user": "ops"})); - let two = inventory("laptop", json!({"address": "box", "port": "2222"})); - let merged = merge_inventory(&[("server".into(), one), ("laptop".into(), two)]); - assert_eq!(merged["all"]["hosts"]["server"]["ansible_user"], "ops"); - assert_eq!(merged["all"]["hosts"]["laptop"]["ansible_port"], 2222); - } -} diff --git a/crates/berth/src/fields.rs b/crates/berth/src/fields.rs deleted file mode 100644 index 4ef2524..0000000 --- a/crates/berth/src/fields.rs +++ /dev/null @@ -1 +0,0 @@ -pub use berth_tui::fields::*; diff --git a/crates/berth/src/key_command.rs b/crates/berth/src/key_command.rs deleted file mode 100644 index 08e7752..0000000 --- a/crates/berth/src/key_command.rs +++ /dev/null @@ -1,330 +0,0 @@ -//! `berth key …`: the key ring, and the logins that put its keys on hosts. -//! With a berth server set, the ring is the server's and its private halves -//! stay there; first contact and host keys are then the server's to do. - -use crate::Result; -use crate::serve::keys::{KeyInfo, KeyList, listing}; -use berth_core::host::Host; -use berth_core::keys::{Once, contact, ring}; -use berth_core::paths::Paths; -use berth_core::server::Server; -use serde_json::json; -use std::io::Write; - -const USAGE: &str = "Usage: berth key [new NAME | add NAME FILE|- | remove NAME | show [NAME] | rotate [NAME] | trust HOST [--key FILE|NAME] [--keep-password] | host HOST [--accept]]"; - -/// The keys of this installation's ring. -pub fn local(paths: &Paths) -> Result> { - Ok(listing(paths)?.keys) -} - -/// The keys of a berth server's ring. -pub fn listed(server: &Server) -> Result> { - let listed: KeyList = serde_json::from_value(server.call("keys.list", json!({}), None)?)?; - Ok(listed.keys) -} - -fn describe(keys: &[KeyInfo], out: &mut dyn Write) -> Result<()> { - for key in keys { - writeln!(out, "{}\t{}\t{}", key.name, key.fingerprint, key.public)?; - if !key.hosts.is_empty() { - writeln!(out, "\tused by {}", key.hosts.join(", "))?; - } - if let Some(next) = &key.pending { - writeln!( - out, - "\tbeing replaced by {}; berth key rotate {} goes on", - next.fingerprint, key.name - )?; - } - } - Ok(()) -} - -/// A private key from a file, or from standard input for `-`. -fn read_private(file: &str) -> Result> { - match file { - "-" => { - let mut data = vec![]; - std::io::Read::read_to_end(&mut std::io::stdin(), &mut data)?; - Ok(data) - } - file => std::fs::read(file).map_err(|error| format!("{file}: {error}").into()), - } -} - -/// `berth key …`. -pub fn command(paths: &Paths, args: &[&str]) -> Result<()> { - let server = Server::configured(paths); - let out = &mut std::io::stdout(); - match (args, &server) { - ([], Some(server)) => describe(&listed(server)?, out), - ([], None) => describe(&local(paths)?, out), - (["show"], _) => show(paths, server.as_ref(), berth_api::profile::BERTH_KEY), - (["show", name], _) => show(paths, server.as_ref(), name), - (["new", name], Some(server)) => { - let key = server.call("keys.add", json!({"name": name}), None)?; - say_added(&serde_json::from_value(key)?, Some(server)) - } - (["new", name], None) => { - let store = berth_core::store::shared(paths)?; - let key = ring::add(paths, &store, name, None)?; - say_added(&from_public(key), None) - } - (["add", name, file], Some(server)) => { - let data = json!({"base64": berth_api::b64(&read_private(file)?)}); - let key = server.call("keys.add", json!({"name": name}), Some(data))?; - say_added(&serde_json::from_value(key)?, Some(server)) - } - (["add", name, file], None) => { - let store = berth_core::store::shared(paths)?; - let key = ring::add(paths, &store, name, Some(&read_private(file)?))?; - say_added(&from_public(key), None) - } - (["remove", name], Some(server)) => { - server.call("keys.remove", json!({"name": name}), None)?; - println!( - "Removed the key {name} from the berth server at {}", - server.url - ); - Ok(()) - } - (["remove", name], None) => { - let store = berth_core::store::shared(paths)?; - ring::remove(paths, &store, name)?; - println!("Removed the key {name}"); - Ok(()) - } - (["rotate", rest @ ..], _) if rest.len() <= 1 => { - let name = rest - .first() - .copied() - .unwrap_or(berth_api::profile::BERTH_KEY); - match &server { - Some(server) => { - let act = berth_api::Act { - arguments: [("name".to_owned(), name.to_owned())].into(), - ..berth_api::Act::default() - }; - let acted = crate::commands::act_on(server, None, "rotate-key", &act)?; - let code = crate::commands::follow_server_job(server, acted, out)?; - std::process::exit(code); - } - None => { - let rotation = contact::rotate(paths, name, out)?; - if !rotation.finished { - std::process::exit(1); - } - Ok(()) - } - } - } - (["trust", host, rest @ ..], _) => trust(paths, host, rest), - (["host", host], _) => host_key(paths, host, false), - (["host", host, "--accept"], _) => host_key(paths, host, true), - _ => Err(USAGE.into()), - } -} - -fn from_public(key: ring::Public) -> KeyInfo { - KeyInfo { - name: key.name, - public: key.public, - fingerprint: key.fingerprint, - pending: None, - hosts: vec![], - } -} - -fn say_added(key: &KeyInfo, server: Option<&Server>) -> Result<()> { - match server { - Some(server) => println!( - "Added the key {} to the berth server at {}: {}", - key.name, server.url, key.fingerprint - ), - None => println!("Added the key {}: {}", key.name, key.fingerprint), - } - println!("{}", key.public); - Ok(()) -} - -/// `berth key show [NAME]`: the public half alone, for an `authorized_keys`. -fn show(paths: &Paths, server: Option<&Server>, name: &str) -> Result<()> { - let keys = match server { - Some(server) => listed(server)?, - None => local(paths)?, - }; - let key = keys - .iter() - .find(|key| key.name == name) - .ok_or_else(|| format!("No key {name}. See: berth key"))?; - println!("{}", key.public); - Ok(()) -} - -/// The one-time credential of a first contact: a private key from a file, -/// or what is typed without echo or comes on standard input, which is a -/// private key when it looks like one and a password otherwise. -fn credential(paths: &Paths, host: &str, key_file: Option<&str>) -> Result { - let text = match key_file { - Some(file) => read_private(file)?, - None => berth_core::registry::read_secret(&format!( - "A password, or a private key on standard input, that logs in to {host} once" - ))? - .into_bytes(), - }; - if String::from_utf8_lossy(&text).contains("PRIVATE KEY-----") { - return Ok(Once::Key(ring::validate(paths, &text)?)); - } - Ok(Once::Password( - String::from_utf8(text).map_err(|_| "A password is text")?, - )) -} - -fn local_host(paths: &Paths, name: &str) -> Result { - let host = Host::load(paths, name)?; - if let Some(server) = host.served(paths) { - return Err(format!( - "{name} is a host of the berth server at {}, which logs in to it with its own keys", - server.url - ) - .into()); - } - Ok(host) -} - -/// `berth key trust HOST`: first contact. -pub(crate) fn trust(paths: &Paths, name: &str, rest: &[&str]) -> Result<()> { - let mut key_file = None; - let mut keep = false; - let mut rest = rest.iter(); - while let Some(arg) = rest.next() { - match *arg { - "--key" => key_file = Some(*rest.next().ok_or("--key needs a file")?), - "--keep-password" => keep = true, - _ => return Err(USAGE.into()), - } - } - let host = Host::load(paths, name)?; - if let Some(server) = host.served(paths) { - return trust_served(&server, name, key_file, keep); - } - let host = local_host(paths, name)?; - // A login the host refused a short while ago is not tried first again: - // the one-time credential is asked for at once. - let refused = berth_core::keys::login_of(paths, &host) - .ok() - .and_then(|login| login.refusal()) - .is_some_and(|refusal| !refusal.penalty); - let first = match refused { - true => Err("refused before; berth key trust".into()), - false => contact::trust_key(paths, &host, None, false), - }; - let trusted = match first { - Ok(trusted) => trusted, - // Refused the key, it needs a login once; anything else is said as it is. - Err(error) if !error.to_string().contains("berth key trust") => return Err(error), - Err(_) => { - let once = credential(paths, name, key_file)?; - contact::trust_key(paths, &host, Some(once), keep)? - } - }; - match trusted.already { - true => println!("{name} takes the key {} already", trusted.key), - false => println!( - "{name} takes the key {} now: {}", - trusted.key, trusted.fingerprint - ), - } - for key in &trusted.host_keys { - println!("Recorded its host key: {}", ring::fingerprint(key)); - } - Ok(()) -} - -/// First contact for a host of a berth server, which logs in to it: its -/// key is tried first, and when the host refuses it a password typed here -/// logs in once. With a server `--key` names a key of the server's ring. -fn trust_served(server: &Server, name: &str, key: Option<&str>, keep: bool) -> Result<()> { - let run = |password: Option| -> Result<(i32, String)> { - let mut arguments = std::collections::BTreeMap::from([( - "keep_password".to_owned(), - if keep { "yes" } else { "no" }.to_owned(), - )]); - if let Some(key) = key { - arguments.insert("key".into(), key.into()); - } - if let Some(password) = password { - arguments.insert("password".into(), password); - } - let act = berth_api::Act { - arguments, - ..berth_api::Act::default() - }; - let acted = crate::commands::act_on(server, Some(name), "trust-key", &act)?; - let mut said = vec![]; - let code = crate::commands::follow_server_job(server, acted, &mut said)?; - let said = String::from_utf8_lossy(&said).into_owned(); - print!("{said}"); - Ok((code, said)) - }; - let (mut code, said) = run(None)?; - // Refused the key, it needs a login once. - if code != 0 && key.is_none() && said.contains("berth key trust") { - let password = - berth_core::registry::read_secret(&format!("A password that logs in to {name} once"))?; - code = run(Some(password))?.0; - } - match code { - 0 => Ok(()), - _ => Err(format!("{name} does not take the server's key").into()), - } -} - -/// `berth key host HOST [--accept]`. -fn host_key(paths: &Paths, name: &str, accept: bool) -> Result<()> { - let host = Host::load(paths, name)?; - let keys: Vec<(String, String)> = match host.served(paths) { - Some(server) => { - let act = berth_api::Act { - arguments: [( - "accept".to_owned(), - if accept { "yes" } else { "no" }.to_owned(), - )] - .into(), - ..berth_api::Act::default() - }; - let acted = crate::commands::act_on(&server, Some(name), "host-key", &act)?; - let result = acted.result.unwrap_or_default(); - result["host_keys"] - .as_array() - .into_iter() - .flatten() - .map(|key| { - let text = |field: &str| key[field].as_str().unwrap_or("").to_owned(); - (text("type"), text("fingerprint")) - }) - .collect() - } - None => { - let host = local_host(paths, name)?; - contact::host_key(paths, &host, accept)? - .iter() - .map(|key| { - let kind = key.split_whitespace().next().unwrap_or("").to_owned(); - (kind, ring::fingerprint(key)) - }) - .collect() - } - }; - if keys.is_empty() { - println!("berth has no host key of {name} yet; the first login records it"); - } - for (kind, fingerprint) in keys { - println!("{kind}\t{fingerprint}"); - } - if accept { - println!("Accepted the host key {name} has now"); - } - Ok(()) -} diff --git a/crates/berth/src/keys.rs b/crates/berth/src/keys.rs deleted file mode 100644 index d5837e3..0000000 --- a/crates/berth/src/keys.rs +++ /dev/null @@ -1,194 +0,0 @@ -//! A terminal's keys and mouse as the shared screens take them, and back. - -use berth_tui::{Click, Code, Event, Key, Mouse}; -use crossterm::event::{ - Event as Native, KeyCode, KeyEvent, KeyEventKind, KeyModifiers, MouseButton, MouseEventKind, -}; - -pub fn key(key: &KeyEvent) -> Option { - let code = match key.code { - KeyCode::Char(ch) => Code::Char(ch), - KeyCode::Enter => Code::Enter, - KeyCode::Esc => Code::Esc, - KeyCode::Tab => Code::Tab, - KeyCode::BackTab => Code::BackTab, - KeyCode::Backspace => Code::Backspace, - KeyCode::Delete => Code::Delete, - KeyCode::Insert => Code::Insert, - KeyCode::Up => Code::Up, - KeyCode::Down => Code::Down, - KeyCode::Left => Code::Left, - KeyCode::Right => Code::Right, - KeyCode::Home => Code::Home, - KeyCode::End => Code::End, - KeyCode::PageUp => Code::PageUp, - KeyCode::PageDown => Code::PageDown, - KeyCode::F(number) => Code::F(number), - _ => return None, - }; - Some(Key { - code, - ctrl: key.modifiers.contains(KeyModifiers::CONTROL), - alt: key.modifiers.contains(KeyModifiers::ALT), - shift: key.modifiers.contains(KeyModifiers::SHIFT), - }) -} - -/// The key as the terminal would have reported it. -pub fn native(key: Key) -> KeyEvent { - let code = match key.code { - Code::Char(ch) => KeyCode::Char(ch), - Code::Enter => KeyCode::Enter, - Code::Esc => KeyCode::Esc, - Code::Tab => KeyCode::Tab, - Code::BackTab => KeyCode::BackTab, - Code::Backspace => KeyCode::Backspace, - Code::Delete => KeyCode::Delete, - Code::Insert => KeyCode::Insert, - Code::Up => KeyCode::Up, - Code::Down => KeyCode::Down, - Code::Left => KeyCode::Left, - Code::Right => KeyCode::Right, - Code::Home => KeyCode::Home, - Code::End => KeyCode::End, - Code::PageUp => KeyCode::PageUp, - Code::PageDown => KeyCode::PageDown, - Code::F(number) => KeyCode::F(number), - }; - let mut modifiers = KeyModifiers::NONE; - for (on, modifier) in [ - (key.ctrl, KeyModifiers::CONTROL), - (key.alt, KeyModifiers::ALT), - (key.shift, KeyModifiers::SHIFT), - ] { - if on { - modifiers |= modifier; - } - } - KeyEvent::new(code, modifiers) -} - -/// How soon a key has to follow Esc to be read as that key with Alt. A -/// terminal sends Alt+R as Esc and `r` together, but a relay on the way, as -/// between Windows and WSL, can pass them on in two writes. -const ALT_WAIT: std::time::Duration = std::time::Duration::from_millis(25); - -thread_local! { - /// An event read while waiting to see whether Esc was Alt. - static AHEAD: std::cell::RefCell> = const { std::cell::RefCell::new(None) }; -} - -/// Whether an event is waiting, for up to `timeout`. -pub fn poll(timeout: std::time::Duration) -> std::io::Result { - if AHEAD.with_borrow(Option::is_some) { - return Ok(true); - } - crossterm::event::poll(timeout) -} - -/// The next event of the terminal, with Esc and a character that come right -/// after it taken as that character with Alt. -pub fn read() -> std::io::Result { - if let Some(event) = AHEAD.take() { - return Ok(event); - } - read_from(crossterm::event::read, || crossterm::event::poll(ALT_WAIT)) -} - -fn read_from( - mut read: impl FnMut() -> std::io::Result, - mut soon: impl FnMut() -> std::io::Result, -) -> std::io::Result { - let event = read()?; - let escape = matches!(&event, Native::Key(key) - if key.code == KeyCode::Esc && key.modifiers.is_empty() && key.kind == KeyEventKind::Press); - if !escape || !soon()? { - return Ok(event); - } - let next = read()?; - if let Native::Key(key) = &next - && let KeyCode::Char(_) = key.code - && key.kind == KeyEventKind::Press - && (key.modifiers - KeyModifiers::SHIFT).is_empty() - { - return Ok(Native::Key(KeyEvent::new_with_kind( - key.code, - key.modifiers | KeyModifiers::ALT, - key.kind, - ))); - } - AHEAD.set(Some(next)); - Ok(event) -} - -/// The event, when it is one the screens take: a key going down, a click, the -/// wheel, a paste. -pub fn event(event: &Native) -> Option { - match event { - Native::Key(pressed) if pressed.kind != KeyEventKind::Release => { - key(pressed).map(Event::Key) - } - Native::Mouse(mouse) => { - let click = match mouse.kind { - MouseEventKind::Down(MouseButton::Left) => Click::Left, - MouseEventKind::Down(MouseButton::Right) => Click::Right, - MouseEventKind::ScrollUp => Click::ScrollUp, - MouseEventKind::ScrollDown => Click::ScrollDown, - _ => return None, - }; - Some(Event::Mouse(Mouse { - click, - column: mouse.column, - row: mouse.row, - })) - } - Native::Paste(text) => Some(Event::Paste(text.clone())), - _ => None, - } -} - -#[cfg(test)] -mod tests { - use super::*; - - fn press(code: KeyCode, modifiers: KeyModifiers) -> Native { - Native::Key(KeyEvent::new(code, modifiers)) - } - - /// What reading gives when the terminal has `events`, each after the - /// first coming soon after the one before when `soon` is set. - fn reads(events: &[Native], soon: bool) -> Vec { - let queue = std::cell::RefCell::new(events.to_vec()); - let mut got = vec![]; - while AHEAD.with_borrow(Option::is_some) || !queue.borrow().is_empty() { - let event = match AHEAD.take() { - Some(event) => event, - None => read_from( - || Ok(queue.borrow_mut().remove(0)), - || Ok(soon && !queue.borrow().is_empty()), - ) - .unwrap(), - }; - got.push(event); - } - got - } - - #[test] - fn esc_and_a_letter_right_after_it_are_alt_and_the_letter() { - let esc = press(KeyCode::Esc, KeyModifiers::NONE); - let r = press(KeyCode::Char('r'), KeyModifiers::NONE); - let alt_r = press(KeyCode::Char('r'), KeyModifiers::ALT); - assert_eq!(reads(&[esc.clone(), r.clone()], true), [alt_r]); - let shifted = press(KeyCode::Char('R'), KeyModifiers::SHIFT); - let alt_shift_r = press(KeyCode::Char('R'), KeyModifiers::SHIFT | KeyModifiers::ALT); - assert_eq!(reads(&[esc.clone(), shifted], true), [alt_shift_r]); - // A letter typed later is a key of its own. - assert_eq!(reads(&[esc.clone(), r.clone()], false), [esc.clone(), r]); - // So is anything but a letter, and it is not lost. - let up = press(KeyCode::Up, KeyModifiers::NONE); - assert_eq!(reads(&[esc.clone(), up.clone()], true), [esc.clone(), up]); - let ctrl_c = press(KeyCode::Char('c'), KeyModifiers::CONTROL); - assert_eq!(reads(&[esc.clone(), ctrl_c.clone()], true), [esc, ctrl_c]); - } -} diff --git a/crates/berth/src/local/mod.rs b/crates/berth/src/local/mod.rs new file mode 100644 index 0000000..d510ef6 --- /dev/null +++ b/crates/berth/src/local/mod.rs @@ -0,0 +1,290 @@ +//! What berth does on the machine its installation is on, which is Linux: +//! `berth serve`, the server the screens and commands use when no other is +//! set, and what only makes sense where the server runs: its setup key, its +//! store, its API documents, and the commands that need the installation's +//! own files, such as `berth ssh` with its key. + +pub mod serve; +mod store_command; + +use crate::Result; +use crate::cli::args::{self, Global}; +use crate::cli::fail::{Fail, Result as Done}; +use crate::cli::{client, out}; +use berth_core::{host::Host, paths::Paths}; +use serde_json::{Value, json}; + +/// The server of this machine: a `berth serve` of this user, or one in this +/// process, whose jobs the second part waits for. With `ask`, a host whose +/// sudo asks for a password has it asked for on the terminal when the +/// server starts, as `berth serve` does. +pub fn server(ask: bool) -> Result> { + let (server, here) = serve::here::local(&Paths::locate()?, ask)?; + if let Some(here) = &here { + berth_page::net::serve_here(here.dial()); + } + Ok(Some((server, Box::new(move || serve::here::finish(here))))) +} + +/// Give the server in this process a host's sudo password for this run, as +/// `berth serve` takes one on its terminal at the start. Whether there is +/// such a server. +pub fn give_sudo_password(name: &str, password: &str) -> bool { + serve::here::give_sudo_password(name, password) +} + +/// ssh runs berth as its askpass helper, the program of SSH_ASKPASS, with +/// the prompt as its one argument and the variable that says where the +/// password is; the answer is all that may go to standard output. Whether +/// this run was one. +pub fn askpass(args: &[String]) -> Result { + let Ok(source) = std::env::var(berth_core::keys::askpass::SOURCE) else { + return Ok(false); + }; + let prompt = match args { + [_, word, prompt @ ..] if word == "askpass" => prompt.join(" "), + [_, prompt] => prompt.clone(), + _ => return Ok(false), + }; + let answer = berth_core::keys::askpass::answer(&prompt, &source, None)?; + println!("{answer}"); + Ok(true) +} + +/// Put back the plugins berth's store has and this machine's plugin +/// directory lacks, as on a machine restored from the store, and say so on +/// standard error. +pub(crate) fn restore_plugins(paths: &Paths) { + match berth_core::plugin_state::restore(paths) { + Ok(notices) => notices.iter().for_each(|notice| eprintln!("{notice}")), + Err(error) => eprintln!("berth: the plugins in berth's store were not checked: {error}"), + } +} + +/// What only works with the installation on this machine is refused when +/// berth is set to use a server elsewhere. +fn here_only(global: &Global, what: &str) -> Done { + if let Some(server) = client::named(global) + && !berth_page::device::local(&server) + { + return Err(Fail::refused(format!( + "{what} works where the berth server runs, and this berth uses the one at {server}. Run it there" + ))); + } + Ok(Paths::locate()?) +} + +/// `berth ssh HOST [--agent] [-- COMMAND…]`: plain SSH with the +/// installation's key, for a host whose controller cannot be reached. +pub fn ssh(global: &Global, ssh: &args::Ssh) -> Done<()> { + let paths = here_only( + global, + "berth ssh, which logs in with the server's own key,", + ) + .map_err(|fail| { + Fail::refused(format!( + "{} berth session attach {} reaches a host through any server", + fail.message, ssh.host + )) + })?; + let host = Host::load(&paths, &ssh.host)?; + if !host.access().reachable() { + return Err(Fail::failed(format!("{} has no SSH address yet", ssh.host))); + } + let mut command = host.ssh(&paths)?; + command.arg(host.target()); + if ssh.agent { + // Root login and the agent's own key are not a given, so it goes + // through the SSH user and sudo; agent-env adds the display and tools. + let agent = host.agent(); + if agent.starts_with('-') + || !agent + .chars() + .all(|ch| ch.is_ascii_alphanumeric() || ch == '_' || ch == '-') + { + return Err(Fail::failed("The agent's account name is not a valid one")); + } + command.arg("-t").arg(format!( + "sudo -iu {agent} sh -c 'command -v agent-env >/dev/null && exec agent-env; exec bash -l'" + )); + } + command.args(&ssh.command); + let status = command.status()?; + match status.code() { + Some(0) => Ok(()), + code => Err(Fail::exit(code.unwrap_or(1))), + } +} + +/// Several hosts' inventories as one. +pub fn merge_inventory(hosts: &[(String, Value)]) -> Value { + let mut all = serde_json::Map::new(); + for (name, inventory) in hosts { + all.insert( + name.clone(), + inventory["all"]["hosts"]["devbox_target"].clone(), + ); + } + json!({"all": {"hosts": all}}) +} + +/// `berth host inventory [NAME…]`: the hosts as an Ansible inventory, with +/// the installation's key as a file of this machine. +pub fn inventory(global: &Global, names: &[String]) -> Done<()> { + let paths = here_only(global, "berth host inventory")?; + let hosts = match names { + [] => Host::all(&paths) + .into_iter() + .filter(|host| !host.address().is_empty()) + .collect(), + names => names + .iter() + .map(|name| Host::load(&paths, name)) + .collect::>>()?, + }; + let mut entries = vec![]; + for host in hosts { + let inventory = host + .inventory(&paths) + .map_err(|error| format!("{}: {error}", host.name))?; + entries.push((host.name, inventory)); + } + out::json(&merge_inventory(&entries)); + Ok(()) +} + +/// The enrollment key an apply asks for, for a host not on the tailnet yet; +/// it goes to the role on standard output. +pub fn tailnet_key(host: &str) -> Done<()> { + let paths = Paths::locate()?; + println!("{}", berth_core::tailnet::key_for(&paths, host)?); + Ok(()) +} + +/// `berth registry publish`: upload the release archive built here, and +/// make it the release address of a host through the server. +pub fn publish( + global: &Global, + destination: &str, + host: Option<&str>, + login: Option<&str>, +) -> Done<()> { + let paths = here_only( + global, + "berth registry publish, which builds the release here,", + )?; + std::env::set_current_dir(paths.source()?)?; + let url = berth_core::registry::publish(&paths, destination)?; + out::done(format!("Published {url}")); + let Some(host) = host else { + return Ok(()); + }; + let client = client::Client::open(global)?; + let mut profile = crate::cli::host_profile(&client, host)?; + profile["release_url"] = json!(url); + profile["release_store"] = json!(destination); + let login = login.map(|name| match name { + "" => destination.split('/').next().unwrap_or(""), + name => name, + }); + profile["release_private"] = json!(login.is_some()); + match login { + Some(name) => { + profile["release_login"] = json!(name); + out::done(format!( + "{host} downloads it from there with the login of {name}" + )); + } + None => { + if let Some(profile) = profile.as_object_mut() { + profile.remove("release_login"); + } + out::done(format!("{host} downloads it from there without a login")); + } + } + crate::cli::save_host(&client, host, &profile) +} + +/// `berth registry push-image`: push berth's container image from here. +pub fn push_image(global: &Global, destination: &str, tag: &str) -> Done<()> { + let paths = here_only(global, "berth registry push-image, which pushes from here,")?; + let image = berth_core::registry::push_image(&paths, destination, tag)?; + out::done(format!("Pushed {image}")); + Ok(()) +} + +/// What `berth serve --web` says, as berth 0.5 had a page in the browser. +pub const WEB_GONE: &str = "--web does nothing any more: berth has no page in the browser, and its screens are berth ui. Take --web out of this command line"; + +/// `berth server …`: the server of this machine itself. +pub fn server_command(command: args::Server) -> Done<()> { + let paths = Paths::locate()?; + match command { + args::Server::Run(run) => { + let mut words = vec![]; + if let Some(listen) = &run.listen { + words.extend(["--listen", listen.as_str()]); + } + if run.expose { + words.push("--expose"); + } + if run.web { + out::warn(WEB_GONE); + } + Ok(serve::command(&paths, &words)?) + } + args::Server::SetupKey { revoke } => { + let words: &[&str] = if revoke { &["--remove"] } else { &[] }; + Ok(serve::setup_key::command(&paths, words)?) + } + args::Server::Store(store) => { + let target; + let file; + let words: Vec<&str> = match &store { + args::Store::Status => vec![], + args::Store::Encrypt => vec!["encrypt"], + args::Store::Rekey => vec!["rekey"], + args::Store::Decrypt => { + crate::cli::ask::confirm("This writes the store's values plain again.")?; + vec!["decrypt", "--yes"] + } + args::Store::Copy { + target: given, + target_file, + into_existing, + } => { + let mut words = vec!["copy"]; + match (given, target_file) { + (Some(given), None) => { + target = given.clone(); + words.push(&target); + } + (None, Some(path)) => { + file = path.display().to_string(); + words.extend(["--target-file", file.as_str()]); + } + _ => { + return Err(Fail::usage( + "berth server store copy takes a TARGET or --target-file FILE", + )); + } + } + if *into_existing { + words.push("--into-existing"); + } + words + } + }; + Ok(store_command::command(&paths, &words)?) + } + args::Server::Api(api) => { + let document = match api { + args::Api::Openapi => serde_json::to_value(serve::openapi())?, + args::Api::Asyncapi => serve::asyncapi(), + args::Api::Operations => serve::operations(), + }; + out::json(&document); + Ok(()) + } + } +} diff --git a/crates/berth/src/serve/actions.rs b/crates/berth/src/local/serve/actions.rs similarity index 50% rename from crates/berth/src/serve/actions.rs rename to crates/berth/src/local/serve/actions.rs index db4cb00..40fb0a2 100644 --- a/crates/berth/src/serve/actions.rs +++ b/crates/berth/src/local/serve/actions.rs @@ -1,12 +1,12 @@ //! The actions of a host and of the installation: what can be done to each, -//! and running one. A host's actions are the tasks its controller lists and +//! and running one. A host's actions are the actions its controller lists and //! berth's own from [`HOST`]; the installation's are [`INSTALLATION`]. An //! action that changes something runs as a job; one the host runs is a //! session there, which a job follows unless it needs a terminal. use super::{ Remote, Serve, - api::{Answer, App, Fail, caller, internal, job_info, remote, report_path}, + api::{Answer, App, Fail, Said, caller, internal, job_info, remote, report_path, turned_down}, auth::Access, setup::{own_place, saved_profile}, }; @@ -18,14 +18,14 @@ use axum::{ }; use berth_api::profile::{self, Auth}; use berth_api::{ - Act, Acted, Action, Argument, ErrorBody, Rollout, RolloutReport, Runs, Session, Task, b64, - client, code, ops::Role, + Act, Acted, Action, Argument, ErrorBody, HostAction, Rollout, RolloutReport, Runs, Session, + b64, client, code, ops::Role, }; use berth_core::{ apply, host::Host, + host_type::{self, Capability, HostType}, keys::{self, Once, contact, ring}, - level::{self, Capability, Level}, profiles::Place, registry::Store, rollout, @@ -44,20 +44,31 @@ pub(super) struct Seen<'a> { /// The host's profile; null for the installation. pub profile: &'a Value, /// What the host's controller lists; `None` when it did not answer. - pub tasks: Option<&'a [Task]>, + pub host_actions: Option<&'a [HostAction]>, /// The names of the secrets the server keeps. pub secrets: &'a [String], - /// The level the host's profile means it to have. - pub level: Level, + /// The type the host's profile means it to be. + pub host_type: HostType, /// What its controller says it is, when it answered. - pub observed: Option, + pub observed: Option, /// Why its controller did not answer, when it was asked and did not. pub unanswered: Option<&'a str>, + /// That its controller is too old for this berth, and how to update it, + /// when that is why it did not answer. + pub older: Option<&'a str>, + /// The caller is on the machine the server runs on: in its process, or + /// on its socket. + pub local: bool, } /// berth's own actions that run through the host's controller, so the host /// is asked before one of them is described. -const THROUGH_CONTROLLER: &[&str] = &["secrets"]; +const THROUGH_CONTROLLER: &[&str] = &["secrets", UPGRADE_SERVER]; + +/// berth's own action that upgrades the berth server in a container on a +/// node. It is offered only where the node's controller lists it, which its +/// role does when it found such a container. +const UPGRADE_SERVER: &str = "upgrade-server"; /// One of berth's own actions. pub(super) struct Builtin { @@ -106,6 +117,16 @@ pub(super) static HOST: &[Builtin] = &[ describe: rename_action, run: rename, }, + Builtin { + name: UPGRADE_SERVER, + describe: upgrade_server_action, + run: upgrade_server, + }, + Builtin { + name: "installer", + describe: installer_action, + run: installer, + }, ]; /// The actions of the installation itself. @@ -120,6 +141,11 @@ pub(super) static INSTALLATION: &[Builtin] = &[ describe: rotate_key_action, run: rotate_key, }, + Builtin { + name: "adopt", + describe: adopt_action, + run: adopt, + }, ]; fn unavailable(mut action: Action, why: Option) -> Action { @@ -148,43 +174,78 @@ pub(super) fn answer(acted: Acted) -> (StatusCode, Json) { (status, Json(acted)) } -/// The tasks a host's controller lists, asked for with a time limit. -async fn host_tasks(remote: &Remote) -> Result, String> { - let link = remote - .link - .as_ref() - .ok_or_else(|| remote.problem.clone().unwrap_or_else(|| "no link".into()))?; - match tokio::time::timeout(Duration::from_secs(15), link.get::("/v1/tasks")).await { - Ok(Ok(tasks)) => Ok(berth_tui::host::list(tasks, "tasks")), - Ok(Err(error)) => Err(error.to_string()), - Err(_) => Err("no answer in time".into()), +/// Why a host's controller did not list its actions. +enum Unlisted { + /// It did not answer. + Unanswered(String), + /// It is older than this berth: what to say, with how to update it. + Older(String), +} + +impl Unlisted { + fn said(self) -> String { + match self { + Unlisted::Unanswered(said) | Unlisted::Older(said) => said, + } } } +/// The actions a host's controller lists, asked for with a time limit. A +/// controller that does not know the call is asked its version, so that one +/// of an older berth is said to be old rather than broken. +async fn controller_actions( + remote: &Remote, + host_type: HostType, +) -> Result, Unlisted> { + let link = remote.link.as_ref().ok_or_else(|| { + Unlisted::Unanswered(remote.problem.clone().unwrap_or_else(|| "no link".into())) + })?; + let error = + match tokio::time::timeout(Duration::from_secs(15), link.get::("/v1/actions")).await + { + Ok(Ok(listed)) => return Ok(berth_tui::host::list(listed, "actions")), + Ok(Err(error)) => error.to_string(), + Err(_) => return Err(Unlisted::Unanswered("no answer in time".into())), + }; + Err(match remote.version().await { + Ok(version) => berth_api::compat::older_controller( + &remote.name, + &version.version, + version.api, + host_type, + ) + .map_or(Unlisted::Unanswered(error), Unlisted::Older), + Err(_) => Unlisted::Unanswered(error), + }) +} + /// What a host's controller says the host is, when it answers. -async fn observed(remote: &Remote) -> Option { +async fn observed(remote: &Remote) -> Option { remote .version() .await .ok() .as_ref() - .and_then(|version| level::seen(Ok(version))) + .and_then(|version| host_type::seen(Ok(version))) } -/// A host's actions: its tasks, then berth's own. A task that takes the +/// A host's actions: its controller's, then berth's own. One that takes the /// workstation's secrets on its input is berth's to run, with the server's. fn host_list(seen: &Seen) -> Vec { + let listed = seen.host_actions.unwrap_or(&[]); let own: Vec = HOST .iter() + .filter(|builtin| { + builtin.name != UPGRADE_SERVER || listed.iter().any(|kept| kept.name == UPGRADE_SERVER) + }) .map(|builtin| (builtin.describe)(seen)) .collect(); - let tasks = seen.tasks.unwrap_or(&[]); - let mut actions: Vec = Action::of_tasks(tasks) + let mut actions: Vec = Action::of_host(listed) .into_iter() .filter(|action| !own.iter().any(|known| known.name == action.name)) .filter(|action| { - let task = tasks.iter().find(|task| task.name == action.name); - task.is_none_or(|task| task.input.is_none()) + let kept = listed.iter().find(|kept| kept.name == action.name); + kept.is_none_or(|kept| kept.input.is_none()) }) .collect(); actions.extend(own); @@ -239,45 +300,49 @@ fn no_action(subject: &str, action: &str, actions: &[Action]) -> Fail { ) } -#[utoipa::path(get, path = "/v1/hosts/{name}/actions", summary = "What can be done to a host: the tasks its controller lists, and berth's own actions.", tag = "hosts", +#[utoipa::path(get, path = "/v1/hosts/{name}/actions", summary = "What can be done to a host: the actions its controller lists, and berth's own.", tag = "hosts", params(("name" = String, Path, description = "A saved host")), responses((status = 200, description = "The actions, each saying whether it can run now", body = Vec), (status = 404, description = "No such host", body = ErrorBody)))] pub async fn host_actions( State(state): App, + access: Option>, Path(name): Path, ) -> Answer>> { state.refresh(false); let remote = remote(&state, &name)?; let profile = profile_of(&state, &name); - // A host that is only reachable has no controller to ask. - let asked = match level::of(&profile) { - Level::Reachable => None, - _ => Some(host_tasks(&remote).await), + // A host of the type ssh has no controller to ask. + let asked = match host_type::of(&profile) { + HostType::Ssh => None, + host_type => Some(controller_actions(&remote, host_type).await), }; - let (tasks, unanswered) = match &asked { - Some(Ok(tasks)) => (Some(tasks.as_slice()), None), - Some(Err(error)) => (None, Some(error.as_str())), - None => (None, None), + let (listed, unanswered, older) = match &asked { + Some(Ok(listed)) => (Some(listed.as_slice()), None, None), + Some(Err(Unlisted::Unanswered(error))) => (None, Some(error.as_str()), None), + Some(Err(Unlisted::Older(said))) => (None, None, Some(said.as_str())), + None => (None, None, None), }; let secrets = secret_names(&state); let observed = observed(&remote).await; Ok(Json(host_list(&Seen { name: &name, profile: &profile, - tasks, + host_actions: listed, secrets: &secrets, - level: level::of(&profile), + host_type: host_type::of(&profile), observed, unanswered, + older, + local: access.is_none(), }))) } -#[utoipa::path(post, path = "/v1/hosts/{name}/actions/{action}", summary = "Run one of a host's actions. One of berth's that changes something is a job; a task of the host is a session there, which a job follows unless it needs a terminal.", tag = "hosts", +#[utoipa::path(post, path = "/v1/hosts/{name}/actions/{action}", summary = "Run one of a host's actions. One of berth's that changes something is a job; an action of the host's controller is a session there, which a job follows unless it needs a terminal.", tag = "hosts", params(("name" = String, Path, description = "A saved host"), ("action" = String, Path, description = "The action, as hosts.actions names it")), request_body(content = Option, description = "Its arguments, and a token that makes asking again safe"), - responses((status = 202, description = "Started as a job; a task of the host also says its session", body = Acted), + responses((status = 202, description = "Started as a job; an action of its controller also says its session", body = Acted), (status = 200, description = "Answered at once: the result, or the terminal session to attach to", body = Acted), (status = 400, description = "An argument is missing, or the body is not one", body = ErrorBody), (status = 403, description = "The caller's role is below the action's", body = ErrorBody), @@ -297,49 +362,70 @@ pub async fn host_act( let profile = profile_of(&state, &name); // berth's own are described without asking the host, which they do themselves. let builtin = HOST.iter().find(|builtin| builtin.name == action); - let reachable_only = level::of(&profile) == Level::Reachable; - let mut unanswered = None; - let tasks = match builtin { - Some(_) if reachable_only || !THROUGH_CONTROLLER.contains(&action.as_str()) => None, + let ssh_only = host_type::of(&profile) == HostType::Ssh; + let (mut unanswered, mut older) = (None, None); + let listed = match builtin { + Some(_) if ssh_only || !THROUGH_CONTROLLER.contains(&action.as_str()) => None, // One that runs through the controller is described as the host answers. - Some(_) => match host_tasks(&remote).await { - Ok(tasks) => Some(tasks), - Err(error) => { + Some(_) => match controller_actions(&remote, host_type::of(&profile)).await { + Ok(listed) => Some(listed), + Err(Unlisted::Unanswered(error)) => { unanswered = Some(error); None } + Err(Unlisted::Older(said)) => { + older = Some(said); + None + } }, - None if reachable_only => Some(vec![]), + None if ssh_only => Some(vec![]), None => Some( - host_tasks(&remote) + controller_actions(&remote, host_type::of(&profile)) .await - .map_err(|error| Fail::new(code::UNREACHABLE, error))?, + .map_err(|unlisted| match unlisted { + Unlisted::Older(said) => Fail::new(code::PRECONDITION, said), + unlisted => Fail::new(code::UNREACHABLE, unlisted.said()), + })?, ), }; let secrets = secret_names(&state); - let observed = observed(&remote).await; + // Only installing the controller asks what the host is, which takes a login there. + let observed = match action.as_str() { + "install-controller" => observed(&remote).await, + _ => None, + }; let actions = host_list(&Seen { name: &name, profile: &profile, - tasks: tasks.as_deref(), + host_actions: listed.as_deref(), secrets: &secrets, - level: level::of(&profile), + host_type: host_type::of(&profile), observed, unanswered: unanswered.as_deref(), + older: older.as_deref(), + local: access.is_none(), }); let found = actions .iter() .find(|known| known.name == action) - .ok_or_else(|| no_action(&name, &action, &actions))?; + .ok_or_else(|| match &older { + // The controller's own actions are not known when it is too old to list them. + Some(said) => Fail::new(code::PRECONDITION, said.clone()), + None => no_action(&name, &action, &actions), + })?; + let local = access.is_none(); admit(&state, &caller(access), found, &act)?; + if local && builtin.is_some() { + take_known_host_key(&state, &name); + } let acted = match builtin { Some(builtin) => (builtin.run)(state, name, act).await?, - None => run_task(&state, &remote, found, act, None).await?, + None => run_action(&state, &remote, found, act, None).await?, }; Ok(answer(acted)) } -#[utoipa::path(get, path = "/v1/actions", summary = "What can be done to the installation as a whole, such as a roll-out over its hosts.", tag = "hosts", +#[utoipa::path(get, path = "/v1/installation/actions", summary = "What can be done to the installation as a whole, such as a roll-out over its hosts.", tag = "hosts", responses((status = 200, description = "The actions", body = Vec)))] pub async fn list() -> Json> { Json(installation_list()) @@ -349,11 +435,13 @@ fn installation_list() -> Vec { let seen = Seen { name: "", profile: &Value::Null, - tasks: None, + host_actions: None, secrets: &[], - level: Level::Workstation, + host_type: HostType::Workstation, observed: None, unanswered: None, + older: None, + local: false, }; INSTALLATION .iter() @@ -361,8 +449,8 @@ fn installation_list() -> Vec { .collect() } -#[utoipa::path(post, path = "/v1/actions/{action}", summary = "Run one of the installation's actions, as a job.", tag = "hosts", - params(("action" = String, Path, description = "The action, as actions.list names it")), +#[utoipa::path(post, path = "/v1/installation/actions/{action}", summary = "Run one of the installation's actions, as a job.", tag = "hosts", + params(("action" = String, Path, description = "The action, as installation.actions names it")), request_body(content = Option, description = "Its arguments, and a token that makes asking again safe"), responses((status = 202, description = "Started as a job", body = Acted), (status = 400, description = "An argument is wrong, or the body is not one", body = ErrorBody), @@ -398,7 +486,7 @@ pub(super) fn refused(error: Box) -> Fail { } } -// ── Tasks of a host ───────────────────────────────────────────────────────── +// ── Actions of a host ───────────────────────────────────────────────────────── /// The host's controller through the server's link to it, for a job that /// waits on it from a thread of its own. @@ -414,11 +502,21 @@ impl Controller for Through { .block_on(link.get::(path)) .map_err(|error| error.to_string()) } + + fn stop(&mut self, id: &str) -> Result<(), String> { + let link = self.remote.link.as_ref().ok_or("no link to the host")?; + // TERM, so the session ends as it would and its transcript stays. + let path = format!("/v1/sessions/{}/signal", client::encode(id)); + self.runtime + .block_on(link.post::(&path, &json!({"signal": "TERM"}))) + .map(|_| ()) + .map_err(|error| error.to_string()) + } } -/// Run a task of the host as a session there, with `input` on its standard +/// Run an action of the host as a session there, with `input` on its standard /// input. A job follows one that needs no terminal, copying its output. -async fn run_task( +async fn run_action( state: &Arc, remote: &Arc, action: &Action, @@ -437,7 +535,7 @@ async fn run_task( if let Some(input) = input { body["input_b64"] = json!(b64(format!("{input}\n").as_bytes())); } - let path = format!("/v1/tasks/{}", client::encode(&action.name)); + let path = format!("/v1/actions/{}", client::encode(&action.name)); // One to attach to is the session alone, with no job to find again. if action.terminal { let session: Session = link.post(&path, &body).await?; @@ -447,8 +545,10 @@ async fn run_task( }); } let mut request = Request::new(Subject::Host(remote.name.clone()), &action.name); + request.arguments = Argument::shown(&action.arguments, &act.arguments); request.token = act.token; - // A task is the host's own session, and runs beside the others there. + request.role = action.role; + // An action is the host's own session, and runs beside the others there. request.changes = false; // The job is written first, so that of the callers with one token only // the one that wrote it starts a session; the others find its job. @@ -509,7 +609,7 @@ fn apply_action(seen: &Seen) -> Action { "Run the installer role against the host again, from the server, with the settings of its profile: for example after changing what is installed. It also installs or updates berth's own tools on the host. It runs as a job of the server.", ) }; - let action = level::fit(action, seen.name, seen.level, Capability::Apply); + let action = host_type::fit(action, seen.name, seen.host_type, Capability::Apply); logs_in_alone(seen, action) } @@ -534,7 +634,7 @@ fn logs_in_alone(seen: &Seen, action: Action) -> Action { } fn install_controller_action(seen: &Seen) -> Action { - let action = level::install_controller_action(seen.name, seen.level, seen.observed); + let action = host_type::install_controller_action(seen.name, seen.host_type, seen.observed); let action = match seen.profile.is_null() { true => unavailable( action, @@ -548,8 +648,7 @@ fn install_controller_action(seen: &Seen) -> Action { fn install_controller(state: Arc, name: String, act: Act) -> Run { Box::pin(async move { let remote = remote(&state, &name)?; - let bad = - |error: Box| Fail::new(code::BAD_REQUEST, error.to_string()); + let bad = |error: Box| turned_down(error); let host = own_place(&name).and_then(|place| Host::load_at(&state.paths, place).map_err(bad))?; let password = act @@ -580,8 +679,7 @@ fn install_controller(state: Arc, name: String, act: Act) -> Run { fn apply(state: Arc, name: String, act: Act) -> Run { Box::pin(async move { let remote = remote(&state, &name)?; - let bad = - |error: Box| Fail::new(code::BAD_REQUEST, error.to_string()); + let bad = |error: Box| turned_down(error); let host = own_place(&name).and_then(|place| Host::load_at(&state.paths, place).map_err(bad))?; let password = act @@ -595,6 +693,7 @@ fn apply(state: Arc, name: String, act: Act) -> Run { )); } let prepared = apply::prepare(&state.paths, &host, password.as_deref()).map_err(bad)?; + let notes = prepared.notes; let mut request = Request::new(Subject::Host(name.clone()), "apply"); request.token = act.token; request.secrets = password.into_iter().collect(); @@ -605,10 +704,13 @@ fn apply(state: Arc, name: String, act: Act) -> Run { }; let paths = state.paths.clone(); let work = Work::Function(Box::new(move |out| { + for note in ¬es { + let _ = writeln!(out, "{note}"); + } if berth_core::runner::run_program(program, out) != Some(0) { return false; } - if let Err(error) = apply::settle(&paths, &host) { + if let Err(error) = apply::settle_with(&paths, &host, out) { let _ = writeln!( out, "! {name} is applied, but its profile was not updated: {error}" @@ -633,7 +735,7 @@ fn seed_action(seen: &Seen) -> Action { "Render the cloud-init user data that installs the host on a new machine, for a provider's field of that name. It answers with the seed.", ) }; - level::fit(action, seen.name, seen.level, Capability::Apply) + host_type::fit(action, seen.name, seen.host_type, Capability::Seed) } fn seed(state: Arc, name: String, _: Act) -> Run { @@ -641,44 +743,240 @@ fn seed(state: Arc, name: String, _: Act) -> Run { let (paths, registries) = (state.paths.clone(), Store::own(state.store.clone())); // It may publish the release and runs the seed builder, and waits for both. let seed = tokio::task::spawn_blocking(move || { - berth_core::seed::cloud(&paths, ®istries, &name, false) - .map_err(|error| error.to_string()) + berth_core::seed::cloud(&paths, ®istries, &name, false).map_err(Said::from) }) .await .map_err(internal)? - .map_err(|error| Fail::new(code::BAD_REQUEST, error))?; + .map_err(turned_down)?; + // A seed's key joins the tailnet for a day, which whoever starts the machine needs to know. + Ok(Acted { + result: Some(json!({"user_data": seed.user_data, "tailnet": seed.tailnet})), + ..Acted::default() + }) + }) +} + +/// Why an action that works with this machine's own drives and files is +/// not offered to a caller elsewhere. +fn only_here(what: &str) -> String { + format!( + "{what} is done on the machine berth serve runs on: there, with berth ui or berth host iso" + ) +} + +fn installer_action(seen: &Seen) -> Action { + // The drives are looked for only for a caller who can write them. + let drives = match seen.local { + true => berth_core::discovery::usb_drives(), + false => vec![], + }; + let kept = seen.profile["builder"]["usb_device"].as_str().unwrap_or(""); + let default = match (kept, drives.as_slice()) { + ("", [(_, only)]) => only.clone(), + (kept, _) => kept.to_owned(), + }; + let found = match drives.is_empty() { + true => "No USB drive is connected now.".to_owned(), + false => format!( + "Connected now: {}.", + drives + .iter() + .map(|(label, _)| label.as_str()) + .collect::>() + .join("; ") + ), + }; + let action = Action { + arguments: vec![ + Argument { + name: "device".into(), + label: "USB drive to write it to".into(), + default, + help: format!( + "The whole USB drive the finished ISO is written to, such as /dev/sdb. It is erased. Empty builds the ISO only. {found}" + ), + ..Argument::default() + }, + Argument { + name: "password".into(), + label: "Your sudo password, to write the drive".into(), + help: "Writing the drive runs as root through sudo. Empty when sudo needs none or no drive is written.".into(), + secret: true, + ..Argument::default() + }, + ], + ..set_up( + "installer", + "Build the USB installer", + "For a blank machine: build an installer image from Ubuntu Server's ISO that installs and sets up the host without questions, and write it to a USB drive. Without an ISO in the profile the current one is downloaded and checked first. Booting the drive erases the machine's chosen disk.", + ) + }; + let action = host_type::fit(action, seen.name, seen.host_type, Capability::Apply); + match (seen.local, seen.profile.is_null()) { + (false, _) => unavailable(action, Some(only_here("Building the USB installer"))), + (_, true) => unavailable( + action, + Some(format!("{} has no profile on the server", seen.name)), + ), + _ => action, + } +} + +fn installer(state: Arc, name: String, act: Act) -> Run { + Box::pin(async move { + let bad = |error: Box| turned_down(error); + let text = |key: &str| act.arguments.get(key).map_or("", |value| value.trim()); + let device = text("device").to_owned(); + let password = act.arguments.get("password").cloned().unwrap_or_default(); + // Only a whole USB drive that is connected now is written, as the script checks again. + if !device.is_empty() + && !berth_core::discovery::usb_drives() + .iter() + .any(|(_, path)| *path == device) + { + return Err(Fail::new( + code::BAD_REQUEST, + format!("{device} is no USB drive connected to this machine"), + )); + } + let profile = profile_of(&state, &name); + let ubuntu = berth_core::seed::ubuntu_iso(&profile); + if let Some(iso) = &ubuntu + && !iso.is_file() + { + return Err(Fail::new( + code::PRECONDITION, + format!("The profile's ISO {} is not there", iso.display()), + )); + } + // The host must be one an installer can be built for before a job starts. + let paths = state.paths.clone(); + let fetch = berth_core::seed::fetch_iso(&paths).map_err(bad)?; + let mut request = Request::new(Subject::Host(name.clone()), "installer"); + request.token = act.token; + request.secrets = Some(password.clone()) + .filter(|password| !password.is_empty()) + .into_iter() + .collect(); + let work = Work::Function(Box::new(move |out| { + let ubuntu = match ubuntu { + Some(iso) => iso, + None => { + let _ = writeln!(out, "Downloading Ubuntu's installer ISO…"); + match fetched(fetch, out) { + Some(iso) => iso, + None => return false, + } + } + }; + let built = match berth_core::seed::installer(&paths, &name, &ubuntu) { + Ok(built) => built, + Err(error) => { + let _ = writeln!(out, "✗ {error}"); + return false; + } + }; + // The builder reads both files while it runs. + let (iso, _profile, key) = (built.iso.clone(), built.profile, built.key); + let program = berth_core::runner::Program { + command: built.command, + input: None, + files: vec![], + }; + if berth_core::runner::run_program(program, out) != Some(0) { + return false; + } + if let Some(key) = &key { + let _ = writeln!(out, "{}", key.note()); + } + if device.is_empty() { + let _ = writeln!(out, "Built {}", iso.display()); + return true; + } + let write = + match berth_core::seed::write_usb(&paths, &iso, &device, !password.is_empty()) { + Ok(write) => write, + Err(error) => { + let _ = writeln!(out, "✗ {error}"); + return false; + } + }; + let program = berth_core::runner::Program { + command: write, + input: (!password.is_empty()).then(|| format!("{password}\n").into_bytes()), + files: vec![], + }; + let written = berth_core::runner::run_program(program, out) == Some(0); + if written { + let _ = writeln!( + out, + "Next: boot the machine from the USB drive. Booting it erases the chosen disk." + ); + } + written + })); + let job = state.runner.start(request, work).map_err(refused)?; Ok(Acted { - result: Some(json!({"user_data": seed.user_data})), + job: Some(job_info(&state.paths, job)), ..Acted::default() }) }) } +/// Download Ubuntu's ISO with the builder: where it is, which it says last. +fn fetched( + command: std::process::Command, + out: &mut berth_core::runner::Output, +) -> Option { + let mut command = command; + let output = match command.stderr(std::process::Stdio::piped()).output() { + Ok(output) => output, + Err(error) => { + let _ = writeln!(out, "✗ python3: {error}"); + return None; + } + }; + let _ = out.write_all(&output.stderr); + let said = String::from_utf8_lossy(&output.stdout); + let iso = said + .lines() + .last() + .map(|line| std::path::PathBuf::from(line.trim())); + match (output.status.success(), iso) { + (true, Some(iso)) if iso.is_file() => Some(iso), + _ => { + let _ = writeln!(out, "✗ Downloading Ubuntu's installer ISO failed"); + None + } + } +} + fn secrets_action(seen: &Seen) -> Action { - let task = seen - .tasks - .and_then(|tasks| tasks.iter().find(|task| task.name == "secrets")); + let kept = seen + .host_actions + .and_then(|listed| listed.iter().find(|kept| kept.name == "secrets")); let mut action = set_up( "secrets", "Send secrets", "Send the host the secrets and registry logins its profile names, from the server's own.", ); action.runs = Runs::Host; - if let Some(task) = task { - action.label = task.label.clone(); - action.help = task.help.clone(); - action.group = task.group.clone(); + if let Some(kept) = kept { + action.label = kept.label.clone(); + action.help = kept.help.clone(); + action.group = kept.group.clone(); } - if let Err(refusal) = level::check(seen.name, seen.level, Capability::Secrets) { + if let Err(refusal) = host_type::check(seen.name, seen.host_type, Capability::Secrets) { return unavailable(action, Some(refusal.why)); } - let why = match (seen.tasks, task, seen.unanswered) { + let why = match (seen.host_actions, kept, seen.unanswered) { + _ if seen.older.is_some() => seen.older.map(str::to_owned), (_, _, Some(error)) => Some(format!( "The controller on {} does not answer, and the secrets go through it: {error}", seen.name )), (Some(_), None, _) => Some(format!( - "The controller on {} has no secrets task yet: apply its configuration", + "The controller on {} has no secrets action yet: apply its configuration", seen.name )), _ => None, @@ -689,8 +987,7 @@ fn secrets_action(seen: &Seen) -> Action { fn secrets(state: Arc, name: String, act: Act) -> Run { Box::pin(async move { let remote = remote(&state, &name)?; - let bad = - |error: Box| Fail::new(code::BAD_REQUEST, error.to_string()); + let bad = |error: Box| turned_down(error); let host = Host::load_at(&state.paths, own_place(&name)?).map_err(bad)?; let payload = Store::own(state.store.clone()) .host_payload(&host.config) @@ -699,7 +996,70 @@ fn secrets(state: Arc, name: String, act: Act) -> Run { name: "secrets".into(), ..Action::default() }; - run_task(&state, &remote, &action, act, Some(payload.to_string())).await + run_action(&state, &remote, &action, act, Some(payload.to_string())).await + }) +} + +fn upgrade_server_action(seen: &Seen) -> Action { + let kept = seen + .host_actions + .and_then(|listed| listed.iter().find(|kept| kept.name == UPGRADE_SERVER)); + let mut action = set_up( + UPGRADE_SERVER, + "Upgrade the berth server", + "Run the berth server in the container on this node from the image of this berth's version, with the same data and the same options as the running container. It is refused when the container has an option it cannot carry over, and the old container is started again when the new one does not come up. A server that upgrades itself restarts during the job.", + ); + action.group = "Server".into(); + action.runs = Runs::Host; + action.confirm = Some(format!( + "Upgrade the berth server on {} to {}? It restarts, and its jobs end", + seen.name, + berth_api::VERSION + )); + action.arguments = vec![Argument { + name: "version".into(), + label: "Version".into(), + help: "The version of berth to run, as its image is tagged: this berth's own unless another is given.".into(), + default: berth_api::VERSION.into(), + ..Argument::default() + }]; + if let Some(kept) = kept { + action.help = match kept.help.is_empty() { + true => action.help, + false => kept.help.clone(), + }; + } + let why = match (seen.host_type, seen.unanswered) { + (HostType::Node, _) if seen.older.is_some() => seen.older.map(str::to_owned), + (HostType::Node, Some(error)) => Some(format!( + "The controller on {} does not answer, and the upgrade runs through it: {error}", + seen.name + )), + (HostType::Node, None) => None, + (other, _) => Some(format!( + "{} is a host of the type {other}: only a node's controller upgrades a berth server it runs. A server of berth 0.5, and one on no node, is updated by its recipe: {}", + seen.name, + berth_api::compat::UPGRADE_DOCS + )), + }; + unavailable(action, why) +} + +fn upgrade_server(state: Arc, name: String, mut act: Act) -> Run { + Box::pin(async move { + let remote = remote(&state, &name)?; + let version = act + .arguments + .get("version") + .map(|version| version.trim().to_owned()) + .filter(|version| !version.is_empty()) + .unwrap_or_else(|| berth_api::VERSION.to_owned()); + act.arguments = [("version".to_owned(), version)].into(); + let action = Action { + name: UPGRADE_SERVER.into(), + ..Action::default() + }; + run_action(&state, &remote, &action, act, None).await }) } @@ -717,6 +1077,10 @@ fn flag(name: &str, label: &str, help: &str) -> Argument { } fn rollout_action(_: &Seen) -> Action { + rollout_declared() +} + +fn rollout_declared() -> Action { Action { name: "rollout".into(), label: "Roll out".into(), @@ -730,8 +1094,14 @@ fn rollout_action(_: &Seen) -> Action { help: "Host names, separated by commas, in the order to take them. All saved hosts when empty.".into(), ..Argument::default() }, + Argument { + name: "except".into(), + label: "Leave out".into(), + help: "Host names, separated by commas, that the roll-out leaves alone.".into(), + ..Argument::default() + }, flag("all", "Also current hosts", "Apply the hosts that are current too."), - flag("update", "Update", "Run each host's update task after applying."), + flag("update", "Update", "Run each host's update action after applying."), flag("reboot", "Reboot", "Reboot a host that says it needs it, and wait for it to come back."), flag("force", "Force", "Reboot although sessions run, which ends them."), ], @@ -745,13 +1115,21 @@ fn rollout(state: Arc, _: String, act: Act) -> Run { let options = Rollout::from_arguments(&act.arguments) .map_err(|error| Fail::new(code::BAD_REQUEST, error))?; let known: Vec = state.hosts.read().unwrap().keys().cloned().collect(); - let names = rollout::chosen(known, &options.hosts) + if let Some(unknown) = options.except.iter().find(|name| !known.contains(name)) { + return Err(Fail::new( + code::NOT_FOUND, + format!("No host {unknown} to leave out"), + )); + } + let mut names = rollout::chosen(known, &options.hosts) .map_err(|error| Fail::new(code::NOT_FOUND, error.to_string()))?; + names.retain(|name| !options.except.contains(name)); let passwords: Vec> = names .iter() .map(|name| state.host(name).and_then(|remote| remote.password.clone())) .collect(); let mut request = Request::new(Subject::Installation, "rollout"); + request.arguments = Argument::shown(&rollout_declared().arguments, &act.arguments); request.host = names.join(", "); request.token = act.token; request.secrets = passwords.iter().flatten().cloned().collect(); @@ -763,9 +1141,9 @@ fn rollout(state: Arc, _: String, act: Act) -> Run { for (name, password) in names.iter().zip(passwords) { let host = berth_core::profiles::Place::own(name) .and_then(|place| Host::load_at(&paths, place)); - // Only a workstation is applied; the others are said and left. + // Only a node or a workstation is applied; the others are said and left. let host = match host { - Ok(host) if level::of(&host.config) != Level::Workstation => { + Ok(host) if !Capability::Apply.allowed(host_type::of(&host.config)) => { skipped.push(host.name); continue; } @@ -829,6 +1207,36 @@ fn given<'a>(act: &'a Act, name: &str) -> Option<&'a str> { .filter(|value| !value.is_empty()) } +/// For a caller on the server's machine, take the host's key from the +/// known-hosts file of the user the server runs as, when berth has none of +/// it yet: that user said yes to it already, so a first login asks nothing. +/// A caller elsewhere sends none of its files; the key the host shows at its +/// first login is recorded and printed, and `berth host verify` shows it. +fn take_known_host_key(state: &Serve, name: &str) { + let Some(known_hosts) = state.known_hosts.lock().unwrap().clone() else { + return; + }; + let Ok(place) = own_place(name) else { + return; + }; + let Ok(host) = Host::load_at(&state.paths, place.clone()) else { + return; + }; + let taken = berth_core::keys::known::take_from_user( + &state.paths, + &state.store, + &place, + &host.config, + &known_hosts, + ); + if let Err(error) = taken { + eprintln!( + "berth: {name}'s host key was not taken from {}: {error}", + known_hosts.display() + ); + } +} + /// What logs in to a host first: it needs only a profile, not a controller. fn first_contact(seen: &Seen, mut action: Action) -> Action { action.group = "Access".into(); @@ -884,7 +1292,10 @@ fn trust_key(state: Arc, name: String, act: Act) -> Run { let store = berth_core::store::shared(&state.paths).map_err(internal)?; let halves = ring::read(&store, key, false).map_err(internal)?; let halves = halves.ok_or_else(|| { - Fail::new(code::NOT_FOUND, format!("No key {key}. See: berth key")) + Fail::new( + code::NOT_FOUND, + format!("No key {key}. See: berth key list"), + ) })?; Some(Once::Key(halves)) } @@ -971,7 +1382,7 @@ fn host_key(state: Arc, name: String, act: Act) -> Run { // ── Its name ──────────────────────────────────────────────────────────────── fn rename_action(seen: &Seen) -> Action { - let action = level::rename_action(); + let action = host_type::rename_action(); let why = seen .profile .is_null() @@ -1066,3 +1477,309 @@ fn rotate_key(state: Arc, _: String, act: Act) -> Run { super::keys::start_rotation(&state, name, act.token) }) } + +// ── Adopting a host ───────────────────────────────────────────────────────── + +fn adopt_action(_: &Seen) -> Action { + let text = |name: &str, label: &str, help: &str| Argument { + name: name.into(), + label: label.into(), + help: help.into(), + ..Argument::default() + }; + Action { + name: "adopt".into(), + label: "Adopt a host".into(), + help: "Take in a host another berth set up, from the profile it keeps, read over SSH with sudo so that it works while berthd is down. Without an address it is the saved host of that name, reached as its profile says. It runs as a job.".into(), + group: "Hosts".into(), + role: Role::Admin, + arguments: vec![ + Argument { + required: true, + ..text("name", "Name", "The host's name here.") + }, + text("address", "Address", "[USER@]ADDRESS to read the profile at; empty for the saved host of that name."), + text("port", "Port", "The SSH port, with an address."), + text("key", "Key file", "A private key file where the server runs, offered as well for reading the profile only."), + flag("force", "Replace", "Replace a host of that name, or a saved workstation's profile."), + ], + ..Action::default() + } +} + +/// What adopting a host is to log in to, and where its profile goes. +fn adopt_reach( + paths: &berth_core::paths::Paths, + name: &str, + act: &Act, +) -> Result<(Place, berth_core::kept_profile::Reach), String> { + use berth_core::kept_profile as kept; + let force = act + .arguments + .get("force") + .is_some_and(|value| value == "yes" || value == "true"); + let address = given(act, "address").unwrap_or(""); + let port = given(act, "port").unwrap_or(""); + let key = given(act, "key"); + let place = Place::own(name).map_err(|error| error.to_string())?; + let saved = place.load(paths).map_err(|error| error.to_string())?; + let reach = match (&saved, address) { + (None, "") => { + return Err(format!( + "There is no host {name} to adopt by its own address. Say where it is: berth host adopt {name} [USER@]ADDRESS" + )); + } + // A host saved below a workstation says less than the host keeps; + // adopting it by its own address is what replacing it means. + (Some(profile), "") => { + if host_type::of(profile) == HostType::Workstation && !force { + return Err(format!( + "{name} is saved as a workstation already. Replace its profile with the one it keeps with --force" + )); + } + let access = profile::Access::of(profile); + if access.link.is_some() || access.address.is_empty() { + return Err(format!( + "{name} has no SSH address to read its profile at: berth host adopt {name} [USER@]ADDRESS" + )); + } + let given = kept::Reach::parse(&access.address, &access.port, key) + .map_err(|error| error.to_string())?; + kept::Reach { + key: given.key, + ..kept::Reach::of(&access) + } + } + (Some(_), _) if !force => { + return Err(format!( + "There is a host {name} already. Choose another name, or replace it with --force" + )); + } + (_, address) => { + kept::Reach::parse(address, port, key).map_err(|error| error.to_string())? + } + }; + Ok((place, reach)) +} + +fn adopt(state: Arc, _: String, act: Act) -> Run { + Box::pin(async move { + use berth_core::kept_profile as kept; + let name = given(&act, "name").unwrap_or_default().to_owned(); + let (place, reach) = adopt_reach(&state.paths, &name, &act) + .map_err(|error| Fail::new(code::BAD_REQUEST, error))?; + let mut request = Request::new(Subject::Host(name.clone()), "adopt"); + request.token = act.token.clone(); + request.secrets = act.sudo_password.iter().cloned().collect(); + let (serve, sudo) = (state.clone(), act.sudo_password.clone()); + let work = Work::Function(Box::new(move |out| { + let paths = &serve.paths; + let target = reach.target(); + let password = || -> berth_core::Result { + sudo.clone().ok_or_else(|| { + format!( + "sudo asks for a password on {target}: give it with --sudo-password-stdin" + ) + .into() + }) + }; + let adopted = kept::read(paths, &place, &reach, &password).and_then(|read| { + let access = profile::Access { + address: reach.address.clone(), + user: reach.user.clone(), + port: reach.port.clone(), + auth: read.auth.clone(), + // The operator's sudo takes none; another account's may. + sudo: read.sudo_asked.then_some(true), + ..profile::Access::default() + }; + let adopted = kept::adopted(paths, &place.title(), &read.kept, &access)?; + kept::save(paths, &place, &adopted)?; + Ok((read, adopted)) + }); + let (read, adopted) = match adopted { + Ok(done) => done, + Err(error) => { + let _ = writeln!(out, "✗ {error}"); + return false; + } + }; + serve.forget(&name); + serve.refresh(false); + let written = match (adopted.berth.as_str(), adopted.written.as_str()) { + ("", _) => String::new(), + (berth, "") => format!(" by berth {berth}"), + (berth, when) => format!(" by berth {berth} at {when}"), + }; + let was = match adopted.host.as_str() { + old if old.is_empty() || old == name => String::new(), + old => format!(", where it was {old}"), + }; + let _ = writeln!( + out, + "✓ Adopted {name} from {target}: its profile was written{written}{was}" + ); + let store = Store::own(serve.store.clone()); + let known = store.secret_names(); + let mut left = vec![]; + for secret in adopted.secrets() { + if !known.contains(&secret) { + left.push(format!( + "the value of secret {secret}: berth secret set {secret}" + )); + } + } + let registries = store.registries().unwrap_or_default(); + for registry in adopted.registries() { + if !registries.contains_key(®istry) { + left.push(format!( + "registry {registry}, with its login: berth registry add {registry} …" + )); + } + } + if profile::joins_tailnet(&adopted.profile) + && !known.iter().any(|known| known == profile::TAILNET_API_KEY) + { + left.push(format!( + "secret {}, which a host that has left the tailnet needs to join again", + profile::TAILNET_API_KEY + )); + } + if !left.is_empty() { + let _ = writeln!( + out, + "What did not come with it, and this berth has not got:" + ); + for line in &left { + let _ = writeln!(out, " {line}"); + } + } + let way = match read.auth { + Auth::Berth => "with this berth's key".to_owned(), + _ => "with the SSH configuration and agent of the server's account, until an apply has given the operator this berth's key".to_owned(), + }; + let _ = writeln!(out, "berth logs in to it {way}."); + let _ = writeln!( + out, + "Next: berth apply {name}. It gives this berth's own key a block of its own in the operator's authorized_keys." + ); + let _ = writeln!( + out, + "The berth that set the host up still has access: its key keeps its BERTH block until you delete that block by hand." + ); + true + })); + let job = state.runner.start(request, work).map_err(refused)?; + Ok(Acted { + job: Some(job_info(&state.paths, job)), + ..Acted::default() + }) + }) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn listed(names: &[&str]) -> Vec { + names + .iter() + .map(|name| HostAction { + name: (*name).into(), + ..HostAction::default() + }) + .collect() + } + + fn seen<'a>( + host_type: HostType, + listed: Option<&'a [HostAction]>, + profile: &'a Value, + ) -> Seen<'a> { + Seen { + name: "big", + profile, + host_actions: listed, + secrets: &[], + host_type, + observed: None, + unanswered: None, + older: None, + local: false, + } + } + + #[test] + fn upgrading_the_server_is_offered_where_the_node_has_its_container() { + let profile = json!({"type": "node", "access": {"address": "192.0.2.20", "user": "root"}}); + let find = |actions: &[Action]| { + actions + .iter() + .find(|action| action.name == UPGRADE_SERVER) + .cloned() + }; + // A node whose role found no berth server's container lists none. + let without = listed(&["status", "update"]); + assert!(find(&host_list(&seen(HostType::Node, Some(&without), &profile))).is_none()); + let with = listed(&["status", UPGRADE_SERVER]); + let actions = host_list(&seen(HostType::Node, Some(&with), &profile)); + let upgrade = find(&actions).unwrap(); + assert!(upgrade.available, "{}", upgrade.why); + assert_eq!(upgrade.arguments[0].default, berth_api::VERSION); + assert_eq!(upgrade.runs, Runs::Host); + // Listed once: berth's own takes the place of the controller's. + assert_eq!( + actions + .iter() + .filter(|action| action.name == UPGRADE_SERVER) + .count(), + 1 + ); + // A controller that does not answer cannot run it. + let mut quiet = seen(HostType::Node, Some(&with), &profile); + quiet.unanswered = Some("no answer in time"); + assert!(!upgrade_server_action(&quiet).available); + // Only a node's controller does it. + let managed = upgrade_server_action(&seen(HostType::Managed, Some(&with), &profile)); + assert!(!managed.available && managed.why.contains("only a node's controller")); + assert!(managed.why.contains(berth_api::compat::UPGRADE_DOCS)); + } + + #[test] + fn a_controller_that_is_too_old_is_said_to_be_old() { + let profile = + json!({"type": "managed", "access": {"address": "192.0.2.12", "user": "root"}}); + let said = + berth_api::compat::older_controller("big", "0.5.2", 1, HostType::Managed).unwrap(); + let mut old = seen(HostType::Managed, None, &profile); + old.older = Some(&said); + let actions = host_list(&old); + let secrets = actions + .iter() + .find(|action| action.name == "secrets") + .unwrap(); + assert!(!secrets.available); + assert_eq!(secrets.why, said); + assert!( + secrets.why.ends_with("berth run big install-controller"), + "{}", + secrets.why + ); + let install = actions + .iter() + .find(|action| action.name == "install-controller") + .unwrap(); + assert!(install.available, "{}", install.why); + } + + #[test] + fn a_node_is_applied_and_makes_no_seed() { + let profile = json!({"type": "node", "access": {"address": "192.0.2.20", "user": "root"}}); + let actions = host_list(&seen(HostType::Node, Some(&[]), &profile)); + let find = |name: &str| actions.iter().find(|action| action.name == name).unwrap(); + assert!(find("apply").available, "{}", find("apply").why); + assert!(!find("seed").available); + assert!(find("seed").why.contains("not a workstation")); + assert!(!find("install-controller").available); + } +} diff --git a/crates/berth/src/serve/api.rs b/crates/berth/src/local/serve/api.rs similarity index 80% rename from crates/berth/src/serve/api.rs rename to crates/berth/src/local/serve/api.rs index 1695864..30d54ee 100644 --- a/crates/berth/src/serve/api.rs +++ b/crates/berth/src/local/serve/api.rs @@ -20,7 +20,7 @@ use axum::{ routing::{any, get}, }; use berth_api::{ - Applied, Behind, Done, ErrorBody, ErrorDetail, HostBehind, HostLevel, JobInfo, Message, Output, + Applied, Behind, Done, ErrorBody, ErrorDetail, HostBehind, HostType, JobInfo, Message, Output, Version, b64, client, code, ops::{self, Role}, streams, @@ -56,13 +56,24 @@ impl Fail { impl IntoResponse for Fail { fn into_response(self) -> Response { let status = StatusCode::from_u16(self.0).unwrap_or(StatusCode::INTERNAL_SERVER_ERROR); + let code = axum::http::HeaderValue::from_str(&self.1).ok(); + let message = + axum::http::HeaderValue::from_str(&berth_api::error_message_header(&self.2)).ok(); let body = ErrorBody { error: ErrorDetail { code: self.1, message: self.2, }, }; - (status, Json(body)).into_response() + let mut response = (status, Json(body)).into_response(); + if let Some(code) = code { + response.headers_mut().insert(berth_api::ERROR_HEADER, code); + } + if let Some(message) = message { + let headers = response.headers_mut(); + headers.insert(berth_api::ERROR_MESSAGE_HEADER, message); + } + response } } @@ -91,6 +102,50 @@ pub(super) fn internal(error: impl std::fmt::Display) -> Fail { Fail::new(code::INTERNAL, error.to_string()) } +/// What the core said when it did not do something: its words, and whether +/// it refused before anything began. It crosses threads, as an error of the +/// core does not. +#[derive(Debug)] +pub(super) struct Said { + pub refused: bool, + pub message: String, +} + +impl From> for Said { + fn from(error: Box) -> Said { + Said { + refused: berth_core::is_refused(error.as_ref()), + message: error.to_string(), + } + } +} + +impl From for Said { + fn from(message: String) -> Said { + Said { + refused: false, + message, + } + } +} + +impl std::fmt::Display for Said { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(&self.message) + } +} + +/// A request the core turned down: refused before anything began is a +/// failed precondition, which berth's commands exit 3 for; anything else is +/// a request that does not fit. +pub(super) fn turned_down(said: impl Into) -> Fail { + let said = said.into(); + match said.refused { + true => Fail::new(code::PRECONDITION, said.message), + false => Fail::new(code::BAD_REQUEST, said.message), + } +} + // ── What the answers look like ────────────────────────────────────────────── #[derive(Serialize, ToSchema)] @@ -115,13 +170,14 @@ pub struct HostInfo { /// What changed since then, when the server would apply something else now. #[serde(skip_serializing_if = "Option::is_none")] pub behind: Option, - /// The level its profile means it to have. - pub level: HostLevel, - /// The level it was seen to have: what its controller's record says, - /// `reachable` when berth logged in and found no controller. Absent - /// when it could not be asked. + /// The type its profile means it to be. + #[serde(rename = "type")] + pub host_type: HostType, + /// The type it was seen to be: what its controller's record says, `ssh` + /// when berth logged in and found no controller. Absent when it could + /// not be asked. #[serde(skip_serializing_if = "Option::is_none")] - pub seen: Option, + pub seen: Option, /// A rotation of a key it logs in with is not done on it yet. #[serde(skip_serializing_if = "std::ops::Not::not")] pub rotating: bool, @@ -187,6 +243,8 @@ pub(super) fn job_info(paths: &berth_core::paths::Paths, job: store::Job) -> Job started: job.started, ended: job.ended, exit_code: job.exit_code, + stopped_by: job.stopped_by, + arguments: job.arguments, } } @@ -262,12 +320,17 @@ fn guarded_routes() -> OpenApiRouter> { .routes(routes!(jobs)) .routes(routes!(job)) .routes(routes!(job_output)) + .routes(routes!(stop_job)) .routes(routes!(inbox)) .routes(routes!(mark_done, mark_open)) .routes(routes!(kv_list)) .routes(routes!(kv_get, kv_put, kv_delete)) } +/// Where a host's controller's operations are passed through: berthd's +/// `/v1/PATH` is `/v1/hosts/NAME/controller/PATH` here. +pub const HOST_PREFIX: &str = "/v1/hosts/{name}/controller/{*rest}"; + pub fn router(state: Arc, guarded: bool) -> Router { let (mut router, _) = guarded_routes().split_for_parts(); // Every stream of the table gets its route here, so none can exist undescribed. @@ -276,10 +339,12 @@ pub fn router(state: Arc, guarded: bool) -> Router { // At the path of a plain GET, whose handler upgrades when asked to. "job" | "inbox" | "kv" | "resource-logs" => router, "events" => router.route(stream.path, get(events)), + "resource-connect" => router.route(stream.path, get(super::connect::connect)), other => panic!("no handler for the stream {other}"), }; } - router = router.route("/v1/hosts/{name}/{*rest}", any(proxy)); + // Under a prefix of its own, so that no route of ours can hide one of the host's. + router = router.route(HOST_PREFIX, any(proxy)); router = router.route(ops::PORTAL, get(portal)); if guarded { router = router.layer(middleware::from_fn_with_state(state.clone(), guard)); @@ -291,6 +356,8 @@ pub fn router(state: Arc, guarded: bool) -> Router { ); } } + // Not behind the guard: an address in a browser, as the recipes of berth 0.5 printed it. + router = router.route("/", get(front)); // Added after the guard, which they are not behind: they check their key themselves. router = router.merge(keyed_routes().split_for_parts().0); router @@ -298,6 +365,23 @@ pub fn router(state: Arc, guarded: bool) -> Router { .with_state(state) } +/// What a browser finds at the server's address. It says nothing of the +/// server, as anyone who reaches it may read it. +pub const FRONT: &str = "This is a berth server. + +berth has no page in the browser any more. Use berth on your own machine, +on Linux, macOS or Windows: + + berth login https://THIS-SERVER + berth ui + +A link that ends in #setup-key=KEY holds a setup key: berth login asks for it. +"; + +async fn front() -> Response { + ([(header::CONTENT_TYPE, "text/plain; charset=utf-8")], FRONT).into_response() +} + /// What a request shows to be let in: a bearer, and the protocols of a /// WebSocket, because a browser cannot set a header on one. fn shown(headers: &HeaderMap) -> Vec { @@ -332,7 +416,7 @@ async fn guard(State(state): App, mut request: Request, next: Next) -> Response .is_some_and(|value| value.as_bytes().eq_ignore_ascii_case(b"websocket")); let allowed = match auth::operation(request.method().as_str(), request.uri().path(), stream) { Ok(Some(op)) => state.auth.allows(&access, op), - // The portal and the documents: each operation on the portal is asked about there. + // The portal, where each operation is asked about as it comes. Ok(None) => Ok(()), Err(()) if state.auth.role(&access) == Role::Admin => Ok(()), Err(()) => Err(( @@ -366,7 +450,8 @@ pub fn caller(access: Option>) -> Access { } /// The portal: every operation over one WebSocket. With `host` an operation -/// is that host's, and goes the way a request under `/v1/hosts/NAME` does. +/// is that host's, and goes the way a request under +/// `/v1/hosts/NAME/controller` does. async fn portal( State(state): App, access: Option>, @@ -383,15 +468,22 @@ async fn portal( let asked = state.clone(); let resolve: berthd::portal::Resolve = Arc::new(move |name, host| { let (table, prefix) = match host { - Some(host) => (ops::HOST, format!("/hosts/{}", client::encode(host))), + Some(host) => ( + ops::HOST, + format!("/hosts/{}/controller", client::encode(host)), + ), None => (ops::SERVE, String::new()), }; let op = ops::find(table, name) - .ok_or_else(|| (code::NOT_FOUND, format!("No operation {name}")))?; + .ok_or_else(|| (code::NOT_FOUND, berth_api::compat::no_operation(name)))?; + if !op.server_only.is_empty() { + return Err((code::FORBIDDEN, op.server_only.into())); + } asked.auth.allows(&access, op)?; Ok(berthd::portal::Target { op, prefix }) }); - ws.on_upgrade(move |socket| berthd::portal::run(socket, inner, resolve)) + berthd::portal::upgrade(ws) + .on_upgrade(move |socket| berthd::portal::run(socket, inner, resolve)) } #[derive(Serialize, ToSchema)] @@ -426,13 +518,13 @@ pub(super) fn compare(state: &Serve) -> Compare { Arc::new(rollout::comparer(&state.paths)) } -/// What the server knows of a host apart from asking it: the level of its +/// What the server knows of a host apart from asking it: the type of its /// profile, and whether a key it logs in with is being rotated. -pub(super) fn known(state: &Serve, name: &str) -> (HostLevel, bool) { +pub(super) fn known(state: &Serve, name: &str) -> (HostType, bool) { let profile = super::setup::saved_profile(state, name).map(|(_, profile)| profile); - let level = profile + let host_type = profile .as_ref() - .map_or(HostLevel::Workstation, berth_api::profile::level); + .map_or(HostType::Workstation, berth_api::profile::host_type); let rotating = profile.is_ok_and(|profile| { let place = berth_core::profiles::Place::own(name); let store = berth_core::store::shared(&state.paths); @@ -442,16 +534,16 @@ pub(super) fn known(state: &Serve, name: &str) -> (HostLevel, bool) { _ => false, } }); - (level, rotating) + (host_type, rotating) } -async fn info(remote: Arc, compare: Compare, known: (HostLevel, bool)) -> HostInfo { +async fn info(remote: Arc, compare: Compare, known: (HostType, bool)) -> HostInfo { let seen = remote.version().await; let applied = seen .as_ref() .ok() .and_then(|version| version.applied.clone()); - let observed = berth_core::level::seen( + let observed = berth_core::host_type::seen( seen.as_ref() .map_err(|error| error.starts_with(super::NO_BERTHD)), ); @@ -464,7 +556,7 @@ async fn info(remote: Arc, compare: Compare, known: (HostLevel, bool)) - error: seen.err(), behind: applied.as_ref().and_then(|applied| compare(applied)), applied: applied.map(|applied| applied.berth), - level: known.0, + host_type: known.0, seen: observed, rotating: known.1, } @@ -505,7 +597,7 @@ async fn version() -> Json { api: berth_api::API, mode: "serve".into(), applied: None, - level: None, + host_type: None, }) } @@ -723,6 +815,53 @@ async fn job_output( Ok(Json(piece(&data, &query, job.ended.is_some()))) } +#[utoipa::path(post, path = "/v1/jobs/{id}/stop", summary = "Stop a running job: its programs end, a session it follows ends on its host, and its record says who stopped it.", tag = "jobs", + params(("id" = String, Path, description = "A job's id")), + responses((status = 202, description = "Asked to stop; it ends soon, as following it shows", body = JobInfo), + (status = 403, description = "The caller may not start a job of its kind, so may not stop it", body = ErrorBody), + (status = 404, description = "No such job", body = ErrorBody), + (status = 409, description = "It has ended already", body = ErrorBody), + (status = 501, description = "It runs in another berth serve of the same store", body = ErrorBody)))] +async fn stop_job( + State(state): App, + access: Option>, + Path(id): Path, +) -> Answer<(StatusCode, Json)> { + let job = find_job(&state, &id)?; + // Who may start a job of its kind may stop it. + if let (Some(Extension(access)), Some(role)) = (&access, state.runner.stop_role(&id)) { + state + .auth + .permits(access, "jobs.stop", role) + .map_err(|(kind, message)| Fail::new(kind, message))?; + } + let by = match &access { + None => "the server's own user".to_owned(), + Some(Extension(access)) => match &access.subject { + Some(subject) => subject.clone(), + None => format!("a caller with the role {}", access.role.name()), + }, + }; + use berth_core::runner::Stopping; + match state.runner.stop(&id, &by).map_err(internal)? { + Stopping::Asked | Stopping::AskedBefore(_) => {} + Stopping::Ended => { + return Err(Fail::new( + code::CONFLICT, + format!("Job {id} has ended already"), + )); + } + Stopping::NotHere => { + return Err(Fail::new( + code::NOT_AVAILABLE, + format!("Job {id} runs in another berth serve of this store, which can stop it"), + )); + } + } + let job = state.store.job(&id)?.unwrap_or(job); + Ok((StatusCode::ACCEPTED, Json(job_info(&state.paths, job)))) +} + /// What is said of a job whose output file is not here. The store has the /// job, but its output stays in the data directory of the machine that ran /// it, which an installation restored on another machine does not have. @@ -800,16 +939,19 @@ async fn forward( ) { loop { tokio::select! { - event = events.recv() => match event { - Ok(event) => { - if let Some(frame) = pick(event) - && socket.send(WsMessage::Text(frame.to_string().into())).await.is_err() - { - return; - } + event = events.recv() => { + let frame = match event { + Ok(event) => pick(event), + // A reader that fell behind is told how much it missed, + // so it can read the current state again. + Err(RecvError::Lagged(missed)) => Some(json!({"dropped": {"events": missed}})), + Err(RecvError::Closed) => return, + }; + if let Some(frame) = frame + && socket.send(WsMessage::Text(frame.to_string().into())).await.is_err() + { + return; } - Err(RecvError::Lagged(_)) => {} - Err(RecvError::Closed) => return, }, frame = socket.recv() => if !matches!(frame, Some(Ok(_))) { return }, } @@ -985,6 +1127,15 @@ struct Follow { jobs: bool, /// Changes in these namespaces of the key-value store. kv: Vec, + /// The kinds of thing whose changes to tell. + changes: Vec, +} + +#[derive(Deserialize, PartialEq)] +#[serde(rename_all = "lowercase")] +enum Changes { + Hosts, + Resources, } async fn events(State(state): App, ws: WebSocketUpgrade) -> Response { @@ -1016,6 +1167,16 @@ async fn events(State(state): App, ws: WebSocketUpgrade) -> Response { Some(value) => json!({"source": "kv", "namespace": namespace, "key": key, "value": value, "revision": revision}), None => json!({"source": "kv", "namespace": namespace, "key": key, "removed": true}), }), + Event::Host { name, change, from } if follow.changes.contains(&Changes::Hosts) => { + let mut frame = json!({"source": "host", "host": name, "change": change}); + if let Some(from) = from { + frame["from"] = json!(from); + } + Some(frame) + } + Event::Resource { id, provisioner, kind, change } if follow.changes.contains(&Changes::Resources) => { + Some(json!({"source": "resource", "resource": id, "provisioner": provisioner, "kind": kind, "change": change})) + } _ => None, }) .await; @@ -1050,12 +1211,30 @@ async fn proxy( ws: Ws, body: Bytes, ) -> Answer { + // What the server calls on a host for itself is not passed through, however it is spelt. + let asked = format!("/v1/{rest}"); + for kind in [ops::Kind::Call, ops::Kind::Stream] { + if let Some((op, _)) = ops::recognise(ops::HOST, kind, method.as_str(), &asked) + && !op.server_only.is_empty() + { + return Err(Fail::new(code::FORBIDDEN, op.server_only)); + } + } let remote = remote(&state, &name)?; + // A host of the type ssh has no controller to pass this to; say what installs one. + let (host_type, _) = known(&state, &name); + if let Err(refusal) = berth_core::host_type::check( + &name, + host_type, + berth_core::host_type::Capability::Sessions, + ) { + return Err(Fail::new(code::PRECONDITION, refusal.why)); + } // The secrets a host gets are the server's own: a caller cannot hand it others. - if method == Method::POST && rest.trim_end_matches('/') == "tasks/secrets" { + if method == Method::POST && rest.trim_end_matches('/') == "actions/secrets" { return Err(Fail::new( code::FORBIDDEN, - "The secrets task takes the server's secrets: run the host's secrets action (POST /v1/hosts/NAME/actions/secrets)", + "The controller's secrets action takes the server's secrets: run the server's secrets action of the host (POST /v1/hosts/NAME/actions/secrets)", )); } let Some(link) = &remote.link else { @@ -1086,32 +1265,41 @@ async fn proxy( Ok(response) } -/// Frames both ways until either side closes. +/// Frames both ways until either side closes, each way on its own, so a +/// side that is slow to take frames does not hold up the other direction. async fn pump(socket: WebSocket, upstream: berth_core::link::LinkStream) { use client::Frame; let (mut to_client, mut from_client) = socket.split(); let (mut to_host, mut from_host) = upstream.split(); - loop { - tokio::select! { - frame = from_client.next() => { - let sent = match frame { - Some(Ok(WsMessage::Text(text))) => to_host.send(Frame::Text(text.as_str().into())).await, - Some(Ok(WsMessage::Binary(data))) => to_host.send(Frame::Binary(data)).await, - Some(Ok(WsMessage::Ping(_) | WsMessage::Pong(_))) => Ok(()), - _ => break, - }; - if sent.is_err() { break } + let towards_host = async { + while let Some(Ok(frame)) = from_client.next().await { + let sent = match frame { + WsMessage::Text(text) => to_host.send(Frame::Text(text.as_str().into())).await, + WsMessage::Binary(data) => to_host.send(Frame::Binary(data)).await, + WsMessage::Ping(_) | WsMessage::Pong(_) => Ok(()), + WsMessage::Close(_) => break, + }; + if sent.is_err() { + break; } - frame = from_host.next() => { - let sent = match frame { - Some(Ok(Frame::Text(text))) => to_client.send(WsMessage::Text(text.as_str().into())).await, - Some(Ok(Frame::Binary(data))) => to_client.send(WsMessage::Binary(data)).await, - Some(Ok(Frame::Ping(_) | Frame::Pong(_) | Frame::Frame(_))) => Ok(()), - _ => break, - }; - if sent.is_err() { break } + } + }; + let towards_client = async { + while let Some(Ok(frame)) = from_host.next().await { + let sent = match frame { + Frame::Text(text) => to_client.send(WsMessage::Text(text.as_str().into())).await, + Frame::Binary(data) => to_client.send(WsMessage::Binary(data)).await, + Frame::Ping(_) | Frame::Pong(_) | Frame::Frame(_) => Ok(()), + Frame::Close(_) => break, + }; + if sent.is_err() { + break; } } + }; + tokio::select! { + () = towards_host => {} + () = towards_client => {} } let _ = to_host.send(Frame::Close(None)).await; let _ = to_client.send(WsMessage::Close(None)).await; diff --git a/crates/berth/src/serve/auth.rs b/crates/berth/src/local/serve/auth.rs similarity index 77% rename from crates/berth/src/serve/auth.rs rename to crates/berth/src/local/serve/auth.rs index 0bc9800..f1720d8 100644 --- a/crates/berth/src/serve/auth.rs +++ b/crates/berth/src/local/serve/auth.rs @@ -78,6 +78,16 @@ impl Auth { }) } + /// Nobody over the network, for a server that has no listener there: + /// its one caller is the owner, who sets it up. + pub fn nobody() -> Auth { + Auth { + token: None, + setup: true, + oauth: None, + } + } + /// Only the server's own token, with set-up open. #[cfg(test)] pub fn token(token: Option) -> Auth { @@ -204,22 +214,39 @@ impl Auth { } /// The operation a request is, for a table lookup of the role it needs. -/// `Err` is a request below a host that no operation is known for. +/// `Err` is a request no operation is known for. What a request says of +/// itself only orders the lookup: one that claims to be a stream at the path +/// of a call is still that call, and a HEAD is the GET that answers it. pub fn operation(method: &str, path: &str, stream: bool) -> Result, ()> { - let kind = if stream { Kind::Stream } else { Kind::Call }; - if let Some((op, _)) = ops::recognise(ops::SERVE, kind, method, path) { + let method = if method.eq_ignore_ascii_case("HEAD") { + "GET" + } else { + method + }; + let kinds = if stream { + [Kind::Stream, Kind::Call] + } else { + [Kind::Call, Kind::Stream] + }; + let found = |table: &'static [Op], path: &str| { + kinds + .iter() + .find_map(|kind| ops::recognise(table, *kind, method, path)) + .map(|(op, _)| op) + }; + if let Some(op) = found(ops::SERVE, path) { return Ok(Some(op)); } - let Some((_, rest)) = path - .strip_prefix("/v1/hosts/") - .and_then(|below| below.split_once('/')) - else { + // Each operation on the portal is asked about there. + if path == ops::PORTAL { return Ok(None); - }; - match ops::recognise(ops::HOST, kind, method, &format!("/v1/{rest}")) { - Some((op, _)) => Ok(Some(op)), - None => Err(()), } + let rest = path + .strip_prefix("/v1/hosts/") + .and_then(|below| below.split_once('/')) + .and_then(|(_, below)| below.strip_prefix("controller/")) + .ok_or(())?; + found(ops::HOST, &format!("/v1/{rest}")).map(Some).ok_or(()) } // ── Access tokens ─────────────────────────────────────────────────────────── @@ -378,8 +405,9 @@ fn metadata_addresses(issuer: &str) -> Vec { } /// The highest role a claim names. The claim is a name, or a way into objects -/// with dots; its value a text of words or a list. In `scope` only `berth:ROLE` -/// counts, in a claim of its own the bare role does too. +/// with dots; its value a text of words, a list, or an object whose keys are +/// the words. In `scope` only `berth:ROLE` counts, in a claim of its own the +/// bare role does too. pub fn role(claims: &Value, claim: &str) -> Option { let value = match claims.get(claim) { Some(value) => value, @@ -392,6 +420,8 @@ pub fn role(claims: &Value, claim: &str) -> Option { .split(|ch: char| ch.is_whitespace() || ch == ',') .collect(), Value::Array(list) => list.iter().filter_map(Value::as_str).collect(), + // Zitadel writes a project's roles so: each role a key, its value where it holds. + Value::Object(roles) => roles.keys().map(String::as_str).collect(), _ => vec![], }; let bare = !matches!(claim, "scope" | "scp"); @@ -427,6 +457,19 @@ mod tests { ); assert_eq!(role(&claims, "groups"), None); assert_eq!(role(&json!({"scope": "openid"}), "scope"), None); + // An object's keys are the words, as Zitadel writes a project's roles. + let zitadel = json!({ + "urn:zitadel:iam:org:project:roles": { + "view": {"317401": "studio.example"}, + "berth:operate": {"317401": "studio.example"}, + "billing": {}, + }, + }); + assert_eq!( + role(&zitadel, "urn:zitadel:iam:org:project:roles"), + Some(Role::Operate) + ); + assert_eq!(role(&json!({"roles": {}}), "roles"), None); } #[test] @@ -438,12 +481,44 @@ mod tests { "hosts.act" ); assert_eq!(name("GET", "/v1/hosts/a/actions", false), "hosts.actions"); - assert_eq!(name("POST", "/v1/actions/rollout", false), "actions.act"); - assert_eq!(name("GET", "/v1/hosts/a/sessions", false), "sessions.list"); assert_eq!( - name("GET", "/v1/hosts/a/sessions/x", true), + name("POST", "/v1/installation/actions/rollout", false), + "installation.act" + ); + assert_eq!( + name("GET", "/v1/hosts/a/controller/sessions", false), + "sessions.list" + ); + assert_eq!( + name("GET", "/v1/hosts/a/controller/sessions/x", true), "sessions.attach" ); + assert_eq!( + name("GET", "/v1/hosts/a/controller/ports/8080", true), + "ports.connect" + ); + // The host's actions and the server's view of them do not meet. + assert_eq!( + name("GET", "/v1/hosts/a/controller/actions", false), + "actions.list" + ); + assert_eq!( + name("POST", "/v1/hosts/a/controller/actions/x", false), + "actions.run" + ); + assert!(operation("GET", "/v1/hosts/a/sessions", false).is_err()); + // A call that claims to be a stream is still that call, and a HEAD is its GET. + assert_eq!(name("PUT", "/v1/secrets/main", true), "secrets.set"); + assert_eq!(name("POST", "/v1/plugins/package", true), "plugins.upload"); + assert_eq!(name("HEAD", "/v1/secrets", false), "secrets.list"); + assert_eq!( + name("POST", "/v1/hosts/a/controller/actions/x", true), + "actions.run" + ); + // What no operation is known for is not let through, below a host or not. + assert!(operation("GET", "/v1/nothing", false).is_err()); + assert!(operation("CONNECT", "/v1/secrets", true).is_err()); + assert!(operation("GET", "/v1/portal/more", true).is_err()); assert_eq!( name("PUT", "/v1/secrets/registry%2Fmain", false), "secrets.set" @@ -454,6 +529,10 @@ mod tests { name("GET", "/v1/resources/r-a/logs", true), "resources.follow" ); + assert_eq!( + name("GET", "/v1/resources/r-a/endpoints/db", true), + "resources.connect" + ); assert!(operation("GET", "/v1/portal", true).unwrap().is_none()); // What nobody knows below a host is not let through on a guess. assert!(operation("POST", "/v1/hosts/a/nothing", false).is_err()); @@ -465,7 +544,7 @@ mod tests { }; let admin = Access::local(); let apply = ops::find(ops::SERVE, "hosts.save").unwrap(); - let run = ops::find(ops::HOST, "tasks.run").unwrap(); + let run = ops::find(ops::HOST, "actions.run").unwrap(); assert_eq!(auth.role(&admin), Role::Operate); assert!(auth.allows(&admin, run).is_ok()); assert!(auth.allows(&admin, apply).unwrap_err().1.contains("set-up")); @@ -474,6 +553,29 @@ mod tests { subject: None, }; assert!(auth.allows(&viewer, run).unwrap_err().1.contains("operate")); + // A port of a host is used, not looked at. + let port = ops::find(ops::HOST, "ports.connect").unwrap(); + assert!( + auth.allows(&viewer, port) + .unwrap_err() + .1 + .contains("operate") + ); + assert!(auth.allows(&admin, port).is_ok()); + // What the server dials on a host for itself is an operation, which its pass-through refuses. + assert_eq!( + name("GET", "/v1/hosts/a/controller/endpoints/192.0.2.1/80", true), + "endpoints.connect" + ); + assert!( + !ops::find(ops::HOST, "endpoints.connect") + .unwrap() + .server_only + .is_empty() + ); + let endpoint = ops::find(ops::SERVE, "resources.connect").unwrap(); + assert!(auth.allows(&viewer, endpoint).is_err()); + assert!(auth.allows(&admin, endpoint).is_ok()); // An action's own role is asked the same way. assert!(auth.permits(&viewer, "apply", Role::Admin).is_err()); assert!( diff --git a/crates/berth/src/local/serve/connect.rs b/crates/berth/src/local/serve/connect.rs new file mode 100644 index 0000000..0cfbcf4 --- /dev/null +++ b/crates/berth/src/local/serve/connect.rs @@ -0,0 +1,447 @@ +//! A byte tunnel to an endpoint of a resource, as `resources.connect` opens +//! it. Only an endpoint the resource's provisioner reported can be opened, +//! and its address is resolved and checked when it is dialled, so the +//! tunnel is no way to an address of the caller's choosing. + +use super::Serve; +use super::api::{Answer, App, Fail, Ws}; +use axum::{extract::Path, response::Response}; +use berth_api::{Endpoint, client, code, tunnel::Tunnel}; +use berth_core::{ + paths::Paths, + resource::{self, Kept}, +}; +use std::{ + net::{IpAddr, Ipv4Addr, Ipv6Addr, SocketAddr}, + time::Duration, +}; +use tokio::io::{AsyncRead, AsyncWrite}; + +/// How long resolving and dialling an endpoint may take. +const DIAL_FOR: Duration = Duration::from_secs(10); + +/// The API number of the first berthd with `endpoints.connect`. +pub const NODE_API: u32 = 2; + +/// A connection to an endpoint, whichever way it was dialled. +pub trait Connection: AsyncRead + AsyncWrite + Send + Unpin + 'static {} + +impl Connection for T {} + +/// Who dials the endpoints of a provisioner's resources. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Dialler { + /// The server, from its own machine: right for an engine there or on a + /// network the server routes to. + Server, + /// The controller of the node the provisioner lives on, whose engine + /// hands out addresses that often only the node routes to. + Node(String), +} + +impl Dialler { + /// The dialler for the provisioner that made a resource. + pub fn of(paths: &Paths, kept: &Kept) -> Dialler { + match berth_core::provisioner::entry(paths, &kept.record.provisioner) { + Ok(Some(entry)) if !entry.node.is_empty() => Dialler::Node(entry.node), + _ => Dialler::Server, + } + } + + /// Resolve, check and dial an endpoint. + pub async fn dial(&self, state: &Serve, endpoint: &Endpoint) -> Answer> { + match self { + Dialler::Server => { + let addresses = resolve(endpoint).await?; + Ok(Box::new(dial_first(endpoint, &addresses).await?)) + } + Dialler::Node(node) => through_node(state, node, endpoint).await, + } + } +} + +/// How a node's controller reaches an endpoint its engine reported: the +/// node's loopback through `ports.connect`, anything else through +/// `endpoints.connect`, which checks the address on the node. +pub fn on_node(endpoint: &Endpoint) -> Result { + let place = format!("{}:{}", endpoint.address, endpoint.port); + let Ok(ip) = endpoint.address.parse::().map(canonical) else { + return Ok(match endpoint.address.eq_ignore_ascii_case("localhost") { + true => format!("/v1/ports/{}", endpoint.port), + false => format!( + "/v1/endpoints/{}/{}", + client::encode(&endpoint.address), + endpoint.port + ), + }); + }; + if metadata(ip) { + return Err(format!( + "{place} is a cloud's metadata address, which berth never dials" + )); + } + // An engine on the node means the node's own loopback, which ports.connect dials as 127.0.0.1. + if ip == IpAddr::V4(Ipv4Addr::LOCALHOST) { + return Ok(format!("/v1/ports/{}", endpoint.port)); + } + if own_machine(ip) { + return Err(format!( + "{place} is a loopback or link-local address that the node's controller does not dial: only 127.0.0.1 of its loopback" + )); + } + Ok(format!("/v1/endpoints/{ip}/{}", endpoint.port)) +} + +/// An endpoint dialled by the controller of `node`, over the link the +/// server has to it. +async fn through_node( + state: &Serve, + node: &str, + endpoint: &Endpoint, +) -> Answer> { + let path = on_node(endpoint).map_err(|why| Fail::new(code::FORBIDDEN, why))?; + let remote = state.host(node).ok_or_else(|| { + Fail::new( + code::UNREACHABLE, + format!("The provisioner lives on the node {node}, which is not a host of this server"), + ) + })?; + let Some(link) = &remote.link else { + let problem = remote.problem.clone().unwrap_or_else(|| "no link".into()); + return Err(Fail::new( + code::UNREACHABLE, + format!("The node {node} cannot be reached: {problem}"), + )); + }; + let too_old = |said: &str| { + Fail::new( + code::NOT_AVAILABLE, + format!( + "The controller of the node {node} is {said}, which cannot reach a resource's endpoint: update its controller or apply {node}" + ), + ) + }; + match remote.version().await { + Err(why) => { + return Err(Fail::new( + code::UNREACHABLE, + format!("The node {node} does not answer: {why}"), + )); + } + Ok(version) if version.api < NODE_API => { + return Err(too_old(&format!( + "berthd {} with API {}", + version.version, version.api + ))); + } + Ok(_) => {} + } + let opened = tokio::time::timeout(DIAL_FOR * 2, link.stream(&path)).await; + match opened { + Ok(Ok(stream)) => Ok(Box::new(Tunnel::new(stream))), + // Only a controller without the route answers so: the stream itself never does. + Ok(Err(error)) if error.status() == 404 => Err(too_old("older than endpoints.connect")), + Ok(Err(error)) => Err(error.into()), + Err(_) => Err(Fail::new( + code::TIMEOUT, + format!( + "The node {node} did not open the endpoint {} in time", + endpoint.name + ), + )), + } +} + +/// The endpoint a caller names: by its name, or else the first with that port. +pub fn chosen<'a>(endpoints: &'a [Endpoint], given: &str) -> Option<&'a Endpoint> { + endpoints + .iter() + .find(|endpoint| endpoint.name == given) + .or_else(|| { + let port = given.parse::().ok()?; + endpoints.iter().find(|endpoint| endpoint.port == port) + }) +} + +/// An address as its IPv4 self when it is an IPv4 address written as IPv6, +/// so that each check sees it once. +fn canonical(ip: IpAddr) -> IpAddr { + match ip { + IpAddr::V6(v6) => v6.to_ipv4_mapped().map_or(IpAddr::V6(v6), IpAddr::V4), + v4 => v4, + } +} + +/// The addresses of clouds' instance metadata services, which hand out +/// credentials to whoever asks from the machine. +fn metadata(ip: IpAddr) -> bool { + match canonical(ip) { + IpAddr::V4(v4) => v4 == Ipv4Addr::new(169, 254, 169, 254), + IpAddr::V6(v6) => v6 == Ipv6Addr::new(0xfd00, 0xec2, 0, 0, 0, 0, 0, 0x254), + } +} + +/// An address that names the dialling machine itself or its own link, not +/// one a provisioner's engine elsewhere hands out. +fn own_machine(ip: IpAddr) -> bool { + match canonical(ip) { + IpAddr::V4(v4) => { + v4.is_loopback() + || v4.is_link_local() + || v4.is_unspecified() + || v4.is_broadcast() + || v4.is_multicast() + } + IpAddr::V6(v6) => { + v6.is_loopback() + || v6.is_unicast_link_local() + || v6.is_unspecified() + || v6.is_multicast() + } + } +} + +/// Whether the server may dial `resolved` for `endpoint`, or why not. +pub fn allowed(endpoint: &Endpoint, resolved: IpAddr) -> Result<(), String> { + let resolved = canonical(resolved); + let place = format!("{}:{}", endpoint.address, endpoint.port); + // Credentials of the server's own machine are never a resource's to hand out. + if metadata(resolved) { + return Err(format!( + "{place} is a cloud's metadata address, which berth never dials" + )); + } + if !own_machine(resolved) { + return Ok(()); + } + let reported = endpoint.address.parse::().ok().map(canonical); + let named_so = reported == Some(resolved) + || (endpoint.address.eq_ignore_ascii_case("localhost") && resolved.is_loopback()); + // Only an engine on this machine can mean this machine by such an address. + if endpoint.local && named_so { + return Ok(()); + } + match (endpoint.local, reported.is_some_and(own_machine)) { + // An address of the engine's own machine, which is not the server's. + (false, true) => Err(format!( + "{place} is an address of the machine the provisioner's engine runs on, which this server cannot reach" + )), + // A name that resolves to the server itself now was not reported as such. + _ => Err(format!( + "{} resolves to {resolved}, an address of the server's own machine, which the provisioner did not report", + endpoint.address + )), + } +} + +/// The addresses an endpoint has now, each one checked. +async fn resolve(endpoint: &Endpoint) -> Answer> { + let addresses: Vec = match endpoint.address.parse::() { + Ok(ip) => vec![SocketAddr::new(ip, endpoint.port)], + Err(_) => { + let looked = tokio::time::timeout( + DIAL_FOR, + tokio::net::lookup_host((endpoint.address.as_str(), endpoint.port)), + ) + .await; + match looked { + Ok(Ok(found)) => found.collect(), + Ok(Err(error)) => { + return Err(Fail::new( + code::UNREACHABLE, + format!("{} does not resolve: {error}", endpoint.address), + )); + } + Err(_) => { + return Err(Fail::new( + code::TIMEOUT, + format!("{} did not resolve in time", endpoint.address), + )); + } + } + } + }; + if addresses.is_empty() { + return Err(Fail::new( + code::UNREACHABLE, + format!("{} has no address", endpoint.address), + )); + } + // One refused address refuses the name, as which of them is dialled is not the caller's to choose. + for address in &addresses { + allowed(endpoint, address.ip()).map_err(|why| Fail::new(code::FORBIDDEN, why))?; + } + Ok(addresses) +} + +/// The first of the checked addresses that answers. +async fn dial_first( + endpoint: &Endpoint, + addresses: &[SocketAddr], +) -> Answer { + let mut last = String::new(); + for address in addresses { + match tokio::time::timeout(DIAL_FOR, tokio::net::TcpStream::connect(address)).await { + Ok(Ok(tcp)) => { + let _ = tcp.set_nodelay(true); + return Ok(tcp); + } + Ok(Err(error)) => last = format!("{address}: {error}"), + Err(_) => { + return Err(Fail::new( + code::TIMEOUT, + format!( + "The endpoint {} at {address} does not answer", + endpoint.name + ), + )); + } + } + } + Err(Fail::new( + code::NOT_SERVED, + format!("Nothing answers on the endpoint {}, {last}", endpoint.name), + )) +} + +/// The endpoint is dialled before the stream opens, so one that nothing +/// answers on is an answer of its own and not a stream that ends at once. +pub async fn connect( + axum::extract::State(state): App, + Path((id, given)): Path<(String, String)>, + ws: Ws, +) -> Answer { + let ws = ws.map_err(|_| { + Fail::new( + code::BAD_REQUEST, + "An endpoint is opened only as a WebSocket", + ) + })?; + // Read now, so the endpoints are those the provisioner reported last. + let kept = resource::find(&state.paths, &id) + .map_err(|error| Fail::new(code::NOT_FOUND, error.to_string()))?; + let endpoints = &kept.record.endpoints; + let endpoint = chosen(endpoints, &given).cloned().ok_or_else(|| { + let names: Vec<&str> = endpoints.iter().map(|known| known.name.as_str()).collect(); + Fail::new( + code::NOT_FOUND, + match names.is_empty() { + true => format!("{} reports no endpoints", kept.title()), + false => format!( + "{} has no endpoint {given}. Its endpoints: {}", + kept.title(), + names.join(", ") + ), + }, + ) + })?; + if endpoint.protocol != Endpoint::TCP { + return Err(Fail::new( + code::BAD_REQUEST, + format!( + "The endpoint {} is {}, and a tunnel carries TCP alone", + endpoint.name, endpoint.protocol + ), + )); + } + let connection = Dialler::of(&state.paths, &kept) + .dial(&state, &endpoint) + .await?; + Ok(ws.on_upgrade(move |socket| berthd::ports::tunnel(socket, connection))) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn endpoint(address: &str, local: bool) -> Endpoint { + Endpoint { + name: "db".into(), + address: address.into(), + port: 5432, + protocol: Endpoint::TCP.into(), + local, + ..Endpoint::default() + } + } + + fn ip(text: &str) -> IpAddr { + text.parse().unwrap() + } + + #[test] + fn the_server_dials_what_was_reported_and_not_its_own_machine_by_another_name() { + // A container's address, or a name that resolves to one like it. + assert!(allowed(&endpoint("192.0.2.20", false), ip("192.0.2.20")).is_ok()); + assert!(allowed(&endpoint("db.example", false), ip("192.0.2.20")).is_ok()); + // Loopback reported by an engine on this machine is this machine's. + assert!(allowed(&endpoint("127.0.0.1", true), ip("127.0.0.1")).is_ok()); + assert!(allowed(&endpoint("localhost", true), ip("::1")).is_ok()); + // Reported by an engine elsewhere, it is that machine's, which the server cannot reach. + let elsewhere = allowed(&endpoint("127.0.0.1", false), ip("127.0.0.1")).unwrap_err(); + assert!(elsewhere.contains("cannot reach"), "{elsewhere}"); + // A name that resolves to the server's own machine now is refused. + for (address, local) in [("db.example", false), ("db.example", true)] { + let refused = allowed(&endpoint(address, local), ip("127.0.0.1")).unwrap_err(); + assert!(refused.contains("did not report"), "{refused}"); + } + assert!(allowed(&endpoint("db.example", true), ip("fe80::1")).is_err()); + assert!(allowed(&endpoint("db.example", false), ip("::ffff:127.0.0.1")).is_err()); + // Metadata services are refused however they are reported. + for address in ["169.254.169.254", "fd00:ec2::254"] { + let refused = allowed(&endpoint(address, true), ip(address)).unwrap_err(); + assert!(refused.contains("metadata"), "{refused}"); + } + assert!(allowed(&endpoint("169.254.10.1", false), ip("169.254.10.1")).is_err()); + } + + #[test] + fn a_nodes_controller_dials_its_loopback_as_a_port_and_the_rest_as_an_address() { + let path = |address: &str, local: bool| on_node(&endpoint(address, local)); + // Reported by the engine on the node, the loopback is the node's, whatever `local` says. + for local in [true, false] { + assert_eq!(path("127.0.0.1", local).unwrap(), "/v1/ports/5432"); + } + assert_eq!(path("localhost", true).unwrap(), "/v1/ports/5432"); + assert_eq!( + path("10.12.0.57", true).unwrap(), + "/v1/endpoints/10.12.0.57/5432" + ); + assert_eq!( + path("::ffff:10.12.0.57", true).unwrap(), + "/v1/endpoints/10.12.0.57/5432" + ); + assert_eq!( + path("db.example", false).unwrap(), + "/v1/endpoints/db.example/5432" + ); + assert!( + path("169.254.169.254", true) + .unwrap_err() + .contains("metadata") + ); + assert!( + path("fd00:ec2::254", true) + .unwrap_err() + .contains("metadata") + ); + for refused in ["127.0.0.2", "::1", "fe80::1", "169.254.3.4"] { + assert!(path(refused, true).is_err(), "{refused}"); + } + } + + #[test] + fn an_endpoint_is_named_or_given_by_its_port() { + let endpoints = [ + Endpoint { + name: "web".into(), + port: 8080, + ..endpoint("192.0.2.20", false) + }, + endpoint("192.0.2.20", false), + ]; + assert_eq!(chosen(&endpoints, "db").unwrap().port, 5432); + assert_eq!(chosen(&endpoints, "8080").unwrap().name, "web"); + assert!(chosen(&endpoints, "22").is_none()); + } +} diff --git a/crates/berth/src/serve/docs.rs b/crates/berth/src/local/serve/docs.rs similarity index 81% rename from crates/berth/src/serve/docs.rs rename to crates/berth/src/local/serve/docs.rs index fd5f87e..5e04337 100644 --- a/crates/berth/src/serve/docs.rs +++ b/crates/berth/src/local/serve/docs.rs @@ -18,10 +18,11 @@ pub fn openapi() -> Value { api.info.version = berth_api::VERSION.into(); api.info.description = Some( "One API over all hosts, with the inbox and a key-value store. Everything under \ - /v1/hosts/{name}/ that is not named here is that host's own API, passed through unchanged; \ - only the `secrets` task is refused, which the host's `secrets` action stands in for. \ + /v1/hosts/{name}/controller/ is that host's own API, passed through unchanged; \ + only the controller's own `secrets` action is refused, which the server's `secrets` action of the host stands in for, \ + and `endpoints.connect`, which the server opens itself for `resources.connect`. \ What can be done to a host or to the installation is listed at /v1/hosts/{name}/actions and \ - /v1/actions, and run by POST to an action's name there. Over TCP every request \ + /v1/installation/actions, and run by POST to an action's name there. Over TCP every request \ carries the server's token or an OAuth 2.0 access token as a bearer, and needs the role its operation names. \ The routes of the setup key take that key as the bearer instead." .into(), @@ -68,13 +69,25 @@ fn upgrade(stream: &Stream, id: &str) -> Value { }) } -/// Put a host's paths under `/v1/hosts/{name}` and its schemas beside ours. +/// Whether a path of berthd is one the server calls only for itself, which +/// it does not pass through and so does not describe as its own. +fn server_only(path: &str) -> bool { + berth_api::ops::HOST + .iter() + .any(|op| op.path == path && !op.server_only.is_empty()) +} + +/// Put a host's paths under `/v1/hosts/{name}/controller` and its schemas +/// beside ours. fn merge_host(document: &mut Value, host: &Value) { let name = json!({"name": "name", "in": "path", "required": true, "description": NAME, "schema": {"type": "string"}}); for (path, item) in host["paths"].as_object().into_iter().flatten() { let Some(rest) = path.strip_prefix("/v1") else { continue; }; + if server_only(path) { + continue; + } let mut item = item.clone(); for (_, operation) in item.as_object_mut().into_iter().flatten() { if !operation.is_object() { @@ -99,11 +112,12 @@ fn merge_host(document: &mut Value, host: &Value) { .collect(); operation["tags"] = json!(tags); } - let target = format!("/v1/hosts/{{name}}{rest}"); - // What berth serve answers itself is not the host's. - if document["paths"].get(&target).is_none() { - document["paths"][target] = item; - } + let target = format!("/v1/hosts/{{name}}/controller{rest}"); + assert!( + document["paths"].get(&target).is_none(), + "{target} is a route of berth serve and of the host" + ); + document["paths"][target] = item; } for (name, schema) in host["components"]["schemas"] .as_object() @@ -123,7 +137,7 @@ pub fn operations() -> Value { let own = serde_json::to_value(own).expect("the document is JSON"); let mut document = berthd::http::operations_of( "berth serve", - "The operations of berth serve. Every operation of a host is one here too: on the portal with `\"host\": \"NAME\"` in the frame, over HTTP under /v1/hosts/NAME. Only the `secrets` task is refused; the host's `secrets` action sends the server's own. An action's own role is asked when it runs. Each operation names the least role a caller over TCP needs: view, operate or admin.", + "The operations of berth serve. Every operation of a host is one here too: on the portal with `\"host\": \"NAME\"` in the frame, over HTTP under /v1/hosts/NAME/controller. Only the controller's own `secrets` action is refused, for which the server's `secrets` action of the host sends the server's secrets, and `endpoints.connect`, which the server opens itself for `resources.connect`. An action's own role is asked when it runs. Each operation names the least role a caller over TCP needs: view, operate or admin.", berth_api::ops::SERVE, &own, &|op| streams::SERVE.iter().find(|stream| stream.path == op.path), @@ -136,10 +150,11 @@ pub fn asyncapi() -> Value { let keep = |text: String| -> &'static str { Box::leak(text.into_boxed_str()) }; let hosted: Vec = streams::HOST .iter() + .filter(|stream| !server_only(stream.path)) .map(|stream| Stream { id: keep(format!("host.{}", stream.id)), path: keep(format!( - "/v1/hosts/{{name}}{}", + "/v1/hosts/{{name}}/controller{}", stream.path.trim_start_matches("/v1") )), summary: stream.summary, @@ -153,7 +168,7 @@ pub fn asyncapi() -> Value { let all: Vec<&Stream> = streams::SERVE.iter().chain(&hosted).collect(); streams::asyncapi( "berth serve", - "The WebSockets of berth serve: its own, and each host's under /v1/hosts/{name}. \ + "The WebSockets of berth serve: its own, and each host's under /v1/hosts/{name}/controller. \ Text frames are JSON, one message each; binary frames carry only session and job bytes.", &all, ) @@ -169,13 +184,13 @@ mod tests { let paths = [ "/v1/hosts", "/v1/hosts/{name}/actions/{action}", - "/v1/actions", + "/v1/installation/actions", "/v1/jobs/{id}/output", "/v1/inbox/{host}/{id}/done", "/v1/kv/{namespace}/{key}", // The host's own, under its name. - "/v1/hosts/{name}/sessions/{id}/output", - "/v1/hosts/{name}/tasks", + "/v1/hosts/{name}/controller/sessions/{id}/output", + "/v1/hosts/{name}/controller/actions", ]; for path in paths { assert!(document["paths"].get(path).is_some(), "{path}"); @@ -222,7 +237,7 @@ mod tests { let streams = asyncapi(); assert_eq!( streams["channels"]["host.session"]["address"], - "/v1/hosts/{name}/sessions/{id}" + "/v1/hosts/{name}/controller/sessions/{id}" ); assert_eq!(streams["channels"]["events"]["address"], "/v1/events"); } @@ -238,16 +253,21 @@ mod tests { "components": {"schemas": {"Done": {"ours": false}, "Session": {}}}, }); merge_host(&mut document, &host); - let get = &document["paths"]["/v1/hosts/{name}/sessions/{id}"]["get"]; + let get = &document["paths"]["/v1/hosts/{name}/controller/sessions/{id}"]["get"]; assert_eq!(get["operationId"], "host_session"); assert_eq!(get["parameters"][0]["name"], "name"); assert_eq!(get["parameters"][1]["name"], "id"); assert_eq!(get["tags"][0], "host: sessions"); + // The host's actions and the server's own of the host are apart. assert!( document["paths"]["/v1/hosts/{name}/actions"]["post"] .get("operationId") .is_none() ); + assert_eq!( + document["paths"]["/v1/hosts/{name}/controller/actions"]["post"]["operationId"], + "host_theirs" + ); assert_eq!(document["components"]["schemas"]["Done"]["ours"], true); assert!(document["components"]["schemas"].get("Session").is_some()); } diff --git a/crates/berth/src/local/serve/here.rs b/crates/berth/src/local/serve/here.rs new file mode 100644 index 0000000..728b387 --- /dev/null +++ b/crates/berth/src/local/serve/here.rs @@ -0,0 +1,372 @@ +//! This installation's API for a client in the same process, which berth's +//! screens use when no server is set: the router of berth serve's own +//! socket, over pipes in memory. Like a caller on that socket, the caller is +//! the user the installation belongs to, so it is an admin and shows no +//! token. Nothing listens anywhere: only this process can reach it. + +use super::{Serve, api, auth}; +use crate::Result; +use berth_api::client::{Dial, Error, Stream}; +use berth_core::{paths::Paths, store}; +use std::sync::Arc; +use std::time::Duration; +use tokio::io::DuplexStream; +use tokio::sync::mpsc::{UnboundedReceiver, UnboundedSender, unbounded_channel}; + +/// How much a pipe holds in each direction before a writer waits. +const PIPE: usize = 256 * 1024; + +/// The socket a running `berth serve` of this user answers on, which is +/// only its user's. +pub fn socket() -> std::path::PathBuf { + berth_core::link::runtime_dir().join("serve.sock") +} + +/// The server berth goes through when it is set to use none: a `berth +/// serve` of this user on this machine, or else one started in this process, +/// which the caller keeps while it is needed. The address names either. +pub fn local(paths: &Paths, ask: bool) -> Result<(String, Option)> { + let socket = socket(); + if std::os::unix::net::UnixStream::connect(&socket).is_ok() { + return Ok((format!("unix:{}", socket.display()), None)); + } + let here = Here::start(paths, ask)?; + Ok((berth_page::device::HERE.to_owned(), Some(here))) +} + +/// Wait for the jobs a command started in its own server, saying on +/// standard error which they are. +pub fn finish(here: Option) { + if let Some(here) = here { + here.wait_for_jobs(|jobs| { + let names: Vec = jobs + .iter() + .map(|job| format!("{} {}", job.action, job.host).trim().to_owned()) + .collect(); + eprintln!( + "berth: waiting for what runs in this berth to end: {}. Ctrl+C stops it unfinished; berth serve keeps jobs running after berth ends.", + names.join(", ") + ); + }); + } +} + +/// The server of this process, while it runs. +static OWN: std::sync::Mutex> = + std::sync::Mutex::new(std::sync::Weak::new()); + +/// Give the server of this process a host's sudo password, which it keeps +/// until the process ends and stores nowhere. Whether one runs. +pub fn give_sudo_password(name: &str, password: &str) -> bool { + let Some(state) = OWN.lock().unwrap().upgrade() else { + return false; + }; + state + .passwords + .lock() + .unwrap() + .insert(name.to_owned(), password.to_owned()); + // Taken up again with the password at the next call. + state.forget(name); + true +} + +/// The pipes clients open, as connections that axum serves. +struct Pipes(UnboundedReceiver); + +impl axum::serve::Listener for Pipes { + type Io = DuplexStream; + type Addr = (); + + async fn accept(&mut self) -> (DuplexStream, ()) { + match self.0.recv().await { + Some(pipe) => (pipe, ()), + // Every client is gone, and none can come. + None => std::future::pending().await, + } + } + + fn local_addr(&self) -> std::io::Result<()> { + Ok(()) + } +} + +pub struct Here { + state: Arc, + connections: UnboundedSender, + /// Runs the server's tasks, and ends them when dropped. + _runtime: tokio::runtime::Runtime, +} + +impl Here { + /// Open the store and serve its API to this process. The hosts' channels + /// are left to `berth serve`, and so are jobs a server before left + /// unfinished, which another berth beside this one may be running. With + /// `ask`, a host whose sudo asks for a password has it asked for on the + /// terminal first, as `berth serve` does when it starts. + pub fn start(paths: &Paths, ask: bool) -> Result { + let store = super::open_store(paths)?; + crate::local::restore_plugins(paths); + let mut state = Serve::with(paths.clone(), store, auth::Auth::nobody()); + if let Some(state) = Arc::get_mut(&mut state) { + state.follows = false; + state.here = true; + } + *OWN.lock().unwrap() = Arc::downgrade(&state); + std::fs::create_dir_all(&state.jobs_dir)?; + let runtime = tokio::runtime::Builder::new_multi_thread() + .worker_threads(2) + .thread_name("berth-here") + .enable_all() + .build()?; + let (connections, pipes) = unbounded_channel(); + runtime.block_on(async { state.refresh(ask) }); + let router = api::router(state.clone(), false); + runtime.spawn(async move { + let _ = axum::serve(Pipes(pipes), router).await; + }); + // Another berth beside this one changes the store too. + let watching = state.clone(); + runtime.spawn(async move { + loop { + tokio::time::sleep(Duration::from_secs(5)).await; + if watching.events.receiver_count() > 0 { + watching.notice(); + } + } + }); + Ok(Here { + state, + connections, + _runtime: runtime, + }) + } + + /// A new connection to the server for each call. + pub fn dial(&self) -> Dial { + let connections = self.connections.clone(); + Arc::new(move || { + let connections = connections.clone(); + Box::pin(async move { + let (client, server) = tokio::io::duplex(PIPE); + connections + .send(server) + .map_err(|_| Error::Transport("berth's own server has stopped".into()))?; + Ok(Box::new(client) as Stream) + }) + }) + } + + /// The jobs that run in this process, which end when it does. + pub fn running(&self) -> Vec { + self.state.runner.running() + } + + /// Wait until no job runs in this process, saying first which do. + pub fn wait_for_jobs(&self, say: impl Fn(&[store::Job])) { + let running = self.running(); + if running.is_empty() { + return; + } + say(&running); + while !self.running().is_empty() { + std::thread::sleep(Duration::from_millis(200)); + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use berth_api::client::{self, Client, Frame}; + use futures_util::StreamExt; + use serde_json::{Value, json}; + + fn scratch() -> (tempfile::TempDir, Paths) { + let dir = tempfile::tempdir().unwrap(); + let paths = Paths { + config_dir: dir.path().join("config"), + data_dir: dir.path().join("data"), + }; + (dir, paths) + } + + #[test] + fn a_client_in_the_process_calls_streams_and_runs_a_job_to_its_end() { + let (_dir, paths) = scratch(); + let profile = json!({"hostname": "dock", "builder": {}, "access": {"address": "192.0.2.7", "link": ["sh", "-c", "exit 1"]}}); + berth_core::profiles::Place::own("dock") + .unwrap() + .save(&paths, &profile) + .unwrap(); + let here = Here::start(&paths, false).unwrap(); + let dial = here.dial(); + // The client has a runtime of its own, as the screens do. + let client_runtime = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .unwrap(); + client_runtime.block_on(async { + // A call, as the owner, with no token. + let mut client = Client::over(dial().await.unwrap()).await.unwrap(); + let access: Value = client.get("/v1/access").await.unwrap(); + assert_eq!( + (&access["role"], &access["setup"]), + (&json!("admin"), &json!(true)) + ); + + // The portal over a connection of its own. + let socket = client::websocket(dial().await.unwrap(), berth_api::ops::PORTAL, "") + .await + .unwrap(); + let portal = berth_api::portal::Portal::over(socket); + let hosts = portal + .call("hosts.list", None, json!({}), None) + .await + .unwrap(); + assert_eq!(hosts[0]["name"], "dock", "{hosts}"); + + // A job, followed on its stream to its end. + let body = json!({"arguments": {"name": "studio", "hostname": "yes"}}); + let answer = client + .send( + "POST", + "/v1/hosts/dock/actions/rename", + &[("content-type", "application/json")], + body.to_string().into(), + ) + .await + .unwrap(); + assert_eq!(answer.status, 202); + let acted: Value = serde_json::from_slice(&answer.body).unwrap(); + let id = acted["job"]["id"].as_str().unwrap().to_owned(); + let mut stream = + client::websocket(dial().await.unwrap(), &format!("/v1/jobs/{id}"), "") + .await + .unwrap(); + let ended = loop { + let frame = tokio::time::timeout(Duration::from_secs(20), stream.next()) + .await + .expect("a frame in time"); + match frame { + Some(Ok(Frame::Text(text))) => { + let said: Value = serde_json::from_str(text.as_str()).unwrap(); + if said["exited"].is_object() { + break said["exited"].clone(); + } + } + Some(Ok(_)) => {} + other => panic!("the stream ended: {other:?}"), + } + }; + assert_eq!(ended["exit_code"], 0, "{ended}"); + }); + here.wait_for_jobs(|_| {}); + assert!(here.running().is_empty()); + assert!( + berth_core::profiles::Place::own("studio") + .unwrap() + .load(&paths) + .unwrap() + .is_some() + ); + } + + fn screen(app: &mut berth_page::App) -> String { + use ratatui::{Terminal, backend::TestBackend}; + let mut terminal = Terminal::new(TestBackend::new(120, 30)).unwrap(); + terminal.draw(|frame| app.draw(frame)).unwrap(); + let buffer = terminal.backend().buffer(); + (0..30) + .map(|y| { + (0..120) + .map(|x| buffer[(x, y)].symbol()) + .collect::() + }) + .collect::>() + .join("\n") + } + + #[test] + fn the_screens_list_and_open_a_host_through_it() { + let (dir, paths) = scratch(); + for (name, address) in [("studio", "192.0.2.10"), ("dock", "192.0.2.11")] { + let profile = json!({"hostname": name, "builder": {}, "access": {"address": address, "link": ["sh", "-c", "exit 1"]}}); + berth_core::profiles::Place::own(name) + .unwrap() + .save(&paths, &profile) + .unwrap(); + } + let here = Here::start(&paths, false).unwrap(); + berth_page::net::serve_here(here.dial()); + let kept = dir.path().join("client"); + let linked = berth_page::device::Linked { + server: Some(berth_page::device::HERE.into()), + ..Default::default() + }; + berth_page::device::install( + linked, + berth_page::settings::Settings::new(Some(kept.clone())), + ); + let mut app = berth_page::App::new(); + app.open_when_listed("dock"); + let started = std::time::Instant::now(); + let shown = loop { + berth_page::pump(); + app.tick(); + let shown = screen(&mut app); + if shown.contains("Connection") && shown.contains("192.0.2.11") { + break shown; + } + assert!( + started.elapsed() < Duration::from_secs(10), + "the host did not open:\n{shown}" + ); + std::thread::sleep(Duration::from_millis(20)); + }; + // Its own server needs no token and says where it is, and is never kept. + assert!(!shown.contains("through the berth server"), "{shown}"); + assert!(!kept.join("config.json").exists()); + app.event(berth_tui::Event::Key(berth_tui::Key::plain( + berth_tui::Code::Esc, + ))); + app.tick(); + let home = screen(&mut app); + assert!( + home.contains("this machine") && home.contains("studio"), + "{home}" + ); + app.event(berth_tui::Event::Key(berth_tui::Key::plain( + berth_tui::Code::Char('q'), + ))); + assert!(app.quit()); + } + + #[test] + fn nothing_listens_for_it() { + let (dir, paths) = scratch(); + let _here = Here::start(&paths, false).unwrap(); + // No socket, no token and no port are made for a server in the process. + assert!(!paths.data_dir.join("serve.token").exists()); + let made: Vec<_> = walk(dir.path()); + assert!( + !made + .iter() + .any(|path| path.extension().is_some_and(|end| end == "sock")), + "{made:?}" + ); + } + + fn walk(dir: &std::path::Path) -> Vec { + let mut found = vec![]; + for entry in std::fs::read_dir(dir).into_iter().flatten().flatten() { + let path = entry.path(); + if path.is_dir() { + found.extend(walk(&path)); + } else { + found.push(path); + } + } + found + } +} diff --git a/crates/berth/src/serve/keys.rs b/crates/berth/src/local/serve/keys.rs similarity index 100% rename from crates/berth/src/serve/keys.rs rename to crates/berth/src/local/serve/keys.rs diff --git a/crates/berth/src/serve/mod.rs b/crates/berth/src/local/serve/mod.rs similarity index 70% rename from crates/berth/src/serve/mod.rs rename to crates/berth/src/local/serve/mod.rs index a020f21..9302ef7 100644 --- a/crates/berth/src/serve/mod.rs +++ b/crates/berth/src/local/serve/mod.rs @@ -1,14 +1,13 @@ //! `berth serve`: one API over all hosts, with the store. It reaches each //! host through a link, as the UI does, and adds nothing to a host's own API. -//! The commands that only read the store (`inbox`, `kv`) are here too. mod actions; mod api; mod auth; +mod connect; mod docs; +pub mod here; pub(crate) mod keys; -#[cfg(feature = "web")] -mod page; mod provisioners; mod resources; mod setup; @@ -20,12 +19,12 @@ use crate::Result; use berth_api::{Message, Version, client}; use berth_core::{ host::Host, - link::{Link, LinkSpec, runtime_dir}, + link::{Link, LinkSpec}, paths::Paths, - store::{self, Filter, Store, Stored, When}, + store::{self, Store, Stored}, }; use futures_util::StreamExt; -use serde_json::{Value, json}; +use serde_json::Value; use std::{ collections::BTreeMap, path::PathBuf, @@ -57,6 +56,130 @@ pub enum Event { value: Option, revision: i64, }, + /// A saved host was added, removed or renamed, or its profile changed. + Host { + name: String, + change: &'static str, + /// Its name before, when it was renamed. + from: Option, + }, + /// A resource was created, removed, or its record changed. + Resource { + id: String, + provisioner: String, + kind: String, + change: &'static str, + }, +} + +/// What the store held of the hosts and resources when last looked at, so a +/// change to it can be told as an event whoever made it. +struct Watch { + paths: Paths, + events: broadcast::Sender, + seen: Mutex, +} + +/// Unknown until the store could first be read, which tells nothing. +#[derive(Default)] +struct Seen { + hosts: Option>, + resources: Option>, +} + +/// How each key of `now` differs from `before`: added, removed or changed. +fn differences<'a, T: PartialEq>( + before: &'a BTreeMap, + now: &'a BTreeMap, +) -> Vec<(&'a str, &'static str)> { + let gone = before + .keys() + .filter(|name| !now.contains_key(*name)) + .map(|name| (name.as_str(), "removed")); + let came = now + .iter() + .filter_map(|(name, value)| match before.get(name) { + None => Some((name.as_str(), "added")), + Some(was) if was != value => Some((name.as_str(), "changed")), + Some(_) => None, + }); + gone.chain(came).collect() +} + +impl Watch { + fn hosts(&self) -> Option> { + berth_core::profiles::list(&self.paths, store::OWN) + .ok() + .map(|saved| saved.into_iter().collect()) + } + + fn resources(&self) -> Option> { + let kept = berth_core::resource::list(&self.paths).ok()?; + Some( + kept.into_iter() + .map(|kept| { + // When it was last looked at is no change of it. + let record = berth_core::resource::Record { + checked: String::new(), + ..kept.record + }; + (record.id.clone(), record) + }) + .collect(), + ) + } + + /// Tell what changed in the store since it was last looked at. + fn notice(&self) { + let (hosts, resources) = (self.hosts(), self.resources()); + // Held while telling, so two callers cannot tell a change twice or out of order. + let mut seen = self.seen.lock().unwrap(); + if let Some(now) = hosts { + if let Some(before) = &seen.hosts { + for (name, change) in differences(before, &now) { + let _ = self.events.send(Event::Host { + name: name.to_owned(), + change, + from: None, + }); + } + } + seen.hosts = Some(now); + } + if let Some(now) = resources { + if let Some(before) = &seen.resources { + for (id, change) in differences(before, &now) { + let record = now.get(id).or_else(|| before.get(id)); + let change = match change { + "added" => "created", + other => other, + }; + let _ = self.events.send(Event::Resource { + id: id.to_owned(), + provisioner: record.map(|r| r.provisioner.clone()).unwrap_or_default(), + kind: record.map(|r| r.kind.clone()).unwrap_or_default(), + change, + }); + } + } + seen.resources = Some(now); + } + } + + /// Tell a rename as one, and not as a host removed and another added. + fn renamed(&self, old: &str, new: &str) { + let mut seen = self.seen.lock().unwrap(); + if let Some(hosts) = seen.hosts.as_mut() + && let Some(profile) = hosts.remove(old) + { + hosts.insert(new.to_owned(), profile); + } + let _ = self.events.send(Event::Host { + name: new.to_owned(), + change: "renamed", + from: Some(old.to_owned()), + }); + } } /// A host as `berth serve` knows it. @@ -103,8 +226,7 @@ impl Remote { if let Some(refused) = link.refused() { return Err(refused); } - let asked = - tokio::time::timeout(Duration::from_secs(15), link.get::("/v1/version")).await; + let asked = tokio::time::timeout(Duration::from_secs(15), link.version()).await; let answer = match asked { Ok(Ok(version)) => Ok(version), Ok(Err(error)) if Link::missing(&error) => Err(format!("{NO_BERTHD}: {error}")), @@ -133,6 +255,17 @@ pub struct Serve { /// How long provisioners may take and how a new machine is waited for, /// which tests make quick. pub provision: fn() -> berth_core::provision::Options, + /// Whether the hosts' channels are taken in, which only `berth serve` + /// does, holding its lock. + pub follows: bool, + /// Whether it serves the process it runs in alone, for a command or the + /// screens, rather than as `berth serve`. + pub here: bool, + /// The known-hosts file of the user the server runs as, which a first + /// login from a caller on this machine takes a host's key from. Tests + /// leave it out or name one of their own. + pub known_hosts: Mutex>, + watch: Arc, } impl Serve { @@ -145,9 +278,17 @@ impl Serve { pub fn with(paths: Paths, store: Arc, auth: auth::Auth) -> Arc { let jobs_dir = berth_core::runner::jobs_dir(&paths); let events = broadcast::channel(1024).0; - let told = events.clone(); + let watch = Arc::new(Watch { + paths: paths.clone(), + events: events.clone(), + seen: Mutex::new(Seen::default()), + }); + watch.notice(); + let (told, watching) = (events.clone(), watch.clone()); let notify = Arc::new(move |job: &store::Job| { let _ = told.send(Event::Job(job.json())); + // A job is what changes hosts and resources, besides the routes. + watching.notice(); }); let runner = berth_core::runner::Runner::new(paths.clone(), store.clone(), jobs_dir.clone(), notify); @@ -162,9 +303,23 @@ impl Serve { followers: Mutex::new(BTreeMap::new()), passwords: Mutex::new(BTreeMap::new()), provision: Default::default, + follows: true, + here: false, + known_hosts: Mutex::new(match cfg!(test) { + true => None, + false => std::env::var_os("HOME") + .map(|home| PathBuf::from(home).join(".ssh/known_hosts")), + }), + watch, }) } + /// Tell whoever follows changes what changed among the saved hosts and + /// the resources since this was last asked. + pub fn notice(&self) { + self.watch.notice(); + } + pub fn host(&self, name: &str) -> Option> { self.hosts.read().unwrap().get(name).cloned() } @@ -176,7 +331,7 @@ impl Serve { .write() .unwrap() .insert(remote.name.clone(), remote.clone()); - if remote.link.is_some() { + if remote.link.is_some() && self.follows { let follower = tokio::spawn(take_in(self.clone(), remote.clone())); let before = self .followers @@ -204,6 +359,7 @@ impl Serve { /// [`Serve::refresh`] takes it up under that name. pub fn renamed(&self, old: &str, new: &str) { self.forget(old); + self.watch.renamed(old, new); let mut passwords = self.passwords.lock().unwrap(); if let Some(password) = passwords.remove(old) { passwords.insert(new.to_owned(), password); @@ -219,6 +375,7 @@ impl Serve { let Ok(saved) = berth_core::profiles::list(&self.paths, store::OWN) else { return; }; + self.notice(); let stale: Vec = { let hosts = self.hosts.read().unwrap(); hosts @@ -262,13 +419,16 @@ impl Serve { .get(&host.name) .cloned() .or_else(|| host.kept_sudo_password(&self.paths)) - .or_else(|| std::env::var("BERTH_SUDO_PASSWORD").ok()) .map(Some) - .ok_or_else(|| { - format!( - "{} asks for a sudo password: start berth serve again", - host.name - ) + .ok_or_else(|| match self.here { + true => format!( + "{name}'s SSH user's sudo asks for a password, and berth was given none. Give it with --sudo-password-stdin, or keep it with berth secret set sudo/{name}; on a terminal berth asks for it", + name = host.name + ), + false => format!( + "{name}'s SSH user's sudo asks for a password, which this server was not given. Keep it with berth secret set sudo/{name}, or run berth serve on a terminal, which asks for it", + name = host.name + ), }), }; if let Ok(Some(password)) = &password { @@ -403,48 +563,26 @@ fn lock(paths: &Paths) -> Result { Ok(file) } -const USAGE: &str = "Usage: berth serve [--listen ADDRESS:PORT [--expose] [--web]]"; - -/// A berth set to use a berth server keeps that server's hosts, so its -/// commands, and the ones a serve starts, would take a host's name for one of -/// those. berth serve serves this installation's own hosts, and does not -/// start like that. -fn refuse_a_server(paths: &Paths) -> Result<()> { - match berth_core::server::Server::configured(paths) { - Some(server) => Err(format!( - "This berth uses the berth server at {}, so its hosts are that server's. berth serve serves the hosts of this machine: stop using the other server with berth config server \"\" (and unset BERTH_SERVER), then start berth serve again", - server.url - ) - .into()), - None => Ok(()), - } -} +const USAGE: &str = "Usage: berth serve [--listen ADDRESS:PORT [--expose]]"; -/// `berth serve [--listen ADDRESS:PORT [--expose] [--web]]` +/// `berth serve [--listen ADDRESS:PORT [--expose]]` /// /// Over TCP every request shows the token, but the connection is plain HTTP, /// so an address other than this machine's own takes `--expose` as well: for a -/// container whose port is published, or behind a proxy that adds TLS. With -/// `--web` the address also gives the hosts as a page for a browser. +/// container whose port is published, or behind a proxy that adds TLS. pub fn command(paths: &Paths, args: &[&str]) -> Result<()> { - let (mut listen, mut expose, mut web) = (None, false, false); + let (mut listen, mut expose) = (None, false); let mut words = args.iter(); while let Some(word) = words.next() { match *word { "--listen" => listen = Some(*words.next().ok_or(USAGE)?), "--expose" => expose = true, - "--web" => web = true, _ => return Err(USAGE.into()), } } - if listen.is_none() && (expose || web) { + if listen.is_none() && expose { return Err(USAGE.into()); } - if web && !cfg!(feature = "web") { - return Err( - "This berth is built without the page for a browser: the berth-web build has it".into(), - ); - } let listen = match listen { None => None, Some(address) => { @@ -460,17 +598,16 @@ pub fn command(paths: &Paths, args: &[&str]) -> Result<()> { Some(address) } }; - refuse_a_server(paths)?; let _lock = lock(paths)?; let store = open_store(paths)?; store.end_lost_jobs()?; // From the next step on the store holds keys and passwords. if store.backend_name() == "postgres" && !store.encrypted() { eprintln!( - "berth serve: the PostgreSQL database holds this installation's keys and passwords readable; berth store encrypt seals them" + "berth serve: the PostgreSQL database holds this installation's keys and passwords readable; berth server store encrypt seals them" ); } - crate::restore_plugins(paths); + crate::local::restore_plugins(paths); let token = match listen { Some(_) => Some(token(paths)?), None => None, @@ -492,8 +629,18 @@ pub fn command(paths: &Paths, args: &[&str]) -> Result<()> { daily.prune(); } }); + // Another berth beside the server changes the store too. + let watching = state.clone(); + tokio::spawn(async move { + loop { + tokio::time::sleep(Duration::from_secs(5)).await; + if watching.events.receiver_count() > 0 { + watching.notice(); + } + } + }); - let socket = runtime_dir().join("serve.sock"); + let socket = here::socket(); let _ = std::fs::remove_file(&socket); let unix = tokio::net::UnixListener::bind(&socket)?; { @@ -512,14 +659,6 @@ pub fn command(paths: &Paths, args: &[&str]) -> Result<()> { state.paths.data_dir.join("serve.token").display() ); let guarded = api::router(state.clone(), true); - #[cfg(feature = "web")] - let guarded = match web { - true => { - println!("The hosts in a browser: http://{}/", tcp.local_addr()?); - guarded.merge(page::router()) - } - false => guarded, - }; Some(tokio::spawn(async move { axum::serve(tcp, guarded).await })) } None => None, @@ -535,69 +674,3 @@ pub fn command(paths: &Paths, args: &[&str]) -> Result<()> { Ok(()) }) } - -/// `berth inbox [HOST] [--topic PREFIX] [--all]`: what the store holds. -pub fn inbox_command(paths: &Paths, args: &[&str]) -> Result<()> { - let (mut host, mut topic, mut all) = (None, None, false); - let mut words = args.iter(); - while let Some(word) = words.next() { - match *word { - "--all" => all = true, - "--topic" => topic = Some(*words.next().ok_or("--topic needs a prefix")?), - word if word.starts_with('-') => { - return Err("Usage: berth inbox [HOST] [--topic PREFIX] [--all]".into()); - } - word => host = Some(word), - } - } - let store = open_store(paths)?; - let filter = Filter { - host, - topic, - before: None, - done: (!all).then_some(false), - limit: 200, - }; - let (messages, next) = store.inbox(&filter)?; - for stored in messages.iter().rev() { - let message = &stored.message; - let mark = if stored.done { "✓" } else { " " }; - println!( - "{mark} {} {}#{} {} {}", - message.time, stored.host, message.id, message.topic, message.body - ); - } - if next.is_some() { - eprintln!("(older messages not shown)"); - } - Ok(()) -} - -/// `berth kv get|set|rm|ls NAMESPACE [KEY] [JSON]` -pub fn kv_command(paths: &Paths, args: &[&str]) -> Result<()> { - let store = open_store(paths)?; - match *args { - ["ls", namespace] => { - for entry in store.list(namespace)? { - println!("{}\t{}\t{}", entry.key, entry.revision, entry.value); - } - } - ["get", namespace, key] => match store.get(namespace, key)? { - Some(entry) => println!("{}", entry.value), - None => return Err(format!("No {key} in {namespace}").into()), - }, - ["set", namespace, key, value] => { - // Anything that is not JSON is taken as text. - let value = serde_json::from_str(value).unwrap_or_else(|_| json!(value)); - let revision = store.put(namespace, key, &value, When::Always)?; - println!("{key} is at revision {revision}"); - } - ["rm", namespace, key] => { - if !store.remove(namespace, key, When::Always)? { - return Err(format!("No {key} in {namespace}").into()); - } - } - _ => return Err("Usage: berth kv get|set|rm|ls NAMESPACE [KEY] [JSON]".into()), - } - Ok(()) -} diff --git a/crates/berth/src/serve/provisioners.rs b/crates/berth/src/local/serve/provisioners.rs similarity index 97% rename from crates/berth/src/serve/provisioners.rs rename to crates/berth/src/local/serve/provisioners.rs index eabf040..17df650 100644 --- a/crates/berth/src/serve/provisioners.rs +++ b/crates/berth/src/local/serve/provisioners.rs @@ -52,7 +52,11 @@ fn known(state: &Serve, store: &Store, name: &str) -> Answer { provision::list(&state.paths, store) .into_iter() .find(|known| known.id == name) - .ok_or_else(|| missing(format!("No provisioner {name}. See: berth provisioners"))) + .ok_or_else(|| { + missing(format!( + "No provisioner {name}. See: berth provisioner list" + )) + }) } #[utoipa::path(get, path = "/v1/provisioners", summary = "What the plugins can create machines and other resources with, what each asks for and can do, and whether it can be used here.", tag = "set-up", @@ -173,7 +177,9 @@ pub async fn provisioner_act( })); } let mut request = Request::new(Subject::Provisioner(name), &action); + request.arguments = berth_api::Argument::shown(&found.arguments, &act.arguments); request.token = act.token; + request.role = found.role; request.secrets = acting.secrets(); let work = Work::Function(Box::new(move |out| { match acting.run(&paths, &options, out) { diff --git a/crates/berth/src/serve/resources.rs b/crates/berth/src/local/serve/resources.rs similarity index 93% rename from crates/berth/src/serve/resources.rs rename to crates/berth/src/local/serve/resources.rs index aeaebad..a99eae0 100644 --- a/crates/berth/src/serve/resources.rs +++ b/crates/berth/src/local/serve/resources.rs @@ -7,7 +7,7 @@ use super::{ Serve, actions::{admit, answer, parse, refused}, - api::{Answer, App, Fail, Ws, caller, internal, job_info}, + api::{Answer, App, Fail, Said, Ws, caller, internal, job_info, turned_down}, auth::Access, }; use axum::{ @@ -39,9 +39,12 @@ fn bad(error: impl std::fmt::Display) -> Fail { /// A refusal of the core as an answer: a missing resource is 404, one that /// is busy 409, anything else a request that does not fit. -fn refusal(error: impl std::fmt::Display) -> Fail { - let message = error.to_string(); - if message.starts_with("No resource ") || message.starts_with("No provisioner ") { +fn refusal(error: impl Into) -> Fail { + let said = error.into(); + let message = said.message.clone(); + if said.refused { + turned_down(said) + } else if message.starts_with("No resource ") || message.starts_with("No provisioner ") { Fail::new(code::NOT_FOUND, message) } else if message.starts_with("Something else is being done to") || message.contains(" already, ") @@ -59,8 +62,8 @@ fn store(state: &Serve) -> Store { /// Run work of the core off the runtime: it waits for the provisioner. async fn blocking( work: impl FnOnce() -> berth_core::Result + Send + 'static, -) -> Answer> { - tokio::task::spawn_blocking(move || work().map_err(|error| error.to_string())) +) -> Answer> { + tokio::task::spawn_blocking(move || work().map_err(Said::from)) .await .map_err(internal) } @@ -119,6 +122,8 @@ pub async fn get(State(state): App, Path(id): Path) -> Answer { return Ok(Acted { @@ -262,15 +275,18 @@ async fn run_action( } Reserved::New(ticket) => ticket, }; - let work = Work::Function(Box::new(move |out| { + // The job ends as the command did, or with 1 when it could not be run. + let work = Work::Coded(Box::new(move |out| { match acting.run(&paths, &options, out) { + // A command's output is its own, and its exit status says how it went. + Ok(answered) if exec => answered.exit_code.unwrap_or(0), Ok(answered) => { say(out, &answered); - true + answered.exit_code.unwrap_or(0) } Err(error) => { let _ = writeln!(out, "✗ {error}"); - false + 1 } } })); diff --git a/crates/berth/src/serve/setup.rs b/crates/berth/src/local/serve/setup.rs similarity index 92% rename from crates/berth/src/serve/setup.rs rename to crates/berth/src/local/serve/setup.rs index 24e3305..084d844 100644 --- a/crates/berth/src/serve/setup.rs +++ b/crates/berth/src/local/serve/setup.rs @@ -5,11 +5,12 @@ use super::{ Serve, api::{Answer, App, Fail, internal}, + auth::Access, }; use axum::{ Json, body::Bytes, - extract::{Path, State}, + extract::{Extension, Path, State}, }; use berth_api::{ Done, ErrorBody, PluginList, PluginOrigin, PluginReport, PluginUpdateSpec, PluginUpdates, code, @@ -68,27 +69,32 @@ pub async fn profile(State(state): App, Path(name): Path) -> Answer, + access: Option>, Json(mut profile): Json, ) -> Answer> { + // A caller on the socket or in berth itself is the server's own user. + let own_user = access.is_none(); let place = own_place(&name)?; if !profile.is_object() { return Err(bad("A profile is an object")); } // A caller of an older layout: its access fields are read as now. A key // file it names is on another machine, so the server logs in with its own. - if let Some(builder) = profile["builder"].as_object_mut() { + if let Some(builder) = profile.get_mut("builder").and_then(Value::as_object_mut) { builder.remove("ssh_key"); } - berth_core::import::convert_profile(&state.paths, &name, &mut profile); - // A workstation's has its set-up's part; a host below that, only how it is reached. + berth_core::import::convert_layout(&state.paths, &name, &mut profile); + // An older caller's pasted tailnet key is never stored: berth makes the keys. + berth_api::profile::take_old_tailnet(&mut profile); + // A workstation's has its set-up's part; a host of another type, only how it is reached. let problems = berth_api::profile::problems(&profile); if !problems.is_empty() { return Err(bad(problems.join("; "))); @@ -98,11 +104,13 @@ pub async fn save_profile( let store = berth_core::store::shared(&state.paths).map_err(internal)?; let known = berth_core::keys::ring::names(&store).map_err(internal)?; if key != berth_api::profile::BERTH_KEY && !known.iter().any(|known| known == key) { - return Err(bad(format!("The server has no key {key}. See: berth key"))); + return Err(bad(format!( + "The server has no key {key}. See: berth key list" + ))); } } // How the caller reached this server is not part of the host. - if let Some(builder) = profile["builder"].as_object_mut() { + if let Some(builder) = profile.get_mut("builder").and_then(Value::as_object_mut) { builder.remove("served"); builder.remove("served_only"); } @@ -111,13 +119,13 @@ pub async fn save_profile( let kept = place.load(&state.paths).map_err(internal)?; let kept_access = kept.as_ref().map(berth_api::profile::Access::of); let link = kept_access.as_ref().and_then(|kept| kept.link.clone()); - if access.link.is_some() && link != access.link { + if !own_user && access.link.is_some() && link != access.link { return Err(bad( "access.link is a command the server runs, and is set on the server itself", )); } let ssh = kept_access.is_some_and(|kept| kept.auth == berth_api::profile::Auth::Ssh); - if access.auth == berth_api::profile::Auth::Ssh && !ssh { + if !own_user && access.auth == berth_api::profile::Auth::Ssh && !ssh { return Err(bad( "access.auth ssh is the SSH configuration of the server's own account, and is set on the server itself", )); @@ -144,7 +152,7 @@ pub async fn remove_host(State(state): App, Path(name): Path) -> Answer< return Err(Fail::new( code::CONFLICT, format!( - "{name} has a machine berth created, {} of {}. Delete it first: berth resource {} remove", + "{name} has a machine berth created, {} of {}. Delete it first: berth resource remove {}", kept.record.external_id, kept.record.provisioner, kept.record.id ), )); @@ -159,6 +167,7 @@ pub async fn remove_host(State(state): App, Path(name): Path) -> Answer< } profiles::remove(&state.paths, &place).map_err(internal)?; state.forget(&name); + state.notice(); Ok(Json(Done { ok: true })) } @@ -366,7 +375,12 @@ pub struct SettingValue { #[utoipa::path(get, path = "/v1/settings", summary = "The server's settings.", tag = "set-up", responses((status = 200, description = "The settings", body = Vec)))] -pub async fn settings(State(state): App) -> Answer>> { +pub async fn settings( + State(state): App, + access: Option>, +) -> Answer>> { + // The user the server runs as changes any of them, as berth config set does. + let local = access.is_none(); let saved = state.paths.settings().map_err(internal)?; let settings = SETTINGS .iter() @@ -379,26 +393,27 @@ pub async fn settings(State(state): App) -> Answer>> { None => String::new(), }, meaning: (*meaning).into(), - open: OPEN_SETTINGS.contains(key), + open: local || OPEN_SETTINGS.contains(key), }) .collect(); Ok(Json(settings)) } -#[utoipa::path(put, path = "/v1/settings/{key}", summary = "Change one of the settings that are open to callers.", tag = "set-up", +#[utoipa::path(put, path = "/v1/settings/{key}", summary = "Change one of the settings that are open to callers, or any for the user the server runs as.", tag = "set-up", params(("key" = String, Path, description = "The setting")), request_body(content = SettingValue, description = "The value"), responses((status = 200, description = "Saved", body = Done), (status = 403, description = "This one is set on the server itself", body = ErrorBody)))] pub async fn set_setting( State(state): App, + access: Option>, Path(key): Path, Json(setting): Json, ) -> Answer> { - if !OPEN_SETTINGS.contains(&key.as_str()) { + if access.is_some() && !OPEN_SETTINGS.contains(&key.as_str()) { return Err(Fail::new( code::FORBIDDEN, - format!("{key} is set on the server itself, with berth config"), + format!("{key} is set on the server itself, with berth config set"), )); } state.paths.set(&key, &setting.value).map_err(bad)?; @@ -561,6 +576,8 @@ pub struct FeatureInfo { /// The group of the Features page it is listed in. pub group: String, pub help: String, + /// The types of host it is for, such as `workstation` or `node`. + pub types: Vec, } #[derive(Serialize, ToSchema)] @@ -583,11 +600,17 @@ pub async fn catalogue(State(state): App) -> Answer> { let features = found .iter() .flat_map(|plugin| { - plugin.features.iter().map(|feature| FeatureInfo { + let hosting = plugins::hosting(plugin); + plugin.features.iter().map(move |feature| FeatureInfo { key: format!("{}/{}", plugin.id, feature.id), label: feature.label.clone(), group: feature.group.clone(), help: feature.help.to_owned(), + types: hosting + .iter() + .find(|known| known.feature == feature.id) + .map(|known| known.types.clone()) + .unwrap_or_else(|| vec![berth_api::HostType::Workstation]), }) }) .collect(); @@ -602,7 +625,7 @@ pub async fn catalogue(State(state): App) -> Answer> { /// Something for the server to make up for a profile. #[derive(Deserialize, ToSchema)] pub struct Generate { - /// `hostname`, `password` or `tailscale_key`. + /// `hostname` or `password`. pub what: String, /// For `password`: the password to hash. Without it one is made up and /// kept in the server's store with `host`, under `user`, for @@ -614,7 +637,7 @@ pub struct Generate { #[derive(Serialize, ToSchema)] pub struct Generated { - /// The host name, the password's hash, or the key. + /// The host name or the password's hash. pub value: String, /// What to tell whoever asked. pub note: String, @@ -623,7 +646,7 @@ pub struct Generated { pub password: Option, } -#[utoipa::path(post, path = "/v1/setup/generate", summary = "Make up a host name, hash or make up a password, or create a one-use enrollment key at the tailnet's control server.", tag = "set-up", +#[utoipa::path(post, path = "/v1/setup/generate", summary = "Make up a host name, or hash or make up a password.", tag = "set-up", request_body(content = Generate, description = "What to make"), responses((status = 200, description = "What was made", body = Generated), (status = 400, description = "It could not be made; the message says why", body = ErrorBody)))] @@ -657,7 +680,7 @@ pub async fn generate(State(state): App, Json(asked): Json) -> Answer< Ok(Generated { value, note: format!( - "Generated password kept in the server's store; see it with: berth credentials {}", + "Generated password kept in the server's store; see it with: berth host credentials {}", place.argument() ), password: Some(password), @@ -667,21 +690,6 @@ pub async fn generate(State(state): App, Json(asked): Json) -> Answer< Err("A made-up password is kept with a host: name the host".into()) } }, - "tailscale_key" => { - let host = asked.host.unwrap_or_default(); - let config = Place::own(&host) - .and_then(|place| place.load(&paths)) - .ok() - .flatten() - .unwrap_or_default(); - let (value, note) = - berth_core::tailnet::generate(&paths, &config, &host).map_err(fail)?; - Ok(Generated { - value, - note, - password: None, - }) - } other => Err(format!("Nothing to make for {other}")), } }) diff --git a/crates/berth/src/serve/setup_key.rs b/crates/berth/src/local/serve/setup_key.rs similarity index 95% rename from crates/berth/src/serve/setup_key.rs rename to crates/berth/src/local/serve/setup_key.rs index 05928c7..efdd066 100644 --- a/crates/berth/src/serve/setup_key.rs +++ b/crates/berth/src/local/serve/setup_key.rs @@ -2,7 +2,7 @@ //! the first user collect the server's token. Whoever installs the server can //! hand it on without ever holding the token, and it ends when the user says //! they are done or after a day. It is kept in the store and read from there -//! on every request, so a running server takes one that `berth setup-key` +//! on every request, so a running server takes one that `berth server setup-key` //! made in another process a moment ago. use super::{ @@ -74,7 +74,7 @@ fn current(store: &Store) -> Option { Some(stored).filter(|stored| stored.key.len() >= 32) } -/// `berth setup-key [--remove]`, where `berth serve` keeps its data. +/// `berth server setup-key [--revoke]`, where `berth serve` keeps its data. pub fn command(paths: &Paths, args: &[&str]) -> Result<()> { match args { [] => { @@ -82,8 +82,7 @@ pub fn command(paths: &Paths, args: &[&str]) -> Result<()> { println!("{key}"); eprintln!( "This setup key lets the first user collect the token of this berth server, until {} or until they are done.\n\ - In a terminal: berth connect URL and paste the key when asked.\n\ - In a browser: URL/#setup-key=KEY\n\ + With berth: berth login URL and paste the key when asked.\n\ URL is the server's address, such as https://studio.example.ts.net.", rfc3339(expires) ); @@ -97,7 +96,7 @@ pub fn command(paths: &Paths, args: &[&str]) -> Result<()> { } Ok(()) } - _ => Err("Usage: berth setup-key [--remove]".into()), + _ => Err("Usage: berth server setup-key [--revoke]".into()), } } diff --git a/crates/berth/src/serve/tests.rs b/crates/berth/src/local/serve/tests.rs similarity index 70% rename from crates/berth/src/serve/tests.rs rename to crates/berth/src/local/serve/tests.rs index e5b8c17..376f3ec 100644 --- a/crates/berth/src/serve/tests.rs +++ b/crates/berth/src/local/serve/tests.rs @@ -227,6 +227,65 @@ async fn the_key_value_store_over_the_api() { assert_eq!(refused.status, 413); } +fn kv_event(key: &str, value: Value) -> super::Event { + super::Event::Kv { + namespace: "ui".into(), + key: key.into(), + value: Some(value), + revision: 1, + } +} + +/// Wait for a frame that says events were dropped, past the ones before it. +async fn dropped(stream: &mut S) -> u64 +where + S: StreamExt> + Unpin, + E: std::fmt::Debug, +{ + loop { + let frame = text(stream).await; + if let Some(missed) = frame["dropped"]["events"].as_u64() { + return missed; + } + } +} + +#[tokio::test(flavor = "multi_thread")] +async fn a_follower_that_falls_behind_is_told_what_it_missed() { + let served = served().await; + // The events stream listens from the handshake on, before it is told + // what to follow, so all of these are waiting when it starts. + let mut events = client::websocket_unix(&served.socket, "/v1/events") + .await + .unwrap(); + for n in 0..1100 { + let _ = served.state.events.send(kv_event("layout", json!(n))); + } + let follow = json!({"kv": ["ui"]}).to_string(); + events.send(Frame::Text(follow.into())).await.unwrap(); + assert_eq!(text(&mut events).await, json!({"dropped": {"events": 76}})); + assert_eq!(text(&mut events).await["value"], 76); + + // A follower of a namespace that reads nothing while the server's buffers + // fill misses the rest, and goes on after the notice. + let mut changes = client::websocket_unix(&served.socket, "/v1/kv/ui") + .await + .unwrap(); + tokio::time::sleep(Duration::from_millis(100)).await; + let large = "x".repeat(4096); + for _ in 0..5000 { + let _ = served.state.events.send(kv_event("layout", json!(large))); + } + let _ = served.state.events.send(kv_event("last", json!(true))); + assert!(dropped(&mut changes).await > 0); + loop { + let frame = text(&mut changes).await; + if frame["key"] == "last" { + break; + } + } +} + fn message(id: u64, topic: &str, keep: &str) -> berth_api::Message { serde_json::from_value(json!({ "id": id, "topic": topic, "body": {"n": id}, "keep": keep, @@ -235,6 +294,43 @@ fn message(id: u64, topic: &str, keep: &str) -> berth_api::Message { .unwrap() } +#[tokio::test] +async fn a_browser_and_a_client_of_berth_0_5_are_told_what_to_do() { + let dir = tempfile::tempdir().unwrap(); + let state = state_in(dir.path(), Some("s3cret-token")); + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let address = listener.local_addr().unwrap(); + let router = api::router(state, true); + tokio::spawn(async move { axum::serve(listener, router).await }); + let connect = || async { tokio::net::TcpStream::connect(address).await.unwrap() }; + + // The address the recipes of 0.5 printed answers without a token, and says only what to run. + let mut client = Client::over(connect().await).await.unwrap(); + let front = client + .send("GET", "/", &[], Default::default()) + .await + .unwrap(); + assert_eq!(front.status, 200); + let text = String::from_utf8(front.body.to_vec()).unwrap(); + assert_eq!(text, api::FRONT); + assert!(text.contains("berth login https://THIS-SERVER") && !text.contains('<')); + assert!(!text.contains(berth_api::VERSION)); + let refused = client.get::("/v1/hosts").await.unwrap_err(); + assert_eq!(refused.status(), 401); + + // A call only berth 0.5 makes says that the client is older. + let socket = client::websocket_at(connect().await, "berth", "/v1/portal", "s3cret-token") + .await + .unwrap(); + let portal = berth_api::portal::Portal::over(socket); + let old = portal + .call("tasks.list", Some("studio"), Value::Null, None) + .await + .unwrap_err(); + assert_eq!(old.status(), 404); + assert!(old.to_string().contains("a call of berth 0.5"), "{old}"); +} + #[tokio::test(flavor = "multi_thread")] async fn hosts_jobs_and_the_inbox_over_the_api() { let served = served().await; @@ -252,7 +348,7 @@ async fn hosts_jobs_and_the_inbox_over_the_api() { ); assert_eq!( client - .get::("/v1/hosts/nope/version") + .get::("/v1/hosts/nope/controller/version") .await .unwrap_err() .status(), @@ -275,7 +371,6 @@ async fn hosts_jobs_and_the_inbox_over_the_api() { "echo 'bash: berthd: command not found' >&2; exit 127".into(), ], first_line: None, - via: None, ..LinkSpec::default() }; served.state.add(Remote::new("broken", "192.0.2.1", broken)); @@ -289,13 +384,13 @@ async fn hosts_jobs_and_the_inbox_over_the_api() { "{hosts}" ); let through = client - .get::("/v1/hosts/broken/tasks") + .get::("/v1/hosts/broken/controller/actions") .await .unwrap_err(); assert_eq!((through.status(), through.code()), (502, "unreachable")); // Secrets are refused before the host is even tried. let secrets = client - .post::("/v1/hosts/broken/tasks/secrets", &json!({})) + .post::("/v1/hosts/broken/controller/actions/secrets", &json!({})) .await .unwrap_err(); assert_eq!((secrets.status(), secrets.code()), (403, "forbidden")); @@ -315,6 +410,8 @@ async fn hosts_jobs_and_the_inbox_over_the_api() { subject: "host:broken".into(), action: "apply".into(), token: None, + stopped_by: None, + arguments: vec![], }; served.state.store.add_job(&job).unwrap(); let jobs: Value = client.get("/v1/jobs").await.unwrap(); @@ -560,11 +657,11 @@ impl Berthd { /// A berthd in single-user mode, or nothing when it is not built or does /// not answer yet. async fn start() -> Option { - Berthd::with_tasks(json!([])).await + Berthd::with_actions(json!([])).await } - /// The same, with these tasks in its `etc/tasks.json`. - async fn with_tasks(tasks: Value) -> Option { + /// The same, with these actions in its `etc/actions.json`. + async fn with_actions(actions: Value) -> Option { let program = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../target/debug/berthd"); if !program.is_file() { eprintln!("skipped: {} is not built", program.display()); @@ -572,7 +669,7 @@ impl Berthd { } let dir = tempfile::tempdir().unwrap(); std::fs::create_dir_all(dir.path().join("etc")).unwrap(); - std::fs::write(dir.path().join("etc/tasks.json"), tasks.to_string()).unwrap(); + std::fs::write(dir.path().join("etc/actions.json"), actions.to_string()).unwrap(); let child = std::process::Command::new(&program) .arg("--single") .arg(dir.path()) @@ -603,7 +700,6 @@ impl Berthd { program: self.program.to_string_lossy().into_owned(), args: vec!["connect".into(), "--dir".into(), dir], first_line: None, - via: None, ..LinkSpec::default() } } @@ -648,12 +744,15 @@ async fn a_hosts_api_is_passed_through() { let hosts: Value = client.get("/v1/hosts").await.unwrap(); assert_eq!(hosts[0]["reachable"], true, "{hosts}"); assert_eq!(hosts[0]["version"], berth_api::VERSION); - let version: Value = client.get("/v1/hosts/box/version").await.unwrap(); + let version: Value = client + .get("/v1/hosts/box/controller/version") + .await + .unwrap(); assert_eq!(version["api"], berth_api::API); // A request with a body and a query, and the host's own error passed back. let start = json!({"kind": "process", "command": "echo through; exit 3"}); let ran: Value = client - .post("/v1/hosts/box/sessions?wait=20", &start) + .post("/v1/hosts/box/controller/sessions?wait=20", &start) .await .unwrap(); assert_eq!(ran["exit_code"], 3, "{ran}"); @@ -662,20 +761,21 @@ async fn a_hosts_api_is_passed_through() { b"through\n" ); let missing = client - .get::("/v1/hosts/box/sessions/snope") + .get::("/v1/hosts/box/controller/sessions/snope") .await .unwrap_err(); assert_eq!((missing.status(), missing.code()), (404, "not_found")); let secrets = client - .post::("/v1/hosts/box/tasks/secrets", &json!({})) + .post::("/v1/hosts/box/controller/actions/secrets", &json!({})) .await .unwrap_err(); assert_eq!(secrets.status(), 403); // A stream, both ways: what is published through it comes back on it. - let mut channel = client::websocket_unix(&served.socket, "/v1/hosts/box/channel?topic=t/") - .await - .unwrap(); + let mut channel = + client::websocket_unix(&served.socket, "/v1/hosts/box/controller/channel?topic=t/") + .await + .unwrap(); let publish = r#"{"topic":"t/x","body":{"hello":1},"keep":"none"}"#; channel.send(Frame::Text(publish.into())).await.unwrap(); let message = text(&mut channel).await; @@ -685,9 +785,12 @@ async fn a_hosts_api_is_passed_through() { ); // A terminal session's bytes come through as binary frames. let start = json!({"kind": "terminal", "command": "printf ready; read line; echo got-$line"}); - let session: Value = client.post("/v1/hosts/box/sessions", &start).await.unwrap(); + let session: Value = client + .post("/v1/hosts/box/controller/sessions", &start) + .await + .unwrap(); let path = format!( - "/v1/hosts/box/sessions/{}?since=0", + "/v1/hosts/box/controller/sessions/{}?since=0", session["id"].as_str().unwrap() ); let mut terminal = client::websocket_unix(&served.socket, &path).await.unwrap(); @@ -719,8 +822,8 @@ async fn job_done(client: &mut Client, id: &str) -> Value { } #[tokio::test(flavor = "multi_thread")] -async fn a_hosts_actions_are_its_tasks_and_berths_own() { - let tasks = json!([ +async fn a_hosts_actions_are_its_controllers_and_berths_own() { + let actions = json!([ {"name": "status", "label": "Status", "group": "System", "user": "agent", "run": "echo up"}, {"name": "demo:status", "label": "Demo", "extends": ["status"], "user": "agent", "run": "echo demo; exit 4"}, {"name": "greet", "label": "Greet", "user": "agent", "run": "echo \"hello $BERTH_ARG_WHO\"", @@ -729,7 +832,7 @@ async fn a_hosts_actions_are_its_tasks_and_berths_own() { {"name": "secrets", "label": "Send the secrets", "user": "agent", "input": "secrets", "run": "cat >/dev/null"}, {"name": "elevated", "label": "As root", "user": "root", "run": "true"}, ]); - let Some(host) = Berthd::with_tasks(tasks).await else { + let Some(host) = Berthd::with_actions(actions).await else { return; }; let served = served().await; @@ -740,8 +843,8 @@ async fn a_hosts_actions_are_its_tasks_and_berths_own() { let actions: Vec = client.get("/v1/hosts/box/actions").await.unwrap(); let names: Vec<&str> = actions.iter().map(|action| action.name.as_str()).collect(); - // The host's tasks, a step under the task it extends, then berth's own; the - // secrets task is berth's, which sends the server's secrets. + // The host's actions, a step under the action it extends, then berth's own; the + // secrets action is berth's, which sends the server's secrets. assert_eq!( names, [ @@ -755,7 +858,8 @@ async fn a_hosts_actions_are_its_tasks_and_berths_own() { "secrets", "trust-key", "host-key", - "rename" + "rename", + "installer" ] ); let status = &actions[0]; @@ -771,7 +875,7 @@ async fn a_hosts_actions_are_its_tasks_and_berths_own() { ); assert!(secrets.available && actions[4].role == Role::Admin); - // A task that needs no terminal is a session there and a job here, which + // An action that needs no terminal is a session there and a job here, which // copies its output and ends with its exit code. let run = |body: Value| { let socket = served.socket.clone(); @@ -802,11 +906,21 @@ async fn a_hosts_actions_are_its_tasks_and_berths_own() { ); // Asked again with its token, it is the same job, and nothing runs again. let sessions = |value: Value| berth_tui::host::list::(value, "sessions").len(); - let before = sessions(client.get("/v1/hosts/box/sessions").await.unwrap()); + let before = sessions( + client + .get("/v1/hosts/box/controller/sessions") + .await + .unwrap(), + ); let again = run(json!({"token": "t-1"})).await.unwrap(); assert_eq!(again["job"]["id"], json!(id)); assert_eq!( - sessions(client.get("/v1/hosts/box/sessions").await.unwrap()), + sessions( + client + .get("/v1/hosts/box/controller/sessions") + .await + .unwrap() + ), before ); @@ -821,11 +935,16 @@ async fn a_hosts_actions_are_its_tasks_and_berths_own() { assert_eq!(ids.len(), 1, "{ids:?}"); job_done(&mut client, ids.first().unwrap()).await; assert_eq!( - sessions(client.get("/v1/hosts/box/sessions").await.unwrap()), + sessions( + client + .get("/v1/hosts/box/controller/sessions") + .await + .unwrap() + ), before + 1 ); - // A task the host will not start ends its job as failed, with the reason, + // An action the host will not start ends its job as failed, with the reason, // and asked again with the token that is the answer. let path = "/v1/hosts/box/actions/elevated"; let refused = client @@ -893,6 +1012,94 @@ async fn a_hosts_actions_are_its_tasks_and_berths_own() { ); } +/// A listener on the loopback that sends back what it gets, and ends its +/// side once the caller has ended theirs. +async fn echo() -> u16 { + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let port = listener.local_addr().unwrap().port(); + tokio::spawn(async move { + while let Ok((connection, _)) = listener.accept().await { + tokio::spawn(async move { + use tokio::io::AsyncWriteExt; + let (mut reading, mut writing) = connection.into_split(); + let _ = tokio::io::copy(&mut reading, &mut writing).await; + let _ = writing.shutdown().await; + }); + } + }); + port +} + +/// Write `data` and end, while what comes back is read: what came back. +async fn through(open: T, data: &[u8]) -> Vec +where + T: tokio::io::AsyncRead + tokio::io::AsyncWrite + Send + 'static, +{ + use tokio::io::{AsyncReadExt, AsyncWriteExt}; + let (mut reading, mut writing) = tokio::io::split(open); + let sent = data.to_vec(); + let writer = tokio::spawn(async move { + writing.write_all(&sent).await.unwrap(); + writing.shutdown().await.unwrap(); + writing + }); + let mut back = vec![]; + tokio::time::timeout(Duration::from_secs(60), reading.read_to_end(&mut back)) + .await + .expect("all of it back in time") + .unwrap(); + drop(writer.await.unwrap()); + back +} + +#[tokio::test(flavor = "multi_thread")] +async fn a_port_of_a_host_is_reached_through_the_server() { + use berth_api::{portal::Portal, tunnel::Tunnel}; + let Some(host) = Berthd::start().await else { + return; + }; + let served = served().await; + served + .state + .add(Remote::new("studio", "192.0.2.10", host.link())); + let port = echo().await; + let data: Vec = (0..3_000_000u32).map(|n| (n % 251) as u8).collect(); + + // As the host's own stream, passed through. + let path = format!("/v1/hosts/studio/controller/ports/{port}"); + let open = client::websocket_unix(&served.socket, &path).await.unwrap(); + assert_eq!(through(Tunnel::new(open), b"hello").await, b"hello"); + let open = client::websocket_unix(&served.socket, &path).await.unwrap(); + assert!(through(Tunnel::new(open), &data).await == data); + + // On the portal, with the host named. + let portal = Portal::over( + client::websocket_unix(&served.socket, "/v1/portal") + .await + .unwrap(), + ); + let open = portal.connect_port(Some("studio"), port).await.unwrap(); + assert!(through(open, &data).await == data); + + // A port nothing listens on is the host's error, either way. + let closed = { + let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap(); + listener.local_addr().unwrap().port() + }; + let path = format!("/v1/hosts/studio/controller/ports/{closed}"); + let refused = client::websocket_unix(&served.socket, &path) + .await + .err() + .unwrap(); + assert_eq!((refused.status(), refused.code()), (503, "not_served")); + let refused = portal + .connect_port(Some("studio"), closed) + .await + .err() + .unwrap(); + assert_eq!((refused.status(), refused.code()), (503, "not_served")); +} + #[tokio::test(flavor = "multi_thread")] async fn the_portal_reaches_serve_and_its_hosts() { use berth_api::portal::{Piece, Portal}; @@ -957,11 +1164,24 @@ async fn the_portal_reaches_serve_and_its_hosts() { other => panic!("the stream ended after {seen:?}: {other:?}"), } } + // The controller's actions are its own, apart from the server's of the host. + let listed = portal + .call("actions.list", Some("box"), Value::Null, None) + .await + .unwrap(); + assert!( + listed + .as_array() + .unwrap() + .iter() + .all(|action| action.get("available").is_none()), + "{listed}" + ); let secrets = portal .call( - "tasks.run", + "actions.run", Some("box"), - json!({"name": "secrets"}), + json!({"action": "secrets"}), Some(json!({})), ) .await; @@ -1043,32 +1263,6 @@ fn the_committed_documents_are_current() { } } -#[cfg(feature = "web")] -#[tokio::test(flavor = "multi_thread")] -async fn the_page_is_given_without_the_token() { - let dir = tempfile::tempdir().unwrap(); - let state = state_in(dir.path(), Some("s3cret-token")); - let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); - let address = listener.local_addr().unwrap(); - let router = api::router(state, true).merge(super::page::router()); - tokio::spawn(async move { axum::serve(listener, router).await }); - let stream = tokio::net::TcpStream::connect(address).await.unwrap(); - let mut client = Client::over(stream).await.unwrap(); - // The page for a browser holds nothing, so it is given without the token; - // what it shows it asks the portal for. - let page = client - .send("GET", "/", &[], Default::default()) - .await - .unwrap(); - let text = String::from_utf8_lossy(&page.body).into_owned(); - assert!(page.status == 200 && text.contains("berth-web.js?v=") && !text.contains("@BUILD@")); - let wasm = client - .send("GET", "/berth-web_bg.wasm", &[], Default::default()) - .await - .unwrap(); - assert!(wasm.status == 200 && wasm.body.starts_with(b"\0asm")); -} - async fn send(client: &mut Client, method: &str, path: &str, body: Value) -> (u16, Value) { let body = match body { Value::Null => String::new(), @@ -1134,12 +1328,13 @@ async fn the_server_is_set_up_through_the_api() { "{name}" ); } + // The server's own user, on its socket, chooses the command that reaches a host. let linked = json!({"builder": {}, "access": {"link": ["sh", "-c", "id"]}}); assert_eq!( - send(client, "PUT", "/v1/hosts/studio/profile", linked) + send(client, "PUT", "/v1/hosts/linked/profile", linked) .await .0, - 400 + 200 ); assert_eq!( send(client, "PUT", "/v1/hosts/studio/profile", json!([])) @@ -1280,8 +1475,29 @@ async fn the_server_is_set_up_through_the_api() { .unwrap() .contains("does not answer") ); + // A key pasted by an older caller is not kept. + let pasted = json!({"builder": {}, "access": {"address": "192.0.2.7"}, "tailscale_auth_key": "tskey-auth-kFAKE-0000", "tailscale_auto_key": false}); + send(client, "PUT", "/v1/hosts/studio/profile", pasted).await; + let (_, kept) = send(client, "GET", "/v1/hosts/studio/profile", Value::Null).await; + assert!(!kept.to_string().contains("tailscale_a"), "{kept}"); + // A seed of a host that joins the tailnet is refused, with what to do, + // while the server has no API key of the tailnet to make its key with. + // It is a refusal before anything began, which the command line exits 3 for. + let (status, said) = send(client, "POST", "/v1/hosts/studio/actions/seed", Value::Null).await; + assert_eq!(status, 412, "{said}"); + assert_eq!( + crate::cli::fail::status_of(status), + crate::cli::fail::REFUSED + ); + let message = said["error"]["message"].as_str().unwrap(); + assert!( + message.contains("berth secret set tailnet_api_key") + && message.contains("switch Tailscale off"), + "{message}" + ); - // Some settings are a caller's to change, who gets in is not. + // The user the server runs as changes any setting on its socket; over + // the network only some are a caller's to change. let value = |value: &str| json!({"value": value}); assert_eq!( send(client, "PUT", "/v1/settings/retention_days", value("30")) @@ -1298,7 +1514,7 @@ async fn the_server_is_set_up_through_the_api() { ) .await .0, - 403 + 200 ); let (_, settings) = send(client, "GET", "/v1/settings", Value::Null).await; let days = settings @@ -1429,7 +1645,7 @@ async fn plugins_are_managed_through_the_api() { &json!("1.0.0") ) ); - assert_eq!(hello["features"][0]["tasks"][0]["name"], "moo"); + assert_eq!(hello["features"][0]["actions"][0]["name"], "moo"); assert!(listed["dir"].as_str().unwrap().ends_with("config/plugins")); let (_, found) = send(client, "GET", "/v1/setup", Value::Null).await; assert!(keys(&found).contains(&"hello/cow".to_owned())); @@ -1456,10 +1672,10 @@ async fn plugins_are_managed_through_the_api() { ); } let asked = json!({"arguments": {"hosts": "nothing"}}); - let (status, said) = send(client, "POST", "/v1/actions/rollout", asked).await; + let (status, said) = send(client, "POST", "/v1/installation/actions/rollout", asked).await; assert_eq!(status, 404, "{said}"); let asked = json!({"arguments": {"force": "yes"}}); - let (status, said) = send(client, "POST", "/v1/actions/rollout", asked).await; + let (status, said) = send(client, "POST", "/v1/installation/actions/rollout", asked).await; assert_eq!(status, 400, "{said}"); let (status, said) = send(client, "DELETE", "/v1/plugins/hello", Value::Null).await; assert_eq!(status, 200, "{said}"); @@ -1618,6 +1834,18 @@ async fn an_access_token_gets_in_with_the_role_it_names() { assert_eq!(status, 403, "{said}"); let (status, said) = as_bearer(address, &admin, "PUT", "/v1/provisioners/vm", registered).await; assert_eq!(status, 200, "{said}"); + // A command that reaches a host is not taken from a caller over the network. + let linked = + json!({"builder": {}, "access": {"address": "192.0.2.7", "link": ["sh", "-c", "id"]}}); + let (status, said) = + as_bearer(address, &admin, "PUT", "/v1/hosts/linked/profile", linked).await; + assert_eq!(status, 400, "{said}"); + assert!( + said["error"]["message"] + .as_str() + .unwrap() + .contains("access.link") + ); let profile = json!({"builder": {}, "access": {"address": "192.0.2.7"}}); // Looking is for every role, using for two, set-up for one. @@ -1636,9 +1864,15 @@ async fn an_access_token_gets_in_with_the_role_it_names() { .contains("operate") ); assert_eq!( - as_bearer(address, &viewer, "POST", "/v1/hosts/a/tasks/x", json!({})) - .await - .0, + as_bearer( + address, + &viewer, + "POST", + "/v1/hosts/a/controller/actions/x", + json!({}) + ) + .await + .0, 403 ); assert_eq!( @@ -1646,7 +1880,7 @@ async fn an_access_token_gets_in_with_the_role_it_names() { address, &viewer, "GET", - "/v1/hosts/a/files?path=x", + "/v1/hosts/a/controller/files?path=x", Value::Null ) .await @@ -1693,7 +1927,7 @@ async fn an_access_token_gets_in_with_the_role_it_names() { for path in [ "/v1/hosts/a/actions/apply", "/v1/hosts/a/actions/seed", - "/v1/actions/rollout", + "/v1/installation/actions/rollout", ] { let (status, said) = as_bearer(address, &operator, "POST", path, Value::Null).await; assert_eq!(status, 403, "{path}: {said}"); @@ -1702,7 +1936,14 @@ async fn an_access_token_gets_in_with_the_role_it_names() { "{said}" ); } - let (status, said) = as_bearer(address, &viewer, "GET", "/v1/actions", Value::Null).await; + let (status, said) = as_bearer( + address, + &viewer, + "GET", + "/v1/installation/actions", + Value::Null, + ) + .await; assert_eq!((status, &said[0]["name"]), (200, &json!("rollout"))); assert_eq!(said[0]["role"], "admin"); // A provisioner is looked at by anyone, and each of its actions has its own role. @@ -1804,17 +2045,21 @@ async fn an_access_token_gets_in_with_the_role_it_names() { assert_eq!(saved.unwrap_err().status(), 403); let run = portal .call( - "tasks.run", + "actions.run", Some("a"), - json!({"name": "x"}), + json!({"action": "x"}), Some(json!({})), ) .await; assert_eq!(run.unwrap_err().status(), 403); assert!( - client::websocket(connect().await, "/v1/hosts/a/sessions/x", &viewer) - .await - .is_err() + client::websocket( + connect().await, + "/v1/hosts/a/controller/sessions/x", + &viewer + ) + .await + .is_err() ); } @@ -1840,7 +2085,7 @@ async fn set_up_can_be_closed_to_the_network() { .unwrap() .contains("set-up") ); - // Installing a plugin is set-up too: its tasks run when a host is applied. + // Installing a plugin is set-up too: its Ansible tasks run when a host is applied. let spec = json!({"spec": "berth-plugin-foo"}); assert_eq!( as_bearer(address, token, "POST", "/v1/plugins", spec) @@ -1851,7 +2096,7 @@ async fn set_up_can_be_closed_to_the_network() { for (method, path) in [ ("POST", "/v1/resources"), ("DELETE", "/v1/resources/r-a"), - ("POST", "/v1/actions/rollout"), + ("POST", "/v1/installation/actions/rollout"), ] { let (status, said) = as_bearer(address, token, method, path, json!({})).await; assert_eq!(status, 403, "{method} {path}: {said}"); @@ -1971,7 +2216,8 @@ fn write_fake_provisioner(home: &std::path::Path) -> PathBuf { "spec": {"format": "text", "label": "Spec"}, "arguments": [{"name": "replicas", "default": "1"}], "actions": [{"name": "services", "changes": false}, - {"name": "restart", "confirm": "Restart?"}]}], + {"name": "restart", "confirm": "Restart?"}]}, + {"id": "box", "label": "Box", "verbs": ["exec"]}], "actions": [ {"name": "images", "label": "Images", "changes": false}, {"name": "prune", "label": "Prune", "confirm": "Prune?", @@ -2074,11 +2320,13 @@ async fn a_provisioner_has_a_page_and_actions_that_run_as_jobs() { names, [ "new-deployment", + "new-box", "settings", "images", "prune", "restart", "secrets", + "adopt", "remove" ] ); @@ -2092,7 +2340,7 @@ async fn a_provisioner_has_a_page_and_actions_that_run_as_jobs() { .await; assert_eq!(status, 404, "{said}"); assert!( - message(&said).contains("Its actions: new-deployment, settings, images"), + message(&said).contains("Its actions: new-deployment, new-box, settings, images"), "{said}" ); let (status, said) = send( @@ -2361,7 +2609,8 @@ async fn machines_are_created_and_deleted_through_the_api() { .unwrap() .save( paths, - &json!({"builder": {}, "access": {"user": "ops", "link": ["sh", "-c", "exit 1"]}}), + &json!({"builder": {}, "access": {"user": "ops", "link": ["sh", "-c", "exit 1"]}, + "ansible_vars": {"devbox_install_tailscale": false}}), ) .unwrap(); let message = |said: &Value| said["error"]["message"].as_str().unwrap_or("").to_owned(); @@ -2382,8 +2631,9 @@ async fn machines_are_created_and_deleted_through_the_api() { // Refusals are answers of their own, before there is a job. let asked = json!({"provisioner": "vm", "host": "studio", "arguments": {"server": "https://192.0.2.10:8443"}}); + // A secret the server lacks is a refusal, which the command line exits 3 for. let (status, said) = send(client, "POST", "/v1/resources", asked.clone()).await; - assert_eq!(status, 400, "{said}"); + assert_eq!(status, 412, "{said}"); assert!(message(&said).contains("api_token"), "{said}"); let secret = json!({"value": "tok-s3cret"}); send(client, "PUT", "/v1/secrets/api_token", secret).await; @@ -2544,7 +2794,7 @@ async fn machines_are_created_and_deleted_through_the_api() { let (status, said) = send(client, "DELETE", "/v1/hosts/studio", Value::Null).await; assert_eq!(status, 409, "{said}"); assert!( - message(&said).contains(&format!("berth resource {id} remove")), + message(&said).contains(&format!("berth resource remove {id}")), "{said}" ); @@ -2957,9 +3207,12 @@ async fn berth_connect_saves_the_server_and_its_token() { data_dir: client.path().join("data"), }; let connect = |key: String| { - let (paths, url) = (paths.clone(), url.clone()); + let (dir, url) = (paths.config_dir.clone(), url.clone()); tokio::task::spawn_blocking(move || { - crate::commands::connect_with(&paths, &url, &key).map_err(|error| error.to_string()) + let settings = berth_page::settings::Settings::new(Some(dir)); + crate::cli::login::claim(&settings, &url, &key) + .map(drop) + .map_err(|error| error.message) }) }; // A wrong key saves nothing. @@ -2997,24 +3250,6 @@ fn a_setup_key_is_refused_when_the_token_is_off() { assert!(super::setup_key::command(&paths, &["--nothing"]).is_err()); } -/// Generate the password of the operator on the screen of `app`, and what -/// the screen said. -fn generate_operator_password(app: &mut crate::app::App) -> String { - use crate::app::Entry; - use crate::fields::{Field, Step}; - let found = Step::SETUP.into_iter().find_map(|step| { - app.step = step; - app.entries() - .iter() - .position(|entry| *entry == Entry::Field(Field::OperatorHash)) - }); - app.list.select(found); - match app.generate() { - Ok(()) => app.status.clone(), - Err(error) => error.to_string(), - } -} - #[tokio::test(flavor = "multi_thread")] async fn a_host_is_renamed_as_a_job_with_what_the_server_keeps_of_it() { let served = served().await; @@ -3089,6 +3324,137 @@ async fn a_host_is_renamed_as_a_job_with_what_the_server_keeps_of_it() { assert_eq!(inbox["messages"][0]["topic"], "agent/done", "{inbox}"); } +/// The frames of an events stream up to the first of `source` with `change`, which is returned. +async fn change_of(stream: &mut S, source: &str, change: &str) -> Value +where + S: StreamExt> + Unpin, + E: std::fmt::Debug, +{ + let mut seen = vec![]; + loop { + let frame = tokio::time::timeout(Duration::from_secs(20), text(stream)) + .await + .unwrap_or_else(|_| panic!("no {source} {change} after {seen:?}")); + if frame["source"] == source && frame["change"] == change { + return frame; + } + seen.push(frame); + } +} + +#[tokio::test(flavor = "multi_thread")] +async fn hosts_and_resources_that_change_are_told_on_the_events_stream() { + let served = served_with(quick).await; + let fake = fake_provisioner(&served); + let mut client = served.client().await; + let client = &mut client; + register_fake(client).await; + let paths = &served.state.paths; + paths.prepare().unwrap(); + let mut events = client::websocket_unix(&served.socket, "/v1/events") + .await + .unwrap(); + let follow = json!({"changes": ["hosts", "resources"]}).to_string(); + events.send(Frame::Text(follow.into())).await.unwrap(); + + // Saved by another berth beside the server: told when the server looks. + let profile = json!({"hostname": "studio", "builder": {}, "access": {"address": "192.0.2.7", "link": ["sh", "-c", "exit 1"]}}); + berth_core::profiles::Place::own("studio") + .unwrap() + .save(paths, &profile) + .unwrap(); + served.state.notice(); + assert_eq!( + change_of(&mut events, "host", "added").await, + json!({"source": "host", "host": "studio", "change": "added"}) + ); + let mut moved = profile.clone(); + moved["access"]["address"] = json!("192.0.2.8"); + let (status, said) = send(client, "PUT", "/v1/hosts/studio/profile", moved).await; + assert_eq!(status, 200, "{said}"); + assert_eq!( + change_of(&mut events, "host", "changed").await["host"], + "studio" + ); + let rename = json!({"arguments": {"name": "dock", "hostname": "no"}}); + let (status, acted) = send(client, "POST", "/v1/hosts/studio/actions/rename", rename).await; + assert_eq!(status, 202, "{acted}"); + assert_eq!( + change_of(&mut events, "host", "renamed").await, + json!({"source": "host", "host": "dock", "change": "renamed", "from": "studio"}) + ); + job_done(client, acted["job"]["id"].as_str().unwrap()).await; + let (status, said) = send(client, "DELETE", "/v1/hosts/dock", Value::Null).await; + assert_eq!(status, 200, "{said}"); + assert_eq!( + change_of(&mut events, "host", "removed").await["host"], + "dock" + ); + + send( + client, + "PUT", + "/v1/secrets/api_token", + json!({"value": "tok-made-up"}), + ) + .await; + std::fs::write( + fake.join("answer.create"), + r#"{"id": "wiki", "state": "running"}"#, + ) + .unwrap(); + let asked = json!({"provisioner": "vm", "kind": "deployment", "name": "wiki", + "arguments": {"server": "x"}, "spec": "services:\n wiki:\n image: wiki\n", + "token": "t-1"}); + let (status, created) = send(client, "POST", "/v1/resources", asked).await; + assert_eq!(status, 202, "{created}"); + let id = created["resource"]["id"].as_str().unwrap().to_owned(); + assert_eq!( + change_of(&mut events, "resource", "created").await, + json!({"source": "resource", "resource": id, "provisioner": "vm", "kind": "deployment", "change": "created"}) + ); + let (_, code) = job_ended(&served.socket, created["job"]["id"].as_str().unwrap()).await; + assert_eq!(code, 0); + // Made, it has the provisioner's id now. + assert_eq!( + change_of(&mut events, "resource", "changed").await["resource"], + json!(id) + ); + let (status, removed) = send( + client, + "DELETE", + &format!("/v1/resources/{id}"), + Value::Null, + ) + .await; + assert_eq!(status, 202, "{removed}"); + assert_eq!( + change_of(&mut events, "resource", "removed").await["resource"], + json!(id) + ); + + // Not asked for, nothing of the kind is told. + let mut quiet = client::websocket_unix(&served.socket, "/v1/events") + .await + .unwrap(); + let follow = json!({"kv": ["ui"]}).to_string(); + quiet.send(Frame::Text(follow.into())).await.unwrap(); + berth_core::profiles::Place::own("pier") + .unwrap() + .save(paths, &profile) + .unwrap(); + served.state.notice(); + put(&served.socket, &[], "1").await; + assert_eq!(text(&mut quiet).await["source"], "kv"); + // A kind it does not know is refused. + let mut wrong = client::websocket_unix(&served.socket, "/v1/events") + .await + .unwrap(); + let follow = json!({"changes": ["plugins"]}).to_string(); + wrong.send(Frame::Text(follow.into())).await.unwrap(); + assert_eq!(text(&mut wrong).await["error"]["code"], "bad_request"); +} + // Not on a runtime of the test's: the screen waits for the server on one of its own. #[test] fn generated_passwords_are_kept_where_the_host_is_and_shown_to_an_admin() { @@ -3118,7 +3484,7 @@ fn generated_passwords_are_kept_where_the_host_is_and_shown_to_an_admin() { made["note"] .as_str() .unwrap() - .contains("berth credentials studio"), + .contains("berth host credentials studio"), "{made}" ); let (status, kept) = ask("GET", "/v1/hosts/studio/credentials", Value::Null); @@ -3134,67 +3500,9 @@ fn generated_passwords_are_kept_where_the_host_is_and_shown_to_an_admin() { let (status, _) = ask("GET", "/v1/hosts/attic/credentials", Value::Null); assert_eq!(status, 404); - // The screen of a berth that uses this server has the server make the - // password, and keeps none of it itself, where a mirror would erase it. - let elsewhere = tempfile::tempdir().unwrap(); - let paths = Paths { - config_dir: elsewhere.path().join("config"), - data_dir: elsewhere.path().join("data"), - }; - let url = format!("unix:{}", served.socket.display()); - paths.set("server", &url).unwrap(); - let place = Place::resolve(&paths, "studio").unwrap(); - assert_eq!(place.server_url(), Some(url.as_str())); - let mut config = profile.clone(); - config["builder"]["served"] = json!(url); - let mut app = crate::app::App::new( - paths.clone(), - place.clone(), - crate::app::parse_config(config.to_string().as_bytes()).unwrap(), - ); - let said = generate_operator_password(&mut app); - assert!(said.contains("kept on the berth server"), "{said}"); - assert!( - app.config["operator_password_hash"] - .as_str() - .unwrap() - .starts_with("$6$") - ); - assert!(credentials::load(&paths, &place).unwrap().is_empty()); - let kept = credentials::load(&served.state.paths, &own).unwrap(); - assert_ne!(kept["ops"], json!(password), "a new one was made"); - - // A host of this berth's own keeps it in this berth's store. - let local = tempfile::tempdir().unwrap(); - let paths = Paths { - config_dir: local.path().join("config"), - data_dir: local.path().join("data"), - }; - let attic = Place::own("attic").unwrap(); - let mut app = crate::app::App::new( - paths.clone(), - attic.clone(), - crate::app::parse_config(b"{}").unwrap(), - ); - let said = generate_operator_password(&mut app); - assert!(said.contains("berth credentials attic"), "{said}"); - assert_eq!(credentials::load(&paths, &attic).unwrap().len(), 1); runtime.shutdown_background(); } -#[test] -fn a_serve_of_a_berth_that_uses_a_berth_server_does_not_start() { - let dir = tempfile::tempdir().unwrap(); - let paths = Paths { - config_dir: dir.path().join("config"), - data_dir: dir.path().join("data"), - }; - paths.set("server", "http://192.0.2.7:7777").unwrap(); - let said = super::command(&paths, &[]).unwrap_err().to_string(); - assert!(said.contains("berth config server \"\""), "{said}"); - assert!(!paths.data_dir.join("serve.lock").exists()); -} - #[test] fn pruning_removes_a_jobs_output_and_its_report() { let dir = tempfile::tempdir().unwrap(); @@ -3234,9 +3542,127 @@ fn pruning_removes_a_jobs_output_and_its_report() { /// The routes of the key ring, with the helpers of these tests. mod keys; -/// berth-remote signed in to a server of the tests. +/// The screens signed in to a server of the tests over the network. mod remote; +#[tokio::test(flavor = "multi_thread")] +async fn a_server_in_the_process_names_both_ways_to_give_a_sudo_password() { + let dir = tempfile::tempdir().unwrap(); + let mut state = state_in(dir.path(), None); + Arc::get_mut(&mut state).unwrap().here = true; + let profile = json!({"builder": {}, "access": {"address": "192.0.2.9", "user": "ops", "sudo": "password"}}); + berth_core::profiles::Place::own("studio") + .unwrap() + .save(&state.paths, &profile) + .unwrap(); + state.refresh(false); + let problem = state + .host("studio") + .unwrap() + .problem + .clone() + .unwrap_or_default(); + assert!(problem.contains("--sudo-password-stdin"), "{problem}"); + assert!( + problem.contains("berth secret set sudo/studio"), + "{problem}" + ); + // Given one for the run, the host is taken up again with it. + state + .passwords + .lock() + .unwrap() + .insert("studio".into(), "pass-word-1".into()); + state.forget("studio"); + state.refresh(false); + let remote = state.host("studio").unwrap(); + assert!( + !remote.problem.clone().unwrap_or_default().contains("sudo"), + "{:?}", + remote.problem + ); + // berth serve says what it says to its own user. + let other = tempfile::tempdir().unwrap(); + let served = state_in(other.path(), None); + berth_core::profiles::Place::own("studio") + .unwrap() + .save(&served.paths, &profile) + .unwrap(); + served.refresh(false); + let problem = served + .host("studio") + .unwrap() + .problem + .clone() + .unwrap_or_default(); + assert!( + problem.contains("run berth serve on a terminal"), + "{problem}" + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn a_caller_on_this_machine_takes_the_host_key_its_user_knows() { + let dir = tempfile::tempdir().unwrap(); + let state = state_in(dir.path(), Some("s3cret-token")); + let known_hosts = dir.path().join("known_hosts"); + std::fs::write(&known_hosts, "192.0.2.8 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEdockdockdockdockdockdockdockdockdockdockdo\n").unwrap(); + *state.known_hosts.lock().unwrap() = Some(known_hosts); + for name in ["dock", "studio"] { + berth_core::profiles::Place::own(name) + .unwrap() + .save( + &state.paths, + &json!({"builder": {}, "access": {"address": "192.0.2.8"}}), + ) + .unwrap(); + } + // Over the network nothing of the server's user's files is taken. + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let address = listener.local_addr().unwrap(); + let router = api::router(state.clone(), true); + tokio::spawn(async move { axum::serve(listener, router).await }); + let mut far = Client::over(tokio::net::TcpStream::connect(address).await.unwrap()) + .await + .unwrap(); + let answer = far + .send( + "POST", + "/v1/hosts/dock/actions/host-key", + &[("authorization", "Bearer s3cret-token")], + Default::default(), + ) + .await + .unwrap(); + assert_eq!(answer.status, 200); + let acted: Value = serde_json::from_slice(&answer.body).unwrap(); + assert_eq!(acted["result"]["host_keys"], json!([]), "{acted}"); + // On the socket the key that file has for the address is the host's. + let socket = dir.path().join("serve.sock"); + let unix = tokio::net::UnixListener::bind(&socket).unwrap(); + let router = api::router(state.clone(), false); + tokio::spawn(async move { axum::serve(unix, router).await }); + let mut client = Client::unix(&socket).await.unwrap(); + let (status, acted) = send( + &mut client, + "POST", + "/v1/hosts/studio/actions/host-key", + Value::Null, + ) + .await; + assert_eq!(status, 200, "{acted}"); + let keys = acted["result"]["host_keys"].as_array().unwrap(); + assert_eq!(keys.len(), 1, "{acted}"); + assert_eq!(keys[0]["type"], "ssh-ed25519"); + assert!( + keys[0]["fingerprint"] + .as_str() + .unwrap() + .starts_with("SHA256:"), + "{acted}" + ); +} + #[tokio::test(flavor = "multi_thread")] async fn first_contact_and_keys_are_actions_of_the_server() { let served = served().await; @@ -3333,42 +3759,58 @@ async fn first_contact_and_keys_are_actions_of_the_server() { assert_eq!(acted["result"], json!({"host_keys": [], "accepted": false})); // The installation rotates a key as keys.rotate does. - let (status, listed) = send(client, "GET", "/v1/actions", Value::Null).await; + let (status, listed) = send(client, "GET", "/v1/installation/actions", Value::Null).await; assert_eq!(status, 200); assert_eq!( action(&listed, "rotate-key")["arguments"][0]["default"], "berth" ); let nothing = json!({"arguments": {"name": "nothing"}}); - let (status, _) = send(client, "POST", "/v1/actions/rotate-key", nothing).await; + let (status, _) = send( + client, + "POST", + "/v1/installation/actions/rotate-key", + nothing, + ) + .await; assert_eq!(status, 404); let asked = json!({"arguments": {"name": "berth"}, "token": "rotate-1"}); - let (status, acted) = send(client, "POST", "/v1/actions/rotate-key", asked.clone()).await; + let (status, acted) = send( + client, + "POST", + "/v1/installation/actions/rotate-key", + asked.clone(), + ) + .await; assert_eq!(status, 202, "{acted}"); assert_eq!(acted["job"]["action"], "rotate-key:berth"); let id = acted["job"]["id"].as_str().unwrap().to_owned(); let (output, _) = job_ended(&served.socket, &id).await; assert!(output.contains("Replacing the key berth"), "{output}"); - let (_, again) = send(client, "POST", "/v1/actions/rotate-key", asked).await; + let (_, again) = send(client, "POST", "/v1/installation/actions/rotate-key", asked).await; assert_eq!(again["job"]["id"], json!(id)); } #[tokio::test(flavor = "multi_thread")] -async fn a_host_added_by_its_address_has_a_level_and_actions_to_match() { +async fn a_host_added_by_its_address_has_a_type_and_actions_to_match() { let served = served().await; let mut client = served.client().await; let client = &mut client; - // Its level and how it is reached are all a host below a workstation needs. - let reachable = - json!({"level": "reachable", "access": {"address": "192.0.2.8", "user": "ops"}}); - let (status, said) = send(client, "PUT", "/v1/hosts/dock/profile", reachable).await; + // Its type and how it is reached are all a host that is not a workstation needs. + let ssh = json!({"type": "ssh", "access": {"address": "192.0.2.8", "user": "ops"}}); + let (status, said) = send(client, "PUT", "/v1/hosts/dock/profile", ssh).await; assert_eq!(status, 200, "{said}"); for (refused, why) in [ ( - json!({"level": "server", "access": {"address": "192.0.2.9"}}), - "level", + json!({"type": "server", "access": {"address": "192.0.2.9"}}), + "type", + ), + (json!({"type": "managed"}), "access.address"), + // The API takes the names of now only. + ( + json!({"level": "reachable", "access": {"address": "192.0.2.9"}}), + "builder", ), - (json!({"level": "controller"}), "access.address"), (json!({"access": {"address": "192.0.2.9"}}), "builder"), ] { let (status, said) = send(client, "PUT", "/v1/hosts/pier/profile", refused).await; @@ -3379,9 +3821,9 @@ async fn a_host_added_by_its_address_has_a_level_and_actions_to_match() { ); } let (_, dock) = send(client, "GET", "/v1/hosts/dock", Value::Null).await; - assert_eq!(dock["level"], "reachable", "{dock}"); + assert_eq!(dock["type"], "ssh", "{dock}"); - // berth's own actions say what the level allows, and how to raise it. + // berth's own actions say what the type allows, and how to change it. let (_, actions) = send(client, "GET", "/v1/hosts/dock/actions", Value::Null).await; let find = |name: &str| { actions @@ -3398,9 +3840,9 @@ async fn a_host_added_by_its_address_has_a_level_and_actions_to_match() { (&json!(true), &json!("Install the controller")) ); for (name, why) in [ - ("apply", "not a workstation"), + ("apply", "neither a node nor a workstation"), ("seed", "not a workstation"), - ("secrets", "berth host install dock"), + ("secrets", "berth run dock install-controller"), ] { let action = find(name); assert_eq!(action["available"], false, "{action}"); @@ -3416,26 +3858,32 @@ async fn a_host_added_by_its_address_has_a_level_and_actions_to_match() { said["error"]["message"] .as_str() .unwrap() - .contains("Set its level to workstation") + .contains("or a workstation in its set-up") ); let (status, _) = send(client, "POST", "/v1/hosts/dock/actions/seed", Value::Null).await; assert_eq!(status, 501); // A roll-out leaves it, and says so. - let (status, acted) = send(client, "POST", "/v1/actions/rollout", Value::Null).await; + let (status, acted) = send( + client, + "POST", + "/v1/installation/actions/rollout", + Value::Null, + ) + .await; assert_eq!(status, 202, "{acted}"); let id = acted["job"]["id"].as_str().unwrap().to_owned(); let (output, code) = job_ended(&served.socket, &id).await; assert_eq!(code, 0, "{output}"); assert!( - output.contains("Skipped 1 host that is not a workstation: dock"), + output.contains("Skipped 1 host that is neither a node nor a workstation: dock"), "{output}" ); - // A controller host is offered an update of its controller, a + // A managed host is offered an update of its controller, a // workstation none, as applying installs it. let mut profile = said_profile(client, "dock").await; - profile["level"] = json!("controller"); + profile["type"] = json!("managed"); assert_eq!( send(client, "PUT", "/v1/hosts/dock/profile", profile.clone()) .await @@ -3451,7 +3899,7 @@ async fn a_host_added_by_its_address_has_a_level_and_actions_to_match() { .cloned() .unwrap(); assert_eq!(install["label"], "Update the controller"); - profile.as_object_mut().unwrap().remove("level"); + profile.as_object_mut().unwrap().remove("type"); profile["builder"] = json!({}); assert_eq!( send(client, "PUT", "/v1/hosts/dock/profile", profile) @@ -3491,7 +3939,7 @@ async fn a_host_seen_to_be_a_workstation_never_gets_the_controller_alone() { .unwrap() .save( &served.state.paths, - &json!({"level": "reachable", "access": {"address": "198.51.100.1", "user": "ops"}}), + &json!({"type": "ssh", "access": {"address": "198.51.100.1", "user": "ops"}}), ) .unwrap(); served @@ -3509,8 +3957,8 @@ async fn a_host_seen_to_be_a_workstation_never_gets_the_controller_alone() { assert_eq!(install["available"], false, "{install}"); let why = install["why"].as_str().unwrap(); assert!( - why.starts_with("box is a workstation; its profile here says reachable") - && why.contains("berth adopt box"), + why.starts_with("box is a workstation; its profile here says ssh") + && why.contains("berth host adopt box"), "{why}" ); let (status, said) = send( @@ -3540,7 +3988,7 @@ async fn a_host_whose_controller_does_not_answer_says_why_and_keeps_berths_own_a let served = served().await; // A link that fails as ssh does for a host it cannot reach. let said = "ssh: connect to host 192.0.2.11 port 22: Network is unreachable"; - let profile = json!({"level": "controller", "access": {"address": "192.0.2.11", + let profile = json!({"type": "managed", "access": {"address": "192.0.2.11", "link": ["sh", "-c", format!("echo '{said}' >&2; exit 255")]}}); berth_core::profiles::Place::own("dock") .unwrap() @@ -3556,7 +4004,13 @@ async fn a_host_whose_controller_does_not_answer_says_why_and_keeps_berths_own_a let socket = served.socket.clone(); async move { let mut client = Client::unix(&socket).await.unwrap(); - send(&mut client, "GET", "/v1/hosts/dock/version", Value::Null).await + send( + &mut client, + "GET", + "/v1/hosts/dock/controller/version", + Value::Null, + ) + .await } }); for (status, answer) in futures_util::future::join_all(asks).await { @@ -3632,7 +4086,7 @@ async fn a_host_removed_or_changed_in_the_store_by_another_process_is_not_listed let (_, hosts) = send(client, "GET", "/v1/hosts", Value::Null).await; assert_eq!(hosts[0]["address"], "192.0.2.40", "{hosts}"); - // `berth remove` there takes it off the list, without a restart. + // `berth host remove` there takes it off the list, without a restart. berth_core::profiles::remove(paths, &place).unwrap(); let (status, hosts) = send(client, "GET", "/v1/hosts", Value::Null).await; assert_eq!((status, hosts), (200, json!([]))); @@ -3640,65 +4094,719 @@ async fn a_host_removed_or_changed_in_the_store_by_another_process_is_not_listed assert_eq!(status, 404); } -// Not on a runtime of the test's: the screen waits for the server on one of its own. -#[test] -fn the_screen_keeps_the_address_a_server_gave_a_new_machine_and_opens_its_overview() { - use crate::fields::Step; - use berth_core::profiles::Place; - let runtime = tokio::runtime::Builder::new_multi_thread() - .enable_all() - .build() +/// What works with this machine's own drives and settings is offered to its +/// user, on the socket or in the process, and not over the network. +#[tokio::test(flavor = "multi_thread")] +async fn the_installer_and_every_setting_are_for_the_user_here() { + let dir = tempfile::tempdir().unwrap(); + let state = state_in(dir.path(), Some("s3cret-token")); + let profile = json!({"hostname": "studio", "builder": {}, "access": {"address": "192.0.2.7", "link": ["sh", "-c", "exit 1"]}}); + berth_core::profiles::Place::own("studio") + .unwrap() + .save(&state.paths, &profile) .unwrap(); - let served = runtime.block_on(served()); - let own = Place::own("studio").unwrap(); - let profile = json!({"hostname": "studio", "builder": {}, "operator_username": "ops"}); - own.save(&served.state.paths, &profile).unwrap(); + let socket = dir.path().join("serve.sock"); + let unix = tokio::net::UnixListener::bind(&socket).unwrap(); + let local = api::router(state.clone(), false); + tokio::spawn(async move { axum::serve(unix, local).await }); + let tcp = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let address = tcp.local_addr().unwrap(); + let guarded = api::router(state.clone(), true); + tokio::spawn(async move { axum::serve(tcp, guarded).await }); + let installer = |listed: &Value| { + listed + .as_array() + .unwrap() + .iter() + .find(|action| action["name"] == "installer") + .cloned() + .unwrap() + }; - let elsewhere = tempfile::tempdir().unwrap(); - let paths = Paths { - config_dir: elsewhere.path().join("config"), - data_dir: elsewhere.path().join("data"), + let mut client = Client::unix(&socket).await.unwrap(); + let (status, listed) = send(&mut client, "GET", "/v1/hosts/studio/actions", Value::Null).await; + assert_eq!(status, 200, "{listed}"); + let here = installer(&listed); + assert_eq!(here["available"], true, "{here}"); + assert_eq!(here["arguments"][1]["secret"], true); + // Only a USB drive that is connected is written. + let act = json!({"arguments": {"device": "/dev/berth-test-none"}}); + let (status, said) = send( + &mut client, + "POST", + "/v1/hosts/studio/actions/installer", + act, + ) + .await; + assert_eq!(status, 400, "{said}"); + assert!( + said["error"]["message"] + .as_str() + .unwrap() + .contains("no USB drive") + ); + + let (status, listed) = as_bearer( + address, + "s3cret-token", + "GET", + "/v1/hosts/studio/actions", + Value::Null, + ) + .await; + assert_eq!(status, 200, "{listed}"); + let elsewhere = installer(&listed); + assert_eq!(elsewhere["available"], false); + assert!( + elsewhere["why"] + .as_str() + .unwrap() + .contains("berth serve runs on"), + "{elsewhere}" + ); + let (status, _) = as_bearer( + address, + "s3cret-token", + "POST", + "/v1/hosts/studio/actions/installer", + json!({}), + ) + .await; + assert_eq!(status, 501); + + // Settings of the machine itself, such as the ISO folder, change only here. + let iso = json!({"value": "/srv/iso"}); + let (status, _) = as_bearer( + address, + "s3cret-token", + "PUT", + "/v1/settings/iso_dir", + iso.clone(), + ) + .await; + assert_eq!(status, 403); + let (status, said) = send(&mut client, "PUT", "/v1/settings/iso_dir", iso).await; + assert_eq!(status, 200, "{said}"); + assert_eq!(state.paths.setting("iso_dir").as_deref(), Some("/srv/iso")); + let (_, listed) = send(&mut client, "GET", "/v1/settings", Value::Null).await; + let open = |listed: &Value| { + listed + .as_array() + .unwrap() + .iter() + .find(|setting| setting["key"] == "iso_dir") + .unwrap()["open"] + .clone() }; - let url = format!("unix:{}", served.socket.display()); - paths.set("server", &url).unwrap(); - let server = berth_core::server::Server::configured(&paths).unwrap(); - server.mirror(&paths).unwrap(); - let place = Place::resolve(&paths, "studio").unwrap(); - let stored = place.load(&paths).unwrap().unwrap(); - let mut app = crate::app::App::new( - paths.clone(), - place.clone(), - crate::app::parse_config(stored.to_string().as_bytes()).unwrap(), - ); - app.set_step(Step::Install); - - // The server made the machine in a job of its own and gave its profile the address. - let mut made = own.load(&served.state.paths).unwrap().unwrap(); - made["access"] = json!({"address": "192.0.2.40", "user": "ops", "auth": "berth"}); - made["profile"] = json!("cloud"); - own.save(&served.state.paths, &made).unwrap(); - let stage = crate::job::Stage { - title: "Create machine".into(), - command: std::process::Command::new("true"), - source: None, + assert_eq!(open(&listed), true); + let (_, listed) = as_bearer(address, "s3cret-token", "GET", "/v1/settings", Value::Null).await; + assert_eq!(open(&listed), false); +} + +// ── Endpoints and commands of resources ───────────────────────────────────── + +/// A port of the loopback that nothing listens on. +fn closed_port() -> u16 { + let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap(); + listener.local_addr().unwrap().port() +} + +/// Make the fake's box `db`, whose `create` answers `answer`: its id. +async fn made_box(served: &Served, fake: &std::path::Path, answer: Value) -> String { + let mut client = served.client().await; + let client = &mut client; + register_fake(client).await; + served.state.paths.prepare().unwrap(); + send( + client, + "PUT", + "/v1/secrets/api_token", + json!({"value": "tok-made-up"}), + ) + .await; + std::fs::write(fake.join("answer.create"), answer.to_string()).unwrap(); + let asked = + json!({"provisioner": "vm", "kind": "box", "name": "db", "arguments": {"server": "x"}}); + let (status, created) = send(client, "POST", "/v1/resources", asked).await; + assert_eq!(status, 202, "{created}"); + let (output, code) = job_ended(&served.socket, created["job"]["id"].as_str().unwrap()).await; + assert_eq!(code, 0, "{output}"); + created["resource"]["id"].as_str().unwrap().to_owned() +} + +#[tokio::test(flavor = "multi_thread")] +async fn a_resource_is_reached_at_the_endpoints_its_provisioner_reports() { + use berth_api::{portal::Portal, tunnel::endpoint_tunnel}; + let served = served_with(quick).await; + let fake = fake_provisioner(&served); + let (port, closed) = (echo().await, closed_port()); + let endpoints = |port: u16| { + json!([ + {"name": "db", "address": "127.0.0.1", "port": port, "hint": "postgres", "local": true}, + {"name": "closed", "address": "127.0.0.1", "port": closed, "local": true}, + {"name": "far", "address": "127.0.0.1", "port": 6000}, + {"name": "stats", "address": "192.0.2.20", "port": 8125, "protocol": "udp"}, + {"name": "Not a name", "address": "192.0.2.20", "port": 1}, + ]) + }; + let answer = json!({"id": "db-1", "state": "running", "endpoints": endpoints(port)}); + let id = made_box(&served, &fake, answer).await; + let mut client = served.client().await; + + // Kept with the record, without the one that does not fit. + let listed: Value = client.get("/v1/resources").await.unwrap(); + let kept = &listed[0]["endpoints"]; + assert_eq!(kept.as_array().unwrap().len(), 4, "{kept}"); + assert_eq!( + kept[0], + json!({"name": "db", "address": "127.0.0.1", "port": port, "protocol": "tcp", "hint": "postgres", "local": true}) + ); + assert_eq!(kept[3]["protocol"], "udp"); + + // Straight to the server, by name and by port, both ways and with each side's end. + let open = |endpoint: String| { + let (socket, id) = (served.socket.clone(), id.clone()); + async move { + let io = tokio::net::UnixStream::connect(&socket).await.unwrap(); + endpoint_tunnel(io, &id, &endpoint, "").await + } + }; + let data: Vec = (0..3_000_000u32).map(|n| (n % 251) as u8).collect(); + assert_eq!( + through(open("db".into()).await.unwrap(), b"hello").await, + b"hello" + ); + assert!(through(open(port.to_string()).await.unwrap(), &data).await == data); + // Nothing on it, not reported, not reachable from here, or not TCP: an answer, not a stream. + for (endpoint, status, code) in [ + ("closed", 503, "not_served"), + ("22", 404, "not_found"), + ("far", 403, "forbidden"), + ("stats", 400, "bad_request"), + ] { + let refused = open(endpoint.into()).await.err().unwrap(); + assert_eq!( + (refused.status(), refused.code()), + (status, code), + "{endpoint}: {refused}" + ); + } + + // Over the portal, with a window of its own. + let portal = Portal::over( + client::websocket_unix(&served.socket, "/v1/portal") + .await + .unwrap(), + ); + let tunnel = portal.connect_endpoint(&id, "db").await.unwrap(); + assert!(through(tunnel, &data).await == data); + let refused = portal.connect_endpoint(&id, "closed").await.err().unwrap(); + assert_eq!((refused.status(), refused.code()), (503, "not_served")); + let refused = portal.connect_endpoint(&id, "6000").await.err().unwrap(); + assert_eq!(refused.status(), 403, "{refused}"); + + // Endpoints that move with a check are a change of the resource. + let mut events = client::websocket_unix(&served.socket, "/v1/events") + .await + .unwrap(); + let follow = json!({"changes": ["resources"]}).to_string(); + events.send(Frame::Text(follow.into())).await.unwrap(); + let moved = echo().await; + let status = json!({"state": "running", "endpoints": endpoints(moved)}); + std::fs::write(fake.join("answer.status"), status.to_string()).unwrap(); + let seen: Value = client.get(&format!("/v1/resources/{id}")).await.unwrap(); + assert_eq!(seen["endpoints"][0]["port"], moved); + assert_eq!( + change_of(&mut events, "resource", "changed").await["resource"], + json!(id) + ); + assert!(through(open("db".into()).await.unwrap(), b"moved").await == b"moved"); + // One that is gone has none. + std::fs::write(fake.join("answer.status"), r#"{"state": "gone"}"#).unwrap(); + let seen: Value = client.get(&format!("/v1/resources/{id}")).await.unwrap(); + assert!(seen.get("endpoints").is_none(), "{seen}"); + let refused = open("db".into()).await.err().unwrap(); + assert_eq!(refused.status(), 404, "{refused}"); +} + +#[tokio::test(flavor = "multi_thread")] +async fn an_endpoint_is_opened_by_an_operator_and_not_by_a_viewer() { + let issuer = Issuer::new(); + let (_dir, address) = trusting(&issuer, true).await; + let with = |scope: &str| issuer.token(json!({"scope": scope})); + // The role is asked before anything of the resource is. + for (scope, status) in [("berth:view", 403), ("berth:operate", 404)] { + let io = tokio::net::TcpStream::connect(address).await.unwrap(); + let refused = berth_api::tunnel::endpoint_tunnel(io, "r-nothing", "db", &with(scope)) + .await + .err() + .unwrap(); + assert_eq!(refused.status(), status, "{scope}: {refused}"); + } + let io = tokio::net::TcpStream::connect(address).await.unwrap(); + let socket = client::websocket(io, "/v1/portal", &with("berth:view")) + .await + .unwrap(); + let portal = berth_api::portal::Portal::over(socket); + let refused = portal + .connect_endpoint("r-nothing", "db") + .await + .err() + .unwrap(); + assert_eq!(refused.status(), 403, "{refused}"); +} + +#[tokio::test(flavor = "multi_thread")] +async fn a_command_runs_in_a_resource_as_a_job_that_ends_as_it_did() { + let served = served_with(quick).await; + let fake = fake_provisioner(&served); + let id = made_box(&served, &fake, json!({"id": "db-1", "state": "running"})).await; + let mut client = served.client().await; + let client = &mut client; + let path = format!("/v1/resources/{id}/actions/exec"); + + // Offered for its kind, to an operator. + let (_, actions) = send( + client, + "GET", + &format!("/v1/resources/{id}/actions"), + Value::Null, + ) + .await; + let exec = actions + .as_array() + .unwrap() + .iter() + .find(|action| action["name"] == "exec") + .cloned() + .unwrap(); + assert_eq!(exec["role"], "operate"); + + // Its output and exit status are the job's. + std::fs::write(fake.join("say.exec"), "hi from db\n").unwrap(); + std::fs::write(fake.join("answer.exec"), r#"{"exit_code": 3}"#).unwrap(); + let asked = json!({"arguments": {"command": r#"["sh", "-c", "echo hi; exit 3"]"#, "stdin": "select 1;\n"}}); + let (status, acted) = send(client, "POST", &path, asked).await; + assert_eq!(status, 202, "{acted}"); + let (output, code) = job_ended(&served.socket, acted["job"]["id"].as_str().unwrap()).await; + assert_eq!(code, 3, "{output}"); + // The output is the command's alone. + assert_eq!(output.trim(), "hi from db", "{output}"); + let request: Value = + serde_json::from_str(&std::fs::read_to_string(fake.join("last.exec")).unwrap()).unwrap(); + assert_eq!(request["command"], json!(["sh", "-c", "echo hi; exit 3"])); + assert_eq!(request["stdin"], "select 1;\n"); + assert_eq!(request["id"], "db-1"); + // The job's record keeps the command, and not what it read. + assert_eq!( + acted["job"]["arguments"], + json!(["sh", "-c", "echo hi; exit 3"]) + ); + let (_, kept) = send( + client, + "GET", + &format!("/v1/jobs/{}", acted["job"]["id"].as_str().unwrap()), + Value::Null, + ) + .await; + assert_eq!( + kept["arguments"], + json!(["sh", "-c", "echo hi; exit 3"]), + "{kept}" + ); + assert!(!kept.to_string().contains("select 1"), "{kept}"); + + // Words split at spaces, no input, and a command that went well. + std::fs::write(fake.join("answer.exec"), r#"{"exit_code": 0}"#).unwrap(); + let asked = json!({"arguments": {"command": "uname -a"}}); + let (_, acted) = send(client, "POST", &path, asked).await; + let (output, code) = job_ended(&served.socket, acted["job"]["id"].as_str().unwrap()).await; + assert_eq!(code, 0, "{output}"); + let request: Value = + serde_json::from_str(&std::fs::read_to_string(fake.join("last.exec")).unwrap()).unwrap(); + assert_eq!(request["command"], json!(["uname", "-a"])); + assert!(request.get("stdin").is_none(), "{request}"); + + // A provisioner that says no exit status fails the job. + std::fs::write(fake.join("answer.exec"), "{}").unwrap(); + let (_, acted) = send( + client, + "POST", + &path, + json!({"arguments": {"command": "true"}}), + ) + .await; + let (output, code) = job_ended(&served.socket, acted["job"]["id"].as_str().unwrap()).await; + assert_eq!(code, 1, "{output}"); + assert!(output.contains("said no exit status"), "{output}"); + + // No command, or one that is no list, is refused before anything runs. + for command in ["", " ", "[\"sh\", 1]"] { + let asked = json!({"arguments": {"command": command}}); + let (status, said) = send(client, "POST", &path, asked).await; + assert_eq!(status, 400, "{command:?}: {said}"); + } + + // A kind without commands has no such action. + let asked = json!({"provisioner": "vm", "kind": "deployment", "name": "wiki", + "arguments": {"server": "x"}, "spec": "services: {}\n"}); + let (status, created) = send(client, "POST", "/v1/resources", asked).await; + assert_eq!(status, 202, "{created}"); + job_ended(&served.socket, created["job"]["id"].as_str().unwrap()).await; + let wiki = created["resource"]["id"].as_str().unwrap(); + let (status, said) = send( + client, + "POST", + &format!("/v1/resources/{wiki}/actions/exec"), + json!({"arguments": {"command": "true"}}), + ) + .await; + assert_eq!(status, 404, "{said}"); +} + +#[tokio::test(flavor = "multi_thread")] +async fn a_running_job_is_stopped_through_the_api() { + use berth_core::runner::{Program, Request, Subject, Work}; + let served = served().await; + let mut command = std::process::Command::new("sh"); + command.args(["-c", "echo started; sleep 30"]); + let work = Work::Program(Box::new(Program { + command, input: None, + files: vec![], + })); + let job = served + .state + .runner + .start(Request::new(Subject::Installation, "rollout"), work) + .unwrap(); + let mut client = served.client().await; + let (status, stopping) = send( + &mut client, + "POST", + &format!("/v1/jobs/{}/stop", job.id), + Value::Null, + ) + .await; + assert_eq!(status, 202, "{stopping}"); + assert_eq!(stopping["stopped_by"], "the server's own user"); + let (output, code) = job_ended(&served.socket, &job.id).await; + assert_eq!(code, 130, "{output}"); + assert!( + output.contains("✗ Stopped by the server's own user"), + "{output}" + ); + // One that has ended is not stopped again. + let (status, refused) = send( + &mut client, + "POST", + &format!("/v1/jobs/{}/stop", job.id), + Value::Null, + ) + .await; + assert_eq!( + (status, &refused["error"]["code"]), + (409, &json!("conflict")) + ); +} + +/// A listener like `echo`, on an address of one of this machine's own +/// networks, or nothing when it has none but the loopback. +async fn echo_on_a_network() -> Option<(String, u16)> { + for network in berthd::endpoints::networks() { + let Ok(listener) = tokio::net::TcpListener::bind((network.address, 0)).await else { + continue; + }; + let port = listener.local_addr().unwrap().port(); + tokio::spawn(async move { + while let Ok((connection, _)) = listener.accept().await { + tokio::spawn(async move { + use tokio::io::AsyncWriteExt; + let (mut reading, mut writing) = connection.into_split(); + let _ = tokio::io::copy(&mut reading, &mut writing).await; + let _ = writing.shutdown().await; + }); + } + }); + return Some((network.address.to_string(), port)); + } + None +} + +/// A controller that answers every request with the version of an older +/// berthd, for a node that has not been updated. +fn older_controller() -> LinkSpec { + let script = r#"import sys +for line in sys.stdin.buffer: + if line in (b"\r\n", b"\n"): + break +body = b'{"version": "0.4.0", "api": 1, "mode": "system"}' +sys.stdout.buffer.write(b"HTTP/1.1 200 OK\r\ncontent-type: application/json\r\nconnection: close\r\ncontent-length: %d\r\n\r\n" % len(body) + body) +sys.stdout.flush() +"#; + LinkSpec { + program: "python3".into(), + args: vec!["-c".into(), script.into()], + ..LinkSpec::default() + } +} + +/// Say that the fake's provisioner `vm` lives on `node`, as an apply of a node does. +fn lives_on(served: &Served, node: &str) { + let paths = &served.state.paths; + let mut entry = berth_core::provisioner::entry(paths, "vm") + .unwrap() + .unwrap(); + entry.node = node.into(); + berth_core::provisioner::save_entry(paths, "vm", &entry).unwrap(); +} + +#[tokio::test(flavor = "multi_thread")] +async fn a_resource_of_a_node_is_reached_through_the_nodes_controller() { + use berth_api::{portal::Portal, tunnel::endpoint_tunnel}; + let Some(node) = Berthd::start().await else { + return; }; - app.job = Some(crate::job::Job::start(vec![stage], "", vec![]).unwrap()); - app.machine_job = true; - for _ in 0..500 { - app.job.as_mut().unwrap().tick(); - if !app.job.as_ref().unwrap().running() { - break; + let served = served_with(quick).await; + let fake = fake_provisioner(&served); + served + .state + .add(Remote::new("big", "192.0.2.30", node.link())); + let (port, closed) = (echo().await, closed_port()); + let network = echo_on_a_network().await; + let mut endpoints = vec![ + json!({"name": "db", "address": "127.0.0.1", "port": port, "local": true}), + json!({"name": "closed", "address": "127.0.0.1", "port": closed, "local": true}), + json!({"name": "far", "address": "192.0.2.1", "port": 5432, "local": true}), + json!({"name": "meta", "address": "169.254.169.254", "port": 80, "local": true}), + json!({"name": "loop", "address": "127.0.0.2", "port": port, "local": true}), + ]; + if let Some((address, port)) = &network { + endpoints.push(json!({"name": "bridge", "address": address, "port": port, "local": true})); + } + let answer = json!({"id": "db-1", "state": "running", "endpoints": endpoints}); + let id = made_box(&served, &fake, answer.clone()).await; + lives_on(&served, "big"); + std::fs::write(fake.join("answer.status"), answer.to_string()).unwrap(); + + // The provisioner is told of the node, so its plugin reports what the node sees. + let status: Value = served + .client() + .await + .get(&format!("/v1/resources/{id}")) + .await + .unwrap(); + assert_eq!(status["state"], "running", "{status}"); + let asked: Value = + serde_json::from_str(&std::fs::read_to_string(fake.join("last.status")).unwrap()).unwrap(); + assert_eq!(asked["node"], "big", "{asked}"); + + let open = |endpoint: &str| { + let (socket, id, endpoint) = (served.socket.clone(), id.clone(), endpoint.to_owned()); + async move { + let io = tokio::net::UnixStream::connect(&socket).await.unwrap(); + endpoint_tunnel(io, &id, &endpoint, "").await + } + }; + // The node's loopback, both ways, with each side's end and a few MB. + let data: Vec = (0..3_000_000u32).map(|n| (n % 251) as u8).collect(); + assert_eq!(through(open("db").await.unwrap(), b"hello").await, b"hello"); + assert!(through(open("db").await.unwrap(), &data).await == data); + let portal = Portal::over( + client::websocket_unix(&served.socket, "/v1/portal") + .await + .unwrap(), + ); + let tunnel = portal.connect_endpoint(&id, "db").await.unwrap(); + assert!(through(tunnel, &data).await == data); + // An address on one of the node's own networks, as a container's on a bridge. + match &network { + Some(_) => { + assert!(through(open("bridge").await.unwrap(), &data).await == data); + let tunnel = portal.connect_endpoint(&id, "bridge").await.unwrap(); + assert_eq!(through(tunnel, b"bridged").await, b"bridged"); } - std::thread::sleep(Duration::from_millis(10)); + None => eprintln!("skipped the bridge: this machine has no network but the loopback"), + } + // Nothing on it, off the node's networks, a metadata address, another loopback address. + for (endpoint, status, said) in [ + ("closed", 503, "Nothing answers"), + ("far", 403, "no network"), + ("meta", 403, "metadata"), + ("loop", 403, "127.0.0.1"), + ] { + let refused = open(endpoint).await.err().unwrap(); + assert_eq!(refused.status(), status, "{endpoint}: {refused}"); + assert!(refused.to_string().contains(said), "{endpoint}: {refused}"); } - app.close_job(); - assert_eq!(app.step, Step::Manage, "{}", app.status); - assert_eq!(app.config["access"]["address"], "192.0.2.40"); - assert!(!app.unsaved()); - - // What the screen saves next, as before it starts the machine, keeps the address. - app.save().unwrap(); - let kept = own.load(&served.state.paths).unwrap().unwrap(); - assert_eq!(kept["access"]["address"], "192.0.2.40", "{kept}"); + + // A caller does not reach the node's stream through the server, however it asks. + for path in [ + format!("/v1/hosts/big/controller/endpoints/127.0.0.1/{port}"), + "/v1/hosts/big/controller/endpoints/192.0.2.1/80".to_owned(), + "/v1/hosts/big/controller/%65ndpoints/192.0.2.1/80".to_owned(), + "/v1/hosts/nobody/controller/endpoints/192.0.2.1/80".to_owned(), + ] { + let refused = client::websocket_unix(&served.socket, &path) + .await + .err() + .unwrap(); + assert_eq!(refused.status(), 403, "{path}: {refused}"); + } + let mut client = served.client().await; + let plain = client + .get::("/v1/hosts/big/controller/endpoints/192.0.2.1/80") + .await + .unwrap_err(); + assert_eq!(plain.status(), 403, "{plain}"); + let args = json!({"address": "192.0.2.1", "port": 80}); + let refused = portal + .open("endpoints.connect", Some("big"), args.clone()) + .await + .err() + .unwrap(); + assert_eq!(refused.status(), 403, "{refused}"); + // Nor through the node's own portal, passed through. + let passed = Portal::over( + client::websocket_unix(&served.socket, "/v1/hosts/big/controller/portal") + .await + .unwrap(), + ); + let refused = passed + .open("endpoints.connect", None, args) + .await + .err() + .unwrap(); + assert_eq!(refused.status(), 403, "{refused}"); + + // A node that is gone, or whose controller is older than the stream, says so. + lives_on(&served, "gone"); + let refused = open("db").await.err().unwrap(); + assert_eq!(refused.status(), 502, "{refused}"); + assert!(refused.to_string().contains("gone"), "{refused}"); + served + .state + .add(Remote::new("old", "192.0.2.31", older_controller())); + lives_on(&served, "old"); + let refused = open("db").await.err().unwrap(); + assert_eq!(refused.status(), 501, "{refused}"); + assert!( + refused.to_string().contains("update its controller"), + "{refused}" + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn an_admin_over_the_network_does_not_reach_a_hosts_stream_for_the_server() { + let issuer = Issuer::new(); + let (_dir, address) = trusting(&issuer, true).await; + let admin = issuer.token(json!({"scope": "berth:admin"})); + let path = "/v1/hosts/big/controller/endpoints/192.0.2.1/80"; + let io = tokio::net::TcpStream::connect(address).await.unwrap(); + let refused = client::websocket(io, path, &admin).await.err().unwrap(); + assert_eq!(refused.status(), 403, "{refused}"); + let (status, said) = as_bearer(address, &admin, "GET", path, Value::Null).await; + assert_eq!(status, 403, "{said}"); + let io = tokio::net::TcpStream::connect(address).await.unwrap(); + let portal = + berth_api::portal::Portal::over(client::websocket(io, "/v1/portal", &admin).await.unwrap()); + let refused = portal + .open( + "endpoints.connect", + Some("big"), + json!({"address": "192.0.2.1", "port": 80}), + ) + .await + .err() + .unwrap(); + assert_eq!(refused.status(), 403, "{refused}"); +} + +#[tokio::test(flavor = "multi_thread")] +async fn a_request_that_claims_to_be_a_stream_needs_its_role_all_the_same() { + let issuer = Issuer::new(); + let (_dir, address) = trusting(&issuer, true).await; + let viewer = issuer.token(json!({"scope": "berth:view"})); + let bearer = format!("Bearer {viewer}"); + for (method, path, body) in [ + ("PUT", "/v1/secrets/main", r#"{"value": "x"}"#), + ("GET", "/v1/secrets", ""), + ("PUT", "/v1/kv/ui/x", "1"), + ("PUT", "/v1/provisioners/vm", r#"{"type": "fake/vm"}"#), + ("POST", "/v1/plugins/package", ""), + ] { + let stream = tokio::net::TcpStream::connect(address).await.unwrap(); + let mut client = Client::over(stream).await.unwrap(); + let headers = [ + ("authorization", bearer.as_str()), + ("content-type", "application/json"), + ("upgrade", "websocket"), + ("connection", "upgrade"), + ]; + let answer = client + .send(method, path, &headers, body.to_owned().into()) + .await + .unwrap(); + assert_eq!(answer.status, 403, "{method} {path}"); + } + let (status, _) = as_bearer(address, &viewer, "HEAD", "/v1/secrets", Value::Null).await; + assert_eq!(status, 403); +} + +#[tokio::test(flavor = "multi_thread")] +async fn a_profile_does_not_say_how_ansible_reaches_the_host() { + let issuer = Issuer::new(); + let (_dir, address) = trusting(&issuer, true).await; + let admin = issuer.token(json!({"scope": "berth:admin"})); + let vars = json!({"builder": {}, "access": {"address": "192.0.2.7"}, + "ansible_vars": {"ansible_ssh_common_args": "-o ProxyCommand=\"sh -c id\"", + "ansible_connection": "local", "ansible_ssh_executable": "/bin/sh", + "devbox_geometry": "1x1"}}); + let (status, said) = as_bearer( + address, + &admin, + "PUT", + "/v1/hosts/studio/profile", + vars.clone(), + ) + .await; + assert_eq!(status, 400, "{said}"); + let message = said["error"]["message"].as_str().unwrap(); + assert!( + message.contains("not ansible_connection, ansible_ssh_common_args, ansible_ssh_executable"), + "{message}" + ); + // Nor from the server's own user. + let served = served_with(quick).await; + let mut client = served.client().await; + let (status, said) = send(&mut client, "PUT", "/v1/hosts/studio/profile", vars).await; + assert_eq!(status, 400, "{said}"); + let feature = json!({"builder": {}, "access": {"address": "192.0.2.7"}, + "ansible_vars": {"devbox_plugin_features": ["../../tmp/x"]}}); + let (status, said) = send(&mut client, "PUT", "/v1/hosts/studio/profile", feature).await; + assert_eq!(status, 400, "{said}"); + assert!( + said["error"]["message"] + .as_str() + .unwrap() + .contains("PLUGIN/FEATURE"), + "{said}" + ); + + // A profile saved before still lists and reads. + let paths = &served.state.paths; + paths.prepare().unwrap(); + berth_core::profiles::Place::own("dock") + .unwrap() + .save( + paths, + &json!({"builder": {}, "access": {"address": "192.0.2.8"}, + "ansible_vars": {"ansible_connection": "local", "devbox_geometry": "1x1"}}), + ) + .unwrap(); + let (status, listed) = send(&mut client, "GET", "/v1/hosts", Value::Null).await; + assert_eq!(status, 200, "{listed}"); + assert!(listed.to_string().contains("\"dock\""), "{listed}"); + let (status, profile) = send(&mut client, "GET", "/v1/hosts/dock/profile", Value::Null).await; + assert_eq!(status, 200, "{profile}"); } diff --git a/crates/berth/src/serve/tests/keys.rs b/crates/berth/src/local/serve/tests/keys.rs similarity index 100% rename from crates/berth/src/serve/tests/keys.rs rename to crates/berth/src/local/serve/tests/keys.rs diff --git a/crates/berth/src/local/serve/tests/remote.rs b/crates/berth/src/local/serve/tests/remote.rs new file mode 100644 index 0000000..3efda10 --- /dev/null +++ b/crates/berth/src/local/serve/tests/remote.rs @@ -0,0 +1,244 @@ +//! The screens against a berth serve of their own over the network: a setup +//! key signs them in, and its portal answers. + +use super::*; +use berth_page::device::Linked; +use berth_page::portal::{Link, Portal}; +use berth_page::settings::{self, Settings}; +use berth_tui::{Code, Event, Key}; +use ratatui::{Terminal as Screen, backend::TestBackend}; +use std::cell::RefCell; +use std::rc::Rc; + +fn screen(app: &mut berth_page::App) -> String { + let mut terminal = Screen::new(TestBackend::new(120, 30)).unwrap(); + terminal.draw(|frame| app.draw(frame)).unwrap(); + let buffer = terminal.backend().buffer(); + (0..30) + .map(|y| { + (0..120) + .map(|x| buffer[(x, y)].symbol()) + .collect::() + }) + .collect::>() + .join("\n") +} + +/// Take in what the network says until `done`, for up to ten seconds. +fn until(what: &str, mut done: impl FnMut() -> bool) { + let started = std::time::Instant::now(); + while !done() { + assert!( + started.elapsed() < Duration::from_secs(10), + "no {what} in time" + ); + berth_page::pump(); + std::thread::sleep(Duration::from_millis(20)); + } +} + +#[tokio::test(flavor = "multi_thread")] +async fn the_screens_sign_in_with_a_setup_key_and_use_the_portal() { + let dir = tempfile::tempdir().unwrap(); + let state = state_in(dir.path(), Some("s3cret-token")); + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let url = format!("http://{}", listener.local_addr().unwrap()); + let router = api::router(state.clone(), true); + tokio::spawn(async move { axum::serve(listener, router).await }); + let (key, _) = super::super::setup_key::make(&state.paths).unwrap(); + + let client = tempfile::tempdir().unwrap(); + let kept = client.path().join("berth"); + let linked = Linked { + server: Some(url.clone()), + token: None, + setup_key: Some(key), + }; + berth_page::device::install(linked, Settings::new(Some(kept.clone()))); + + // The key collects the token, which is kept for the next run. + let mut app = berth_page::App::new(); + until("token", || { + app.tick(); + screen(&mut app).contains("is installed and serves at") + }); + let settings = Settings::new(Some(kept.clone())); + assert_eq!( + settings.get(settings::TOKEN).as_deref(), + Some("s3cret-token") + ); + assert_eq!( + settings.get(settings::SERVER).as_deref(), + Some(url.as_str()) + ); + + // Done ends the key and shows the hosts of the server. + app.event(Event::Key(Key { + code: Code::Enter, + ctrl: false, + alt: false, + shift: false, + })); + until("end of the key", || { + app.tick(); + state + .store + .read(kind::TOKEN, store::OWN, super::super::setup_key::NAME) + .unwrap() + .is_none() + }); + until("hosts", || { + app.tick(); + let shown = screen(&mut app); + shown.contains("Hosts") && !shown.contains("is installed") + }); + + // The portal answers with the token, and a wrong one is refused. + let portal = Portal::open(&url, "s3cret-token").unwrap(); + let hosts = Rc::new(RefCell::new(None)); + let answer = hosts.clone(); + portal.call("hosts.list", None, json!({}), Value::Null, move |result| { + *answer.borrow_mut() = Some(result) + }); + until("answer", || hosts.borrow().is_some()); + assert_eq!(portal.link(), Link::Open); + assert_eq!(hosts.borrow_mut().take().unwrap(), Ok(json!([]))); + let refused = Portal::open(&url, "wrong").unwrap(); + until("refusal", || refused.link() != Link::Connecting); + assert_eq!(refused.link(), Link::Refused); +} + +/// A server of berth 0.5.2 as its portal answers: every call with its version. +async fn older_server(token: &'static str) -> String { + use axum::extract::ws::{Message, WebSocketUpgrade}; + let portal = axum::routing::get(move |ws: WebSocketUpgrade| async move { + ws.protocols([token]).on_upgrade(|mut socket| async move { + while let Some(Ok(Message::Text(text))) = socket.recv().await { + let asked: Value = serde_json::from_str(text.as_str()).unwrap(); + let said = json!({"id": asked["id"], "result": {"version": "0.5.2", "api": 1, "mode": "serve"}}); + if socket.send(Message::Text(said.to_string().into())).await.is_err() { + break; + } + } + }) + }); + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let url = format!("http://{}", listener.local_addr().unwrap()); + let router = axum::Router::new().route(berth_api::ops::PORTAL, portal); + tokio::spawn(async move { axum::serve(listener, router).await }); + url +} + +#[tokio::test(flavor = "multi_thread")] +async fn a_client_refuses_an_older_server_and_still_reads_its_version() { + use crate::cli::{args::Global, client::Client, fail}; + let url = older_server("s3cret-token").await; + let dir = tempfile::tempdir().unwrap(); + let token = dir.path().join("token"); + std::fs::write(&token, "s3cret-token").unwrap(); + let global = Global { + server: Some(url.clone()), + token_file: Some(token), + ..Global::default() + }; + let kept = Settings::new(Some(dir.path().join("berth"))); + let (opened, version) = tokio::task::spawn_blocking(move || { + let opened = Client::open_in(&global, &kept).map(|_| ()); + let version = Client::open_any(&global, &kept).and_then(|client| client.server_version()); + (opened, version) + }) + .await + .unwrap(); + let refused = opened.unwrap_err(); + assert_eq!(refused.status, fail::REFUSED); + assert!( + refused + .message + .contains("is berth 0.5.2 (API 1), older than this berth") + && refused.message.contains("Update the server first"), + "{}", + refused.message + ); + assert_eq!(version.unwrap(), ("0.5.2".to_owned(), 1)); +} + +#[tokio::test(flavor = "multi_thread")] +async fn the_screens_say_that_an_older_server_is_updated_first() { + let url = older_server("s3cret-token").await; + let client = tempfile::tempdir().unwrap(); + let kept = client.path().join("berth"); + let linked = Linked { + server: Some(url.clone()), + token: Some("s3cret-token".into()), + setup_key: None, + }; + berth_page::device::install(linked, Settings::new(Some(kept.clone()))); + let mut app = berth_page::App::new(); + until("the older server", || { + app.tick(); + screen(&mut app).contains("older than this berth") + }); + // The token stays for when the server is updated. + assert_eq!( + Settings::new(Some(kept)).get(settings::TOKEN).as_deref(), + Some("s3cret-token") + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn the_screens_keep_the_token_of_a_server_that_does_not_answer_or_refuses_it() { + // Nobody listens at this address. + let closed = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let away = format!("http://{}", closed.local_addr().unwrap()); + drop(closed); + let client = tempfile::tempdir().unwrap(); + let kept = client.path().join("berth"); + let settings = Settings::new(Some(kept.clone())); + settings + .set(&[(settings::SERVER, &away), (settings::TOKEN, "s3cret-kept")]) + .unwrap(); + let linked = Linked { + server: None, + token: None, + setup_key: None, + }; + berth_page::device::install(linked, Settings::new(Some(kept.clone()))); + let mut app = berth_page::App::new(); + let started = std::time::Instant::now(); + while started.elapsed() < Duration::from_secs(2) { + app.tick(); + berth_page::pump(); + std::thread::sleep(Duration::from_millis(20)); + } + assert_eq!( + settings.get(settings::TOKEN).as_deref(), + Some("s3cret-kept") + ); + drop(app); + + // A server that refuses it asks for another, and the kept one stays until then. + let dir = tempfile::tempdir().unwrap(); + let state = state_in(dir.path(), Some("s3cret-token")); + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let url = format!("http://{}", listener.local_addr().unwrap()); + let router = api::router(state, true); + tokio::spawn(async move { axum::serve(listener, router).await }); + settings + .set(&[(settings::SERVER, &url), (settings::TOKEN, "s3cret-kept")]) + .unwrap(); + let linked = Linked { + server: None, + token: None, + setup_key: None, + }; + berth_page::device::install(linked, Settings::new(Some(kept.clone()))); + let mut app = berth_page::App::new(); + until("the refusal", || { + app.tick(); + screen(&mut app).contains("did not take that token") + }); + assert_eq!( + settings.get(settings::TOKEN).as_deref(), + Some("s3cret-kept") + ); +} diff --git a/crates/berth/src/store_command.rs b/crates/berth/src/local/store_command.rs similarity index 96% rename from crates/berth/src/store_command.rs rename to crates/berth/src/local/store_command.rs index 1dc4186..5f67e26 100644 --- a/crates/berth/src/store_command.rs +++ b/crates/berth/src/local/store_command.rs @@ -1,4 +1,4 @@ -//! `berth store`: where the store is, sealing its values with a key, and +//! `berth server store`: where the store is, sealing its values with a key, and //! copying it to another database. use crate::Result; @@ -11,14 +11,14 @@ use std::io::{BufRead, IsTerminal, Write}; /// Asks a yes-or-no question. type Confirm<'a> = &'a mut dyn FnMut(&str) -> Result; -const USAGE: &str = "Usage: berth store [encrypt | rekey | decrypt [--yes]] - berth store copy TARGET | - | --target-file FILE [--into-existing] +const USAGE: &str = "Usage: berth server store [status | encrypt | rekey | decrypt] + berth server store copy TARGET | - | --target-file FILE [--into-existing] copy writes everything of this store into TARGET, a SQLite file or a postgres:// URL, as the store setting takes it. A URL with a password shows up in the process list: give it on standard input (-) or in a file instead."; -/// `berth store [encrypt | rekey | decrypt [--yes] | copy …]` +/// `berth server store [status | encrypt | rekey | decrypt | copy …]` pub fn command(paths: &Paths, args: &[&str]) -> Result<()> { let installation = Installation::of(paths); let mut out = std::io::stdout().lock(); @@ -70,7 +70,7 @@ fn status(installation: &Installation, out: &mut dyn Write) -> Result<()> { )?, None => writeln!( out, - "Encrypted no; berth store encrypt would keep its key in {file}" + "Encrypted no; berth server store encrypt would keep its key in {file}" )?, } let report = store.seal_report()?; @@ -125,7 +125,7 @@ fn encrypt(paths: &Paths, installation: &Installation, out: &mut dyn Write) -> R let file = &installation.key_file; if store.encrypted() || file.exists() { return Err(format!( - "The store is encrypted already, with the key in {}. berth store rekey changes the key", + "The store is encrypted already, with the key in {}. berth server store rekey changes the key", file.display() ) .into()); @@ -146,7 +146,7 @@ fn rekey(paths: &Paths, installation: &Installation, out: &mut dyn Write) -> Res let _serve = hold_serve(paths)?; let store = installation.open()?; if !store.encrypted() { - return Err("The store is not encrypted: berth store encrypt makes its key".into()); + return Err("The store is not encrypted: berth server store encrypt makes its key".into()); } let _keys = hold_keys(installation)?; let (count, id) = rewrite_with_new_key(installation, &store)?; @@ -217,7 +217,7 @@ fn same(one: &Target, other: &Target) -> bool { } } -/// `berth store copy TARGET | - | --target-file FILE [--into-existing]`: +/// `berth server store copy TARGET | - | --target-file FILE [--into-existing]`: /// every table of this store into another. It does not switch to it. fn copy( paths: &Paths, @@ -295,7 +295,7 @@ fn copy( }; writeln!( out, - "This berth still uses the {}. To switch: berth config store {shown}, or set BERTH_STORE.", + "This berth still uses the {}. To switch: berth config set store {shown}, or set BERTH_STORE.", installation.target.describe() )?; if from.encrypted() || from.seal_report()?.sealed > 0 { @@ -463,7 +463,7 @@ mod tests { assert!(said.contains(" kv 1 rows"), "{said}"); assert!(said.contains("Left out 1 rows"), "{said}"); assert!( - said.contains(&format!("berth config store {}", target.display())), + said.contains(&format!("berth config set store {}", target.display())), "{said}" ); assert!(said.contains("key file has to be there"), "{said}"); @@ -601,7 +601,7 @@ mod tests { ) .unwrap(); paths.own_key().unwrap(); - let token = crate::serve::token(paths).unwrap(); + let token = crate::local::serve::token(paths).unwrap(); // A plugin from an upload and one copied in as a folder. let own = berth_core::plugins::own_dir(&paths.config_dir); fs::create_dir_all(own.join("uploads")).unwrap(); @@ -656,7 +656,7 @@ mod tests { let file = paths.own_key().unwrap(); assert_eq!(file, paths.data_dir.join("keys/berth")); assert_eq!(fs::read(&file).unwrap(), key); - assert_eq!(crate::serve::token(paths).unwrap(), token); + assert_eq!(crate::local::serve::token(paths).unwrap(), token); assert_eq!( fs::read_to_string(paths.data_dir.join("serve.token")) .unwrap() diff --git a/crates/berth/src/main.rs b/crates/berth/src/main.rs index f59283e..fec16e3 100644 --- a/crates/berth/src/main.rs +++ b/crates/berth/src/main.rs @@ -1,517 +1,22 @@ -mod app; -mod commands; -mod fields; -mod key_command; -mod keys; -mod mirror; -mod picker; -mod provisioner_command; -mod provisioner_tab; -mod remote; -mod resource_command; -mod serve; -mod session; -mod store_command; -mod ui; +//! berth: set up and look after agent workstations. Its commands and its +//! screens are clients of a berth server's API, on Linux, macOS and Windows. +//! On Linux the program has the server in it as well: `berth serve`, and +//! one in the process for a command when no other is set. + +mod cli; +#[cfg(target_os = "linux")] +mod local; +#[cfg(not(target_os = "linux"))] +#[path = "client_only.rs"] +mod local; mod update; -use app::{App, parse_config}; -use berth_core::{credentials, discovery, job, paths, plugins, registry}; -pub(crate) use berth_tui::{markdown, theme}; -use crossterm::{cursor::SetCursorStyle, event}; -use fields::{Action, Field, Step}; -use paths::{Paths, layout}; -use serde_json::json; -use std::{error::Error, fs, io::Write, process::Command}; +use std::error::Error; type Result = std::result::Result>; -const USAGE: &str = "berth: set up and look after agent workstations. - -Usage: - berth choose a host, or create one - berth HOST open a host (a saved name, or a path to a profile) - berth list saved hosts - berth host add NAME ADDRESS [--user USER] [--port PORT] [--auth AUTH] - [--level reachable|controller|workstation] [--no-contact] - save a machine that exists already, by its address, and - install berth's key there; it is reachable unless told - berth host install HOST install berth's controller alone on a host below a - workstation: sessions, tasks and files, no desktop - berth host level HOST reachable|controller|workstation - say what the host is, as for one added by its address - that berth set up before; nothing on the host changes - berth host rename HOST NAME [--hostname] - give a host another name in berth, with what berth keeps - of it; the machine is left as it is, and --hostname makes - NAME its profile's hostname for the next apply - berth iso HOST [--write /dev/sdX] - build the unattended installer; write it to a USB drive - berth seed HOST render the cloud-init seed - berth apply HOST apply the host's profile to it over SSH - berth apply --behind [--update] [--reboot [--force]] - apply every host that is behind, one at a time, stopping - at the first that fails; --all takes current ones too, - --update runs their update task, --reboot reboots those - that need it unless sessions run (--force: even then) - berth connect URL use the berth server at URL: give it the setup key made - there with berth setup-key, on standard input - berth remove HOST forget a host; the machine is left as it is - berth profile HOST the host's profile, as JSON on standard output - berth profile HOST --set FILE - save the profile in FILE (- for standard input) as the - host's; a host of a berth server keeps it on the server - berth credentials HOST [--show] - the passwords berth generated for the host's accounts; - only to a terminal, unless --show - berth adopt HOST [[USER@]ADDRESS] [--port PORT] [--key FILE] [--force] - save a host from the profile it keeps, read over SSH - with sudo; without an address, a saved host by its own; - --force replaces a host of that name - berth provisioners what can create machines, and what each asks for - berth provisioner the provisioners, and whether each can be used here - berth provisioner types the types of provisioner the plugins bring - berth provisioner add NAME TYPE [SETTING=VALUE…] - register a provisioner of a type, such as incus/vm, - under a name of its own, with its settings - berth provisioner remove NAME - forget a provisioner that holds nothing - berth provisioner NAME [SETTING=VALUE…] - how it is now, what berth made with it, and its - actions; the values are of its own arguments, this once - berth provisioner NAME actions - berth provisioner NAME run ACTION [NAME=VALUE…] [--yes] - run one of its actions, such as settings, which keeps - where it is, secrets, new-deployment, or the plugin's own - berth resource [--provisioner NAME] [--kind KIND] [--host HOST] - what provisioners made for berth: machines, deployments - berth resource ID one of them as its provisioner says it is now - berth resource ID actions | run ACTION [NAME=VALUE…] [--yes] | remove [--yes] - berth resource ID logs [--tail N] [--follow] - berth resource ID change [--spec FILE|-] [--secret NAME[:VARIABLE]]… [NAME=VALUE…] - berth deploy NAME --provisioner PROVISIONER --spec FILE|- [--secret NAME[:VARIABLE]]… - [NAME=VALUE…] a deployment from a specification, such as a Compose - file; again with the same name gives it the new one - berth provision HOST PROVISIONER [NAME=VALUE…] [--yes] - create the machine for a saved host and set it up - berth instance HOST the host's machine, as its resource - berth instance HOST start|stop - berth instance HOST destroy [--yes] - delete the machine; the profile stays - berth key this berth's own SSH public key, which the operator of a - host it sets up takes - berth tasks HOST what the host can do: its tasks and those of its plugins - berth run HOST TASK [NAME=VALUE…] [--yes] - run one of them and leave with its exit code - (berth manage HOST TASK is the same) - berth exec HOST [--user agent|operator|root] [--keep] -- COMMAND… - run a command on the host; its exit code is berth's - berth attach HOST [ID] this terminal onto a terminal session of the host, or onto - a new shell as the agent; Ctrl+Q leaves it running - berth sessions HOST the host's sessions, running and ended - berth output HOST ID [--tail N] [--follow] - what a session wrote - berth channel HOST [TOPIC-PREFIX] [--since ID] - messages on the host's channel as they come - berth call HOST SERVICE METHOD [JSON] - call a service a plugin runs on the host - berth inventory [HOST…] the hosts as an Ansible inventory - berth ssh HOST [COMMAND] a shell on the host over plain SSH, or run a command there - berth ssh HOST --agent the same as the agent account - berth registry registries (also the Registries tab of the home screen): - Nexus servers with repositories, npm, file and Docker registries - berth registry add NAME nexus|npm|raw|docker URL [USERNAME] - berth registry repo NAME npm|raw|docker REPOSITORY [HOST[:PORT]] - berth registry login NAME store the password or token of a registry - berth registry check [NAME] - whether a registry answers and accepts its login - berth registry remove NAME - berth secret [set NAME | remove NAME] - secret values, kept in berth's store (see berth store); - tailnet_api_key is the control server's API key, for - enrollment keys - berth tailnet-key HOST create a one-use enrollment key for HOST, valid for an - hour, at the control server of the tailnet_ settings - (HOST may also be the path of its profile) - berth host HOST what a host is given - berth host HOST secret NAME env VARIABLE | file PATH [MODE] | off - berth host HOST registry NAME on|off - berth publish REGISTRY[/REPOSITORY] [HOST [--login [REGISTRY]]] - upload the release archive to a Nexus raw repository or - file server and use it as HOST's release address; with - --login the cloud seed carries that registry's login - berth image push REGISTRY[/REPOSITORY] [TAG] - push the container image - berth plugins plugins and where they come from - berth plugins add SPEC install a plugin with npm: a name, a tarball or a folder - berth plugins remove NAME remove one of yours, by its id or npm name - berth plugins update [ID] [--check] [--apply [--reboot [--force]]] - update one plugin or all to their registry's latest - version (--check only says which have one), then say - which hosts are behind; --apply brings them up to date - berth --server URL [--token TOKEN] [COMMAND…] - this once, use the hosts of a berth serve elsewhere - (http://HOST:PORT, https://HOST or unix:/path); the - server setting does the same for every run - berth --version the version - berth update [--check] install the latest release in this program's place; hosts - get its berthd and box when they are applied again - berth config show where things are kept and the settings - berth config KEY [VALUE] change a setting; no value clears it - berth serve [--listen ADDRESS:PORT [--expose]] - an API over all hosts on a socket of this user; with - --listen also over HTTP behind a token, on this machine - only unless --expose allows another address - berth setup-key [--remove] make a key, valid for a day, that lets the first user - collect the token of the berth serve on this machine; - --remove ends it - berth inbox [HOST] [--topic PREFIX] [--all] - messages taken in from the hosts' channels - berth kv get|set|rm|ls NAMESPACE [KEY] [JSON] - the key-value store - berth store where berth keeps its state, and whether it is encrypted - berth store encrypt | rekey | decrypt [--yes] - seal the store's values with a new key file, change the - key, or write them plain again; berth serve is stopped first - berth store copy TARGET | - | --target-file FILE [--into-existing] - copy the store into another SQLite file or PostgreSQL - database, which berth config store then switches to; a - URL as TARGET shows up in the process list, so - reads - it from standard input - berth openapi | asyncapi | operations - the API documents of berth serve"; - -fn show_config(paths: &Paths) -> Result<()> { - // The settings of the file are shown when the store does not open. - let (settings, unopened) = match paths.settings() { - Ok(settings) => (settings, None), - Err(error) => (paths.local_settings()?, Some(error)), - }; - println!( - "Settings {}, and the store", - paths.settings_file().display() - ); - let store = berth_core::store::Installation::of(paths).target.describe(); - match &unopened { - None => println!("Store {store}: settings, hosts, registries and secrets"), - Some(error) => println!("Store {store}, which does not open: {error}"), - } - match berth_core::server::Server::configured(paths) { - Some(server) => println!( - "Hosts on the berth server at {}, with a copy in the store", - server.url - ), - None => println!("Hosts in the store, with their passwords and machines"), - } - println!("Builds {}", paths.build_dir().display()); - match paths.source() { - Ok(source) => println!("Source {}", source.display()), - Err(error) => println!("Source {error}"), - } - println!(); - for (key, help) in paths::SETTINGS { - // What lets someone in is not for a screen. - let value = match settings.get(*key).and_then(|value| value.as_str()) { - Some(_) if paths::is_secret(key) => Some("(set)"), - value => value, - }; - let value = match (&unopened, paths::is_local(key)) { - (Some(_), false) => Some("(in the store, which does not open)"), - _ => value, - }; - println!("{key} = {}\n {help}", value.unwrap_or("(not set)")); - } - Ok(()) -} - -/// `berth config KEY VALUE`: the tailnet settings of a berth server this -/// berth uses are the server's, and change there. -fn change_setting(paths: &Paths, key: &str, value: &str) -> Result<()> { - match berth_core::server::Server::configured(paths) { - Some(server) if berth_core::tailnet::SETTINGS.contains(&key) => { - server.call( - "settings.set", - json!({"key": key}), - Some(json!({"value": value})), - )?; - println!("Changed {key} on the berth server at {}", server.url); - Ok(()) - } - _ => paths.set(key, value), - } -} - -fn enter() -> Result { - let terminal = ratatui::init(); - crossterm::execute!( - std::io::stdout(), - event::EnableBracketedPaste, - event::EnableMouseCapture, - SetCursorStyle::BlinkingBar - )?; - Ok(terminal) -} - -fn leave() { - let _ = crossterm::execute!( - std::io::stdout(), - event::DisableBracketedPaste, - event::DisableMouseCapture, - SetCursorStyle::DefaultUserShape - ); - ratatui::restore(); -} - -/// Load a host's profile, starting from the example for a new one. -/// Whether these arguments open the terminal screen: none, or a host's name. -fn opens_screen(args: &[&str]) -> bool { - // The commands that are one word. - const ALONE: &[&str] = &[ - "list", - "registry", - "secret", - "plugins", - "provisioners", - "provisioner", - "resource", - "inventory", - "inbox", - "kv", - "store", - "help", - ]; - match args { - [] => true, - [one] => !one.starts_with('-') && !ALONE.contains(one), - _ => false, - } -} - -/// Whether a command reads this machine's own plugins: the screen, and the -/// commands that apply, build or provision with them or list them. -fn loads_plugins(args: &[&str]) -> bool { - const VERBS: &[&str] = &[ - "apply", - "plugins", - "provisioners", - "provisioner", - "provision", - "instance", - "resource", - "deploy", - "seed", - "iso", - "publish", - "host", - ]; - opens_screen(args) || args.first().is_some_and(|verb| VERBS.contains(verb)) -} - -/// Put back the plugins berth's store has and this machine's plugin -/// directory lacks, as on a machine restored from the store, and say so on -/// standard error. -pub(crate) fn restore_plugins(paths: &Paths) { - match berth_core::plugin_state::restore(paths) { - Ok(notices) => notices.iter().for_each(|notice| eprintln!("{notice}")), - Err(error) => eprintln!("berth: the plugins in berth's store were not checked: {error}"), - } -} - -fn open(paths: &Paths, host: &str) -> Result { - let place = berth_core::profiles::Place::resolve(paths, host)?; - paths.prepare()?; - // Actions call the builder scripts and playbook by relative path. - std::env::set_current_dir(paths.source()?)?; - let (data, existing) = match place.load(paths)? { - Some(profile) => (profile.to_string().into_bytes(), true), - None => (fs::read(layout::EXAMPLE)?, false), - }; - let served = place.server_url().is_some(); - let mut app = App::new(paths.clone(), place, parse_config(&data)?); - // With a server set, a new host is one of the server's. - if let (false, true, Some(server)) = ( - existing, - served, - berth_core::server::Server::configured(paths), - ) { - if !server.admin(paths) { - return Err(format!( - "No host {host} on the berth server at {}. Adding one needs the role admin there.", - server.url - ) - .into()); - } - app.config["builder"]["served"] = json!(server.url); - } - app.build_dir = paths.build_dir(); - app.iso_dir = paths.iso_dir(); - app.load_plugins(paths.config_dir.join("plugins")); - // A host is reached without the store; what needs it says why it is missing. - app.access = registry::Store::new(paths) - .map_err(|error| format!("berth's store does not open: {error}")); - // A server's host is set up from the server, with the server's plugins. - if app.served().is_some() { - let known = app - .host_profile() - .served(paths) - .and_then(|server| server.call("setup.catalogue", json!({}), None).ok()); - // The server's plugins are what its hosts can have. - if let Some(features) = known - .as_ref() - .and_then(|found| found["features"].as_array()) - { - let text = |feature: &serde_json::Value, key: &str| { - feature[key].as_str().unwrap_or("").to_owned() - }; - app.set_features( - features - .iter() - .map(|feature| berth_tui::setup::Feature { - key: text(feature, "key"), - label: text(feature, "label"), - group: text(feature, "group"), - help: text(feature, "help"), - }) - .collect(), - ); - } - } - // The keys of the ring that logs in to the host: the server's for its hosts. - let ring = match app.host_profile().served(paths) { - Some(server) => key_command::listed(&server).unwrap_or_default(), - None => key_command::local(paths).unwrap_or_default(), - }; - app.own_public = ring - .iter() - .find(|key| key.name == berth_api::profile::BERTH_KEY) - .map(|key| key.public.clone()) - .unwrap_or_default(); - app.ring_keys = ring.into_iter().map(|key| key.name).collect(); - app.load_access(); - app.detect(); - // A host that is installed opens on its maintenance tab. - if existing && !app.text(Field::SshHost).is_empty() { - app.set_step(Step::Manage); - } - Ok(app) -} - -/// Stop with the list of problems when the profile cannot run the action. -fn require_ready(app: &App) -> Result<()> { - // A seed and an installer make a whole workstation. - berth_core::level::guard(&app.host_profile(), berth_core::level::Capability::Apply)?; - let problems = app.problems(); - if problems.is_empty() { - return Ok(()); - } - Err(format!("Not ready:\n {}", problems.join("\n ")).into()) -} - -fn succeed(mut command: Command) -> Result<()> { - let status = command.status()?; - if !status.success() { - let program = command.get_program().to_string_lossy().into_owned(); - return Err(format!("{program} failed ({status})").into()); - } - Ok(()) -} - -/// `berth iso HOST [--write DEVICE]`: build the installer without the UI. -fn build_installer(paths: &Paths, host: &str, device: Option<&str>) -> Result<()> { - let mut app = open(paths, host)?; - app.action = Action::Iso; - if app.text(Field::Iso).is_empty() { - let output = Command::new("python3") - .args([layout::BUILDER, "fetch-iso", "--dir"]) - .arg(&app.iso_dir) - .stderr(std::process::Stdio::inherit()) - .output()?; - if !output.status.success() { - return Err("Downloading the Ubuntu installer failed".into()); - } - app.commit(Field::Iso, String::from_utf8(output.stdout)?.trim()); - } - require_ready(&app)?; - app.execute()?; - let Some(device) = device else { - return Ok(()); - }; - let name = format!("{}-autoinstall.iso", app.text(Field::Hostname)); - print!("Erase {device} and write the installer to it? [y/N] "); - std::io::stdout().flush()?; - let mut answer = String::new(); - std::io::stdin().read_line(&mut answer)?; - if !answer.trim().eq_ignore_ascii_case("y") { - return Err("Not written".into()); - } - let mut write = Command::new("sudo"); - write - .args(["bash", layout::WRITE_USB]) - .arg(app.build_dir.join(name)) - .arg(device); - succeed(write) -} - -/// `berth host HOST ...`: the secrets and registries a host is given. -fn host_command(paths: &Paths, host: &str, args: &[&str]) -> Result<()> { - let mut app = open(paths, host)?; - let list = |value: &serde_json::Value| value.as_array().cloned().unwrap_or_default(); - match *args { - [] => { - for secret in list(&app.config["secrets"]) { - let place = match (secret["env"].as_str(), secret["file"].as_str()) { - (Some(variable), _) => format!("env {variable}"), - (_, Some(path)) => format!("file {path}"), - _ => String::new(), - }; - println!( - "secret {} {place}", - secret["name"].as_str().unwrap_or("") - ); - } - for name in list(&app.config["registries"]) { - println!("registry {}", name.as_str().unwrap_or("")); - } - println!("Send them with: berth run {host} secrets"); - return Ok(()); - } - ["secret", name, ref how @ ..] => { - let mut secrets = list(&app.config["secrets"]); - secrets.retain(|known| known["name"] != name); - match *how { - ["env", variable] => secrets.push(json!({"name": name, "env": variable})), - ["file", path] => secrets.push(json!({"name": name, "file": path})), - ["file", path, mode] => { - secrets.push(json!({"name": name, "file": path, "mode": mode})) - } - ["off"] => {} - _ => { - return Err( - "Usage: berth host HOST secret NAME env VARIABLE | file PATH [MODE] | off" - .into(), - ); - } - } - app.config["secrets"] = json!(secrets); - } - ["registry", name, switch @ ("on" | "off")] => { - let mut registries = list(&app.config["registries"]); - registries.retain(|known| known != name); - if switch == "on" { - if !registry::Store::new(paths)? - .registries()? - .contains_key(name) - { - return Err(format!("No registry {name}. See: berth registry").into()); - } - registries.push(json!(name)); - } - app.config["registries"] = json!(registries); - } - _ => return Err("Usage: berth host HOST [secret NAME ... | registry NAME on|off]".into()), - } - app.save() -} +/// A server on this machine, and what to do once it is no longer needed. +type Local = (String, Box); /// End quietly when what reads berth's output has gone, as `head` does after /// its lines: printing then panics, which is no fault of berth's to report. @@ -526,786 +31,57 @@ fn end_quietly_on_a_closed_pipe() { })); } -/// What to say instead of opening berth's screens where there is no terminal -/// for them, as in a script or over ssh without one. `word` is what was -/// given for a host, which may have been meant as a command. -fn no_terminal(paths: &Paths, word: Option<&str>) -> String { - let known = |word: &str| paths.hosts().iter().any(|(name, _)| name == word); - match word { - Some(word) if !known(word) => format!( - "there is no command `{word}`, and no terminal to set up a host of that name in. `berth help` lists the commands" - ), - _ => "berth's screens need a terminal, and there is none here. `berth help` lists the commands that need none".into(), - } -} - -fn main() { - end_quietly_on_a_closed_pipe(); - if let Err(error) = run() { - eprintln!("berth: {error}"); - std::process::exit(1); - } -} - -fn run() -> Result<()> { - let args: Vec = std::env::args().skip(1).collect(); - let mut args: Vec<&str> = args.iter().map(String::as_str).collect(); - // ssh runs berth as its askpass helper, the program of SSH_ASKPASS, with - // the prompt as its one argument and the variable that says where the - // password is; the answer is all that may go to standard output. - if let Ok(source) = std::env::var(berth_core::keys::askpass::SOURCE) { - let prompt = match args[..] { - ["askpass", ref prompt @ ..] => Some(prompt.join(" ")), - [prompt] => Some(prompt.to_owned()), - _ => None, - }; - if let Some(prompt) = prompt { - let answer = berth_core::keys::askpass::answer(&prompt, &source, None)?; - println!("{answer}"); - return Ok(()); - } - } - // `--server URL [--token TOKEN]` first: this run only, through that berth serve. - while let [flag @ ("--server" | "--token"), value, ..] = args[..] { - let variable = if flag == "--server" { - "BERTH_SERVER" - } else { - "BERTH_TOKEN" - }; - // SAFETY: nothing else runs yet. - unsafe { std::env::set_var(variable, value) }; - args.drain(..2); - } - let paths = Paths::locate()?; - // The first open of the store takes in the files of an older berth and - // says so on standard error, which the screen would hide. A store that - // does not open is reported where it is needed. - if opens_screen(&args) { - let _ = berth_core::store::shared(&paths); - } - if loads_plugins(&args) && berth_core::server::Server::configured(&paths).is_none() { - restore_plugins(&paths); - } - // With a server set, the hosts are the server's: fetch them before anything looks. - let local = [ - "config", - "serve", - "--version", - "-V", - "version", - "openapi", - "asyncapi", - "operations", - "update", - "key", - "askpass", - "tailnet-key", - "credentials", - "connect", - "setup-key", - "store", - ]; - // The screen opens at once on what the last mirror left; the commands - // that print are given current data. - let mut mirroring = None; - if let Some(server) = berth_core::server::Server::configured(&paths) - && !args.first().is_some_and(|verb| local.contains(verb)) - { - if opens_screen(&args) { - // A host the last mirror does not have cannot be opened without one. - if let [name] = args[..] - && !paths.hosts().iter().any(|(known, _)| known == name) - { - server - .mirror(&paths) - .map_err(|error| format!("the berth server at {}: {error}", server.url))?; - } - mirroring = Some(mirror::Mirroring::of(&paths, server)); - } else { - server - .mirror(&paths) - .map_err(|error| format!("the berth server at {}: {error}", server.url))?; - } - } - let host = match args[..] { - ["list"] => { - for (name, address) in paths.hosts() { - println!("{name:<24} {address}"); - } - return Ok(()); - } - ["registry", ref rest @ ..] => return registry::registry_command(&paths, rest), - ["secret", ref rest @ ..] => return registry::secret_command(&paths, rest), - ["image", "push", destination, ref rest @ ..] => { - let image = - registry::push_image(&paths, destination, rest.first().unwrap_or(&"latest"))?; - println!("Pushed {image}"); - return Ok(()); - } - ["publish", destination, ref rest @ ..] => { - std::env::set_current_dir(paths.source()?)?; - let url = registry::publish(&paths, destination)?; - println!("Published {url}"); - let (host, login) = - match *rest { - [] => (None, None), - [host] => (Some(host), None), - [host, "--login"] => (Some(host), destination.split('/').next()), - [host, "--login", registry] => (Some(host), Some(registry)), - _ => return Err( - "Usage: berth publish REGISTRY[/REPOSITORY] [HOST [--login [REGISTRY]]]" - .into(), - ), - }; - if let Some(host) = host { - let mut app = open(&paths, host)?; - app.config["release_url"] = json!(url); - app.config["release_store"] = json!(destination); - app.config["release_private"] = json!(login.is_some()); - match login { - Some(name) => { - if registry::Store::new(&paths)?.curl_login(name)?.is_none() { - return Err(format!("Registry {name} has no stored login").into()); - } - app.config["release_login"] = json!(name); - println!("{host} downloads it from there with the login of {name}"); - } - None => { - if let Some(profile) = app.config.as_object_mut() { - profile.remove("release_login"); - } - println!("{host} downloads it from there without a login"); - } - } - app.save()?; - } - return Ok(()); - } - // A host's own subcommands start with secret or registry. - ["host", "add", name, address, ref rest @ ..] - if !["secret", "registry"].contains(&name) => - { - return commands::add_host(&paths, name, address, rest); - } - ["host", "install", host] => return commands::install_controller(&paths, host), - ["host", "level", host, level] => return commands::set_level(&paths, host, level), - ["host", "rename", host, name] => return commands::rename(&paths, host, name, false), - ["host", "rename", host, name, "--hostname"] - | ["host", "rename", host, "--hostname", name] => { - return commands::rename(&paths, host, name, true); - } - ["host", host, ref rest @ ..] => return host_command(&paths, host, rest), - ["--version" | "-V" | "version"] => { - println!("berth {}", berth_api::VERSION); - return Ok(()); - } - ["update", ref rest @ ..] => return update::command(rest), - ["serve", ref rest @ ..] => return serve::command(&paths, rest), - ["setup-key", ref rest @ ..] => return serve::setup_key::command(&paths, rest), - ["connect", url] => return commands::connect(&paths, url), - ["inbox", ref rest @ ..] => return serve::inbox_command(&paths, rest), - ["kv", ref rest @ ..] => return serve::kv_command(&paths, rest), - ["store", ref rest @ ..] => return store_command::command(&paths, rest), - ["openapi"] => { - println!("{}", serde_json::to_string_pretty(&serve::openapi())?); - return Ok(()); - } - ["operations"] => { - println!("{}", serde_json::to_string_pretty(&serve::operations())?); - return Ok(()); - } - ["asyncapi"] => { - println!("{}", serde_json::to_string_pretty(&serve::asyncapi())?); - return Ok(()); - } - ["plugins"] => { - let listing = plugins::Manager::new(&paths).list()?; - print!("{}", plugins::describe(&listing.plugins)); - return Ok(()); - } - ["plugins", "add", spec] => { - for line in plugins::Manager::new(&paths).add(spec)? { - println!("{line}"); - } - return Ok(()); - } - ["plugins", "update", ref rest @ ..] => return commands::update_plugins(&paths, rest), - ["plugins", "remove", name] => { - let manager = plugins::Manager::new(&paths); - // A name npm knows that berth does not, such as a broken package. - let known = manager - .list()? - .plugins - .iter() - .any(|plugin| plugin.id == *name || plugin.package == *name); - if !known && manager.server().is_none() { - let mut npm = Command::new("npm"); - npm.args([ - "uninstall", - "--ignore-scripts", - "--no-audit", - "--no-fund", - "--prefix", - ]) - .arg(plugins::own_dir(&paths.config_dir)) - .arg(name); - let done = succeed(npm); - if let Some(line) = plugins::remembered(&paths) { - eprintln!("{line}"); - } - return done; - } - for line in manager.remove(name)? { - println!("{line}"); - } - return Ok(()); - } - ["config"] => return show_config(&paths), - ["config", key] => return change_setting(&paths, key, ""), - ["config", key, value] => return change_setting(&paths, key, value), - ["tailnet-key", host] => { - // An apply runs this on the machine that runs berth, for a host - // not on the tailnet yet; the key goes to the role on stdout. - println!("{}", berth_core::tailnet::key_for(&paths, host)?); - return Ok(()); - } - ["profile", host] => return commands::show_profile(&paths, host), - ["profile", host, "--set", file] => return commands::set_profile(&paths, host, file), - ["credentials", host] => return commands::credentials(&paths, host, false), - ["credentials", host, "--show"] | ["credentials", "--show", host] => { - return commands::credentials(&paths, host, true); - } - ["iso", host] => return build_installer(&paths, host, None), - ["iso", host, "--write", device] => return build_installer(&paths, host, Some(device)), - ["key", ref rest @ ..] => return key_command::command(&paths, rest), - ["apply", "--behind" | "--all", ..] => return commands::apply_hosts(&paths, &args[1..]), - ["apply", host] => return commands::apply(&paths, host), - ["remove", host] => return commands::remove(&paths, host), - ["adopt", host, ref rest @ ..] => return commands::adopt(&paths, host, rest), - ["provisioners"] => return commands::provisioners(&paths), - ["provisioner", ref rest @ ..] => return provisioner_command::command(&paths, rest), - ["provision", host, provisioner, ref rest @ ..] => { - return resource_command::provision(&paths, host, provisioner, rest); - } - ["instance", host, ref rest @ ..] => return resource_command::instance(&paths, host, rest), - ["resource", ref rest @ ..] => return resource_command::command(&paths, rest), - ["deploy", name, ref rest @ ..] => return resource_command::deploy(&paths, name, rest), - ["seed", host] if berth_core::server::Server::configured(&paths).is_some() => { - return commands::seed(&paths, host); - } - ["seed", host] => { - let mut app = open(&paths, host)?; - app.action = Action::CloudSeed; - require_ready(&app)?; - return app.execute(); - } - ["tasks", host] => return commands::tasks(&paths, host), - ["run" | "manage", host, task, ref rest @ ..] => { - return commands::run(&paths, host, task, rest); - } - ["exec", host, ref rest @ ..] => return commands::exec(&paths, host, rest), - ["attach", host] => return commands::attach(&paths, host, None), - ["attach", host, id] => return commands::attach(&paths, host, Some(id)), - ["sessions", host] => return commands::sessions(&paths, host), - ["output", host, id, ref rest @ ..] => return commands::output(&paths, host, id, rest), - ["channel", host, ref rest @ ..] => return commands::channel(&paths, host, rest), - ["call", host, service, method] => { - return commands::call(&paths, host, service, method, None); - } - ["call", host, service, method, body] => { - return commands::call(&paths, host, service, method, Some(body)); - } - ["inventory", ref hosts @ ..] => return commands::inventory(&paths, hosts), - ["ssh", host, "--agent"] | ["ssh", "--agent", host] => { - let status = open(&paths, host)?.agent_terminal()?.status()?; - std::process::exit(status.code().unwrap_or(1)); - } - ["ssh", host, ref remote @ ..] => { - let status = open(&paths, host)?.terminal(remote)?.status()?; - std::process::exit(status.code().unwrap_or(1)); - } - [] => None, - [host] if !host.starts_with('-') && host != "help" => Some(host.to_owned()), - _ => { - println!("{USAGE}"); - return Ok(()); - } +/// `berth ui [HOST]`: the screens of a server: the one this berth is set to +/// use, a `berth serve` of this user on this machine, or else one in this +/// process for as long as they are open, which waits for the jobs it runs +/// before berth ends. +fn screens(global: &cli::args::Global, host: Option<&str>) -> cli::fail::Result<()> { + use berth_page::settings::{self, Settings}; + use cli::fail::Fail; + use std::io::IsTerminal; + if !std::io::stdin().is_terminal() || !std::io::stdout().is_terminal() { + return Err(Fail::usage( + "berth's screens need a terminal, and there is none here. berth help lists the commands that need none", + )); + } + let var = |name: &str| std::env::var(name).ok(); + let settings = Settings::new(settings::folder(var)); + cli::client::take_in_remote(&settings, settings::old_folder(var).as_deref()); + let mut linked = berth_page::device::Linked { + server: global + .server + .clone() + .or_else(|| var("BERTH_SERVER").filter(|server| !server.trim().is_empty())), + token: var("BERTH_TOKEN").filter(|token| !token.trim().is_empty()), + setup_key: None, }; - { - use std::io::IsTerminal; - if !std::io::stdin().is_terminal() || !std::io::stdout().is_terminal() { - return Err(no_terminal(&paths, host.as_deref()).into()); - } + if let Some(file) = &global.token_file { + linked.token = Some(std::fs::read_to_string(file)?.trim().to_owned()); } - // Find the source before the screen is taken over, so the error is readable. - paths.source()?; - if let Some(running) = &mut mirroring { - running.start(); + let mut finish: Box = Box::new(|| {}); + // Without a server here, the screens ask for one and its token. + if linked.local(&settings) + && let Some((server, then)) = local::server(true)? + { + linked.server = Some(server); + finish = then; } - let mut terminal = enter()?; - let outcome = (|| -> Result<()> { - loop { - let name = match &host { - Some(host) => host.clone(), - None => match ui::home(&mut terminal, &paths, &mut mirroring)? { - Some(name) => name, - None => return Ok(()), - }, - }; - let mut app = open(&paths, &name)?; - app.start(); - // Terminal actions (Ansible) suspend the UI and come back to it. - while ui::run(&mut terminal, &mut app, &mut mirroring)? { - leave(); - match app.execute() { - Ok(()) => println!("\nDone."), - Err(error) => println!("\n{error}"), - } - println!("Press Enter to return to berth."); - let _ = std::io::stdin().read_line(&mut String::new()); - terminal = enter()?; - // A host renamed meanwhile is opened again under its new name. - if let Some(new) = app.renamed.take() { - app = open(&paths, &new)?; - app.start(); - } - } - if host.is_some() { - return Ok(()); - } - } - })(); - leave(); - outcome + berth_page::device::install(linked, settings); + let shown = berth_page::screens::run(host.map(str::to_owned)); + finish(); + Ok(shown?) } -#[cfg(test)] -mod tests { - use super::opens_screen; - - #[test] - fn only_the_screen_opens_before_the_mirror_is_made() { - assert!(opens_screen(&[]) && opens_screen(&["studio"])); - assert!(!opens_screen(&["list"]) && !opens_screen(&["--version"])); - assert!(!opens_screen(&["run", "studio", "task"])); - } - - use super::*; - use crate::fields::{Action, Field, Step}; - use crossterm::event::{Event, KeyCode, KeyEvent, KeyModifiers}; - use tui_input::{Input, backend::crossterm::EventHandler}; - - /// Paths of a folder that is never made, for an app that saves nothing. - fn nowhere() -> Paths { - Paths { - config_dir: "/nonexistent/berth-test/config".into(), - data_dir: "/nonexistent/berth-test/data".into(), - } - } - - /// An app of a new profile that is never saved, in a file whose name is - /// no host name, so that the profile needs a hostname of its own. - fn app(config: &str) -> App { - App::new( - nowhere(), - berth_core::profiles::Place::File("/nonexistent/berth-test/new profile.json".into()), - parse_config(config.as_bytes()).unwrap(), - ) - } - - fn selectable(app: &App) -> Vec { - app.rows() - .iter() - .enumerate() - .filter_map(|(position, row)| row.entry.map(|_| position)) - .collect() - } - - fn demo_plugin() -> plugins::Plugin { - let feature = |id: &str, group: &str, action: &str| plugins::Feature { - id: id.into(), - group: group.into(), - label: id.into(), - help: "", - actions: vec![plugins::PluginAction { - name: action.into(), - label: "Say hello".into(), - help: "", - run: "echo 'hi'".into(), - user: "agent".into(), - terminal: false, - group: String::new(), - extends: vec![], - agent: false, - arguments: vec![], - confirm: None, - }], - services: vec![], - channels: vec![], - }; - plugins::Plugin { - id: "demo".into(), - label: "Demo".into(), - package: "berth-plugin-demo".into(), - version: "1.0.0".into(), - description: String::new(), - dir: "/plugins/demo".into(), - found: "/plugins/demo".into(), - features: vec![ - feature("one", "development tools", "hello"), - feature("two", "Games", "other"), - ], - provisioners: vec![], - } - } - - #[test] - fn plugin_features_join_groups() { - use app::{Entry, Item}; - let mut app = app(r#"{"access":{"address":"box"},"builder":{"shipped_plugins":true}}"#); - app.set_plugins(vec![demo_plugin()]); - app.step = Step::Features; - let groups = app.feature_groups(); - // One feature joins a built-in group, the other brings its own. The - // groups of the coding agents are not shown without their plugins. - assert_eq!(groups[0].title, "Development Tools"); - assert_eq!(groups[0].items.last(), Some(&Item::Feature(0))); - assert_eq!(groups.last().unwrap().title, "Games"); - assert_eq!(groups.last().unwrap().items, [Item::Feature(1)]); - - // Chosen out of order, stored in the plugin's order. - app.set_item(Item::Feature(1), true); - app.set_item(Item::Feature(0), true); - assert_eq!( - app.config["ansible_vars"]["devbox_plugin_features"], - json!(["demo/one", "demo/two"]) - ); - // What the features can do on the host is the host's to say: the - // overview has no rows for them until it has been asked. - app.set_step(Step::Manage); - assert!( - !app.entries() - .iter() - .any(|entry| matches!(entry, Entry::HostTask(_))) - ); - app.set_item(Item::Feature(0), false); - assert_eq!( - app.config["ansible_vars"]["devbox_plugin_features"], - json!(["demo/two"]) - ); - } - - #[test] - fn old_agent_switches_become_plugin_features() { - // Everything was on unless switched off; T3 followed the agents. - let features = |config: &str| app(config).config["ansible_vars"].clone(); - assert_eq!( - features("{}")["devbox_plugin_features"], - json!([ - "claude/cli", - "codex/cli", - "github/cli", - "pi/cli", - "t3/server" - ]) - ); - let old = features( - r#"{"ansible_vars": {"devbox_agent_codex": false, "devbox_install_t3": false, - "devbox_plugin_features": ["unity/cli"], "devbox_tools_web": true}}"#, - ); - assert_eq!( - old, - json!({"devbox_tools_web": true, - "devbox_plugin_features": ["claude/cli", "github/cli", "pi/cli", "unity/cli"]}) - ); - let none = features(r#"{"ansible_vars": {"devbox_install_agents": false}}"#); - assert_eq!(none["devbox_plugin_features"], json!(["github/cli"])); - // Done once: what is switched off afterwards stays off. - let kept = features( - r#"{"builder": {"shipped_plugins": true}, "ansible_vars": {"devbox_plugin_features": []}}"#, - ); - assert_eq!(kept["devbox_plugin_features"], json!([])); - } - - #[test] - fn groups_are_selectable_and_switch_everything_in_them() { - use app::Entry; - let mut app = app(r#"{"builder":{"shipped_plugins":true}}"#); - app.set_plugins(vec![demo_plugin()]); - app.step = Step::Features; - let rows = app.rows(); - assert!( - rows[0] - .line - .to_string() - .starts_with("Development Tools 6 of 7") - ); - assert_eq!(rows[0].entry, Some(Entry::Group(0))); - assert!(rows[7].line.to_string().starts_with(" └─ one")); - - // Not everything is on, so the group is switched on first, then off. - assert!(!app.group_all_on(0)); - app.toggle_group(0); - assert!(app.group_all_on(0)); - assert_eq!( - app.config["ansible_vars"]["devbox_plugin_features"], - json!(["demo/one"]) - ); - app.toggle_group(0); - assert!(!app.enabled(Field::ToolsWeb) && !app.enabled(Field::ToolsScientific)); - assert_eq!( - app.config["ansible_vars"]["devbox_plugin_features"], - json!([]) - ); - - // The accounts on the Settings tab start with a heading. - app.step = Step::Settings; - assert_eq!(selectable(&app)[0], 1); - app.action = Action::Docker; - assert_eq!(app.fields(), [Field::AgentUser]); - } - - #[test] - fn settings_follow_the_action_and_disk_mode() { - let mut app = app("{}"); - app.step = Step::Install; - app.action = Action::Docker; - assert!(app.fields().is_empty()); - // What an installer picks the disk and network with has defaults. - app.action = Action::UsbSeed; - assert!(!app.fields().contains(&Field::DiskMode)); - app.step = Step::Settings; - assert!(app.fields().contains(&Field::DiskType)); - assert!(!app.fields().contains(&Field::DiskSerial)); - assert!(!app.fields().contains(&Field::WifiSsid)); - app.commit(Field::DiskMode, "serial"); - app.activate(Field::Wifi); - let fields = app.fields(); - assert!(fields.contains(&Field::DiskSerial) && !fields.contains(&Field::DiskType)); - assert!(fields.contains(&Field::WifiSsid)); - app.step = Step::Install; - app.action = Action::Ssh; - assert!(app.fields().contains(&Field::SshHost)); - assert!(!app.fields().contains(&Field::Hostname)); - } - - #[test] - fn new_profiles_default_to_auto_with_wifi_off() { - let config = parse_config(br#"{}"#).unwrap(); - assert_eq!(config["target_disk_mode"], "auto"); - assert_eq!(config["wifi_enabled"], false); - assert!(config.get("operator_passwordless_sudo").is_none()); - assert!(config["access"]["sudo"].is_null()); - // The switches of a profile from before are dropped, whatever they said. - let old = parse_config(br#"{"operator_passwordless_sudo": false, "berth_ssh_key": false}"#) - .unwrap(); - assert!( - old.get("operator_passwordless_sudo").is_none() && old.get("berth_ssh_key").is_none() - ); - let legacy = parse_config( - br#"{"target_disk_serial":"OLD_SSD","wifi":{"ssid":"saved"}, - "release_url":"https://REPLACE.example/x"}"#, - ) - .unwrap(); - assert_eq!(legacy["target_disk_mode"], "serial"); - assert_eq!(legacy["wifi_enabled"], true); - assert_eq!(legacy["release_url"], ""); - // A profile of the qemu kind there once was shows as cloud, reached from the QEMU host. - let qemu = parse_config(br#"{"profile": "qemu"}"#).unwrap(); - assert_eq!(qemu["profile"], "cloud"); - assert_eq!( - qemu["ansible_vars"]["devbox_bootstrap_ssh_cidr"], - "10.0.2.2/32" - ); - let app = app(r#"{"profile": "qemu"}"#); - assert_eq!(app.text(Field::RecoveryCidr), "10.0.2.2/32"); - } - - #[test] - fn asking_for_the_sudo_password_is_off_until_chosen() { - let directory = tempfile::tempdir().unwrap(); - let path = directory.path().join("studio.json"); - let profile = r#"{"access":{"address":"192.0.2.10","user":"ops"}}"#; - let paths = Paths { - config_dir: directory.path().join("config"), - data_dir: directory.path().join("data"), - }; - let mut app = App::new( - paths, - berth_core::profiles::Place::File(path.clone()), - parse_config(profile.as_bytes()).unwrap(), - ); - // An edit elsewhere leaves it unset. - app.commit(Field::Hostname, "studio"); - app.save().unwrap(); - let host = |path: &std::path::Path| berth_core::host::Host { - name: "studio".into(), - place: berth_core::profiles::Place::File(path.to_owned()), - config: serde_json::from_slice(&fs::read(path).unwrap()).unwrap(), - }; - assert!(!app.enabled(Field::AskBecomePass)); - assert!(!host(&path).asks_password()); - assert!(app.config["builder"]["ask_become_pass"].is_null()); - // A first apply as an account whose sudo asks for a password. - app.activate(Field::AskBecomePass); - app.save().unwrap(); - assert!(app.enabled(Field::AskBecomePass) && host(&path).asks_password()); - // Once chosen, the choice stays. - app.activate(Field::AskBecomePass); - app.save().unwrap(); - assert!(!app.enabled(Field::AskBecomePass) && !host(&path).asks_password()); - assert_eq!(app.config["access"]["sudo"], "none"); - } - - #[test] - fn edits_land_in_the_profile() { - let mut app = app(r#"{"ansible_vars":{"devbox_install_dev_tools":false}}"#); - app.commit(Field::WifiSsid, " home "); - assert_eq!(app.config["wifi"]["ssid"], "home"); - app.commit(Field::WifiSsid, ""); - assert!(app.config["wifi"].is_null()); - - app.commit(Field::Geometry, "1920x1080"); - assert_eq!(app.config["ansible_vars"]["devbox_geometry"], "1920x1080"); - app.commit(Field::Geometry, ""); - assert!(app.config["ansible_vars"].get("devbox_geometry").is_none()); - - // Tool categories inherit the older single switch until set themselves. - assert!(!app.enabled(Field::ToolsJvm)); - assert!(app.enabled(Field::Docker)); - app.activate(Field::ToolsJvm); - assert_eq!(app.config["ansible_vars"]["devbox_tools_jvm"], true); - - app.activate(Field::Profile); - assert_eq!(app.text(Field::Profile), "cloud"); - app.commit(Field::Profile, "desktop"); - assert_eq!(app.text(Field::Profile), "cloud"); - assert!(app.unsaved()); - } - - #[test] - fn review_reports_what_would_fail() { - let mut app = app("{}"); - app.action = Action::Save; - assert!(app.problems().is_empty()); - app.action = Action::UsbSeed; - let problems = app.problems().join("\n"); - assert!(problems.contains("Hostname")); - assert!(problems.contains("Operator needs an SSH key")); - assert!(problems.contains("Operator password is required")); - // The key of this berth is one, which an older profile cannot turn down. - app.own_public = "ssh-ed25519 AAAA berth@here".into(); - assert!(!app.problems().join("\n").contains("SSH key")); - app.config["berth_ssh_key"] = json!(false); - assert!(!app.problems().join("\n").contains("SSH key")); - app.config.as_object_mut().unwrap().remove("berth_ssh_key"); - app.own_public.clear(); - - app.config["hostname"] = json!("devbox-1"); - app.config["operator_ssh_public_key"] = json!("ssh-ed25519 AAAA me"); - app.config["operator_password_hash"] = json!(format!("$6$salt${}", "a".repeat(86))); - assert_eq!(app.problems(), Vec::::new()); - - // An agent without a privilege level types a password for sudo, so it needs one. - app.commit(Field::AgentPrivileges, "limited"); - app.activate(Field::AgentSudo); - assert_eq!(app.problems(), Vec::::new()); - app.config["agent_privileges"] = json!(null); - assert_eq!(app.problems().len(), 1); - app.activate(Field::AgentSudo); - app.commit(Field::AgentUser, "ops"); - assert_eq!(app.problems().len(), 1); - // Ubuntu already has a system group with this name, so the account cannot be created. - app.commit(Field::AgentUser, "operator"); - assert_eq!(app.problems().len(), 1); - app.commit(Field::AgentUser, "agent"); - - // Only the USB installer insists on an operator password. - let hash = app.config["operator_password_hash"].take(); - assert_eq!(app.problems(), ["Operator password is required (F4 or F5)"]); - app.action = Action::CloudSeed; - assert!(!app.problems().join("\n").contains("password")); - app.action = Action::UsbSeed; - app.config["operator_password_hash"] = hash; - - // A cloud seed needs somewhere to get the release from, but no SSH network. - app.action = Action::CloudSeed; - assert_eq!(app.problems().len(), 1); - let step = std::mem::replace(&mut app.step, Step::Settings); - assert!(app.fields().contains(&Field::ReleaseUrl)); - app.config["release_store"] = json!("nexus/releases"); - assert!(app.problems().is_empty()); - assert!(app.fields().contains(&Field::ReleasePrivate)); - assert!(!app.fields().contains(&Field::ReleaseUrl)); - app.step = step; - app.action = Action::Ssh; - assert_eq!(app.problems().len(), 1); - app.commit(Field::SshPort, "22"); - app.config["access"]["address"] = json!("devbox.example"); - assert!(app.problems().is_empty()); - } - - #[test] - fn choosing_an_iso_records_its_checksum() { - let directory = tempfile::tempdir().unwrap(); - let iso = directory.path().join("ubuntu.iso"); - fs::write(&iso, "abc").unwrap(); - let mut app = app("{}"); - app.commit(Field::Iso, iso.to_str().unwrap()); - assert_eq!( - app.text(Field::IsoSha), - "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad" - ); - } - - #[test] - fn wifi_logins_come_from_and_are_checked_against_this_machine() { - let mut app = app("{}"); - app.wifi_networks = vec![("home".into(), "secret".into()), ("cafe".into(), "".into())]; - app.commit(Field::WifiSsid, "cafe"); - assert_eq!(app.text(Field::WifiPassword), ""); - assert_eq!(app.wifi_password_matches(), None); - app.commit(Field::WifiSsid, "home"); - assert_eq!(app.text(Field::WifiPassword), "secret"); - assert_eq!(app.wifi_password_matches(), Some(true)); - app.commit(Field::WifiPassword, "typo"); - assert_eq!(app.wifi_password_matches(), Some(false)); - assert!(app.status.contains("Differs")); - } - - #[test] - fn input_widget_inserts_and_deletes_at_cursor() { - let mut input = Input::new("abde".into()); - for code in [ - KeyCode::Home, - KeyCode::Right, - KeyCode::Right, - KeyCode::Char('c'), - KeyCode::Delete, - KeyCode::End, - KeyCode::Backspace, - ] { - input.handle_event(&Event::Key(KeyEvent::new(code, KeyModifiers::NONE))); +fn main() { + end_quietly_on_a_closed_pipe(); + let args: Vec = std::env::args().collect(); + match local::askpass(&args) { + Ok(true) => return, + Ok(false) => {} + Err(error) => { + eprintln!("berth: {error}"); + std::process::exit(1); } - assert_eq!(input.value(), "abc"); - assert_eq!(input.cursor(), 3); - } - - #[test] - fn discovers_custom_public_key_names_and_config_includes() { - let directory = tempfile::tempdir().unwrap(); - let write = |name: &str, content: &str| { - fs::write(directory.path().join(name), content).unwrap(); - }; - write("storage.pub", "ssh-ed25519 AAAATEST storage\n"); - write("rsa.pub", "ssh-rsa AAAATEST laptop\n"); - write("invalid.pub", "not a key"); - write("config", "Include config.d/*\nHost main * !excluded\n"); - fs::create_dir(directory.path().join("config.d")).unwrap(); - write("config.d/cloud", "Host cloud-server\n User operator\n"); - let (public, _) = discovery::keys(directory.path()); - assert_eq!(public.len(), 2); - assert!(public.iter().any(|(_, key)| key.ends_with("storage"))); - assert_eq!(discovery::hosts(&directory.path().join("config")).len(), 2); } + std::process::exit(cli::main(args)); } diff --git a/crates/berth/src/mirror.rs b/crates/berth/src/mirror.rs deleted file mode 100644 index 2e49772..0000000 --- a/crates/berth/src/mirror.rs +++ /dev/null @@ -1,102 +0,0 @@ -//! The mirror of a berth server, made while the screen is open. The server -//! may wait many seconds for a host that does not answer, and the screen -//! shows what the last mirror left until the new one is there. - -use std::sync::{ - Arc, - mpsc::{Receiver, TryRecvError, channel}, -}; - -/// What making a mirror comes to: nothing to say, or why it failed. -pub type Made = Result<(), String>; - -/// A mirror being made on a thread of its own, at most one at a time. -pub struct Mirroring { - make: Arc Made + Send + Sync>, - running: Option>, -} - -impl Mirroring { - /// `make` is the work of one mirror. - pub fn new(make: impl Fn() -> Made + Send + Sync + 'static) -> Mirroring { - Mirroring { - make: Arc::new(make), - running: None, - } - } - - /// The mirror of the berth server this berth uses. - pub fn of(paths: &berth_core::paths::Paths, server: berth_core::server::Server) -> Mirroring { - let paths = paths.clone(); - Mirroring::new(move || { - server - .mirror(&paths) - .map_err(|error| format!("the berth server at {}: {error}", server.url)) - }) - } - - /// Start a mirror, unless one is under way. Returns whether it started. - pub fn start(&mut self) -> bool { - if self.running.is_some() { - return false; - } - let (sender, receiver) = channel(); - let make = self.make.clone(); - std::thread::spawn(move || { - let _ = sender.send(make()); - }); - self.running = Some(receiver); - true - } - - pub fn running(&self) -> bool { - self.running.is_some() - } - - /// How the mirror ended, once, when it has. - pub fn finished(&mut self) -> Option { - let receiver = self.running.as_ref()?; - let made = match receiver.try_recv() { - Ok(made) => made, - Err(TryRecvError::Empty) => return None, - Err(TryRecvError::Disconnected) => Err("the mirror stopped without an answer".into()), - }; - self.running = None; - Some(made) - } -} - -#[cfg(test)] -mod tests { - use super::*; - use std::sync::atomic::{AtomicUsize, Ordering}; - - #[test] - fn one_mirror_runs_at_a_time_and_its_end_is_told_once() { - let (release, wait) = channel::<()>(); - let wait = std::sync::Mutex::new(wait); - let made = Arc::new(AtomicUsize::new(0)); - let count = made.clone(); - let mut mirroring = Mirroring::new(move || { - count.fetch_add(1, Ordering::SeqCst); - wait.lock() - .unwrap() - .recv() - .map_err(|error| error.to_string()) - }); - assert!(mirroring.start() && mirroring.running()); - // Asking again while it runs starts nothing, and nothing has ended. - assert!(!mirroring.start()); - assert!(mirroring.finished().is_none()); - release.send(()).unwrap(); - let ended = loop { - if let Some(ended) = mirroring.finished() { - break ended; - } - std::thread::yield_now(); - }; - assert_eq!(ended, Ok(())); - assert!(mirroring.finished().is_none() && !mirroring.running()); - assert_eq!(made.load(Ordering::SeqCst), 1); - } -} diff --git a/crates/berth/src/picker.rs b/crates/berth/src/picker.rs deleted file mode 100644 index 5313ef5..0000000 --- a/crates/berth/src/picker.rs +++ /dev/null @@ -1,109 +0,0 @@ -use crate::{Result, discovery::expand_path, theme}; -use ratatui::{ - layout::Rect, - style::Style, - widgets::{Clear, List, ListState}, -}; -use std::{fs, path::PathBuf}; - -pub struct Picker { - pub directory: PathBuf, - pub entries: Vec, - pub state: ListState, - pub area: Rect, -} - -impl Picker { - pub fn new(value: &str) -> Result { - let path = expand_path(value); - let directory = if path.is_dir() { - path - } else { - path.parent() - .filter(|p| p.is_dir()) - .unwrap_or(std::path::Path::new(".")) - .to_owned() - }; - let mut picker = Self { - directory, - entries: vec![], - state: ListState::default(), - area: Rect::default(), - }; - picker.reload()?; - Ok(picker) - } - fn reload(&mut self) -> Result<()> { - self.directory = self.directory.canonicalize()?; - let mut entries = fs::read_dir(&self.directory)? - .filter_map(|entry| entry.ok().map(|e| e.path())) - .collect::>(); - entries.sort_by_key(|path| (!path.is_dir(), path.file_name().map(|n| n.to_owned()))); - self.entries = self - .directory - .parent() - .map(|p| vec![p.to_owned()]) - .unwrap_or_default(); - self.entries.extend(entries); - self.state = ListState::default().with_selected(Some(0)); - Ok(()) - } - pub fn choose(&mut self) -> Result> { - let Some(path) = self - .entries - .get(self.state.selected().unwrap_or(0)) - .cloned() - else { - return Ok(None); - }; - if path.is_dir() { - self.directory = path; - self.reload()?; - Ok(None) - } else { - Ok(Some(path.display().to_string())) - } - } - pub fn move_by(&mut self, up: bool) { - let index = self.state.selected().unwrap_or(0); - self.state.select(Some(if up { - index.saturating_sub(1) - } else { - (index + 1).min(self.entries.len().saturating_sub(1)) - })); - } - pub fn draw(&mut self, frame: &mut ratatui::Frame) { - self.area = frame.area().inner(ratatui::layout::Margin { - horizontal: 4, - vertical: 3, - }); - frame.render_widget(Clear, self.area); - let entries = self - .entries - .iter() - .map(|path| { - let name = if path == self.directory.parent().unwrap_or(&self.directory) { - "..".into() - } else { - path.file_name() - .unwrap_or_default() - .to_string_lossy() - .into_owned() - }; - format!("{} {name}", if path.is_dir() { "▸" } else { " " }) - }) - .collect::>(); - frame.render_stateful_widget( - List::new(entries) - .block( - theme::popup(&self.directory.display().to_string(), theme::ACCENT) - .title_bottom(theme::hints( - "Enter open/select · Ctrl+Enter use directory · Esc cancel", - )), - ) - .highlight_style(Style::new().bg(theme::RAISED)), - self.area, - &mut self.state, - ); - } -} diff --git a/crates/berth/src/provisioner_command.rs b/crates/berth/src/provisioner_command.rs deleted file mode 100644 index 840d6af..0000000 --- a/crates/berth/src/provisioner_command.rs +++ /dev/null @@ -1,864 +0,0 @@ -//! `berth provisioner …`: a provisioner as a subject of its own, its page and -//! its actions. With a berth server set the provisioners are the server's, -//! and their actions run there; otherwise here, with this berth's secrets. -//! The Provisioners tab of the terminal screen goes the same way. - -use crate::Result; -use crate::commands::{follow_job, parse_arguments, ready}; -use berth_api::{ - self as api, Act, Action, NewResource, ProvisionerInfo, ProvisionerState, Resource, - ResourceSecret, SavedProvisioner, -}; -use berth_core::{ - paths::Paths, - provision, provisioner, registry, resource, - runner::{self, Request, Subject, Work}, - server::Server, -}; -pub use berth_tui::provisioners::answer_lines; -use serde_json::{Value, json}; -use std::collections::BTreeMap; -use std::io::Write; - -const USAGE: &str = "Usage: berth provisioner [types | add NAME TYPE [SETTING=VALUE…] | remove NAME | NAME [SETTING=VALUE…] | NAME actions | NAME run ACTION [ARGUMENT=VALUE…] [--yes]]"; - -/// Where the provisioners are: here, or on the berth server berth uses. -pub enum Source { - Here(Paths), - Server(Server), -} - -/// What running an action came to. -pub enum Ran { - /// It only read, and answered with a message and rows. - Answered(Value), - /// It ran as a job to its end, with this exit code. - Ended(i32), -} - -impl Source { - pub fn of(paths: &Paths) -> Source { - match Server::configured(paths) { - Some(server) => Source::Server(server), - None => Source::Here(paths.clone()), - } - } - - /// Where they are, for people. - pub fn place(&self) -> String { - match self { - Source::Here(_) => "this machine".into(), - Source::Server(server) => format!("the berth server at {}", server.url), - } - } - - pub fn list(&self) -> Result> { - match self { - Source::Here(paths) => Ok(provision::list(paths, ®istry::Store::new(paths)?)), - Source::Server(server) => Ok(serde_json::from_value(server.call( - "provisioners.list", - Value::Null, - None, - )?)?), - } - } - - /// The types the plugins bring, which a provisioner is registered as. - pub fn types(&self) -> Result> { - match self { - Source::Here(paths) => Ok(provision::types(paths)), - Source::Server(server) => Ok(serde_json::from_value(server.call( - "provisioners.types", - Value::Null, - None, - )?)?), - } - } - - /// Register a provisioner, or change the settings of one that is. - pub fn save(&self, name: &str, saved: &SavedProvisioner) -> Result<()> { - match self { - Source::Here(paths) => provisioner::register(paths, name, saved).map(|_| ()), - Source::Server(server) => server - .call( - "provisioners.save", - json!({"name": name}), - Some(serde_json::to_value(saved)?), - ) - .map(|_| ()), - } - } - - /// Forget a registered provisioner. - pub fn remove(&self, name: &str) -> Result<()> { - match self { - Source::Here(paths) => { - provisioner::remove_entry(paths, ®istry::Store::new(paths)?, name) - } - Source::Server(server) => server - .call("provisioners.remove", json!({"name": name}), None) - .map(|_| ()), - } - } - - /// Its page; `given` are values of its own arguments for this once. - pub fn state(&self, name: &str, given: &BTreeMap) -> Result { - match self { - Source::Here(paths) => provisioner::state( - paths, - ®istry::Store::new(paths)?, - name, - given, - &Default::default(), - ), - Source::Server(server) => { - let mut asked = json!(given); - asked["name"] = json!(name); - Ok(serde_json::from_value(server.call( - "provisioners.get", - asked, - None, - )?)?) - } - } - } - - pub fn actions(&self, name: &str) -> Result> { - match self { - Source::Here(paths) => provisioner::actions(paths, ®istry::Store::new(paths)?, name), - Source::Server(server) => Ok(serde_json::from_value(server.call( - "provisioners.actions", - json!({"name": name}), - None, - )?)?), - } - } - - /// Run an action of `name`. A job's output goes to `out` as it comes; a - /// provisioner's progress for one that only reads goes to `progress`. - pub fn run( - &self, - name: &str, - action: &str, - act: Act, - out: &mut dyn Write, - progress: &mut dyn Write, - ) -> Result { - match self { - Source::Here(paths) => { - let store = registry::Store::new(paths)?; - let acting = provisioner::prepare(paths, &store, name, action, &act.arguments)?; - let options = provision::Options::default(); - if !acting.changes() { - let answered = acting.run(paths, &options, progress)?; - return Ok(Ran::Answered(answered.value())); - } - let mut request = Request::new(Subject::Provisioner(name.to_owned()), action); - request.secrets = acting.secrets(); - let held = paths.clone(); - let work = Work::Function(Box::new(move |out| { - match acting.run(&held, &options, out) { - Ok(answered) => { - for line in answered.lines() { - let _ = writeln!(out, "{line}"); - } - true - } - Err(error) => { - let _ = writeln!(out, "✗ {error}"); - false - } - } - })); - let code = runner::here(paths, request, work, out)?; - Ok(Ran::Ended(code.unwrap_or(1))) - } - Source::Server(server) => { - let acted = server.call( - "provisioners.act", - json!({"name": name, "action": action}), - Some(serde_json::to_value(&act)?), - )?; - let acted: api::Acted = serde_json::from_value(acted)?; - let Some(job) = acted.job else { - return Ok(Ran::Answered(acted.result.unwrap_or(Value::Null))); - }; - let _ = writeln!( - progress, - "Job {} on the berth server at {}", - job.id, server.url - ); - let code = berth_core::link::runtime()?.block_on(async { - let portal = server.portal().await?; - follow_job(&portal, &job.id, out).await - })?; - Ok(Ran::Ended(code)) - } - } - } -} - -/// What an action of a resource is given: its arguments, and for `change` -/// a new specification and other secrets. -#[derive(Debug, Clone, Default)] -pub struct Change { - pub arguments: BTreeMap, - pub spec: Option, - pub secrets: Option>, -} - -/// The text of an answer that is the bytes of a page, as the portal gives them. -fn text_of(answer: &Value) -> String { - match answer { - Value::String(text) => text.clone(), - other => other["base64"] - .as_str() - .and_then(api::unb64) - .map(|bytes| String::from_utf8_lossy(&bytes).into_owned()) - .unwrap_or_default(), - } -} - -impl Source { - /// The resources kept, narrowed by provisioner, kind and host. - pub fn resources( - &self, - provisioner: Option<&str>, - kind: Option<&str>, - host: Option<&str>, - ) -> Result> { - match self { - Source::Here(paths) => resource::listed(paths, provisioner, kind, host), - Source::Server(server) => { - let mut asked = json!({}); - for (key, value) in [("provisioner", provisioner), ("kind", kind), ("host", host)] { - if let Some(value) = value { - asked[key] = json!(value); - } - } - Ok(serde_json::from_value(server.call( - "resources.list", - asked, - None, - )?)?) - } - } - } - - /// The id of a resource given by its id or by a name only it has. - pub fn resolve(&self, given: &str) -> Result { - if resource::valid_id(given) { - return Ok(given.to_owned()); - } - let named: Vec = self - .resources(None, None, None)? - .into_iter() - .filter(|known| known.name == given) - .collect(); - match named.as_slice() { - [one] => Ok(one.id.clone()), - [] => Err(format!("No resource {given}. See: berth resource").into()), - _ => Err(format!( - "Several resources are called {given}: name one by its id ({})", - named - .iter() - .map(|known| known.id.as_str()) - .collect::>() - .join(", ") - ) - .into()), - } - } - - /// One resource, as its provisioner says it is now. - pub fn resource(&self, id: &str) -> Result { - match self { - Source::Here(paths) => resource::status( - paths, - ®istry::Store::new(paths)?, - id, - &Default::default(), - ), - Source::Server(server) => Ok(serde_json::from_value(server.call( - "resources.get", - json!({"id": id}), - None, - )?)?), - } - } - - pub fn resource_actions(&self, id: &str) -> Result> { - match self { - Source::Here(paths) => { - let kept = resource::find(paths, id)?; - Ok(resource::actions( - paths, - ®istry::Store::new(paths)?, - &kept, - )) - } - Source::Server(server) => Ok(serde_json::from_value(server.call( - "resources.actions", - json!({"id": id}), - None, - )?)?), - } - } - - /// Follow a job of the berth server to its end, its output to `out`. - fn follow( - server: &Server, - job: &str, - out: &mut dyn Write, - progress: &mut dyn Write, - ) -> Result { - let _ = writeln!(progress, "Job {job} on the berth server at {}", server.url); - berth_core::link::runtime()?.block_on(async { - let portal = server.portal().await?; - follow_job(&portal, job, out).await - }) - } - - /// Create a resource: its record, and how its making ended. - pub fn create( - &self, - asked: &NewResource, - out: &mut dyn Write, - progress: &mut dyn Write, - ) -> Result<(Resource, i32)> { - match self { - Source::Here(paths) => { - let store = registry::Store::new(paths)?; - let mut creating = match resource::prepare(paths, &store, asked)? { - resource::Asked::Again(kept) => { - let plugins = provision::list_plugins(paths); - let _ = writeln!(progress, "It was asked for with this token before"); - return Ok((resource::info(paths, &kept, &plugins, false), 0)); - } - resource::Asked::New(creating) => creating, - }; - let kept = creating.kept().clone(); - let mut request = Request::new(kept.subject(), "create"); - request.held = creating.take_lock(); - request.secrets = creating.secrets(); - let held = paths.clone(); - let work = Work::Function(Box::new(move |out| { - let done = creating.run(&held, &store, &Default::default(), out); - if let Err(error) = &done { - let _ = writeln!(out, "✗ {error}"); - } - done.is_ok() - })); - let code = runner::here(paths, request, work, out)?.unwrap_or(1); - let plugins = provision::list_plugins(paths); - let kept = resource::find(paths, kept.id()).unwrap_or(kept); - Ok((resource::info(paths, &kept, &plugins, false), code)) - } - Source::Server(server) => { - let created: api::Created = serde_json::from_value(server.call( - "resources.create", - Value::Null, - Some(serde_json::to_value(asked)?), - )?)?; - let code = Source::follow(server, &created.job.id, out, progress)?; - Ok((created.resource, code)) - } - } - } - - /// Run an action of a resource. A job's output goes to `out` as it - /// comes; the provisioner's progress for one that only reads to `progress`. - pub fn resource_run( - &self, - id: &str, - action: &str, - given: Change, - out: &mut dyn Write, - progress: &mut dyn Write, - ) -> Result { - match self { - Source::Here(paths) => { - let store = registry::Store::new(paths)?; - let given = resource::Given { - arguments: given.arguments, - spec: given.spec, - secrets: given.secrets, - }; - let mut acting = resource::prepare_action(paths, &store, id, action, &given)?; - let options = provision::Options::default(); - if !acting.changes() { - let answered = acting.run(paths, &options, progress)?; - return Ok(Ran::Answered(answered.value())); - } - let mut request = Request::new(acting.kept().subject(), action); - request.held = acting.take_lock(); - request.secrets = acting.secrets(); - let held = paths.clone(); - let work = Work::Function(Box::new(move |out| { - match acting.run(&held, &options, out) { - Ok(answered) => { - for line in answered.lines() { - let _ = writeln!(out, "{line}"); - } - true - } - Err(error) => { - let _ = writeln!(out, "✗ {error}"); - false - } - } - })); - let code = runner::here(paths, request, work, out)?; - Ok(Ran::Ended(code.unwrap_or(1))) - } - Source::Server(server) => { - let acted = match action { - resource::CHANGE => server.call( - "resources.change", - json!({"id": id}), - Some(serde_json::to_value(api::ResourceChange { - arguments: given.arguments, - spec: given.spec, - secrets: given.secrets, - token: None, - })?), - )?, - resource::REMOVE => server.call("resources.remove", json!({"id": id}), None)?, - _ => server.call( - "resources.act", - json!({"id": id, "action": action}), - Some(serde_json::to_value(Act { - arguments: given.arguments, - ..Act::default() - })?), - )?, - }; - let acted: api::Acted = serde_json::from_value(acted)?; - let Some(job) = acted.job else { - return Ok(Ran::Answered(acted.result.unwrap_or(Value::Null))); - }; - Ok(Ran::Ended(Source::follow(server, &job.id, out, progress)?)) - } - } - } - - /// A resource's log to `out`: the last `tail` lines, and with `follow` - /// what comes after. - pub fn logs(&self, id: &str, tail: u32, follow: bool, out: &mut dyn Write) -> Result<()> { - match self { - Source::Here(paths) => { - let limit = match follow { - true => std::time::Duration::from_secs(24 * 60 * 60), - false => std::time::Duration::from_secs(60), - }; - resource::logs( - paths, - ®istry::Store::new(paths)?, - id, - tail, - follow, - limit, - out, - ) - } - Source::Server(server) if !follow => { - let answer = - server.call("resources.logs", json!({"id": id, "tail": tail}), None)?; - out.write_all(text_of(&answer).as_bytes())?; - Ok(()) - } - Source::Server(server) => berth_core::link::runtime()?.block_on(async { - let portal = server.portal().await?; - let mut stream = portal - .open("resources.follow", None, json!({"id": id, "tail": tail})) - .await?; - while let Some(piece) = stream.recv().await { - match piece? { - api::portal::Piece::Binary(data) => { - out.write_all(&data)?; - out.flush()?; - } - api::portal::Piece::Text(said) => { - if let Some(error) = said["error"].as_str() { - return Err(error.to_owned().into()); - } - } - } - } - Ok(()) - }), - } - } -} - -/// The line a provisioner has in the list: its name, and why it cannot be -/// used here when it cannot. -pub fn summary(provisioner: &ProvisionerInfo) -> String { - let named = match provisioner.kind.is_empty() || provisioner.kind == provisioner.id { - true => format!("{} {}", provisioner.id, provisioner.label), - false => format!( - "{} {} ({})", - provisioner.id, provisioner.label, provisioner.kind - ), - }; - match provisioner.problem.is_empty() { - true => named, - false => format!("{named} · not usable: {}", provisioner.problem), - } -} - -/// What `berth provisioner NAME` says of it. -pub fn describe( - state: &ProvisionerState, - actions: &[Action], - given: &BTreeMap, -) -> String { - let info = &state.provisioner; - let mut text = format!("{} {}\n", info.id, info.label); - if !info.description.is_empty() { - text += &format!("{}\n", info.description); - } - // A reason that is long, or has several lines, goes below in full. - let pieces: Vec<&str> = state - .message - .lines() - .flat_map(|line| line.split(" | ")) - .map(str::trim) - .filter(|piece| !piece.is_empty()) - .collect(); - let long = pieces.len() > 1 || state.message.chars().count() > 60; - match (state.message.is_empty(), long) { - (true, _) => text += &format!("\nState {}\n", state.state), - (false, false) => text += &format!("\nState {} · {}\n", state.state, state.message), - (false, true) => { - text += &format!("\nState {}\n", state.state); - for piece in pieces { - text += &format!(" {piece}\n"); - } - } - } - let width = state - .details - .iter() - .map(|row| row.label.chars().count()) - .max() - .unwrap_or(0) - .max(8); - for row in &state.details { - text += &format!("{: = info - .own - .iter() - .map(|name| match (given.get(name), info.settings.get(name)) { - (Some(value), _) => format!("{name}={value} (given for this call, not saved)"), - (None, Some(value)) => format!("{name}={value}"), - (None, None) => format!("{name} not set"), - }) - .collect(); - text += &format!("Settings {}\n", set.join(" · ")); - if info.own.iter().any(|name| given.contains_key(name)) { - text += &format!( - " To keep them: berth provisioner {} run settings NAME=VALUE\n", - info.id - ); - } - } - let mut kinds: Vec<&str> = vec![]; - for held in &state.resources { - if !kinds.contains(&held.kind.as_str()) { - kinds.push(&held.kind); - } - } - if kinds.is_empty() { - text += "\nWhat berth made with it\n none\n"; - } - for kind in kinds { - let label = info - .kinds - .iter() - .find(|known| known.id == kind) - .map(|known| known.label.as_str()) - .filter(|label| !label.is_empty() && kind != "machine"); - text += &format!( - "\n{} berth made with it\n", - match (kind, label) { - ("machine", _) => "Machines", - (_, Some(label)) => label, - (kind, None) => kind, - } - ); - for held in state.resources.iter().filter(|held| held.kind == kind) { - let mut notes = vec![]; - if !held.state.is_empty() { - notes.push(held.state.clone()); - } - match held.resource.is_empty() { - true => notes.push("berth has no record of it".into()), - false => notes.push(held.resource.clone()), - } - let id = match held.id.as_str() { - "" => "(being made)", - id => id, - }; - let name = match held.host.is_empty() { - true => &held.name, - false => &held.host, - }; - text += &format!(" {name:<16} {id} {}\n", notes.join(" · ")); - } - } - let mut groups: Vec<(&str, Vec<&Action>)> = vec![]; - for action in actions { - let title = berth_tui::page::group_of(action); - match groups - .iter_mut() - .find(|(known, _)| known.eq_ignore_ascii_case(title)) - { - Some((_, list)) => list.push(action), - None => groups.push((title, vec![action])), - } - } - if !groups.is_empty() { - text += &format!("\nActions (berth provisioner {} run ACTION)\n", info.id); - } - for (title, list) in groups { - text += &format!(" {title}\n"); - for action in list { - let why = match action.available { - true => String::new(), - false => format!(" ({})", action.why), - }; - text += &format!(" {:<18} {}{why}\n", action.name, action.title()); - } - } - text -} - -/// `berth provisioner …`. -pub fn command(paths: &Paths, args: &[&str]) -> Result<()> { - let source = Source::of(paths); - match args { - [] => { - let listed = source.list()?; - if listed.is_empty() { - println!( - "No provisioner is registered. Add one: berth provisioner add NAME TYPE, with a type of: berth provisioner types" - ); - } - for provisioner in &listed { - println!("{}", summary(provisioner)); - } - Ok(()) - } - ["types"] => { - let types = source.types()?; - if types.is_empty() { - println!("No plugin brings a provisioner. See: berth plugins"); - } - for kind in &types { - let mut makes: Vec<&str> = kind.kinds.iter().map(|made| made.id.as_str()).collect(); - if makes.is_empty() { - makes.push("machine"); - } - println!( - "{:<20} {} · makes {}", - kind.id, - kind.label, - makes.join(", ") - ); - } - Ok(()) - } - ["add", name, kind, rest @ ..] if rest.iter().all(|arg| arg.contains('=')) => { - let (settings, _) = parse_arguments(rest)?; - if source.list()?.iter().any(|known| known.id == *name) { - return Err(format!( - "There is a provisioner {name} already. Change its settings with: berth provisioner {name} run settings NAME=VALUE" - ) - .into()); - } - source.save( - name, - &SavedProvisioner { - kind: (*kind).to_owned(), - settings: Some(settings), - }, - )?; - println!("Registered {name}, a {kind}. See it with: berth provisioner {name}"); - Ok(()) - } - ["remove", name] => { - source.remove(name)?; - println!("Removed the provisioner {name}"); - Ok(()) - } - [name, "actions"] => { - for action in source.actions(name)? { - let arguments: Vec = action - .arguments - .iter() - .map(|argument| match argument.required { - true => format!("{}=…", argument.name), - false => format!("[{}=…]", argument.name), - }) - .collect(); - let why = match action.available { - true => String::new(), - false => format!(" ({})", action.why), - }; - println!( - "{:<18} {} {}{why}", - action.name, - action.title(), - arguments.join(" ") - ); - } - Ok(()) - } - [name, "run", action, rest @ ..] => { - let (values, yes) = parse_arguments(rest)?; - let actions = source.actions(name)?; - if !actions.iter().any(|known| known.name == *action) { - let names: Vec<&str> = actions.iter().map(|known| known.name.as_str()).collect(); - return Err(format!( - "{name} has no action {action}. Its actions: {}", - names.join(", ") - ) - .into()); - } - let (found, values) = ready(name, &actions, action, values, yes)?; - let act = Act { - arguments: values, - ..Act::default() - }; - let ran = source.run( - name, - &found.name, - act, - &mut std::io::stdout(), - &mut std::io::stderr(), - )?; - match ran { - Ran::Answered(answer) => { - for line in answer_lines(&answer) { - println!("{line}"); - } - Ok(()) - } - Ran::Ended(code) => std::process::exit(code), - } - } - [name, rest @ ..] if rest.iter().all(|arg| arg.contains('=')) => { - let (given, _) = parse_arguments(rest)?; - let state = source.state(name, &given)?; - let actions = source.actions(name).unwrap_or_default(); - print!("{}", describe(&state, &actions, &given)); - Ok(()) - } - _ => Err(USAGE.into()), - } -} - -#[cfg(test)] -mod tests { - use super::*; - use berth_api::{Detail, ListedResource}; - - #[test] - fn a_provisioners_page_says_how_it_is_what_it_made_and_what_it_can_do() { - let state = ProvisionerState { - provisioner: ProvisionerInfo { - id: "incus/vm".into(), - label: "Incus virtual machine".into(), - own: vec!["server".into(), "project".into()], - settings: [("server".to_owned(), "https://192.0.2.10:8443".to_owned())].into(), - ..Default::default() - }, - state: "ready".into(), - message: "Incus 6.18 answers".into(), - details: vec![Detail { - label: "Kernel".into(), - value: "Linux 6.17".into(), - }], - resources: vec![ - ListedResource { - id: "berth-studio".into(), - kind: "machine".into(), - name: "studio".into(), - host: "studio".into(), - state: "running".into(), - resource: "r-studio".into(), - }, - ListedResource { - id: "berth-attic".into(), - kind: "machine".into(), - name: "attic".into(), - host: "attic".into(), - ..ListedResource::default() - }, - ], - ..Default::default() - }; - let actions = [ - Action { - name: "pools".into(), - label: "Storage pools".into(), - group: "Storage".into(), - ..Action::default() - }, - Action { - name: "settings".into(), - label: "Settings".into(), - group: "Set-Up".into(), - available: false, - why: "no".into(), - ..Action::default() - }, - ]; - let text = describe(&state, &actions, &BTreeMap::new()); - for line in [ - "State ready · Incus 6.18 answers\n", - "Kernel Linux 6.17\n", - "Settings server=https://192.0.2.10:8443 · project not set\n", - " studio berth-studio running · r-studio\n", - " attic berth-attic berth has no record of it\n", - "Actions (berth provisioner incus/vm run ACTION)\n", - " pools Storage pools\n", - " settings Settings (no)\n", - ] { - assert!(text.contains(line), "{line:?} in\n{text}"); - } - let given: BTreeMap = - [("server".to_owned(), "https://192.0.2.11:8443".to_owned())].into(); - let long = ProvisionerState { - state: "unreachable".into(), - message: "docker version failed: Cannot connect to the Docker daemon | error during connect: ssh: handshake failed".into(), - ..state.clone() - }; - let text = describe(&long, &[], &given); - for line in [ - "State unreachable\n", - " docker version failed: Cannot connect to the Docker daemon\n", - " error during connect: ssh: handshake failed\n", - "server=https://192.0.2.11:8443 (given for this call, not saved) · project not set\n", - "berth provisioner incus/vm run settings NAME=VALUE\n", - ] { - assert!(text.contains(line), "{line:?} in\n{text}"); - } - let lines = answer_lines(&json!({"message": "2 pools", "details": [ - {"label": "default", "value": "zfs"}, {"label": "fast-ssd", "value": "dir"}]})); - assert_eq!(lines, ["2 pools", " default zfs", " fast-ssd dir"]); - let refused = ProvisionerInfo { - id: "lab/old".into(), - label: "Old".into(), - problem: "lab/old speaks version 1".into(), - ..Default::default() - }; - assert_eq!( - summary(&refused), - "lab/old Old · not usable: lab/old speaks version 1" - ); - } -} diff --git a/crates/berth/src/provisioner_tab.rs b/crates/berth/src/provisioner_tab.rs deleted file mode 100644 index 10ca11c..0000000 --- a/crates/berth/src/provisioner_tab.rs +++ /dev/null @@ -1,501 +0,0 @@ -//! The Provisioners tab of the terminal's home screen: the shared tab of -//! [`berth_tui::provisioners`], with the provisioners of this machine or of -//! the berth server berth uses. What it asks for runs in a thread of its -//! own, so the screen goes on drawing while a provisioner is asked. - -use crate::provisioner_command::{Change, Ran, Source, answer_lines}; -use berth_api::{Act, Action, ProvisionerInfo, ProvisionerState, Resource}; -use berth_core::paths::Paths; -use berth_tui::host::View as Panel; -use berth_tui::provisioners::{Asked, Shown, Tab}; -use berth_tui::screen::Platform; -use crossterm::event::{KeyCode, KeyEvent}; -use ratatui::Frame; -use std::sync::{ - Arc, - atomic::{AtomicBool, Ordering}, - mpsc::{Receiver, Sender, channel}, -}; - -/// What came back from a thread. -enum Arrived { - List(Result, String>), - Page( - String, - Box>, - Result, String>, - ), - Resource( - String, - Box>, - Result, String>, - ), - /// The types of provisioner, to register one. - Types(Result, String>), - /// A provisioner was registered under this name, or why not. - Saved(Result), - Output(Vec), - /// A job ended: the subject's page to ask again, a resource's or a - /// provisioner's, and how it ended. - Ended(Subject, Option), -} - -/// What a job was of, whose page is asked again when it ends. -enum Subject { - Provisioner(String), - Resource(String), -} - -/// Where a job's output goes: on to the screen, a piece at a time, until -/// the panel is closed. -struct Lines(Sender, Arc); - -impl std::io::Write for Lines { - fn write(&mut self, data: &[u8]) -> std::io::Result { - if self.1.load(Ordering::Relaxed) { - return Err(std::io::ErrorKind::BrokenPipe.into()); - } - let _ = self.0.send(Arrived::Output(data.to_vec())); - Ok(data.len()) - } - - fn flush(&mut self) -> std::io::Result<()> { - Ok(()) - } -} - -/// What a key on the tab leads to. -pub enum Left { - Stay, - Tab(usize), - Quit, -} - -pub struct View { - pub tab: Tab, - listing: Option, String>>, - asked: bool, - sender: Sender, - arrived: Receiver, - /// A job that runs or ran, in a panel over the tab. - pub job: Option, - /// Set when the panel is closed, so a log that is followed stops. - closed: Arc, - /// The kind to make once the list has come, as [`View::create`] asks. - creating: Option, -} - -impl Default for View { - fn default() -> View { - let (sender, arrived) = channel(); - View { - tab: Tab::default(), - listing: None, - asked: false, - sender, - arrived, - job: None, - closed: Arc::new(AtomicBool::new(false)), - creating: None, - } - } -} - -impl View { - pub fn list(&self) -> &[ProvisionerInfo] { - match &self.listing { - Some(Ok(list)) => list, - _ => &[], - } - } - - /// Ask for the list, unless it was asked for. - pub fn load(&mut self, paths: &Paths) { - if self.listing.is_some() || self.asked { - return; - } - self.asked = true; - let (paths, sender) = (paths.clone(), self.sender.clone()); - std::thread::spawn(move || { - let listed = Source::of(&paths).list().map_err(|error| error.to_string()); - let _ = sender.send(Arrived::List(listed)); - }); - } - - /// Open the form that makes a resource of `kind` that is no host, such - /// as an instance, with the provisioner that makes it; once the list has - /// come, when it has not. - pub fn create(&mut self, paths: &Paths, kind: &str, status: &mut String) { - let list = match &self.listing { - Some(Ok(list)) => list.clone(), - _ => { - self.creating = Some(kind.to_owned()); - self.listing = None; - self.load(paths); - return; - } - }; - match self.tab.create(&list, kind) { - Asked::Open(id) => self.open(paths, id), - Asked::Nothing(Some(said)) => *status = said, - _ => {} - } - } - - fn open(&self, paths: &Paths, id: String) { - let (paths, sender) = (paths.clone(), self.sender.clone()); - std::thread::spawn(move || { - let source = Source::of(&paths); - let state = source - .state(&id, &Default::default()) - .map_err(|error| error.to_string()); - let actions = source.actions(&id).map_err(|error| error.to_string()); - let _ = sender.send(Arrived::Page(id, Box::new(state), actions)); - }); - } - - fn types(&self, paths: &Paths) { - let (paths, sender) = (paths.clone(), self.sender.clone()); - std::thread::spawn(move || { - let types = Source::of(&paths) - .types() - .map_err(|error| error.to_string()); - let _ = sender.send(Arrived::Types(types)); - }); - } - - fn save(&self, paths: &Paths, name: String, kind: String) { - let (paths, sender) = (paths.clone(), self.sender.clone()); - std::thread::spawn(move || { - let saved = berth_api::SavedProvisioner { - kind, - settings: None, - }; - let said = Source::of(&paths) - .save(&name, &saved) - .map(|()| name) - .map_err(|error| error.to_string()); - let _ = sender.send(Arrived::Saved(said)); - }); - } - - fn open_resource(&self, paths: &Paths, id: String) { - let (paths, sender) = (paths.clone(), self.sender.clone()); - std::thread::spawn(move || { - let source = Source::of(&paths); - let seen = source.resource(&id).map_err(|error| error.to_string()); - let actions = source - .resource_actions(&id) - .map_err(|error| error.to_string()); - let _ = sender.send(Arrived::Resource(id, Box::new(seen), actions)); - }); - } - - /// A panel over the tab for what runs, and the flag that says it closed. - fn panel(&mut self, title: &str) -> Arc { - let (cols, rows) = crossterm::terminal::size().unwrap_or((100, 30)); - self.job = Some(Panel::plain( - title, - rows.saturating_sub(6).max(8), - cols.saturating_sub(8).max(20), - )); - self.closed = Arc::new(AtomicBool::new(false)); - self.closed.clone() - } - - /// Show a resource's log in a panel, following it until the panel closes. - fn logs(&mut self, paths: &Paths, id: String) { - let closed = self.panel(&format!("Log · {id}")); - let (paths, sender) = (paths.clone(), self.sender.clone()); - std::thread::spawn(move || { - let mut out = Lines(sender.clone(), closed); - let code = match Source::of(&paths).logs(&id, 200, true, &mut out) { - Ok(()) => Some(0), - Err(error) => { - let _ = sender.send(Arrived::Output(format!("✗ {error}\n").into_bytes())); - None - } - }; - let _ = sender.send(Arrived::Ended(Subject::Resource(id), code)); - }); - } - - /// Run an action of a resource, as `run` does one of a provisioner. - fn run_resource( - &mut self, - paths: &Paths, - id: String, - action: Action, - values: Change, - status: &mut String, - ) { - let (paths, sender) = (paths.clone(), self.sender.clone()); - *status = format!("Asking {id}…"); - let closed = self.panel(&format!("{} · {id}", action.title())); - std::thread::spawn(move || { - let mut out = Lines(sender.clone(), closed); - let code = match Source::of(&paths).resource_run( - &id, - &action.name, - values, - &mut out, - &mut std::io::sink(), - ) { - Ok(Ran::Ended(code)) => Some(code), - Ok(Ran::Answered(answer)) => { - let lines = answer_lines(&answer).join("\n"); - let _ = sender.send(Arrived::Output(format!("{lines}\n").into_bytes())); - Some(0) - } - Err(error) => { - let _ = sender.send(Arrived::Output(format!("✗ {error}\n").into_bytes())); - None - } - }; - let _ = sender.send(Arrived::Ended(Subject::Resource(id), code)); - }); - } - - /// Take in what has come back, and say what there is to say. - pub fn pump(&mut self, paths: &Paths, status: &mut String) { - while let Ok(arrived) = self.arrived.try_recv() { - match arrived { - Arrived::List(listed) => { - if let Err(error) = &listed { - *status = format!("✗ {error}"); - } - let ready = listed.is_ok(); - self.listing = Some(listed); - self.asked = false; - if let Some(kind) = self.creating.take() - && ready - { - self.create(paths, &kind, status); - } - } - // One that was removed takes its page back to the list. - Arrived::Page(_, state, _) - if state - .as_ref() - .as_ref() - .is_err_and(|error| error.starts_with("No provisioner ")) => - { - self.tab.page = None; - self.listing = None; - self.load(paths); - } - Arrived::Page(id, state, actions) => self.tab.opened(&id, *state, actions), - Arrived::Types(types) => { - if let Some(said) = self.tab.adding(types) { - *status = said; - } - } - Arrived::Saved(Ok(name)) => { - *status = format!("Registered {name}. Its page has its settings and secrets"); - self.listing = None; - self.load(paths); - } - Arrived::Saved(Err(error)) => *status = format!("✗ {error}"), - // One that was removed takes its page back to its provisioner's. - Arrived::Resource(id, seen, _) - if seen - .as_ref() - .as_ref() - .is_err_and(|error| error.starts_with("No resource ")) => - { - if let Some(provisioner) = self.tab.back_from(&id) { - self.open(paths, provisioner); - } - } - Arrived::Resource(id, seen, actions) => { - self.tab.opened_resource(&id, *seen, actions) - } - Arrived::Output(data) => { - if let Some(job) = &mut self.job { - job.feed(&data); - } - } - Arrived::Ended(subject, code) => { - if let Some(job) = &mut self.job { - job.ended = Some(code); - } - // What the action changed shows on the page. - match subject { - Subject::Provisioner(id) => self.open(paths, id), - Subject::Resource(id) => self.open_resource(paths, id), - } - } - } - } - } - - pub fn shown(&self, paths: &Paths) -> Shown { - let mut shown = self.tab.shown(self.list(), &Source::of(paths).place()); - if let Some(Err(error)) = &self.listing { - shown.help = format!("✗ {error}\n\n{}", shown.help); - } - shown - } - - /// Draw what is over the tab: a job's panel, a form, a question, the - /// arguments. `status` is the last message, which a form shows itself. - pub fn draw_over(&self, frame: &mut Frame, status: &str) { - if let Some(job) = &self.job { - let keys = match job.outcome() { - None => " It runs · its output comes as it is written ", - Some(true) => " ✓ Done · Enter close ", - Some(false) => " ✗ It did not finish · Enter close ", - }; - berth_tui::screen::draw_panel(frame, &job.title, job.screen(), job.outcome(), keys, ""); - return; - } - if let Some(form) = self.tab.form() { - form.draw(frame, status); - } - if let Some(question) = self.tab.question() { - berth_tui::screen::draw_question(frame, question); - } - if let Some((actions, asked)) = self.tab.arguments() { - berth_tui::screen::draw_action_arguments(frame, actions, asked); - } - } - - /// Run an action, in a thread. Its output, or its answer, fills a panel. - fn run(&mut self, paths: &Paths, id: String, action: Action, values: Act, status: &mut String) { - let (paths, sender) = (paths.clone(), self.sender.clone()); - *status = format!("Asking {id}…"); - let closed = self.panel(&format!("{} · {id}", action.title())); - std::thread::spawn(move || { - let mut out = Lines(sender.clone(), closed); - let code = match Source::of(&paths).run( - &id, - &action.name, - values, - &mut out, - &mut std::io::sink(), - ) { - Ok(Ran::Ended(code)) => Some(code), - Ok(Ran::Answered(answer)) => { - let lines = answer_lines(&answer).join("\n"); - let _ = sender.send(Arrived::Output(format!("{lines}\n").into_bytes())); - Some(0) - } - Err(error) => { - let _ = sender.send(Arrived::Output(format!("✗ {error}\n").into_bytes())); - None - } - }; - let _ = sender.send(Arrived::Ended(Subject::Provisioner(id), code)); - }); - } - - /// Whether keys go to a dialog or a panel, so none of them changes tabs. - pub fn typing(&self) -> bool { - self.job.is_some() || self.tab.typing() - } - - /// Handle a key on the tab. - pub fn key( - &mut self, - paths: &Paths, - key: KeyEvent, - status: &mut String, - platform: &mut dyn Platform, - ) -> Left { - if let Some(job) = &self.job { - // A log goes on until it is closed; a job's panel closes once it ended. - let log = job.title.starts_with("Log · "); - if (log || !job.running()) - && matches!(key.code, KeyCode::Enter | KeyCode::Esc | KeyCode::Char('q')) - { - self.closed.store(true, Ordering::Relaxed); - self.job = None; - } - return Left::Stay; - } - let Some(shared) = crate::keys::key(&key) else { - return Left::Stay; - }; - let list = self.list().to_vec(); - let asked = self.tab.key(shared, &list, platform); - if asked == Asked::Other { - match key.code { - KeyCode::Char(digit @ '1'..='6') => { - return Left::Tab(digit as usize - '1' as usize); - } - // Next to the hosts: Tab goes on to the settings, Shift+Tab back. - KeyCode::Tab => return Left::Tab(2), - KeyCode::BackTab | KeyCode::Esc => return Left::Tab(0), - KeyCode::Char('q') => return Left::Quit, - _ => {} - } - } - self.asked(paths, asked, status) - } - - /// Put pasted text into what is being typed. - pub fn paste(&mut self, text: &str) { - self.tab.paste(text); - } - - /// A click while a form is open, which the form takes. - pub fn click( - &mut self, - paths: &Paths, - (column, row): (u16, u16), - status: &mut String, - platform: &mut dyn Platform, - ) -> Left { - let list = self.list().to_vec(); - let asked = self.tab.click(column, row, &list, platform); - self.asked(paths, asked, status) - } - - /// Do what a key or a click on the tab asked for. - fn asked(&mut self, paths: &Paths, asked: Asked, status: &mut String) -> Left { - match asked { - Asked::Nothing(said) => { - if let Some(said) = said { - *status = said; - } - } - Asked::List => { - self.listing = None; - self.load(paths); - } - Asked::Add => self.types(paths), - Asked::Save { name, kind } => { - *status = format!("Registering {name}…"); - self.save(paths, name, kind); - } - Asked::Open(id) => self.open(paths, id), - Asked::Run { - provisioner, - action, - values, - } => { - let act = Act { - arguments: values, - ..Act::default() - }; - self.run(paths, provisioner, *action, act, status); - } - Asked::OpenResource(id) => self.open_resource(paths, id), - Asked::RunResource { - resource, - action, - values, - } => { - let change = Change { - arguments: values, - ..Change::default() - }; - self.run_resource(paths, resource, *action, change, status); - } - Asked::Logs(id) => self.logs(paths, id), - Asked::Hosts => return Left::Tab(0), - Asked::Other => {} - } - Left::Stay - } -} diff --git a/crates/berth/src/remote.rs b/crates/berth/src/remote.rs deleted file mode 100644 index 100801b..0000000 --- a/crates/berth/src/remote.rs +++ /dev/null @@ -1,532 +0,0 @@ -//! The host's own API, for the UI: one background thread owns the runtime and -//! the link, takes requests and sends back what happened. The UI never waits -//! for the network. - -use berth_api::{self as api, client::Frame}; -use berth_core::link::{Link, LinkSpec}; -pub use berth_tui::host::{Event, HostView, NEEDS_API, Pending, Request, State, Transport, list}; -use futures_util::{SinkExt, StreamExt}; -use serde_json::{Value, json}; -use std::sync::{Arc, mpsc}; -use tokio::sync::mpsc::{UnboundedReceiver, UnboundedSender, unbounded_channel}; - -/// The exit code an `exited` frame carries, when the text is one. -pub fn exited(text: &str) -> Option> { - berth_tui::host::exited(&serde_json::from_str(text).ok()?) -} - -enum Out { - Input(Vec), - Resize(u16, u16), -} - -/// How a host's version is compared with what would be applied now. It may -/// wait on the network, and runs off the runtime. -pub type Compare = Arc Option + Send + Sync>; - -async fn refresh( - link: Arc, - events: mpsc::Sender, - all: bool, - compare: Option, -) { - if all { - match link.get::("/v1/version").await { - Ok(version) => { - let old = version.api < NEEDS_API; - if let Some(compare) = compare { - let asked = version.clone(); - let events = events.clone(); - tokio::task::spawn_blocking(move || { - let _ = events.send(Event::Behind(compare(&asked))); - }); - } - let _ = events.send(Event::Version(version)); - if old { - return; - } - } - Err(error) => { - let _ = events.send(Event::Failed { - missing: Link::missing(&error), - message: error.to_string(), - }); - return; - } - } - if let Ok(tasks) = link.get::("/v1/tasks").await { - let _ = events.send(Event::Tasks(list(tasks, "tasks"))); - } - } - if let Ok(sessions) = link.get::("/v1/sessions").await { - let _ = events.send(Event::Sessions(list(sessions, "sessions"))); - } - if let Ok(stats) = link.get::("/v1/stats").await { - let _ = events.send(Event::Stats(Box::new(stats))); - } -} - -async fn follow( - link: Arc, - session: api::Session, - events: mpsc::Sender, - mut out: UnboundedReceiver, -) { - let terminal = session.kind == api::SessionKind::Terminal; - let id = session.id.clone(); - // A terminal replays its screen; a process is read from the start. - let path = match terminal { - true => format!("/v1/sessions/{id}"), - false => format!("/v1/sessions/{id}?since=0"), - }; - let mut stream = match link.stream(&path).await { - Ok(stream) => stream, - Err(error) => { - // An ended session can still be read back. - let output = link - .get::(&format!("/v1/sessions/{id}/output")) - .await; - match output.ok().and_then(|output| api::unb64(&output.data_b64)) { - Some(bytes) if !session.running() => { - let _ = events.send(Event::Output(id.clone(), bytes)); - let _ = events.send(Event::Exited(id, session.exit_code)); - } - _ => { - let _ = events.send(Event::Error(error.to_string())); - let _ = events.send(Event::Closed(id)); - } - } - return; - } - }; - loop { - tokio::select! { - frame = stream.next() => match frame { - Some(Ok(Frame::Binary(bytes))) => { - let event = match (terminal, bytes.split_first()) { - (true, _) => Event::Output(id.clone(), bytes.to_vec()), - (false, Some((_, rest))) => Event::Output(id.clone(), rest.to_vec()), - (false, None) => continue, - }; - if events.send(event).is_err() { - return; - } - } - Some(Ok(Frame::Text(text))) => { - if let Some(code) = exited(text.as_str()) { - let _ = events.send(Event::Exited(id.clone(), code)); - } - } - Some(Ok(_)) => {} - _ => { - let _ = events.send(Event::Closed(id)); - return; - } - }, - message = out.recv() => { - let frame = match message { - Some(Out::Input(bytes)) if terminal => Frame::Binary(bytes.into()), - Some(Out::Input(bytes)) => Frame::Binary([vec![0], bytes].concat().into()), - Some(Out::Resize(rows, cols)) => { - Frame::Text(json!({"resize": {"rows": rows, "cols": cols}}).to_string().into()) - } - None => { - let _ = stream.close().await; - return; - } - }; - if stream.send(frame).await.is_err() { - let _ = events.send(Event::Closed(id)); - return; - } - } - } - } -} - -async fn worker( - link: Arc, - mut requests: UnboundedReceiver, - events: mpsc::Sender, - compare: Option, -) { - let mut stream: Option> = None; - while let Some(request) = requests.recv().await { - let (link, events) = (link.clone(), events.clone()); - match request { - Request::Refresh => drop(tokio::spawn(refresh(link, events, true, compare.clone()))), - Request::Poll => drop(tokio::spawn(refresh(link, events, false, None))), - Request::Run { name, body } => drop(tokio::spawn(async move { - let _ = events.send(match link.run_task(&name, &body).await { - Ok(session) => Event::Opened(Box::new(session)), - Err(error) => Event::Error(error.to_string()), - }); - })), - Request::Start { body } => drop(tokio::spawn(async move { - let _ = events.send( - match link.post::("/v1/sessions", &body).await { - Ok(session) => Event::Opened(Box::new(session)), - Err(error) => Event::Error(error.to_string()), - }, - ); - })), - Request::Open(session) => { - let (sender, receiver) = unbounded_channel(); - stream = Some(sender); - tokio::spawn(follow(link, *session, events, receiver)); - } - Request::Input(bytes) => { - if let Some(stream) = &stream { - let _ = stream.send(Out::Input(bytes)); - } - } - Request::Resize(rows, cols) => { - if let Some(stream) = &stream { - let _ = stream.send(Out::Resize(rows, cols)); - } - } - Request::Close => stream = None, - Request::Delete(id) => drop(tokio::spawn(async move { - let path = format!("/v1/sessions/{id}"); - if let Err(error) = link.call::("DELETE", &path, None).await { - let _ = events.send(Event::Error(error.to_string())); - } - refresh(link, events, false, None).await; - })), - Request::Choices { task, argument } => drop(tokio::spawn(async move { - let path = format!( - "/v1/tasks/{}/choices/{}", - api::client::encode(&task), - api::client::encode(&argument) - ); - let choices = match link.get::(&path).await { - Ok(value) => list(value, "choices"), - Err(_) => vec![], - }; - let _ = events.send(Event::Choices { - task, - argument, - choices, - }); - })), - } - } -} - -/// The thread that talks to one host. -pub struct Remote { - requests: UnboundedSender, - events: mpsc::Receiver, -} - -impl Remote { - pub fn start(spec: LinkSpec, compare: Option) -> std::io::Result { - let runtime = berth_core::link::runtime()?; - let (requests, receiver) = unbounded_channel(); - let (sender, events) = mpsc::channel(); - std::thread::spawn(move || { - runtime.block_on(worker(Arc::new(Link::new(spec)), receiver, sender, compare)); - }); - Ok(Remote { requests, events }) - } -} - -impl Transport for Remote { - fn send(&self, request: Request) { - let _ = self.requests.send(request); - } - - fn next(&self) -> Option { - self.events.try_recv().ok() - } -} - -/// Start talking to the host over `spec`. Returns why not, if it cannot be. -#[cfg(test)] -pub fn connect(host: &mut HostView, spec: LinkSpec) -> Option { - connect_with(host, spec, None) -} - -/// The same, comparing the host with what would be applied now. -pub fn connect_with( - host: &mut HostView, - spec: LinkSpec, - compare: Option, -) -> Option { - match Remote::start(spec, compare) { - Ok(remote) => { - host.connect(Box::new(remote)); - None - } - Err(error) => { - host.unreachable(error.to_string()); - Some(error.to_string()) - } - } -} - -#[cfg(test)] -mod tests { - use super::*; - use berth_tui::host::{Arguments, View}; - use std::time::Duration; - - #[test] - fn lists_come_bare_or_under_a_key() { - let bare: Vec = list(json!([{"name": "status"}]), "tasks"); - let keyed: Vec = list( - json!({"tasks": [{"name": "status"}, {"no": "name"}]}), - "tasks", - ); - assert_eq!((bare.len(), keyed.len()), (1, 1)); - assert_eq!(exited(r#"{"exited":{"exit_code":3}}"#), Some(Some(3))); - assert_eq!(exited(r#"{"exited":{}}"#), Some(None)); - assert_eq!(exited(r#"{"dropped":{"frames":1}}"#), None); - } - - #[test] - fn a_transcript_is_shown_line_by_line() { - let session: api::Session = serde_json::from_value(json!({ - "id": "s1", "kind": "process", "title": "demo", "command": ["true"], "user": "agent", - "started_by": {"level": "host", "uid": 0}, "started": "2026-11-12T00:00:00Z", - })) - .unwrap(); - let mut view = View::new(&session, 4, 20, true, false); - assert!(view.feed(b"one\ntwo").is_empty()); - view.feed(b"!\nthree\n"); - assert_eq!(view.screen().contents(), "one\ntwo!\nthree"); - assert!(view.running() && view.outcome().is_none()); - view.ended = Some(Some(2)); - assert_eq!(view.outcome(), Some(false)); - // A terminal answers a question about the cursor, also one in two pieces. - let mut terminal = View::new(&session, 4, 20, true, false); - terminal.terminal = true; - assert!(terminal.feed(b"ab\x1b[6").is_empty()); - assert_eq!(terminal.feed(b"n"), b"\x1b[1;3R"); - } - - /// The overview against a real controller, when one has been built - /// (`cargo build -p berthd`): connect, list, run tasks, type into one. - #[test] - fn the_overview_talks_to_a_controller() { - let built = std::env::current_exe() - .unwrap() - .parent() - .unwrap() - .parent() - .unwrap() - .join("berthd"); - if !built.is_file() { - eprintln!("skipped: {} is not built", built.display()); - return; - } - let dir = tempfile::tempdir().unwrap(); - std::fs::create_dir_all(dir.path().join("etc")).unwrap(); - let tasks = json!([ - {"name": "status", "label": "Status", "group": "System", "user": "agent", "run": "echo up; echo warn >&2"}, - {"name": "demo:status", "label": "Demo", "extends": ["status"], "user": "agent", "run": "echo demo; exit 4"}, - {"name": "greet", "label": "Greet", "user": "agent", "run": "echo \"hello $BERTH_ARG_WHO\"", - "arguments": [{"name": "who", "required": true, "choices": "printf 'ann\\nbob\\n'"}]}, - {"name": "ask", "label": "Ask", "user": "agent", "terminal": true, "run": "printf 'code? '; read c; echo got $c"}, - ]); - std::fs::write(dir.path().join("etc/tasks.json"), tasks.to_string()).unwrap(); - let mut daemon = std::process::Command::new(&built) - .arg("--single") - .arg(dir.path()) - .spawn() - .unwrap(); - struct Stop<'a>(&'a mut std::process::Child); - impl Drop for Stop<'_> { - fn drop(&mut self) { - let _ = self.0.kill(); - let _ = self.0.wait(); - } - } - let _stop = Stop(&mut daemon); - for _ in 0..100 { - if dir.path().join("control.sock").exists() { - break; - } - std::thread::sleep(Duration::from_millis(50)); - } - let spec = LinkSpec { - program: built.to_string_lossy().into_owned(), - args: vec![ - "connect".into(), - "--dir".into(), - dir.path().to_string_lossy().into_owned(), - ], - first_line: None, - via: None, - ..LinkSpec::default() - }; - let mut host = HostView::default(); - assert!(connect(&mut host, spec).is_none()); - fn until(host: &mut HostView, what: &str, done: impl Fn(&HostView) -> bool) { - for _ in 0..400 { - if let Some(note) = host.pump() { - eprintln!("note: {note}"); - } - if done(host) { - return; - } - std::thread::sleep(Duration::from_millis(25)); - } - panic!("never: {what}; state {:?}", host.state); - } - until(&mut host, "ready with tasks", |host| { - matches!(host.state, State::Ready(_)) && host.tasks.len() == 4 - }); - until(&mut host, "stats", |host| host.stats.is_some()); - assert!(host.stats_line().unwrap().contains("load ")); - assert!(host.tasks[0].run.is_empty() && host.tasks[1].extends == ["status"]); - - // A task and the step a plugin adds, as one transcript; the first - // failing step is the outcome. - host.run("status", json!({})); - until(&mut host, "status ended", |host| { - host.view.as_ref().is_some_and(|view| !view.running()) - }); - let view = host.view.as_ref().unwrap(); - assert!(view.ours && !view.terminal && view.title == "Status"); - let text = view.screen().contents(); - assert!( - text.contains("up") && text.contains("warn") && text.contains("demo"), - "{text}" - ); - assert_eq!(view.ended, Some(Some(4))); - let id = view.id.clone(); - // Closing a finished one that was started here forgets it. - host.close(true); - until(&mut host, "forgotten", |host| { - host.sessions.iter().all(|session| session.id != id) - }); - - // Arguments reach the command; the host offers choices for one. - host.send(Request::Choices { - task: "greet".into(), - argument: "who".into(), - }); - host.arguments = Some(Arguments { - action: 1, - index: 0, - values: Default::default(), - input: Default::default(), - choices: vec![], - chosen: None, - secret: false, - text: false, - }); - until(&mut host, "choices", |host| { - host.arguments.as_ref().unwrap().choices == ["ann", "bob"] - }); - host.arguments = None; - host.run("greet", json!({"arguments": {"who": "a 'quoted' b"}})); - until(&mut host, "greet ended", |host| { - host.view.as_ref().is_some_and(|view| !view.running()) - }); - assert!( - host.view - .as_ref() - .unwrap() - .screen() - .contents() - .contains("hello a 'quoted' b") - ); - host.close(false); - until(&mut host, "kept", |host| { - host.sessions.iter().any(|session| session.title == "Greet") - }); - - // A terminal task takes typing; left with Ctrl+Q it runs on and can be - // opened again. - host.run("ask", json!({"rows": 10, "cols": 40})); - until(&mut host, "the question", |host| { - host.view - .as_ref() - .is_some_and(|view| view.screen().contents().contains("code?")) - }); - assert!(host.view.as_ref().unwrap().terminal); - host.close(false); - until(&mut host, "listed as running", |host| { - host.sessions - .iter() - .any(|session| session.title == "Ask" && session.running()) - }); - let session = host - .sessions - .iter() - .find(|session| session.title == "Ask") - .unwrap() - .clone(); - host.open(&session); - until(&mut host, "the question again", |host| { - host.view - .as_ref() - .is_some_and(|view| view.screen().contents().contains("code?")) - }); - host.input(b"42\r".to_vec()); - until(&mut host, "the answer", |host| { - host.view.as_ref().is_some_and(|view| !view.running()) - }); - let view = host.view.as_ref().unwrap(); - assert!(view.screen().contents().contains("got 42") && view.outcome() == Some(true)); - assert!(!view.ours); - - // A task the host does not have says so. - host.close(true); - host.run("nothing", json!({})); - let mut said = None; - for _ in 0..200 { - said = host.pump(); - if said.is_some() { - break; - } - std::thread::sleep(Duration::from_millis(25)); - } - assert!(said.is_some() && host.view.is_none()); - } - - #[test] - fn a_host_without_a_controller_is_told_apart() { - let spec = |script: &str| LinkSpec { - program: "sh".into(), - args: vec!["-c".into(), script.into()], - first_line: None, - via: None, - ..LinkSpec::default() - }; - let state = |script: &str, password: Option<&str>| { - let mut host = HostView::default(); - host.password = password.map(str::to_owned); - connect(&mut host, spec(script)); - for _ in 0..400 { - host.pump(); - if host.state != State::Connecting { - break; - } - std::thread::sleep(Duration::from_millis(25)); - } - (host.state, host.password) - }; - assert_eq!( - state("echo 'sudo: berthd: command not found' >&2; exit 1", None).0, - State::Missing - ); - assert_eq!( - state("echo 'sudo: a password is required' >&2; exit 1", None).0, - State::Password - ); - // A password sudo did not take is forgotten, to be asked for again. - let wrong = state( - "echo 'Sorry, try again.' >&2; echo 'sudo: 1 incorrect password attempt' >&2; exit 1", - Some("x"), - ); - assert_eq!(wrong, (State::Password, None)); - let (other, _) = state( - "echo 'ssh: connect to host box port 22: Connection timed out' >&2; exit 255", - None, - ); - assert!(matches!(other, State::Unreachable(message) if message.contains("timed out"))); - } -} diff --git a/crates/berth/src/resource_command.rs b/crates/berth/src/resource_command.rs deleted file mode 100644 index e856f1b..0000000 --- a/crates/berth/src/resource_command.rs +++ /dev/null @@ -1,456 +0,0 @@ -//! `berth resource …`, `berth deploy …`, and the short forms for a host's -//! machine, `berth provision` and `berth instance`. With a berth server set -//! the resources are the server's, and what changes them runs there as jobs; -//! otherwise here, with this berth's secrets. - -use crate::Result; -use crate::commands::{parse_arguments, ready}; -use crate::provisioner_command::{Change, Ran, Source, answer_lines}; -use berth_api::{Action, NewResource, Resource, ResourceSecret}; -use berth_core::{paths::Paths, resource}; -use std::io::{IsTerminal, Read, Write}; - -const USAGE: &str = "Usage: berth resource [--provisioner PROVISIONER] [--kind KIND] [--host HOST] - berth resource ID [actions | run ACTION [NAME=VALUE…] [--yes] | logs [--tail N] [--follow] - | change [--spec FILE|-] [--secret NAME[:VARIABLE]]… [NAME=VALUE…] | remove [--yes]]"; - -const DEPLOY: &str = "Usage: berth deploy NAME --provisioner PROVISIONER --spec FILE|- [--secret NAME[:VARIABLE]]… [--token TEXT] [NAME=VALUE…]"; - -fn ask(question: &str) -> Result { - print!("{question} "); - std::io::stdout().flush()?; - let mut answer = String::new(); - std::io::stdin().read_line(&mut answer)?; - Ok(answer.trim().to_owned()) -} - -/// A berth secret given as `NAME` or `NAME:VARIABLE`. -fn secret(given: &str) -> ResourceSecret { - match given.split_once(':') { - Some((name, variable)) => ResourceSecret { - name: name.to_owned(), - env: variable.to_owned(), - }, - None => ResourceSecret { - name: given.to_owned(), - env: String::new(), - }, - } -} - -/// A specification from a file, or from standard input for `-`. -fn read_spec(from: &str) -> Result { - let mut text = String::new(); - match from { - "-" => { - std::io::stdin().read_to_string(&mut text)?; - } - path => { - text = std::fs::read_to_string(path).map_err(|error| format!("{path}: {error}"))?; - } - } - Ok(text) -} - -/// What the line of a resource in a list says. -pub fn line(found: &Resource) -> String { - let name = match &found.host { - Some(host) if found.kind == "machine" => format!("{} (host {host})", found.name), - _ => found.name.clone(), - }; - format!( - "{:<12} {:<11} {:<26} {:<20} {}", - found.id, found.kind, name, found.provisioner, found.state - ) -} - -/// What `berth resource ID` says of it. -pub fn describe(found: &Resource, actions: &[Action]) -> String { - let mut text = format!( - "{} {} {} of {}\n", - found.id, found.kind, found.name, found.provisioner - ); - let mut row = |label: &str, value: &str| { - if !value.is_empty() { - text += &format!("{label:<11}{value}\n"); - } - }; - row("State", &found.state); - row("Problem", &found.problem); - row("Made as", &found.external_id); - row("Host", found.host.as_deref().unwrap_or("")); - row("Address", &found.address); - row("Created", &found.created); - row("Asked", &found.checked); - row("Spec", &found.spec_sha256); - let secrets: Vec = found - .secrets - .iter() - .map(|secret| format!("{} as {}", secret.name, secret.variable())) - .collect(); - row("Secrets", &secrets.join(", ")); - let arguments: Vec = found - .arguments - .iter() - .filter(|(_, value)| !value.is_empty()) - .map(|(key, value)| format!("{key}={value}")) - .collect(); - row("Arguments", &arguments.join(" ")); - if !actions.is_empty() { - text += &format!("\nActions (berth resource {} run ACTION)\n", found.id); - } - for action in actions { - let why = match action.available { - true => String::new(), - false => format!(" ({})", action.why), - }; - text += &format!(" {:<18} {}{why}\n", action.name, action.title()); - } - if found.verbs.iter().any(|verb| verb == "logs") { - text += &format!("\nIts log: berth resource {} logs [--follow]\n", found.id); - } - text -} - -/// Leave with how a run of an action ended, after saying what it answered. -fn finish(ran: Ran) -> Result<()> { - match ran { - Ran::Answered(answer) => { - for line in answer_lines(&answer) { - println!("{line}"); - } - Ok(()) - } - Ran::Ended(code) => std::process::exit(code), - } -} - -/// The options of `change` and `deploy`: the spec, the secrets, a token and -/// the arguments. -#[derive(Default)] -struct Given { - provisioner: Option, - spec: Option, - secrets: Option>, - token: Option, - values: std::collections::BTreeMap, - yes: bool, -} - -fn options(args: &[&str], usage: &str) -> Result { - let mut given = Given::default(); - let mut plain = vec![]; - let mut rest = args.iter(); - while let Some(arg) = rest.next() { - let mut value = || rest.next().copied().ok_or(usage); - match *arg { - "--provisioner" => given.provisioner = Some(value()?.to_owned()), - "--spec" => given.spec = Some(read_spec(value()?)?), - "--secret" => given - .secrets - .get_or_insert_with(Vec::new) - .push(secret(value()?)), - "--token" => given.token = Some(value()?.to_owned()), - other => plain.push(other), - } - } - (given.values, given.yes) = parse_arguments(&plain)?; - Ok(given) -} - -/// `berth resource …`. -pub fn command(paths: &Paths, args: &[&str]) -> Result<()> { - let source = Source::of(paths); - let filtered = |args: &[&str]| -> Result<()> { - let (mut provisioner, mut kind, mut host) = (None, None, None); - let mut rest = args.iter(); - while let Some(arg) = rest.next() { - let value = rest.next().copied().ok_or(USAGE)?; - match *arg { - "--provisioner" => provisioner = Some(value), - "--kind" => kind = Some(value), - "--host" => host = Some(value), - _ => return Err(USAGE.into()), - } - } - let listed = source.resources(provisioner, kind, host)?; - if listed.is_empty() { - println!("berth made nothing with a provisioner yet. See: berth provisioner"); - } - for found in &listed { - println!("{}", line(found)); - } - Ok(()) - }; - match args { - [] => filtered(&[]), - [first, ..] if first.starts_with("--") => filtered(args), - [given] => { - let id = source.resolve(given)?; - let found = source.resource(&id)?; - let actions = source.resource_actions(&id).unwrap_or_default(); - print!("{}", describe(&found, &actions)); - Ok(()) - } - [given, "actions"] => { - let id = source.resolve(given)?; - for action in source.resource_actions(&id)? { - let arguments: Vec = action - .arguments - .iter() - .map(|argument| format!("[{}=…]", argument.name)) - .collect(); - let why = match action.available { - true => String::new(), - false => format!(" ({})", action.why), - }; - println!( - "{:<18} {} {}{why}", - action.name, - action.title(), - arguments.join(" ") - ); - } - Ok(()) - } - [given, "logs", rest @ ..] => { - let id = source.resolve(given)?; - let (mut tail, mut follow) = (100u32, false); - let mut rest = rest.iter(); - while let Some(arg) = rest.next() { - match *arg { - "--follow" | "-f" => follow = true, - "--tail" => { - tail = rest - .next() - .ok_or("--tail needs a number of lines")? - .parse() - .map_err(|_| "--tail needs a number of lines")? - } - _ => return Err(USAGE.into()), - } - } - source.logs(&id, tail, follow, &mut std::io::stdout()) - } - [given, "change", rest @ ..] => { - let id = source.resolve(given)?; - let given = options(rest, USAGE)?; - let change = Change { - arguments: given.values, - spec: given.spec, - secrets: given.secrets, - }; - finish(source.resource_run( - &id, - resource::CHANGE, - change, - &mut std::io::stdout(), - &mut std::io::stderr(), - )?) - } - [given, "remove", rest @ ..] => { - let id = source.resolve(given)?; - let (_, yes) = parse_arguments(rest)?; - let actions = source.resource_actions(&id)?; - ready(&id, &actions, resource::REMOVE, Default::default(), yes)?; - finish(source.resource_run( - &id, - resource::REMOVE, - Change::default(), - &mut std::io::stdout(), - &mut std::io::stderr(), - )?) - } - [given, "run", action, rest @ ..] => { - let id = source.resolve(given)?; - let (values, yes) = parse_arguments(rest)?; - let actions = source.resource_actions(&id)?; - let (found, values) = ready(&id, &actions, action, values, yes)?; - finish(source.resource_run( - &id, - &found.name, - Change { - arguments: values, - ..Change::default() - }, - &mut std::io::stdout(), - &mut std::io::stderr(), - )?) - } - _ => Err(USAGE.into()), - } -} - -/// `berth deploy NAME --provisioner PROVISIONER --spec FILE|- …`: make a -/// deployment from a specification, or give the one of that name the new -/// specification. -pub fn deploy(paths: &Paths, name: &str, args: &[&str]) -> Result<()> { - let given = options(args, DEPLOY)?; - let provisioner = given.provisioner.ok_or(DEPLOY)?; - let spec = given.spec.ok_or(DEPLOY)?; - let source = Source::of(paths); - let existing = source - .resources(Some(&provisioner), Some("deployment"), None)? - .into_iter() - .find(|known| known.name == name && !known.external_id.is_empty()); - if let Some(existing) = existing { - eprintln!( - "There is a deployment {name} already, {}: giving it this specification", - existing.id - ); - let change = Change { - arguments: given.values, - spec: Some(spec), - secrets: given.secrets, - }; - return finish(source.resource_run( - &existing.id, - resource::CHANGE, - change, - &mut std::io::stdout(), - &mut std::io::stderr(), - )?); - } - let asked = NewResource { - provisioner, - kind: "deployment".into(), - name: name.to_owned(), - arguments: given.values, - spec: Some(spec), - secrets: given.secrets.unwrap_or_default(), - token: given.token, - ..NewResource::default() - }; - let (made, code) = source.create(&asked, &mut std::io::stdout(), &mut std::io::stderr())?; - eprintln!("{}: berth resource {}", made.name, made.id); - std::process::exit(code); -} - -/// `berth provision HOST PROVISIONER [NAME=VALUE…] [--yes]`: create the -/// machine for a saved host and set it up. -pub fn provision(paths: &Paths, name: &str, provisioner: &str, args: &[&str]) -> Result<()> { - let (mut values, yes) = parse_arguments(args)?; - let source = Source::of(paths); - let known = source.list()?; - let found = known - .iter() - .find(|known| known.id == provisioner) - .ok_or_else(|| format!("No provisioner {provisioner}. See: berth provisioners"))?; - // What is required and has neither a default nor a kept setting is asked for here, as `berth run` does. - if std::io::stdin().is_terminal() { - for argument in &found.arguments { - if argument.required - && argument.default.is_empty() - && found - .settings - .get(&argument.name) - .is_none_or(String::is_empty) - && !values.contains_key(&argument.name) - { - let label = match argument.label.as_str() { - "" => &argument.name, - label => label, - }; - values.insert(argument.name.clone(), ask(&format!("{label}:"))?); - } - } - } - if !yes - && !ask(&format!( - "Create a machine for {name} with {} ({provisioner})? [y/N]", - found.label - ))? - .eq_ignore_ascii_case("y") - { - return Err("Not created".into()); - } - let asked = NewResource { - provisioner: provisioner.to_owned(), - kind: "machine".into(), - host: Some(name.to_owned()), - arguments: values, - ..NewResource::default() - }; - let (_, code) = source.create(&asked, &mut std::io::stdout(), &mut std::io::stderr())?; - std::process::exit(code); -} - -/// The machine of a host, as a resource. -fn machine_of(source: &Source, name: &str) -> Result { - source - .resources(None, Some("machine"), Some(name))? - .into_iter() - .next() - .ok_or_else(|| format!("{name} has no machine that berth created").into()) -} - -/// `berth instance HOST [start|stop|destroy [--yes]]`: the machine berth -/// created for the host, or a change to it, as its resource. -pub fn instance(paths: &Paths, name: &str, args: &[&str]) -> Result<()> { - let source = Source::of(paths); - let id = machine_of(&source, name)?.id; - let run = |action: &str| { - source.resource_run( - &id, - action, - Change::default(), - &mut std::io::stdout(), - &mut std::io::stderr(), - ) - }; - match args { - [] => { - let found = source.resource(&id)?; - let actions = source.resource_actions(&id).unwrap_or_default(); - print!("{}", describe(&found, &actions)); - Ok(()) - } - [verb @ ("start" | "stop")] => finish(run(verb)?), - ["destroy", rest @ ..] => { - let (_, yes) = parse_arguments(rest)?; - if !yes && ask(&format!("Delete {name}'s machine? Type {name} to say so:"))? != name { - return Err("Not deleted".into()); - } - finish(run(resource::REMOVE)?) - } - _ => Err("Usage: berth instance HOST [start|stop|destroy [--yes]]".into()), - } -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn a_resource_is_described_with_what_can_be_done_to_it() { - let found = Resource { - id: "r-wiki".into(), - provisioner: "docker/container".into(), - kind: "deployment".into(), - name: "wiki".into(), - state: "running".into(), - external_id: "wiki".into(), - secrets: vec![secret("wiki_db:DB_PASSWORD")], - spec_sha256: "ab12".into(), - verbs: vec!["logs".into()], - ..Resource::default() - }; - let actions = [Action { - name: "remove".into(), - label: "Remove the deployment".into(), - ..Action::default() - }]; - let text = describe(&found, &actions); - for line in [ - "r-wiki deployment wiki of docker/container\n", - "State running\n", - "Secrets wiki_db as DB_PASSWORD\n", - " remove Remove the deployment\n", - "Its log: berth resource r-wiki logs [--follow]\n", - ] { - assert!(text.contains(line), "{line:?} in\n{text}"); - } - assert_eq!(secret("token").variable(), "token"); - assert!(line(&found).starts_with("r-wiki deployment wiki")); - } -} diff --git a/crates/berth/src/serve/page.rs b/crates/berth/src/serve/page.rs deleted file mode 100644 index 7d2b029..0000000 --- a/crates/berth/src/serve/page.rs +++ /dev/null @@ -1,51 +0,0 @@ -//! berth in a browser: the page built from `crates/berth-web/`, carried in the program. -//! It is served to anyone who asks, since it holds nothing: what it shows it -//! gets through the portal, with the token its user gives it. - -use axum::{ - Router, - http::{HeaderValue, header}, - response::{Html, IntoResponse, Response}, - routing::get, -}; - -const BUILD: &str = env!("BERTH_WEB_BUILD"); -const PAGE: &str = include_str!("../../../berth-web/index.html"); -const SCRIPT: &str = include_str!(concat!(env!("OUT_DIR"), "/web/berth-web.js")); -const WASM: &[u8] = include_bytes!(concat!(env!("OUT_DIR"), "/web/berth-web_bg.wasm")); - -/// Script and wasm are asked for by the build they are of, so they can be kept. -const KEPT: &str = "public, max-age=31536000, immutable"; - -fn file(kind: &'static str, body: &'static [u8]) -> Response { - ( - [ - (header::CONTENT_TYPE, HeaderValue::from_static(kind)), - (header::CACHE_CONTROL, HeaderValue::from_static(KEPT)), - ], - body, - ) - .into_response() -} - -async fn page() -> Response { - ( - [(header::CACHE_CONTROL, HeaderValue::from_static("no-cache"))], - // `self`: this server is the one the page is of. - Html(PAGE.replace("@BUILD@", BUILD).replace("@SERVER@", "self")), - ) - .into_response() -} - -pub fn router() -> Router { - Router::new() - .route("/", get(page)) - .route( - "/berth-web.js", - get(|| async { file("text/javascript; charset=utf-8", SCRIPT.as_bytes()) }), - ) - .route( - "/berth-web_bg.wasm", - get(|| async { file("application/wasm", WASM) }), - ) -} diff --git a/crates/berth/src/serve/tests/remote.rs b/crates/berth/src/serve/tests/remote.rs deleted file mode 100644 index f782dc1..0000000 --- a/crates/berth/src/serve/tests/remote.rs +++ /dev/null @@ -1,112 +0,0 @@ -//! berth-remote against a berth serve of its own: a setup key signs it in, -//! and its portal answers. - -use super::*; -use berth_page::device::{Kept, Linked}; -use berth_page::portal::{Link, Portal}; -use berth_remote::{device::Terminal, settings::Settings}; -use berth_tui::{Code, Event, Key}; -use ratatui::{Terminal as Screen, backend::TestBackend}; -use std::cell::RefCell; -use std::rc::Rc; - -fn screen(app: &mut berth_page::App) -> String { - let mut terminal = Screen::new(TestBackend::new(120, 30)).unwrap(); - terminal.draw(|frame| app.draw(frame)).unwrap(); - let buffer = terminal.backend().buffer(); - (0..30) - .map(|y| { - (0..120) - .map(|x| buffer[(x, y)].symbol()) - .collect::() - }) - .collect::>() - .join("\n") -} - -/// Take in what the network says until `done`, for up to ten seconds. -fn until(what: &str, mut done: impl FnMut() -> bool) { - let started = std::time::Instant::now(); - while !done() { - assert!( - started.elapsed() < Duration::from_secs(10), - "no {what} in time" - ); - berth_remote::pump(); - std::thread::sleep(Duration::from_millis(20)); - } -} - -#[tokio::test(flavor = "multi_thread")] -async fn berth_remote_signs_in_with_a_setup_key_and_uses_the_portal() { - let dir = tempfile::tempdir().unwrap(); - let state = state_in(dir.path(), Some("s3cret-token")); - let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); - let url = format!("http://{}", listener.local_addr().unwrap()); - let router = api::router(state.clone(), true); - tokio::spawn(async move { axum::serve(listener, router).await }); - let (key, _) = super::super::setup_key::make(&state.paths).unwrap(); - - let client = tempfile::tempdir().unwrap(); - let kept = client.path().join("berth-remote"); - let linked = Linked { - server: Some(url.clone()), - token: None, - setup_key: Some(key), - }; - let terminal = Terminal::new(linked, Settings::new(Some(kept.clone())), false).unwrap(); - berth_page::device::install(terminal); - - // The key collects the token, which is kept for the next run. - let mut app = berth_page::App::new(); - until("token", || { - app.tick(); - screen(&mut app).contains("is installed and serves at") - }); - let settings = Settings::new(Some(kept.clone())).read(); - assert_eq!(settings["token"], "s3cret-token"); - assert_eq!(settings["server"], url.as_str()); - - // Done ends the key and shows the hosts of the server. - app.event(Event::Key(Key { - code: Code::Enter, - ctrl: false, - alt: false, - shift: false, - })); - until("end of the key", || { - app.tick(); - state - .store - .read(kind::TOKEN, store::OWN, super::super::setup_key::NAME) - .unwrap() - .is_none() - }); - until("hosts", || { - app.tick(); - let shown = screen(&mut app); - shown.contains("Hosts") && !shown.contains("is installed") - }); - - // The portal answers with the token, and a wrong one is refused. - let portal = Portal::open(&berth_page::device::portal_url(&url), "s3cret-token").unwrap(); - let hosts = Rc::new(RefCell::new(None)); - let answer = hosts.clone(); - portal.call("hosts.list", None, json!({}), Value::Null, move |result| { - *answer.borrow_mut() = Some(result) - }); - until("answer", || hosts.borrow().is_some()); - assert_eq!(portal.link(), Link::Open); - assert_eq!(hosts.borrow_mut().take().unwrap(), Ok(json!([]))); - let refused = Portal::open(&berth_page::device::portal_url(&url), "wrong").unwrap(); - until("refusal", || refused.link() != Link::Connecting); - assert_eq!(refused.link(), Link::Refused); - - // Forgetting removes what was kept. - let forgot = Terminal::new(Linked::default(), Settings::new(Some(kept.clone())), true).unwrap(); - assert!(!kept.exists()); - assert_eq!( - berth_page::device::Device::stored(&forgot, Kept::Token), - None - ); -} diff --git a/crates/berth/src/session.rs b/crates/berth/src/session.rs deleted file mode 100644 index 49f12ce..0000000 --- a/crates/berth/src/session.rs +++ /dev/null @@ -1,206 +0,0 @@ -//! A command that needs a terminal, run inside the UI: a shell on a host, or a -//! sign-in that shows a link and waits for a code. It gets a pseudo-terminal of -//! its own; what it draws there is kept here and shown in a panel. - -use crate::Result; -use crossterm::event::KeyEvent; -use std::{ - fs::File, - io::{Read, Write}, - process::{Child, Command}, - sync::mpsc::{Receiver, channel}, -}; - -/// Lines kept above the screen, for scrolling back. -const HISTORY: usize = 2000; - -pub struct Session { - pub title: String, - /// Close the panel when the command ends well, as for a shell. - pub closes: bool, - parser: vt100::Parser, - child: Child, - /// Our end of the pseudo-terminal. - terminal: File, - receiver: Receiver>, - /// The last bytes of output, for a question that arrives in two pieces. - tail: Vec, - /// Whether it succeeded, once it has ended. - pub outcome: Option, -} - -impl Session { - pub fn start(title: &str, command: Command, rows: u16, cols: u16) -> Result { - let (ours, child) = berth_pty::spawn(command, rows, cols)?; - let (sender, receiver) = channel(); - let mut reader = ours.try_clone()?; - std::thread::spawn(move || { - let mut buffer = [0; 4096]; - while let Ok(count) = reader.read(&mut buffer) - && count > 0 - && sender.send(buffer[..count].to_vec()).is_ok() - {} - }); - Ok(Session { - title: title.to_owned(), - closes: false, - parser: vt100::Parser::new(rows, cols, HISTORY), - child, - terminal: ours, - receiver, - tail: vec![], - outcome: None, - }) - } - - /// Take in what the command has written and notice when it has ended. - pub fn tick(&mut self) { - while let Ok(bytes) = self.receiver.try_recv() { - self.parser.process(&bytes); - // Prompts ask where the cursor is and wait for the answer. - self.tail.extend_from_slice(&bytes); - let asked = self - .tail - .windows(4) - .filter(|window| window == b"\x1b[6n") - .count(); - let keep = self.tail.len().saturating_sub(3); - self.tail.drain(..keep); - for _ in 0..asked { - let (row, col) = self.screen().cursor_position(); - self.send(format!("\x1b[{};{}R", row + 1, col + 1).as_bytes()); - } - } - if self.outcome.is_none() - && let Ok(Some(status)) = self.child.try_wait() - { - self.outcome = Some(status.success()); - } - } - - pub fn running(&self) -> bool { - self.outcome.is_none() - } - - pub fn screen(&self) -> &vt100::Screen { - self.parser.screen() - } - - /// Give the terminal a new size; the command is told by the system. - pub fn resize(&mut self, rows: u16, cols: u16) { - if self.screen().size() != (rows, cols) { - self.parser.screen_mut().set_size(rows, cols); - berth_pty::resize(&self.terminal, rows, cols); - } - } - - /// Look further back (positive) or forward again, by `lines`. - pub fn scroll(&mut self, lines: isize) { - let now = self.screen().scrollback() as isize; - self.parser - .screen_mut() - .set_scrollback((now + lines).max(0) as usize); - } - - pub fn send(&mut self, bytes: &[u8]) { - if self.running() && !bytes.is_empty() { - self.parser.screen_mut().set_scrollback(0); - let _ = self.terminal.write_all(bytes); - } - } - - pub fn paste(&mut self, text: &str) { - if self.screen().bracketed_paste() { - self.send(format!("\x1b[200~{text}\x1b[201~").as_bytes()); - } else { - self.send(text.as_bytes()); - } - } - - /// What a terminal sends for a key, in the mode the command has asked for. - pub fn key(&mut self, key: &KeyEvent) { - let mut bytes = key_bytes(key); - if self.screen().application_cursor() && bytes.len() == 3 && bytes.starts_with(b"\x1b[") { - bytes[1] = b'O'; - } - self.send(&bytes); - } -} - -impl Drop for Session { - /// Interrupt what still runs, as Ctrl+C would: through sudo on a host a - /// hang-up alone does not reach it. Then end the command. - fn drop(&mut self) { - if self.running() { - self.send(&[3]); - std::thread::sleep(std::time::Duration::from_millis(200)); - } - let _ = self.child.kill(); - let _ = self.child.wait(); - } -} - -/// What a terminal sends for a key. -pub fn key_bytes(key: &KeyEvent) -> Vec { - crate::keys::key(key).map_or(vec![], |key| berth_tui::event::key_bytes(&key)) -} - -#[cfg(test)] -mod tests { - use super::*; - use crossterm::event::{KeyCode, KeyModifiers}; - - #[test] - fn a_session_shows_output_and_takes_input() { - let mut command = Command::new("sh"); - command.args([ - "-c", - "printf 'code? '; read code; printf 'got %s\\r\\n' \"$code\"", - ]); - let mut session = Session::start("demo", command, 5, 40).unwrap(); - let wait = |session: &mut Session, text: &str| { - for _ in 0..200 { - session.tick(); - if session.screen().contents().contains(text) { - return; - } - std::thread::sleep(std::time::Duration::from_millis(10)); - } - panic!("no {text:?} in {:?}", session.screen().contents()); - }; - wait(&mut session, "code?"); - assert!(session.running()); - for key in [KeyCode::Char('4'), KeyCode::Char('2'), KeyCode::Enter] { - session.key(&KeyEvent::new(key, KeyModifiers::NONE)); - } - wait(&mut session, "got 42"); - for _ in 0..200 { - session.tick(); - if !session.running() { - break; - } - std::thread::sleep(std::time::Duration::from_millis(10)); - } - assert_eq!(session.outcome, Some(true)); - // A question about the cursor position is answered. - let mut command = Command::new("sh"); - command.args([ - "-c", - "stty raw -echo; printf 'ab\\033[6n'; dd bs=1 count=6 2>/dev/null | od -c", - ]); - let mut session = Session::start("demo", command, 5, 40).unwrap(); - wait(&mut session, "[ 1 ; 3"); - // The command sees the size, and a new one. - let mut command = Command::new("sh"); - command.args(["-c", "stty size; read go; stty size"]); - let mut session = Session::start("demo", command, 5, 40).unwrap(); - wait(&mut session, "5 40"); - session.resize(7, 50); - session.send(b"\r"); - wait(&mut session, "7 50"); - let alt = KeyEvent::new(KeyCode::Char('b'), KeyModifiers::ALT); - assert_eq!(key_bytes(&alt), b"\x1bb"); - let control = KeyEvent::new(KeyCode::Char('c'), KeyModifiers::CONTROL); - assert_eq!(key_bytes(&control), vec![3]); - } -} diff --git a/crates/berth/src/ui.rs b/crates/berth/src/ui.rs deleted file mode 100644 index f0540fa..0000000 --- a/crates/berth/src/ui.rs +++ /dev/null @@ -1,5414 +0,0 @@ -//! Drawing and input handling for the wizard. - -use crate::{ - Result, - app::{App, Entry, Item, Row}, - fields::{Action, Field, Kind, Step}, - job::Job, - paths::Paths, - registry::Store, - theme, -}; -use berth_tui::provisioners::MAKES; -use berth_tui::screen::{ - Platform, Spots, clear_hints, dialog, draw_arguments, draw_footer, draw_header, draw_panel, - draw_question, draw_view, ellipsize, hint_at, inside, note_hints, pasted, with_links, words, -}; -use crossterm::event::{ - Event, KeyCode, KeyEvent, KeyEventKind, KeyModifiers, MouseButton, MouseEvent, MouseEventKind, -}; -use ratatui::{ - DefaultTerminal, Frame, - layout::{Constraint, Layout, Margin, Rect}, - style::Style, - text::{Line, Span}, - widgets::{Block, Clear, Gauge, List, ListItem, ListState, Padding, Paragraph, Wrap}, -}; -use serde_json::json; -use tui_input::{Input, InputRequest, backend::crossterm::EventHandler}; - -/// A clickable region of the last frame. -#[derive(Clone, Copy)] -enum Target { - Step(Step), - /// Index into the selectable rows. - Row(usize), - Suggestion(usize), - Button(KeyCode), - /// Text area of the editor, with its horizontal scroll. - Editor(usize), -} - -type Hits = Vec<(Rect, Target)>; - -enum Flow { - Continue, - Quit, - Run, -} - -fn buttons(app: &App) -> Vec<(&'static str, KeyCode)> { - let mut buttons = vec![]; - if app.path_selected() { - buttons.push(("[Browse F3]", KeyCode::F(3))); - } - if let Some(group) = app.selected_group() { - buttons.push(if app.group_all_on(group) { - ("[All off Space]", KeyCode::Char(' ')) - } else { - ("[All on Space]", KeyCode::Char(' ')) - }); - } - if app.selected_entry() == Some(Entry::Task("terminal")) { - buttons.push(if app.as_agent { - ("[As the SSH user Space]", KeyCode::Char(' ')) - } else { - ("[As the agent Space]", KeyCode::Char(' ')) - }); - } - if let Some(field) = app.selected_field() { - if field.can_generate() { - buttons.push(("[Generate F4]", KeyCode::F(4))); - } - if field.is_password_hash() { - buttons.push(("[Set password F5]", KeyCode::F(5))); - } - if field.uses_saved_wifi() && !app.privileged { - buttons.push(("[Unlock saved passwords F5]", KeyCode::F(5))); - } - } - buttons -} - -fn draw(frame: &mut Frame, app: &mut App, rows: &[Row], display: &mut ListState, hits: &mut Hits) { - hits.clear(); - clear_hints(); - frame.render_widget(theme::backdrop(), frame.area()); - let area = frame.area().inner(Margin { - horizontal: 2, - vertical: 1, - }); - let [header, _, panel, _, footer] = Layout::vertical([ - Constraint::Length(1), - Constraint::Length(1), - Constraint::Min(4), - Constraint::Length(1), - Constraint::Length(1), - ]) - .areas(area); - let host = app.place.title(); - // The overview has no tabs; the set-up flow under it does. - let setup = app.step != Step::Manage; - let labels = Step::SETUP.map(Step::label); - let current = Step::SETUP.iter().position(|step| *step == app.step); - let tabs = draw_header( - frame, - header, - if setup { &labels[..] } else { &[][..] }, - current.unwrap_or(0), - &if setup { - format!("{host} › set-up") - } else { - host.to_string() - }, - ); - hits.extend( - tabs.into_iter() - .zip(Step::SETUP) - .map(|(rect, step)| (rect, Target::Step(step))), - ); - let [left, _, right] = Layout::horizontal([ - Constraint::Percentage(62), - Constraint::Length(2), - Constraint::Min(0), - ]) - .areas(panel); - - let editing = app.editing.is_some(); - let [body, editor, controls] = Layout::vertical([ - Constraint::Min(1), - Constraint::Length(if editing { 3 } else { 0 }), - Constraint::Length(if editing { 0 } else { 1 }), - ]) - .areas(left); - let buttons = buttons(app); - if !editing { - let mut x = controls.x; - for &(label, code) in &buttons { - let rect = Rect::new( - x, - controls.y, - (label.len() as u16).min(controls.right().saturating_sub(x)), - 1, - ); - frame.render_widget( - Paragraph::new(label).style(Style::new().fg(theme::ACCENT)), - rect, - ); - hits.push((rect, Target::Button(code))); - x += label.len() as u16 + 2; - } - } - - // Right column: detected values for the selected field above its description. - let [choices, help] = Layout::vertical([ - Constraint::Length(if app.suggestions.is_empty() { - 0 - } else { - (app.suggestions.len() as u16 + 2).min(10) - }), - Constraint::Min(3), - ]) - .areas(right); - if !app.suggestions.is_empty() { - let block = theme::panel("Suggestions · → to choose", app.suggestion_focus); - let width = choices.width.saturating_sub(4) as usize; - let labels = app - .suggestions - .iter() - .map(|(label, _)| ellipsize(label, width)) - .collect::>(); - frame.render_stateful_widget( - List::new(labels) - .block(block) - .highlight_style(Style::new().bg(theme::RAISED)), - choices, - &mut app.suggestion_list, - ); - for index in app.suggestion_list.offset()..app.suggestions.len() { - let y = choices.y + 1 + (index - app.suggestion_list.offset()) as u16; - if y + 1 >= choices.bottom() { - break; - } - hits.push(( - Rect::new(choices.x + 1, y, choices.width.saturating_sub(2), 1), - Target::Suggestion(index), - )); - } - } - frame.render_widget( - Paragraph::new(crate::markdown::render(&app.help())) - .block(theme::panel("Information", false)) - .wrap(Wrap { trim: false }), - help, - ); - - // Left column: the rows of the step. The cursor only stops on selectable ones. - let positions = rows - .iter() - .enumerate() - .filter_map(|(position, row)| row.entry.map(|_| position)) - .collect::>(); - display.select( - app.list - .selected() - .and_then(|index| positions.get(index).copied()), - ); - frame.render_stateful_widget( - List::new(rows.iter().map(|row| ListItem::new(row.line.clone()))) - .highlight_symbol(theme::cursor()) - .highlight_style(Style::new().bg(theme::RAISED)), - body, - display, - ); - for (index, &position) in positions.iter().enumerate() { - if position < display.offset() { - continue; - } - let y = body.y + (position - display.offset()) as u16; - if y >= body.bottom() { - break; - } - hits.push((Rect::new(body.x, y, body.width, 1), Target::Row(index))); - } - - if let Some(input) = &app.editing { - let secret = app.password_entry - || app - .selected_field() - .is_some_and(|field| field.spec().kind == Kind::Secret); - let width = editor.width.saturating_sub(2).max(1) as usize; - let (cursor, scroll, value) = if secret { - let cursor = input.cursor(); - ( - cursor, - cursor.saturating_sub(width - 1), - "•".repeat(input.value().chars().count()), - ) - } else { - ( - input.visual_cursor(), - input.visual_scroll(width), - input.value().to_owned(), - ) - }; - let name = match (app.password_entry, app.selected_entry()) { - (true, _) => " New password ".to_owned(), - (false, Some(Entry::Machine(pick))) => berth_tui::machine::editor_title(pick, &host), - _ => " Edit ".to_owned(), - }; - let mut title = vec![Span::raw(name.clone())]; - let mut x = editor.x + 1 + name.len() as u16; - for &(label, code) in &buttons { - title.push(Span::styled( - format!("{label} "), - Style::new().fg(theme::ACCENT).bold(), - )); - hits.push(( - Rect::new( - x, - editor.y, - (label.len() as u16).min(editor.right().saturating_sub(x + 1)), - 1, - ), - Target::Button(code), - )); - x += label.len() as u16 + 1; - } - frame.render_widget( - Paragraph::new(value) - .scroll((0, scroll.min(u16::MAX as usize) as u16)) - .block( - Block::bordered() - .title(Line::from(title)) - .border_style(Style::new().fg(theme::ACCENT)), - ), - editor, - ); - hits.push(( - editor.inner(Margin { - horizontal: 1, - vertical: 1, - }), - Target::Editor(scroll), - )); - if editor.width > 2 && editor.height > 2 && !app.suggestion_focus { - frame.set_cursor_position(( - editor.x + 1 + (cursor - scroll).min(width - 1) as u16, - editor.y + 1, - )); - } - } - - let keys = if editing { - format!( - "Enter accept · ↑↓ field · Tab next tab · Esc cancel · {}", - berth_tui::screen::COPY_FIELD - ) - } else { - match app.step { - Step::Manage => "↑↓ select · Enter choose · x unset · Ctrl+S save · q hosts", - _ => "↑↓ select · Enter choose · 1-4 tab · x unset · Ctrl+S save · Esc overview", - } - .to_owned() - }; - draw_footer(frame, footer, &keys, &app.status, theme::ACCENT); - if let Some(picker) = &mut app.picker { - picker.draw(frame); - } - if let Some(job) = &app.job { - // A job of the machine is named by what it does, not by the install method. - let action = match app.machine_job { - true => job.title.clone(), - false => app.action.label().to_owned(), - }; - (app.links, app.words) = draw_job(frame, job, &action, &app.status, area); - } - if let Some(session) = &app.session { - (app.links, app.words) = draw_session(frame, session, &app.status); - } - if let Some(view) = &app.host.view { - (app.links, app.words) = draw_view(frame, view, &app.status, "Ctrl+Q"); - } - if let Some((question, _)) = &app.host.question { - draw_question(frame, question); - } - if let Some(question) = &app.machine.question { - draw_question(frame, question); - } - draw_arguments(frame, &app.host); - if app.choosing_drive { - let lines = app - .usb_drives - .iter() - .take(9) - .enumerate() - .map(|(index, (label, _))| Line::raw(ellipsize(&format!("{} {label}", index + 1), 56))) - .collect(); - dialog( - frame, - " Write to which USB drive? ", - " 1-9 choose · Esc cancel ", - lines, - ); - } - if let Some(question) = &app.confirm { - let lines = vec![ - Line::raw(ellipsize(question, 56)), - Line::styled( - "Everything on that drive will be lost.", - Style::new().fg(theme::WARN), - ), - ]; - dialog(frame, " Write to USB drive ", " y erase · n cancel ", lines); - } - if let Some(input) = &app.unlock { - let local = std::env::var("USER").unwrap_or_else(|_| "this user".into()); - let user = match (app.unlock_remote, app.text(Field::SshUser)) { - (false, _) => local, - (true, "") => format!("{local} on {}", app.text(Field::SshHost)), - (true, user) => format!("{user} on {}", app.text(Field::SshHost)), - }; - let kept = match app.unlock_remote { - true => "Kept until berth is closed. It is not stored.", - false => "Used once for sudo. It is not stored.", - }; - draw_unlock( - frame, - input, - app.unlock_failed, - app.unlock_reason, - &user, - kept, - ); - } - if let Some(qr) = &app.qr { - qr.draw(frame); - } -} - -/// Put text on the clipboard: with the desktop's own tool when there is one, -/// otherwise by asking the terminal (OSC 52). -fn copy_text(text: &str) -> bool { - use std::io::Write; - use std::process::{Command, Stdio}; - for (program, args) in [ - ("wl-copy", &[][..]), - ("xclip", &["-selection", "clipboard"][..]), - ("xsel", &["--clipboard", "--input"][..]), - ] { - if program == "wl-copy" && std::env::var_os("WAYLAND_DISPLAY").is_none() { - continue; - } - let child = Command::new(program) - .args(args) - .stdin(Stdio::piped()) - .stdout(Stdio::null()) - .stderr(Stdio::null()) - .spawn(); - if let Ok(mut child) = child { - let written = child - .stdin - .take() - .is_some_and(|mut stdin| stdin.write_all(text.as_bytes()).is_ok()); - // xclip keeps running to serve the selection; only wl-copy is waited for. - if written - && (program != "wl-copy" || child.wait().is_ok_and(|status| status.success())) - { - return true; - } - } - } - const ALPHABET: &[u8; 64] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"; - let mut encoded = String::new(); - for chunk in text.as_bytes().chunks(3) { - let bits = chunk.iter().enumerate().fold(0u32, |bits, (index, byte)| { - bits | (*byte as u32) << (16 - 8 * index) - }); - for index in 0..4 { - encoded.push(if index <= chunk.len() { - ALPHABET[(bits >> (18 - 6 * index) & 63) as usize] as char - } else { - '=' - }); - } - } - let mut out = std::io::stdout(); - write!(out, "\x1b]52;c;{encoded}\x07").is_ok() && out.flush().is_ok() -} - -/// What is on the desktop's clipboard, if it can be read. -fn paste_text() -> Option { - for (program, args) in [ - ("wl-paste", &["--no-newline"][..]), - ("xclip", &["-selection", "clipboard", "-o"][..]), - ("xsel", &["--clipboard", "--output"][..]), - ] { - if program == "wl-paste" && std::env::var_os("WAYLAND_DISPLAY").is_none() { - continue; - } - let output = std::process::Command::new(program) - .args(args) - .stdin(std::process::Stdio::null()) - .stderr(std::process::Stdio::null()) - .output(); - if let Ok(output) = output - && output.status.success() - { - return Some(String::from_utf8_lossy(&output.stdout).trim().to_owned()); - } - } - None -} - -/// Draw a running or finished job. Returns where its web addresses and its -/// words are, for opening and copying with the mouse. -fn draw_job(frame: &mut Frame, job: &Job, action: &str, note: &str, area: Rect) -> (Spots, Spots) { - let (keys, color) = match &job.outcome { - None => (" Esc cancel ", theme::ACCENT), - Some(Ok(())) if job.offers_usb(action, Action::Iso.label()) => { - (" w write to USB drive · Enter close ", theme::GOOD) - } - Some(Ok(())) => ( - " Enter close · Click opens a link · Right-click copies a value ", - theme::GOOD, - ), - Some(Err(_)) => (" Enter close · Right-click copies a value ", theme::BAD), - }; - // Output with a web address in it offers its QR code. - let shown = job.lines.iter().rev().take(area.height as usize); - let keys = match berth_tui::qr::any_link(shown.map(String::as_str)) { - true => format!(" {} · {} ", keys.trim(), berth_tui::qr::HINT), - false => keys.to_owned(), - }; - let title = if job.title == action { - format!(" {action} ") - } else { - format!(" {action} · {} ", job.title) - }; - note_hints( - keys.trim(), - area.x + 1, - area.bottom().saturating_sub(1), - true, - ); - let block = theme::popup(title.trim(), color) - .title_bottom(theme::hints(keys.trim())) - .title_bottom( - Line::styled( - if note.is_empty() { - String::new() - } else { - format!(" {note} ") - }, - Style::new().fg(theme::WARN), - ) - .right_aligned(), - ) - .padding(Padding::horizontal(1)); - let [meter, _, log] = Layout::vertical([ - Constraint::Length(2), - Constraint::Length(1), - Constraint::Min(0), - ]) - .areas(block.inner(area)); - frame.render_widget(Clear, area); - frame.render_widget(block, area); - match &job.outcome { - None => match job.progress { - Some(ratio) => frame.render_widget( - Gauge::default() - .ratio(ratio.clamp(0.0, 1.0)) - .gauge_style(Style::new().fg(theme::ACCENT).bg(theme::RAISED)), - Rect { height: 1, ..meter }, - ), - None => frame.render_widget( - Paragraph::new("Working…").style(Style::new().fg(theme::MUTED)), - meter, - ), - }, - Some(Ok(())) => frame.render_widget( - Paragraph::new(format!("✓ Done. {}", job.next)) - .wrap(Wrap { trim: true }) - .style(Style::new().fg(theme::GOOD)), - meter, - ), - Some(Err(error)) => frame.render_widget( - Paragraph::new(format!("✗ {error}")) - .wrap(Wrap { trim: true }) - .style(Style::new().fg(theme::BAD)), - meter, - ), - } - let first = job.lines.len().saturating_sub(log.height as usize); - let (mut links, mut copyable) = (vec![], vec![]); - let lines: Vec = job.lines[first..] - .iter() - .enumerate() - .map(|(row, line)| { - let mut x = log.x; - let shown = ellipsize(line, log.width as usize); - for (column, word) in words(&shown) { - if !word.ends_with('…') { - let width = Span::raw(word.as_str()).width() as u16; - copyable.push(( - Rect::new(log.x + column, log.y + row as u16, width, 1), - word, - )); - } - } - let spans = with_links(&shown) - .into_iter() - .map(|(text, link)| { - let span = if link { - Span::styled(text.clone(), Style::new().fg(theme::ACCENT).underlined()) - } else { - Span::raw(text.clone()) - }; - let width = span.width() as u16; - // A shortened address would open the wrong page. - if link && !text.ends_with('…') { - links.push((Rect::new(x, log.y + row as u16, width, 1), text)); - } - x += width; - span - }) - .collect::>(); - Line::from(spans) - }) - .collect(); - frame.render_widget( - Paragraph::new(lines).style(Style::new().fg(theme::MUTED)), - log, - ); - (links, copyable) -} - -/// Draw a command's terminal. Returns where its web addresses and its words -/// are, for opening and copying with the mouse. -fn draw_session( - frame: &mut Frame, - session: &crate::session::Session, - note: &str, -) -> (Spots, Spots) { - let keys = match session.outcome { - None => { - " Ctrl+V paste · Ctrl+Q close · Click opens a link · Right-click copies · Wheel scrolls " - } - Some(true) => " ✓ Done · Enter close ", - Some(false) => " ✗ It did not finish · Enter close ", - }; - draw_panel( - frame, - &session.title, - session.screen(), - session.outcome, - keys, - note, - ) -} - -/// The desktop this terminal runs on: its clipboard, and its way to open a link. -struct Desktop; - -impl Platform for Desktop { - fn copy(&mut self, text: &str) -> bool { - copy_text(text) - } - - fn paste(&mut self) -> Option { - paste_text() - } - - fn open(&mut self, link: &str) -> std::result::Result<(), String> { - std::process::Command::new("xdg-open") - .arg(link) - .stdin(std::process::Stdio::null()) - .stdout(std::process::Stdio::null()) - .stderr(std::process::Stdio::null()) - .spawn() - .map(drop) - .map_err(|error| error.to_string()) - } -} - -/// A click on a key hint is that key. -fn hint_click(event: Event) -> Event { - if let Event::Mouse(mouse) = &event - && mouse.kind == MouseEventKind::Down(MouseButton::Left) - && let Some(key) = hint_at(mouse.column, mouse.row) - { - return Event::Key(crate::keys::native(key)); - } - event -} - -/// Alt+R over a panel shows its links as QR codes, and the overlay takes -/// every key while it is open. -fn qr_event(app: &mut App, event: &Event) -> bool { - let Some(event) = crate::keys::event(event) else { - return app.qr.is_some(); - }; - let mut said = None; - let taken = berth_tui::qr::take(&mut app.qr, &app.links, &event, &mut said); - if let Some(said) = said { - app.status = said; - } - taken -} - -/// A click on a web address opens it here; a right-click on a word copies it. -fn spot_event(app: &mut App, event: &Event) -> bool { - let said = crate::keys::event(event).and_then(|event| { - berth_tui::screen::spot_event(&app.links, &app.words, &event, &mut Desktop) - }); - match said { - Some(said) => { - if !said.is_empty() { - app.status = said; - } - true - } - None => false, - } -} - -fn session_event(app: &mut App, event: &Event) { - if qr_event(app, event) || spot_event(app, event) { - return; - } - let Some(session) = &mut app.session else { - return; - }; - match event { - Event::Mouse(mouse) if mouse.kind == MouseEventKind::ScrollUp => session.scroll(3), - Event::Mouse(mouse) if mouse.kind == MouseEventKind::ScrollDown => session.scroll(-3), - Event::Paste(text) => { - session.paste(text); - app.status = pasted(text); - } - Event::Key(key) if key.kind != KeyEventKind::Release => { - let control = key.modifiers.contains(KeyModifiers::CONTROL); - if control && key.code == KeyCode::Char('q') { - app.session = None; - } else if control && key.code == KeyCode::Char('v') { - // Terminals that keep Ctrl+V to themselves paste by their own means. - app.status = match paste_text() { - Some(text) => { - session.paste(&text); - pasted(&text) - } - None => "Could not read the clipboard".into(), - }; - } else if session.running() { - session.key(key); - } else if matches!(key.code, KeyCode::Enter | KeyCode::Esc | KeyCode::Char('q')) { - let title = session.title.clone(); - app.status = match session.outcome { - Some(true) => format!("{title} finished"), - _ => format!("{title} did not finish"), - }; - app.session = None; - } - } - _ => {} - } -} - -/// Keys and the mouse while a session of the host is in the panel. -fn view_event(app: &mut App, event: &Event) { - if qr_event(app, event) { - return; - } - if let Some(event) = crate::keys::event(event) - && let Some(said) = berth_tui::screen::view_event( - &mut app.host, - &app.links, - &app.words, - &event, - &mut Desktop, - ) - { - app.status = said; - } -} - -/// The yes-or-no question before a task or before applying. -fn question_event(app: &mut App, event: &Event) -> Flow { - let Event::Key(key) = event else { - return Flow::Continue; - }; - match key.code { - KeyCode::Char('y' | 'Y') => match app.host.question.take() { - Some((_, crate::remote::Pending::Action(index))) => app.begin_task(index, true), - Some((_, crate::remote::Pending::Apply)) => return apply(app, true), - None => {} - }, - KeyCode::Char('n' | 'N') | KeyCode::Esc => app.host.question = None, - _ => {} - } - Flow::Continue -} - -/// The dialog that asks for a task's arguments. -fn arguments_event(app: &mut App, event: &Event, platform: &mut dyn Platform) { - let Some(event) = crate::keys::event(event) else { - return; - }; - let mut said = None; - let accepted = berth_tui::screen::arguments_event(&mut app.host, &event, platform, &mut said); - if let Some(said) = said { - app.status = said; - } - if accepted { - app.accept_argument(); - } -} - -/// Applying the configuration is the SSH install, run from the overview. It -/// may replace the host's controller, which ends the sessions running there. -fn apply(app: &mut App, confirmed: bool) -> Flow { - let running = app - .host - .sessions - .iter() - .filter(|session| session.running()) - .count(); - if running > 0 && !confirmed { - let question = format!( - "{running} session{} on the host. Applying can restart berth's controller there, which ends them. Apply anyway?", - if running == 1 { - " is running" - } else { - "s are running" - } - ); - app.host.question = Some((question, crate::remote::Pending::Apply)); - return Flow::Continue; - } - app.action = Action::Ssh; - let problems = app.problems(); - app.action = Action::Manage; - if problems.is_empty() { - app.terminal_action = Some(Action::Ssh); - // What the host can do may be different afterwards. - app.host.disconnect(); - return Flow::Run; - } - app.status = problems[0].clone(); - Flow::Continue -} - -fn job_event(app: &mut App, event: &Event) { - if qr_event(app, event) || spot_event(app, event) { - return; - } - let (Some(job), Event::Key(key)) = (&mut app.job, event) else { - return; - }; - if key.kind == KeyEventKind::Release { - return; - } - let interrupt = key.code == KeyCode::Char('c') && key.modifiers.contains(KeyModifiers::CONTROL); - if job.running() { - if key.code == KeyCode::Esc || interrupt { - job.cancel(); - } - } else if key.code == KeyCode::Char('w') - && job.offers_usb(app.action.label(), Action::Iso.label()) - { - app.write_built_iso(None); - } else if matches!(key.code, KeyCode::Enter | KeyCode::Esc | KeyCode::Char('q')) || interrupt { - app.close_job(); - } -} - -fn confirm_event(app: &mut App, event: &Event) { - let Event::Key(key) = event else { - return; - }; - match key.code { - KeyCode::Char('y' | 'Y') => { - app.confirm = None; - app.run(true); - } - KeyCode::Char('n' | 'N') | KeyCode::Esc => { - app.confirm = None; - app.write_only = false; - } - _ => {} - } -} - -fn draw_unlock( - frame: &mut Frame, - input: &Input, - failed: bool, - reason: &str, - user: &str, - kept: &'static str, -) { - let typed = input.value().chars().count().min(54); - let note = if failed { - Line::styled("Wrong password. Try again.", Style::new().fg(theme::BAD)) - } else { - Line::styled(kept, Style::new().fg(theme::MUTED)) - }; - let lines = vec![ - Line::raw(reason.to_owned()), - note, - Line::raw(""), - Line::from(vec![ - Span::styled("› ", Style::new().fg(theme::ACCENT)), - Span::raw("•".repeat(typed)), - ]), - ]; - let title = ellipsize(&format!(" Password for {user} "), 58); - let area = dialog( - frame, - &title, - " Enter unlock · Esc cancel · Ctrl+O copy ", - lines, - ); - if area.width > 6 && area.height > 5 { - frame.set_cursor_position((area.x + 4 + typed as u16, area.y + 4)); - } -} - -fn unlock_event(app: &mut App, event: &Event, platform: &mut dyn Platform) { - let Some(input) = &mut app.unlock else { - return; - }; - match event { - Event::Key(key) if key.kind != KeyEventKind::Release => match key.code { - KeyCode::Char('o') if key.modifiers.contains(KeyModifiers::CONTROL) => { - app.status = berth_tui::screen::copy_field(platform, input); - } - KeyCode::Esc => { - app.unlock = None; - app.unlock_remote = false; - app.pending_run = false; - app.write_only = false; - } - KeyCode::Enter => app.submit_unlock(), - KeyCode::Char('u') if key.modifiers.contains(KeyModifiers::CONTROL) => input.reset(), - _ => { - input.handle_event(event); - } - }, - Event::Paste(text) => { - for ch in text.chars().filter(|ch| !ch.is_control()) { - input.handle(InputRequest::InsertChar(ch)); - } - } - _ => {} - } -} - -fn picker_event(app: &mut App, event: &Event) { - let Some(picker) = &mut app.picker else { - return; - }; - let mut choose = false; - let mut selected = None; - match event { - Event::Key(key) if key.kind != KeyEventKind::Release => match key.code { - KeyCode::Esc => { - app.picker = None; - return; - } - KeyCode::Up => picker.move_by(true), - KeyCode::Down => picker.move_by(false), - KeyCode::Enter if key.modifiers.contains(KeyModifiers::CONTROL) => { - selected = Some(picker.directory.display().to_string()) - } - KeyCode::Enter => choose = true, - _ => {} - }, - Event::Mouse(mouse) if inside(picker.area, mouse.column, mouse.row) => match mouse.kind { - MouseEventKind::ScrollUp => picker.move_by(true), - MouseEventKind::ScrollDown => picker.move_by(false), - MouseEventKind::Down(MouseButton::Left) - if mouse.row > picker.area.y && mouse.row + 1 < picker.area.bottom() => - { - let index = picker.state.offset() + (mouse.row - picker.area.y - 1) as usize; - if index < picker.entries.len() { - picker.state.select(Some(index)); - choose = true; - } - } - _ => {} - }, - _ => {} - } - if choose { - match picker.choose() { - Ok(value) => selected = value, - Err(error) => app.status = error.to_string(), - } - } - if let Some(path) = selected { - app.picker = None; - app.password_entry = false; - app.editing = Some(Input::new(path)); - app.accept_edit(); - } -} - -fn move_suggestion(app: &mut App, up: bool) { - let index = app.suggestion_list.selected().unwrap_or(0); - app.suggestion_list.select(Some(if up { - index.saturating_sub(1) - } else { - (index + 1).min(app.suggestions.len().saturating_sub(1)) - })); -} - -/// Handle a mouse event. Clicks on rows and buttons come back as the key they stand for. -fn mouse_event(app: &mut App, mouse: MouseEvent, hits: &Hits) -> Option { - let target = hits - .iter() - .rev() - .find(|(rect, _)| inside(*rect, mouse.column, mouse.row)) - .copied(); - let key = |code| Some(KeyEvent::new(code, KeyModifiers::NONE)); - match mouse.kind { - MouseEventKind::ScrollUp | MouseEventKind::ScrollDown => { - let up = mouse.kind == MouseEventKind::ScrollUp; - if matches!(target, Some((_, Target::Suggestion(_)))) { - move_suggestion(app, up); - app.suggestion_focus = true; - } else { - app.accept_edit(); - app.move_selection(up); - } - None - } - MouseEventKind::Down(MouseButton::Left) => match target? { - (_, Target::Step(step)) => { - app.accept_edit(); - app.set_step(step); - None - } - (_, Target::Row(index)) => { - app.accept_edit(); - app.list.select(Some(index)); - app.suggestion_focus = false; - key(KeyCode::Enter) - } - (_, Target::Suggestion(index)) => { - app.suggestion_list.select(Some(index)); - app.select_suggestion(); - None - } - (_, Target::Button(code)) => key(code), - (rect, Target::Editor(scroll)) => { - if let Some(input) = &mut app.editing { - let column = (mouse.column - rect.x) as usize + scroll; - let mut width = 0; - let cursor = input - .value() - .chars() - .take_while(|ch| { - width += Span::raw(ch.to_string()).width(); - width <= column - }) - .count(); - input.handle(InputRequest::SetCursor(cursor)); - } - None - } - }, - _ => None, - } -} - -fn choose(app: &mut App, entry: Entry) -> Flow { - match entry { - Entry::Action(action) => { - app.action = action.choose(&mut app.config); - berth_tui::setup::fit_profile(&mut app.config, app.action); - } - Entry::Field(field) => app.activate(field), - Entry::Setup => app.set_step(Step::Install), - Entry::HostRegistry(index) => app.toggle_host_registry(index), - Entry::HostSecret(index) => app.editing = Some(Input::new(app.host_secret(index))), - // A key is pasted whole, not changed letter by letter. - Entry::Key(..) => app.editing = Some(Input::default()), - Entry::Group(_) => app.status = "Space switches the whole group".into(), - Entry::Feature(index) => { - let item = Item::Feature(index); - app.set_item(item, !app.item_enabled(item)); - } - Entry::HostTask(index) => app.begin_task(index, false), - Entry::HostSession(index) => { - if let Some(session) = app.host.sessions.get(index).cloned() { - app.host.open(&session); - } - } - Entry::Task("terminal") => app.start_host_shell(), - Entry::Task("ssh") => { - let (rows, cols) = session_size(); - app.start_shell(rows, cols); - } - Entry::Task("connect") => app.reconnect_host(), - Entry::Task("address") => app.edit_address(), - Entry::Task("apply") => return apply(app, false), - Entry::Task("install-controller") => { - let _ = app.save(); - app.own_command = Some( - ["host", "install", &app.place.argument()] - .map(str::to_owned) - .to_vec(), - ); - } - Entry::Machine(pick) => app.machine_enter(pick), - Entry::Save => { - save(app); - } - Entry::Task(_) => {} - Entry::Run => { - if !app.problems().is_empty() { - app.status = "Fix the listed problems first".into(); - } else if app.leaves_ui() { - return Flow::Run; - } else { - app.write_only = false; - app.run(false); - } - } - } - Flow::Continue -} - -/// Save the profile and say how that went. -fn save(app: &mut App) -> bool { - let saved = app.save(); - let done = saved.is_ok(); - app.status = match saved { - Ok(()) => app.saved_note(), - Err(error) => format!("Save failed · {error}"), - }; - done -} - -/// The size of the terminal panel for the window as it is now. -fn session_size() -> (u16, u16) { - let (width, height) = crossterm::terminal::size().unwrap_or((100, 30)); - ( - height.saturating_sub(4).max(4), - width.saturating_sub(8).max(20), - ) -} - -fn key_event( - app: &mut App, - key: KeyEvent, - quit_armed: &mut bool, - platform: &mut dyn Platform, -) -> Flow { - let armed = std::mem::take(quit_armed); - let control = key.modifiers.contains(KeyModifiers::CONTROL); - match key.code { - KeyCode::Char('s') if control => { - app.accept_edit(); - save(app); - return Flow::Continue; - } - KeyCode::F(3) => { - app.open_picker(); - return Flow::Continue; - } - KeyCode::F(4) if app.selected_field().is_some_and(Field::can_generate) => { - app.cancel_edit(); - if let Err(error) = app.generate() { - app.status = error.to_string(); - } - return Flow::Continue; - } - KeyCode::F(5) if app.selected_field().is_some_and(Field::is_password_hash) => { - app.password_entry = true; - app.editing = Some(Input::default()); - return Flow::Continue; - } - KeyCode::F(5) - if !app.privileged && app.selected_field().is_some_and(Field::uses_saved_wifi) => - { - app.accept_edit(); - app.request_unlock("Read the Wi-Fi passwords saved on this machine."); - return Flow::Continue; - } - KeyCode::Tab | KeyCode::BackTab => { - app.accept_edit(); - let step = app - .step - .offset(if key.code == KeyCode::Tab { 1 } else { -1 }); - if step != app.step { - app.set_step(step); - } - return Flow::Continue; - } - KeyCode::F(2) if !app.suggestions.is_empty() => { - app.suggestion_focus = !app.suggestion_focus; - return Flow::Continue; - } - _ => {} - } - if app.suggestion_focus { - match key.code { - KeyCode::Up => move_suggestion(app, true), - KeyCode::Down => move_suggestion(app, false), - KeyCode::Enter => app.select_suggestion(), - KeyCode::Left | KeyCode::Esc => app.suggestion_focus = false, - _ => {} - } - return Flow::Continue; - } - if let Some(input) = &mut app.editing { - match key.code { - KeyCode::Esc => { - app.retry_address = false; - app.cancel_edit(); - } - KeyCode::Enter => { - let retry = std::mem::take(&mut app.retry_address); - app.accept_edit(); - if retry && save(app) { - // Back on the overview, which asks the host at its new address. - app.set_step(Step::Manage); - app.reconnect_host(); - } - } - KeyCode::Up | KeyCode::Down => { - app.accept_edit(); - app.move_selection(key.code == KeyCode::Up); - } - KeyCode::Char('u') if control => input.reset(), - KeyCode::Char('o') if control => { - app.status = berth_tui::screen::copy_field(platform, input); - } - _ => { - input.handle_event(&Event::Key(key)); - } - } - return Flow::Continue; - } - match key.code { - KeyCode::Up => app.move_selection(true), - KeyCode::Down => app.move_selection(false), - KeyCode::Right if !app.suggestions.is_empty() => app.suggestion_focus = true, - KeyCode::Char(digit @ '1'..='9') => { - if app.step != Step::Manage - && let Some(&step) = Step::SETUP.get(digit as usize - '1' as usize) - { - app.set_step(step); - } - } - KeyCode::Char(' ') => match app.selected_entry() { - Some(Entry::Group(group)) => app.toggle_group(group), - Some(Entry::Task("terminal")) => app.as_agent = !app.as_agent, - Some( - entry @ (Entry::Field(_) - | Entry::Feature(..) - | Entry::HostRegistry(_) - | Entry::Machine( - berth_tui::machine::Pick::Provisioner | berth_tui::machine::Pick::Argument(_), - )), - ) => { - return choose(app, entry); - } - _ => {} - }, - KeyCode::Char('x') | KeyCode::Delete => { - if let Some(Entry::Key(account, Some(index))) = app.selected_entry() - && berth_tui::setup::set_key(&mut app.config, account, Some(index), "").is_ok() - { - app.status = "SSH key removed".into(); - } - if let Some(Entry::HostSecret(index)) = app.selected_entry() { - app.set_host_secret(index, ""); - } - if let Some(Entry::HostSession(index)) = app.selected_entry() - && let Some(session) = app.host.sessions.get(index) - { - if session.running() { - app.status = "It is still running. Open it and end it first.".into(); - } else { - app.status = format!("Forgot {}", session.title); - app.host - .send(crate::remote::Request::Delete(session.id.clone())); - } - } - if let Some(field) = app.selected_field() { - app.clear(field); - } - } - // Leaving the set-up flow returns to the host's overview. - KeyCode::Char('q') | KeyCode::Esc if app.step != Step::Manage => app.set_step(Step::Manage), - KeyCode::Char('q') | KeyCode::Esc => { - if armed || !app.unsaved() { - return Flow::Quit; - } - *quit_armed = true; - app.status = "Unsaved changes · Ctrl+S saves, q again discards".into(); - } - KeyCode::Enter => { - if let Some(entry) = app.selected_entry() { - return choose(app, entry); - } - } - _ => {} - } - Flow::Continue -} - -/// What the home screen is showing. -enum HomeMode { - Hosts, - Naming(Input), - /// The selected setting, and its editor while it is being changed. - Settings(usize, Option), - Registries(RegistryView), - Secrets(SecretView), - Plugins(PluginView), - Provisioners(Box), -} - -/// A clickable region of the home screen. -#[derive(Clone, Copy)] -enum HomeTarget { - /// Hosts, Settings or Registries. - Tab(usize), - Row(usize), -} - -/// The Registries tab: the selected row, its editor while something is typed, -/// and whether a registry's removal waits for a second x. -#[derive(Default)] -struct RegistryView { - selected: usize, - editor: Option, - armed: bool, - /// The result of the last check, shown until the selection moves. - report: Vec, -} - -/// A row of the Registries tab. Registries are named, repositories numbered. -#[derive(Clone, PartialEq, Debug)] -enum RegistryRow { - Name(String), - Type(String), - Address(String), - User(String), - Password(String), - Repository(String, usize), - Format(String, usize), - DockerAddress(String, usize), - AddRepository(String), - AddRegistry, -} - -impl RegistryRow { - /// The registry a row belongs to. - fn registry(&self) -> Option<&str> { - use RegistryRow::*; - match self { - Name(name) - | Type(name) - | Address(name) - | User(name) - | Password(name) - | Repository(name, _) - | Format(name, _) - | DockerAddress(name, _) - | AddRepository(name) => Some(name), - AddRegistry => None, - } - } - - fn help(&self) -> &'static str { - use RegistryRow::*; - match self { - Name(_) => { - "A registry: a server packages, files or images come from and go to. Enter renames it, x removes it with its stored login. Hosts get a registry with: berth host HOST registry NAME on." - } - Type(_) => { - "Enter switches the kind. Sonatype Nexus: one server and login with repositories under it. npm registry: any npm-compatible registry at this address. File server: an address files are uploaded to with HTTP PUT, for the release archive. Docker registry: any registry for container images." - } - Address(_) => { - "Where the registry is. With https:// for Nexus, npm and file servers; a Docker registry is HOST or HOST:PORT." - } - User(_) => { - "The account to log in with. Leave it empty when the registry takes a token alone, such as an npm token. x clears it." - } - Password(_) => { - "The password or token. It is kept in berth's store with the secrets, and is never shown. x forgets it." - } - Repository(..) => { - "A repository on the Nexus server, by its name there. Enter renames it, x removes it from this list." - } - Format(..) => { - "Enter switches the format. npm: becomes the agent's npm registry. raw: a place to publish the release archive. docker: container images." - } - DockerAddress(..) => { - "Leave it empty when the repository uses path based routing in Nexus: images are then at HOST/REPOSITORY/IMAGE. With a connector on a port or host name of its own, enter that as HOST:PORT." - } - AddRepository(_) => { - "Add a repository of this Nexus server. Type its name; it starts as npm and Format changes that." - } - AddRegistry => { - "Add a registry. Type a short name for it (lower-case letters, digits, - and _), then set its kind and address." - } - } - } -} - -/// The rows of the Registries tab with what they show. -fn registry_rows(store: Option<&Store>) -> Vec<(RegistryRow, Line<'static>)> { - use RegistryRow::*; - let muted = Style::new().fg(theme::MUTED); - let field = |prefix: &str, label: &str, value: &str| { - let mut line = crate::app::value_line(label, value, Kind::Text); - line.spans.insert(0, Span::styled(prefix.to_owned(), muted)); - line - }; - let mut rows = vec![]; - let registries = store.and_then(|store| store.registries().ok()); - for (name, registry) in registries.unwrap_or_default() { - let text = |key: &str| registry[key].as_str().unwrap_or("").to_owned(); - let nexus = registry["type"] == "nexus"; - rows.push(( - Name(name.clone()), - Line::styled(name.clone(), Style::new().fg(theme::ACCENT).bold()), - )); - rows.push(( - Type(name.clone()), - field(" ├─ ", "Kind", crate::registry::type_label(&text("type"))), - )); - rows.push(( - Address(name.clone()), - field(" ├─ ", "Address", &text("url")), - )); - rows.push(( - User(name.clone()), - field(" ├─ ", "User name", &text("username")), - )); - let stored = store - .and_then(|store| store.has_secret(&format!("registry/{name}")).ok()) - .unwrap_or(false); - let mut password = Line::from(vec![ - Span::styled(if nexus { " ├─ " } else { " └─ " }, muted), - Span::styled("Password or token ", Style::new().fg(theme::TEXT)), - ]); - password.spans.push(match stored { - true => Span::styled("● stored", Style::new().fg(theme::GOOD)), - false => Span::styled("○ not set", muted), - }); - rows.push((Password(name.clone()), password)); - if !nexus { - continue; - } - let repositories = registry["repositories"] - .as_array() - .cloned() - .unwrap_or_default(); - for (index, repository) in repositories.iter().enumerate() { - let repo = repository["name"].as_str().unwrap_or(""); - let format = repository["format"].as_str().unwrap_or(""); - let mut line = field(" ├─ ", "Repository", repo); - if format != "docker" { - line.spans.push(Span::styled( - format!(" {}", crate::registry::repository_url(®istry, repo)), - muted, - )); - } - rows.push((Repository(name.clone(), index), line)); - let last = if format == "docker" { - " │ ├─ " - } else { - " │ └─ " - }; - rows.push((Format(name.clone(), index), field(last, "Format", format))); - if format == "docker" { - let line = match repository["address"] - .as_str() - .filter(|address| !address.is_empty()) - { - Some(address) => field(" │ └─ ", "Docker address", address), - None => Line::from(vec![ - Span::styled(" │ └─ ", muted), - Span::styled("Docker address ", Style::new().fg(theme::TEXT)), - Span::styled( - format!( - "Default · {} (path based routing)", - crate::registry::docker_address(®istry, repository) - ), - muted.italic(), - ), - ]), - }; - rows.push((DockerAddress(name.clone(), index), line)); - } - } - rows.push(( - AddRepository(name.clone()), - Line::from(vec![ - Span::styled(" └─ ", muted), - Span::styled("+ Add repository", Style::new().fg(theme::ACCENT)), - ]), - )); - } - rows.push(( - AddRegistry, - Line::styled("+ Add registry", Style::new().fg(theme::ACCENT)), - )); - rows -} - -/// The Plugins tab: its rows and editor, the plugins of this machine or of -/// the berth server as they were last read, and what the last change said. -#[derive(Default)] -struct PluginView { - tab: berth_tui::plugins::Tab, - editor: Option, - listing: Option>, - report: Vec, - /// The lines of a roll-out that runs, as they come. - rolling: Option>, -} - -/// The lines a roll-out writes, sent on one by one; only its own, not what -/// Ansible says. -struct RolloutLines { - sender: std::sync::mpsc::Sender, - pending: Vec, -} - -impl std::io::Write for RolloutLines { - fn write(&mut self, data: &[u8]) -> std::io::Result { - self.pending.extend_from_slice(data); - while let Some(end) = self.pending.iter().position(|byte| *byte == b'\n') { - let line: Vec = self.pending.drain(..=end).collect(); - let line = String::from_utf8_lossy(&line).trim_end().to_owned(); - if berth_tui::plugins::rollout_line(&line) { - let _ = self.sender.send(line); - } - } - Ok(data.len()) - } - - fn flush(&mut self) -> std::io::Result<()> { - Ok(()) - } -} - -impl PluginView { - /// Read the plugins, unless they were read since the last change. - fn load(&mut self, paths: &Paths) { - if self.listing.is_none() { - let listed = crate::plugins::Manager::new(paths).list(); - self.listing = Some(listed.map_err(|error| error.to_string())); - } - } - - fn plugins(&self) -> &[berth_api::PluginInfo] { - match &self.listing { - Some(Ok(listing)) => &listing.plugins, - _ => &[], - } - } - - /// Take in what a roll-out that runs has said since. - fn pump(&mut self, status: &mut String) { - let Some(lines) = &self.rolling else { - return; - }; - loop { - match lines.try_recv() { - Ok(line) => { - *status = line.clone(); - self.report.push(line); - } - Err(std::sync::mpsc::TryRecvError::Empty) => return, - Err(std::sync::mpsc::TryRecvError::Disconnected) => { - self.rolling = None; - return; - } - } - } - } - - /// Start a roll-out in the background, its lines shown as they come. - fn roll_out(&mut self, paths: &Paths, options: berth_api::Rollout) { - let (sender, receiver) = std::sync::mpsc::channel(); - let paths = paths.clone(); - self.report = vec!["Rolling out…".into()]; - self.tab.behind.clear(); - self.rolling = Some(receiver); - std::thread::spawn(move || { - let mut out = RolloutLines { - sender: sender.clone(), - pending: vec![], - }; - let said = match crate::commands::rollout(&paths, &options, false, &mut out) { - Ok(report) => crate::commands::rollout_summary(&report), - Err(error) => format!("✗ {error}"), - }; - for line in said.lines() { - let _ = sender.send(line.to_owned()); - } - }); - } - - /// Show what a change said, and read the plugins again. - fn changed(&mut self, done: Result>, status: &mut String) { - self.report = match done { - Ok(lines) => lines, - Err(error) => error.to_string().lines().map(str::to_owned).collect(), - }; - *status = berth_tui::plugins::status(&self.report); - self.listing = None; - } -} - -/// Where the plugins of the Plugins tab are: here, or on the berth server. -fn plugin_place(server: &Option) -> berth_tui::plugins::Place<'_> { - match server { - Some(server) => berth_tui::plugins::Place::Server(&server.url), - None => berth_tui::plugins::Place::Here, - } -} - -/// Handle a key on the Plugins tab. -fn plugins_key( - view: &mut PluginView, - paths: &Paths, - key: KeyEvent, - status: &mut String, -) -> Outcome { - use berth_tui::plugins::Asked; - view.load(paths); - let server = berth_core::server::Server::configured(paths); - let plugins = view.plugins().to_vec(); - let Some(shared) = crate::keys::key(&key) else { - return Outcome::Stay; - }; - let place = plugin_place(&server); - match view - .tab - .key(shared, &mut view.editor, &plugins, &mut view.report, &place) - { - Asked::Nothing(said) => { - if let Some(said) = said { - *status = said; - } - } - Asked::Install(spec) => { - let done = crate::plugins::Manager::new(paths).add(&spec); - view.changed(done, status); - } - Asked::Remove(id) => { - let done = crate::plugins::Manager::new(paths).remove(&id); - view.changed(done, status); - } - Asked::Check => match crate::plugins::Manager::new(paths).check() { - Ok(checked) => { - view.report = view.tab.checked(checked.plugins); - *status = berth_tui::plugins::status(&view.report); - } - Err(error) => view.changed(Err(error), status), - }, - Asked::Update(id) => { - let manager = crate::plugins::Manager::new(paths); - match manager.update(id.as_deref()) { - Ok(mut report) => { - if manager.server().is_none() { - report.behind = berth_core::rollout::survey(paths); - } - let lines = view.tab.updated(report); - view.changed(Ok(lines), status); - } - Err(error) => view.changed(Err(error), status), - } - } - Asked::Rollout(_) if view.rolling.is_some() => { - *status = "A roll-out runs already".into(); - } - Asked::Rollout(options) => view.roll_out(paths, options), - Asked::Other => match leave_tab(key, tabs::PLUGINS) { - Outcome::Stay => {} - outcome => return outcome, - }, - } - Outcome::Stay -} - -/// The Secrets tab: the selected row, the editor, the name a typed value is -/// for, and whether a removal waits for a second x. -#[derive(Default)] -struct SecretView { - selected: usize, - editor: Option, - /// A new secret's name, while its value is being typed. - naming: Option, - armed: bool, -} - -/// Handle a key on the Secrets tab. The row after the last secret adds one. -fn secrets_key( - view: &mut SecretView, - paths: &Paths, - key: KeyEvent, - status: &mut String, -) -> Outcome { - let store = match Store::new(paths) { - Ok(store) => store, - Err(error) => { - *status = error.to_string(); - return leave_tab(key, tabs::SECRETS); - } - }; - let names = store.secret_names(); - view.selected = view.selected.min(names.len()); - let selected = names.get(view.selected).cloned(); - if let Some(input) = &mut view.editor { - match key.code { - KeyCode::Esc => { - view.editor = None; - view.naming = None; - } - KeyCode::Enter => { - let typed = input.value().to_owned(); - // A name was typed for a new secret: its value comes next. - if selected.is_none() && view.naming.is_none() { - let name = typed.trim(); - if !crate::plugins::valid_name(name) { - *status = "Use lower-case letters, digits, - and _, starting with a letter" - .into(); - } else if names.iter().any(|known| known == name) { - *status = format!("There is a secret {name} already"); - } else { - view.naming = Some(name.to_owned()); - view.editor = Some(Input::default()); - *status = format!("Now the value of {name}"); - } - return Outcome::Stay; - } - let name = view.naming.take().or(selected).unwrap_or_default(); - view.editor = None; - if typed.is_empty() { - *status = "Nothing entered, kept as it was".into(); - return Outcome::Stay; - } - *status = match store.set_secret(&name, Some(&typed)) { - Ok(()) => format!("Stored {name}"), - Err(error) => error.to_string(), - }; - let listed = store.secret_names(); - view.selected = listed - .iter() - .position(|known| *known == name) - .unwrap_or(view.selected); - } - _ => { - input.handle_event(&Event::Key(key)); - } - } - return Outcome::Stay; - } - let armed = std::mem::take(&mut view.armed); - match key.code { - KeyCode::Up => view.selected = view.selected.saturating_sub(1), - KeyCode::Down => view.selected = (view.selected + 1).min(names.len()), - KeyCode::Enter => view.editor = Some(Input::default()), - KeyCode::Char('x') | KeyCode::Delete => match selected { - Some(name) if armed => { - *status = match store.set_secret(&name, None) { - Ok(()) => format!("Removed {name}"), - Err(error) => error.to_string(), - } - } - Some(name) => { - view.armed = true; - *status = format!("x again removes {name}"); - } - None => {} - }, - _ => return leave_tab(key, tabs::SECRETS), - } - Outcome::Stay -} - -/// The tabs of the home screen, in their order; a digit key is one more -/// than the index. -pub(crate) mod tabs { - pub const HOSTS: usize = 0; - pub const PROVISIONERS: usize = 1; - pub const SETTINGS: usize = 2; - pub const REGISTRIES: usize = 3; - pub const SECRETS: usize = 4; - pub const PLUGINS: usize = 5; - pub const NAMES: [&str; 6] = berth_tui::screen::HOME_TABS; -} - -/// The keys that leave a tab for another, the same on every tab: a digit, -/// Tab and Shift+Tab, Esc for the hosts, and q. `here` is the tab's index. -fn leave_tab(key: KeyEvent, here: usize) -> Outcome { - let count = tabs::NAMES.len(); - match key.code { - KeyCode::Char(digit @ '1'..='6') => Outcome::Tab(digit as usize - '1' as usize), - KeyCode::Esc => Outcome::Tab(tabs::HOSTS), - KeyCode::Tab => Outcome::Tab((here + 1) % count), - KeyCode::BackTab => Outcome::Tab((here + count - 1) % count), - KeyCode::Char('q') => Outcome::Quit, - _ => Outcome::Stay, - } -} - -/// Where the registries and secrets of `store` are kept, for the help of their tabs. -fn kept_in(paths: &Paths, store: &Store) -> String { - match store.server() { - Some(server) => format!( - "Kept on the berth server at {}. This berth has the registries and the names of the secrets in its store.", - server.url - ), - None => format!( - "Kept in berth's store, the {}, {}. See: berth store", - berth_core::store::Installation::of(paths).target.describe(), - match store.encrypted() { - true => "encrypted", - false => "not encrypted", - } - ), - } -} - -/// What a key on the Registries tab leads to. -enum Outcome { - Stay, - Tab(usize), - Quit, -} - -/// Take what was typed for a row. The message is shown in the footer. -fn registry_accept(store: &Store, row: &RegistryRow, value: &str) -> crate::Result { - use RegistryRow::*; - let mut registries = store.registries()?; - let valid = crate::plugins::valid_name; - let message = match row { - AddRegistry | Name(_) => { - if !valid(value) { - return Err( - "Use lower-case letters, digits, - and _, starting with a letter".into(), - ); - } - if registries.contains_key(value) { - return Err(format!("There is a registry {value} already").into()); - } - if let Name(old) = row { - // The stored login moves with the name, in the same change. - return Ok(match store.rename_registry(old, value)? { - true => format!("Registry {value}"), - false => format!( - "Registry {value}. The berth server drops a login with the old name: enter its password again" - ), - }); - } - registries.insert(value.to_owned(), json!({"type": "nexus", "url": ""})); - format!("Registry {value}") - } - Address(name) => { - let registry = &mut registries[name.as_str()]; - if registry["type"] != "docker" - && !value.starts_with("https://") - && !value.starts_with("http://") - { - return Err("Start the address with https://".into()); - } - registry["url"] = json!(value.trim_end_matches('/')); - "Address saved".into() - } - User(name) => { - match value { - "" => registries[name.as_str()] - .as_object_mut() - .and_then(|registry| registry.remove("username")), - user => registries[name.as_str()] - .as_object_mut() - .and_then(|registry| registry.insert("username".into(), json!(user))), - }; - "User name saved".into() - } - Password(name) => { - if value.is_empty() { - return Ok("Nothing entered, kept as it was".into()); - } - store.set_secret(&format!("registry/{name}"), Some(value))?; - return Ok("Stored".into()); - } - AddRepository(name) | Repository(name, _) => { - if value.is_empty() || value.contains(['/', ' ']) { - return Err("The repository's name on the server, without / or spaces".into()); - } - let registry = &mut registries[name.as_str()]; - let mut repositories = registry["repositories"] - .as_array() - .cloned() - .unwrap_or_default(); - if repositories.iter().any(|known| known["name"] == value) { - return Err(format!("{value} is listed already").into()); - } - match row { - Repository(_, index) => repositories[*index]["name"] = json!(value), - _ => repositories.push(json!({"format": "npm", "name": value})), - } - registry["repositories"] = json!(repositories); - format!("Repository {value}") - } - DockerAddress(name, index) => { - let address = value.split_once("://").map_or(value, |(_, rest)| rest); - let repository = &mut registries[name.as_str()]["repositories"][*index]; - match address.trim_end_matches('/') { - "" => repository - .as_object_mut() - .and_then(|repository| repository.remove("address")), - address => repository - .as_object_mut() - .and_then(|repository| repository.insert("address".into(), json!(address))), - }; - "Docker address saved".into() - } - Type(_) | Format(..) => String::new(), - }; - store.save_registries(registries)?; - Ok(message) -} - -/// Handle a key on the Registries tab. -fn registries_key( - view: &mut RegistryView, - paths: &Paths, - key: KeyEvent, - status: &mut String, -) -> Outcome { - use RegistryRow::*; - let store = match Store::new(paths) { - Ok(store) => store, - Err(error) => { - *status = error.to_string(); - return leave_tab(key, tabs::REGISTRIES); - } - }; - let rows = registry_rows(Some(&store)); - view.selected = view.selected.min(rows.len() - 1); - let row = rows[view.selected].0.clone(); - if let Some(input) = &mut view.editor { - match key.code { - KeyCode::Esc => view.editor = None, - KeyCode::Enter => { - let typed = input.value().trim().to_owned(); - match registry_accept(&store, &row, &typed) { - Ok(message) => { - *status = message; - view.editor = None; - // A new or renamed registry stays selected, ready for its kind and address. - if matches!(row, AddRegistry | Name(_)) { - let named = registry_rows(Some(&store)) - .iter() - .position(|(row, _)| *row == Name(typed.clone())); - view.selected = named.unwrap_or(view.selected); - } - } - Err(error) => *status = error.to_string(), - } - } - _ => { - input.handle_event(&Event::Key(key)); - } - } - return Outcome::Stay; - } - let armed = std::mem::take(&mut view.armed); - let report = std::mem::take(&mut view.report); - if key.code == KeyCode::Char('c') { - match row.registry() { - Some(name) => { - view.report = crate::registry::check(&store, name) - .unwrap_or_else(|error| vec![format!("✗ {error}")]); - view.report.insert(0, format!("Checked {name}")); - *status = format!("Checked {name}"); - } - None => *status = "Select a registry to check".into(), - } - return Outcome::Stay; - } - // Moving on to something else drops the report; other keys keep it on screen. - if !matches!(key.code, KeyCode::Up | KeyCode::Down) { - view.report = report; - } - let change = |edit: &dyn Fn(&mut serde_json::Map)| { - let mut registries = store.registries().unwrap_or_default(); - edit(&mut registries); - store - .save_registries(registries) - .map_or_else(|error| error.to_string(), |()| String::new()) - }; - let cycle = |all: &[&str], current: &str| { - let index = all - .iter() - .position(|known| *known == current) - .map_or(0, |index| index + 1); - all[index % all.len()].to_owned() - }; - match key.code { - KeyCode::Up => view.selected = view.selected.saturating_sub(1), - KeyCode::Down => view.selected = (view.selected + 1).min(rows.len() - 1), - KeyCode::Enter | KeyCode::Char(' ') => match &row { - Type(name) => { - *status = change(&|registries| { - let next = cycle( - crate::registry::TYPES, - registries[name.as_str()]["type"].as_str().unwrap_or(""), - ); - registries[name.as_str()]["type"] = json!(next); - }) - } - Format(name, index) => { - *status = change(&|registries| { - let repository = &mut registries[name.as_str()]["repositories"][*index]; - let next = cycle( - crate::registry::FORMATS, - repository["format"].as_str().unwrap_or(""), - ); - repository["format"] = json!(next); - }) - } - // Editing starts from the current value, except for what is secret or new. - other => { - let registries = store.registries().unwrap_or_default(); - let current = match other { - Name(name) => name.clone(), - Address(name) => registries[name.as_str()]["url"] - .as_str() - .unwrap_or("") - .to_owned(), - User(name) => registries[name.as_str()]["username"] - .as_str() - .unwrap_or("") - .to_owned(), - Repository(name, index) => { - registries[name.as_str()]["repositories"][*index]["name"] - .as_str() - .unwrap_or("") - .to_owned() - } - DockerAddress(name, index) => registries[name.as_str()]["repositories"][*index] - ["address"] - .as_str() - .unwrap_or("") - .to_owned(), - _ => String::new(), - }; - view.editor = Some(Input::new(current)); - } - }, - KeyCode::Char('x') | KeyCode::Delete => match &row { - Name(name) if !armed => { - view.armed = true; - *status = format!("x again removes {name} and its stored login"); - } - Name(name) => { - *status = change(&|registries| { - registries.remove(name.as_str()); - }); - let _ = store.set_secret(&format!("registry/{name}"), None); - } - User(name) => { - *status = change(&|registries| { - if let Some(registry) = registries[name.as_str()].as_object_mut() { - registry.remove("username"); - } - }) - } - Password(name) => { - *status = match store.set_secret(&format!("registry/{name}"), None) { - Ok(()) => "Forgotten".into(), - Err(error) => error.to_string(), - } - } - Repository(name, index) => { - *status = change(&|registries| { - if let Some(repositories) = - registries[name.as_str()]["repositories"].as_array_mut() - { - repositories.remove(*index); - } - }) - } - _ => {} - }, - _ => return leave_tab(key, tabs::REGISTRIES), - } - Outcome::Stay -} - -/// Open one of the home screen's tabs. -fn open_tab(home: &mut Home, tab: usize) { - home.mode = match tab { - tabs::PROVISIONERS => HomeMode::Provisioners(Default::default()), - tabs::SETTINGS => HomeMode::Settings(0, None), - tabs::REGISTRIES => HomeMode::Registries(RegistryView::default()), - tabs::SECRETS => HomeMode::Secrets(SecretView::default()), - tabs::PLUGINS => HomeMode::Plugins(PluginView::default()), - _ => HomeMode::Hosts, - }; -} - -/// Whether the server's last word is that a host answers: a small marker in -/// front of its name. Nothing without a server, and a hollow one when the -/// server has not said. -fn host_marker(known: Option<&berth_core::server::Known>, name: &str) -> Span<'static> { - match known.map(|known| known.hosts.get(name)) { - None => Span::raw(""), - Some(Some(reach)) if reach.reachable => Span::styled("● ", Style::new().fg(theme::GOOD)), - Some(Some(_)) => Span::styled("● ", Style::new().fg(theme::BAD)), - Some(None) => Span::styled("○ ", Style::new().fg(theme::MUTED)), - } -} - -/// What the Information panel says of the selected host on the Hosts tab. -fn host_help(home: &Home) -> String { - let Some(known) = &home.known else { - return String::new(); - }; - let mut help = String::new(); - if let Some(note) = server_note(known) { - help = format!("{note}\n\n"); - } - if let Some((name, _)) = home.hosts.get(home.selected) - && known.hosts.get(name).is_some_and(|reach| reach.rotating) - { - help.push_str(&format!("A key rotation has not reached {name} yet: it is offered the old key and the new one until it has. Run the rotation again when it answers.\n\n")); - } - help.push_str(&match home.hosts.get(home.selected) { - None => "● answers, ● does not answer, ○ not known yet: what the berth server found when it last asked.".to_owned(), - Some((name, _)) => match known.hosts.get(name) { - Some(reach) if reach.reachable => match &reach.version { - Some(version) => format!("{name} answers, with berthd {version}."), - None => format!("{name} answers."), - }, - Some(reach) => format!( - "{name} does not answer the berth server.\n\n{}", - reach.error.as_deref().unwrap_or("It gave no reason.") - ), - None => format!("The berth server has not said yet whether {name} answers."), - }, - }); - help -} - -struct Home { - hosts: Vec<(String, String)>, - selected: usize, - mode: HomeMode, - status: String, - /// The water moves with the time since then. - started: std::time::Instant, - /// The link that opens the server's page, as a QR code, while it is shown. - qr: Option, - /// The SSH keys of this berth's key ring, or of the berth server it - /// uses, by name and public half, and that server. Asked for when - /// Settings is first shown; empty when unknown. - berth_key: Option, - /// The tailnet settings of the berth server this berth uses, which are - /// the server's to keep. Asked for with the key. - served: Option>, - /// What the last mirror found of the berth server and its hosts, with a - /// server set. - known: Option, - /// Whether a mirror is being made now. - asking: bool, - /// `r` was pressed on the Hosts tab: the loop reloads, which needs the mirror. - refresh: bool, - /// The hosts of this berth a key rotation has not reached yet. - rotating: Vec, -} - -/// The status line while a mirror is being made, if nothing else is said. -const ASKING: &str = "Asking the berth server…"; - -/// What the last mirror left, for the home screen. -fn known_of(paths: &Paths) -> Option { - berth_core::server::Server::configured(paths).map(|server| server.known(paths)) -} - -/// The line for a server older than this berth. -fn server_note(known: &berth_core::server::Known) -> Option { - let version = known.version.as_deref()?; - crate::update::newer(berth_api::VERSION, version).then(|| { - format!( - "The berth server runs {version} and this berth {}. Update the berth server.", - berth_api::VERSION - ) - }) -} - -/// Take in a finished mirror: the hosts and what is known of them are read -/// again, and what came from the server on the Settings tab is asked for again. -fn home_mirror(home: &mut Home, paths: &Paths, mirroring: &mut Option) { - if std::mem::take(&mut home.refresh) { - match mirroring { - Some(mirroring) => { - if !mirroring.start() { - home.status = "Still asking the berth server".into(); - } - } - None => { - reload_hosts(home, paths); - home.status = "Read the hosts again".into(); - } - } - } - let Some(mirroring) = mirroring else { - return; - }; - if let Some(made) = mirroring.finished() { - match made { - Ok(()) => reload_hosts(home, paths), - Err(error) => home.status = error, - } - } - home.asking = mirroring.running(); -} - -/// Read the saved hosts and what is known of them again, keeping the selected -/// host selected. -fn reload_hosts(home: &mut Home, paths: &Paths) { - let chosen = home.hosts.get(home.selected).map(|(name, _)| name.clone()); - home.hosts = paths.hosts(); - home.selected = chosen - .and_then(|name| home.hosts.iter().position(|(known, _)| *known == name)) - .unwrap_or(home.selected.min(home.hosts.len())); - home.known = known_of(paths); - home.rotating = rotating(paths); - home.berth_key = None; - home.served = None; -} - -/// The hosts of this berth that a key rotation has not reached yet: they -/// are offered both keys until it has. A server's hosts are the server's. -fn rotating(paths: &Paths) -> Vec { - if berth_core::server::Server::configured(paths).is_some() { - return vec![]; - } - let Ok(store) = berth_core::store::shared(paths) else { - return vec![]; - }; - // Nothing to look at while no key is being replaced. - let pending = berth_core::keys::ring::names(&store).is_ok_and(|names| { - names.iter().any(|name| { - berth_core::keys::ring::read(&store, name, true).is_ok_and(|next| next.is_some()) - }) - }); - if !pending { - return vec![]; - } - berth_core::host::Host::all(paths) - .into_iter() - .filter(|host| { - berth_core::keys::contact::waiting(&store, &host.place, &host.config) - .is_ok_and(|keys| !keys.is_empty()) - }) - .map(|host| host.name) - .collect() -} - -/// The keys of a key ring by name and public half, and the berth server -/// whose ring it is. -type RingKeys = (Vec<(String, String)>, Option); - -/// The rows of the Settings tab after berth's own settings. -#[derive(Clone, Copy, PartialEq, Eq, Debug)] -enum SettingsRow { - /// A key of the ring, by its place in the list. - Key(usize), - Connect, -} - -fn settings_extras(home: &Home, paths: &Paths) -> Vec { - let mut extras = vec![]; - if let Some((keys, _)) = &home.berth_key { - extras.extend((0..keys.len()).map(SettingsRow::Key)); - } - if connect_link(paths).is_some() { - extras.push(SettingsRow::Connect); - } - extras -} - -/// The SSH keys hosts set up from here are logged in to with: the server's -/// for a berth that uses one, which only someone who may set the server up -/// is told. -fn berth_key(paths: &Paths) -> RingKeys { - let server = berth_core::server::Server::configured(paths); - let keys = match &server { - Some(server) => crate::key_command::listed(server), - None => crate::key_command::local(paths), - }; - let keys = keys - .unwrap_or_default() - .into_iter() - .map(|key| (key.name, key.public)) - .collect(); - (keys, server.map(|server| server.url)) -} - -/// The tailnet settings of the berth server this berth uses, by key. -fn served_settings(paths: &Paths) -> Option> { - let server = berth_core::server::Server::configured(paths)?; - let listed = server.call("settings.list", json!({}), None).ok()?; - Some( - listed - .as_array()? - .iter() - .filter_map(|setting| { - let key = setting["key"].as_str()?; - berth_core::tailnet::SETTINGS - .contains(&key) - .then(|| (key.to_owned(), setting["value"].clone())) - }) - .collect(), - ) -} - -/// Change a setting: on the berth server this berth uses for one that is -/// the server's, here otherwise. -fn change_setting(home: &mut Home, paths: &Paths, key: &str, value: &str) -> crate::Result<()> { - match ( - berth_core::server::Server::configured(paths), - &mut home.served, - ) { - (Some(server), Some(served)) if berth_core::tailnet::SETTINGS.contains(&key) => { - server.call( - "settings.set", - json!({"key": key}), - Some(json!({"value": value})), - )?; - served.insert(key.into(), json!(value)); - Ok(()) - } - _ => paths.set(key, value), - } -} - -/// What a setting is called on screen. -fn setting_label(key: &str) -> &str { - match key { - "source" => "Provisioning source", - "default_host" => "Default host", - "iso_dir" => "Installer ISO folder", - other => other, - } -} - -/// A setting's stored value, and what applies when it is not set. `served` -/// has the settings a berth server keeps. -fn setting( - paths: &Paths, - served: Option<&serde_json::Map>, - key: &str, -) -> (String, String) { - // One at a time, so a store that does not open leaves the settings of the file. - let stored = match served { - Some(served) if berth_core::tailnet::SETTINGS.contains(&key) => served - .get(key) - .and_then(|value| value.as_str()) - .map(str::to_owned), - _ => paths.setting_with(key, None), - }; - let fallback = match key { - "source" => paths - .source() - .map(|dir| dir.display().to_string()) - .unwrap_or_else(|_| "not found".into()), - "iso_dir" => paths.iso_dir().display().to_string(), - _ => "none".into(), - }; - (stored.unwrap_or_default(), fallback) -} - -/// The berth server this berth uses, and the link that opens its page signed -/// in with its token. None without a token: a login is not handed on. -fn connect_link(paths: &Paths) -> Option<(String, String)> { - let server = berth_core::server::Server::configured(paths)?; - let link = berth_tui::qr::served_link(&server.url, server.token.as_deref()?)?; - Some((server.url, link)) -} - -/// Clears a status line once it has been showing for a while. -#[derive(Default)] -struct Fade { - text: String, - since: Option, -} - -impl Fade { - const SHOWN: std::time::Duration = std::time::Duration::from_secs(6); - - /// Clear `status` when its time is up. Otherwise, how long it has left. - fn check(&mut self, status: &mut String) -> Option { - if *status != self.text { - self.text = status.clone(); - self.since = Some(std::time::Instant::now()); - } - if status.is_empty() { - return None; - } - let shown = self.since.map_or(Self::SHOWN, |since| since.elapsed()); - if shown >= Self::SHOWN { - status.clear(); - self.text.clear(); - return None; - } - Some(Self::SHOWN - shown) - } -} - -/// While the mirror runs the host's own screen says so, and a failure is told. -/// What the host shows is read again unless it holds changes not yet saved. -fn host_mirror(app: &mut App, mirroring: &mut Option) { - let Some(mirroring) = mirroring else { - return; - }; - if let Some(made) = mirroring.finished() { - match made { - Ok(()) => { - if app.status == ASKING { - app.status.clear(); - } - if let Some(said) = app.take_in_mirror() { - app.status = said; - } - } - Err(error) => app.status = error, - } - } else if app.status.is_empty() { - app.status = ASKING.into(); - } -} - -/// The home screen: pick a saved host, name a new one or change berth's own -/// settings. `None` quits. -pub fn home( - terminal: &mut DefaultTerminal, - paths: &Paths, - mirroring: &mut Option, -) -> Result> { - // A blinking cursor restarts its blink with every frame of the water, which - // reads as flicker; a steady one does not. - use crossterm::cursor::SetCursorStyle; - crossterm::execute!(std::io::stdout(), SetCursorStyle::SteadyBar)?; - let chosen = home_loop(terminal, paths, mirroring); - crossterm::execute!(std::io::stdout(), SetCursorStyle::BlinkingBar)?; - chosen -} - -fn home_loop( - terminal: &mut DefaultTerminal, - paths: &Paths, - mirroring: &mut Option, -) -> Result> { - let hosts = paths.hosts(); - let preferred = setting(paths, None, "default_host").0; - let mut home = Home { - selected: hosts - .iter() - .position(|(name, _)| *name == preferred) - .unwrap_or(0), - hosts, - mode: HomeMode::Hosts, - status: String::new(), - started: std::time::Instant::now(), - qr: None, - berth_key: None, - served: None, - known: None, - asking: false, - refresh: false, - rotating: rotating(paths), - }; - let mut hits = vec![]; - let mut fade = Fade::default(); - home.known = known_of(paths); - loop { - home_mirror(&mut home, paths, mirroring); - fade.check(&mut home.status); - if matches!(home.mode, HomeMode::Settings(..)) && home.berth_key.is_none() { - home.berth_key = Some(berth_key(paths)); - home.served = served_settings(paths); - } - if let HomeMode::Plugins(view) = &mut home.mode { - view.load(paths); - view.pump(&mut home.status); - } - if let HomeMode::Provisioners(view) = &mut home.mode { - view.load(paths); - view.pump(paths, &mut home.status); - } - // The water redraws many times a second. Each frame is applied at once, - // so the text cursor is not seen travelling to the water and back. - crossterm::execute!( - std::io::stdout(), - crossterm::terminal::BeginSynchronizedUpdate - )?; - let drawn = terminal.draw(|frame| draw_home(frame, paths, &home, &mut hits)); - crossterm::execute!( - std::io::stdout(), - crossterm::terminal::EndSynchronizedUpdate - )?; - drawn?; - if !crate::keys::poll(std::time::Duration::from_millis(50))? { - continue; - } - let event = hint_click(crate::keys::read()?); - if home.qr.is_some() { - if let Some(event) = crate::keys::event(&event) { - berth_tui::qr::take(&mut home.qr, &vec![], &event, &mut None); - } - continue; - } - let left = match event { - Event::Key(key) if key.kind != KeyEventKind::Release => { - if key.code == KeyCode::Char('c') && key.modifiers.contains(KeyModifiers::CONTROL) { - return Ok(None); - } - home_key(&mut home, paths, key, &mut Desktop) - } - Event::Mouse(mouse) => home_mouse(&mut home, paths, mouse, &hits, &mut Desktop), - Event::Paste(text) => { - if let Some(input) = home_editor(&mut home.mode) { - for ch in text.chars().filter(|ch| !ch.is_control()) { - input.handle(InputRequest::InsertChar(ch)); - } - } else if let HomeMode::Provisioners(view) = &mut home.mode { - view.paste(&text); - } - None - } - _ => None, - }; - if let Some(host) = left { - return Ok(host); - } - } -} - -/// The text field of the home screen that is being edited, if one is. -fn home_editor(mode: &mut HomeMode) -> Option<&mut Input> { - match mode { - HomeMode::Naming(input) => Some(input), - HomeMode::Settings(_, editor) => editor.as_mut(), - HomeMode::Registries(view) => view.editor.as_mut(), - HomeMode::Secrets(view) => view.editor.as_mut(), - HomeMode::Plugins(view) => view.editor.as_mut(), - HomeMode::Hosts | HomeMode::Provisioners(_) => None, - } -} - -/// Handle the mouse on the home screen: the wheel moves the selection, a click -/// switches tab or chooses a row. -fn home_mouse( - home: &mut Home, - paths: &Paths, - mouse: MouseEvent, - hits: &[(Rect, HomeTarget)], - platform: &mut dyn Platform, -) -> Option> { - let key = |code| KeyEvent::new(code, KeyModifiers::NONE); - match mouse.kind { - MouseEventKind::ScrollUp => home_key(home, paths, key(KeyCode::Up), platform), - MouseEventKind::ScrollDown => home_key(home, paths, key(KeyCode::Down), platform), - MouseEventKind::Down(MouseButton::Left) => { - // A form covers the tab, and takes the click itself. - if let HomeMode::Provisioners(view) = &mut home.mode - && view.tab.form().is_some() - { - let at = (mouse.column, mouse.row); - let left = view.click(paths, at, &mut home.status, platform); - if let crate::provisioner_tab::Left::Tab(tab) = left { - open_tab(home, tab); - } - return None; - } - let (_, target) = hits - .iter() - .find(|(rect, _)| inside(*rect, mouse.column, mouse.row))?; - match *target { - HomeTarget::Tab(1) if matches!(home.mode, HomeMode::Settings(..)) => None, - HomeTarget::Tab(2) if matches!(home.mode, HomeMode::Registries(..)) => None, - HomeTarget::Tab(3) if matches!(home.mode, HomeMode::Secrets(..)) => None, - HomeTarget::Tab(4) if matches!(home.mode, HomeMode::Plugins(..)) => None, - HomeTarget::Tab(5) if matches!(home.mode, HomeMode::Provisioners(..)) => None, - HomeTarget::Tab(tab) => { - open_tab(home, tab); - None - } - HomeTarget::Row(index) => { - // A click elsewhere drops what was being typed. - match &mut home.mode { - HomeMode::Registries(view) => { - view.selected = index; - view.editor = None; - } - HomeMode::Secrets(view) => { - view.selected = index; - view.editor = None; - view.naming = None; - } - HomeMode::Plugins(view) => { - view.tab.selected = index; - view.editor = None; - } - // A click selects a row, and on the row selected chooses it. - HomeMode::Provisioners(view) => { - if view.typing() { - return None; - } - let before = view.tab.selected_row(); - view.tab.select_row(index); - if view.tab.page.is_some() && before != index { - return None; - } - } - HomeMode::Settings(selected, editor) => { - *selected = index; - *editor = None; - } - mode => { - *mode = HomeMode::Hosts; - home.selected = index; - } - } - home_key(home, paths, key(KeyCode::Enter), platform) - } - } - } - _ => None, - } -} - -/// Handle a key on the home screen. `Some` leaves it: with a host, or to quit. -fn home_key( - home: &mut Home, - paths: &Paths, - key: KeyEvent, - platform: &mut dyn Platform, -) -> Option> { - let extras = match home.mode { - HomeMode::Settings(..) => settings_extras(home, paths), - _ => vec![], - }; - let extra = |selected: usize| { - let index = selected.checked_sub(crate::paths::SETTINGS.len())?; - extras.get(index).copied() - }; - if let Some(typed) = crate::keys::key(&key) - && berth_tui::screen::is_copy_field(&typed) - && let Some(input) = home_editor(&mut home.mode) - { - home.status = berth_tui::screen::copy_field(platform, input); - return None; - } - let outcome = match &mut home.mode { - HomeMode::Registries(view) => Some(registries_key(view, paths, key, &mut home.status)), - HomeMode::Secrets(view) => Some(secrets_key(view, paths, key, &mut home.status)), - HomeMode::Plugins(view) => Some(plugins_key(view, paths, key, &mut home.status)), - HomeMode::Provisioners(view) => { - Some(match view.key(paths, key, &mut home.status, platform) { - crate::provisioner_tab::Left::Stay => Outcome::Stay, - crate::provisioner_tab::Left::Tab(tab) => Outcome::Tab(tab), - crate::provisioner_tab::Left::Quit => Outcome::Quit, - }) - } - _ => None, - }; - match outcome { - Some(Outcome::Stay) => return None, - Some(Outcome::Tab(tab)) => { - open_tab(home, tab); - return None; - } - Some(Outcome::Quit) => return Some(None), - None => {} - } - match &mut home.mode { - HomeMode::Registries(_) - | HomeMode::Secrets(_) - | HomeMode::Plugins(_) - | HomeMode::Provisioners(_) => {} - HomeMode::Naming(input) => match key.code { - KeyCode::Esc => home.mode = HomeMode::Hosts, - KeyCode::Enter => { - let name = input.value().trim().to_owned(); - if name.contains('/') || name.ends_with(".json") { - home.status = "Use letters, digits, - and _".into(); - } else if let Err(error) = berth_core::profiles::Place::resolve(paths, &name) { - home.status = error.to_string(); - } else { - return Some(Some(name)); - } - } - _ => { - input.handle_event(&Event::Key(key)); - } - }, - HomeMode::Settings(selected, editor @ Some(_)) => match key.code { - KeyCode::Esc => *editor = None, - KeyCode::Enter => { - let (name, _) = crate::paths::SETTINGS[*selected]; - let value = editor.take().unwrap_or_default(); - home.status = match change_setting(home, paths, name, value.value().trim()) { - Ok(()) => format!("Saved {}", setting_label(name)), - Err(error) => error.to_string(), - }; - } - _ => { - if let Some(input) = editor { - input.handle_event(&Event::Key(key)); - } - } - }, - HomeMode::Settings(selected, editor) => match key.code { - KeyCode::Up => *selected = selected.saturating_sub(1), - KeyCode::Down => { - // The key of berth and the row that opens the server's page - // come after the settings. - let last = crate::paths::SETTINGS.len() + extras.len() - 1; - *selected = (*selected + 1).min(last); - } - KeyCode::Enter if *selected < crate::paths::SETTINGS.len() => { - let (name, _) = crate::paths::SETTINGS[*selected]; - *editor = Some(Input::new(setting(paths, home.served.as_ref(), name).0)); - } - KeyCode::Enter | KeyCode::Char('c') - if matches!(extra(*selected), Some(SettingsRow::Key(_))) => - { - if let (Some(SettingsRow::Key(index)), Some((keys, _))) = - (extra(*selected), &home.berth_key) - && let Some((_, public)) = keys.get(index) - { - home.status = berth_tui::berth_key::copy(platform, public); - } - } - KeyCode::Enter | KeyCode::Char('r') - if key.code == KeyCode::Enter || key.modifiers.contains(KeyModifiers::ALT) => - { - if extra(*selected) == Some(SettingsRow::Connect) - && let Some((_, link)) = connect_link(paths) - { - home.qr = Some(berth_tui::qr::Overlay::connect(link)); - } - } - _ => match leave_tab(key, tabs::SETTINGS) { - Outcome::Tab(tab) => open_tab(home, tab), - Outcome::Quit => return Some(None), - Outcome::Stay => {} - }, - }, - HomeMode::Hosts => match key.code { - KeyCode::Up => home.selected = home.selected.saturating_sub(1), - KeyCode::Down => { - home.selected = (home.selected + 1).min(home.hosts.len() + MAKES.len()) - } - KeyCode::Char('n') => home.mode = HomeMode::Naming(Input::default()), - KeyCode::Char('s') => open_tab(home, tabs::SETTINGS), - KeyCode::Char('r') => home.refresh = true, - KeyCode::Char('1'..='6') | KeyCode::Tab | KeyCode::BackTab => { - if let Outcome::Tab(tab) = leave_tab(key, tabs::HOSTS) { - open_tab(home, tab); - } - } - KeyCode::Char('d') => { - if let Some((name, _)) = home.hosts.get(home.selected) { - home.status = match paths.set("default_host", name) { - Ok(()) => format!("{name} is selected first from now on"), - Err(error) => error.to_string(), - }; - } - } - KeyCode::Enter => match home.hosts.get(home.selected) { - Some((name, _)) => return Some(Some(name.clone())), - None => match home.selected.checked_sub(home.hosts.len() + 1) { - // An instance or a compose project is made on its - // provisioner's page, whose form opens at once. - Some(index) if index < MAKES.len() => { - let mut view = Box::::default(); - view.create(paths, MAKES[index].0, &mut home.status); - home.mode = HomeMode::Provisioners(view); - } - _ => home.mode = HomeMode::Naming(Input::default()), - }, - }, - KeyCode::Char('q') | KeyCode::Esc => return Some(None), - _ => {} - }, - } - None -} - -fn draw_home(frame: &mut Frame, paths: &Paths, home: &Home, hits: &mut Vec<(Rect, HomeTarget)>) { - hits.clear(); - clear_hints(); - frame.render_widget(theme::backdrop(), frame.area()); - let area = frame.area().inner(Margin { - horizontal: 2, - vertical: 1, - }); - let [header, _, logo, main, water, footer] = Layout::vertical([ - Constraint::Length(1), - Constraint::Length(1), - Constraint::Length(4), - Constraint::Min(1), - Constraint::Length(2), - Constraint::Length(1), - ]) - .areas(area); - let tab = match home.mode { - HomeMode::Provisioners(..) => tabs::PROVISIONERS, - HomeMode::Settings(..) => tabs::SETTINGS, - HomeMode::Registries(..) => tabs::REGISTRIES, - HomeMode::Secrets(..) => tabs::SECRETS, - HomeMode::Plugins(..) => tabs::PLUGINS, - _ => tabs::HOSTS, - }; - // The Information panel is for the settings, and for a host with a server's word on it. - let settings = tab != 0 || home.known.is_some(); - let tabs = draw_header( - frame, - header, - &tabs::NAMES, - tab, - home.known - .as_ref() - .and_then(server_note) - .as_deref() - .unwrap_or(""), - ); - hits.extend( - tabs.into_iter() - .enumerate() - .map(|(tab, rect)| (rect, HomeTarget::Tab(tab))), - ); - // What is typed for a password is not shown. - let mut masked = false; - frame.render_widget(Paragraph::new(theme::logo()), logo); - frame.render_widget( - Paragraph::new(theme::swell( - water.width, - home.started.elapsed().as_secs_f32(), - )), - water, - ); - - // Settings use the same arrangement as a host's tabs: rows on the left with - // the editor under them, what the selected row means on the right. - let [left, right] = Layout::horizontal([ - Constraint::Percentage(if settings { 64 } else { 100 }), - Constraint::Min(0), - ]) - .areas(main); - let muted = Style::new().fg(theme::MUTED); - let (rows, selected, editor, help, keys): (Vec, usize, _, String, _) = match &home.mode { - HomeMode::Settings(selected, editor) => { - let connect = connect_link(paths); - let extras = settings_extras(home, paths); - let (keys, server) = home.berth_key.clone().unwrap_or_default(); - let mut rows: Vec = crate::paths::SETTINGS - .iter() - .map(|(name, _)| { - let (stored, fallback) = setting(paths, home.served.as_ref(), name); - if stored.is_empty() { - Line::from(vec![ - Span::styled( - format!("{} ", setting_label(name)), - Style::new().fg(theme::TEXT), - ), - Span::styled(format!("Default · {fallback}"), muted.italic()), - ]) - } else if crate::paths::is_secret(name) { - // A secret is not shown whole in a list; its field shows it. - crate::app::value_line( - setting_label(name), - &berth_tui::screen::short_secret(&stored), - Kind::Text, - ) - } else { - crate::app::value_line(setting_label(name), &stored, Kind::Text) - } - }) - .collect(); - for extra in &extras { - rows.push(match extra { - SettingsRow::Key(index) => { - let (name, public) = &keys[*index]; - berth_tui::berth_key::row(name, public) - } - SettingsRow::Connect => Line::from(vec![ - Span::styled("Open on another device ", Style::new().fg(theme::TEXT)), - Span::styled("QR code and link, with the token", muted), - ]), - }); - } - let selected = (*selected).min(rows.len() - 1); - let extra = selected - .checked_sub(crate::paths::SETTINGS.len()) - .and_then(|index| extras.get(index)); - let (help, keys) = match (crate::paths::SETTINGS.get(selected), extra, &connect) { - (Some((name, help)), ..) => ( - format!( - "{}.\n\nLeave it empty to use the default. Also: berth config {name} VALUE\n\nKept {}", - berth_tui::screen::describe_setting(help), - match (&home.served, berth_core::tailnet::SETTINGS.contains(name)) { - (Some(_), true) => format!( - "on the berth server at {server}", - server = server.as_deref().unwrap_or("") - ), - _ if crate::paths::is_local(name) => { - format!("in {}", paths.settings_file().display()) - } - _ => format!( - "in berth's store, the {}", - berth_core::store::Installation::of(paths).target.describe() - ), - } - ), - "↑↓ select · Enter change · q quit", - ), - (None, Some(SettingsRow::Key(index)), _) => ( - berth_tui::berth_key::help(&keys[*index].0, &keys[*index].1, server.as_deref()), - "↑↓ select · Enter copy the key · q quit", - ), - (None, Some(SettingsRow::Connect), Some((url, _))) => ( - berth_tui::qr::connect_help(url, true), - "↑↓ select · Enter show the QR code · q quit", - ), - _ => (String::new(), ""), - }; - ( - rows, - selected, - editor.as_ref().map(|input| (" Edit ", input)), - help, - if editor.is_some() { - "Enter accept · Esc cancel · Ctrl+O copy" - } else { - keys - }, - ) - } - HomeMode::Registries(view) => { - let store = Store::new(paths); - let listed = registry_rows(store.as_ref().ok()); - let selected = view.selected.min(listed.len() - 1); - let row = &listed[selected].0; - masked = matches!(row, RegistryRow::Password(_)); - let title = match row { - RegistryRow::AddRegistry => " Name of the new registry ", - RegistryRow::AddRepository(_) => " Name of the repository on the server ", - RegistryRow::Password(_) => " Password or token ", - _ => " Edit ", - }; - let help = match view.report.is_empty() { - true => format!( - "{}\n\nc checks that this registry answers and accepts its login.\n\n{}", - row.help(), - match &store { - Ok(store) => kept_in(paths, store), - Err(error) => error.to_string(), - } - ), - false => view.report.join("\n\n"), - }; - ( - listed.iter().map(|(_, line)| line.clone()).collect(), - selected, - view.editor.as_ref().map(|input| (title, input)), - help, - if view.editor.is_some() { - "Enter accept · Esc cancel · Ctrl+O copy" - } else { - "↑↓ select · Enter change · c check · x remove · q quit" - }, - ) - } - HomeMode::Secrets(view) => { - let store = Store::new(paths); - let names = store - .as_ref() - .map(|store| store.secret_names()) - .unwrap_or_default(); - let selected = view.selected.min(names.len()); - let mut rows: Vec = names - .iter() - .map(|name| { - Line::from(vec![ - Span::styled(format!("{name} "), Style::new().fg(theme::TEXT)), - Span::styled("● stored", Style::new().fg(theme::GOOD)), - ]) - }) - .collect(); - rows.push(Line::styled("+ Add secret", Style::new().fg(theme::ACCENT))); - // A value is not shown while it is typed; a new secret's name is. - masked = view.naming.is_some() || selected < names.len(); - let title = match (&view.naming, names.get(selected)) { - (Some(_), _) => " Value of the new secret ", - (None, Some(_)) => " New value ", - (None, None) => " Name of the new secret ", - }; - let about = match names.get(selected) { - Some(name) if let Some(help) = berth_tui::setup::own_secret_help(name) => help, - Some(_) => { - "A token, password or key file. Enter replaces its value, x twice removes it. Values are never shown." - } - None => { - "Add a secret: first a short name (lower-case letters, digits, - and _), then its value. Paste works. The name tailnet_api_key is for the API key of the tailnet's control server, which berth keeps for itself." - } - }; - let help = format!( - "{about}\n\nWhich host gets a secret, and whether as an environment variable or a file, is chosen on the Secrets page of that host.\n\n{}", - match &store { - Ok(store) => kept_in(paths, store), - Err(error) => error.to_string(), - } - ); - ( - rows, - selected, - view.editor.as_ref().map(|input| (title, input)), - help, - if view.editor.is_some() { - "Enter accept · Esc cancel · Ctrl+O copy" - } else { - "↑↓ select · Enter set value · x remove · q quit" - }, - ) - } - HomeMode::Provisioners(view) => { - let shown = view.shown(paths); - (shown.rows, shown.selected, None, shown.help, shown.keys) - } - HomeMode::Plugins(view) => { - let plugins = view.plugins(); - let listed = berth_tui::plugins::rows(plugins, &view.tab.updates); - let selected = view.tab.selected.min(listed.len() - 1); - let server = berth_core::server::Server::configured(paths); - let place = plugin_place(&server); - let report = match &view.listing { - Some(Err(error)) if view.report.is_empty() => vec![format!("✗ {error}")], - _ => view.report.clone(), - }; - let dir = match &view.listing { - Some(Ok(listing)) => listing.dir.clone(), - _ => crate::plugins::own_dir(&paths.config_dir) - .display() - .to_string(), - }; - let help = berth_tui::plugins::help( - listed[selected].0.map(|index| &plugins[index]), - &report, - &place.adding(&dir), - ); - ( - listed.into_iter().map(|(_, line)| line).collect(), - selected, - view.editor - .as_ref() - .map(|input| (place.editor_title(), input)), - help, - if view.editor.is_some() { - "Enter install · Esc cancel · Ctrl+O copy" - } else { - "↑↓ select · Enter add · c check · U update · A all · x remove · q quit" - }, - ) - } - mode => { - let preferred = setting(paths, None, "default_host").0; - let mut rows: Vec = home - .hosts - .iter() - .map(|(name, address)| { - let address = if address.is_empty() { - "no address yet" - } else { - address - }; - let mut line = vec![ - host_marker(home.known.as_ref(), name), - Span::styled(format!("{name:<24}"), Style::new().fg(theme::TEXT)), - Span::styled(format!("{address:<28}"), muted), - ]; - if *name == preferred { - line.push(Span::styled("default ", Style::new().fg(theme::STEEL))); - } - let served = home - .known - .as_ref() - .and_then(|known| known.hosts.get(name)) - .is_some_and(|reach| reach.rotating); - if served || home.rotating.contains(name) { - line.push(Span::styled( - "key rotation pending", - Style::new().fg(theme::WARN), - )); - } - Line::from(line) - }) - .collect(); - rows.push(Line::styled("+ New host", Style::new().fg(theme::ACCENT))); - for (_, label) in MAKES { - rows.push(Line::styled( - format!("+ New {label}"), - Style::new().fg(theme::ACCENT), - )); - } - let help = host_help(home); - let naming = match mode { - HomeMode::Naming(input) => Some((" New host name ", input)), - _ => None, - }; - ( - rows, - home.selected, - naming, - help, - if naming.is_some() { - "Enter create · Esc cancel · Ctrl+O copy" - } else { - "↑↓ select · Enter open · n new host · d default · r refresh · q quit" - }, - ) - } - }; - let [body, edit] = Layout::vertical([ - Constraint::Min(1), - Constraint::Length(if editor.is_some() { 3 } else { 0 }), - ]) - .areas(left); - for index in 0..(rows.len() as u16).min(body.height) { - hits.push(( - Rect::new(body.x, body.y + index, body.width, 1), - HomeTarget::Row(index as usize), - )); - } - frame.render_stateful_widget( - List::new(rows) - .highlight_symbol(theme::cursor()) - .highlight_style(Style::new().bg(theme::RAISED)), - body, - &mut ListState::default().with_selected(Some(selected)), - ); - if settings { - frame.render_widget( - Paragraph::new(crate::markdown::render(&help)) - .block(theme::panel("Information", false)) - .wrap(Wrap { trim: false }), - right, - ); - } - if let Some((title, input)) = editor { - // The panel has a border and one column of padding on each side. - let width = edit.width.saturating_sub(4).max(1) as usize; - let scroll = input.visual_scroll(width); - frame.render_widget( - Paragraph::new(match masked { - true => "•".repeat(input.value().chars().count()), - false => input.value().to_owned(), - }) - .scroll((0, scroll.min(u16::MAX as usize) as u16)) - .block(theme::panel(title.trim(), true)), - edit, - ); - frame.set_cursor_position(( - edit.x + 2 + (input.visual_cursor() - scroll).min(width) as u16, - edit.y + 1, - )); - } - let status = match home.status.is_empty() && home.asking { - true => ASKING, - false => &home.status, - }; - draw_footer(frame, footer, keys, status, theme::WARN); - if let HomeMode::Provisioners(view) = &home.mode { - view.draw_over(frame, status); - } - if let Some(qr) = &home.qr { - qr.draw(frame); - } -} - -/// Run a host's screen. Returns true when the chosen action has to run in the -/// normal terminal, and false when the user left the host. -pub fn run( - terminal: &mut DefaultTerminal, - app: &mut App, - mirroring: &mut Option, -) -> Result { - let mut display = ListState::default(); - let mut hits: Hits = vec![]; - let mut quit_armed = false; - let mut fade = Fade::default(); - loop { - host_mirror(app, mirroring); - // "x again to quit" and the like stay until the next key. - let fading = match quit_armed { - true => None, - false => fade.check(&mut app.status), - }; - app.update_suggestions(); - app.watch_host(); - app.watch_machine(); - app.watch_found_level(); - // One of berth's own actions on the host runs in the normal terminal. - if app.own_command.is_some() { - return Ok(true); - } - let (panel_rows, panel_cols) = session_size(); - app.host.resize(panel_rows, panel_cols); - let rows = app.rows(); - terminal.draw(|frame| draw(frame, app, &rows, &mut display, &mut hits))?; - if let Some(job) = &mut app.job { - let running = job.running(); - job.tick(); - if running && !crate::keys::poll(std::time::Duration::from_millis(100))? { - continue; - } - } - if let Some(session) = &mut app.session { - session.tick(); - let (rows, cols) = session_size(); - session.resize(rows, cols); - // A shell that was left needs no key to close its panel. - if session.closes && session.outcome == Some(true) { - app.session = None; - continue; - } - if !crate::keys::poll(std::time::Duration::from_millis(40))? { - continue; - } - let event = hint_click(crate::keys::read()?); - session_event(app, &event); - continue; - } - if let Some(view) = &app.host.view { - // A shell that was left needs no key to close its panel. - if view.shell && view.outcome() == Some(true) { - app.host.close(true); - continue; - } - if !crate::keys::poll(std::time::Duration::from_millis(40))? { - continue; - } - let event = hint_click(crate::keys::read()?); - view_event(app, &event); - continue; - } - // While the overview talks to the host, what it says is taken in - // without waiting for a key. - if app.step == Step::Manage - && app.host.connected() - && !crate::keys::poll(std::time::Duration::from_millis(200))? - { - continue; - } - // The machine's state is taken in when it comes, too. - if app.machine_awaited() && !crate::keys::poll(std::time::Duration::from_millis(200))? { - continue; - } - if let Some(left) = fading - && !crate::keys::poll(left)? - { - continue; - } - let event = hint_click(crate::keys::read()?); - // The overlay can outlast the panel it came from. - if app.qr.is_some() { - qr_event(app, &event); - continue; - } - if app.unlock.is_some() { - unlock_event(app, &event, &mut Desktop); - continue; - } - if app.choosing_drive { - if let Event::Key(key) = event { - let chosen = match key.code { - KeyCode::Char(digit @ '1'..='9') => { - app.usb_drives.get(digit as usize - '1' as usize).cloned() - } - KeyCode::Esc => { - app.choosing_drive = false; - None - } - _ => None, - }; - if let Some((_, device)) = chosen { - app.write_built_iso(Some(device)); - } - } - continue; - } - if app.job.is_some() { - job_event(app, &event); - continue; - } - if app.confirm.is_some() { - confirm_event(app, &event); - continue; - } - if app.machine.question.is_some() { - if let Event::Key(key) = &event { - match key.code { - KeyCode::Char('y' | 'Y') => app.machine_answer(true), - KeyCode::Char('n' | 'N') | KeyCode::Esc => app.machine_answer(false), - _ => {} - } - } - continue; - } - if app.host.question.is_some() { - if let Flow::Run = question_event(app, &event) { - return Ok(true); - } - continue; - } - if app.host.arguments.is_some() { - arguments_event(app, &event, &mut Desktop); - continue; - } - if app.picker.is_some() { - picker_event(app, &event); - continue; - } - let key = match event { - Event::Key(key) if key.kind != KeyEventKind::Release => key, - Event::Mouse(mouse) => match mouse_event(app, mouse, &hits) { - Some(key) => key, - None => continue, - }, - Event::Paste(text) => { - if let Some(input) = &mut app.editing { - for ch in text.chars().filter(|ch| !ch.is_control()) { - input.handle(InputRequest::InsertChar(ch)); - } - } - continue; - } - _ => continue, - }; - match key_event(app, key, &mut quit_armed, &mut Desktop) { - Flow::Continue => {} - Flow::Quit => return Ok(false), - Flow::Run => return Ok(true), - } - } -} - -#[cfg(test)] -mod tests { - use super::*; - use crate::app::parse_config; - use berth_core::profiles::Place; - use ratatui::{Terminal, backend::TestBackend}; - - /// Paths of a folder that is never made, for an app that saves nothing. - fn nowhere() -> Paths { - Paths { - config_dir: "/nonexistent/berth-test/config".into(), - data_dir: "/nonexistent/berth-test/data".into(), - } - } - - /// An app of a new profile that is never saved. - fn unsaved_app(config: &[u8]) -> App { - App::new( - nowhere(), - Place::File("/nonexistent/berth-test/studio.json".into()), - parse_config(config).unwrap(), - ) - } - - /// An app of a profile in the file `name` of `directory`, which also - /// holds the installation. - fn file_app(directory: &std::path::Path, name: &str, config: &[u8]) -> App { - App::new( - Paths { - config_dir: directory.join("config"), - data_dir: directory.join("data"), - }, - Place::File(directory.join(name)), - parse_config(config).unwrap(), - ) - } - - #[test] - fn applying_from_the_ui_holds_the_hosts_lock() { - use berth_core::runner::{Subject, lock}; - use std::process::Command; - let dir = tempfile::tempdir().unwrap(); - let mut app = file_app(dir.path(), "studio.json", b"{}"); - app.action = Action::Ssh; - let subject = Subject::Host(app.place.argument()); - let held = lock(&app.paths, &subject, "rollout").unwrap(); - let refused = app - .run_command(Command::new("true"), vec![]) - .unwrap_err() - .to_string(); - assert!(refused.contains("(rollout, process "), "{refused}"); - drop(held); - app.run_command(Command::new("true"), vec![]).unwrap(); - let failed = app.run_command(Command::new("false"), vec![]).unwrap_err(); - assert!(failed.to_string().contains("exit status: 1"), "{failed}"); - // Once it has ended, the lock is free for the next one. - lock(&app.paths, &subject, "apply").unwrap(); - } - - /// A clipboard of the test's own, so none of the desktop's is touched. - #[derive(Default)] - struct Clipboard(Option); - - impl Platform for Clipboard { - fn copy(&mut self, text: &str) -> bool { - self.0 = Some(text.to_owned()); - true - } - - fn paste(&mut self) -> Option { - self.0.clone() - } - - fn open(&mut self, _: &str) -> std::result::Result<(), String> { - Err("no browser in tests".into()) - } - } - - fn screen(app: &mut App) -> String { - let mut terminal = Terminal::new(TestBackend::new(100, 24)).unwrap(); - app.update_suggestions(); - let rows = app.rows(); - let (mut display, mut hits) = (ListState::default(), vec![]); - terminal - .draw(|frame| draw(frame, app, &rows, &mut display, &mut hits)) - .unwrap(); - let buffer = terminal.backend().buffer(); - (0..24) - .map(|y| { - (0..100) - .map(|x| buffer[(x, y)].symbol()) - .collect::() - }) - .collect::>() - .join("\n") - } - - #[test] - fn space_on_a_group_heading_switches_the_group() { - let mut app = unsaved_app(b"{}"); - // The coding agents are plugins that come with berth. - app.set_plugins(crate::plugins::load(&["../../plugins".into()])); - let chosen = |app: &App| app.config["ansible_vars"]["devbox_plugin_features"].to_string(); - app.set_step(Step::Features); - // The heading itself is selected first and offers the key. - assert_eq!(app.selected_group(), Some(0)); - let shown = screen(&mut app); - println!("{shown}"); - assert!(shown.contains("Coding Agents 3 of 3")); - assert!(shown.contains("[All off Space]")); - press(&mut app, &[KeyCode::Char(' ')]); - assert!(!chosen(&app).contains("claude/cli") && !chosen(&app).contains("pi/cli")); - assert!(chosen(&app).contains("t3/server")); - assert!(screen(&mut app).contains("[All on Space]")); - // Enter leaves a heading alone; on an entry Space and Enter both switch it. - press(&mut app, &[KeyCode::Enter]); - assert!(!chosen(&app).contains("claude/cli")); - press(&mut app, &[KeyCode::Down, KeyCode::Char(' ')]); - assert!(chosen(&app).contains("claude/cli") && !chosen(&app).contains("codex/cli")); - } - - #[test] - fn secrets_are_stored_on_the_home_screen_and_given_on_a_host_page() { - let directory = tempfile::tempdir().unwrap(); - let paths = Paths { - config_dir: directory.path().join("config"), - data_dir: directory.path().join("data"), - }; - let mut home = Home { - hosts: vec![], - selected: 0, - mode: HomeMode::Hosts, - status: String::new(), - started: std::time::Instant::now(), - qr: None, - berth_key: None, - served: None, - known: None, - asking: false, - refresh: false, - rotating: vec![], - }; - let keys = |home: &mut Home, codes: &[KeyCode]| { - for &code in codes { - home_key( - home, - &paths, - KeyEvent::new(code, KeyModifiers::NONE), - &mut Clipboard::default(), - ); - } - }; - let typed = |text: &str| text.chars().map(KeyCode::Char).collect::>(); - // The Secrets tab: a name, then the value, which the screen does not show. - keys(&mut home, &[KeyCode::Char('5'), KeyCode::Enter]); - keys(&mut home, &typed("github")); - keys(&mut home, &[KeyCode::Enter]); - keys(&mut home, &typed("ghp_value")); - let mut terminal = Terminal::new(TestBackend::new(110, 24)).unwrap(); - terminal - .draw(|frame| draw_home(frame, &paths, &home, &mut vec![])) - .unwrap(); - let buffer = terminal.backend().buffer(); - let screen: String = (0..24) - .flat_map(|y| (0..110).map(move |x| (x, y))) - .map(|position| buffer[position].symbol()) - .collect(); - assert!(screen.contains("•••••••••") && !screen.contains("ghp_value")); - keys(&mut home, &[KeyCode::Enter]); - let store = Store::new(&paths).unwrap(); - assert_eq!( - store.secret("github").unwrap().as_deref(), - Some("ghp_value") - ); - store.set_secret("registry/nexus", Some("pw")).unwrap(); - assert_eq!(store.secret_names(), ["github"]); - - // On a host: its Settings tab lists it, and takes how the host gets it. - let mut app = unsaved_app(b"{}"); - app.access = Ok(store.clone()); - app.set_step(Step::Settings); - let secrets: Vec = app - .entries() - .into_iter() - .filter(|entry| matches!(entry, Entry::HostSecret(_))) - .collect(); - assert_eq!(secrets, [Entry::HostSecret(0)]); - let at = app - .entries() - .iter() - .position(|entry| *entry == Entry::HostSecret(0)); - app.list.select(at); - press(&mut app, &[KeyCode::Enter]); - for code in typed("GH_TOKEN") { - press(&mut app, &[code]); - } - press(&mut app, &[KeyCode::Enter]); - assert_eq!( - app.config["secrets"], - json!([{"name": "github", "env": "GH_TOKEN"}]) - ); - assert!(screen_text(&mut app).contains("github ● variable GH_TOKEN")); - app.set_host_secret(0, "~/.config/gh/token"); - assert_eq!( - app.config["secrets"], - json!([{"name": "github", "file": "~/.config/gh/token"}]) - ); - app.set_host_secret(0, "not a name or path"); - assert_eq!(app.config["secrets"][0]["file"], "~/.config/gh/token"); - press(&mut app, &[KeyCode::Char('x')]); - assert_eq!(app.config["secrets"], json!([])); - - // Removing the secret takes x twice. - keys(&mut home, &[KeyCode::Char('x')]); - assert!(store.secret("github").unwrap().is_some()); - keys(&mut home, &[KeyCode::Char('x')]); - assert!(store.secret("github").unwrap().is_none()); - } - - #[test] - fn plugins_tab_lists_and_removes_a_copied_plugin() { - let directory = tempfile::tempdir().unwrap(); - let paths = Paths { - config_dir: directory.path().join("config"), - data_dir: directory.path().join("data"), - }; - let own = paths.config_dir.join("plugins/aa-hello"); - std::fs::create_dir_all(&own).unwrap(); - std::fs::write( - own.join("package.json"), - r#"{"name":"berth-plugin-aa-hello","version":"1.2.3","displayName":"Hello", - "berth":{"features":[{"id":"cow","label":"cowsay","group":"Development Tools"}]}}"#, - ) - .unwrap(); - let mut home = Home { - hosts: vec![], - selected: 0, - mode: HomeMode::Hosts, - status: String::new(), - started: std::time::Instant::now(), - qr: None, - berth_key: None, - served: None, - known: None, - asking: false, - refresh: false, - rotating: vec![], - }; - let key = |home: &mut Home, code| { - home_key( - home, - &paths, - KeyEvent::new(code, KeyModifiers::NONE), - &mut Clipboard::default(), - ); - }; - key(&mut home, KeyCode::Char('6')); - if let HomeMode::Plugins(view) = &mut home.mode { - view.load(&paths); - } - let mut terminal = Terminal::new(TestBackend::new(120, 40)).unwrap(); - terminal - .draw(|frame| draw_home(frame, &paths, &home, &mut vec![])) - .unwrap(); - let buffer = terminal.backend().buffer(); - let text: String = (0..40) - .flat_map(|y| (0..120).map(move |x| (x, y))) - .map(|position| buffer[position].symbol()) - .collect(); - // The plugin of the source this program carries, and the copied one. - assert!(text.contains("Unity") && text.contains("comes with berth")); - assert!(text.contains("Hello berth-plugin-aa-hello 1.2.3 · yours")); - assert!(text.contains("cowsay in Development Tools") && text.contains("+ Add plugin")); - // Hello sorts first. Shipped plugins stay; a copied one goes after x twice. - key(&mut home, KeyCode::Char('x')); - assert!(own.exists()); - key(&mut home, KeyCode::Char('x')); - assert!(!own.exists()); - key(&mut home, KeyCode::Char('x')); - assert!(home.status.contains("comes with berth")); - } - - #[test] - fn the_provisioners_tab_opens_a_page_and_runs_its_actions_here() { - use std::os::unix::fs::PermissionsExt; - let directory = tempfile::tempdir().unwrap(); - let paths = Paths { - config_dir: directory.path().join("config"), - data_dir: directory.path().join("data"), - }; - paths.prepare().unwrap(); - // The checkout is the provisioning source, so none is unpacked behind the test's back. - let checkout = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../.."); - paths - .set("source", checkout.canonicalize().unwrap().to_str().unwrap()) - .unwrap(); - let own = paths.config_dir.join("plugins/aa-lab"); - std::fs::create_dir_all(own.join("bin")).unwrap(); - std::fs::write( - own.join("bin/provision"), - "#!/bin/sh\ncat > /dev/null\n[ -f \"say.$1\" ] && cat \"say.$1\" >&2\n[ -f \"answer.$1\" ] && cat \"answer.$1\"\nexit 0\n", - ) - .unwrap(); - std::fs::set_permissions( - own.join("bin/provision"), - std::fs::Permissions::from_mode(0o755), - ) - .unwrap(); - std::fs::write( - own.join("package.json"), - r#"{"name": "berth-plugin-aa-lab", "berth": {"provisioners": [ - {"id": "box", "label": "Lab box", "command": "bin/provision", "protocol": 2, - "kinds": [{"id": "machine", "setup": "apply"}, - {"id": "instance", "label": "Instance", "arguments": [ - {"name": "image", "label": "Image", "default": "debian:12"}, - {"name": "notes", "label": "Notes", "kind": "text"}]}], - "actions": [{"name": "look", "label": "Look around", "changes": false}, - {"name": "sweep", "label": "Sweep", "confirm": "Sweep the lab?"}]}]}}"#, - ) - .unwrap(); - std::fs::write( - own.join("answer.status"), - r#"{"message": "The lab answers", "details": [{"label": "Benches", "value": "3"}]}"#, - ) - .unwrap(); - std::fs::write(own.join("answer.act"), r#"{"message": "All clean"}"#).unwrap(); - std::fs::write(own.join("say.act"), "sweeping\n").unwrap(); - let mut home = Home { - hosts: vec![], - selected: 0, - mode: HomeMode::Hosts, - status: String::new(), - started: std::time::Instant::now(), - qr: None, - berth_key: None, - served: None, - known: None, - asking: false, - refresh: false, - rotating: vec![], - }; - let key = |home: &mut Home, code| { - home_key( - home, - &paths, - KeyEvent::new(code, KeyModifiers::NONE), - &mut Clipboard::default(), - ); - }; - // What runs in a thread is taken in as the screen would, until `done` says so. - let settle = |home: &mut Home, done: &dyn Fn(&str) -> bool| -> String { - for _ in 0..500 { - if let HomeMode::Provisioners(view) = &mut home.mode { - view.load(&paths); - view.pump(&paths, &mut home.status); - } - let screen = home_screen(&paths, home).join("\n"); - if done(&screen) { - return screen; - } - std::thread::sleep(std::time::Duration::from_millis(10)); - } - panic!( - "the screen did not settle:\n{}", - home_screen(&paths, home).join("\n") - ); - }; - // The hosts' list makes instances and compose projects too, with - // the provisioner that makes them; here none does yet. - let screen = settle(&mut home, &|screen| { - screen.contains("+ New compose project") - }); - assert!(screen.contains("+ New instance"), "{screen}"); - home.selected = home.hosts.len() + 1; - key(&mut home, KeyCode::Enter); - assert!(matches!(home.mode, HomeMode::Provisioners(_))); - settle(&mut home, &|screen| { - screen.contains("No provisioner makes instances") - }); - key(&mut home, KeyCode::Char('2')); - settle(&mut home, &|screen| screen.contains("+ Add provisioner")); - // A provisioner is registered under a name of its own, of a type a plugin brings. - key(&mut home, KeyCode::Char('a')); - settle(&mut home, &|screen| { - screen.contains("Add provisioner · 1 of 2") - }); - for ch in "lab".chars() { - key(&mut home, KeyCode::Char(ch)); - } - key(&mut home, KeyCode::Enter); - // The lab's type sorts first, so it is offered first. - let screen = settle(&mut home, &|screen| { - screen.contains("Add provisioner · 2 of 2") - }); - assert!(screen.contains("aa-lab/box"), "{screen}"); - key(&mut home, KeyCode::Enter); - let screen = settle(&mut home, &|screen| screen.contains("Lab box")); - // Its type's label names it; the type itself is on its page. - assert!( - screen.contains("lab") && !screen.contains("(aa-lab/box)"), - "{screen}" - ); - assert_eq!( - berth_core::provisioner::entry(&paths, "lab") - .unwrap() - .unwrap() - .kind, - "aa-lab/box" - ); - // Its page asks it how it is. - key(&mut home, KeyCode::Enter); - let screen = settle(&mut home, &|screen| screen.contains("The lab answers")); - assert!( - screen.contains("● ready") && screen.contains("Benches 3"), - "{screen}" - ); - assert!(screen.contains("▸ Look around") && screen.contains("▸ Sweep")); - // What it makes is asked for in a form over the whole screen, laid - // out as a host's set-up is. Esc goes back to the page. - assert!(screen.contains("▸ New instance"), "{screen}"); - key(&mut home, KeyCode::Enter); - let screen = home_screen(&paths, &home).join("\n"); - assert!(screen.contains("lab › new instance"), "{screen}"); - assert!( - screen.contains("Image debian:12") && screen.contains("Notes Not set"), - "{screen}" - ); - assert!( - screen.contains("[Run · New instance]") && screen.contains("╭ Information "), - "{screen}" - ); - assert!(!screen.contains("1 Hosts"), "{screen}"); - // A text is edited in the editor, which has the screen to itself. - for code in [KeyCode::Down, KeyCode::Down, KeyCode::Enter] { - key(&mut home, code); - } - key(&mut home, KeyCode::Char('1')); - let screen = home_screen(&paths, &home).join("\n"); - assert!( - screen.contains("╭ Notes · line 1 of 1, column 2 "), - "{screen}" - ); - assert!(!screen.contains("Information"), "{screen}"); - key(&mut home, KeyCode::Esc); - key(&mut home, KeyCode::Esc); - let screen = home_screen(&paths, &home).join("\n"); - assert!(screen.contains("▸ Look around"), "{screen}"); - // One that only reads answers in a panel too, which Esc closes. - key(&mut home, KeyCode::Down); - key(&mut home, KeyCode::Enter); - let screen = settle(&mut home, &|screen| screen.contains("All clean")); - assert!(!screen.contains("sweeping"), "{screen}"); - key(&mut home, KeyCode::Esc); - assert!(matches!(home.mode, HomeMode::Provisioners(_))); - // One that changes something asks first, and is a job in a panel. - key(&mut home, KeyCode::Down); - key(&mut home, KeyCode::Enter); - assert!( - home_screen(&paths, &home) - .join("\n") - .contains("Sweep the lab?") - ); - key(&mut home, KeyCode::Char('y')); - let screen = settle(&mut home, &|screen| screen.contains("✓ Done")); - assert!( - screen.contains("sweeping") && screen.contains("✓ All clean"), - "{screen}" - ); - // Keys go to the panel until it is closed, and then back to the page. - key(&mut home, KeyCode::Char('1')); - assert!(matches!(home.mode, HomeMode::Provisioners(_))); - key(&mut home, KeyCode::Enter); - key(&mut home, KeyCode::Esc); - key(&mut home, KeyCode::Esc); - assert!(matches!(home.mode, HomeMode::Hosts)); - // The hosts' list opens the same form, which chooses the provisioner - // too; Esc there goes back to the hosts. - home.selected = home.hosts.len() + 1; - key(&mut home, KeyCode::Enter); - let screen = settle(&mut home, &|screen| screen.contains("[Run · New instance]")); - assert!( - screen.contains("Provisioner Lab box · lab") && screen.contains("Name * Not set"), - "{screen}" - ); - key(&mut home, KeyCode::Esc); - assert!(matches!(home.mode, HomeMode::Hosts)); - } - - #[test] - fn hints_can_be_clicked() { - let keys = - "↑↓ select · Enter open · Ctrl+V paste · x remove · F3 browse · Click opens a link"; - note_hints(keys, 4, 9, true); - let click = |column| { - hint_click(Event::Mouse(MouseEvent { - kind: MouseEventKind::Down(MouseButton::Left), - column, - row: 9, - modifiers: KeyModifiers::NONE, - })) - }; - let key = |code, modifiers| Event::Key(KeyEvent::new(code, modifiers)); - // The arrows are not one key, so a click there stays a click. - assert!(matches!(click(5), Event::Mouse(_))); - assert_eq!(click(15), key(KeyCode::Enter, KeyModifiers::NONE)); - assert_eq!(click(27), key(KeyCode::Char('v'), KeyModifiers::CONTROL)); - assert_eq!(click(45), key(KeyCode::Char('x'), KeyModifiers::NONE)); - // Between two hints. - assert!(matches!(click(49), Event::Mouse(_))); - assert_eq!(click(51), key(KeyCode::F(3), KeyModifiers::NONE)); - assert!(matches!(click(70), Event::Mouse(_))); - assert_eq!(pasted("AB12-CD34"), "Pasted 9 characters"); - } - - #[test] - fn a_session_draws_its_terminal_and_finds_a_wrapped_link() { - let address = format!("https://example.com/login?code={}", "a".repeat(40)); - let mut command = std::process::Command::new("printf"); - command - .arg("Open %s now\\r\\ncode: AB12-CD34\\r\\n") - .arg(&address); - let mut session = crate::session::Session::start("demo", command, 6, 30).unwrap(); - for _ in 0..200 { - session.tick(); - if !session.running() { - break; - } - std::thread::sleep(std::time::Duration::from_millis(10)); - } - // The last output can arrive a moment after the process has ended. - for _ in 0..5 { - std::thread::sleep(std::time::Duration::from_millis(10)); - session.tick(); - } - let mut terminal = Terminal::new(TestBackend::new(60, 12)).unwrap(); - let (mut links, mut words) = (vec![], vec![]); - terminal - .draw(|frame| (links, words) = draw_session(frame, &session, "")) - .unwrap(); - // Every cell of the address opens the whole of it, over three rows. - assert_eq!(links.len(), address.len()); - assert!(links.iter().all(|(_, link)| *link == address)); - assert_eq!(links.last().unwrap().0.y - links[0].0.y, 2); - assert!(words.iter().any(|(_, word)| word == "AB12-CD34")); - } - - fn screen_text(app: &mut App) -> String { - screen(app) - } - - #[test] - fn registries_are_edited_on_their_own_tab() { - let directory = tempfile::tempdir().unwrap(); - let paths = Paths { - config_dir: directory.path().join("config"), - data_dir: directory.path().join("data"), - }; - let mut home = Home { - hosts: vec![], - selected: 0, - mode: HomeMode::Hosts, - status: String::new(), - started: std::time::Instant::now(), - qr: None, - berth_key: None, - served: None, - known: None, - asking: false, - refresh: false, - rotating: vec![], - }; - let keys = |home: &mut Home, codes: &[KeyCode]| { - for &code in codes { - home_key( - home, - &paths, - KeyEvent::new(code, KeyModifiers::NONE), - &mut Clipboard::default(), - ); - } - }; - let typed = |text: &str| text.chars().map(KeyCode::Char).collect::>(); - use KeyCode::{Down, Enter, Up}; - // Hosts, then the Registries tab: only "+ Add registry" is there. - keys(&mut home, &[KeyCode::Char('4'), Enter]); - keys(&mut home, &typed("Bad Name")); - keys(&mut home, &[Enter]); - assert!(home.status.starts_with("Use lower-case")); - for _ in 0..8 { - keys(&mut home, &[KeyCode::Backspace]); - } - keys(&mut home, &typed("nexus")); - keys(&mut home, &[Enter, Down, Down, Enter]); - keys(&mut home, &typed("https://nexus.example/")); - // User name, then the password, which the screen does not show. - keys(&mut home, &[Enter, Down, Enter]); - keys(&mut home, &typed("ci")); - keys(&mut home, &[Enter, Down, Enter]); - keys(&mut home, &typed("hunter2")); - let mut terminal = Terminal::new(TestBackend::new(110, 30)).unwrap(); - let mut shown = |home: &Home| { - terminal - .draw(|frame| draw_home(frame, &paths, home, &mut vec![])) - .unwrap(); - let buffer = terminal.backend().buffer(); - (0..30) - .map(|y| { - (0..110) - .map(|x| buffer[(x, y)].symbol()) - .collect::() - }) - .collect::>() - .join("\n") - }; - let screen = shown(&home); - assert!(screen.contains("•••••••") && !screen.contains("hunter2")); - keys(&mut home, &[Enter]); - // A repository: added as npm, switched to raw and on to docker, which asks for an address. - keys(&mut home, &[Down, Enter]); - keys(&mut home, &typed("images")); - keys(&mut home, &[Enter, Down, Enter, Enter, Down, Enter]); - keys(&mut home, &typed("nexus.example:8083")); - keys(&mut home, &[Enter]); - let store = Store::new(&paths).unwrap(); - assert_eq!( - store.registries().unwrap()["nexus"], - json!({"type": "nexus", "url": "https://nexus.example", "username": "ci", "repositories": [ - {"format": "docker", "name": "images", "address": "nexus.example:8083"}]}) - ); - assert_eq!( - store.secret("registry/nexus").unwrap().as_deref(), - Some("hunter2") - ); - let screen = shown(&home); - println!("{screen}"); - assert!(screen.contains("Registries") && screen.contains("● stored")); - assert!(screen.contains("Docker address nexus.example:8083")); - - // A second registry of another kind: a plain npm registry with a token. - keys(&mut home, &[Down, Down, Enter]); - keys(&mut home, &typed("verdaccio")); - keys(&mut home, &[Enter, Down, Enter]); - assert_eq!(store.registries().unwrap()["verdaccio"]["type"], "npm"); - // Removing a registry takes x twice and forgets its login. - for _ in 0..12 { - keys(&mut home, &[Up]); - } - keys(&mut home, &[KeyCode::Char('x')]); - assert!(store.registries().unwrap().contains_key("nexus")); - keys(&mut home, &[KeyCode::Char('x')]); - assert!(!store.registries().unwrap().contains_key("nexus")); - assert_eq!(store.secret("registry/nexus").unwrap(), None); - } - - fn press(app: &mut App, codes: &[KeyCode]) -> bool { - let mut armed = false; - for &code in codes { - app.update_suggestions(); - let key = KeyEvent::new(code, KeyModifiers::NONE); - if matches!( - key_event(app, key, &mut armed, &mut Clipboard::default()), - Flow::Run - ) { - return true; - } - } - false - } - - #[test] - fn settings_offer_the_link_to_the_servers_page_as_a_qr_code() { - let directory = tempfile::tempdir().unwrap(); - let paths = Paths { - config_dir: directory.path().join("config"), - data_dir: directory.path().join("data"), - }; - let mut home = Home { - hosts: vec![], - selected: 0, - mode: HomeMode::Settings(0, None), - status: String::new(), - started: std::time::Instant::now(), - qr: None, - berth_key: None, - served: None, - known: None, - asking: false, - refresh: false, - rotating: vec![], - }; - let last = berth_core::paths::SETTINGS.len(); - let down = |home: &mut Home| { - for _ in 0..last + 2 { - home_key( - home, - &paths, - KeyEvent::new(KeyCode::Down, KeyModifiers::NONE), - &mut Clipboard::default(), - ); - } - }; - // Without a server, or with one reached through a socket, there is no such row. - down(&mut home); - assert!(matches!(home.mode, HomeMode::Settings(index, _) if index == last - 1)); - paths.set("server", "https://berth.example.ts.net").unwrap(); - paths.set("server_token", "a1b2").unwrap(); - down(&mut home); - assert!(matches!(home.mode, HomeMode::Settings(index, _) if index == last)); - let mut terminal = Terminal::new(TestBackend::new(110, 50)).unwrap(); - let mut shown = |home: &Home| { - terminal - .draw(|frame| draw_home(frame, &paths, home, &mut vec![])) - .unwrap(); - let buffer = terminal.backend().buffer(); - (0..50) - .map(|y| { - (0..110) - .map(|x| buffer[(x, y)].symbol()) - .collect::() - }) - .collect::>() - .join("\n") - }; - let screen = shown(&home); - assert!(screen.contains("Open on another device") && screen.contains("server's token")); - home_key( - &mut home, - &paths, - KeyEvent::new(KeyCode::Enter, KeyModifiers::NONE), - &mut Clipboard::default(), - ); - let overlay = home.qr.clone().unwrap(); - assert_eq!(overlay.link(), "https://berth.example.ts.net/#token=a1b2"); - let screen = shown(&home); - assert!( - screen.contains('▀') && screen.contains("#token=a1b2"), - "{screen}" - ); - } - - #[test] - fn alt_r_shows_the_links_of_a_panel_as_qr_codes() { - let mut app = unsaved_app(b"{}"); - app.links = vec![( - Rect::new(0, 0, 1, 1), - "https://studio.example.ts.net/vnc.html".into(), - )]; - let alt_r = Event::Key(KeyEvent::new(KeyCode::Char('r'), KeyModifiers::ALT)); - assert!(qr_event(&mut app, &alt_r)); - assert_eq!( - app.qr.as_ref().map(|qr| qr.link()), - Some("https://studio.example.ts.net/vnc.html") - ); - assert!(qr_event( - &mut app, - &Event::Key(KeyEvent::new(KeyCode::Esc, KeyModifiers::NONE)) - )); - assert!(app.qr.is_none()); - // A plain r is the panel's. - let r = Event::Key(KeyEvent::new(KeyCode::Char('r'), KeyModifiers::NONE)); - assert!(!qr_event(&mut app, &r)); - } - - #[test] - fn settings_are_edited_from_the_home_screen() { - let directory = tempfile::tempdir().unwrap(); - let paths = Paths { - config_dir: directory.path().join("config"), - data_dir: directory.path().join("data"), - }; - let mut home = Home { - hosts: vec![("laptop".into(), "100.64.0.1".into())], - selected: 0, - mode: HomeMode::Hosts, - status: String::new(), - started: std::time::Instant::now(), - qr: None, - berth_key: None, - served: None, - known: None, - asking: false, - refresh: false, - rotating: vec![], - }; - let press = |home: &mut Home, code| { - home_key( - home, - &paths, - KeyEvent::new(code, KeyModifiers::NONE), - &mut Clipboard::default(), - ) - }; - assert!(press(&mut home, KeyCode::Char('d')).is_none()); - assert_eq!(paths.settings().unwrap()["default_host"], "laptop"); - for code in [ - KeyCode::Char('3'), - KeyCode::Down, - KeyCode::Down, - KeyCode::Enter, - ] { - press(&mut home, code); - } - for ch in "/srv/iso".chars() { - press(&mut home, KeyCode::Char(ch)); - } - press(&mut home, KeyCode::Enter); - assert_eq!(paths.settings().unwrap()["iso_dir"], "/srv/iso"); - let mut terminal = Terminal::new(TestBackend::new(100, 24)).unwrap(); - terminal - .draw(|frame| draw_home(frame, &paths, &home, &mut vec![])) - .unwrap(); - let text = format!("{:?}", terminal.backend().buffer()); - assert!(text.contains("█ ▄▄ █") && text.contains("▂") && text.contains("/srv/iso")); - // Clicking the Hosts tab and then the host opens it. - let mut hits = vec![]; - let mut click = |home: &mut Home, column, row| { - terminal - .draw(|frame| draw_home(frame, &paths, home, &mut hits)) - .unwrap(); - let mouse = MouseEvent { - kind: MouseEventKind::Down(MouseButton::Left), - column, - row, - modifiers: KeyModifiers::NONE, - }; - home_mouse(home, &paths, mouse, &hits, &mut Clipboard::default()) - }; - assert!(click(&mut home, 15, 1).is_none()); - assert!(matches!(home.mode, HomeMode::Hosts)); - assert_eq!(click(&mut home, 10, 7), Some(Some("laptop".into()))); - } - - /// A home screen on the Settings tab of fresh scratch directories, and the - /// text of the screen after drawing it. - fn settings_home(directory: &std::path::Path) -> (Paths, Home) { - let paths = Paths { - config_dir: directory.join("config"), - data_dir: directory.join("data"), - }; - let home = Home { - hosts: vec![], - selected: 0, - mode: HomeMode::Settings(0, None), - status: String::new(), - started: std::time::Instant::now(), - qr: None, - berth_key: None, - served: None, - known: None, - asking: false, - refresh: false, - rotating: vec![], - }; - (paths, home) - } - - fn home_screen(paths: &Paths, home: &Home) -> Vec { - let mut terminal = Terminal::new(TestBackend::new(140, 30)).unwrap(); - terminal - .draw(|frame| draw_home(frame, paths, home, &mut vec![])) - .unwrap(); - let buffer = terminal.backend().buffer(); - (0..30) - .map(|y| (0..140).map(|x| buffer[(x, y)].symbol()).collect()) - .collect() - } - - #[test] - fn the_footers_leave_the_tabs_to_the_tab_bar() { - let directory = tempfile::tempdir().unwrap(); - let (paths, mut home) = settings_home(directory.path()); - for mode in [ - HomeMode::Hosts, - HomeMode::Settings(0, None), - HomeMode::Registries(RegistryView::default()), - HomeMode::Secrets(SecretView::default()), - HomeMode::Plugins(PluginView::default()), - ] { - home.mode = mode; - let rows = home_screen(&paths, &home); - let footer = rows - .iter() - .rev() - .find(|row| !row.trim().is_empty()) - .unwrap(); - for tab in &berth_tui::screen::HOME_TABS[1..] { - assert!( - !footer.to_lowercase().contains(&tab.to_lowercase()), - "{footer}" - ); - } - } - // The tab bar has them all, with the digit that opens each. - let rows = home_screen(&paths, &home).join("\n"); - for tab in berth_tui::screen::HOME_TABS { - assert!(rows.contains(tab), "{tab}"); - } - } - - #[test] - fn the_text_of_a_field_is_copied_with_its_hint_or_its_key() { - let directory = tempfile::tempdir().unwrap(); - let (paths, mut home) = settings_home(directory.path()); - let mut clipboard = Clipboard::default(); - home.mode = HomeMode::Settings(0, Some(Input::new("/srv/provision".into()))); - let rows = home_screen(&paths, &home); - let last = rows - .iter() - .position(|row| row.contains("Ctrl+O copy")) - .expect("the hint is shown"); - let column = rows[last].find("Ctrl+O copy").unwrap(); - assert!(rows[last].contains("Enter accept") && rows[last].contains("Esc cancel")); - // The rows hold only one-cell characters up to the hint, so a byte is a column. - let click = |column: usize| { - hint_click(Event::Mouse(MouseEvent { - kind: MouseEventKind::Down(MouseButton::Left), - column: column as u16, - row: last as u16, - modifiers: KeyModifiers::NONE, - })) - }; - // Enter and Esc are hints like it. - assert_eq!( - click(rows[last].find("Esc cancel").unwrap()), - Event::Key(KeyEvent::new(KeyCode::Esc, KeyModifiers::NONE)) - ); - let Event::Key(copy) = click(column + 2) else { - panic!("a click on the hint is a key"); - }; - assert_eq!( - copy, - KeyEvent::new(KeyCode::Char('o'), KeyModifiers::CONTROL) - ); - assert!(home_key(&mut home, &paths, copy, &mut clipboard).is_none()); - assert_eq!(clipboard.0.as_deref(), Some("/srv/provision")); - assert_eq!(home.status, "Copied the text of the field"); - // The field is still open and unchanged, and the status does not say the text. - assert!( - matches!(&home.mode, HomeMode::Settings(0, Some(input)) if input.value() == "/srv/provision") - ); - } - - #[test] - fn a_secret_setting_is_shortened_in_the_list_and_whole_in_its_field() { - let directory = tempfile::tempdir().unwrap(); - let (paths, mut home) = settings_home(directory.path()); - let token = "tok-0123456789abcdefghijklmnopqrstuvwxyz"; - paths.set("server_token", token).unwrap(); - paths.set("default_host", "studio").unwrap(); - let rows = home_screen(&paths, &home).join("\n"); - assert!(!rows.contains(token) && !rows.contains("0123456789abcdef")); - assert!(rows.contains("server_token …wxyz")); - // What is not a secret stays whole. - assert!(rows.contains("Default host studio")); - // The description reads right, with no "The" in front. - let at = crate::paths::SETTINGS - .iter() - .position(|(name, _)| *name == "server_token") - .unwrap(); - home.mode = HomeMode::Settings(at, None); - home_key( - &mut home, - &paths, - KeyEvent::new(KeyCode::Enter, KeyModifiers::NONE), - &mut Clipboard::default(), - ); - let edited = home_screen(&paths, &home).join("\n"); - assert!(edited.contains(token)); - // The description reads right, with no "The" in front of "the token". - assert!(!edited.contains("The the") && edited.contains("The token of that berth")); - assert!(crate::paths::is_secret("store")); - assert!(!crate::paths::is_secret("serve_token")); - } - - #[test] - fn the_home_screen_draws_while_the_mirror_has_not_returned() { - let directory = tempfile::tempdir().unwrap(); - let (paths, mut home) = settings_home(directory.path()); - home.mode = HomeMode::Hosts; - // A mirror that waits for the test to let it end. - let (release, wait) = std::sync::mpsc::channel::(); - let wait = std::sync::Mutex::new(wait); - let mut mirroring = Some(crate::mirror::Mirroring::new(move || { - wait.lock().unwrap().recv().unwrap() - })); - mirroring.as_mut().unwrap().start(); - home_mirror(&mut home, &paths, &mut mirroring); - assert!(home.asking); - assert!(home_screen(&paths, &home).join("\n").contains(ASKING)); - // Starting another while one runs starts nothing. - assert!(!mirroring.as_mut().unwrap().start()); - // A host the mirror brings is listed once it is there, and not before. - Place::own("studio") - .unwrap() - .save(&paths, &json!({"access": {"address": "192.0.2.10"}})) - .unwrap(); - assert!(home.hosts.is_empty()); - release.send(Ok(())).unwrap(); - while mirroring.as_ref().unwrap().running() { - home_mirror(&mut home, &paths, &mut mirroring); - std::thread::yield_now(); - } - assert!(!home.asking); - assert_eq!(home.hosts, [("studio".to_owned(), "192.0.2.10".to_owned())]); - assert!(!home_screen(&paths, &home).join("\n").contains(ASKING)); - // A mirror that fails says why and keeps what was there. - mirroring.as_mut().unwrap().start(); - release - .send(Err( - "the berth server at http://192.0.2.7:7777: timed out".into() - )) - .unwrap(); - while mirroring.as_ref().unwrap().running() { - home_mirror(&mut home, &paths, &mut mirroring); - std::thread::yield_now(); - } - assert!(home.status.contains("timed out") && home.hosts.len() == 1); - } - - #[test] - fn hosts_show_whether_the_server_reaches_them() { - let directory = tempfile::tempdir().unwrap(); - let (paths, mut home) = settings_home(directory.path()); - home.mode = HomeMode::Hosts; - paths.set("server", "http://192.0.2.7:7777").unwrap(); - let server = berth_core::server::Server::configured(&paths).unwrap(); - for (name, address) in [ - ("attic", "192.0.2.20"), - ("studio", "192.0.2.10"), - ("yard", "192.0.2.30"), - ] { - Place::stored(&server.scope(), name) - .unwrap() - .save(&paths, &json!({"access": {"address": address}})) - .unwrap(); - } - let state = |version: &str| { - format!( - r#"{{"version":"{version}","hosts":{{"studio":{{"reachable":true,"error":null,"version":"0.3.0"}},"attic":{{"reachable":false,"error":"ssh: connect to host 192.0.2.20: timed out","version":null}}}}}}"# - ) - }; - // What a mirror keeps of the server's state is in the store, under its scope. - let store = berth_core::store::shared(&paths).unwrap(); - let mirrored = |state: String| { - store - .write( - berth_core::store::kind::MIRROR, - &server.scope(), - "state", - state.as_bytes(), - ) - .unwrap() - }; - mirrored(state("0.3.0")); - home.hosts = paths.hosts(); - home.known = known_of(&paths); - let rows = home_screen(&paths, &home); - let line = |name: &str| rows.iter().find(|row| row.contains(name)).unwrap().clone(); - assert!(line("studio").contains("● studio")); - assert!(line("attic").contains("● attic")); - assert!(line("yard").contains("○ yard")); - // The selected host, the first, is attic: its error is in the panel. - let text = rows.join("\n"); - assert!(text.contains("does not answer") && text.contains("timed out")); - // The server is older than this berth, so the screen says so. - assert!(text.contains("The berth server runs 0.3.0")); - assert!(text.contains("Update the berth server")); - for same in [berth_api::VERSION.to_owned(), "99.0.0".to_owned()] { - mirrored(state(&same)); - home.known = known_of(&paths); - let text = home_screen(&paths, &home).join("\n"); - assert!(!text.contains("The berth server runs")); - } - // With no server there is no marker and no panel. - paths.set("server", "").unwrap(); - home.known = known_of(&paths); - assert!(home.known.is_none()); - assert!(!home_screen(&paths, &home).join("\n").contains('●')); - } - - #[test] - fn an_unreachable_host_can_have_its_address_changed() { - use crate::remote::State; - let directory = tempfile::tempdir().unwrap(); - let mut app = file_app(directory.path(), "studio.json", b"{}"); - app.set_step(Step::Manage); - app.commit(Field::SshHost, "192.0.2.20"); - let error = "ssh: connect to host 192.0.2.20 port 22: Connection timed out and a long text that goes on past the width of the list so that it has to be wrapped over lines\nsecond line of the error"; - app.host.state = State::Unreachable(error.into()); - let text = screen_text(&mut app); - assert!(text.contains("This machine could not reach the host")); - assert!(text.contains("second line of the error") && text.contains("wrapped over lines")); - assert!(text.contains("Change the SSH address") && text.contains("Try again")); - // A host of the berth server is reached by the server. - app.config["builder"]["served"] = json!("http://192.0.2.7:7777"); - let text = screen_text(&mut app); - assert!(text.contains("The berth server at http://192.0.2.7:7777 could not")); - app.config["builder"] - .as_object_mut() - .unwrap() - .remove("served"); - // The action opens the address for editing. - choose(&mut app, Entry::Task("address")); - assert!(app.retry_address && app.editing.is_some(), "{}", app.status); - assert_eq!(app.selected_field(), Some(Field::SshHost)); - let mut armed = false; - let mut clipboard = Clipboard::default(); - for code in [KeyCode::Char('9'), KeyCode::Enter] { - let event = KeyEvent::new(code, KeyModifiers::NONE); - key_event(&mut app, event, &mut armed, &mut clipboard); - } - // Saved, and the overview asks the host again. - assert!(!app.retry_address && app.step == Step::Manage); - assert!(app.text(Field::SshHost).ends_with('9')); - assert!(directory.path().join("studio.json").is_file()); - assert!(matches!(app.host.state, State::Idle | State::Connecting)); - } - - #[test] - fn the_hosts_tab_refreshes_with_r_and_the_hint_is_clickable() { - let directory = tempfile::tempdir().unwrap(); - let (paths, mut home) = settings_home(directory.path()); - home.mode = HomeMode::Hosts; - let press = |home: &mut Home, ch| { - home_key( - home, - &paths, - KeyEvent::new(KeyCode::Char(ch), KeyModifiers::NONE), - &mut Clipboard::default(), - ) - }; - // Without a server it reads the saved hosts again. - Place::own("studio") - .unwrap() - .save(&paths, &json!({"access": {"address": "192.0.2.10"}})) - .unwrap(); - let rows = home_screen(&paths, &home); - let last = rows - .iter() - .position(|row| row.contains("r refresh")) - .unwrap(); - let column = rows[last].find("r refresh").unwrap(); - let Event::Key(clicked) = hint_click(Event::Mouse(MouseEvent { - kind: MouseEventKind::Down(MouseButton::Left), - column: column as u16, - row: last as u16, - modifiers: KeyModifiers::NONE, - })) else { - panic!("a click on the hint is a key"); - }; - assert_eq!(clicked.code, KeyCode::Char('r')); - assert!(home.hosts.is_empty()); - home_key(&mut home, &paths, clicked, &mut Clipboard::default()); - home_mirror(&mut home, &paths, &mut None); - assert_eq!(home.hosts.len(), 1); - // With a server it starts the mirror, and not a second while one runs. - let (release, wait) = std::sync::mpsc::channel::(); - let wait = std::sync::Mutex::new(wait); - let mut mirroring = Some(crate::mirror::Mirroring::new(move || { - wait.lock().unwrap().recv().unwrap() - })); - press(&mut home, 'r'); - home_mirror(&mut home, &paths, &mut mirroring); - assert!(home.asking); - press(&mut home, 'r'); - home_mirror(&mut home, &paths, &mut mirroring); - assert!(home.status.contains("Still asking")); - release.send(Ok(())).unwrap(); - while mirroring.as_ref().unwrap().running() { - home_mirror(&mut home, &paths, &mut mirroring); - std::thread::yield_now(); - } - // Typing a name is not a refresh, and r is no longer the Registries tab. - press(&mut home, 'n'); - press(&mut home, 'r'); - assert!(matches!(&home.mode, HomeMode::Naming(input) if input.value() == "r")); - assert!(!home.refresh); - home.mode = HomeMode::Hosts; - press(&mut home, 'r'); - assert!(matches!(home.mode, HomeMode::Hosts)); - home.mode = HomeMode::Settings(0, None); - press(&mut home, 'r'); - assert!(matches!(home.mode, HomeMode::Settings(..))); - } - - #[test] - fn the_copy_hint_of_a_host_edit_can_be_clicked() { - let directory = tempfile::tempdir().unwrap(); - let mut app = file_app(directory.path(), "studio.json", b"{}"); - app.commit(Field::SshHost, "192.0.2.20"); - app.editing = Some(Input::new("192.0.2.20".into())); - let rows: Vec = screen(&mut app).lines().map(str::to_owned).collect(); - let at = rows - .iter() - .position(|row| row.contains("Ctrl+O copy")) - .unwrap(); - let column = rows[at].find("Ctrl+O copy").unwrap(); - let click = |column: usize| { - hint_click(Event::Mouse(MouseEvent { - kind: MouseEventKind::Down(MouseButton::Left), - column: column as u16, - row: at as u16, - modifiers: KeyModifiers::NONE, - })) - }; - let Event::Key(copy) = click(column + 1) else { - panic!("a click on the hint is a key"); - }; - let mut clipboard = Clipboard::default(); - key_event(&mut app, copy, &mut false, &mut clipboard); - assert_eq!(clipboard.0.as_deref(), Some("192.0.2.20")); - assert!(app.editing.is_some()); - // Enter is a hint of the line too. - let enter = rows[at].find("Enter accept").unwrap(); - assert!(matches!(click(enter), Event::Key(key) if key.code == KeyCode::Enter)); - } - - #[test] - fn a_mirror_is_taken_in_unless_the_host_has_unsaved_changes() { - let directory = tempfile::tempdir().unwrap(); - let paths = Paths { - config_dir: directory.path().join("config"), - data_dir: directory.path().join("data"), - }; - let place = Place::stored("served:http://192.0.2.7:7777", "studio").unwrap(); - // What a mirror writes: the profile under the server's scope. - let write = |address: &str| { - place - .save(&paths, &json!({"access": {"address": address}})) - .unwrap() - }; - write("192.0.2.20"); - let stored = place.load(&paths).unwrap().unwrap(); - let mut app = App::new( - paths.clone(), - place.clone(), - parse_config(stored.to_string().as_bytes()).unwrap(), - ); - assert_eq!(app.take_in_mirror(), None); - // A new address in the mirror is taken in. - write("192.0.2.21"); - assert!(app.take_in_mirror().unwrap().contains("Took in")); - assert_eq!(app.text(Field::SshHost), "192.0.2.21"); - assert!(!app.unsaved()); - // With a change of its own the screen keeps it and says so. - app.commit(Field::SshHost, "192.0.2.99"); - write("192.0.2.22"); - let said = app.take_in_mirror().unwrap(); - assert!(said.contains("Saving sends this one")); - assert_eq!(app.text(Field::SshHost), "192.0.2.99"); - } - - #[test] - fn what_a_command_saved_of_the_host_is_not_overwritten_by_the_screen() { - let directory = tempfile::tempdir().unwrap(); - let paths = Paths { - config_dir: directory.path().join("config"), - data_dir: directory.path().join("data"), - }; - let place = Place::own("studio").unwrap(); - place - .save(&paths, &json!({"access": {"user": "ops"}})) - .unwrap(); - let open = || { - let stored = place.load(&paths).unwrap().unwrap(); - App::new( - paths.clone(), - place.clone(), - parse_config(stored.to_string().as_bytes()).unwrap(), - ) - }; - // What `berth provision` does from the screen: the machine's address - // and the cloud profile go into the stored profile. - let provisioned = |port: &str| { - let mut kept = place.load(&paths).unwrap().unwrap(); - kept["access"]["address"] = json!("192.0.2.40"); - kept["access"]["port"] = json!(port); - kept["profile"] = json!("cloud"); - place.save(&paths, &kept).unwrap(); - }; - let mut app = open(); - assert_eq!(app.take_in_command(), None); - provisioned("2222"); - assert!(app.take_in_command().unwrap().contains("Took in")); - assert!(!app.unsaved()); - app.save().unwrap(); - let kept = place.load(&paths).unwrap().unwrap(); - assert_eq!(kept["access"]["address"], "192.0.2.40"); - - // With changes not saved yet the address joins them, and saving keeps both. - let mut app = open(); - app.commit(Field::Keyboard, "us"); - provisioned("2200"); - let said = app.take_in_command().unwrap(); - assert!(said.contains("not saved yet"), "{said}"); - assert!(app.unsaved()); - assert_eq!(app.text(Field::SshPort), "2200"); - app.save().unwrap(); - let kept = place.load(&paths).unwrap().unwrap(); - assert_eq!( - (&kept["access"]["port"], &kept["keyboard"], &kept["profile"]), - (&json!("2200"), &json!("us"), &json!("cloud")) - ); - } - - #[test] - fn words_of_output_can_be_copied() { - assert_eq!( - words("App: code K79HUTDTLL75"), - [ - (0, "App".to_owned()), - (5, "code".to_owned()), - (15, "K79HUTDTLL75".to_owned()) - ] - ); - // The address inside brackets, without them or the semicolon. - assert_eq!( - words("(or https://100.64.0.1:9090; log in)")[1], - (4, "https://100.64.0.1:9090".to_owned()) - ); - } - - #[test] - fn web_addresses_in_output_are_found() { - let parts = - with_links("Cockpit https://box.ts.net:9090 (or https://100.64.0.1:9090; log in)"); - let links: Vec<_> = parts - .iter() - .filter(|part| part.1) - .map(|part| &part.0) - .collect(); - assert_eq!( - links, - ["https://box.ts.net:9090", "https://100.64.0.1:9090"] - ); - let text: String = parts.iter().map(|part| part.0.as_str()).collect(); - assert!(text.ends_with("; log in)") && text.starts_with("Cockpit ")); - assert_eq!( - with_links("ssh ops@host"), - [("ssh ops@host".to_owned(), false)] - ); - } - - /// The text of the rows of the current tab. - fn lines(app: &App) -> Vec { - app.rows().iter().map(|row| row.line.to_string()).collect() - } - - fn select(app: &mut App, entry: Entry) { - let at = app.entries().iter().position(|known| *known == entry); - app.list - .select(Some(at.unwrap_or_else(|| panic!("no {entry:?}")))); - } - - #[test] - fn the_set_up_tabs_go_from_install_to_a_blocked_run() { - let directory = tempfile::tempdir().unwrap(); - let mut app = file_app(directory.path(), "profile.json", b"{}"); - use KeyCode::*; - // A new host opens on Install; the tabs say nothing but their names. - assert_eq!(app.step, Step::Install); - let install = screen(&mut app); - let header = install.lines().nth(1).unwrap(); - let place = |text: &str| header.find(text).unwrap(); - assert!(place("Install") < place("Features") && place("Features") < place("Settings")); - assert!(place("Settings") < place("Run") && !header.contains('✗')); - assert!(install.contains("1-4 tab")); - // The USB installer is the default method; what it needs chosen follows. - press(&mut app, &[Down, Enter, Down, Down, Down, Down, Down, Down]); - assert_eq!(app.action, Action::Iso); - let install = screen(&mut app); - assert!(install.contains("● USB installer")); - assert!(install.contains("▌ Host profile laptop"), "{install}"); - assert!(install.contains("Suggestions")); - assert!(!install.contains("Disk selection") && !install.contains("Source ISO")); - - // Type a hostname here, then cycle the disk mode on Settings from the suggestions. - press( - &mut app, - &[Down, Enter, Char('b'), Char('o'), Char('x'), Enter], - ); - assert_eq!(app.text(Field::Hostname), "box"); - press(&mut app, &[Char('3')]); - assert_eq!(app.step, Step::Settings); - select(&mut app, Entry::Field(Field::DiskMode)); - press(&mut app, &[Right, Down, Enter]); - assert_eq!(app.text(Field::DiskMode), "serial"); - let settings = lines(&app); - assert!(settings.contains(&" │ └─ Target disk serial Not set".to_owned())); - assert_eq!( - settings.last().unwrap(), - "✗ Serial disk selection needs the target disk serial" - ); - - // Run has the ISO, every problem under its tab, and a button that refuses. - press(&mut app, &[Char('4')]); - let run = lines(&app); - assert!(run[0].starts_with("Source ISO")); - let at = |text: &str| run.iter().position(|row| row.contains(text)).unwrap(); - assert!(at("Settings") < at(" ✗ Serial disk selection needs the target disk serial")); - assert!(at("Run") < at(" ✗ Source ISO file does not exist")); - select(&mut app, Entry::Run); - assert!(!press(&mut app, &[Enter])); - assert!(screen(&mut app).contains("[Run · USB installer]")); - assert!(app.status.contains("Fix the listed problems")); - // Save does what Ctrl+S does. - select(&mut app, Entry::Save); - press(&mut app, &[Enter]); - assert!(directory.path().join("profile.json").is_file() && !app.unsaved()); - app.commit(Field::Keyboard, "us"); - - // Unsaved changes need a second q; Ctrl+S clears that. - let mut armed = false; - let q = KeyEvent::new(Char('q'), KeyModifiers::NONE); - // The first q only leaves the set-up pages for the host's overview. - assert!(matches!( - key_event(&mut app, q, &mut armed, &mut Clipboard::default()), - Flow::Continue - )); - assert!(!armed && app.step == Step::Manage); - assert!(matches!( - key_event(&mut app, q, &mut armed, &mut Clipboard::default()), - Flow::Continue - )); - assert!(armed); - let save = KeyEvent::new(Char('s'), KeyModifiers::CONTROL); - key_event(&mut app, save, &mut armed, &mut Clipboard::default()); - assert!(directory.path().join("profile.json").is_file() && !app.unsaved()); - assert!(matches!( - key_event(&mut app, q, &mut armed, &mut Clipboard::default()), - Flow::Quit - )); - } - - #[test] - fn unlock_dialog_masks_the_password_and_closes_on_escape() { - let mut app = unsaved_app(b"{}"); - app.unlock = Some(Input::new("hunter2".into())); - app.unlock_failed = true; - let text = screen(&mut app); - assert!(text.contains("Password for ")); - assert!(text.contains("› •••••••")); - assert!(text.contains("Wrong password")); - assert!(!text.contains("hunter2")); - let escape = Event::Key(KeyEvent::new(KeyCode::Esc, KeyModifiers::NONE)); - unlock_event(&mut app, &escape, &mut Desktop); - assert!(app.unlock.is_none()); - } - - #[test] - fn a_running_action_shows_its_progress_and_output() { - let mut app = unsaved_app(b"{}"); - app.action = Action::Iso; - let mut command = std::process::Command::new("sh"); - command.args([ - "-c", - "echo copying; echo 'xorriso : UPDATE : Writing: 1s 46.4% fifo'; sleep 5", - ]); - let stage = crate::job::Stage { - title: "Build unattended ISO".into(), - command, - source: None, - input: None, - }; - app.job = Some(Job::start(vec![stage], "", vec![]).unwrap()); - let mut text = String::new(); - for _ in 0..200 { - app.job.as_mut().unwrap().tick(); - text = screen(&mut app); - if text.contains("46%") { - break; - } - std::thread::sleep(std::time::Duration::from_millis(10)); - } - assert!(text.contains("46%") && text.contains("copying"), "{text}"); - assert!(text.contains("Esc cancel")); - let escape = Event::Key(KeyEvent::new(KeyCode::Esc, KeyModifiers::NONE)); - job_event(&mut app, &escape); - for _ in 0..200 { - app.job.as_mut().unwrap().tick(); - if !app.job.as_ref().unwrap().running() { - break; - } - std::thread::sleep(std::time::Duration::from_millis(10)); - } - assert!(screen(&mut app).contains("Enter close")); - job_event( - &mut app, - &Event::Key(KeyEvent::new(KeyCode::Enter, KeyModifiers::NONE)), - ); - assert!(app.job.is_none()); - } - - #[test] - fn writing_a_usb_drive_asks_before_erasing() { - let mut app = unsaved_app(b"{}"); - app.confirm = Some("Erase /dev/sdz · Stick · 14.6G?".into()); - let text = screen(&mut app); - assert!(text.contains("Erase /dev/sdz") && text.contains("y erase")); - confirm_event( - &mut app, - &Event::Key(KeyEvent::new(KeyCode::Char('n'), KeyModifiers::NONE)), - ); - assert!(app.confirm.is_none() && app.job.is_none()); - } - - /// A host that has answered, with a few tasks as it would list them. - fn host_tasks() -> Vec { - serde_json::from_value(json!([ - {"name": "status", "label": "Status", "group": "System", "user": "root", "help": "How the host is."}, - {"name": "claude:status", "label": "Claude Code sign-in", "extends": ["sign-in:status"], "help": "Whether Claude is signed in."}, - {"name": "sign-in:status", "label": "Show who is signed in", "group": "Sign-In", "help": "Accounts."}, - {"name": "claude:login", "label": "Sign in to Claude Code", "group": "sign-in", "terminal": true}, - {"name": "reboot", "label": "Reboot", "group": "System", "user": "root", "confirm": "Restart the host?"}, - {"name": "backup:restore", "label": "Restore", "group": "Backup", - "arguments": [{"name": "snapshot", "label": "Snapshot", "required": true, "choices": "restic snapshots"}, - {"name": "target", "default": "/"}]}, - ])) - .unwrap() - } - - #[test] - fn a_host_below_a_workstation_is_offered_the_controller_and_no_apply() { - use crate::remote::State; - let mut app = unsaved_app(br#"{"level": "reachable", "access": {"address": "192.0.2.4"}}"#); - app.set_step(Step::Manage); - app.host.level = berth_api::HostLevel::Reachable; - app.host.state = State::Missing; - app.host.seen = Some(berth_api::HostLevel::Reachable); - app.host.own = - berth_core::level::local_actions("devbox", berth_api::HostLevel::Reachable, None); - let text = screen_text(&mut app); - assert!(text.contains("Level · reachable"), "{text}"); - assert!(text.contains("▸ Install the controller") && text.contains("▸ Terminal over SSH")); - assert!(!app.entries().contains(&Entry::Task("apply"))); - // Choosing it, and saying yes, runs berth host install in the normal terminal. - let at = app - .entries() - .iter() - .position(|entry| *entry == Entry::HostTask(0)) - .unwrap(); - app.list.select(Some(at)); - press(&mut app, &[KeyCode::Enter]); - question_event( - &mut app, - &Event::Key(KeyEvent::new(KeyCode::Char('y'), KeyModifiers::NONE)), - ); - let words = app.own_command.clone().unwrap(); - assert_eq!(words[..2], ["host".to_owned(), "install".to_owned()]); - app.own_command = None; - - // A host that cannot be reached still lists berth's own actions. - app.host.state = State::Unreachable( - "ssh: connect to host 192.0.2.4 port 22: Network is unreachable".into(), - ); - let rows: Vec = app.rows().iter().map(|row| row.line.to_string()).collect(); - let text = rows.join("\n"); - assert!( - text.contains("Network is unreachable") && text.contains("▸ Install the controller"), - "{text}" - ); - assert!(app.entries().contains(&Entry::HostTask(0))); - - // A controller host lists berth's own beside its tasks; one it cannot - // run is marked, and its reason is in the information box. - app.own_command = None; - app.config["level"] = json!("controller"); - app.host.level = berth_api::HostLevel::Controller; - app.host.state = State::Ready(berth_api::VERSION.into()); - app.host.tasks = host_tasks(); - app.host.own = vec![ - berth_core::level::install_controller_action( - "devbox", - berth_api::HostLevel::Controller, - None, - ), - berth_api::Action { - name: "host-key".into(), - label: "Host key".into(), - group: "Access".into(), - available: false, - why: "devbox has no profile on the server".into(), - ..berth_api::Action::default() - }, - ]; - let rows: Vec = app.rows().iter().map(|row| row.line.to_string()).collect(); - assert!( - rows.iter() - .any(|row| row.contains("▸ Update the controller")) - ); - assert!(rows.iter().any(|row| row.contains("▸ Host key · not now"))); - assert_eq!( - rows.iter().filter(|row| row.as_str() == "Set-Up").count(), - 1, - "{rows:?}" - ); - assert!(!app.entries().contains(&Entry::Task("apply"))); - let index = app - .host - .actions() - .iter() - .position(|action| action.name == "host-key") - .unwrap(); - app.begin_task(index, false); - assert!(app.status.contains("cannot run now: devbox has no profile")); - assert!(app.own_command.is_none()); - } - - #[test] - fn an_existing_host_is_offered_the_level_it_says_it_has() { - let mut app = unsaved_app(br#"{"access": {"address": "192.0.2.4"}}"#); - app.offer_level(Some(berth_api::HostLevel::Controller)); - assert_eq!(app.config["level"], "controller"); - assert!(app.status.contains("answers as a controller host")); - // Nothing found, or what it is already, changes nothing. - app.offer_level(None); - app.offer_level(Some(berth_api::HostLevel::Controller)); - assert_eq!(app.config["level"], "controller"); - // A level chosen on its row stays; the host's answer is only told. - app.commit(Field::Level, "reachable"); - app.offer_level(Some(berth_api::HostLevel::Workstation)); - assert_eq!(app.config["level"], "reachable"); - assert!( - app.status.contains("its Level here says reachable"), - "{}", - app.status - ); - } - - #[test] - fn the_overview_is_what_the_host_lists() { - use crate::remote::{Pending, State}; - let mut app = unsaved_app(b"{}"); - app.set_step(Step::Manage); - assert_eq!(app.action, Action::Manage); - // Without an address there is nothing to ask. - let manage = screen(&mut app); - assert!(manage.contains("✗ Enter an SSH hostname") && !manage.contains("▸ Status")); - assert!(app.entries().contains(&Entry::Task("apply"))); - - app.commit(Field::SshHost, "devbox.example"); - app.config["agent_username"] = json!("robot"); - assert!(screen_text(&mut app).contains("Asking the host")); - // A host without berth's controller offers to apply, once. - app.host.state = State::Missing; - let applies = |app: &App| { - app.entries() - .iter() - .filter(|entry| **entry == Entry::Task("apply")) - .count() - }; - assert!(screen_text(&mut app).contains("not on this host yet") && applies(&app) == 1); - app.host.state = - State::Unreachable("ssh: connect to host devbox.example: timed out\nmore".into()); - let text = screen_text(&mut app); - assert!(text.contains("✗ ssh: connect to host") && text.contains("▸ Terminal over SSH")); - assert!(app.entries().contains(&Entry::Task("connect"))); - - // One that answered lists its tasks in their groups. - app.host.state = State::Ready("0.2.0".into()); - app.host.tasks = host_tasks(); - let rows: Vec = app.rows().iter().map(|row| row.line.to_string()).collect(); - let place = |text: &str| rows.iter().position(|row| row.contains(text)).unwrap(); - assert!( - place("System") < place("▸ Status") - && place("▸ Status") < place("▸ Terminal · as robot") - ); - assert!(place("▸ Terminal") < place("▸ Reboot")); - // Groups are matched whatever their case; a step has no row of its own. - assert!(place("Sign-In") < place("▸ Show who is signed in")); - assert!(place("▸ Show who is signed in") < place("▸ Sign in to Claude Code")); - assert!(place("Backup") > place("▸ Sign in to Claude Code")); - assert!(!rows.iter().any(|row| row.contains("Claude Code sign-in"))); - assert!(!rows.iter().any(|row| row.contains("Sessions"))); - let select = |app: &mut App, entry: Entry| { - let at = app.entries().iter().position(|known| *known == entry); - app.list.select(Some(at.unwrap())); - }; - // The information box says what the plugins' steps add. - select(&mut app, Entry::HostTask(1)); - let help = app.help(); - assert!( - help.contains("### From plugins") && help.contains("**Claude Code sign-in**: Whether") - ); - assert!(help.contains("berth run HOST sign-in:status")); - - // A task with a question waits for the yes. - select(&mut app, Entry::HostTask(3)); - assert!(!press(&mut app, &[KeyCode::Enter])); - assert_eq!( - app.host.question, - Some(("Restart the host?".into(), Pending::Action(3))) - ); - assert!(screen_text(&mut app).contains("Restart the host?")); - question_event( - &mut app, - &Event::Key(KeyEvent::new(KeyCode::Char('n'), KeyModifiers::NONE)), - ); - assert!(app.host.question.is_none()); - - // A task with arguments asks for each; a needed one cannot be left empty. - select(&mut app, Entry::HostTask(4)); - press(&mut app, &[KeyCode::Enter]); - let key = |app: &mut App, code| { - arguments_event( - app, - &Event::Key(KeyEvent::new(code, KeyModifiers::NONE)), - &mut Desktop, - ) - }; - assert!(screen_text(&mut app).contains("Restore · 1 of 2")); - key(&mut app, KeyCode::Enter); - assert!(app.status.contains("Snapshot is needed")); - app.host.arguments.as_mut().unwrap().choices = vec!["abc123".into(), "def456".into()]; - assert!(screen_text(&mut app).contains("○ def456")); - key(&mut app, KeyCode::Down); - key(&mut app, KeyCode::Down); - key(&mut app, KeyCode::Enter); - let asked = app.host.arguments.as_ref().unwrap(); - assert_eq!((asked.index, asked.input.value()), (1, "/")); - assert_eq!(asked.values["snapshot"], "def456"); - key(&mut app, KeyCode::Esc); - assert!(app.host.arguments.is_none()); - - // Sessions of the host get a group; a running one is not forgotten with x. - app.host.sessions = serde_json::from_value(json!([ - {"id": "sabc", "kind": "terminal", "title": "robot on devbox", "command": ["bash", "-l"], "user": "agent", - "started_by": {"level": "host", "uid": 0}, "started": "2026-11-12T10:00:00Z"}, - {"id": "sdef", "kind": "process", "title": "Update", "command": ["update"], "user": "root", - "started_by": {"level": "host", "uid": 0}, "started": "2026-11-12T10:00:00Z", - "ended": "2026-11-12T10:05:00Z", "exit_code": 2}, - ])) - .unwrap(); - let text: String = app - .rows() - .iter() - .map(|row| row.line.to_string() + "\n") - .collect(); - assert!( - text.contains("▸ robot on devbox · running") && text.contains("▸ Update · failed (2)") - ); - select(&mut app, Entry::HostSession(0)); - assert!( - app.help().contains("still running") && app.help().contains("berth attach HOST sabc") - ); - press(&mut app, &[KeyCode::Char('x')]); - assert!(app.status.contains("still running")); - // Opening one puts it in the panel; a transcript takes its lines. - select(&mut app, Entry::HostSession(1)); - press(&mut app, &[KeyCode::Enter]); - let view = app.host.view.as_ref().unwrap(); - assert!(view.id == "sdef" && !view.terminal && !view.ours); - assert!(screen_text(&mut app).contains("Ctrl+Q leave it running")); - view_event( - &mut app, - &Event::Key(KeyEvent::new(KeyCode::Char('q'), KeyModifiers::CONTROL)), - ); - assert!(app.host.view.is_none()); - - // Applying with a session running asks first; then it is the SSH install. - app.config["hostname"] = json!("devbox-1"); - app.config["operator_ssh_public_key"] = json!("ssh-ed25519 AAAA me"); - select(&mut app, Entry::Task("apply")); - assert!(!press(&mut app, &[KeyCode::Enter])); - assert!(matches!(app.host.question, Some((_, Pending::Apply)))); - let yes = Event::Key(KeyEvent::new(KeyCode::Char('y'), KeyModifiers::NONE)); - assert!(matches!(question_event(&mut app, &yes), Flow::Run)); - assert_eq!(app.terminal_action, Some(Action::Ssh)); - assert_eq!(app.host.state, State::Idle); - app.terminal_action = None; - - // Space switches the Terminal row between the two accounts. - app.host.state = State::Ready("0.2.0".into()); - select(&mut app, Entry::Task("terminal")); - press(&mut app, &[KeyCode::Char(' ')]); - assert!(!app.as_agent && screen_text(&mut app).contains("the SSH user")); - press(&mut app, &[KeyCode::Char(' ')]); - // ssh's options come from berth's key ring, which is in a store. - let scratch = tempfile::tempdir().unwrap(); - app.paths = Paths { - config_dir: scratch.path().join("config"), - data_dir: scratch.path().join("data"), - }; - let command = format!("{:?}", app.agent_terminal().unwrap()); - assert!(command.contains("devbox.example") && command.contains("sudo -iu robot ")); - // The set-up tabs are a flow under the overview, opening on Install. - select(&mut app, Entry::Setup); - press(&mut app, &[KeyCode::Enter]); - assert_eq!((app.step, app.action), (Step::Install, Action::Iso)); - press(&mut app, &[KeyCode::Char('4')]); - assert_eq!(app.step, Step::Run); - press(&mut app, &[KeyCode::Esc]); - assert_eq!(app.step, Step::Manage); - // x removes the value of the selected field, and with the address - // gone what was known of the host is too. - app.host.state = State::Ready("0.2.0".into()); - select(&mut app, Entry::Field(Field::SshHost)); - press(&mut app, &[KeyCode::Char('x')]); - assert_eq!(app.text(Field::SshHost), ""); - assert_eq!(app.host.state, State::Idle); - } - - const BERTH_KEY: &str = - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIExampleKeyOfBerth1234 berth@studio"; - - #[test] - fn settings_show_the_key_of_berth_and_copy_it() { - let directory = tempfile::tempdir().unwrap(); - let paths = Paths { - config_dir: directory.path().join("config"), - data_dir: directory.path().join("data"), - }; - paths.set("server", "https://192.0.2.7:8443").unwrap(); - paths.set("server_token", "a1b2").unwrap(); - let mut home = Home { - hosts: vec![], - selected: 0, - mode: HomeMode::Settings(0, None), - status: String::new(), - started: std::time::Instant::now(), - qr: None, - berth_key: Some(( - vec![("berth".into(), BERTH_KEY.into())], - Some("https://192.0.2.7:8443".into()), - )), - served: None, - known: None, - asking: false, - refresh: false, - rotating: vec![], - }; - let mut clipboard = Clipboard::default(); - let settings = berth_core::paths::SETTINGS.len(); - for _ in 0..settings { - home_key( - &mut home, - &paths, - KeyEvent::new(KeyCode::Down, KeyModifiers::NONE), - &mut clipboard, - ); - } - // The key comes after the settings, before the row with the QR code. - assert!(matches!(home.mode, HomeMode::Settings(index, _) if index == settings)); - let mut terminal = Terminal::new(TestBackend::new(130, 50)).unwrap(); - terminal - .draw(|frame| draw_home(frame, &paths, &home, &mut vec![])) - .unwrap(); - let buffer = terminal.backend().buffer(); - let screen = (0..50) - .map(|y| { - (0..130) - .map(|x| buffer[(x, y)].symbol()) - .collect::() - }) - .collect::>() - .join("\n"); - assert!( - screen.contains("This berth's SSH key ssh-ed25519 …erth1234 berth@studio"), - "{screen}" - ); - assert!(screen.contains("Enter copy the key") && screen.contains("berth server at")); - assert!(screen.contains("Open on another device")); - home_key( - &mut home, - &paths, - KeyEvent::new(KeyCode::Enter, KeyModifiers::NONE), - &mut clipboard, - ); - assert_eq!(clipboard.0.as_deref(), Some(BERTH_KEY)); - assert_eq!(home.status, "Copied this berth's SSH public key"); - assert!(home.qr.is_none()); - // The row after it still shows the QR code. - home_key( - &mut home, - &paths, - KeyEvent::new(KeyCode::Down, KeyModifiers::NONE), - &mut clipboard, - ); - home_key( - &mut home, - &paths, - KeyEvent::new(KeyCode::Enter, KeyModifiers::NONE), - &mut clipboard, - ); - assert!(home.qr.is_some()); - // Without a key to give there is no row for it. - home.berth_key = Some((vec![], None)); - home.mode = HomeMode::Settings(settings, None); - home.qr = None; - home_key( - &mut home, - &paths, - KeyEvent::new(KeyCode::Enter, KeyModifiers::NONE), - &mut clipboard, - ); - assert!(home.qr.is_some()); - } - - fn lab(id: &str, label: &str, resources: &[&str]) -> berth_api::ProvisionerInfo { - let mut arguments = vec![berth_api::Argument { - name: "server".into(), - label: "Lab server".into(), - required: true, - ..Default::default() - }]; - arguments.extend(resources.iter().map(|name| berth_api::Argument { - name: (*name).into(), - label: format!("Lab {name}"), - default: "4".into(), - kind: berth_api::Argument::RESOURCE.into(), - ..Default::default() - })); - berth_api::ProvisionerInfo { - id: id.into(), - label: label.into(), - setup: "apply".into(), - arguments, - ..Default::default() - } - } - - #[test] - fn the_tabs_create_and_delete_the_machine() { - use berth_tui::machine::Pick; - let profile = format!( - r#"{{"hostname": "studio", "operator_password_hash": "$6${}", - "operator_ssh_public_keys": ["ssh-ed25519 AAAA studio"]}}"#, - "a".repeat(40) - ); - let mut app = unsaved_app(profile.as_bytes()); - app.set_step(Step::Install); - app.action = Action::Machine; - let machine_rows = |app: &App| -> Vec { - app.entries() - .into_iter() - .filter(|entry| matches!(entry, Entry::Machine(_))) - .collect() - }; - // Nothing was asked without berth's own folders, so nothing shows. - assert!(machine_rows(&app).is_empty()); - app.machine.asking(); - app.machine.listed(Ok(vec![ - lab("lab/vm", "Lab machine", &["cpus", "memory"]), - lab("lab/box", "Lab box", &["cpus"]), - ])); - app.machine.found(Ok(None)); - // The provisioner is one row, and what it requires is under it. - assert!(lines(&app).contains( - &"Provisioner Lab machine · berth applies the profile over SSH".to_owned() - )); - assert_eq!( - machine_rows(&app), - [ - Entry::Machine(Pick::Provisioner), - Entry::Machine(Pick::Argument(0)) - ] - ); - let install = lines(&app); - assert!(install.contains(&" └─ Lab server * Not set".to_owned())); - assert_eq!(install.last().unwrap(), "✗ Lab server is required"); - // Its size is under Machine on Settings, and goes with the provisioner. - press(&mut app, &[KeyCode::Char('3')]); - let settings = lines(&app); - assert_eq!( - settings[..3], - ["Machine", " ├─ Lab cpus 4", " └─ Lab memory 4"] - ); - press(&mut app, &[KeyCode::Char('1')]); - select(&mut app, Entry::Machine(Pick::Provisioner)); - press(&mut app, &[KeyCode::Enter, KeyCode::Char('3')]); - assert_eq!(lines(&app)[..2], ["Machine", " └─ Lab cpus 4"]); - press(&mut app, &[KeyCode::Char('1')]); - select(&mut app, Entry::Machine(Pick::Provisioner)); - press(&mut app, &[KeyCode::Enter]); - assert!( - lines(&app) - .iter() - .any(|row| row.contains("Provisioner Lab machine")) - ); - - // Creating with a required argument empty is refused before asking. - press(&mut app, &[KeyCode::Char('4')]); - let run = lines(&app); - assert!(run.contains(&" ✗ Lab server is required".to_owned())); - select(&mut app, Entry::Machine(Pick::Create)); - press(&mut app, &[KeyCode::Enter]); - assert!(app.machine.question.is_none() && app.status.contains("Fix the listed problems")); - press(&mut app, &[KeyCode::Char('1')]); - select(&mut app, Entry::Machine(Pick::Argument(0))); - press(&mut app, &[KeyCode::Enter]); - for ch in "192.0.2.10".chars() { - press(&mut app, &[KeyCode::Char(ch)]); - } - press(&mut app, &[KeyCode::Enter]); - assert_eq!(app.machine.values["server"], "192.0.2.10"); - // A size that is no size is a problem of Settings, and blocks Create. - press(&mut app, &[KeyCode::Char('3')]); - select(&mut app, Entry::Machine(Pick::Argument(2))); - press( - &mut app, - &[KeyCode::Enter, KeyCode::Char('x'), KeyCode::Enter], - ); - let settings = lines(&app); - assert_eq!( - settings.last().unwrap(), - "✗ Lab memory must be a size with its unit, such as 8GiB, 512MiB or 4g" - ); - press(&mut app, &[KeyCode::Char('4')]); - assert!( - lines(&app) - .iter() - .any(|row| row.contains(" ✗ Lab memory must be a size")) - ); - select(&mut app, Entry::Machine(Pick::Create)); - press(&mut app, &[KeyCode::Enter]); - assert!(app.machine.question.is_none() && app.status.contains("Fix the listed problems")); - press(&mut app, &[KeyCode::Char('3')]); - select(&mut app, Entry::Machine(Pick::Argument(2))); - press(&mut app, &[KeyCode::Enter, KeyCode::Backspace]); - for ch in "8GiB".chars() { - press(&mut app, &[KeyCode::Char(ch)]); - } - press(&mut app, &[KeyCode::Enter, KeyCode::Char('4')]); - assert!(lines(&app).contains(&"✓ Ready".to_owned())); - select(&mut app, Entry::Machine(Pick::Create)); - press(&mut app, &[KeyCode::Enter]); - let shown = screen_text(&mut app); - assert!( - shown.contains("Create a machine for studio with Lab machine?"), - "{shown}" - ); - app.machine_answer(false); - assert!(app.machine.question.is_none() && app.job.is_none()); - - // Another way to install has no provisioner to choose. - app.action = Action::Ssh; - app.set_step(Step::Install); - assert!(machine_rows(&app).is_empty()); - // With a machine, its state and what can be done to it are on Run, whatever the way. - app.machine.found(Ok(Some(berth_api::Resource { - provisioner: "lab/vm".into(), - id: "r-studio".into(), - external_id: "studio".into(), - state: "stopped".into(), - verbs: vec!["start".into()], - ..Default::default() - }))); - app.set_step(Step::Run); - assert_eq!( - machine_rows(&app), - [Entry::Machine(Pick::Start), Entry::Machine(Pick::Delete)] - ); - select(&mut app, Entry::Machine(Pick::Delete)); - press(&mut app, &[KeyCode::Enter]); - assert!(screen_text(&mut app).contains("Type studio to delete its machine")); - for ch in "studi".chars() { - press(&mut app, &[KeyCode::Char(ch)]); - } - press(&mut app, &[KeyCode::Enter]); - assert!(app.status.starts_with("Not deleted") && app.job.is_none()); - } - - #[test] - fn the_provisioner_is_a_choice_like_the_others() { - use KeyCode::*; - use berth_tui::machine::Pick; - let mut app = unsaved_app(br#"{"profile": "laptop"}"#); - app.machine.asking(); - app.machine.listed(Ok(vec![ - lab("lab/vm", "Lab machine", &["cpus"]), - lab("lab/box", "Lab box", &[]), - ])); - app.machine.found(Ok(None)); - // Choosing the way makes the profile a cloud one, and offers none to choose. - select(&mut app, Entry::Action(Action::Machine)); - press(&mut app, &[Enter]); - assert_eq!(app.config["profile"], "cloud"); - assert!(!lines(&app).iter().any(|row| row.contains("Host profile"))); - // The row offers the provisioners in the panel, at the chosen one. - select(&mut app, Entry::Machine(Pick::Provisioner)); - let shown = screen(&mut app); - assert!( - shown.contains("Suggestions") && shown.contains("Lab box"), - "{shown}" - ); - assert_eq!( - app.suggestions, - [ - ("Lab machine".to_owned(), "lab/vm".to_owned()), - ("Lab box".to_owned(), "lab/box".to_owned()) - ] - ); - assert_eq!(app.suggestion_list.selected(), Some(0)); - // Space goes on to the next, and the panel follows it. - press(&mut app, &[Char(' ')]); - assert_eq!(app.machine.chosen.as_deref(), Some("lab/box")); - app.update_suggestions(); - assert_eq!(app.suggestion_list.selected(), Some(1)); - // → enters the panel, and Enter there chooses. - press(&mut app, &[Right, Up, Enter]); - assert_eq!(app.machine.chosen.as_deref(), Some("lab/vm")); - assert!(!app.suggestion_focus); - assert!( - app.help() - .contains("Space or Enter switches to the next provisioner") - ); - } -} diff --git a/crates/berth/src/update.rs b/crates/berth/src/update.rs index 783ac03..edce7a6 100644 --- a/crates/berth/src/update.rs +++ b/crates/berth/src/update.rs @@ -6,23 +6,35 @@ //! them to the new version. use crate::Result; +use crate::cli::args::Update; +use crate::cli::out; use std::{fs, path::Path, process::Command}; const REPOSITORY: &str = "https://github.com/helightdev/berth"; -/// The archive of this build's kind: with the page for a browser, or without. -#[cfg(not(feature = "web"))] -const ASSET: &str = "berth-x86_64-linux.tar.gz"; -#[cfg(feature = "web")] -const ASSET: &str = "berth-web-x86_64-linux.tar.gz"; -/// Where the releases are: GitHub's, or `BERTH_RELEASES` for a mirror. -fn releases() -> String { - match std::env::var("BERTH_RELEASES") { - Ok(base) if !base.is_empty() => base.trim_end_matches('/').to_owned(), - _ => format!("{REPOSITORY}/releases"), +/// The archive of the release for this system, when there is one. +fn asset() -> Option<&'static str> { + match (std::env::consts::OS, std::env::consts::ARCH) { + ("linux", "x86_64") => Some("berth-x86_64-linux.tar.gz"), + ("linux", "aarch64") => Some("berth-aarch64-linux.tar.gz"), + ("macos", "aarch64") => Some("berth-aarch64-macos.tar.gz"), + ("macos", "x86_64") => Some("berth-x86_64-macos.tar.gz"), + _ => None, } } +/// The setting of this user's that names a mirror of the releases. +pub const RELEASES: &str = "releases"; + +/// Where the releases are: a mirror given with `--from` or kept as the +/// `releases` setting, or else GitHub's. +fn releases(from: Option<&str>) -> Option { + from.map(str::to_owned) + .or_else(|| crate::cli::client::settings().get(RELEASES)) + .filter(|base| !base.is_empty()) + .map(|base| base.trim_end_matches('/').to_owned()) +} + fn curl(args: &[&str]) -> Result { let output = Command::new("curl").args(["-fsSL"]).args(args).output()?; if !output.status.success() { @@ -33,14 +45,14 @@ fn curl(args: &[&str]) -> Result { } /// The newest release's version, from where `releases/latest` leads. -fn latest() -> Result { - let base = releases(); +fn latest(mirror: Option<&str>) -> Result { // A mirror says it in a file; GitHub redirects to the tag. - if std::env::var_os("BERTH_RELEASES").is_some() { + if let Some(base) = mirror { return Ok(curl(&[&format!("{base}/latest/download/VERSION")])? .trim_start_matches('v') .to_owned()); } + let base = format!("{REPOSITORY}/releases"); let page = curl(&[ "-o", "/dev/null", @@ -70,27 +82,33 @@ pub(crate) fn newer(candidate: &str, current: &str) -> bool { parts(candidate) > parts(current) } -/// `berth update [--check]` -pub fn command(args: &[&str]) -> Result<()> { - let check = match args { - [] => false, - ["--check"] => true, - _ => return Err("Usage: berth update [--check]".into()), - }; - if !cfg!(all(target_os = "linux", target_arch = "x86_64")) { +/// `berth update [--check] [--from URL]` +pub fn command(update: &Update) -> crate::cli::fail::Result<()> { + Ok(install(update)?) +} + +fn install(update: &Update) -> Result<()> { + let check = update.check; + let mirror = releases(update.from.as_deref()); + if cfg!(windows) { + return Err("On Windows the new release is installed with install.ps1: irm https://raw.githubusercontent.com/helightdev/berth/main/install.ps1 | iex".into()); + } + let Some(asset) = asset() else { return Err(format!( - "Releases are built for Linux on x86_64. Update this one with: cargo install --locked --git {REPOSITORY} berth" + "No release is built for this system. Update this one with: cargo install --locked --git {REPOSITORY} berth" ) .into()); - } + }; let current = berth_api::VERSION; - let latest = latest()?; + let latest = latest(mirror.as_deref())?; if !newer(&latest, current) { - println!("berth {current} is the latest"); + out::line(format!("berth {current} is the latest")); return Ok(()); } if check { - println!("berth {latest} is out; this is {current}. Run berth update to install it."); + out::line(format!( + "berth {latest} is out; this is {current}. berth update installs it" + )); return Ok(()); } let program = fs::canonicalize(std::env::current_exe()?)?; @@ -100,13 +118,14 @@ pub fn command(args: &[&str]) -> Result<()> { .prefix(".berth-update") .tempdir_in(folder) .map_err(|error| format!("{}: {error}", folder.display()))?; - let url = format!("{}/download/v{latest}", releases()); + let base = mirror.unwrap_or_else(|| format!("{REPOSITORY}/releases")); + let url = format!("{base}/download/v{latest}"); let fetch = |name: &str| -> Result<()> { let to = work.path().join(name); curl(&["-o", &to.to_string_lossy(), &format!("{url}/{name}")]).map(|_| ()) }; - fetch(ASSET)?; - fetch(&format!("{ASSET}.sha256"))?; + fetch(asset)?; + fetch(&format!("{asset}.sha256"))?; let run = |program: &str, args: &[&str]| -> Result<()> { let status = Command::new(program) .args(args) @@ -119,20 +138,30 @@ pub fn command(args: &[&str]) -> Result<()> { false => Err(format!("{program} failed ({status})").into()), } }; - run("sha256sum", &["-c", &format!("{ASSET}.sha256")]) - .map_err(|_| "the download does not match its checksum; nothing was changed")?; - run("tar", &["-xzf", ASSET, "berth"])?; + let sums = format!("{asset}.sha256"); + // macOS has shasum rather than sha256sum. + let checked = match cfg!(target_os = "macos") { + true => run("shasum", &["-a", "256", "-c", &sums]), + false => run("sha256sum", &["-c", &sums]), + }; + checked.map_err(|_| "the download does not match its checksum; nothing was changed")?; + run("tar", &["-xzf", asset, "berth"])?; replace(&work.path().join("berth"), &program)?; - println!("berth {current} → {latest}"); - println!("Hosts get the new berthd and box when their configuration is applied again."); + out::done(format!("berth {current} → {latest}")); + if cfg!(target_os = "linux") { + out::hint("Hosts get the new berthd and box when they are applied again: berth rollout"); + } Ok(()) } /// Put the new program in the old one's place in one step, so a running /// berth keeps the file it has open and the next start finds the new one. fn replace(new: &Path, program: &Path) -> Result<()> { - use std::os::unix::fs::PermissionsExt; - fs::set_permissions(new, fs::Permissions::from_mode(0o755))?; + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + fs::set_permissions(new, fs::Permissions::from_mode(0o755))?; + } fs::rename(new, program).map_err(|error| format!("{}: {error}", program.display()))?; Ok(()) } diff --git a/crates/berth/tests/cli.rs b/crates/berth/tests/cli.rs new file mode 100644 index 0000000..935628d --- /dev/null +++ b/crates/berth/tests/cli.rs @@ -0,0 +1,224 @@ +//! The command line as a script sees it: the exit statuses of the documented +//! table, and `-o json` printing the API's own documents. It runs the berth +//! that was built, against a `berth serve` and a server in its own process, +//! in scratch directories, with made-up hosts. +#![cfg(target_os = "linux")] + +use serde_json::Value; +use std::io::{Read, Write}; +use std::path::{Path, PathBuf}; +use std::process::{Child, Command, Stdio}; +use std::time::{Duration, Instant}; + +/// A scratch installation: its directories, and the `berth serve` of it +/// while one runs. +struct Scratch { + dir: tempfile::TempDir, + serve: Option, +} + +impl Scratch { + fn new() -> Scratch { + let dir = tempfile::Builder::new().prefix("bc").tempdir().unwrap(); + for sub in ["r", "c", "d", "home"] { + std::fs::create_dir_all(dir.path().join(sub)).unwrap(); + } + Scratch { dir, serve: None } + } + + fn command(&self) -> Command { + let path = self.dir.path(); + let mut command = Command::new(env!("CARGO_BIN_EXE_berth")); + command + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap_or_default()) + .env("HOME", path.join("home")) + .env("XDG_RUNTIME_DIR", path.join("r")) + .env("XDG_CONFIG_HOME", path.join("c")) + .env("XDG_DATA_HOME", path.join("d")) + .env("NO_COLOR", "1") + .stdin(Stdio::null()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()); + command + } + + /// Run berth with `args`: its exit status, standard output and error. + fn run(&self, args: &[&str]) -> (i32, String, String) { + let done = self.command().args(args).output().unwrap(); + ( + done.status.code().unwrap_or(-1), + String::from_utf8_lossy(&done.stdout).into_owned(), + String::from_utf8_lossy(&done.stderr).into_owned(), + ) + } + + fn socket(&self) -> PathBuf { + self.dir.path().join("r/berth/serve.sock") + } + + /// Start `berth serve` on its socket and wait until it answers. + fn serve(&mut self) { + let child = self + .command() + .args(["serve"]) + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .spawn() + .unwrap(); + self.serve = Some(child); + let start = Instant::now(); + while std::os::unix::net::UnixStream::connect(self.socket()).is_err() { + assert!( + start.elapsed() < Duration::from_secs(20), + "berth serve did not start" + ); + std::thread::sleep(Duration::from_millis(50)); + } + } + + /// What the server answers to `GET path`, as JSON. + fn get(&self, path: &str) -> Value { + get(&self.socket(), path) + } +} + +impl Drop for Scratch { + fn drop(&mut self) { + if let Some(mut serve) = self.serve.take() { + let _ = serve.kill(); + let _ = serve.wait(); + } + } +} + +fn get(socket: &Path, path: &str) -> Value { + let mut stream = std::os::unix::net::UnixStream::connect(socket).unwrap(); + write!(stream, "GET {path} HTTP/1.0\r\nHost: berth\r\n\r\n").unwrap(); + let mut answer = String::new(); + stream.read_to_string(&mut answer).unwrap(); + let (head, body) = answer.split_once("\r\n\r\n").unwrap(); + assert!( + head.starts_with("HTTP/1.0 200") || head.starts_with("HTTP/1.1 200"), + "{path}: {head}" + ); + serde_json::from_str(body).unwrap_or_else(|error| panic!("{path}: {error}: {body}")) +} + +#[test] +fn exit_statuses_follow_the_documented_table() { + let scratch = Scratch::new(); + let (status, _, said) = scratch.run(&[ + "host", + "add", + "studio", + "192.0.2.10", + "--type", + "ssh", + "--no-contact", + ]); + assert_eq!(status, 0, "{said}"); + let table: &[(&[&str], i32)] = &[ + // 0: done. + (&["--version"], 0), + (&[], 0), + (&["help"], 0), + (&["host", "list"], 0), + (&["-o", "json", "job", "list"], 0), + // 1: the work failed. + (&["host", "show", "dock"], 1), + (&["host", "add", "studio", "192.0.2.11", "--no-contact"], 1), + // 2: the command line was wrong. + (&["aply", "studio"], 2), + (&["host", "show"], 2), + (&["host", "list", "--colour"], 2), + (&["-o", "yaml", "host", "list"], 2), + (&["host", "create", "--no-input"], 2), + // 3: refused. + (&["host", "apply", "studio", "--no-input"], 3), + (&["host", "remove", "studio", "--no-input"], 3), + // 4: the server could not be reached. + (&["--server", "http://127.0.0.1:9", "host", "list"], 4), + ( + &["--server", "unix:/nonexistent/serve.sock", "job", "list"], + 4, + ), + ]; + for (args, expected) in table { + let (status, _, said) = scratch.run(args); + assert_eq!(status, *expected, "berth {}: {said}", args.join(" ")); + } + // With -o json, a failure is an object on standard error with the same status. + let (status, out, said) = scratch.run(&["-o", "json", "aply"]); + assert_eq!((status, out.as_str()), (2, "")); + let error: Value = serde_json::from_str(said.trim()).unwrap(); + assert_eq!(error["error"]["exit"], 2); + let (status, _, said) = scratch.run(&["-o", "json", "host", "apply", "studio", "--no-input"]); + assert_eq!(status, 3); + let error: Value = serde_json::from_str(said.trim()).unwrap(); + assert_eq!(error["error"]["exit"], 3, "{error}"); + // A confirmation answered with -y goes on. + let (status, _, said) = scratch.run(&["host", "remove", "studio", "-y"]); + assert_eq!(status, 0, "{said}"); +} + +#[test] +fn json_is_the_documents_of_the_api() { + let mut scratch = Scratch::new(); + let (status, _, said) = scratch.run(&[ + "host", + "add", + "studio", + "192.0.2.10", + "--type", + "ssh", + "--no-contact", + ]); + assert_eq!(status, 0, "{said}"); + let (status, _, said) = scratch.run(&["key", "create", "spare"]); + assert_eq!(status, 0, "{said}"); + scratch.serve(); + let json = |args: &[&str]| -> Value { + let mut all = vec!["-o", "json"]; + all.extend(args); + let (status, out, said) = scratch.run(&all); + assert_eq!(status, 0, "berth {}: {said}", all.join(" ")); + serde_json::from_str(&out) + .unwrap_or_else(|error| panic!("berth {}: {error}: {out}", all.join(" "))) + }; + // What does not change between two reads is the same document. + let same: &[(&[&str], &str)] = &[ + (&["key", "list"], "/v1/keys"), + (&["secret", "list"], "/v1/secrets"), + (&["registry", "list"], "/v1/registries"), + (&["provisioner", "list"], "/v1/provisioners"), + (&["provisioner", "types"], "/v1/provisioner-types"), + (&["resource", "list"], "/v1/resources"), + (&["plugin", "list"], "/v1/plugins"), + (&["job", "list"], "/v1/jobs?limit=20"), + ]; + for (args, path) in same { + assert_eq!(json(args), scratch.get(path), "berth {}", args.join(" ")); + } + // A host's live state may move between two reads; what it is does not. + let keys = |value: &Value| -> Vec { + let mut keys: Vec = value.as_object().unwrap().keys().cloned().collect(); + keys.sort(); + keys + }; + let listed = json(&["host", "list"]); + let served = scratch.get("/v1/hosts"); + assert_eq!( + listed.as_array().unwrap().len(), + served.as_array().unwrap().len() + ); + assert_eq!(keys(&listed[0]), keys(&served[0])); + assert_eq!(listed[0]["name"], "studio"); + let shown = json(&["host", "show", "studio"]); + let got = scratch.get("/v1/hosts/studio"); + assert_eq!(keys(&shown), keys(&got)); + assert_eq!( + (&shown["name"], &shown["type"]), + (&got["name"], &got["type"]) + ); +} diff --git a/crates/berthd/src/tasks.rs b/crates/berthd/src/actions.rs similarity index 65% rename from crates/berthd/src/tasks.rs rename to crates/berthd/src/actions.rs index ae401f7..b22fc9f 100644 --- a/crates/berthd/src/tasks.rs +++ b/crates/berthd/src/actions.rs @@ -1,5 +1,5 @@ -//! Tasks: the host's named operations. Running one starts a session whose -//! steps are the task and the tasks that extend it, each as its own user. +//! Actions: the host's named operations. Running one starts a session whose +//! steps are the action and the actions that extend it, each as its own user. use crate::error::{E, R}; use crate::sessions::{Plan, Wait, new_id, started}; @@ -8,24 +8,24 @@ use axum::{ Json, extract::{Extension, Path, Query, State as AppState}, }; -use berth_api::{self as api, CommandLine, SessionKind, Task}; +use berth_api::{self as api, CommandLine, HostAction, SessionKind}; use std::{collections::BTreeMap, sync::Arc}; -fn allowed(caller: &Caller, task: &Task) -> bool { - caller.host() || task.agent +fn allowed(caller: &Caller, action: &HostAction) -> bool { + caller.host() || action.agent } -fn find(state: &State, caller: &Caller, name: &str) -> R { +fn find(state: &State, caller: &Caller, name: &str) -> R { let config = state.config(); - let task = config - .tasks + let action = config + .actions .iter() - .find(|task| task.name == name) - .ok_or_else(|| E::missing(format!("task {name}")))?; - if !allowed(caller, task) { + .find(|action| action.name == name) + .ok_or_else(|| E::missing(format!("action {name}")))?; + if !allowed(caller, action) { return Err(E::forbidden(format!("The agent may not run {name}"))); } - Ok(task.clone()) + Ok(action.clone()) } /// The name an argument has in the command's environment. @@ -46,14 +46,17 @@ pub fn variable(name: &str) -> String { pub async fn list( AppState(state): AppState>, Extension(caller): Extension, -) -> Json> { +) -> Json> { let config = state.config(); - let tasks = config.tasks.iter().filter(|task| allowed(&caller, task)); + let actions = config + .actions + .iter() + .filter(|action| allowed(&caller, action)); Json( - tasks - .map(|task| Task { + actions + .map(|action| HostAction { run: String::new(), - ..task.clone() + ..action.clone() }) .collect(), ) @@ -64,12 +67,16 @@ pub async fn run( Extension(caller): Extension, Path(name): Path, Query(wait): Query, - body: Option>, + body: Option>, ) -> R> { let request = body.map(|Json(request)| request).unwrap_or_default(); - let task = find(&state, &caller, &name)?; - let mut env = BTreeMap::from([("BERTH_TASK".to_owned(), task.name.clone())]); - for argument in &task.arguments { + let action = find(&state, &caller, &name)?; + // BERTH_TASK is what plugins written for berth 0.5 read. + let mut env = BTreeMap::from([ + ("BERTH_ACTION".to_owned(), action.name.clone()), + ("BERTH_TASK".to_owned(), action.name.clone()), + ]); + for argument in &action.arguments { let value = request .arguments .get(&argument.name) @@ -83,26 +90,27 @@ pub async fn run( } env.insert(variable(&argument.name), value); } - if let Some(unknown) = request - .arguments - .keys() - .find(|key| task.arguments.iter().all(|argument| &argument.name != *key)) - { + if let Some(unknown) = request.arguments.keys().find(|key| { + action + .arguments + .iter() + .all(|argument| &argument.name != *key) + }) { return Err(E::bad(format!("{name} has no argument {unknown}"))); } - let input = match (&task.input, &request.input_b64) { + let input = match (&action.input, &request.input_b64) { (Some(what), None) => return Err(E::bad(format!("{name} needs {what} sent as input_b64"))), (_, Some(data)) => Some(api::unb64(data).ok_or_else(|| E::bad("input_b64 is not base64"))?), (None, None) => None, }; - // The task itself, then what extends it, in the order of the file. + // The action itself, then what extends it, in the order of the file. let config = state.config(); let extenders = config - .tasks + .actions .iter() - .filter(|other| other.extends.contains(&task.name) && other.name != task.name); + .filter(|other| other.extends.contains(&action.name) && other.name != action.name); let mut steps = vec![]; - for step in std::iter::once(&task).chain(extenders) { + for step in std::iter::once(&action).chain(extenders) { let command = crate::users::command( &state, &step.user, @@ -121,19 +129,19 @@ pub async fn run( } let plan = Plan { id: new_id(), - kind: if task.terminal { + kind: if action.terminal { SessionKind::Terminal } else { SessionKind::Process }, - title: if task.label.is_empty() { - task.name.clone() + title: if action.label.is_empty() { + action.name.clone() } else { - task.label.clone() + action.label.clone() }, - command: vec![task.name.clone()], - user: task.user.clone(), - task: Some(task.name.clone()), + command: vec![action.name.clone()], + user: action.user.clone(), + action: Some(action.name.clone()), steps, input, close_input: true, @@ -150,8 +158,8 @@ pub async fn choices( Extension(caller): Extension, Path((name, argument)): Path<(String, String)>, ) -> R> { - let task = find(&state, &caller, &name)?; - let argument = task + let action = find(&state, &caller, &name)?; + let argument = action .arguments .iter() .find(|known| known.name == argument) @@ -161,7 +169,7 @@ pub async fn choices( } let line = CommandLine::Shell(argument.choices.clone()); let (code, output, errors) = - crate::users::capture(&state, &task.user, &line, 15, 256 << 10).await?; + crate::users::capture(&state, &action.user, &line, 15, 256 << 10).await?; if code != 0 { return Err(E::internal(format!( "The choices command failed ({code}): {errors}" diff --git a/crates/berthd/src/endpoints.rs b/crates/berthd/src/endpoints.rs new file mode 100644 index 0000000..e24f7b3 --- /dev/null +++ b/crates/berthd/src/endpoints.rs @@ -0,0 +1,316 @@ +//! A TCP connection to an address on a network of the host's own, as a +//! stream: how `berth serve` reaches a resource of an engine on this host, +//! such as a container on a bridge that only this host routes to. +//! +//! It reaches further than `ports.connect`, so it is kept narrow: `berth +//! serve` calls it itself for an endpoint a provisioner reported and never +//! passes it through for a caller, and the address has to be on a network +//! that one of the host's interfaces is attached to. + +use crate::error::{E, R}; +use axum::{ + extract::{ + Path, + ws::{WebSocketUpgrade, rejection::WebSocketUpgradeRejection}, + }, + response::Response, +}; +use berth_api::code; +use std::{ + net::{IpAddr, Ipv4Addr, Ipv6Addr, SocketAddr}, + time::Duration, +}; +use tokio::net::TcpStream; + +/// How long resolving and dialling may take, as for a port. +const DIAL_FOR: Duration = Duration::from_secs(10); + +/// A network one of the host's interfaces is attached to: an address of the +/// interface and the length of its prefix. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct Network { + pub address: IpAddr, + pub prefix: u8, +} + +impl Network { + fn holds(&self, ip: IpAddr) -> bool { + match (self.address, ip) { + (IpAddr::V4(own), IpAddr::V4(ip)) => { + let mask = u32::MAX + .checked_shl(32 - u32::from(self.prefix)) + .unwrap_or(0); + u32::from(own) & mask == u32::from(ip) & mask + } + (IpAddr::V6(own), IpAddr::V6(ip)) => { + let mask = u128::MAX + .checked_shl(128 - u32::from(self.prefix)) + .unwrap_or(0); + u128::from(own) & mask == u128::from(ip) & mask + } + _ => false, + } + } +} + +/// An IPv4 address written as IPv6 as its IPv4 self, so each check sees it once. +fn canonical(ip: IpAddr) -> IpAddr { + match ip { + IpAddr::V6(v6) => v6.to_ipv4_mapped().map_or(IpAddr::V6(v6), IpAddr::V4), + v4 => v4, + } +} + +/// The addresses of clouds' instance metadata services, which hand out the +/// host's credentials to whoever asks from it. +fn metadata(ip: IpAddr) -> bool { + match ip { + IpAddr::V4(v4) => v4 == Ipv4Addr::new(169, 254, 169, 254), + IpAddr::V6(v6) => v6 == Ipv6Addr::new(0xfd00, 0xec2, 0, 0, 0, 0, 0, 0x254), + } +} + +fn link_local(ip: IpAddr) -> bool { + match ip { + IpAddr::V4(v4) => v4.is_link_local(), + IpAddr::V6(v6) => v6.is_unicast_link_local(), + } +} + +fn special(ip: IpAddr) -> bool { + match ip { + IpAddr::V4(v4) => v4.is_unspecified() || v4.is_broadcast() || v4.is_multicast(), + IpAddr::V6(v6) => v6.is_unspecified() || v6.is_multicast(), + } +} + +/// Whether `ip` may be dialled from this host, given the networks of its +/// interfaces, or why not. +pub fn allowed(ip: IpAddr, networks: &[Network]) -> Result<(), String> { + let ip = canonical(ip); + if metadata(ip) { + return Err(format!( + "{ip} is a cloud's metadata address, which berth never dials" + )); + } + // The host's loopback is what ports.connect is for. + if ip.is_loopback() { + return Err(format!( + "{ip} is the host's loopback: open the port with ports.connect" + )); + } + if link_local(ip) || special(ip) { + return Err(format!("{ip} is no address of a network berth dials")); + } + match networks.iter().any(|network| network.holds(ip)) { + true => Ok(()), + // Anything further is reached through a router, which is not the host's to offer. + false => Err(format!( + "{ip} is on no network of the host's own interfaces" + )), + } +} + +/// The networks the host's interfaces that are up are attached to, without +/// the loopback and link-local ones. +pub fn networks() -> Vec { + let mut found = vec![]; + let mut list: *mut libc::ifaddrs = std::ptr::null_mut(); + // SAFETY: getifaddrs fills `list`, which is freed below and read only before that. + if unsafe { libc::getifaddrs(&mut list) } != 0 { + return found; + } + let mut at = list; + while !at.is_null() { + // SAFETY: `at` is an entry of the list getifaddrs made. + let entry = unsafe { &*at }; + at = entry.ifa_next; + let flags = entry.ifa_flags; + if flags & libc::IFF_UP as u32 == 0 || flags & libc::IFF_LOOPBACK as u32 != 0 { + continue; + } + // SAFETY: the address and the mask are of the family the address says, or null. + let (address, mask) = unsafe { (ip_of(entry.ifa_addr), ip_of(entry.ifa_netmask)) }; + let (Some(address), Some(mask)) = (address, mask) else { + continue; + }; + let prefix = match mask { + IpAddr::V4(mask) => u32::from(mask).leading_ones() as u8, + IpAddr::V6(mask) => u128::from(mask).leading_ones() as u8, + }; + // A prefix this short is a route to much of the internet, not a network the host is on. + let wide = match address { + IpAddr::V4(_) => prefix < 8, + IpAddr::V6(_) => prefix < 16, + }; + if wide || link_local(address) || address.is_loopback() { + continue; + } + found.push(Network { address, prefix }); + } + // SAFETY: the list came from getifaddrs and is not read after this. + unsafe { libc::freeifaddrs(list) }; + found +} + +/// The address a `sockaddr` holds, when it is IPv4 or IPv6. +/// +/// # Safety +/// `address` is null or points to a `sockaddr` of the family it names. +unsafe fn ip_of(address: *const libc::sockaddr) -> Option { + if address.is_null() { + return None; + } + // SAFETY: as the caller promises, the family says which struct it is. + unsafe { + match i32::from((*address).sa_family) { + libc::AF_INET => { + let v4 = &*(address as *const libc::sockaddr_in); + Some(IpAddr::V4(Ipv4Addr::from(u32::from_be(v4.sin_addr.s_addr)))) + } + libc::AF_INET6 => { + let v6 = &*(address as *const libc::sockaddr_in6); + Some(IpAddr::V6(Ipv6Addr::from(v6.sin6_addr.s6_addr))) + } + _ => None, + } + } +} + +/// The addresses `address` names now, each one checked. +async fn resolve(address: &str, port: u16) -> R> { + let found: Vec = match address.parse::() { + Ok(ip) => vec![SocketAddr::new(ip, port)], + Err(_) => { + match tokio::time::timeout(DIAL_FOR, tokio::net::lookup_host((address, port))).await { + Ok(Ok(found)) => found.collect(), + Ok(Err(error)) => { + return Err(E( + code::UNREACHABLE, + format!("{address} does not resolve: {error}"), + )); + } + Err(_) => { + return Err(E( + code::TIMEOUT, + format!("{address} did not resolve in time"), + )); + } + } + } + }; + if found.is_empty() { + return Err(E(code::UNREACHABLE, format!("{address} has no address"))); + } + let networks = networks(); + // One refused address refuses the name, as which of them is dialled is not the caller's to choose. + for each in &found { + allowed(each.ip(), &networks).map_err(|why| match address.parse::() { + Ok(_) => E::forbidden(why), + Err(_) => E::forbidden(format!("{address} resolves to {}: {why}", each.ip())), + })?; + } + Ok(found) +} + +/// The address is resolved and checked once, and what was checked is what +/// is dialled; it is dialled before the stream opens, so one that nothing +/// answers on is an answer of its own. +pub async fn connect( + Path((address, port)): Path<(String, String)>, + upgrade: Result, +) -> R { + let upgrade = upgrade.map_err(|_| E::bad("An endpoint is opened only as a WebSocket"))?; + let port = port + .parse::() + .ok() + .filter(|port| *port > 0) + .ok_or_else(|| E::bad(format!("{port} is no port: a number from 1 to 65535")))?; + let addresses = resolve(&address, port).await?; + let mut last = String::new(); + for each in &addresses { + match tokio::time::timeout(DIAL_FOR, TcpStream::connect(each)).await { + Ok(Ok(tcp)) => { + let _ = tcp.set_nodelay(true); + return Ok(upgrade.on_upgrade(move |socket| crate::ports::tunnel(socket, tcp))); + } + Ok(Err(error)) => last = format!("{each}: {error}"), + Err(_) => { + return Err(E( + code::TIMEOUT, + format!("{each} does not answer from the host"), + )); + } + } + } + Err(E( + code::NOT_SERVED, + format!("Nothing answers on {address} port {port} from the host: {last}"), + )) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn ip(text: &str) -> IpAddr { + text.parse().unwrap() + } + + #[test] + fn only_addresses_on_the_hosts_own_networks_are_dialled() { + let networks = [ + Network { + address: ip("10.12.0.1"), + prefix: 24, + }, + Network { + address: ip("192.0.2.10"), + prefix: 24, + }, + Network { + address: ip("fd42:1::1"), + prefix: 64, + }, + ]; + for on in [ + "10.12.0.57", + "192.0.2.10", + "192.0.2.254", + "fd42:1::99", + "::ffff:10.12.0.2", + ] { + assert!(allowed(ip(on), &networks).is_ok(), "{on}"); + } + for (off, why) in [ + ("10.12.1.5", "no network"), + ("198.51.100.7", "no network"), + ("fd42:2::1", "no network"), + ("127.0.0.1", "ports.connect"), + ("::1", "ports.connect"), + ("169.254.169.254", "metadata"), + ("fd00:ec2::254", "metadata"), + ("169.254.3.4", "no address"), + ("fe80::1", "no address"), + ("0.0.0.0", "no address"), + ("224.0.0.1", "no address"), + ] { + let refused = allowed(ip(off), &networks).unwrap_err(); + assert!(refused.contains(why), "{off}: {refused}"); + } + // A metadata address is refused even on a network the host is on. + let cloud = [Network { + address: ip("169.254.0.1"), + prefix: 16, + }]; + assert!(allowed(ip("169.254.169.254"), &cloud).is_err()); + } + + #[test] + fn the_hosts_networks_leave_out_loopback() { + for network in networks() { + assert!(!network.address.is_loopback(), "{network:?}"); + assert!(!link_local(network.address), "{network:?}"); + } + } +} diff --git a/crates/berthd/src/error.rs b/crates/berthd/src/error.rs index 56ae8a2..990232d 100644 --- a/crates/berthd/src/error.rs +++ b/crates/berthd/src/error.rs @@ -38,12 +38,17 @@ impl From for E { impl IntoResponse for E { fn into_response(self) -> Response { let status = StatusCode::from_u16(self.0.0).unwrap_or(StatusCode::INTERNAL_SERVER_ERROR); + let message = berth_api::error_message_header(&self.1); let body = ErrorBody { error: ErrorDetail { code: self.0.1.into(), message: self.1, }, }; - (status, Json(body)).into_response() + let headers = [ + (berth_api::ERROR_HEADER, self.0.1.to_owned()), + (berth_api::ERROR_MESSAGE_HEADER, message), + ]; + (status, headers, Json(body)).into_response() } } diff --git a/crates/berthd/src/http.rs b/crates/berthd/src/http.rs index fdc86c7..fbea5cd 100644 --- a/crates/berthd/src/http.rs +++ b/crates/berthd/src/http.rs @@ -3,7 +3,7 @@ use crate::error::E; use crate::state::{Caller, Mode, State}; -use crate::{channel, files, services, sessions, system, tasks}; +use crate::{actions, channel, endpoints, files, ports, services, sessions, system}; use axum::{ Json, Router, extract::{Extension, Request, State as AppState}, @@ -46,7 +46,7 @@ pub const ROUTES: &[Route] = &[ Route { method: "POST", path: "/v1/sessions", summary: "Start a session.", level: "host", query: &[WAIT], body: "StartSession", reply: "Started", stream: "", handler: || post(sessions::start) }, Route { method: "GET", path: "/v1/sessions", summary: "All sessions, running and ended.", level: "host", query: &[], body: "", reply: "Session[]", stream: "", handler: || get(sessions::list) }, - Route { method: "GET", path: "/v1/sessions/{id}", summary: "One session's state. The agent sees those it started through a task.", level: "both", query: &[], body: "", reply: "Session", stream: "session", handler: || get(sessions::one) }, + Route { method: "GET", path: "/v1/sessions/{id}", summary: "One session's state. The agent sees those it started through an action.", level: "both", query: &[], body: "", reply: "Session", stream: "session", handler: || get(sessions::one) }, Route { method: "DELETE", path: "/v1/sessions/{id}", summary: "End it if it runs (HUP, after five seconds KILL) and forget it with its transcript.", level: "host", query: &[], body: "", reply: "Done", stream: "", handler: || delete(sessions::delete) }, Route { method: "GET", path: "/v1/sessions/{id}/output", summary: "The transcript, by bytes or by lines.", level: "both", query: &[("stream", "stdout, stderr or merged (default), for a process."), ("since", "Byte offset to read from."), ("limit", "Most bytes to return; default 1 MB."), ("line", "Read lines from this line number (a process only)."), ("count", "How many lines; default 100."), ("tail", "The last so many lines.")], body: "", reply: "Output", stream: "", handler: || get(sessions::output) }, Route { method: "GET", path: "/v1/sessions/{id}/screen", summary: "A terminal as it looks now, as text.", level: "both", query: &[("format", "text (default) or ansi, which keeps colours."), ("tail", "That many lines of screen and scrollback instead.")], body: "", reply: "Screen", stream: "", handler: || get(sessions::screen) }, @@ -54,9 +54,9 @@ pub const ROUTES: &[Route] = &[ Route { method: "POST", path: "/v1/sessions/{id}/resize", summary: "Give a terminal a new size.", level: "host", query: &[], body: "Resize", reply: "Done", stream: "", handler: || post(sessions::resize) }, Route { method: "POST", path: "/v1/sessions/{id}/signal", summary: "Send INT, TERM, HUP or KILL to the session's process group.", level: "host", query: &[], body: "Signal", reply: "Done", stream: "", handler: || post(sessions::signal) }, - Route { method: "GET", path: "/v1/tasks", summary: "The tasks the caller may run, in the order of the file.", level: "both", query: &[], body: "", reply: "Task[]", stream: "", handler: || get(tasks::list) }, - Route { method: "POST", path: "/v1/tasks/{name}", summary: "Run a task. It and what extends it are the steps of one session.", level: "both", query: &[WAIT], body: "RunTask", reply: "Started", stream: "", handler: || post(tasks::run) }, - Route { method: "GET", path: "/v1/tasks/{name}/choices/{argument}", summary: "The values to offer for an argument.", level: "both", query: &[], body: "", reply: "Choices", stream: "", handler: || get(tasks::choices) }, + Route { method: "GET", path: "/v1/actions", summary: "The actions the caller may run, in the order of the file.", level: "both", query: &[], body: "", reply: "HostAction[]", stream: "", handler: || get(actions::list) }, + Route { method: "POST", path: "/v1/actions/{action}", summary: "Run an action. It and what extends it are the steps of one session.", level: "both", query: &[WAIT], body: "RunAction", reply: "Started", stream: "", handler: || post(actions::run) }, + Route { method: "GET", path: "/v1/actions/{action}/choices/{argument}", summary: "The values to offer for an argument.", level: "both", query: &[], body: "", reply: "Choices", stream: "", handler: || get(actions::choices) }, Route { method: "GET", path: "/v1/stats", summary: "Load, CPU, memory, filesystems, network, GPU and unit states, read now.", level: "host", query: &[], body: "", reply: "Stats", stream: "", handler: || get(system::stats) }, Route { method: "GET", path: "/v1/screenshot", summary: "The shared desktop as PNG.", level: "host", query: &[], body: "", reply: "png", stream: "", handler: || get(system::screenshot) }, @@ -77,6 +77,10 @@ pub const ROUTES: &[Route] = &[ Route { method: "GET", path: "/v1/services", summary: "The services that can be called, and whether each is up.", level: "both", query: &[], body: "", reply: "Service[]", stream: "", handler: || get(services::list) }, Route { method: "POST", path: "/v1/services/{plugin}/{name}/{method}", summary: "Call a method of the service PLUGIN/NAME: JSON in, JSON out.", level: "both", query: &[], body: "json", reply: "json", stream: "", handler: || post(services::call) }, Route { method: "GET", path: "/v1/services/{plugin}/{name}/serve", summary: "Only as a WebSocket: for the program that answers the service's calls.", level: "both", query: &[], body: "", reply: "", stream: "serve", handler: || get(services::serve) }, + + Route { method: "GET", path: "/v1/ports/{port}", summary: "Only as a WebSocket: a TCP connection to a port on the host's loopback. A port that nothing listens on is an error before the handshake.", level: "host", query: &[], body: "", reply: "", stream: "port", handler: || get(ports::connect) }, + // berth serve calls it itself, for an endpoint a provisioner reported, and passes it through for nobody. + Route { method: "GET", path: "/v1/endpoints/{address}/{port}", summary: "Only as a WebSocket, and only for berth serve itself: a TCP connection to an address on a network of the host's own interfaces, such as a container on a bridge. Never loopback, link-local or a metadata address. One that nothing answers on is an error before the handshake.", level: "host", query: &[], body: "", reply: "", stream: "endpoint", handler: || get(endpoints::connect) }, ]; async fn version(AppState(state): AppState>) -> Json { @@ -93,22 +97,22 @@ async fn version(AppState(state): AppState>) -> Json { let applied = record .clone() .and_then(|record| serde_json::from_value(record).ok()); - // A record without a level is of a whole profile, as before levels, and - // only a workstation's apply or first boot leaves the profile it keeps. + // A record without a level is of a whole profile, as before there were + // types, and so is a kept profile without a record: the first boot of a + // workstation leaves one. The role records the type as `level` with its + // older values, and a node as `node`. let kept = state.config_dir.join("profile.json").exists(); - let level = match kept { - true => Some(api::HostLevel::Workstation), - false => record.map(|record| match record["level"].as_str() { - Some(level) => api::HostLevel::parse(level).unwrap_or_default(), - None => api::HostLevel::Workstation, - }), + let host_type = match record.as_ref().map(|record| record["level"].as_str()) { + Some(Some(level)) => Some(api::HostType::from_level(level).unwrap_or_default()), + Some(None) => Some(api::HostType::Workstation), + None => kept.then_some(api::HostType::Workstation), }; Json(api::Version { version: api::VERSION.into(), api: api::API, mode: mode.into(), applied, - level, + host_type, }) } @@ -149,7 +153,7 @@ async fn portal( upgrade: axum::extract::ws::WebSocketUpgrade, ) -> Response { let inner = router(state).layer(Extension(caller)); - upgrade.on_upgrade(move |socket| { + crate::portal::upgrade(upgrade).on_upgrade(move |socket| { crate::portal::run(socket, inner, crate::portal::host_operations()) }) } @@ -320,8 +324,8 @@ pub fn paths(prefix: &str) -> serde_json::Map { api::Resize, api::Signal, api::Argument, - api::Task, - api::RunTask, + api::HostAction, + api::RunAction, api::Choices, api::Memory, api::Filesystem, @@ -377,11 +381,15 @@ const PATH_ARGUMENTS: &[(&str, &str, &str)] = &[ "id", "A session's id, as starting it or the list gives it.", ), - ("tasks.", "name", "A task's name, as the list gives it."), ( - "tasks.", + "actions.", + "action", + "An action's name, as the list gives it.", + ), + ( + "actions.", "argument", - "The name of one of the task's arguments.", + "The name of one of the action's arguments.", ), ("system.power", "verb", "`reboot` or `poweroff`."), ( @@ -403,6 +411,21 @@ const PATH_ARGUMENTS: &[(&str, &str, &str)] = &[ ("services.", "plugin", "The plugin the service belongs to."), ("services.", "name", "The service's name within its plugin."), ("services.call", "method", "One of the service's methods."), + ( + "endpoints.", + "address", + "An IP address or a DNS name, on a network of the host's own interfaces.", + ), + ( + "endpoints.", + "port", + "A TCP port: a number from 1 to 65535.", + ), + ( + "ports.", + "port", + "A TCP port on the host's loopback, 127.0.0.1: a number from 1 to 65535.", + ), ("jobs.", "id", "A job's id."), ("kv.", "namespace", "A namespace."), ("kv.", "key", "A key."), diff --git a/crates/berthd/src/lib.rs b/crates/berthd/src/lib.rs index 4f7079a..034d124 100644 --- a/crates/berthd/src/lib.rs +++ b/crates/berthd/src/lib.rs @@ -1,16 +1,18 @@ -//! berthd: the host's API. Sessions, tasks, system calls, a channel and +//! berthd: the host's API. Sessions, actions, system calls, a channel and //! services, over unix sockets. +mod actions; mod channel; +pub mod endpoints; mod error; mod files; pub mod http; pub mod portal; +pub mod ports; mod services; mod sessions; pub mod state; mod system; -mod tasks; mod transcript; mod users; diff --git a/crates/berthd/src/portal.rs b/crates/berthd/src/portal.rs index 69f3329..8dfe278 100644 --- a/crates/berthd/src/portal.rs +++ b/crates/berthd/src/portal.rs @@ -12,14 +12,20 @@ use axum::{ use berth_api::{ ErrorBody, b64, client, code, ops::{self, Kind, Op}, - portal::{tagged, untagged}, + portal::{self, Credit, Taken, WINDOWS, tagged, untagged}, unb64, }; use futures_util::{SinkExt, StreamExt}; use hyper_util::{rt::TokioIo, service::TowerToHyperService}; use serde_json::{Map, Value, json}; -use std::{collections::HashMap, sync::Arc}; -use tokio::sync::mpsc; +use std::{ + collections::HashMap, + sync::{ + Arc, + atomic::{AtomicUsize, Ordering}, + }, +}; +use tokio::sync::{OwnedSemaphorePermit, Semaphore, mpsc}; use tower::ServiceExt; /// The messages of the WebSocket an operation is on its own. @@ -42,6 +48,10 @@ pub type Resolve = Arc) -> Result + S pub fn host_operations() -> Resolve { Arc::new(|name, host| match (ops::find(ops::HOST, name), host) { (_, Some(_)) => Err((code::BAD_REQUEST, "A host has no hosts".into())), + // berth serve opens these at their own path, never through a portal. + (Some(op), None) if !op.server_only.is_empty() => { + Err((code::FORBIDDEN, op.server_only.into())) + } (Some(op), None) => Ok(Target { op, prefix: String::new(), @@ -52,6 +62,8 @@ pub fn host_operations() -> Resolve { fn failure(id: u64, problem: Problem) -> Value { let ((status, code), message) = problem; + // A message may carry what the caller sent, which is not sent back whole. + let message = portal::shorten(&message); json!({"id": id, "error": {"code": code, "message": message, "status": status}}) } @@ -231,6 +243,8 @@ async fn open( code::NOT_FOUND, code::BAD_REQUEST, code::UNREACHABLE, + code::NOT_SERVED, + code::TIMEOUT, ]; let kind = known .into_iter() @@ -240,34 +254,200 @@ async fn open( }) } +/// What one portal may hold for its caller. A caller that does not read +/// cannot make it hold more: the reader stops reading once its own replies +/// fill `control`, and a call or a stream's input that does not fit in +/// `held` is refused. +#[derive(Debug, Clone, Copy)] +pub struct Limits { + /// Bytes of frames of streams and of answers waiting for the socket. + pub room: usize, + /// Bytes of the reader's own replies and of credits waiting for the socket. + pub control: usize, + /// Bytes of calls in flight, of their answers until they are written, + /// and of input waiting for its stream. + pub held: usize, + /// Calls in flight at once. + pub calls: usize, + /// Streams open at once. + pub streams: usize, +} + +impl Default for Limits { + fn default() -> Limits { + Limits { + room: 1 << 20, + control: 64 * 1024, + held: portal::HELD_BYTES, + calls: portal::CALLS, + streams: portal::STREAMS, + } + } +} + +/// Set the most a WebSocket of the portal takes in one message. +pub fn upgrade( + upgrade: axum::extract::ws::WebSocketUpgrade, +) -> axum::extract::ws::WebSocketUpgrade { + upgrade + .max_message_size(portal::FRAME_BYTES) + .max_frame_size(portal::FRAME_BYTES) +} + +/// The most each budget of a portal held at once, and the frames it read: +/// what a test looks at. +#[derive(Debug, Default)] +pub struct Gauge { + pub read: AtomicUsize, + pub room: AtomicUsize, + pub control: AtomicUsize, + pub held: AtomicUsize, +} + +impl Gauge { + fn saw(&self, which: &AtomicUsize, budget: &Semaphore, size: usize) { + which.fetch_max( + size.saturating_sub(budget.available_permits()), + Ordering::Relaxed, + ); + } +} + +/// A message on its way to the socket, with the room it takes until it is written. +struct Queued { + message: Message, + _taken: Vec, +} + +/// The frames on their way to the socket, in the order they were sent. Only +/// the writer gives room back, and it waits on nothing but the socket, so a +/// task that waits for room waits for the network and never for another +/// task. +#[derive(Clone)] +struct Out { + queue: mpsc::UnboundedSender, + room: Arc, + control: Arc, + limits: Limits, + gauge: Arc, +} + +fn size_of(message: &Message) -> usize { + match message { + Message::Text(text) => text.len(), + Message::Binary(bytes) => bytes.len(), + _ => 0, + } +} + +impl Out { + /// A frame of a stream or the answer to a call: it waits for room, and + /// keeps what it holds of the portal's budget until it is written. + async fn data(&self, message: Message, held: Option) -> bool { + let size = size_of(&message).clamp(1, self.limits.room) as u32; + let Ok(permit) = self.room.clone().acquire_many_owned(size).await else { + return false; + }; + self.gauge + .saw(&self.gauge.room, &self.room, self.limits.room); + let mut taken = vec![permit]; + taken.extend(held); + self.queue + .send(Queued { + message, + _taken: taken, + }) + .is_ok() + } + + /// A small frame of the portal's own, such as an error or a credit. It + /// waits only for the writer, which waits only for the socket. + async fn control(&self, frame: Value) -> bool { + let message = Message::Text(frame.to_string().into()); + let size = size_of(&message).clamp(1, self.limits.control) as u32; + let Ok(permit) = self.control.clone().acquire_many_owned(size).await else { + return false; + }; + self.gauge + .saw(&self.gauge.control, &self.control, self.limits.control); + self.queue + .send(Queued { + message, + _taken: vec![permit], + }) + .is_ok() + } +} + +/// What the caller sends towards an open stream, each frame with what it +/// holds of the portal's budget until the stream took it. +struct Inbound { + input: mpsc::UnboundedSender<(upstream::Frame, OwnedSemaphorePermit)>, + credit: Arc, +} + +/// An error the portal answers with for a caller's frame, as the frame +/// that says it. +fn limit(id: u64, message: String) -> Value { + failure(id, (code::LIMIT, message)) +} + /// Serve a portal until its socket closes. `router` answers as the caller /// the portal was opened by. pub async fn run(socket: WebSocket, router: Router, resolve: Resolve) { + serve(socket, router, resolve, Limits::default(), Arc::default()).await +} + +/// [`run`], within `limits`, saying in `gauge` what it held. +pub async fn serve( + socket: WebSocket, + router: Router, + resolve: Resolve, + limits: Limits, + gauge: Arc, +) { let (mut sink, mut source) = socket.split(); - let (out, mut outgoing) = mpsc::unbounded_channel::(); - tokio::spawn(async move { - while let Some(message) = outgoing.recv().await { - if sink.send(message).await.is_err() { + let (queue, mut outgoing) = mpsc::unbounded_channel::(); + let room = Arc::new(Semaphore::new(limits.room)); + let control = Arc::new(Semaphore::new(limits.control)); + let held = Arc::new(Semaphore::new(limits.held)); + let calls = Arc::new(Semaphore::new(limits.calls)); + let out = Out { + queue, + room: room.clone(), + control: control.clone(), + limits, + gauge: gauge.clone(), + }; + let writer = tokio::spawn(async move { + while let Some(queued) = outgoing.recv().await { + if sink.send(queued.message).await.is_err() { break; } } + // Whoever waits for room would wait for a socket that is gone. + room.close(); + control.close(); }); - let say = |out: &mpsc::UnboundedSender, frame: Value| { - let _ = out.send(Message::Text(frame.to_string().into())); + let hold = |size: usize| { + let size = u32::try_from(size.max(1)).unwrap_or(u32::MAX); + let permit = held.clone().try_acquire_many_owned(size).ok(); + gauge.saw(&gauge.held, &held, limits.held); + permit }; - // What the caller sends towards each open stream. - let mut streams: HashMap> = HashMap::new(); + let mut streams: HashMap = HashMap::new(); while let Some(Ok(message)) = source.next().await { + gauge.read.fetch_add(1, Ordering::Relaxed); + let size = size_of(&message); let frame: Value = match message { Message::Text(text) => match serde_json::from_str(text.as_str()) { Ok(frame) => frame, Err(_) => continue, }, Message::Binary(bytes) => { - if let Some((id, bytes)) = untagged(&bytes) - && let Some(stream) = streams.get(&id) - { - let _ = stream.send(upstream::Frame::Binary(bytes.to_vec().into())); + if let Some((id, bytes)) = untagged(&bytes) { + let frame = upstream::Frame::Binary(bytes.to_vec().into()); + feed(&mut streams, id, frame, size, &hold, &out).await; } continue; } @@ -279,7 +459,14 @@ pub async fn run(socket: WebSocket, router: Router, resolve: Resolve) { if frame["end"].as_bool() == Some(true) { streams.remove(&id); } else if let Some(stream) = streams.get(&id) { - let _ = stream.send(upstream::Frame::Text(frame["data"].to_string().into())); + match frame["credit"].as_u64() { + Some(bytes) => stream.credit.grant(bytes), + None => { + let text = frame["data"].to_string(); + let frame = upstream::Frame::Text(text.into()); + feed(&mut streams, id, frame, size, &hold, &out).await; + } + } } continue; } @@ -287,79 +474,461 @@ pub async fn run(socket: WebSocket, router: Router, resolve: Resolve) { continue; }; let name = frame["op"].as_str().unwrap_or(""); + // The name is the caller's: a long one is not sent back. + if name.len() > portal::NAME_BYTES { + let problem = ( + code::BAD_REQUEST, + format!( + "An operation's name is at most {} bytes", + portal::NAME_BYTES + ), + ); + if !out.control(failure(id, problem)).await { + break; + } + continue; + } let target = match resolve(name, frame["host"].as_str()) { Ok(target) => target, Err(problem) => { - say(&out, failure(id, problem)); + if !out.control(failure(id, problem)).await { + break; + } continue; } }; - streams.retain(|_, stream| !stream.is_closed()); + streams.retain(|_, stream| !stream.input.is_closed()); match target.op.kind { Kind::Call => { - let (router, out) = (router.clone(), out.clone()); + let Ok(call_permit) = calls.clone().try_acquire_owned() else { + let said = format!( + "More than {} calls at once on this portal: wait for an answer", + limits.calls + ); + if !out.control(limit(id, said)).await { + break; + } + continue; + }; + let Some(asked) = hold(size) else { + let said = format!( + "More than {} bytes wait on this portal: read what came, or wait for an answer", + limits.held + ); + if !out.control(limit(id, said)).await { + break; + } + continue; + }; + let (router, out, held, gauge) = + (router.clone(), out.clone(), held.clone(), gauge.clone()); tokio::spawn(async move { - let frame = match call(router, target, &frame).await { + let answer = match call(router, target, &frame).await { Ok(result) => json!({"id": id, "result": result}), Err(problem) => failure(id, problem), }; - let _ = out.send(Message::Text(frame.to_string().into())); + drop((frame, asked)); + let mut answer = answer.to_string(); + let size = u32::try_from(answer.len().max(1)).unwrap_or(u32::MAX); + let taken = held.clone().try_acquire_many_owned(size).ok(); + gauge.saw(&gauge.held, &held, limits.held); + if taken.is_none() { + answer = limit(id, format!( + "The answer is {} bytes, and more than {} bytes wait on this portal: read what came, or ask over HTTP", + answer.len(), + limits.held + )) + .to_string(); + } + out.data(Message::Text(answer.into()), taken).await; + drop(call_permit); }); } Kind::Stream => { - let (towards, mut input) = mpsc::unbounded_channel(); - streams.insert(id as u32, towards); + if streams.len() >= limits.streams { + let said = format!( + "More than {} streams at once on this portal: end one first", + limits.streams + ); + if !out.control(limit(id, said)).await { + break; + } + continue; + } + let window = frame["window"] + .as_u64() + .map(|window| window.clamp(WINDOWS.0.into(), WINDOWS.1.into()) as u32); + let (towards, input) = mpsc::unbounded_channel(); + let credit = Arc::new(Credit::with(window)); + let inbound = Inbound { + input: towards, + credit: credit.clone(), + }; + streams.insert(id as u32, inbound); let (router, out) = (router.clone(), out.clone()); tokio::spawn(async move { let stream = match open(router, &target, &frame["args"]).await { Ok(stream) => stream, Err(problem) => { - let _ = - out.send(Message::Text(failure(id, problem).to_string().into())); + out.control(failure(id, problem)).await; return; } }; - let opened = json!({"id": id, "result": {"open": true}}); - let _ = out.send(Message::Text(opened.to_string().into())); - let (mut up, mut down) = stream.split(); - let sub = id as u32; - loop { - tokio::select! { - frame = down.next() => match frame { - Some(Ok(upstream::Frame::Text(text))) => { - let data = serde_json::from_str::(text.as_str()) - .unwrap_or_else(|_| json!(text.as_str())); - let frame = json!({"sub": sub, "data": data}); - if out.send(Message::Text(frame.to_string().into())).is_err() { - return; - } - } - Some(Ok(upstream::Frame::Binary(bytes))) => { - if out.send(Message::Binary(tagged(sub, &bytes).into())).is_err() { - return; - } - } - Some(Ok(upstream::Frame::Close(_))) | Some(Err(_)) | None => break, - Some(Ok(_)) => {} - }, - frame = input.recv() => match frame { - Some(frame) => { - if up.send(frame).await.is_err() { - break; - } - } - // The caller ended it, or left. - None => { - let _ = up.close().await; - return; - } - }, - } + let mut opened = json!({"id": id, "result": {"open": true}}); + if let Some(window) = window { + opened["result"]["window"] = json!(window); + } + if !out + .data(Message::Text(opened.to_string().into()), None) + .await + { + return; } - let end = json!({"sub": sub, "end": true}); - let _ = out.send(Message::Text(end.to_string().into())); + pipe(id as u32, stream, input, &out, &credit, window).await; }); } } } + drop(streams); + drop(out); + let _ = writer.await; +} + +/// Give the stream `id` a frame of the caller's, holding its size of the +/// portal's budget until the stream took it. A stream whose caller sends +/// more than the budget holds is ended, as a caller within its window never +/// does. +async fn feed( + streams: &mut HashMap, + id: u32, + frame: upstream::Frame, + size: usize, + hold: &impl Fn(usize) -> Option, + out: &Out, +) { + let Some(stream) = streams.get(&id) else { + return; + }; + match hold(size) { + Some(permit) => { + let _ = stream.input.send((frame, permit)); + } + None => { + streams.remove(&id); + let ((status, code), message) = ( + code::LIMIT, + format!( + "More than {} bytes wait on this portal: send a stream no more than its window", + out.limits.held + ), + ); + let end = json!({"sub": id, "end": true, + "error": {"code": code, "message": message, "status": status}}); + out.control(end).await; + } + } +} + +/// One open stream: its frames to the caller and the caller's to it, each way +/// on its own, so a direction that waits does not hold up the other. +async fn pipe( + sub: u32, + stream: client::WebSocketStream, + mut input: mpsc::UnboundedReceiver<(upstream::Frame, OwnedSemaphorePermit)>, + out: &Out, + credit: &Credit, + window: Option, +) { + let (mut up, mut down) = stream.split(); + let downstream = async { + while let Some(Ok(frame)) = down.next().await { + let message = match frame { + upstream::Frame::Text(text) => { + let data = serde_json::from_str::(text.as_str()) + .unwrap_or_else(|_| json!(text.as_str())); + Message::Text(json!({"sub": sub, "data": data}).to_string().into()) + } + upstream::Frame::Binary(bytes) => { + credit.room().await; + credit.spend(bytes.len()); + Message::Binary(tagged(sub, &bytes).into()) + } + upstream::Frame::Close(_) => break, + _ => continue, + }; + if !out.data(message, None).await { + return false; + } + } + true + }; + let upstream = async { + let mut taken = Taken::new(window); + while let Some((frame, held)) = input.recv().await { + let size = match &frame { + upstream::Frame::Binary(bytes) => bytes.len(), + _ => 0, + }; + // A stream that takes nothing more ends from its side, downstream. + if up.send(frame).await.is_err() { + return std::future::pending().await; + } + drop(held); + if let Some(credit) = taken.took(sub, size) { + out.control(credit).await; + } + } + // The caller ended it, or left. + let _ = up.close().await; + }; + tokio::select! { + ended = downstream => if ended { + let end = json!({"sub": sub, "end": true}); + out.data(Message::Text(end.to_string().into()), None).await; + }, + () = upstream => {} + } +} + +#[cfg(test)] +mod tests { + use super::*; + use axum::{extract::ws::WebSocketUpgrade, routing::get}; + use berth_api::{client::Frame, ops::Role, portal::Portal}; + use std::time::Duration; + use tokio::net::TcpStream; + + const fn op(name: &'static str, kind: Kind, path: &'static str) -> Op { + Op { + name, + kind, + method: "GET", + path, + body: ops::Body::Json, + headers: &[], + http_only: "", + role: Role::View, + key: "", + server_only: "", + } + } + + static QUICK: Op = op("quick", Kind::Call, "/v1/quick"); + static SLOW: Op = op("slow", Kind::Call, "/v1/slow"); + static SINK: Op = op("sink", Kind::Stream, "/v1/sink"); + + /// A portal on a port of the loopback, within `limits`, over a router + /// with a quick call, a slow one, and a stream that never reads. + async fn serving(limits: Limits) -> (std::net::SocketAddr, Arc) { + let inner = Router::new() + .route( + "/v1/quick", + get(|| async { axum::Json(json!({"ok": true})) }), + ) + .route( + "/v1/slow", + get(|| async { + tokio::time::sleep(Duration::from_millis(300)).await; + axum::Json(json!({"ok": true})) + }), + ) + .route( + "/v1/sink", + get(|upgrade: WebSocketUpgrade| async { + upgrade.on_upgrade(|socket| async move { + std::future::pending::<()>().await; + drop(socket); + }) + }), + ); + let resolve: Resolve = Arc::new(|name, _| { + [&QUICK, &SLOW, &SINK] + .into_iter() + .find(|known| known.name == name) + .map(|op| Target { + op, + prefix: String::new(), + }) + .ok_or_else(|| (code::NOT_FOUND, format!("No operation {name}"))) + }); + let gauge = Arc::new(Gauge::default()); + let watched = gauge.clone(); + let outer = Router::new().route( + "/v1/portal", + get(move |ws: WebSocketUpgrade| { + let (inner, resolve, gauge) = (inner.clone(), resolve.clone(), watched.clone()); + async move { + upgrade(ws) + .on_upgrade(move |socket| serve(socket, inner, resolve, limits, gauge)) + } + }), + ); + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let address = listener.local_addr().unwrap(); + tokio::spawn(async move { axum::serve(listener, outer).await }); + (address, gauge) + } + + async fn socket(address: std::net::SocketAddr) -> client::WebSocketStream { + let stream = TcpStream::connect(address).await.unwrap(); + client::websocket(stream, ops::PORTAL, "").await.unwrap() + } + + async fn portal(address: std::net::SocketAddr) -> Portal { + Portal::over(socket(address).await) + } + + async fn next_text(socket: &mut client::WebSocketStream) -> Value { + loop { + match tokio::time::timeout(Duration::from_secs(10), socket.next()) + .await + .unwrap() + { + Some(Ok(Frame::Text(text))) => return serde_json::from_str(text.as_str()).unwrap(), + Some(Ok(_)) => continue, + other => panic!("{other:?}"), + } + } + } + + #[tokio::test(flavor = "multi_thread")] + async fn a_caller_that_does_not_read_is_held_within_the_limits() { + let limits = Limits::default(); + let (address, gauge) = serving(limits).await; + let (mut sink, mut source) = socket(address).await.split(); + // A long name is not sent back. + let long = "x".repeat(60_000); + sink.send(Frame::Text(json!({"id": 1, "op": long}).to_string().into())) + .await + .unwrap(); + let mut back = None; + while back.is_none() { + if let Some(Ok(Frame::Text(text))) = source.next().await { + back = Some(text.to_string()); + } + } + let back = back.unwrap(); + assert!(back.len() < 300, "{back}"); + assert!( + back.contains("An operation's name is at most 128 bytes"), + "{back}" + ); + // Then it sends without reading: far more replies than the socket's buffers hold. + let sent = 400_000; + let flood = tokio::spawn(async move { + for id in 0..sent { + let frame = json!({"id": id, "op": "no-such-operation"}).to_string(); + if sink.send(Frame::Text(frame.into())).await.is_err() { + break; + } + } + }); + tokio::time::sleep(Duration::from_secs(3)).await; + let read = gauge.read.load(Ordering::Relaxed); + tokio::time::sleep(Duration::from_secs(1)).await; + // The reader stopped: it waits for room for its replies. + assert_eq!(gauge.read.load(Ordering::Relaxed), read); + assert!(read < sent, "the portal read all {read} frames"); + assert!(gauge.control.load(Ordering::Relaxed) <= limits.control); + assert!(gauge.room.load(Ordering::Relaxed) <= limits.room); + assert!(gauge.held.load(Ordering::Relaxed) <= limits.held); + // Another caller of the same server is answered meanwhile. + let other = portal(address).await; + let answered = tokio::time::timeout( + Duration::from_secs(5), + other.call("quick", None, Value::Null, None), + ) + .await + .unwrap() + .unwrap(); + assert_eq!(answered, json!({"ok": true})); + // Once it reads, it gets its answers. + let mut got = 0; + while got < 1000 { + if let Some(Ok(Frame::Text(_))) = source.next().await { + got += 1; + } + } + flood.abort(); + } + + #[tokio::test(flavor = "multi_thread")] + async fn many_quick_calls_all_come_back() { + let (address, gauge) = serving(Limits::default()).await; + let portal = portal(address).await; + for _ in 0..10 { + let calls = (0..200).map(|_| portal.call("quick", None, Value::Null, None)); + for answered in futures_util::future::join_all(calls).await { + assert_eq!(answered.unwrap(), json!({"ok": true})); + } + } + assert!(gauge.held.load(Ordering::Relaxed) <= Limits::default().held); + } + + #[tokio::test(flavor = "multi_thread")] + async fn calls_and_streams_at_once_are_limited() { + let limits = Limits { + calls: 3, + streams: 2, + ..Limits::default() + }; + let (address, _) = serving(limits).await; + let portal = portal(address).await; + let calls = (0..6).map(|_| portal.call("slow", None, Value::Null, None)); + let answered = futures_util::future::join_all(calls).await; + let refused: Vec = answered + .iter() + .filter_map(|answered| answered.as_ref().err()) + .map(|error| error.to_string()) + .collect(); + assert_eq!(refused.len(), 3, "{answered:?}"); + assert!( + refused[0].contains("More than 3 calls at once"), + "{refused:?}" + ); + assert_eq!(answered[0].as_ref().unwrap(), &json!({"ok": true})); + // Once they are answered, there is room again. + portal.call("slow", None, Value::Null, None).await.unwrap(); + + let first = portal.open("sink", None, Value::Null).await.unwrap(); + let _second = portal.open("sink", None, Value::Null).await.unwrap(); + let third = portal.open("sink", None, Value::Null).await.err().unwrap(); + assert!( + third.to_string().contains("More than 2 streams at once"), + "{third}" + ); + drop(first); + tokio::time::sleep(Duration::from_millis(200)).await; + portal.open("sink", None, Value::Null).await.unwrap(); + } + + #[tokio::test(flavor = "multi_thread")] + async fn a_stream_sent_past_what_the_portal_holds_ends() { + let limits = Limits { + held: 256 * 1024, + ..Limits::default() + }; + let (address, gauge) = serving(limits).await; + let mut socket = socket(address).await; + // Without a window, nothing but the portal's budget stops it. + socket + .send(Frame::Text( + json!({"id": 1, "op": "sink"}).to_string().into(), + )) + .await + .unwrap(); + assert_eq!(next_text(&mut socket).await["result"]["open"], true); + let chunk = vec![7u8; 32 * 1024]; + for _ in 0..64 { + socket + .send(Frame::Binary(tagged(1, &chunk).into())) + .await + .unwrap(); + } + let ended = next_text(&mut socket).await; + assert_eq!(ended["sub"], 1, "{ended}"); + assert_eq!(ended["end"], true, "{ended}"); + assert_eq!(ended["error"]["code"], "limit", "{ended}"); + assert!(gauge.held.load(Ordering::Relaxed) <= limits.held); + } } diff --git a/crates/berthd/src/ports.rs b/crates/berthd/src/ports.rs new file mode 100644 index 0000000..51495f4 --- /dev/null +++ b/crates/berthd/src/ports.rs @@ -0,0 +1,124 @@ +//! A TCP connection to a port on the host's loopback, as a stream. + +use crate::error::{E, R}; +use axum::{ + extract::{ + Path, + ws::{Message, WebSocket, WebSocketUpgrade, rejection::WebSocketUpgradeRejection}, + }, + response::Response, +}; +use berth_api::{code, streams::EOF}; +use futures_util::{SinkExt, StreamExt}; +use std::{net::Ipv4Addr, time::Duration}; +use tokio::{ + io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt}, + net::TcpStream, +}; + +/// How much is read from the port at a time, which is the most one frame carries. +const FRAME: usize = 64 * 1024; + +/// The port is dialled before the stream opens, so one that nothing listens +/// on is an answer of its own and not a stream that ends at once. +pub async fn connect( + Path(port): Path, + upgrade: Result, +) -> R { + let upgrade = upgrade.map_err(|_| E::bad("A port is opened only as a WebSocket"))?; + let port = port + .parse::() + .ok() + .filter(|port| *port > 0) + .ok_or_else(|| E::bad(format!("{port} is no port: a number from 1 to 65535")))?; + let dialled = tokio::time::timeout( + Duration::from_secs(10), + TcpStream::connect((Ipv4Addr::LOCALHOST, port)), + ) + .await; + let tcp = match dialled { + Ok(Ok(tcp)) => tcp, + Ok(Err(error)) => { + return Err(E( + code::NOT_SERVED, + format!("Nothing answers on port {port} of the host: {error}"), + )); + } + Err(_) => { + return Err(E( + code::TIMEOUT, + format!("Port {port} of the host does not answer"), + )); + } + }; + let _ = tcp.set_nodelay(true); + Ok(upgrade.on_upgrade(move |socket| tunnel(socket, tcp))) +} + +fn said_end(text: &str) -> bool { + serde_json::from_str::(text).is_ok_and(|frame| frame["eof"] == true) +} + +/// Bytes both ways between a stream and a connection, each direction on its +/// own. Each side's end is a frame, so the other direction goes on until it +/// ends too; the caller leaving, or the connection failing, ends both at once. +pub async fn tunnel(socket: WebSocket, connection: C) +where + C: AsyncRead + AsyncWrite + Send + 'static, +{ + let (mut sink, mut source) = socket.split(); + let (reader, writer) = tokio::io::split(connection); + let (said, heard) = tokio::sync::oneshot::channel::<()>(); + let upstream = async move { + let (mut writer, mut said) = (Some(writer), Some(said)); + while let Some(Ok(message)) = source.next().await { + match message { + Message::Binary(bytes) => { + if let Some(open) = writer.as_mut() + && open.write_all(&bytes).await.is_err() + { + writer = None; + } + } + Message::Text(text) if said_end(text.as_str()) => { + if let Some(mut open) = writer.take() { + let _ = open.shutdown().await; + } + if let Some(said) = said.take() { + let _ = said.send(()); + } + } + Message::Close(_) => break, + _ => {} + } + } + }; + let downstream = async move { + let mut reader = reader; + let mut buffer = vec![0; FRAME]; + let ended = loop { + match reader.read(&mut buffer).await { + Ok(0) => break true, + Ok(count) => { + let bytes = buffer[..count].to_vec(); + if sink.send(Message::Binary(bytes.into())).await.is_err() { + return; + } + } + Err(_) => break false, + } + }; + if ended { + if sink.send(Message::Text(EOF.into())).await.is_err() { + return; + } + // Until the caller has said its end too. + let _ = heard.await; + } + let _ = sink.send(Message::Close(None)).await; + }; + tokio::select! { + () = upstream => {} + () = downstream => {} + } +} diff --git a/crates/berthd/src/sessions.rs b/crates/berthd/src/sessions.rs index 5de8359..d08dd0e 100644 --- a/crates/berthd/src/sessions.rs +++ b/crates/berthd/src/sessions.rs @@ -82,7 +82,7 @@ pub struct Plan { pub title: String, pub command: Vec, pub user: String, - pub task: Option, + pub action: Option, /// The commands to run one after another, with the step each is. A plain /// session has one command and no step. pub steps: Vec<(Option, Command)>, @@ -280,7 +280,7 @@ impl Sessions { uid: caller.uid, session: None, }, - task: plan.task, + action: plan.action, started: api::now(), ended: None, exit_code: None, @@ -450,7 +450,7 @@ impl Session { self.meta.lock().unwrap().kind } - /// The agent may look at the sessions it started through a task. + /// The agent may look at the sessions it started through an action. pub fn visible_to(&self, caller: &Caller) -> bool { let meta = self.meta.lock().unwrap(); caller.host() @@ -785,7 +785,7 @@ pub async fn start( title: request.title.clone().unwrap_or_else(|| words.join(" ")), command: words, user: request.user.clone(), - task: None, + action: None, steps: vec![(None, command)], // Nobody can type into a command that is waited for in one request. close_input: input.is_some() || wait.wait.is_some(), @@ -806,7 +806,7 @@ fn visible(state: &State, caller: &Caller, id: &str) -> R> { let session = state.sessions.get(id)?; if !session.visible_to(caller) { return Err(E::forbidden( - "The agent sees only the sessions it started through a task", + "The agent sees only the sessions it started through an action", )); } Ok(session) diff --git a/crates/berthd/src/state.rs b/crates/berthd/src/state.rs index 54ccc15..85c2fc3 100644 --- a/crates/berthd/src/state.rs +++ b/crates/berthd/src/state.rs @@ -1,4 +1,4 @@ -use berth_api::{Config, Level, ServiceSpec, Task, Topic}; +use berth_api::{Config, HostAction, Level, ServiceSpec, Topic}; use std::{ path::{Path, PathBuf}, sync::{Arc, RwLock}, @@ -30,7 +30,7 @@ impl Caller { #[derive(Debug, Default)] pub struct Loaded { pub berthd: Config, - pub tasks: Vec, + pub actions: Vec, pub services: Vec, pub topics: Vec, } @@ -60,7 +60,11 @@ impl Loaded { let mut problems = vec![]; let loaded = Loaded { berthd: read(dir, "berthd.json", &mut problems), - tasks: read(dir, "tasks.json", &mut problems), + actions: match dir.join("actions.json").exists() { + true => read(dir, "actions.json", &mut problems), + // What a role of berth 0.5 wrote, until the host is applied again. + false => read(dir, "tasks.json", &mut problems), + }, services: read(dir, "services.json", &mut problems), topics: read(dir, "channels.json", &mut problems), }; diff --git a/crates/berthd/tests/api.rs b/crates/berthd/tests/api.rs index 958a200..647f967 100644 --- a/crates/berthd/tests/api.rs +++ b/crates/berthd/tests/api.rs @@ -431,31 +431,31 @@ async fn a_terminal_keeps_its_screen_for_viewers_that_come_and_go() { } #[tokio::test] -async fn tasks_run_as_steps_with_arguments() { - let tasks = json!([ +async fn actions_run_as_steps_with_arguments() { + let actions = json!([ {"name": "greet", "label": "Greet", "group": "Demo", "agent": true, "confirm": "Really?", "arguments": [{"name": "who", "required": true}, {"name": "how-loud", "default": "quiet"}, {"name": "pick", "choices": "echo one; echo; echo 'two words'"}], "run": "printf '%s|%s\\n' \"$BERTH_ARG_WHO\" \"$BERTH_ARG_HOW_LOUD\"; exit 2"}, - {"name": "plug:extra", "extends": ["greet"], "run": "echo extra for $BERTH_TASK"}, + {"name": "plug:extra", "extends": ["greet"], "run": "echo extra for $BERTH_ACTION and $BERTH_TASK"}, {"name": "plug:last", "extends": ["greet", "other"], "run": "exit 5"}, {"name": "private", "run": "echo host only"}, {"name": "feed", "input": "secrets", "run": "cat"}, {"name": "ask", "terminal": true, "agent": true, "run": "printf 'go? '; read answer; echo went $answer"}, ]); - let daemon = Daemon::start(&[("tasks.json", tasks)]); + let daemon = Daemon::start(&[("actions.json", actions)]); let mut host = daemon.host().await; let mut agent = daemon.agent().await; - let listed: Vec = host.get("/v1/tasks").await.unwrap(); + let listed: Vec = host.get("/v1/actions").await.unwrap(); assert_eq!(listed.len(), 6); - assert!(listed.iter().all(|task| task.run.is_empty())); + assert!(listed.iter().all(|action| action.run.is_empty())); assert_eq!(listed[0].confirm.as_deref(), Some("Really?")); - let listed: Vec = agent.get("/v1/tasks").await.unwrap(); + let listed: Vec = agent.get("/v1/actions").await.unwrap(); assert_eq!( listed .iter() - .map(|task| task.name.as_str()) + .map(|action| action.name.as_str()) .collect::>(), ["greet", "ask"] ); @@ -463,10 +463,10 @@ async fn tasks_run_as_steps_with_arguments() { // Arguments arrive in the environment, quotes and spaces untouched. All // steps run; the exit code is the first that was not 0. let run = json!({"arguments": {"who": "it's \"me\" $HOME; ls"}}); - let done: Value = host.post("/v1/tasks/greet?wait=10s", &run).await.unwrap(); + let done: Value = host.post("/v1/actions/greet?wait=10s", &run).await.unwrap(); assert_eq!( text(&done["stdout_b64"]), - "it's \"me\" $HOME; ls|quiet\nextra for greet\n" + "it's \"me\" $HOME; ls|quiet\nextra for greet and greet\n" ); assert_eq!(done["exit_code"], 2); let steps: Vec<(String, i64)> = done["steps"] @@ -488,13 +488,16 @@ async fn tasks_run_as_steps_with_arguments() { ("plug:last".to_owned(), 5) ] ); - assert_eq!(done["task"], "greet"); + assert_eq!(done["action"], "greet"); - assert_eq!(code(host.post("/v1/tasks/greet", &json!({})).await).0, 400); + assert_eq!( + code(host.post("/v1/actions/greet", &json!({})).await).0, + 400 + ); assert_eq!( code( host.post( - "/v1/tasks/greet", + "/v1/actions/greet", &json!({"arguments": {"who": "x", "nope": "y"}}) ) .await @@ -503,17 +506,17 @@ async fn tasks_run_as_steps_with_arguments() { 400 ); assert_eq!( - code(host.post("/v1/tasks/missing", &json!({})).await).0, + code(host.post("/v1/actions/missing", &json!({})).await).0, 404 ); - let choices: api::Choices = host.get("/v1/tasks/greet/choices/pick").await.unwrap(); + let choices: api::Choices = host.get("/v1/actions/greet/choices/pick").await.unwrap(); assert_eq!(choices.choices, ["one", "two words"]); // What the workstation sends as input reaches the command and nothing else does. - assert_eq!(code(host.post("/v1/tasks/feed", &json!({})).await).0, 400); + assert_eq!(code(host.post("/v1/actions/feed", &json!({})).await).0, 400); let done: Value = host .post( - "/v1/tasks/feed?wait=10s", + "/v1/actions/feed?wait=10s", &json!({"input_b64": api::b64(b"{\"k\":1}\n")}), ) .await @@ -522,11 +525,11 @@ async fn tasks_run_as_steps_with_arguments() { // The agent runs what is marked for it, reads and follows that session, and nothing else. assert_eq!( - code(agent.post("/v1/tasks/private", &json!({})).await).0, + code(agent.post("/v1/actions/private", &json!({})).await).0, 403 ); let hosts: Value = host - .post("/v1/tasks/private?wait=10s", &json!({})) + .post("/v1/actions/private?wait=10s", &json!({})) .await .unwrap(); assert_eq!( @@ -539,7 +542,7 @@ async fn tasks_run_as_steps_with_arguments() { 403 ); let started: Value = agent - .post("/v1/tasks/ask", &json!({"rows": 5, "cols": 40})) + .post("/v1/actions/ask", &json!({"rows": 5, "cols": 40})) .await .unwrap(); assert_eq!( @@ -1061,6 +1064,7 @@ async fn services_answer_calls_through_their_socket() { #[tokio::test] async fn a_restart_ends_sessions_and_keeps_their_transcripts() { + // What the role of berth 0.5 wrote is read until there is the file of now. let mut daemon = Daemon::start(&[("tasks.json", json!([{"name": "one", "run": "echo first"}]))]); let mut host = daemon.host().await; @@ -1074,18 +1078,20 @@ async fn a_restart_ends_sessions_and_keeps_their_transcripts() { let id = started["id"].as_str().unwrap().to_owned(); tokio::time::sleep(Duration::from_millis(200)).await; - // A reload takes up new tasks and leaves the session running. + let actions: Vec = host.get("/v1/actions").await.unwrap(); + assert_eq!(actions[0].name, "one"); + // A reload takes up new actions and leaves the session running. std::fs::write( - daemon.dir.path().join("etc/tasks.json"), + daemon.dir.path().join("etc/actions.json"), json!([{"name": "two", "run": "true"}]).to_string(), ) .unwrap(); let _: Value = host.post("/v1/reload", &json!({})).await.unwrap(); - let tasks: Vec = host.get("/v1/tasks").await.unwrap(); - assert_eq!(tasks[0].name, "two"); + let actions: Vec = host.get("/v1/actions").await.unwrap(); + assert_eq!(actions[0].name, "two"); let state: api::Session = host.get(&format!("/v1/sessions/{id}")).await.unwrap(); assert!(state.running()); - std::fs::write(daemon.dir.path().join("etc/tasks.json"), "not json").unwrap(); + std::fs::write(daemon.dir.path().join("etc/actions.json"), "not json").unwrap(); assert_eq!(code(host.post("/v1/reload", &json!({})).await).0, 400); drop(host); @@ -1228,7 +1234,7 @@ async fn the_portal_carries_calls_and_streams_together() { let agent = Portal::over(daemon.stream("agent", "/v1/portal").await); assert!( agent - .call("tasks.list", None, Value::Null, None) + .call("actions.list", None, Value::Null, None) .await .is_ok() ); @@ -1236,6 +1242,219 @@ async fn the_portal_carries_calls_and_streams_together() { assert_eq!(refused.unwrap_err().status(), 403); } +/// A listener on the loopback that sends back what it gets, and ends its +/// side once the caller has ended theirs. +async fn echo() -> u16 { + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let port = listener.local_addr().unwrap().port(); + tokio::spawn(async move { + while let Ok((connection, _)) = listener.accept().await { + tokio::spawn(async move { + use tokio::io::AsyncWriteExt; + let (mut reading, mut writing) = connection.into_split(); + let _ = tokio::io::copy(&mut reading, &mut writing).await; + let _ = writing.shutdown().await; + }); + } + }); + port +} + +/// Write `data` and end, while what comes back is read: what came back. +async fn through(open: T, data: &[u8]) -> Vec +where + T: tokio::io::AsyncRead + tokio::io::AsyncWrite + Send + 'static, +{ + use tokio::io::{AsyncReadExt, AsyncWriteExt}; + let (mut reading, mut writing) = tokio::io::split(open); + let sent = data.to_vec(); + let writer = tokio::spawn(async move { + writing.write_all(&sent).await.unwrap(); + writing.shutdown().await.unwrap(); + writing + }); + let mut back = vec![]; + tokio::time::timeout(Duration::from_secs(60), reading.read_to_end(&mut back)) + .await + .expect("all of it back in time") + .unwrap(); + drop(writer.await.unwrap()); + back +} + +#[tokio::test(flavor = "multi_thread")] +async fn a_port_on_the_loopback_is_a_stream() { + use berth_api::{portal::Portal, tunnel}; + use tokio::io::AsyncWriteExt; + let daemon = Daemon::start(&[]); + let port = echo().await; + let control = || tokio::net::UnixStream::connect(daemon.socket("control")); + + // What is sent comes back, and after this side's end the rest still arrives. + let open = tunnel::tunnel(control().await.unwrap(), port) + .await + .unwrap(); + assert_eq!(through(open, b"hello").await, b"hello"); + // A few MB at once, over the stream's own WebSocket and over the portal's windows. + let data: Vec = (0..4_000_000u32).map(|n| (n % 251) as u8).collect(); + let open = tunnel::tunnel(control().await.unwrap(), port) + .await + .unwrap(); + assert!(through(open, &data).await == data); + let portal = Portal::over(daemon.stream("control", "/v1/portal").await); + let open = portal.connect_port(None, port).await.unwrap(); + assert!(through(open, &data).await == data); + + // A stream that is not read holds up neither calls nor other streams. + // It takes more than the system's buffers hold, which the window then keeps. + let mut stuck = portal.connect_port(None, port).await.unwrap(); + let writing = vec![7u8; 64 << 20]; + let writer = tokio::spawn(async move { + stuck.write_all(&writing).await.unwrap(); + stuck + }); + tokio::time::sleep(Duration::from_millis(300)).await; + assert!(!writer.is_finished(), "the window let it all through"); + let version = tokio::time::timeout( + Duration::from_secs(10), + portal.call("version", None, Value::Null, None), + ) + .await + .expect("an answer beside the stuck stream"); + assert_eq!(version.unwrap()["api"], api::API); + let other = portal.connect_port(None, port).await.unwrap(); + assert_eq!(through(other, b"beside").await, b"beside"); + drop(writer); + + // A port nothing listens on is an error before the stream opens. + let closed = { + let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap(); + listener.local_addr().unwrap().port() + }; + let refused = tunnel::tunnel(control().await.unwrap(), closed) + .await + .err() + .unwrap(); + assert_eq!((refused.status(), refused.code()), (503, "not_served")); + let refused = portal.connect_port(None, closed).await.err().unwrap(); + assert_eq!((refused.status(), refused.code()), (503, "not_served")); + // Only a port of the loopback, by its number. + for wrong in ["0", "65536", "http", "192.0.2.1:22", "-1"] { + let path = format!("/v1/ports/{wrong}"); + let refused = client::websocket_unix(&daemon.socket("control"), &path) + .await + .err() + .unwrap(); + assert_eq!(refused.status(), 400, "{wrong}"); + } + // A plain request is no stream, and the agent may not open one. + let plain = daemon + .host() + .await + .get::(&format!("/v1/ports/{port}")) + .await; + assert_eq!(code(plain).0, 400); + let agent = client::websocket_unix(&daemon.socket("agent"), &format!("/v1/ports/{port}")) + .await + .err() + .unwrap(); + assert_eq!(agent.status(), 403); +} + +/// A listener like `echo`, on an address of one of the host's own networks, +/// or nothing when the machine has none but the loopback. +async fn echo_on_a_network() -> Option<(String, u16)> { + for network in berthd::endpoints::networks() { + let Ok(listener) = tokio::net::TcpListener::bind((network.address, 0)).await else { + continue; + }; + let port = listener.local_addr().unwrap().port(); + tokio::spawn(async move { + while let Ok((connection, _)) = listener.accept().await { + tokio::spawn(async move { + use tokio::io::AsyncWriteExt; + let (mut reading, mut writing) = connection.into_split(); + let _ = tokio::io::copy(&mut reading, &mut writing).await; + let _ = writing.shutdown().await; + }); + } + }); + return Some((network.address.to_string(), port)); + } + None +} + +#[tokio::test(flavor = "multi_thread")] +async fn an_address_on_the_hosts_own_networks_is_a_stream_for_the_server_alone() { + use berth_api::{portal::Portal, tunnel}; + let daemon = Daemon::start(&[]); + let control = || tokio::net::UnixStream::connect(daemon.socket("control")); + let open = |address: String, port: u16| async move { + let io = control().await.unwrap(); + tunnel::address_tunnel(io, &address, port).await + }; + + match echo_on_a_network().await { + Some((address, port)) => { + let data: Vec = (0..3_000_000u32).map(|n| (n % 251) as u8).collect(); + let opened = open(address.clone(), port).await.unwrap(); + assert_eq!(through(opened, b"hello").await, b"hello"); + let opened = open(address.clone(), port).await.unwrap(); + assert!(through(opened, &data).await == data); + // A port nothing listens on there is an error before the stream opens. + let closed = { + let listener = std::net::TcpListener::bind((address.as_str(), 0)).unwrap(); + listener.local_addr().unwrap().port() + }; + let refused = open(address.clone(), closed).await.err().unwrap(); + assert_eq!((refused.status(), refused.code()), (503, "not_served")); + } + None => eprintln!("skipped the round trip: this machine has no network but the loopback"), + } + + // Off the host's networks, its loopback, a metadata address, and a name for the loopback. + let port = echo().await; + for (address, why) in [ + ("192.0.2.1", "no network"), + ("127.0.0.1", "ports.connect"), + ("169.254.169.254", "metadata"), + ("fd00:ec2::254", "metadata"), + ("fe80::1", "no address"), + ("localhost", "ports.connect"), + ] { + let refused = open(address.into(), port).await.err().unwrap(); + assert_eq!(refused.status(), 403, "{address}: {refused}"); + assert!(refused.to_string().contains(why), "{address}: {refused}"); + } + for wrong in ["0", "65536", "http"] { + let path = format!("/v1/endpoints/192.0.2.1/{wrong}"); + let refused = client::websocket_unix(&daemon.socket("control"), &path) + .await + .err() + .unwrap(); + assert_eq!(refused.status(), 400, "{wrong}"); + } + + // The agent may not open it, and no portal carries it. + let agent = client::websocket_unix(&daemon.socket("agent"), "/v1/endpoints/192.0.2.1/80") + .await + .err() + .unwrap(); + assert_eq!(agent.status(), 403); + let portal = Portal::over(daemon.stream("control", "/v1/portal").await); + let args = json!({"address": "192.0.2.1", "port": 80}); + let refused = portal + .open("endpoints.connect", None, args) + .await + .err() + .unwrap(); + assert_eq!(refused.status(), 403, "{refused}"); + assert!( + refused.to_string().contains("resources.connect"), + "{refused}" + ); +} + /// The committed API documents are what the program generates. Run with /// BERTH_UPDATE_DOCS=1 to write them after a change to the API. #[test] @@ -1259,3 +1478,81 @@ fn the_committed_documents_are_current() { ); } } + +#[tokio::test] +async fn a_node_says_it_is_one_and_what_its_features_offer() { + let offer = json!({"feature": "incus/server", "provisioners": ["vm"], + "settings": {"server": "https://{address}:8444"}, "secrets": {"incus_server_cert": "CERT"}}); + // A node keeps its profile as a workstation does; its record says what it is. + let daemon = Daemon::start(&[ + ( + "applied.json", + json!({"berth": "0.6.0", "plugins": [], "level": "node", "offers": [offer]}), + ), + ( + "profile.json", + json!({"format": 2, "berth": "0.6.0", "host": "big", "profile": {}}), + ), + ]); + let mut host = daemon.host().await; + let version: api::Version = host.get("/v1/version").await.unwrap(); + assert_eq!(version.host_type, Some(api::HostType::Node)); + let applied = version.applied.unwrap(); + assert_eq!(applied.offers[0].feature, "incus/server"); + assert_eq!(applied.offers[0].secrets["incus_server_cert"], "CERT"); + // A kept profile without a record is a workstation's first boot. + let daemon = Daemon::start(&[( + "profile.json", + json!({"format": 2, "berth": "0.6.0", "host": "studio", "profile": {}}), + )]); + let mut host = daemon.host().await; + let version: api::Version = host.get("/v1/version").await.unwrap(); + assert_eq!(version.host_type, Some(api::HostType::Workstation)); +} + +#[tokio::test(flavor = "multi_thread")] +async fn an_agent_that_floods_its_portal_does_not_stop_berthd() { + use berth_api::portal::Portal; + let daemon = Daemon::start(&[]); + let (mut sink, mut source) = daemon.stream("agent", "/v1/portal").await.split(); + // A long name it sends is not sent back. + let name = "x".repeat(60_000); + sink.send(Frame::Text(json!({"id": 0, "op": name}).to_string().into())) + .await + .unwrap(); + let back = loop { + if let Some(Ok(Frame::Text(text))) = source.next().await { + break text.to_string(); + } + }; + assert!(back.len() < 300, "{back}"); + assert!(back.contains("\"code\":\"bad_request\""), "{back}"); + // It sends calls and never reads their answers. + let flood = tokio::spawn(async move { + for id in 1..200_000u64 { + let frame = json!({"id": id, "op": "no-such-operation"}).to_string(); + if sink.send(Frame::Text(frame.into())).await.is_err() { + break; + } + } + sink + }); + tokio::time::sleep(Duration::from_secs(2)).await; + // Every other caller is answered meanwhile, the agent's other portals too. + for level in ["agent", "control"] { + let portal = Portal::over(daemon.stream(level, "/v1/portal").await); + let version = tokio::time::timeout( + Duration::from_secs(10), + portal.call("version", None, Value::Null, None), + ) + .await + .unwrap() + .unwrap(); + assert_eq!(version["version"], api::VERSION, "{version}"); + } + let mut host = daemon.host().await; + let version: api::Version = host.get("/v1/version").await.unwrap(); + assert_eq!(version.version, api::VERSION); + flood.abort(); + drop(source); +} diff --git a/crates/box/src/main.rs b/crates/box/src/main.rs index 119ae01..3700a78 100644 --- a/crates/box/src/main.rs +++ b/crates/box/src/main.rs @@ -11,8 +11,8 @@ const USAGE: &str = "usage: box desktop screenshot [--output PATH|-] | click X Y [BUTTON] | type TEXT | key CHORD | scroll up|down [STEPS] | clipboard | pause | resume box mcp the desktop tools as an MCP server on stdio box exec [COMMAND...] run a command, or a shell, with the desktop's environment - box tasks the tasks this user may run - box run TASK [NAME=VALUE...] run one; output here, its exit code as box's + box actions the actions this user may run + box run ACTION [NAME=VALUE...] run one; output here, its exit code as box's box publish TOPIC [JSON] [--keep none|last|log] box listen TOPIC-PREFIX [--since ID] one JSON message per line box services what can be called, and whether it is up diff --git a/crates/box/src/net.rs b/crates/box/src/net.rs index 8a7c819..940588d 100644 --- a/crates/box/src/net.rs +++ b/crates/box/src/net.rs @@ -2,7 +2,7 @@ use crate::{assignments, json_or_string, serve, take_flag}; use berth_api::client::{self, Client, Frame, encode, encode_path}; -use berth_api::{Service, Session, SessionKind, Task}; +use berth_api::{HostAction, Service, Session, SessionKind}; use futures_util::{SinkExt, StreamExt}; use serde_json::{Value, json}; use std::io::{IsTerminal, Read, Write}; @@ -41,10 +41,10 @@ fn e(error: impl std::fmt::Display) -> String { async fn verb_async(verb: &str, mut args: Vec) -> Result { let usage = |text: &str| Err(format!("usage: box {text}")); match (verb, args.len()) { - ("tasks", 0) | ("services", 0) => {} + ("actions", 0) | ("services", 0) => {} ("run", 1..) | ("publish", 1..) | ("listen", 1..) | ("call", 2..) | ("serve", 3..) => {} - ("tasks" | "services", _) => return usage(verb), - ("run", _) => return usage("run TASK [NAME=VALUE…]"), + ("actions" | "services", _) => return usage(verb), + ("run", _) => return usage("run ACTION [NAME=VALUE…]"), ("publish", _) => return usage("publish TOPIC [JSON] [--keep none|last|log]"), ("listen", _) => return usage("listen TOPIC-PREFIX [--since ID]"), ("call", _) => return usage("call SERVICE METHOD [JSON]"), @@ -57,12 +57,12 @@ async fn verb_async(verb: &str, mut args: Vec) -> Result { let sock = socket()?; let mut api = Client::unix(&sock).await.map_err(e)?; match verb { - "tasks" => { - let tasks: Vec = api.get("/v1/tasks").await.map_err(e)?; - for task in tasks.iter().filter(|t| t.extends.is_empty()) { - let mut line = format!("{}\t{}", task.name, task.label); - if !task.arguments.is_empty() { - let names: Vec = task + "actions" => { + let actions: Vec = api.get("/v1/actions").await.map_err(e)?; + for action in actions.iter().filter(|a| a.extends.is_empty()) { + let mut line = format!("{}\t{}", action.name, action.label); + if !action.arguments.is_empty() { + let names: Vec = action .arguments .iter() .map(|a| format!("{}{}", a.name, if a.required { "*" } else { "" })) @@ -84,8 +84,8 @@ async fn verb_async(verb: &str, mut args: Vec) -> Result { } } "run" => { - let task = args.remove(0); - return run_task(&sock, &mut api, &task, &args).await; + let action = args.remove(0); + return run_action(&sock, &mut api, &action, &args).await; } "publish" => { let keep = take_flag(&mut args, "--keep")?; @@ -205,10 +205,10 @@ fn exit_status(code: Option) -> i32 { code.map_or(1, |n| (n & 0xff) as i32) } -async fn run_task( +async fn run_action( sock: &std::path::Path, api: &mut Client, - task: &str, + action: &str, words: &[String], ) -> Result { let mut body = json!({"arguments": assignments(words)?}); @@ -218,7 +218,7 @@ async fn run_task( body["cols"] = cols.into(); } let session: Session = api - .post(&format!("/v1/tasks/{}", encode(task)), &body) + .post(&format!("/v1/actions/{}", encode(action)), &body) .await .map_err(e)?; let mut ws = client::websocket_unix( diff --git a/crates/box/tests/berthd.rs b/crates/box/tests/berthd.rs index 3a1b6e1..44729bd 100644 --- a/crates/box/tests/berthd.rs +++ b/crates/box/tests/berthd.rs @@ -26,7 +26,7 @@ fn verbs_against_berthd() { let dir = tempfile::tempdir().unwrap(); std::fs::create_dir(dir.path().join("etc")).unwrap(); std::fs::write( - dir.path().join("etc/tasks.json"), + dir.path().join("etc/actions.json"), r#"[{"name":"hello","label":"Hello","user":"agent","agent":true,"arguments":[{"name":"who","required":true}],"run":"echo hi $BERTH_ARG_WHO; echo err >&2; exit 3"}]"#, ) .unwrap(); @@ -51,11 +51,11 @@ fn verbs_against_berthd() { cmd.env("BERTH_DIR", dir.path()).args(args); cmd }; - let out = bx(&["tasks"]).output().unwrap(); - let tasks = String::from_utf8_lossy(&out.stdout).into_owned(); - if !tasks.contains("hello\tHello\twho*") { + let out = bx(&["actions"]).output().unwrap(); + let actions = String::from_utf8_lossy(&out.stdout).into_owned(); + if !actions.contains("hello\tHello\twho*") { return eprintln!( - "skipped: berthd does not serve tasks yet: {tasks} {}", + "skipped: berthd does not serve actions yet: {actions} {}", String::from_utf8_lossy(&out.stderr) ); } diff --git a/docs/.gitignore b/docs/.gitignore index 1bae26f..3644477 100644 --- a/docs/.gitignore +++ b/docs/.gitignore @@ -11,6 +11,3 @@ npm-debug.log* /content/docs/api/host/ /content/docs/api/serve/ /public/openapi/ - -# the page of berth serve, copied from target/web-dist (npm run gather) -/public/web/ diff --git a/docs/content/docs/(guide)/deploy/ansible.mdx b/docs/content/docs/(guide)/deploy/ansible.mdx index 51e6376..0e8454c 100644 --- a/docs/content/docs/(guide)/deploy/ansible.mdx +++ b/docs/content/docs/(guide)/deploy/ansible.mdx @@ -57,17 +57,17 @@ Remote runs also leave the current SSH source address open in the firewall when -## Profiles +## Targets -`devbox_profile` selects defaults for the kind of machine: +`devbox_profile` selects defaults for the target, the kind of machine the host is installed on. A berth profile says it as `target`: -| Profile | For | +| Target | For | | --- | --- | | `laptop` | Physical machines, with the LightDM login and power defaults | | `cloud` | Cloud instances, virtual machines and tests. Set a bootstrap SSH CIDR to limit SSH, such as `10.0.2.2/32` for a QEMU guest | | `container` | The container image | -Cloud networking and disks are left as they are in every profile. The agent's username and home are configurable, and its UID is discovered rather than assumed. +Cloud networking and disks are left as they are on every target. The agent's username and home are configurable, and its UID is discovered rather than assumed. ## Settings diff --git a/docs/content/docs/(guide)/deploy/cloud.mdx b/docs/content/docs/(guide)/deploy/cloud.mdx index 3570536..e910f85 100644 --- a/docs/content/docs/(guide)/deploy/cloud.mdx +++ b/docs/content/docs/(guide)/deploy/cloud.mdx @@ -17,7 +17,7 @@ The seed gives the operator [the key of berth itself](/docs/reference/profile#th ### Set the profile -Create the host. On **Install**, choose **Cloud-init seed**, which switches a `laptop` **Host profile** to `cloud`. For the host to join the tailnet on first boot, give it an enrollment key there or switch on **Automatic enrollment key** ([joining the tailnet](/docs/use/access#joining-the-tailnet)). +Create the host. On **Install**, choose **Cloud-init seed**, which switches a `laptop` **Target** to `cloud`. With **Tailscale** on, berth puts a one-use key it makes into the seed, and the host joins the tailnet on first boot. That needs the secret `tailnet_api_key` ([joining the tailnet](/docs/use/access#joining-the-tailnet)). The key lasts 24 hours. When berth writes the seed, it says until when the key works. @@ -44,7 +44,7 @@ On **Settings**, enter your network in **Recovery SSH CIDR** under **Access**, s On **Run**, check that no problem is listed and press Enter on **Run · Cloud-init seed**, or: ```bash -berth seed cloud-1 +berth host seed cloud-1 ``` diff --git a/docs/content/docs/(guide)/deploy/container.mdx b/docs/content/docs/(guide)/deploy/container.mdx index 7b74c00..164d5be 100644 --- a/docs/content/docs/(guide)/deploy/container.mdx +++ b/docs/content/docs/(guide)/deploy/container.mdx @@ -42,7 +42,7 @@ Ports are published on host loopback only. If Docker runs on another machine, tu ## Reaching it from berth -The container runs `berthd` for the agent account, without systemd and without root. Sessions, tasks that run as the agent, the channel and files work. Calls that need systemd or another user are refused. A profile connects to it with a `link` command in its `builder` section: +The container runs `berthd` for the agent account, without systemd and without root. Sessions, actions that run as the agent, the channel and files work. Calls that need systemd or another user are refused. A profile connects to it with a `link` command in its `builder` section: ```json title="In the host's profile" { "builder": { "link": ["docker", "exec", "-i", "CONTAINER", "berthd", "connect"] } } diff --git a/docs/content/docs/(guide)/deploy/deployments.mdx b/docs/content/docs/(guide)/deploy/deployments.mdx index 78ce8ab..ce1ffbd 100644 --- a/docs/content/docs/(guide)/deploy/deployments.mdx +++ b/docs/content/docs/(guide)/deploy/deployments.mdx @@ -35,7 +35,7 @@ Store the password as a berth secret, then deploy: ```bash berth secret set notes_db_password -berth deploy notes --provisioner docker \ +berth resource create notes --provisioner docker \ --spec notes.compose.yaml --secret notes_db_password:DB_PASSWORD ``` @@ -43,7 +43,7 @@ berth deploy notes --provisioner docker \ ▸ Creating the deployment notes with docker … ✓ docker created notes: running -notes: berth resource r-p5x2hd7a +notes is r-p5x2hd7a: berth resource show r-p5x2hd7a ``` The output of Compose follows as it runs. The last line gives the new resource's id. @@ -54,26 +54,25 @@ The output of Compose follows as it runs. The last line gives the new resource's | `--provisioner NAME` | The provisioner that makes it, here `docker` | | `--spec FILE` | The Compose file. `-` reads it from standard input | | `--secret NAME[:VARIABLE]` | A berth secret it gets, as the variable `VARIABLE`, or as `NAME` when none is given. Give it once for each secret | -| `--token TEXT` | Any text of yours. The same token again answers with the first job, and nothing is made twice | | `NAME=VALUE` | Values of its arguments, such as `keep_volumes=yes`, or `host=…` for an engine other than the setting's | berth refuses before anything is made when a secret has no value, or when the name of a variable is not one a shell takes. A deployment of that name that exists already is changed instead, with the new file. ### From the provisioner's page -The Hosts tab has the row **+ New compose project**, and the provisioner's page the action of that name, `new-deployment`, in the terminal UI and in the page of [`berth serve --web`](/docs/use/server#in-a-browser). Both open a form laid out like a host's set-up, with the provisioner, the name, the Compose file, the secrets and the deployment's own arguments each on a row. Enter on a row edits it, and Enter on **Run** makes the deployment. The Compose file opens in a text editor that takes the whole screen. Ctrl+S keeps the file, and Esc leaves it as it was. The file is kept in berth's store, so no file on your machine is needed. Give the secrets as `NAME` or `NAME:VARIABLE`, separated by commas. The deployment is made as a job of the provisioner, and is listed on its page when the job ends. +The Hosts tab has the row **+ New compose project**, and the provisioner's page the action of that name, `new-deployment`, on every system. Both open a form laid out like a host's set-up, with the provisioner, the name, the Compose file, the secrets and the deployment's own arguments each on a row. Enter on a row edits it, and Enter on **Run** makes the deployment. The Compose file opens in a text editor that takes the whole screen. Ctrl+S keeps the file, and Esc leaves it as it was. The file is kept in berth's store, so no file on your machine is needed. Give the secrets as `NAME` or `NAME:VARIABLE`, separated by commas. The deployment is made as a job of the provisioner, and is listed on its page when the job ends. From the command line the action takes the same values: ```bash -berth provisioner docker run new-deployment name=notes \ +berth provisioner run docker new-deployment name=notes \ spec="$(cat notes.compose.yaml)" secrets=notes_db_password:DB_PASSWORD --yes ``` ## Look after it ```bash -berth resource notes +berth resource show notes ``` ```text @@ -86,7 +85,7 @@ Spec 5f0c1d9e8a7b6c5d4e3f2a1b0c9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a3b2c1d Secrets notes_db_password as DB_PASSWORD Arguments host=ssh://ops@192.0.2.10 keep_volumes=no -Actions (berth resource r-p5x2hd7a run ACTION) +Actions (berth resource run r-p5x2hd7a ACTION) start Start stop Stop change Change @@ -95,14 +94,14 @@ Actions (berth resource r-p5x2hd7a run ACTION) pull Pull and update remove Remove the deployment -Its log: berth resource r-p5x2hd7a logs [--follow] +Its log: berth resource logs r-p5x2hd7a [--follow] ``` ```bash -berth resource notes logs --follow # every service's log -berth resource notes run services # each container, with its state -berth resource notes run pull # newer images, then up again -berth resource notes run stop +berth resource logs notes --follow # every service's log +berth resource run notes services # each container, with its state +berth resource run notes pull # newer images, then up again +berth resource run notes stop ``` ## Change it @@ -110,9 +109,9 @@ berth resource notes run stop Deploy again with the same name, or use `change`: ```bash -berth deploy notes --provisioner docker --spec notes.compose.yaml -berth resource notes change --spec notes.compose.yaml -berth resource notes change keep_volumes=yes +berth resource create notes --provisioner docker --spec notes.compose.yaml +berth resource set notes --spec notes.compose.yaml +berth resource set notes keep_volumes=yes ``` The plugin runs `docker compose up -d --remove-orphans` with the file. With the same file nothing changes. Without `--spec`, the file berth has is used again. The secrets stay as they are unless `--secret` is given, which replaces the whole list. @@ -120,7 +119,7 @@ The plugin runs `docker compose up -d --remove-orphans` with the file. With the ## Remove it ```bash -berth resource notes remove +berth resource remove notes ``` berth asks `Remove the deployment notes with what it holds? [y/N]`, which `--yes` answers. The plugin runs `docker compose down`, which removes the project's named volumes too unless `keep_volumes` is `yes`. berth then forgets the record and the Compose file. @@ -139,7 +138,7 @@ Compose puts the values into the containers as the file says. A variable in a co ## With a berth server -With a server set, `berth deploy` sends the Compose file to the server. The server keeps it and makes the deployment as a job of the subject `resource:ID`. The secrets it gets are the server's. Making, changing and removing one need the role `admin`, and reading its log `operate` ([Resources](/docs/deploy/resources#with-a-berth-server)). +With a server set, `berth resource create` sends the Compose file to the server. The server keeps it and makes the deployment as a job of the subject `resource:ID`. The secrets it gets are the server's. Making, changing and removing one need the role `admin`, and reading its log `operate` ([Resources](/docs/deploy/resources#with-a-berth-server)). ## The Docker plugin's part diff --git a/docs/content/docs/(guide)/deploy/index.mdx b/docs/content/docs/(guide)/deploy/index.mdx index f5b372c..470c9c8 100644 --- a/docs/content/docs/(guide)/deploy/index.mdx +++ b/docs/content/docs/(guide)/deploy/index.mdx @@ -10,9 +10,9 @@ The install methods all run the same Ansible role with the settings of the host' | Method | For | Command | | --- | --- | --- | -| USB installer | A blank laptop or PC; erases a disk and installs Ubuntu | `berth iso HOST` | -| Cloud-init seed | A fresh cloud machine at a provider that takes cloud-init | `berth seed HOST` | -| Create a machine | A container or virtual machine that a plugin creates, such as on Docker or Incus | Install tab, or `berth provision HOST PROVISIONER` | +| USB installer | A blank laptop or PC; erases a disk and installs Ubuntu | `berth host iso HOST` | +| Cloud-init seed | A fresh cloud machine at a provider that takes cloud-init | `berth host seed HOST` | +| Create a machine | A container or virtual machine that a plugin creates, such as on Docker or Incus | Install tab, or `berth host create HOST --provisioner PROVISIONER` | | Existing host over SSH | An Ubuntu 26.04 machine you can log in to | Install tab | | This machine or an inventory | The machine berth runs on, or your own Ansible inventory | Install tab, or `ansible-playbook` | | Container image | Trying the desktop on Docker | Install tab, or `docker compose` | @@ -55,15 +55,15 @@ The set-up of a host has four tabs. Each tab depends only on the tabs before it. | Tab | What it holds | | --- | --- | -| Install | How to install, with the host profile (not for a created machine, which is always `cloud`), the provisioner and its required arguments, the SSH address, the hostname and Tailscale | +| Install | How to install, with the target (not for a created machine, which is always `cloud`), the provisioner and its required arguments, the SSH address, the hostname and Tailscale | | Features | What gets installed, including Cockpit and Docker on the host | | Settings | Everything with a default, such as the machine's resources, the accounts, registries and secrets, and the installer's disk and Wi-Fi | | Run | The last files for the method, every problem, and the buttons | A setting shows only when the method and features it belongs to are chosen. Each of the first three tabs lists its own problems at the bottom. **Run** lists all of them under the name of their tab, and does not start while one is listed. Warnings, such as SSH being open to any address, are listed too but do not stop it. **Save** on Run, or Ctrl+S anywhere, saves the profile. -Builds run inside berth with their output and a progress bar. Esc cancels and Enter returns. Installing over SSH and *This machine or an inventory* continue in the normal terminal, where Ansible and SSH can ask for passwords, and return to berth when they finish. +What Run starts is a job of the server, whose output fills a panel. Esc hides the panel and the job goes on. It stays in the server's list. A password the job needs, such as a sudo password, is asked for in a dialog first. To provision this machine or an inventory, run the role [with Ansible directly](/docs/deploy/ansible). - The role installs services, changes the firewall and grants the agent sudo. With a blank inventory, *This machine or an inventory* provisions **the machine you are sitting at**. Only do that on a dedicated box. + The role installs services, changes the firewall and grants the agent sudo. Applied to `localhost`, it provisions **the machine you are sitting at**. Only do that on a dedicated box. diff --git a/docs/content/docs/(guide)/deploy/provisioners.mdx b/docs/content/docs/(guide)/deploy/provisioners.mdx index 53b21c8..933b751 100644 --- a/docs/content/docs/(guide)/deploy/provisioners.mdx +++ b/docs/content/docs/(guide)/deploy/provisioners.mdx @@ -14,6 +14,8 @@ Each host has at most one machine made this way. ## Register a provisioner +A [node](/docs/use/nodes) registers the provisioners of its engines by itself when it is applied, such as `big-incus-vm`. Register one by hand for an engine that is not a node's. + See the types the plugins bring: ```bash @@ -36,7 +38,7 @@ berth provisioner add docker docker/container Names take lowercase letters, digits, `-` and `_`, and start with a letter. The type stays as it was registered. To use another, register one under another name. See which provisioners you have, and whether each can be used here: ```bash -berth provisioner +berth provisioner list ``` ```text @@ -44,7 +46,7 @@ docker Docker container (docker/container) ovh-incus Incus virtual machine (incus/vm) · not usable: ovh-incus needs the secrets incus_client_cert, incus_client_key: berth secret set NAME ``` -One that cannot be used says what to change, such as a [secret](#secrets) to set or a plugin to update. `berth provisioners` lists the same with what each asks for: +One that cannot be used says what to change, such as a [secret](#secrets) to set or a plugin to update. `berth provisioner list` lists the same with what each asks for: ```text ovh-incus Incus virtual machine @@ -61,7 +63,7 @@ The listing is shortened here. A secret marked *not set* is required and has no In the terminal UI, `a` on the **Provisioners** tab adds one. It asks for the name and then the type. -`berth provisioner remove NAME` forgets a provisioner, with its settings and its own secrets. berth refuses while it holds anything, so remove what it made first. The action `remove` on its page does the same. +`berth provisioner remove NAME` forgets a provisioner, with its settings and its own secrets. berth refuses while it holds anything, so remove what it made first. The action `remove` on its page does the same. The action `adopt` makes one you registered the provisioner of a [node](/docs/use/nodes#take-in-a-server-that-exists) that reaches the same server. ### From an earlier version @@ -70,7 +72,7 @@ berth before named each provisioner by its type and kept one set of settings for ## A provisioner's page ```bash -berth provisioner docker +berth provisioner show docker ``` ```text @@ -89,12 +91,13 @@ Machines berth made with it Compose project berth made with it notes notes running · r-p5x2hd7a -Actions (berth provisioner docker run ACTION) +Actions (berth provisioner run docker ACTION) Create new-deployment New compose project - new-instance New instance + new-instance New container from an image Set-Up settings Change settings + adopt Make it a node's remove Remove provisioner Engine engine Check the engine @@ -124,12 +127,12 @@ What berth made is listed by kind, each with berth's id of the resource and the A provisioner's own arguments say where it is, such as the Incus `server` or the Docker `host`. berth keeps their values as the provisioner's settings, in its store: ```bash -berth provisioner ovh-incus run settings server=https://192.0.2.10:8443 project=lab +berth provisioner set ovh-incus server=https://192.0.2.10:8443 project=lab ``` -Every call to the provisioner uses them, and so does each machine made with it unless `berth provision` is given another value. The record of a machine keeps the values it was made with, and an empty setting is forgotten. While a setting it needs has no value, berth prompts for it before each action. +Every call to the provisioner uses them, and so does each machine made with it unless `berth host create --provisioner` is given another value. The record of a machine keeps the values it was made with, and an empty setting is forgotten. While a setting it needs has no value, berth prompts for it before each action. -`berth provisioner NAME SETTING=VALUE…` asks another place this once, such as `berth provisioner ovh-incus server=https://192.0.2.11:8443`. +`berth provisioner show NAME SETTING=VALUE…` asks another place this once, such as `berth provisioner show ovh-incus server=https://192.0.2.11:8443`. ### Secrets @@ -144,14 +147,14 @@ The action `secrets` on the provisioner's page takes each in a hidden field. An ### Actions ```bash -berth provisioner docker actions -berth provisioner docker run images -berth provisioner docker run prune volumes=yes +berth provisioner actions docker +berth provisioner run docker images +berth provisioner run docker prune volumes=yes ``` In a terminal, `run` prompts for a required argument you leave out. It puts the action's question first, unless `--yes` is given. An action that only reads, such as `images`, answers at once. One that changes something runs as a job and shows its output as it comes. Only one such job runs at a time for a provisioner. -In the terminal UI, the **Provisioners** tab of the home screen (`2`) lists the provisioners, and `a` registers one. Enter opens a page, which lists the actions in their groups and what berth made there, by kind. Enter on an action shows its question and then a dialog for its arguments. A job's output fills a panel. An argument of several lines, such as a Compose file, keeps the line breaks of what you paste. Each shows as `⏎`. Enter on something berth made opens its [resource's page](/docs/deploy/resources#in-the-terminal-ui), and Esc comes back. The page of [`berth serve --web`](/docs/use/server#in-a-browser) has the same tab. +In the terminal UI, the **Provisioners** tab of the home screen (`2`) lists the provisioners, and `a` registers one. Enter opens a page, which lists the actions in their groups and what berth made there, by kind. Enter on an action shows its question and then a dialog for its arguments. A job's output fills a panel. An argument of several lines, such as a Compose file, keeps the line breaks of what you paste. Each shows as `⏎`. Enter on something berth made opens its [resource's page](/docs/deploy/resources#in-the-terminal-ui), and Esc comes back. ## Seed and apply @@ -159,10 +162,10 @@ A provisioner sets up the new machine in one of two ways. The plugin chooses. | Kind | The new machine | berth waits for | What the profile needs | | --- | --- | --- | --- | -| `seed` | Boots with berth's [cloud seed](/docs/deploy/cloud) made from the profile and sets itself up | berthd to answer, up to 30 minutes. It then prints the machine's provision state | What `berth seed` needs. A provisioner that puts the release archive into the machine itself does without a `release_url` | +| `seed` | Boots with berth's [cloud seed](/docs/deploy/cloud) made from the profile and sets itself up | berthd to answer, up to 30 minutes. It then prints the machine's provision state | What `berth host seed` needs. A provisioner that puts the release archive into the machine itself does without a `release_url` | | `apply` | Starts with SSH and an account for berth | SSH to answer, up to 5 minutes. berth then applies the profile as `berth apply` does | An SSH user, `access.user` | -For an `apply` provisioner the account is the profile's SSH user. It takes berth's [own key](/docs/reference/profile#the-key-of-berth-itself) and the keys of the profile's operator. If that user's sudo asks for a password, set `BERTH_SUDO_PASSWORD` for the command, or apply later with `berth apply HOST`. +For an `apply` provisioner the account is the profile's SSH user. It takes berth's [own key](/docs/reference/profile#the-key-of-berth-itself) and the keys of the profile's operator. If that user's sudo asks for a password, berth asks for it at the terminal. A script gives it with `--sudo-password-stdin`, or keeps it as the secret `sudo/HOST`. A provisioner may report the machine's address and SSH port. berth writes them to `access.address` and `access.port` of the profile, but only where those are empty. A machine just made has a new SSH host key, so berth forgets the host key it recorded for the host and records the new one at the first login. @@ -192,7 +195,7 @@ The first boot of a seeded machine installs everything and can take most of the │ Ansible task 87: Install the desktop … studio-2: still waiting, 12 minutes so far │ Ansible task 214: Configure the browser - │ first boot: configured-needs-enrollment + │ first boot: configured ✓ studio-2: berthd answers ``` @@ -229,7 +232,7 @@ berth refuses to start while a required secret has no value. ### Create it ```bash -berth provision studio-2 ovh-incus cpus=8 +berth host create studio-2 --provisioner ovh-incus cpus=8 ``` berth asks `Create a machine for studio-2 with Incus virtual machine (ovh-incus)? [y/N]`. `--yes` answers it. Arguments are `NAME=VALUE`. An argument that is required and has no default is asked for in a terminal when you leave it out. The others take their defaults. @@ -245,18 +248,18 @@ In the terminal UI, choose **Create a machine** under **How To Install** on the | --- | --- | | Install | **Provisioner**, a choice like any other. Space or Enter switches to the next one, and → chooses from the suggestions. Under it the required arguments, marked `*` | | Settings | Under **Machine**, the resources it has, such as CPUs, memory and disk. Under **Provisioner Options**, its other arguments | -| Run | **Create machine**, and every problem | +| Run | **Create with PROVISIONER**, and every problem | -The first provisioner is chosen when the list arrives. There is no **Host profile** row. A created machine is always a `cloud` host. berth writes that into the profile before it creates the machine, on the command line as well. Enter on an argument changes it. A missing secret is a problem of the Install tab, and a bad resource value one of the Settings tab. **Create machine** on the Run tab asks once before it saves the profile and starts, and does not start while a problem is listed. The output is shown as it runs. The page of [`berth serve --web`](/docs/use/server#in-a-browser) has the same tabs. +The first provisioner is chosen when the list arrives. There is no **Target** row. A created machine always has the target `cloud`. berth writes that into the profile before it creates the machine, on the command line as well. Enter on an argument changes it. A missing secret is a problem of the Install tab, and a bad resource value one of the Settings tab. **Create with PROVISIONER** on the Run tab asks once before it saves the profile and starts, and does not start while a problem is listed. The output is shown as it runs. The screens are the same with a [berth server](/docs/use/server) elsewhere. Only one thing runs at a time for a host. A second command for the same host is refused until the first ends. ## Look after the machine -The machine is a [resource](/docs/deploy/resources) of the kind `machine`. `berth instance HOST` shows it by the host's name, as `berth resource ID` does: +The machine is a [resource](/docs/deploy/resources) of the kind `machine`. `berth resource show` shows it by the host's name, as it does by its id: ```bash -berth instance studio-2 +berth resource show studio-2 ``` ```text @@ -269,18 +272,18 @@ Created 2026-11-12T20:14:03Z Asked 2026-11-13T08:02:11Z Arguments cpus=8 disk=60GiB image=ubuntu/26.04/cloud memory=8GiB pool=default project=default server=https://192.0.2.10:8443 -Actions (berth resource r-k3q7wz2m run ACTION) +Actions (berth resource run r-k3q7wz2m ACTION) start Start stop Stop change Change remove Delete the machine -Its log: berth resource r-k3q7wz2m logs [--follow] +Its log: berth resource logs r-k3q7wz2m [--follow] ``` The state comes from the provisioner each time you ask. A machine the platform no longer has is `missing`, and its record stays until you delete it through berth ([States](/docs/deploy/resources#states)). -`berth instance HOST start` and `stop` work only when the provisioner offers them. The actions above say which. The Run tab shows the machine's state and a **Start machine** and **Stop machine** row in the same cases, whatever install method is chosen. +`berth host start HOST` and `stop` work only when the provisioner offers them. The actions above say which. The Run tab shows the machine's state and a **Start** and **Stop** row in the same cases, whatever install method is chosen. ## When it fails @@ -288,22 +291,22 @@ What happens depends on how far it got. | It failed | What is left | To go on | | --- | --- | --- | -| Before the provisioner gave an id | A record with a token, and perhaps a machine | `berth provision` again with the same provisioner. The provisioner finds the machine by the token and does not make a second one | -| After it gave one, such as a boot that did not finish or a failed apply | The machine and its record. The message says so | Fix the cause and wait, or `berth apply HOST` for a failed apply. `berth resource ID remove` deletes the machine to start again | +| Before the provisioner gave an id | A record with a token, and perhaps a machine | `berth host create HOST --provisioner PROVISIONER` again with the same provisioner. The provisioner finds the machine by the token and does not make a second one | +| After it gave one, such as a boot that did not finish or a failed apply | The machine and its record. The message says so | Fix the cause and wait, or `berth apply HOST` for a failed apply. `berth resource remove ID` deletes the machine to start again | -A record without a machine is shown with a problem that says creating it did not end. Asking for a different provisioner while it exists is refused. `berth resource ID remove` then forgets the record only. A machine the provisioner made anyway is not deleted. +A record without a machine is shown with a problem that says creating it did not end. Asking for a different provisioner while it exists is refused. `berth resource remove ID` then forgets the record only. A machine the provisioner made anyway is not deleted. ## Delete the machine ```bash -berth instance studio-2 destroy +berth host remove studio-2 --destroy ``` -berth asks you to type the host's name. `--yes` skips that. `berth resource ID remove` does the same and asks yes or no. It runs the provisioner's `destroy` with the id it kept, never another, and then removes the record. A machine that is gone already counts as deleted. If deleting fails, the record stays so you can try again. +berth asks before it deletes anything, and `-y` answers yes. It deletes the machine and then forgets the host. To delete the machine alone and keep the profile, use `berth resource remove ID`. Either runs the provisioner's `destroy` with the id it kept, never another, and then removes the record. A machine that is gone already counts as deleted. If deleting fails, the record stays so you can try again. -The profile stays. The host's node on the tailnet is not removed. Remove it in the tailnet's admin console. +The host's node on the tailnet is not removed. Remove it in the tailnet's admin console. -Deleting needs the provisioner's plugin and its required secrets. Removing a host with `berth remove` is refused while its machine exists. Delete the machine first. +Deleting needs the provisioner's plugin and its required secrets. Removing a host with `berth host remove` is refused while its machine exists. Delete the machine first. ## With a berth server @@ -352,7 +355,7 @@ For an engine at `ssh://user@address`, the provisioner only sets `DOCKER_HOST`, The container runs privileged by default. With `runtime=sysbox` it runs under [Sysbox](https://github.com/nestybox/sysbox) instead, which the Docker host needs to have. Read the limits in `plugins/docker/guide.md` before you use an engine that others share. In the container Chromium uses its setuid sandbox helper, as the [self-test](/docs/reference/on-the-host#the-self-test) describes. ```bash -berth provision studio-2 docker runtime=sysbox +berth host create studio-2 --provisioner docker runtime=sysbox ``` ### Incus @@ -365,5 +368,5 @@ Their actions only read: `server` says the server's version and whether it trust ```bash berth provisioner add ovh-incus incus/vm server=https://192.0.2.10:8443 -berth provision studio-2 ovh-incus +berth host create studio-2 --provisioner ovh-incus ``` diff --git a/docs/content/docs/(guide)/deploy/qemu.mdx b/docs/content/docs/(guide)/deploy/qemu.mdx index 1496a6a..fe6a56a 100644 --- a/docs/content/docs/(guide)/deploy/qemu.mdx +++ b/docs/content/docs/(guide)/deploy/qemu.mdx @@ -12,7 +12,7 @@ QEMU, OVMF firmware and `cloud-image-utils` on your machine. Optionally Canonica ## Make a test host -Create a separate host with `profile: cloud` and the recovery SSH CIDR `10.0.2.2/32`, QEMU's address for your machine. For the USB installer, use disk mode `auto`, or `serial` with the virtual serial `DEVBOX_TEST_DISK`. Check the virtio disk's udev identifier in the VM installer if it differs. +Create a separate host with `target: cloud` and the recovery SSH CIDR `10.0.2.2/32`, QEMU's address for your machine. For the USB installer, use disk mode `auto`, or `serial` with the virtual serial `DEVBOX_TEST_DISK`. Check the virtio disk's udev identifier in the VM installer if it differs. Never put a physical machine's disk serial into a VM seed, or a VM's into a physical one. @@ -53,7 +53,7 @@ python3 provision/bin/qemu.py cloud --directory build/vm-cloud \ -To test the role without publishing a release, boot the cloud image with a minimal user data that only creates your login. Then use **Existing host over SSH** against `127.0.0.1` port 2222 with the `qemu` profile. +To test the role without publishing a release, boot the cloud image with a minimal user data that only creates your login. Then use **Existing host over SSH** against `127.0.0.1` port 2222 with the target `cloud` and the recovery SSH CIDR `10.0.2.2/32`. diff --git a/docs/content/docs/(guide)/deploy/resources.mdx b/docs/content/docs/(guide)/deploy/resources.mdx index e89e039..e81dd06 100644 --- a/docs/content/docs/(guide)/deploy/resources.mdx +++ b/docs/content/docs/(guide)/deploy/resources.mdx @@ -7,7 +7,7 @@ icon: Shapes A resource is something a [provisioner](/docs/deploy/provisioners) made for berth. A host's machine is one. A plugin can declare other kinds, such as the [deployments](/docs/deploy/deployments) of the Docker plugin. berth keeps a record of each resource in its store and looks after all of them the same way. ```bash -berth resource +berth resource list ``` ```text @@ -21,10 +21,10 @@ The list comes from berth's records, with each resource as it was last seen. ber | Kind | Belongs to | Made with | | --- | --- | --- | -| `machine` | A saved host, at most one machine for each | `berth provision HOST PROVISIONER` ([Create the machine](/docs/deploy/provisioners#create-the-machine)) | -| A kind its plugin declares, such as `deployment` or `instance` | No host | `berth deploy` for a deployment, the provisioner's `new-KIND` action, or `resources.create` for any kind | +| `machine` | A saved host, at most one machine for each | `berth host create HOST --provisioner PROVISIONER` ([Create the machine](/docs/deploy/provisioners#create-the-machine)) | +| A kind its plugin declares, such as `deployment` or `instance` | No host | `berth resource create` for a deployment, the provisioner's `new-KIND` action, or `resources.create` for any kind | -A provisioner's page says which kinds it makes. Each kind lists which of `start`, `stop`, `change` and `logs` its provisioner takes for it, and can bring actions of its own. +A provisioner's page says which kinds it makes. Each kind lists which of `start`, `stop`, `change`, `logs` and `exec` its provisioner takes for it, and can bring actions of its own. ## What berth keeps of one @@ -36,6 +36,7 @@ A provisioner's page says which kinds it makes. Each kind lists which of `start` | `host` | The saved host, for a machine | | `state`, `checked` | How it was when berth last asked, and when that was (`Asked`) | | `address` | Where it is reached, when the provisioner says | +| `endpoints` | The addresses and ports it serves on, as the provisioner last said ([Endpoints](#endpoints)) | | `arguments` | The values it was made or last changed with | | `secrets` | The names of the berth secrets it is given, and the variable each becomes. Never their values | | `spec_sha256` | The SHA-256 of its specification, for a kind made from one | @@ -54,10 +55,10 @@ A missing resource keeps its record. berth does not forget a resource that was r ## Look at one -`berth resource ID` asks the provisioner how the resource is now, and keeps what it says. A name works in place of the id while only one resource has that name. +`berth resource show ID` asks the provisioner how the resource is now, and keeps what it says. A name works in place of the id while only one resource has that name. ```bash -berth resource r-k3q7wz2m +berth resource show r-k3q7wz2m ``` ```text @@ -70,13 +71,13 @@ Created 2026-11-12T20:14:03Z Asked 2026-11-13T08:02:11Z Arguments cpus=8 disk=60GiB image=ubuntu/26.04/cloud memory=8GiB pool=default project=default server=https://192.0.2.10:8443 -Actions (berth resource r-k3q7wz2m run ACTION) +Actions (berth resource run r-k3q7wz2m ACTION) start Start stop Stop change Change remove Delete the machine -Its log: berth resource r-k3q7wz2m logs [--follow] +Its log: berth resource logs r-k3q7wz2m [--follow] ``` An action that cannot run now says why beside its name, such as a secret of the provisioner that has no value. @@ -87,15 +88,16 @@ An action that cannot run now says why beside its name, such as a secret of the | --- | --- | --- | | `start`, `stop` | The kind lists that verb | Start or stop it. Stop asks first | | `change` | The kind lists `change` | Give it other values of the kind's arguments. A kind made from a specification also takes a new one, and other secrets | +| `exec` | The kind lists `exec` | Run a command in it and wait for it to end ([Run a command in it](#run-a-command-in-it)) | | The kind's own | Its plugin declares them | What the plugin does, such as `services` of a deployment | | `remove` | Always | Remove it with its provisioner and forget the record. For a machine, the host's profile stays | ```bash -berth resource r-k3q7wz2m actions -berth resource r-k3q7wz2m run stop -berth resource r-k3q7wz2m change cpus=4 memory=16GiB -berth resource r-k3q7wz2m logs --tail 50 --follow -berth resource r-k3q7wz2m remove +berth resource actions r-k3q7wz2m +berth resource run r-k3q7wz2m stop +berth resource set r-k3q7wz2m cpus=4 memory=16GiB +berth resource logs r-k3q7wz2m --tail 50 --follow +berth resource remove r-k3q7wz2m ``` `run` and `remove` put the action's question first, unless `--yes` is given. In a terminal, `run` prompts for a required argument you leave out. An action that only reads answers at once. One that changes something runs as a job and shows its output as it comes. berth then ends with the job's exit code. @@ -104,18 +106,84 @@ berth resource r-k3q7wz2m remove `logs` prints the last 100 lines, or `--tail N`. `--follow` goes on with what comes after them until the log ends or you stop it. A berth server ends a followed log after an hour. Lines that contain a secret show `[secret]` in its place. +## Endpoints + +A resource's endpoints say where it listens, as a list of addresses and ports. Its provisioner reports them each time it makes, changes or checks the resource. After a container restarts with a new address, its next check reports the new one. When the resource is gone, berth clears the list. + +| Field | Holds | +| --- | --- | +| `name` | Unique within the resource, such as `8080` or `web-80` | +| `address` | An IP address or a DNS name | +| `port` | The port | +| `protocol` | `tcp` or `udp` | +| `hint` | What speaks there, for people, such as `http` or `postgres`. It may be empty | +| `local` | `true` when the provisioner's engine runs on the machine that dials it: the server's own, or the node the provisioner is on | + +An address is as seen from the machine the provisioner's engine runs on, such as the Docker engine or the Incus server. With `local`, that machine is the one that dials. For a provisioner on the server's machine, the server then dials the address as it is, `127.0.0.1` among them. Without `local`, `127.0.0.1` names the engine's machine, and the server refuses to dial it. For an engine elsewhere, a provisioner reports addresses that other machines can connect to, such as the engine's own name. + +A change in the endpoints is a change of the resource for `events.follow`, so a follower of `resources` hears of it. + +### Reach an endpoint + +`resources.connect` opens a TCP connection to one endpoint, through the server ([The stream of an endpoint](/docs/extend/streams#the-stream-of-an-endpoint)). The caller names the endpoint by its name or its port, and needs the role `operate`. + +The server dials only what the provisioner reported. When it dials, it reads the resource's endpoints again and resolves the address then. It refuses: + +| What | Why | +| --- | --- | +| An endpoint the provisioner did not report | The tunnel is not a way to an address of the caller's choosing | +| A UDP endpoint | A tunnel is a TCP connection | +| A cloud's metadata address, `169.254.169.254` or `fd00:ec2::254` | It hands credentials of the server's machine to whoever asks | +| A loopback or link-local address, unless the endpoint is `local` and reports that very address | Such an address is the server's own. A name that resolves to one now, or one an engine elsewhere reported, is not what the provisioner meant | + +Who dials depends on where the provisioner is: + +| Provisioner | Who dials | How | +| --- | --- | --- | +| On a [node](/docs/use/nodes#reach-what-runs-on-a-node), registered by its apply | The node's controller | The server opens the connection through the link it has to the node. `127.0.0.1` is the node's loopback, and any other address has to be on a network of the node's own | +| Any other | The server | From its own machine, with the checks above | + +The server is right for an engine on its own machine or on a network it routes to. A node's controller is right for a container on a bridge such as `incusbr0` or `docker0`, which only the node routes to. + +For a provisioner on a node, the server still refuses an endpoint that was not reported, a UDP endpoint and a metadata address. The node's controller then refuses: + +| What | Why | +| --- | --- | +| A loopback address other than `127.0.0.1` | The controller reaches the node's loopback at `127.0.0.1` alone | +| A link-local or metadata address | It hands credentials of the node to whoever asks | +| An address on none of the node's networks | Such an address is reached through a router, which is not the node's to offer | + +When the node cannot be reached, the stream fails with `502 unreachable`. A node whose controller is older than this gives `501 not_available`: update its controller or apply the node. + +### Run a command in it + +The action `exec` runs a command in a resource whose kind has it, such as a Docker or an Incus instance. Its provisioner runs it, and the server waits for it to end. + +| Argument | Holds | +| --- | --- | +| `command` | The words of the command, split at spaces, or a JSON list such as `["sh", "-c", "echo hi"]` for a word with spaces in it | +| `stdin` | What the command reads on its standard input. Empty gives it none | + +It runs as a job of the resource, and needs the role `operate`. It does not hold the resource's lock, so several commands can run at once. The job's output is the command's, and the job ends with the command's exit status. The job records the command line, but not the standard input. Give a secret on standard input, never on the command line. + +```bash +curl -X POST "$BERTH/v1/resources/r-m4t8qv2c/actions/exec" \ + -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \ + -d '{"arguments": {"command": "psql -U app", "stdin": "select count(*) from notes;"}}' +``` + ## A host's machine A host's machine is a resource of the kind `machine`. These short forms go by the host's name: | Command | Is the same as | | --- | --- | -| `berth provision HOST PROVISIONER [NAME=VALUE…]` | Making the machine and setting it up as the host | -| `berth instance HOST` | `berth resource ID` | -| `berth instance HOST start\|stop` | `berth resource ID run start\|stop` | -| `berth instance HOST destroy [--yes]` | `berth resource ID remove`, but it asks you to type the host's name | +| `berth host create HOST --provisioner PROVISIONER [NAME=VALUE…]` | Making the machine and setting it up as the host | +| `berth resource show HOST` | `berth resource show ID` | +| `berth host start HOST`, `berth host stop HOST` | `berth resource run ID start`, `berth resource run ID stop` | +| `berth host remove HOST --destroy` | `berth resource remove ID`, and then `berth host remove HOST` | -`berth remove HOST` is refused while the host has a machine berth made. Its message gives the command that deletes the machine. +`berth host remove HOST` is refused while the host has a machine berth made. Its message gives the command that deletes the machine. ## Records, tokens and locks @@ -125,11 +193,11 @@ A host's machine is kept beside the host's profile, and every other resource in Each change is a job whose subject is `resource:ID`, and it holds the resource's lock while it runs. A second change to the same resource is refused until the first has ended. A host's machine shares the host's lock, so applying the host and changing its machine exclude each other. -A caller can give a token of its own, such as `berth deploy --token TEXT` or `token` in a request. The same token again answers with the job the first call started, and nothing runs a second time. +A caller of the API can give a token of its own, as `token` in a request. The same token again answers with the job the first call started, and nothing runs a second time. ### From an older berth -An older berth kept the record of a host's machine beside its profile. The first run of this berth makes each of those a resource of the kind `machine`, with its provisioner's id and token. It says how many it converted, and does it once. `berth resource` lists them afterwards. +An older berth kept the record of a host's machine beside its profile. The first run of this berth makes each of those a resource of the kind `machine`, with its provisioner's id and token. It says how many it converted, and does it once. `berth resource list` lists them afterwards. ## In the terminal UI @@ -145,13 +213,13 @@ Compose project berth made with it An entry the provisioner lists with berth's labels, but that berth has no record of, is marked *no record of it*. berth does not remove it. -Enter on an entry opens the resource's page. It shows the resource's state and fields, and its actions in their groups. **Show its log** is there when its kind gives logs, and follows the log in a panel until you close it. Esc goes back to the provisioner's page. The page of [`berth serve --web`](/docs/use/server#in-a-browser) has the same tab. +Enter on an entry opens the resource's page. It shows the resource's state and fields, and its actions in their groups. **Show its log** is there when its kind gives logs, and follows the log in a panel until you close it. Esc goes back to the provisioner's page. berth on macOS and Windows has the same tab ([On other systems](/docs/use/server#on-other-systems)). A host's machine is also on the host's Run tab, with its state and rows to start, stop and delete it. ## With a berth server -With a [berth server](/docs/use/server) set, `berth resource`, `berth deploy` and the short forms use the server's resources. What changes one runs as a job there, with the server's plugins, secrets and provisioner settings. +With a [berth server](/docs/use/server) set, `berth resource list`, `berth resource create` and the short forms use the server's resources. What changes one runs as a job there, with the server's plugins, secrets and provisioner settings. | Operation | Route | Role | | --- | --- | --- | @@ -164,7 +232,8 @@ With a [berth server](/docs/use/server) set, `berth resource`, `berth deploy` an | `resources.act` | `POST /v1/resources/{id}/actions/{action}` | The action's own | | `resources.logs` | `GET /v1/resources/{id}/logs?tail=N`, as plain text | `operate` | | `resources.follow` | A WebSocket at the same path. The lines come as binary frames, then `{"exited": …}` | `operate` | +| `resources.connect` | A WebSocket at `/v1/resources/{id}/endpoints/{endpoint}`: a TCP connection to the endpoint | `operate` | -An action of a resource needs `admin` unless its plugin says otherwise, including one that only reads. Requests and answers are in the [API reference](/docs/api/serve/resources). +An action of a resource needs `admin` unless its plugin says otherwise, including one that only reads. `exec` needs `operate`. Requests and answers are in the [API reference](/docs/api/serve/resources). With `serve_setup off`, a server takes changes to resources only on its own socket, not over the network ([Closing set-up](/docs/extend/serve#closing-set-up)). diff --git a/docs/content/docs/(guide)/deploy/ssh.mdx b/docs/content/docs/(guide)/deploy/ssh.mdx index cf4e806..85ec56f 100644 --- a/docs/content/docs/(guide)/deploy/ssh.mdx +++ b/docs/content/docs/(guide)/deploy/ssh.mdx @@ -6,7 +6,7 @@ icon: Server This method runs the role over SSH against a machine that already has Ubuntu. The role adds accounts, packages and services to the existing system without partitioning or reinstalling. -A machine that should stay as it is can be added with only an address instead, and get berth's controller alone or nothing at all ([Host levels](/docs/use/levels)). The **Level** row on the Install tab chooses. +A machine that should stay as it is can be added with only an address instead, and get berth's controller alone or nothing at all ([Host types](/docs/use/types)). The **Type** row on the Install tab chooses. The role installs services, changes the firewall policy and grants the agent sudo for the entire machine. Do not point it at a computer you use for anything else. @@ -34,7 +34,7 @@ On the host's **Install** tab, choose **Existing host over SSH**. The SSH rows f ### Give the address -Enter a hostname, an IP address or a `Host` alias from your SSH configuration in **SSH host**. Machines on your tailnet and aliases from `~/.ssh/config` are suggested. Accepting the host fills blank user and port fields from OpenSSH's effective configuration. +Enter a hostname, an IP address or a `Host` alias from your SSH configuration in **SSH host**. berth logs in from the machine its server runs on, with that machine's SSH configuration and its aliases. diff --git a/docs/content/docs/(guide)/deploy/usb.mdx b/docs/content/docs/(guide)/deploy/usb.mdx index b63cd60..73837c3 100644 --- a/docs/content/docs/(guide)/deploy/usb.mdx +++ b/docs/content/docs/(guide)/deploy/usb.mdx @@ -27,16 +27,16 @@ The USB installer is the Ubuntu Server 26.04 ISO with the host's profile built i berth garage-laptop ``` -It opens on **Install**. Choose **USB installer** under **How To Install**, and keep **Host profile** at `laptop`. **Hostname** is the name the machine gets. F4 generates one. +It opens on **Install**. Choose **USB installer** under **How To Install**, and keep **Target** at `laptop`. **Hostname** is the name the machine gets. F4 generates one. ### Optionally, join the tailnet -The Tailscale settings are on **Install** too. Give a short-lived, one-use Tailscale key in **Tailscale enrollment key**. Or switch on **Automatic enrollment key**: with the control server's API key in the secret `tailnet_api_key`, berth creates a one-use key valid for 24 hours each time it builds the installer. F4 on the key field does the same for a key to paste. Tailnets that require tags take them from `berth config tailnet_tags tag:devbox`. For a Headscale server, see [Headscale](/docs/use/access#headscale). +**Tailscale** is on **Install** too. With it on, berth makes a one-use key through the control server's API each time it builds the installer, from the secret `tailnet_api_key` ([joining the tailnet](/docs/use/access#joining-the-tailnet)). Tailnets that require tags take them from `berth config set tailnet_tags tag:devbox`. For a Headscale server, see [Headscale](/docs/use/access#headscale). -Reusable keys and existing machine identities do not belong in installer images. +The key lasts 24 hours. When berth builds the image, it says until when the image can join the tailnet. A machine installed later is set up without the tailnet, and the next apply from berth joins it. @@ -57,21 +57,23 @@ Under **Installer** on **Settings**, set **Disk selection**. `auto`, the default ### Set up the network -Wired DHCP is always configured. To add Wi-Fi, switch **Wi-Fi** on under **Installer** and give the SSID and password. Networks saved in NetworkManager on your machine are suggested, and F5 unlocks their saved passwords so choosing one fills in its password. Leave the interface blank: the installer finds the wireless interface on the target and falls back to wired networking when there is none. +Wired DHCP is always configured. To add Wi-Fi, switch **Wi-Fi** on under **Installer** and give the SSID and password. Leave the interface blank: the installer finds the wireless interface on the target and falls back to wired networking when there is none. ### Choose the ISO -On **Run**, F4 on **Source ISO** downloads the current Ubuntu Server ISO and checks it against Ubuntu's `SHA256SUMS`. When you choose a file you already have, berth computes its SHA-256. Compare that with Ubuntu's signed `SHA256SUMS`, because a checksum taken from the file only proves it has not changed since. +**Source ISO** on **Run** is the Ubuntu Server ISO to build from, a path on the machine the server runs on. Left empty, the build downloads the current one and checks it against Ubuntu's `SHA256SUMS`. For a file you already have, put the SHA-256 from Ubuntu's signed `SHA256SUMS` in **ISO SHA256**. The build checks the file against it. ### Build and write -Choose the drive under **Then write to USB drive** on **Run**, or press `w` when the build has finished. Then press Enter on **Run · USB installer**. berth asks which drive to use when several are connected and asks you to confirm erasing it. It then asks for your `sudo` password and writes the image with a progress bar. +Press Enter on **Run · USB installer**. berth saves the profile and opens the host's overview with its action **Build the USB installer**. Its dialog takes the drive to write and your `sudo` password for writing it. The help of the drive lists the USB drives connected now. When one drive is connected, it's filled in. Leave the drive empty to build the ISO alone. The build and the writing run as a job, whose output fills a panel. + +The installer is built on the machine the server runs on, which writes one of its own drives. berth on your machine does it with the server inside berth, or with a `berth serve` of yours there. From another machine the action is listed as unavailable, with that reason. @@ -79,8 +81,8 @@ Choose the drive under **Then write to USB drive** on **Run**, or press `w` when The same from the command line: ```bash -berth iso garage-laptop # downloads the ISO when none is set -berth iso garage-laptop --write /dev/sdX # and writes it, after asking +berth host iso garage-laptop # downloads the ISO when none is set +berth host iso garage-laptop --write /dev/sdX # and writes it, after asking ``` The image is `~/.local/share/berth/build/HOSTNAME-autoinstall.iso`. To write it by hand, use the script from the provisioning source. It accepts only a whole USB disk that is large enough and has no system mount on it, and unmounts anything else first: @@ -101,10 +103,10 @@ cat /var/lib/devbox/provision-state sudo systemctl start devbox-provision.service # start it again after a failure ``` -`configured-needs-enrollment` means the role finished but nobody has signed in yet. The `ready` marker means Ansible converged, not that every acceptance test passed. If a dependency fails, the state is `failed` and a timer retries. The success marker is only written after a successful run. +`configured` means the role finished. The `ready` marker means Ansible converged, not that every acceptance test passed. If a dependency fails, the state is `failed` and a timer retries. The success marker is only written after a successful run. - The installed machine accepts SSH only from the tailnet, unless you set a Tailscale key or a **Recovery SSH CIDR**. Set one of them, or be ready to use the console. + The installed machine accepts SSH only from the tailnet, unless you set a **Recovery SSH CIDR**. Keep **Tailscale** on with the secret `tailnet_api_key` set, set the CIDR, or be ready to use the console. ## Disk selection @@ -120,7 +122,7 @@ sudo systemctl start devbox-provision.service # start it again after a failu | `largest` / `smallest` | By capacity among eligible disks | | `ssd` / `hdd` | The only eligible non-rotating or rotating disk | -For `auto`, `model`, `path`, `largest` and `smallest`, `target_disk_ssd` narrows the candidates to `any`, `ssd` or `hdd`. Each mode needs exactly one final candidate, including when capacities tie. The Settings tab shows the fields of the chosen mode and offers the values in Suggestions. +For `auto`, `model`, `path`, `largest` and `smallest`, `target_disk_ssd` narrows the candidates to `any`, `ssd` or `hdd`. Each mode needs exactly one final candidate, including when capacities tie. The Settings tab shows the fields of the chosen mode. ```json title="In the host profile" { @@ -135,7 +137,7 @@ A serial comes from `udevadm info --query=property --name=/dev/nvme0n1`. The dis The ISO keeps the source image's boot configuration through `xorriso` replay and patches GRUB to install automatically. It embeds the seed and the release archive. The bootstrap checks the archive against the digest in the seed. -The USB drive and its seed can still hold credentials after the installed copies have been used. One-use enrollment keys limit how long they are useful. Secure Boot with the remastered image has not been qualified. +The USB drive and its seed can still hold credentials after the installed copies have been used. The enrollment key berth puts in it works once and for 24 hours. Secure Boot with the remastered image has not been qualified. ## Without berth diff --git a/docs/content/docs/(guide)/extend/index.mdx b/docs/content/docs/(guide)/extend/index.mdx index fb07e69..199f25c 100644 --- a/docs/content/docs/(guide)/extend/index.mdx +++ b/docs/content/docs/(guide)/extend/index.mdx @@ -6,7 +6,7 @@ icon: Puzzle The plugins that come with berth, installing your own, and switching features on. - The package format with its features, actions, tasks and guides. + The package format with its features, actions, Ansible tasks and guides. A program on the host that answers calls from the agent and from you. One local API over all hosts, with an inbox and a key-value store. The WebSockets of berthd and berth serve: frames, resuming, reconnecting. diff --git a/docs/content/docs/(guide)/extend/plugins.mdx b/docs/content/docs/(guide)/extend/plugins.mdx index 437593c..43efe75 100644 --- a/docs/content/docs/(guide)/extend/plugins.mdx +++ b/docs/content/docs/(guide)/extend/plugins.mdx @@ -1,10 +1,10 @@ --- title: Plugins -description: Optional software, tasks and guides for hosts, switched on per host. +description: Optional software, actions and guides for hosts, switched on per host. icon: Blocks --- -A plugin adds optional things to a host. Each thing is a **feature**: packages, set-up steps, a guide for the agents working there, and tasks on the host's overview. A plugin can have several features, each switched on by itself. Even the coding agents are plugins. +A plugin adds optional things to a host. Each thing is a **feature**: packages, set-up steps, a guide for the agents working there, and actions on the host's overview. A plugin can have several features, each switched on by itself. Even the coding agents are plugins. ## Plugins that come with berth @@ -44,41 +44,41 @@ The **Plugins** tab of the home screen (`6`) lists both with their features. Ent From the command line: ```bash -berth plugins add @acme/berth-plugin-foo # from a registry -berth plugins add ./foo-1.0.0.tgz # a tarball from npm pack -berth plugins add ~/src/berth-plugin-foo # a folder -berth plugins # what is found, and from where -berth plugins remove @acme/berth-plugin-foo +berth plugin add @acme/berth-plugin-foo # from a registry +berth plugin add ./foo-1.0.0.tgz # a tarball from npm pack +berth plugin add ~/src/berth-plugin-foo # a folder +berth plugin list # what is found, and from where +berth plugin remove @acme/berth-plugin-foo ``` `add` runs `npm install --ignore-scripts --prefix ~/.config/berth/plugins`, so registries and credentials come from your npm configuration. You can also copy a folder to `~/.config/berth/plugins/NAME/` by hand. Npm is not needed for that. -After each change berth makes there, it writes the folder's `package.json`, npm's lock file, uploaded `.tgz` files and the folders you copied in to its [store](/docs/use/store). A machine restored from the store gets them back, and npm installs the rest. A folder you copy in by hand is in the store after berth's next change of the plugins. Remove a plugin with `berth plugins remove`, as one deleted by hand comes back from the store. +After each change berth makes there, it writes the folder's `package.json`, npm's lock file, uploaded `.tgz` files and the folders you copied in to its [store](/docs/use/store). A machine restored from the store gets them back, and npm installs the rest. A folder you copy in by hand is in the store after berth's next change of the plugins. Remove a plugin with `berth plugin remove`, as one deleted by hand comes back from the store. ### Updating a plugin ```bash -berth plugins update --check # which have a newer version -berth plugins update foo # one, by its id or npm name -berth plugins update # all of them +berth plugin update --check # which have a newer version +berth plugin update foo # one, by its id or npm name +berth plugin update # all of them ``` `--check` runs `npm outdated` with your npm configuration, so it asks the registry each plugin came from. `update` installs the registry's latest version and then lists the hosts that are behind. On the Plugins tab, `c` checks, `U` updates the selected plugin and `A` all of them. A plugin with a newer version says `update available: 1.0.0 → 1.1.0`. | Came from | Updated by | | --- | --- | -| A registry | `berth plugins update` | +| A registry | `berth plugin update` | | A `.tgz`, an `https://` address or a folder | Adding the new one again, which replaces it | | A folder copied by hand | Copying the new one over it | | berth itself | `berth update`, or a newer server image | -A host keeps what it was applied with until it is applied again. After an update, `berth plugins update --apply` or `r` on the Plugins tab applies the hosts that are behind ([Updates](/docs/use/updates#bringing-hosts-up-to-date)). +A host keeps what it was applied with until it is applied again. After an update, `berth rollout` or `r` on the Plugins tab applies the hosts that are behind ([Updates](/docs/use/updates#bringing-hosts-up-to-date)). ### On a berth server -With a [berth server](/docs/use/server) set, the Plugins tab and `berth plugins` show the server's plugins and change those, which needs the role `admin` there. The server installs a package name or an `https://` address with its own npm and npm configuration. A `.tgz` or a folder on your machine is packed and sent to the server. The page in a browser has the same tab, and `u` there sends a `.tgz` from the device. +With a [berth server](/docs/use/server) set, the Plugins tab and `berth plugin list` show the server's plugins and change those, which needs the role `admin` there. The server installs a package name or an `https://` address with its own npm and npm configuration. `berth plugin add` packs a `.tgz` or a folder on your machine and sends it to the server, and `u` on the tab sends a `.tgz` chosen in the file dialog. -The server's hosts can have a new plugin's features as soon as it is installed. Its tasks run on the server when a host is applied, so installing is part of set-up. A server with `serve_setup off` takes it only on its own socket ([Closing set-up](/docs/extend/serve#closing-set-up)). +The server's hosts can have a new plugin's features as soon as it is installed. Its Ansible tasks run on the server when a host is applied, so installing is part of set-up. A server with `serve_setup off` takes it only on its own socket ([Closing set-up](/docs/extend/serve#closing-set-up)). A plugin of yours with the same id as a built-in one replaces it. berth reads a plugin's `package.json` and hands its tasks to Ansible. It never runs the package's npm scripts. diff --git a/docs/content/docs/(guide)/extend/serve.mdx b/docs/content/docs/(guide)/extend/serve.mdx index 459feb2..f0ea353 100644 --- a/docs/content/docs/(guide)/extend/serve.mdx +++ b/docs/content/docs/(guide)/extend/serve.mdx @@ -16,42 +16,43 @@ It prints where it listens and how many hosts it knows. Only one runs per user. | Operations | For | | --- | --- | -| [`hosts.*`](/docs/api/serve/hosts) | The saved hosts and their profiles, and whether each is behind. Each says its [level](/docs/use/levels), the level it was seen to have, and whether a key rotation has yet to reach it. `hosts.save` takes a host below a workstation with only `level` and `access`. `hosts.actions` lists what can be done to a host and `hosts.act` runs one of those [actions](#actions) | +| [`hosts.*`](/docs/api/serve/hosts) | The saved hosts and their profiles, and whether each is behind. Each says its [type](/docs/use/types), the type it was seen to be, and whether a key rotation has yet to reach it. `hosts.save` takes a host that is not a workstation with only `type` and `access`. `hosts.actions` lists what can be done to a host and `hosts.act` runs one of those [actions](#actions) | | [`actions.*`](/docs/api/serve/actions) | The actions of the installation itself, such as `rollout`, which applies the hosts that are behind one after another | | [`secrets.*`](/docs/api/serve/secrets), [`registries.*`](/docs/api/serve/registries), [`settings.*`](/docs/api/serve/settings) | What the server keeps beside the hosts. A secret's value goes in and never comes back out | | [`plugins.*`](/docs/api/serve/plugins) | The server's plugins. One is installed from its npm registry, from an `https://` address, or from a `.tgz` that is sent, and one of its own is removed. `plugins.check` asks the registries for newer versions and `plugins.update` installs them. `setup.catalogue` lists the new features straight away | | [`provisioners.*`](/docs/api/serve/provisioners) | The provisioners of the plugins with their pages and actions. A page lists what berth made with the provisioner, by kind. An action of a provisioner has its own role ([Provisioners](/docs/deploy/provisioners#with-a-berth-server)) | | [`resources.*`](/docs/api/serve/resources) | What provisioners made for berth, such as a host's machine or a deployment. Creating, changing and removing one are jobs of the subject `resource:ID` and need the role `admin`. Its log needs `operate`, and its actions have their own roles ([Resources](/docs/deploy/resources#with-a-berth-server)) | -| Every [berthd operation](/docs/api/host) | With a host: `"host": "NAME"` on the portal, `/v1/hosts/NAME/…` over HTTP. Only the `secrets` task is refused; the host's `secrets` action sends the server's own | -| [`jobs.*`](/docs/api/serve/jobs) | Jobs, one for each run of an action that runs here or follows a task, with their output. Each says its subject and its action. A roll-out's job has a report, host by host | +| Every [berthd operation](/docs/api/host) | With a host: `"host": "NAME"` on the portal, `/v1/hosts/NAME/controller/…` over HTTP. Only the controller's own `secrets` action is refused; the server's `secrets` action of the host sends the server's own | +| [`jobs.*`](/docs/api/serve/jobs) | Jobs, one for each run of an action that runs here or follows one of a host's controller, with their output. Each says its subject and its action. A roll-out's job has a report, host by host | | [`inbox.*`](/docs/api/serve/inbox) | Messages the hosts published on their channels and kept, taken in while `berth serve` runs; each can be marked done | | [`kv.*`](/docs/api/serve/kv) | A key-value store with revisions, for user interfaces to keep state in | -| [`events.follow`](/docs/api/serve/events) | One stream of channel messages, jobs and key-value changes, at `/v1/events` | +| [`events.follow`](/docs/api/serve/events) | One stream of channel messages, jobs, key-value changes and changes to hosts and resources, at `/v1/events` | All of it is also on the [portal](/docs/api/portal), the one WebSocket at `/v1/portal` for calls and streams together. ## Actions -What berth does to a host or to the installation is an *action*. `GET /v1/hosts/NAME/actions` lists a host's, and `GET /v1/actions` lists those of the installation. A form can be built from an action's description. It has the arguments with their defaults and choices, a question to confirm, the least role, and whether it can run now. +What berth does to a host or to the installation is an *action*. `GET /v1/hosts/NAME/actions` lists a host's, and `GET /v1/installation/actions` lists those of the installation. A host's controller has actions of its own, which are among the host's here. Its own list, `actions.list`, is at `/v1/hosts/NAME/controller/actions`. A form can be built from an action's description. It has the arguments with their defaults and choices, a question to confirm, the least role, and whether it can run now. | Action | Of | Runs | Answers | | --- | --- | --- | --- | -| A task of the host, such as `status` | The host | On the host, as a session | `202` with the session and a job that copies its output. A task that needs a terminal answers `200` with the session alone, to attach to | +| An action of the host's controller, such as `status` | The host | On the host, as a session | `202` with the session and a job that copies its output. One that needs a terminal answers `200` with the session alone, to attach to | | `apply` | The host | Here, as a job | `202` with the job | -| `install-controller` | The host | Here, as a job | `202` with the job. It installs berthd and box alone on a host below a workstation | +| `install-controller` | The host | Here, as a job | `202` with the job. It installs berthd and box alone on an `ssh` or `managed` host | | `seed` | The host | Here, at once | `200` with the seed's `user_data` as the result | | `secrets` | The host | On the host, with the server's secrets | `202` with the session and its job | | `rename` | The host | Here, as a job | `202` with the job. It gives the host the name in its `name` argument, with what the server keeps of it ([Renaming a host](/docs/use/moving#renaming-a-host)) | +| `upgrade-server` | A node with a berth server's container | On the node, as a session | `202` with the session and its job. It runs the server from the image of `version`, this server's own unless given ([The berth server on a node](/docs/use/nodes#the-berth-server-on-a-node)) | | `rollout` | The installation | Here, as a job | `202` with the job; its report says how each host went | -An action that the host's level rules out is listed with `available` false, and its `why` says which action raises the level. Run anyway, it is refused with `501 not_available`. +An action that the host's type rules out is listed with `available` false, and its `why` says which action changes the type. Run anyway, it is refused with `501 not_available`. To run one, POST to its name with its arguments: ```bash curl --unix-socket "$XDG_RUNTIME_DIR/berth/serve.sock" -X POST \ -d '{"arguments": {"update": "yes"}, "token": "nightly-2026-11-13"}' \ - http://berth/v1/actions/rollout + http://berth/v1/installation/actions/rollout ``` The token is optional. The same token again for the same subject and action answers with the job the first request started, and nothing runs a second time. That lets a caller that lost the answer ask again. While an action changes a host, a second one that changes it is refused with `409` until the first has ended, and so is a second roll-out. A job is followed with `jobs.get`, `jobs.output` and the `jobs.follow` stream, and the events stream says when it starts and ends. @@ -59,24 +60,24 @@ The token is optional. The same token again for the same subject and action answ The inbox, jobs and key-value store are kept in berth's store for `retention_days` (90 by default), and are also readable without the server: ```bash -berth inbox studio --topic ask/ # what is open; --all includes what is done +berth inbox list studio --topic ask/ # what is open; --all includes what is done berth kv set ui layout '{"columns": 2}' -berth kv ls ui +berth kv list ui ``` ## The store The store is where berth keeps the host profiles, with their generated passwords and the records of the machines it created. It also keeps the registries, secrets and settings. berth's own SSH key, the server's token and setup key and the list of your own plugins are there too, with the inbox, jobs and key-value store. It is `berth.db`, a SQLite file in berth's data directory, unless the `store` setting or `BERTH_STORE` names another file or a PostgreSQL database. [The store](/docs/use/store) says how to choose it, encrypt it, move it to another database and restore a server from it. -`berth serve` serves the hosts of its own installation. It does not start on a berth that has the `server` setting, or `BERTH_SERVER`, as that berth's hosts are the other server's. Clear it with `berth config server ""`. +`berth serve` serves the hosts of its own installation. | Command | Does | | --- | --- | -| `berth profile HOST` | Prints a host's profile as JSON | -| `berth profile HOST --set FILE` | Saves the profile in FILE as the host's, `-` for standard input | -| `berth credentials HOST` | Prints the passwords berth generated for the host's accounts. `hosts.credentials` gives them to an admin of the server | +| `berth host export HOST` | Prints a host's profile as JSON | +| `berth host import HOST FILE` | Saves the profile in FILE as the host's, `-` for standard input | +| `berth host credentials HOST` | Prints the passwords berth generated for the host's accounts. `hosts.credentials` gives them to an admin of the server | -`berth store` says where the store is and whether its values are encrypted, and `berth store encrypt` seals them with a key file ([Encrypt it](/docs/use/store#encrypt-it)). +`berth server store status` says where the store is and whether its values are encrypted, and `berth server store encrypt` seals them with a key file ([Encrypt it](/docs/use/store#encrypt-it)). ## Where it listens @@ -127,10 +128,10 @@ A caller over TCP shows the server's token or an OAuth 2.0 access token, and act | Role | May | | --- | --- | | `view` | Read what the hosts and the server report, such as sessions with their output, logs, jobs and the inbox | -| `operate` | Work on the hosts with tasks, commands, terminals and files. Units, power and the key-value store are included | +| `operate` | Work on the hosts with actions, commands, terminals and files. Units, power and the key-value store are included | | `admin` | Change profiles, secrets, registries, settings and plugins, apply a profile to a host, and create or delete the machine of a host | -Each action has a role of its own, which is checked when it runs. A host's tasks need `operate`, berth's own actions `admin`. +Each action has a role of its own, which is checked when it runs. The actions of a host's controller need `operate`, berth's own actions `admin`. The server's token is `admin`. The socket doesn't ask for a token and is `admin` too. The [API reference](/docs/api/serve) lists the role of each operation, and [`access`](/docs/api/serve/general) tells a caller its own. @@ -139,8 +140,8 @@ The server's token is `admin`. The socket doesn't ask for a token and is `admin` `berth serve` can act as an OAuth 2.0 resource server. It then takes access tokens signed by an issuer you already run, such as Keycloak or Zitadel. berth doesn't keep users or sessions. The issuer decides who gets which role. ```bash title="On the server" -berth config oauth_issuer https://id.example.com/realms/main -berth config oauth_audience berth +berth config set oauth_issuer https://id.example.com/realms/main +berth config set oauth_audience berth ``` | Setting | Meaning | @@ -153,12 +154,20 @@ berth config oauth_audience berth A token is a JWT signed with one of the issuer's keys (RSA, ECDSA or Ed25519). It needs `iss` and `exp`, and the role in the claim: `berth:view`, `berth:operate` or `berth:admin`. In a claim other than `scope` the bare `view`, `operate` and `admin` count too. The highest role named is the caller's. +The roles can be written in the claim three ways: + +| Claim's value | Example | Roles read | +| --- | --- | --- | +| Text | `"openid berth:operate"` | The words, split at spaces and commas | +| List | `["berth:view", "admin"]` | Each entry | +| Object | `{"admin": {…}, "billing": {…}}` | Each key. Zitadel writes project roles this way, in `urn:zitadel:iam:org:project:roles` | + ```bash curl -H "Authorization: Bearer $ACCESS_TOKEN" https://berth.example.ts.net/v1/access # {"role":"operate","setup":true,"subject":"ada"} ``` -An access token is used like the server's token. It works as a bearer and as the WebSocket protocol. berth takes it as `--token` or `server_token`, and the page for a browser takes it too. +An access token is used like the server's token. It works as a bearer and as the WebSocket protocol. berth takes it with `--token-file` or `--token-stdin`, as `BERTH_TOKEN`, or kept by `berth login`, and the screens when they ask for it. The server says which issuer it trusts at `/.well-known/oauth-protected-resource`, without a token. @@ -184,7 +193,7 @@ docker run -d -e BERTH_OAUTH_ISSUER=https://id.example.com/realms/main \ By default an `admin` can set the server up over the network. To keep that on the server itself: ```bash title="On the server" -berth config serve_setup off +berth config set serve_setup off ``` A caller over TCP is then `operate` at most, and the socket stays `admin`. Some things are refused from a caller of any role: @@ -196,7 +205,7 @@ A caller over TCP is then `operate` at most, and the socket stays `admin`. Some ## In a container -The `ghcr.io/helightdev/berth` image runs `berth serve --listen 0.0.0.0:7777 --expose --web` by default, as user `berth` (uid 1000): +The `ghcr.io/helightdev/berth` image runs `berth serve --listen 0.0.0.0:7777 --expose` by default, as user `berth` (uid 1000): ```bash docker run -d --name berth -p 127.0.0.1:7777:7777 \ @@ -217,9 +226,9 @@ The image contains ansible-core, the OpenSSH client and Python, so it can also a ## The API documents ```bash -berth operations # every operation, for the portal and HTTP -berth openapi # OpenAPI 3 for the HTTP routes -berth asyncapi # AsyncAPI 3 for the streams +berth server api operations # every operation, for the portal and HTTP +berth server api openapi # OpenAPI 3 for the HTTP routes +berth server api asyncapi # AsyncAPI 3 for the streams ``` The [API reference](/docs/api) is generated from them. diff --git a/docs/content/docs/(guide)/extend/services.mdx b/docs/content/docs/(guide)/extend/services.mdx index 7c99256..5f80c2a 100644 --- a/docs/content/docs/(guide)/extend/services.mdx +++ b/docs/content/docs/(guide)/extend/services.mdx @@ -6,7 +6,7 @@ icon: Workflow A service is a program on a host that runs all the time and answers calls. The plugin declares it, berth runs it, and `berthd` routes each call to it. The -caller is the agent on the host, with `box call`, or you, from your machine, with `berth call`. +caller is the agent on the host, with `box call`, or you, from your machine, with `berth service call`. ## Declare it @@ -177,15 +177,15 @@ On your machine, a person watches the topic and answers. The `id` is the one in the question: ```sh -berth channel studio ask/ +berth channel listen studio ask/ # {"id":4812,"topic":"ask/questions","body":{"id":3107,"question":"Which branch do I release from?"}, ...} -berth call studio ask/desk answer '{"id":3107,"text":"main"}' +berth service call studio ask/desk answer '{"id":3107,"text":"main"}' ``` The agent's `box call` then prints `{"answer":"main"}` and exits with status 0. On the host, a script can list what is waiting: `box listen ask/` -prints each question as a line of JSON, and `berth inbox studio --topic ask/` lists +prints each question as a line of JSON, and `berth inbox list studio --topic ask/` lists the ones `berth serve` has taken in from the host. If nobody answers within 590 seconds, the handler exits with an error and the diff --git a/docs/content/docs/(guide)/extend/streams.mdx b/docs/content/docs/(guide)/extend/streams.mdx index 13f0178..631d4b3 100644 --- a/docs/content/docs/(guide)/extend/streams.mdx +++ b/docs/content/docs/(guide)/extend/streams.mdx @@ -13,8 +13,8 @@ output is at `/v1/sessions/{id}`, the channel at `/v1/channel`, and a service's serving connection at `/v1/services/{service}/serve`. Open it with a `GET` and the usual upgrade headers, on the same unix socket as the plain requests. -`berth serve` passes a host's streams through under `/v1/hosts/{name}`, so the -session above is at `/v1/hosts/{name}/sessions/{id}` there, with the same +`berth serve` passes a host's streams through under `/v1/hosts/{name}/controller`, +so the session above is at `/v1/hosts/{name}/controller/sessions/{id}` there, with the same frames. Over TCP the token from `serve.token` goes in an `Authorization: Bearer` header. A browser cannot set a header on a WebSocket, so the token is also accepted as the `Sec-WebSocket-Protocol` value. @@ -61,6 +61,80 @@ Text frames are the same for both kinds: Any number of viewers can be connected to one session. A terminal has the size that the last viewer to resize it asked for. +## The stream of a port + +[`ports.connect`](/docs/api/host/ports) is a TCP connection to a port on the +host's loopback, `127.0.0.1`. It is at `/v1/ports/{port}`, and through +`berth serve` at `/v1/hosts/{name}/controller/ports/{port}`. berthd connects to the port +before the handshake, so a port that nothing listens on gives the error `503 +not_served`. It needs the role `operate` and the host +level. + +Binary frames are the connection's bytes in both directions. Either side ends +its direction with a text frame: + +| Frame | From | Meaning | +|---|---|---| +| `{"eof":true}` | client | The client sends no more. The program on the port reads the end, and what it sends still comes back | +| `{"eof":true}` | server | The program closed its side. Bytes sent towards it still arrive | + +The stream ends once both sides have sent `{"eof":true}`. It ends at once when +the client closes the WebSocket or the connection to the port breaks. A stream +that closes before its `{"eof":true}` has lost its connection. +`berth_api::tunnel` reads and writes such a stream like a socket. + +## The stream of an address on a host + +[`endpoints.connect`](/docs/api/host/endpoints) of berthd is a TCP connection +to an address on one of the host's own networks, such as a container on a +bridge. It is at `/v1/endpoints/{address}/{port}`, and its frames are those of +the stream of a port. Only `berth serve` opens it, for an endpoint of a +resource on a node. It does not pass it through under +`/v1/hosts/{name}/controller`, and a portal refuses it. + +berthd resolves the address once, checks every address it has and connects to +the one it checked, all before the handshake: + +| Error | When | +|---|---| +| `403 forbidden` | The address is a loopback, link-local or metadata address, or on none of the host's networks | +| `502 unreachable`, `504 timeout` | The name does not resolve, or not in time | +| `503 not_served`, `504 timeout` | The address refuses the connection, or doesn't answer in time | + +A network of the host is the subnet of an address of one of its interfaces +that are up, without the loopback. A prefix shorter than `/8` for IPv4 or +`/16` for IPv6 doesn't count. + +## The stream of an endpoint + +[`resources.connect`](/docs/api/serve/resources) of `berth serve` is a TCP +connection to an endpoint of a resource, one that its provisioner reported +([Endpoints](/docs/deploy/resources#endpoints)). It is at +`/v1/resources/{id}/endpoints/{endpoint}`, where `endpoint` is the endpoint's +name or its port. Its frames are those of the stream of a port. + +The server reads the endpoints from the resource's record, then resolves the +address, checks it and connects, all before the handshake. It needs the role +`operate`. What it can refuse: + +| Error | When | +|---|---| +| `404 not_found` | No such resource, or its provisioner did not report that endpoint | +| `400 bad_request` | The endpoint is UDP | +| `403 forbidden` | The address is one berth does not dial from the server | +| `503 not_served`, `504 timeout` | The endpoint refuses the connection, or doesn't answer in time | +| `502 unreachable` | The provisioner is on a node that cannot be reached | +| `501 not_available` | The provisioner is on a node whose controller is older than `endpoints.connect` | + +Who dials the endpoint: + +| Where the provisioner is | Who dials | +|---|---| +| On a node | The node's controller: `ports.connect` for `127.0.0.1`, `endpoints.connect` for any other address | +| Anywhere else | The server itself | + +`Portal::connect_endpoint` and `berth_api::tunnel::endpoint_tunnel` open one. + ## Resuming A stream that can be resumed takes `since=` and starts there. The page of each @@ -104,6 +178,11 @@ A client that cannot keep up does not slow the others down: `GET /v1/sessions/{id}/output?since=OFFSET`, or reconnect with `since=`. - On the channel, a listener that falls 1,000 messages behind is closed. Open it again with `since` set to the last id it saw. +- On `events.follow`, `inbox.follow` and `kv.follow` of `berth serve`, a + follower that is too slow gets `{"dropped":{"events":12}}` and the stream + goes on. The number counts all the server's events it missed, also those it + does not follow. Read the current state again with `inbox.list`, `kv.list` or + `jobs.list`. ## Events on your machine @@ -116,19 +195,39 @@ GET /v1/events The first frame the client sends says what to follow. It is sent once, first: ```json -{ "hosts": ["studio"], "jobs": true, "kv": ["ui"], "topics": ["myplugin/"] } +{ "hosts": ["studio"], "jobs": true, "kv": ["ui"], "topics": ["myplugin/"], "changes": ["hosts", "resources"] } ``` -`hosts` and `topics` select channel messages by host and by topic prefix. `jobs` -turns on job starts and ends. `kv` lists the key-value namespaces to watch. +| Field | Follows | +| --- | --- | +| `hosts`, `topics` | Channel messages, by host and by topic prefix. All hosts when `hosts` is empty | +| `jobs` | Job starts and ends | +| `kv` | Changes in these key-value namespaces | +| `changes` | Changes to the saved `hosts` and to `resources` | -Each frame the server sends names where it came from in `source`, which is -`channel`, `job` or `kv`: +Each frame the server sends names where it came from in `source`: ```json { "source": "kv", "namespace": "ui", "key": "layout", "revision": 4, "value": {} } ``` -A channel frame also has the `host` and the `message`, and a job frame has the -`job`. Events are not resumable. After a reconnect, read what was missed with -`inbox.list` and `kv.list`, then follow again. `inbox.follow` and `kv.follow` follow one of them alone. +| `source` | The frame also has | +| --- | --- | +| `channel` | The `host` and the `message` | +| `job` | The `job` | +| `kv` | The `namespace`, the `key`, and the `value` and `revision`, or `removed` | +| `host` | The `host` and the `change`: `added`, `removed`, `renamed` or `changed`. A rename has the old name in `from` | +| `resource` | The `resource` id, its `provisioner` and `kind`, and the `change`: `created`, `removed` or `changed` | + +A host or resource frame only names what changed. Read the rest with `hosts.get` +or `resources.get`. A host changes when its profile is saved, as by +`hosts.save` or an action, and a resource when its record does, as when a job +creates it or its state is checked. A change that another berth makes in the +store is told within a few seconds. + +```json +{ "source": "host", "host": "dock", "change": "renamed", "from": "studio" } +``` + +Events are not resumable. After a reconnect, read what was missed with +`inbox.list`, `kv.list`, `hosts.list` and `resources.list`, then follow again. `inbox.follow` and `kv.follow` follow one of them alone. diff --git a/docs/content/docs/(guide)/extend/writing-plugins.mdx b/docs/content/docs/(guide)/extend/writing-plugins.mdx index a3d7283..a911e99 100644 --- a/docs/content/docs/(guide)/extend/writing-plugins.mdx +++ b/docs/content/docs/(guide)/extend/writing-plugins.mdx @@ -91,15 +91,18 @@ The plugin's id is the package name without its scope and without a leading `ber commands: { type: 'string[]', description: 'Commands the host must have afterwards; added to /etc/devbox/expected-commands, which the self-test checks.' }, tasks: { type: 'string', description: 'Path of an Ansible task list in the package.' }, guide: { type: 'string', description: 'Path of a guide for agents in the package.' }, - actions: { type: 'Action[]', description: "Tasks for the host's overview." }, + actions: { type: 'Action[]', description: "Actions for the host's overview." }, services: { type: 'Service[]', description: 'Programs that keep running on the host and answer calls. See Writing a service.' }, channels: { type: 'Channel[]', description: "Topics of the host's channel the feature uses, with how long messages are kept." }, + types: { type: 'string[]', default: '["workstation"]', description: 'The host types it is for: workstation, node or both. berth refuses to apply a host whose profile lists a feature that is not for its type.' }, + offers: { type: 'string[]', description: "Ids of the plugin's provisioners the feature can offer once it is applied on a node. See Offering provisioners." }, + trusts: { type: '{certificate, key}', description: "The names of two berth secrets, a client certificate and its key, that the host is to trust. berth makes them when it has neither, and the tasks get the certificate alone." }, }} /> ## Actions -An action becomes a task of the host named `PLUGIN:ACTION`. Applying the configuration writes the actions of the chosen features to the host, and they appear on the overview in their group. From the command line: `berth run HOST PLUGIN:ACTION`. +An action becomes an action of the host's controller named `PLUGIN:ACTION`. Applying the configuration writes the actions of the chosen features to the host, and they appear on the overview in their group. From the command line: `berth run HOST PLUGIN:ACTION`. + ## On each host diff --git a/docs/content/docs/(guide)/reference/box.mdx b/docs/content/docs/(guide)/reference/box.mdx index 99565be..b7588a6 100644 --- a/docs/content/docs/(guide)/reference/box.mdx +++ b/docs/content/docs/(guide)/reference/box.mdx @@ -1,6 +1,6 @@ --- title: box -description: The command on every host for the agent's side of berth, from desktop control to tasks, the channel and services. +description: The command on every host for the agent's side of berth, from desktop control to actions, the channel and services. icon: Box --- @@ -10,8 +10,8 @@ icon: Box box desktop screenshot [--output PATH|-] | click X Y [BUTTON] | type TEXT | key CHORD | scroll up|down [STEPS] | clipboard | pause | resume box mcp the desktop tools as an MCP server on stdio box exec [COMMAND...] run a command, or a shell, with the desktop's environment -box tasks the tasks this user may run -box run TASK [NAME=VALUE...] run one; output here, its exit code as box's +box actions the actions this user may run +box run ACTION [NAME=VALUE...] run one; output here, its exit code as box's box publish TOPIC [JSON] [--keep none|last|log] box listen TOPIC-PREFIX [--since ID] one JSON message per line box services what can be called, and whether it is up @@ -58,12 +58,12 @@ box exec xvfb-run -a COMMAND box exec devbox-smoke ``` -## Tasks +## Actions -`box tasks` lists the host's tasks the agent may run, those marked `agent: true`, and `box run` runs one with its plugin steps: +`box actions` lists the host's actions the agent may run, those marked `agent: true`, and `box run` runs one with its plugin steps: ```bash title="As the agent" -box tasks +box actions box run smoke box run sign-in:status ``` @@ -78,7 +78,7 @@ box listen myplugin/ box listen myplugin/ --since 120 ``` -From your machine, `berth channel HOST PREFIX` follows the same messages. +From your machine, `berth channel listen HOST PREFIX` follows the same messages. ## Services diff --git a/docs/content/docs/(guide)/reference/building.mdx b/docs/content/docs/(guide)/reference/building.mdx index 3931f55..b9aed2f 100644 --- a/docs/content/docs/(guide)/reference/building.mdx +++ b/docs/content/docs/(guide)/reference/building.mdx @@ -8,10 +8,10 @@ icon: Hammer | Folder | Holds | | --- | --- | -| `crates/berth/` | `berth`: the terminal UI, the commands and `berth serve` | -| `crates/berth-core/`, `berth-api/`, `berth-pty/`, `berth-tui/` | Shared libraries. `berth-tui` has the screens the terminal UI and the page share | +| `crates/berth/` | `berth`. On Linux the commands and `berth serve`, in `src/local/`; on every system the screens | +| `crates/berth-page/` | The screens in a terminal, which do everything through the API of a server | +| `crates/berth-core/`, `berth-api/`, `berth-pty/`, `berth-tui/` | Shared libraries. `berth-tui` has the parts of the screens with no terminal in them | | `crates/vendor/` | The vendored `tui-input` crate | -| `crates/berth-web/` | The page `berth serve --web` shows in a browser, built for WebAssembly. The docs build copies it to `/web` | | `crates/berthd/`, `crates/box/` | `berthd` and `box`, built static for hosts | | `provision/ansible/` | The devbox role | | `provision/bootstrap/` | First-boot provisioning | @@ -33,12 +33,11 @@ cargo install --locked --path crates/berth # into ~/.cargo/bin The build first makes `berthd` and `box` as static musl binaries and places them in the role's files. It then packs the provisioning source and embeds it in `berth`. It needs Python 3 for the packing. -A release build embeds release builds of `berthd`, `box` and the page. A development or test build (`cargo build`, `cargo test`) embeds quicker ones: +A release build embeds release builds of `berthd` and `box`. A development or test build (`cargo build`, `cargo test`) embeds quicker ones: | Part | Profile | Built | | --- | --- | --- | | `berthd`, `box` | `host-dev` | Optimised for size, incrementally, with debug assertions | -| The page | `web-dev` | Optimised, without the link-time optimisation of `web` | | berth itself | `dev` | Line numbers, no other debug information | For a debugger, `CARGO_PROFILE_DEV_DEBUG=true` gives berth's own crates full debug information. @@ -46,7 +45,7 @@ For a debugger, `CARGO_PROFILE_DEV_DEBUG=true` gives berth's own crates full deb While changing the role or plugins, point berth at the checkout instead of the built-in copy: ```bash -berth config source "$PWD" +berth config set source "$PWD" ``` ## Check @@ -126,53 +125,35 @@ GitHub Actions build on every push: | CI | The checks above and the end-to-end tests, both with `ci/run.sh`, and Vale. berth is built once, and the tests of each test host run in a job of their own | | Boot | The boot tests. `boot cloud` runs for changes to provisioning. `boot iso` runs once a week and when started by hand, because the installer takes longest and is used least | | Docs | The docs build, and publishing it to GitHub Pages from `main` | -| Build | The static `berth` for Linux x86_64 as artifact `berth-x86_64-linux`, the host programs as `host-programs-x86_64-linux`, `berth-remote` for each system as `berth-remote-`; the container image `ghcr.io/helightdev/berth`, tagged `edge` from `main` | +| Build | `berth` for each system as artifact `berth-`, the host programs as `host-programs-x86_64-linux`, and the container image `ghcr.io/helightdev/berth`, tagged `edge` from `main` | To release, push a tag such as `v0.3.0`, or create the release on GitHub's release page with a new tag. Build then: -1. Builds `berth`, `berthd`, `box` and the page with the version of the tag, and checks that each says it. Runs `boot cloud`. When it fails, nothing is published. -2. Publishes the release with both archives, the archives of `berth-remote`, their checksum files, `install.sh`, `install-remote.sh`, `install.ps1` and `VERSION`. A release made on the page gets the files added. +1. Builds `berth`, `berthd` and `box` with the version of the tag, and checks that each says it. Runs `boot cloud`. When it fails, nothing is published. +2. Publishes the release with the archives of `berth`, their checksum files, `install.sh`, `install.ps1` and `VERSION`. A release made on the page gets the files added. 3. Pushes the image tagged `0.3.0`, `0.3` and `latest`. 4. Opens a pull request that sets the version on `main` in `Cargo.toml`, `Cargo.lock` and `docs/openapi/`. Merge it once its checks have passed. -You don't edit a version anywhere. A build from a checkout says the version in `Cargo.toml`, which is that of the latest release. That way `berth update` never offers an older version. `berth`, `berthd`, `box` and the page take the version from `berth_api::VERSION`. A release build sets it with `BERTH_BUILD_VERSION`, and `docker build` takes the same name as a build argument. +You don't edit a version anywhere. A build from a checkout says the version in `Cargo.toml`, which is that of the latest release. That way `berth update` never offers an older version. `berth`, `berthd` and `box` take the version from `berth_api::VERSION`. A release build sets it with `BERTH_BUILD_VERSION`, and `docker build` takes the same name as a build argument. A tag with a suffix, such as `v0.4.0-rc.1`, makes a pre-release. Its programs say `0.4.0-rc.1` and its image is tagged only with that. `main` stays as it is. The install scripts and `berth update` pass over pre-releases, and `BERTH_VERSION=v0.4.0-rc.1` installs one. -## With the page for a browser +## On every system -berth comes in two builds. The usual one has no page. The `web` feature adds the page `berth serve --web` shows, built from `crates/berth-web/` as WebAssembly and carried in the program (about 2 MB more): +`berth` builds on Linux, macOS and Windows. Its installation, `berth serve` and the commands are compiled for Linux alone, in `crates/berth/src/local/`. On macOS and Windows it is the screens, `berth login` and `berth update`, a client of a server elsewhere. The program builds on the system it is for: ```bash -rustup target add wasm32-unknown-unknown # once -cargo build --release -p berth --features web -``` - -| Build | Release archive | Install script | -| --- | --- | --- | -| Usual | `berth-x86_64-linux.tar.gz` | the default | -| With the page | `berth-web-x86_64-linux.tar.gz` | `BERTH_VARIANT=web` | - -`berth update` stays with the build it is. The container image is the one with the page. The docs build takes the page from `target/web-dist` for `/web`. It builds the page with the `web` profile, as a release does, by setting `BERTH_WEB_PROFILE=web`. - -## berth-remote - -`berth-remote` is the client for a [berth server](/docs/use/server#with-berth-remote) in a terminal. The screens of the page are in `crates/berth-page/`, which `crates/berth-web/` builds for a browser and `crates/berth-remote/` for a terminal. Each program gives the screens its own `Device`, from `crates/berth-page/src/device.rs`. It covers the clipboard, links, files, what is kept and the portal's socket. - -It builds on the system it is for: - -```bash -cargo build --release --locked -p berth-remote +cargo build --release --locked -p berth ``` A release builds it on a runner of each system and checks that it says its version: | System | Target | Archive | | --- | --- | --- | -| Windows x86_64 | `x86_64-pc-windows-msvc`, with the C runtime linked in | `berth-remote-x86_64-windows.zip` | -| macOS on Apple silicon | `aarch64-apple-darwin` | `berth-remote-aarch64-macos.tar.gz` | -| macOS on Intel | `x86_64-apple-darwin` | `berth-remote-x86_64-macos.tar.gz` | -| Linux x86_64 | `x86_64-unknown-linux-musl` | `berth-remote-x86_64-linux.tar.gz` | -| Linux arm64 | `aarch64-unknown-linux-musl` | `berth-remote-aarch64-linux.tar.gz` | +| Linux x86_64 | `x86_64-unknown-linux-musl`, with berthd and box | `berth-x86_64-linux.tar.gz` | +| Linux arm64 | `aarch64-unknown-linux-musl`, with berthd and box for x86_64 hosts | `berth-aarch64-linux.tar.gz` | +| macOS on Apple silicon | `aarch64-apple-darwin` | `berth-aarch64-macos.tar.gz` | +| macOS on Intel | `x86_64-apple-darwin` | `berth-x86_64-macos.tar.gz` | +| Windows x86_64 | `x86_64-pc-windows-msvc`, with the C runtime linked in | `berth-x86_64-windows.zip` | On Windows, the build needs Rust from rustup and the Visual Studio build tools. diff --git a/docs/content/docs/(guide)/reference/cli.mdx b/docs/content/docs/(guide)/reference/cli.mdx new file mode 100644 index 0000000..5bbbb7f --- /dev/null +++ b/docs/content/docs/(guide)/reference/cli.mdx @@ -0,0 +1,1530 @@ +--- +title: Command reference +description: Every command of berth with its arguments, made from the same definition as berth --help. +icon: SquareTerminal +--- + +This page is generated from the commands themselves. [The command line](/docs/use/cli) explains them by what they do. + +## Flags of every command + +| Flag | What it does | +| --- | --- | +| `--server URL` | Use this berth server for this command: https://HOST, http://HOST:PORT or unix:/path. berth login keeps one. | +| `--token-file FILE` | Read the server's token from this file. | +| `--token-stdin` | Read the server's token from standard input. | +| `-o, --output OUTPUT` | What is printed on standard output: a table for people, or the API's own JSON. One of `table`, `json`. By default `table`. | +| `--no-input` | Never ask. What is missing is an error that names the flag to pass. | +| `-y, --yes` | Answer yes to a confirmation. | +| `-q, --quiet` | Print only the result. | +| `--sudo-password-stdin` | Read the SSH user's sudo password from standard input, for a host whose sudo asks for one. | +| `--color WHEN` | Colours: on a terminal, always or never. NO_COLOR turns them off as well. One of `auto`, `always`, `never`. By default `auto`. | + +Flags go before or after the names, as `--tail 50` or `--tail=50`. `--` ends the flags; what follows it is passed on as it is. + +## Exit statuses + +| Status | Means | +| --- | --- | +| 0 | done | +| 1 | the work failed: a job failed, or a host did not answer | +| 2 | the command line was wrong: an unknown word, a missing value, a bad flag | +| 3 | refused: the role is too low, the host or this server cannot do it now, or a confirmation was declined | +| 4 | the server could not be reached | + +A command run on a host, such as `berth exec`, ends with that command's own status. + +## Environment + +| Variable | What it does | +| --- | --- | +| `BERTH_SERVER` | The server, when no --server is given. It wins over the one berth login kept. | +| `BERTH_TOKEN` | Its token, when no --token-file or --token-stdin is given. For containers and CI. | +| `NO_COLOR` | No colours, as --color never. | +| `BERTH_ and a setting's name` | A setting of the server, in capitals, such as BERTH_RETENTION_DAYS, for a server in a container. | +| `XDG_CONFIG_HOME, XDG_DATA_HOME, XDG_RUNTIME_DIR` | Where berth keeps its settings, its data and its socket. | +| `VISUAL, EDITOR` | The editor of berth host edit. | + +## berth ls + +Short for berth host list. + +```text +berth ls +``` + +## berth apply + +Short for berth host apply. + +```text +berth apply NAME [--detach] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `--detach` | Print the job's id and return; berth job wait ID waits for it. Needs a berth serve. | + +## berth exec + +Run a command on a host and pass its exit status through. + +```text +berth exec HOST [--user USER] [--detach] [--keep] -- COMMAND… +``` + +| Argument | What it is | +| --- | --- | +| `HOST` | A host's name. | +| `--user USER` | The account it runs as: agent, operator or root. By default `agent`. | +| `--detach` | Start it and return; berth session logs HOST ID shows what it writes. | +| `--keep` | Keep the session once it has ended, for berth session logs. | +| `COMMAND…` | The command and its arguments, after --. | + +## berth ssh + +A shell on a host over plain SSH, or a command run there. Needs the installation on this machine. + +```text +berth ssh HOST [--agent] [-- COMMAND…] +``` + +| Argument | What it is | +| --- | --- | +| `HOST` | A host's name. | +| `--agent` | Log in as the agent account. | +| `COMMAND…` | A command to run there, after --. | + +## berth run + +Run one of a host's actions: berth action list HOST names them. + +```text +berth run HOST ACTION [NAME=VALUE…] [--detach] +``` + +| Argument | What it is | +| --- | --- | +| `HOST` | A host's name. | +| `ACTION` | The action's name. | +| `NAME=VALUE…` | The action's arguments. | +| `--detach` | Print the job's id and return; berth job wait ID waits for it. Needs a berth serve. | + +## berth ui + +Open berth's screens: the home screen, or the page of one host. + +```text +berth ui [HOST] +``` + +| Argument | What it is | +| --- | --- | +| `HOST` | The host to open. | + +## berth host + +The hosts: list, create, add, change, install and remove them. + +### berth host list + +The hosts, with their address and whether each answers. + +`berth host ls` is the same. + +```text +berth host list +``` + +### berth host show + +One host: its type, address, state and actions. + +```text +berth host show NAME +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | + +### berth host create + +Describe a new host, then install it. Asks what is missing on a terminal. + +```text +berth host create NAME [--method METHOD] [--from PROFILE] [--type TYPE] [--feature FEATURE] [--no-feature FEATURE] [--set KEY=VALUE] [--generate-passwords] [--provisioner PROVISIONER] [NAME=VALUE…] [--run] [--detach] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `--method METHOD` | How it is installed: provisioner, iso, cloud-seed, ssh or ssh-only. | +| `--from PROFILE` | Start from a profile: a host's name, a JSON file, or - for standard input. | +| `--type TYPE` | What berth does with it: workstation, or node for a server of machines, applied over SSH. | +| `--feature FEATURE` | Switch on a plugin's feature, PLUGIN/FEATURE. Again for more. | +| `--no-feature FEATURE` | Switch off a plugin's feature. | +| `--set KEY=VALUE` | Set a field of the profile. Again for more; the keys are listed below. | +| `--generate-passwords` | Generate the accounts' passwords; berth host credentials NAME shows them. | +| `--provisioner PROVISIONER` | Make the machine with this provisioner, which also sets it up. | +| `NAME=VALUE…` | The provisioner's arguments, such as cpus=2. | +| `--run` | Install it once it is saved: apply, render the seed or build the installer. | +| `--detach` | Print the job's id and return; berth job wait ID waits for it. Needs a berth serve. | + +The ways to install, for `--method`: + +| Method | What it is | +| --- | --- | +| `provisioner` | From a provisioner | +| `iso` | USB installer | +| `cloud-seed` | Cloud-init seed | +| `ssh` | Existing host over SSH | +| `ssh-only` | Existing host, SSH only | + +The fields of a profile, for `--set KEY=VALUE` here and `berth host set`. A switch takes `on` or `off`. `feature.PLUGIN/FEATURE=on`, `secret.NAME=VARIABLE` and `registry.NAME=on` give a host those as well. + +| Key | Field | +| --- | --- | +| `type` | Type | +| `target` | Target | +| `address` | SSH address | +| `user` | SSH user | +| `port` | SSH port | +| `auth` | Log in with | +| `sudo` | Ask for sudo password | +| `hostname` | Hostname | +| `install_tailscale` | Tailscale | +| `tools_web` | JavaScript runtimes | +| `tools_native` | Native development | +| `tools_jvm` | JVM development | +| `tools_dotnet` | .NET development | +| `tools_scripting` | Scripting languages | +| `tools_scientific` | Scientific computing | +| `tools_chrome` | Google Chrome | +| `install_playwright` | Playwright | +| `software_rendering` | Software graphics | +| `install_cockpit` | Cockpit web console | +| `install_docker` | Docker on host | +| `operator_username` | Operator username | +| `operator_ssh_public_keys` | SSH key | +| `operator_password_hash` | Operator password hash | +| `agent_username` | Agent username | +| `agent_password_hash` | Agent password hash | +| `agent_ssh_public_keys` | SSH key | +| `agent_privileges` | Agent privileges | +| `agent_passwordless_sudo` | Agent passwordless sudo | +| `harden_ssh` | SSH hardening | +| `bootstrap_ssh_cidr` | Recovery SSH CIDR | +| `geometry` | Desktop size | +| `target_disk_mode` | Disk selection | +| `target_disk_serial` | Target disk serial | +| `target_disk_model` | Disk model pattern | +| `target_disk_path` | Disk path pattern | +| `target_disk_ssd` | Disk type filter | +| `keyboard` | Keyboard layout | +| `wifi_enabled` | Wi-Fi | +| `wifi_ssid` | Wi-Fi SSID | +| `wifi_password` | Wi-Fi password | +| `wifi_interface` | Wi-Fi interface | +| `release_store` | Publish release to | +| `release_url` | Release HTTPS URL | +| `release_private` | Download with login | +| `iso` | Source ISO | +| `iso_sha256` | ISO SHA256 | +| `usb_device` | Then write to USB drive | +| `inventory` | Ansible inventory | + +### berth host add + +Take in a machine that exists, by its address, and log in to it once. + +```text +berth host add NAME ADDRESS [--user USER] [--port PORT] [--auth AUTH] [--type TYPE] [--feature FEATURE] [--no-contact] [--password-stdin] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `ADDRESS` | Its name or IP address, or a Host alias of the server's SSH configuration. | +| `--user USER` | The account berth logs in as. | +| `--port PORT` | Its SSH port, when not 22. | +| `--auth AUTH` | How berth logs in: berth, key:NAME, password or ssh. | +| `--type TYPE` | What berth does with it: ssh, managed, node or workstation. By default `ssh`. | +| `--feature FEATURE` | Switch on a plugin's feature of a node or a workstation, PLUGIN/FEATURE, such as incus/server. Again for more. | +| `--no-contact` | Save it without logging in to it. | +| `--password-stdin` | Read a password from standard input that installs berth's key on it, once. | + +### berth host set + +Change settings of a host's profile: KEY=VALUE, as berth host create --help lists the keys. + +```text +berth host set NAME KEY=VALUE… +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `KEY=VALUE…` | KEY=VALUE, where KEY is a field (berth host create --help), feature.PLUGIN/FEATURE, secret.NAME or registry.NAME. | + +### berth host edit + +Change a host's profile in $EDITOR. + +```text +berth host edit NAME +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | + +### berth host export + +A host's profile as JSON on standard output. + +```text +berth host export NAME +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | + +### berth host import + +Save the profile in FILE (- for standard input) as the host's, new or changed. + +```text +berth host import NAME FILE +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `FILE` | The profile as JSON, such as berth host export wrote; - for standard input. | + +### berth host apply + +Apply the host's profile to it over SSH, or install its controller alone when it is managed. + +```text +berth host apply NAME [--detach] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `--detach` | Print the job's id and return; berth job wait ID waits for it. Needs a berth serve. | + +### berth host seed + +The cloud-init user data that installs the host on first boot, on standard output. + +```text +berth host seed NAME [--file FILE] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `--file FILE` | Write it to FILE instead. | + +### berth host iso + +Build the host's unattended USB installer where the server runs, and write it to a drive. + +```text +berth host iso NAME [--write DEVICE] [--password-stdin] [--detach] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `--write DEVICE` | Write it to this USB drive, which is erased, such as /dev/sdb. | +| `--password-stdin` | Read your sudo password for writing the drive from standard input. | +| `--detach` | Print the job's id and return; berth job wait ID waits for it. Needs a berth serve. | + +### berth host rename + +Give a host another name in berth. The machine is left as it is. + +```text +berth host rename NAME NEW [--hostname] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `NEW` | Its new name. | +| `--hostname` | Make NEW its profile's hostname as well, for the next apply. | + +### berth host remove + +Forget a host. The machine is left as it is, unless --destroy. + +`berth host rm` is the same. + +```text +berth host remove NAME [--destroy] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `--destroy` | Delete the machine a provisioner made for it as well. | + +### berth host adopt + +Take in a host berth set up before, from the profile it keeps. + +```text +berth host adopt NAME [ADDRESS] [--port PORT] [--key FILE] [--force] [--detach] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `ADDRESS` | [USER@]ADDRESS; without it, the saved host of that name. | +| `--port PORT` | Its SSH port, when not 22. | +| `--key FILE` | A private key file where the server runs, offered as well for reading the profile only. | +| `--force` | Replace a host of that name. | +| `--detach` | Print the job's id and return; berth job wait ID waits for it. Needs a berth serve. | + +### berth host credentials + +The passwords berth generated for a host's accounts. + +```text +berth host credentials NAME [--show] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `--show` | Print them even when standard output is no terminal. | + +### berth host verify + +Check the host's SSH host key, and accept a new one. + +```text +berth host verify NAME [--accept] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `--accept` | Accept the key the host shows now. | + +### berth host start + +Start the machine a provisioner made for the host. + +```text +berth host start NAME [--detach] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `--detach` | Print the job's id and return; berth job wait ID waits for it. Needs a berth serve. | + +### berth host stop + +Stop the machine a provisioner made for the host. + +```text +berth host stop NAME [--detach] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `--detach` | Print the job's id and return; berth job wait ID waits for it. Needs a berth serve. | + +### berth host inventory + +The hosts as an Ansible inventory. Needs the installation on this machine. + +```text +berth host inventory [NAMES…] +``` + +| Argument | What it is | +| --- | --- | +| `NAMES…` | The hosts; all with an address when none is named. | + +## berth rollout + +Bring the hosts up to date, one at a time. + +```text +berth rollout [--only HOST] [--except HOST] [--all] [--update] [--reboot] [--force] [--detach] +``` + +| Argument | What it is | +| --- | --- | +| `--only HOST` | Only these hosts. Again for more. | +| `--except HOST` | Leave these hosts out. Again for more. | +| `--all` | Take hosts that are up to date as well. | +| `--update` | Run each host's update action as well. | +| `--reboot` | Reboot the hosts that need it, unless sessions run there. | +| `--force` | With --reboot: even when sessions run. | +| `--detach` | Print the job's id and return; berth job wait ID waits for it. Needs a berth serve. | + +## berth action + +What a host can do. + +### berth action list + +What a host can do, or the installation without a host. + +`berth action ls` is the same. + +```text +berth action list [HOST] +``` + +| Argument | What it is | +| --- | --- | +| `HOST` | A host's name. | + +## berth session + +The terminal and process sessions of a host. + +### berth session list + +A host's sessions, running and ended. + +`berth session ls` is the same. + +```text +berth session list HOST +``` + +| Argument | What it is | +| --- | --- | +| `HOST` | A host's name. | + +### berth session attach + +This terminal onto a terminal session, or onto a new shell as the agent. Ctrl+Q leaves it running. + +```text +berth session attach HOST [ID] +``` + +| Argument | What it is | +| --- | --- | +| `HOST` | A host's name. | +| `ID` | Its id. | + +### berth session logs + +What a session wrote. + +```text +berth session logs HOST ID [-f, --follow] [--tail N] +``` + +| Argument | What it is | +| --- | --- | +| `HOST` | A host's name. | +| `ID` | Its id. | +| `-f, --follow` | Go on as it writes more. | +| `--tail N` | Only the last N lines. | + +### berth session stop + +End a session. + +```text +berth session stop HOST ID +``` + +| Argument | What it is | +| --- | --- | +| `HOST` | A host's name. | +| `ID` | Its id. | + +## berth provisioner + +What can make hosts and resources, such as Incus or Docker. + +### berth provisioner list + +The provisioners, and whether each can be used here. + +`berth provisioner ls` is the same. + +```text +berth provisioner list +``` + +### berth provisioner types + +The types of provisioner the plugins bring. + +```text +berth provisioner types +``` + +### berth provisioner add + +Register a provisioner of a type, such as incus/vm, under a name of its own. + +```text +berth provisioner add NAME TYPE [NAME=VALUE…] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `TYPE` | Its type, as berth provisioner types lists them. | +| `NAME=VALUE…` | Its settings. | + +### berth provisioner show + +How it is now and what it holds. NAME=VALUE gives one of its settings another value for this once. + +```text +berth provisioner show NAME [NAME=VALUE…] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `NAME=VALUE…` | A setting with another value for this once. | + +### berth provisioner set + +Change its settings. + +```text +berth provisioner set NAME NAME=VALUE… +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `NAME=VALUE…` | A setting and its new value. | + +### berth provisioner remove + +Forget a provisioner that holds nothing. + +`berth provisioner rm` is the same. + +```text +berth provisioner remove NAME +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | + +### berth provisioner actions + +What it can do. + +```text +berth provisioner actions NAME +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | + +### berth provisioner run + +Run one of its actions. + +```text +berth provisioner run NAME ACTION [NAME=VALUE…] [--detach] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `ACTION` | The action's name. | +| `NAME=VALUE…` | The action's arguments; a required one that is missing is asked for. | +| `--detach` | Print the job's id and return; berth job wait ID waits for it. Needs a berth serve. | + +## berth resource + +What provisioners made that is not a host, such as a deployment. + +### berth resource list + +What provisioners made. + +`berth resource ls` is the same. + +```text +berth resource list [--kind KIND] [--provisioner PROVISIONER] [--host HOST] +``` + +| Argument | What it is | +| --- | --- | +| `--kind KIND` | Only those of this kind, such as deployment or machine. | +| `--provisioner PROVISIONER` | Only those this provisioner made. | +| `--host HOST` | Only the machine of this host. | + +### berth resource show + +One of them as its provisioner says it is now. + +```text +berth resource show ID +``` + +| Argument | What it is | +| --- | --- | +| `ID` | Its id. | + +### berth resource create + +Make a resource, such as a deployment from a Compose file. + +```text +berth resource create NAME --provisioner PROVISIONER [--kind KIND] [--spec FILE] [--secret NAME[:VARIABLE]] [NAME=VALUE…] [--detach] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `--provisioner PROVISIONER` | The provisioner that makes it. | +| `--kind KIND` | Its kind, where the provisioner makes several. | +| `--spec FILE` | The specification, such as a Compose file; - for standard input. | +| `--secret NAME[:VARIABLE]` | A secret it gets, NAME or NAME:VARIABLE. Again for more. | +| `NAME=VALUE…` | The arguments of its kind. | +| `--detach` | Print the job's id and return; berth job wait ID waits for it. Needs a berth serve. | + +### berth resource set + +Change a resource: other arguments, a new specification, other secrets. + +```text +berth resource set ID [--spec FILE] [--secret NAME[:VARIABLE]] [NAME=VALUE…] [--detach] +``` + +| Argument | What it is | +| --- | --- | +| `ID` | Its id. | +| `--spec FILE` | A new specification; - for standard input. | +| `--secret NAME[:VARIABLE]` | The secrets it gets from now on, NAME or NAME:VARIABLE. Again for more. | +| `NAME=VALUE…` | Arguments with new values. | +| `--detach` | Print the job's id and return; berth job wait ID waits for it. Needs a berth serve. | + +### berth resource remove + +Remove a resource with its provisioner. + +`berth resource rm` is the same. + +```text +berth resource remove ID [--detach] +``` + +| Argument | What it is | +| --- | --- | +| `ID` | Its id. | +| `--detach` | Print the job's id and return; berth job wait ID waits for it. Needs a berth serve. | + +### berth resource logs + +What it wrote. + +```text +berth resource logs ID [-f, --follow] [--tail N] +``` + +| Argument | What it is | +| --- | --- | +| `ID` | Its id. | +| `-f, --follow` | Go on as it writes more. | +| `--tail N` | Only the last N lines. | + +### berth resource exec + +Run a command in it, for a kind that has exec, and exit with its exit status. + +```text +berth resource exec ID [--stdin] -- COMMAND… [--detach] +``` + +| Argument | What it is | +| --- | --- | +| `ID` | Its id. | +| `--stdin` | Give the command what berth reads on standard input, to its end. | +| `COMMAND…` | The command and its arguments, after --. | +| `--detach` | Print the job's id and return; berth job wait ID waits for it. Needs a berth serve. | + +### berth resource actions + +What it can do. + +```text +berth resource actions ID +``` + +| Argument | What it is | +| --- | --- | +| `ID` | Its id. | + +### berth resource run + +Run one of its actions. + +```text +berth resource run ID ACTION [NAME=VALUE…] [--detach] +``` + +| Argument | What it is | +| --- | --- | +| `ID` | Its id. | +| `ACTION` | The action's name. | +| `NAME=VALUE…` | The action's arguments; a required one that is missing is asked for. | +| `--detach` | Print the job's id and return; berth job wait ID waits for it. Needs a berth serve. | + +## berth job + +The work the server does: applies, installers, roll-outs. + +### berth job list + +The jobs, newest first. + +`berth job ls` is the same. + +```text +berth job list [--limit LIMIT] +``` + +| Argument | What it is | +| --- | --- | +| `--limit LIMIT` | By default `20`. | + +### berth job show + +One job's state. + +```text +berth job show ID +``` + +| Argument | What it is | +| --- | --- | +| `ID` | Its id. | + +### berth job logs + +What a job wrote. + +```text +berth job logs ID [-f, --follow] +``` + +| Argument | What it is | +| --- | --- | +| `ID` | Its id. | +| `-f, --follow` | Go on as it writes more, until it ends. | + +### berth job wait + +Wait for a job to end and exit with its result. + +```text +berth job wait ID +``` + +| Argument | What it is | +| --- | --- | +| `ID` | Its id. | + +### berth job stop + +Stop a running job: its programs end, and its record says who stopped it. + +```text +berth job stop ID +``` + +| Argument | What it is | +| --- | --- | +| `ID` | Its id. | + +## berth key + +The SSH keys berth logs in to hosts with. + +### berth key list + +The keys of berth's key ring, by their public halves. + +`berth key ls` is the same. + +```text +berth key list +``` + +### berth key show + +One key's public half, which a host's operator takes. + +```text +berth key show [NAME] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. By default `berth`. | + +### berth key create + +Make a new ed25519 key. + +```text +berth key create NAME +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | + +### berth key import + +Add an OpenSSH private key without a passphrase from FILE (- for standard input). + +```text +berth key import NAME FILE +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `FILE` | The private key file, in OpenSSH format. | + +### berth key remove + +Remove a key no host logs in with. + +`berth key rm` is the same. + +```text +berth key remove NAME +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | + +### berth key rotate + +Replace a key on every host that takes it. + +```text +berth key rotate NAME [--detach] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `--detach` | Print the job's id and return; berth job wait ID waits for it. Needs a berth serve. | + +### berth key trust + +Install a key on a host that does not take it yet, with a password used once. + +```text +berth key trust HOST [--key NAME] [--password-stdin] +``` + +| Argument | What it is | +| --- | --- | +| `HOST` | A host's name. | +| `--key NAME` | Log in once with this key of the ring, to install the one the host logs in with. | +| `--password-stdin` | Read the password from standard input. | + +## berth secret + +Secret values the server keeps and gives to hosts. + +### berth secret list + +The names of the secrets. + +`berth secret ls` is the same. + +```text +berth secret list +``` + +### berth secret set + +Set a secret's value: asked for, or read from a file or standard input. + +```text +berth secret set NAME [--value-file FILE] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `--value-file FILE` | Read the value from FILE. | + +### berth secret remove + +Remove a secret. + +`berth secret rm` is the same. + +```text +berth secret remove NAME +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | + +## berth registry + +Registries: Nexus servers, npm, file and Docker registries. + +### berth registry list + +The registries. + +`berth registry ls` is the same. + +```text +berth registry list +``` + +### berth registry add + +Add a registry, or change its address or user. + +```text +berth registry add NAME TYPE URL [--user USER] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `TYPE` | nexus, npm, raw or docker. | +| `URL` | The address. | +| `--user USER` | The user of its login; without one the login is a token. | + +### berth registry add-repo + +Add a repository of a Nexus registry. + +```text +berth registry add-repo NAME FORMAT REPOSITORY [--address ADDRESS] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `FORMAT` | npm, raw or docker. | +| `REPOSITORY` | The repository's name on the registry. | +| `--address ADDRESS` | Its own address, HOST[:PORT], for a Docker repository. | + +### berth registry login + +Keep the password or token of a registry. + +```text +berth registry login NAME [--password-stdin] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | +| `--password-stdin` | Read it from standard input. | + +### berth registry check + +Whether a registry answers and takes its login. + +```text +berth registry check [NAME] +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | + +### berth registry remove + +Remove a registry with its login. + +`berth registry rm` is the same. + +```text +berth registry remove NAME +``` + +| Argument | What it is | +| --- | --- | +| `NAME` | Its name. | + +### berth registry publish + +Upload the release archive to a raw repository or file server, and use it as HOST's release address. + +```text +berth registry publish DESTINATION [--host HOST] [--login REGISTRY] +``` + +| Argument | What it is | +| --- | --- | +| `DESTINATION` | REGISTRY or REGISTRY/REPOSITORY. | +| `--host HOST` | A host's name. | +| `--login REGISTRY` | The cloud seed carries this registry's login; the destination's without a name. | + +### berth registry push-image + +Push berth's container image. + +```text +berth registry push-image DESTINATION [TAG] +``` + +| Argument | What it is | +| --- | --- | +| `DESTINATION` | REGISTRY or REGISTRY/REPOSITORY. | +| `TAG` | By default `latest`. | + +## berth plugin + +Plugins and what they install. + +### berth plugin list + +The plugins and where they come from. + +`berth plugin ls` is the same. + +```text +berth plugin list +``` + +### berth plugin show + +What a plugin brings: its features with their actions, services and channels, and its provisioners. + +```text +berth plugin show ID +``` + +| Argument | What it is | +| --- | --- | +| `ID` | Its id. | + +### berth plugin add + +Install a plugin with npm: a name, or the address of a tarball. + +```text +berth plugin add SPEC +``` + +| Argument | What it is | +| --- | --- | +| `SPEC` | A plugin's npm name, or the address of its .tgz. | + +### berth plugin remove + +Remove one of the server's own plugins. + +`berth plugin rm` is the same. + +```text +berth plugin remove ID +``` + +| Argument | What it is | +| --- | --- | +| `ID` | Its id. | + +### berth plugin update + +Update one plugin or all to their registry's latest version. + +```text +berth plugin update [ID] [--check] +``` + +| Argument | What it is | +| --- | --- | +| `ID` | Its id. | +| `--check` | Only say which have a newer version. | + +## berth config + +The settings: the server's, and this berth's own, such as the server it uses. + +### berth config list + +The settings, with what each means and where it is kept. + +`berth config ls` is the same. + +```text +berth config list +``` + +### berth config get + +One setting's value. + +```text +berth config get KEY +``` + +| Argument | What it is | +| --- | --- | +| `KEY` | The setting or key. | + +### berth config set + +Change a setting. + +```text +berth config set KEY VALUE +``` + +| Argument | What it is | +| --- | --- | +| `KEY` | The setting or key. | +| `VALUE` | The value. | + +### berth config unset + +Set a setting back to its default. + +```text +berth config unset KEY +``` + +| Argument | What it is | +| --- | --- | +| `KEY` | The setting or key. | + +## berth port + +Reach a port of a host, or an endpoint of a resource, from this machine. + +### berth port forward + +Listen on this machine and pass each connection to a port of a host, or to an endpoint of a resource. + +TARGET is a host's name. A resource's id or name is taken when no host has that name, or always with --resource. + +```text +berth port forward TARGET [LOCAL:]PORT|ENDPOINT… [--resource] [--address ADDRESS] +``` + +| Argument | What it is | +| --- | --- | +| `TARGET` | A host's name, or a resource's id or name. | +| `[LOCAL:]PORT|ENDPOINT…` | [LOCAL:]PORT for a host's port. For a resource, [LOCAL:]ENDPOINT, an endpoint's name or port; LOCAL is the endpoint's port unless given. | +| `--resource` | TARGET is a resource, even where a host has the same name. | +| `--address ADDRESS` | The address to listen on. By default `127.0.0.1`. | + +## berth inbox + +Messages taken in from the hosts' channels. + +### berth inbox list + +The messages, oldest first. + +`berth inbox ls` is the same. + +```text +berth inbox list [HOST] [--topic PREFIX] [--all] +``` + +| Argument | What it is | +| --- | --- | +| `HOST` | A host's name. | +| `--topic PREFIX` | Only topics that start with PREFIX. | +| `--all` | Done ones as well. | + +## berth kv + +The server's key-value store. + +### berth kv list + +The keys of a namespace with their values. + +`berth kv ls` is the same. + +```text +berth kv list NAMESPACE +``` + +| Argument | What it is | +| --- | --- | +| `NAMESPACE` | The namespace. | + +### berth kv get + +One key's value. + +```text +berth kv get NAMESPACE KEY +``` + +| Argument | What it is | +| --- | --- | +| `NAMESPACE` | The namespace. | +| `KEY` | The setting or key. | + +### berth kv set + +Write a key: JSON, or else text. + +```text +berth kv set NAMESPACE KEY VALUE +``` + +| Argument | What it is | +| --- | --- | +| `NAMESPACE` | The namespace. | +| `KEY` | The setting or key. | +| `VALUE` | The value. | + +### berth kv remove + +Remove a key. + +`berth kv rm` is the same. + +```text +berth kv remove NAMESPACE KEY +``` + +| Argument | What it is | +| --- | --- | +| `NAMESPACE` | The namespace. | +| `KEY` | The setting or key. | + +## berth channel + +A host's channel. + +### berth channel listen + +Messages on a host's channel as they come, one JSON object per line. + +```text +berth channel listen HOST [--topic PREFIX] [--since ID] +``` + +| Argument | What it is | +| --- | --- | +| `HOST` | A host's name. | +| `--topic PREFIX` | Only topics that start with PREFIX. | +| `--since ID` | From after this message on. | + +## berth service + +Services that plugins run on a host. + +### berth service call + +Call a method of a service a plugin runs on the host. + +```text +berth service call HOST SERVICE METHOD [BODY] +``` + +| Argument | What it is | +| --- | --- | +| `HOST` | A host's name. | +| `SERVICE` | PLUGIN/NAME of the service. | +| `METHOD` | The service's method. | +| `BODY` | The body, as JSON. | + +## berth server + +The berth server of this machine: running it, its setup key, its store and its API documents. + +### berth server run + +Serve the API over all hosts on a socket of this user, and over HTTP with --listen. + +```text +berth server run [--listen ADDRESS:PORT] [--expose] +``` + +| Argument | What it is | +| --- | --- | +| `--listen ADDRESS:PORT` | Also listen over HTTP, behind a token. | +| `--expose` | Allow an address other than this machine's own. | + +### berth server setup-key + +Make a key, valid for a day, that lets the first user collect the server's token. + +```text +berth server setup-key [--revoke] +``` + +| Argument | What it is | +| --- | --- | +| `--revoke` | End the key that is open. | + +### berth server store + +Where the server keeps its state, and sealing it. + +#### berth server store status + +Where the store is, and whether it is sealed. + +```text +berth server store status +``` + +#### berth server store encrypt + +Seal the store's values with a new key file. + +```text +berth server store encrypt +``` + +#### berth server store rekey + +Seal them with another key. + +```text +berth server store rekey +``` + +#### berth server store decrypt + +Write them plain again. + +```text +berth server store decrypt +``` + +#### berth server store copy + +Copy the store into another SQLite file or PostgreSQL database, and switch to it. + +```text +berth server store copy [TARGET] [--target-file FILE] [--into-existing] +``` + +| Argument | What it is | +| --- | --- | +| `TARGET` | A file, a PostgreSQL URL, or - to read it from standard input. | +| `--target-file FILE` | Read the target from FILE. | +| `--into-existing` | Copy into a database that holds a store already. | + +### berth server api + +The server's API documents. + +#### berth server api openapi + +The OpenAPI document. + +```text +berth server api openapi +``` + +#### berth server api asyncapi + +The AsyncAPI document of the streams. + +```text +berth server api asyncapi +``` + +#### berth server api operations + +The operations, with their routes and roles. + +```text +berth server api operations +``` + +## berth serve + +Short for berth server run. + +```text +berth serve [--listen ADDRESS:PORT] [--expose] +``` + +| Argument | What it is | +| --- | --- | +| `--listen ADDRESS:PORT` | Also listen over HTTP, behind a token. | +| `--expose` | Allow an address other than this machine's own. | + +## berth login + +Use the berth server at URL, with a setup key made there. + +```text +berth login URL [--setup-key-stdin] +``` + +| Argument | What it is | +| --- | --- | +| `URL` | https://HOST or http://HOST:PORT. | +| `--setup-key-stdin` | Read the setup key from standard input. | + +## berth logout + +Stop using the berth server berth login kept. + +```text +berth logout +``` + +## berth update + +Install the latest release in this program's place. + +```text +berth update [--check] [--from URL] +``` + +| Argument | What it is | +| --- | --- | +| `--check` | Only say whether there is a newer release. | +| `--from URL` | Where releases are, instead of berth's own. | + +## berth version + +The version of this berth, and of the server it uses. + +```text +berth version +``` + +## berth completion + +The completions of a shell, to source from its start-up file. + +```text +berth completion SHELL +``` + +| Argument | What it is | +| --- | --- | +| `SHELL` | bash, zsh, fish, elvish or powershell. One of `bash`, `elvish`, `fish`, `powershell`, `zsh`. | diff --git a/docs/content/docs/(guide)/reference/index.mdx b/docs/content/docs/(guide)/reference/index.mdx index 1ba4875..d32fa53 100644 --- a/docs/content/docs/(guide)/reference/index.mdx +++ b/docs/content/docs/(guide)/reference/index.mdx @@ -7,7 +7,7 @@ icon: Library The tool inventory, categories, mise and versions. CPU or GPU rendering, and Xvfb. - The agent's command for the desktop, MCP, tasks, the channel and services. + The agent's command for the desktop, MCP, actions, the channel and services. Files, units, and what agents read. Every field of a host profile. The checks and the upstream references. diff --git a/docs/content/docs/(guide)/reference/meta.json b/docs/content/docs/(guide)/reference/meta.json index 39653c2..7af1de5 100644 --- a/docs/content/docs/(guide)/reference/meta.json +++ b/docs/content/docs/(guide)/reference/meta.json @@ -1,5 +1,5 @@ { "title": "Reference", "icon": "Library", - "pages": ["tools", "graphics", "box", "on-the-host", "profile", "validation", "building"] + "pages": ["cli", "tools", "graphics", "box", "on-the-host", "profile", "validation", "building"] } diff --git a/docs/content/docs/(guide)/reference/on-the-host.mdx b/docs/content/docs/(guide)/reference/on-the-host.mdx index a952c59..291226a 100644 --- a/docs/content/docs/(guide)/reference/on-the-host.mdx +++ b/docs/content/docs/(guide)/reference/on-the-host.mdx @@ -20,7 +20,7 @@ icon: FolderTree - + @@ -58,10 +58,10 @@ icon: FolderTree | `/etc/devbox/guides/` | Longer guides for the desktop, graphics, and each plugin feature with a guide | | `/etc/devbox/expected-commands` | Commands the self-test requires, including those plugins list | | `/etc/devbox/mcp.json` | The desktop MCP server, to merge into an MCP client's configuration | -| `/etc/berth/*.json` | berthd's settings, tasks, services and channel topics | +| `/etc/berth/*.json` | berthd's settings, actions, services and channel topics | | `/etc/berth/applied.json` | The versions of berth and its plugins at the last apply | | `/etc/berth/profile.json` | The host's profile, for root only, which [another berth can adopt](/docs/use/moving) | -| `/usr/local/lib/berth/tasks/` | The scripts of berth's own tasks | +| `/usr/local/lib/berth/tasks/` | The scripts of berth's own actions | | `/var/lib/devbox/provision-state` | The state of the first-boot provisioning | | `/var/lib/devbox/ready` | Written when Ansible converged, with the release's digest | | `/var/lib/devbox/toolchains.json` | The runtime versions that were installed | @@ -96,7 +96,7 @@ GUI programs, T3 Code and the commands it starts share the desktop's `DISPLAY`, ## The self-test -`devbox-smoke` is the **Self-Test** task. It checks: +`devbox-smoke` is the **Self-Test** action. It checks: - that the expected commands exist - llvmpipe on the shared desktop and in Xvfb, and lavapipe enumeration diff --git a/docs/content/docs/(guide)/reference/profile.mdx b/docs/content/docs/(guide)/reference/profile.mdx index b36819f..30c3c4d 100644 --- a/docs/content/docs/(guide)/reference/profile.mdx +++ b/docs/content/docs/(guide)/reference/profile.mdx @@ -7,20 +7,20 @@ icon: FileBraces A profile is a JSON document that describes one host. berth keeps it in its store and writes it from the set-up tabs. The fields below can also be edited by hand: write the profile to a file, change it and save it back. ```bash -berth profile studio > studio.json -berth profile studio --set studio.json +berth host export studio > studio.json +berth host import studio studio.json ``` `--set -` reads the profile from standard input. berth checks it as the terminal UI reads a profile and saves it as it is. The profile of a berth server's host is saved on the server. -A profile can also stay in a file of its own. berth reads a host given as a path, such as `berth ./studio.json`, from that file and saves it there. +A profile can also stay in a file of its own. `berth host import studio studio.json` saves one from a file, and `berth host export studio` writes it out again. Settings of the Ansible role go under `ansible_vars`, and how berth reaches and installs the host under `builder`. ```json title="studio's profile (excerpt)" { "hostname": "studio", - "profile": "laptop", + "target": "laptop", "operator_username": "ops", "operator_ssh_public_keys": ["ssh-ed25519 AAAA... you@workstation"], "agent_username": "agent", @@ -39,11 +39,8 @@ Settings of the Ansible role go under `ansible_vars`, and how berth reaches and -A host below a workstation needs only its level and `access`. The other fields are for a workstation. See [Host levels](/docs/use/levels). +An `ssh` or `managed` host needs only its type and `access`. A node also takes `target`, the Tailscale fields and `ansible_vars`, and refuses the fields of the agent and the operator ([Nodes](/docs/use/nodes)). The other fields are for a workstation. See [Host types](/docs/use/types). ```json title="dock's profile" { - "level": "reachable", + "type": "ssh", "access": { "address": "192.0.2.4", "user": "ops", "auth": "berth" } } ``` @@ -158,6 +161,19 @@ A profile of berth 0.4 kept these under `builder`: berth converts stored profiles once, and reads a profile in a file in the new layout. The file keeps the old layout until you save it. A key file that `ssh_key` named becomes a key of the key ring, and the host logs in with it. If berth cannot read the file, the host logs in with `ssh`, and berth says so. A host without a key file logs in with `ssh`, which is what it did before. +## Older profiles + +A profile of berth 0.5 or earlier names two fields differently: + +| berth 0.5 | Now | +| --- | --- | +| `level`: `reachable` | `type`: `ssh` | +| `level`: `controller` | `type`: `managed` | +| `level`: `workstation` | No `type`, which means a workstation | +| `profile` | `target` | + +berth converts its stored profiles once, the profiles of a berth server's hosts with them. It reads the old names wherever a profile comes from: a file that `berth host import HOST FILE` reads, and the profile a host keeps, which [`berth host adopt`](/docs/use/moving) reads. A file keeps the old names until you save it. berth writes `type` and `target` and nothing else, and the API of `berth serve` takes no other spelling. + ## Install (`builder`) How berth installs the host. These settings stay on your machine. @@ -177,9 +193,9 @@ berth keeps more of a host in its store, beside the profile: | What | Holds | How to see it | | --- | --- | --- | -| Generated passwords | The passwords F4 made for the host's accounts | `berth credentials HOST` | -| The machine's record | The [resource](/docs/deploy/resources) of a machine berth created for the host, with its provisioner and id | `berth instance HOST` | +| Generated passwords | The passwords F4 made for the host's accounts | `berth host credentials HOST` | +| The machine's record | The [resource](/docs/deploy/resources) of a machine berth created for the host, with its provisioner and id | `berth host show HOST` | -`berth credentials` prints an account and its password to a line, and only to a terminal unless `--show` is given. For a berth server's host it asks the server, which keeps them. `berth remove HOST` forgets the profile and both of these at once. It refuses while berth knows of a machine it created for the host. +`berth host credentials` prints an account and its password to a line, and only to a terminal unless `--show` is given. For a berth server's host it asks the server, which keeps them. `berth host remove HOST` forgets the profile and both of these at once. It refuses while berth knows of a machine it created for the host. Secrets and registries a host gets are listed in the profile by name (`secrets`, `registries`). Their values stay in berth's store. diff --git a/docs/content/docs/(guide)/reference/validation.mdx b/docs/content/docs/(guide)/reference/validation.mdx index bec893f..b7d8c26 100644 --- a/docs/content/docs/(guide)/reference/validation.mdx +++ b/docs/content/docs/(guide)/reference/validation.mdx @@ -16,7 +16,7 @@ CI runs these on every push, without network access to hosts: | `ansible-playbook --syntax-check` | The role | | `cargo fmt`, `cargo clippy` | Formatting and lints | | `vale` | The prose of the README, these docs and the plugin guides | -| `tests/e2e` | berth against a test host in a container. A profile is applied over SSH, and a second time without changes. Then the tests use the host with SSH keys, tasks, terminals, files, secrets and a plugin from a folder. `berth serve` is tested with its roles, its set-up and its page, and with a restore of the installation from its database and key file. They use a development build of berth | +| `tests/e2e` | berth against a test host in a container. A profile is applied over SSH, and a second time without changes. Then the tests use the host with SSH keys, actions, terminals, files, secrets and a plugin from a folder. `berth serve` is tested with its roles, its set-up and its page, and with a restore of the installation from its database and key file. They use a development build of berth | See [Building and releasing](/docs/reference/building#in-containers) for running them locally. @@ -26,15 +26,15 @@ The container suite can't boot anything. A cloud seed and the USB installer are | Test | What happens | | --- | --- | -| `boot cloud` | The Ubuntu 26.04 cloud image boots with the seed of `berth seed` as its NoCloud user data. It downloads the release archive over HTTPS and applies the role on its first boot | -| `boot iso` | The installer of `berth iso` boots with only a disk of another serial, and must refuse it without writing to it. It then boots with that disk and the right one, erases and installs the right one and powers off. The installed system boots without the installer and sets itself up | +| `boot cloud` | The Ubuntu 26.04 cloud image boots with the seed of `berth host seed` as its NoCloud user data. It downloads the release archive over HTTPS and applies the role on its first boot | +| `boot iso` | The installer of `berth host iso` boots with only a disk of another serial, and must refuse it without writing to it. It then boots with that disk and the right one, erases and installs the right one and powers off. The installed system boots without the installer and sets itself up | After the first boot, berth reaches the host over SSH and checks: - berthd and a plugin's service run as systemd units, and berthd starts, stops and restarts units and reads the journal - the firewall lets in SSH and nothing else, and SSH takes keys only and refuses root -- the status task ends without failed units, and a second apply in a row doesn't change anything -- `berth apply --all --reboot` refuses while a session runs, then reboots the host, waits for it and checks it +- the status action ends without failed units, and a second apply in a row doesn't change anything +- `berth rollout --all --reboot` refuses while a session runs, then reboots the host, waits for it and checks it - the end-to-end tests of commands, berthd's API and secrets The guest runs on QEMU's user network with a package cache and a release server on the container's side. The cache is in `build/boot/cache` and makes a second run a lot quicker. The profile leaves out the coding agents, toolchains, Docker and Cockpit, and keeps the desktop. The programs a host can do without, such as the office and graphics programs, are left out too unless `BERTH_CI_BOOT_EXTRAS=1` is set. On GitHub the Boot workflow runs either one when started by hand. Started by hand or by its weekly schedule, it installs those programs as well. Logs, timings and the disk and memory used are in `build/boot/KIND/`. diff --git a/docs/content/docs/(guide)/start/concepts.mdx b/docs/content/docs/(guide)/start/concepts.mdx index 8b5ae25..d503f46 100644 --- a/docs/content/docs/(guide)/start/concepts.mdx +++ b/docs/content/docs/(guide)/start/concepts.mdx @@ -21,10 +21,10 @@ berth has a side on your machine, the *workstation*, and a side on each host. | --- | --- | --- | | `berth` | your machine | Keeps a profile per host and installs it. Manages hosts afterwards from its terminal UI or commands | | The role | the host, once per install or apply | The Ansible role that sets up the host. Every install method runs it | -| `berthd` | the host, as root | Runs tasks, sessions and services and answers the [API](/docs/api/host) | -| `box` | the host, for the agent | Desktop control, tasks the agent may run, the channel and services, from the agent's side ([box](/docs/reference/box)) | +| `berthd` | the host, as root | Runs actions, sessions and services and answers the [API](/docs/api/host) | +| `box` | the host, for the agent | Desktop control, actions the agent may run, the channel and services, from the agent's side ([box](/docs/reference/box)) | | `berth serve` | your machine or a server | One API over all hosts, with an inbox and a key-value store. Other machines' berth can use its hosts ([A berth server](/docs/use/server)) | -| Plugins | both | Optional software, tasks and services, switched on per host ([Plugins](/docs/extend/plugins)) | +| Plugins | both | Optional software, actions and services, switched on per host ([Plugins](/docs/extend/plugins)) | The software berth puts on a host is called *devbox*. The name appears in paths and settings there, such as `/etc/devbox` and `devbox_` variables. @@ -32,11 +32,13 @@ The software berth puts on a host is called *devbox*. The name appears in paths | Term | Meaning | | --- | --- | -| Host | A laptop, server, VM or container that berth sets up and manages | -| Profile | The JSON document that describes one host, from what to install to how it is reached. berth keeps it in its store. See [Profile](/docs/reference/profile) | +| Host | Anything berth reaches and logs in to, such as a laptop, server, VM or container | +| Type | What berth does with a host: `ssh` only logs in, `managed` adds berth's controller, `workstation` applies the whole profile. See [Host types](/docs/use/types) | +| Profile | The JSON document that describes one host, from what to install to how it is reached, or a template to start one from. berth keeps it in its store. See [Profile](/docs/reference/profile) | +| Target | The kind of machine a host is installed on, such as a laptop, a cloud machine or a container. A profile says it as `target` | | Operator | Your account on the host, `ops` by default, for SSH and Cockpit | | Agent | The account the coding agents work in, `agent` by default. The virtual desktop runs as this account. Its sudo rights are its [privileges](/docs/use/accounts) | -| Task | Something a host can do, such as Status or a sign-in, run from its overview or with `berth run HOST TASK`. Plugins add tasks | +| Action | Something a host, a provisioner, a resource or the installation can do. A host's controller has actions such as Status or a sign-in, run from its overview or with `berth run HOST ACTION`. Plugins add actions | | Host level, agent level | berthd answers on two sockets. berth connects at host level, as root. `box` connects at agent level, as the agent. Each call says which levels may use it | ## Connecting to a host diff --git a/docs/content/docs/(guide)/start/install.mdx b/docs/content/docs/(guide)/start/install.mdx index da556d3..904e4e0 100644 --- a/docs/content/docs/(guide)/start/install.mdx +++ b/docs/content/docs/(guide)/start/install.mdx @@ -4,16 +4,16 @@ description: Put the berth command on the machine you work from. It is one file. icon: Download --- -berth runs on Linux. The program contains everything it puts on a host, so installing it means placing one file on your `PATH`. On Windows and macOS, `berth-remote` uses the hosts of a berth server. +berth sets hosts up from Linux. The program contains everything it puts on a host, so installing it means placing one file on your `PATH`. On macOS and Windows the same program is a client of a [berth server](/docs/use/server) on another machine. - + -```bash title="Linux x86_64" +```bash title="Linux and macOS" curl -fsSL https://raw.githubusercontent.com/helightdev/berth/main/install.sh | sh ``` -The script downloads the latest release for Linux x86_64 from GitHub, checks it against its SHA-256 and installs it to `~/.local/bin/berth`. +The script downloads the latest release for this machine from GitHub, checks it against its SHA-256 and installs it to `~/.local/bin/berth`. Releases have it for Linux on x86_64 and arm64, and for macOS on Apple silicon and Intel. | Variable | Effect | | --- | --- | @@ -22,6 +22,15 @@ The script downloads the latest release for Linux x86_64 from GitHub, checks it On other architectures, build it with cargo. + + + +```powershell title="Windows, in PowerShell" +irm https://raw.githubusercontent.com/helightdev/berth/main/install.ps1 | iex +``` + +The script installs `berth.exe` to `%LOCALAPPDATA%\Programs\berth`, checked in the same way. `BERTH_INSTALL_DIR` and `BERTH_VERSION` work as for `install.sh`. See [From macOS or Windows](/docs/use/server#on-other-systems). + @@ -32,7 +41,7 @@ rustup target add x86_64-unknown-linux-musl cargo install --locked --git https://github.com/helightdev/berth berth ``` -From a checkout, `cargo install --locked --path crates/berth` does the same. +From a checkout, `cargo install --locked --path crates/berth` does the same. On macOS and Windows the build is the client alone, which only needs Rust. @@ -46,37 +55,24 @@ docker run -d --name berth -p 127.0.0.1:7777:7777 \ ghcr.io/helightdev/berth ``` -Any other command goes after the image name, and the terminal UI needs the default command cleared: +Any other command goes after the image name, such as `ls`, or `ui` for the screens: ```bash -docker run --rm -it -v berth-config:/config -v berth-data:/data ghcr.io/helightdev/berth list -docker run --rm -it --entrypoint berth -v berth-config:/config -v berth-data:/data \ - -v ~/.ssh:/home/berth/.ssh:ro ghcr.io/helightdev/berth +docker run --rm -it -v berth-config:/config -v berth-data:/data ghcr.io/helightdev/berth ls +docker run --rm -it -v berth-config:/config -v berth-data:/data \ + -v ~/.ssh:/home/berth/.ssh:ro ghcr.io/helightdev/berth ui ``` -See [berth serve](/docs/extend/serve#in-a-container) for the volumes and the token. A berth installed on your machine can use the container's hosts with its token: +See [berth serve](/docs/extend/serve#in-a-container) for the volumes and the token. A berth installed on your machine uses the container's hosts once it has logged in with a setup key made there: ```bash -berth --server http://127.0.0.1:7777 --token "$(docker exec berth cat /data/berth/serve.token)" list +docker exec berth berth server setup-key +berth login http://127.0.0.1:7777 +berth ls ``` See [A berth server](/docs/use/server). - - - -berth itself needs Linux. `berth-remote` is the client for a [berth server](/docs/use/server) on another machine, with the screens of its page in a terminal. It runs on Windows, macOS and Linux: - -```powershell title="Windows, in PowerShell" -irm https://raw.githubusercontent.com/helightdev/berth/main/install.ps1 | iex -``` - -```bash title="macOS and Linux" -curl -fsSL https://raw.githubusercontent.com/helightdev/berth/main/install-remote.sh | sh -``` - -Each script downloads the latest release for this machine, checks it against its SHA-256 and installs it: to `%LOCALAPPDATA%\Programs\berth-remote` on Windows, and to `~/.local/bin/berth-remote` elsewhere. `BERTH_INSTALL_DIR` and `BERTH_VERSION` work as for `install.sh`. Releases have it for Windows x86_64, macOS on Apple silicon and Intel, and Linux x86_64 and arm64. See [With berth-remote](/docs/use/server#with-berth-remote). - @@ -108,7 +104,7 @@ python3 -m venv .venv && .venv/bin/pip install -r provision/requirements.txt | --- | --- | | Settings of this machine, listed below | `~/.config/berth/config.json` | | The other settings, registries, secret values and registry passwords | berth's store, by default `~/.local/share/berth/berth.db` | -| One profile per host, its generated passwords and the record of a machine berth created for it | berth's store too. `berth profile HOST` prints a profile and `berth credentials HOST` the passwords | +| One profile per host, its generated passwords and the record of a machine berth created for it | berth's store too. `berth host export HOST` prints a profile and `berth host credentials HOST` the passwords | | berth's SSH keys, and the token and setup key of `berth serve` | berth's store. A key is written to `~/.local/share/berth/keys/NAME` when ssh needs it, and the token to `serve.token` | | The key file of an encrypted store | `~/.config/berth/store.key` (setting `store_key`) | | Locks held while berth creates or changes a host's machine | `~/.local/share/berth/locks/` | @@ -117,14 +113,14 @@ python3 -m venv .venv && .venv/bin/pip install -r provision/requirements.txt | Build output such as the release archive, seeds and installer images | `~/.local/share/berth/build/` | | Downloaded Ubuntu installer ISOs | `~/.local/share/berth/iso/` (setting `iso_dir`) | -`XDG_CONFIG_HOME` and `XDG_DATA_HOME` are honoured. `berth config` prints the paths and settings, and `berth config KEY VALUE` changes one, in the file or in the store. The Settings tab of the home screen does the same. `berth store` says where the store is and whether it is encrypted. See [The store](/docs/use/store), which also says how to restore an installation from it. +`XDG_CONFIG_HOME` and `XDG_DATA_HOME` are honoured. `berth config list` prints the paths and settings, and `berth config set KEY VALUE` changes one, in the file or in the store. The Settings tab of the home screen does the same. `berth server store status` says where the store is and whether it is encrypted. See [The store](/docs/use/store), which also says how to restore an installation from it. These settings stay in `config.json`, as they belong to this machine or are needed to open the store: | Settings | What they are for | | --- | --- | | `store`, `store_key`, `store_ca` | Where the store is and how it is opened | -| `server`, `server_token` | The berth server this machine uses | +| `server`, `server_token` | The berth server this machine uses, kept by `berth login` | | `source`, `iso_dir` | Folders on this machine | A key berth does not know stays in `config.json` as it is, such as the OAuth client settings of berth 0.4, which berth has no use for any more. @@ -132,7 +128,7 @@ A key berth does not know stays in `config.json` as it is, such as the OAuth cli An older berth kept secrets in `secrets.json`, and registries and all settings in `config.json`. It kept each host's profile in `~/.config/berth/hosts/NAME.json`, with `NAME.credentials.json` and `NAME.instance.json` beside it. The first run of this berth moves all of them into the store. It keeps the old files in `~/.config/berth/migrated/`, and removes the copies of a berth server's profiles in `~/.local/share/berth/served/`. Delete the old files once the store has what you need. It also puts berth's own SSH key, the token of `berth serve` and your own plugins into the store, and leaves their files where they are. The setup key, which was in `serve.setup-key`, is in the store alone from then on. - Profiles hold Wi-Fi passwords and enrollment keys, and the store holds generated passwords. They are in clear text unless the store is encrypted, see [The store](/docs/use/store#encrypt-it). Keep the store and its key file private, and keep files that `berth profile` writes out of version control. + Profiles hold Wi-Fi passwords, and the store holds generated passwords and the tailnet's API key. They are in clear text unless the store is encrypted, see [The store](/docs/use/store#encrypt-it). Keep the store and its key file private, and keep files that `berth host export` writes out of version control. ## The provisioning source @@ -149,6 +145,6 @@ berth looks for the source in this order: To work on the role, point berth at your checkout, and clear the setting to go back: ```bash -berth config source ~/src/berth -berth config source "" +berth config set source ~/src/berth +berth config set source "" ``` diff --git a/docs/content/docs/(guide)/start/quick-start.mdx b/docs/content/docs/(guide)/start/quick-start.mdx index 8be3e0c..a20374c 100644 --- a/docs/content/docs/(guide)/start/quick-start.mdx +++ b/docs/content/docs/(guide)/start/quick-start.mdx @@ -4,7 +4,7 @@ description: Create a profile for a host, install it, and open its desktop. icon: Zap --- -This walks through a first host from berth's terminal UI. The example uses an Ubuntu 26.04 server you can already reach over SSH. The other ways to install differ only in what you choose on the Install tab. +This walks through a first host, in berth's screens and then with the commands alone. The example uses an Ubuntu 26.04 server you can already reach over SSH, at `192.0.2.10`. The other ways to install differ only in what you choose on the Install tab. With no server set, berth keeps everything on your machine. No server has to run. The [container](/docs/deploy/container) runs the same desktop on Docker, and [QEMU](/docs/deploy/qemu) runs a whole host in a VM. @@ -16,10 +16,10 @@ This walks through a first host from berth's terminal UI. The example uses an Ub ### Create the host ```bash -berth studio +berth ui studio ``` -A name berth does not know starts a new profile, which berth saves in its store under the name `studio`. Running `berth` alone opens the home screen, where `n` creates a host. `berth profile studio` prints the profile as JSON. +A name berth doesn't know starts a new host, whose profile berth saves in its store under the name `studio`. `berth ui` alone opens the home screen, where `n` creates a host. `berth host create studio` asks the same questions on the command line, as the last section shows. `berth host export studio` prints the profile as JSON. @@ -53,9 +53,9 @@ Tab and Shift+Tab switch tabs too. Ctrl+S saves the profile at any point. ### Check the settings -**Settings** (key `3`) holds what has a default. That covers the *operator*, your account on the host, and the *agent* account the coding agents use. SSH keys found in `~/.ssh` and in ssh-agent are offered as suggestions (Right or F2). Each account can take several, and the operator also takes [the key of berth itself](/docs/reference/profile#the-key-of-berth-itself). `c` on that field copies it. +**Settings** (key `3`) holds what has a default. That covers the *operator*, your account on the host, and the *agent* account the coding agents use. Paste an SSH public key into an account's key row. Each account can take several, and the operator also takes [the key of berth itself](/docs/reference/profile#the-key-of-berth-itself). `c` on that field copies it. -On a password hash field, F4 generates a password and stores its hash, and F5 hashes one you type. berth keeps a generated password in its store, and `berth credentials studio` shows it. +On a password hash field, F4 generates a password and stores its hash, and F5 hashes one you type. berth keeps a generated password in its store, and `berth host credentials studio` shows it. @@ -75,9 +75,9 @@ The other methods are on [Deploying](/docs/deploy). ### Use the host -Once the host has an SSH address, `berth studio` opens its **overview**. Run **Status** to check it, then **Access links and desktop password** for the address of the desktop. +Once the host has an SSH address, `berth ui studio` opens its **overview**. Run **Status** to check it, then **Access links and desktop password** for the address of the desktop. - + @@ -89,10 +89,23 @@ Installing a coding agent does not sign it in. Under **Sign-In**, **Sign in to C +## The same with the commands + +`berth host create` asks the questions of the four tabs in their order, below the command. Each flag answers one. With every answer given as a flag and `--no-input`, it asks nothing: + +```bash +berth host create studio --method ssh --set address=192.0.2.10 --set user=ops \ + --feature claude/cli --generate-passwords --no-input +berth host apply studio +berth host credentials studio +``` + +`berth host apply` follows the install as a job until it ends. `berth ls` lists the hosts and whether each answers, and `berth action list studio` names what it can do. `berth run studio status` runs one. A step that fails says which flag or field to fix, and exits with a status of its own. The reference lists [the exit statuses](/docs/reference/cli#exit-statuses). + ## Next Tailnet addresses, Cockpit and SSH. Give the agent a GitHub token. - The same tasks without the UI. + The same work without the UI. diff --git a/docs/content/docs/(guide)/use/access.mdx b/docs/content/docs/(guide)/use/access.mdx index 66df2ff..23539c4 100644 --- a/docs/content/docs/(guide)/use/access.mdx +++ b/docs/content/docs/(guide)/use/access.mdx @@ -19,66 +19,65 @@ berth run studio access In the panel, **Alt+R** shows the links as QR codes. Scan one with a phone to open it there. The pairing link of **Use T3 from another device** works the same way. -The desktop link printed by the task includes the password after `#` and signs in by itself, so treat it like the password. The names are the ones `tailscale status` reports for the host, such as `HOST.TAILNET.ts.net` on Tailscale. They need the tailnet's DNS on the machine you browse from, while the tailnet address works without it. berth configures no Funnel or public proxy for them. +The desktop link printed by the action includes the password after `#` and signs in by itself, so treat it like the password. The names are the ones `tailscale status` reports for the host, such as `HOST.TAILNET.ts.net` on Tailscale. They need the tailnet's DNS on the machine you browse from, while the tailnet address works without it. berth configures no Funnel or public proxy for them. ## Joining the tailnet -A host joins the tailnet while it is set up, with an enrollment key. The key fields are under **Tailscale** on the host's Install tab: +A host joins the tailnet while it is set up, with a one-use key berth makes through the API of the tailnet's control server. Nobody pastes a key. The one choice in a profile is **Tailscale** on the host's Install tab (`devbox_install_tailscale`): whether the host joins at all. -| Field | Profile | What it does | -| --- | --- | --- | -| Login server | `tailscale_login_server` | The Headscale server the host joins. Empty: berth's `tailnet_login_server` setting, or Tailscale's own service when that is empty too | -| Automatic enrollment key | `tailscale_auto_key` | berth creates a key itself whenever the host needs one | -| Tailscale enrollment key | `tailscale_auth_key` | A key you made, pasted in. Prefer a short-lived, one-use one | - -A host that is on the tailnet already is left alone, so applying again doesn't need a key. Once it has joined, it publishes the desktop and T3 there. That is `devbox_tailscale_serve`, on by default, and it needs HTTPS certificates enabled for the tailnet. +berth needs the control server's API key in the secret `tailnet_api_key`: -Without a key, enroll by hand and then run **Publish desktop and T3 on the tailnet**, or do it on the host: - -```bash title="On the host" -sudo tailscale up -sudo tailscale serve --bg --https=443 http://127.0.0.1:6080 -sudo tailscale serve --bg --https=8443 http://127.0.0.1:3773 +```bash +berth secret set tailnet_api_key +berth config set tailnet_tags tag:berth # if the tailnet wants tags ``` -Enable tailnet HTTPS if Tailscale asks for it. The publish task also gives Cockpit the tailnet's trusted certificate in place of its self-signed one. - -### Automatic enrollment keys +The secret is one of these: -With an API key of the control server, berth makes the keys and nobody pastes one. Store the API key as the secret `tailnet_api_key`. For Tailscale it's an API access token from the admin console. `TS_API_KEY` in the environment also works. For Headscale, make one with `headscale apikeys create`. +| Control server | The secret | Lifetime | +| --- | --- | --- | +| Tailscale | An OAuth client with the `auth_keys` scope, as `CLIENT_ID:CLIENT_SECRET` | The client doesn't expire. berth trades it for an access token of an hour each time it makes a key | +| Tailscale | An API access token from the admin console | 1 to 90 days, chosen when you make it | +| Headscale | An API key from `headscale apikeys create` | 90 days unless you choose otherwise | -```bash -berth secret set tailnet_api_key -berth config tailnet_tags tag:berth # if the tailnet wants tags -``` +Prefer an OAuth client for Tailscale, so the secret doesn't run out. Keys it makes have tags, from those the client may give, so set `tailnet_tags` to them. On the Secrets tab of the home screen, Enter replaces the value. With a berth server, the server keeps the secret and the `tailnet_` settings. -Then switch on **Automatic enrollment key** for the host. berth creates a key only at the moment it's used: +berth makes a key only at the moment it's used: | When | Key valid for | | --- | --- | | A seed or installer is made | 24 hours, for writing the installer and the first boot | | An apply finds the host not on the tailnet | 1 hour | -Every key joins one host once. It's pre-approved, not reusable and not ephemeral, and it gets the tags of `tailnet_tags`. A seed has its key written into its files, which only you can read, and the installed host deletes its copy after it has joined. Until then, whoever has the seed or the USB stick can join a machine to the tailnet with it. The API key never leaves the machine that runs berth. With a berth server, that's the server, and its settings and secret are set there. +Every key joins one host once. It's pre-approved, not reusable and not ephemeral, and it gets the tags of `tailnet_tags`. The role reads it from a private file, never from the command line. A seed has its key written into its files, which only you can read, and the installed host deletes its copy after it has joined. Until then, whoever has the seed or the USB stick can join a machine to the tailnet with it. The API key never leaves the machine that runs berth. + +When berth writes a seed or an installer, it says until when the key works. A machine first started later is set up without the tailnet, and the next apply from berth joins it with a new key. + +A host that's on the tailnet already is left alone, so applying again goes without a key and leaves the host logged in. Once it has joined, it publishes the desktop and T3 there. That's `devbox_tailscale_serve`, on by default, and it needs HTTPS certificates enabled for the tailnet. Enable them if Tailscale asks, then run **Publish desktop and T3 on the tailnet**. That action also gives Cockpit the tailnet's trusted certificate in place of its self-signed one. + +### When berth has no API key + +A host whose profile wants the tailnet needs the secret. Without it, berth stops before anything runs and says what to do: set the secret, or switch **Tailscale** off for the host. A seed, an installer or a new machine isn't made, and the set-up tabs show the problem for them. An apply stops before its first change. A host that's on the tailnet already is applied as usual, as it doesn't need a key. -F4 on **Tailscale enrollment key** creates a key the same way, valid for 24 hours, for pasting where you need one. +A new host on a berth without the secret starts with **Tailscale** off. ### Headscale [Headscale](https://headscale.net/) is a control server you run yourself, for the same `tailscale` client. Point berth at it: ```bash -berth config tailnet_login_server https://hs.example.org -berth config tailnet_user agents # the user new hosts belong to +berth config set tailnet_login_server https://hs.example.org +berth config set tailnet_user agents # the user new hosts belong to berth secret set tailnet_api_key # from: headscale apikeys create ``` -Hosts then join with `tailscale up --login-server https://hs.example.org`. Headscale keys need a user or tags (`tailnet_tags`). Some things differ from Tailscale: +berth makes its keys through Headscale's API with that key, and hosts join with `--login-server https://hs.example.org`. Headscale keys need a user or tags (`tailnet_tags`). Some things differ from Tailscale: | | On Headscale | | --- | --- | +| The secret | An API key. Headscale has no OAuth clients | | Names | Those of Headscale's MagicDNS, `HOST.BASE_DOMAIN`. berth shows what `tailscale status` reports | -| `tailscale serve`, Funnel | Headscale doesn't offer them, so the desktop and T3 aren't published. The access task shows SSH tunnels for them instead | +| `tailscale serve`, Funnel | Headscale doesn't offer them, so the desktop and T3 aren't published. The access action shows SSH tunnels for them instead | | HTTPS certificates | Headscale doesn't issue them. Cockpit stays on its self-signed certificate, at `https://TAILNET-ADDRESS:9090` | ## The desktop @@ -115,7 +114,7 @@ For which addresses may reach SSH, see [the firewall](/docs/deploy/ssh#the-firew ## Without a tailnet -Tunnel the ports over SSH and open them on your own machine: +Switch **Tailscale** off for a host, and berth reaches it over SSH at its address. The access action says so where the tailnet's links would be, and shows the tunnels. Tunnel the ports over SSH and open them on your own machine: ```bash ssh -L 16080:127.0.0.1:6080 -L 13773:127.0.0.1:3773 ops@HOST diff --git a/docs/content/docs/(guide)/use/accounts.mdx b/docs/content/docs/(guide)/use/accounts.mdx index 9f3e49b..0c7de14 100644 --- a/docs/content/docs/(guide)/use/accounts.mdx +++ b/docs/content/docs/(guide)/use/accounts.mdx @@ -33,14 +33,14 @@ The operator's password is optional and only for local logins: the console, the ## Passwords -On a password hash field, F4 generates a password and stores its hash, and F5 hashes a password you type. berth keeps a generated password in its store, beside the profile, and `berth credentials NAME` shows it. For a berth server's host the server generates it and keeps it. The profile itself only stores hashes. +On a password hash field, F4 generates a password and stores its hash, and F5 hashes a password you type. berth keeps a generated password in its store, beside the profile, and `berth host credentials NAME` shows it. For a berth server's host the server generates it and keeps it. The profile itself only stores hashes. - An empty agent password locks password login for the agent. - Turning off the agent's passwordless sudo requires an agent password. ## The first apply as another account -An existing machine may not have the operator yet. berth then applies as an account of the machine's own, the profile's SSH user. When sudo prompts that account for a password, switch on **Ask for sudo password** on the Run tab, or set `access.sudo` to `password`. berth asks for the password when it applies and when it connects, or takes the secret `sudo/HOST` when you keep it there. `BERTH_SUDO_PASSWORD` gives it where nobody can type it. +An existing machine may not have the operator yet. berth then applies as an account of the machine's own, the profile's SSH user. When sudo prompts that account for a password, switch on **Ask for sudo password** on the Run tab, or set `access.sudo` to `password`. berth asks for the password when it applies and when it connects, or takes the secret `sudo/HOST` when you keep it there. `--sudo-password-stdin` gives it where nobody can type it. When the SSH user is the operator, berth writes what the apply made true once it has worked: `access.sudo` becomes `none`, a login with `password` becomes one with berth's key, and a kept `sudo/HOST` or `ssh/HOST` is deleted. The operator's sudo doesn't prompt from then on, and its SSH takes keys only. Any other SSH user keeps its setting. @@ -96,12 +96,12 @@ A host that berth reaches through `access.link`, such as a container, doesn't us ### Keys -berth keeps SSH keys in its store, by name, sealed when the [store is encrypted](/docs/use/store#encrypt-it). The key `berth` always exists. berth writes a private half to `keys/NAME` in its data directory when ssh or Ansible needs it, and `berth inventory` names that file. Private halves never leave through the API or a command. +berth keeps SSH keys in its store, by name, sealed when the [store is encrypted](/docs/use/store#encrypt-it). The key `berth` always exists. berth writes a private half to `keys/NAME` in its data directory when ssh or Ansible needs it, and `berth host inventory` names that file. Private halves never leave through the API or a command. ```bash -berth key # the keys, the hosts that use each, and what is pending -berth key new deploy # make an ed25519 key -berth key add deploy ~/.ssh/deploy # take in a private key; - reads standard input +berth key list # the keys, the hosts that use each, and what is pending +berth key create deploy # make an ed25519 key +berth key import deploy ~/.ssh/deploy # take in a private key; - reads standard input berth key show deploy # the public half, for an authorized_keys berth key remove deploy # refused while a host logs in with it ``` @@ -129,13 +129,20 @@ When sshd closes the connection before the login, berth says the host is turning ### Host keys -berth records each host's SSH host key in its store and gives ssh a known-hosts file of its own for each host, under `known_hosts/` of the data directory, written from the store. So a restored installation knows its hosts. The first login records the key. A key that changes later is refused: +berth records each host's SSH host key in its store and gives ssh a known-hosts file of its own for each host, under `known_hosts/` of the data directory, written from the store. So a restored installation knows its hosts. The first login records the key and prints its fingerprint. A key that changes later is refused: ```text -The SSH host key of studio is not the one berth recorded: the machine was set up again or replaced, or something is in between. If you know which, accept the new key with: berth key host studio --accept +The SSH host key of studio is not the one berth recorded: the machine was set up again or replaced, or something is in between. If you know which, accept the new key with: berth host verify studio --accept ``` -`berth key host studio` shows the recorded key. When berth itself creates a host's machine again, it forgets the old key and records the new one without asking. When a profile of berth 0.4 is converted, a host keeps the key your `~/.ssh/known_hosts` has for its address. +Where the first key comes from depends on where the server runs: + +| You run berth | The first host key | +| --- | --- | +| With no server set, or with a `berth serve` of your own on this machine | The key `~/.ssh/known_hosts` of the server's user has for the host's address, when it has one. Otherwise the key the host shows at the first login | +| With a server elsewhere | The key the host shows the server at the first login. Nothing of your own files is sent to the server. Compare the fingerprint it prints, or `berth host verify studio`, with the host's own | + +`berth host verify studio` shows the recorded key. When berth itself creates a host's machine again, it forgets the old key and records the new one without asking. When a profile of berth 0.4 is converted, a host keeps the key your `~/.ssh/known_hosts` has for its address. ### Rekeying @@ -143,7 +150,7 @@ The SSH host key of studio is not the one berth recorded: the machine was set up 1. berth makes a new key and stores it beside the current one as pending. 2. On each host that takes the key, it logs in and adds the new key. Once a login with the new key works, it removes the old one. -3. A host that does not answer stays pending. berth offers it both keys until then, and an apply gives the operator both. The Hosts tab marks it, in the terminal UI and on the browser page. +3. A host that does not answer stays pending. berth offers it both keys until then, and an apply gives the operator both. The Hosts tab marks it. 4. When no host is pending, the new key replaces the old one, which is deleted. Running it again goes on where it stopped. With a berth server set, the rotation runs as a job on the server, and its output is shown as it comes. diff --git a/docs/content/docs/(guide)/use/backups.mdx b/docs/content/docs/(guide)/use/backups.mdx index 8965973..55f3fa7 100644 --- a/docs/content/docs/(guide)/use/backups.mdx +++ b/docs/content/docs/(guide)/use/backups.mdx @@ -20,7 +20,7 @@ On the host's **Features** tab, switch on **Backups** under System. ### Set the repository -Set `devbox_backup_repository` in the profile's `ansible_vars`, in restic's rclone form. Edit the profile with `berth profile studio > studio.json`, then `berth profile studio --set studio.json`: +Set `devbox_backup_repository` in the profile's `ansible_vars`, in restic's rclone form. Edit the profile with `berth host export studio > studio.json`, then `berth host import studio studio.json`: ```json title="In studio's profile" { "ansible_vars": { "devbox_backup_repository": "rclone:b2:my-bucket/studio" } } @@ -36,8 +36,8 @@ The repository password and the rclone configuration go to the host as files: ```bash berth secret set backup-password berth secret set rclone-conf < ~/.config/rclone/rclone.conf -berth host studio secret backup-password file ~/.config/berth-backup/password 0600 -berth host studio secret rclone-conf file ~/.config/rclone/rclone.conf 0600 +berth host set studio secret.backup-password=~/.config/berth-backup/password:0600 +berth host set studio secret.rclone-conf=~/.config/rclone/rclone.conf:0600 berth run studio secrets ``` @@ -55,9 +55,9 @@ Run **Apply this configuration**. The schedule is switched on only when a reposi Keep a copy of the repository password somewhere else. Without it the snapshots cannot be read. -## Tasks +## Actions -| Task | Does | +| Action | Does | | --- | --- | | Back up now (`backup:snapshot`) | Takes a snapshot and applies the retention policy; the first run creates the repository | | List snapshots (`backup:list`) | The snapshots in the repository, of this host and of others that share it | @@ -108,6 +108,6 @@ This page is about the agent's data on a host. On the machine that runs berth, b | The key file, when the store is encrypted | `~/.config/berth/store.key`. Keep it apart from the database | | The settings of this machine | `~/.config/berth/config.json` | -With these, an installation can be [restored on another machine](/docs/use/store#restore-on-a-new-machine). `berth profile HOST` prints one host's profile, for a copy of that alone. +With these, an installation can be [restored on another machine](/docs/use/store#restore-on-a-new-machine). `berth host export HOST` prints one host's profile, for a copy of that alone. Copy a SQLite store with `sqlite3 berth.db ".backup copy.db"`, which takes a consistent copy while berth runs. diff --git a/docs/content/docs/(guide)/use/cli.mdx b/docs/content/docs/(guide)/use/cli.mdx index ad61670..3b74023 100644 --- a/docs/content/docs/(guide)/use/cli.mdx +++ b/docs/content/docs/(guide)/use/cli.mdx @@ -1,167 +1,211 @@ --- -title: Commands -description: berth's commands, grouped by what they are for. Everything the terminal UI does for a running host can be scripted with them. +title: The command line +description: berth's commands by what they do. Each one is a call of the server's API, so a script does what the screens do. icon: Terminal --- -`berth --help` prints the full list. Commands that take a `HOST` accept a saved host's name or a path to a profile. +A command is written `berth THING ACTION [NAME] [FLAGS]`, such as `berth host show studio` or `berth secret set github`. A few that you type often have a short form. `berth --help` lists the things, and `berth THING --help` lists what each can do. The [command reference](/docs/reference/cli) has every command with its flags. -With a [berth server](/docs/use/server) set, by `--server` or the `server` setting, the host commands use the server's hosts. Installing, secrets and registries then stay on the server. +| Short form | Means | +| --- | --- | +| `berth ls` | `berth host list` | +| `berth apply HOST` | `berth host apply HOST` | +| `berth exec HOST -- COMMAND…` | Run a command on the host | +| `berth ssh HOST` | A shell over plain SSH | +| `berth run HOST ACTION` | Run one of the host's actions | +| `berth ui [HOST]` | Open [the screens](/docs/use/terminal-ui) | +| `berth serve` | `berth server run` | + +`berth` alone prints a short help. A word berth doesn't know stops with status 2 and the command it was likely meant to be. A spelling of berth 0.5, such as `berth list` or `berth provision`, stops in the same way and names the command to use now. + +## Which server + +Each command calls the API of a berth server. berth picks the first of these: + +1. `--server URL` on the command line. +2. `BERTH_SERVER` in the environment. +3. The server that `berth login` kept. +4. The `berth serve` of this machine, on its socket. +5. A server inside berth itself, with this machine's installation. This one is on Linux alone. - +```bash +berth login https://berth.example.ts.net # asks for a setup key, then keeps the server and its token +berth logout # forgets them again +``` + +The token comes from a [setup key](/docs/use/server#with-a-setup-key), from `--token-file FILE` or from `--token-stdin`. It never goes on the command line, where other users of the machine could read it. A command can also use a server once: `berth --server URL --token-file FILE ls`. -## Hosts and profiles +## Hosts -| Command | Does | + + +| What | Command | | --- | --- | -| `berth` | The home screen | -| `berth HOST` | Open a host in the terminal UI; a new name starts a new profile | -| `berth list` | Saved hosts and their addresses | -| `berth host add NAME ADDRESS [--user USER] [--port PORT] [--auth AUTH] [--level LEVEL] [--no-contact]` | Save a machine that exists by its address, reachable unless `--level` says otherwise, and install berth's key there ([Host levels](/docs/use/levels#adding-a-machine-that-exists)) | -| `berth profile HOST` | The host's profile, as JSON on standard output | -| `berth profile HOST --set FILE` | Save the profile in FILE as the host's, checked as the terminal UI reads one. `-` reads standard input. With a [berth server](/docs/use/server) set, it is saved on the server | -| `berth credentials HOST [--show]` | The passwords berth generated for the host's accounts, an account and its password to a line. Only to a terminal unless `--show` is given. For a server's host they come from the server | -| `berth inventory [HOST…]` | The hosts as an Ansible inventory | -| `berth config` | Where things are kept, and the settings | -| `berth config KEY [VALUE]` | Change a setting; no value clears it | -| `berth --version` | The version | -| `berth --server URL [--token TOKEN] [COMMAND…]` | Run once with the hosts of a berth serve elsewhere ([A berth server](/docs/use/server)) | - - +| See the hosts | `berth ls` | +| See the type, address, state and actions of one host | `berth host show studio` | +| Describe a new host and install it | `berth host create studio` | +| Make the host with a provisioner | `berth host create studio --provisioner ovh-incus` | +| Take in a machine that exists | `berth host add build-vm 192.0.2.40 --user ops` | +| Change a setting | `berth host set studio hostname=studio-2` | +| Edit the profile in `$EDITOR` | `berth host edit studio` | +| Save and load the profile as JSON | `berth host export studio > studio.json`, `berth host import studio studio.json` | +| Apply the profile | `berth apply studio` | +| Bring every host that is behind up to date | `berth rollout` ([Updates](/docs/use/updates#bringing-hosts-up-to-date)) | +| Rename a host | `berth host rename studio attic` ([Renaming a host](/docs/use/moving#renaming-a-host)) | +| Take over a host from the profile it keeps | `berth host adopt studio 192.0.2.40` ([Moving a host](/docs/use/moving)) | +| Start or stop the host's machine | `berth host start studio`, `berth host stop studio` | +| Forget a host | `berth host remove studio` | +| Forget it and delete its machine | `berth host remove studio --destroy` | + +`berth host create` asks what it needs to know when you're at a terminal. These are the questions of the set-up tabs of the screens, and the same rules check the answers. Each question has a flag. A script gives the answers with them: + +```bash +berth host create studio --method cloud-seed --set operator_username=ops \ + --set hostname=studio --feature claude/cli --generate-passwords +``` + +`berth host create --help` lists every key of `--set`. With `--no-input`, or without a terminal, a missing answer stops with status 2 and names its flag. ## Installing -| Command | Does | +| What | Command | | --- | --- | -| `berth iso HOST [--write /dev/sdX]` | Build the USB installer, downloading the ISO if none is set; write it to a drive after asking | -| `berth seed HOST` | Render the cloud-init seed | -| `berth apply HOST` | Apply the host's profile to it over SSH. With a [berth server](/docs/use/server) set, as a job there. Only a workstation is applied | -| `berth host install HOST` | Install berth's controller alone on a host below a workstation ([Installing the controller](/docs/use/levels#installing-the-controller)) | -| `berth host level HOST LEVEL` | Save the level the host's profile says, such as for a host with berth's controller added by its address. A workstation is adopted instead ([Adding a machine that exists](/docs/use/levels#adding-a-machine-that-exists)) | -| `berth apply --behind [--update] [--reboot [--force]]` | Apply every host that is behind, one at a time ([Updates](/docs/use/updates#bringing-hosts-up-to-date)). `--all` takes the current ones too | -| `berth host rename HOST NAME [--hostname]` | Give a host another name in berth, with everything berth keeps of it. The machine is left as it is. With `--hostname`, the profile's hostname changes to NAME as well, for the next apply ([Renaming a host](/docs/use/moving#renaming-a-host)) | -| `berth remove HOST` | Forget a host. The machine itself is left as it is, and a machine berth created for the host must be deleted first | -| `berth adopt HOST [[USER@]ADDRESS] [--port PORT] [--key FILE] [--force]` | Save a host from the profile it keeps, read over SSH with sudo. Without an address, a saved host by its own ([Moving a host](/docs/use/moving)) | -| `berth key` | The SSH keys of berth's [key ring](/docs/use/accounts#keys), with their public halves, the hosts that use each, and what is being replaced. With a berth server set, the server's | -| `berth key new NAME`, `berth key add NAME FILE\|-` | Make a key, or take in a private key from a file or standard input | -| `berth key show [NAME]`, `berth key remove NAME` | Print a public half, the key `berth`'s by default. Remove a key, which berth refuses while a host logs in with it | -| `berth key rotate [NAME]` | Replace a key on the hosts that take it ([Rekeying](/docs/use/accounts#rekeying)) | -| `berth key trust HOST [--key FILE] [--keep-password]` | Install the host's key with a password or a key used once ([First contact](/docs/use/accounts#first-contact)) | -| `berth key host HOST [--accept]` | The host's recorded SSH host key; accept a changed one | -| `berth connect URL` | Use the [berth server](/docs/use/server#with-a-setup-key) at URL, with the token a setup key collects. The key comes on standard input | -| `berth publish REGISTRY[/REPOSITORY] [HOST [--login [REGISTRY]]]` | Upload the release archive and use it as HOST's release address ([Registries](/docs/use/registries#publishing-the-release-archive)) | -| `berth image push REGISTRY[/REPOSITORY] [TAG]` | Push the container image | - -Both build commands refuse to start while the profile has problems, and list them. - -## Machines, provisioners and resources - -| Command | Does | +| Write the cloud seed | `berth host seed studio > user-data`, or `--file user-data` | +| Build the USB installer | `berth host iso studio`, with `--write /dev/sdX` to write it to a drive | +| Install berth's controller alone | `berth run build-vm install-controller` ([Host types](/docs/use/types#installing-the-controller)) | +| Change the type a profile says | `berth host set build-vm type=managed` | +| Make a server a node, for Incus and Docker machines | `berth host add big 192.0.2.20 --user root --type node` ([Nodes](/docs/use/nodes)) | +| Install berth's key on a host | `berth key trust build-vm` ([First contact](/docs/use/accounts#first-contact)) | +| Check the host's SSH host key, or accept a changed one | `berth host verify studio`, `berth host verify studio --accept` | +| The passwords berth made for the accounts | `berth host credentials studio` | + +For a host whose sudo takes a password, berth asks for it at the terminal, once in a command, and keeps it only while the command runs. A script gives it with `--sudo-password-stdin`, which every command takes, or keeps it as the secret `sudo/HOST`. A `berth serve` asks for it on its own terminal when it starts, or takes the kept secret. + +## Jobs + +What changes a host runs as a job of the server, such as an apply or a roll-out. berth follows the job with a line that shows the step it is at and the last lines of its log, and ends with one line that says how it went. `Ctrl+C` stops following. The job goes on in a `berth serve`. + +| What | Command | | --- | --- | -| `berth provisioner` | The registered provisioners, and whether each can be used here | -| `berth provisioner types` | The types of provisioner the plugins bring, such as `incus/vm` | -| `berth provisioner add NAME TYPE [SETTING=VALUE…]` | Register a provisioner of a type under a name of its own, with its settings ([Provisioners](/docs/deploy/provisioners#register-a-provisioner)) | -| `berth provisioner remove NAME` | Forget a provisioner that doesn't hold anything, with its settings and its own secrets | -| `berth provisioner NAME [SETTING=VALUE…]` | How it is now, what it can do, and what berth made with it, by kind. Each `NAME=VALUE` sets one of the provisioner's own arguments for this call alone | -| `berth provisioner NAME actions` | Its actions, with their arguments | -| `berth provisioner NAME run ACTION [NAME=VALUE…] [--yes]` | Run one. `settings` keeps the values of its own arguments, `secrets` its own secrets, and `new-deployment` makes a deployment. One that changes something runs as a job, and berth ends with its exit code | -| `berth provisioners` | The registered provisioners, and what each asks for | -| `berth resource [--provisioner NAME] [--kind KIND] [--host HOST]` | What provisioners made for berth, such as machines and deployments, as berth's records have them ([Resources](/docs/deploy/resources)) | -| `berth resource ID` | One of them, as its provisioner says it is now, with its actions. A name works while only one resource has it | -| `berth resource ID actions` | Its actions, with their arguments | -| `berth resource ID run ACTION [NAME=VALUE…] [--yes]` | Run one, such as `start`, `stop` or an action of its kind | -| `berth resource ID logs [--tail N] [--follow]` | What it wrote, as its provisioner gives it | -| `berth resource ID change [--spec FILE\|-] [--secret NAME[:VARIABLE]]… [NAME=VALUE…]` | Give it other values, a new specification or other secrets. The rest stays as it is | -| `berth resource ID remove [--yes]` | Remove it with its provisioner and forget it. A host's profile stays | -| `berth deploy NAME --provisioner PROVISIONER --spec FILE\|- [--secret NAME[:VARIABLE]]… [--token TEXT] [NAME=VALUE…]` | Make a deployment from a specification, such as a Compose file. With the name of one that exists, change it ([Deployments](/docs/deploy/deployments)) | -| `berth provision HOST PROVISIONER [NAME=VALUE…] [--yes]` | Create the machine for a saved host and set it up | -| `berth instance HOST` | The machine berth created for the host, as `berth resource ID` shows it | -| `berth instance HOST start\|stop` | Start or stop it, when the provisioner can | -| `berth instance HOST destroy [--yes]` | Delete the machine, after you type the host's name. The profile stays | - -With a [berth server](/docs/use/server) set, the server's provisioners, plugins and resources are used. Changing a resource needs the role `admin`, reading its log `operate`, and an action of a provisioner or a resource its own role. See [Provisioners](/docs/deploy/provisioners) and [Resources](/docs/deploy/resources#with-a-berth-server). - -## Tasks +| Start a job and leave it running | `berth apply studio --detach`, which needs a `berth serve` | +| See the jobs | `berth job list` | +| See one job | `berth job show ID` | +| Read its log, and go on as it writes | `berth job logs ID -f` | +| Wait for it, and take its status | `berth job wait ID` | -```bash -berth tasks studio # what the host can do -berth run studio status # run a task; its exit code is berth's -berth run studio backup:restore snapshot=latest --yes -berth run studio claude:login # a sign-in runs in your terminal -``` +## Actions -`berth run HOST TASK` runs a task and its plugin steps and leaves with the task's exit code. A task's arguments are given as `NAME=VALUE`, and `--yes` answers its confirmation. `berth manage HOST TASK` is the same command. Tasks that need a terminal, such as sign-ins, run in your terminal over `ssh -t`. + -`berth tasks` lists berth's own actions beside the host's tasks, such as `install-controller`, and says why one cannot run now. A host that is only reachable has no tasks yet. +```bash +berth action list studio # what the host can do +berth run studio status # run an action; its exit status is berth's +berth run studio backup:restore snapshot=latest -y +berth run studio claude:login # a sign-in runs in your terminal +``` -When a [berth server](/docs/use/server) keeps the host, `berth tasks` lists the server's actions for the host: its tasks, and berth's own such as `apply` and `seed`. `berth run` runs any of them through the server, which keeps a job of each run ([Actions](/docs/extend/serve#actions)). +An action's arguments are `NAME=VALUE` words, and `-y` answers its question. A required argument you leave out is asked for at a terminal. `berth action list` lists berth's own actions beside the controller's, such as `install-controller`, and says why one can't run now. A host whose type is `ssh` has none of its controller's yet. ## Sessions and commands ```bash -berth exec studio -- id # as the agent +berth exec studio -- id # as the agent berth exec studio --user root -- journalctl -u berthd -n 20 -berth attach studio # a new shell as the agent -berth sessions studio -berth output studio ID --tail 50 --follow +berth session attach studio # a new shell as the agent +berth session list studio +berth session logs studio ID --tail 50 -f +berth ssh studio # plain SSH as the SSH user ``` -| Command | Does | -| --- | --- | -| `berth exec HOST [--user agent\|operator\|root] [--keep] -- COMMAND…` | Run a command on the host and leave with its exit code. The session is removed afterwards unless `--keep` is given | -| `berth attach HOST [ID]` | Put this terminal onto a terminal session of the host, or onto a new shell as the agent; Ctrl+Q leaves it running | -| `berth sessions HOST` | The host's sessions, running and ended | -| `berth output HOST ID [--tail N] [--follow]` | What a session wrote | -| `berth ssh HOST [COMMAND]` | A shell over plain SSH as the SSH user, or one command | -| `berth ssh HOST --agent` | The same as the agent account | +`berth exec` ends with the command's own exit status. Its session is removed afterwards, unless `--keep` is given. `Ctrl+Q` leaves an attached terminal running. - + -## Secrets and registries +## Secrets, registries and keys -| Command | Does | +| What | Command | | --- | --- | -| `berth secret [set NAME \| remove NAME]` | Secret values, kept in berth's store | -| `berth host HOST` | The secrets and registries a host gets | -| `berth host HOST secret NAME env VARIABLE \| file PATH [MODE] \| off` | Give a secret to a host, or stop | -| `berth host HOST registry NAME on\|off` | Give a registry's login to a host, or stop | -| `berth registry` | The registries | -| `berth registry add NAME nexus\|npm\|raw\|docker URL [USERNAME]` | Add one | -| `berth registry repo NAME npm\|raw\|docker REPOSITORY [HOST[:PORT]]` | Add a repository to a Nexus server | -| `berth registry login NAME` | Store its password or token | -| `berth registry check [NAME]` | Whether it answers and accepts the login | -| `berth registry remove NAME` | Remove it | - - +| Keep a secret | `berth secret set github`, which reads the value at a prompt or on standard input | +| Give it to a host as a variable | `berth host set studio secret.github=GH_TOKEN` | +| Give it to a host as a file | `berth host set studio secret.backup=~/.config/backup/password:0600` | +| Stop giving it | `berth host set studio secret.github=` | +| Add a registry | `berth registry add nexus nexus https://nexus.example.com --user ci-user` | +| Give its login to a host | `berth host set studio registry.nexus=on` | +| See the keys of the key ring | `berth key list` ([Keys](/docs/use/accounts#keys)) | +| Replace a key on the hosts that take it | `berth key rotate berth` ([Rekeying](/docs/use/accounts#rekeying)) | + + See [Secrets](/docs/use/secrets) and [Registries](/docs/use/registries). +## Provisioners and resources + +| What | Command | +| --- | --- | +| See the provisioners | `berth provisioner list` | +| See the types the plugins bring | `berth provisioner types` | +| Register one | `berth provisioner add ovh-incus incus/vm server=https://192.0.2.10:8443` ([Provisioners](/docs/deploy/provisioners)) | +| See how it is and what it made | `berth provisioner show ovh-incus` | +| Change its settings | `berth provisioner set ovh-incus project=agents` | +| See what provisioners made | `berth resource list` ([Resources](/docs/deploy/resources)) | +| Make a deployment from a Compose file | `berth resource create wiki --provisioner docker --spec compose.yaml` ([Deployments](/docs/deploy/deployments)) | +| Give it a new Compose file | `berth resource set wiki --spec compose.yaml` | +| Read its log | `berth resource logs wiki -f` | +| Remove it | `berth resource remove wiki` | + ## Plugins -| Command | Does | +| What | Command | +| --- | --- | +| See the plugins | `berth plugin list` | +| See what one brings | `berth plugin show claude` | +| Install one | `berth plugin add @acme/berth-plugin-tools`, or a `.tgz` or folder of this machine, which is sent to the server | +| Check for newer versions | `berth plugin update --check` | +| Update them | `berth plugin update`, then `berth rollout` | + +## Ports, services and messages + +| What | Command | | --- | --- | -| `berth plugins` | Plugins and where they come from | -| `berth plugins add SPEC` | Install a plugin with npm from a name, a tarball or a folder | -| `berth plugins remove NAME` | Remove one of yours, by its id or npm name | -| `berth plugins update --check` | Which of yours have a newer version in their registry | -| `berth plugins update [ID] [--apply [--reboot [--force]]]` | Update one plugin or all, then list the hosts that are behind. `--apply` brings them up to date | +| Reach a port of a host from this machine | `berth port forward studio 8080`, or `8080:3000` for another local port | +| Call a service a plugin runs | `berth service call studio notes/store list` | +| Read a host's channel as it comes | `berth channel listen studio` | +| The messages the server took in | `berth inbox list` | +| The server's key-value store | `berth kv list NAMESPACE`, `berth kv set NAMESPACE KEY JSON` | + +## The server of this machine -With a [berth server](/docs/use/server) set, these show and change the server's plugins. A `.tgz` or folder named in `add` is sent from your machine. +| What | Command | +| --- | --- | +| Run it | `berth serve`, with `--listen ADDRESS:PORT [--expose]` for the network ([berth serve](/docs/extend/serve)) | +| Make a setup key for the first user | `berth server setup-key` ([A berth server](/docs/use/server#with-a-setup-key)) | +| The store and its key | `berth server store status`, `encrypt`, `rekey`, `decrypt`, `copy` ([The store](/docs/use/store)) | +| The API documents | `berth server api openapi`, `asyncapi`, `operations` ([API](/docs/api#the-documents)) | +| The server's settings | `berth config list`, `berth config set KEY VALUE` | +| Update berth itself | `berth update` ([Updates](/docs/use/updates#berth-itself)) | -## Services, channel and the local API +## In scripts + +| Flag | What it does | +| --- | --- | +| `-o json` | Print the API's own JSON documents instead of tables. Errors go to standard error as JSON as well | +| `--no-input` | Never ask; a missing value stops with status 2 and names its flag | +| `-y` | Answer yes to every question | +| `-q` | Print only the result | +| `--color never` | Print without colours, as `NO_COLOR` does | -| Command | Does | +| Status | Means | | --- | --- | -| `berth call HOST SERVICE METHOD [JSON]` | Call a service a plugin runs on the host | -| `berth channel HOST [TOPIC-PREFIX] [--since ID]` | Messages on the host's channel as they come | -| `berth serve [--listen ADDRESS:PORT [--expose] [--web]]` | The API over all hosts ([berth serve](/docs/extend/serve)). `--expose` allows an address other than loopback. `--web` also shows [the hosts in a browser](/docs/use/server#in-a-browser) | -| `berth update [--check]` | Replace this program with the latest release ([Updates](/docs/use/updates#berth-itself)) | -| `berth setup-key [--remove]` | Make a [setup key](/docs/use/server#with-a-setup-key) for the `berth serve` whose data is on this machine, valid for 24 hours, in place of any earlier one. `--remove` ends it | -| `berth inbox [HOST] [--topic PREFIX] [--all]` | Messages `berth serve` took in from the hosts' channels | -| `berth kv get\|set\|rm\|ls NAMESPACE [KEY] [JSON]` | The key-value store of `berth serve` | -| `berth store` | Where the store is, and whether its state is encrypted ([The store](/docs/use/store)) | -| `berth store encrypt\|rekey\|decrypt [--yes]` | Seal the store's state with a new key file, change the key, or write it plain again. Stop `berth serve` first | -| `berth store copy TARGET\|-\|--target-file FILE [--into-existing]` | Copy the store into another SQLite file or PostgreSQL database ([Move to another database](/docs/use/store#move-to-another-database)). `-` reads a URL from standard input, which keeps its password out of the process list | -| `berth operations`, `berth openapi`, `berth asyncapi` | The API documents of `berth serve` ([API](/docs/api#the-documents)) | +| 0 | Done | +| 1 | A job failed, or a host didn't answer | +| 2 | The command line was wrong | +| 3 | Refused. The role is too low, the host or the server can't do it now, or a question had the answer no | +| 4 | The server couldn't be reached | + +`berth completion bash` prints the completions of a shell. Source them from its start-up file: + +```bash +echo 'source <(berth completion bash)' >> ~/.bashrc +``` diff --git a/docs/content/docs/(guide)/use/index.mdx b/docs/content/docs/(guide)/use/index.mdx index 96613e6..66815e5 100644 --- a/docs/content/docs/(guide)/use/index.mdx +++ b/docs/content/docs/(guide)/use/index.mdx @@ -10,7 +10,7 @@ Once a host is installed, you look after it from berth, from sign-ins and tokens The home screen, the set-up tabs, keys and the mouse. - Tasks, the terminal panel and sessions. + Actions, the terminal panel and sessions. Everything without the UI, for scripts. Use the hosts of a berth serve from another machine. Links, the tailnet, Cockpit and SSH. @@ -20,4 +20,5 @@ Once a host is installed, you look after it from berth, from sign-ins and tokens npm, Docker and Nexus logins, and publishing the release. Nightly restic snapshots of the agent's data. What updates itself, what you update, and the limits. + The order of the updates since berth 0.5, and the way back. diff --git a/docs/content/docs/(guide)/use/levels.mdx b/docs/content/docs/(guide)/use/levels.mdx deleted file mode 100644 index 0effb20..0000000 --- a/docs/content/docs/(guide)/use/levels.mdx +++ /dev/null @@ -1,116 +0,0 @@ ---- -title: Host levels -description: A host has a level, from a machine berth only logs in to up to a whole workstation. What each level allows, and how to add a host by its address. -icon: Layers ---- - -berth does not have to set a machine up to work with it. Each host has one of three levels: - -| Level | What is on the host | What berth does with it | -| --- | --- | --- | -| `reachable` | Nothing of berth | Logs in over SSH for a terminal, commands and the inventory | -| `controller` | berthd and box | Also sessions, the channel, files, tasks and secrets | -| `workstation` | The whole profile | Also the desktop, the agents, plugins, applying and roll-outs | - -The level is part of the host's [profile](/docs/reference/profile#level), as `level`. A profile that doesn't say is a workstation, so profiles from before levels mean what they always did. What the host has right now is asked of it. The overview and the Hosts tab show both when they differ, such as for a controller installed by hand. - -## Capabilities - -| Capability | `reachable` | `controller` | `workstation` | -| --- | --- | --- | --- | -| Terminal and commands over SSH, inventory | yes | yes | yes | -| Install the controller alone | yes | update it | by applying | -| Sessions, channel, files, units, logs, processes | no | yes | yes | -| The tasks status, update and reboot | no | yes | yes | -| Secrets and registry logins sent to the host | no | yes | yes | -| Plugin features, their tasks and services | no | no | yes | -| Apply, roll-out, cloud seed, USB installer | no | no | yes | -| Desktop and access links | no | no | yes | - -The terminal UI, the browser page, the commands and the API all use the same check in berth. An action the level rules out is still listed, as unavailable, with the reason and what raises the level. Asked for anyway, the server refuses it with `501 not_available`. A roll-out leaves hosts that aren't workstations and says how many it left. - -## Adding a machine that exists - -A machine you can already log in to is added by its address, and berth leaves it as it is. - -```bash -berth host add dock 192.0.2.4 --user ops -``` - -| Option | Default | For | -| --- | --- | --- | -| `--user USER` | Your SSH configuration | The SSH user | -| `--port PORT` | Your SSH configuration | The SSH port | -| `--auth AUTH` | `berth` | How berth [logs in](/docs/use/accounts#logging-in-to-hosts): `berth`, `key:NAME`, `password` or `ssh` | -| `--level LEVEL` | `reachable` | The level it is meant to have | -| `--no-contact` | | Save it without logging in | - -After saving the host, berth logs in the way [first contact](/docs/use/accounts#first-contact) does. It tries its own key, and if the machine refuses it, it asks for a password or a private key that logs in once to install the key. It records the host's SSH host key and prints its fingerprint. When your `~/.ssh/known_hosts` knows the address, berth takes the host key from there. With a [berth server](/docs/use/server) set, the host is saved on the server, which logs in with its own keys. - -Then berth asks the host what it is. A machine that berth set up before has a controller that says its level, and berth offers to change the profile here to a higher one than `--level` gave. Without a terminal it prints the command. - -A workstation has its whole profile on the host as well. berth offers to [adopt](/docs/use/moving) it from there: - -```bash -berth adopt dock -``` - -A host with berth's controller alone has nothing more to keep, and berth offers to save its level: - -```bash -berth host level dock controller -``` - -`berth host level HOST LEVEL` only changes the profile here. A host's page says the same when its controller reports more than its profile says. berth refuses to install the controller alone on a host that is a workstation, whatever its profile here says. It asks the host's controller first, and the playbook checks the host's own record and the profile it keeps. - -In the terminal UI, create a host on the home screen and choose **Existing host, SSH only** on the **Install** tab. The address and the login are all the set-up asks for, and **Run** saves the host. For a controller, choose **Existing host over SSH** and set **Level** to `controller`. **Run** then installs the controller. The browser page does the same through the server. - -Over the API, `hosts.save` takes a profile with only the level and `access`: - -```bash -curl --unix-socket "$XDG_RUNTIME_DIR/berth/serve.sock" -X PUT \ - -d '{"level": "reachable", "access": {"address": "192.0.2.4", "user": "ops"}}' \ - http://berth/v1/hosts/dock/profile -``` - -The host's `trust-key` action then installs the server's key, with a password used once in its arguments ([Actions](/docs/extend/serve#actions)). - -## Add the server to berth - -The recipes leave a plain Ubuntu server where the account `root` takes the public key you gave `rescue-install`. Add it to berth from your own machine. This example uses the made-up name `big` and the address `192.0.2.20`: - -```bash -berth host add big 192.0.2.20 --user root --auth ssh -berth host install big -``` - -`berth host add` saves `big` as a reachable host. `--auth ssh` makes berth log in with your own SSH configuration and agent, so the key that works for `ssh root@192.0.2.20` works for berth. The default, `--auth berth`, uses berth's own key, which berth installs on the first contact with a password or a private key that logs in once. `--auth ssh` works with a local berth only. With a [berth server](/docs/use/server) set, the server logs in with its own keys. - -`berth host install big` puts the controller alone on the server. berth can then report its status and run commands there, and it can update and reboot the server. Sessions run as the SSH user, here `root`. berth does not apply such a server as a workstation. It stays a reachable or controller host, and the machines on it are created with the Incus or Docker provisioner. - -The recipes are in [`recipes/`](https://github.com/helightdev/berth/tree/main/recipes): `rescue-install` installs Ubuntu, and `incus-host` makes the server a host for machines. - -## Installing the controller - -```bash -berth host install dock -``` - -The controller is berthd and box with the files they read, from the same part of the role that a full apply runs. Nothing else of a workstation is installed: no accounts, desktop, agents or plugins. The SSH user is the account sessions run as, and its home is where the file calls may go. The host's level becomes `controller`, and its tasks are status, update, reboot and secrets. - -The same is the host's action **Install the controller** on its overview, in the browser page and over the API as `install-controller`. On a controller host it's called **Update the controller**: run it after updating berth, as berth's own version is all such a host is compared with. A workstation's controller comes with applying its configuration, so the action is not offered there. - -berth also refuses it on a host that is a workstation whatever its profile here says. It asks the host's controller first. The role checks again on the host before it changes anything. It stops when the host's record says workstation. It also stops when berthd there serves an account other than the SSH user. Installing the controller alone on such a host would take the workstation's controller apart. The overview and `berth tasks` say when a host is more than its profile, and how to adopt it. - - - On a host with systemd the controller is a service that starts at boot. Without systemd, as in most containers, the controller is installed but nothing on the host starts it. berth says so and waits a minute for it to answer. The install fails when it doesn't. Something on the host then has to run `/usr/local/bin/berthd`, as the init of a container can. - - -## Raising a level - -| From | To | How | -| --- | --- | --- | -| `reachable` | `controller` | Install the controller | -| `reachable` or `controller` | `workstation` | Set **Level** to `workstation` in the set-up, fill in the rest of the tabs, and apply | - -berth never lowers a level, and never removes what it installed. diff --git a/docs/content/docs/(guide)/use/meta.json b/docs/content/docs/(guide)/use/meta.json index a689f03..54e29e4 100644 --- a/docs/content/docs/(guide)/use/meta.json +++ b/docs/content/docs/(guide)/use/meta.json @@ -1,5 +1,22 @@ { "title": "Using berth", "icon": "SquareTerminal", - "pages": ["terminal-ui", "overview", "cli", "levels", "server", "store", "access", "accounts", "sign-ins", "secrets", "registries", "backups", "moving", "updates"] + "pages": [ + "terminal-ui", + "overview", + "cli", + "types", + "nodes", + "server", + "store", + "access", + "accounts", + "sign-ins", + "secrets", + "registries", + "backups", + "moving", + "updates", + "upgrading" + ] } diff --git a/docs/content/docs/(guide)/use/moving.mdx b/docs/content/docs/(guide)/use/moving.mdx index bfa3bbf..07add59 100644 --- a/docs/content/docs/(guide)/use/moving.mdx +++ b/docs/content/docs/(guide)/use/moving.mdx @@ -4,7 +4,7 @@ description: Take a host over in another berth, or get a lost profile back, from icon: ArrowRightLeft --- -Every apply and every first boot from a seed leaves the host's profile on the host, at `/etc/berth/profile.json`. Only root can read it. A berth that does not have the profile can take the host over from it with `berth adopt`. This helps when: +Every apply and every first boot from a seed leaves the host's profile on the host, at `/etc/berth/profile.json`. Only root can read it. A berth that does not have the profile can take the host over from it with `berth host adopt`. This helps when: - you move to another berth installation - a berth lost its profiles @@ -16,50 +16,52 @@ The file contains the profile as berth stores it, with a few fields left out: | Field | Kept | Why | | --- | --- | --- | -| `tailscale_auth_key` | No | An enrollment key works once. The new berth makes a new one when the host has to join again | -| `access` | No | `berth adopt` sets the address, user and port from how it reached the host. A key name means a key of the old berth's key ring, and a link command is a program of the old berth's machine | +| `tailscale_auth_key`, `tailscale_auto_key`, `tailscale_login_server` | No | Fields of older berths. berth makes a key when the host has to join the tailnet again, at the control server of its own settings | +| `access` | No | `berth host adopt` sets the address, user and port from how it reached the host. A key name means a key of the old berth's key ring, and a link command is a program of the old berth's machine | | `builder.served` | No | A server of the old berth means nothing to another one | | `secrets`, `registries` | Names only | Their values stay in each berth's own store and are never in a profile | | `operator_password_hash`, `agent_password_hash` | Yes | They are in `/etc/shadow` already. Without its hash, an apply would lock the agent's password | | `wifi` | Yes | The USB installer wrote the same network into the host's network settings | | The machine a provisioner made | Yes | Its record, so it is still a [resource](/docs/deploy/resources) of the host. Its token is a label on the machine and does not grant access | -The file also says its format, the version of the berth that wrote it, the time, and the host's name there. A file of berth 0.4.1 has format 1 and the profile layout of then. berth converts it when it adopts the host. +The file also says its format, the version of the berth that wrote it, the time, and the host's name there. A file of berth 0.4.1 has format 1 and the profile layout of then. Format 2, of berth 0.5, has the [older names](/docs/reference/profile#older-profiles) `level` and `profile`. `berth host adopt` converts both, and writes format 3. A berth that knows only format 2 refuses a file of format 3 and asks to be updated. -berthd serves the same file to an administrator as `GET /v1/applied/profile`. Through a berth server that is `GET /v1/hosts/NAME/applied/profile`. +berthd serves the same file to an administrator as `GET /v1/applied/profile`. Through a berth server that is `GET /v1/hosts/NAME/controller/applied/profile`. ## Adopt a host ```bash -berth adopt studio ops@192.0.2.10 +berth host adopt studio ops@192.0.2.10 berth apply studio ``` -`berth adopt NAME [USER@]ADDRESS` logs in over SSH as a user who can become root, reads the file with sudo and saves it as host `NAME`. It does not need berthd. `--port PORT` gives another SSH port. `--key FILE` offers one more private key for this read only. `--force` replaces a saved host of the same name. Without it, berth refuses. +`berth host adopt NAME [USER@]ADDRESS` logs in over SSH as a user who can become root, reads the file with sudo and saves it as host `NAME`. It does not need berthd. `--port PORT` gives another SSH port. `--key FILE` offers one more private key for this read only. `--force` replaces a saved host of the same name. Without it, berth refuses. -Without an address, `berth adopt NAME` adopts the saved host `NAME` at the address its profile has. This is the way for a workstation that was saved by its address with [`berth host add`](/docs/use/levels#adding-a-machine-that-exists), whose profile here says less than the host keeps. `berth host add` suggests the command when it finds such a host. A host saved as a workstation already needs `--force`. +Without an address, `berth host adopt NAME` adopts the saved host `NAME` at the address its profile has. This is the way for a workstation that was saved by its address with [`berth host add`](/docs/use/types#adding-a-machine-that-exists), whose profile here says less than the host keeps. `berth host add` suggests the command when it finds such a host. A host saved as a workstation already needs `--force`. The profile's `access` comes from how berth got in. It logs in with `berth` when this berth's own key worked, and with `ssh` when your SSH configuration, your agent or the key of `--key` did. `sudo` is `password` when sudo asked for one. After the first apply, the operator takes this berth's key, and an apply as the operator sets `sudo` to `none` ([Logging in to hosts](/docs/use/accounts#logging-in-to-hosts)). -berth asks on the terminal for a sudo password when sudo on the host needs one. `BERTH_SUDO_PASSWORD` gives it without a terminal. +berth asks on the terminal for a sudo password when sudo on the host needs one. Without a terminal, `--sudo-password-stdin` reads it from standard input. The output lists what the profile names and this berth lacks: secret values and registries. Set them before you apply. `berth apply` then makes the host this berth's. -With a [berth server](/docs/use/server) set, `berth adopt` reads the file from this machine and saves the host on the server. The server applies with its own key, and it does not take the record of a machine a provisioner made. +With a [berth server](/docs/use/server), `berth host adopt` runs on the server: the server logs in to the host, reads the file and saves the host. The server applies with its own key, and it does not take the record of a machine a provisioner made. ## Getting in -`berth adopt` logs in with this berth's own key, your SSH agent and your usual keys. A new berth's key is not on the host yet, so use one of these two ways. +`berth host adopt` logs in with this berth's own key, your SSH agent and your usual keys. A new berth's key is not on the host yet, so use one of these two ways. ### Bring the old berth's key -Give `berth adopt` the old berth's private key with `--key FILE`. A berth since 0.5 keeps its key in its store, and writes it to `keys/berth` in its data directory, `~/.local/share/berth` by default, when it logs in to a host. A berth of 0.4 kept it as `id_ed25519` there. Copy it through a pipe, so the key is never part of a command line: +Give `berth host adopt` the old berth's private key with `--key FILE`. A berth since 0.5 keeps its key in its store, and writes it to `keys/berth` in its data directory, `~/.local/share/berth` by default, when it logs in to a host. A berth of 0.4 kept it as `id_ed25519` there. Copy it through a pipe, so the key is never part of a command line: ```bash umask 077 ssh old-machine 'cat ~/.local/share/berth/keys/berth' > old-berth.key -berth adopt studio ops@192.0.2.10 --key old-berth.key -berth key trust studio --key old-berth.key +berth host adopt studio ops@192.0.2.10 --key old-berth.key +berth key import old-berth old-berth.key +berth key trust studio --key old-berth +berth key remove old-berth rm old-berth.key ``` @@ -122,4 +124,4 @@ berth moves everything it keeps under the old name to the new one, in one change With `--hostname`, the new name becomes the profile's hostname as well. The machine takes it at its next apply, which also writes the profile the host keeps with the new name. The name has to suit a hostname then: lower-case letters, digits and `-`. -berth refuses when a host has the new name already. A job that is creating the host's machine blocks the rename until that job ends. When a [berth server](/docs/use/server) keeps the host, the server renames it as a job. On the server's page and over its API it's the host's action **Rename**, `rename`. +berth refuses when a host has the new name already. A job that is creating the host's machine blocks the rename until that job ends. When a [berth server](/docs/use/server) keeps the host, the server renames it as a job. On its overview and over the server's API it's the host's action **Rename**, `rename`. diff --git a/docs/content/docs/(guide)/use/nodes.mdx b/docs/content/docs/(guide)/use/nodes.mdx new file mode 100644 index 0000000..a71e60e --- /dev/null +++ b/docs/content/docs/(guide)/use/nodes.mdx @@ -0,0 +1,156 @@ +--- +title: Nodes +description: A node is a server that berth applies without a desktop, such as one that hosts Incus and Docker machines. Creating one, taking in a server that exists, and what berth leaves to you. +icon: Server +--- + +A node is a server that berth sets up and keeps up to date, without a desktop. Its plugin features install what the server is for, such as Incus or a Docker engine for berth's machines. berth then has a provisioner for each, and the provisioner records which node it is on. + +| Part | On a node | +| --- | --- | +| System | Ubuntu 24.04 or later on x86_64, its time zone and locale, and Ubuntu's security updates without reboots | +| SSH | Keys only. The SSH user takes berth's key, and root keeps a login with keys when berth logs in as root or root had one before | +| Controller | berthd and box, with the actions status, update, reboot and secrets. Sessions run as the SSH user | +| Firewall | `ufw`: incoming traffic is refused except SSH, the tailnet and what the features open | +| Tailnet | Tailscale, joined as a workstation joins ([Tailnet](/docs/use/access)) | +| Features | Only those the profile lists, such as `incus/server` and `docker/engine` | +| Not on a node | A desktop, the agent and operator accounts, VNC, T3, Cockpit, and the agents' features | + +A node is a [host type](/docs/use/types) between `managed` and `workstation`. It is applied and rolled out like a workstation. It has no cloud seed or USB installer, because it is a server you install with Ubuntu first. + +## Create one + +Install Ubuntu on the server from your provider's panel, or with the [`rescue-install`](https://github.com/helightdev/berth/tree/main/recipes/rescue-install) recipe when that installer does not work. Then add it by its address as a node. This example uses the made-up name `big` and the address `192.0.2.20`: + +```bash +berth host add big 192.0.2.20 --user root --auth ssh --type node \ + --feature docker/engine --feature incus/server +berth apply big +``` + +`--feature` names the features it gets, which land in the profile's `ansible_vars` as `devbox_plugin_features`. `berth host set big feature.incus/server=off` changes them later. `berth host create big --type node` asks for the same at a terminal: the address, the login and the features, without a desktop or accounts. The screens show the same for a node: the type `node` on the Install tab, and only the features of a node on the Features tab. + +The first apply installs berth's key for the SSH user. A node added with `--auth password` is logged in to with berth's key afterwards, and berth forgets the password. `--auth ssh` stays as it is. + +An `ssh` or `managed` host becomes a node with `berth host set big type=node`. A node is not applied over a workstation that berth set up: the role stops before it changes anything. + +| Profile field | On a node | +| --- | --- | +| `type` | `node` | +| `access` | Where berth reaches it and as whom. The SSH user is the account sessions run as | +| `target` | `cloud`, or `container` for a node without systemd. `laptop` is refused | +| `ansible_vars` | `devbox_plugin_features`, the settings of the features, and the node's own settings below | +| The agent's and the operator's fields | Refused, as a node has neither account. `berth host set HOST type=node` takes them out | + +| Setting | Default | Meaning | +| --- | --- | --- | +| `devbox_ssh_from` | `any` | Who reaches SSH from outside the tailnet: `any`, or one address or network. The address an apply comes from always does. A server taken in keeps SSH as limited as its firewall had it | +| `devbox_harden_ssh` | `true` | Keys only for SSH. It is skipped while the SSH user has no key | +| `devbox_manage_firewall` | `true` | Set the node's firewall | +| `devbox_install_tailscale` | `true` | Install Tailscale and join the tailnet with a key berth makes ([Joining the tailnet](/docs/use/access#joining-the-tailnet)) | +| `devbox_timezone` | `Europe/Berlin` | The node's time zone | + +## Features for a node + +| Feature | What it sets up | Provisioners berth registers | +| --- | --- | --- | +| `incus/server` | Incus, the bridge `incusbr0` at `10.77.0.1/24`, a storage pool in a directory, and the API on port 8444 that trusts berth's client certificate | `big-incus-vm` and `big-incus-container` | +| `docker/engine` | The Docker engine, and Sysbox when asked | `big-docker-container`, which reaches the engine over SSH | + +The guides of the [Incus](https://github.com/helightdev/berth/blob/main/plugins/incus/guide.md) and [Docker](https://github.com/helightdev/berth/blob/main/plugins/docker/guide.md) plugins list their settings. A feature says which host types it is for. berth refuses to apply a node that lists a feature for workstations, and the reverse. + +### Provisioners the node offers + +At the end of an apply the role records what each feature offers in `/etc/berth/applied.json`, and berthd reports it. berth then registers a provisioner for each: + +| What | Where it comes from | +| --- | --- | +| The name | `NODE-PLUGIN-TYPE`, such as `big-incus-vm` | +| The node | The host it is on, on the provisioner's page (`node` over the API), and kept when the node is renamed | +| Settings | Where berth reaches the node, such as `server=https://192.0.2.20:8444` | +| Secrets | Public values only, such as the Incus server's certificate, kept as the provisioner's own `incus_server_cert` | + +berth makes its Incus client certificate and key once, as the secrets `incus_client_cert` and `incus_client_key`. Only the certificate goes to the node, which trusts it. The key stays in berth's store. + +### Reach what runs on a node + +The endpoints of a resource on a node's provisioner are dialled by the node's controller, not by the server ([Reach an endpoint](/docs/deploy/resources#reach-an-endpoint)). So `resources.connect` reaches a container on `incusbr0` or `docker0`, and a port published on the node's `127.0.0.1`, also when the server has no route to them. Each request to the provisioner has the node's name in `node`, and the Docker and Incus plugins then report addresses as the node sees them. + +| Address of the endpoint | The node's controller | +| --- | --- | +| `127.0.0.1` | Connects to that port of its loopback | +| On a network of one of the node's interfaces, such as `10.77.0.57` | Connects to it | +| Any other, a link-local or a metadata address | Refuses it | + +The node's firewall allows outgoing connections. The controller's connections to its bridges pass. + +berth takes from a node's report only what is the node's own, as the node is a machine it manages: + +| Part of the report | Taken when | +| --- | --- | +| A feature | The node's profile names it | +| A provisioner | The feature's manifest lists it in `offers` | +| A setting | It is one of the provisioner's own arguments. A URL in it has to point at the node's address, or berth leaves the provisioner out | +| A secret | It is one the provisioner declares | + +The node's provisioners are those berth registered for it or made its own. A node's report never changes or removes any other provisioner. + +An apply with the same features doesn't change them. A feature taken out of the profile takes its provisioners with it at the next apply. While one of them still holds machines or deployments, berth refuses to apply the node without the feature, and it refuses to remove the node. Remove those resources first, or keep the feature. + +## Take in a server that exists + +A server that the `incus-host` recipe of berth 0.5 set up, or that has Incus or Docker installed by hand, is added and applied as a node like a new one. The features take what is there: + +- Incus with a storage pool keeps its pools, networks, profiles and trusted clients. Only the API address, when it has none, and berth's certificate are added. +- An installed Docker engine stays, with its containers and volumes, also one from Docker's own packages. +- A firewall rule made by hand that lets SSH in from some addresses only stays, and the node does not open SSH to every address. The address the apply comes from is let in too. Delete the rule by hand to open SSH wider. +- Root keeps a login with keys it had, such as the one the berth server in its container uses. The node never opens root to passwords. +- A provisioner registered by hand for the node's server becomes the node's instead of a second one, with its settings as they were. That happens only when berth itself can tell: the server it names is on the node's address, the same name or a name that berth's server finds at the same address. The recipe's `https://incus-host:8444` passes when the server reaches the node as `node`, as both resolve to the gateway of the server's container. +- berth leaves a provisioner as it is when only the node says it's the same, and also one of another node. The node's word may be the provisioner's certificate, which is public. It may also be the provisioner's server among the aliases. The apply says so in a line and doesn't register a second one. If you know it reaches the node, make it the node's and apply again: + + ```bash + berth provisioner run incus-vm adopt node=big + berth apply big + ``` + + This is also the way for the provisioner of the local Docker engine of a berth server in a container on the node. + +Apply the node from the berth server that holds the provisioners registered for this machine, such as the server of the recipe. Those provisioners are that server's. A berth elsewhere doesn't see them, and registers new ones beside them. + +Then check the node and its provisioners: + +```bash +berth run big status +berth provisioner list +``` + +## The berth server on a node + +The `berth-server` recipe installs berth's server in a container named `berth`. When a node has that container, it offers the action **Upgrade the berth server** (`upgrade-server`). It runs the server from the image of the version of the berth that runs the action, or of the `version` it is given. The node's controller does it, and a job follows it: + +- The new container takes over what the running one has: + + | Taken over | Such as | + | --- | --- | + | Name and data | `berth`, and the folders or volumes it mounts | + | Ports and networks | `--publish 127.0.0.1:7777:7777` | + | Restart policy | `--restart unless-stopped` | + | Groups and extra hosts | `--group-add`, `--add-host incus-host:host-gateway` | + | User and environment | The variables the image does not set | + + A Docker socket mounted into it stays. +- Logging, such as the `max-size` and `max-file` a `daemon.json` gives every container, is carried over too. +- An option it cannot carry over, such as `--privileged` or devices, stops it before anything changes. Upgrade such a container by hand. +- The old container is stopped and kept until the new one keeps running. If the new one stops, the old one runs again. +- It refuses to go to an older version than the one running. + +When the server upgrades its own container, it restarts during the job. The output stays in the node's session. + +The action needs the machine to be a node, which needs a server of this berth. So the recipe makes the first update of a server of berth 0.5, as [Upgrading from 0.5](/docs/use/upgrading) shows. + +## What berth does not do + +- It never makes a storage pool on a disk, which would erase it. Create such a pool with `incus storage create` and name it in `devbox_incus_pool`. +- Taking a feature out does not remove Incus, Docker or their guests from the node. Only the provisioner goes. +- Docker opens the ports its containers publish by itself, past `ufw`. Publish a port on `127.0.0.1` or on the tailnet address to keep it private. +- It does not take a workstation to a node, or a node back to `managed`. diff --git a/docs/content/docs/(guide)/use/overview.mdx b/docs/content/docs/(guide)/use/overview.mdx index 7bc13cc..7f4acba 100644 --- a/docs/content/docs/(guide)/use/overview.mdx +++ b/docs/content/docs/(guide)/use/overview.mdx @@ -1,18 +1,18 @@ --- title: A host's overview -description: The page of a running host, with its tasks, the terminal panel and its sessions. +description: The page of a running host, with its actions, the terminal panel and its sessions. icon: LayoutDashboard --- -Opening an installed host shows its overview, with the connection at the top and the host's tasks in groups below. The host reports its own tasks, and the list changes with what is installed there. +Opening an installed host shows its overview, with the connection at the top and the host's actions in groups below. The host's controller reports its own actions, and the list changes with what is installed there. -The line under the title is the host's load, CPU, memory, disk and uptime. Under **Connection**, a line says the host's [level](/docs/use/levels), and what the host itself was seen to be when that differs. +The line under the title is the host's load, CPU, memory, disk and uptime. Under **Connection**, a line says the host's [type](/docs/use/types), and what the host itself was seen to be when that differs. -## Tasks +## Actions -| Group | Task | Does | +| Group | Action | Does | | --- | --- | --- | | System | Status | Provisioning state and services, graphics and disk space, failed units and the access links, then what plugins report | | | Terminal | A shell on the host in a panel; Space switches between the agent and the SSH user | @@ -30,11 +30,11 @@ The line under the title is the host's load, CPU, memory, disk and uptime. Under When the host was applied before a plugin or berth itself changed, a **Configuration** group at the top says which, such as `claude 1.0.0 → 1.1.0`, with **Apply this configuration** below it ([Updates](/docs/use/updates#hosts-that-are-behind)). -berth's own actions on the host are listed beside its tasks, such as **Install the controller** for a host below a workstation. One the host's level or state rules out is marked `not now`, and the Information panel says why and what to do first. A host with the controller alone has only Status, Update, Reboot and the secrets. +berth's own actions on the host are listed beside its controller's, such as **Install the controller** for a host that is not a workstation. One the host's type or state rules out is marked `not now`, and the Information panel says why and what to do first. A host with the controller alone has only Status, Update, Reboot and the secrets. -Plugins add tasks to these groups or bring groups of their own, such as **Backup**. They also add steps to built-in tasks such as Status and Update. A task's output stays on the page. +Plugins add actions to these groups or bring groups of their own, such as **Backup**. They also add steps to built-in actions such as Status and Update. An action's output stays on the page. -Some tasks take arguments, such as which snapshot to restore. Tasks that cannot be undone ask for confirmation first. When the profile says to ask for the SSH user's sudo password, berth asks once and keeps it until it closes. +Some actions take arguments, such as which snapshot to restore. Actions that cannot be undone ask for confirmation first. When the profile says to ask for the SSH user's sudo password, berth asks once and keeps it until it closes. ## Panels @@ -56,11 +56,11 @@ A long web address that runs over several rows is one link, even when the progra ## Sessions -A terminal is a session of the host and keeps running when you leave it. Running sessions appear in a **Sessions** group on the overview, where Enter opens one again. `berth sessions HOST` lists them, and `berth attach HOST ID` puts your own terminal onto one ([Commands](/docs/use/cli#sessions-and-commands)). +A terminal is a session of the host and keeps running when you leave it. Running sessions appear in a **Sessions** group on the overview, where Enter opens one again. `berth session list HOST` lists them, and `berth session attach HOST ID` puts your own terminal onto one ([Commands](/docs/use/cli#sessions-and-commands)). ## Unreachable hosts -Instead of the tasks, the overview then says why, and offers: +Instead of the actions, the overview then says why, and offers: | Row | Does | | --- | --- | diff --git a/docs/content/docs/(guide)/use/registries.mdx b/docs/content/docs/(guide)/use/registries.mdx index cc87d4b..4f2d1e9 100644 --- a/docs/content/docs/(guide)/use/registries.mdx +++ b/docs/content/docs/(guide)/use/registries.mdx @@ -15,22 +15,22 @@ A registry is a server with one address and one login. Hosts get npm and Docker A registry with a user name logs in with user name and password. Without a user name, what is stored is used as a token: `_authToken` for npm, a bearer token for uploads. -Registries, with their addresses and user names, are kept in berth's store. Passwords and tokens are kept there as secrets, sealed when the store is encrypted. `berth store` says where the store is. +Registries, with their addresses and user names, are kept in berth's store. Passwords and tokens are kept there as secrets, sealed when the store is encrypted. `berth server store status` says where the store is. ## Adding registries ```bash -berth registry add nexus nexus https://nexus.example.com ci-user +berth registry add nexus nexus https://nexus.example.com --user ci-user berth registry login nexus # asks for the password or token -berth registry repo nexus npm npm-group -berth registry repo nexus raw berth-releases -berth registry repo nexus docker docker-hosted nexus.example.com:8083 +berth registry add-repo nexus npm npm-group +berth registry add-repo nexus raw berth-releases +berth registry add-repo nexus docker docker-hosted --address nexus.example.com:8083 berth registry add verdaccio npm https://npm.example.com # token login -berth registry add files raw https://files.example.com/berth ci-user -berth registry add ghcr docker ghcr.io my-user +berth registry add files raw https://files.example.com/berth --user ci-user +berth registry add ghcr docker ghcr.io --user my-user -berth registry # the list +berth registry list # the list berth registry check # do they answer and accept the login? ``` @@ -51,7 +51,7 @@ A Docker repository with path-based routing in Nexus does not need an address: i ## Giving a host a registry ```bash -berth host studio registry nexus on +berth host set studio registry.nexus=on berth run studio secrets ``` @@ -62,8 +62,8 @@ Or switch it on under **Registries** on the host's **Settings** tab and run **Se A machine set up from a [cloud seed](/docs/deploy/cloud) downloads the release archive. With **Cloud-init seed** chosen on a host's Install tab, **Publish release to** under Release on the Settings tab switches between your file servers and raw repositories. **Download with login** decides whether the machine logs in for the download. Making the seed then uploads the archive and points the seed at it. From the command line: ```bash -berth publish nexus/berth-releases cloud-1 -berth publish files cloud-1 +berth registry publish nexus/berth-releases --host cloud-1 +berth registry publish files --host cloud-1 ``` Either way the seed pins the archive's SHA-256, and the archive contains no profiles, keys or secrets. The new machine downloads it with or without a login. @@ -77,8 +77,8 @@ The default. The repository has to allow anonymous reads. ```bash -berth publish nexus/berth-releases cloud-1 --login # the login of nexus -berth publish nexus/berth-releases cloud-1 --login nexus-read # another registry's +berth registry publish nexus/berth-releases --host cloud-1 --login # the login of nexus +berth registry publish nexus/berth-releases --host cloud-1 --login nexus-read # another registry's ``` The cloud seed then contains that login, in a file only root can read that is deleted after the download. The login is taken from your machine when the seed is rendered, so it is never written into the host's profile. @@ -93,8 +93,8 @@ The cloud seed then contains that login, in a file only root can read that is de ## Pushing the container image ```bash -berth image push nexus/docker-hosted # NEXUS-DOCKER-ADDRESS/berth-desktop:latest -berth image push ghcr v1 # ghcr.io/berth-desktop:v1 +berth registry push-image nexus/docker-hosted # NEXUS-DOCKER-ADDRESS/berth-desktop:latest +berth registry push-image ghcr v1 # ghcr.io/berth-desktop:v1 ``` pushes the image built by the **Container image** install method. diff --git a/docs/content/docs/(guide)/use/secrets.mdx b/docs/content/docs/(guide)/use/secrets.mdx index ca04435..9f98263 100644 --- a/docs/content/docs/(guide)/use/secrets.mdx +++ b/docs/content/docs/(guide)/use/secrets.mdx @@ -11,7 +11,7 @@ berth keeps secret values in its store and sends them to a host when you say so. | Secret values and registry passwords | berth's store, by default `~/.local/share/berth/berth.db` | The SQLite file is readable by you only. Values are plain unless the store is encrypted | | Which secrets and registries a host gets | The host's profile | Names only | -`berth store` says where the store is and whether it is encrypted. `berth store encrypt` seals the values with a key file. See [The store](/docs/use/store). With a [berth server](/docs/use/server), the values are kept there and this machine has only their names. +`berth server store status` says where the store is and whether it is encrypted. `berth server store encrypt` seals the values with a key file. See [The store](/docs/use/store). With a [berth server](/docs/use/server), the values are kept there and this machine has only their names. On the host everything belongs to the agent account, in plain files. The agent can read everything you send it. @@ -33,6 +33,8 @@ echo "$TOKEN" | berth secret set github Or use the **Secrets** tab of the home screen (`5`). Enter on **+ Add secret** asks for a name and the value, Enter on a secret replaces its value, `x` twice removes it. Values are never shown. +The secret `tailnet_api_key` is berth's own, for making the keys hosts join the tailnet with, and is never given to a host. See [joining the tailnet](/docs/use/access#joining-the-tailnet) for what it holds. + @@ -41,8 +43,8 @@ Or use the **Secrets** tab of the home screen (`5`). Enter on **+ Add secret** a As an environment variable or as a file, with an optional mode: ```bash -berth host studio secret github env GH_TOKEN -berth host studio secret deploy-cert file ~/.config/acme/cert.pem 0600 +berth host set studio secret.github=GH_TOKEN +berth host set studio secret.deploy-cert=~/.config/acme/cert.pem:0600 ``` Or under **Secrets** on the host's **Settings** tab, where you type a variable name such as `GH_TOKEN` or a path such as `~/.config/app/token`. @@ -63,7 +65,7 @@ Or **Secrets and registry logins** under Sync on the overview. -Check what a host is given with `berth host studio`, and stop giving one with `berth host studio secret github off`. List the stored names with `berth secret`, and delete a value with `berth secret remove github`. +Check what a host is given with `berth host show studio`, and stop giving one with `berth host set studio secret.github=`. List the stored names with `berth secret list`, and delete a value with `berth secret remove github`. ## On the host @@ -77,7 +79,7 @@ Create a fine-grained token limited to the repositories the agent should work on ```bash berth secret set github -berth host studio secret github env GH_TOKEN +berth host set studio secret.github=GH_TOKEN berth run studio secrets ``` diff --git a/docs/content/docs/(guide)/use/server.mdx b/docs/content/docs/(guide)/use/server.mdx index c9111bf..b3d3c8d 100644 --- a/docs/content/docs/(guide)/use/server.mdx +++ b/docs/content/docs/(guide)/use/server.mdx @@ -4,7 +4,7 @@ description: Run berth serve on one machine and use its hosts from berth on othe icon: ServerCog --- -A server machine keeps the hosts with their profiles and secrets, and runs `berth serve`. berth on another machine, such as your laptop, then uses that server's hosts instead of its own: the terminal UI, `berth list`, `berth run` and the other host commands go through the server's [portal](/docs/api/portal). +A server machine keeps the hosts with their profiles and secrets, and runs `berth serve`. berth on another machine, such as your laptop, then uses that server: the [screens](/docs/use/terminal-ui), `berth ls`, `berth run` and every other command call the server's API through its [portal](/docs/api/portal). On macOS and Windows berth is only that: a client of a server. ```text your laptop the server hosts @@ -23,10 +23,10 @@ docker run -d --name berth -p 127.0.0.1:7777:7777 \ -v berth-config:/config -v berth-data:/data \ -v ~/.ssh:/home/berth/.ssh:ro \ ghcr.io/helightdev/berth -docker exec berth cat /data/berth/serve.token +docker exec berth berth server setup-key ``` -Without the image, run `berth serve --listen 0.0.0.0:7777 --expose` where berth is installed. The token is in `~/.local/share/berth/serve.token`. Set the hosts up on the server as usual, with the terminal UI there or `docker run --rm -it --entrypoint berth …` for the container. +Without the image, run `berth serve --listen 0.0.0.0:7777 --expose` where berth is installed. The last command makes a [setup key](#with-a-setup-key), which hands the server's token to its first user. The server keeps its state in one store, a SQLite file by default. [The store](/docs/use/store) says how to keep it in PostgreSQL, encrypt it and restore the server on another machine from it. @@ -42,54 +42,38 @@ sudo tailscale serve --bg --https=443 http://127.0.0.1:7777 ## Connect to it - - - -`--server` and `--token` apply to one run and leave your own hosts and settings alone: +`berth login` keeps the server and its token, and every command after it uses that server: ```bash -berth --server https://berth.example.ts.net --token "$(cat token)" # the terminal UI -berth --server https://berth.example.ts.net --token "$(cat token)" list -berth --server https://berth.example.ts.net --token "$(cat token)" run studio status +berth login https://berth.example.ts.net # asks for a setup key +berth ls +berth run studio status +berth logout # back to the berth of this machine ``` - - - -Store the address and token as settings, and berth uses the server on every run: +With the token in a file instead, `berth login https://berth.example.ts.net --token-file token` keeps it. For one command alone, give the server on its command line. That leaves the kept one alone: ```bash -berth config server https://berth.example.ts.net -berth config server_token "$(cat token)" -berth config server "" # back to the hosts of this machine +berth --server https://berth.example.ts.net --token-file token ls ``` -`BERTH_SERVER` and `BERTH_TOKEN` in the environment take precedence over the settings. - - - +`BERTH_SERVER` and `BERTH_TOKEN` in the environment take precedence over what `berth login` kept. A token never goes on the command line, where other users of the machine could read it. ### With a setup key A setup key lets the first user collect the token without reading `serve.token` on the server. Whoever installs the server, such as a coding agent, hands on the key and never sees the token. Make one where the server keeps its data: ```bash title="On the server" -docker exec berth berth setup-key # or: berth setup-key +docker exec berth berth server setup-key # or: berth server setup-key ``` -It prints the key. On your own machine, give it to berth when it asks: +It prints the key. On your own machine, give it to `berth login` when it asks, or on standard input with `--setup-key-stdin`: ```bash -berth connect https://berth.example.ts.net +berth login https://berth.example.ts.net ``` -berth checks the key, collects the token and saves `server` and `server_token`. Or open the server's page with the key in the link: - -```text -https://berth.example.ts.net/#setup-key=KEY -``` - -The page signs the browser in and shows the token, with the two `berth config` commands for a terminal. The key lasts 24 hours. It ends sooner when you choose **Done** on the page, or when `berth connect` has saved the token. Until then it's as good as the token, so keep it to yourself. A new key replaces the old one, and `berth setup-key --remove` ends it. A server with `serve_token` off has no token to hand out, so it doesn't make a key. +berth checks the key, collects the token and keeps the server and the token. The key lasts 24 hours, and ends once `berth login` has saved the token. Until then it's as good as the token, so keep it to yourself. A new key replaces the old one, and `berth server setup-key --revoke` ends it. A server with `serve_token` off has no token to hand out, so it doesn't make a key. | Address | Reaches | | --- | --- | @@ -97,133 +81,69 @@ The page signs the browser in and shows the token, with the two `berth config` c | `http://HOST:PORT` | A server's port directly | | `unix:/path/to/serve.sock` | A `berth serve` on this machine, without a token | -## In a browser - -With `--web`, `berth serve` also shows its hosts as a page. It needs the build of berth with the page in it. The container image is one, and the install script takes it with `BERTH_VARIANT=web` ([Building](/docs/reference/building#with-the-page-for-a-browser)). - -```bash title="On the server" -berth serve --listen 0.0.0.0:7777 --expose --web -``` - -Open the server's address in a browser and enter the token: - -``` -https://berth.example.ts.net/ -https://berth.example.ts.net/#token=TOKEN # a link that signs in -``` - -To open the page on a phone, choose **Open on another device** on the Settings tab of the home screen. It shows a QR code and the link with the token in it, so anyone who scans it gets in. The row is there when berth reaches the server with its token, such as `server_token`, and in the page for the role `admin`. From the terminal UI the link opens the server's own page, which needs `--web`. From the page it opens the same page you are on, such as the one at /web. - -The page is the terminal UI drawn in the browser. It has the host list and the overview of each host, where tasks run and sessions open in the terminal panel. The browser keeps the token until **o** on the host list signs out. A token in a link is taken out of the address bar when the page opens. - -With the role `admin` the page also sets the server up, with the same screens as the terminal UI: +## On other systems -- **Provisioners**, **Settings**, **Registries**, **Secrets** and **Plugins** are tabs of the home screen. On **Plugins**, `u` sends a `.tgz` from the device. A provisioner's page opens the pages of what berth made with it ([Resources](/docs/deploy/resources#in-the-terminal-ui)). -- **New host** starts a profile, and a host's overview opens its Install, Features, Settings and Run tabs. A new host opens on Install with **Create a machine** chosen, and offers only Create a machine, Cloud-init seed and Existing host over SSH. -- **Run** does what the chosen method does on the server. That is creating the machine, a cloud-init seed to copy, or an apply over SSH. The output is shown as it runs. -- **Install secrets and registry logins** sends a host what its profile names. - -Some keys differ: - -| In the page | Instead of | -| --- | --- | -| **Alt+Q** leaves a running session | `Ctrl+Q`, which closes some browsers | -| **Ctrl+V** and the browser's paste | The desktop clipboard tools | -| A click on a link opens a tab | `xdg-open` | -| **g** generates and **p** takes a password to hash | `F4` and `F5`, which reloads a page | - -A password the server makes up is shown once in the page. The server keeps a copy in its store, and `berth credentials HOST` shows it again to an admin of the server. - -What needs your own machine stays in the terminal UI. That is the USB installer and the container image, values read from your local keys, and sending a plugin's folder. - -Keys the browser keeps for itself, such as `Ctrl+W` and `Ctrl+T`, never reach a shell in the page. A phone can tap rows and key hints, but has no keyboard for a terminal. - -### Without `--web` - -The same page is part of this site, at [/web](/web/). It asks which server to use, or takes it from its address: - -``` -/web/?server=https://berth.example.ts.net -/web/?server=https://berth.example.ts.net#token=TOKEN -``` - -Your browser talks to the server you name, and to nothing else. It keeps the server and the token. A page loaded over `https` can only reach a server over `https`. - -## With berth-remote - -berth runs on Linux. `berth-remote` uses the hosts of a server from a terminal on Windows, macOS or Linux. It shows the screens of the page, and works with a server that runs without `--web`. Install it on Windows in PowerShell: +berth on macOS and Windows is the client of a server: the screens and the commands, which all call a server. Install it on Windows in PowerShell: ```powershell irm https://raw.githubusercontent.com/helightdev/berth/main/install.ps1 | iex ``` -On macOS or Linux: +On macOS: ```bash -curl -fsSL https://raw.githubusercontent.com/helightdev/berth/main/install-remote.sh | sh +curl -fsSL https://raw.githubusercontent.com/helightdev/berth/main/install.sh | sh ``` -Each script downloads the latest release for this machine, checks it against its SHA-256 and installs it, to `%LOCALAPPDATA%\Programs\berth-remote` on Windows and to `~/.local/bin` elsewhere. `BERTH_INSTALL_DIR` and `BERTH_VERSION` work as for `install.sh`. To download it by hand, take the archive for your system and its `.sha256` file from the [latest release](https://github.com/helightdev/berth/releases/latest). +Each script downloads the latest release for this machine, checks it against its SHA-256 and installs it, to `%LOCALAPPDATA%\Programs\berth` on Windows and to `~/.local/bin` on macOS. `BERTH_INSTALL_DIR` and `BERTH_VERSION` work as on Linux. To download it by hand, take the archive for your system and its `.sha256` file from the [latest release](https://github.com/helightdev/berth/releases/latest). -Then start it with the server's address. On Windows, use Windows Terminal or the console of Windows 10 or later. +Then connect it with a setup key, and open the screens. On Windows, use Windows Terminal or the console of Windows 10 or later. ```bash -berth-remote --server https://berth.example.ts.net +berth login https://berth.example.ts.net +berth ui ``` -It asks for the token and keeps the server and the token for the next run. A token never goes on the command line: +berth keeps the server and the token in the folder of [berth's settings](/docs/use/terminal-ui#your-machine-and-the-server). When none is kept, the screens open with a form for a server's address and token. -| Variable or option | Effect | +| Variable | Effect | | --- | --- | +| `BERTH_SERVER` | The server, for this run | | `BERTH_TOKEN` | The token, so it isn't asked for | -| `BERTH_SETUP_KEY` | A [setup key](#with-a-setup-key), which collects the token | -| `BERTH_SERVER` | The server, instead of `--server` | -| `--forget` | Remove the kept server and token, and keep nothing from this run | -| `BERTH_REMOTE_FILE` | The plugin package to send, where there is no file dialog | -| `--version` | Say the version | - -The server and the token are in `settings.json` in a folder of your own: +| `BERTH_FILE` | The plugin package the screens send, where there is no file dialog | -| System | Folder | -| --- | --- | -| Windows | `%LOCALAPPDATA%\berth-remote`. Its access list lets in you, SYSTEM and the administrators | -| macOS | `~/Library/Application Support/berth-remote`, which only you can open | -| Linux | `~/.config/berth-remote`, or under `$XDG_CONFIG_HOME`, which only you can open | +berth talks to a server over `https` only with a certificate that the system or the Mozilla roots trust. It opens only `http` and `https` links, in the system's browser. **o** on the host list signs out. -It talks to a server over `https` only with a certificate that the system or the Mozilla roots trust. It opens only `http` and `https` links, in the system's browser. +A password the server makes up is shown once. The server keeps a copy in its store, and `berth host credentials HOST` shows it again to an admin of the server. -The screens and keys are those of the page. **Ctrl+Q** quits, and `u` on **Plugins** opens the system's file dialog: on Linux that of zenity or kdialog. +What works with the server's own machine is offered there only. That is the USB installer, which writes a drive of that machine, and the settings that aren't open to the network. Elsewhere the screens list them as unavailable, with the reason. ## What works from another machine -| Works through the server | Stays on a machine with the host's profile | +Every command calls the server. Each works from another machine as it does on the server. These are the exceptions: they need the server's own machine, and say so elsewhere. + +| Needs the server's machine | Why | | --- | --- | -| `berth list`, and the host list of the home screen and of the page | USB installers: `berth iso` and writing a drive | -| The host's overview with its tasks, sessions and terminal panel | Applying to this machine or an inventory, and container images | -| The host commands such as `berth run`, `exec`, `attach` and `channel` | Publishing a release or an image by hand | -| The set-up tabs of a host, new hosts, and `berth remove` | | -| `berth profile` and `berth credentials` | | -| `berth apply`, `berth seed`, and the **Secrets and registry logins** task | | -| `berth provisioner`, `berth provision`, `berth instance`, `berth resource` and `berth deploy` | | -| `berth secret`, `berth registry` and `berth host` | | -| The Plugins tab and `berth plugins` | | - -berth keeps the server's hosts apart from its own. Its own hosts are hidden while a server is set and return when the setting is cleared. +| Writing the USB installer to a drive | The drive is a device of that machine | +| `berth ssh HOST` | It uses that machine's SSH configuration and keys | +| `berth registry publish` and `berth registry push-image` | They upload files of that machine | +| `berth server …` and `berth serve` | They are the server itself | +| The settings that aren't open to the network | The server keeps them to its own users | ### Setting the server up from here -With the role `admin` on the server, which its token has, berth sets the server up as if the hosts were its own. What you change is saved on the server: +With the role `admin` on the server, which its token has, berth sets the server up from anywhere. What you change is saved on the server: -- A host's set-up tabs, a new host from the home screen, and `berth profile HOST --set FILE` save the profile there. -- A password generated on the set-up tabs is made and kept by the server. `berth credentials HOST` asks the server for it. -- **Apply this configuration** and `berth apply HOST` run as a job on the server. Its output is shown as it comes. -- `berth seed HOST` has the server render the cloud-init seed, and writes it here. -- `berth provision HOST` and `berth instance HOST` use the server's provisioners and secrets, and create the machine from the server ([Provisioners](/docs/deploy/provisioners#with-a-berth-server)). `berth provisioner` shows the server's provisioners, and their actions run there. -- `berth resource` and `berth deploy` use the server's resources. A deployment's specification is sent to the server and kept there, and it gets the server's secrets ([Deployments](/docs/deploy/deployments#with-a-berth-server)). -- A secret's value is sent to the server and kept only there. Here berth knows its name. -- The `tailnet_` settings and the secret `tailnet_api_key` are the server's, which creates the [enrollment keys](/docs/use/access#automatic-enrollment-keys) for its hosts. Settings and `berth config` change them there. +- A host's set-up tabs, `berth host create`, `berth host set` and `berth host import` save the profile there. +- A password the server generates is kept there. `berth host credentials HOST` asks the server for it. +- `berth apply HOST` and **Apply this configuration** run as a job on the server. berth follows its output as it comes ([Jobs](/docs/use/cli#jobs)). +- `berth host seed HOST` has the server render the cloud-init seed, and writes it here. +- `berth host create HOST --provisioner NAME`, `berth host start` and `berth host stop` use the server's provisioners and secrets ([Provisioners](/docs/deploy/provisioners#with-a-berth-server)). +- `berth resource` uses the server's resources. A deployment's specification is sent to the server and kept there, and it gets the server's secrets ([Deployments](/docs/deploy/deployments#with-a-berth-server)). +- A secret's value is sent to the server and kept only there. +- The `tailnet_` settings and the secret `tailnet_api_key` are the server's, which creates the [keys its hosts join with](/docs/use/access#joining-the-tailnet). Settings and `berth config set` change them there. - A plugin is installed on the server, and its features show on the Features tab of the server's hosts. A `.tgz` or folder of yours is sent there first. -- The server logs in to hosts with the keys of [its own key ring](/docs/use/accounts#keys), by default its key `berth`, which a host it installs takes. `berth key` and its commands act on the server's ring, and its private halves stay there. For a host that was set up elsewhere, add the key `berth` to the operator first. Settings lists the keys and copies one to the clipboard, and **Log in with** on a host's Install tab chooses among them. -- `berth key rotate` replaces a key of the server's on the hosts that take it, as a job of the server ([Rekeying](/docs/use/accounts#rekeying)). First contact and accepting a changed host key are done on the server, with `berth key trust HOST` and `berth key host HOST --accept` there. +- The server logs in to hosts with the keys of [its own key ring](/docs/use/accounts#keys), by default its key `berth`. `berth key` acts on the server's ring, and its private halves stay there. `berth key trust HOST` and `berth host verify HOST --accept` make first contact and accept a changed host key from the server. +- `berth key rotate` replaces a key on the hosts that take it, as a job of the server ([Rekeying](/docs/use/accounts#rekeying)). With a lesser role the set-up tabs say *This host is set up on the berth server at URL*, and the host's overview works as before. The roles, access tokens of an OAuth 2.0 issuer in place of the token, and how to keep set-up on the server itself are in [berth serve](/docs/extend/serve#who-gets-in). diff --git a/docs/content/docs/(guide)/use/sign-ins.mdx b/docs/content/docs/(guide)/use/sign-ins.mdx index 27b7838..dcf9f01 100644 --- a/docs/content/docs/(guide)/use/sign-ins.mdx +++ b/docs/content/docs/(guide)/use/sign-ins.mdx @@ -10,7 +10,7 @@ A new host has the agents installed but nobody signed in. Each sign-in runs on t Under **Sign-In**: -| Task | Signs in | +| Action | Signs in | | --- | --- | | Sign in to Claude Code | Claude Code, from the `claude` plugin | | Sign in to Codex | Codex, from the `codex` plugin | @@ -25,7 +25,7 @@ berth run studio claude:login berth run studio sign-in:status ``` -Pi has no sign-in task. Start `pi` in the terminal and complete its own sign-in. +Pi has no sign-in action. Start `pi` in the terminal and complete its own sign-in. ## GitHub without signing in diff --git a/docs/content/docs/(guide)/use/store.mdx b/docs/content/docs/(guide)/use/store.mdx index b2f60e2..547e0e8 100644 --- a/docs/content/docs/(guide)/use/store.mdx +++ b/docs/content/docs/(guide)/use/store.mdx @@ -15,13 +15,13 @@ berth keeps its state in one store: the hosts, their passwords, secrets, berth's | `store_ca` | A PEM file of certificate authorities to trust as well as the public ones, for a database with a private one | ```bash -berth config store "/srv/berth/berth.db" -berth config store "" # back to berth.db in the data directory +berth config set store "/srv/berth/berth.db" +berth config set store "" # back to berth.db in the data directory ``` `BERTH_STORE`, `BERTH_STORE_KEY` and `BERTH_STORE_CA` in the environment take precedence over the settings. These three settings stay in `config.json`, as berth needs them to open the store. -With `sslmode=require`, berth connects over TLS and checks the server's certificate and its host name. `verify-ca` and `verify-full` do the same. Without an `sslmode`, or with any other, the connection is plain. A URL can contain the password, and `berth config store URL` shows it in the process list for a moment. Write it into `config.json` yourself to avoid that. +With `sslmode=require`, berth connects over TLS and checks the server's certificate and its host name. `verify-ca` and `verify-full` do the same. Without an `sslmode`, or with any other, the connection is plain. A URL can contain the password, and `berth config set store URL` shows it in the process list for a moment. Write it into `config.json` yourself to avoid that. ## Encrypt it @@ -29,10 +29,10 @@ Without a key, every secret, password and key in the store is readable to anyone | Command | Does | | --- | --- | -| `berth store` | Says where the store is, and how many of its values are sealed | -| `berth store encrypt` | Makes the key file and seals every value with it | -| `berth store rekey` | Seals the values with a new key, and keeps the previous one beside it as `store.key.old` | -| `berth store decrypt [--yes]` | Writes the values plain again and moves the key file aside to `store.key.old` | +| `berth server store status` | Says where the store is, and how many of its values are sealed | +| `berth server store encrypt` | Makes the key file and seals every value with it | +| `berth server store rekey` | Seals the values with a new key, and keeps the previous one beside it as `store.key.old` | +| `berth server store decrypt [--yes]` | Writes the values plain again and moves the key file aside to `store.key.old` | Stop `berth serve` before you run them, as it would go on sealing with the key it loaded. The inbox, the jobs and the key-value store of `berth serve` are not sealed. @@ -58,12 +58,12 @@ On a machine whose plugin directory lacks plugins of the store, berth writes the ## Move to another database -`berth store copy` copies everything into another store, such as from the SQLite file to PostgreSQL: +`berth server store copy` copies everything into another store, such as from the SQLite file to PostgreSQL: ```bash -berth store copy - <<< "postgres://berth:PASSWORD@db.example.org/berth?sslmode=require" -berth store copy /srv/berth/berth.db -berth store copy --target-file ~/berth-database-url +berth server store copy - <<< "postgres://berth:PASSWORD@db.example.org/berth?sslmode=require" +berth server store copy /srv/berth/berth.db +berth server store copy --target-file ~/berth-database-url ``` `-` reads the target from standard input and `--target-file` from a file's first line. A URL as an argument shows up in the process list. The copy honours `store_ca`. @@ -73,7 +73,7 @@ berth store copy --target-file ~/berth-database-url - Values are copied as they are, sealed or plain, and keep their times. - What this berth knows of berth servers is left out, as their next mirror fills it again. -The copy does not switch. When it's done, switch with `berth config store TARGET` or `BERTH_STORE`. With an encrypted store, the key file has to be wherever the store is used. +The copy does not switch. When it's done, switch with `berth config set store TARGET` or `BERTH_STORE`. With an encrypted store, the key file has to be wherever the store is used. ## Restore on a new machine @@ -103,7 +103,7 @@ For PostgreSQL, set `store` to the database's URL. For SQLite, copy the file and ### Start it -Run `berth list` or start `berth serve`. The hosts, secrets and settings are there. berth writes its SSH key and the token back. It logs in to the hosts with the same key, and clients of `berth serve` stay connected. It reinstalls your own plugins with npm. +Run `berth ls` or start `berth serve`. The hosts, secrets and settings are there. berth writes its SSH key and the token back. It logs in to the hosts with the same key, and clients of `berth serve` stay connected. It reinstalls your own plugins with npm. @@ -129,4 +129,4 @@ docker run -d --name berth -p 127.0.0.1:7777:7777 \ ghcr.io/helightdev/berth ``` -The variables are visible to anyone who can inspect the container. To keep the password out of them, write `store` into `/config/berth/config.json` instead. Mount the key's folder rather than the file alone, as a rekey writes beside it. To encrypt a new store, run `docker exec berth berth store encrypt` with `BERTH_STORE_KEY` set, after stopping the server. +The variables are visible to anyone who can inspect the container. To keep the password out of them, write `store` into `/config/berth/config.json` instead. Mount the key's folder rather than the file alone, as a rekey writes beside it. To encrypt a new store, run `docker exec berth berth server store encrypt` with `BERTH_STORE_KEY` set, after stopping the server. diff --git a/docs/content/docs/(guide)/use/terminal-ui.mdx b/docs/content/docs/(guide)/use/terminal-ui.mdx index d152323..6c6455d 100644 --- a/docs/content/docs/(guide)/use/terminal-ui.mdx +++ b/docs/content/docs/(guide)/use/terminal-ui.mdx @@ -1,28 +1,44 @@ --- title: The terminal UI -description: The screens of berth's terminal UI and the keys that move between them. +description: The screens of berth, the server they use, and the keys that move between them. icon: AppWindow --- -`berth` with no arguments opens the home screen. `berth NAME` opens a host directly: its overview when it has an SSH address, its set-up tabs when it is new. +`berth` with no arguments opens the home screen. `berth NAME` opens a host directly, on its overview. `berth ui` and `berth ui NAME` do the same. ```bash berth # home screen: pick a host or create one berth studio # open a host directly ``` +The screens do everything through the API of a berth server, the same one on every system. Which server that is: + +| When | The screens use | +| --- | --- | +| `--server`, or `BERTH_SERVER` is set | That server, for this run | +| A server is kept by `berth login` | That server ([A berth server](/docs/use/server)) | +| Neither, and a `berth serve` of yours runs on this machine | Its socket, with no token | +| Neither, on Linux | A server inside berth itself, for as long as the screens are open | +| Neither, on macOS or Windows | The Connect dialog, for a server's address and its token | + +The server inside berth is the one `berth serve` runs, served to that berth alone over pipes in memory. It's yours, so it takes no token. It doesn't take in the hosts' channels, which is what `berth serve` is for. A host whose sudo asks for a password has it asked for in the terminal before the screens open. + +A job that still runs when you quit, such as an apply, would end with berth. berth waits for it instead and says so. Ctrl+C there stops it unfinished. With `berth serve` running, the screens use it, and its jobs go on after they close. + ## The home screen The home screen has six tabs, switched with the number keys, or with Tab and Shift+Tab in this order. | Tab | Holds | | --- | --- | -| `1` Hosts | The saved hosts and their addresses. With a berth server, a dot shows whether each host answered the server, and the Information panel says why not. A host that still takes the old key of a [key rotation](/docs/use/accounts#rekeying) is marked `key rotation pending`. `n` creates one, `d` makes the selected one the default, `r` asks the server again, or reads the saved hosts again when there is no server | -| `2` Provisioners | The provisioners of the plugins, and whether each can be used here. Enter opens the page of one, with how it is and what berth made with it, grouped by kind. Enter on one of its actions runs it. Enter on something it made opens that [resource's page](/docs/deploy/resources#in-the-terminal-ui), with its state, its actions and its log. Esc goes back a page ([Provisioners](/docs/deploy/provisioners#a-provisioners-page)). With a server set, the server's | -| `3` Settings | The settings of `berth config`, the SSH keys of berth's key ring to copy, and with a berth server a QR code that opens its page on a phone. With a server, the `tailnet_` settings are the server's | +| `1` Hosts | The server's hosts and their addresses, with a dot for whether each answered and why not. A host that still takes the old key of a [key rotation](/docs/use/accounts#rekeying) is marked `key rotation pending`. `n` creates one, `d` makes the selected one the default, `x` twice removes one, `r` asks again | +| `2` Provisioners | The provisioners of the plugins, and whether each can be used. Enter opens the page of one, with how it is and what berth made with it, grouped by kind. Enter on one of its actions runs it. Enter on something it made opens that [resource's page](/docs/deploy/resources#in-the-terminal-ui), with its state, its actions and its log. Esc goes back a page ([Provisioners](/docs/deploy/provisioners#a-provisioners-page)) | +| `3` Settings | The settings of `berth config list` and the SSH keys of the server's key ring to copy. Over the network, only some settings change there; on the server's own machine, every one | | `4` Registries | npm, Docker, file and Nexus registries ([Registries](/docs/use/registries)) | -| `5` Secrets | Secret values, kept in berth's store ([Secrets](/docs/use/secrets)) | -| `6` Plugins | Built-in and installed plugins, and adding, updating and removing your own ([Plugins](/docs/extend/plugins#updating-a-plugin)). With a server set, the server's | +| `5` Secrets | Secret values, kept in the server's store ([Secrets](/docs/use/secrets)) | +| `6` Plugins | Built-in and installed plugins, and adding, updating and removing your own ([Plugins](/docs/extend/plugins#updating-a-plugin)) | + +The tabs after Hosts are for the role `admin`, which you have on your own machine. @@ -34,10 +50,20 @@ Each tab depends only on the tabs before it. Install holds what you have to choo | Set-up tab | For | | --- | --- | -| `1` Install | How to install, with the host profile (not for a created machine, which is always `cloud`), the provisioner and its required arguments, the SSH address, the hostname and Tailscale | +| `1` Install | How to install, with the target (not for a created machine, which is always `cloud`), the provisioner and its required arguments, the SSH address, the hostname and Tailscale | | `2` Features | The coding agents and T3 Code, with tool categories, browsers, graphics, Cockpit, Docker on the host and plugin features | | `3` Settings | Everything with a default. See the list below | -| `4` Run | The last files for the method, every problem grouped by tab, warnings, and the Run, Create machine and Save buttons | +| `4` Run | The last fields for the method, every problem grouped by tab, warnings, and the Run, Create with PROVISIONER and Save buttons | + +The ways to install a host: + +| Method | Run does, on the server | +| --- | --- | +| Create a machine | Has a provisioner create it, and sets it up ([Provisioners](/docs/deploy/provisioners)) | +| USB installer | Builds the unattended installer and writes it to a USB drive, on the server's own machine ([USB installer](/docs/deploy/usb)) | +| Cloud-init seed | Renders the seed to paste into a provider's user data field ([Cloud](/docs/deploy/cloud)) | +| Existing host over SSH | Applies the profile over SSH, or installs the controller alone on a managed host | +| Existing host, SSH only | Saves the host, which berth only logs in to ([Host types](/docs/use/types)) | The Settings tab has these groups, in this order: @@ -45,14 +71,14 @@ The Settings tab has these groups, in this order: | --- | --- | --- | | Machine | The provisioner's CPUs, memory and disk | Create a machine | | Provisioner Options | The provisioner's other arguments | Create a machine | -| Operator Account, Agent Account | The accounts. Container image has only the agent's username | Every method | -| Access | SSH hardening, and Recovery SSH CIDR | Every method but Container image. No recovery CIDR for Create a machine | +| Operator Account, Agent Account | The accounts | Every method | +| Access | SSH hardening, and Recovery SSH CIDR | Every method. No recovery CIDR for Create a machine | | Registries, Secrets | The logins and secrets the host gets | Every method | -| Desktop | Desktop size | Every method but Container image | +| Desktop | Desktop size | Every method | | Installer | Disk selection, keyboard layout and Wi-Fi | USB installer | | Release | Where the release is published | Cloud-init seed | -A row shows only when the method and features it belongs to are chosen. Each of the first three tabs lists its own problems at the bottom. Run lists all of them, grouped by tab, and does not start while one is listed. +A row shows only when the method and features it belongs to are chosen. Each of the first three tabs lists its own problems at the bottom. Run lists all of them, grouped by tab, and doesn't start while one is listed. The **Information** panel on the right explains the selected row. **Apply this configuration** on the overview sends changes to a running host. @@ -64,47 +90,30 @@ The **Information** panel on the right explains the selected row. **Apply this c | --- | --- | | Up, Down, Enter | Select a row. Enter edits the row or runs it | | Tab, Shift+Tab, number keys | Next or previous tab, or a tab by number; also accepts the current edit | -| Right or F2, Left | Into the *Suggestions* list (detected keys, hosts, choices) and back | -| F3 | Browse for a file on path fields; Ctrl+Enter picks the directory itself | -| F4 | Generate a value, such as a hostname, a password hash or a Tailscale key, or download the ISO | -| F5 | Type a password and store its hash; on Wi-Fi fields, unlock the saved passwords | +| F4 | Generate a value, such as a hostname or a password hash | +| F5 | Type a password and store its hash | | c | On a key of the key ring, such as **This berth's SSH key**, copy its public half to the clipboard | | Alt+R | In a panel, show its links as QR codes to scan with a phone. Left and Right go to the other links, any other key closes it. Where Alt+R is taken by other software, click its hint at the bottom of the panel | +| Ctrl+O | Copy the text of the field being edited | | Ctrl+S | Save the profile, as **Save** on the Run tab does | | Ctrl+U, Esc | Clear the field being edited, cancel the edit | | x or Delete | Unset the selected text, path or secret field | | q or Esc | From set-up back to the overview, from there to the home screen; asks again when there are unsaved changes | +| q | On the hosts of the home screen, quit | +| Ctrl+Q | In a session, leave it running. Anywhere else, quit | -Tabs, rows, buttons and suggestions take mouse clicks, and lists scroll with the wheel. +Tabs, rows and buttons take mouse clicks, and lists scroll with the wheel. On Plugins, `u` opens the system's file dialog to send a `.tgz` to the server. On Linux that is zenity's or kdialog's; without one, `BERTH_FILE` names the package. -## What berth reads from your machine +## Your machine and the server -berth fills suggestions and blank fields from what it finds locally. It reads, and never changes: +The screens take what they show from the server. Of your machine they use the clipboard and the file you choose to send. The server reads what its own machine has: its keys, its plugins and its USB drives. A path in a profile, such as the Ubuntu ISO of a USB installer, is a path on the server's machine. - - +The server and its token are kept as the settings `server` and `server_token`, in `config.json` of berth's folder of settings. Only you can read the file. -Every `.pub` file in `~/.ssh`, keys held by ssh-agent, the ways to log in with the keys of berth's key ring, and `Host` aliases from the SSH configuration, including `Include` files. - - - - -The Linux machines in `tailscale status`, as address suggestions. With the secret `tailnet_api_key` or `TS_API_KEY` set, F4 on the Tailscale key field creates a one-use enrollment key through the API of Tailscale or of the Headscale server in `tailnet_login_server`. - - - - -The names of networks saved in NetworkManager. Their passwords only after F5 on a Wi-Fi field, which asks for your password, passes it to `sudo` once and keeps that authorisation alive while berth runs. berth never asks for it on its own. - - - - -Connected USB disks, for **Then write to USB drive**. - - - - -Blank fields can be filled from `DEVBOX_HOSTNAME`, `DEVBOX_TARGET_DISK_SERIAL`, `DEVBOX_RELEASE_URL`, `DEVBOX_ISO` and `DEVBOX_INVENTORY`. +| System | Folder | +| --- | --- | +| Linux | `~/.config/berth`, or under `$XDG_CONFIG_HOME` | +| macOS | `~/Library/Application Support/berth` | +| Windows | `%LOCALAPPDATA%\berth`. Its access list lets in you, SYSTEM and the administrators | - - +The first time berth starts its screens it takes in the server and token that `berth-remote` kept, and removes that program's file. diff --git a/docs/content/docs/(guide)/use/types.mdx b/docs/content/docs/(guide)/use/types.mdx new file mode 100644 index 0000000..7a97f5f --- /dev/null +++ b/docs/content/docs/(guide)/use/types.mdx @@ -0,0 +1,110 @@ +--- +title: Host types +description: A host has a type, from a machine berth only logs in to up to a whole workstation. What each type allows, and how to add a host by its address. +icon: Layers +--- + +berth does not have to set a machine up to work with it. A host is anything berth reaches and logs in to, and its type says what berth does with it: + +| Type | What is on the host | What berth does with it | +| --- | --- | --- | +| `ssh` | Nothing of berth | Logs in over SSH for a terminal, commands and the inventory | +| `managed` | berthd and box | Also sessions, the channel, files, actions and secrets | +| `node` | A server's part of the profile: berthd, a firewall and plugin features | Also applying, roll-outs and plugin features, such as the engines that host machines ([Nodes](/docs/use/nodes)) | +| `workstation` | The whole profile | Also the desktop, the agents and the accounts, cloud seeds and the USB installer | + +The type is part of the host's [profile](/docs/reference/profile#type), as `type`. A profile that doesn't say is a workstation, so older profiles mean what they always did. What the host has right now is asked of it. The overview and the Hosts tab show both when they differ, such as for a controller installed by hand. + +A profile of berth 0.5 or earlier calls the type `level`, with the values `reachable`, `controller` and `workstation`. berth reads it with the names of now ([Older profiles](/docs/reference/profile#older-profiles)). + +## Capabilities + +| Capability | `ssh` | `managed` | `node` | `workstation` | +| --- | --- | --- | --- | --- | +| Terminal and commands over SSH, inventory | yes | yes | yes | yes | +| Install the controller alone | yes | update it | by applying | by applying | +| Sessions, channel, files, units, logs, processes | no | yes | yes | yes | +| The actions status, update and reboot | no | yes | yes | yes | +| Secrets and registry logins sent to the host | no | yes | yes | yes | +| Plugin features, their actions and services | no | no | yes | yes | +| Apply and roll-out | no | no | yes | yes | +| Cloud seed and USB installer | no | no | no | yes | +| Desktop and access links | no | no | no | yes | + +The screens, the commands and the API all use the same check in berth. An action the type rules out is still listed, as unavailable, with the reason and what changes the type. Asked for anyway, the server refuses it with `501 not_available`. A roll-out leaves hosts that aren't workstations and says how many it left. + +## Adding a machine that exists + +A machine you can already log in to is added by its address, and berth leaves it as it is. + +```bash +berth host add dock 192.0.2.4 --user ops +``` + +| Option | Default | For | +| --- | --- | --- | +| `--user USER` | Your SSH configuration | The SSH user | +| `--port PORT` | Your SSH configuration | The SSH port | +| `--auth AUTH` | `berth` | How berth [logs in](/docs/use/accounts#logging-in-to-hosts): `berth`, `key:NAME`, `password` or `ssh` | +| `--type TYPE` | `ssh` | The type it is meant to be | +| `--no-contact` | | Save it without logging in | + +After saving the host, berth logs in the way [first contact](/docs/use/accounts#first-contact) does. It tries its own key, and if the machine refuses it, it asks for a password or a private key that logs in once to install the key. It records the host's SSH host key and prints its fingerprint. When your `~/.ssh/known_hosts` knows the address and the server runs on your machine, berth takes the host key from there. With a [berth server](/docs/use/server) elsewhere, the host is saved on the server, which logs in with its own keys and records the key the host shows it. [Host keys](/docs/use/accounts#host-keys) has more. + +Then berth asks the host what it is. A machine that berth set up before has a controller that says its type, and berth offers to change the profile here when the host has more than `--type` gave. Without a terminal it prints the command. + +A workstation or a node keeps its profile on the host as well. berth offers to [adopt](/docs/use/moving) it from there: + +```bash +berth host adopt dock +``` + +A host with berth's controller alone has nothing more to keep, and berth offers to save its type: + +```bash +berth host set dock type=managed +``` + +`berth host set HOST type=TYPE` only changes the profile here. A host's page says the same when its controller reports more than its profile says. berth refuses to install the controller alone on a host that is a workstation or a node, whatever its profile here says. It asks the host's controller first, and the playbook checks the host's own record and the profile it keeps. + +In the terminal UI, create a host on the home screen and choose **Existing host, SSH only** on the **Install** tab. The address and the login are all the set-up asks for, and **Run** saves the host. For a managed host, choose **Existing host over SSH** and set **Type** to `managed`. **Run** then installs the controller. + +Over the API, `hosts.save` takes a profile with only the type and `access`: + +```bash +curl --unix-socket "$XDG_RUNTIME_DIR/berth/serve.sock" -X PUT \ + -d '{"type": "ssh", "access": {"address": "192.0.2.4", "user": "ops"}}' \ + http://berth/v1/hosts/dock/profile +``` + +The host's `trust-key` action then installs the server's key, with a password used once in its arguments ([Actions](/docs/extend/serve#actions)). + +## A server for machines + +A server that hosts Incus or Docker machines is a [node](/docs/use/nodes). berth applies it like a workstation, without the desktop and the accounts, and its features install the engines and register their provisioners. A server that the recipes set up before is taken in the same way. + +## Installing the controller + +```bash +berth run dock install-controller +``` + +The controller is berthd and box with the files they read, from the same part of the role that a full apply runs. Nothing else of a workstation is installed: no accounts, desktop, agents or plugins. The SSH user is the account sessions run as, and its home is where the file calls may go. The host's type becomes `managed`, and its actions are status, update, reboot and secrets. + +The same is the host's action **Install the controller** on its overview, and over the API as `install-controller`. On a managed host it's called **Update the controller**: run it after updating berth, as berth's own version is all such a host is compared with. A workstation's controller comes with applying its configuration, so the action is not offered there. + +berth also refuses it on a host that is a workstation or a node whatever its profile here says. It asks the host's controller first. The role checks again on the host before it changes anything. It stops when the host's record says workstation or node. It also stops when berthd there serves an account other than the SSH user. Installing the controller alone on such a host would take the workstation's controller apart. The overview and `berth action list` say when a host is more than its profile, and how to adopt it. + + + On a host with systemd the controller is a service that starts at boot. Without systemd, as in most containers, the controller is installed but nothing on the host starts it. berth says so and waits a minute for it to answer. The install fails when it doesn't. Something on the host then has to run `/usr/local/bin/berthd`, as the init of a container can. + + +## Changing the type + +| From | To | How | +| --- | --- | --- | +| `ssh` | `managed` | Install the controller | +| `ssh` or `managed` | `node` | `berth host set HOST type=node`, list its features, and apply ([Nodes](/docs/use/nodes)) | +| `ssh`, `managed` or `node` | `workstation` | Set **Type** to `workstation` in the set-up, fill in the rest of the tabs, and apply | + +berth never takes a host back to a type with less, and never removes what it installed. A node is not applied over a workstation that berth set up, as nothing would look after its desktop, agent and accounts any more. Making a profile a node's takes the fields of the agent and the operator out of it. diff --git a/docs/content/docs/(guide)/use/updates.mdx b/docs/content/docs/(guide)/use/updates.mdx index 82581c7..da2c7f5 100644 --- a/docs/content/docs/(guide)/use/updates.mdx +++ b/docs/content/docs/(guide)/use/updates.mdx @@ -14,8 +14,8 @@ Ubuntu security updates and Firefox's vendor updates install automatically. The | --- | --- | | **Update** on the overview, or `berth run HOST update` | `apt` update and `dist-upgrade`, then Claude Code, Codex, Pi, T3 Code and the Unity CLI to their latest versions, through their plugins | | **Reboot**, or `berth run HOST reboot` | Restarts the host to finish updates | -| **Apply this configuration** | Runs the role again, which brings new or changed features and settings, berthd and box, and plugin tasks and services | -| `berth plugins update` | Your plugins, to the latest version of their registry ([Updating a plugin](/docs/extend/plugins#updating-a-plugin)) | +| **Apply this configuration** | Runs the role again, which brings new or changed features and settings, berthd and box, and plugin actions and services | +| `berth plugin update` | Your plugins, to the latest version of their registry ([Updating a plugin](/docs/extend/plugins#updating-a-plugin)) | The versions in the defaults (`devbox_claude_version`, `devbox_codex_version`, `devbox_pi_version`, `devbox_t3_version`) are the least ones, so applying the configuration again does not go back to them after an update. Claude Code's own self-update is off. @@ -26,7 +26,11 @@ berth update --check # is there a newer release? berth update # download it, check its SHA-256 and replace this program ``` -`berth update` takes the same archive as the install script. It offers only full releases, not pre-releases such as `0.4.0-rc.1`, and moves from a pre-release to its release. A container is updated by pulling a newer image. `berthd` and `box` travel inside `berth`. A host gets the new ones when you apply its configuration. Its overview says when its tools are of another version than your berth. `BERTH_RELEASES` names a mirror of the releases page. +`berth update` takes the same archive as the install script. It offers only full releases, not pre-releases such as `0.4.0-rc.1`, and moves from a pre-release to its release. A container is updated by pulling a newer image. `berthd` and `box` travel inside `berth`. A host gets the new ones when you apply its configuration. Its overview says when its tools are of another version than your berth. + +A berthd of berth 0.5 says the host's type by its older name, and berth reads it. Its sessions, files and `berth exec` work. Its actions and port forwarding need the berthd of API 2: until then `berth ls` marks the host `old controller`, and the host's page and its actions say what updates it, `berth apply HOST` or, for a managed host, `berth run HOST install-controller`. An older berth that meets a newer berthd is out of date: update it. It refuses the profile a newer berth left on the host rather than misread it. `berth update --from URL` takes the releases from a mirror instead, and the setting `releases` keeps one. + +A client refuses a server with an older API, and says to update the server first. [Upgrading from 0.5](/docs/use/upgrading) has the order for a server, its clients and its hosts. ## Hosts that are behind @@ -37,16 +41,16 @@ A host applied by an older berth, or with an older berthd, says nothing yet. App ## Bringing hosts up to date ```bash -berth apply --behind # every host that is behind -berth apply --behind --update --reboot # and run Update, and reboot where needed -berth plugins update --apply # update the plugins, then the same +berth rollout # every host that is behind +berth rollout --update --reboot # and run Update, and reboot where needed +berth plugin update && berth rollout # update the plugins, then the same ``` -The hosts are taken one at a time, by name. Each is applied and then checked. Its berthd has to answer with this berth's version and the host must no longer be behind. Its Status task has to end well too. Hosts that are current are skipped. The first host that fails stops the roll-out, and the hosts after it are not touched. +The hosts are taken one at a time, by name. Each is applied and then checked. Its berthd has to answer with this berth's version and the host must no longer be behind. Its Status action has to end well too. Hosts that are current are skipped. The first host that fails stops the roll-out, and the hosts after it are not touched. | Option | Does | | --- | --- | -| `--update` | Runs the host's Update task after applying. If it ends badly, that host is finished and the roll-out stops after it | +| `--update` | Runs the host's Update action after applying. If it ends badly, that host is finished and the roll-out stops after it | | `--reboot` | Reboots a host that says it needs it, waits up to ten minutes for it to answer again, then checks it. A host with running sessions is not rebooted, and the roll-out stops there | | `--force` | Reboots also while sessions run, which ends them | | `--all` | Applies the hosts that are current too | @@ -63,7 +67,7 @@ A task, service or channel topic a plugin no longer has is gone after applying. Applying does not restart desktops that are already running or turns an agent is in the middle of. After changing versions, a service's environment or unit files, restart the agent's user services in a quiet moment, or reboot. -Applying restarts berthd only when its program or unit changed, because a restart ends the sessions it holds. New tasks, services and channel topics only need a reload, which applying does. +Applying restarts berthd only when its program or unit changed, because a restart ends the sessions it holds. New actions, services and channel topics only need a reload, which applying does. ## What berth does not do diff --git a/docs/content/docs/(guide)/use/upgrading.mdx b/docs/content/docs/(guide)/use/upgrading.mdx new file mode 100644 index 0000000..dd06321 --- /dev/null +++ b/docs/content/docs/(guide)/use/upgrading.mdx @@ -0,0 +1,208 @@ +--- +title: Upgrading from 0.5 +description: What changed since berth 0.5, the order of the updates, and the way back. +icon: CircleArrowUp +--- + +This berth has one set of screens and one command line, both over a server's API. Many commands and names changed, and the store is converted the first time this berth opens it. Update in the order below, and make the backup first. + +## The order + +| Step | What | Why then | +| --- | --- | --- | +| 1 | Back up the store | berth 0.5 cannot read the store once this berth has converted it | +| 2 | Update the server | A client of this berth refuses a server of berth 0.5 | +| 3 | Update the clients | A client of berth 0.5 cannot work with the new server | +| 4 | Update the hosts' controllers | berth needs their actions, which the controller of berth 0.5 does not have | +| 5 | Make the server's machine a node, if it hosts Incus or Docker | Optional. It then gets `upgrade-server` for the next update | + +### 1. Back up + +For a server of the `berth-server` recipe, stop it and copy its folder: + +```bash +sudo docker stop berth +sudo tar -C /srv -czf ~/berth-0.5.tar.gz berth +sudo docker start berth +``` + +For a berth without a server, copy `~/.local/share/berth` and `~/.config/berth` while no berth runs. A store in PostgreSQL needs a dump, with the password in `~/.pgpass` rather than on the command line: + +```bash +pg_dump --format=custom --file=berth-0.5.dump --host=192.0.2.40 --username=berth berth +``` + +### 2. The server + +Fetch the recipe of the new version and run it with `berth_server_update`. It pulls the new image and makes the container again with the data and every option of the old one, such as the name `incus-host` and the Docker socket. The old container runs again if the new one does not come up: + +```bash +curl -fsSL https://github.com/helightdev/berth/archive/refs/heads/main.tar.gz \ + | tar xz --strip-components=1 --wildcards '*/recipes' \ + '*/provision/ansible/roles/devbox/files/berth-server-upgrade.py' +cd recipes/berth-server +sudo ./install.sh -e berth_server_update=true +``` + +Pass the options you gave the first run again, such as `-e berth_server_node=true`. Do not remove the container by hand. A new container from the recipe gets the recipe's options alone. [Update it](https://github.com/helightdev/berth/blob/main/recipes/berth-server/README.md#update-it) in the recipe's guide has the details. + +For a server in a container you made yourself, run the same helper with the version to run: + +```bash +sudo python3 provision/ansible/roles/devbox/files/berth-server-upgrade.py berth 0.6.0 +``` + +A `berth serve` that runs as a service is updated with `berth update`, then restarted. Its command line may keep `--web`, which does nothing now. + +### 3. The clients + +```bash +berth update +``` + +Install berth where berth-remote was used: the [installation](/docs/start/install) replaces it. berth takes in the server and token berth-remote kept, on the first command that needs a server: + +| berth has | What happens to berth-remote's settings | +| --- | --- | +| No server | Its server and token become berth's, and its file goes | +| The same server | Its token is taken when berth has none, and its file goes | +| Another server, or the same with another token | Its file stays. berth says so once, with `berth login` for the other server | + +A server that does not answer never costs berth the token it kept. Only `berth logout`, signing out in the screens, or a new token in place of one the server refused replaces it. + +### 4. The hosts + +`berth ls` marks each host whose controller is of berth 0.5 as `old controller`. Update them: + +| Host type | Command | +| --- | --- | +| `workstation` or `node` | `berth apply HOST` | +| `managed` | `berth run HOST install-controller` | + +Until then sessions, files and `berth exec` work, and the host's actions and its page in the screens say that the controller is older. + +### 5. The server's machine as a node + +The `incus-host` recipe is gone. A machine it set up becomes a [node](/docs/use/nodes). Run the server recipe with `berth_server_node`. Add the machine from a berth connected to that server, because the provisioners of the machine are that server's: + +```bash +sudo ./install.sh -e berth_server_node=true -e berth_server_update=true +berth host add big node --user root --type node \ + --feature docker/engine --feature incus/server +berth apply big +``` + +The apply makes the recipe's Incus provisioners the node's. It says so for each, and names any it left alone. The provisioner of the server's local Docker engine is one of those, as berth cannot tell that it is the node's engine. Make it the node's by hand, and apply once more: + +```bash +berth provisioner run docker adopt node=big +berth apply big +``` + +Use the name `berth provisioner list` shows for it. + +From then on, `berth run big upgrade-server version=0.7.0` updates the server. + +## What is converted, and when + +The first start of this berth on a store converts it, whether that is the server in its container or a command on your machine. + +| What | Becomes | +| --- | --- | +| The schema of the SQLite file or the database | The schema of this berth | +| `level` and `profile` in every profile | `type` and `target` | +| `controller` as a type | `managed` | +| Keys pasted into profiles to join the tailnet | Removed. berth makes the keys with the secret `tailnet_api_key` | +| A host berth 0.5 never joined to the tailnet itself | Keeps applying without the tailnet until berth can make its key: its profile gets `devbox_tailnet_optional` | + +The conversion happens once. A second start doesn't change the store. + +## Going back + +Before it converts a SQLite store, berth writes a copy of it beside the store, named for the version and schema it had, such as `berth.db.berth-0.5.2-schema-4`, and says where. Only the latest copy is kept. With no room for it, berth says so and converts the store without one. + +berth 0.5 cannot read a converted store, and in a container it stops again and again. To go back, stop the new berth, put the copy in place and start berth 0.5: + +```bash +sudo docker rm -f berth +cd /srv/berth/data/berth +sudo mv berth.db berth.db.converted +sudo rm -f berth.db-wal berth.db-shm +sudo mv berth.db.berth-0.5.2-schema-4 berth.db +``` + +Then run the recipe of berth 0.5 again, or `berth serve` of that version. What changed after the upgrade is lost. A store in PostgreSQL goes back from the dump of step 1 with `pg_restore`. berth doesn't copy it, and prints a line when it converts one. + +## Commands + +Each old spelling fails with the new one named, so a script does not do something it did not mean. `berth serve --web` is the exception: it works and says that `--web` does nothing. + +{/* old-spellings:start */} + +| berth 0.5 | Now | +| --- | --- | +| `berth list` | `berth host list, or berth ls` | +| `berth remove` | `berth host remove NAME` | +| `berth profile` | `berth host export NAME, berth host import NAME FILE or berth host set NAME KEY=VALUE` | +| `berth credentials` | `berth host credentials NAME` | +| `berth adopt` | `berth host adopt NAME [ADDRESS]` | +| `berth iso` | `berth host iso NAME` | +| `berth seed` | `berth host seed NAME` | +| `berth inventory` | `berth host inventory [NAME…]` | +| `berth provisioners` | `berth provisioner list` | +| `berth provision` | `berth host create NAME --provisioner PROVISIONER` | +| `berth instance` | `berth host start NAME, berth host stop NAME or berth host remove NAME --destroy` | +| `berth deploy` | `berth resource create NAME --provisioner PROVISIONER --spec FILE` | +| `berth actions` | `berth action list HOST` | +| `berth manage` | `berth run HOST ACTION` | +| `berth attach` | `berth session attach HOST [ID]` | +| `berth sessions` | `berth session list HOST` | +| `berth output` | `berth session logs HOST ID` | +| `berth call` | `berth service call HOST SERVICE METHOD` | +| `berth plugins` | `berth plugin list, add, remove or update` | +| `berth connect` | `berth login URL` | +| `berth setup-key` | `berth server setup-key` | +| `berth store` | `berth server store` | +| `berth openapi` | `berth server api openapi` | +| `berth asyncapi` | `berth server api asyncapi` | +| `berth operations` | `berth server api operations` | +| `berth publish` | `berth registry publish DESTINATION --host HOST` | +| `berth image` | `berth registry push-image DESTINATION` | +| `berth host install` | `berth host apply NAME for a managed host, or berth run HOST install-controller` | +| `berth host type` | `berth host set NAME type=TYPE` | +| `berth key new` | `berth key create NAME` | +| `berth key add` | `berth key import NAME FILE` | +| `berth key host` | `berth host verify NAME [--accept]` | +| `berth registry repo` | `berth registry add-repo NAME FORMAT REPOSITORY` | +| `berth apply --behind` | `berth rollout` | +| `berth apply --all` | `berth rollout --all` | +| `berth channel` | `berth channel listen HOST` | + +{/* old-spellings:end */} + +## Names + +| Before | Now | +| --- | --- | +| A host's `level` | Its `type` | +| The level `controller` | The type `managed` | +| A profile's `profile`, such as `laptop` | Its `target` | +| Tasks of a host | Its actions: `berth run HOST ACTION` | +| `berth connect URL` | `berth login URL` | +| berth-remote | berth: `berth ui` opens the screens | +| The page in the browser | The screens and the command line. The server's address shows a line about `berth login` | +| The `incus-host` recipe | A node with the features `incus/server` and `docker/engine` | +| `tailscale_auth_key` and `tailscale_auto_key` in a profile | The secret `tailnet_api_key`, from which berth makes each host's key | + +## API clients + +The API is number 2 now, and `GET /v1/version` says so in `api`. A client of berth's own refuses a server with an older number. A call that only berth 0.5 makes gets an error that names berth 0.5. + +| Before | Now | +| --- | --- | +| `level` in hosts and profiles | `type` | +| `profile` in a profile | `target` | +| `/v1/tasks`, `tasks.list`, `tasks.run` | `/v1/actions`, `actions.list`, `actions.run` | +| The server's `actions.act` | `hosts.act`, `installation.act`, `provisioners.act` and `resources.act` | + +The [API reference](/docs/api/serve) lists every operation. diff --git a/docs/content/docs/api/index.mdx b/docs/content/docs/api/index.mdx index 253abf5..6204b9d 100644 --- a/docs/content/docs/api/index.mdx +++ b/docs/content/docs/api/index.mdx @@ -11,7 +11,7 @@ berthd and `berth serve` answer the same kind of API. Each of their *operations* | berthd | every host, as root | unix sockets in `/run/berth` | [berthd](/docs/api/host) | | `berth serve` | your machine, or a server | a unix socket of your user, and optionally TCP | [berth serve](/docs/api/serve) | -`berth serve` passes berthd's operations through to the host it names, so a program that talks to `berth serve` reaches all hosts. It also lists what can be done to each host and to the installation as [actions](/docs/extend/serve#actions), and runs one as a job it keeps. +`berth serve` passes berthd's operations through to the host it names, under `/v1/hosts/NAME/controller`, so a program that talks to `berth serve` reaches all hosts. It does not pass through `endpoints.connect`, which it opens itself for `resources.connect`. It also lists what can be done to each host and to the installation as [actions](/docs/extend/serve#actions), and runs one as a job it keeps. ## Ways to call an operation @@ -27,6 +27,39 @@ Every operation page shows the portal frame and the HTTP request side by side. A *call* takes arguments and input and gives back a result. After it opens, a *stream* keeps sending frames, such as a session's output or new channel messages, and some streams take frames back, such as typed input. On the portal a stream is a subscription. See [Streams](/docs/extend/streams) for what all streams have in common. +## The API number + +`version` answers with `api`, a number that rises when something is added to the API. A client compares it to see what a host or a server can do. In `berth serve`, `hosts.list` gives each host's number too. + +| `api` | Added | +| --- | --- | +| 1 | The API as first released | +| 2 | What the list below names. It also renames, and breaks what a client of number 1 calls: see below | + +What number 2 adds: + +- `ports.connect`, with windows on the portal's streams +- Roles from the keys of an object claim +- The `dropped` frame of `events.follow`, `inbox.follow` and `kv.follow` +- Host and resource changes on `events.follow` +- The endpoints of a resource, and `resources.connect` to them +- `endpoints.connect` of berthd, which `berth serve` alone opens, to reach an endpoint of a resource on a node +- The `exec` action of a resource + +What number 2 renames, with no old name kept: + +| Number 1 | Number 2 | +| --- | --- | +| `tasks.list`, `tasks.run`, `tasks.choices` at `/v1/tasks…` | `actions.list`, `actions.run`, `actions.choices` at `/v1/actions…`, with the action's name as `action` | +| The schemas `Task`, `RunTask`, `PluginTask` and a feature's `tasks` | `HostAction`, `RunAction`, `PluginAction` and `actions` | +| A session's `task` | `action` | +| `actions.list`, `actions.act` of `berth serve` at `/v1/actions…` | `installation.actions`, `installation.act` at `/v1/installation/actions…` | +| A host's operations through `berth serve` at `/v1/hosts/NAME/…` | `/v1/hosts/NAME/controller/…`, so that none meets the server's own routes of a host | +| `level` of `version` and `hosts.get`, with `reachable`, `controller`, `workstation` | `type`, with `ssh`, `managed`, `workstation` | +| `level` and `profile` in a profile that `hosts.save` takes | `type` and `target` | + +berth reads what a berthd of number 1 says of the host's type. It needs number 2 for everything else, and says to update the controller or apply the host until it has it. + ## Who may call berthd answers on two sockets, and each operation says which may use it: @@ -48,6 +81,8 @@ Over HTTP an error is a status code with a JSON body; on the portal the same obj { "error": { "code": "not_found", "message": "No session k3v9qz2m8w1x", "status": 404 } } ``` +Over HTTP the header `berth-error` repeats the code, and `berth-error-message` the start of the message, percent-encoded. A WebSocket client that is refused during the handshake often sees only the head of the answer, so it reads both there. + | Status | Code | Means | | --- | --- | --- | | 400 | `bad_request` | An argument or the input is missing or malformed | @@ -72,11 +107,11 @@ The reference is generated from documents that the programs print and the tests | Document | Printed by | Download | | --- | --- | --- | | Operations of berthd | `berthd operations` | [berthd.operations.json](/openapi/berthd.operations.json) | -| Operations of berth serve | `berth operations` | [berth.operations.json](/openapi/berth.operations.json) | +| Operations of berth serve | `berth server api operations` | [berth.operations.json](/openapi/berth.operations.json) | | OpenAPI 3 of berthd | `berthd openapi` | [berthd.openapi.json](/openapi/berthd.openapi.json) | -| OpenAPI 3 of berth serve | `berth openapi` | [berth.openapi.json](/openapi/berth.openapi.json) | +| OpenAPI 3 of berth serve | `berth server api openapi` | [berth.openapi.json](/openapi/berth.openapi.json) | | AsyncAPI 3 of berthd | `berthd asyncapi` | [berthd.asyncapi.json](/openapi/berthd.asyncapi.json) | -| AsyncAPI 3 of berth serve | `berth asyncapi` | [berth.asyncapi.json](/openapi/berth.asyncapi.json) | +| AsyncAPI 3 of berth serve | `berth server api asyncapi` | [berth.asyncapi.json](/openapi/berth.asyncapi.json) | An operations document describes each operation with its name and HTTP route, arguments and errors, and input and result schemas. For streams it adds the frames each way. `components` holds the JSON schemas the `$ref`s point at. The `berth serve` document has `"hosts": true`: every berthd operation is available there with a host. diff --git a/docs/content/docs/api/portal.mdx b/docs/content/docs/api/portal.mdx index bd82d26..ae693ae 100644 --- a/docs/content/docs/api/portal.mdx +++ b/docs/content/docs/api/portal.mdx @@ -43,11 +43,11 @@ An unknown operation is `404`, and a missing argument is `400`. On the portal of `berth serve`, `host` names the host and the rest of the frame is the host's operation, exactly as berthd takes it: ```js -→ {"id": 9, "host": "studio", "op": "tasks.run", "args": {"name": "status"}} +→ {"id": 9, "host": "studio", "op": "actions.run", "args": {"action": "status"}} ← {"id": 9, "result": {"id": "a81kq0wd3m5c"}} ``` -`berth serve` refuses the `secrets` task, because the secrets stay on the machine that runs it. +`berth serve` refuses the controller's own `secrets` action, because the secrets stay on the machine that runs it. ## A stream @@ -71,6 +71,25 @@ The reply to a streaming operation is `{"open": true}`. The server then sends fr The frames inside `data` and the bytes after the four-byte prefix are those of the stream's own WebSocket, listed on each stream's page. Calls go on beside open streams. +## A window on a stream's bytes + +When nobody reads a stream, its bytes would fill the other side's memory or hold up the frames of the other streams. A client asks for a window when it opens the stream to prevent this: + +```js +→ {"id": 12, "op": "ports.connect", "args": {"port": 8080}, "window": 262144} +← {"id": 12, "result": {"open": true, "window": 262144}} +→ binary: 00 00 00 0c, then up to 262144 bytes +← {"sub": 12, "credit": 131072} +``` + +| Rule | Means | +| --- | --- | +| `window` in the request | How many bytes of binary frames each side may send before the other side credits them. The server takes 16 KB to 4 MB | +| `window` in the answer | The server keeps to that window. Without it, no window applies to the stream | +| `{"sub": N, "credit": BYTES}` | Either side has taken so many bytes, and the other may send that many more | + +A side may send one frame beyond the window, so a large frame isn't held forever. JSON frames don't count against it. Each side says what it took once it has taken half the window. berth's client asks for a window on every stream it opens, and a client that asks for none gets the streams as before. + ## Bytes Bytes in input or a result are `{"base64": "…"}`, and a result adds `content_type`. The portal takes up to 8 MB of bytes per operation, and a larger result is a `413` error. Use HTTP for larger transfers, such as big files. @@ -80,6 +99,24 @@ Bytes in input or a result are `{"base64": "…"}`, and a result adds `content_t ← {"id": 11, "result": {"ok": true}} ``` +## Limits + +The server keeps a fixed amount of memory for each portal, so a caller that sends without reading can't fill it. berthd and `berth serve` keep to the same limits: + +| Limit | Value | Past it | +| --- | --- | --- | +| One message | The 8 MB of an operation as base64, and 1 MB more | The portal closes | +| The name of an operation | 128 bytes | A `400` error that doesn't repeat the name | +| An error's message | 2 KB | The message is cut short | +| Calls in flight | 256 | A `429` error with the code `limit` | +| Streams open | 256, such as the connections of a forwarded port | A `429` error with the code `limit` | +| Calls, their answers until they are sent, and input waiting for its stream | 64 MB | A call is refused with a `429` error with the code `limit`, and a stream ends with that error in its last frame | +| The portal's own frames waiting to be sent, such as errors and credits | 64 KB | The server reads nothing more from the caller until they are sent | + +A caller that keeps to its windows and reads what comes never reaches these limits. + +berth's client holds up to 16 MB of a stream that it has not read. A server that sends more ends that stream with an error. + ## When to use HTTP instead - One-off calls from a shell, with `curl`. diff --git a/docs/next.config.mjs b/docs/next.config.mjs index 0d031e6..b0d450b 100644 --- a/docs/next.config.mjs +++ b/docs/next.config.mjs @@ -17,10 +17,4 @@ const config = { env: { NEXT_PUBLIC_BASE_PATH: basePath ?? '' }, }; -// A file host gives public/web/index.html for /web/ by itself; the -// development server has to be told. -if (process.env.NODE_ENV === 'development') { - config.rewrites = async () => [{ source: '/web/', destination: '/web/index.html' }]; -} - export default withMDX(config); diff --git a/docs/openapi/berth.asyncapi.json b/docs/openapi/berth.asyncapi.json index 43af0d7..4d9c304 100644 --- a/docs/openapi/berth.asyncapi.json +++ b/docs/openapi/berth.asyncapi.json @@ -11,19 +11,28 @@ } } }, - "description": "Channel messages of chosen hosts, jobs and key-value changes together. The first frame sent names what to follow; each frame received says where it came from.", + "description": "Channel messages of chosen hosts, jobs, key-value changes and changes to hosts and resources together. The first frame sent names what to follow; each frame received says where it came from.", "messages": { "events.channel": { "$ref": "#/components/messages/events.channel" }, + "events.dropped": { + "$ref": "#/components/messages/events.dropped" + }, "events.follow": { "$ref": "#/components/messages/events.follow" }, + "events.host": { + "$ref": "#/components/messages/events.host" + }, "events.job": { "$ref": "#/components/messages/events.job" }, "events.kv": { "$ref": "#/components/messages/events.kv" + }, + "events.resource": { + "$ref": "#/components/messages/events.resource" } }, "parameters": {}, @@ -31,7 +40,7 @@ "x-berth-resumable": false }, "host.channel": { - "address": "/v1/hosts/{name}/channel", + "address": "/v1/hosts/{name}/controller/channel", "bindings": { "ws": { "query": { @@ -67,7 +76,7 @@ "x-berth-resumable": true }, "host.logs": { - "address": "/v1/hosts/{name}/logs", + "address": "/v1/hosts/{name}/controller/logs", "bindings": { "ws": { "query": { @@ -99,8 +108,38 @@ "x-berth-level": "host", "x-berth-resumable": true }, + "host.port": { + "address": "/v1/hosts/{name}/controller/ports/{port}", + "bindings": { + "ws": { + "query": { + "properties": {}, + "type": "object" + } + } + }, + "description": "A TCP connection to a port on the host's loopback, 127.0.0.1. The port is dialled first, so one that nothing listens on is an error and not a stream.", + "messages": { + "host.port.bytes": { + "$ref": "#/components/messages/host.port.bytes" + }, + "host.port.eof": { + "$ref": "#/components/messages/host.port.eof" + } + }, + "parameters": { + "name": { + "description": "name" + }, + "port": { + "description": "port" + } + }, + "x-berth-level": "host", + "x-berth-resumable": false + }, "host.serve": { - "address": "/v1/hosts/{name}/services/{service}/serve", + "address": "/v1/hosts/{name}/controller/services/{service}/serve", "bindings": { "ws": { "query": { @@ -133,7 +172,7 @@ "x-berth-resumable": false }, "host.session": { - "address": "/v1/hosts/{name}/sessions/{id}", + "address": "/v1/hosts/{name}/controller/sessions/{id}", "bindings": { "ws": { "query": { @@ -203,6 +242,9 @@ }, "description": "Messages as they are taken in from the hosts.", "messages": { + "inbox.dropped": { + "$ref": "#/components/messages/inbox.dropped" + }, "inbox.message": { "$ref": "#/components/messages/inbox.message" } @@ -255,6 +297,9 @@ }, "description": "Changes to a namespace of the key-value store.", "messages": { + "kv.dropped": { + "$ref": "#/components/messages/kv.dropped" + }, "kv.removed": { "$ref": "#/components/messages/kv.removed" }, @@ -270,6 +315,36 @@ "x-berth-level": "host", "x-berth-resumable": false }, + "resource-connect": { + "address": "/v1/resources/{id}/endpoints/{endpoint}", + "bindings": { + "ws": { + "query": { + "properties": {}, + "type": "object" + } + } + }, + "description": "A TCP connection to an endpoint its provisioner reported for a resource, named by the endpoint's name or its port. The server dials it first, so one that nothing answers on, or an address berth refuses, is an error and not a stream.", + "messages": { + "resource-connect.bytes": { + "$ref": "#/components/messages/resource-connect.bytes" + }, + "resource-connect.eof": { + "$ref": "#/components/messages/resource-connect.eof" + } + }, + "parameters": { + "endpoint": { + "description": "endpoint" + }, + "id": { + "description": "id" + } + }, + "x-berth-level": "host", + "x-berth-resumable": false + }, "resource-logs": { "address": "/v1/resources/{id}/logs", "bindings": { @@ -332,11 +407,32 @@ }, "summary": "A message from a host's channel." }, + "events.dropped": { + "contentType": "application/json", + "examples": [ + { + "payload": { + "dropped": { + "events": 12 + } + } + } + ], + "name": "dropped", + "payload": { + "type": "object" + }, + "summary": "This follower was too slow. `events` counts the server's events it missed, of every kind. Read what it follows again." + }, "events.follow": { "contentType": "application/json", "examples": [ { "payload": { + "changes": [ + "hosts", + "resources" + ], "hosts": [ "server" ], @@ -356,6 +452,24 @@ }, "summary": "What to follow. Sent once, first." }, + "events.host": { + "contentType": "application/json", + "examples": [ + { + "payload": { + "change": "renamed", + "from": "desk", + "host": "studio", + "source": "host" + } + } + ], + "name": "host", + "payload": { + "type": "object" + }, + "summary": "A saved host was `added`, `removed`, `renamed` (with the name it had `from`) or its profile `changed`. Read the host itself with hosts.get." + }, "events.job": { "contentType": "application/json", "examples": [ @@ -397,6 +511,25 @@ }, "summary": "A key changed." }, + "events.resource": { + "contentType": "application/json", + "examples": [ + { + "payload": { + "change": "created", + "kind": "deployment", + "provisioner": "docker", + "resource": "r-k3v9qz2m", + "source": "resource" + } + } + ], + "name": "resource", + "payload": { + "type": "object" + }, + "summary": "A resource was `created`, `removed` or its record `changed`. Read it with resources.get." + }, "host.channel.message": { "contentType": "application/json", "examples": [ @@ -460,6 +593,30 @@ }, "summary": "One journal entry." }, + "host.port.bytes": { + "contentType": "application/octet-stream", + "name": "bytes", + "payload": { + "format": "binary", + "type": "string" + }, + "summary": "Bytes for the program on the port." + }, + "host.port.eof": { + "contentType": "application/json", + "examples": [ + { + "payload": { + "eof": true + } + } + ], + "name": "eof", + "payload": { + "type": "object" + }, + "summary": "The client sends no more, and the program on the port reads the end. What it sends still comes back." + }, "host.serve.call": { "contentType": "application/json", "examples": [ @@ -618,6 +775,23 @@ }, "summary": "INT, TERM, HUP or KILL to the session's process group." }, + "inbox.dropped": { + "contentType": "application/json", + "examples": [ + { + "payload": { + "dropped": { + "events": 12 + } + } + } + ], + "name": "dropped", + "payload": { + "type": "object" + }, + "summary": "This follower was too slow. `events` counts the server's events it missed, of every kind. Read the inbox again." + }, "inbox.message": { "contentType": "application/json", "examples": [ @@ -669,6 +843,23 @@ }, "summary": "The job's output." }, + "kv.dropped": { + "contentType": "application/json", + "examples": [ + { + "payload": { + "dropped": { + "events": 12 + } + } + } + ], + "name": "dropped", + "payload": { + "type": "object" + }, + "summary": "This follower was too slow. `events` counts the server's events it missed, of every kind. List the namespace again." + }, "kv.removed": { "contentType": "application/json", "examples": [ @@ -704,6 +895,30 @@ }, "summary": "A key was written." }, + "resource-connect.bytes": { + "contentType": "application/octet-stream", + "name": "bytes", + "payload": { + "format": "binary", + "type": "string" + }, + "summary": "Bytes for the endpoint." + }, + "resource-connect.eof": { + "contentType": "application/json", + "examples": [ + { + "payload": { + "eof": true + } + } + ], + "name": "eof", + "payload": { + "type": "object" + }, + "summary": "The client sends no more, and the endpoint reads the end. What it sends still comes back." + }, "resource-logs.exited": { "contentType": "application/json", "examples": [ @@ -734,7 +949,7 @@ }, "defaultContentType": "application/json", "info": { - "description": "The WebSockets of berth serve: its own, and each host's under /v1/hosts/{name}. Text frames are JSON, one message each; binary frames carry only session and job bytes.", + "description": "The WebSockets of berth serve: its own, and each host's under /v1/hosts/{name}/controller. Text frames are JSON, one message each; binary frames carry only session and job bytes.", "title": "berth serve", "version": "0.5.2" }, @@ -764,6 +979,15 @@ }, { "$ref": "#/channels/events/messages/events.kv" + }, + { + "$ref": "#/channels/events/messages/events.host" + }, + { + "$ref": "#/channels/events/messages/events.resource" + }, + { + "$ref": "#/channels/events/messages/events.dropped" } ] }, @@ -800,6 +1024,34 @@ } ] }, + "host.port.receive": { + "action": "receive", + "channel": { + "$ref": "#/channels/host.port" + }, + "messages": [ + { + "$ref": "#/channels/host.port/messages/host.port.bytes" + }, + { + "$ref": "#/channels/host.port/messages/host.port.eof" + } + ] + }, + "host.port.send": { + "action": "send", + "channel": { + "$ref": "#/channels/host.port" + }, + "messages": [ + { + "$ref": "#/channels/host.port/messages/host.port.bytes" + }, + { + "$ref": "#/channels/host.port/messages/host.port.eof" + } + ] + }, "host.serve.receive": { "action": "receive", "channel": { @@ -870,6 +1122,9 @@ "messages": [ { "$ref": "#/channels/inbox/messages/inbox.message" + }, + { + "$ref": "#/channels/inbox/messages/inbox.dropped" } ] }, @@ -898,6 +1153,37 @@ }, { "$ref": "#/channels/kv/messages/kv.removed" + }, + { + "$ref": "#/channels/kv/messages/kv.dropped" + } + ] + }, + "resource-connect.receive": { + "action": "receive", + "channel": { + "$ref": "#/channels/resource-connect" + }, + "messages": [ + { + "$ref": "#/channels/resource-connect/messages/resource-connect.bytes" + }, + { + "$ref": "#/channels/resource-connect/messages/resource-connect.eof" + } + ] + }, + "resource-connect.send": { + "action": "send", + "channel": { + "$ref": "#/channels/resource-connect" + }, + "messages": [ + { + "$ref": "#/channels/resource-connect/messages/resource-connect.bytes" + }, + { + "$ref": "#/channels/resource-connect/messages/resource-connect.eof" } ] }, diff --git a/docs/openapi/berth.openapi.json b/docs/openapi/berth.openapi.json index d3c2cde..a246cdb 100644 --- a/docs/openapi/berth.openapi.json +++ b/docs/openapi/berth.openapi.json @@ -182,11 +182,17 @@ "type": "object" }, "Applied": { - "description": "What berth recorded on a host when it applied the configuration: its own\nversion and the plugins whose features the host has.", + "description": "What berth recorded on a host when it applied the configuration: its own\nversion, the plugins whose features the host has, and the provisioners\nthose features offer.", "properties": { "berth": { "type": "string" }, + "offers": { + "items": { + "$ref": "#/components/schemas/Offer" + }, + "type": "array" + }, "plugins": { "items": { "$ref": "#/components/schemas/AppliedPlugin" @@ -496,6 +502,43 @@ ], "type": "object" }, + "Endpoint": { + "description": "Where a resource can be reached: an address and a port, as its\nprovisioner reports it. The address is as seen from the machine the\nprovisioner's engine runs on, such as a Docker engine or an Incus server.", + "properties": { + "address": { + "description": "An IP address or a DNS name.", + "type": "string" + }, + "hint": { + "description": "What speaks there, for people, such as `http` or `postgres`.", + "type": "string" + }, + "local": { + "description": "The engine runs on the server's own machine, so the address is one\nthe server reaches as it is, a loopback address among them.", + "type": "boolean" + }, + "name": { + "description": "Unique among the resource's endpoints: lowercase letters, digits, `-`, `_` and `.`.", + "type": "string" + }, + "port": { + "format": "int32", + "minimum": 0, + "type": "integer" + }, + "protocol": { + "description": "`tcp` or `udp`.", + "type": "string" + } + }, + "required": [ + "name", + "address", + "port", + "protocol" + ], + "type": "object" + }, "ErrorBody": { "properties": { "error": { @@ -538,13 +581,21 @@ }, "label": { "type": "string" + }, + "types": { + "description": "The types of host it is for, such as `workstation` or `node`.", + "items": { + "$ref": "#/components/schemas/HostType" + }, + "type": "array" } }, "required": [ "key", "label", "group", - "help" + "help", + "types" ], "type": "object" }, @@ -658,7 +709,7 @@ ] }, "what": { - "description": "`hostname`, `password` or `tailscale_key`.", + "description": "`hostname` or `password`.", "type": "string" } }, @@ -681,7 +732,7 @@ ] }, "value": { - "description": "The host name, the password's hash, or the key.", + "description": "The host name or the password's hash.", "type": "string" } }, @@ -720,6 +771,66 @@ ], "type": "object" }, + "HostAction": { + "description": "An action of the host's controller: an entry of `/etc/berth/actions.json`.\nListed without `run`.", + "properties": { + "agent": { + "description": "The agent may run it.", + "type": "boolean" + }, + "arguments": { + "items": { + "$ref": "#/components/schemas/Argument" + }, + "type": "array" + }, + "confirm": { + "type": [ + "string", + "null" + ] + }, + "extends": { + "description": "Actions it is a step of; it then has no row of its own.", + "items": { + "type": "string" + }, + "type": "array" + }, + "group": { + "type": "string" + }, + "help": { + "type": "string" + }, + "input": { + "description": "What the workstation sends on standard input: `secrets`.", + "type": [ + "string", + "null" + ] + }, + "label": { + "type": "string" + }, + "name": { + "type": "string" + }, + "run": { + "type": "string" + }, + "terminal": { + "type": "boolean" + }, + "user": { + "type": "string" + } + }, + "required": [ + "name" + ], + "type": "object" + }, "HostBehind": { "allOf": [ { @@ -778,10 +889,6 @@ "null" ] }, - "level": { - "$ref": "#/components/schemas/HostLevel", - "description": "The level its profile means it to have." - }, "name": { "type": "string" }, @@ -795,14 +902,18 @@ "seen": { "oneOf": [ { - "$ref": "#/components/schemas/HostLevel", - "description": "The level it was seen to have: what its controller's record says,\n`reachable` when berth logged in and found no controller. Absent\nwhen it could not be asked." + "$ref": "#/components/schemas/HostType", + "description": "The type it was seen to be: what its controller's record says, `ssh`\nwhen berth logged in and found no controller. Absent when it could\nnot be asked." }, { "type": "null" } ] }, + "type": { + "$ref": "#/components/schemas/HostType", + "description": "The type its profile means it to be." + }, "version": { "description": "The version of its berthd.", "type": [ @@ -815,15 +926,16 @@ "name", "address", "reachable", - "level" + "type" ], "type": "object" }, - "HostLevel": { - "description": "How much of berth a host has: what its profile means it to have, or\nwhat it was seen to have. Each level can do what the ones before it can.", + "HostType": { + "description": "What berth does with a host: what its profile means it to be, or what it\nwas seen to be. They are in the order of how much of berth each has.", "enum": [ - "reachable", - "controller", + "ssh", + "managed", + "node", "workstation" ], "type": "string" @@ -930,6 +1042,13 @@ "description": "The action it runs, such as `apply`, `rollout` or `provision`.", "type": "string" }, + "arguments": { + "description": "What it was started with, as people may see it: the command of a\nresource's `exec`, or an action's arguments as `NAME=VALUE`. Never a\nsecret argument, a password or standard input.", + "items": { + "type": "string" + }, + "type": "array" + }, "ended": { "type": [ "string", @@ -964,6 +1083,13 @@ "started": { "type": "string" }, + "stopped_by": { + "description": "Who stopped it before it ended by itself.", + "type": [ + "string", + "null" + ] + }, "subject": { "description": "What it was done to: `host:NAME`, `provisioner:NAME`,\n`resource:ID` or `installation`. Empty for a job from before 0.5.", "type": "string" @@ -1367,6 +1493,64 @@ ], "type": "object" }, + "Offer": { + "description": "Provisioners a feature of a host offers, as its tasks found them: berth\nregisters each under a name of its own that names the host. The host\nserves this to anyone who may read it, so it holds public values only,\nsuch as a server's certificate, never a key.", + "properties": { + "aliases": { + "additionalProperties": { + "items": { + "type": "string" + }, + "type": "array" + }, + "description": "Other values of a setting that reach the same server, such as the\nname a berth server on the host gives it: a provisioner registered by\nhand with one of them is named to its owner, who can make it the\nhost's.", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "feature": { + "description": "The feature, as `PLUGIN/FEATURE`.", + "type": "string" + }, + "fingerprint": { + "description": "What the host says identifies the server, such as the fingerprint of\nits certificate, for the person who reads the report.", + "type": "string" + }, + "provisioners": { + "description": "The plugin's provisioners it offers, by their ids in the plugin.", + "items": { + "type": "string" + }, + "type": "array" + }, + "secrets": { + "additionalProperties": { + "type": "string" + }, + "description": "Values of the provisioners' secrets, such as the certificate of the\nserver to pin, kept as the provisioners' own.", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "settings": { + "additionalProperties": { + "type": "string" + }, + "description": "Values of the provisioners' own arguments. `{address}`, `{user}` and\n`{port}` stand for where berth reaches the host and as whom.", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "feature", + "provisioners" + ], + "type": "object" + }, "Output": { "properties": { "data_b64": { @@ -1421,9 +1605,55 @@ ], "type": "object" }, + "PluginAction": { + "description": "An action a feature adds to a host.", + "properties": { + "agent": { + "description": "The agent may run it with `box run`.", + "type": "boolean" + }, + "arguments": { + "description": "The names of the arguments it asks for.", + "items": { + "type": "string" + }, + "type": "array" + }, + "extends": { + "description": "Actions of berth it runs after, with no row of its own.", + "items": { + "type": "string" + }, + "type": "array" + }, + "label": { + "type": "string" + }, + "name": { + "description": "The action is `PLUGIN:NAME` on the host.", + "type": "string" + }, + "user": { + "description": "`agent`, `operator` or `root`.", + "type": "string" + } + }, + "required": [ + "name", + "label", + "user" + ], + "type": "object" + }, "PluginFeature": { "description": "Something a plugin installs, switched on per host.", "properties": { + "actions": { + "items": { + "$ref": "#/components/schemas/PluginAction" + }, + "type": "array" + }, "channels": { "description": "Channel topics, without the plugin's id.", "items": { @@ -1450,12 +1680,6 @@ "$ref": "#/components/schemas/PluginService" }, "type": "array" - }, - "tasks": { - "items": { - "$ref": "#/components/schemas/PluginTask" - }, - "type": "array" } }, "required": [ @@ -1602,46 +1826,6 @@ ], "type": "object" }, - "PluginTask": { - "description": "A task a feature adds to a host's overview.", - "properties": { - "agent": { - "description": "The agent may run it with `box run`.", - "type": "boolean" - }, - "arguments": { - "description": "The names of the arguments it asks for.", - "items": { - "type": "string" - }, - "type": "array" - }, - "extends": { - "description": "Tasks of berth it runs after, with no row of its own.", - "items": { - "type": "string" - }, - "type": "array" - }, - "label": { - "type": "string" - }, - "name": { - "description": "The task is `PLUGIN:NAME` on the host.", - "type": "string" - }, - "user": { - "description": "`agent`, `operator` or `root`.", - "type": "string" - } - }, - "required": [ - "name", - "label", - "user" - ], - "type": "object" - }, "PluginUpdate": { "description": "Whether a newer version of a plugin is there, as its registry says.", "properties": { @@ -1782,6 +1966,10 @@ }, "type": "array" }, + "node": { + "description": "The node it lives on, when a feature of that host offered it.", + "type": "string" + }, "own": { "description": "The names of the provisioner's own arguments, which say where it is,\nsuch as an Incus server. Their values are kept as its settings.", "items": { @@ -2045,6 +2233,13 @@ "description": "When it was asked for, RFC 3339.", "type": "string" }, + "endpoints": { + "description": "Where it can be reached, as its provisioner last said: when it was\nmade, changed or checked.", + "items": { + "$ref": "#/components/schemas/Endpoint" + }, + "type": "array" + }, "external_id": { "description": "The provisioner's id of it; empty until `create` said it.", "type": "string" @@ -2092,7 +2287,7 @@ "type": "string" }, "verbs": { - "description": "What can be done to it besides being removed: `start`, `stop`,\n`change`, `logs`, as its kind declares.", + "description": "What can be done to it besides being removed: `start`, `stop`,\n`change`, `logs`, `exec`, as its kind declares.", "items": { "type": "string" }, @@ -2238,7 +2433,7 @@ ], "type": "string" }, - "RunTask": { + "RunAction": { "properties": { "arguments": { "additionalProperties": { @@ -2452,6 +2647,13 @@ }, "Session": { "properties": { + "action": { + "description": "The action it was started for, if any. The sessions an older\nberthd kept on the host say `task`.", + "type": [ + "string", + "null" + ] + }, "cols": { "format": "int32", "minimum": 0, @@ -2520,13 +2722,6 @@ }, "type": "array" }, - "task": { - "description": "The task it was started for, if any.", - "type": [ - "string", - "null" - ] - }, "title": { "type": "string" }, @@ -2877,68 +3072,8 @@ ], "type": "object" }, - "Task": { - "description": "An entry of `/etc/berth/tasks.json`. Listed without `run`.", - "properties": { - "agent": { - "description": "The agent may run it.", - "type": "boolean" - }, - "arguments": { - "items": { - "$ref": "#/components/schemas/Argument" - }, - "type": "array" - }, - "confirm": { - "type": [ - "string", - "null" - ] - }, - "extends": { - "description": "Tasks it is a step of; it then has no row of its own.", - "items": { - "type": "string" - }, - "type": "array" - }, - "group": { - "type": "string" - }, - "help": { - "type": "string" - }, - "input": { - "description": "What the workstation sends on standard input: `secrets`.", - "type": [ - "string", - "null" - ] - }, - "label": { - "type": "string" - }, - "name": { - "type": "string" - }, - "run": { - "type": "string" - }, - "terminal": { - "type": "boolean" - }, - "user": { - "type": "string" - } - }, - "required": [ - "name" - ], - "type": "object" - }, - "Topic": { - "description": "An entry of `/etc/berth/channels.json`.", + "Topic": { + "description": "An entry of `/etc/berth/channels.json`.", "properties": { "days": { "format": "int64", @@ -3035,21 +3170,21 @@ } ] }, - "level": { + "mode": { + "description": "`system` or `single`.", + "type": "string" + }, + "type": { "oneOf": [ { - "$ref": "#/components/schemas/HostLevel", - "description": "What was installed on the host, by that record: `controller` when\nonly the controller was, `workstation` for a whole profile. Absent\nwhen the host keeps no record." + "$ref": "#/components/schemas/HostType", + "description": "What was installed on the host, by that record: `managed` when\nonly the controller was, `workstation` for a whole profile. Absent\nwhen the host keeps no record." }, { "type": "null" } ] }, - "mode": { - "description": "`system` or `single`.", - "type": "string" - }, "version": { "type": "string" } @@ -3063,7 +3198,7 @@ } }, "info": { - "description": "One API over all hosts, with the inbox and a key-value store. Everything under /v1/hosts/{name}/ that is not named here is that host's own API, passed through unchanged; only the `secrets` task is refused, which the host's `secrets` action stands in for. What can be done to a host or to the installation is listed at /v1/hosts/{name}/actions and /v1/actions, and run by POST to an action's name there. Over TCP every request carries the server's token or an OAuth 2.0 access token as a bearer, and needs the role its operation names. The routes of the setup key take that key as the bearer instead.", + "description": "One API over all hosts, with the inbox and a key-value store. Everything under /v1/hosts/{name}/controller/ is that host's own API, passed through unchanged; only the controller's own `secrets` action is refused, which the server's `secrets` action of the host stands in for, and `endpoints.connect`, which the server opens itself for `resources.connect`. What can be done to a host or to the installation is listed at /v1/hosts/{name}/actions and /v1/installation/actions, and run by POST to an action's name there. Over TCP every request carries the server's token or an OAuth 2.0 access token as a bearer, and needs the role its operation names. The routes of the setup key take that key as the bearer instead.", "title": "berth serve", "version": "0.5.2" }, @@ -3090,119 +3225,6 @@ ] } }, - "/v1/actions": { - "get": { - "operationId": "list", - "responses": { - "200": { - "content": { - "application/json": { - "schema": { - "items": { - "$ref": "#/components/schemas/Action" - }, - "type": "array" - } - } - }, - "description": "The actions" - } - }, - "summary": "What can be done to the installation as a whole, such as a roll-out over its hosts.", - "tags": [ - "hosts" - ] - } - }, - "/v1/actions/{action}": { - "post": { - "operationId": "act", - "parameters": [ - { - "description": "The action, as actions.list names it", - "in": "path", - "name": "action", - "required": true, - "schema": { - "type": "string" - } - } - ], - "requestBody": { - "content": { - "application/json": { - "schema": { - "oneOf": [ - { - "$ref": "#/components/schemas/Act" - }, - { - "type": "null" - } - ] - } - } - }, - "description": "Its arguments, and a token that makes asking again safe" - }, - "responses": { - "202": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/Acted" - } - } - }, - "description": "Started as a job" - }, - "400": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/ErrorBody" - } - } - }, - "description": "An argument is wrong, or the body is not one" - }, - "403": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/ErrorBody" - } - } - }, - "description": "The caller's role is below the action's" - }, - "404": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/ErrorBody" - } - } - }, - "description": "No such action, or a host it names" - }, - "409": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/ErrorBody" - } - } - }, - "description": "Another one is running" - } - }, - "summary": "Run one of the installation's actions, as a job.", - "tags": [ - "hosts" - ] - } - }, "/v1/events": { "get": { "operationId": "stream_events", @@ -3212,7 +3234,7 @@ "description": "A WebSocket; its frames are in the AsyncAPI document" } }, - "summary": "Channel messages of chosen hosts, jobs and key-value changes together. The first frame sent names what to follow; each frame received says where it came from.", + "summary": "Channel messages of chosen hosts, jobs, key-value changes and changes to hosts and resources together. The first frame sent names what to follow; each frame received says where it came from.", "tags": [ "streams" ] @@ -3373,7 +3395,7 @@ "description": "No such host" } }, - "summary": "What can be done to a host: the tasks its controller lists, and berth's own actions.", + "summary": "What can be done to a host: the actions its controller lists, and berth's own.", "tags": [ "hosts" ] @@ -3438,7 +3460,7 @@ } } }, - "description": "Started as a job; a task of the host also says its session" + "description": "Started as a job; an action of its controller also says its session" }, "400": { "content": { @@ -3501,13 +3523,186 @@ "description": "The host could not be reached" } }, - "summary": "Run one of a host's actions. One of berth's that changes something is a job; a task of the host is a session there, which a job follows unless it needs a terminal.", + "summary": "Run one of a host's actions. One of berth's that changes something is a job; an action of the host's controller is a session there, which a job follows unless it needs a terminal.", + "tags": [ + "hosts" + ] + } + }, + "/v1/hosts/{name}/controller/actions": { + "get": { + "description": "Host and agent level.", + "operationId": "host_get_v1_actions", + "parameters": [ + { + "description": "A saved host", + "in": "path", + "name": "name", + "required": true, + "schema": { + "type": "string" + } + } + ], + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "items": { + "$ref": "#/components/schemas/HostAction" + }, + "type": "array" + } + } + }, + "description": "Done." + }, + "default": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorBody" + } + } + }, + "description": "An error." + } + }, + "summary": "The actions the caller may run, in the order of the file.", + "tags": [ + "host: actions" + ] + } + }, + "/v1/hosts/{name}/controller/actions/{action}": { + "post": { + "description": "Host and agent level.", + "operationId": "host_post_v1_actions__action_", + "parameters": [ + { + "description": "A saved host", + "in": "path", + "name": "name", + "required": true, + "schema": { + "type": "string" + } + }, + { + "in": "path", + "name": "action", + "required": true, + "schema": { + "type": "string" + } + }, + { + "description": "A duration such as 30s, at most 10m: answer when the command has ended, with what it wrote.", + "in": "query", + "name": "wait", + "schema": { + "type": "string" + } + } + ], + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/RunAction" + } + } + } + }, + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Started" + } + } + }, + "description": "Done." + }, + "default": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorBody" + } + } + }, + "description": "An error." + } + }, + "summary": "Run an action. It and what extends it are the steps of one session.", + "tags": [ + "host: actions" + ] + } + }, + "/v1/hosts/{name}/controller/actions/{action}/choices/{argument}": { + "get": { + "description": "Host and agent level.", + "operationId": "host_get_v1_actions__action__choices__argument_", + "parameters": [ + { + "description": "A saved host", + "in": "path", + "name": "name", + "required": true, + "schema": { + "type": "string" + } + }, + { + "in": "path", + "name": "action", + "required": true, + "schema": { + "type": "string" + } + }, + { + "in": "path", + "name": "argument", + "required": true, + "schema": { + "type": "string" + } + } + ], + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Choices" + } + } + }, + "description": "Done." + }, + "default": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorBody" + } + } + }, + "description": "An error." + } + }, + "summary": "The values to offer for an argument.", "tags": [ - "hosts" + "host: actions" ] } }, - "/v1/hosts/{name}/applied/profile": { + "/v1/hosts/{name}/controller/applied/profile": { "get": { "description": "Host level only.", "operationId": "host_get_v1_applied_profile", @@ -3550,7 +3745,7 @@ ] } }, - "/v1/hosts/{name}/channel": { + "/v1/hosts/{name}/controller/channel": { "get": { "description": "Host and agent level.", "operationId": "host_get_v1_channel", @@ -3620,7 +3815,7 @@ ] } }, - "/v1/hosts/{name}/channel/{topic}": { + "/v1/hosts/{name}/controller/channel/{topic}": { "post": { "description": "Host and agent level.", "operationId": "host_post_v1_channel___topic_", @@ -3680,49 +3875,7 @@ ] } }, - "/v1/hosts/{name}/credentials": { - "get": { - "operationId": "credentials", - "parameters": [ - { - "description": "A saved host", - "in": "path", - "name": "name", - "required": true, - "schema": { - "type": "string" - } - } - ], - "responses": { - "200": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/Credentials" - } - } - }, - "description": "The passwords by account; none when none were generated" - }, - "404": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/ErrorBody" - } - } - }, - "description": "No such host" - } - }, - "summary": "The passwords the server generated for a host's accounts.", - "tags": [ - "set-up" - ] - } - }, - "/v1/hosts/{name}/files": { + "/v1/hosts/{name}/controller/files": { "delete": { "description": "Host level only.", "operationId": "host_delete_v1_files", @@ -3898,7 +4051,7 @@ ] } }, - "/v1/hosts/{name}/logs": { + "/v1/hosts/{name}/controller/logs": { "get": { "description": "Host level only.", "operationId": "host_get_v1_logs", @@ -3971,7 +4124,51 @@ ] } }, - "/v1/hosts/{name}/power/{verb}": { + "/v1/hosts/{name}/controller/ports/{port}": { + "get": { + "description": "Host level only.", + "operationId": "host_get_v1_ports__port_", + "parameters": [ + { + "description": "A saved host", + "in": "path", + "name": "name", + "required": true, + "schema": { + "type": "string" + } + }, + { + "in": "path", + "name": "port", + "required": true, + "schema": { + "type": "string" + } + } + ], + "responses": { + "101": { + "description": "With a WebSocket handshake: the stream `port` of the AsyncAPI document." + }, + "default": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorBody" + } + } + }, + "description": "An error." + } + }, + "summary": "Only as a WebSocket: a TCP connection to a port on the host's loopback. A port that nothing listens on is an error before the handshake.", + "tags": [ + "host: ports" + ] + } + }, + "/v1/hosts/{name}/controller/power/{verb}": { "post": { "description": "Host level only.", "operationId": "host_post_v1_power__verb_", @@ -4030,7 +4227,7 @@ ] } }, - "/v1/hosts/{name}/processes": { + "/v1/hosts/{name}/controller/processes": { "get": { "description": "Host level only.", "operationId": "host_get_v1_processes", @@ -4076,7 +4273,7 @@ ] } }, - "/v1/hosts/{name}/processes/{pid}/signal": { + "/v1/hosts/{name}/controller/processes/{pid}/signal": { "post": { "description": "Host level only.", "operationId": "host_post_v1_processes__pid__signal", @@ -4136,96 +4333,7 @@ ] } }, - "/v1/hosts/{name}/profile": { - "get": { - "operationId": "profile", - "parameters": [ - { - "description": "A saved host", - "in": "path", - "name": "name", - "required": true, - "schema": { - "type": "string" - } - } - ], - "responses": { - "200": { - "content": { - "application/json": { - "schema": {} - } - }, - "description": "The profile, with the passwords and keys in it" - }, - "404": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/ErrorBody" - } - } - }, - "description": "No such host" - } - }, - "summary": "A host's profile, as it is saved.", - "tags": [ - "set-up" - ] - }, - "put": { - "operationId": "save_profile", - "parameters": [ - { - "description": "The name to save it under: letters, digits, `-`, `_` and `.`", - "in": "path", - "name": "name", - "required": true, - "schema": { - "type": "string" - } - } - ], - "requestBody": { - "content": { - "application/json": { - "schema": {} - } - }, - "description": "The whole profile. A workstation's has a builder object; a reachable or controller host's needs only level and access", - "required": true - }, - "responses": { - "200": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/Done" - } - } - }, - "description": "Saved" - }, - "400": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/ErrorBody" - } - } - }, - "description": "Not a profile, or not a name" - } - }, - "summary": "Save a host's profile: a new host, or a changed one. A host below a workstation needs only its level and access.", - "tags": [ - "set-up" - ] - } - }, - "/v1/hosts/{name}/reload": { + "/v1/hosts/{name}/controller/reload": { "post": { "description": "Host level only.", "operationId": "host_post_v1_reload", @@ -4268,7 +4376,7 @@ ] } }, - "/v1/hosts/{name}/screenshot": { + "/v1/hosts/{name}/controller/screenshot": { "get": { "description": "Host level only.", "operationId": "host_get_v1_screenshot", @@ -4312,7 +4420,7 @@ ] } }, - "/v1/hosts/{name}/services": { + "/v1/hosts/{name}/controller/services": { "get": { "description": "Host and agent level.", "operationId": "host_get_v1_services", @@ -4358,7 +4466,7 @@ ] } }, - "/v1/hosts/{name}/services/{plugin}/{name}/serve": { + "/v1/hosts/{name}/controller/services/{plugin}/{name}/serve": { "get": { "description": "Host and agent level.", "operationId": "host_get_v1_services__plugin___name__serve", @@ -4410,7 +4518,7 @@ ] } }, - "/v1/hosts/{name}/services/{plugin}/{name}/{method}": { + "/v1/hosts/{name}/controller/services/{plugin}/{name}/{method}": { "post": { "description": "Host and agent level.", "operationId": "host_post_v1_services__plugin___name___method_", @@ -4482,7 +4590,7 @@ ] } }, - "/v1/hosts/{name}/sessions": { + "/v1/hosts/{name}/controller/sessions": { "get": { "description": "Host level only.", "operationId": "host_get_v1_sessions", @@ -4586,7 +4694,7 @@ ] } }, - "/v1/hosts/{name}/sessions/{id}": { + "/v1/hosts/{name}/controller/sessions/{id}": { "delete": { "description": "Host level only.", "operationId": "host_delete_v1_sessions__id_", @@ -4683,13 +4791,13 @@ "description": "An error." } }, - "summary": "One session's state. The agent sees those it started through a task.", + "summary": "One session's state. The agent sees those it started through an action.", "tags": [ "host: sessions" ] } }, - "/v1/hosts/{name}/sessions/{id}/input": { + "/v1/hosts/{name}/controller/sessions/{id}/input": { "post": { "description": "Host level only.", "operationId": "host_post_v1_sessions__id__input", @@ -4749,7 +4857,7 @@ ] } }, - "/v1/hosts/{name}/sessions/{id}/output": { + "/v1/hosts/{name}/controller/sessions/{id}/output": { "get": { "description": "Host and agent level.", "operationId": "host_get_v1_sessions__id__output", @@ -4848,7 +4956,7 @@ ] } }, - "/v1/hosts/{name}/sessions/{id}/resize": { + "/v1/hosts/{name}/controller/sessions/{id}/resize": { "post": { "description": "Host level only.", "operationId": "host_post_v1_sessions__id__resize", @@ -4908,7 +5016,7 @@ ] } }, - "/v1/hosts/{name}/sessions/{id}/screen": { + "/v1/hosts/{name}/controller/sessions/{id}/screen": { "get": { "description": "Host and agent level.", "operationId": "host_get_v1_sessions__id__screen", @@ -4975,7 +5083,7 @@ ] } }, - "/v1/hosts/{name}/sessions/{id}/signal": { + "/v1/hosts/{name}/controller/sessions/{id}/signal": { "post": { "description": "Host level only.", "operationId": "host_post_v1_sessions__id__signal", @@ -5035,7 +5143,7 @@ ] } }, - "/v1/hosts/{name}/stats": { + "/v1/hosts/{name}/controller/stats": { "get": { "description": "Host level only.", "operationId": "host_get_v1_stats", @@ -5078,10 +5186,10 @@ ] } }, - "/v1/hosts/{name}/tasks": { + "/v1/hosts/{name}/controller/units": { "get": { - "description": "Host and agent level.", - "operationId": "host_get_v1_tasks", + "description": "Host level only.", + "operationId": "host_get_v1_units", "parameters": [ { "description": "A saved host", @@ -5099,7 +5207,7 @@ "application/json": { "schema": { "items": { - "$ref": "#/components/schemas/Task" + "$ref": "#/components/schemas/Unit" }, "type": "array" } @@ -5118,16 +5226,16 @@ "description": "An error." } }, - "summary": "The tasks the caller may run, in the order of the file.", + "summary": "The units and their state.", "tags": [ - "host: tasks" + "host: units" ] } }, - "/v1/hosts/{name}/tasks/{name}": { - "post": { - "description": "Host and agent level.", - "operationId": "host_post_v1_tasks__name_", + "/v1/hosts/{name}/controller/units/{unit}": { + "get": { + "description": "Host level only.", + "operationId": "host_get_v1_units__unit_", "parameters": [ { "description": "A saved host", @@ -5140,36 +5248,22 @@ }, { "in": "path", - "name": "name", + "name": "unit", "required": true, "schema": { "type": "string" } - }, - { - "description": "A duration such as 30s, at most 10m: answer when the command has ended, with what it wrote.", - "in": "query", - "name": "wait", - "schema": { - "type": "string" - } } ], - "requestBody": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/RunTask" - } - } - } - }, "responses": { "200": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/Started" + "additionalProperties": { + "type": "string" + }, + "type": "object" } } }, @@ -5186,16 +5280,16 @@ "description": "An error." } }, - "summary": "Run a task. It and what extends it are the steps of one session.", + "summary": "One unit's properties, as systemctl show gives them.", "tags": [ - "host: tasks" + "host: units" ] } }, - "/v1/hosts/{name}/tasks/{name}/choices/{argument}": { - "get": { - "description": "Host and agent level.", - "operationId": "host_get_v1_tasks__name__choices__argument_", + "/v1/hosts/{name}/controller/units/{unit}/{verb}": { + "post": { + "description": "Host level only.", + "operationId": "host_post_v1_units__unit___verb_", "parameters": [ { "description": "A saved host", @@ -5208,7 +5302,7 @@ }, { "in": "path", - "name": "name", + "name": "unit", "required": true, "schema": { "type": "string" @@ -5216,7 +5310,7 @@ }, { "in": "path", - "name": "argument", + "name": "verb", "required": true, "schema": { "type": "string" @@ -5228,7 +5322,7 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/Choices" + "$ref": "#/components/schemas/Done" } } }, @@ -5245,16 +5339,16 @@ "description": "An error." } }, - "summary": "The values to offer for an argument.", + "summary": "Start, stop, restart, reload, enable or disable a unit.", "tags": [ - "host: tasks" + "host: units" ] } }, - "/v1/hosts/{name}/units": { + "/v1/hosts/{name}/controller/version": { "get": { - "description": "Host level only.", - "operationId": "host_get_v1_units", + "description": "Host and agent level.", + "operationId": "host_get_v1_version", "parameters": [ { "description": "A saved host", @@ -5271,10 +5365,7 @@ "content": { "application/json": { "schema": { - "items": { - "$ref": "#/components/schemas/Unit" - }, - "type": "array" + "$ref": "#/components/schemas/Version" } } }, @@ -5291,16 +5382,15 @@ "description": "An error." } }, - "summary": "The units and their state.", + "summary": "The version of berthd and of its API.", "tags": [ - "host: units" + "host: version" ] } }, - "/v1/hosts/{name}/units/{unit}": { + "/v1/hosts/{name}/credentials": { "get": { - "description": "Host level only.", - "operationId": "host_get_v1_units__unit_", + "operationId": "credentials", "parameters": [ { "description": "A saved host", @@ -5310,14 +5400,6 @@ "schema": { "type": "string" } - }, - { - "in": "path", - "name": "unit", - "required": true, - "schema": { - "type": "string" - } } ], "responses": { @@ -5325,16 +5407,13 @@ "content": { "application/json": { "schema": { - "additionalProperties": { - "type": "string" - }, - "type": "object" + "$ref": "#/components/schemas/Credentials" } } }, - "description": "Done." + "description": "The passwords by account; none when none were generated" }, - "default": { + "404": { "content": { "application/json": { "schema": { @@ -5342,19 +5421,18 @@ } } }, - "description": "An error." + "description": "No such host" } }, - "summary": "One unit's properties, as systemctl show gives them.", + "summary": "The passwords the server generated for a host's accounts.", "tags": [ - "host: units" + "set-up" ] } }, - "/v1/hosts/{name}/units/{unit}/{verb}": { - "post": { - "description": "Host level only.", - "operationId": "host_post_v1_units__unit___verb_", + "/v1/hosts/{name}/profile": { + "get": { + "operationId": "profile", "parameters": [ { "description": "A saved host", @@ -5364,36 +5442,18 @@ "schema": { "type": "string" } - }, - { - "in": "path", - "name": "unit", - "required": true, - "schema": { - "type": "string" - } - }, - { - "in": "path", - "name": "verb", - "required": true, - "schema": { - "type": "string" - } } ], "responses": { "200": { "content": { "application/json": { - "schema": { - "$ref": "#/components/schemas/Done" - } + "schema": {} } }, - "description": "Done." + "description": "The profile, with the passwords and keys in it" }, - "default": { + "404": { "content": { "application/json": { "schema": { @@ -5401,22 +5461,19 @@ } } }, - "description": "An error." + "description": "No such host" } }, - "summary": "Start, stop, restart, reload, enable or disable a unit.", + "summary": "A host's profile, as it is saved.", "tags": [ - "host: units" + "set-up" ] - } - }, - "/v1/hosts/{name}/version": { - "get": { - "description": "Host and agent level.", - "operationId": "host_get_v1_version", + }, + "put": { + "operationId": "save_profile", "parameters": [ { - "description": "A saved host", + "description": "The name to save it under: letters, digits, `-`, `_` and `.`", "in": "path", "name": "name", "required": true, @@ -5425,18 +5482,27 @@ } } ], + "requestBody": { + "content": { + "application/json": { + "schema": {} + } + }, + "description": "The whole profile. A workstation's has a builder object; a host of the type ssh or managed needs only type and access", + "required": true + }, "responses": { "200": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/Version" + "$ref": "#/components/schemas/Done" } } }, - "description": "Done." + "description": "Saved" }, - "default": { + "400": { "content": { "application/json": { "schema": { @@ -5444,12 +5510,12 @@ } } }, - "description": "An error." + "description": "Not a profile, or not a name" } }, - "summary": "The version of berthd and of its API.", + "summary": "Save a host's profile: a new host, or a changed one. A host that is not a workstation needs only its type and access.", "tags": [ - "host: version" + "set-up" ] } }, @@ -5630,6 +5696,119 @@ ] } }, + "/v1/installation/actions": { + "get": { + "operationId": "list", + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "items": { + "$ref": "#/components/schemas/Action" + }, + "type": "array" + } + } + }, + "description": "The actions" + } + }, + "summary": "What can be done to the installation as a whole, such as a roll-out over its hosts.", + "tags": [ + "hosts" + ] + } + }, + "/v1/installation/actions/{action}": { + "post": { + "operationId": "act", + "parameters": [ + { + "description": "The action, as installation.actions names it", + "in": "path", + "name": "action", + "required": true, + "schema": { + "type": "string" + } + } + ], + "requestBody": { + "content": { + "application/json": { + "schema": { + "oneOf": [ + { + "$ref": "#/components/schemas/Act" + }, + { + "type": "null" + } + ] + } + } + }, + "description": "Its arguments, and a token that makes asking again safe" + }, + "responses": { + "202": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Acted" + } + } + }, + "description": "Started as a job" + }, + "400": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorBody" + } + } + }, + "description": "An argument is wrong, or the body is not one" + }, + "403": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorBody" + } + } + }, + "description": "The caller's role is below the action's" + }, + "404": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorBody" + } + } + }, + "description": "No such action, or a host it names" + }, + "409": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorBody" + } + } + }, + "description": "Another one is running" + } + }, + "summary": "Run one of the installation's actions, as a job.", + "tags": [ + "hosts" + ] + } + }, "/v1/jobs": { "get": { "operationId": "jobs", @@ -5808,6 +5987,78 @@ ] } }, + "/v1/jobs/{id}/stop": { + "post": { + "operationId": "stop_job", + "parameters": [ + { + "description": "A job's id", + "in": "path", + "name": "id", + "required": true, + "schema": { + "type": "string" + } + } + ], + "responses": { + "202": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/JobInfo" + } + } + }, + "description": "Asked to stop; it ends soon, as following it shows" + }, + "403": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorBody" + } + } + }, + "description": "The caller may not start a job of its kind, so may not stop it" + }, + "404": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorBody" + } + } + }, + "description": "No such job" + }, + "409": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorBody" + } + } + }, + "description": "It has ended already" + }, + "501": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorBody" + } + } + }, + "description": "It runs in another berth serve of the same store" + } + }, + "summary": "Stop a running job: its programs end, a session it follows ends on its host, and its record says who stopped it.", + "tags": [ + "jobs" + ] + } + }, "/v1/keys": { "get": { "operationId": "list", @@ -7502,6 +7753,21 @@ ] } }, + "/v1/resources/{id}/endpoints/{endpoint}": { + "get": { + "operationId": "stream_resource-connect", + "parameters": [], + "responses": { + "101": { + "description": "A WebSocket; its frames are in the AsyncAPI document" + } + }, + "summary": "A TCP connection to an endpoint its provisioner reported for a resource, named by the endpoint's name or its port. The server dials it first, so one that nothing answers on, or an address berth refuses, is an error and not a stream.", + "tags": [ + "streams" + ] + } + }, "/v1/resources/{id}/logs": { "get": { "operationId": "logs", @@ -7753,7 +8019,7 @@ "description": "This one is set on the server itself" } }, - "summary": "Change one of the settings that are open to callers.", + "summary": "Change one of the settings that are open to callers, or any for the user the server runs as.", "tags": [ "set-up" ] @@ -7915,7 +8181,7 @@ "description": "It could not be made; the message says why" } }, - "summary": "Make up a host name, hash or make up a password, or create a one-use enrollment key at the tailnet's control server.", + "summary": "Make up a host name, or hash or make up a password.", "tags": [ "set-up" ] diff --git a/docs/openapi/berth.operations.json b/docs/openapi/berth.operations.json index d7ed772..5471386 100644 --- a/docs/openapi/berth.operations.json +++ b/docs/openapi/berth.operations.json @@ -182,11 +182,17 @@ "type": "object" }, "Applied": { - "description": "What berth recorded on a host when it applied the configuration: its own\nversion and the plugins whose features the host has.", + "description": "What berth recorded on a host when it applied the configuration: its own\nversion, the plugins whose features the host has, and the provisioners\nthose features offer.", "properties": { "berth": { "type": "string" }, + "offers": { + "items": { + "$ref": "#/components/schemas/Offer" + }, + "type": "array" + }, "plugins": { "items": { "$ref": "#/components/schemas/AppliedPlugin" @@ -399,6 +405,43 @@ ], "type": "object" }, + "Endpoint": { + "description": "Where a resource can be reached: an address and a port, as its\nprovisioner reports it. The address is as seen from the machine the\nprovisioner's engine runs on, such as a Docker engine or an Incus server.", + "properties": { + "address": { + "description": "An IP address or a DNS name.", + "type": "string" + }, + "hint": { + "description": "What speaks there, for people, such as `http` or `postgres`.", + "type": "string" + }, + "local": { + "description": "The engine runs on the server's own machine, so the address is one\nthe server reaches as it is, a loopback address among them.", + "type": "boolean" + }, + "name": { + "description": "Unique among the resource's endpoints: lowercase letters, digits, `-`, `_` and `.`.", + "type": "string" + }, + "port": { + "format": "int32", + "minimum": 0, + "type": "integer" + }, + "protocol": { + "description": "`tcp` or `udp`.", + "type": "string" + } + }, + "required": [ + "name", + "address", + "port", + "protocol" + ], + "type": "object" + }, "ErrorBody": { "properties": { "error": { @@ -441,13 +484,21 @@ }, "label": { "type": "string" + }, + "types": { + "description": "The types of host it is for, such as `workstation` or `node`.", + "items": { + "$ref": "#/components/schemas/HostType" + }, + "type": "array" } }, "required": [ "key", "label", "group", - "help" + "help", + "types" ], "type": "object" }, @@ -498,7 +549,7 @@ ] }, "what": { - "description": "`hostname`, `password` or `tailscale_key`.", + "description": "`hostname` or `password`.", "type": "string" } }, @@ -521,7 +572,7 @@ ] }, "value": { - "description": "The host name, the password's hash, or the key.", + "description": "The host name or the password's hash.", "type": "string" } }, @@ -589,10 +640,6 @@ "null" ] }, - "level": { - "$ref": "#/components/schemas/HostLevel", - "description": "The level its profile means it to have." - }, "name": { "type": "string" }, @@ -606,14 +653,18 @@ "seen": { "oneOf": [ { - "$ref": "#/components/schemas/HostLevel", - "description": "The level it was seen to have: what its controller's record says,\n`reachable` when berth logged in and found no controller. Absent\nwhen it could not be asked." + "$ref": "#/components/schemas/HostType", + "description": "The type it was seen to be: what its controller's record says, `ssh`\nwhen berth logged in and found no controller. Absent when it could\nnot be asked." }, { "type": "null" } ] }, + "type": { + "$ref": "#/components/schemas/HostType", + "description": "The type its profile means it to be." + }, "version": { "description": "The version of its berthd.", "type": [ @@ -626,15 +677,16 @@ "name", "address", "reachable", - "level" + "type" ], "type": "object" }, - "HostLevel": { - "description": "How much of berth a host has: what its profile means it to have, or\nwhat it was seen to have. Each level can do what the ones before it can.", + "HostType": { + "description": "What berth does with a host: what its profile means it to be, or what it\nwas seen to be. They are in the order of how much of berth each has.", "enum": [ - "reachable", - "controller", + "ssh", + "managed", + "node", "workstation" ], "type": "string" @@ -697,6 +749,13 @@ "description": "The action it runs, such as `apply`, `rollout` or `provision`.", "type": "string" }, + "arguments": { + "description": "What it was started with, as people may see it: the command of a\nresource's `exec`, or an action's arguments as `NAME=VALUE`. Never a\nsecret argument, a password or standard input.", + "items": { + "type": "string" + }, + "type": "array" + }, "ended": { "type": [ "string", @@ -731,6 +790,13 @@ "started": { "type": "string" }, + "stopped_by": { + "description": "Who stopped it before it ended by itself.", + "type": [ + "string", + "null" + ] + }, "subject": { "description": "What it was done to: `host:NAME`, `provisioner:NAME`,\n`resource:ID` or `installation`. Empty for a job from before 0.5.", "type": "string" @@ -985,6 +1051,64 @@ ], "type": "object" }, + "Offer": { + "description": "Provisioners a feature of a host offers, as its tasks found them: berth\nregisters each under a name of its own that names the host. The host\nserves this to anyone who may read it, so it holds public values only,\nsuch as a server's certificate, never a key.", + "properties": { + "aliases": { + "additionalProperties": { + "items": { + "type": "string" + }, + "type": "array" + }, + "description": "Other values of a setting that reach the same server, such as the\nname a berth server on the host gives it: a provisioner registered by\nhand with one of them is named to its owner, who can make it the\nhost's.", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "feature": { + "description": "The feature, as `PLUGIN/FEATURE`.", + "type": "string" + }, + "fingerprint": { + "description": "What the host says identifies the server, such as the fingerprint of\nits certificate, for the person who reads the report.", + "type": "string" + }, + "provisioners": { + "description": "The plugin's provisioners it offers, by their ids in the plugin.", + "items": { + "type": "string" + }, + "type": "array" + }, + "secrets": { + "additionalProperties": { + "type": "string" + }, + "description": "Values of the provisioners' secrets, such as the certificate of the\nserver to pin, kept as the provisioners' own.", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "settings": { + "additionalProperties": { + "type": "string" + }, + "description": "Values of the provisioners' own arguments. `{address}`, `{user}` and\n`{port}` stand for where berth reaches the host and as whom.", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "feature", + "provisioners" + ], + "type": "object" + }, "Output": { "properties": { "data_b64": { @@ -1039,9 +1163,55 @@ ], "type": "object" }, + "PluginAction": { + "description": "An action a feature adds to a host.", + "properties": { + "agent": { + "description": "The agent may run it with `box run`.", + "type": "boolean" + }, + "arguments": { + "description": "The names of the arguments it asks for.", + "items": { + "type": "string" + }, + "type": "array" + }, + "extends": { + "description": "Actions of berth it runs after, with no row of its own.", + "items": { + "type": "string" + }, + "type": "array" + }, + "label": { + "type": "string" + }, + "name": { + "description": "The action is `PLUGIN:NAME` on the host.", + "type": "string" + }, + "user": { + "description": "`agent`, `operator` or `root`.", + "type": "string" + } + }, + "required": [ + "name", + "label", + "user" + ], + "type": "object" + }, "PluginFeature": { "description": "Something a plugin installs, switched on per host.", "properties": { + "actions": { + "items": { + "$ref": "#/components/schemas/PluginAction" + }, + "type": "array" + }, "channels": { "description": "Channel topics, without the plugin's id.", "items": { @@ -1068,12 +1238,6 @@ "$ref": "#/components/schemas/PluginService" }, "type": "array" - }, - "tasks": { - "items": { - "$ref": "#/components/schemas/PluginTask" - }, - "type": "array" } }, "required": [ @@ -1220,46 +1384,6 @@ ], "type": "object" }, - "PluginTask": { - "description": "A task a feature adds to a host's overview.", - "properties": { - "agent": { - "description": "The agent may run it with `box run`.", - "type": "boolean" - }, - "arguments": { - "description": "The names of the arguments it asks for.", - "items": { - "type": "string" - }, - "type": "array" - }, - "extends": { - "description": "Tasks of berth it runs after, with no row of its own.", - "items": { - "type": "string" - }, - "type": "array" - }, - "label": { - "type": "string" - }, - "name": { - "description": "The task is `PLUGIN:NAME` on the host.", - "type": "string" - }, - "user": { - "description": "`agent`, `operator` or `root`.", - "type": "string" - } - }, - "required": [ - "name", - "label", - "user" - ], - "type": "object" - }, "PluginUpdate": { "description": "Whether a newer version of a plugin is there, as its registry says.", "properties": { @@ -1360,6 +1484,10 @@ }, "type": "array" }, + "node": { + "description": "The node it lives on, when a feature of that host offered it.", + "type": "string" + }, "own": { "description": "The names of the provisioner's own arguments, which say where it is,\nsuch as an Incus server. Their values are kept as its settings.", "items": { @@ -1540,6 +1668,13 @@ "description": "When it was asked for, RFC 3339.", "type": "string" }, + "endpoints": { + "description": "Where it can be reached, as its provisioner last said: when it was\nmade, changed or checked.", + "items": { + "$ref": "#/components/schemas/Endpoint" + }, + "type": "array" + }, "external_id": { "description": "The provisioner's id of it; empty until `create` said it.", "type": "string" @@ -1587,7 +1722,7 @@ "type": "string" }, "verbs": { - "description": "What can be done to it besides being removed: `start`, `stop`,\n`change`, `logs`, as its kind declares.", + "description": "What can be done to it besides being removed: `start`, `stop`,\n`change`, `logs`, `exec`, as its kind declares.", "items": { "type": "string" }, @@ -1793,6 +1928,13 @@ }, "Session": { "properties": { + "action": { + "description": "The action it was started for, if any. The sessions an older\nberthd kept on the host say `task`.", + "type": [ + "string", + "null" + ] + }, "cols": { "format": "int32", "minimum": 0, @@ -1861,13 +2003,6 @@ }, "type": "array" }, - "task": { - "description": "The task it was started for, if any.", - "type": [ - "string", - "null" - ] - }, "title": { "type": "string" }, @@ -2026,21 +2161,21 @@ } ] }, - "level": { + "mode": { + "description": "`system` or `single`.", + "type": "string" + }, + "type": { "oneOf": [ { - "$ref": "#/components/schemas/HostLevel", - "description": "What was installed on the host, by that record: `controller` when\nonly the controller was, `workstation` for a whole profile. Absent\nwhen the host keeps no record." + "$ref": "#/components/schemas/HostType", + "description": "What was installed on the host, by that record: `managed` when\nonly the controller was, `workstation` for a whole profile. Absent\nwhen the host keeps no record." }, { "type": "null" } ] }, - "mode": { - "description": "`system` or `single`.", - "type": "string" - }, "version": { "type": "string" } @@ -2053,7 +2188,7 @@ } } }, - "description": "The operations of berth serve. Every operation of a host is one here too: on the portal with `\"host\": \"NAME\"` in the frame, over HTTP under /v1/hosts/NAME. Only the `secrets` task is refused; the host's `secrets` action sends the server's own. An action's own role is asked when it runs. Each operation names the least role a caller over TCP needs: view, operate or admin.", + "description": "The operations of berth serve. Every operation of a host is one here too: on the portal with `\"host\": \"NAME\"` in the frame, over HTTP under /v1/hosts/NAME/controller. Only the controller's own `secrets` action is refused, for which the server's `secrets` action of the host sends the server's secrets, and `endpoints.connect`, which the server opens itself for `resources.connect`. An action's own role is asked when it runs. Each operation names the least role a caller over TCP needs: view, operate or admin.", "hosts": true, "operations": [ { @@ -2297,7 +2432,7 @@ } ], "data": { - "description": "The whole profile. A workstation's has a builder object; a reachable or controller host's needs only level and access", + "description": "The whole profile. A workstation's has a builder object; a host of the type ssh or managed needs only type and access", "schema": {} }, "errors": [ @@ -2320,7 +2455,7 @@ } }, "role": "admin", - "summary": "Save a host's profile: a new host, or a changed one. A host below a workstation needs only its level and access." + "summary": "Save a host's profile: a new host, or a changed one. A host that is not a workstation needs only its type and access." }, { "args": [ @@ -2385,7 +2520,7 @@ } }, "role": "view", - "summary": "What can be done to a host: the tasks its controller lists, and berth's own actions." + "summary": "What can be done to a host: the actions its controller lists, and berth's own." }, { "args": [ @@ -2455,17 +2590,17 @@ } }, "role": "view", - "summary": "Run one of a host's actions. One of berth's that changes something is a job; a task of the host is a session there, which a job follows unless it needs a terminal." + "summary": "Run one of a host's actions. One of berth's that changes something is a job; an action of the host's controller is a session there, which a job follows unless it needs a terminal." }, { "args": [], - "group": "actions", + "group": "installation", "http": { "method": "GET", - "path": "/v1/actions" + "path": "/v1/installation/actions" }, "kind": "call", - "name": "actions.list", + "name": "installation.actions", "portal": true, "result": { "schema": { @@ -2481,7 +2616,7 @@ { "args": [ { - "description": "The action, as actions.list names it", + "description": "The action, as installation.actions names it", "in": "path", "name": "action", "required": true @@ -2518,13 +2653,13 @@ "status": "409" } ], - "group": "actions", + "group": "installation", "http": { "method": "POST", - "path": "/v1/actions/{action}" + "path": "/v1/installation/actions/{action}" }, "kind": "call", - "name": "actions.act", + "name": "installation.act", "portal": true, "result": null, "role": "view", @@ -2913,7 +3048,7 @@ } }, "role": "admin", - "summary": "Change one of the settings that are open to callers." + "summary": "Change one of the settings that are open to callers, or any for the user the server runs as." }, { "args": [], @@ -3136,7 +3271,7 @@ } }, "role": "admin", - "summary": "Make up a host name, hash or make up a password, or create a one-use enrollment key at the tailnet's control server." + "summary": "Make up a host name, or hash or make up a password." }, { "args": [], @@ -3776,6 +3911,65 @@ "sends": [], "summary": "What a resource writes, as its provisioner gives it, with secrets masked: the last lines, then what follows." }, + { + "args": [ + { + "description": null, + "in": "path", + "name": "id", + "required": true + }, + { + "description": null, + "in": "path", + "name": "endpoint", + "required": true + } + ], + "group": "resources", + "http": { + "method": "GET", + "path": "/v1/resources/{id}/endpoints/{endpoint}" + }, + "kind": "stream", + "name": "resources.connect", + "portal": true, + "receives": [ + { + "binary": true, + "description": "What the endpoint sent.", + "example": null, + "name": "bytes" + }, + { + "binary": false, + "description": "The endpoint closed its side. Bytes sent towards it still arrive. The stream ends once both sides have sent this.", + "example": { + "eof": true + }, + "name": "eof" + } + ], + "resumable": false, + "role": "operate", + "sends": [ + { + "binary": true, + "description": "Bytes for the endpoint.", + "example": null, + "name": "bytes" + }, + { + "binary": false, + "description": "The client sends no more, and the endpoint reads the end. What it sends still comes back.", + "example": { + "eof": true + }, + "name": "eof" + } + ], + "summary": "A TCP connection to an endpoint its provisioner reported for a resource, named by the endpoint's name or its port. The server dials it first, so one that nothing answers on, or an address berth refuses, is an error and not a stream." + }, { "args": [ { @@ -3948,6 +4142,45 @@ "role": "view", "summary": "What a job wrote, by bytes or by lines." }, + { + "args": [ + { + "description": "A job's id", + "in": "path", + "name": "id", + "required": true + } + ], + "errors": [ + { + "description": "The caller may not start a job of its kind, so may not stop it", + "status": "403" + }, + { + "description": "No such job", + "status": "404" + }, + { + "description": "It has ended already", + "status": "409" + }, + { + "description": "It runs in another berth serve of the same store", + "status": "501" + } + ], + "group": "jobs", + "http": { + "method": "POST", + "path": "/v1/jobs/{id}/stop" + }, + "kind": "call", + "name": "jobs.stop", + "portal": true, + "result": null, + "role": "view", + "summary": "Stop a running job: its programs end, a session it follows ends on its host, and its record says who stopped it." + }, { "args": [ { @@ -4043,6 +4276,16 @@ "topic": "myplugin/progress" }, "name": "message" + }, + { + "binary": false, + "description": "This follower was too slow. `events` counts the server's events it missed, of every kind. Read the inbox again.", + "example": { + "dropped": { + "events": 12 + } + }, + "name": "dropped" } ], "resumable": false, @@ -4187,6 +4430,16 @@ "removed": true }, "name": "removed" + }, + { + "binary": false, + "description": "This follower was too slow. `events` counts the server's events it missed, of every kind. List the namespace again.", + "example": { + "dropped": { + "events": 12 + } + }, + "name": "dropped" } ], "resumable": false, @@ -4395,6 +4648,39 @@ "value": {} }, "name": "kv" + }, + { + "binary": false, + "description": "A saved host was `added`, `removed`, `renamed` (with the name it had `from`) or its profile `changed`. Read the host itself with hosts.get.", + "example": { + "change": "renamed", + "from": "desk", + "host": "studio", + "source": "host" + }, + "name": "host" + }, + { + "binary": false, + "description": "A resource was `created`, `removed` or its record `changed`. Read it with resources.get.", + "example": { + "change": "created", + "kind": "deployment", + "provisioner": "docker", + "resource": "r-k3v9qz2m", + "source": "resource" + }, + "name": "resource" + }, + { + "binary": false, + "description": "This follower was too slow. `events` counts the server's events it missed, of every kind. Read what it follows again.", + "example": { + "dropped": { + "events": 12 + } + }, + "name": "dropped" } ], "resumable": false, @@ -4404,6 +4690,10 @@ "binary": false, "description": "What to follow. Sent once, first.", "example": { + "changes": [ + "hosts", + "resources" + ], "hosts": [ "server" ], @@ -4418,7 +4708,7 @@ "name": "follow" } ], - "summary": "Channel messages of chosen hosts, jobs and key-value changes together. The first frame sent names what to follow; each frame received says where it came from." + "summary": "Channel messages of chosen hosts, jobs, key-value changes and changes to hosts and resources together. The first frame sent names what to follow; each frame received says where it came from." } ], "portal": { diff --git a/docs/openapi/berthd.asyncapi.json b/docs/openapi/berthd.asyncapi.json index ebe4ad0..fe91273 100644 --- a/docs/openapi/berthd.asyncapi.json +++ b/docs/openapi/berthd.asyncapi.json @@ -33,6 +33,36 @@ "x-berth-level": "both", "x-berth-resumable": true }, + "endpoint": { + "address": "/v1/endpoints/{address}/{port}", + "bindings": { + "ws": { + "query": { + "properties": {}, + "type": "object" + } + } + }, + "description": "For berth serve alone: a TCP connection to an address on a network of the host's own interfaces, such as a container on a bridge. The address is resolved once, every address it has is checked, and the one checked is dialled first, so one that nothing answers on is an error and not a stream.", + "messages": { + "endpoint.bytes": { + "$ref": "#/components/messages/endpoint.bytes" + }, + "endpoint.eof": { + "$ref": "#/components/messages/endpoint.eof" + } + }, + "parameters": { + "address": { + "description": "address" + }, + "port": { + "description": "port" + } + }, + "x-berth-level": "host", + "x-berth-resumable": false + }, "logs": { "address": "/v1/logs", "bindings": { @@ -62,6 +92,33 @@ "x-berth-level": "host", "x-berth-resumable": true }, + "port": { + "address": "/v1/ports/{port}", + "bindings": { + "ws": { + "query": { + "properties": {}, + "type": "object" + } + } + }, + "description": "A TCP connection to a port on the host's loopback, 127.0.0.1. The port is dialled first, so one that nothing listens on is an error and not a stream.", + "messages": { + "port.bytes": { + "$ref": "#/components/messages/port.bytes" + }, + "port.eof": { + "$ref": "#/components/messages/port.eof" + } + }, + "parameters": { + "port": { + "description": "port" + } + }, + "x-berth-level": "host", + "x-berth-resumable": false + }, "serve": { "address": "/v1/services/{service}/serve", "bindings": { @@ -186,6 +243,30 @@ }, "summary": "Publish a message." }, + "endpoint.bytes": { + "contentType": "application/octet-stream", + "name": "bytes", + "payload": { + "format": "binary", + "type": "string" + }, + "summary": "Bytes for the endpoint." + }, + "endpoint.eof": { + "contentType": "application/json", + "examples": [ + { + "payload": { + "eof": true + } + } + ], + "name": "eof", + "payload": { + "type": "object" + }, + "summary": "The client sends no more, and the endpoint reads the end. What it sends still comes back." + }, "logs.entry": { "contentType": "application/json", "examples": [ @@ -205,6 +286,30 @@ }, "summary": "One journal entry." }, + "port.bytes": { + "contentType": "application/octet-stream", + "name": "bytes", + "payload": { + "format": "binary", + "type": "string" + }, + "summary": "Bytes for the program on the port." + }, + "port.eof": { + "contentType": "application/json", + "examples": [ + { + "payload": { + "eof": true + } + } + ], + "name": "eof", + "payload": { + "type": "object" + }, + "summary": "The client sends no more, and the program on the port reads the end. What it sends still comes back." + }, "serve.call": { "contentType": "application/json", "examples": [ @@ -394,6 +499,34 @@ } ] }, + "endpoint.receive": { + "action": "receive", + "channel": { + "$ref": "#/channels/endpoint" + }, + "messages": [ + { + "$ref": "#/channels/endpoint/messages/endpoint.bytes" + }, + { + "$ref": "#/channels/endpoint/messages/endpoint.eof" + } + ] + }, + "endpoint.send": { + "action": "send", + "channel": { + "$ref": "#/channels/endpoint" + }, + "messages": [ + { + "$ref": "#/channels/endpoint/messages/endpoint.bytes" + }, + { + "$ref": "#/channels/endpoint/messages/endpoint.eof" + } + ] + }, "logs.send": { "action": "send", "channel": { @@ -405,6 +538,34 @@ } ] }, + "port.receive": { + "action": "receive", + "channel": { + "$ref": "#/channels/port" + }, + "messages": [ + { + "$ref": "#/channels/port/messages/port.bytes" + }, + { + "$ref": "#/channels/port/messages/port.eof" + } + ] + }, + "port.send": { + "action": "send", + "channel": { + "$ref": "#/channels/port" + }, + "messages": [ + { + "$ref": "#/channels/port/messages/port.bytes" + }, + { + "$ref": "#/channels/port/messages/port.eof" + } + ] + }, "serve.receive": { "action": "receive", "channel": { diff --git a/docs/openapi/berthd.openapi.json b/docs/openapi/berthd.openapi.json index 5da3811..77b4e37 100644 --- a/docs/openapi/berthd.openapi.json +++ b/docs/openapi/berthd.openapi.json @@ -2,11 +2,17 @@ "components": { "schemas": { "Applied": { - "description": "What berth recorded on a host when it applied the configuration: its own\nversion and the plugins whose features the host has.", + "description": "What berth recorded on a host when it applied the configuration: its own\nversion, the plugins whose features the host has, and the provisioners\nthose features offer.", "properties": { "berth": { "type": "string" }, + "offers": { + "items": { + "$ref": "#/components/schemas/Offer" + }, + "type": "array" + }, "plugins": { "items": { "$ref": "#/components/schemas/AppliedPlugin" @@ -337,11 +343,72 @@ ], "type": "object" }, - "HostLevel": { - "description": "How much of berth a host has: what its profile means it to have, or\nwhat it was seen to have. Each level can do what the ones before it can.", + "HostAction": { + "description": "An action of the host's controller: an entry of `/etc/berth/actions.json`.\nListed without `run`.", + "properties": { + "agent": { + "description": "The agent may run it.", + "type": "boolean" + }, + "arguments": { + "items": { + "$ref": "#/components/schemas/Argument" + }, + "type": "array" + }, + "confirm": { + "type": [ + "string", + "null" + ] + }, + "extends": { + "description": "Actions it is a step of; it then has no row of its own.", + "items": { + "type": "string" + }, + "type": "array" + }, + "group": { + "type": "string" + }, + "help": { + "type": "string" + }, + "input": { + "description": "What the workstation sends on standard input: `secrets`.", + "type": [ + "string", + "null" + ] + }, + "label": { + "type": "string" + }, + "name": { + "type": "string" + }, + "run": { + "type": "string" + }, + "terminal": { + "type": "boolean" + }, + "user": { + "type": "string" + } + }, + "required": [ + "name" + ], + "type": "object" + }, + "HostType": { + "description": "What berth does with a host: what its profile means it to be, or what it\nwas seen to be. They are in the order of how much of berth each has.", "enum": [ - "reachable", - "controller", + "ssh", + "managed", + "node", "workstation" ], "type": "string" @@ -585,6 +652,64 @@ ], "type": "object" }, + "Offer": { + "description": "Provisioners a feature of a host offers, as its tasks found them: berth\nregisters each under a name of its own that names the host. The host\nserves this to anyone who may read it, so it holds public values only,\nsuch as a server's certificate, never a key.", + "properties": { + "aliases": { + "additionalProperties": { + "items": { + "type": "string" + }, + "type": "array" + }, + "description": "Other values of a setting that reach the same server, such as the\nname a berth server on the host gives it: a provisioner registered by\nhand with one of them is named to its owner, who can make it the\nhost's.", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "feature": { + "description": "The feature, as `PLUGIN/FEATURE`.", + "type": "string" + }, + "fingerprint": { + "description": "What the host says identifies the server, such as the fingerprint of\nits certificate, for the person who reads the report.", + "type": "string" + }, + "provisioners": { + "description": "The plugin's provisioners it offers, by their ids in the plugin.", + "items": { + "type": "string" + }, + "type": "array" + }, + "secrets": { + "additionalProperties": { + "type": "string" + }, + "description": "Values of the provisioners' secrets, such as the certificate of the\nserver to pin, kept as the provisioners' own.", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "settings": { + "additionalProperties": { + "type": "string" + }, + "description": "Values of the provisioners' own arguments. `{address}`, `{user}` and\n`{port}` stand for where berth reaches the host and as whom.", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "feature", + "provisioners" + ], + "type": "object" + }, "Output": { "properties": { "data_b64": { @@ -762,7 +887,7 @@ ], "type": "object" }, - "RunTask": { + "RunAction": { "properties": { "arguments": { "additionalProperties": { @@ -918,6 +1043,13 @@ }, "Session": { "properties": { + "action": { + "description": "The action it was started for, if any. The sessions an older\nberthd kept on the host say `task`.", + "type": [ + "string", + "null" + ] + }, "cols": { "format": "int32", "minimum": 0, @@ -986,13 +1118,6 @@ }, "type": "array" }, - "task": { - "description": "The task it was started for, if any.", - "type": [ - "string", - "null" - ] - }, "title": { "type": "string" }, @@ -1255,66 +1380,6 @@ ], "type": "object" }, - "Task": { - "description": "An entry of `/etc/berth/tasks.json`. Listed without `run`.", - "properties": { - "agent": { - "description": "The agent may run it.", - "type": "boolean" - }, - "arguments": { - "items": { - "$ref": "#/components/schemas/Argument" - }, - "type": "array" - }, - "confirm": { - "type": [ - "string", - "null" - ] - }, - "extends": { - "description": "Tasks it is a step of; it then has no row of its own.", - "items": { - "type": "string" - }, - "type": "array" - }, - "group": { - "type": "string" - }, - "help": { - "type": "string" - }, - "input": { - "description": "What the workstation sends on standard input: `secrets`.", - "type": [ - "string", - "null" - ] - }, - "label": { - "type": "string" - }, - "name": { - "type": "string" - }, - "run": { - "type": "string" - }, - "terminal": { - "type": "boolean" - }, - "user": { - "type": "string" - } - }, - "required": [ - "name" - ], - "type": "object" - }, "Topic": { "description": "An entry of `/etc/berth/channels.json`.", "properties": { @@ -1413,21 +1478,21 @@ } ] }, - "level": { + "mode": { + "description": "`system` or `single`.", + "type": "string" + }, + "type": { "oneOf": [ { - "$ref": "#/components/schemas/HostLevel", - "description": "What was installed on the host, by that record: `controller` when\nonly the controller was, `workstation` for a whole profile. Absent\nwhen the host keeps no record." + "$ref": "#/components/schemas/HostType", + "description": "What was installed on the host, by that record: `managed` when\nonly the controller was, `workstation` for a whole profile. Absent\nwhen the host keeps no record." }, { "type": "null" } ] }, - "mode": { - "description": "`system` or `single`.", - "type": "string" - }, "version": { "type": "string" } @@ -1450,6 +1515,151 @@ }, "openapi": "3.1.0", "paths": { + "/v1/actions": { + "get": { + "description": "Host and agent level.", + "operationId": "get_v1_actions", + "parameters": [], + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "items": { + "$ref": "#/components/schemas/HostAction" + }, + "type": "array" + } + } + }, + "description": "Done." + }, + "default": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorBody" + } + } + }, + "description": "An error." + } + }, + "summary": "The actions the caller may run, in the order of the file.", + "tags": [ + "actions" + ] + } + }, + "/v1/actions/{action}": { + "post": { + "description": "Host and agent level.", + "operationId": "post_v1_actions__action_", + "parameters": [ + { + "in": "path", + "name": "action", + "required": true, + "schema": { + "type": "string" + } + }, + { + "description": "A duration such as 30s, at most 10m: answer when the command has ended, with what it wrote.", + "in": "query", + "name": "wait", + "schema": { + "type": "string" + } + } + ], + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/RunAction" + } + } + } + }, + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Started" + } + } + }, + "description": "Done." + }, + "default": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorBody" + } + } + }, + "description": "An error." + } + }, + "summary": "Run an action. It and what extends it are the steps of one session.", + "tags": [ + "actions" + ] + } + }, + "/v1/actions/{action}/choices/{argument}": { + "get": { + "description": "Host and agent level.", + "operationId": "get_v1_actions__action__choices__argument_", + "parameters": [ + { + "in": "path", + "name": "action", + "required": true, + "schema": { + "type": "string" + } + }, + { + "in": "path", + "name": "argument", + "required": true, + "schema": { + "type": "string" + } + } + ], + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Choices" + } + } + }, + "description": "Done." + }, + "default": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorBody" + } + } + }, + "description": "An error." + } + }, + "summary": "The values to offer for an argument.", + "tags": [ + "actions" + ] + } + }, "/v1/applied/profile": { "get": { "description": "Host level only.", @@ -1595,6 +1805,49 @@ ] } }, + "/v1/endpoints/{address}/{port}": { + "get": { + "description": "Host level only.", + "operationId": "get_v1_endpoints__address___port_", + "parameters": [ + { + "in": "path", + "name": "address", + "required": true, + "schema": { + "type": "string" + } + }, + { + "in": "path", + "name": "port", + "required": true, + "schema": { + "type": "string" + } + } + ], + "responses": { + "101": { + "description": "With a WebSocket handshake: the stream `endpoint` of the AsyncAPI document." + }, + "default": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorBody" + } + } + }, + "description": "An error." + } + }, + "summary": "Only as a WebSocket, and only for berth serve itself: a TCP connection to an address on a network of the host's own interfaces, such as a container on a bridge. Never loopback, link-local or a metadata address. One that nothing answers on is an error before the handshake.", + "tags": [ + "endpoints" + ] + } + }, "/v1/files": { "delete": { "description": "Host level only.", @@ -1808,6 +2061,41 @@ ] } }, + "/v1/ports/{port}": { + "get": { + "description": "Host level only.", + "operationId": "get_v1_ports__port_", + "parameters": [ + { + "in": "path", + "name": "port", + "required": true, + "schema": { + "type": "string" + } + } + ], + "responses": { + "101": { + "description": "With a WebSocket handshake: the stream `port` of the AsyncAPI document." + }, + "default": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorBody" + } + } + }, + "description": "An error." + } + }, + "summary": "Only as a WebSocket: a TCP connection to a port on the host's loopback. A port that nothing listens on is an error before the handshake.", + "tags": [ + "ports" + ] + } + }, "/v1/power/{verb}": { "post": { "description": "Host level only.", @@ -2318,7 +2606,7 @@ "description": "An error." } }, - "summary": "One session's state. The agent sees those it started through a task.", + "summary": "One session's state. The agent sees those it started through an action.", "tags": [ "sessions" ] @@ -2658,151 +2946,6 @@ ] } }, - "/v1/tasks": { - "get": { - "description": "Host and agent level.", - "operationId": "get_v1_tasks", - "parameters": [], - "responses": { - "200": { - "content": { - "application/json": { - "schema": { - "items": { - "$ref": "#/components/schemas/Task" - }, - "type": "array" - } - } - }, - "description": "Done." - }, - "default": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/ErrorBody" - } - } - }, - "description": "An error." - } - }, - "summary": "The tasks the caller may run, in the order of the file.", - "tags": [ - "tasks" - ] - } - }, - "/v1/tasks/{name}": { - "post": { - "description": "Host and agent level.", - "operationId": "post_v1_tasks__name_", - "parameters": [ - { - "in": "path", - "name": "name", - "required": true, - "schema": { - "type": "string" - } - }, - { - "description": "A duration such as 30s, at most 10m: answer when the command has ended, with what it wrote.", - "in": "query", - "name": "wait", - "schema": { - "type": "string" - } - } - ], - "requestBody": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/RunTask" - } - } - } - }, - "responses": { - "200": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/Started" - } - } - }, - "description": "Done." - }, - "default": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/ErrorBody" - } - } - }, - "description": "An error." - } - }, - "summary": "Run a task. It and what extends it are the steps of one session.", - "tags": [ - "tasks" - ] - } - }, - "/v1/tasks/{name}/choices/{argument}": { - "get": { - "description": "Host and agent level.", - "operationId": "get_v1_tasks__name__choices__argument_", - "parameters": [ - { - "in": "path", - "name": "name", - "required": true, - "schema": { - "type": "string" - } - }, - { - "in": "path", - "name": "argument", - "required": true, - "schema": { - "type": "string" - } - } - ], - "responses": { - "200": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/Choices" - } - } - }, - "description": "Done." - }, - "default": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/ErrorBody" - } - } - }, - "description": "An error." - } - }, - "summary": "The values to offer for an argument.", - "tags": [ - "tasks" - ] - } - }, "/v1/units": { "get": { "description": "Host level only.", diff --git a/docs/openapi/berthd.operations.json b/docs/openapi/berthd.operations.json index 9720f93..6203557 100644 --- a/docs/openapi/berthd.operations.json +++ b/docs/openapi/berthd.operations.json @@ -2,11 +2,17 @@ "components": { "schemas": { "Applied": { - "description": "What berth recorded on a host when it applied the configuration: its own\nversion and the plugins whose features the host has.", + "description": "What berth recorded on a host when it applied the configuration: its own\nversion, the plugins whose features the host has, and the provisioners\nthose features offer.", "properties": { "berth": { "type": "string" }, + "offers": { + "items": { + "$ref": "#/components/schemas/Offer" + }, + "type": "array" + }, "plugins": { "items": { "$ref": "#/components/schemas/AppliedPlugin" @@ -337,11 +343,72 @@ ], "type": "object" }, - "HostLevel": { - "description": "How much of berth a host has: what its profile means it to have, or\nwhat it was seen to have. Each level can do what the ones before it can.", + "HostAction": { + "description": "An action of the host's controller: an entry of `/etc/berth/actions.json`.\nListed without `run`.", + "properties": { + "agent": { + "description": "The agent may run it.", + "type": "boolean" + }, + "arguments": { + "items": { + "$ref": "#/components/schemas/Argument" + }, + "type": "array" + }, + "confirm": { + "type": [ + "string", + "null" + ] + }, + "extends": { + "description": "Actions it is a step of; it then has no row of its own.", + "items": { + "type": "string" + }, + "type": "array" + }, + "group": { + "type": "string" + }, + "help": { + "type": "string" + }, + "input": { + "description": "What the workstation sends on standard input: `secrets`.", + "type": [ + "string", + "null" + ] + }, + "label": { + "type": "string" + }, + "name": { + "type": "string" + }, + "run": { + "type": "string" + }, + "terminal": { + "type": "boolean" + }, + "user": { + "type": "string" + } + }, + "required": [ + "name" + ], + "type": "object" + }, + "HostType": { + "description": "What berth does with a host: what its profile means it to be, or what it\nwas seen to be. They are in the order of how much of berth each has.", "enum": [ - "reachable", - "controller", + "ssh", + "managed", + "node", "workstation" ], "type": "string" @@ -585,6 +652,64 @@ ], "type": "object" }, + "Offer": { + "description": "Provisioners a feature of a host offers, as its tasks found them: berth\nregisters each under a name of its own that names the host. The host\nserves this to anyone who may read it, so it holds public values only,\nsuch as a server's certificate, never a key.", + "properties": { + "aliases": { + "additionalProperties": { + "items": { + "type": "string" + }, + "type": "array" + }, + "description": "Other values of a setting that reach the same server, such as the\nname a berth server on the host gives it: a provisioner registered by\nhand with one of them is named to its owner, who can make it the\nhost's.", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "feature": { + "description": "The feature, as `PLUGIN/FEATURE`.", + "type": "string" + }, + "fingerprint": { + "description": "What the host says identifies the server, such as the fingerprint of\nits certificate, for the person who reads the report.", + "type": "string" + }, + "provisioners": { + "description": "The plugin's provisioners it offers, by their ids in the plugin.", + "items": { + "type": "string" + }, + "type": "array" + }, + "secrets": { + "additionalProperties": { + "type": "string" + }, + "description": "Values of the provisioners' secrets, such as the certificate of the\nserver to pin, kept as the provisioners' own.", + "propertyNames": { + "type": "string" + }, + "type": "object" + }, + "settings": { + "additionalProperties": { + "type": "string" + }, + "description": "Values of the provisioners' own arguments. `{address}`, `{user}` and\n`{port}` stand for where berth reaches the host and as whom.", + "propertyNames": { + "type": "string" + }, + "type": "object" + } + }, + "required": [ + "feature", + "provisioners" + ], + "type": "object" + }, "Output": { "properties": { "data_b64": { @@ -762,7 +887,7 @@ ], "type": "object" }, - "RunTask": { + "RunAction": { "properties": { "arguments": { "additionalProperties": { @@ -918,6 +1043,13 @@ }, "Session": { "properties": { + "action": { + "description": "The action it was started for, if any. The sessions an older\nberthd kept on the host say `task`.", + "type": [ + "string", + "null" + ] + }, "cols": { "format": "int32", "minimum": 0, @@ -986,13 +1118,6 @@ }, "type": "array" }, - "task": { - "description": "The task it was started for, if any.", - "type": [ - "string", - "null" - ] - }, "title": { "type": "string" }, @@ -1255,66 +1380,6 @@ ], "type": "object" }, - "Task": { - "description": "An entry of `/etc/berth/tasks.json`. Listed without `run`.", - "properties": { - "agent": { - "description": "The agent may run it.", - "type": "boolean" - }, - "arguments": { - "items": { - "$ref": "#/components/schemas/Argument" - }, - "type": "array" - }, - "confirm": { - "type": [ - "string", - "null" - ] - }, - "extends": { - "description": "Tasks it is a step of; it then has no row of its own.", - "items": { - "type": "string" - }, - "type": "array" - }, - "group": { - "type": "string" - }, - "help": { - "type": "string" - }, - "input": { - "description": "What the workstation sends on standard input: `secrets`.", - "type": [ - "string", - "null" - ] - }, - "label": { - "type": "string" - }, - "name": { - "type": "string" - }, - "run": { - "type": "string" - }, - "terminal": { - "type": "boolean" - }, - "user": { - "type": "string" - } - }, - "required": [ - "name" - ], - "type": "object" - }, "Topic": { "description": "An entry of `/etc/berth/channels.json`.", "properties": { @@ -1413,21 +1478,21 @@ } ] }, - "level": { + "mode": { + "description": "`system` or `single`.", + "type": "string" + }, + "type": { "oneOf": [ { - "$ref": "#/components/schemas/HostLevel", - "description": "What was installed on the host, by that record: `controller` when\nonly the controller was, `workstation` for a whole profile. Absent\nwhen the host keeps no record." + "$ref": "#/components/schemas/HostType", + "description": "What was installed on the host, by that record: `managed` when\nonly the controller was, `workstation` for a whole profile. Absent\nwhen the host keeps no record." }, { "type": "null" } ] }, - "mode": { - "description": "`system` or `single`.", - "type": "string" - }, "version": { "type": "string" } @@ -1578,7 +1643,7 @@ } }, "role": "view", - "summary": "One session's state. The agent sees those it started through a task." + "summary": "One session's state. The agent sees those it started through an action." }, { "args": [ @@ -1903,32 +1968,32 @@ }, { "args": [], - "group": "tasks", + "group": "actions", "http": { "method": "GET", - "path": "/v1/tasks" + "path": "/v1/actions" }, "kind": "call", "level": "Host and agent level.", - "name": "tasks.list", + "name": "actions.list", "portal": true, "result": { "schema": { "items": { - "$ref": "#/components/schemas/Task" + "$ref": "#/components/schemas/HostAction" }, "type": "array" } }, "role": "view", - "summary": "The tasks the caller may run, in the order of the file." + "summary": "The actions the caller may run, in the order of the file." }, { "args": [ { - "description": "A task's name, as the list gives it.", + "description": "An action's name, as the list gives it.", "in": "path", - "name": "name", + "name": "action", "required": true }, { @@ -1942,17 +2007,17 @@ "data": { "description": null, "schema": { - "$ref": "#/components/schemas/RunTask" + "$ref": "#/components/schemas/RunAction" } }, - "group": "tasks", + "group": "actions", "http": { "method": "POST", - "path": "/v1/tasks/{name}" + "path": "/v1/actions/{action}" }, "kind": "call", "level": "Host and agent level.", - "name": "tasks.run", + "name": "actions.run", "portal": true, "result": { "schema": { @@ -1960,31 +2025,31 @@ } }, "role": "operate", - "summary": "Run a task. It and what extends it are the steps of one session." + "summary": "Run an action. It and what extends it are the steps of one session." }, { "args": [ { - "description": "A task's name, as the list gives it.", + "description": "An action's name, as the list gives it.", "in": "path", - "name": "name", + "name": "action", "required": true }, { - "description": "The name of one of the task's arguments.", + "description": "The name of one of the action's arguments.", "in": "path", "name": "argument", "required": true } ], - "group": "tasks", + "group": "actions", "http": { "method": "GET", - "path": "/v1/tasks/{name}/choices/{argument}" + "path": "/v1/actions/{action}/choices/{argument}" }, "kind": "call", "level": "Host and agent level.", - "name": "tasks.choices", + "name": "actions.choices", "portal": true, "result": { "schema": { @@ -2650,6 +2715,121 @@ } ], "summary": "For the program that answers a service's calls. While it is open the service is up." + }, + { + "args": [ + { + "description": "A TCP port on the host's loopback, 127.0.0.1: a number from 1 to 65535.", + "in": "path", + "name": "port", + "required": true + } + ], + "group": "ports", + "http": { + "method": "GET", + "path": "/v1/ports/{port}" + }, + "kind": "stream", + "level": "Host level only.", + "name": "ports.connect", + "portal": true, + "receives": [ + { + "binary": true, + "description": "What the program on the port sent.", + "example": null, + "name": "bytes" + }, + { + "binary": false, + "description": "The program on the port closed its side. Bytes sent towards it still arrive. The stream ends once both sides have sent this.", + "example": { + "eof": true + }, + "name": "eof" + } + ], + "resumable": false, + "role": "operate", + "sends": [ + { + "binary": true, + "description": "Bytes for the program on the port.", + "example": null, + "name": "bytes" + }, + { + "binary": false, + "description": "The client sends no more, and the program on the port reads the end. What it sends still comes back.", + "example": { + "eof": true + }, + "name": "eof" + } + ], + "summary": "A TCP connection to a port on the host's loopback, 127.0.0.1. The port is dialled first, so one that nothing listens on is an error and not a stream." + }, + { + "args": [ + { + "description": "An IP address or a DNS name, on a network of the host's own interfaces.", + "in": "path", + "name": "address", + "required": true + }, + { + "description": "A TCP port: a number from 1 to 65535.", + "in": "path", + "name": "port", + "required": true + } + ], + "group": "endpoints", + "http": { + "method": "GET", + "path": "/v1/endpoints/{address}/{port}" + }, + "http_only": "berth serve dials it itself, only for an endpoint a provisioner reported: open the endpoint with resources.connect", + "kind": "stream", + "level": "Host level only.", + "name": "endpoints.connect", + "portal": false, + "receives": [ + { + "binary": true, + "description": "What the endpoint sent.", + "example": null, + "name": "bytes" + }, + { + "binary": false, + "description": "The endpoint closed its side. Bytes sent towards it still arrive. The stream ends once both sides have sent this.", + "example": { + "eof": true + }, + "name": "eof" + } + ], + "resumable": false, + "role": "operate", + "sends": [ + { + "binary": true, + "description": "Bytes for the endpoint.", + "example": null, + "name": "bytes" + }, + { + "binary": false, + "description": "The client sends no more, and the endpoint reads the end. What it sends still comes back.", + "example": { + "eof": true + }, + "name": "eof" + } + ], + "summary": "For berth serve alone: a TCP connection to an address on a network of the host's own interfaces, such as a container on a bridge. The address is resolved once, every address it has is checked, and the one checked is dialled first, so one that nothing answers on is an error and not a stream." } ], "portal": { diff --git a/docs/public/screens/accounts.png b/docs/public/screens/accounts.png index bcda744..ce120fd 100644 Binary files a/docs/public/screens/accounts.png and b/docs/public/screens/accounts.png differ diff --git a/docs/public/screens/actions.png b/docs/public/screens/actions.png new file mode 100644 index 0000000..8b4eab7 Binary files /dev/null and b/docs/public/screens/actions.png differ diff --git a/docs/public/screens/features.png b/docs/public/screens/features.png index 6d3bee9..a5f6e64 100644 Binary files a/docs/public/screens/features.png and b/docs/public/screens/features.png differ diff --git a/docs/public/screens/home-plugins.png b/docs/public/screens/home-plugins.png index 30044d9..f3a1bb5 100644 Binary files a/docs/public/screens/home-plugins.png and b/docs/public/screens/home-plugins.png differ diff --git a/docs/public/screens/home.png b/docs/public/screens/home.png index fd89c88..edf6938 100644 Binary files a/docs/public/screens/home.png and b/docs/public/screens/home.png differ diff --git a/docs/public/screens/host.png b/docs/public/screens/host.png index 4ad40c2..7731812 100644 Binary files a/docs/public/screens/host.png and b/docs/public/screens/host.png differ diff --git a/docs/public/screens/install-cloud.png b/docs/public/screens/install-cloud.png index a47a1a6..4ccc936 100644 Binary files a/docs/public/screens/install-cloud.png and b/docs/public/screens/install-cloud.png differ diff --git a/docs/public/screens/list.png b/docs/public/screens/list.png index 42c91f8..17dc623 100644 Binary files a/docs/public/screens/list.png and b/docs/public/screens/list.png differ diff --git a/docs/public/screens/overview.png b/docs/public/screens/overview.png index 9bcfded..f8518af 100644 Binary files a/docs/public/screens/overview.png and b/docs/public/screens/overview.png differ diff --git a/docs/public/screens/secrets.png b/docs/public/screens/secrets.png index e1b95ba..9430f21 100644 Binary files a/docs/public/screens/secrets.png and b/docs/public/screens/secrets.png differ diff --git a/docs/public/screens/sessions.png b/docs/public/screens/sessions.png index 5809eca..67736cb 100644 Binary files a/docs/public/screens/sessions.png and b/docs/public/screens/sessions.png differ diff --git a/docs/public/screens/tasks.png b/docs/public/screens/tasks.png deleted file mode 100644 index 8194fce..0000000 Binary files a/docs/public/screens/tasks.png and /dev/null differ diff --git a/docs/public/screens/terminal.png b/docs/public/screens/terminal.png index 5aaa4c9..e37b743 100644 Binary files a/docs/public/screens/terminal.png and b/docs/public/screens/terminal.png differ diff --git a/docs/scripts/gather.mjs b/docs/scripts/gather.mjs index 2150a27..1c902dc 100644 --- a/docs/scripts/gather.mjs +++ b/docs/scripts/gather.mjs @@ -1,5 +1,4 @@ // Generates the API reference before every build, from the committed operation // documents in docs/openapi. The guide, and the overview and portal pages of -// the API, are written by hand. Then puts the page of berth serve under /web. +// the API, are written by hand. await import('./reference.mjs'); -await import('./web.mjs'); diff --git a/docs/scripts/reference.mjs b/docs/scripts/reference.mjs index f1ab55e..7190cc6 100644 --- a/docs/scripts/reference.mjs +++ b/docs/scripts/reference.mjs @@ -1,5 +1,5 @@ // The API reference, by operation, from the committed operation documents in -// docs/openapi (`berthd operations`, `berth operations`). One folder per +// docs/openapi (`berthd operations`, `berth server api operations`). One folder per // program under content/docs/api, one page per group of operations. The // overview and the portal page beside them are written by hand. // Plain Node, no packages. @@ -29,7 +29,7 @@ const programs = [ const groups = { general: ['General', 'The version, and reading the configuration again.'], sessions: ['Sessions', 'Commands and terminals on the host, their output and input.'], - tasks: ['Tasks', "The host's tasks and their arguments."], + applied: ['Applied', 'The profile the host keeps of itself, as the last apply wrote it.'], system: ['System', 'Load, memory and disks, and power.'], desktop: ['Desktop', 'The shared desktop of the agent.'], files: ['Files', 'Files under the configured roots, as the agent.'], @@ -38,17 +38,29 @@ const groups = { processes: ['Processes', 'The processes of the host.'], channel: ['Channel', 'Messages between programs on the host and berth.'], services: ['Services', 'Programs of plugins that answer calls.'], + ports: ['Ports', "TCP connections to ports on the host's loopback."], + endpoints: ['Endpoints', "TCP connections to addresses on the host's own networks, which only berth serve opens, for a resource's endpoint."], setup_key: ['Setup key', 'Collecting the token with a short-lived key, after an install.'], hosts: ['Hosts', 'The hosts berth serve knows, their profiles, and what can be done to each.'], - actions: ['Actions', 'What can be done to the installation as a whole, such as a roll-out.'], + installation: ['Installation', 'What can be done to the installation as a whole, such as a roll-out.'], + keys: ['Keys', 'The SSH keys berth logs in to hosts with.'], + secrets: ['Secrets', 'Secret values the server keeps and gives to hosts.'], + registries: ['Registries', 'Package registries, their logins and their repositories.'], + plugins: ['Plugins', 'The plugins the server has, and updating them.'], + settings: ['Settings', "The server's settings."], + setup: ['Set-up', 'What the set-up of a new host starts from: the starting profile and the features.'], provisioners: ['Provisioners', 'What the plugins can create machines and deployments with, their pages and their actions.'], - resources: ['Resources', 'What provisioners made for berth, a host\'s machine among them: creating, changing, removing, their actions and logs.'], + resources: ['Resources', 'What provisioners made for berth, a host\'s machine among them: creating, changing, removing, their actions, logs and endpoints.'], jobs: ['Jobs', 'Long-running work of berth serve, such as an apply: one run of an action.'], inbox: ['Inbox', "Messages taken in from the hosts' channels."], kv: ['Key-value store', 'Small JSON values with revisions, for user interfaces.'], - events: ['Events', 'One stream for channel messages, jobs and key-value changes.'], + events: ['Events', 'One stream for channel messages, jobs, key-value changes and changes to hosts and resources.'], }; -const groupOrder = Object.keys(groups); +// A group of the same name means something else in each program. +const hostGroups = { + actions: ['Actions', "The host's actions and their arguments."], +}; +const groupOrder = [...Object.keys(hostGroups), ...Object.keys(groups)]; // --------------------------------------------------------------------------- // Text @@ -400,7 +412,7 @@ function typesSection(names, schemas) { } function groupPage(group, ops, program, schemas) { - const [title, description] = groups[group] ?? [group, '']; + const [title, description] = (program.serve ? undefined : hostGroups[group]) ?? groups[group] ?? [group, '']; const used = new Set(); let body = ''; for (const op of ops) body += operationSection(op, program, schemas, used); diff --git a/docs/scripts/screens.py b/docs/scripts/screens.py index 3802777..315930e 100644 --- a/docs/scripts/screens.py +++ b/docs/scripts/screens.py @@ -2,7 +2,7 @@ """Terminal screenshots for the documentation, in docs/public/screens/. Runs the built berth binary (target/debug/berth) in a pseudo-terminal against an -invented set-up and writes what the terminal shows as PNG, in the UI's colours. +invented set-up, its screens with the server inside berth, and writes what the terminal shows as PNG, in the UI's colours. Each picture is drawn as SVG first, then rendered by headless Chrome at twice the size with JetBrains Mono (docs/scripts/fonts, SIL OFL) embedded, so the text looks the same wherever the PNG is shown. Only the PNGs are kept. @@ -182,20 +182,19 @@ def make_demo(root: Path) -> dict: access={"address": "192.0.2.31", "user": "ops"}, ) for host in (studio, cloud, laptop): - run(env, "profile", host["hostname"], "--set", "-", stdin=json.dumps(host, indent=2) + "\n") - run(env, "config", "default_host", "studio") + run(env, "host", "import", host["hostname"], "-", stdin=json.dumps(host, indent=2) + "\n") + run(env, "config", "set", "default_host", "studio") # Registries and a secret, through the commands; nothing is contacted. - run(env, "registry", "add", "nexus", "nexus", "https://nexus.example.com", "ci-user") - run(env, "registry", "repo", "nexus", "npm", "npm-group") - run(env, "registry", "repo", "nexus", "raw", "berth-releases") - run(env, "registry", "repo", "nexus", "docker", "docker-hosted", "nexus.example.com:8083") - run(env, "registry", "add", "ghcr", "docker", "ghcr.io", "demo-user") + run(env, "registry", "add", "nexus", "nexus", "https://nexus.example.com", "--user", "ci-user") + run(env, "registry", "add-repo", "nexus", "npm", "npm-group") + run(env, "registry", "add-repo", "nexus", "raw", "berth-releases") + run(env, "registry", "add-repo", "nexus", "docker", "docker-hosted", "--address", "nexus.example.com:8083") + run(env, "registry", "add", "ghcr", "docker", "ghcr.io", "--user", "demo-user") run(env, "secret", "set", "github", stdin="github_pat_demo_not_a_real_token\n") run(env, "secret", "set", "backup-password", stdin="correct-horse-demo\n") for host in ("studio", "cloud-1"): - run(env, "host", host, "secret", "github", "env", "GH_TOKEN") - run(env, "host", host, "registry", "nexus", "on") - run(env, "host", "studio", "secret", "backup-password", "file", "~/.config/berth-backup/password", "0600") + run(env, "host", "set", host, "secret.github=GH_TOKEN", "registry.nexus=on") + run(env, "host", "set", "studio", "secret.backup-password=~/.config/berth-backup/password:0600") return env @@ -503,7 +502,7 @@ def save(name: str, screen: pyte.Screen, title: str, trim=True): def tui(env, name, args, keys, title, cols=120, rows=30, settle=1.5): - term = Terminal([str(BERTH), *args], env, cols, rows, env["HOME"]) + term = Terminal([str(BERTH), "ui", *args], env, cols, rows, env["HOME"]) try: term.pump(settle) for step in keys: @@ -537,7 +536,12 @@ def shlex_quote(text): # Downs from the top of a host's Settings tab to the fourth row of its registries # and secrets: past the six rows of the operator with its one key, the five of # the agent, and SSH hardening. -SECRETS_DOWN = 15 +SECRETS_DOWN = 13 + + +# The keys from a new host's overview to its set-up tabs: past Try again to +# Change what is installed. +SETUP = [2.0, "", "", 1.0] def pictures(env): @@ -545,22 +549,23 @@ def pictures(env): picture shows what an earlier one left behind, such as sessions.""" return { "home": (False, lambda: tui(env, "home", [], [], "berth", rows=20)), - "home-plugins": (False, lambda: tui(env, "home-plugins", [], ["5"], "berth: plugins")), - "features": (False, lambda: tui(env, "features", ["cloud-1"], ["2"], "berth cloud-1: features", rows=34)), - "accounts": (False, lambda: tui(env, "accounts", ["cloud-1"], ["3"], "berth cloud-1: settings", rows=26)), - "secrets": (False, lambda: tui(env, "secrets", ["cloud-1"], ["3"] + [""] * SECRETS_DOWN, + "home-plugins": (False, lambda: tui(env, "home-plugins", [], ["6"], "berth: plugins")), + "features": (False, lambda: tui(env, "features", ["cloud-1"], SETUP + ["2"], "berth cloud-1: features", rows=34)), + "accounts": (False, lambda: tui(env, "accounts", ["cloud-1"], SETUP + ["3"], "berth cloud-1: settings", rows=26)), + "secrets": (False, lambda: tui(env, "secrets", ["cloud-1"], SETUP + ["3"] + [""] * SECRETS_DOWN, "berth cloud-1: settings", rows=22)), - "install-cloud": (False, lambda: tui(env, "install-cloud", ["cloud-1"], ["", "", "", "4"], + "install-cloud": (False, lambda: tui(env, "install-cloud", ["cloud-1"], + SETUP + ["", "", "", "4"], "berth cloud-1: run", rows=34)), - "list": (False, lambda: command(env, "list", ["list"])), - "overview": (True, lambda: tui(env, "overview", ["studio"], [2.5, ""], "berth studio", rows=46)), + "list": (False, lambda: command(env, "list", ["ls"])), + "overview": (True, lambda: tui(env, "overview", ["studio"], [2.5], "berth studio", rows=46)), "terminal": (True, lambda: tui(env, "terminal", ["studio"], - [2.5, "", 1.5, "id\r", "ls --color=never /srv/agent\r", 3.0], + [2.5, "", "", 1.5, "id\r", "ls --color=never /srv/agent\r", 3.0], "berth studio: terminal", rows=20)), - "tasks": (True, lambda: command(env, "tasks", ["tasks", "studio"])), + "actions": (True, lambda: command(env, "actions", ["action", "list", "studio"])), "exec": (True, lambda: command(env, "exec", ["exec", "studio", "--", "id"])), - "sessions": (True, lambda: command(env, "sessions", ["sessions", "studio"])), - "host": (False, lambda: command(env, "host", ["host", "studio"])), + "sessions": (True, lambda: command(env, "sessions", ["session", "list", "studio"])), + "host": (False, lambda: command(env, "host", ["host", "show", "studio"])), } diff --git a/docs/scripts/web.mjs b/docs/scripts/web.mjs deleted file mode 100644 index 39e755c..0000000 --- a/docs/scripts/web.mjs +++ /dev/null @@ -1,44 +0,0 @@ -// Puts the page of `berth serve` into public/web, so the site carries it at -// /web as plain files. The page is built with berth (cargo build -p berth -// --features web leaves it in target/web-dist); without it the site still builds, and /web -// says how to get it. -import { execFileSync } from 'node:child_process'; -import { createHash } from 'node:crypto'; -import { cpSync, existsSync, mkdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; -import { dirname, join } from 'node:path'; -import { fileURLToPath } from 'node:url'; - -const root = join(dirname(fileURLToPath(import.meta.url)), '..', '..'); -const dist = join(root, 'target', 'web-dist'); -const out = join(root, 'docs', 'public', 'web'); -const files = ['berth-web.js', 'berth-web_bg.wasm']; -const built = () => files.every((file) => existsSync(join(dist, file))); - -if (!built() && process.env.BERTH_WEB_BUILD !== '0') { - try { - // The page of a release, though berth itself is a development build. - execFileSync('cargo', ['build', '--locked', '-p', 'berth', '--features', 'web'], { - cwd: root, - stdio: 'inherit', - env: { ...process.env, BERTH_WEB_PROFILE: 'web' }, - }); - } catch { - // No Rust here, or no wasm target: the site is built without the page. - } -} - -rmSync(out, { recursive: true, force: true }); -mkdirSync(out, { recursive: true }); -if (built()) { - for (const file of files) cpSync(join(dist, file), join(out, file)); - const build = createHash('sha256').update(readFileSync(join(dist, files[1]))).digest('hex').slice(0, 16); - const page = readFileSync(join(root, 'crates', 'berth-web', 'index.html'), 'utf8'); - // No server of its own: the page asks for one, or takes ?server= from its address. - writeFileSync(join(out, 'index.html'), page.replaceAll('@BUILD@', build).replaceAll('@SERVER@', '')); -} else { - console.warn('docs: the page for /web is not built (cargo build -p berth --features web); a note is put in its place'); - writeFileSync( - join(out, 'index.html'), - 'berthThis copy of the site was built without the page. berth serve --listen ADDRESS --web shows it.\n', - ); -} diff --git a/docs/src/lib/layout.shared.tsx b/docs/src/lib/layout.shared.tsx index 4114494..a2d6edb 100644 --- a/docs/src/lib/layout.shared.tsx +++ b/docs/src/lib/layout.shared.tsx @@ -15,8 +15,6 @@ export function baseOptions(): BaseLayoutProps { ), }, - // The page of berth serve, as plain files under public/web: not a route of the site. - links: [{ text: 'Web', url: asset('/web/'), external: true }], githubUrl: github, // Dark only. themeSwitch: { enabled: false }, diff --git a/install-remote.sh b/install-remote.sh deleted file mode 100755 index 7defe7d..0000000 --- a/install-remote.sh +++ /dev/null @@ -1,59 +0,0 @@ -#!/bin/sh -# Installs berth-remote for this user on macOS or Linux: the released program -# for this machine, checked against its SHA-256, into ~/.local/bin (or -# BERTH_INSTALL_DIR). berth-remote uses the hosts of a berth serve elsewhere. -# -# curl -fsSL https://raw.githubusercontent.com/helightdev/berth/main/install-remote.sh | sh -# -# BERTH_VERSION picks a release tag such as v0.5.0; the default is the latest. -set -eu - -repo=helightdev/berth -dir=${BERTH_INSTALL_DIR:-$HOME/.local/bin} -version=${BERTH_VERSION:-latest} - -fail() { printf 'install: %s\n' "$1" >&2; exit 1; } - -case "$(uname -s)" in - Linux) system=linux ;; - Darwin) system=macos ;; - *) fail "no released program for $(uname -s): on Windows use install.ps1" ;; -esac -case "$(uname -m)" in - x86_64|amd64) arch=x86_64 ;; - arm64|aarch64) arch=aarch64 ;; - *) fail "no released program for $(uname -m): build it with cargo install --locked --git https://github.com/$repo berth-remote" ;; -esac -asset=berth-remote-$arch-$system.tar.gz -for tool in curl tar; do - command -v "$tool" >/dev/null || fail "$tool is needed" -done -# macOS has shasum rather than sha256sum. -if command -v sha256sum >/dev/null; then - check() { sha256sum -c "$1"; } -elif command -v shasum >/dev/null; then - check() { shasum -a 256 -c "$1"; } -else - fail "sha256sum or shasum is needed" -fi - -if [ "$version" = latest ]; then - url=https://github.com/$repo/releases/latest/download -else - url=https://github.com/$repo/releases/download/$version -fi - -work=$(mktemp -d) -trap 'rm -rf "$work"' EXIT -curl -fsSL -o "$work/$asset" "$url/$asset" || fail "could not download $url/$asset" -curl -fsSL -o "$work/$asset.sha256" "$url/$asset.sha256" || fail "could not download the checksum" -(cd "$work" && check "$asset.sha256" >/dev/null) || fail "the download does not match its checksum" -tar -xzf "$work/$asset" -C "$work" berth-remote -mkdir -p "$dir" -install -m 755 "$work/berth-remote" "$dir/berth-remote" - -printf 'Installed %s to %s\n' "$("$dir/berth-remote" --version 2>/dev/null || echo berth-remote)" "$dir/berth-remote" -case ":$PATH:" in - *":$dir:"*) ;; - *) printf 'Add %s to your PATH to run it as berth-remote.\n' "$dir" ;; -esac diff --git a/install.ps1 b/install.ps1 index d3dc03f..f69fa7e 100644 --- a/install.ps1 +++ b/install.ps1 @@ -1,6 +1,6 @@ -# Installs berth-remote for this user on Windows: the released program, -# checked against its SHA-256, into %LOCALAPPDATA%\Programs\berth-remote (or -# BERTH_INSTALL_DIR). berth-remote uses the hosts of a berth serve elsewhere. +# Installs berth for this user on Windows: the released program, checked +# against its SHA-256, into %LOCALAPPDATA%\Programs\berth (or +# BERTH_INSTALL_DIR). On Windows berth uses the hosts of a berth serve elsewhere. # # irm https://raw.githubusercontent.com/helightdev/berth/main/install.ps1 | iex # @@ -9,8 +9,8 @@ $ErrorActionPreference = 'Stop' $ProgressPreference = 'SilentlyContinue' $repo = 'helightdev/berth' -$asset = 'berth-remote-x86_64-windows.zip' -$dir = if ($env:BERTH_INSTALL_DIR) { $env:BERTH_INSTALL_DIR } else { Join-Path $env:LOCALAPPDATA 'Programs\berth-remote' } +$asset = 'berth-x86_64-windows.zip' +$dir = if ($env:BERTH_INSTALL_DIR) { $env:BERTH_INSTALL_DIR } else { Join-Path $env:LOCALAPPDATA 'Programs\berth' } $version = if ($env:BERTH_VERSION) { $env:BERTH_VERSION } else { 'latest' } function Fail([string] $message) { @@ -20,7 +20,7 @@ function Fail([string] $message) { } if (-not [Environment]::Is64BitOperatingSystem -or $env:PROCESSOR_ARCHITECTURE -eq 'ARM64') { - Fail "no released program for this processor: build it with cargo install --locked --git https://github.com/$repo berth-remote" + Fail "no released program for this processor: build it with cargo install --locked --git https://github.com/$repo berth" } $url = if ($version -eq 'latest') { @@ -29,7 +29,7 @@ $url = if ($version -eq 'latest') { "https://github.com/$repo/releases/download/$version" } -$work = Join-Path ([IO.Path]::GetTempPath()) ("berth-remote-" + [Guid]::NewGuid()) +$work = Join-Path ([IO.Path]::GetTempPath()) ("berth-" + [Guid]::NewGuid()) New-Item -ItemType Directory -Path $work | Out-Null try { $zip = Join-Path $work $asset @@ -39,16 +39,17 @@ try { if ((Get-FileHash -Algorithm SHA256 $zip).Hash -ne $want) { Fail 'the download does not match its checksum' } Expand-Archive -Path $zip -DestinationPath (Join-Path $work 'unpacked') New-Item -ItemType Directory -Force -Path $dir | Out-Null - $program = Join-Path $dir 'berth-remote.exe' - Copy-Item -Force (Join-Path $work 'unpacked\berth-remote.exe') $program + $program = Join-Path $dir 'berth.exe' + Copy-Item -Force (Join-Path $work 'unpacked\berth.exe') $program } finally { Remove-Item -Recurse -Force $work -ErrorAction SilentlyContinue } -$said = try { & $program --version } catch { 'berth-remote' } +$said = try { & $program --version } catch { 'berth' } Write-Host "Installed $said to $program" $path = [Environment]::GetEnvironmentVariable('Path', 'User') if (-not (($path -split ';') -contains $dir)) { - Write-Host "Add $dir to your Path to run it as berth-remote:" + Write-Host "Add $dir to your Path to run it as berth:" Write-Host " [Environment]::SetEnvironmentVariable('Path', [Environment]::GetEnvironmentVariable('Path', 'User') + ';$dir', 'User')" } +Write-Host "Next: berth login URL, with the setup key of your berth server." diff --git a/install.sh b/install.sh index 73c7849..d2db5e1 100755 --- a/install.sh +++ b/install.sh @@ -1,6 +1,8 @@ #!/bin/sh -# Installs berth for this user: the released program for this machine, checked -# against its SHA-256, into ~/.local/bin (or BERTH_INSTALL_DIR). +# Installs berth for this user on Linux or macOS: the released program for +# this machine, checked against its SHA-256, into ~/.local/bin (or +# BERTH_INSTALL_DIR). On macOS berth is a client of a berth serve elsewhere; +# on Windows use install.ps1. # # curl -fsSL https://raw.githubusercontent.com/helightdev/berth/main/install.sh | sh # @@ -10,24 +12,31 @@ set -eu repo=helightdev/berth dir=${BERTH_INSTALL_DIR:-$HOME/.local/bin} version=${BERTH_VERSION:-latest} -# BERTH_VARIANT=web takes the build that carries the page for a browser. -variant=${BERTH_VARIANT:-} fail() { printf 'install: %s\n' "$1" >&2; exit 1; } -[ "$(uname -s)" = Linux ] || fail "berth runs on Linux; on another system use the container image ghcr.io/$repo, or install-remote.sh for berth-remote, which uses a berth server elsewhere" +case "$(uname -s)" in + Linux) system=linux ;; + Darwin) system=macos ;; + *) fail "no released program for $(uname -s): on Windows use install.ps1" ;; +esac case "$(uname -m)" in x86_64|amd64) arch=x86_64 ;; + arm64|aarch64) arch=aarch64 ;; *) fail "no released program for $(uname -m): build it with cargo install --locked --git https://github.com/$repo berth" ;; esac -case "$variant" in - "") asset=berth-$arch-linux.tar.gz ;; - web) asset=berth-web-$arch-linux.tar.gz ;; - *) fail "no variant $variant: leave BERTH_VARIANT unset, or set it to web" ;; -esac -for tool in curl tar sha256sum; do +asset=berth-$arch-$system.tar.gz +for tool in curl tar; do command -v "$tool" >/dev/null || fail "$tool is needed" done +# macOS has shasum rather than sha256sum. +if command -v sha256sum >/dev/null; then + check() { sha256sum -c "$1"; } +elif command -v shasum >/dev/null; then + check() { shasum -a 256 -c "$1"; } +else + fail "sha256sum or shasum is needed" +fi if [ "$version" = latest ]; then url=https://github.com/$repo/releases/latest/download @@ -39,7 +48,7 @@ work=$(mktemp -d) trap 'rm -rf "$work"' EXIT curl -fsSL -o "$work/$asset" "$url/$asset" || fail "could not download $url/$asset" curl -fsSL -o "$work/$asset.sha256" "$url/$asset.sha256" || fail "could not download the checksum" -(cd "$work" && sha256sum -c "$asset.sha256" >/dev/null) || fail "the download does not match its checksum" +(cd "$work" && check "$asset.sha256" >/dev/null) || fail "the download does not match its checksum" tar -xzf "$work/$asset" -C "$work" berth mkdir -p "$dir" install -m 755 "$work/berth" "$dir/berth" @@ -49,6 +58,12 @@ case ":$PATH:" in *":$dir:"*) ;; *) printf 'Add %s to your PATH to run it as berth.\n' "$dir" ;; esac +# The hosts are set up from Linux; elsewhere berth uses a server. +if [ "$system" != linux ]; then + printf 'Next: berth login URL, with the setup key of your berth server.\n' + exit 0 +fi +printf 'Next: berth host create NAME, or berth ui for the screens. berth login URL uses a berth server instead.\n' missing= for tool in ssh python3 ansible-playbook; do command -v "$tool" >/dev/null || missing="$missing $tool" diff --git a/plugins/docker/bin/provision b/plugins/docker/bin/provision index 440db2b..1646a59 100755 --- a/plugins/docker/bin/provision +++ b/plugins/docker/bin/provision @@ -14,6 +14,7 @@ import sys import tempfile import threading import time +import urllib.parse from pathlib import Path PACKAGE = Path(__file__).resolve().parent.parent @@ -26,6 +27,9 @@ RUNTIMES = ('privileged', 'sysbox') ROWS = 60 # How many of the latest lines `logs` gives when the request does not say. LOG_LINES = 100 +# What usually speaks on a port of a container, as a hint for people. +HINTS = {22: 'ssh', 80: 'http', 443: 'https', 3000: 'http', 3306: 'mysql', 5432: 'postgres', 6379: 'redis', + 8000: 'http', 8080: 'http', 8443: 'https', 9000: 'http', 27017: 'mongodb'} # Runs inside the container with the public keys on stdin. It does not depend on # the image, so a custom one needs only sh, useradd, sudo and sshd. @@ -97,10 +101,12 @@ def stream(argv, env, what, cwd=None, mask=lambda text: text): class Docker: - def __init__(self, endpoint): + def __init__(self, endpoint, node=''): self.env = dict(os.environ) if endpoint: self.env['DOCKER_HOST'] = endpoint + # The node the engine runs on, whose controller dials the endpoints, when berth names one. + self.node = node def run(self, *args, input=None, check=True, merge=False, env=None): try: @@ -156,6 +162,20 @@ class Docker: def where(self): return self.env.get('DOCKER_HOST') or 'the local Docker' + def engine_host(self): + """The machine the engine runs on, as berth reaches it, or None when it is this one.""" + if not hasattr(self, '_engine_host'): + endpoint = self.env.get('DOCKER_HOST') or '' + if not endpoint: + # A context of the docker CLI can name an engine elsewhere too. + done = self.run('context', 'inspect', '--format', '{{.Endpoints.docker.Host}}', check=False) + endpoint = done.stdout.strip() if done.returncode == 0 else '' + if not endpoint or endpoint.startswith(('unix://', 'npipe://')): + self._engine_host = None + else: + self._engine_host = urllib.parse.urlsplit(endpoint).hostname or '' + return self._engine_host + def valid_name(host): if not re.fullmatch(r'[A-Za-z0-9][A-Za-z0-9_.-]*', host or ''): @@ -185,7 +205,7 @@ def settings(request): arguments = request.get('arguments') or {} host = request.get('host') or '' return { - 'docker': Docker((arguments.get('host') or '').strip()), + 'docker': Docker((arguments.get('host') or '').strip(), request.get('node') or ''), 'image': (arguments.get('image') or '').strip(), 'port': (arguments.get('ssh_port') or '').strip(), 'address': (arguments.get('publish_address') or '').strip() or '127.0.0.1', @@ -221,6 +241,82 @@ def where(cfg, info): return container_address(info), 22 +def slug(text): + return re.sub(r'[^a-z0-9_.-]+', '-', (text or '').lower()).strip('-._') + + +def endpoints_of(docker, info, prefix=''): + """Where a container is reached: each published port, at the engine's machine, and on an engine + of the machine that dials also each exposed port at each of its network addresses, which only + that machine routes to. That machine is this one, or the node berth names, whose controller + dials.""" + host = docker.engine_host() + local = host is None or bool(docker.node) + found = [] + + def add(name, address, port, protocol, target): + name = slug(prefix + name)[:63].strip('-._') + if not name or not address or any(known['name'] == name for known in found): + return + entry = {'name': name, 'address': address, 'port': int(port), 'protocol': protocol} + if protocol == 'tcp' and int(target) in HINTS: + entry['hint'] = HINTS[int(target)] + if local: + entry['local'] = True + found.append(entry) + + settings_of = (info or {}).get('NetworkSettings') or {} + for key, bindings in sorted((settings_of.get('Ports') or {}).items()): + target, _, protocol = key.partition('/') + for binding in bindings or []: + published = binding.get('HostPort') or '' + if not published.isdigit() or not target.isdigit(): + continue + address = binding.get('HostIp') or '' + # Published on every address: the dialling machine's loopback, or the engine's machine by name. + if address in ('', '0.0.0.0', '::'): + address = '127.0.0.1' if local else host + add(published + ('' if protocol == 'tcp' else f'-{protocol}'), address, published, protocol, target) + if local: + exposed = sorted(((info or {}).get('Config') or {}).get('ExposedPorts') or {}) + for network, attached in sorted((settings_of.get('Networks') or {}).items()): + address = (attached or {}).get('IPAddress') or '' + for key in exposed: + target, _, protocol = key.partition('/') + if target.isdigit(): + add(f'{network}-{target}' + ('' if protocol == 'tcp' else f'-{protocol}'), address, target, + protocol, target) + return found + + +def run_in(docker, name, request): + """Run the request's command in a running container, its output on standard error, which berth + shows as the job's: the command's exit status.""" + command = request.get('command') + if not isinstance(command, list) or not command or not all(isinstance(word, str) for word in command): + raise Failure('the request carries no command as a list of words') + info = docker.inspect(name) + if info is None: + raise Failure(f'{name} does not exist') + if not info.get('State', {}).get('Running'): + raise Failure(f'{name} is not running: start it first') + stdin = request.get('stdin') + stdin = stdin if isinstance(stdin, str) and stdin else None + argv = ['docker', 'exec', *(['-i'] if stdin is not None else []), name, *command] + try: + process = subprocess.Popen(argv, env=docker.env, + stdin=subprocess.PIPE if stdin is not None else subprocess.DEVNULL, + stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + except FileNotFoundError: + raise Failure('docker is not installed or not on the PATH') + if stdin is not None: + threading.Thread(target=feed, args=(process.stdin, stdin.encode()), daemon=True).start() + for chunk in iter(lambda: process.stdout.read1(65536), b''): + sys.stderr.buffer.write(chunk) + sys.stderr.flush() + return {'exit_code': process.wait()} + + def package_version(): return json.loads((PACKAGE / 'package.json').read_text())['version'] @@ -396,12 +492,12 @@ def status(request): if info is None: return {'state': 'gone'} if info.get('State', {}).get('Running'): - answer = {'state': 'running'} + answer = {'state': 'running', 'endpoints': endpoints_of(cfg['docker'], info)} address, _ = where(cfg, info) if address: answer['address'] = address return answer - return {'state': 'stopped'} + return {'state': 'stopped', 'endpoints': []} def ours(docker, name): @@ -491,8 +587,9 @@ def change(request): if args: say(f'Changing the limits of {name}.') docker.run('update', *args, name) - running = (docker.inspect(name) or {}).get('State', {}).get('Running') - return {'state': 'running' if running else 'stopped'} + info = docker.inspect(name) or {} + running = info.get('State', {}).get('Running') + return {'state': 'running' if running else 'stopped', 'endpoints': endpoints_of(docker, info)} def logs(request): @@ -701,7 +798,7 @@ def instance_create(request): if not (docker.inspect(name) or {}).get('State', {}).get('Running'): say(f'Starting {name}.') docker.run('start', name) - return {'id': name, 'state': 'running'} + return {'id': name, 'state': 'running', 'endpoints': endpoints_of(docker, docker.inspect(name))} name = instance_name(request) if docker.inspect(name) is not None: raise Failure(f'{docker.where()} already has a container named {name}. Choose another name.') @@ -740,14 +837,16 @@ def instance_create(request): for made in docker.names(f'label=berth-token={token}'): docker.run('rm', '--force', made, check=False) raise failure('docker run', done.stderr or done.stdout) - return {'id': name, 'state': 'running'} + return {'id': name, 'state': 'running', 'endpoints': endpoints_of(docker, docker.inspect(name))} def instance_status(request): - info = settings(request)['docker'].inspect(identify(request)) + docker = settings(request)['docker'] + info = docker.inspect(identify(request)) if info is None: return {'state': 'gone'} - return {'state': 'running' if info.get('State', {}).get('Running') else 'stopped'} + return {'state': 'running' if info.get('State', {}).get('Running') else 'stopped', + 'endpoints': endpoints_of(docker, info)} def instance_start(request): @@ -756,7 +855,7 @@ def instance_start(request): if instance_ours(docker, name) is None: raise Failure(f'{name} does not exist') docker.run('start', name) - return {} + return {'endpoints': endpoints_of(docker, docker.inspect(name))} def instance_stop(request): @@ -765,7 +864,7 @@ def instance_stop(request): if instance_ours(docker, name) is None: raise Failure(f'{name} does not exist') docker.run('stop', '--time', '30', name) - return {} + return {'endpoints': []} def instance_destroy(request): @@ -794,8 +893,9 @@ def instance_change(request): if args: say(f'Changing the limits of {name}.') docker.run('update', *args, name) - running = (docker.inspect(name) or {}).get('State', {}).get('Running') - return {'state': 'running' if running else 'stopped'} + info = docker.inspect(name) or {} + running = info.get('State', {}).get('Running') + return {'state': 'running' if running else 'stopped', 'endpoints': endpoints_of(docker, info)} def instance_logs(request): @@ -814,6 +914,14 @@ def instance_logs(request): return None +def instance_exec(request): + docker = settings(request)['docker'] + name = identify(request) + if instance_ours(docker, name) is None: + raise Failure(f'{name} does not exist') + return run_in(docker, name, request) + + def instances(docker): resources = [] for name in docker.names('label=berth-resource'): @@ -828,7 +936,7 @@ def instances(docker): INSTANCE_VERBS = {'create': instance_create, 'status': instance_status, 'start': instance_start, 'stop': instance_stop, 'destroy': instance_destroy, 'change': instance_change, - 'logs': instance_logs} + 'logs': instance_logs, 'exec': instance_exec} # Deployments: one Compose project each, named after the deployment. @@ -977,7 +1085,25 @@ class Compose: if line.strip()] say(f'Bringing up {self.project} at {self.where()}.') self.run('up', '-d', '--remove-orphans', spec=spec, labels=labels, services=services, progress=True) - return project_state(self.containers()) + return self.state(self.containers()) + + def state(self, rows): + """The project's state, with where each of its containers is reached by its service's name.""" + answer = project_state(rows) + if answer['state'] != 'gone': + endpoints = [] + for row in rows: + if not running(row) or not row.get('Name'): + continue + service = slug(row.get('Service') or row['Name']) + # Where one container is reached is not worth failing its project's state over. + try: + info = self.docker.inspect(row['Name']) + except (Failure, ValueError, IndexError): + continue + endpoints += endpoints_of(self.docker, info, f'{service}-') + answer['endpoints'] = endpoints + return answer def running(row): @@ -1030,7 +1156,8 @@ def deployment_change(request): def deployment_status(request): - return project_state(Compose(request, deployment_id(request)).containers()) + compose = Compose(request, deployment_id(request)) + return compose.state(compose.containers()) def deployment_destroy(request): @@ -1132,8 +1259,13 @@ def act(request): return ACTIONS[name](settings(request), arguments) +def exec_elsewhere(request): + raise Failure(f'a {request.get("kind") or "machine"} runs no commands here; an instance does') + + VERBS = {'create': create, 'status': status, 'start': start, 'stop': stop, - 'destroy': destroy, 'list': list_all, 'change': change, 'logs': logs, 'act': act} + 'destroy': destroy, 'list': list_all, 'change': change, 'logs': logs, 'act': act, + 'exec': exec_elsewhere} def handler(verb, request): diff --git a/plugins/docker/guide.md b/plugins/docker/guide.md index ced82f0..9521346 100644 --- a/plugins/docker/guide.md +++ b/plugins/docker/guide.md @@ -9,6 +9,38 @@ engine you already have. It also runs Compose projects on the same engine, as [deployments](#deployments), and plain containers from any image, as [instances](#instances). +## A Docker engine on a node + +The feature `docker/engine` installs the Docker engine on a [node](https://github.com/helightdev/berth/blob/main/docs/content/docs/%28guide%29/use/nodes.mdx), +a server that berth applies without a desktop. berth then registers the +provisioner `NODE-docker-container` for it, with `host=ssh://USER@ADDRESS:PORT` +for the account and address berth reaches the node at. List it among the +node's features: + + "ansible_vars": {"devbox_plugin_features": ["docker/engine"]} + +| Part | Set-up | +| --- | --- | +| Docker | `docker.io` with Compose and Buildx, from Ubuntu's packages | +| An engine installed already | Kept as it is, also one from Docker's own packages, with its containers and volumes | +| The SSH user | Joins the group `docker` unless it is root | +| Sysbox | With `devbox_docker_sysbox` set to `true`, installed beside Docker. See [Sysbox](#sysbox) | + +| Setting | Default | Meaning | +| --- | --- | --- | +| `devbox_docker_sysbox` | `false` | Install Sysbox | +| `devbox_docker_sysbox_version` | `0.7.1` | The Sysbox release | +| `devbox_docker_sysbox_checksum_amd64`, `devbox_docker_sysbox_checksum_arm64` | those of `0.7.1` | The SHA-256 of the package. The download is always checked, and another version needs its own sums | + +Docker opens the ports its containers publish by itself, past the node's +firewall. Publish a port on `127.0.0.1` or on the tailnet address to keep it +private. + +A berth server in a container on the node, with the engine's socket, calls +this engine its local one. When that server applies the node, its +provisioner of the local engine becomes the node's, and no second one is +registered. + ## What it makes - A container named `berth-HOST`, with the labels `berth-host` and @@ -35,7 +67,7 @@ each engine, under a name of your own: `host` is the provisioner's own argument. berth keeps it as a setting, which every machine and action uses unless it is given another: - berth provisioner build-box run settings host=ssh://ops@192.0.2.20 + berth provisioner set build-box host=ssh://ops@192.0.2.20 | Name | Default | Meaning | | --- | --- | --- | @@ -78,7 +110,8 @@ the socket's group: --group-add "$(stat --format %g /var/run/docker.sock)" ghcr.io/helightdev/berth Leave `host` empty then. The socket gives root on that machine. The -`incus-host` recipe sets its berth server up this way. +`berth-server` recipe sets its berth server up this way with +`berth_server_node`. ## Machines @@ -97,10 +130,10 @@ matches. An instance is one plain container from any image, such as `nginx:1.27` or `debian:12`. It becomes no berth host, and berth puts nothing of its own in it. -Make one with **+ New instance** under the hosts, or the action New instance -on the provisioner's page. Both open a form laid out like a host's set-up, with every field on a row. Or: +Make one with **+ New resource from an image** under the hosts, or the action +New container from an image on the provisioner's page. Both open a form laid out like a host's set-up, with every field on a row. Or: - berth provisioner docker run new-instance name=web image=nginx:1.27 ports=8080:80 + berth provisioner run docker new-instance name=web image=nginx:1.27 ports=8080:80 | Argument | Default | Meaning | | --- | --- | --- | @@ -120,6 +153,32 @@ to another argument is refused, so make a new instance for it. The values of `environment` reach Docker in its environment, not on its command line. Removing an instance removes its anonymous volumes and keeps named ones. +### Endpoints and commands + +The plugin reports where a container is reached as its endpoints, each time +berth makes, changes, starts or checks it. Each published port is one, named +after the port it is published on, such as `8080`. On the local engine, each +port the image exposes is one too, at each of the container's network +addresses, such as `bridge-80`. A deployment reports those of each running +container, named after its service, such as `web-8080`. + +| Engine | Address of a published port | Network addresses | +| --- | --- | --- | +| On this machine | `127.0.0.1`, or the address it is published on | Reported, as this machine routes to them | +| On a node, through the node's provisioner | `127.0.0.1` of the node, or the address it is published on | Reported, as the node's controller dials them | +| Elsewhere, by `host` or a context | The engine's host name from its address, or the address it is published on | Left out, as only the engine's machine routes to them | + +berth opens a tunnel to an endpoint with `resources.connect`. A port published +on the loopback of an engine elsewhere is reported too, and the server refuses +to dial it. For the provisioner a node registers, berth names the node in each +request, and the node's controller dials the endpoints, so its loopback and +its bridges are reached. + +An instance also takes the action `exec`. It runs `docker exec` with the +command, and `-i` with its standard input when it has some. The container has +to be running. The command's output is the job's, and the job ends with the +command's exit status. + ## Deployments A deployment is a Compose project. Its spec is a Compose file, and its name is @@ -148,7 +207,7 @@ have them. | Destroy | `docker compose down`. It removes the project's named volumes too, unless `keep_volumes` is `yes`. | | List | The projects whose containers have berth's labels. | -Make one with `berth deploy NAME --provisioner NAME --spec FILE`, or with the +Make one with `berth resource create NAME --provisioner NAME --spec FILE`, or with the action `new-deployment` on the provisioner's page, which takes the Compose file pasted whole. @@ -166,7 +225,7 @@ A deployment's page lists these actions: ## Actions -The provisioner's page in berth, and `berth provisioner NAME`, list these +The provisioner's page in berth, and `berth provisioner show NAME`, list these actions: | Action | What it does | @@ -191,14 +250,13 @@ container is not root on the Docker host. With `runtime=sysbox` the plugin starts the container with `--runtime sysbox-runc` in place of `--privileged` and its cgroup and `/run` options: - berth provision studio-2 docker runtime=sysbox + berth host create studio-2 --provisioner docker runtime=sysbox The Docker host needs Sysbox installed. The plugin checks for the runtime `sysbox-runc` before it makes anything, and says what to install when it is missing. The action `engine` lists the runtimes the engine has, and the -provisioner's page offers `sysbox` only where the engine has it. The recipe -`incus-host` in the berth source installs Sysbox beside Docker when you ask it -to. Sysbox's own documentation lists the distributions and kernels it supports. +provisioner's page offers `sysbox` only where the engine has it. The feature +`docker/engine` installs Sysbox beside Docker on a node when you ask it to. Sysbox's own documentation lists the distributions and kernels it supports. Whether berth's role applies cleanly inside a Sysbox container has not been checked yet. These are the parts to watch: the firewall, the tailnet's tunnel diff --git a/plugins/docker/package.json b/plugins/docker/package.json index fc39698..de5aefb 100644 --- a/plugins/docker/package.json +++ b/plugins/docker/package.json @@ -9,10 +9,40 @@ ], "files": [ "bin", + "tasks", "Dockerfile", "guide.md" ], "berth": { + "features": [ + { + "id": "engine", + "group": "Servers", + "label": "Docker engine", + "description": "Install the Docker engine on a node for berth's machines, Compose projects and instances, with Sysbox beside it when devbox_docker_sysbox is true in the profile's ansible_vars. An engine that is installed already is kept as it is. berth registers the provisioner docker/container of this node after the apply, which reaches the engine over SSH.", + "summary": "The Docker engine (`docker`) runs containers here.", + "types": [ + "node" + ], + "offers": [ + "container" + ], + "commands": [ + "docker" + ], + "tasks": "tasks/engine.yml", + "actions": [ + { + "name": "engine-status", + "label": "Docker", + "user": "root", + "extends": "status", + "help": "The version of the Docker engine, its runtimes and its containers. Part of Status.", + "run": "echo; echo '── Docker'; docker version | sed -n '/^Server/,$p' | sed -n '1,3p'; docker info 2>/dev/null | grep -i '^ *runtimes:'; docker ps --all" + } + ] + } + ], "provisioners": [ { "id": "container", @@ -57,8 +87,8 @@ }, { "id": "instance", - "label": "Instance", - "verbs": ["start", "stop", "change", "logs"], + "label": "Container from an image", + "verbs": ["start", "stop", "change", "logs", "exec"], "arguments": [ {"name": "image", "label": "Image", "default": "", "help": "The image to start from, such as debian:12 or any image reference. It becomes no berth host: nothing of berth is put in it."}, {"name": "command", "label": "Command", "default": "", "help": "The command to run in the container, such as sleep infinity. Empty runs the image's own command."}, diff --git a/plugins/docker/tasks/engine.yml b/plugins/docker/tasks/engine.yml new file mode 100644 index 0000000..e7ef63f --- /dev/null +++ b/plugins/docker/tasks/engine.yml @@ -0,0 +1,118 @@ +--- +# A Docker engine on a node, for the plugin's machines, deployments and +# instances. An engine that is installed already, from Ubuntu's packages or +# Docker's own, is taken as it is, with its containers, volumes and settings. +# Sysbox is installed beside it when the profile asks. +- name: Take the Docker settings of the profile + ansible.builtin.set_fact: + devbox_docker: + sysbox: "{{ devbox_docker_sysbox | default(false) | bool }}" + sysbox_version: "{{ devbox_docker_sysbox_version | default('0.7.1') }}" + sysbox_checksums: + amd64: "{{ devbox_docker_sysbox_checksum_amd64 | default('9d6d5484f980d0a17f86c492c1262015c2afb66280bdb97215b79fde6a0261c5') }}" + arm64: "{{ devbox_docker_sysbox_checksum_arm64 | default('04ca894ae0b53f0fa54eaacc173ce40363c9a95ea5450f773716a84ef650a69b') }}" + +- name: See whether Docker is installed + ansible.builtin.command: docker version --format '{% raw %}{{.Client.Version}}{% endraw %}' + register: devbox_docker_found + changed_when: false + failed_when: false + +# Docker's own packages conflict with Ubuntu's, so an engine that is there stays. +- name: Install Docker from Ubuntu's packages + ansible.builtin.apt: + name: [docker.io, docker-buildx, docker-compose-v2] + state: present + lock_timeout: 300 + environment: + DEBIAN_FRONTEND: noninteractive + when: devbox_docker_found.rc != 0 + +- name: Start Docker now and at boot + ansible.builtin.systemd_service: + name: docker.service + enabled: true + state: started + when: devbox_start_services + +- name: Let the SSH user use Docker + ansible.builtin.user: + name: "{{ devbox_user }}" + groups: docker + append: true + when: devbox_user != 'root' + +# Its installer restarts Docker when it has to change Docker's network +# settings, unless containers exist (then it leaves Docker running and says +# so). A host that has this version is left as it is. +- name: Install Sysbox beside Docker + when: devbox_docker.sysbox | bool + vars: + devbox_sysbox_arch: "{{ 'arm64' if ansible_facts.architecture == 'aarch64' else 'amd64' }}" + devbox_sysbox_sum: "{{ devbox_docker.sysbox_checksums[devbox_sysbox_arch] }}" + devbox_sysbox_package: "/root/sysbox-ce_{{ devbox_docker.sysbox_version }}.linux_{{ devbox_sysbox_arch }}.deb" + block: + - name: Require a checksum for the Sysbox package + ansible.builtin.assert: + that: devbox_sysbox_sum is match('^[0-9a-f]{64}$') + fail_msg: >- + Sysbox is installed as root, so its package must be checked. Set + devbox_docker_sysbox_checksum_{{ devbox_sysbox_arch }} to the SHA-256 of version + {{ devbox_docker.sysbox_version }} from its release page. + - name: See which Sysbox is installed + ansible.builtin.command: dpkg-query -W -f=${Version} sysbox-ce + register: devbox_sysbox_installed + changed_when: false + failed_when: false + - name: Download Sysbox + ansible.builtin.get_url: + url: "https://github.com/nestybox/sysbox/releases/download/v{{ devbox_docker.sysbox_version }}/sysbox-ce_{{ devbox_docker.sysbox_version }}.linux_{{ devbox_sysbox_arch }}.deb" + dest: "{{ devbox_sysbox_package }}" + mode: '0600' + checksum: "sha256:{{ devbox_sysbox_sum }}" + when: not devbox_sysbox_installed.stdout.startswith(devbox_docker.sysbox_version) + - name: Install Sysbox + ansible.builtin.apt: + deb: "{{ devbox_sysbox_package }}" + when: not devbox_sysbox_installed.stdout.startswith(devbox_docker.sysbox_version) + - name: Remove the package file + ansible.builtin.file: + path: "{{ devbox_sysbox_package }}" + state: absent + - name: Start Sysbox now and at boot + ansible.builtin.systemd_service: + name: sysbox.service + enabled: true + state: started + when: devbox_start_services + +# A berth server in a container here, with this engine's socket, calls it +# its local engine: when the apply comes from that container, a provisioner +# of its local engine is this one. +- name: Find the addresses and mounts of a berth server's container here + ansible.builtin.command: >- + docker container inspect --format + '{% raw %}{{range .NetworkSettings.Networks}}{{.IPAddress}} {{end}}|{{range .Mounts}}{{.Destination}} {{end}}{% endraw %}' + {{ devbox_berth_server_container | default('berth') | quote }} + register: devbox_docker_server + changed_when: false + failed_when: false + +# A server applying from its container with the socket reaches the engine +# without SSH; any other berth reaches it over SSH as the account it applies as. +- name: Offer the provisioner of this engine + ansible.builtin.set_fact: + devbox_node_units: "{{ devbox_node_units + ['docker'] }}" + devbox_offer: + provisioners: [container] + settings: + host: "{{ '' if devbox_docker_local else 'ssh://{user}@{address}:{port}' }}" + aliases: + host: "{{ [''] if devbox_docker_here else [] }}" + vars: + devbox_docker_found: "{{ (devbox_docker_server.stdout ~ '|').split('|') if devbox_docker_server.rc == 0 else ['', ''] }}" + devbox_docker_here: >- + {{ (devbox_apply_from | default('')) | length > 0 + and (devbox_apply_from | default('')) in devbox_docker_found[0].split() }} + devbox_docker_local: "{{ devbox_docker_here and '/var/run/docker.sock' in devbox_docker_found[1].split() }}" + diff --git a/plugins/incus/bin/provision b/plugins/incus/bin/provision index 4679292..6a0a12a 100755 --- a/plugins/incus/bin/provision +++ b/plugins/incus/bin/provision @@ -32,6 +32,10 @@ NAME_LIMIT = 63 PROTOCOL = 2 # Rows of a listing an action answers with, at most. ROWS = 60 +# What usually speaks on a port, as a hint for people. +HINTS = {22: 'ssh', 80: 'http', 443: 'https', 3000: 'http', 3306: 'mysql', 5432: 'postgres', 6379: 'redis', + 8000: 'http', 8080: 'http', 8443: 'https', 27017: 'mongodb'} +LOOPBACK = ('127.0.0.1', '::1', 'localhost') class Failure(Exception): @@ -62,6 +66,8 @@ class Incus: self.host = url.hostname self.port = url.port or 8443 self.project = project or 'default' + # The node the server runs on, whose controller dials the endpoints, when berth names one. + self.node = '' cert, key = secrets.get('incus_client_cert') or '', secrets.get('incus_client_key') or '' if not cert.strip() or not key.strip(): raise Failure('The secrets incus_client_cert and incus_client_key are needed to talk to Incus.') @@ -245,6 +251,43 @@ def state_of(instance): return 'stopped' if status in ('Stopped', 'Frozen', 'Error') else 'running' +def split_address(text): + """The protocol, address and port of a proxy device's listen or connect, such as + tcp:0.0.0.0:8080 or tcp:[::]:443, or None for a socket file or a range of ports.""" + protocol, _, rest = (text or '').partition(':') + address, _, port = rest.rpartition(':') + if protocol not in ('tcp', 'udp') or not port.isdigit() or not 0 < int(port) < 65536: + return None + return protocol, address.strip('[]'), int(port) + + +def endpoints_of(incus, instance): + """The instance's proxy devices, each a port of the Incus server that leads into it. An instance's + own address is its address; berth is not told which of its ports serve anything. On a node + berth names, its controller dials, so the server's loopback is the node's.""" + local = incus.host in LOOPBACK or bool(incus.node) + found = [] + for name, device in sorted(((instance or {}).get('expanded_devices') or {}).items()): + if (device or {}).get('type') != 'proxy': + continue + listen, connect = split_address(device.get('listen')), split_address(device.get('connect')) + if not listen: + continue + protocol, address, port = listen + # Listening on every address: the server's own name for it, or this machine's loopback. + if address in ('', '0.0.0.0', '::'): + address = '127.0.0.1' if local else incus.host + endpoint = {'name': re.sub(r'[^a-z0-9_.-]+', '-', name.lower()).strip('-._')[:63] or 'proxy', + 'address': address, 'port': port, 'protocol': protocol} + target = connect[2] if connect else port + if protocol == 'tcp' and target in HINTS: + endpoint['hint'] = HINTS[target] + if local: + endpoint['local'] = True + found.append(endpoint) + return found + + def checked_archive(seed): archive = (seed or {}).get('archive') or {} path, expected = archive.get('path') or '', archive.get('sha256') or '' @@ -355,7 +398,7 @@ def create(incus, request, arguments): raise Failure(f'There is no image {image} on {IMAGES} for {TYPES[kind]} instances: {reason}') from None raise Failure(f'Incus could not create {name}: {reason}') from None deliver(incus, name, kind, archive, False) - answer = {'id': name} + answer = {'id': name, 'endpoints': endpoints_of(incus, incus.instance(name))} address = wait_for_address(incus, name) if address: answer['address'] = address @@ -415,7 +458,8 @@ def create_instance(incus, request, arguments): if 'image' in reason.lower() and ('not found' in reason.lower() or "couldn't be found" in reason.lower()): raise Failure(f'There is no image {image} on {IMAGES} for {TYPES[kind]} instances: {reason}') from None raise Failure(f'Incus could not create {name}: {reason}') from None - answer = {'id': name, 'state': state_of(incus.instance(name))} + instance = incus.instance(name) + answer = {'id': name, 'state': state_of(instance), 'endpoints': endpoints_of(incus, instance)} address = wait_for_address(incus, name) if address: answer['address'] = address @@ -425,6 +469,8 @@ def create_instance(incus, request, arguments): def status(incus, request, arguments): instance = incus.instance(request['id']) answer = {'state': state_of(instance)} + if instance: + answer['endpoints'] = endpoints_of(incus, instance) if answer['state'] == 'running' else [] if answer['state'] == 'running': address = incus.address(request['id']) if address: @@ -534,7 +580,8 @@ def change(incus, request, arguments): if patch['config'] or 'devices' in patch: say(f'Changing {name}…') incus.call('PATCH', f'/1.0/instances/{name}', patch) - return {'state': state_of(incus.instance(name))} + instance = incus.instance(name) + return {'state': state_of(instance), 'endpoints': endpoints_of(incus, instance)} def logs(incus, request, arguments): @@ -555,6 +602,35 @@ def logs(incus, request, arguments): return None +def run_in(incus, request, arguments): + """Run the request's command in a running instance, with its output recorded by Incus and then + written on standard error, which berth shows as the job's: the command's exit status.""" + name = request['id'] + command = request.get('command') + if not isinstance(command, list) or not command or not all(isinstance(word, str) for word in command): + raise Failure('The request carries no command as a list of words.') + if request.get('stdin'): + raise Failure('Incus runs the command here without standard input: give it none.') + instance = incus.instance(name) + if not instance: + raise Failure(f'{name} is no longer on {incus.server}.') + if state_of(instance) != 'running': + raise Failure(f'{name} is not running: start it first.') + done = incus.call('POST', f'/1.0/instances/{name}/exec', { + 'command': command, 'environment': {}, 'interactive': False, + 'wait-for-websocket': False, 'record-output': True, + }) or {} + for stream in ('1', '2'): + path = ((done.get('output') or {}).get(stream) or '').split('?')[0] + if not path.startswith(f'/1.0/instances/{name}/logs/'): + continue + sys.stderr.write(incus.text(path)) + sys.stderr.flush() + # The recorded output stays on the server until it is removed. + incus.exchange('DELETE', path) + return {'exit_code': int(done.get('return', -1))} + + def counted(count, word): return f'{count} {word}' + ('' if count == 1 else 's') @@ -698,9 +774,9 @@ def status_of(incus, request, arguments): VERBS = {'create': create, 'status': status_of, 'destroy': destroy, 'start': power('start'), 'stop': power('stop'), 'list': listed, - 'change': change, 'logs': logs, 'act': act} + 'change': change, 'logs': logs, 'act': act, 'exec': run_in} # The verbs that are about one instance, and need its id. -ABOUT_ONE = ('destroy', 'start', 'stop', 'change', 'logs') +ABOUT_ONE = ('destroy', 'start', 'stop', 'change', 'logs', 'exec') def main(): @@ -727,6 +803,7 @@ def main(): arguments = request.get('arguments') or {} with tempfile.TemporaryDirectory(prefix='berth-incus-') as scratch: incus = Incus(arguments.get('server') or '', arguments.get('project'), request.get('secrets') or {}, scratch) + incus.node = request.get('node') or '' incus.check_trust() answer = VERBS[verb](incus, request, arguments) except Failure as exc: diff --git a/plugins/incus/guide.md b/plugins/incus/guide.md index af3cb51..6c105ab 100644 --- a/plugins/incus/guide.md +++ b/plugins/incus/guide.md @@ -26,6 +26,54 @@ While berth waits for a new machine to answer on SSH, it shows the latest lines of the machine's console log. When the console has none, the plugin reads cloud-init's output in the machine instead. +## An Incus server on a node + +The feature `incus/server` installs Incus on a [node](https://github.com/helightdev/berth/blob/main/docs/content/docs/%28guide%29/use/nodes.mdx), +a server that berth applies without a desktop, and sets it up for berth's +machines. berth then registers the provisioners `NODE-incus-vm` and +`NODE-incus-container` for it, so the steps of the next section are done for +you. List it among the node's features: + + "ansible_vars": {"devbox_plugin_features": ["incus/server"]} + +| Part | Set-up | +| --- | --- | +| Incus | From Ubuntu's packages, with QEMU for virtual machines | +| Network | The bridge `incusbr0` at `10.77.0.1/24`. The node routes the guests' traffic out, and lets nothing in to them. No IPv6 | +| Storage | A pool named `default` with the driver `dir`, in a folder on the node's root file system | +| Default profile | Nesting is on. A container's systemd needs it, and so does Docker in a guest | +| API | `[::]:8444`, open in the node's firewall on the tailnet, to the address the apply came from, and to Docker's bridge | +| Trust | berth's client certificate. berth makes it once, as the secrets `incus_client_cert` and `incus_client_key`. The certificate goes to the node, and the key stays with berth | + +The settings go in the profile's `ansible_vars`: + +| Setting | Default | Meaning | +| --- | --- | --- | +| `devbox_incus_virtual_machines` | `true` | Install QEMU, and offer `incus/vm`. The node needs KVM | +| `devbox_incus_pool` | `default` | The storage pool's name | +| `devbox_incus_pool_driver` | `dir` | `dir`, `btrfs`, `zfs` or `lvm` | +| `devbox_incus_pool_source` | empty | An existing folder, btrfs file system, ZFS pool or LVM volume group. Never a disk | +| `devbox_incus_bridge` | `incusbr0` | The guests' bridge | +| `devbox_incus_bridge_address` | `10.77.0.1/24` | The node's address on the bridge. It must not be a network the node is on already | +| `devbox_incus_api_address` | empty | Where the API listens. Empty keeps the address Incus has, or `[::]:8444` when it has none | +| `devbox_incus_api_interfaces` | `[tailscale0]` | The interfaces the firewall opens the API on | +| `devbox_incus_api_from` | the apply's address | An address or network the firewall opens the API to besides | + +The feature does not erase a disk. Making a storage pool on a disk would, so +it never does: it makes a pool in a folder, or uses the source you name. For +a pool on a disk of its own, create it on the node first and name it: + + incus storage create fast zfs source=/dev/nvme1n1 + +An Incus that has a storage pool is set up already, and the feature keeps it +as it is. Its pools, networks, profiles and trusted clients stay. Only an API +address, when it has none, and berth's certificate are added. A server the +`incus-host` recipe set up is taken in so, with its guests. A +provisioner registered there by hand becomes the node's. + +berth reaches the API at the address it has for the node, and pins the +server's certificate. The fingerprint is in the apply's output. + ## Let the server trust berth Incus accepts a client whose certificate is in its trust store. Make a key and @@ -68,7 +116,7 @@ size are under Machine on the Settings tab. `server` and `project` are the provisioner's own arguments. berth keeps them as its settings, which every machine and action uses unless the call gives a value: - berth provisioner ovh-incus run settings server=https://192.0.2.10:8443 + berth provisioner set ovh-incus server=https://192.0.2.10:8443 | Argument | Default | Meaning | | --- | --- | --- | @@ -94,10 +142,11 @@ host. It starts from any image on `https://images.linuxcontainers.org`, such as `debian/12`, and berth puts nothing of its own in it. Cloud-init user data that you paste sets it up on its first boot, if the image has cloud-init. -Make one with **+ New instance** under the hosts, or the action New instance -on the provisioner's page. Both open a form laid out like a host's set-up, with every field on a row. Or: +Make one with **+ New resource from an image** under the hosts. Or use the +provisioner's action New container from an image, or New virtual machine from +an image for a provisioner of virtual machines. Both open a form laid out like a host's set-up, with every field on a row. Or: - berth provisioner ovh-incus run new-instance name=web image=debian/12 + berth provisioner run ovh-incus new-instance name=web image=debian/12 berth keeps it as a resource with a page of its own. There it starts, stops, changes and removes it, and shows its console log. The plugin marks it with @@ -116,9 +165,27 @@ instance for it. | `network` | | Network for the instance. Left empty, it uses the network of the `default` profile. | | `nesting` | `yes` | Lets a container run containers of its own, such as Docker's. | +### Endpoints and commands + +The plugin reports each proxy device of an instance or a machine as an +endpoint, named after the device. A proxy device is a port of the Incus server +that leads into the instance, such as `listen: tcp:0.0.0.0:8080`. A device that +listens on every address is reported at the server's host name from `server`. +An instance's own address is its address; Incus does not say which of its +ports serve anything, so those are no endpoints. berth opens a tunnel to an +endpoint with `resources.connect`. For a provisioner a node registers, berth +names the node in each request. The node's controller then dials, so a device +that listens on every address is reported at the node's `127.0.0.1`. + +An instance also takes the action `exec`. Incus runs the command and records +its output, which the plugin then gives as the job's and removes from the +server. The job ends with the command's exit status. The instance has to be +running. Incus runs it without standard input, and the plugin refuses a command that +comes with some. + ## Actions -The provisioner's page in berth, and `berth provisioner NAME`, list these +The provisioner's page in berth, and `berth provisioner show NAME`, list these actions: | Action | What it does | @@ -148,5 +215,6 @@ The page also offers the server's pools and networks for `pool` and `network`. the machine. - The server needs to reach `https://images.linuxcontainers.org` to download the image the first time. -- berth reaches the machine through its tailnet once it's set up. The address - Incus reports is only the first way in. +- Once it's set up, berth reaches the machine through the tailnet when it has + joined one. Without the tailnet, berth reaches it over SSH at the address + Incus reports. diff --git a/plugins/incus/package.json b/plugins/incus/package.json index da700b3..2658729 100644 --- a/plugins/incus/package.json +++ b/plugins/incus/package.json @@ -9,9 +9,44 @@ ], "files": [ "bin", + "tasks", "guide.md" ], "berth": { + "features": [ + { + "id": "server", + "group": "Servers", + "label": "Incus server", + "description": "Install Incus on a node and set it up for berth's machines: the bridge incusbr0 for the guests, a storage pool in a directory, and the API on port 8444 for clients it trusts by certificate. berth's own client certificate is trusted, and berth registers the provisioners incus/vm and incus/container of this node after the apply. An Incus that is set up already keeps its pools, networks and trust. The settings are devbox_incus_* in the profile's ansible_vars; the guide has them.", + "summary": "Incus (`incus`) hosts virtual machines and containers here, on the bridge `incusbr0`.", + "types": [ + "node" + ], + "offers": [ + "vm", + "container" + ], + "trusts": { + "certificate": "incus_client_cert", + "key": "incus_client_key" + }, + "commands": [ + "incus" + ], + "tasks": "tasks/server.yml", + "actions": [ + { + "name": "server-status", + "label": "Incus", + "user": "root", + "extends": "status", + "help": "The version of Incus, its storage pools and its instances. Part of Status.", + "run": "echo; echo '── Incus'; incus version; incus storage list --format compact; incus list --format compact --columns nst" + } + ] + } + ], "provisioners": [ { "id": "vm", @@ -43,8 +78,8 @@ }, { "id": "instance", - "label": "Instance", - "verbs": ["start", "stop", "change", "logs"], + "label": "Virtual machine from an image", + "verbs": ["start", "stop", "change", "logs", "exec"], "arguments": [ {"name": "image", "label": "Image", "default": "ubuntu/24.04", "help": "The image on images.linuxcontainers.org to start from, such as debian/12 or alpine/3.20. It becomes no berth host: nothing of berth is put in it."}, {"name": "user_data", "label": "Cloud-init user data", "kind": "text", "help": "What cloud-init does on its first boot, such as packages to install and users to add, for an image that has cloud-init. Empty leaves the image as it is."}, @@ -119,8 +154,8 @@ }, { "id": "instance", - "label": "Instance", - "verbs": ["start", "stop", "change", "logs"], + "label": "Container from an image", + "verbs": ["start", "stop", "change", "logs", "exec"], "arguments": [ {"name": "image", "label": "Image", "default": "ubuntu/24.04", "help": "The image on images.linuxcontainers.org to start from, such as debian/12 or alpine/3.20. It becomes no berth host: nothing of berth is put in it."}, {"name": "user_data", "label": "Cloud-init user data", "kind": "text", "help": "What cloud-init does on its first boot, such as packages to install and users to add, for an image that has cloud-init. Empty leaves the image as it is."}, diff --git a/plugins/incus/tasks/server.yml b/plugins/incus/tasks/server.yml new file mode 100644 index 0000000..23971fa --- /dev/null +++ b/plugins/incus/tasks/server.yml @@ -0,0 +1,180 @@ +--- +# Incus on a node, with the bridge its guests are on, a storage pool, and its +# API over HTTPS for clients it trusts by certificate. An Incus that is set up +# already is taken as it is: its pools, networks, profiles and trust stay, and +# only the API address and berth's certificate are added when missing. A +# storage pool is never made on a disk, which would erase it. +- name: Take the Incus settings of the profile + ansible.builtin.set_fact: + devbox_incus: + virtual_machines: "{{ devbox_incus_virtual_machines | default(true) | bool }}" + pool: "{{ devbox_incus_pool | default('default') }}" + pool_driver: "{{ devbox_incus_pool_driver | default('dir') }}" + pool_source: "{{ devbox_incus_pool_source | default('') }}" + bridge: "{{ devbox_incus_bridge | default('incusbr0') }}" + bridge_address: "{{ devbox_incus_bridge_address | default('10.77.0.1/24') }}" + api_address: "{{ devbox_incus_api_address | default('') }}" + api_interfaces: "{{ devbox_incus_api_interfaces | default(['tailscale0']) }}" + api_from: "{{ devbox_incus_api_from | default(devbox_apply_from | default('')) }}" + client: "{{ (devbox_client_certificates | default({}))[devbox_plugin_feature] | default({}) }}" + +- name: Check the Incus settings + ansible.builtin.assert: + that: + - devbox_incus.pool is match('^[a-zA-Z0-9][a-zA-Z0-9_.-]*$') + - devbox_incus.pool_driver in ['dir', 'btrfs', 'zfs', 'lvm'] + - devbox_incus.bridge is match('^[a-z][a-z0-9-]{0,14}$') + # A source on a device would be formatted. + - not devbox_incus.pool_source.startswith('/dev/') + - devbox_incus.pool_driver in ['dir', 'btrfs'] or devbox_incus.pool_source | length > 0 + fail_msg: >- + devbox_incus_pool_driver is dir, btrfs, zfs or lvm, and devbox_incus_pool_source names + an existing directory, btrfs file system, ZFS pool or LVM volume group, never a disk: + making a pool on a disk erases it, so the feature does not. A zfs or lvm pool needs its + source. Create a pool on a disk yourself with incus storage create and name it in + devbox_incus_pool. + +- name: Install Incus + ansible.builtin.apt: + name: >- + {{ ['incus'] + + (['qemu-system-x86', 'ovmf'] if devbox_incus.virtual_machines | bool else []) + + (['btrfs-progs'] if devbox_incus.pool_driver == 'btrfs' else []) + + (['zfsutils-linux'] if devbox_incus.pool_driver == 'zfs' else []) + + (['lvm2'] if devbox_incus.pool_driver == 'lvm' else []) }} + state: present + lock_timeout: 300 + environment: + DEBIAN_FRONTEND: noninteractive + +- name: Start Incus now and at boot + ansible.builtin.systemd_service: + name: "{{ item }}" + enabled: true + state: started + loop: [incus.socket, incus.service] + when: devbox_start_services + +- name: See what Incus has + ansible.builtin.command: "incus {{ item }} list --format csv --columns n" + loop: [storage, network] + register: devbox_incus_found + changed_when: false + +- name: Note what Incus has + ansible.builtin.set_fact: + devbox_incus_pools: "{{ devbox_incus_found.results[0].stdout_lines | map('trim') | select | list }}" + devbox_incus_networks: "{{ devbox_incus_found.results[1].stdout_lines | map('trim') | select | list }}" + +# Only an Incus with no storage pool is new: one that has a pool keeps its +# set-up, so its guests are never touched. +- name: Set a new Incus up with the guests' bridge, a storage pool and the default profile + ansible.builtin.command: incus admin init --preseed + args: + stdin: "{{ devbox_incus_preseed | to_nice_yaml }}" + when: devbox_incus_pools | length == 0 + changed_when: true + vars: + devbox_incus_preseed: + networks: >- + {{ [] if devbox_incus.bridge in devbox_incus_networks else + [{'name': devbox_incus.bridge, 'type': 'bridge', + 'config': {'ipv4.address': devbox_incus.bridge_address, 'ipv4.nat': 'true', 'ipv6.address': 'none'}}] }} + storage_pools: + - name: "{{ devbox_incus.pool }}" + driver: "{{ devbox_incus.pool_driver }}" + config: "{{ {'source': devbox_incus.pool_source} if devbox_incus.pool_source | length > 0 else {} }}" + profiles: + - name: default + # A container's systemd does not come up without it, and Docker in a guest needs it too. + config: + security.nesting: "true" + devices: + eth0: {name: eth0, network: "{{ devbox_incus.bridge }}", type: nic} + root: {path: /, pool: "{{ devbox_incus.pool }}", type: disk} + +- name: Say what an Incus set up before keeps + ansible.builtin.debug: + msg: >- + Incus was set up already, with the storage pools {{ devbox_incus_pools | join(', ') }} + and the networks {{ devbox_incus_networks | join(', ') or 'none' }}. They stay as they are. + when: devbox_incus_pools | length > 0 + +- name: Read where the Incus API listens + ansible.builtin.command: incus config get core.https_address + register: devbox_incus_listens + changed_when: false + +# An address chosen before stays unless the profile names another. +- name: Set where the Incus API listens + ansible.builtin.command: + argv: [incus, config, set, core.https_address, "{{ devbox_incus_wanted }}"] + when: devbox_incus_listens.stdout | trim != devbox_incus_wanted + changed_when: true + vars: + devbox_incus_wanted: >- + {{ devbox_incus.api_address if devbox_incus.api_address | length > 0 + else (devbox_incus_listens.stdout | trim or '[::]:8444') }} + +- name: Read where the Incus API listens now + ansible.builtin.command: incus config get core.https_address + register: devbox_incus_listening + changed_when: false + +- name: Read the clients Incus trusts + ansible.builtin.command: incus config trust list --format json + register: devbox_incus_trusted + changed_when: false + +# Only berth's certificate comes here; its key stays with berth. +- name: Trust berth's client certificate + when: + - devbox_incus.client.fingerprint | default('') | length > 0 + - devbox_incus.client.fingerprint not in (devbox_incus_trusted.stdout | from_json | map(attribute='fingerprint') | list) + block: + - name: Put berth's certificate where Incus reads it + ansible.builtin.copy: + dest: /root/berth-incus-client.crt + content: "{{ devbox_incus.client.certificate }}" + mode: '0600' + - name: Add berth's certificate to the trust store + ansible.builtin.command: + argv: [incus, config, trust, add-certificate, /root/berth-incus-client.crt, --name, "{{ devbox_incus.client.name }}"] + changed_when: true + always: + - name: Remove the copy of berth's certificate + ansible.builtin.file: + path: /root/berth-incus-client.crt + state: absent + +- name: Read the server's certificate + ansible.builtin.slurp: + src: /var/lib/incus/server.crt + register: devbox_incus_server_cert + +- name: Read the fingerprint of the server's certificate + ansible.builtin.command: openssl x509 -noout -fingerprint -sha256 -in /var/lib/incus/server.crt + register: devbox_incus_fingerprint + changed_when: false + +- name: Open the firewall to the guests and to the API, and offer the provisioners + ansible.builtin.set_fact: + devbox_firewall_rules: >- + {{ devbox_firewall_rules + + ['allow in on ' ~ devbox_incus.bridge, 'route allow in on ' ~ devbox_incus.bridge, 'route allow out on ' ~ devbox_incus.bridge] + + (devbox_incus.api_interfaces | map('regex_replace', '^(.*)$', 'allow in on \\1 to any port ' ~ devbox_incus_port ~ ' proto tcp') | list) + + (['allow from ' ~ devbox_incus.api_from ~ ' to any port ' ~ devbox_incus_port ~ ' proto tcp'] if devbox_incus.api_from | length > 0 else []) + + (['allow in on docker0 to any port ' ~ devbox_incus_port ~ ' proto tcp'] if 'docker0' in ansible_facts.interfaces else []) }} + devbox_node_units: "{{ devbox_node_units + ['incus'] }}" + devbox_offer: + provisioners: "{{ ['vm', 'container'] if devbox_incus.virtual_machines | bool else ['container'] }}" + settings: + server: "https://{address}:{{ devbox_incus_port }}" + secrets: + incus_server_cert: "{{ devbox_incus_server_cert.content | b64decode }}" + # The name a berth server in a container on this node gave it. + aliases: + server: ["https://incus-host:{{ devbox_incus_port }}"] + fingerprint: "{{ devbox_incus_fingerprint.stdout.split('=')[-1] | replace(':', '') | lower | trim }}" + vars: + devbox_incus_port: "{{ (devbox_incus_listening.stdout | trim).rsplit(':', 1)[-1] }}" diff --git a/provision/ansible/roles/devbox/defaults/main.yml b/provision/ansible/roles/devbox/defaults/main.yml index a299d49..b4514c5 100644 --- a/provision/ansible/roles/devbox/defaults/main.yml +++ b/provision/ansible/roles/devbox/defaults/main.yml @@ -109,11 +109,16 @@ devbox_base_commands: [git, rg, jq, curl, python3, pip, uv, node, npm, gcc, g++, devbox_install_playwright: true devbox_install_docker: true devbox_install_tailscale: true +# An apply of a host off the tailnet goes on without it when berth cannot +# make the host's key, rather than stop. berth sets it once for the hosts of +# berth 0.5 that it never joined by itself; it joins them once it can. +devbox_tailnet_optional: false # Web console on port 9090, reachable over the tailnet or an SSH tunnel. devbox_install_cockpit: true devbox_cockpit_packages: [cockpit, cockpit-storaged, cockpit-packagekit, cockpit-files] devbox_start_services: true -# How much of the role runs: workstation for all of it, controller for +# How much of the role runs: workstation for all of it, node for a server +# without a desktop (berth sets it for a host of that type), controller for # berthd and box alone (the playbook controller.yml sets it). devbox_level: workstation # Where berthd's files are, which installing the controller alone reads first. @@ -127,13 +132,46 @@ devbox_berthd_roots: ["{{ devbox_home }}", /srv/agent] devbox_berthd_units: "{{ ['ssh', 'tailscaled', 'berthd', 'cockpit.socket', 'docker', 'lightdm'] + devbox_service_units }}" # The account sessions run as on a host with the controller alone: the SSH user. devbox_controller_user: "{{ ansible_facts.env.SUDO_USER | default(ansible_facts.user_id) }}" +# A node's account berth logs in as, whom sessions run as: the SSH user too. +# berth sets it from the profile's access.user. +devbox_node_user: "{{ devbox_controller_user }}" +# Who reaches SSH on a node from outside the tailnet: any, or one address or +# network such as 198.51.100.7. The address an apply comes from always does. +devbox_ssh_from: any +# What a node needs of its own; its features install the rest. +devbox_node_packages: + - sudo + - ca-certificates + - curl + - gnupg + - locales + - tzdata + - git + - jq + - rsync + - unzip + - xz-utils + - zstd + - iproute2 + - openssh-server + - ufw + - unattended-upgrades + - python3 + - python3-psutil +# The name of a berth server's container on a node, which the action +# upgrade-server runs from a newer image. +devbox_berth_server_container: berth +# berth sets these when the host joins the tailnet (devbox_install_tailscale); +# a profile cannot. The one-use key berth made for a seed: devbox_tailscale_auth_key: "" # The Headscale server the host joins, such as https://hs.example.org; empty # for Tailscale's own service. devbox_tailscale_login_server: "" -# Set by berth for a host that joins automatically: the command, run where -# berth runs, that prints a new one-use key. +# For an apply: the command, run where berth runs, that prints a new one-use key. devbox_tailscale_key_command: [] +# For an apply when berth cannot make keys: why not, said before anything +# changes on a host that is not on the tailnet. +devbox_tailnet_problem: "" # Publish the desktop and T3 on the tailnet over HTTPS once the host is enrolled. devbox_tailscale_serve: true # An agent laptop keeps running with the lid closed. Set to suspend on battery instead. diff --git a/provision/ansible/roles/devbox/files/berth-server-upgrade.py b/provision/ansible/roles/devbox/files/berth-server-upgrade.py new file mode 100644 index 0000000..8349cfe --- /dev/null +++ b/provision/ansible/roles/devbox/files/berth-server-upgrade.py @@ -0,0 +1,386 @@ +#!/usr/bin/env python3 +"""Run the berth server in a container from the image of another version, or +make it again with options of its own, keeping the data and the options of +the running container. + + berth-server-upgrade CONTAINER the version in BERTH_ARG_VERSION + berth-server-upgrade CONTAINER VERSION + berth-server-upgrade CONTAINER --image IMAGE [--pull] [--keep-image] + [--publish P]… [--volume SRC:DST]… [--add-host NAME:ADDRESS]… [--group-add GROUP]… + +The first two are the node's action upgrade-server: the image is the +container's own repository at VERSION. The third is the server recipe's: it +takes the image as given, or keeps the container's own with --keep-image, and +sets the options it names over those of the container. An option it names +replaces the container's one for the same port, mount point or host name. +Nothing changes when the container runs that image with those options +already. + +The new container gets the same name, mounts, published ports, restart +policy, groups, extra hosts, network, logging, user and environment as the +old one, so it keeps the server's data and what it reaches. An option this +does not know how to carry over stops it before anything changes. The old +container is renamed and stopped, not removed, until the new one runs; if the +new one does not come up, the old one is started again. Environment values +go to docker in a private file, never on its command line. +""" +import argparse +import json +import os +import re +import subprocess +import sys +import tempfile +import time + +DOCKER = os.environ.get('DEVBOX_DOCKER', 'docker') +VERSION = re.compile(r'^[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$') +# How long the new container has to stay running before the old one goes. +SETTLE = float(os.environ.get('DEVBOX_UPGRADE_SETTLE', '10')) +# What `docker run` gives a container without --shm-size. +DEFAULT_SHM = 64 * 1024 * 1024 +UNCHANGED = 'is unchanged' + + +class Refused(Exception): + """A reason to change nothing, said in a sentence.""" + + +def docker(*args, check=True, capture=True): + done = subprocess.run([DOCKER, *args], text=True, capture_output=capture) + if check and done.returncode != 0: + raise Refused(f'docker {args[0]} {args[1] if len(args) > 1 else ""} failed: ' + f'{(done.stderr or done.stdout or "").strip()}') + return done + + +def inspect(kind, name): + return json.loads(docker(kind, 'inspect', name).stdout)[0] + + +def repository(image): + """An image reference without its tag or digest.""" + image = image.split('@', 1)[0] + last = image.rsplit('/', 1)[-1] + if ':' in last: + image = image[:len(image) - len(last)] + last.split(':', 1)[0] + return image + + +def version_key(text): + numbers = re.match(r'^([0-9]+)\.([0-9]+)\.([0-9]+)', text or '') + return tuple(int(part) for part in numbers.groups()) if numbers else None + + +def image_version(image, ref): + labels = (image.get('Config') or {}).get('Labels') or {} + return labels.get('org.opencontainers.image.version') or ref.rsplit(':', 1)[-1] + + +# Options of a container that would change what it is if they were dropped, +# and which this does not carry over: one that is set refuses the upgrade. +# What docker itself fills in for every container, such as the masked paths, +# the cgroup namespace or the console size, is not among them. +UNCARRIED = { + 'Privileged': 'it runs privileged', + 'CapAdd': 'it has capabilities added', + 'CapDrop': 'it has capabilities dropped', + 'Devices': 'it has devices', + 'DeviceRequests': 'it asks for devices such as GPUs', + 'DeviceCgroupRules': 'it has device rules', + 'SecurityOpt': 'it has security options', + 'Sysctls': 'it has sysctls', + 'Tmpfs': 'it has tmpfs mounts', + 'VolumesFrom': 'it takes volumes from another container', + 'Links': 'it has links', + 'Dns': 'it has DNS servers of its own', + 'DnsSearch': 'it has DNS search domains of its own', + 'DnsOptions': 'it has DNS options of its own', + 'Ulimits': 'it has ulimits', + 'PidMode': 'it shares a process namespace', + 'UTSMode': 'it shares a UTS namespace', + 'UsernsMode': 'it has a user namespace mode', + 'CgroupParent': 'it has a cgroup parent of its own', + 'CpuShares': 'it has CPU shares', + 'CpuQuota': 'it has a CPU quota', + 'CpuPeriod': 'it has a CPU period', + 'CpusetCpus': 'it is pinned to CPUs', + 'CpusetMems': 'it is pinned to memory nodes', + 'BlkioWeight': 'it has a block I/O weight', + 'MemoryReservation': 'it has a memory reservation', + 'OomKillDisable': 'it is kept from the OOM killer', + 'OomScoreAdj': 'it has an OOM score of its own', + 'ReadonlyRootfs': 'its root file system is read-only', + 'AutoRemove': 'it is removed when it stops', + 'StorageOpt': 'it has storage options', +} + + +def refusals_of(container, image_config): + config, host = container['Config'], container['HostConfig'] + refusals = [why for key, why in UNCARRIED.items() + if host.get(key) not in (None, '', 0, False, [], {})] + ipc = host.get('IpcMode') + if ipc not in (None, '', 'private', 'shareable'): + refusals.append(f'it shares an IPC namespace ({ipc})') + swap = host.get('MemorySwap') + if swap not in (None, 0, -1) and swap != 2 * (host.get('Memory') or 0): + refusals.append('it has a swap limit of its own') + runtime = host.get('Runtime') or 'runc' + if runtime != 'runc': + refusals.append(f'it runs under the runtime {runtime}') + if (config.get('Entrypoint') or None) != (image_config.get('Entrypoint') or None): + refusals.append('its entry point is not its image\'s') + if (config.get('WorkingDir') or '') != (image_config.get('WorkingDir') or ''): + refusals.append('its working directory is not its image\'s') + if config.get('Healthcheck') and config.get('Healthcheck') != image_config.get('Healthcheck'): + refusals.append('it has a health check of its own') + if len(((container.get('NetworkSettings') or {}).get('Networks') or {})) > 1: + refusals.append('it is on more than one network') + for mount in container.get('Mounts') or []: + if mount.get('Type') not in ('bind', 'volume'): + refusals.append(f"it has a {mount.get('Type')} mount at {mount.get('Destination')}") + return refusals + + +def mount_option(mount): + source = mount['Source'] if mount.get('Type') == 'bind' else mount['Name'] + # An anonymous volume too is named, so the new container takes the same data. + modes = [part for part in (mount.get('Mode') or '').split(',') if part and part not in ('rw', 'ro')] + if not mount.get('RW', True): + modes.insert(0, 'ro') + return f"{source}:{mount['Destination']}" + (':' + ','.join(modes) if modes else '') + + +def run_options(container, old_image): + """The options of `docker run` that make a container like this one, its + environment beyond what its image sets, and its command when it is not + the image's. Refused when it has one this does not carry over.""" + config, host = container['Config'], container['HostConfig'] + image_config = old_image.get('Config') or {} + refusals = refusals_of(container, image_config) + if refusals: + raise Refused('The container cannot be carried over as it is: ' + '; '.join(refusals) + + '. Upgrade it by hand.') + if (config.get('Cmd') or None) != (image_config.get('Cmd') or None) and config.get('Cmd'): + command = config['Cmd'] + else: + command = [] + mode = host.get('NetworkMode') or 'default' + + options = ['--detach', '--name', container['Name'].lstrip('/')] + policy = host.get('RestartPolicy') or {} + if policy.get('Name') and policy['Name'] != 'no': + retries = policy.get('MaximumRetryCount') or 0 + options += ['--restart', policy['Name'] + (f':{retries}' if policy['Name'] == 'on-failure' and retries else '')] + if mode not in ('default', 'bridge'): + options += ['--network', mode] + if config.get('Hostname') and not container['Id'].startswith(config['Hostname']) and mode != 'host': + options += ['--hostname', config['Hostname']] + if config.get('Domainname'): + options += ['--domainname', config['Domainname']] + for port, bindings in sorted((host.get('PortBindings') or {}).items()): + target = port.removesuffix('/tcp') + for binding in bindings or []: + address, published = binding.get('HostIp') or '', binding.get('HostPort') or '' + # An IPv6 address is published in brackets. + if ':' in address: + address = f'[{address}]' + options += ['--publish', f'{address}:{published}:{target}' if address else + (f'{published}:{target}' if published else target)] + if host.get('PublishAllPorts'): + options += ['--publish-all'] + for mount in container.get('Mounts') or []: + options += ['--volume', mount_option(mount)] + for group in host.get('GroupAdd') or []: + options += ['--group-add', group] + for extra in host.get('ExtraHosts') or []: + options += ['--add-host', extra] + if host.get('Init'): + options += ['--init'] + if host.get('Memory'): + options += ['--memory', str(host['Memory'])] + if host.get('NanoCpus'): + options += ['--cpus', str(host['NanoCpus'] / 1e9)] + if host.get('ShmSize') and host['ShmSize'] != DEFAULT_SHM: + options += ['--shm-size', str(host['ShmSize'])] + # The daemon's own log options, such as max-size in daemon.json, are set on + # every container: they are carried as they are, like any other. + log = host.get('LogConfig') or {} + if log.get('Type'): + options += ['--log-driver', log['Type']] + for key, value in sorted((log.get('Config') or {}).items()): + options += ['--log-opt', f'{key}={value}'] + if config.get('StopSignal') and config['StopSignal'] != image_config.get('StopSignal'): + options += ['--stop-signal', config['StopSignal']] + if config.get('StopTimeout') is not None and config.get('StopTimeout') != image_config.get('StopTimeout'): + options += ['--stop-timeout', str(config['StopTimeout'])] + if (config.get('User') or '') != (image_config.get('User') or ''): + options += ['--user', config['User']] + image_labels = image_config.get('Labels') or {} + for key, value in sorted((config.get('Labels') or {}).items()): + if image_labels.get(key) != value: + options += ['--label', f'{key}={value}'] + image_env = set(image_config.get('Env') or []) + environment = [line for line in config.get('Env') or [] if line not in image_env] + return options, environment, command + + +def pairs(options): + """The options after --detach and the name, as (flag, value) with None + for a flag without one.""" + found, rest = [], options[3:] + alone = {'--init', '--publish-all'} + while rest: + flag = rest.pop(0) + found.append((flag, None if flag in alone else rest.pop(0))) + return found + + +def key_of(flag, value): + """What an option is about, so one asked for replaces the container's.""" + if flag == '--publish': + return flag, value.rsplit(':', 1)[-1] + if flag == '--volume': + return flag, value.split(':')[1] if ':' in value else value + if flag == '--add-host': + return flag, re.split('[:=]', value, maxsplit=1)[0] + return flag, value + + +def with_asked(options, asked): + """The container's options with those asked for in place of the ones for + the same port, mount point or host name, and the others added.""" + head, kept = options[:3], pairs(options) + keys = {key_of(flag, value) for flag, value in asked} + kept = [(flag, value) for flag, value in kept + if flag not in ('--publish', '--volume', '--add-host', '--group-add') or key_of(flag, value) not in keys] + merged = kept + [option for option in asked if option not in kept] + out = list(head) + for flag, value in merged: + out += [flag] if value is None else [flag, value] + return out + + +def running(name): + done = docker('container', 'inspect', '--format', '{{.State.Running}}', name, check=False) + return done.returncode == 0 and done.stdout.strip() == 'true' + + +def replace(name, container, current, target, options, environment, command, label): + """Run `target` as `name` with these options, in place of the container.""" + previous = f'{name}-before-{label}' + docker('container', 'rm', '--force', previous, check=False) + with tempfile.NamedTemporaryFile('w', prefix='berth-env-', delete=False) as env_file: + os.chmod(env_file.name, 0o600) + env_file.write(''.join(line + '\n' for line in environment)) + try: + print(f'Stopping {name}, which runs {current}', flush=True) + docker('container', 'rename', name, previous) + docker('container', 'stop', previous) + started = docker('run', *options, '--env-file', env_file.name, target, *command, check=False) + time.sleep(SETTLE) + if started.returncode != 0 or not running(name): + said = (started.stderr or '').strip() + logs = docker('logs', '--tail', '20', name, check=False) + docker('container', 'rm', '--force', name, check=False) + docker('container', 'rename', previous, name) + docker('container', 'start', name) + raise Refused(f'The new container did not keep running, so {name} runs {current} again. ' + f'{said}\n{(logs.stdout + logs.stderr).strip()}'.strip()) + finally: + os.unlink(env_file.name) + docker('container', 'rm', previous) + + +def found_container(name): + found = docker('container', 'inspect', name, check=False) + if found.returncode != 0: + raise Refused(f'There is no container {name} here.') + return json.loads(found.stdout)[0] + + +def refuse_older(name, running_version, version): + if version_key(running_version) and version_key(version) and version_key(version) < version_key(running_version): + raise Refused(f'{name} runs berth {running_version}, which is newer than {version}. ' + 'An older server may not read what a newer one stored, so it is not done here.') + + +def upgrade(name, version): + """The node's action: the container's repository at `version`.""" + if not VERSION.match(version): + raise Refused(f'{version!r} is not a version of berth, such as 0.6.0') + container = found_container(name) + current = container['Config']['Image'] + target = f'{repository(current)}:{version}' + old_image = inspect('image', container['Image']) + refuse_older(name, image_version(old_image, current), version) + options, environment, command = run_options(container, old_image) + print(f'Pulling {target}', flush=True) + docker('pull', target, capture=False) + if inspect('image', target)['Id'] == container['Image']: + print(f'{name} runs {target} already.') + return + replace(name, container, current, target, options, environment, command, version) + print(f'{name} runs {target} now, with its data and options as they were.') + + +def remake(name, image, pull, keep_image, asked): + """The recipe's: `image`, or the container's own, with `asked` set over + the container's options.""" + container = found_container(name) + current = container['Config']['Image'] + old_image = inspect('image', container['Image']) + options, environment, command = run_options(container, old_image) + target = current if keep_image else image + if pull and not keep_image: + print(f'Pulling {target}', flush=True) + docker('pull', target, capture=False) + found = docker('image', 'inspect', target, check=False) + if found.returncode != 0: + if keep_image: + raise Refused(f'The image {target} of {name} is not here any more. Update it with the image to run.') + docker('pull', target, capture=False) + found = docker('image', 'inspect', target) + new_image = json.loads(found.stdout)[0] + if keep_image and new_image['Id'] != container['Image']: + # The tag was pulled again since: making the container again would update it unasked. + target = container['Image'] + new_image = old_image + refuse_older(name, image_version(old_image, current), image_version(new_image, target)) + wanted = with_asked(options, asked) + same = sorted(pairs(wanted), key=str) == sorted(pairs(options), key=str) + if new_image['Id'] == container['Image'] and same: + print(f'{name} {UNCHANGED}: it runs {current} with the options asked for.') + return + replace(name, container, current, target, wanted, environment, command, 'update') + print(f'{name} runs {target} now, with its data and options as they were and those asked for.') + + +def main(): + if len(sys.argv) > 2 and sys.argv[2].startswith('--'): + parser = argparse.ArgumentParser(prog='berth-server-upgrade') + parser.add_argument('container') + parser.add_argument('--image', required=True) + parser.add_argument('--pull', action='store_true') + parser.add_argument('--keep-image', action='store_true') + for flag in ('--publish', '--volume', '--add-host', '--group-add'): + parser.add_argument(flag, action='append', default=[]) + args = parser.parse_args() + asked = [(f'--{flag}', value) for flag in ('publish', 'volume', 'add-host', 'group-add') + for value in getattr(args, flag.replace('-', '_')) if value] + work = lambda: remake(args.container, args.image, args.pull, args.keep_image, asked) # noqa: E731 + elif len(sys.argv) in (2, 3): + version = sys.argv[2] if len(sys.argv) == 3 else os.environ.get('BERTH_ARG_VERSION', '') + work = lambda: upgrade(sys.argv[1], version.strip()) # noqa: E731 + else: + sys.exit('Usage: berth-server-upgrade CONTAINER [VERSION | --image IMAGE …]') + try: + work() + except Refused as refused: + print(f'✗ {refused}', file=sys.stderr) + sys.exit(1) + + +if __name__ == '__main__': + main() diff --git a/provision/ansible/roles/devbox/files/node-firewall.sh b/provision/ansible/roles/devbox/files/node-firewall.sh new file mode 100644 index 0000000..0d85836 --- /dev/null +++ b/provision/ansible/roles/devbox/files/node-firewall.sh @@ -0,0 +1,69 @@ +#!/usr/bin/env bash +# The firewall of a node: incoming traffic is refused except what the rules on +# standard input allow, one line of ufw arguments each, such as +# `allow in on incusbr0`. berth's rules carry the comment berth: one that is no +# longer asked for is deleted, and rules added by hand stay. Rules are added +# before the firewall is switched on, so SSH is never closed in between. It +# says on standard output what it changed, and nothing when nothing did. +set -euo pipefail +ufw=${DEVBOX_UFW:-ufw} +mark=berth + +mapfile -t wanted +rules=() +for rule in "${wanted[@]}"; do + rule=$(xargs <<<"$rule") + [[ -n $rule ]] && rules+=("$rule") +done + +# `ufw show added` lists each rule as the command that added it, with its comment. +added=$("$ufw" show added 2>/dev/null || true) +has() { grep -qxF -- "ufw $1 comment '$mark'" <<<"$added"; } + +for rule in "${rules[@]}"; do + if ! has "$rule"; then + # shellcheck disable=SC2086 # a rule is words for ufw + "$ufw" $rule comment "$mark" >/dev/null + echo "added: $rule" + fi +done + +while IFS= read -r line; do + [[ $line == "ufw "*" comment '$mark'" ]] || continue + rule=${line#ufw } + rule=${rule% comment \'$mark\'} + keep=0 + for wanted_rule in "${rules[@]}"; do + [[ $wanted_rule == "$rule" ]] && keep=1 + done + if [[ $keep == 0 ]]; then + # A routed rule is deleted as `ufw route delete RULE`. + # shellcheck disable=SC2086 # a rule is words for ufw + case $rule in + "route "*) "$ufw" route delete ${rule#route } >/dev/null ;; + *) "$ufw" delete $rule >/dev/null ;; + esac + echo "removed: $rule" + fi +done <<<"$added" + +# Should ufw have written a rule otherwise than it was given, deleting the old +# form took the rule away: it goes back in, so what is asked for always stays. +added=$("$ufw" show added 2>/dev/null || true) +for rule in "${rules[@]}"; do + if ! has "$rule"; then + # shellcheck disable=SC2086 # a rule is words for ufw + "$ufw" $rule comment "$mark" >/dev/null + fi +done + +status=$("$ufw" status 2>/dev/null || true) +if ! grep -q '^Status: active' <<<"$status"; then + "$ufw" default deny incoming >/dev/null + "$ufw" default allow outgoing >/dev/null + "$ufw" --force enable >/dev/null + echo 'switched the firewall on: incoming traffic is refused except the rules above' +elif ! "$ufw" status verbose 2>/dev/null | grep -q 'deny (incoming)'; then + "$ufw" default deny incoming >/dev/null + echo 'refused incoming traffic by default' +fi diff --git a/provision/ansible/roles/devbox/files/tasks/common.sh b/provision/ansible/roles/devbox/files/tasks/common.sh index 7727780..b9c623a 100644 --- a/provision/ansible/roles/devbox/files/tasks/common.sh +++ b/provision/ansible/roles/devbox/files/tasks/common.sh @@ -1,5 +1,6 @@ -# Shared by berth's task scripts. Sourced, not run. The tasks run on the host -# as root, started by berthd; `berth run HOST TASK` or the overview runs them. +# Shared by the scripts of berth's own actions. Sourced, not run. They run on +# the host as root, started by berthd; `berth run HOST ACTION` or the overview +# runs them. set -euo pipefail agent=$(. /etc/devbox/environment 2>/dev/null; echo "${DEVBOX_USER:-agent}") operator=$(jq -r '.operator // empty' /etc/berth/berthd.json 2>/dev/null || true) @@ -12,20 +13,33 @@ own_control() { [[ -n $control && ! $control =~ ^https://[a-z0-9.-]*tailscale\.c as_agent() { sudo -iu "$agent" agent-env "$@"; } +# What a host off the tailnet says where a tailnet address would be. +off_tailnet='Not on a tailnet: berth reaches this host over SSH at its address. With the secret tailnet_api_key set, applying it from berth joins it.' + +tunnels() { + echo "Desktop ssh -L 6080:127.0.0.1:6080 ${operator:-OPERATOR}@$1, then http://127.0.0.1:6080/vnc.html" + echo "T3 ssh -L 3773:127.0.0.1:3773 ${operator:-OPERATOR}@$1, then http://127.0.0.1:3773/" +} + access() { - if [[ -z $address ]]; then echo 'Not on a tailnet. Run: sudo tailscale up'; return; fi password=$(cat "/home/$agent/.config/tigervnc/password" 2>/dev/null || true) + if [[ -z $address ]]; then + echo "$off_tailnet Through an SSH tunnel, with the address berth uses:" + tunnels ADDRESS + echo "SSH ssh ${operator:-OPERATOR}@ADDRESS" + echo "Desktop (VNC) password: ${password:-not created yet}" + return + fi if own_control; then echo "The tailnet's control server, $control, does not offer tailscale serve. Through an SSH tunnel:" - echo "Desktop ssh -L 6080:127.0.0.1:6080 ${operator:-OPERATOR}@$address, then http://127.0.0.1:6080/vnc.html" - echo "T3 ssh -L 3773:127.0.0.1:3773 ${operator:-OPERATOR}@$address, then http://127.0.0.1:3773/" + tunnels "$address" elif tailscale serve status 2>/dev/null | grep -q 127.0.0.1:6080; then # After the #, so the password stays in the browser and reaches no server or log. encoded=$(python3 -c 'import sys, urllib.parse; print(urllib.parse.quote(sys.argv[1], safe=""))' "$password") echo "Desktop https://$name/vnc.html${password:+#autoconnect=1&password=$encoded}" echo "T3 https://$name:8443/" else - echo 'Desktop and T3 are not published on the tailnet yet (task: publish).' + echo 'Desktop and T3 are not published on the tailnet yet (action: publish).' fi if systemctl is-active -q cockpit.socket && own_control; then echo "Cockpit https://$address:9090 (its own certificate; log in as the operator)" diff --git a/provision/ansible/roles/devbox/files/tasks/pair.sh b/provision/ansible/roles/devbox/files/tasks/pair.sh index 27af8fc..b6758c1 100644 --- a/provision/ansible/roles/devbox/files/tasks/pair.sh +++ b/provision/ansible/roles/devbox/files/tasks/pair.sh @@ -2,10 +2,10 @@ # T3 asks every new browser to pair once. Its own pairing link points at this # machine only, so the token goes into the published tailnet address. . "$(dirname "$0")/common.sh" -[[ -n $name ]] || { echo 'Not on a tailnet. Run: sudo tailscale up' >&2; exit 1; } -tailscale serve status 2>/dev/null | grep -q 127.0.0.1:3773 || { echo 'T3 is not published on the tailnet yet (task: publish).' >&2; exit 1; } +[[ -n $name ]] || { echo "$off_tailnet" >&2; exit 1; } +tailscale serve status 2>/dev/null | grep -q 127.0.0.1:3773 || { echo 'T3 is not published on the tailnet yet (action: publish).' >&2; exit 1; } token=$(as_agent t3 pair --ttl 10m --label "berth $(date +%F)" | sed -n 's/^Token: *//p') -[[ -n $token ]] || { echo 'T3 gave no pairing token. Is it running? (task: status)' >&2; exit 1; } +[[ -n $token ]] || { echo 'T3 gave no pairing token. Is it running? (action: status)' >&2; exit 1; } echo "Browser https://$name:8443/pair#token=$token" echo "App: host https://$name:8443" echo "App: code $token" diff --git a/provision/ansible/roles/devbox/files/tasks/publish.sh b/provision/ansible/roles/devbox/files/tasks/publish.sh index 82dfcc2..88fe292 100644 --- a/provision/ansible/roles/devbox/files/tasks/publish.sh +++ b/provision/ansible/roles/devbox/files/tasks/publish.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash . "$(dirname "$0")/common.sh" -[[ -n $address ]] || { echo 'Not on a tailnet. Run: sudo tailscale up' >&2; exit 1; } +[[ -n $address ]] || { echo "$off_tailnet" >&2; exit 1; } if own_control; then echo "Nothing published: the tailnet's control server, $control, offers neither tailscale serve nor HTTPS certificates." >&2 access diff --git a/provision/ansible/roles/devbox/files/tasks/status.sh b/provision/ansible/roles/devbox/files/tasks/status.sh index 4f76085..10c9b03 100644 --- a/provision/ansible/roles/devbox/files/tasks/status.sh +++ b/provision/ansible/roles/devbox/files/tasks/status.sh @@ -3,7 +3,7 @@ # A host with berth's controller alone has no desktop, agents or tailnet of berth's. if [[ ${1:-} == controller ]]; then echo "Host $(hostname), up since $(uptime -s)" - echo "Level controller: berthd and box, nothing of a workstation" + echo "Type managed: berthd and box, nothing of a workstation" for unit in ssh berthd; do systemctl cat "$unit" >/dev/null 2>&1 && printf '%-11s %s\n' "$unit" "$(systemctl is-active "$unit" || true)" done @@ -12,6 +12,20 @@ if [[ ${1:-} == controller ]]; then { systemctl --failed --no-legend 2>/dev/null || true; } | sed 's/^/Failed /' exit 0 fi +# A node is a server: its controller, firewall, tailnet and what its features run. +if [[ ${1:-} == node ]]; then + echo "Host $(hostname), up since $(uptime -s)" + echo "Type node: berthd, firewall and plugin features, no desktop" + echo "Tailscale ${address:-not enrolled} ${name}" + for unit in ssh berthd tailscaled docker incus; do + systemctl cat "$unit" >/dev/null 2>&1 && printf '%-11s %s\n' "$unit" "$(systemctl is-active "$unit" || true)" + done + echo "Firewall $(ufw status 2>/dev/null | sed -n 's/^Status: //p' || echo unknown)" + echo "Disk $(df -h --output=avail,pcent / | tail -1 | xargs) free/used on /" + [[ -f /var/run/reboot-required ]] && echo 'Reboot required by installed updates' + { systemctl --failed --no-legend 2>/dev/null || true; } | sed 's/^/Failed /' + exit 0 +fi echo "Host $(hostname), up since $(uptime -s)" echo "Provision $(cat /var/lib/devbox/provision-state 2>/dev/null || echo unknown)" echo "Tailscale ${address:-not enrolled} ${name}" diff --git a/provision/ansible/roles/devbox/files/tasks/update.sh b/provision/ansible/roles/devbox/files/tasks/update.sh index 0b2db94..cd12ab2 100644 --- a/provision/ansible/roles/devbox/files/tasks/update.sh +++ b/provision/ansible/roles/devbox/files/tasks/update.sh @@ -4,4 +4,4 @@ export DEBIAN_FRONTEND=noninteractive apt-get update apt-get -y -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold dist-upgrade apt-get -y autoremove -[[ -f /var/run/reboot-required ]] && echo 'A reboot is required to finish (task: reboot).' || echo 'Up to date.' +[[ -f /var/run/reboot-required ]] && echo 'A reboot is required to finish (action: reboot).' || echo 'Up to date.' diff --git a/provision/ansible/roles/devbox/tasks/applied.yml b/provision/ansible/roles/devbox/tasks/applied.yml index 5093099..1df428c 100644 --- a/provision/ansible/roles/devbox/tasks/applied.yml +++ b/provision/ansible/roles/devbox/tasks/applied.yml @@ -2,15 +2,16 @@ # Which berth and plugins this configuration came from, for berthd to report: # berth compares it with what it would apply now to tell whether the host is # behind. Only the plugins whose features the host has are recorded. A host -# with the controller alone says so; a whole profile is recorded as before. +# with the controller alone says so, and so does a node, with the +# provisioners its features offer; a whole profile is recorded as before. - name: Record which berth and plugins this configuration came from ansible.builtin.copy: dest: /etc/berth/applied.json mode: '0644' - content: "{{ ({'berth': devbox_berth_applied.berth, 'plugins': devbox_berth_applied.plugins | selectattr('id', 'in', devbox_plugin_features | map('regex_replace', '/.*$', '') | list) | list} | combine({'level': devbox_level} if devbox_level != 'workstation' else {})) | to_nice_json }}\n" + content: "{{ ({'berth': devbox_berth_applied.berth, 'plugins': devbox_berth_applied.plugins | selectattr('id', 'in', devbox_plugin_features | map('regex_replace', '/.*$', '') | list) | list} | combine({'level': devbox_level} if devbox_level != 'workstation' else {}) | combine({'offers': devbox_plugin_offers} if devbox_plugin_offers | default([]) | length > 0 else {})) | to_nice_json }}\n" when: devbox_berth_applied is defined # The profile itself, as berth holds it, so that a berth without it can take -# the host over: berth adopt reads it over SSH, and berthd serves it to an +# the host over: berth host adopt reads it over SSH, and berthd serves it to an # administrator. It holds the accounts' password hashes, so only root reads it. # The time of the write leaves the comparison, so applying the same profile # again changes nothing. diff --git a/provision/ansible/roles/devbox/tasks/berth.yml b/provision/ansible/roles/devbox/tasks/berth.yml index 1f8a13e..94621fd 100644 --- a/provision/ansible/roles/devbox/tasks/berth.yml +++ b/provision/ansible/roles/devbox/tasks/berth.yml @@ -52,13 +52,19 @@ operator: "{{ '' if devbox_profile == 'container' else devbox_operator_user }}" roots: "{{ devbox_berthd_roots }}" units: "{{ devbox_berthd_units }}" - agent_env: /usr/local/bin/agent-env - - {name: tasks, value: "{{ devbox_tasks }}"} + # Only a workstation's agent has the wrapper with its display and tools. + agent_env: "{{ '/usr/local/bin/agent-env' if devbox_level == 'workstation' else '' }}" + - {name: actions, value: "{{ devbox_tasks }}"} - {name: services, value: "{{ devbox_plugin_services }}"} - {name: channels, value: "{{ devbox_plugin_channels }}"} loop_control: label: "{{ item.name }}" register: devbox_berth_files +# berthd read the actions from tasks.json before, and still would if it stayed. +- name: Remove the file of actions of berth 0.5 + ansible.builtin.file: + path: /etc/berth/tasks.json + state: absent # The agent's rights follow the services of plugins, so they are settled here # and not with the accounts. - name: Set what the agent may do as root diff --git a/provision/ansible/roles/devbox/tasks/controller_guard.yml b/provision/ansible/roles/devbox/tasks/controller_guard.yml index c1e5e5b..d914d67 100644 --- a/provision/ansible/roles/devbox/tasks/controller_guard.yml +++ b/provision/ansible/roles/devbox/tasks/controller_guard.yml @@ -1,6 +1,7 @@ --- -# A workstation's controller describes the agent and its desktop: installing -# the controller alone over it would take that apart, so it is refused here, +# A workstation's controller describes the agent and its desktop, and a +# node's its features: installing the controller alone over either would +# take that apart, so it is refused here, # whatever the profile on berth's side says. Nothing skips this check. - name: Read what berth installed on the host before ansible.builtin.slurp: @@ -18,17 +19,17 @@ ansible.builtin.set_fact: devbox_controller_recorded: "{{ (devbox_controller_found.results[0].content | b64decode | from_json) if 'content' in devbox_controller_found.results[0] else {} }}" devbox_controller_config: "{{ (devbox_controller_found.results[1].content | b64decode | from_json) if 'content' in devbox_controller_found.results[1] else {} }}" -- name: Refuse a host that berth set up as a workstation +- name: Refuse a host that berth set up as a workstation or a node ansible.builtin.fail: msg: >- - {{ inventory_hostname }} is a workstation that berth set up, and its - profile here says otherwise. Installing the controller alone would take - its controller apart, so nothing was changed. To adopt it from the - profile it keeps: berth adopt NAME. + {{ inventory_hostname }} is a {{ 'node' if devbox_controller_recorded.level | default('') == 'node' else 'workstation' }} + that berth set up, and its profile here says otherwise. Installing the + controller alone would take its controller apart, so nothing was + changed. To adopt it from the profile it keeps: berth host adopt NAME. when: >- devbox_controller_kept.stat.exists or (devbox_controller_recorded | length > 0 - and devbox_controller_recorded.level | default('workstation') == 'workstation') + and devbox_controller_recorded.level | default('workstation') in ['workstation', 'node']) - name: Refuse a host whose controller serves another account ansible.builtin.fail: msg: >- @@ -36,7 +37,7 @@ {{ devbox_controller_config.agent }}, not the SSH user {{ devbox_controller_user }}: berth set it up as a workstation, and its profile here says otherwise. Nothing was changed. To adopt it from the - profile it keeps: berth adopt NAME. + profile it keeps: berth host adopt NAME. when: - devbox_controller_config.agent | default('') | length > 0 - devbox_controller_config.agent != devbox_controller_user diff --git a/provision/ansible/roles/devbox/tasks/host.yml b/provision/ansible/roles/devbox/tasks/host.yml index e90cc0d..65be5f7 100644 --- a/provision/ansible/roles/devbox/tasks/host.yml +++ b/provision/ansible/roles/devbox/tasks/host.yml @@ -187,33 +187,8 @@ enabled: false state: stopped when: not devbox_install_cockpit and devbox_cockpit_unit.stat.exists and devbox_start_services -- name: Configure timezone - ansible.builtin.file: - src: "/usr/share/zoneinfo/{{ devbox_timezone }}" - dest: /etc/localtime - state: link - force: true -- name: Enable English locale - ansible.builtin.lineinfile: - path: /etc/locale.gen - regexp: '^#?\s*en_US.UTF-8 UTF-8' - line: en_US.UTF-8 UTF-8 - register: locale_config -- name: Generate locale - ansible.builtin.command: locale-gen - when: locale_config.changed -- name: Configure unattended security updates without reboots - ansible.builtin.copy: - dest: /etc/apt/apt.conf.d/52devbox - mode: '0644' - content: | - APT::Periodic::Update-Package-Lists "1"; - APT::Periodic::Unattended-Upgrade "1"; - Unattended-Upgrade::Automatic-Reboot "false"; - Unattended-Upgrade::Origins-Pattern { - "origin=Ubuntu,codename=${distro_codename},label=Ubuntu-Security"; - "site=packages.mozilla.org"; - }; +- name: Configure the time zone, the locale and security updates + ansible.builtin.import_tasks: system.yml # Ubuntu only lets programs with an AppArmor profile create user namespaces. # Playwright's Chromium and its headless shell have none, so their sandbox # would refuse to start. @@ -255,24 +230,8 @@ groups: docker append: true when: devbox_install_docker -- name: Add Tailscale package signing key - ansible.builtin.get_url: - url: https://pkgs.tailscale.com/stable/ubuntu/resolute.noarmor.gpg - dest: /usr/share/keyrings/devbox-tailscale.gpg - mode: '0644' - when: devbox_install_tailscale -- name: Configure Tailscale repository - ansible.builtin.copy: - dest: /etc/apt/sources.list.d/devbox-tailscale.list - content: "deb [signed-by=/usr/share/keyrings/devbox-tailscale.gpg] https://pkgs.tailscale.com/stable/ubuntu resolute main\n" - mode: '0644' - register: tailscale_repo - when: devbox_install_tailscale - name: Install Tailscale - ansible.builtin.apt: - name: tailscale - state: present - update_cache: "{{ tailscale_repo.changed }}" + ansible.builtin.import_tasks: tailscale.yml when: devbox_install_tailscale - name: Enable machine services ansible.builtin.systemd_service: @@ -362,10 +321,12 @@ msg: >- {% if devbox_serve.rc == 5 %}The desktop and T3 are not published on the tailnet ({{ devbox_serve.stdout | trim }}). Reach them through an SSH tunnel, as the - access task of the UI's Manage action shows.{% else %}The desktop and T3 were - not published on the tailnet ({{ devbox_serve.stdout | default('') | trim }}). - Enroll the host and enable HTTPS certificates for the tailnet, then run the - publish task from the UI's Manage action or apply again.{% endif %} + host's access action shows.{% elif devbox_serve.rc == 3 %}The host is not on the + tailnet, so the desktop and T3 are not published on it. Reach them through an + SSH tunnel, as the host's access action shows; applying it from berth joins + it.{% else %}The desktop and T3 were not published on the tailnet + ({{ devbox_serve.stdout | default('') | trim }}). Enable HTTPS certificates for + the tailnet, then run the host's publish action or apply again.{% endif %} when: devbox_serve is not skipped and devbox_serve.rc | default(0) != 0 # The timer first fires five minutes after a boot, which has long passed during # an install; without this the self-signed certificate stays until a reboot. diff --git a/provision/ansible/roles/devbox/tasks/main.yml b/provision/ansible/roles/devbox/tasks/main.yml index b8d8a0f..3ae906f 100644 --- a/provision/ansible/roles/devbox/tasks/main.yml +++ b/provision/ansible/roles/devbox/tasks/main.yml @@ -1,82 +1,9 @@ --- -- name: Check target and profile before any changes - ansible.builtin.assert: - that: - - ansible_facts.distribution == 'Ubuntu' - - ansible_facts.distribution_version == '26.04' - - ansible_facts.architecture == 'x86_64' - - devbox_profile in ['laptop', 'cloud', 'container'] - - devbox_user is match('^[a-z_][a-z0-9_-]{0,31}$') - - devbox_home == '/home/' ~ devbox_user - - devbox_operator_user is match('^[a-z_][a-z0-9_-]{0,31}$') - - devbox_operator_user != devbox_user - - devbox_user not in devbox_reserved_names - - devbox_operator_user not in devbox_reserved_names - - devbox_agent_privileges in ['', 'full', 'limited', 'none'] - - devbox_agent_privileges | length > 0 or devbox_agent_passwordless_sudo or devbox_agent_password_hash | length > 0 - - devbox_agent_password_hash == '' or devbox_agent_password_hash is match('^\$[6y]\$') - - devbox_operator_password_hash == '' or devbox_operator_password_hash is match('^\$[6y]\$') - - devbox_berth_ssh_public_key.splitlines() | reject('match', '^(ssh-|ecdsa-|sk-)[^ ]+ [A-Za-z0-9+/=]+( .*)?$') | list | length == 0 - - devbox_geometry is match('^[0-9]{3,4}x[0-9]{3,4}$') - fail_msg: This version supports dedicated Ubuntu 26.04 amd64 hosts with distinct operator and agent accounts whose names Ubuntu does not already use. -- name: Install common packages - ansible.builtin.apt: - name: "{{ devbox_packages }}" - state: present - update_cache: true - cache_valid_time: 3600 - lock_timeout: 300 - environment: - DEBIAN_FRONTEND: noninteractive -# A program that cannot be installed does not stop the set-up: the rest are -# installed one at a time, and the next apply tries the missing ones again. -- name: Install further programs - when: devbox_extra_packages | length > 0 - environment: - DEBIAN_FRONTEND: noninteractive - block: - - name: Install further programs together - ansible.builtin.apt: - name: "{{ devbox_extra_packages }}" - state: present - lock_timeout: 300 - rescue: - - name: Install further programs one at a time - ansible.builtin.apt: - name: "{{ item }}" - state: present - lock_timeout: 300 - loop: "{{ devbox_extra_packages }}" - register: devbox_extra_installed - ignore_errors: true - - name: Name the programs that are missing - ansible.builtin.debug: - msg: >- - Not installed: {{ devbox_extra_installed.results | selectattr('failed') | map(attribute='item') | join(', ') }}. - The host is set up without them, and the next apply tries them again. -- name: Configure accounts and paths - ansible.builtin.import_tasks: accounts.yml -- name: Install application runtimes - ansible.builtin.import_tasks: toolchains.yml -- name: Install desktop applications and coding clients - ansible.builtin.import_tasks: applications.yml -- name: Apply selected plugins - ansible.builtin.import_tasks: plugins.yml -- name: Configure the shared desktop - ansible.builtin.import_tasks: desktop.yml -- name: Install berthd, box and what they read - ansible.builtin.import_tasks: berth.yml -- name: Configure full host services - ansible.builtin.include_tasks: host.yml - when: devbox_profile != 'container' -- name: Configure laptop integration - ansible.builtin.include_tasks: laptop.yml - when: devbox_profile == 'laptop' -- name: Record provisioned configuration (authentication is separate) - ansible.builtin.copy: - dest: /var/lib/devbox/status.json - mode: '0644' - content: "{{ {'state': 'configured-needs-enrollment', 'profile': devbox_profile, 'agents_installed': devbox_install_agents, 't3': devbox_t3_version, 'claude': devbox_claude_version, 'codex': devbox_codex_version, 'pi': devbox_pi_version, 'tool_categories': devbox_enabled_categories, 'plugin_features': devbox_plugin_features, 'agent_privileges': devbox_privileges, 'software_rendering': devbox_software_rendering, 'cockpit': devbox_install_cockpit and devbox_profile != 'container', 'ssh_hardened': devbox_ssh_hardened | default(false)} | to_nice_json }}\n" -# Last, so a run that failed leaves the host recorded as it was before. -- name: Record what this configuration came from - ansible.builtin.import_tasks: applied.yml +# A node is a server berth applies without a desktop: node.yml. Everything +# else of the role is a workstation's: workstation.yml. +- name: Set up a node + ansible.builtin.include_tasks: node.yml + when: devbox_level == 'node' +- name: Set up a workstation + ansible.builtin.include_tasks: workstation.yml + when: devbox_level != 'node' diff --git a/provision/ansible/roles/devbox/tasks/node.yml b/provision/ansible/roles/devbox/tasks/node.yml new file mode 100644 index 0000000..8044089 --- /dev/null +++ b/provision/ansible/roles/devbox/tasks/node.yml @@ -0,0 +1,43 @@ +--- +# A node: a server berth applies without a desktop. It gets the base system, +# updates, SSH that takes keys, berth's controller, a firewall and the plugin +# features its profile names, such as the engines that host machines. There +# are no agent or operator accounts: sessions run as the SSH user, and berth +# keeps logging in as that user with its own key. +- name: Check the node and find the account sessions run as + ansible.builtin.import_tasks: node_facts.yml +- name: Check that the node can join the tailnet + ansible.builtin.include_tasks: tailnet_check.yml + when: devbox_install_tailscale and devbox_start_services and devbox_profile != 'container' +- name: Install what a node needs + ansible.builtin.apt: + name: "{{ devbox_node_packages }}" + state: present + update_cache: true + cache_valid_time: 3600 + lock_timeout: 300 + environment: + DEBIAN_FRONTEND: noninteractive +- name: Configure the time zone, the locale and security updates + ansible.builtin.import_tasks: system.yml + +- name: Let berth in with its key, and nobody with a password + ansible.builtin.import_tasks: node_ssh.yml + +# ── The tailnet ───────────────────────────────────────────────────────────── + +- name: Install Tailscale + ansible.builtin.import_tasks: tailscale.yml + when: devbox_install_tailscale +- name: Start Tailscale now and at boot + ansible.builtin.systemd_service: + name: tailscaled + enabled: true + state: started + when: devbox_install_tailscale and devbox_start_services +- name: Join the tailnet + ansible.builtin.include_tasks: tailnet.yml + when: devbox_install_tailscale and devbox_start_services + +- name: Apply the features, berth's controller and the firewall + ansible.builtin.import_tasks: node_berth.yml diff --git a/provision/ansible/roles/devbox/tasks/node_berth.yml b/provision/ansible/roles/devbox/tasks/node_berth.yml new file mode 100644 index 0000000..3e16d57 --- /dev/null +++ b/provision/ansible/roles/devbox/tasks/node_berth.yml @@ -0,0 +1,90 @@ +--- +# A node's features, berth's controller with the node's actions, its +# firewall and the record of the apply, which says what the features offer. +- name: Apply the features of the node + ansible.builtin.import_tasks: plugins.yml +- name: See whether a berth server runs here in a container + ansible.builtin.command: docker container inspect --format {% raw %}'{{.Config.Image}}'{% endraw %} {{ devbox_berth_server_container | quote }} + register: devbox_node_server + changed_when: false + failed_when: false +- name: Describe the node's actions and units to berthd + ansible.builtin.set_fact: + # Rebooting asks systemd, and only a node with a berth server's container can upgrade it. + devbox_tasks: >- + {{ (devbox_node_tasks + | rejectattr('name', 'equalto', 'reboot' if ansible_facts.service_mgr != 'systemd' else '') + | rejectattr('name', 'equalto', 'upgrade-server' if devbox_node_server.rc != 0 else '') + | list) + devbox_plugin_tasks }} + devbox_berthd_units: "{{ ['ssh', 'berthd', 'ufw'] + (['tailscaled'] if devbox_install_tailscale else []) + devbox_node_units + devbox_service_units }}" + devbox_berthd_roots: ["{{ devbox_node_home }}"] + devbox_berthd_systemd: "{{ ansible_facts.service_mgr == 'systemd' }}" +- name: Install the helper that takes in secrets + ansible.builtin.copy: + src: secrets.py + dest: /usr/local/bin/devbox-secrets + mode: '0755' +- name: Install berthd, box and what they read + ansible.builtin.import_tasks: berth.yml +- name: Install the helper that upgrades a berth server's container + ansible.builtin.copy: + src: berth-server-upgrade.py + dest: /usr/local/lib/berth/berth-server-upgrade + mode: '0755' +- name: Say who starts berthd on a node without systemd + ansible.builtin.debug: + msg: This node has no systemd, so nothing here starts berthd by itself. Something on the host has to run /usr/local/bin/berthd, as the init of a container does. + when: not devbox_berthd_systemd | bool + +# ── The firewall ──────────────────────────────────────────────────────────── + +# Last, once the features have said what they open. The address this apply +# came from keeps SSH, so the run never locks berth out. +- name: Install the node's firewall helper + ansible.builtin.copy: + src: node-firewall.sh + dest: /usr/local/sbin/devbox-node-firewall + mode: '0755' + when: devbox_manage_firewall +# A machine taken in may let SSH in from some addresses only, by rules made by +# hand such as those of the incus-host recipe of 0.5. Those stay and are not +# widened: SSH stays open to them and to where this apply came from. +- name: Read the firewall's rules + ansible.builtin.command: ufw show added + register: devbox_node_ufw_added + changed_when: false + failed_when: false + when: devbox_manage_firewall +- name: Keep SSH as limited as it was + ansible.builtin.set_fact: + devbox_node_ssh_limited: >- + {{ devbox_ssh_from == 'any' + and (devbox_node_ufw_added.stdout_lines | default([]) | select('match', '^ufw allow from [^ ]+ to any port 22( proto tcp)?$') | list | length > 0) + and (devbox_node_ufw_added.stdout_lines | default([]) | select('match', '^ufw allow (in )?(22|22/tcp|ssh|OpenSSH)( |$)') | list | length == 0) }} + when: devbox_manage_firewall +- name: Say that SSH stays limited + ansible.builtin.debug: + msg: SSH stays open only to the addresses the firewall's own rules name, and to the address this apply came from. Delete those rules by hand to open it to every address. + when: devbox_manage_firewall and devbox_node_ssh_limited | bool +- name: Set the node's firewall + ansible.builtin.command: /usr/local/sbin/devbox-node-firewall + args: + stdin: "{{ devbox_node_firewall | join('\n') }}" + register: devbox_node_firewall_set + changed_when: devbox_node_firewall_set.stdout | length > 0 + when: devbox_manage_firewall + vars: + devbox_node_firewall: >- + {{ (([] if devbox_node_ssh_limited | bool else [('allow 22/tcp' if devbox_ssh_from == 'any' else 'allow from ' ~ devbox_ssh_from ~ ' to any port 22 proto tcp')]) + + (['allow from ' ~ devbox_apply_from ~ ' to any port 22 proto tcp'] if devbox_apply_from | length > 0 and (devbox_ssh_from != 'any' or devbox_node_ssh_limited | bool) else []) + + (['allow from ' ~ devbox_bootstrap_ssh_cidr ~ ' to any port 22 proto tcp'] if devbox_bootstrap_ssh_cidr | length > 0 else []) + + (['allow in on tailscale0 to any port 22 proto tcp', 'allow 41641/udp'] if devbox_install_tailscale else []) + + devbox_firewall_rules) | unique }} +- name: Say what the firewall changed + ansible.builtin.debug: + msg: "{{ devbox_node_firewall_set.stdout_lines }}" + when: devbox_node_firewall_set is changed + +# Last, so a run that failed leaves the host recorded as it was before. +- name: Record what this configuration came from + ansible.builtin.import_tasks: applied.yml diff --git a/provision/ansible/roles/devbox/tasks/node_facts.yml b/provision/ansible/roles/devbox/tasks/node_facts.yml new file mode 100644 index 0000000..6f24f93 --- /dev/null +++ b/provision/ansible/roles/devbox/tasks/node_facts.yml @@ -0,0 +1,44 @@ +--- +# What a node is before anything changes: the checks, the guard against a +# workstation berth set up, and the account sessions run as. +- name: Check the node before any changes + ansible.builtin.assert: + that: + - ansible_facts.distribution == 'Ubuntu' + - ansible_facts.distribution_version is version('24.04', '>=') + - ansible_facts.architecture == 'x86_64' + - devbox_profile in ['cloud', 'container'] + - devbox_node_user is match('^[a-z_][a-z0-9_-]{0,31}$') + - devbox_ssh_from == 'any' or devbox_ssh_from is match('^[0-9a-fA-F.:/]+$') + - devbox_berth_ssh_public_key.splitlines() | reject('match', '^(ssh-|ecdsa-|sk-)[^ ]+ [A-Za-z0-9+/=]+( .*)?$') | list | length == 0 + fail_msg: A node is an Ubuntu 24.04 or later server on x86_64, installed on a cloud machine or in a container, that berth reaches as an account of its own. + +- name: Read what berth installed on the host before + ansible.builtin.slurp: + src: /etc/berth/applied.json + register: devbox_node_found + failed_when: false +# A workstation's desktop, agent and accounts would stay behind, unlooked after. +- name: Refuse a host that berth set up as a workstation + ansible.builtin.fail: + msg: >- + {{ inventory_hostname }} is a workstation that berth set up. Applying it + as a node would leave its desktop, its agent and its accounts behind, so + nothing was changed. Keep it a workstation, or install the server again + and apply it as a node. + when: + - "'content' in devbox_node_found" + - (devbox_node_found.content | b64decode | from_json).level | default('workstation') == 'workstation' + +- name: Find the account berth logs in as + ansible.builtin.getent: + database: passwd + key: "{{ devbox_node_user }}" +# The role's other tasks and the features name the account sessions run as +# devbox_user and its home devbox_home. +- name: Describe the node to the role and to berthd + ansible.builtin.set_fact: + devbox_user: "{{ devbox_node_user }}" + devbox_operator_user: "{{ devbox_node_user }}" + devbox_home: "{{ ansible_facts.getent_passwd[devbox_node_user][4] }}" + devbox_node_home: "{{ ansible_facts.getent_passwd[devbox_node_user][4] }}" diff --git a/provision/ansible/roles/devbox/tasks/node_ssh.yml b/provision/ansible/roles/devbox/tasks/node_ssh.yml new file mode 100644 index 0000000..b5a8bac --- /dev/null +++ b/provision/ansible/roles/devbox/tasks/node_ssh.yml @@ -0,0 +1,87 @@ +--- +# A node's SSH: berth's key for the SSH user, then keys only. +- name: Read the SSH session this apply came in on + ansible.builtin.command: printenv SSH_CONNECTION + become: false + register: devbox_ssh_session + changed_when: false + failed_when: false +- name: Note where this apply came from + ansible.builtin.set_fact: + devbox_apply_from: "{{ (devbox_ssh_session.stdout | default('') | trim).split(' ')[0] | default('') }}" +- name: Make the SSH user's folder of keys + ansible.builtin.file: + path: "{{ devbox_node_home }}/.ssh" + state: directory + owner: "{{ devbox_node_user }}" + group: "{{ ansible_facts.getent_passwd[devbox_node_user][2] }}" + mode: '0700' +# As for a workstation's operator: a block of its own for each berth, named +# after its key, which no later apply removes. +- name: Give the SSH user the keys of the berth that applies this + ansible.builtin.blockinfile: + path: "{{ devbox_node_home }}/.ssh/authorized_keys" + create: true + owner: "{{ devbox_node_user }}" + group: "{{ ansible_facts.getent_passwd[devbox_node_user][2] }}" + mode: '0600' + marker: "# {mark} BERTH {{ devbox_berth_ssh_public_key.split()[1] | hash('sha1') }}" + block: "{{ devbox_berth_ssh_public_key }}" + when: devbox_berth_ssh_public_key | length > 0 +- name: Check that the SSH user takes the keys of the berth that applies this + ansible.builtin.command: + argv: [grep, -qxF, --, "{{ item }}", "{{ devbox_node_home }}/.ssh/authorized_keys"] + loop: "{{ devbox_berth_ssh_public_key.splitlines() }}" + changed_when: false +- name: See whether the SSH user has a key to log in with + ansible.builtin.command: grep -qE '^(ssh-|ecdsa-|sk-)' {{ (devbox_node_home ~ '/.ssh/authorized_keys') | quote }} + register: devbox_node_keys + changed_when: false + failed_when: false +# A machine taken in may let root in with keys already, as the one the berth +# server recipes of 0.5 set up does for the server in its container. What +# berth wrote before is kept as it is. +- name: See whether root logs in with keys here already + ansible.builtin.shell: | + if [ -f /etc/ssh/sshd_config.d/00-devbox.conf ]; then + awk 'tolower($1) == "permitrootlogin" {print $2}' /etc/ssh/sshd_config.d/00-devbox.conf + elif grep -qE '^(ssh-|ecdsa-|sk-)' /root/.ssh/authorized_keys 2>/dev/null; then + /usr/sbin/sshd -T 2>/dev/null | awk '$1 == "permitrootlogin" {print $2}' + else + echo no + fi + register: devbox_node_root_login + changed_when: false + failed_when: false +# Only keys log in. Root keeps its login with keys when berth logs in as root, +# so that this berth is never locked out, and when it had one before; any +# other SSH user leaves root none. It is never opened to passwords. +- name: Require SSH keys, and keep root's login as berth uses it + ansible.builtin.copy: + # sshd uses the first value it reads, so this has to sort before other drop-ins. + dest: /etc/ssh/sshd_config.d/00-devbox.conf + mode: '0644' + content: | + PasswordAuthentication no + KbdInteractiveAuthentication no + PermitRootLogin {{ 'prohibit-password' if devbox_node_user == 'root' or (devbox_node_root_login.stdout | default('no') | trim) in ['yes', 'prohibit-password', 'without-password'] else 'no' }} + validate: /usr/sbin/sshd -t -f %s + register: devbox_node_sshd + when: devbox_harden_ssh and devbox_node_keys.rc == 0 +- name: Remove SSH hardening when it is switched off + ansible.builtin.file: + path: /etc/ssh/sshd_config.d/00-devbox.conf + state: absent + register: devbox_node_sshd_relaxed + when: not (devbox_harden_ssh and devbox_node_keys.rc == 0) +- name: Explain skipped SSH hardening + ansible.builtin.debug: + msg: >- + SSH hardening skipped: {{ devbox_node_user }} has no SSH key to log in + with, so turning passwords off would lock this node. + when: devbox_harden_ssh and devbox_node_keys.rc != 0 +- name: Apply SSH server settings + ansible.builtin.systemd_service: + name: ssh + state: reloaded + when: devbox_start_services and (devbox_node_sshd.changed or devbox_node_sshd_relaxed.changed) diff --git a/provision/ansible/roles/devbox/tasks/plugin.yml b/provision/ansible/roles/devbox/tasks/plugin.yml index 8513d8e..b9f5512 100644 --- a/provision/ansible/roles/devbox/tasks/plugin.yml +++ b/provision/ansible/roles/devbox/tasks/plugin.yml @@ -22,6 +22,15 @@ - name: Take the feature's description ansible.builtin.set_fact: devbox_feature: "{{ devbox_plugin_matches[0] }}" + # Its tasks set this to the provisioners they found, once they are sure of them. + devbox_offer: {} +# A feature is for a workstation unless its manifest names other types. +- name: Require the feature to be for this type of host + ansible.builtin.assert: + that: ('node' if devbox_level == 'node' else 'workstation') in (devbox_feature.types | default(['workstation'])) + fail_msg: >- + {{ devbox_plugin_feature }} is for {{ (devbox_feature.types | default(['workstation'])) | join(' or ') }} hosts, + not for a {{ 'node' if devbox_level == 'node' else 'workstation' }}. Take it out of the profile's features. - name: Install the feature's packages ansible.builtin.apt: name: "{{ devbox_feature.packages }}" @@ -31,6 +40,17 @@ - name: Run the feature's tasks ansible.builtin.include_tasks: "{{ devbox_plugin_dir }}/{{ devbox_feature.tasks }}" when: devbox_feature.tasks | default('') | length > 0 +- name: Record the provisioners the feature offers + ansible.builtin.set_fact: + devbox_plugin_offers: >- + {{ devbox_plugin_offers + [{ + 'feature': devbox_plugin_feature, + 'provisioners': devbox_offer.provisioners | default(devbox_feature.offers) | intersect(devbox_feature.offers), + 'settings': devbox_offer.settings | default({}), + 'secrets': devbox_offer.secrets | default({}), + 'aliases': devbox_offer.aliases | default({}), + 'fingerprint': devbox_offer.fingerprint | default('')}] }} + when: devbox_feature.offers | default([]) | length > 0 and devbox_offer | length > 0 - name: Install the feature's guide for agents ansible.builtin.template: src: "{{ devbox_plugin_dir }}/{{ devbox_feature.guide }}" diff --git a/provision/ansible/roles/devbox/tasks/plugins.yml b/provision/ansible/roles/devbox/tasks/plugins.yml index 19fb39f..f86cf29 100644 --- a/provision/ansible/roles/devbox/tasks/plugins.yml +++ b/provision/ansible/roles/devbox/tasks/plugins.yml @@ -6,6 +6,11 @@ devbox_plugin_tasks: [] devbox_plugin_services: [] devbox_plugin_channels: [] + # What a node's features add: the provisioners they offer, the rules of + # its firewall and the units whose state berthd reports. + devbox_plugin_offers: [] + devbox_firewall_rules: [] + devbox_node_units: [] - name: Create the directory for detailed guides ansible.builtin.file: path: /etc/devbox/guides diff --git a/provision/ansible/roles/devbox/tasks/system.yml b/provision/ansible/roles/devbox/tasks/system.yml new file mode 100644 index 0000000..7f8b19e --- /dev/null +++ b/provision/ansible/roles/devbox/tasks/system.yml @@ -0,0 +1,30 @@ +--- +# What every host berth applies gets: its time zone, an English locale and +# Ubuntu's security updates without reboots. A workstation and a node share it. +- name: Configure timezone + ansible.builtin.file: + src: "/usr/share/zoneinfo/{{ devbox_timezone }}" + dest: /etc/localtime + state: link + force: true +- name: Enable English locale + ansible.builtin.lineinfile: + path: /etc/locale.gen + regexp: '^#?\s*en_US.UTF-8 UTF-8' + line: en_US.UTF-8 UTF-8 + register: locale_config +- name: Generate locale + ansible.builtin.command: locale-gen + when: locale_config.changed +- name: Configure unattended security updates without reboots + ansible.builtin.copy: + dest: /etc/apt/apt.conf.d/52devbox + mode: '0644' + content: | + APT::Periodic::Update-Package-Lists "1"; + APT::Periodic::Unattended-Upgrade "1"; + Unattended-Upgrade::Automatic-Reboot "false"; + Unattended-Upgrade::Origins-Pattern { + "origin=Ubuntu,codename=${distro_codename},label=Ubuntu-Security"; + "site=packages.mozilla.org"; + }; diff --git a/provision/ansible/roles/devbox/tasks/tailnet.yml b/provision/ansible/roles/devbox/tasks/tailnet.yml index c8728fc..75ba579 100644 --- a/provision/ansible/roles/devbox/tasks/tailnet.yml +++ b/provision/ansible/roles/devbox/tasks/tailnet.yml @@ -1,16 +1,10 @@ --- -# Join the tailnet once. A host that is on it already is left alone, so -# applying again needs no key. The key is pasted into the profile, or, with -# devbox_tailscale_key_command, created by berth on the machine that runs -# this play, only when the host is about to use it. -- name: Inspect Tailscale enrollment - ansible.builtin.command: tailscale status --json - register: tailscale_status - changed_when: false - failed_when: false -- name: Tell whether the host is on the tailnet - ansible.builtin.set_fact: - devbox_tailnet_enrolled: "{{ (tailscale_status.stdout | default('{}', true) | from_json).get('BackendState', '') == 'Running' }}" +# Join the tailnet once, with a one-use key berth made: the one of a seed, +# or, with devbox_tailscale_key_command, one berth makes on the machine that +# runs this play when the host is about to use it. A host on the tailnet +# already is left alone, so applying again needs no key and never logs it out. +- name: Check that the host can join the tailnet + ansible.builtin.include_tasks: tailnet_check.yml - name: Create a one-use enrollment key with berth ansible.builtin.command: argv: "{{ devbox_tailscale_key_command }}" @@ -22,7 +16,6 @@ no_log: true when: - not devbox_tailnet_enrolled | bool - - devbox_tailscale_auth_key | length == 0 - devbox_tailscale_key_command | length > 0 # Kept out of the task above, which hides its output for the key's sake. - name: Explain why berth made no enrollment key @@ -31,7 +24,7 @@ when: devbox_tailscale_made_key.rc | default(0) != 0 - name: Choose the enrollment key ansible.builtin.set_fact: - devbox_tailnet_key: "{{ devbox_tailscale_auth_key if devbox_tailscale_auth_key | length > 0 else (devbox_tailscale_made_key.stdout | default('') | trim) }}" + devbox_tailnet_key: "{{ (devbox_tailscale_made_key.stdout | default('') | trim) if devbox_tailscale_key_command | length > 0 else devbox_tailscale_auth_key }}" no_log: true - name: Enroll Tailscale only when not already enrolled when: not devbox_tailnet_enrolled | bool and devbox_tailnet_key | length > 0 @@ -42,10 +35,21 @@ content: "{{ devbox_tailnet_key }}" mode: '0600' no_log: true + # The key of a seed lasts a day. A machine first started later is set up + # without the tailnet, and berth's next apply joins it with a new key. - name: Consume enrollment key ansible.builtin.command: argv: "{{ ['tailscale', 'up', '--timeout=3m', '--auth-key=file:/run/devbox-tailscale-key'] + (['--login-server=' ~ devbox_tailscale_login_server] if devbox_tailscale_login_server | length > 0 else []) }}" - changed_when: true + register: devbox_tailnet_up + changed_when: devbox_tailnet_up.rc == 0 + failed_when: devbox_tailnet_up.rc != 0 and devbox_tailscale_key_command | length > 0 + - name: Explain why the host is not on the tailnet + ansible.builtin.debug: + msg: >- + The host did not join the tailnet with the key of its seed, which lasts 24 hours + ({{ devbox_tailnet_up.stderr | default('') | trim }}). It is reached over SSH at its address; + apply it from berth to join it with a new key. + when: devbox_tailnet_up.rc != 0 always: - name: Remove temporary enrollment credential ansible.builtin.file: diff --git a/provision/ansible/roles/devbox/tasks/tailnet_check.yml b/provision/ansible/roles/devbox/tasks/tailnet_check.yml new file mode 100644 index 0000000..66ab22d --- /dev/null +++ b/provision/ansible/roles/devbox/tasks/tailnet_check.yml @@ -0,0 +1,37 @@ +--- +# A host that joins the tailnet and is not on it needs a one-use key: the +# one of its seed, or berth's command for one. Without either the play stops +# here, before it changes anything, with what berth said is missing. A host +# on the tailnet already needs no key, and one with devbox_tailnet_optional +# goes on without the tailnet, as berth 0.5 did. +- name: Inspect Tailscale enrollment + ansible.builtin.command: tailscale status --json + register: devbox_tailnet_status + changed_when: false + failed_when: false + check_mode: false +- name: Tell whether the host is on the tailnet + ansible.builtin.set_fact: + devbox_tailnet_enrolled: "{{ devbox_tailnet_status.stdout | default('', true) is search('\"BackendState\":\\s*\"Running\"') }}" +- name: Stop when the host cannot join the tailnet + ansible.builtin.fail: + msg: >- + {{ devbox_tailnet_problem if devbox_tailnet_problem | length > 0 else + 'This host joins the tailnet, and nothing gave it a key to join with. Apply it with berth, + which makes one from the secret tailnet_api_key, or set devbox_install_tailscale to false.' }} + when: + - not devbox_tailnet_enrolled | bool + - devbox_tailscale_auth_key | length == 0 + - devbox_tailscale_key_command | length == 0 + - not devbox_tailnet_optional | default(false) | bool +- name: Say that the host stays off the tailnet + ansible.builtin.debug: + msg: >- + This host is not on the tailnet, and berth cannot make its key yet + ({{ devbox_tailnet_problem if devbox_tailnet_problem | length > 0 else 'nothing gave it one' }}). + The apply goes on without the tailnet, and berth reaches the host over SSH at its address. + when: + - not devbox_tailnet_enrolled | bool + - devbox_tailscale_auth_key | length == 0 + - devbox_tailscale_key_command | length == 0 + - devbox_tailnet_optional | default(false) | bool diff --git a/provision/ansible/roles/devbox/tasks/tailscale.yml b/provision/ansible/roles/devbox/tasks/tailscale.yml new file mode 100644 index 0000000..00a624b --- /dev/null +++ b/provision/ansible/roles/devbox/tasks/tailscale.yml @@ -0,0 +1,19 @@ +--- +# Tailscale from its own repository, for the release the host runs. Joining +# is tailnet.yml's. +- name: Add Tailscale package signing key + ansible.builtin.get_url: + url: https://pkgs.tailscale.com/stable/ubuntu/{{ ansible_facts.distribution_release }}.noarmor.gpg + dest: /usr/share/keyrings/devbox-tailscale.gpg + mode: '0644' +- name: Configure Tailscale repository + ansible.builtin.copy: + dest: /etc/apt/sources.list.d/devbox-tailscale.list + content: "deb [signed-by=/usr/share/keyrings/devbox-tailscale.gpg] https://pkgs.tailscale.com/stable/ubuntu {{ ansible_facts.distribution_release }} main\n" + mode: '0644' + register: tailscale_repo +- name: Install Tailscale + ansible.builtin.apt: + name: tailscale + state: present + update_cache: "{{ tailscale_repo.changed }}" diff --git a/provision/ansible/roles/devbox/tasks/workstation.yml b/provision/ansible/roles/devbox/tasks/workstation.yml new file mode 100644 index 0000000..24c1dc0 --- /dev/null +++ b/provision/ansible/roles/devbox/tasks/workstation.yml @@ -0,0 +1,85 @@ +--- +- name: Check target and profile before any changes + ansible.builtin.assert: + that: + - ansible_facts.distribution == 'Ubuntu' + - ansible_facts.distribution_version == '26.04' + - ansible_facts.architecture == 'x86_64' + - devbox_profile in ['laptop', 'cloud', 'container'] + - devbox_user is match('^[a-z_][a-z0-9_-]{0,31}$') + - devbox_home == '/home/' ~ devbox_user + - devbox_operator_user is match('^[a-z_][a-z0-9_-]{0,31}$') + - devbox_operator_user != devbox_user + - devbox_user not in devbox_reserved_names + - devbox_operator_user not in devbox_reserved_names + - devbox_agent_privileges in ['', 'full', 'limited', 'none'] + - devbox_agent_privileges | length > 0 or devbox_agent_passwordless_sudo or devbox_agent_password_hash | length > 0 + - devbox_agent_password_hash == '' or devbox_agent_password_hash is match('^\$[6y]\$') + - devbox_operator_password_hash == '' or devbox_operator_password_hash is match('^\$[6y]\$') + - devbox_berth_ssh_public_key.splitlines() | reject('match', '^(ssh-|ecdsa-|sk-)[^ ]+ [A-Za-z0-9+/=]+( .*)?$') | list | length == 0 + - devbox_geometry is match('^[0-9]{3,4}x[0-9]{3,4}$') + fail_msg: This version supports dedicated Ubuntu 26.04 amd64 hosts with distinct operator and agent accounts whose names Ubuntu does not already use. +- name: Check that the host can join the tailnet + ansible.builtin.include_tasks: tailnet_check.yml + when: devbox_install_tailscale and devbox_start_services and devbox_profile != 'container' +- name: Install common packages + ansible.builtin.apt: + name: "{{ devbox_packages }}" + state: present + update_cache: true + cache_valid_time: 3600 + lock_timeout: 300 + environment: + DEBIAN_FRONTEND: noninteractive +# A program that cannot be installed does not stop the set-up: the rest are +# installed one at a time, and the next apply tries the missing ones again. +- name: Install further programs + when: devbox_extra_packages | length > 0 + environment: + DEBIAN_FRONTEND: noninteractive + block: + - name: Install further programs together + ansible.builtin.apt: + name: "{{ devbox_extra_packages }}" + state: present + lock_timeout: 300 + rescue: + - name: Install further programs one at a time + ansible.builtin.apt: + name: "{{ item }}" + state: present + lock_timeout: 300 + loop: "{{ devbox_extra_packages }}" + register: devbox_extra_installed + ignore_errors: true + - name: Name the programs that are missing + ansible.builtin.debug: + msg: >- + Not installed: {{ devbox_extra_installed.results | selectattr('failed') | map(attribute='item') | join(', ') }}. + The host is set up without them, and the next apply tries them again. +- name: Configure accounts and paths + ansible.builtin.import_tasks: accounts.yml +- name: Install application runtimes + ansible.builtin.import_tasks: toolchains.yml +- name: Install desktop applications and coding clients + ansible.builtin.import_tasks: applications.yml +- name: Apply selected plugins + ansible.builtin.import_tasks: plugins.yml +- name: Configure the shared desktop + ansible.builtin.import_tasks: desktop.yml +- name: Install berthd, box and what they read + ansible.builtin.import_tasks: berth.yml +- name: Configure full host services + ansible.builtin.include_tasks: host.yml + when: devbox_profile != 'container' +- name: Configure laptop integration + ansible.builtin.include_tasks: laptop.yml + when: devbox_profile == 'laptop' +- name: Record provisioned configuration (authentication is separate) + ansible.builtin.copy: + dest: /var/lib/devbox/status.json + mode: '0644' + content: "{{ {'state': 'configured', 'profile': devbox_profile, 'agents_installed': devbox_install_agents, 't3': devbox_t3_version, 'claude': devbox_claude_version, 'codex': devbox_codex_version, 'pi': devbox_pi_version, 'tool_categories': devbox_enabled_categories, 'plugin_features': devbox_plugin_features, 'agent_privileges': devbox_privileges, 'software_rendering': devbox_software_rendering, 'cockpit': devbox_install_cockpit and devbox_profile != 'container', 'ssh_hardened': devbox_ssh_hardened | default(false)} | to_nice_json }}\n" +# Last, so a run that failed leaves the host recorded as it was before. +- name: Record what this configuration came from + ansible.builtin.import_tasks: applied.yml diff --git a/provision/ansible/roles/devbox/templates/AGENT.md.j2 b/provision/ansible/roles/devbox/templates/AGENT.md.j2 index 2db3072..362e251 100644 --- a/provision/ansible/roles/devbox/templates/AGENT.md.j2 +++ b/provision/ansible/roles/devbox/templates/AGENT.md.j2 @@ -41,10 +41,10 @@ resolved versions and `/etc/devbox/expected-commands` every command this machine is expected to have. `yq` is the Python/jq-based Ubuntu package, not mikefarah/yq. Not listed here means not installed; install what a task needs. -## Tasks +## Actions -`box tasks` lists the named operations of this machine you may run, such as the -self-test; `box run TASK` runs one and shows its output. `box --help` has the +`box actions` lists the named operations of this machine you may run, such as +the self-test; `box run ACTION` runs one and shows its output. `box --help` has the rest: publishing and reading messages on the machine's channel, and calling the services plugins provide. diff --git a/provision/ansible/roles/devbox/vars/main.yml b/provision/ansible/roles/devbox/vars/main.yml index 88e905b..0dcaf87 100644 --- a/provision/ansible/roles/devbox/vars/main.yml +++ b/provision/ansible/roles/devbox/vars/main.yml @@ -1,6 +1,6 @@ --- -# berth's own tasks, in the order the overview shows them. berthd reads them -# from /etc/berth/tasks.json, where the actions of the plugins follow. +# berth's own actions, in the order the overview shows them. berthd reads them +# from /etc/berth/actions.json, where the actions of the plugins follow. devbox_task_dir: /usr/local/lib/berth/tasks devbox_core_tasks: - name: status @@ -61,7 +61,7 @@ devbox_core_tasks: agent: true help: "The accounts the tools on this host are signed in to for the agent: the coding agents, GitHub and whatever else the plugins of this host know about." run: "true" -# The tasks of a host with the controller alone, which assume nothing of a +# The actions of a host with the controller alone, which assume nothing of a # desktop, its accounts or plugins. devbox_controller_tasks: - name: status @@ -90,4 +90,46 @@ devbox_controller_tasks: input: secrets help: "Send this host's secrets and the logins of its registries to the account sessions run as: tokens in ~/.config/devbox/secrets.env or as files, npm and Docker logins. Replaces what was installed before." run: /usr/local/bin/devbox-secrets +# The actions of a node, which has berth's controller and plugin features but +# no desktop or accounts of its own. The role leaves out upgrade-server where +# it finds no berth server's container. +devbox_node_tasks: + - name: status + label: Status + group: System + user: root + help: The node, how long it has been up, the state of SSH, berthd, the firewall and the tailnet, disk space and failed units, followed by what its features report. + run: "{{ devbox_task_dir }}/status.sh node" + - name: update + label: Update + group: System + user: root + help: apt update and dist-upgrade on the node, then what its features keep up to date. Says whether a reboot is needed afterwards. + run: "{{ devbox_task_dir }}/update.sh" + - name: reboot + label: Reboot + group: System + user: root + confirm: Restart the node? Everything running on it ends, sessions and the machines it hosts included. + help: Restart the node. + run: "{{ devbox_task_dir }}/reboot.sh" + - name: secrets + label: Secrets and registry logins + group: Sync + user: agent + input: secrets + help: "Send this host's secrets and the logins of its registries to the account sessions run as: tokens in ~/.config/devbox/secrets.env or as files, npm and Docker logins. Replaces what was installed before." + run: /usr/local/bin/devbox-secrets + - name: upgrade-server + label: Upgrade the berth server + group: Server + user: root + confirm: Run the berth server in its container from the image of another version? It restarts, and its jobs end. + help: Run the berth server in the container on this node from the image of the version given, with the same data and the same options as the running container. It is refused when the container has an option it cannot carry over, and the old container is started again when the new one does not come up. + arguments: + - name: version + label: Version + required: true + help: The version of berth to run, as its image is tagged, such as 0.6.0. + run: "/usr/local/lib/berth/berth-server-upgrade {{ devbox_berth_server_container | quote }}" devbox_service_dir: /usr/local/lib/berth/services diff --git a/provision/bin/devbox.py b/provision/bin/devbox.py index 9cd928c..13b17ee 100755 --- a/provision/bin/devbox.py +++ b/provision/bin/devbox.py @@ -32,7 +32,7 @@ 'crates/berth/Cargo.toml', 'crates/berth/build.rs', 'crates/berth/src', 'crates/vendor/tui-input', 'crates/vendor/README.md', ] -PROFILES = {'laptop', 'cloud'} +TARGETS = {'laptop', 'cloud'} # Where a QEMU guest on the user-mode network sees the machine it runs on. QEMU_HOST = '10.0.2.2/32' DEFAULT_OPERATOR = 'ops' @@ -144,13 +144,53 @@ def release_delivered(): return os.environ.get('BERTH_RELEASE_DELIVERED') == '1' +def take_old_names(c): + """A profile of berth 0.5 or earlier with the names its fields have now: + `type` for `level`, whose values were reachable, controller and + workstation, and `target` for `profile`. What a later berth wrote stays.""" + c = dict(c) + level = c.pop('level', None) + if level is not None and 'type' not in c: + host_type = {'reachable': 'ssh', 'controller': 'managed', 'workstation': None}.get(level, level) + if host_type is not None: + c['type'] = host_type + if isinstance(c.get('profile'), str): + c.setdefault('target', c.pop('profile')) + c.pop('profile', None) + # berth makes a host's key to join the tailnet and gives it to the seed + # itself; what older berths kept of joining in other ways is not read. + for key in ('tailscale_auth_key', 'tailscale_auto_key', 'tailscale_login_server'): + c.pop(key, None) + return c + + +# The role's variables of joining the tailnet, which berth sets and a profile cannot. +TAILNET_VARS = ('devbox_tailscale_auth_key', 'devbox_tailscale_login_server', + 'devbox_tailscale_key_command', 'devbox_tailnet_problem') + + +def tailnet_given(): + """The key berth made for this seed to join the tailnet with, and the login + server, from the private file named in BERTH_TAILNET_FILE; empty without one.""" + path = os.environ.get('BERTH_TAILNET_FILE', '') + if not path: + return {} + given = json.loads(Path(path).read_text()) + key, server = given.get('key'), given.get('login_server') or '' + if not isinstance(key, str) or not key or any(ch in key for ch in '\n\r "\''): + raise ValueError('BERTH_TAILNET_FILE holds no usable key') + if server and not re.fullmatch(r'https?://[^\s/]+(/[^\s]*)?', server): + raise ValueError('The login server of BERTH_TAILNET_FILE must be an http:// or https:// address') + return {'key': key, 'login_server': server} + + def retire_qemu(c): - """A profile of the qemu kind there once was, as the cloud one it is now. - That kind let SSH in only from the QEMU host when no recovery network was + """A profile of the qemu target there once was, as the cloud one it is now. + That target let SSH in only from the QEMU host when no recovery network was named, so it gets that network rather than SSH open to every address.""" - if c.get('profile') != 'qemu': + if c.get('target') != 'qemu': return c - c = dict(c, profile='cloud') + c = dict(c, target='cloud') variables = dict(c.get('ansible_vars') or {}) if not variables.get('devbox_bootstrap_ssh_cidr'): variables['devbox_bootstrap_ssh_cidr'] = QEMU_HOST @@ -163,11 +203,11 @@ def checked_config(path, kind): c = json.loads(Path(path).read_text()) if not isinstance(c, dict): raise ValueError('Configuration must be a JSON object') - c = retire_qemu(c) + c = retire_qemu(take_old_names(c)) if not HOSTNAME.fullmatch(text(c, 'hostname')): raise ValueError('Invalid hostname: use lowercase letters, digits and hyphens, starting with a letter') - if text(c, 'profile') not in PROFILES: - raise ValueError('profile must be laptop or cloud') + if text(c, 'target') not in TARGETS: + raise ValueError('target must be laptop or cloud') operator = text(c, 'operator_username', DEFAULT_OPERATOR) agent = text(c, 'agent_username', 'agent') @@ -199,15 +239,14 @@ def checked_config(path, kind): variables = c.setdefault('ansible_vars', {}) if not isinstance(variables, dict) or any(not k.startswith('devbox_') for k in variables): raise ValueError('ansible_vars must contain only devbox_ settings') - text(c, 'tailscale_auth_key') - login_server = text(c, 'tailscale_login_server') - if login_server and not re.fullmatch(r'https?://[^\s/]+(/[^\s]*)?', login_server): - raise ValueError('tailscale_login_server must be an http:// or https:// address') - # berth puts a new key into the profile it renders; without berth there is none. - if flag(c, 'tailscale_auto_key', False) and not c.get('tailscale_auth_key') \ - and variables.get('devbox_install_tailscale', True): - raise ValueError('tailscale_auto_key needs berth to create the key: render the seed with berth, ' - 'or paste a key into tailscale_auth_key') + for name in TAILNET_VARS: + variables.pop(name, None) + # berth makes the key when it renders a seed; without berth there is none, + # and a host would come up off the tailnet it was meant to be on. + if variables.get('devbox_install_tailscale', True) and not tailnet_given(): + raise ValueError('The host joins the tailnet, and the seed has no key for it: render the seed with berth, ' + 'which makes one from the secret tailnet_api_key, or set ' + 'ansible_vars.devbox_install_tailscale to false') if kind == 'usb': c['target_disk_mode'] = disk_guard.policy(c)['mode'] @@ -298,14 +337,15 @@ def deployment(c, sha): """Everything the first-boot provisioner needs, read by bootstrap.sh on the target.""" variables = dict(c.get('ansible_vars') or {}) variables.update(account_vars(c)) - if c.get('tailscale_auth_key'): - variables['devbox_tailscale_auth_key'] = c['tailscale_auth_key'] - if c.get('tailscale_login_server'): - variables['devbox_tailscale_login_server'] = c['tailscale_login_server'] + tailnet = tailnet_given() if variables.get('devbox_install_tailscale', True) else {} + if tailnet: + variables['devbox_tailscale_auth_key'] = tailnet['key'] + if tailnet.get('login_server'): + variables['devbox_tailscale_login_server'] = tailnet['login_server'] # berth hands over the profile as the host keeps it, and the role writes it. if isinstance(c.get('berth_profile'), dict): variables['devbox_berth_profile'] = c['berth_profile'] - return {'profile': c['profile'], 'release_sha256': sha, + return {'target': c['target'], 'release_sha256': sha, 'release_url': '' if release_delivered() else c.get('release_url', ''), 'ansible_vars': variables} diff --git a/provision/bootstrap/bootstrap.sh b/provision/bootstrap/bootstrap.sh index 5a6571a..8805cb3 100755 --- a/provision/bootstrap/bootstrap.sh +++ b/provision/bootstrap/bootstrap.sh @@ -90,7 +90,8 @@ import json, os from pathlib import Path d=json.loads(Path('/opt/devbox-bootstrap/deployment.json').read_text()) p=Path('/etc/devbox'); p.mkdir(parents=True,exist_ok=True) -v=d['ansible_vars']; v['devbox_profile']=d['profile'] +# The role names the target by its older name, as did the seeds of berth 0.5. +v=d['ansible_vars']; v['devbox_profile']=d.get('target', d.get('profile')) # A seed of the qemu kind there once was is a cloud host that SSH reaches only from the QEMU host. if v['devbox_profile']=='qemu': v['devbox_profile']='cloud' @@ -107,10 +108,10 @@ step 'Applying the role' ln -sfn "$release" /opt/devbox/current printf '%s\n' "$digest" > "$state/ready" chmod 0644 "$state/ready" -set_state configured-needs-enrollment +set_state configured step 'The role is applied' -# Successful enrollment consumes the temporary key. The original USB/cloud user data -# may still contain it; use short-lived one-use keys, never a reusable credential. +# Successful enrollment consumes the key. The original USB/cloud user data may +# still contain it, which is why berth makes it for one use and a day. python3 - <<'PY' import json from pathlib import Path diff --git a/provision/config/example.json b/provision/config/example.json index e075f9e..ff4176a 100644 --- a/provision/config/example.json +++ b/provision/config/example.json @@ -1,6 +1,6 @@ { "hostname": "", - "profile": "laptop", + "target": "laptop", "operator_username": "ops", "operator_ssh_public_keys": [], "operator_password_hash": "", @@ -14,7 +14,6 @@ "target_disk_serial": "", "keyboard": "de", "release_url": "", - "tailscale_auth_key": "", "wifi_enabled": false, "wifi": null, "ansible_vars": {}, diff --git a/recipes/README.md b/recipes/README.md index 6813f8b..8f432d8 100644 --- a/recipes/README.md +++ b/recipes/README.md @@ -6,24 +6,28 @@ recipe with its own guide and a script that runs it. | Recipe | What it sets up | | --- | --- | | [`rescue-install`](rescue-install/) | Ubuntu on a server from its provider's rescue system, on two disks as a RAID 1. For when the provider's installer does not work. | -| [`incus-host`](incus-host/) | Turns an empty server into a host for machines, with Incus, Docker and a firewall. berth creates its hosts there as guests. | -| [`berth-server`](berth-server/) | Turns a small machine into a berth server, in a Docker container and on your tailnet with TLS. | +| [`berth-server`](berth-server/) | Turns a machine into a berth server, in a Docker container and on your tailnet with TLS. With `berth_server_node`, the server can then apply its own machine as a node. | A recipe runs without berth. The playbooks use only the modules that come with `ansible-core`, and share the tasks in `shared/`. +A server that hosts Incus and Docker machines is not a recipe: berth applies +it as a [node](../docs/content/docs/%28guide%29/use/nodes.mdx), whose features +install and update the engines and register their provisioners. A server the +earlier `incus-host` recipe set up is taken in as a node in the same way. + The recipes that set up a berth server end with a setup key. With it the -first user collects the server's token, with `berth connect URL` or in a -browser, so the token never has to be read out or passed on. +first user collects the server's token, with `berth login URL`, so the token never has to be read out or passed on. ## Get the recipes -A recipe needs only this folder. This fetches it into `recipes/` in the -current directory: +A recipe needs this folder, and `berth-server` also the helper that updates +the server's container. This fetches them into the current directory: ```bash curl -fsSL https://github.com/helightdev/berth/archive/refs/heads/main.tar.gz \ - | tar xz --strip-components=1 --wildcards '*/recipes' + | tar xz --strip-components=1 --wildcards '*/recipes' \ + '*/provision/ansible/roles/devbox/files/berth-server-upgrade.py' ``` The recipes share the tasks in `shared/`, so fetch the folder as a whole. diff --git a/recipes/berth-server/README.md b/recipes/berth-server/README.md index f866544..9d95d92 100644 --- a/recipes/berth-server/README.md +++ b/recipes/berth-server/README.md @@ -25,7 +25,8 @@ On the machine itself: ```bash curl -fsSL https://github.com/helightdev/berth/archive/refs/heads/main.tar.gz \ - | tar xz --strip-components=1 --wildcards '*/recipes' + | tar xz --strip-components=1 --wildcards '*/recipes' \ + '*/provision/ansible/roles/devbox/files/berth-server-upgrade.py' cd recipes/berth-server sudo ./install.sh -e tailscale_auth_key=tskey-auth-EXAMPLE ``` @@ -39,44 +40,79 @@ From another machine that has Ansible: ansible-playbook -i 192.0.2.10, -u root site.yml -e tailscale_auth_key=tskey-auth-EXAMPLE ``` -A second run changes nothing that is already in place. It leaves a container -named `berth` as it is, and starts it when it was stopped. +A second run changes nothing that is already in place. A container named +`berth` keeps its image, data and options. A stopped one is started. It is made again only when it lacks one of the recipe's +options, such as the port or a folder in `/srv/berth`. ## Use it The first run makes a setup key and prints it at the end, with the server's address. The key lets the first user collect the server's token. It lasts 24 -hours and ends when that user chooses Done. Whoever runs the recipe can hand on +hours and ends once `berth login` has collected the token. Whoever runs the recipe can hand on the key and never sees the token. On your own machine, give berth the key when it asks: ```bash -berth connect https://THIS-MACHINE.YOUR-TAILNET.ts.net +berth login https://THIS-MACHINE.YOUR-TAILNET.ts.net ``` -Or open the server's page with the key in the link: +Or open berth's screens with the key in `BERTH_SETUP_KEY`, on Linux, macOS +or Windows: -```text -https://THIS-MACHINE.YOUR-TAILNET.ts.net/#setup-key=KEY +```bash +read -rs BERTH_SETUP_KEY && export BERTH_SETUP_KEY +berth --server https://THIS-MACHINE.YOUR-TAILNET.ts.net ``` -The page signs the browser in and shows the token. `tailscale serve status` on +The screens sign in and show the token. `tailscale serve status` on the machine says the address. Make a new key on the machine at any time: ```bash -sudo docker exec berth berth setup-key +sudo docker exec berth berth server setup-key +``` + +The token itself gives full control of every host. Without a key, pass it to +`berth login` on standard input, on the machine itself: + +```bash +sudo cat /srv/berth/data/berth/serve.token | berth login https://THIS-MACHINE.YOUR-TAILNET.ts.net --token-stdin +``` + +## The server on a node + +A large server can run the berth server and host the machines too. Install +Ubuntu, run the recipe with `berth_server_node`, and the server can apply its +own machine as a [node](../../docs/content/docs/%28guide%29/use/nodes.mdx): + +```bash +sudo ./install.sh -e berth_server_node=true -e tailscale_auth_key=tskey-auth-EXAMPLE ``` -The token itself gives full control of every host. Without a key, read it on -the machine and set it by hand: +| Part | Set-up | +| --- | --- | +| Login | The server's own key logs in as `root` here | +| Name | The server finds this machine at the name `node` | +| Docker | The container gets the socket of this machine's engine, and joins its group | + +Then, on a berth connected to the server, add the machine as a node, list +its features and apply it. This example calls it `big`: ```bash -sudo cat /srv/berth/data/berth/serve.token -berth config server https://THIS-MACHINE.YOUR-TAILNET.ts.net -berth config server_token +berth host add big node --user root --type node \ + --feature docker/engine --feature incus/server +berth apply big ``` +The apply installs Incus and Docker, and the server registers their +provisioners, which reach them on this machine. Docker's socket and the root +login give root on this machine, so whoever has the server's token controls +the whole machine. A container from an earlier run keeps its data and its +own options and gets these: the run makes it again with both. + +Once the machine is a node, its action `upgrade-server` updates the server +in place. See [Update it](#update-it). + ## Settings The settings are in [`vars.yml`](vars.yml), each with a comment. Change them @@ -88,6 +124,8 @@ there, or keep your own file and pass it with `-e @my-vars.yml`. | `berth_server_dir` | `/srv/berth` | Where the server keeps its settings, data and SSH keys | | `berth_server_port` | `7777` | The port on this machine only | | `berth_server_tailscale_serve` | `true` | Put the server on the tailnet with TLS | +| `berth_server_update` | `false` | Pull the image and run the server from it when it is newer. See [Update it](#update-it) | +| `berth_server_node` | `false` | Let the server apply this machine as a node. See [The server on a node](#the-server-on-a-node) | | `tailscale_auth_key` | empty | An enrollment key. Without one Tailscale is not installed | | `tailscale_login_server` | empty | The address of a Headscale server | | `ssh_from` | `any` | Who reaches SSH from outside the tailnet: `any`, or an address or network | @@ -95,17 +133,53 @@ there, or keep your own file and pass it with `-e @my-vars.yml`. Pass the enrollment key on the command line. Do not save it in a file you commit. +This machine takes a key you make, in Tailscale's admin console or with +`headscale preauthkeys create`, because the recipe runs before berth does. +Make it one-use and short-lived. The hosts berth sets up later get their keys +from berth, which makes them through the tailnet's API with the secret +`tailnet_api_key`. + ## Update it -The data is in `/srv/berth`. A new container picks up where the old one -stopped. Pull the image, remove the container and run the recipe again: +Run the recipe again with `berth_server_update`. It pulls the image and, +when it is newer, makes the container again from it: + +```bash +sudo ./install.sh -e berth_server_update=true +``` + +The new container keeps the data in `/srv/berth` and every option of the old +one: its mounts, ports, extra host names, groups, environment, logging and +restart policy. An option the recipe sets, such as the port, takes the value +of this run. The old container is stopped and kept until the new one runs, +and starts again if the new one does not come up. A container with an option +the helper cannot carry over, such as `--privileged`, is left as it is, and +the run says which option it was. + +On a machine that runs Incus, the container keeps the name `incus-host`, at +which the provisioners of berth 0.5 reach it. A run without +`berth_server_update` leaves the image as it is. + +Pin a version with its tag: ```bash -sudo docker pull ghcr.io/helightdev/berth -sudo docker rm -f berth -sudo ./install.sh +sudo ./install.sh -e berth_server_update=true -e berth_server_image=ghcr.io/helightdev/berth:0.6.0 ``` +A server never goes back to an older version this way. Download the recipe +of the new version first, with the command in [Run it](#run-it), as each +version's recipe knows its own server. + +Once the machine is a node, its action `upgrade-server` does the same from a +berth of the new version: + +```bash +berth run big upgrade-server version=0.6.0 +``` + +Do not remove the container by hand to update it. A new one from the recipe +has only the recipe's options, and loses those passed by hand. + ## Good to know - The tailnet address needs HTTPS certificates switched on for your tailnet. diff --git a/recipes/berth-server/site.yml b/recipes/berth-server/site.yml index fa4db5b..dd77077 100644 --- a/recipes/berth-server/site.yml +++ b/recipes/berth-server/site.yml @@ -9,6 +9,20 @@ vars_files: - vars.yml tasks: + - name: Check that the helper that updates the server's container came with the recipe + ansible.builtin.stat: + path: "{{ playbook_dir }}/../../provision/ansible/roles/devbox/files/berth-server-upgrade.py" + delegate_to: localhost + become: false + register: berth_server_helper_found + + - name: Stop when it did not + ansible.builtin.assert: + that: berth_server_helper_found.stat.exists + fail_msg: >- + provision/ansible/roles/devbox/files/berth-server-upgrade.py is not beside recipes/. + Download the recipe with the command in its README, which takes that file too. + - name: Check that this is an Ubuntu the recipe knows ansible.builtin.assert: that: @@ -61,9 +75,58 @@ when: "'Status: active' not in firewall_state.stdout" changed_when: true + # A machine the incus-host recipe of berth 0.5 set up runs Incus, and the + # provisioners registered then reach it as incus-host, and Docker through + # its socket: the container keeps both. + - name: See whether this machine runs Incus + ansible.builtin.stat: + path: /var/lib/incus/server.crt + register: berth_server_incus + + - name: Find the group of Docker's socket + ansible.builtin.command: stat --format %g /var/run/docker.sock + register: docker_socket_group + changed_when: false + when: berth_server_node | bool or berth_server_incus.stat.exists + - name: Run the berth server ansible.builtin.include_tasks: ../shared/berth-server.yml + vars: + berth_server_helper: "{{ playbook_dir }}/../../provision/ansible/roles/devbox/files/berth-server-upgrade.py" + # On a node the server reaches this machine as `node`, and its Docker + # engine through the socket. The server in the image is no root, so it + # joins the socket's group. + berth_server_engine: "{{ berth_server_node | bool or berth_server_incus.stat.exists }}" + berth_server_options: >- + {{ ['--publish', '127.0.0.1:' ~ berth_server_port ~ ':7777', + '--volume', berth_server_dir ~ '/config:/config', + '--volume', berth_server_dir ~ '/data:/data', + '--volume', berth_server_dir ~ '/ssh:/home/berth/.ssh'] + + (['--add-host', 'node:host-gateway'] if berth_server_node | bool else []) + + (['--add-host', 'incus-host:host-gateway'] if berth_server_incus.stat.exists else []) + + (['--volume', '/var/run/docker.sock:/var/run/docker.sock', + '--group-add', docker_socket_group.stdout | trim] if berth_server_engine | bool else []) }} + + # Only the public half comes out of the container; berth makes its key there. + - name: Read the berth server's public key + ansible.builtin.command: docker exec berth berth key show + register: berth_server_key + changed_when: false + when: berth_server_node | bool + + - name: Let the berth server log in to this machine as root + ansible.builtin.blockinfile: + path: /root/.ssh/authorized_keys + create: true + mode: "0600" + marker: "# {mark} BERTH SERVER IN ITS CONTAINER" + block: "{{ berth_server_key.stdout | trim }}" + when: berth_server_node | bool - name: Say how to use the berth server ansible.builtin.debug: - msg: "{{ berth_server_handover }}" + msg: "{{ berth_server_handover + (berth_server_as_node if berth_server_node | bool else []) }}" + vars: + berth_server_as_node: + - "To make this machine a node of the server, on a berth connected to it: berth host add NAME node --user root --type node --feature docker/engine --feature incus/server" + - "Then apply it: berth apply NAME" diff --git a/recipes/berth-server/vars.yml b/recipes/berth-server/vars.yml index c032be9..323cd8d 100644 --- a/recipes/berth-server/vars.yml +++ b/recipes/berth-server/vars.yml @@ -12,11 +12,25 @@ berth_server_port: 7777 # It needs the tailnet's HTTPS certificates switched on, and does nothing # on a machine that is on no tailnet. berth_server_tailscale_serve: true +# Pull berth_server_image and run the server from it when it is newer. A +# container that is there keeps its data and its options either way, and +# gets those of this file that it lacks. +# ./install.sh -e berth_server_update=true +berth_server_update: false + +# Make this machine a node of the server, a server berth applies itself: +# the server gets root's login here with its own key, reaches this machine by +# the name `node`, and gets the socket of its Docker engine. Then add it from +# the server, as the README says. A container from an earlier run gets these +# and keeps its own. +berth_server_node: false # ── The tailnet ─────────────────────────────────────────────────────────── # An enrollment key joins the machine to your tailnet. Without one Tailscale # is not installed: pass it on the command line rather than saving it here. +# This recipe runs before berth does, so it takes a key made by hand; the +# hosts berth sets up get theirs from berth, with the secret tailnet_api_key. # ./install.sh -e tailscale_auth_key=tskey-auth-… tailscale_auth_key: "" tailscale_hostname: "{{ ansible_facts['hostname'] }}" diff --git a/recipes/incus-host/README.md b/recipes/incus-host/README.md deleted file mode 100644 index f93f579..0000000 --- a/recipes/incus-host/README.md +++ /dev/null @@ -1,210 +0,0 @@ -# An Incus and Docker host - -This recipe turns an empty Ubuntu server into a host for machines. It installs -Incus and Docker and sets up a firewall. The guests get a network and a -storage pool. berth can then create its hosts on the server as Incus guests, -so several agent machines run on one large server. - -It was run on Ubuntu 26.04 and needs 24.04 or later. - -## What you get - -| Part | Set-up | -| --- | --- | -| Incus | From Ubuntu's packages, with QEMU for virtual machines | -| Storage | A btrfs pool named `default`: a 120 GiB file, or a disk you name | -| Network | The bridge `incusbr0` at `10.77.0.1/24`. Guests get an address from it and go out through the server. No IPv6 | -| Default profile | Nesting is on. A container's systemd needs it, and so does Docker in a guest | -| Incus API | Port 8444, open on the tailnet interface only | -| Docker | `docker.io` with Compose and Buildx, from Ubuntu's packages | -| Sysbox | On request, the Sysbox runtime beside Docker, for containers that run systemd without `--privileged` | -| Firewall | `ufw`: incoming traffic is refused except SSH, the tailnet and the guests' bridge | -| Tailscale | Installed and joined when you pass an enrollment key | -| berth server | On request, berth's server in a Docker container, set up to create machines on this Incus | - -## Before you start - -The recipe does not wipe anything, so start from a fresh install. Reinstall -the server with plain Ubuntu and your SSH key from your provider's panel, or -with the [`rescue-install`](../rescue-install/) recipe when that installer -does not work. - -Virtual machines need KVM. A dedicated server has it. A rented virtual server -often does not, and then only containers work. Set `incus_virtual_machines` to -`false` there. - -## Run it - -On the server itself: - -```bash -curl -fsSL https://github.com/helightdev/berth/archive/refs/heads/main.tar.gz \ - | tar xz --strip-components=1 --wildcards '*/recipes' -cd recipes/incus-host -sudo ./install.sh -e tailscale_auth_key=tskey-auth-EXAMPLE -``` - -`install.sh` installs `ansible-core` from Ubuntu's packages and applies -`site.yml` to the machine it runs on. Its arguments go to `ansible-playbook`. - -From another machine that has Ansible: - -```bash -ansible-playbook -i 192.0.2.10, -u root site.yml -e tailscale_auth_key=tskey-auth-EXAMPLE -``` - -A second run changes nothing that is already in place. It leaves a server -alone that has a storage pool, so it never touches existing guests. - -## Add the server to berth - -The recipes leave a plain Ubuntu server where the account `root` takes the public key the server was installed with, such as the one you gave `rescue-install`. Add it to berth from your own machine. This example uses the made-up name `big` and the address `192.0.2.20`: - -```bash -berth host add big 192.0.2.20 --user root --auth ssh -berth host install big -``` - -`berth host add` saves `big` as a reachable host. `--auth ssh` makes berth log in with your own SSH configuration and agent, so the key that works for `ssh root@192.0.2.20` works for berth. The default, `--auth berth`, uses berth's own key, which berth installs on the first contact with a password or a private key that logs in once. `--auth ssh` works with a local berth only. With a berth server set, the server logs in with its own keys. - -`berth host install big` puts the controller alone on the server. berth can then report its status and run commands there, and it can update and reboot the server. Sessions run as the SSH user, here `root`. berth never applies such a server as a workstation: it stays a reachable or controller host, and the machines on it are created with the Incus or Docker provisioner. - -## Settings - -The settings are in [`vars.yml`](vars.yml), each with a comment. Change them -there, or keep your own file and pass it with `-e @my-vars.yml`. - -| Setting | Default | Meaning | -| --- | --- | --- | -| `incus_virtual_machines` | `true` | Install QEMU for virtual machines | -| `incus_pool_size` | `120GiB` | Size of the storage pool's file | -| `incus_pool_source` | empty | An empty disk or partition to use instead of a file. Its contents are lost | -| `incus_bridge_address` | `10.77.0.1/24` | The server's address on the guests' bridge | -| `incus_api_address` | `[::]:8444` | Where the Incus API listens | -| `incus_api_interface` | `tailscale0` | The one interface the firewall opens the API on. Empty opens it nowhere | -| `incus_trusted_clients` | none | Clients that may use the API, each with a name and a certificate file | -| `sysbox` | `false` | Also install Sysbox, for berth's Docker provisioner with `runtime=sysbox` | -| `sysbox_version` | `0.7.1` | The Sysbox release to install | -| `sysbox_checksum_amd64`, `sysbox_checksum_arm64` | those of `0.7.1` | The SHA-256 of the package for each architecture. The download is always checked, and another version needs its own sums | -| `berth_server` | `false` | Also run a berth server here. See below | -| `tailscale_auth_key` | empty | An enrollment key. Without one Tailscale is not installed | -| `tailscale_login_server` | empty | The address of a Headscale server | -| `ssh_from` | `any` | Who reaches SSH from outside the tailnet: `any`, or an address or network | - -Pass the enrollment key on the command line. Do not save it in a file you -commit. - -## A berth server on the same machine - -With `berth_server` set, the recipe also runs berth's server in a Docker -container: - -```bash -sudo ./install.sh -e berth_server=true -e tailscale_auth_key=tskey-auth-EXAMPLE -``` - -| Part | Set-up | -| --- | --- | -| Container | `berth`, from the image `ghcr.io/helightdev/berth`, restarted with the machine | -| Data | `/srv/berth` with the folders `config`, `data` and `ssh` | -| Port | 7777 on this machine only | -| Tailnet | `tailscale serve` puts it at `https://THIS-MACHINE.YOUR-TAILNET.ts.net` with TLS | -| Incus | The server gets a key and a certificate that this Incus trusts, stored as its secrets | -| Docker | The container gets the socket of this machine's engine. The server creates Docker machines and deployments here | - -The first run makes a setup key and prints it at the end, with the server's -address. The key lets the first user collect the server's token, for 24 hours -or until they choose Done. On your own machine: - -```bash -berth connect https://THIS-MACHINE.YOUR-TAILNET.ts.net -``` - -berth asks for the key. In a browser, open -`https://THIS-MACHINE.YOUR-TAILNET.ts.net/#setup-key=KEY` instead. Make a new -key with `sudo docker exec berth berth setup-key`. - -The token itself gives full control of every host. Without a key, read it with -`sudo cat /srv/berth/data/berth/serve.token`, then set it with -`berth config server` and `berth config server_token`. - -The server's provisioners are ready. The recipe registers three of them, -unless the server has provisioners of those names already: - -| Name | Type | Setting | -| --- | --- | --- | -| `incus-vm` | `incus/vm` | `server=https://incus-host:8444` | -| `incus-container` | `incus/container` | `server=https://incus-host:8444` | -| `docker` | `docker/container` | none: the engine of this machine | - -The container connects to the guests at their addresses on the bridge. berth -can set up a new machine there before it has joined a tailnet. - -Docker's socket gives root on this machine, as the Incus certificate does. So -whoever has the server's token controls the whole machine. - -A server from an earlier run keeps the container it has. To give it the -socket, run `sudo docker rm -f berth` and then the recipe again. The data in -`/srv/berth` stays. - -The tailnet address needs HTTPS certificates switched on for your tailnet. -Set `berth_server_tailscale_serve` to `false` to leave the server on this -machine only. A private image needs `docker login ghcr.io` before the run. - -## Sysbox for Docker machines - -berth's Docker provisioner runs its containers privileged, which gives root in -them the rights of root on the server. With `sysbox` set, the recipe installs -[Sysbox](https://github.com/nestybox/sysbox) from its release package, and a -machine made with `runtime=sysbox` runs without `--privileged`: - -```bash -sudo ./install.sh -e sysbox=true -berth provision studio-2 docker host=ssh://root@192.0.2.10 runtime=sysbox -``` - -The recipe downloads the package from the Sysbox release page on GitHub and -checks it against `sysbox_checksum_amd64` or `sysbox_checksum_arm64`. It -refuses to install without a sum. For another version, take the sums from that -release's page and pass them with `-e`. - -The installer starts `sysbox.service`, which runs `sysbox-mgr.service` and -`sysbox-fs.service`, and tells Docker about the runtime. It restarts Docker -when it has to change Docker's network settings, so set `sysbox` on the first -run, before any container exists. With containers present it leaves Docker -running and prints how to restart it. The installer needs a kernel that Sysbox -supports and stops when it finds an older one. Sysbox lists Ubuntu 22.04 and -24.04. It does not list 26.04 yet. On an unlisted release the installer only -checks the kernel. - -## Let another berth create machines here - -berth talks to the Incus API with a client certificate. Make a key and a -certificate on the machine that runs berth: - -```bash -openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:secp384r1 -sha384 \ - -nodes -days 3650 -subj "/CN=berth" -keyout berth.key -out berth.crt -``` - -Then let the server trust it. Either name it in `incus_trusted_clients` and -run the recipe again, or copy `berth.crt` to the server and add it there: - -```bash -incus config trust add-certificate berth.crt --name berth -``` - -Store the key and the certificate as berth's secrets `incus_client_key` and -`incus_client_cert`. The guide of berth's Incus plugin has the rest. - -## Good to know - -- The bridge's network must not be one the server is on already. Change - `incus_bridge_address` if `10.77.0.0/24` is taken. -- The guests' addresses are only reachable from the server. A guest that - berth sets up joins your tailnet, and berth reaches it there. -- Docker opens the ports its containers publish by itself, past `ufw`. - Publish a port on `127.0.0.1` or on the tailnet address to keep it private. -- With `ssh_from` set to one address, SSH from anywhere else is refused. - Your provider's console is the way back in if the address was wrong. -- To look at the result: `incus info`, `incus list`, `sudo ufw status`. diff --git a/recipes/incus-host/install.sh b/recipes/incus-host/install.sh deleted file mode 100755 index 5c045d3..0000000 --- a/recipes/incus-host/install.sh +++ /dev/null @@ -1,20 +0,0 @@ -#!/usr/bin/env bash -# Run the recipe on this server: install Ansible from Ubuntu's packages and -# apply site.yml to the machine itself. Arguments go to ansible-playbook: -# -# sudo ./install.sh -# sudo ./install.sh -e tailscale_auth_key=tskey-auth-… -e @my-vars.yml -# sudo ./install.sh --check --diff what a run would change -set -euo pipefail -cd "$(dirname "$0")" - -if [[ $EUID -ne 0 ]]; then - echo 'Run it as root: sudo ./install.sh' >&2 - exit 1 -fi -if ! command -v ansible-playbook >/dev/null; then - export DEBIAN_FRONTEND=noninteractive - apt-get update - apt-get install -y ansible-core -fi -exec ansible-playbook --connection local --inventory localhost, site.yml "$@" diff --git a/recipes/incus-host/site.yml b/recipes/incus-host/site.yml deleted file mode 100644 index a294c23..0000000 --- a/recipes/incus-host/site.yml +++ /dev/null @@ -1,276 +0,0 @@ -# A server that only hosts machines: Incus for containers and virtual -# machines, Docker beside it, and a firewall that lets the guests out and -# nobody but the tailnet in to the Incus API. -# -# ./install.sh on the server itself -# ansible-playbook -i SERVER, -u root site.yml from another machine -- name: Set up an Incus and Docker host - hosts: all - become: true - vars_files: - - vars.yml - tasks: - - name: Check that this is an Ubuntu the recipe knows - ansible.builtin.assert: - that: - - ansible_facts['distribution'] == 'Ubuntu' - - ansible_facts['distribution_version'] is version('24.04', '>=') - fail_msg: The recipe is written for Ubuntu 24.04 and later, and was run on 26.04. - - - name: Install Incus, Docker and what they need - ansible.builtin.apt: - name: "{{ base_packages + (vm_packages if incus_virtual_machines | bool else []) }}" - state: present - update_cache: true - cache_valid_time: 3600 - vars: - base_packages: - - incus - - btrfs-progs - - docker.io - - docker-buildx - - docker-compose-v2 - - ufw - - curl - - ca-certificates - - unattended-upgrades - vm_packages: - - qemu-system-x86 - - ovmf - - - name: Start Incus and Docker now and at boot - ansible.builtin.systemd_service: - name: "{{ item }}" - enabled: true - state: started - loop: - - incus.socket - - incus.service - - docker.service - - # ── Sysbox, when asked for ────────────────────────────────────────────── - - # Its installer restarts Docker when it has to change Docker's network - # settings, unless containers exist (then it leaves Docker running and - # says so), so it comes before the berth server's container. A host that - # has this version is left as it is. - - name: Install Sysbox beside Docker - when: sysbox | bool - vars: - sysbox_arch: "{{ 'arm64' if ansible_facts['architecture'] == 'aarch64' else 'amd64' }}" - sysbox_sum: "{{ sysbox_checksum_arm64 if sysbox_arch == 'arm64' else sysbox_checksum_amd64 }}" - sysbox_package: "/root/sysbox-ce_{{ sysbox_version }}.linux_{{ sysbox_arch }}.deb" - block: - - name: Require a checksum for the Sysbox package - ansible.builtin.assert: - that: - - ansible_facts['architecture'] in ['x86_64', 'aarch64'] - - sysbox_sum is match('^[0-9a-f]{64}$') - fail_msg: >- - Sysbox is installed as root, so its package must be checked. Set - sysbox_checksum_{{ sysbox_arch }} to the SHA-256 of version - {{ sysbox_version }} from the release page, on an amd64 or arm64 machine. - - - name: See which Sysbox is installed - ansible.builtin.command: dpkg-query -W -f=${Version} sysbox-ce - register: sysbox_installed - changed_when: false - failed_when: false - - - name: Download Sysbox - ansible.builtin.get_url: - url: "https://github.com/nestybox/sysbox/releases/download/v{{ sysbox_version }}/sysbox-ce_{{ sysbox_version }}.linux_{{ sysbox_arch }}.deb" - dest: "{{ sysbox_package }}" - mode: "0600" - checksum: "sha256:{{ sysbox_sum }}" - when: not sysbox_installed.stdout.startswith(sysbox_version) - - - name: Install Sysbox - ansible.builtin.apt: - deb: "{{ sysbox_package }}" - when: not sysbox_installed.stdout.startswith(sysbox_version) - - - name: Remove the package file - ansible.builtin.file: - path: "{{ sysbox_package }}" - state: absent - - - name: Start Sysbox now and at boot - ansible.builtin.systemd_service: - name: sysbox.service - enabled: true - state: started - - # ── The tailnet ───────────────────────────────────────────────────────── - - - name: Join the tailnet - ansible.builtin.include_tasks: ../shared/tailscale.yml - when: tailscale_auth_key | length > 0 - - # ── Incus ─────────────────────────────────────────────────────────────── - - - name: See whether Incus has been set up - ansible.builtin.command: incus storage list --format csv - register: incus_pools - changed_when: false - - # A second run leaves a server that has a storage pool as it is. - - name: Set Incus up with a storage pool, the guests' bridge and the default profile - ansible.builtin.command: incus admin init --preseed - args: - stdin: "{{ lookup('ansible.builtin.template', 'preseed.yaml.j2') }}" - when: incus_pools.stdout | trim | length == 0 - changed_when: true - - - name: Read where the Incus API listens - ansible.builtin.command: incus config get core.https_address - register: incus_listens - changed_when: false - - - name: Set where the Incus API listens - ansible.builtin.command: "incus config set core.https_address {{ incus_api_address | quote }}" - when: incus_listens.stdout | trim != incus_api_address - changed_when: true - - - name: Read the clients Incus trusts - ansible.builtin.command: incus config trust list --format csv - register: incus_trusted - changed_when: false - - - name: Copy the certificates of the clients to trust - ansible.builtin.copy: - src: "{{ item.certificate }}" - dest: "/root/incus-trust-{{ item.name }}.crt" - mode: "0600" - loop: "{{ incus_trusted_clients }}" - loop_control: - label: "{{ item.name }}" - when: (item.name ~ ',') not in incus_trusted.stdout - - - name: Trust the clients - ansible.builtin.command: "incus config trust add-certificate /root/incus-trust-{{ item.name }}.crt --name {{ item.name | quote }}" - loop: "{{ incus_trusted_clients }}" - loop_control: - label: "{{ item.name }}" - when: (item.name ~ ',') not in incus_trusted.stdout - changed_when: true - - - name: Remove the copies - ansible.builtin.file: - path: "/root/incus-trust-{{ item.name }}.crt" - state: absent - loop: "{{ incus_trusted_clients }}" - loop_control: - label: "{{ item.name }}" - - # ── The firewall ──────────────────────────────────────────────────────── - - # SSH is allowed before the firewall is switched on, so the run does not lock itself out. - - name: Allow what the server is for - ansible.builtin.command: "ufw {{ item }}" - loop: "{{ rules | select | list }}" - register: firewall_rule - changed_when: "'Skipping' not in firewall_rule.stdout" - vars: - api_port: "{{ incus_api_address.rsplit(':', 1)[-1] }}" - rules: - - "{{ 'allow 22/tcp' if ssh_from == 'any' else 'allow from ' ~ ssh_from ~ ' to any port 22 proto tcp' }}" - - "{{ 'allow in on tailscale0 to any port 22 proto tcp' if tailscale_auth_key else '' }}" - - "{{ 'allow 41641/udp' if tailscale_auth_key else '' }}" - # The guests ask the bridge for their address and for names. - - "allow in on {{ incus_bridge }}" - - "route allow in on {{ incus_bridge }}" - - "route allow out on {{ incus_bridge }}" - - "{{ 'allow in on ' ~ incus_api_interface ~ ' to any port ' ~ api_port ~ ' proto tcp' if incus_api_interface else '' }}" - # The berth server is a container, which reaches the Incus API over Docker's bridge. - - "{{ 'allow in on docker0 to any port ' ~ api_port ~ ' proto tcp' if berth_server | bool else '' }}" - - "{{ 'allow in on tailscale0 to any port 443 proto tcp' if (berth_server | bool) and (berth_server_tailscale_serve | bool) else '' }}" - - - name: See whether the firewall is on - ansible.builtin.command: ufw status - register: firewall_state - changed_when: false - - - name: Switch the firewall on - ansible.builtin.command: ufw --force enable - when: "'Status: active' not in firewall_state.stdout" - changed_when: true - - # ── A berth server, when asked for ────────────────────────────────────── - - - name: Find the group of Docker's socket - ansible.builtin.command: stat --format %g /var/run/docker.sock - register: docker_socket_group - changed_when: false - when: berth_server | bool - - - name: Run a berth server - ansible.builtin.include_tasks: ../shared/berth-server.yml - when: berth_server | bool - vars: - # The Incus API and the Docker engine are on the machine the container - # runs on. The server in the image is no root, so it joins the socket's group. - berth_server_run_options: >- - --add-host incus-host:host-gateway - --volume /var/run/docker.sock:/var/run/docker.sock - --group-add {{ docker_socket_group.stdout | trim }} - - - name: Set the berth server up for this Incus - when: berth_server | bool - vars: - api_port: "{{ incus_api_address.rsplit(':', 1)[-1] }}" - block: - # The key is made in the container and stays there: only the certificate comes out. - - name: Let the berth server create machines on this Incus - ansible.builtin.shell: | - set -e - docker exec berth sh -c 'umask 077 && cd /tmp && openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:secp384r1 -sha384 -nodes -days 3650 -subj /CN=berth-server -keyout incus.key -out incus.crt 2>/dev/null && berth secret set incus_client_key < incus.key >/dev/null && berth secret set incus_client_cert < incus.crt >/dev/null && rm incus.key' - docker cp berth:/tmp/incus.crt /root/incus-trust-berth-server.crt - docker exec berth rm /tmp/incus.crt - docker exec -i berth berth secret set incus_server_cert < /var/lib/incus/server.crt >/dev/null - incus config trust add-certificate /root/incus-trust-berth-server.crt --name berth-server - rm /root/incus-trust-berth-server.crt - args: - executable: /bin/bash - when: "'berth-server,' not in incus_trusted.stdout" - changed_when: true - - # A provisioner registered already stays as it is, with the server somebody chose. - - name: Register the berth server's provisioners of this Incus and this Docker engine - ansible.builtin.shell: | - set -e - registered=$(docker exec berth berth provisioner) - add() { - if ! grep -q "^$1 " <<<"$registered"; then - docker exec berth berth provisioner add "$@" >/dev/null - echo "added $1" - fi - } - add incus-vm incus/vm server=https://incus-host:{{ api_port }} - add incus-container incus/container server=https://incus-host:{{ api_port }} - add docker docker/container - args: - executable: /bin/bash - register: provisioners_added - changed_when: "'added' in provisioners_added.stdout" - - - name: Say how to use the berth server - ansible.builtin.debug: - msg: "{{ berth_server_handover + ['It creates machines on this Incus and on this Docker engine'] }}" - - # ── A look at the result ──────────────────────────────────────────────── - - - name: Ask Incus and Docker how they are - ansible.builtin.command: "{{ item }}" - loop: "{{ ['incus info', 'docker info'] + (['sysbox-runc --version'] if sysbox | bool else []) }}" - changed_when: false - register: answers - loop_control: - label: "{{ item }}" - - - name: Say what there is - ansible.builtin.debug: - msg: - - "Incus API: {{ incus_api_address }}{{ ', open on ' ~ incus_api_interface if incus_api_interface else ', closed by the firewall' }}" - - "Guests: {{ incus_bridge }} at {{ incus_bridge_address }}, storage pool {{ incus_pool }} ({{ incus_pool_source or incus_pool_size ~ ' in a file' }})" - - "Trusted clients: {{ (incus_trusted_clients | map(attribute='name') | list + (['berth-server'] if berth_server | bool else [])) | join(', ') or 'none yet: incus config trust add-certificate FILE' }}" diff --git a/recipes/incus-host/templates/preseed.yaml.j2 b/recipes/incus-host/templates/preseed.yaml.j2 deleted file mode 100644 index e8874af..0000000 --- a/recipes/incus-host/templates/preseed.yaml.j2 +++ /dev/null @@ -1,30 +0,0 @@ -networks: -- name: {{ incus_bridge }} - type: bridge - config: - ipv4.address: {{ incus_bridge_address }} - ipv4.nat: "true" - ipv6.address: none -storage_pools: -- name: {{ incus_pool }} - driver: btrfs - config: -{% if incus_pool_source %} - source: {{ incus_pool_source }} -{% else %} - size: {{ incus_pool_size }} -{% endif %} -profiles: -- name: default - config: - # A container's systemd does not come up without it, and Docker in a guest needs it too. - security.nesting: "true" - devices: - eth0: - name: eth0 - network: {{ incus_bridge }} - type: nic - root: - path: / - pool: {{ incus_pool }} - type: disk diff --git a/recipes/incus-host/vars.yml b/recipes/incus-host/vars.yml deleted file mode 100644 index 389bfa4..0000000 --- a/recipes/incus-host/vars.yml +++ /dev/null @@ -1,76 +0,0 @@ -# What the recipe sets up. Change it here, or pass your own file: -# ./install.sh -e @my-vars.yml - -# ── Incus ───────────────────────────────────────────────────────────────── - -# Also install QEMU, for virtual machines. The server needs KVM for them. -incus_virtual_machines: true - -# The storage pool for the guests' disks. With no source it is a btrfs file -# of this size on the root file system, which every server can do. -incus_pool: default -incus_pool_size: 120GiB -# A whole empty disk or partition instead, such as /dev/nvme1n1. Its -# contents are lost. -incus_pool_source: "" - -# The bridge the guests are on, with the address the server has on it. They -# reach the internet through the server and nothing reaches them from outside. -incus_bridge: incusbr0 -incus_bridge_address: 10.77.0.1/24 - -# Where the Incus API listens. The firewall opens its port on one interface -# only: the tailnet's, so nobody else reaches it. Empty opens it nowhere. -incus_api_address: "[::]:8444" -incus_api_interface: tailscale0 - -# Clients that may use the API, such as a berth that creates machines here: -# a name and the path of its certificate on the machine that runs the recipe. -# - name: berth -# certificate: ~/berth.crt -incus_trusted_clients: [] - -# ── Docker ──────────────────────────────────────────────────────────────── - -# Also install Sysbox, a runtime under which a container runs systemd and -# Docker without --privileged. berth's Docker provisioner uses it with -# runtime=sysbox. The package is installed before any container of the recipe -# runs, because it may restart Docker. -sysbox: false -sysbox_version: 0.7.1 -# The SHA-256 of the package for each architecture, from the release page. The -# download is always checked against it, and the recipe refuses to install -# Sysbox without one. Another sysbox_version needs its own sums. -sysbox_checksum_amd64: 9d6d5484f980d0a17f86c492c1262015c2afb66280bdb97215b79fde6a0261c5 -sysbox_checksum_arm64: 04ca894ae0b53f0fa54eaacc173ce40363c9a95ea5450f773716a84ef650a69b - -# ── A berth server ──────────────────────────────────────────────────────── - -# Also run a berth server on this machine, in a Docker container. It is -# given a certificate that this Incus trusts, so it can create machines here. -berth_server: false -berth_server_image: ghcr.io/helightdev/berth -# Where it keeps its settings, profiles and data. -berth_server_dir: /srv/berth -# The port on this machine only. Nothing outside reaches it. -berth_server_port: 7777 -# Put the server on the tailnet with TLS, at https://THIS-MACHINE.YOUR-TAILNET.ts.net. -# It needs the tailnet's HTTPS certificates switched on, and does nothing -# on a machine that is on no tailnet. -berth_server_tailscale_serve: true - -# ── The tailnet ─────────────────────────────────────────────────────────── - -# An enrollment key joins the server to your tailnet. Without one Tailscale -# is not installed: pass it on the command line rather than saving it here. -# ./install.sh -e tailscale_auth_key=tskey-auth-… -tailscale_auth_key: "" -tailscale_hostname: "{{ ansible_facts['hostname'] }}" -# The address of a Headscale server; empty for Tailscale's own service. -tailscale_login_server: "" - -# ── The firewall ────────────────────────────────────────────────────────── - -# Who may reach SSH from outside the tailnet: any, or one address or network -# such as 198.51.100.7. The provider's console is the way in if this is wrong. -ssh_from: any diff --git a/recipes/rescue-install/README.md b/recipes/rescue-install/README.md index 14c90df..81ccb1e 100644 --- a/recipes/rescue-install/README.md +++ b/recipes/rescue-install/README.md @@ -2,8 +2,9 @@ This recipe installs Ubuntu on a server from its provider's rescue system. It is for a server whose provider has no working installer for the Ubuntu -release you want. The result is a plain server with SSH, which the -[`incus-host`](../incus-host/) recipe can then set up. +release you want. The result is a plain server with SSH, which berth can then +apply as a [node](../../docs/content/docs/%28guide%29/use/nodes.mdx): a server +for Incus and Docker machines. It was run for Ubuntu 26.04, from a rescue system that is Debian or Ubuntu. diff --git a/recipes/shared/berth-server.yml b/recipes/shared/berth-server.yml index 6c29de6..542d486 100644 --- a/recipes/shared/berth-server.yml +++ b/recipes/shared/berth-server.yml @@ -23,20 +23,33 @@ # Its port stays on this machine: the token gives full control of every host. - name: Start the berth server in a container - ansible.builtin.command: >- - docker run --detach --name berth --restart unless-stopped - --publish 127.0.0.1:{{ berth_server_port }}:7777 - --volume {{ berth_server_dir }}/config:/config - --volume {{ berth_server_dir }}/data:/data - --volume {{ berth_server_dir }}/ssh:/home/berth/.ssh - {{ berth_server_run_options | default('') }} - {{ berth_server_image | quote }} + ansible.builtin.command: + argv: "{{ ['docker', 'run', '--detach', '--name', 'berth', '--restart', 'unless-stopped'] + + berth_server_options + [berth_server_image] }}" when: berth_container.rc != 0 changed_when: true +# A container from an earlier run, by this recipe or by hand, keeps its data +# and every option of its own, such as a socket or a name it reaches: the +# helper makes it again only when the image or the options above differ, and +# starts the old one again when the new one does not come up. +- name: Bring the berth server's container up to date, with its options + ansible.builtin.script: + cmd: >- + {{ berth_server_helper | quote }} berth --image {{ berth_server_image | quote }} + {{ '--pull' if berth_server_update | bool else '--keep-image' }} + {% for flag in berth_server_options | batch(2) %}{{ flag[0] }} {{ flag[1] | quote }} {% endfor %} + executable: python3 + register: berth_container_remade + changed_when: "'is unchanged' not in berth_container_remade.stdout" + when: berth_container.rc == 0 + - name: Start a berth server that was stopped ansible.builtin.command: docker start berth - when: berth_container.rc == 0 and berth_container.stdout | trim != 'true' + when: + - berth_container.rc == 0 + - berth_container_remade is not changed + - berth_container.stdout | trim != 'true' changed_when: true - name: See whether the tailnet serves the berth server @@ -61,12 +74,12 @@ host: 127.0.0.1 port: "{{ berth_server_port }}" timeout: 60 - when: berth_container.rc != 0 + when: berth_container.rc != 0 or berth_container_remade is changed # The first user collects the token with this key, so whoever ran the # recipe hands on the key and never the token. Later runs leave it be. - name: Make a setup key for the first user of the new berth server - ansible.builtin.command: docker exec berth berth setup-key + ansible.builtin.command: docker exec berth berth server setup-key register: berth_setup_key when: berth_container.rc != 0 changed_when: berth_setup_key.rc == 0 @@ -99,11 +112,10 @@ with_key: - "berth server: {{ berth_url }}{{ berth_here }}" - "Setup key for the first user, valid for 24 hours: {{ berth_key }}" - - "In a terminal: berth connect {{ berth_url }} and paste the setup key when asked" - - "In a browser: {{ berth_url }}/#setup-key={{ berth_key }}" - - "The key ends when the user chooses Done. A new one: sudo docker exec berth berth setup-key" + - "With berth: berth login {{ berth_url }} and paste the setup key when asked" + - "The key ends once berth login has the token. A new one: sudo docker exec berth berth server setup-key" without_key: - "berth server: {{ berth_url }}{{ berth_here }}" - - "A setup key for a user to collect the token: sudo docker exec berth berth setup-key" - - "With it: berth connect {{ berth_url }}, or open {{ berth_url }}/#setup-key=KEY in a browser" + - "A setup key for a user to collect the token: sudo docker exec berth berth server setup-key" + - "With it: berth login {{ berth_url }}" - "Or the token itself: sudo cat {{ berth_server_dir }}/data/berth/serve.token" diff --git a/tests/container_link.py b/tests/container_link.py index 276dece..29b13c9 100644 --- a/tests/container_link.py +++ b/tests/container_link.py @@ -4,7 +4,7 @@ python3 tests/container_link.py CONTAINER -Starts a session and runs a task, and fails when either does not end well. +Starts a session and runs an action, and fails when either does not end well. Not a unit test: run it against the built image, started with docker compose. """ import base64 @@ -68,13 +68,13 @@ def main(): sys.exit(f'Unexpected session result: {session}') print(f'session {session["id"]}: output and exit code as expected') - names = [task['name'] for task in call(container, 'GET', '/v1/tasks')] + names = [task['name'] for task in call(container, 'GET', '/v1/actions')] if 'sign-in:status' not in names: - sys.exit(f'No sign-in:status among the tasks: {names}') - task = call(container, 'POST', '/v1/tasks/sign-in:status?wait=120s', {}) + sys.exit(f'No sign-in:status among the actions: {names}') + task = call(container, 'POST', '/v1/actions/sign-in:status?wait=120s', {}) if task.get('exit_code') != 0: - sys.exit(f'Task sign-in:status failed: {task}') - print(f'task sign-in:status: ran as session {task["id"]}, steps {[step["name"] for step in task.get("steps", [])]}') + sys.exit(f'Action sign-in:status failed: {task}') + print(f'action sign-in:status: ran as session {task["id"]}, steps {[step["name"] for step in task.get("steps", [])]}') refused = None try: diff --git a/tests/e2e/boot.py b/tests/e2e/boot.py index 53556d9..0d35985 100644 --- a/tests/e2e/boot.py +++ b/tests/e2e/boot.py @@ -2,8 +2,8 @@ """Boot a real install in QEMU, set it up on first boot, and test it with berth over SSH. - ci/run.sh boot cloud the Ubuntu cloud image with the seed of berth seed - ci/run.sh boot iso the installer of berth iso, on a blank virtual disk + ci/run.sh boot cloud the Ubuntu cloud image with the seed of berth host seed + ci/run.sh boot iso the installer of berth host iso, on a blank virtual disk ci/run.sh boot cloud --keep -- boot_host.FirstBoot Only meant for the container of ci/boot.compose.yml. Logs, the timings and the @@ -47,11 +47,23 @@ def control_server(): """A tailnet control server to enrol the guest with on its first boot, as - (URL, key), when one is given.""" + (URL, API key), when one is given. The URL tailscale means Tailscale's own.""" url, key = os.environ.get('BERTH_BOOT_CONTROL_URL'), os.environ.get('BERTH_BOOT_CONTROL_KEY') return (url, key) if url and key else None +def use_control_server(station): + """berth's settings and secret for the control server, so that the seed it + renders carries a key berth made there.""" + url, key = control_server() + def setting(key, value): + station.berth('config', *(('set', key, value) if value else ('unset', key))) + setting('tailnet_login_server', '' if url == 'tailscale' else url) + for name in ('user', 'tags'): + setting(f'tailnet_{name}', os.environ.get(f'BERTH_BOOT_CONTROL_{name.upper()}', '')) + station.berth('secret', 'set', 'tailnet_api_key', input=key + '\n') + + def boot_vars(): """The role as someone would set up a quick host: the desktop and berth's own parts, with the plugin of the tests, and without coding agents, @@ -69,8 +81,7 @@ def boot_vars(): if not os.environ.get('BERTH_BOOT_EXTRAS'): variables['devbox_extra_packages'] = [] if control_server(): - # Enrolment on first boot. The role's option for a control server of - # one's own is not there yet; its name goes here when it is. + # Enrolment on first boot, with the key berth makes for the seed. variables['devbox_install_tailscale'] = True return variables @@ -83,7 +94,7 @@ def password_hash(): def boot_profile(station, kind, **values): profile = json.loads((SRC / 'provision/config/example.json').read_text()) profile.update({ - 'hostname': HOST, 'profile': 'cloud', 'keyboard': 'us', + 'hostname': HOST, 'target': 'cloud', 'keyboard': 'us', # As berth writes a new host: the shipped plugins' features are chosen # in the list, not switched on for a profile from before them. 'builder': {'shipped_plugins': True}, @@ -97,8 +108,6 @@ def boot_profile(station, kind, **values): if kind != 'cloud': # The installed machine is reached only from this container, as QEMU's host. profile['ansible_vars']['devbox_bootstrap_ssh_cidr'] = f'{vm.GATEWAY}/32' - if control_server(): - profile['tailscale_auth_key'] = control_server()[1] profile.update(values) return profile @@ -147,7 +156,7 @@ def wait_for_provisioning(station, machine, timeout): check=False).stdout raise AssertionError(f'The first-boot provisioning failed. The end of its journal, all of which' f' is in provision.log:\n{tail}') - if ready == 'ready' and state == 'configured-needs-enrollment' and active != 'activating': + if ready == 'ready' and state == 'configured' and active != 'activating': return line = lines[3] if len(lines) > 3 else '' if (state, line) != shown: @@ -165,13 +174,14 @@ def boot_cloud(run, side, station): with run.phase('fetch the cloud image'): image = vm.cloud_image() with run.phase('render the seed'): - station.berth('plugins', 'add', str(FIXTURE)) + station.berth('plugin', 'add', str(FIXTURE)) + if control_server(): + use_control_server(station) station.write_profile(boot_profile(station, 'cloud', release_url=f'https://{vm.GATEWAY}/devbox-release.tar.gz')) - station.berth('seed', HOST, timeout=300) + user_data = station.berth('host', 'seed', HOST, timeout=300).stdout build = station.data / 'berth/build' side.publish(build / 'devbox-release.tar.gz') - user_data = (build / HOST / 'cloud/user-data').read_text() - shutil.copyfile(build / HOST / 'cloud/user-data', run.logs / 'user-data') + (run.logs / 'user-data').write_text(user_data) machine = vm.Machine(HOST, run) # The provider's part: the instance's name, and trust in the test side's # certificate authority, which the release server's certificate is from. @@ -208,11 +218,13 @@ def install_iso(run, side, station): with run.phase('fetch the installer ISO'): iso, sha = vm.server_iso() with run.phase('build the installer'): - station.berth('plugins', 'add', str(FIXTURE)) + station.berth('plugin', 'add', str(FIXTURE)) + if control_server(): + use_control_server(station) station.write_profile(boot_profile(station, 'iso', target_disk_mode='serial', target_disk_serial=TARGET, builder={'shipped_plugins': True, 'iso': str(iso), 'iso_sha256': sha})) - station.berth('iso', HOST, timeout=1800) + station.berth('host', 'iso', HOST, timeout=1800) built = station.data / 'berth/build' / f'{HOST}-autoinstall.iso' run.facts['installer_gb'] = round(built.stat().st_size / 2**30, 2) diff --git a/tests/e2e/boot_host.py b/tests/e2e/boot_host.py index 19d1e29..6648f03 100644 --- a/tests/e2e/boot_host.py +++ b/tests/e2e/boot_host.py @@ -51,7 +51,7 @@ def root(self, command): return self.station.root_on_host(command).stdout def test_the_first_boot_applied_the_release_it_was_given(self): - self.assertEqual(self.root('cat /var/lib/devbox/provision-state').strip(), 'configured-needs-enrollment') + self.assertEqual(self.root('cat /var/lib/devbox/provision-state').strip(), 'configured') deployment = json.loads(self.root('cat /opt/devbox-bootstrap/deployment.json')) self.assertEqual(self.root('cat /var/lib/devbox/ready').strip(), deployment['release_sha256']) self.assertEqual(self.root('readlink /opt/devbox/current').strip(), @@ -78,7 +78,7 @@ def test_the_plugin_service_is_a_unit_that_answers(self): unit = show(self.station, SERVICE, 'ActiveState', 'UnitFileState', 'User') self.assertEqual((unit['ActiveState'], unit['UnitFileState'], unit['User']), ('active', 'enabled', AGENT)) answer = wait_for('the service answering', lambda: (done := self.station.berth( - 'call', HOST, 'e2e/echo', 'say', '{"text": "hi"}', check=False)).returncode == 0 and done, timeout=60) + 'service', 'call', HOST, 'e2e/echo', 'say', '{"text": "hi"}', check=False)).returncode == 0 and done, timeout=60) self.assertEqual(json.loads(answer.stdout), {'method': 'say', 'body': {'text': 'hi'}, 'level': 'host'}) def test_ssh_takes_keys_only_and_no_root(self): @@ -132,7 +132,7 @@ def test_the_firewall_lets_in_ssh_and_nothing_else(self): def test_the_status_task_ends_well(self): done = self.station.berth('run', HOST, 'status', check=False, timeout=180) self.assertEqual(done.returncode, 0, done.stdout + done.stderr) - for line in (f'Host {HOST}', 'Provision configured-needs-enrollment', 'ssh active', + for line in (f'Host {HOST}', 'Provision configured', 'ssh active', 'berthd active'): self.assertIn(line, done.stdout) self.assertNotRegex(done.stdout, r'(?m)^Failed', done.stdout) @@ -159,7 +159,7 @@ def test_a_service_is_stopped_started_and_restarted(self): before = self.state(SERVICE)['MainPID'] self.link.post(f'/v1/units/{SERVICE}/stop') self.assertEqual(self.state(SERVICE)['ActiveState'], 'inactive') - refused = self.station.berth('call', HOST, 'e2e/echo', 'say', '{}', check=False) + refused = self.station.berth('service', 'call', HOST, 'e2e/echo', 'say', '{}', check=False) self.assertNotEqual(refused.returncode, 0) self.link.post(f'/v1/units/{SERVICE}/start') self.assertEqual(self.state(SERVICE)['ActiveState'], 'active') @@ -168,7 +168,7 @@ def test_a_service_is_stopped_started_and_restarted(self): restarted = self.state(SERVICE)['MainPID'] self.assertEqual(len({before, started, restarted}), 3, (before, started, restarted)) wait_for('the service answering again', lambda: self.station.berth( - 'call', HOST, 'e2e/echo', 'say', '{}', check=False).returncode == 0, timeout=60) + 'service', 'call', HOST, 'e2e/echo', 'say', '{}', check=False).returncode == 0, timeout=60) status, answer = self.link.request('POST', f'/v1/units/{SERVICE}/mask', {}) self.assertEqual(status, 400, answer) @@ -200,7 +200,7 @@ def test_applying_again_changes_nothing(self): @needs_boot class Reboot(unittest.TestCase): - """berth apply --all --reboot on a host that needs it.""" + """berth rollout --all --reboot on a host that needs it.""" @classmethod def setUpClass(cls): @@ -213,7 +213,7 @@ def test_a_reboot_waits_for_running_sessions(self): sleeper = self.link.post('/v1/sessions', {'kind': 'process', 'title': 'busy', 'command': ['sleep', '900']}) boot = station.root_on_host('cat /proc/sys/kernel/random/boot_id').stdout try: - refused = station.berth('apply', '--all', '--reboot', check=False, timeout=1800) + refused = station.berth('rollout', '--all', '--reboot', check=False, timeout=1800) self.assertEqual(refused.returncode, 1, refused.stdout) self.assertRegex(refused.stdout, rf'(?m)^{HOST}: failed: needs a reboot, but 1 session is running there') self.assertEqual(station.root_on_host('cat /proc/sys/kernel/random/boot_id').stdout, boot) @@ -224,7 +224,7 @@ def test_a_rollout_reboots_the_host_and_checks_it(self): station = self.station station.root_on_host('touch /var/run/reboot-required') boot = station.root_on_host('cat /proc/sys/kernel/random/boot_id').stdout - done = station.berth('apply', '--all', '--reboot', check=False, timeout=2400) + done = station.berth('rollout', '--all', '--reboot', check=False, timeout=2400) self.assertEqual(done.returncode, 0, done.stdout[-3000:] + done.stderr[-2000:]) self.assertIn(f'▸ {HOST}: rebooting', done.stdout) self.assertIn(f'✓ {HOST} is up to date', done.stdout) @@ -235,8 +235,9 @@ def test_a_rollout_reboots_the_host_and_checks_it(self): # Enrolment in a tailnet on first boot, against a control server of the tests' -# own. boot.py gives the profile the key and switches Tailscale on when -# BERTH_CI_BOOT_CONTROL_URL and BERTH_CI_BOOT_CONTROL_KEY are set. +# own. boot.py gives berth its API key and switches Tailscale on when +# BERTH_CI_BOOT_CONTROL_URL and BERTH_CI_BOOT_CONTROL_KEY are set, and the +# seed carries a key berth made there. @needs_boot class Enrolment(unittest.TestCase): @unittest.skipUnless(os.environ.get('BERTH_BOOT_CONTROL_URL') and os.environ.get('BERTH_BOOT_CONTROL_KEY'), @@ -247,4 +248,6 @@ def test_the_host_enrolled_on_first_boot(self): self.assertEqual(status['BackendState'], 'Running') self.assertEqual(status.get('CurrentTailnet', {}).get('Name') is not None, True) deployment = station.root_on_host('cat /opt/devbox-bootstrap/deployment.json').stdout + # Neither the API key nor the key berth made for the seed stays. self.assertNotIn(os.environ['BERTH_BOOT_CONTROL_KEY'], deployment) + self.assertNotIn('devbox_tailscale_auth_key', deployment) diff --git a/tests/e2e/boot_install.py b/tests/e2e/boot_install.py index c3d061a..d8ff322 100644 --- a/tests/e2e/boot_install.py +++ b/tests/e2e/boot_install.py @@ -1,4 +1,4 @@ -"""The unattended installer of berth iso: its disk guard refused a disk with +"""The unattended installer of berth host iso: its disk guard refused a disk with another serial, erased the one it was given and left the other alone, and the installed system boots without the installer. boot.py ran the installer and noted what it saw from outside the guest; these check that and the guest.""" diff --git a/tests/e2e/e2e_access.py b/tests/e2e/e2e_access.py index fd8e89b..7372c48 100644 --- a/tests/e2e/e2e_access.py +++ b/tests/e2e/e2e_access.py @@ -7,6 +7,7 @@ It runs on `dock` before e2e_own_key, which wants that host as a provider hands it over: at the end the operator's password, its authorized_keys, the provided key and the host's SSH host keys are as they were.""" +import json import subprocess import tempfile import time @@ -99,7 +100,7 @@ def logs_in(self): """What `berth ssh` does with the profile as it is: the account it reached, or its failure. It runs without a terminal, so ssh can ask nobody for a password.""" - return subprocess.run([str(BERTH), 'ssh', HOST, 'id -un'], env=self.station.env, cwd=self.station.home, + return subprocess.run([str(BERTH), 'ssh', HOST, '--', 'id -un'], env=self.station.env, cwd=self.station.home, stdin=subprocess.DEVNULL, capture_output=True, text=True, timeout=60, start_new_session=True) @@ -115,13 +116,13 @@ def test_1_first_contact_installs_berths_key_with_a_password_used_once(self): refused = self.logs_in() self.assertNotEqual(refused.returncode, 0, refused.stdout) own = self.station.own_public() - done = self.berth('key', 'trust', HOST, input=self.password + '\n') + done = self.berth('key', 'trust', HOST, '--password-stdin', input=self.password + '\n') self.assertIn(f'{HOST} takes the key berth now', done.stdout) self.assertIn(own, self.authorized()) self.assertEqual(self.logs_in().stdout.strip(), OPERATOR) # The host key it met is recorded, and the password is not kept. - self.assertIn(fingerprint(self.original_host_key), self.berth('key', 'host', HOST).stdout) - self.assertNotIn(f'ssh/{HOST}', self.berth('secret').stdout) + self.assertIn(fingerprint(self.original_host_key), self.berth('host', 'verify', HOST).stdout) + self.assertNotIn(f'ssh/{HOST}', self.berth('secret', 'list').stdout) self.assertEqual(self.station.read_profile()['access']['auth'], 'berth') # Once it takes the key, nothing more is asked or added. again = self.berth('key', 'trust', HOST) @@ -132,21 +133,21 @@ def test_2_a_key_of_the_ring_logs_in_alone(self): file = self.station.root / 'deploy' public = keygen(file, 'deploy@access') self.admin.ssh(OPERATOR, 'cat >> ~/.ssh/authorized_keys', input=public + '\n') - added = self.berth('key', 'add', 'deploy', str(file)) + added = self.berth('key', 'import', 'deploy', str(file)) self.assertIn('Added the key deploy', added.stdout) - self.assertNotIn('PRIVATE KEY', added.stdout + self.berth('key').stdout) + self.assertNotIn('PRIVATE KEY', added.stdout + self.berth('key', 'list').stdout) self.set_auth('key:deploy') try: self.assertEqual(self.logs_in().stdout.strip(), OPERATOR) - options = self.berth('inventory', HOST).stdout + options = self.berth('host', 'inventory', HOST).stdout self.assertIn('keys/deploy', options) self.assertNotIn('keys/berth', options) - refused = self.berth('key', 'remove', 'deploy', check=False) + refused = self.berth('key', 'remove', 'deploy', '-y', check=False) self.assertNotEqual(refused.returncode, 0) self.assertIn(HOST, refused.stderr) finally: self.set_auth('berth') - self.berth('key', 'remove', 'deploy') + self.berth('key', 'remove', 'deploy', '-y') def test_3_a_password_goes_through_berths_askpass_helper(self): self.set_auth('password') @@ -156,13 +157,13 @@ def test_3_a_password_goes_through_berths_askpass_helper(self): self.assertNotEqual(without.returncode, 0) self.berth('secret', 'set', f'ssh/{HOST}', input=self.password + '\n') self.assertEqual(self.logs_in().stdout.strip(), OPERATOR) - self.assertNotIn(self.password, self.berth('inventory', HOST).stdout) + self.assertNotIn(self.password, self.berth('host', 'inventory', HOST).stdout) finally: - self.berth('secret', 'remove', f'ssh/{HOST}', check=False) + self.berth('secret', 'remove', f'ssh/{HOST}', '-y', check=False) self.set_auth('berth') def test_4_a_changed_host_key_is_refused_until_it_is_accepted(self): - recorded = self.berth('key', 'host', HOST).stdout + recorded = self.berth('host', 'verify', HOST).stdout self.root(f'rm -f /etc/ssh/ssh_host_* && ssh-keygen -A && {RELOAD_SSHD}') new = wait_for('the host showing a new host key', lambda: (now := scanned()) and now != self.original_host_key and now) @@ -171,17 +172,17 @@ def test_4_a_changed_host_key_is_refused_until_it_is_accepted(self): self.assertNotEqual(refused.returncode, 0) self.assertRegex(refused.stderr, r'HOST IDENTIFICATION HAS CHANGED|Host key verification failed') # berth's own logins say what happened and what to do. - said = self.berth('tasks', HOST, check=False) - self.assertNotEqual(said.returncode, 0) - self.assertIn(f'berth key host {HOST} --accept', said.stderr) - self.assertEqual(self.berth('key', 'host', HOST).stdout, recorded) - accepted = self.berth('key', 'host', HOST, '--accept') + said = json.loads(self.berth('host', 'show', HOST, '-o', 'json').stdout) + self.assertFalse(said['reachable']) + self.assertIn(f'berth host verify {HOST} --accept', said['error']) + self.assertEqual(self.berth('host', 'verify', HOST).stdout, recorded) + accepted = self.berth('host', 'verify', HOST, '--accept', '-y') self.assertIn(fingerprint(new), accepted.stdout) self.assertEqual(self.logs_in().stdout.strip(), OPERATOR) def test_5_a_rotation_replaces_berths_key_on_the_host(self): before = self.station.own_public() - rotated = self.berth('key', 'rotate', timeout=300) + rotated = self.berth('key', 'rotate', 'berth', '-y', timeout=300) self.assertIn(f'{HOST} takes the new key', rotated.stdout) after = self.station.own_public() self.assertNotEqual(after, before) @@ -190,7 +191,7 @@ def test_5_a_rotation_replaces_berths_key_on_the_host(self): self.assertNotIn(before.split()[1], keys) # The other lines stay: the deploy key the provider added. self.assertIn('deploy@access', keys) - self.assertNotIn('being replaced', self.berth('key').stdout) + self.assertNotIn('being replaced', self.berth('key', 'list').stdout) self.assertEqual(self.logs_in().stdout.strip(), OPERATOR) self.assertEqual(Path(f'{self.station.own_key}.pub').read_text().strip(), after) @@ -200,21 +201,20 @@ def test_6_automatic_logins_stop_at_a_refusal_so_sshd_does_not_shut_berth_out(se try: server = Workstation('access-penalty', ssh_key=False, trust=False) server.write_profile(profile(access={'address': HOST, 'user': OPERATOR, 'auth': 'berth'})) - serve = Serve(server, web=False) + serve = Serve(server) try: client = Workstation('access-penalty-client', ssh_key=False, trust=False) - client.berth('config', 'server', serve.url) - client.berth('config', 'server_token', serve.token) + client.berth('login', serve.url, '--token-stdin', input=serve.token) # The server's channel follower and the host list ask by themselves; each # refused login would count against this address at the host's sshd. for _ in range(5): - client.berth('list') + client.berth('ls') time.sleep(3) host = next(found for found in serve.call('GET', '/v1/hosts') if found['name'] == HOST) self.assertFalse(host['reachable']) self.assertIn('did not take berth', host['error'] or '') # One first contact with the password goes through. - done = server.berth('key', 'trust', HOST, input=self.password + '\n', timeout=120) + done = server.berth('key', 'trust', HOST, '--password-stdin', input=self.password + '\n', timeout=120) self.assertIn(f'{HOST} takes the key berth now', done.stdout, done.stderr) self.assertNotIn('PerSourcePenalties', done.stdout + done.stderr) self.assertEqual(self.admin.ssh(OPERATOR, 'true', check=False).returncode, 0) diff --git a/tests/e2e/e2e_adopt.py b/tests/e2e/e2e_adopt.py index f7d8212..db16740 100644 --- a/tests/e2e/e2e_adopt.py +++ b/tests/e2e/e2e_adopt.py @@ -1,5 +1,5 @@ """The profile a host keeps of itself, and another berth taking the host over -from it: the file the role writes, berthd serving it, berth adopt reading it +from it: the file the role writes, berthd serving it, berth host adopt reading it over SSH while berthd is broken, and the next applies of the new berth. It looks at the host as provisioned() applied it, and leaves it so: the new @@ -42,16 +42,15 @@ def setUpClass(cls): def test_the_host_keeps_its_profile_for_root_only(self): self.assertEqual(self.station.root_on_host(f'stat -c %a:%U:%G {KEPT}').stdout.strip(), '600:root:root') self.assertNotEqual(self.station.ssh(OPERATOR, f'cat {KEPT}', check=False).returncode, 0) - self.assertEqual((self.kept['format'], self.kept['host']), (2, HOST)) + self.assertEqual((self.kept['format'], self.kept['host']), (3, HOST)) self.assertRegex(self.kept['written'], r'^\d{4}-\d\d-\d\dT\d\d:\d\d:\d\dZ$') saved = self.station.read_profile() kept = self.kept['profile'] # How this berth logs in means nothing to another one. self.assertNotIn('access', kept) self.assertEqual(kept['builder'], {key: value for key, value in saved['builder'].items() if key not in LOCAL}) - self.assertNotIn('tailscale_auth_key', kept) for key, value in saved.items(): - if key not in ('access', 'builder', 'tailscale_auth_key'): + if key not in ('access', 'builder'): # An empty list a test saved since the last apply is not on the host yet. self.assertEqual(kept.get(key, []), value, key) @@ -60,7 +59,7 @@ def test_berthd_serves_it(self): link = BerthdLink(self.station, key=self.station.key) self.assertEqual(link.get('/v1/applied/profile'), self.kept) # The controller says the host is a workstation, as the file shows. - self.assertEqual(link.get('/v1/version')['level'], 'workstation') + self.assertEqual(link.get('/v1/version')['type'], 'workstation') def test_another_berth_adopts_the_host_while_berthd_is_broken(self): adopter = Workstation('adopter') @@ -68,16 +67,16 @@ def test_another_berth_adopts_the_host_while_berthd_is_broken(self): old_own = self.station.own_public() self.assertEqual(block(self.station, marker(old_own)), [old_own]) # Its keys are not on the host yet, and it says the ways in. - refused = adopter.berth('adopt', HOST, f'{OPERATOR}@{HOST}', check=False) + refused = adopter.berth('host', 'adopt', HOST, f'{OPERATOR}@{HOST}', check=False) self.assertNotEqual(refused.returncode, 0) - self.assertIn('berth key show', refused.stderr) - self.assertIn('--key', refused.stderr) - self.assertNotEqual(adopter.berth('profile', HOST, check=False).returncode, 0) + self.assertIn('berth key show', refused.output) + self.assertIn('--key', refused.output) + self.assertNotEqual(adopter.berth('host', 'export', HOST, check=False).returncode, 0) # Its public key put into authorized_keys by hand, as from the console. self.station.ssh(OPERATOR, f'echo {shell_quote(own)} >> ~/.ssh/authorized_keys') self.station.root_on_host('mv /usr/local/bin/berthd /usr/local/bin/berthd.away') try: - adopted = adopter.berth('adopt', HOST, f'{OPERATOR}@{HOST}') + adopted = adopter.berth('host', 'adopt', HOST, f'{OPERATOR}@{HOST}') except AssertionError: self.station.root_on_host('mv /usr/local/bin/berthd.away /usr/local/bin/berthd') raise @@ -88,10 +87,10 @@ def test_another_berth_adopts_the_host_while_berthd_is_broken(self): profile = adopter.read_profile() self.assertEqual(profile['access'], {'address': HOST, 'user': OPERATOR, 'auth': 'berth'}) # Once there, it is not replaced without being asked. - again = adopter.berth('adopt', HOST, f'{OPERATOR}@{HOST}', check=False) + again = adopter.berth('host', 'adopt', HOST, f'{OPERATOR}@{HOST}', check=False) self.assertNotEqual(again.returncode, 0) - self.assertIn('--force', again.stderr) - adopter.berth('adopt', HOST, f'{OPERATOR}@{HOST}', '--force') + self.assertIn('--force', again.output) + adopter.berth('host', 'adopt', HOST, f'{OPERATOR}@{HOST}', '--force') # Its apply repairs berthd and gives its key a block of its own. adopter.berth('apply', HOST, timeout=900) diff --git a/tests/e2e/e2e_apply.py b/tests/e2e/e2e_apply.py index 0727105..3d9d722 100644 --- a/tests/e2e/e2e_apply.py +++ b/tests/e2e/e2e_apply.py @@ -23,7 +23,7 @@ def test_the_first_apply_installs_berthd(self): self.assertEqual(version['mode'], 'system') berthd = json.loads(self.station.root_on_host('cat /etc/berth/berthd.json').stdout) self.assertEqual((berthd['agent'], berthd['operator']), (AGENT, OPERATOR)) - tasks = json.loads(self.station.root_on_host('cat /etc/berth/tasks.json').stdout) + tasks = json.loads(self.station.root_on_host('cat /etc/berth/actions.json').stdout) self.assertIn('secrets', [task['name'] for task in tasks]) print(f'\n first apply: {self.station.apply_seconds:.0f}s, {recap}', end=' ') diff --git a/tests/e2e/e2e_client.py b/tests/e2e/e2e_client.py index a20e4ab..660f20e 100644 --- a/tests/e2e/e2e_client.py +++ b/tests/e2e/e2e_client.py @@ -1,6 +1,8 @@ """berth on another machine, with its hosts those of a berth serve: the commands go through the server, which reaches the host. ClientApply looks at the apply of the shared state of berth serve (harness.served_extended()).""" +import json +import re import unittest from harness import AGENT, HOST, Workstation, needs_e2e, served, served_extended @@ -14,27 +16,31 @@ class Client(unittest.TestCase): def setUpClass(cls): cls.serve = served() cls.station = Workstation('client', ssh_key=False) - cls.station.berth('config', 'server', cls.serve.url) - cls.station.berth('config', 'server_token', cls.serve.token) + cls.station.berth('login', cls.serve.url, '--token-stdin', input=cls.serve.token) def test_the_hosts_are_the_servers(self): - self.assertRegex(self.station.berth('list').stdout, rf'(?m)^{HOST}\s+{HOST}$') - shown = self.station.berth('config').stdout - self.assertIn(f'server = {self.serve.url}', shown) - self.assertIn('server_token = (set)', shown) + listed = json.loads(self.station.berth('ls', '-o', 'json').stdout) + self.assertEqual([(host['name'], host['address']) for host in listed], [(HOST, HOST)]) + kept = json.loads((self.station.berth_dir / 'config.json').read_text()) + self.assertEqual(kept['server'], self.serve.url) + # The server's settings are listed, and no token among them; the + # client's own say which server it uses, and that it has a token. + shown = self.station.berth('config', 'list').stdout self.assertNotIn(self.serve.token, shown) + self.assertRegex(shown, rf'(?m)^ server +{re.escape(self.serve.url)}$') + self.assertRegex(shown, r'(?m)^ server_token +\(set\)$') def test_commands_go_through_the_server(self): - self.assertIn('secrets', self.station.berth('tasks', HOST).stdout) + self.assertIn('secrets', self.station.berth('action', 'list', HOST).stdout) self.assertEqual(self.station.berth('exec', HOST, '--', 'id', '-un').stdout.strip(), AGENT) done = self.station.berth('exec', HOST, '--', 'sh', '-c', 'cat; exit 5', input='piped\n', check=False) self.assertEqual((done.returncode, done.stdout), (5, 'piped\n')) self.assertEqual(self.station.berth('run', HOST, 'sign-in:status').returncode, 0) self.station.berth('exec', HOST, '--keep', '--', 'echo', 'kept through the server') - self.assertIn('echo kept through the server', self.station.berth('sessions', HOST).stdout) + self.assertIn('echo kept through the server', self.station.berth('session', 'list', HOST).stdout) def test_attach_through_the_server(self): - terminal = self.station.terminal('attach', HOST) + terminal = self.station.terminal('session', 'attach', HOST) try: terminal.pump(2) terminal.send('echo "via $((40 + 2))"\r') @@ -51,8 +57,7 @@ def test_the_seed_is_rendered_by_the_server(self): 'release_url': 'https://192.0.2.30/berth-release.tar.gz'} self.serve.call('PUT', '/v1/hosts/cloud-3/profile', draft) try: - written = self.station.berth('seed', 'cloud-3').stdout.strip() - seed = open(written).read() + seed = self.station.berth('host', 'seed', 'cloud-3').stdout self.assertTrue(seed.startswith('#cloud-config')) own = self.serve.station.own_public().split()[1] self.assertIn(own, seed) @@ -61,9 +66,9 @@ def test_the_seed_is_rendered_by_the_server(self): def test_once_with_a_server_given_on_the_command_line(self): once = Workstation('once', ssh_key=False) - listed = once.berth('--server', self.serve.url, '--token', self.serve.token, 'list').stdout + listed = once.berth('--server', self.serve.url, '--token-stdin', 'ls', input=self.serve.token).stdout self.assertRegex(listed, rf'(?m)^{HOST}\s') - refused = once.berth('--server', self.serve.url, '--token', 'wrong', 'list', check=False) + refused = once.berth('--server', self.serve.url, '--token-stdin', 'ls', input='wrong', check=False) self.assertNotEqual(refused.returncode, 0) self.assertIn(self.serve.url, refused.stderr) @@ -79,7 +84,6 @@ def setUpClass(cls): def test_apply_is_a_job_of_the_server(self): done = self.applied.first - self.assertIn(f'on the berth server at {self.applied.serve.url}', done.stderr) self.assertIn('PLAY RECAP', done.stdout) self.assertRegex(done.stdout, r'failed=0') @@ -92,7 +96,7 @@ class AccessToken(unittest.TestCase): def setUpClass(cls): cls.serve = served() cls.station = Workstation('service', ssh_key=False) - cls.station.berth('config', 'server', cls.serve.url) + cls.station.env['BERTH_SERVER'] = cls.serve.url def test_the_token_gives_the_role_it_names(self): env = {'BERTH_TOKEN': self.serve.issuer.token('berth:operate', subject='berth-client')} @@ -104,5 +108,5 @@ def test_the_token_gives_the_role_it_names(self): def test_a_token_of_another_issuer_is_refused(self): env = {'BERTH_TOKEN': self.serve.issuer.token('berth:admin', issuer='http://192.0.2.1')} - refused = self.station.berth('list', env=env, check=False) + refused = self.station.berth('ls', env=env, check=False) self.assertNotEqual(refused.returncode, 0) diff --git a/tests/e2e/e2e_controller.py b/tests/e2e/e2e_controller.py index 003f0d6..7a8f2f8 100644 --- a/tests/e2e/e2e_controller.py +++ b/tests/e2e/e2e_controller.py @@ -19,6 +19,11 @@ '/usr/local/lib/berth /etc/berth /var/lib/berth /run/berth') +def actions(station, host): + """A host's actions by their names, as the server lists them.""" + return {action['name']: action for action in json.loads(station.berth('action', 'list', host, '-o', 'json').stdout)} + + @needs_e2e class Controller(unittest.TestCase): """The tests build on each other, in the order of their names.""" @@ -47,32 +52,33 @@ def berth(self, *args, **kwargs): kwargs.setdefault('timeout', 120) return self.station.berth(*args, **kwargs) - def test_1_a_machine_added_by_its_address_is_only_reachable(self): - self.assertIn(f'Added {HOST} as a reachable host', self.added.stdout) + def test_1_a_machine_added_by_its_address_is_only_logged_in_to(self): + self.assertIn(f'Added {HOST} as a host of the type ssh', self.added.stdout) self.assertIn(f'{HOST} takes the key berth already', self.added.stdout) self.assertEqual(self.station.read_profile(), { - 'level': 'reachable', 'access': {'address': HOST, 'user': OPERATOR, 'auth': 'berth'}}) - self.assertEqual(self.berth('ssh', HOST, 'id -un').stdout.strip(), OPERATOR) - listed = self.berth('tasks', HOST).stdout - self.assertRegex(listed, r'(?m)^ install-controller\s+Install the controller') - # What needs more than a login is refused, and says what raises the level. + 'type': 'ssh', 'access': {'address': HOST, 'user': OPERATOR, 'auth': 'berth'}}) + self.assertEqual(self.berth('ssh', HOST, '--', 'id -un').stdout.strip(), OPERATOR) + listed = actions(self.station, HOST) + self.assertEqual(listed['install-controller']['label'], 'Install the controller') + # What needs more than a login is refused, and says what changes the type. refused = self.berth('apply', HOST, check=False) self.assertNotEqual(refused.returncode, 0) - self.assertIn('not a workstation', refused.stderr) + self.assertIn('neither a node nor a workstation', refused.stderr) refused = self.berth('exec', HOST, '--', 'id', check=False) self.assertNotEqual(refused.returncode, 0) - self.assertIn(f'berth host install {HOST}', refused.stderr) + self.assertIn(f'berth run {HOST} install-controller', refused.stderr) self.assertNotEqual(self.root('test -e /usr/local/bin/berthd', check=False).returncode, 0) def test_2_installing_the_controller_puts_berthd_and_box_there_alone(self): - done = self.berth('host', 'install', HOST, timeout=600) - self.assertIn(f'{HOST} is a controller host now', done.stdout) + done = self.berth('run', HOST, 'install-controller', '-y', timeout=600) + self.assertIn(f'{HOST} is a managed host now', done.stdout) self.assertIn(f"berth's controller answers on {HOST}", done.stdout) - self.assertEqual(self.station.read_profile()['level'], 'controller') + self.assertEqual(self.station.read_profile()['type'], 'managed') + # The role's record keeps the older name of the type. applied = json.loads(self.root('cat /etc/berth/applied.json').stdout) self.assertEqual((applied['level'], applied['plugins']), ('controller', [])) version = self.station.link().get('/v1/version') - self.assertEqual(version['level'], 'controller') + self.assertEqual(version['type'], 'managed') # Nothing of a workstation: no agent account, desktop or role record. for absent in ('id agent', 'test -e /etc/devbox', 'test -e /var/lib/devbox/status.json', 'test -e /etc/sudoers.d/90-devbox-agent', 'command -v Xvnc'): @@ -81,22 +87,22 @@ def test_2_installing_the_controller_puts_berthd_and_box_there_alone(self): def test_3_sessions_and_tasks_run_as_the_ssh_user(self): self.assertEqual(self.berth('exec', HOST, '--', 'id', '-un').stdout.strip(), OPERATOR) self.assertEqual(self.berth('exec', HOST, '--user', 'root', '--', 'id', '-un').stdout.strip(), 'root') - listed = self.berth('tasks', HOST).stdout + listed = actions(self.station, HOST) for name in ('status', 'update', 'secrets'): - self.assertRegex(listed, rf'(?m)^ {name}\s+', name) - self.assertNotRegex(listed, r'(?m)^ smoke\s+') - self.assertRegex(listed, r'(?m)^ install-controller\s+Update the controller') + self.assertIn(name, listed) + self.assertNotIn('smoke', listed) + self.assertEqual(listed['install-controller']['label'], 'Update the controller') status = self.berth('run', HOST, 'status') - self.assertIn('Level controller', status.stdout) + self.assertIn('Type managed', status.stdout) marker = random_text('controller') self.berth('exec', HOST, '--keep', '--', 'echo', marker) - line = next(line for line in self.berth('sessions', HOST).stdout.splitlines() if marker in line) - self.assertEqual(self.berth('output', HOST, line.split()[0]).stdout.strip(), marker) + line = next(line for line in self.berth('session', 'list', HOST).stdout.splitlines() if marker in line) + self.assertEqual(self.berth('session', 'logs', HOST, line.split()[0]).stdout.strip(), marker) def test_4_secrets_go_to_the_ssh_user(self): value = random_text('controller-secret') self.berth('secret', 'set', 'controller-token', input=value + '\n') - self.berth('host', HOST, 'secret', 'controller-token', 'env', 'CONTROLLER_TOKEN') + self.berth('host', 'set', HOST, 'secret.controller-token=CONTROLLER_TOKEN') sent = self.berth('run', HOST, 'secrets') self.assertNotIn(value, sent.stdout + sent.stderr) env = self.root(f'cat /home/{OPERATOR}/.config/devbox/secrets.env').stdout @@ -109,28 +115,27 @@ class OnAWorkstation(unittest.TestCase): a profile saved here under another name says the host is less: on `studio`, as provisioned() applied it.""" - def test_a_workstation_added_again_as_reachable_keeps_its_controller(self): + def test_a_workstation_added_again_as_ssh_keeps_its_controller(self): station = provisioned() - files = 'cat /etc/berth/applied.json /etc/berth/berthd.json /etc/berth/tasks.json' + files = 'cat /etc/berth/applied.json /etc/berth/berthd.json /etc/berth/actions.json' before = station.root_on_host(files).stdout - added = station.berth('host', 'add', 'adopted', HOST, '--user', OPERATOR, '--level', 'reachable', input='') + added = station.berth('host', 'add', 'adopted', HOST, '--user', OPERATOR, '--type', 'ssh', input='') try: # berth sees what the host is, and says how to adopt it. - self.assertIn('adopted is a workstation; its profile here says reachable', added.stdout) - self.assertIn('berth adopt adopted', added.stdout) - listed = station.berth('tasks', 'adopted').stdout - self.assertIn('adopted is a workstation; its profile here says reachable', listed) - self.assertRegex(listed, r'(?m)^ install-controller\s+.*take its controller apart') - refused = station.berth('host', 'install', 'adopted', check=False, timeout=600) + self.assertIn('adopted is a workstation; its profile here says ssh', added.stdout) + self.assertIn('berth host adopt adopted', added.stdout) + listed = actions(station, 'adopted') + self.assertIn('take its controller apart', listed['install-controller']['why']) + refused = station.berth('run', 'adopted', 'install-controller', '-y', check=False, timeout=600) self.assertNotEqual(refused.returncode, 0) self.assertIn('would take its controller apart', refused.stdout + refused.stderr) self.assertEqual(station.root_on_host(files).stdout, before) - self.assertEqual(station.read_profile('adopted')['level'], 'reachable') + self.assertEqual(station.read_profile('adopted')['type'], 'ssh') # Adopted from the profile the host keeps, by the address it was added at. - adopted = station.berth('adopt', 'adopted') + adopted = station.berth('host', 'adopt', 'adopted') self.assertIn('Adopted adopted', adopted.stdout) profile = station.read_profile('adopted') - self.assertNotIn('level', profile) + self.assertNotIn('type', profile) saved = station.read_profile() for key in ('hostname', 'operator_ssh_public_keys', 'agent_ssh_public_keys', 'ansible_vars'): self.assertEqual(profile[key], saved[key], key) diff --git a/tests/e2e/e2e_host.py b/tests/e2e/e2e_host.py index 1bf3cd4..ae4e7b0 100644 --- a/tests/e2e/e2e_host.py +++ b/tests/e2e/e2e_host.py @@ -14,16 +14,18 @@ def setUpClass(cls): cls.station = provisioned() def test_tasks_are_listed_by_group(self): - listed = self.station.berth('tasks', HOST).stdout - for line in ('System', 'Sync', 'Sign-In'): - self.assertRegex(listed, rf'(?m)^{line}$') - self.assertRegex(listed, r'(?m)^ secrets\s+Secrets and registry logins') + listed = json.loads(self.station.berth('action', 'list', HOST, '-o', 'json').stdout) + groups = {action['group'] or 'System' for action in listed} + self.assertLessEqual({'System', 'Sync', 'Sign-In'}, groups) + secrets = next(action for action in listed if action['name'] == 'secrets') + self.assertEqual(secrets['label'], 'Secrets and registry logins') + self.assertRegex(self.station.berth('action', 'list', HOST).stdout, r'(?m)^secrets\s+') def test_a_task_runs_and_its_exit_code_is_berths(self): self.assertEqual(self.station.berth('run', HOST, 'sign-in:status').returncode, 0) missing = self.station.berth('run', HOST, 'nothing-like-this', check=False) - self.assertEqual(missing.returncode, 1) - self.assertIn('has no task nothing-like-this', missing.stderr) + self.assertEqual(missing.returncode, 2) + self.assertIn('has no action nothing-like-this', missing.stderr) def test_the_status_task_copes_without_systemd(self): done = self.station.berth('run', HOST, 'status', check=False, timeout=120) @@ -45,16 +47,16 @@ def test_a_command_gets_input_and_leaves_with_its_code(self): def test_a_kept_session_is_listed_with_its_output(self): marker = random_text('kept') self.station.berth('exec', HOST, '--keep', '--', 'echo', marker) - listed = self.station.berth('sessions', HOST).stdout + listed = self.station.berth('session', 'list', HOST).stdout line = next(line for line in listed.splitlines() if marker in line) self.assertIn('exit 0', line) session = line.split()[0] - output = self.station.berth('output', HOST, session).stdout + output = self.station.berth('session', 'logs', HOST, session).stdout self.assertEqual(output.strip(), marker) - self.assertEqual(self.station.berth('output', HOST, session, '--tail', '1').stdout.strip(), marker) + self.assertEqual(self.station.berth('session', 'logs', HOST, session, '--tail', '1').stdout.strip(), marker) def test_attach_is_a_terminal_on_the_host(self): - terminal = self.station.terminal('attach', HOST) + terminal = self.station.terminal('session', 'attach', HOST) try: terminal.pump(2) terminal.send('echo "$(id -un) $((6 * 7))"\r') @@ -124,7 +126,7 @@ def test_files_outside_the_roots_are_refused(self): for path in ('/etc/shadow', '/root', f'/home/{AGENT}/../{OPERATOR}/.ssh/authorized_keys'): status, answer = self.link.request('GET', f'/v1/files?path={path}') self.assertIn(status, (403, 404), (path, answer)) - status, _ = self.link.request('PUT', '/v1/files?path=/etc/berth/tasks.json&overwrite=true', b'[]') + status, _ = self.link.request('PUT', '/v1/files?path=/etc/berth/actions.json&overwrite=true', b'[]') self.assertEqual(status, 403) def test_units_without_systemd_are_an_answer_not_a_crash(self): @@ -146,11 +148,11 @@ def test_the_hosts_state(self): self.assertTrue(any('berthd' in process['command'] for process in processes)) def test_a_task_runs_as_steps(self): - tasks = {task['name']: task for task in self.link.get('/v1/tasks')} + tasks = {task['name']: task for task in self.link.get('/v1/actions')} self.assertEqual(tasks['status']['user'], 'root') - session = self.link.post('/v1/tasks/sign-in:status?wait=60s', {}) + session = self.link.post('/v1/actions/sign-in:status?wait=60s', {}) self.assertEqual(session['exit_code'], 0) - self.assertEqual(session['task'], 'sign-in:status') + self.assertEqual(session['action'], 'sign-in:status') def test_the_channel_keeps_what_it_is_told_to(self): topic = random_text('e2e-host/notes') @@ -162,7 +164,7 @@ def test_the_channel_keeps_what_it_is_told_to(self): self.assertEqual([message['body'] for message in messages], [body]) def test_the_agent_sees_its_own_tasks(self): - listed = self.station.berth('exec', HOST, '--', 'box', 'tasks').stdout + listed = self.station.berth('exec', HOST, '--', 'box', 'actions').stdout self.assertIn('sign-in:status', listed) self.assertNotIn('reboot', re.findall(r'^\S+', listed, re.M)) self.assertEqual(self.station.berth('exec', HOST, '--', 'box', 'run', 'sign-in:status').returncode, 0) diff --git a/tests/e2e/e2e_local.py b/tests/e2e/e2e_local.py index 36e9f3c..d75cc9b 100644 --- a/tests/e2e/e2e_local.py +++ b/tests/e2e/e2e_local.py @@ -14,19 +14,20 @@ def test_berth_makes_its_own_key_once(self): first = station.own_public() self.assertRegex(first, r'^ssh-ed25519 \S+ berth@\S+$') self.assertEqual(station.own_public(), first) - listed = station.berth('key').stdout - self.assertRegex(listed, r'(?m)^berth\tSHA256:\S+\tssh-ed25519 ') - self.assertIn(first, listed) + listed = json.loads(station.berth('key', 'list', '-o', 'json').stdout)['keys'] + self.assertEqual([(key['name'], key['public']) for key in listed], [('berth', first)]) + self.assertRegex(listed[0]['fingerprint'], r'^SHA256:\S+$') + self.assertRegex(station.berth('key', 'list').stdout, r'(?m)^berth\s+SHA256:\S+') # The key lives in the store; it is a file only once ssh needs one. self.assertFalse(station.own_key.exists()) station.write_profile(profile(hostname='attic', access={'address': '192.0.2.61'}), name='attic') - inventory = json.loads(station.berth('inventory', 'attic').stdout)['all']['hosts']['attic'] + inventory = json.loads(station.berth('host', 'inventory', 'attic').stdout)['all']['hosts']['attic'] self.assertIn(f'IdentityFile={station.own_key}', inventory['ansible_ssh_common_args']) self.assertEqual(station.own_key.stat().st_mode & 0o777, 0o600) self.assertEqual(station.own_key.with_suffix('.pub').read_text().strip(), first) # The store keeps both halves, so a lost file comes back as it was. station.own_key.with_suffix('.pub').unlink() - station.berth('inventory', 'attic') + station.berth('host', 'inventory', 'attic') self.assertEqual(station.own_key.with_suffix('.pub').read_text().strip(), first) self.assertEqual(station.own_public(), first) @@ -41,9 +42,10 @@ def test_the_cloud_seed_carries_the_operators_keys(self): # What F5 or mkpasswd would store; nobody logs in with it. operator_password_hash='$6$e2e$' + 'x' * 86, release_url='https://192.0.2.30/berth-release.tar.gz'), name='cloud-1') - done = station.berth('seed', 'cloud-1', timeout=300) - seeds = list((station.data / 'berth/build').rglob('user-data')) - self.assertEqual(len(seeds), 1, done.stdout + done.stderr) + written = station.root / 'user-data' + done = station.berth('host', 'seed', 'cloud-1', '--file', str(written), timeout=300) + seeds = [written] + self.assertTrue(written.exists(), done.stdout + done.stderr) seed = seeds[0].read_text() self.assertTrue(seed.startswith('#cloud-config'), seed[:200]) self.assertIn(station.public.split()[1], seed) @@ -52,6 +54,11 @@ def test_the_cloud_seed_carries_the_operators_keys(self): self.assertEqual(seeds[0].stat().st_mode & 0o077, 0) +def listed(station): + """The hosts berth lists, by name and address.""" + return [(host['name'], host['address']) for host in json.loads(station.berth('ls', '-o', 'json').stdout)] + + @needs_e2e class Profiles(unittest.TestCase): """Profiles live in berth's store, and come in and out as JSON.""" @@ -61,20 +68,20 @@ def test_a_profile_goes_out_and_in_and_is_removed_with_what_is_kept_of_it(self): kept = profile(hostname='yard', access={'address': '192.0.2.50', 'user': OPERATOR}) station.write_profile(kept, name='yard') self.assertEqual(station.read_profile('yard'), kept) - self.assertRegex(station.berth('list').stdout, r'(?m)^yard\s+192\.0\.2\.50$') + self.assertIn(('yard', '192.0.2.50'), listed(station)) self.assertFalse((station.berth_dir / 'hosts').exists()) # What is not a profile is refused, and the saved one stays. - refused = station.berth('profile', 'yard', '--set', '-', input='[]', check=False) + refused = station.berth('host', 'import', 'yard', '-', input='[]', check=False) self.assertNotEqual(refused.returncode, 0) self.assertEqual(station.read_profile('yard'), kept) # Passwords go to a terminal, or to a pipe only when asked to. - piped = station.berth('credentials', 'yard', check=False) + piped = station.berth('host', 'credentials', 'yard', check=False) self.assertNotEqual(piped.returncode, 0) self.assertIn('--show', piped.stderr) - self.assertIn('no passwords', station.berth('credentials', 'yard', '--show').stderr) + self.assertIn('no passwords', station.berth('host', 'credentials', 'yard', '--show').stderr) station.remove_host('yard') - self.assertNotIn('yard', station.berth('list').stdout) - self.assertNotEqual(station.berth('profile', 'yard', check=False).returncode, 0) + self.assertNotIn('yard', station.berth('ls').stdout) + self.assertNotEqual(station.berth('host', 'export', 'yard', check=False).returncode, 0) def test_the_hosts_of_an_older_berth_move_into_the_store(self): station = Workstation('older-hosts', ssh_key=False, trust=False) @@ -89,31 +96,32 @@ def test_the_hosts_of_an_older_berth_move_into_the_store(self): mirrored = station.data / 'berth/served/http---192-0-2-7-7777/hosts' mirrored.mkdir(parents=True) (mirrored / 'yard.json').write_text('{}') - listed = station.berth('list') - self.assertRegex(listed.stdout, r'(?m)^attic\s+192\.0\.2\.60$') - self.assertIn('moved 1 host into berth', listed.stderr) + moved = station.berth('ls', '-o', 'json') + self.assertIn(('attic', '192.0.2.60'), [(host['name'], host['address']) for host in json.loads(moved.stdout)]) + self.assertIn('moved 1 host into berth', moved.stderr) self.assertFalse(hosts.exists()) self.assertTrue((station.berth_dir / 'migrated/hosts/attic.json').is_file()) self.assertFalse((station.data / 'berth/served').exists()) # It is kept in the layout of now; without a key file it logs in as it did. self.assertEqual(station.read_profile('attic'), { **old, 'builder': {'shipped_plugins': True}, 'access': {'address': '192.0.2.60', 'auth': 'ssh'}}) - shown = station.berth('credentials', 'attic', '--show').stdout - self.assertEqual(shown, f'{OPERATOR} made-up-password\n') + shown = json.loads(station.berth('host', 'credentials', 'attic', '-o', 'json').stdout) + self.assertEqual(shown['credentials'], {OPERATOR: 'made-up-password'}) # Once is enough. - self.assertNotIn('moved', station.berth('list').stderr) + self.assertNotIn('moved', station.berth('ls').stderr) @needs_e2e class Screens(unittest.TestCase): - """The terminal UI starts, shows the host and what berthd says of it, and quits.""" + """The screens start with a server in berth itself, show the host and what + berthd says of it, and quit.""" @classmethod def setUpClass(cls): cls.station = provisioned() def test_the_home_screen_lists_the_host(self): - terminal = self.station.terminal(cols=120, rows=30) + terminal = self.station.terminal('ui', cols=120, rows=30) try: terminal.expect(HOST) terminal.send('q') @@ -122,10 +130,13 @@ def test_the_home_screen_lists_the_host(self): terminal.close() def test_the_overview_shows_the_hosts_tasks(self): - terminal = self.station.terminal(HOST, cols=120, rows=46) + terminal = self.station.terminal('ui', HOST, cols=120, rows=46) try: # The rows of the tasks come from berthd on the host. terminal.expect('Secrets and registry logins') + # q goes back to the hosts, and there it quits. + terminal.send('q') + terminal.expect('New host') terminal.send('q') self.assertEqual(terminal.wait(), 0) finally: @@ -139,24 +150,26 @@ def setUpClass(cls): cls.station = provisioned() def test_hosts_and_their_inventory(self): - self.assertRegex(self.station.berth('list').stdout, rf'(?m)^{HOST}\s+{HOST}$') - inventory = json.loads(self.station.berth('inventory', HOST).stdout) + self.assertIn((HOST, HOST), listed(self.station)) + inventory = json.loads(self.station.berth('host', 'inventory', HOST).stdout) target = inventory['all']['hosts'][HOST] self.assertEqual((target['ansible_host'], target['ansible_user']), (HOST, OPERATOR)) self.assertIn(f'IdentityFile={self.station.own_key}', target['ansible_ssh_common_args']) self.assertIn('HostKeyAlias=', target['ansible_ssh_common_args']) def test_the_settings_say_where_things_are(self): - shown = self.station.berth('config').stdout - self.assertIn('Hosts in the store', shown) - self.assertIn('server = (not set)', shown) - self.assertIn(f"Store SQLite file {self.station.data / 'berth/berth.db'}", shown) + shown = self.station.berth('config', 'list').stdout + self.assertRegex(shown, r'(?m)^ retention_days +\(not set\)$') + # The client's own settings are listed apart, the token never shown. + self.assertRegex(shown, r'(?m)^This berth, on this machine\n server +\(not set\)$') + store = self.station.berth('server', 'store', 'status').stdout + self.assertIn(f"Store SQLite file {self.station.data / 'berth/berth.db'}", store) def test_the_api_documents(self): - operations = json.loads(self.station.berth('operations').stdout) + operations = json.loads(self.station.berth('server', 'api', 'operations').stdout) self.assertTrue(operations) - openapi = json.loads(self.station.berth('openapi').stdout) + openapi = json.loads(self.station.berth('server', 'api', 'openapi').stdout) self.assertIn('/v1/hosts', openapi['paths']) - asyncapi = json.loads(self.station.berth('asyncapi').stdout) + asyncapi = json.loads(self.station.berth('server', 'api', 'asyncapi').stdout) self.assertTrue(asyncapi.get('asyncapi')) self.assertRegex(self.station.berth('--version').stdout, r'^berth \d+\.\d+\.\d+') diff --git a/tests/e2e/e2e_node.py b/tests/e2e/e2e_node.py new file mode 100644 index 0000000..0a8abaa --- /dev/null +++ b/tests/e2e/e2e_node.py @@ -0,0 +1,276 @@ +"""A node on the bare test host `dock`: berth applies it without a desktop, +with a made-up plugin's feature for a node that offers a provisioner, and +registers that provisioner, naming the node. Applying again changes nothing +of it; a feature that is taken out takes its provisioner along, but not while +it holds a deployment; and the node is not forgotten while one is there. The +endpoints of a deployment of that provisioner are reached through the node's +controller, on its loopback and on its own network. + +The test host is a container without systemd and without the internet, so +tests/e2e/site.yml applies the node's tasks that need neither: the guard, +its SSH, the features, berthd and the record of the apply. The packages, the +firewall and Tailscale are left out, and so are the features incus/server and +docker/engine, whose engines do not run here: tests/test_node.py reads them, +and runs the firewall's helper and the server's upgrade against fakes. + +It runs last on `dock`, as it leaves a node behind.""" +import json +import stat +import unittest + +from harness import (HOST, OPERATOR, Serve, WebSocket, Workstation, hand_over_operator_key, needs_e2e, shell_quote, + wait_for) + +# The port of the listener on the node that sends back what it gets. +ECHO = 7071 + +# A provisioner that makes deployments and says nothing else of note. +# A deployment's endpoints are a listener on the node's loopback, the same +# one at the node's own address, and an address no network of the node holds. +PROGRAM = r'''#!/usr/bin/env python3 +import json, socket, sys +verb = sys.argv[1] +request = json.load(sys.stdin) +with open(__file__ + '.node', 'w') as said: + said.write(request.get('node', '')) +if verb == 'status' and not request.get('id'): + print(json.dumps({'message': 'The crane at ' + request['arguments'].get('place', 'nowhere') + ' answers'})) +elif verb in ('create', 'change'): + print(json.dumps({'id': 'load-' + request['name'], 'state': 'running'})) +elif verb == 'status': + endpoints = [{'name': 'db', 'address': '127.0.0.1', 'port': ECHO, 'local': True}, + {'name': 'lan', 'address': socket.gethostbyname(HOST), 'port': ECHO, 'local': True}, + {'name': 'far', 'address': '192.0.2.1', 'port': ECHO, 'local': True}] + print(json.dumps({'state': 'running', 'endpoints': endpoints})) +elif verb == 'destroy': + print(json.dumps({'state': 'gone'})) +else: + print(json.dumps({})) +''' + +# The crane's tasks: what a feature of a node does to offer its provisioner. +TASKS = '''--- +- name: Keep the certificate berth gave the crane to trust + ansible.builtin.copy: + dest: /etc/yard-crane.crt + content: "{{ devbox_client_certificates[devbox_plugin_feature].certificate }}" + mode: '0644' +- name: Offer the crane's provisioner + ansible.builtin.set_fact: + devbox_node_units: "{{ devbox_node_units + ['yard-crane'] }}" + devbox_offer: + settings: + place: "https://{address}:9999" + secrets: + yard_server_cert: "PUBLIC-CERT-OF-THE-CRANE" + fingerprint: "0123abcd" +''' + +MANIFEST = { + 'name': 'berth-plugin-yard', 'version': '1.0.0', 'displayName': 'Yard', 'files': ['bin', 'tasks'], + 'berth': { + 'features': [ + {'id': 'crane', 'label': 'Crane', 'types': ['node'], 'offers': ['box'], + 'trusts': {'certificate': 'yard_client_cert', 'key': 'yard_client_key'}, + 'tasks': 'tasks/crane.yml', + 'actions': [{'name': 'crane-status', 'label': 'Crane', 'user': 'root', 'extends': 'status', + 'run': 'echo crane'}]}, + {'id': 'bench', 'label': 'Bench'}, + ], + 'provisioners': [{ + 'id': 'box', 'label': 'Yard box', 'command': 'bin/provision', 'protocol': 2, + 'secrets': ['yard_client_cert', 'yard_client_key', {'name': 'yard_server_cert', 'required': False}], + 'arguments': [{'name': 'place', 'label': 'Place', 'required': True}], + 'kinds': [{'id': 'deployment', 'label': 'Load', 'spec': {'format': 'text', 'label': 'Load'}}], + }], + }, +} + + +@needs_e2e +class Node(unittest.TestCase): + """The tests build on each other, in the order of their names.""" + + # What the workstations of the tests before this one left on `dock`: a node + # is not applied over a workstation that berth set up, so it goes first. + LEFT = ('pkill -x berthd; userdel -r agent; rm -rf /etc/berth /var/lib/berth /etc/devbox /var/lib/devbox ' + '/etc/sudoers.d/90-devbox-agent /etc/ssh/sshd_config.d/00-devbox.conf; true') + + @classmethod + def setUpClass(cls): + cls.station = Workstation('node', ssh_key=False) + folder = cls.station.berth_dir / 'plugins/yard' + (folder / 'bin').mkdir(parents=True) + (folder / 'tasks').mkdir() + (folder / 'package.json').write_text(json.dumps(MANIFEST)) + (folder / 'tasks/crane.yml').write_text(TASKS) + program = folder / 'bin/provision' + program.write_text(PROGRAM.replace('ECHO', str(ECHO)).replace('HOST', repr(HOST))) + program.chmod(program.stat().st_mode | stat.S_IXUSR) + hand_over_operator_key(cls.station.own_public()) + key = cls.station.own_key_file() + wait_for('the operator taking the key of berth', lambda: cls.station.ssh( + OPERATOR, 'true', key=key, check=False).returncode == 0, timeout=60) + cls.station.ssh(OPERATOR, f'sudo -n sh -c {shell_quote(cls.LEFT)}', key=key, check=False) + cls.station.berth('host', 'add', HOST, HOST, '--user', OPERATOR, '--type', 'node', + '--feature', 'yard/crane', input='') + + @classmethod + def tearDownClass(cls): + hand_over_operator_key() + + def berth(self, *args, **kwargs): + kwargs.setdefault('input', '') + return self.station.berth(*args, **kwargs) + + def root(self, command, check=True): + return self.station.ssh(OPERATOR, f'sudo -n sh -c {shell_quote(command)}', key=self.station.own_key, + check=check) + + def profile(self, features): + profile = self.station.read_profile() + profile['ansible_vars'] = {'devbox_plugin_features': list(features), 'devbox_start_services': False, + 'devbox_install_tailscale': False, 'devbox_manage_firewall': False} + self.station.write_profile(profile) + return profile + + def provisioners(self): + listed = json.loads(self.berth('provisioner', 'list', '-o', 'json').stdout) + if isinstance(listed, dict): + listed = listed['provisioners'] + return [provisioner['id'] for provisioner in listed] + + def test_1_a_feature_for_a_workstation_is_refused_before_anything_runs(self): + added = self.station.read_profile() + self.assertEqual(added['type'], 'node') + self.assertEqual(added['ansible_vars']['devbox_plugin_features'], ['yard/crane']) + # A feature that is not for a node is refused when it is set, too. + refused = self.berth('host', 'set', HOST, 'feature.yard/bench=on', check=False) + self.assertEqual(refused.returncode, 2, refused.output) + self.assertIn('not for a host of the type node', refused.stderr) + self.profile(['yard/crane', 'yard/bench']) + refused = self.berth('apply', HOST, check=False) + self.assertNotEqual(refused.returncode, 0) + self.assertIn(f'{HOST} is a node, and yard/bench is for a workstation', refused.stderr) + self.assertNotEqual(self.root('test -e /etc/berth/applied.json', check=False).returncode, 0) + + def test_2_applying_the_node_registers_what_its_feature_offers(self): + self.profile(['yard/crane']) + done = self.berth('apply', HOST, timeout=900) + name = f'{HOST}-yard-box' + self.assertIn(f'✓ Registered the provisioner {name} (yard/box) of {HOST}', done.stdout) + self.assertIn('pins the server\'s certificate 0123abcd', done.stdout) + page = self.berth('provisioner', 'show', name).stdout + self.assertIn(f'place=https://{HOST}:9999', page) + self.assertIn(f'The crane at https://{HOST}:9999 answers', page) + self.assertIn(f'Node {HOST}, whose feature offers it', page) + # The host records what it is and what it offers, and says so. + applied = json.loads(self.root('cat /etc/berth/applied.json').stdout) + self.assertEqual(applied['level'], 'node') + self.assertEqual(applied['offers'][0]['feature'], 'yard/crane') + self.assertEqual(self.station.link().get('/v1/version')['type'], 'node') + # Only the certificate berth made went to the node, never its key. + trusted = self.root('cat /etc/yard-crane.crt').stdout + self.assertTrue(trusted.startswith('-----BEGIN CERTIFICATE-----')) + self.assertNotIn('PRIVATE KEY', self.root('cat /etc/berth/*.json /etc/yard-crane.crt').stdout) + # Sessions run as the SSH user; nothing of a workstation is there. + self.assertEqual(self.berth('exec', HOST, '--', 'id', '-un').stdout.strip(), OPERATOR) + for absent in ('id agent', 'test -e /etc/devbox/AGENT.md', 'test -e /etc/sudoers.d/90-devbox-agent', + 'command -v Xvnc'): + self.assertNotEqual(self.root(absent, check=False).returncode, 0, absent) + listed = self.berth('action', 'list', HOST).stdout + for action in ('status', 'update', 'secrets'): + self.assertRegex(listed, rf'(?m)^{action}\s+') + # No berth server's container here, so no upgrade of one. + self.assertNotRegex(listed, r'(?m)^upgrade-server\s+') + # The list names the node it lives on. + self.assertRegex(self.berth('provisioner', 'list').stdout, rf'(?m)^{name}\s+yard/box\s+{HOST}\s') + self.assertIn('crane', self.berth('run', HOST, 'status').stdout) + # SSH takes keys only, and leaves root none, as berth logs in as the operator. + sshd = self.root('cat /etc/ssh/sshd_config.d/00-devbox.conf').stdout + self.assertIn('PasswordAuthentication no', sshd) + self.assertIn('PermitRootLogin no', sshd) + + def test_3_applying_again_registers_nothing_new(self): + done = self.berth('apply', HOST, timeout=900) + self.assertNotIn('Registered the provisioner', done.stdout) + self.assertEqual(self.provisioners().count(f'{HOST}-yard-box'), 1) + + def test_4_a_provisioner_that_holds_a_deployment_stays_with_its_node(self): + name = f'{HOST}-yard-box' + self.berth('resource', 'create', 'crate', '--provisioner', name, '--spec', '-', input='one crate\n') + self.profile([]) + refused = self.berth('apply', HOST, check=False) + self.assertNotEqual(refused.returncode, 0) + self.assertIn(f'offers the provisioner {name} through yard/crane, which still holds', refused.stderr) + removed = self.station.remove_host(check=False) + self.assertNotEqual(removed.returncode, 0) + self.assertIn(f'is the node of the provisioner {name}', removed.stderr) + self.assertIn(name, self.provisioners()) + + def test_5_a_deployments_endpoints_are_reached_through_the_nodes_controller(self): + echo = ('import socket, threading\n' + 's = socket.create_server(("0.0.0.0", ECHO))\n' + 'def serve(c):\n' + ' while True:\n' + ' data = c.recv(65536)\n' + ' if not data: break\n' + ' c.sendall(data)\n' + ' c.shutdown(socket.SHUT_WR)\n' + 'while True: threading.Thread(target=serve, args=(s.accept()[0],)).start()\n').replace( + 'ECHO', str(ECHO)) + self.root(f'setsid python3 -c {shell_quote(echo)} berth-e2e-echo /dev/null 2>&1 &') + serve = Serve(self.station) + try: + resources = serve.call('GET', '/v1/resources') + [crate] = [resource for resource in resources if resource['name'] == 'crate'] + seen = serve.call('GET', f'/v1/resources/{crate["id"]}') + self.assertEqual([endpoint['name'] for endpoint in seen['endpoints']], ['db', 'lan', 'far'], seen) + node = (self.station.berth_dir / 'plugins/yard/bin/provision.node').read_text() + self.assertEqual(node, HOST) + path = f'/v1/resources/{crate["id"]}/endpoints' + data = bytes(range(256)) * 4096 + for endpoint in ('db', 'lan'): + back = wait_for(f'the endpoint {endpoint}', lambda endpoint=endpoint: self.through(serve, f'{path}/{endpoint}', data), + timeout=30) + self.assertEqual(back, data, endpoint) + # Off the node's networks, and the node's stream for the server alone, which no caller reaches. + for refused in (f'{path}/far', f'/v1/hosts/{HOST}/controller/endpoints/127.0.0.1/{ECHO}'): + with self.assertRaises(PermissionError) as raised: + WebSocket(serve.port, refused, protocols=[serve.token]) + self.assertTrue(str(raised.exception).startswith('403'), raised.exception) + finally: + serve.stop() + self.root('pkill -f berth-e2e-echo', check=False) + + @staticmethod + def through(serve, path, data): + """What comes back from an endpoint for `data` and this side's end, or None while it is refused.""" + try: + tunnel = WebSocket(serve.port, path, protocols=[serve.token]) + except PermissionError: + return None + try: + for start in range(0, len(data), 65536): + tunnel.send(data[start:start + 65536]) + tunnel.send({'eof': True}) + back = b'' + while True: + frame = tunnel.recv() + if frame is None or isinstance(frame, dict): + return back + back += frame + finally: + tunnel.close() + + def test_6_without_its_feature_the_provisioner_goes(self): + name = f'{HOST}-yard-box' + self.berth('resource', 'remove', 'crate', '-y') + done = self.berth('apply', HOST, timeout=900) + self.assertIn(f'✓ Forgot the provisioner {name}', done.stdout) + self.assertNotIn(name, self.provisioners()) + self.assertNotIn('offers', json.loads(self.root('cat /etc/berth/applied.json').stdout)) + + +if __name__ == '__main__': + unittest.main() diff --git a/tests/e2e/e2e_own_key.py b/tests/e2e/e2e_own_key.py index 06242d8..48213eb 100644 --- a/tests/e2e/e2e_own_key.py +++ b/tests/e2e/e2e_own_key.py @@ -23,7 +23,7 @@ def bare_profile(template): """A profile for the test host as a new host starts, with no SSH key in it.""" profile = json.loads(json.dumps(template)) - profile.update({'hostname': HOST, 'profile': 'cloud', 'keyboard': 'us', 'ansible_vars': light_vars()}) + profile.update({'hostname': HOST, 'target': 'cloud', 'keyboard': 'us', 'ansible_vars': light_vars()}) profile['builder'] = {'shipped_plugins': True} profile['access'] = {'address': HOST, 'user': OPERATOR, 'auth': 'berth'} return profile @@ -85,7 +85,7 @@ def test_applying_asks_for_nothing_and_again_changes_nothing(self): self.assertEqual(changed_tasks(self.second_apply.stdout), []) def test_tasks_are_listed_and_run(self): - self.assertRegex(self.berth('tasks', HOST).stdout, r'(?m)^ secrets\s+') + self.assertRegex(self.berth('action', 'list', HOST).stdout, r'(?m)^secrets\s+') self.assertEqual(self.berth('run', HOST, 'sign-in:status').returncode, 0) def test_commands_run_as_each_account(self): @@ -98,9 +98,9 @@ def test_commands_run_as_each_account(self): def test_sessions_are_kept_and_attached_to(self): marker = random_text('own-key') self.berth('exec', HOST, '--keep', '--', 'echo', marker) - line = next(line for line in self.berth('sessions', HOST).stdout.splitlines() if marker in line) - self.assertEqual(self.berth('output', HOST, line.split()[0]).stdout.strip(), marker) - terminal = self.station.terminal('attach', HOST) + line = next(line for line in self.berth('session', 'list', HOST).stdout.splitlines() if marker in line) + self.assertEqual(self.berth('session', 'logs', HOST, line.split()[0]).stdout.strip(), marker) + terminal = self.station.terminal('session', 'attach', HOST) try: terminal.pump(2) terminal.send('echo "$(id -un) $((6 * 7))"\r') @@ -113,14 +113,14 @@ def test_sessions_are_kept_and_attached_to(self): def test_secrets_are_sent_to_the_agent(self): value = random_text('own-key-secret') self.berth('secret', 'set', 'own-key-token', input=value + '\n') - self.berth('host', HOST, 'secret', 'own-key-token', 'env', 'OWN_KEY_TOKEN') + self.berth('host', 'set', HOST, 'secret.own-key-token=OWN_KEY_TOKEN') try: sent = self.berth('run', HOST, 'secrets') self.assertNotIn(value, sent.stdout + sent.stderr) env = self.root_on_host(f'cat /home/{AGENT}/.config/devbox/secrets.env') self.assertIn(f'export OWN_KEY_TOKEN={value}', env) finally: - self.berth('host', HOST, 'secret', 'own-key-token', 'off') + self.berth('host', 'set', HOST, 'secret.own-key-token=') self.berth('run', HOST, 'secrets') @@ -152,12 +152,12 @@ def setUpClass(cls): profile = bare_profile(template) profile['access'].update(user=owner, sudo='password') cls.station.write_profile(profile) - typed = {'BERTH_SUDO_PASSWORD': cls.password} - cls.first_apply = cls.station.berth('apply', HOST, input='', env=typed, timeout=900) + typed = cls.password + '\n' + cls.first_apply = cls.station.berth('apply', HOST, '--sudo-password-stdin', input=typed, timeout=900) cls.owner_profile = cls.station.read_profile() profile['access']['user'] = OPERATOR cls.station.write_profile(profile) - cls.second_apply = cls.station.berth('apply', HOST, input='', env=typed, timeout=900) + cls.second_apply = cls.station.berth('apply', HOST, '--sudo-password-stdin', input=typed, timeout=900) cls.operator_profile = cls.station.read_profile() @classmethod @@ -181,7 +181,7 @@ def test_an_apply_as_the_operator_settles_the_profile(self): self.assertEqual(done.stdout.strip(), AGENT) def test_plain_ssh_and_the_files_of_the_agent(self): - self.assertEqual(self.berth('ssh', HOST, 'id -un').stdout.strip(), OPERATOR) + self.assertEqual(self.berth('ssh', HOST, '--', 'id -un').stdout.strip(), OPERATOR) link = BerthdLink(self.station) path = f'/home/{AGENT}/own-key.txt' content = random_text('written').encode() @@ -200,12 +200,11 @@ class Served(OwnKeyOnly, unittest.TestCase): @classmethod def setUpClass(cls): cls.keyed = keyed() - cls.serve = Serve(cls.keyed, web=False) + cls.serve = Serve(cls.keyed) catalogue = cls.serve.call('GET', '/v1/setup') cls.serve.call('PUT', f'/v1/hosts/{HOST}/profile', bare_profile(catalogue['template'])) cls.station = Workstation('own-key-client', ssh_key=False) - cls.station.berth('config', 'server', cls.serve.url) - cls.station.berth('config', 'server_token', cls.serve.token) + cls.station.berth('login', cls.serve.url, '--token-stdin', input=cls.serve.token) cls.first_apply = cls.station.berth('apply', HOST, input='', timeout=900) cls.second_apply = cls.station.berth('apply', HOST, input='', timeout=900) @@ -225,7 +224,7 @@ def test_the_server_hands_out_its_key_and_no_sudo_switch(self): self.assertEqual(catalogue['template']['operator_ssh_public_keys'], []) def test_the_agents_files_through_the_server(self): - path = f'/v1/hosts/{HOST}/files?path=/home/{AGENT}/own-key-served.txt' + path = f'/v1/hosts/{HOST}/controller/files?path=/home/{AGENT}/own-key-served.txt' content = random_text('served').encode() status, _ = self.serve.request('PUT', path + '&overwrite=true', content, headers={'Content-Type': 'application/octet-stream'}) diff --git a/tests/e2e/e2e_plugins.py b/tests/e2e/e2e_plugins.py index f8788f5..bbc8e3e 100644 --- a/tests/e2e/e2e_plugins.py +++ b/tests/e2e/e2e_plugins.py @@ -20,20 +20,25 @@ def setUpClass(cls): cls.link = cls.station.link() def test_berth_lists_it_with_what_it_brings(self): - listed = self.station.berth('plugins').stdout + plugins = json.loads(self.station.berth('plugin', 'list', '-o', 'json').stdout)['plugins'] + self.assertIn(('e2e', PACKAGE, '1.0.0'), [(p['id'], p['package'], p['version']) for p in plugins]) + listed = self.station.berth('plugin', 'show', 'e2e').stdout self.assertRegex(listed, rf'(?m)^e2e {PACKAGE} 1\.0\.0 ') self.assertIn('e2e/hello Hello (Testing)', listed) - self.assertIn('task e2e:hello Say hello (as agent; the agent may run it; asks name)', listed) + self.assertIn('action e2e:hello Say hello (as agent; the agent may run it; asks name)', listed) self.assertIn('service e2e/echo as agent: say', listed) self.assertIn('channel e2e/notes', listed) def test_its_task_runs_with_an_argument(self): - self.assertRegex(self.station.berth('tasks', HOST).stdout, r'e2e:hello\s+Say hello\s+name=…') + listed = json.loads(self.station.berth('action', 'list', HOST, '-o', 'json').stdout) + hello = next(action for action in listed if action['name'] == 'e2e:hello') + self.assertEqual((hello['label'], [argument['name'] for argument in hello['arguments']]), ('Say hello', ['name'])) done = self.station.berth('run', HOST, 'e2e:hello', 'name=ada') self.assertEqual(done.stdout.strip(), f'hello ada from {AGENT}') missing = self.station.berth('run', HOST, 'e2e:hello', check=False) - self.assertIn('e2e:hello needs name=VALUE', missing.stderr) - choices = self.link.get('/v1/tasks/e2e:hello/choices/name') + self.assertEqual(missing.returncode, 2) + self.assertIn('is missing: pass name=VALUE', missing.stderr) + choices = self.link.get('/v1/actions/e2e:hello/choices/name') self.assertEqual(choices['choices'], ['ada', 'grace']) # The agent may run it too. done = self.station.berth('exec', HOST, '--', 'box', 'run', 'e2e:hello', 'name=grace') @@ -49,7 +54,7 @@ def test_its_ansible_tasks_and_guide_are_applied(self): self.assertIn('box run e2e:hello', guide) def test_its_service_answers_once_something_serves_it(self): - refused = self.station.berth('call', HOST, 'e2e/echo', 'say', '{}', check=False) + refused = self.station.berth('service', 'call', HOST, 'e2e/echo', 'say', '{}', check=False) self.assertNotEqual(refused.returncode, 0) # systemd would run the service; here the tests do, as the agent. served = self.link.post('/v1/sessions', { @@ -57,7 +62,7 @@ def test_its_service_answers_once_something_serves_it(self): 'command': ['bash', '/usr/local/lib/berth/services/e2e-echo.sh']}) try: def answered(): - done = self.station.berth('call', HOST, 'e2e/echo', 'say', '{"text": "hi"}', check=False) + done = self.station.berth('service', 'call', HOST, 'e2e/echo', 'say', '{"text": "hi"}', check=False) return done.returncode == 0 and json.loads(done.stdout) self.assertEqual(wait_for('the service answering', answered), {'method': 'say', 'body': {'text': 'hi'}, 'level': 'host'}) @@ -85,7 +90,7 @@ def setUpClass(cls): cls.station = plain().station def test_its_tasks_are_gone_from_berth_and_the_host(self): - self.assertNotIn('e2e/hello', self.station.berth('plugins').stdout) - self.assertNotIn('e2e:hello', self.station.berth('tasks', HOST).stdout) + self.assertNotIn('e2e', self.station.berth('plugin', 'list').stdout.split()) + self.assertNotIn('e2e:hello', self.station.berth('action', 'list', HOST).stdout) services = json.loads(self.station.root_on_host('cat /etc/berth/services.json').stdout) self.assertEqual(services, []) diff --git a/tests/e2e/e2e_provisioners.py b/tests/e2e/e2e_provisioners.py index e6fb438..06412bb 100644 --- a/tests/e2e/e2e_provisioners.py +++ b/tests/e2e/e2e_provisioners.py @@ -1,19 +1,24 @@ """Provisioners as subjects of their own, with the real berth: a made-up provisioner of a plugin in the workstation's own plugin folder, its page, its settings and its actions, and a deployment it makes as a resource, from the -command line and through berth serve. It needs no test host, and runs on -`dock` beside the modules there. +command line and through berth serve, with the endpoints it reports and the +commands it runs. It needs no test host, and runs on `dock` beside the +modules there. With BERTH_E2E_DOCKER_HOST set to a Docker engine the runner may use, such as -tcp://docker:2375, the Docker plugin's own actions are asked of it too, and a -small Compose deployment is made, changed and removed there. Without one those -cases are skipped.""" +tcp://docker:2375, the Docker plugin's own actions are asked of it too, a +small Compose deployment is made, changed and removed there, and an instance +with a published port is reached through its endpoint and runs a command. +Without one those cases are skipped.""" import json import os +import socket import stat +import struct +import threading import unittest -from harness import Serve, Workstation, needs_e2e +from harness import Serve, WebSocket, Workstation, needs_e2e # A provisioner that answers from what it is asked: it writes each request, # secrets left out, to requests.jsonl of its folder. @@ -40,7 +45,16 @@ print(f'deploying {request["name"]} with {request.get("environment", {}).get("TOKEN")}', file=sys.stderr) with open(f'spec.{request["name"]}', 'w') as spec: spec.write(request['spec']) - print(json.dumps({'id': f'project-{request["name"]}', 'state': 'running'})) + answer = {'id': f'project-{request["name"]}', 'state': 'running'} + # A spec that names a port of this machine is reached there. + for line in request['spec'].splitlines(): + if line.startswith('listen: '): + answer['endpoints'] = [{'name': 'bench', 'address': '127.0.0.1', 'port': int(line[8:]), + 'hint': 'echo', 'local': True}] + print(json.dumps(answer)) +elif verb == 'exec': + print(f'ran {" ".join(request["command"])} reading {request.get("stdin", "nothing")!r}', file=sys.stderr) + print(json.dumps({'exit_code': 4})) elif verb == 'destroy': print(json.dumps({'state': 'gone'})) elif verb == 'logs': @@ -67,7 +81,7 @@ 'id': 'box', 'label': 'Lab box', 'command': 'bin/provision', 'protocol': 2, 'verbs': ['list'], 'arguments': [{'name': 'place', 'label': 'Place', 'required': True}], 'kinds': [{'id': 'machine', 'setup': 'apply'}, - {'id': 'deployment', 'label': 'Bench setup', 'verbs': ['change', 'logs'], + {'id': 'deployment', 'label': 'Bench setup', 'verbs': ['change', 'logs', 'exec'], 'spec': {'format': 'text', 'label': 'Setup'}, 'actions': [{'name': 'services', 'label': 'Services', 'changes': False}]}], 'actions': [ @@ -113,43 +127,46 @@ def berth(self, *args, **kwargs): return self.station.berth(*args, **kwargs) def test_a_provisioner_is_listed_with_why_it_cannot_be_used(self): - listed = self.berth('provisioner').stdout - self.assertIn('box Lab box (lab/box) · not usable: box needs place', listed) - self.assertIn('old Old box (lab/old) · not usable: lab/old speaks version 1', listed) + listed = {found['id']: found for found in json.loads(self.berth('provisioner', 'list', '-o', 'json').stdout)} + self.assertEqual(listed['box']['type'], 'lab/box') + self.assertEqual(listed['box']['problem'], 'box needs place: berth provisioner set box place=VALUE') + self.assertEqual(listed['old']['type'], 'lab/old') + self.assertTrue(listed['old']['problem'].startswith('lab/old speaks version 1'), listed['old']['problem']) + self.assertRegex(self.berth('provisioner', 'list').stdout, r'(?m)^box\s+lab/box\s+no: box needs place: .*\s+Lab box$') types = self.berth('provisioner', 'types').stdout - self.assertRegex(types, r'(?m)^lab/box\s+Lab box · makes machine, deployment$') - page = self.berth('provisioner', 'old').stdout + self.assertRegex(types, r'(?m)^lab/box\s+Lab box\s+machine, deployment$') + page = self.berth('provisioner', 'show', 'old').stdout self.assertIn('unusable', page) def test_its_settings_page_and_actions_from_the_command_line(self): - self.berth('provisioner', 'box', 'run', 'settings', 'place=attic', '--yes') - page = self.berth('provisioner', 'box').stdout + self.berth('provisioner', 'run', 'box', 'settings', 'place=attic', '--yes') + page = self.berth('provisioner', 'show', 'box').stdout for line in ('State ready · The lab at attic answers', 'Benches 3', 'Settings place=attic', 'Machines berth made with it', ' attic box-1 running · berth has no record of it', ' look'): self.assertIn(line, page) # For this once, another place. - self.assertIn('The lab at cellar answers', self.berth('provisioner', 'box', 'place=cellar').stdout) - looked = self.berth('provisioner', 'box', 'run', 'look').stdout + self.assertIn('The lab at cellar answers', self.berth('provisioner', 'show', 'box', 'place=cellar').stdout) + looked = self.berth('provisioner', 'run', 'box', 'look').stdout self.assertEqual(looked.splitlines(), ['Looked around attic', ' Dust little']) - swept = self.berth('provisioner', 'box', 'run', 'sweep', 'shelf=bottom', '--yes') + swept = self.berth('provisioner', 'run', 'box', 'sweep', 'shelf=bottom', '--yes') self.assertIn('sweeping bottom with \n', swept.stdout) self.assertIn('✓ Swept', swept.stdout) - refused = self.berth('provisioner', 'box', 'run', 'sweep', 'shelf=middle', '--yes', check=False) + refused = self.berth('provisioner', 'run', 'box', 'sweep', 'shelf=middle', '--yes', check=False) self.assertNotEqual(refused.returncode, 0) self.assertIn('shelf=top or bottom only', refused.stderr) self.assertEqual(requests(self.folder)[-1]['verb'], 'act') def test_through_berth_serve_an_action_that_changes_something_is_a_job(self): - self.berth('provisioner', 'box', 'run', 'settings', 'place=attic', '--yes') - serve = Serve(self.station, web=False) + self.berth('provisioner', 'run', 'box', 'settings', 'place=attic', '--yes') + serve = Serve(self.station) try: page = serve.call('GET', '/v1/provisioners/box') self.assertEqual((page['state'], page['message']), ('ready', 'The lab at attic answers')) self.assertEqual(page['values'], {'shelf': ['top', 'bottom']}) actions = serve.call('GET', '/v1/provisioners/box/actions') self.assertEqual([action['name'] for action in actions], - ['new-deployment', 'settings', 'look', 'sweep', 'remove']) + ['new-deployment', 'settings', 'look', 'sweep', 'adopt', 'remove']) looked = serve.call('POST', '/v1/provisioners/box/actions/look') self.assertEqual(looked['result']['message'], 'Looked around attic') acted = serve.call('POST', '/v1/provisioners/box/actions/sweep', @@ -170,8 +187,8 @@ def test_through_berth_serve_an_action_that_changes_something_is_a_job(self): self.assertEqual(len(requests(self.folder)), before) # And from a client of the server. client = Workstation('provisioners-client', ssh_key=False, trust=False) - listed = client.berth('--server', serve.url, '--token', serve.token, 'provisioner').stdout - self.assertIn('box Lab box (lab/box)', listed) + listed = client.berth('--server', serve.url, '--token-stdin', 'provisioner', 'list', input=serve.token).stdout + self.assertRegex(listed, r'(?m)^box\s+lab/box\s') portal.close() finally: serve.stop() @@ -185,15 +202,14 @@ class Deployments(unittest.TestCase): def setUpClass(cls): cls.station = Workstation('deployments', ssh_key=False, trust=False) cls.folder = lab(cls.station) - cls.station.berth('provisioner', 'box', 'run', 'settings', 'place=attic', '--yes') + cls.station.berth('provisioner', 'run', 'box', 'settings', 'place=attic', '--yes') cls.station.berth('secret', 'set', 'bench_token', input='b3nch-t0ken-made-up\n') def berth(self, *args, **kwargs): return self.station.berth(*args, **kwargs) def test_1_a_deployment_is_made_from_its_spec_with_a_secret_by_name(self): - made = self.berth('deploy', 'bench', '--provisioner', 'box', '--spec', '-', - '--secret', 'bench_token:TOKEN', input='benches: 3\n') + made = self.berth('resource', 'create', 'bench', '--provisioner', 'box', '--spec', '-', '--secret', 'bench_token:TOKEN', input='benches: 3\n') output = made.stdout + made.stderr self.assertIn('deploying bench with [secret]', output) self.assertNotIn('b3nch-t0ken', output) @@ -202,28 +218,28 @@ def test_1_a_deployment_is_made_from_its_spec_with_a_secret_by_name(self): create = next(asked for asked in requests(self.folder) if asked['verb'] == 'create') self.assertEqual(list(create['environment']), ['TOKEN']) self.assertNotIn('b3nch-t0ken', create['spec'] + json.dumps(create['arguments'])) - listed = self.berth('resource').stdout - self.assertRegex(listed, r'(?m)^r-\S+\s+deployment\s+bench\s+box\s+running$') - page = self.berth('provisioner', 'box').stdout + listed = self.berth('resource', 'list').stdout + self.assertRegex(listed, r'(?m)^r-\S+\s+deployment\s+bench\s+box\s+running\s*$') + page = self.berth('provisioner', 'show', 'box').stdout self.assertIn('Bench setup berth made with it', page) self.assertRegex(page, r' bench\s+project-bench r-') def test_2_its_page_actions_log_and_change(self): - shown = self.berth('resource', 'bench').stdout + shown = self.berth('resource', 'show', 'bench').stdout for line in ('State running', 'Made as project-bench', 'Secrets bench_token as TOKEN', ' services', ' change', ' remove'): self.assertIn(line, shown) - said = self.berth('resource', 'bench', 'run', 'services').stdout + said = self.berth('resource', 'run', 'bench', 'services').stdout self.assertEqual(said.splitlines(), ['deployment project-bench has 1 service', ' web running']) - log = self.berth('resource', 'bench', 'logs', '--tail', '5').stdout + log = self.berth('resource', 'logs', 'bench', '--tail', '5').stdout self.assertIn('project-bench started, tail 5', log) (self.station.root / 'bench.txt').write_text('benches: 4\n') - self.berth('deploy', 'bench', '--provisioner', 'box', '--spec', str(self.station.root / 'bench.txt')) + self.berth('resource', 'create', 'bench', '--provisioner', 'box', '--spec', str(self.station.root / 'bench.txt')) self.assertEqual((self.folder / 'spec.bench').read_text(), 'benches: 4\n') self.assertEqual(requests(self.folder)[-1]['verb'], 'change') def test_3_through_berth_serve_with_a_token_asked_twice(self): - serve = Serve(self.station, web=False) + serve = Serve(self.station) try: body = {'provisioner': 'box', 'kind': 'deployment', 'name': 'yard', 'spec': 'yard: 1\n', 'secrets': [{'name': 'bench_token', 'env': 'TOKEN'}], 'token': 'yard-1'} @@ -248,36 +264,183 @@ def test_3_through_berth_serve_with_a_token_asked_twice(self): serve.stop() def test_4_removing_it_forgets_it(self): - removed = self.berth('resource', 'bench', 'remove', '--yes') + removed = self.berth('resource', 'remove', 'bench', '-y') self.assertIn('Removed the deployment bench', removed.stdout) - self.assertNotIn('bench', self.berth('resource').stdout) + self.assertNotIn('bench', self.berth('resource', 'list').stdout) def test_5_the_provisioners_own_action_makes_one_and_it_is_kept_while_it_holds_one(self): - self.berth('provisioner', 'box', 'run', 'new-deployment', 'name=shed', 'spec=shed: 1\nbenches: 2\n', - 'secrets=bench_token:TOKEN', '--yes') + self.berth('provisioner', 'run', 'box', 'new-deployment', 'name=shed', 'spec=shed: 1\nbenches: 2\n', 'secrets=bench_token:TOKEN', '--yes') self.assertEqual((self.folder / 'spec.shed').read_text(), 'shed: 1\nbenches: 2\n') - self.assertIn('shed', self.berth('resource').stdout) - refused = self.berth('provisioner', 'remove', 'box', check=False) + self.assertIn('shed', self.berth('resource', 'list').stdout) + refused = self.berth('provisioner', 'remove', 'box', '-y', check=False) self.assertNotEqual(refused.returncode, 0) self.assertIn('box still holds', refused.stderr) - self.berth('resource', 'shed', 'remove', '--yes') - self.assertIn('Removed the provisioner box', self.berth('provisioner', 'remove', 'box').stdout) - self.assertNotIn('Lab box', self.berth('provisioner').stdout) + self.berth('resource', 'remove', 'shed', '-y') + self.assertIn('Removed the provisioner box', self.berth('provisioner', 'remove', 'box', '-y').stdout) + self.assertNotIn('Lab box', self.berth('provisioner', 'list').stdout) + + +def echo(): + """A listener on the loopback that sends back what it gets and ends its side after the + caller has ended theirs: its port.""" + listener = socket.create_server(('127.0.0.1', 0)) + + def serve(): + while True: + connection, _ = listener.accept() + with connection: + while data := connection.recv(65536): + connection.sendall(data) + connection.shutdown(socket.SHUT_WR) + + threading.Thread(target=serve, daemon=True).start() + return listener.getsockname()[1] + + +def through_endpoint(serve, rid, endpoint, data): + """Send `data` to an endpoint of a resource on its own stream, end, and read what comes back + until the other side has ended too.""" + ws = WebSocket(serve.port, f'/v1/resources/{rid}/endpoints/{endpoint}', protocols=[serve.token]) + try: + ws.send(data) + ws.send({'eof': True}) + back = b'' + while True: + frame = ws.recv() + if frame is None or frame == {'eof': True}: + return back + back += frame + finally: + ws.close() + + +def through_portal(portal, rid, endpoint, data): + """The same over the portal, as a stream of its own number.""" + opened = portal.result('resources.connect', {'id': rid, 'endpoint': endpoint}) + assert opened.get('open'), opened + number = portal.next - 1 + tag = struct.pack('!I', number) + portal.send(tag + data) + portal.send({'sub': number, 'data': {'eof': True}}) + back = b'' + while True: + frame = portal._take(lambda got: (got[:4] == tag) if isinstance(got, bytes) else got.get('sub') == number, + 60) + if isinstance(frame, bytes): + back += frame[4:] + elif frame.get('data') == {'eof': True} or frame.get('end'): + portal.send({'sub': number, 'end': True}) + return back + + +def job_output(portal, job): + """A job's output and the exit code it ended with.""" + texts, data, _ = portal.stream('jobs.follow', {'id': job['id']}, + until=lambda texts, data: any('exited' in t for t in texts if t)) + exited = next(t['exited'] for t in texts if t and 'exited' in t) + return data.decode(errors='replace'), exited['exit_code'] + + +@needs_e2e +class Endpoints(unittest.TestCase): + """A deployment of the lab provisioner that says where it listens, reached through berth + serve, and a command run in it.""" + + @classmethod + def setUpClass(cls): + cls.station = Workstation('endpoints', ssh_key=False, trust=False) + cls.folder = lab(cls.station) + cls.station.berth('provisioner', 'set', 'box', 'place=attic') + cls.serve = Serve(cls.station) + cls.portal = cls.serve.portal() + cls.port = echo() + body = {'provisioner': 'box', 'kind': 'deployment', 'name': 'db', 'spec': f'listen: {cls.port}\n'} + created = cls.serve.call('POST', '/v1/resources', body, expect=202) + cls.rid = created['resource']['id'] + _, code = job_output(cls.portal, created['job']) + assert code == 0, code + + @classmethod + def tearDownClass(cls): + cls.portal.close() + cls.serve.stop() + + def test_its_endpoints_are_kept_with_it(self): + seen = self.serve.call('GET', f'/v1/resources/{self.rid}') + self.assertEqual(seen['endpoints'], [{'name': 'bench', 'address': '127.0.0.1', 'port': self.port, + 'protocol': 'tcp', 'hint': 'echo', 'local': True}]) + + def test_an_endpoint_is_reached_on_its_own_stream_and_over_the_portal(self): + data = os.urandom(1 << 20) + self.assertEqual(through_endpoint(self.serve, self.rid, 'bench', b'hello'), b'hello') + self.assertEqual(through_endpoint(self.serve, self.rid, str(self.port), data), data) + self.assertEqual(through_portal(self.portal, self.rid, 'bench', data), data) + # Only what the provisioner reported is opened. + with self.assertRaisesRegex(PermissionError, '^404 '): + WebSocket(self.serve.port, f'/v1/resources/{self.rid}/endpoints/22', protocols=[self.serve.token]) + refused = self.portal.call('resources.connect', {'id': self.rid, 'endpoint': '22'}) + self.assertEqual(refused['error']['code'], 'not_found') + + def test_a_command_runs_in_it_as_a_job_that_ends_as_the_command_did(self): + acted = self.serve.call('POST', f'/v1/resources/{self.rid}/actions/exec', + {'arguments': {'command': 'count --all', 'stdin': 'benches'}}, expect=202) + output, code = job_output(self.portal, acted['job']) + self.assertEqual(code, 4, output) + self.assertIn("ran count --all reading 'benches'", output) + # The output is the command's alone. + self.assertNotIn('exited with', output) + + def test_the_command_line_shows_its_endpoints(self): + page = self.station.berth('resource', 'show', 'db').stdout + self.assertIn(f'Endpoints (berth port forward {self.rid} ENDPOINT)', page) + self.assertRegex(page, rf'(?m)^ bench 127\.0\.0\.1:{self.port}/tcp echo · on the server\'s machine$') + shown = json.loads(self.station.berth('resource', 'show', 'db', '-o', 'json').stdout) + self.assertEqual([endpoint['name'] for endpoint in shown['endpoints']], ['bench']) + + def test_the_command_line_runs_a_command_in_it_with_its_exit_status(self): + ran = self.station.berth('resource', 'exec', 'db', '--stdin', '--', 'count', '--all', + input='benches', check=False) + self.assertEqual(ran.returncode, 4, ran.output) + self.assertIn("ran count --all reading 'benches'", ran.stdout) + self.assertNotIn('exited with', ran.stdout) + + def test_port_forward_reaches_an_endpoint_by_its_name(self): + free = socket.create_server(('127.0.0.1', 0)) + local = free.getsockname()[1] + free.close() + forward = self.station.spawn('port', 'forward', 'db', f'{local}:bench') + try: + said = forward.stdout.readline() + self.assertIn(f'127.0.0.1:{local} → db:bench', said) + with socket.create_connection(('127.0.0.1', local), timeout=10) as connection: + connection.sendall(b'hello') + connection.shutdown(socket.SHUT_WR) + got = b'' + while data := connection.recv(65536): + got += data + self.assertEqual(got, b'hello') + finally: + forward.terminate() + forward.wait(timeout=10) + # A name that is no host and no resource says so. + refused = self.station.berth('port', 'forward', 'nowhere', '80', check=False) + self.assertNotEqual(refused.returncode, 0) + self.assertIn('There is no host or resource nowhere', refused.stderr) @needs_e2e @unittest.skipUnless(os.environ.get('BERTH_E2E_DOCKER_HOST'), 'no Docker engine on the test network') class DockerEngine(unittest.TestCase): - """The Docker plugin's actions that only read, against the engine of the test network.""" + """The Docker plugin against the engine of the test network.""" def test_the_engine_answers_and_lists_its_runtimes(self): station = Workstation('provisioners-docker', ssh_key=False, trust=False) endpoint = os.environ['BERTH_E2E_DOCKER_HOST'] station.berth('provisioner', 'add', 'docker', 'docker/container', f'host={endpoint}') - page = station.berth('provisioner', 'docker').stdout + page = station.berth('provisioner', 'show', 'docker').stdout self.assertIn(f'answers at {endpoint}', page) self.assertIn('Runtimes', page) - volumes = station.berth('provisioner', 'docker', 'run', 'volumes').stdout + volumes = station.berth('provisioner', 'run', 'docker', 'volumes').stdout self.assertIn(f'volumes at {endpoint}', volumes) def test_a_compose_deployment_is_made_changed_and_removed(self): @@ -289,19 +452,58 @@ def test_a_compose_deployment_is_made_changed_and_removed(self): ' command: ["sh", "-c", "echo token-is-$$TOKEN; sleep 600"]\n' ' environment:\n TOKEN: ${TOKEN}\n') name = 'berth-test-e2e' - made = station.berth('deploy', name, '--provisioner', 'docker', '--spec', '-', - '--secret', 'compose_token:TOKEN', input=spec, timeout=300) + made = station.berth('resource', 'create', name, '--provisioner', 'docker', '--spec', '-', '--secret', 'compose_token:TOKEN', input=spec, timeout=300) self.assertNotIn('c0mpose-t0ken', made.stdout + made.stderr) try: - self.assertIn('running', station.berth('resource', name).stdout) - log = station.berth('resource', name, 'logs', '--tail', '20').stdout + self.assertIn('running', station.berth('resource', 'show', name).stdout) + log = station.berth('resource', 'logs', name, '--tail', '20').stdout self.assertIn('token-is-[secret]', log) - station.berth('resource', name, 'change', '--spec', '-', input=spec.replace('600', '601'), timeout=300) - self.assertIn('running', station.berth('resource', name).stdout) + station.berth('resource', 'set', name, '--spec', '-', input=spec.replace('600', '601'), timeout=300) + self.assertIn('running', station.berth('resource', 'show', name).stdout) finally: - removed = station.berth('resource', name, 'remove', '--yes', timeout=300) + removed = station.berth('resource', 'remove', name, '-y', timeout=300) self.assertIn(f'Removed the deployment {name}', removed.stdout) + def test_an_instance_is_reached_at_its_published_port_and_runs_a_command(self): + station = Workstation('provisioners-instance', ssh_key=False, trust=False) + endpoint = os.environ['BERTH_E2E_DOCKER_HOST'] + station.berth('provisioner', 'add', 'docker', 'docker/container', f'host={endpoint}') + serve = Serve(station) + portal = serve.portal() + command = "sh -c 'mkdir -p /www && echo hello-from-box > /www/index.html && httpd -f -p 8080 -h /www'" + body = {'provisioner': 'docker', 'kind': 'instance', 'name': 'berth-test-web', + 'arguments': {'image': 'busybox:1.37', 'command': command, 'ports': '8080'}} + created = serve.call('POST', '/v1/resources', body, expect=202) + rid = created['resource']['id'] + try: + output, code = job_output(portal, created['job']) + self.assertEqual(code, 0, output) + # The engine is elsewhere, so the port Docker chose is at the engine's own name. + [published] = serve.call('GET', f'/v1/resources/{rid}')['endpoints'] + host = endpoint.split('://', 1)[1].rsplit(':', 1)[0] + self.assertEqual((published['address'], published['protocol']), (host, 'tcp')) + self.assertNotIn('local', published) + answer = through_endpoint(serve, rid, published['name'], b'GET / HTTP/1.0\r\n\r\n') + self.assertIn(b'200 OK', answer) + self.assertIn(b'hello-from-box', answer) + answer = through_portal(portal, rid, str(published['port']), b'GET / HTTP/1.0\r\n\r\n') + self.assertIn(b'hello-from-box', answer) + acted = serve.call('POST', f'/v1/resources/{rid}/actions/exec', + {'arguments': {'command': '["sh", "-c", "cat /www/index.html; exit 5"]'}}, + expect=202) + output, code = job_output(portal, acted['job']) + self.assertEqual(code, 5, output) + self.assertIn('hello-from-box', output) + # The same from the command line: its output and its exit status are the command's. + ran = station.berth('resource', 'exec', rid, '--', 'sh', '-c', 'cat /www/index.html; exit 5', + check=False) + self.assertEqual((ran.returncode, ran.stdout.strip()), (5, 'hello-from-box'), ran.output) + finally: + removed = serve.call('DELETE', f'/v1/resources/{rid}', expect=202) + job_output(portal, removed['job']) + portal.close() + serve.stop() + if __name__ == '__main__': unittest.main() diff --git a/tests/e2e/e2e_restore.py b/tests/e2e/e2e_restore.py index a07d157..4cbe8a5 100644 --- a/tests/e2e/e2e_restore.py +++ b/tests/e2e/e2e_restore.py @@ -56,18 +56,18 @@ def setUpClass(cls): folder.mkdir(mode=0o700) station.env.update(BERTH_STORE=str(keep / 'berth.db'), BERTH_STORE_KEY=str(keep / 'store.key')) # What the store says of itself when it is sealed. - cls.encrypted = station.berth('store', 'encrypt') + cls.encrypted = station.berth('server', 'store', 'encrypt') cls.own_public = station.own_public() cls.give_the_host_the_keys() key = station.own_key_file() wait_for('the operator taking the key of the server', lambda: station.ssh( OPERATOR, 'true', key=key, check=False).returncode == 0) - serve = Serve(station, web=False) + serve = Serve(station) try: cls.fill(serve) finally: serve.stop() - cls.status = station.berth('store').stdout + cls.status = station.berth('server', 'store', 'status').stdout cls.sealed_file = (keep / 'berth.db').read_bytes() # The database and the key file are all that is kept. for name in ('berth.db', 'store.key'): @@ -76,7 +76,7 @@ def setUpClass(cls): wipe(station) shutil.rmtree(keep) station.env.update(BERTH_STORE=str(cls.moved / 'berth.db'), BERTH_STORE_KEY=str(cls.moved / 'store.key')) - cls.serve = Serve(station, web=False) + cls.serve = Serve(station) @classmethod def give_the_host_the_keys(cls): @@ -93,7 +93,7 @@ def fill(cls, serve): host = cls.worker.read_profile() host['access']['auth'] = 'berth' host['ansible_vars'] = light_vars() - for key in ('secrets', 'registries', 'tailscale_auto_key', 'tailscale_auth_key', 'tailscale_login_server'): + for key in ('secrets', 'registries'): host.pop(key, None) serve.call('PUT', f'/v1/hosts/{HOST}/profile', host) wait_for('the host answering through the server', lambda: [ @@ -155,8 +155,8 @@ def test_4_the_secret_and_the_setting_are_there(self): profile['secrets'] = [{'name': self.secret, 'env': 'RESTORED_SECRET'}] self.serve.call('PUT', f'/v1/hosts/{HOST}/profile', profile) session = self.serve.call('POST', f'/v1/hosts/{HOST}/actions/secrets', expect=202)['session'] - wait_for('the secrets task to end', lambda: self.serve.call( - 'GET', f'/v1/hosts/{HOST}/sessions/{session["id"]}').get('exit_code') == 0) + wait_for('the secrets action to end', lambda: self.serve.call( + 'GET', f'/v1/hosts/{HOST}/controller/sessions/{session["id"]}').get('exit_code') == 0) env = self.worker.root_on_host(f'cat /home/{AGENT}/.config/devbox/secrets.env').stdout self.assertIn(f'export RESTORED_SECRET={self.secret_value}', env) @@ -183,7 +183,9 @@ def test_6_without_the_key_file_berth_serve_refuses_and_writes_nothing(self): self.assertIn(str(absent), said) self.assertNotIn('Listening on', said) self.assertTrue(digest(database) == before, 'berth serve changed the database') - # SQLite's own files beside the database are no change; a key file or a new database would be. - self.assertEqual({path.name for path in self.keyless.iterdir()} - {'berth.db-wal', 'berth.db-shm'}, {'berth.db'}) + # SQLite's own files and berth's empty lock beside the database are no change; a key file, a copy or a new database would be. + beside = {'berth.db-wal', 'berth.db-shm', 'berth.db.convert.lock'} + self.assertEqual({path.name for path in self.keyless.iterdir()} - beside, {'berth.db'}) + self.assertEqual((self.keyless / 'berth.db.convert.lock').stat().st_size, 0) self.assertFalse((station.data / 'berth/serve.token').exists()) self.assertFalse((station.data / 'berth/keys').exists()) diff --git a/tests/e2e/e2e_secrets.py b/tests/e2e/e2e_secrets.py index 316df01..fd0c778 100644 --- a/tests/e2e/e2e_secrets.py +++ b/tests/e2e/e2e_secrets.py @@ -18,14 +18,14 @@ def setUpClass(cls): berth = cls.station.berth berth('secret', 'set', 'github', input=cls.token + '\n') berth('secret', 'set', 'deploy-key', input=cls.key) - berth('host', HOST, 'secret', 'github', 'env', 'GH_TOKEN') - berth('host', HOST, 'secret', 'deploy-key', 'file', '~/.config/e2e/deploy.key', '0640') - berth('registry', 'add', 'npm-e2e', 'npm', cls.registry.url, 'ci-user') + berth('host', 'set', HOST, 'secret.github=GH_TOKEN') + berth('host', 'set', HOST, 'secret.deploy-key=~/.config/e2e/deploy.key:0640') + berth('registry', 'add', 'npm-e2e', 'npm', cls.registry.url, '--user', 'ci-user') berth('registry', 'login', 'npm-e2e', input=cls.password + '\n') - berth('registry', 'add', 'images', 'docker', 'https://192.0.2.20:5000', 'pusher') + berth('registry', 'add', 'images', 'docker', 'https://192.0.2.20:5000', '--user', 'pusher') berth('registry', 'login', 'images', input='image-password\n') for name in ('npm-e2e', 'images'): - berth('host', HOST, 'registry', name, 'on') + berth('host', 'set', HOST, f'registry.{name}=on') @classmethod def tearDownClass(cls): @@ -38,13 +38,13 @@ def agent_file(self, path): return self.station.root_on_host(f'cat /home/{AGENT}/{path}').stdout def test_secret_values_stay_out_of_listings_and_the_profile(self): - listed = self.station.berth('secret').stdout + listed = self.station.berth('secret', 'list').stdout self.assertIn('github', listed) self.assertNotIn(self.token, listed) - self.assertNotIn(self.token, self.station.berth('profile', HOST).stdout) - given = self.station.berth('host', HOST).stdout - self.assertRegex(given, r'secret\s+github\s+env GH_TOKEN') - self.assertRegex(given, r'registry\s+npm-e2e') + self.assertNotIn(self.token, self.station.berth('host', 'export', HOST).stdout) + given = json.loads(self.station.berth('host', 'export', HOST).stdout) + self.assertIn({'name': 'github', 'env': 'GH_TOKEN'}, given['secrets']) + self.assertIn('npm-e2e', given['registries']) def test_a_registry_login_is_checked(self): checked = self.station.berth('registry', 'check', 'npm-e2e').stdout @@ -52,8 +52,9 @@ def test_a_registry_login_is_checked(self): self.assertIn('✓ Login: accepted', checked) self.station.berth('registry', 'login', 'npm-e2e', input='not-the-password\n') try: - checked = self.station.berth('registry', 'check', 'npm-e2e').stdout - self.assertIn('login not accepted', checked) + refused = self.station.berth('registry', 'check', 'npm-e2e', check=False) + self.assertEqual(refused.returncode, 1) + self.assertIn('login not accepted', refused.stdout) finally: self.station.berth('registry', 'login', 'npm-e2e', input=self.password + '\n') @@ -77,10 +78,10 @@ def test_the_secrets_task_installs_them_for_the_agent(self): self.assertEqual(shown.strip(), self.token[:15]) def test_a_secret_turned_off_is_gone_after_the_next_send(self): - self.station.berth('host', HOST, 'secret', 'github', 'off') + self.station.berth('host', 'set', HOST, 'secret.github=') try: self.station.berth('run', HOST, 'secrets') self.assertNotIn('GH_TOKEN', self.agent_file('.config/devbox/secrets.env')) finally: - self.station.berth('host', HOST, 'secret', 'github', 'env', 'GH_TOKEN') + self.station.berth('host', 'set', HOST, 'secret.github=GH_TOKEN') self.station.berth('run', HOST, 'secrets') diff --git a/tests/e2e/e2e_serve.py b/tests/e2e/e2e_serve.py index 84e07fd..6a8841f 100644 --- a/tests/e2e/e2e_serve.py +++ b/tests/e2e/e2e_serve.py @@ -1,6 +1,5 @@ """berth serve over TCP: its routes and the portal, the roles of access -tokens, setting the server up through the API, applying as a job, and the -page for a browser. +tokens, setting the server up through the API, and applying as a job. All but ApplyJob use berth serve as harness.served() starts it. ApplyJob looks at the job of the server's own shared state (harness.served_extended()).""" @@ -37,18 +36,18 @@ def test_nothing_without_the_token(self): self.assertEqual(answer['error']['code'], 'unauthorized') def test_a_hosts_operations_go_through(self): - started = self.serve.call('POST', f'/v1/hosts/{HOST}/sessions?wait=30s', + started = self.serve.call('POST', f'/v1/hosts/{HOST}/controller/sessions?wait=30s', {'kind': 'process', 'command': ['id', '-un']}) self.assertEqual((started['exit_code'], text_of(started)), (0, f'{AGENT}\n')) - tasks = self.serve.call('GET', f'/v1/hosts/{HOST}/tasks') + tasks = self.serve.call('GET', f'/v1/hosts/{HOST}/controller/actions') self.assertIn('secrets', [task['name'] for task in tasks]) - path = f'/v1/hosts/{HOST}/files?path=/home/{AGENT}/through-serve.txt&overwrite=true' + path = f'/v1/hosts/{HOST}/controller/files?path=/home/{AGENT}/through-serve.txt&overwrite=true' status, _ = self.serve.request('PUT', path, b'written through berth serve', headers={'Content-Type': 'application/octet-stream'}) self.assertEqual(status, 200) status, read, _ = self.serve.request('GET', path.split('&')[0], raw=True) self.assertEqual((status, read), (200, b'written through berth serve')) - status, _ = self.serve.request('GET', '/v1/hosts/nowhere/sessions') + status, _ = self.serve.request('GET', '/v1/hosts/nowhere/controller/sessions') self.assertEqual(status, 404) def test_the_key_value_store(self): @@ -68,7 +67,7 @@ def test_the_key_value_store(self): def test_the_inbox_takes_in_what_the_host_keeps(self): body = {'note': random_text('inbox')} - self.serve.call('POST', f'/v1/hosts/{HOST}/sessions?wait=30s', { + self.serve.call('POST', f'/v1/hosts/{HOST}/controller/sessions?wait=30s', { 'kind': 'process', 'command': ['box', 'publish', 'e2e/inbox', json.dumps(body), '--keep', 'log']}) def arrived(): @@ -135,9 +134,9 @@ def test_each_role_may_do_what_it_names(self): process = {'kind': 'process', 'command': ['true']} asked = [ ('GET', '/v1/hosts', None, 'view'), - ('GET', f'/v1/hosts/{HOST}/sessions', None, 'view'), - ('POST', f'/v1/hosts/{HOST}/sessions?wait=10s', process, 'operate'), - ('GET', f'/v1/hosts/{HOST}/files?path=/home/{AGENT}', None, 'operate'), + ('GET', f'/v1/hosts/{HOST}/controller/sessions', None, 'view'), + ('POST', f'/v1/hosts/{HOST}/controller/sessions?wait=10s', process, 'operate'), + ('GET', f'/v1/hosts/{HOST}/controller/files?path=/home/{AGENT}', None, 'operate'), ('GET', '/v1/secrets', None, 'admin'), ('GET', f'/v1/hosts/{HOST}/profile', None, 'admin'), ] @@ -217,8 +216,11 @@ def test_what_a_new_host_starts_from(self): self.assertTrue(hashed['value'].startswith('$6$')) def test_a_host_is_made_seeded_and_removed(self): + profile = provisioned().read_profile() + # A seed of a host that joins the tailnet needs a key no test network can make. + apart = {**profile.get('ansible_vars', {}), 'devbox_install_tailscale': False} self.serve.call('PUT', '/v1/hosts/cloud-2/profile', { - **provisioned().read_profile(), 'hostname': 'cloud-2', 'builder': {}, 'access': {}, + **profile, 'hostname': 'cloud-2', 'builder': {}, 'access': {}, 'ansible_vars': apart, 'operator_password_hash': '$6$e2e$' + 'x' * 86, 'release_url': 'https://192.0.2.30/berth-release.tar.gz'}) self.assertFalse(self.serve.call('GET', '/v1/hosts/cloud-2')['reachable']) @@ -239,8 +241,8 @@ def test_the_server_sends_its_secrets_to_the_host(self): self.serve.call('PUT', f'/v1/hosts/{HOST}/profile', profile) try: session = self.serve.call('POST', f'/v1/hosts/{HOST}/actions/secrets', expect=202)['session'] - wait_for('the secrets task to end', lambda: self.serve.call( - 'GET', f'/v1/hosts/{HOST}/sessions/{session["id"]}').get('exit_code') == 0) + wait_for('the secrets action to end', lambda: self.serve.call( + 'GET', f'/v1/hosts/{HOST}/controller/sessions/{session["id"]}').get('exit_code') == 0) env = provisioned().root_on_host(f'cat /home/{AGENT}/.config/devbox/secrets.env').stdout self.assertIn(f'export SERVER_TOKEN={value}', env) finally: @@ -285,35 +287,11 @@ def test_applying_is_a_job_of_the_server(self): class ClosedSetUp(unittest.TestCase): def test_set_up_can_be_closed_to_the_network(self): station = Workstation('closed', ssh_key=False) - closed = Serve(station, web=False, env={'BERTH_SERVE_SETUP': 'off'}) + closed = Serve(station, env={'BERTH_SERVE_SETUP': 'off'}) try: self.assertEqual(closed.call('GET', '/v1/access'), {'role': 'operate', 'setup': False}) status, answer = closed.request('PUT', '/v1/secrets/x', {'value': 'y'}) self.assertEqual(status, 403) self.assertIn('set-up', answer['error']['message']) - # The page is only there with --web. - status, _ = closed.request('GET', '/', token='') - self.assertNotEqual(status, 200) finally: closed.stop() - - -@needs_e2e -class Page(unittest.TestCase): - @classmethod - def setUpClass(cls): - cls.serve = served() - - def test_the_page_is_given_to_anyone(self): - status, page, headers = self.serve.request('GET', '/', token='', raw=True) - self.assertEqual(status, 200) - self.assertTrue(headers['content-type'].startswith('text/html'), headers) - self.assertIn(b'berth-web', page) - status, script, headers = self.serve.request('GET', '/berth-web.js', token='', raw=True) - self.assertEqual(status, 200) - self.assertIn('javascript', headers['content-type']) - status, wasm, headers = self.serve.request('GET', '/berth-web_bg.wasm', token='', raw=True) - self.assertEqual((status, wasm[:4], headers['content-type']), (200, b'\0asm', 'application/wasm')) - # What the page shows it fetches with a token. - self.assertEqual(self.serve.request('GET', '/v1/hosts', token='')[0], 401) - diff --git a/tests/e2e/e2e_tailnet.py b/tests/e2e/e2e_tailnet.py index 11baa73..1ea040f 100644 --- a/tests/e2e/e2e_tailnet.py +++ b/tests/e2e/e2e_tailnet.py @@ -1,11 +1,12 @@ -"""Joining a tailnet at a Headscale server: with a key berth creates when the -host needs one, through berth and through berth serve, and with a key pasted -into the profile. The test host runs the real tailscale client. - -Each way of joining is a shared state of the host, which leaves the tailnet -before the next one joins: AutomaticKey is the extended state -(harness.extended()), PastedKey the plain one (harness.plain()) and -ThroughTheServer the one of berth serve (harness.served_extended()).""" +"""Joining a tailnet at a Headscale server, with a key berth creates when the +host needs one, through berth and through berth serve; and a host an older +berth left joined by hand, which stays as it is. The test host runs the real +tailscale client. + +Each is a shared state of the host, which leaves the tailnet before the next +one joins: AutomaticKey is the extended state (harness.extended()), +JoinedByHand the plain one (harness.plain()) and ThroughTheServer the one of +berth serve (harness.served_extended()).""" import json import sqlite3 import time @@ -79,17 +80,34 @@ def test_the_host_knows_its_name_on_the_tailnet(self): @needs_e2e -class PastedKey(Joined, unittest.TestCase): - """A key made by hand and pasted into the profile, with the login server.""" +class JoinedByHand(unittest.TestCase): + """A host joined by hand, with the profile of an older berth that held a + pasted key, and no API key of the tailnet on the workstation.""" @classmethod def setUpClass(cls): applied = plain() - cls.station, cls.before = applied.station, applied.before + cls.station, cls.first, cls.saved, cls.refused = applied.station, applied.first, applied.saved, applied.refused + cls.nodes, cls.after, cls.before = applied.nodes, applied.after, applied.before - def test_the_host_joins_with_the_pasted_key(self): - self.assert_joined(self.before) - self.assertEqual(last_key_id(), self.before + 1) + def test_a_host_on_the_tailnet_is_left_on_it(self): + self.assertIn('failed=0', self.first.stdout) + self.assertEqual([node['id'] for node in self.after], [node['id'] for node in self.nodes]) + self.assertEqual([node['givenName'] for node in self.after], [HOST]) + self.assertNotIn('Consume enrollment key', changed(self.first.stdout)) + self.assertEqual(last_key_id(), self.before) + + def test_the_pasted_key_is_not_kept(self): + for old in ('tailscale_auth_key', 'tailscale_auto_key', 'tailscale_login_server'): + self.assertNotIn(old, self.saved) + self.assertNotIn('devbox_tailscale_auth_key', self.saved.get('ansible_vars', {})) + + def test_off_the_tailnet_without_the_api_key_nothing_runs(self): + output = self.refused.stdout + self.refused.stderr + self.assertNotEqual(self.refused.returncode, 0, output) + self.assertIn('berth secret set tailnet_api_key', output) + self.assertIn('switch Tailscale off', output) + self.assertEqual(changed(self.refused.stdout), []) @needs_e2e @@ -129,14 +147,16 @@ def test_the_host_joins_and_applying_again_makes_no_key(self): self.assertNotIn('Consume enrollment key', changed(self.second)) def test_a_seed_carries_a_key_made_for_it(self): - draft = {**self.kept, 'hostname': 'cloud-5', 'builder': {}, 'access': {}, 'tailscale_auto_key': True, + draft = {**self.kept, 'hostname': 'cloud-5', 'builder': {}, 'access': {}, 'operator_password_hash': '$6$e2e$' + 'x' * 86, 'release_url': 'https://192.0.2.30/berth-release.tar.gz'} self.serve.call('PUT', '/v1/hosts/cloud-5/profile', joining(draft)) try: before = last_key_id() - written = self.client.berth('seed', 'cloud-5').stdout.strip() - seed = open(written).read() + seeded = self.client.berth('host', 'seed', 'cloud-5') + # Whoever starts the machine learns until when it can join. + self.assertIn('joins the tailnet until', seeded.stderr) + seed = seeded.stdout made = [key for key in headscale().keys() if int(key['id']) > before] self.assertEqual(len(made), 1) self.assertFalse(made[0]['used']) @@ -146,6 +166,7 @@ def test_a_seed_carries_a_key_made_for_it(self): self.assertIn(headscale().url, seed) self.assertNotIn(headscale().api_key, seed) # The profile keeps no key: the next seed gets a new one. - self.assertEqual(self.serve.call('GET', '/v1/hosts/cloud-5/profile').get('tailscale_auth_key', ''), '') + self.assertNotIn(made[0]['key'].removesuffix('***'), + json.dumps(self.serve.call('GET', '/v1/hosts/cloud-5/profile'))) finally: self.serve.call('DELETE', '/v1/hosts/cloud-5') diff --git a/tests/e2e/e2e_update.py b/tests/e2e/e2e_update.py index e236436..9711f7d 100644 --- a/tests/e2e/e2e_update.py +++ b/tests/e2e/e2e_update.py @@ -7,14 +7,19 @@ (harness.extended()), which applied `up` 1.0.0 from the registry and the shipped `shipit`; Update leaves `up` at 2.0.0 and the host current, which ShippedChanged starts from. UpdateServed uses the shared state of berth serve -(harness.served_extended()), which applied `served` 1.0.0.""" +(harness.served_extended()), which applied `served` 1.0.0. FromBerth052 needs +no host: it opens a store that berth 0.5.2 wrote.""" import json +import shutil import unittest +from pathlib import Path from harness import (AGENT, HOST, SCRATCH, Workstation, extended, light_vars, needs_e2e, package_registry, plugin_version, random_text, served_extended, unb64, wait_for) PACKAGE = 'berth-plugin-up' +# A store berth 0.5.2 wrote, with made-up hosts and fake keys. +STORE_052 = Path(__file__).resolve().parents[2] / 'crates/berth-core/tests/fixtures/store-0.5.2.db' def version_of(applied, plugin): @@ -54,8 +59,8 @@ def setUpClass(cls): @classmethod def tearDownClass(cls): cls.station.remove_host('alpha', check=False) - cls.station.berth('host', HOST, 'secret', 'up-key', 'off', check=False) - cls.station.berth('secret', 'remove', 'up-key', check=False) + cls.station.berth('host', 'set', HOST, 'secret.up-key=', check=False) + cls.station.berth('secret', 'remove', 'up-key', '-y', check=False) def run_as_agent(self, command): return self.station.berth('exec', HOST, '--', 'bash', '-c', command).stdout @@ -76,22 +81,23 @@ def test_an_update_reaches_the_host_and_keeps_what_was_there(self): kept.plant(station, self.run_as_agent) secret = random_text('up-secret') station.berth('secret', 'set', 'up-key', input=secret + '\n') - station.berth('host', HOST, 'secret', 'up-key', 'file', '~/.config/e2e/up.key', '0600') + station.berth('host', 'set', HOST, 'secret.up-key=~/.config/e2e/up.key:0600') station.berth('run', HOST, 'secrets') session = station.berth('exec', HOST, '--keep', '--', 'echo', f'session {kept.mark}') self.assertIn(kept.mark, session.stdout) # A newer version in the registry. self.registry.publish(plugin_version('2.0.0')) - checked = station.berth('plugins', 'update', '--check', env=self.npm).stdout - self.assertRegex(checked, r'(?m)^up\s+1\.0\.0 → 2\.0\.0$') - updated = station.berth('plugins', 'update', 'up', env=self.npm).stdout + checked = station.berth('plugin', 'update', '--check', env=self.npm).stdout + self.assertRegex(checked, r'(?m)^up\s+1\.0\.0\s+2\.0\.0\s*$') + updated = station.berth('plugin', 'update', 'up', env=self.npm).output self.assertIn('✓ up 1.0.0 → 2.0.0', updated) self.assertIn(f'{HOST} needs applying: up 1.0.0 → 2.0.0', updated) - self.assertRegex(station.berth('plugins').stdout, rf'(?m)^up {PACKAGE} 2\.0\.0 ') + plugins = json.loads(station.berth('plugin', 'list', '-o', 'json').stdout)['plugins'] + self.assertIn(('up', PACKAGE, '2.0.0'), [(p['id'], p['package'], p['version']) for p in plugins]) # The overview says so too. - overview = station.terminal(HOST, cols=160) + overview = station.terminal('ui', HOST, cols=160) try: overview.expect(r'Applied before these changed: up 1\.0\.0 → 2\.0\.0', timeout=60) finally: @@ -100,9 +106,8 @@ def test_an_update_reaches_the_host_and_keeps_what_was_there(self): # A roll-out that fails at its first host leaves the next as it was. alpha = station.read_profile() alpha['ansible_vars'] = light_vars(['up/missing']) - alpha.pop('tailscale_auto_key', None) station.write_profile(alpha, name='alpha') - stopped = station.berth('apply', '--behind', check=False, timeout=900) + stopped = station.berth('rollout', check=False, timeout=900) self.assertEqual(stopped.returncode, 1, stopped.stdout) self.assertRegex(stopped.stdout, r'(?m)^alpha: failed: applying failed') self.assertRegex(stopped.stdout, rf'(?m)^{HOST}: not started$') @@ -110,16 +115,16 @@ def test_an_update_reaches_the_host_and_keeps_what_was_there(self): station.remove_host('alpha') # Brought up to date. - rolled = station.berth('apply', '--behind', timeout=900).stdout + rolled = station.berth('rollout', timeout=900).stdout self.assertIn(f'▸ {HOST}: applying (up 1.0.0 → 2.0.0)', rolled) self.assertIn(f'✓ {HOST} is up to date', rolled) self.assertEqual(version_of(self.applied(), 'up'), '2.0.0') - self.assertIn(f'✓ {HOST} is current', station.berth('apply', '--behind').stdout) + self.assertIn(f'✓ {HOST} is current', station.berth('rollout').stdout) # What changed is there, and what went is gone. self.assertEqual(station.berth('run', HOST, 'up:hello').stdout.strip(), 'hello from 2.0.0') self.assertEqual(station.berth('run', HOST, 'up:added').stdout.strip(), 'added task') - tasks = station.berth('tasks', HOST).stdout + tasks = station.berth('action', 'list', HOST).stdout self.assertNotIn('up:old', tasks) services = json.loads(station.root_on_host('cat /etc/berth/services.json').stdout) ours = [service['run'] for service in services if 'box serve up/echo' in service['run']] @@ -132,7 +137,7 @@ def test_an_update_reaches_the_host_and_keeps_what_was_there(self): # And what the agent had is still there. kept.check(self, self.run_as_agent) self.assertEqual(self.run_as_agent('cat ~/.config/e2e/up.key').strip(), secret) - sessions = station.berth('sessions', HOST).stdout + sessions = station.berth('session', 'list', HOST).stdout self.assertIn(f'session {kept.mark}', sessions) notes = self.link.get('/v1/channel?topic=up/notes') messages = notes['messages'] if isinstance(notes, dict) else notes @@ -149,8 +154,8 @@ class ShippedChanged(unittest.TestCase): def setUpClass(cls): cls.station = station = extended().station cls.link = station.link() - cls.current = station.berth('apply', '--behind').stdout - cls.refused = station.berth('plugins', 'update', 'shipit', check=False).stderr + cls.current = station.berth('rollout').stdout + cls.refused = station.berth('plugin', 'update', 'shipit', check=False).stderr # The same version, another task. source = SCRATCH / 'source-shipped/plugins/shipit/package.json' manifest = json.loads(source.read_text()) @@ -159,7 +164,7 @@ def setUpClass(cls): station.root_on_host('touch /var/run/reboot-required') sleeper = cls.link.post('/v1/sessions', {'kind': 'process', 'title': 'busy', 'command': ['sleep', '600']}) try: - cls.rolled = station.berth('apply', '--behind', '--reboot', check=False, timeout=900) + cls.rolled = station.berth('rollout', '--reboot', check=False, timeout=900) finally: cls.link.post(f'/v1/sessions/{sleeper["id"]}/signal', {'signal': 'KILL'}) station.root_on_host('rm -f /var/run/reboot-required') @@ -189,8 +194,7 @@ def setUpClass(cls): assert ended['exit_code'] == 0, unb64( cls.serve.call('GET', f'/v1/jobs/{applied.job["id"]}/output')['data_b64'])[-3000:] cls.station = Workstation('update-client', ssh_key=False) - cls.station.berth('config', 'server', cls.serve.url) - cls.station.berth('config', 'server_token', cls.serve.token) + cls.station.berth('login', cls.serve.url, '--token-stdin', input=cls.serve.token) def host(self): return self.serve.call('GET', f'/v1/hosts/{HOST}') @@ -209,7 +213,7 @@ def run(command): self.registry.publish(plugin_version('2.0.0', 'berth-plugin-served')) # The tab of the terminal client asks the server. It lists every plugin # with its features before the row that adds one, so it is tall. - home = station.terminal(cols=160, rows=46) + home = station.terminal('ui', cols=160, rows=46) try: home.expect(HOST) home.send('6') @@ -218,7 +222,7 @@ def run(command): home.expect(r'update available: 1\.0\.0 → 2\.0\.0', timeout=60) finally: home.close() - updated = station.berth('plugins', 'update').stdout + updated = station.berth('plugin', 'update').output self.assertIn('✓ served 1.0.0 → 2.0.0', updated) self.assertIn(f'{HOST} needs applying: served 1.0.0 → 2.0.0', updated) behind = self.host()['behind'] @@ -226,8 +230,7 @@ def run(command): listed = self.serve.call('GET', '/v1/hosts') self.assertTrue(any(host['name'] == HOST and 'behind' in host for host in listed)) - rolled = station.berth('apply', '--behind', timeout=900) - self.assertIn(f'on the berth server at {self.serve.url}', rolled.stdout) + rolled = station.berth('rollout', timeout=900) self.assertIn(f'✓ {HOST} is up to date', rolled.stdout) self.assertRegex(rolled.stdout, rf'(?m)^{HOST}: up to date$') jobs = self.serve.call('GET', '/v1/jobs') @@ -240,3 +243,38 @@ def run(command): kept.check(self, run) self.assertEqual(self.serve.call('GET', '/v1/kv/update/mark')['value'], {'mark': mark}) self.assertEqual(run('id -un').strip(), AGENT) + + +@needs_e2e +class FromBerth052(unittest.TestCase): + """The first start of this berth on the store of berth 0.5.2 copies it, + converts it and says so once; the next start says nothing.""" + + @classmethod + def setUpClass(cls): + cls.station = Workstation('from-0.5.2', ssh_key=False, trust=False) + cls.store = cls.station.data / 'berth/berth.db' + cls.store.parent.mkdir(parents=True, exist_ok=True, mode=0o700) + shutil.copy(STORE_052, cls.store) + cls.store.chmod(0o600) + + def test_the_store_is_copied_converted_and_left_alone_after(self): + first = self.station.berth('host', 'export', 'studio') + copy = self.store.with_name('berth.db.berth-0.5.2-schema-4') + self.assertIn(f'a copy of it as berth 0.5.2 left it is in {copy}', first.stderr) + self.assertIn('3 profiles say type for level and target for profile now', first.stderr) + self.assertEqual(copy.stat().st_mode & 0o777, 0o600) + studio = json.loads(first.stdout) + self.assertNotIn('level', studio) + self.assertNotIn('tailscale_auth_key', studio) + pier = json.loads(self.station.berth('host', 'export', 'pier').stdout) + self.assertEqual(pier['type'], 'managed') + self.assertTrue(pier['ansible_vars']['devbox_tailnet_optional']) + # The next start finds nothing to convert or copy. + copy.unlink() + again = self.station.berth('host', 'export', 'studio') + self.assertEqual(json.loads(again.stdout), studio) + self.assertNotIn('berth:', again.stderr) + self.assertFalse(copy.exists()) + secrets = self.station.berth('secret', 'list').stdout + self.assertIn('tailnet_api_key', secrets) diff --git a/tests/e2e/harness.py b/tests/e2e/harness.py index c3dec1d..afd56f1 100644 --- a/tests/e2e/harness.py +++ b/tests/e2e/harness.py @@ -135,6 +135,18 @@ def source_with_plugins(name, plugins): # A workstation: a HOME and XDG directories of its own +def follows_a_job(args): + """Whether berth follows a job of its server for these arguments.""" + words = [arg for arg in args if not str(arg).startswith('-')] + first, second = (words + ['', ''])[:2] + return (first in ('apply', 'rollout') + or (first, second) in {('host', 'add'), ('host', 'apply'), ('host', 'adopt'), ('host', 'iso'), ('host', 'create'), + ('host', 'rename'), ('host', 'start'), ('host', 'stop'), ('key', 'trust'), + ('key', 'rotate'), ('resource', 'create'), ('resource', 'set'), + ('resource', 'remove'), ('resource', 'run'), ('provisioner', 'run'), + ('provisioner', 'set')}) + + class Workstation: """A user's machine with berth: its own HOME, XDG directories and SSH key.""" @@ -186,7 +198,7 @@ def own_key_file(self): of a host that is only made for that and removed again.""" probe = 'key-file' self.write_profile({'hostname': probe, 'builder': {}, 'access': {'address': '192.0.2.1'}}, name=probe) - self.berth('inventory', probe) + self.berth('host', 'inventory', probe) self.remove_host(probe) return self.own_key @@ -210,6 +222,11 @@ def trust_host(self): def berth(self, *args, input=None, check=True, timeout=600, env=None): done = subprocess.run([str(BERTH), *args], env={**self.env, **(env or {})}, cwd=self.home, input=input, capture_output=True, text=True, timeout=timeout) + # What a job wrote goes to standard error, and the result to standard + # output; the tests of a command that follows a job read both. + done.output = done.stdout + done.stderr + if follows_a_job(args): + done.stdout = done.output if check and done.returncode != 0: raise AssertionError(f'berth {" ".join(args)} exited {done.returncode}:\n' f'{done.stdout[-4000:]}\n{done.stderr[-4000:]}') @@ -234,15 +251,15 @@ def root_on_host(self, command, check=True): return self.ssh(OPERATOR, f'sudo -n sh -c {shell_quote(command)}', check=check) def write_profile(self, profile, name=HOST): - """Save a host's profile in berth's store, as `berth profile --set` does.""" - self.berth('profile', name, '--set', '-', input=json.dumps(profile, indent=2) + '\n') + """Save a host's profile in berth's store, as `berth host import` does.""" + self.berth('host', 'import', name, '-', input=json.dumps(profile, indent=2) + '\n') def read_profile(self, name=HOST): - return json.loads(self.berth('profile', name).stdout) + return json.loads(self.berth('host', 'export', name).stdout) def remove_host(self, name=HOST, check=True): """Forget a host, with what berth keeps of it.""" - return self.berth('remove', name, check=check) + return self.berth('host', 'remove', name, '-y', check=check) def link(self): return BerthdLink(self) @@ -269,7 +286,7 @@ def profile(**values): """A profile like berth writes for a new host, with the test host's address.""" base = json.loads((SRC / 'provision/config/example.json').read_text()) base.update({ - 'hostname': HOST, 'profile': 'cloud', 'keyboard': 'us', + 'hostname': HOST, 'target': 'cloud', 'keyboard': 'us', # The operator's sudo needs no password, as by default, so berth asks # for none. shipped_plugins says the features are chosen already, as # in every profile berth saves. berth logs in with the user's SSH @@ -329,7 +346,7 @@ def provisioned(): started = time.monotonic() first = station.berth('apply', HOST, timeout=900) station.apply_seconds = time.monotonic() - started - station.first_apply = first.stdout + station.first_apply = first.output wait_for('berthd answering', lambda: station.link().get('/v1/version'), timeout=60) _provisioned['workstation'] = station return station @@ -355,8 +372,8 @@ def served(): wait_for('the operator taking the server\'s key', lambda: server.ssh( OPERATOR, 'true', key=key, check=False).returncode == 0) issuer = Issuer() - serve = Serve(server, web=True, env={'BERTH_OAUTH_ISSUER': issuer.url, 'BERTH_OAUTH_AUDIENCE': 'berth', - **package_registry().env()}) + serve = Serve(server, env={'BERTH_OAUTH_ISSUER': issuer.url, 'BERTH_OAUTH_AUDIENCE': 'berth', + **package_registry().env()}) serve.issuer = issuer # The host's profile as the workstation has it, so applying it from the # server keeps the workstation's keys; the server adds its own. What the @@ -366,7 +383,7 @@ def served(): # The server logs in with its own key; the SSH configuration is its own account's. host['access']['auth'] = 'berth' host['ansible_vars'] = light_vars() - for key in ('secrets', 'registries', 'tailscale_auto_key', 'tailscale_auth_key', 'tailscale_login_server'): + for key in ('secrets', 'registries'): host.pop(key, None) serve.call('PUT', f'/v1/hosts/{HOST}/profile', host) _provisioned['serve'] = serve @@ -493,14 +510,14 @@ def _enter_extended(): station.env['BERTH_SOURCE'] = str(SCRATCH / 'source-shipped' if (SCRATCH / 'source-shipped').exists() else source_with_plugins('source-shipped', { 'shipit': plugin_version('1.0.0', 'berth-plugin-shipit')})) - station.berth('plugins', 'add', str(FIXTURE)) + station.berth('plugin', 'add', str(FIXTURE)) registry.publish(plugin_version('1.0.0')) - station.berth('plugins', 'add', 'berth-plugin-up', env=registry.env()) + station.berth('plugin', 'add', 'berth-plugin-up', env=registry.env()) leave_the_tailnet(station) - station.berth('config', 'tailnet_login_server', headscale().url) - station.berth('config', 'tailnet_user', headscale().user) + station.berth('config', 'set', 'tailnet_login_server', headscale().url) + station.berth('config', 'set', 'tailnet_user', headscale().user) station.berth('secret', 'set', 'tailnet_api_key', input=headscale().api_key + '\n') - station.write_profile(joining({**kept, 'ansible_vars': light_vars(EXTENDED_FEATURES)}, tailscale_auto_key=True)) + station.write_profile(joining({**kept, 'ansible_vars': light_vars(EXTENDED_FEATURES)})) before = last_key_id() first = station.berth('apply', HOST, timeout=900) after_first = last_key_id() @@ -510,38 +527,60 @@ def leave(): leave_the_tailnet(station) station.write_profile(kept) for key in ('tailnet_login_server', 'tailnet_user'): - station.berth('config', key) - station.berth('secret', 'remove', 'tailnet_api_key', check=False) + station.berth('config', 'unset', key) + station.berth('secret', 'remove', 'tailnet_api_key', '-y', check=False) for package in ('berth-plugin-e2e', 'berth-plugin-up'): - station.berth('plugins', 'remove', package, check=False) + station.berth('plugin', 'remove', package, '-y', check=False) return Applied(station=station, kept=kept, first=first, second=second, before=before, after_first=after_first), leave def plain(): """The host applied from the workstation again without the extended - state's plugins, joining the tailnet with a key made by hand and pasted - into the profile. `first` is the apply, `before` the newest Headscale key - before the key was made.""" + state's plugins, as an older berth left it: joined to the tailnet by hand, + with that berth's profile, which held a pasted key and had berth's keys + switched off, and no API key of the tailnet. `first` is that apply, + `nodes` and `after` Headscale's nodes before and after it, and `before` + the newest Headscale key after the host joined. Then the host leaves the tailnet, and `refused` is + the apply that finds it off the tailnet with no way to a key.""" return state('plain', _enter_plain) +def join_by_hand(station): + """Join the host to Headscale with a key made by hand, as one did before + berth made the keys.""" + key = headscale().new_key() + station.root_on_host(f"umask 077; printf %s {shell_quote(key)} > /run/e2e-key; " + f"tailscale up --timeout=60s --auth-key=file:/run/e2e-key " + f"--login-server={shell_quote(headscale().url)}; status=$?; rm -f /run/e2e-key; exit $status") + + def _enter_plain(): station = provisioned() for package in ('berth-plugin-e2e', 'berth-plugin-up'): if (station.berth_dir / 'plugins/node_modules' / package).exists(): - station.berth('plugins', 'remove', package) + station.berth('plugin', 'remove', package, '-y') kept = station.read_profile() leave_the_tailnet(station) - before = last_key_id() + join_by_hand(station) + station.berth('config', 'set', 'tailnet_login_server', headscale().url) + station.berth('config', 'set', 'tailnet_user', headscale().user) station.write_profile(joining({**kept, 'ansible_vars': light_vars()}, tailscale_auth_key=headscale().new_key(), - tailscale_login_server=headscale().url)) + tailscale_auto_key=False, tailscale_login_server=headscale().url)) + nodes, before = headscale().nodes(), last_key_id() first = station.berth('apply', HOST, timeout=900) + after = headscale().nodes() + saved = station.read_profile() + leave_the_tailnet(station) + refused = station.berth('apply', HOST, timeout=900, check=False) def leave(): leave_the_tailnet(station) station.write_profile(kept) - return Applied(station=station, kept=kept, first=first, before=before), leave + for key in ('tailnet_login_server', 'tailnet_user'): + station.berth('config', 'unset', key) + return Applied(station=station, kept=kept, first=first, saved=saved, nodes=nodes, after=after, + before=before, refused=refused), leave def served_extended(): @@ -562,14 +601,13 @@ def _enter_served_extended(): registry.publish(plugin_version('1.0.0', 'berth-plugin-served')) serve.call('POST', '/v1/plugins', {'spec': 'berth-plugin-served'}) client = Workstation('served-client', ssh_key=False) - client.berth('config', 'server', serve.url) - client.berth('config', 'server_token', serve.token) - client.berth('config', 'tailnet_login_server', headscale().url) - client.berth('config', 'tailnet_user', headscale().user) + client.berth('login', serve.url, '--token-stdin', input=serve.token) + client.berth('config', 'set', 'tailnet_login_server', headscale().url) + client.berth('config', 'set', 'tailnet_user', headscale().user) client.berth('secret', 'set', 'tailnet_api_key', input=headscale().api_key + '\n') kept = serve.call('GET', f'/v1/hosts/{HOST}/profile') serve.call('PUT', f'/v1/hosts/{HOST}/profile', - joining({**kept, 'ansible_vars': light_vars(['served/main'])}, tailscale_auto_key=True)) + joining({**kept, 'ansible_vars': light_vars(['served/main'])})) before = last_key_id() first = client.berth('apply', HOST, timeout=900) after_first = last_key_id() @@ -688,11 +726,11 @@ def unb64(text): class Serve: """`berth serve --listen` of a workstation, on a port of its own.""" - def __init__(self, station, web=True, env=None): + def __init__(self, station, env=None): MADE['serve'] += 1 self.station = station self.port = free_port() - args = ['serve', '--listen', f'127.0.0.1:{self.port}'] + (['--web'] if web else []) + args = ['serve', '--listen', f'127.0.0.1:{self.port}'] self.process = station.spawn(*args, env=env) self.lines = [] self._reader = threading.Thread(target=self._read, daemon=True) diff --git a/tests/e2e/run.py b/tests/e2e/run.py index baaf3d1..edf5089 100644 --- a/tests/e2e/run.py +++ b/tests/e2e/run.py @@ -56,16 +56,15 @@ 'e2e_tailnet.AutomaticKey', 'e2e_update.Update', 'e2e_update.ShippedChanged', - # plain(): without them, with a pasted key. + # plain(): without them, joined by hand by an older berth. 'e2e_plugins.PluginRemoved', - 'e2e_tailnet.PastedKey', + 'e2e_tailnet.JoinedByHand', # served(): berth serve, applying nothing. 'e2e_serve.Routes', 'e2e_serve.Portal', 'e2e_serve.Roles', 'e2e_serve.SetUp', 'e2e_serve.ClosedSetUp', - 'e2e_serve.Page', 'e2e_client.Client', 'e2e_client.AccessToken', # served_extended(): applied by the server, with a plugin and the tailnet. @@ -78,8 +77,10 @@ ], # The bare host first: the ways berth logs in, and berth's controller # alone, which leave it as they found it. Then a host that only ever - # takes berth's own key. The provisioners' tests need no host at all. - 'dock': ['e2e_access', 'e2e_controller.Controller', 'e2e_own_key', 'e2e_provisioners'], + # takes berth's own key. The provisioners' tests and the one of a store + # of berth 0.5.2 need no host at all. Last, a node, which stays one. + 'dock': ['e2e_access', 'e2e_controller.Controller', 'e2e_own_key', 'e2e_provisioners', + 'e2e_update.FromBerth052', 'e2e_node'], } diff --git a/tests/e2e/site.yml b/tests/e2e/site.yml index 2b1dd3e..0038bb5 100644 --- a/tests/e2e/site.yml +++ b/tests/e2e/site.yml @@ -2,17 +2,48 @@ # The role as the end-to-end tests apply it: the tasks berth itself talks to # (accounts and keys, plugins, the agent's helpers, berthd and its files), # without the desktop packages, toolchains and browsers that need the internet. -# The tests put this playbook where berth looks for provision/ansible/site.yml. +# A node gets its own tasks of the same kinds, without the packages and the +# Tailscale install that need the internet. The tests put this playbook where +# berth looks for provision/ansible/site.yml. +- name: Configure what berth talks to on a node + hosts: all + become: true + gather_facts: true + tasks: + - name: Set the node up without the internet + when: devbox_level | default('workstation') == 'node' + block: + - name: Check the node and find the account sessions run as + ansible.builtin.include_role: + name: devbox + tasks_from: node_facts.yml + - name: Let berth in with its key, and nobody with a password + ansible.builtin.include_role: + name: devbox + tasks_from: node_ssh.yml + - name: Apply the features, berth's controller and the record + ansible.builtin.include_role: + name: devbox + tasks_from: node_berth.yml + - name: Configure what berth talks to on a host hosts: all become: true gather_facts: true tasks: + - name: End here for a node + ansible.builtin.meta: end_host + when: devbox_level | default('workstation') == 'node' - name: Check target and profile before any changes ansible.builtin.assert: that: - ansible_facts.distribution == 'Ubuntu' - devbox_profile in ['laptop', 'cloud', 'container'] + - name: Check that the host can join the tailnet + ansible.builtin.include_role: + name: devbox + tasks_from: tailnet_check.yml + when: devbox_install_tailscale | default(false) - name: Configure accounts and paths ansible.builtin.include_role: name: devbox diff --git a/tests/fixtures/berth-server-0.5.2.json b/tests/fixtures/berth-server-0.5.2.json new file mode 100644 index 0000000..540455a --- /dev/null +++ b/tests/fixtures/berth-server-0.5.2.json @@ -0,0 +1,243 @@ +{ + "_about": "docker inspect of a berth 0.5.2 server container that the incus-host recipe of 0.5.2 ran, on a Docker daemon with max-size and max-file in daemon.json; paths, ids and the group are made up", + "container": { + "Config": { + "AttachStderr": true, + "AttachStdin": false, + "AttachStdout": true, + "Cmd": [ + "serve", + "--listen", + "0.0.0.0:7777", + "--expose", + "--web" + ], + "Domainname": "", + "Entrypoint": [ + "tini", + "--", + "berth" + ], + "Env": [ + "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", + "XDG_CONFIG_HOME=/config", + "XDG_DATA_HOME=/data", + "TERM=xterm-256color" + ], + "ExposedPorts": { + "7777/tcp": {} + }, + "Hostname": "4d1c0ffee000", + "Image": "ghcr.io/helightdev/berth:0.5.2", + "Labels": {}, + "OpenStdin": false, + "StdinOnce": false, + "Tty": false, + "User": "berth", + "Volumes": { + "/config": {}, + "/data": {} + }, + "WorkingDir": "" + }, + "HostConfig": { + "AutoRemove": false, + "Binds": [ + "/srv/berth/ssh:/home/berth/.ssh", + "/var/run/docker.sock:/var/run/docker.sock", + "/srv/berth/config:/config", + "/srv/berth/data:/data" + ], + "BlkioDeviceReadBps": [], + "BlkioDeviceReadIOps": [], + "BlkioDeviceWriteBps": [], + "BlkioDeviceWriteIOps": [], + "BlkioWeight": 0, + "BlkioWeightDevice": [], + "CapAdd": null, + "CapDrop": null, + "Cgroup": "", + "CgroupParent": "", + "CgroupnsMode": "private", + "ConsoleSize": [ + 0, + 0 + ], + "ContainerIDFile": "", + "CpuCount": 0, + "CpuPercent": 0, + "CpuPeriod": 0, + "CpuQuota": 0, + "CpuRealtimePeriod": 0, + "CpuRealtimeRuntime": 0, + "CpuShares": 0, + "CpusetCpus": "", + "CpusetMems": "", + "DeviceCgroupRules": null, + "DeviceRequests": null, + "Devices": [], + "Dns": null, + "DnsOptions": [], + "DnsSearch": [], + "ExtraHosts": [ + "incus-host:host-gateway" + ], + "GroupAdd": [ + "989" + ], + "IOMaximumBandwidth": 0, + "IOMaximumIOps": 0, + "IpcMode": "private", + "Isolation": "", + "Links": null, + "LogConfig": { + "Config": { + "max-file": "5", + "max-size": "10m" + }, + "Type": "json-file" + }, + "MaskedPaths": [ + "/proc/acpi", + "/proc/asound", + "/proc/interrupts", + "/proc/kcore", + "/proc/keys", + "/proc/latency_stats", + "/proc/sched_debug", + "/proc/scsi", + "/proc/timer_list", + "/proc/timer_stats", + "/sys/devices/virtual/powercap", + "/sys/firmware", + "/sys/devices/system/cpu/cpu0/thermal_throttle" + ], + "Memory": 0, + "MemoryReservation": 0, + "MemorySwap": 0, + "MemorySwappiness": null, + "NanoCpus": 0, + "NetworkMode": "bridge", + "OomKillDisable": false, + "OomScoreAdj": 0, + "PidMode": "", + "PidsLimit": null, + "PortBindings": { + "7777/tcp": [ + { + "HostIp": "127.0.0.1", + "HostPort": "7777" + } + ] + }, + "Privileged": false, + "PublishAllPorts": false, + "ReadonlyPaths": [ + "/proc/bus", + "/proc/fs", + "/proc/irq", + "/proc/sys", + "/proc/sysrq-trigger" + ], + "ReadonlyRootfs": false, + "RestartPolicy": { + "MaximumRetryCount": 0, + "Name": "unless-stopped" + }, + "Runtime": "runc", + "SecurityOpt": null, + "ShmSize": 67108864, + "UTSMode": "", + "Ulimits": [], + "UsernsMode": "", + "VolumeDriver": "", + "VolumesFrom": null + }, + "Id": "4d1c0ffee0000000000000000000000000000000000000000000000000000000", + "Image": "sha256:0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a", + "Mounts": [ + { + "Destination": "/data", + "Mode": "", + "Propagation": "rprivate", + "RW": true, + "Source": "/srv/berth/data", + "Type": "bind" + }, + { + "Destination": "/home/berth/.ssh", + "Mode": "", + "Propagation": "rprivate", + "RW": true, + "Source": "/srv/berth/ssh", + "Type": "bind" + }, + { + "Destination": "/var/run/docker.sock", + "Mode": "", + "Propagation": "rprivate", + "RW": true, + "Source": "/var/run/docker.sock", + "Type": "bind" + }, + { + "Destination": "/config", + "Mode": "", + "Propagation": "rprivate", + "RW": true, + "Source": "/srv/berth/config", + "Type": "bind" + } + ], + "Name": "/berth", + "NetworkSettings": { + "Networks": { + "bridge": { + "Gateway": "172.17.0.1", + "IPAddress": "172.17.0.2", + "MacAddress": "02:42:ac:11:00:02", + "NetworkID": "" + } + } + }, + "State": { + "Running": true, + "Status": "running" + } + }, + "image": { + "Config": { + "ArgsEscaped": true, + "Cmd": [ + "serve", + "--listen", + "0.0.0.0:7777", + "--expose", + "--web" + ], + "Entrypoint": [ + "tini", + "--", + "berth" + ], + "Env": [ + "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", + "XDG_CONFIG_HOME=/config", + "XDG_DATA_HOME=/data", + "TERM=xterm-256color" + ], + "ExposedPorts": { + "7777/tcp": {} + }, + "User": "berth", + "Volumes": { + "/config": {}, + "/data": {} + } + }, + "Id": "sha256:0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a", + "RepoTags": [ + "ghcr.io/helightdev/berth:0.5.2" + ] + } +} diff --git a/tests/test_controller_guard.py b/tests/test_controller_guard.py index 7c4aa5a..9913931 100644 --- a/tests/test_controller_guard.py +++ b/tests/test_controller_guard.py @@ -60,6 +60,12 @@ def test_a_workstation_is_refused(self): code, said = self.run_guard({'applied': {'berth': '0.5.0', 'plugins': [], 'level': 'workstation'}}) self.assertNotEqual(code, 0) + def test_a_node_is_refused(self): + code, said = self.run_guard({'applied': {'berth': '0.6.0', 'plugins': [], 'level': 'node'}, + 'berthd': {'agent': 'ops'}}) + self.assertNotEqual(code, 0) + self.assertIn('is a node that berth set up', said) + def test_a_controller_that_serves_another_account_is_refused(self): code, said = self.run_guard({'berthd': {'agent': 'agent', 'operator': 'ops'}}) self.assertNotEqual(code, 0) diff --git a/tests/test_devbox.py b/tests/test_devbox.py index d56306a..83661f5 100755 --- a/tests/test_devbox.py +++ b/tests/test_devbox.py @@ -20,11 +20,18 @@ def load(name,path): guard=load('guard','provision/bin/check-target-disk.py') def config(): - return {'hostname':'test-devbox','profile':'cloud', + return {'hostname':'test-devbox','target':'cloud', 'operator_ssh_public_key':'ssh-ed25519 '+base64.b64encode(b'\0'*64).decode(), 'operator_password_hash':'$6$salt$'+'a'*86,'target_disk_serial':'DEVBOX_TEST_DISK', 'release_url':'https://example.com/devbox.tar.gz', - 'ansible_vars':{'devbox_bootstrap_ssh_cidr':'10.0.2.2/32'}} + 'ansible_vars':{'devbox_bootstrap_ssh_cidr':'10.0.2.2/32','devbox_install_tailscale':False}} + + +def joining(): + """config() of a host that joins the tailnet.""" + c=config() + c['ansible_vars']={**c['ansible_vars'],'devbox_install_tailscale':True} + return c class DiskGuardTests(unittest.TestCase): def test_exact_single_nonremovable_disk(self): @@ -98,7 +105,7 @@ def test_cloud_seed_carries_a_release_login_only_when_given(self): self.assertRaises(ValueError,builder.cloud_config,config(),'a'*64) def test_only_the_usb_installer_needs_an_operator_password(self): with tempfile.TemporaryDirectory() as d: - path=Path(d)/'c.json'; c=config(); c['profile']='cloud'; del c['operator_password_hash'] + path=Path(d)/'c.json'; c=config(); c['target']='cloud'; del c['operator_password_hash'] path.write_text(json.dumps(c)) operator=builder.cloud_config(builder.checked_config(path,'cloud'),'a'*64)['users'][1] # Locked with `*`, not `!`: sshd without PAM refuses keys for an account locked with `!`. @@ -107,7 +114,7 @@ def test_only_the_usb_installer_needs_an_operator_password(self): with self.assertRaises(ValueError): builder.checked_config(path,'usb') # A profile from when the operator's sudo could ask still renders, and gets no password. for laptop in ('cloud','laptop'): - c['operator_passwordless_sudo']=False; c['profile']=laptop; path.write_text(json.dumps(c)) + c['operator_passwordless_sudo']=False; c['target']=laptop; path.write_text(json.dumps(c)) builder.checked_config(path,'cloud') operator=builder.cloud_config(config(),'a'*64)['users'][1] self.assertEqual((operator['lock_passwd'],operator['passwd']),(False,config()['operator_password_hash'])) @@ -198,39 +205,53 @@ def test_iso_checksum_is_checked_when_given(self): def test_cloud_recovery_network_is_optional_but_checked(self): with tempfile.TemporaryDirectory() as temp: path=Path(temp)/'host.json' - c=config(); c['ansible_vars']={}; path.write_text(json.dumps(c)) + c=config(); c['ansible_vars']={'devbox_install_tailscale':False}; path.write_text(json.dumps(c)) builder.checked_config(path,'cloud') - c['ansible_vars']={'devbox_bootstrap_ssh_cidr':'not a network'}; path.write_text(json.dumps(c)) + c['ansible_vars']={'devbox_bootstrap_ssh_cidr':'not a network','devbox_install_tailscale':False}; path.write_text(json.dumps(c)) with self.assertRaises(ValueError): builder.checked_config(path,'cloud') - def test_a_qemu_profile_is_a_cloud_one_reached_from_the_qemu_host(self): + def test_a_qemu_target_is_a_cloud_one_reached_from_the_qemu_host(self): with tempfile.TemporaryDirectory() as temp: path=Path(temp)/'host.json' - c=config(); c['profile']='qemu'; c['ansible_vars']={}; path.write_text(json.dumps(c)) + c=config(); c['target']='qemu'; c['ansible_vars']={'devbox_install_tailscale':False}; path.write_text(json.dumps(c)) checked=builder.checked_config(path,'cloud') - self.assertEqual(checked['profile'],'cloud') + self.assertEqual(checked['target'],'cloud') self.assertEqual(checked['ansible_vars']['devbox_bootstrap_ssh_cidr'],'10.0.2.2/32') - self.assertEqual(builder.deployment(checked,'a'*64)['profile'],'cloud') + self.assertEqual(builder.deployment(checked,'a'*64)['target'],'cloud') # A network it names stays; laptop and cloud are left alone. - c['ansible_vars']={'devbox_bootstrap_ssh_cidr':'192.0.2.0/24'}; path.write_text(json.dumps(c)) + c['ansible_vars']={'devbox_bootstrap_ssh_cidr':'192.0.2.0/24','devbox_install_tailscale':False}; path.write_text(json.dumps(c)) self.assertEqual(builder.checked_config(path,'cloud')['ansible_vars']['devbox_bootstrap_ssh_cidr'],'192.0.2.0/24') for kept in ['laptop','cloud']: - self.assertEqual(builder.retire_qemu({'profile':kept}),{'profile':kept}) - c['profile']='desktop'; path.write_text(json.dumps(c)) + self.assertEqual(builder.retire_qemu({'target':kept}),{'target':kept}) + c['target']='desktop'; path.write_text(json.dumps(c)) with self.assertRaisesRegex(ValueError,'laptop or cloud'): builder.checked_config(path,'cloud') + def test_a_profile_of_berth_05_is_read_with_the_names_of_now(self): + with tempfile.TemporaryDirectory() as temp: + path=Path(temp)/'host.json' + c=config(); del c['target']; c['profile']='laptop'; c['level']='workstation' + path.write_text(json.dumps(c)) + checked=builder.checked_config(path,'cloud') + self.assertEqual(checked['target'],'laptop') + self.assertNotIn('profile',checked); self.assertNotIn('level',checked); self.assertNotIn('type',checked) + self.assertEqual(builder.deployment(checked,'a'*64)['target'],'laptop') + self.assertEqual(builder.take_old_names({'level':'controller','profile':'cloud'}),{'type':'managed','target':'cloud'}) + self.assertEqual(builder.take_old_names({'type':'ssh','level':'controller','target':'laptop','profile':'cloud'}), + {'type':'ssh','target':'laptop'}) def test_bootstrap_hands_a_qemu_seed_to_the_role_as_cloud(self): script=(ROOT/'provision/bootstrap/bootstrap.sh').read_text() start=script.index("python3 - <<'PY'\n")+len("python3 - <<'PY'\n") handover=script[start:script.index('\nPY\n',start)] with tempfile.TemporaryDirectory() as temp: root=Path(temp) - deployment={'profile':'qemu','ansible_vars':{'devbox_user':'agent'}} code=(handover.replace("/opt/devbox-bootstrap/deployment.json",str(root/'deployment.json')) .replace("Path('/etc/devbox')","Path(%r)"%str(root/'etc'))) - (root/'deployment.json').write_text(json.dumps(deployment)) - subprocess.run([sys.executable,'-c',code],check=True) - handed=json.loads((root/'etc/host.json').read_text()) - self.assertEqual(handed['devbox_profile'],'cloud') - self.assertEqual(handed['devbox_bootstrap_ssh_cidr'],'10.0.2.2/32') + # A seed of berth 0.5 named the target profile. + for key in ('target','profile'): + deployment={key:'qemu','ansible_vars':{'devbox_user':'agent'}} + (root/'deployment.json').write_text(json.dumps(deployment)) + subprocess.run([sys.executable,'-c',code],check=True) + handed=json.loads((root/'etc/host.json').read_text()) + self.assertEqual(handed['devbox_profile'],'cloud',key) + self.assertEqual(handed['devbox_bootstrap_ssh_cidr'],'10.0.2.2/32',key) def test_accounts_take_several_keys_and_the_operator_this_berths(self): key=lambda byte:'ssh-ed25519 '+base64.b64encode(bytes([byte])*64).decode() c={**config(),'operator_ssh_public_keys':[key(1),key(1)],'agent_ssh_public_keys':[key(2),key(3)]} @@ -266,7 +287,7 @@ def test_accounts_take_several_keys_and_the_operator_this_berths(self): path=Path(temp)/'host.json'; path.write_text(json.dumps(only)) with self.assertRaises(ValueError): builder.checked_config(path,'cloud') def test_malformed_profiles_are_rejected_cleanly(self): - for change in [{'hostname':None},{'hostname':'trailing-'},{'profile':'container'}, + for change in [{'hostname':None},{'hostname':'trailing-'},{'target':'container'}, {'agent_ssh_public_key':'not a key'},{'operator_username':'root'}, {'agent_username':'ops'},{'operator_username':'operator'},{'agent_username':'admin'},{'agent_passwordless_sudo':'no'}, {'ansible_vars':{'other':1}},{'target_disk_mode':'serial','target_disk_serial':''}]: @@ -335,28 +356,45 @@ def test_release_is_deterministic_and_excludes_local_secrets(self): 'provision/ansible/site.yml','provision/config/example.json']: self.assertIn(need,names) self.assertFalse(any('secrets/' in x or 'config/local.json' in x for x in names)) - def test_seed_joins_the_login_server_with_the_key_berth_put_in(self): + def test_seed_joins_the_login_server_with_the_key_berth_gave_it(self): with tempfile.TemporaryDirectory() as temp: - path=Path(temp)/'host.json' - # A profile as berth renders it for a host that joins automatically. - path.write_text(json.dumps({**config(),'tailscale_auto_key':True, - 'tailscale_auth_key':'hskey-auth-new','tailscale_login_server':'http://hs.test:8080'})) - variables=builder.deployment(builder.checked_config(path,'cloud'),'0'*64)['ansible_vars'] - self.assertEqual(variables['devbox_tailscale_auth_key'],'hskey-auth-new') + path,given=Path(temp)/'host.json',Path(temp)/'tailnet.json' + given.write_text(json.dumps({'key':'hskey-auth-FAKE','login_server':'http://hs.test:8080'})) + path.write_text(json.dumps(joining())) + with patch.dict(os.environ,{'BERTH_TAILNET_FILE':str(given)}): + variables=builder.deployment(builder.checked_config(path,'cloud'),'0'*64)['ansible_vars'] + self.assertEqual(variables['devbox_tailscale_auth_key'],'hskey-auth-FAKE') self.assertEqual(variables['devbox_tailscale_login_server'],'http://hs.test:8080') self.assertNotIn('devbox_tailscale_key_command',variables) - # Without berth nothing makes the key. - path.write_text(json.dumps({**config(),'tailscale_auto_key':True})) - with self.assertRaisesRegex(ValueError,'needs berth'): builder.checked_config(path,'cloud') - path.write_text(json.dumps({**config(),'tailscale_login_server':'hs.test'})) - with self.assertRaises(ValueError): builder.checked_config(path,'cloud') - # A pasted key without a login server stays as it was. - path.write_text(json.dumps({**config(),'tailscale_auth_key':'tskey-auth-old'})) - variables=builder.deployment(builder.checked_config(path,'cloud'),'0'*64)['ansible_vars'] - self.assertEqual(variables['devbox_tailscale_auth_key'],'tskey-auth-old') + given.write_text(json.dumps({'key':'tskey-auth-FAKE','login_server':''})) + with patch.dict(os.environ,{'BERTH_TAILNET_FILE':str(given)}): + variables=builder.deployment(builder.checked_config(path,'cloud'),'0'*64)['ansible_vars'] self.assertNotIn('devbox_tailscale_login_server',variables) + given.write_text(json.dumps({'key':'tskey-auth-FAKE','login_server':'hs.test'})) + with patch.dict(os.environ,{'BERTH_TAILNET_FILE':str(given)}): + with self.assertRaises(ValueError): builder.checked_config(path,'cloud') + def test_a_seed_that_joins_needs_the_key_berth_makes(self): + with tempfile.TemporaryDirectory() as temp, patch.dict(os.environ,{'BERTH_TAILNET_FILE':''}): + path=Path(temp)/'host.json' + path.write_text(json.dumps(joining())) + with self.assertRaisesRegex(ValueError,'tailnet_api_key'): builder.checked_config(path,'cloud') + # A key pasted into the profile, as older berths took it, is not read. + path.write_text(json.dumps({**joining(),'tailscale_auth_key':'tskey-auth-FAKE','tailscale_auto_key':False})) + with self.assertRaisesRegex(ValueError,'tailnet_api_key'): builder.checked_config(path,'cloud') + pasted={**config(),'tailscale_auth_key':'tskey-auth-FAKE','tailscale_login_server':'http://hs.test:8080'} + pasted['ansible_vars']={**pasted['ansible_vars'],'devbox_tailscale_auth_key':'tskey-auth-FAKE', + 'devbox_tailscale_key_command':['sh','-c','id']} + path.write_text(json.dumps(pasted)) + checked=builder.checked_config(path,'cloud') + self.assertNotIn('tailscale_auth_key',checked) + variables=builder.deployment(checked,'0'*64)['ansible_vars'] + for name in builder.TAILNET_VARS: + self.assertNotIn(name,variables) + # A host off the tailnet needs no key. + path.write_text(json.dumps(config())) + builder.checked_config(path,'cloud') def test_release_carries_plugins_berth_installed(self): - # berth plugins add puts a plugin in node_modules of berth's plugins folder. + # berth plugin add puts a plugin in node_modules of berth's plugins folder. with tempfile.TemporaryDirectory() as temp, patch.object(builder,'BUILD',Path(temp)/'build'): plugin=Path(temp)/'plugins/node_modules/berth-plugin-demo' (plugin/'tasks').mkdir(parents=True); (plugin/'node_modules/dep').mkdir(parents=True) diff --git a/tests/test_docker_plugin.py b/tests/test_docker_plugin.py index 74e0d06..e189954 100644 --- a/tests/test_docker_plugin.py +++ b/tests/test_docker_plugin.py @@ -118,16 +118,18 @@ def fresh_rules(self): class Provision(Harness): - def test_manifest_has_provisioner_and_no_features(self): + def test_manifest_has_provisioner_and_the_engine_of_a_node(self): package = json.loads((PLUGIN / 'package.json').read_text()) - self.assertNotIn('features', package['berth']) + # Its one feature is for a node; a workstation has Docker from the role. + self.assertEqual([(feature['id'], feature['types']) for feature in package['berth']['features']], + [('engine', ['node'])]) [entry] = package['berth']['provisioners'] self.assertEqual((entry['id'], entry['protocol'], entry['command']), ('container', 2, 'bin/provision')) self.assertEqual(entry['verbs'], ['list']) self.assertNotIn('secrets', entry) self.assertEqual([a['name'] for a in entry['arguments']], ['host']) machine, deployment, instance = entry['kinds'] - self.assertEqual((instance['id'], instance['verbs']), ('instance', ['start', 'stop', 'change', 'logs'])) + self.assertEqual((instance['id'], instance['verbs']), ('instance', ['start', 'stop', 'change', 'logs', 'exec'])) self.assertEqual(instance['arguments'][0]['name'], 'image') self.assertEqual((machine['id'], machine['setup']), ('machine', 'apply')) self.assertEqual(machine['verbs'], ['start', 'stop', 'change', 'logs']) @@ -225,8 +227,8 @@ def test_create_refuses_a_name_taken_by_a_stranger(self): self.assertEqual(self.argv('run'), []) def test_status_mapping(self): - cases = [(inspect(OURS), {'state': 'running', 'address': '172.17.0.2'}), - (inspect(OURS, running=False), {'state': 'stopped'}), + cases = [(inspect(OURS), {'state': 'running', 'endpoints': [], 'address': '172.17.0.2'}), + (inspect(OURS, running=False), {'state': 'stopped', 'endpoints': []}), (MISSING, {'state': 'gone'})] for result, expected in cases: with self.subTest(expected=expected): @@ -238,7 +240,7 @@ def test_status_mapping(self): def test_status_with_published_port_answers_the_publish_address(self): done = self.run_verb('status', {'id': 'berth-studio'}, arguments={'ssh_port': '2222', 'publish_address': '192.0.2.5'}, rules=[{'prefix': ['container', 'inspect'], 'results': [inspect(OURS)]}]) - self.assertEqual(json.loads(done.stdout), {'state': 'running', 'address': '192.0.2.5'}) + self.assertEqual(json.loads(done.stdout), {'state': 'running', 'endpoints': [], 'address': '192.0.2.5'}) def test_logs_without_follow_are_the_latest_console_lines(self): console = ''.join(f'[ OK ] Started unit-{n}.service.\n' for n in range(100)) @@ -384,7 +386,7 @@ def test_change_sets_new_limits(self): rules = [{'prefix': ['container', 'inspect'], 'results': [inspect(OURS)]}] done = self.run_verb('change', {'id': 'berth-studio'}, arguments={'cpus': '2', 'memory': '1GiB'}, rules=rules) self.assertEqual(done.returncode, 0, done.stderr) - self.assertEqual(json.loads(done.stdout), {'state': 'running'}) + self.assertEqual(json.loads(done.stdout), {'state': 'running', 'endpoints': []}) self.assertEqual(self.argv('update'), [['update', '--cpus', '2', '--memory', str(1024 ** 3), '--memory-swap', str(2 * 1024 ** 3), 'berth-studio']]) done = self.run_verb('change', {'id': 'berth-studio'}, rules=[ @@ -484,7 +486,7 @@ def test_create_runs_a_plain_container_from_the_image_with_no_berth_host(self): 'ports': '8080:80, 9000:9000', 'restart': 'unless-stopped', 'memory': '2GiB'}, rules=rules) self.assertEqual(done.returncode, 0, done.stderr) - self.assertEqual(json.loads(done.stdout), {'id': 'web-1', 'state': 'running'}) + self.assertEqual(json.loads(done.stdout), {'id': 'web-1', 'state': 'running', 'endpoints': []}) [call] = [c for c in self.calls() if c['args'][:1] == ['run']] run = call['args'] self.assertIn('berth-token=tok-web', run) @@ -502,7 +504,7 @@ def test_create_again_with_its_token_is_the_same_container(self): 'results': [[0, 'web-1\n', '']]}, {'prefix': ['container', 'inspect'], 'results': [inspect(OURS)]}] done = self.make('create', rules=rules) - self.assertEqual(json.loads(done.stdout), {'id': 'web-1', 'state': 'running'}) + self.assertEqual(json.loads(done.stdout), {'id': 'web-1', 'state': 'running', 'endpoints': []}) self.assertEqual(self.argv('run'), []) def test_a_taken_name_or_a_bad_image_is_refused(self): @@ -583,6 +585,74 @@ def test_stop_start_and_destroy_only_what_berth_made_as_an_instance(self): self.assertEqual(done.returncode, 1) self.assertIn('was not made by berth as an instance', done.stderr) + def published(self): + """An instance of ours with ports published on every address, on one address and over UDP.""" + info = json.loads(inspect({'berth-resource': 'r1'})[1]) + info[0]['Config']['ExposedPorts'] = {'80/tcp': {}, '53/udp': {}} + info[0]['NetworkSettings']['Ports'] = { + '80/tcp': [{'HostIp': '0.0.0.0', 'HostPort': '8080'}, {'HostIp': '::', 'HostPort': '8080'}], + '5432/tcp': [{'HostIp': '192.0.2.7', 'HostPort': '15432'}], + '53/udp': [{'HostIp': '0.0.0.0', 'HostPort': '5353'}], + '9000/tcp': None, + } + return [{'prefix': ['container', 'inspect', 'web-1'], 'results': [[0, json.dumps(info), '']]}] + + def test_status_reports_published_ports_and_network_addresses_as_endpoints(self): + done = self.make('status', rules=self.published()) + self.assertEqual(done.returncode, 0, done.stderr) + local = {'protocol': 'tcp', 'local': True} + self.assertEqual(json.loads(done.stdout)['endpoints'], [ + {'name': '5353-udp', 'address': '127.0.0.1', 'port': 5353, 'protocol': 'udp', 'local': True}, + {'name': '15432', 'address': '192.0.2.7', 'port': 15432, 'hint': 'postgres', **local}, + {'name': '8080', 'address': '127.0.0.1', 'port': 8080, 'hint': 'http', **local}, + {'name': 'bridge-53-udp', 'address': '172.17.0.2', 'port': 53, 'protocol': 'udp', 'local': True}, + {'name': 'bridge-80', 'address': '172.17.0.2', 'port': 80, 'hint': 'http', **local}, + ]) + + def test_an_engine_elsewhere_has_its_ports_at_its_own_name_and_no_network_addresses(self): + done = self.make('status', rules=self.published(), arguments={'host': 'ssh://ops@192.0.2.10'}) + self.assertEqual(done.returncode, 0, done.stderr) + self.assertEqual(json.loads(done.stdout)['endpoints'], [ + {'name': '5353-udp', 'address': '192.0.2.10', 'port': 5353, 'protocol': 'udp'}, + {'name': '15432', 'address': '192.0.2.7', 'port': 15432, 'protocol': 'tcp', 'hint': 'postgres'}, + {'name': '8080', 'address': '192.0.2.10', 'port': 8080, 'protocol': 'tcp', 'hint': 'http'}, + ]) + + def test_an_engine_on_a_node_reports_its_endpoints_as_the_node_sees_them(self): + done = self.make('status', {'node': 'big'}, rules=self.published(), + arguments={'host': 'ssh://ops@192.0.2.10'}) + self.assertEqual(done.returncode, 0, done.stderr) + local = {'protocol': 'tcp', 'local': True} + self.assertEqual(json.loads(done.stdout)['endpoints'], [ + {'name': '5353-udp', 'address': '127.0.0.1', 'port': 5353, 'protocol': 'udp', 'local': True}, + {'name': '15432', 'address': '192.0.2.7', 'port': 15432, 'hint': 'postgres', **local}, + {'name': '8080', 'address': '127.0.0.1', 'port': 8080, 'hint': 'http', **local}, + {'name': 'bridge-53-udp', 'address': '172.17.0.2', 'port': 53, 'protocol': 'udp', 'local': True}, + {'name': 'bridge-80', 'address': '172.17.0.2', 'port': 80, 'hint': 'http', **local}, + ]) + + def test_a_command_runs_in_the_container_with_its_input_and_exit_status(self): + rules = [{'prefix': ['container', 'inspect', 'web-1'], 'results': [inspect({'berth-resource': 'r1'})]}, + {'prefix': ['exec'], 'results': [[3, 'hi from web\n', '']]}] + done = self.make('exec', {'command': ['sh', '-c', 'cat; exit 3'], 'stdin': 'select 1;\n'}, rules=rules) + self.assertEqual(done.returncode, 0, done.stderr) + self.assertEqual(json.loads(done.stdout), {'exit_code': 3}) + self.assertIn('hi from web', done.stderr) + [call] = [c for c in self.calls() if c['args'][:1] == ['exec']] + self.assertEqual(call['args'], ['exec', '-i', 'web-1', 'sh', '-c', 'cat; exit 3']) + self.assertEqual(call['stdin'], 'select 1;\n') + # Without input it reads none, and a stopped container or no command is refused. + done = self.make('exec', {'command': ['true']}, rules=rules) + self.assertEqual(self.argv('exec')[-1], ['exec', 'web-1', 'true']) + stopped = [{'prefix': ['container', 'inspect', 'web-1'], 'results': [inspect({'berth-resource': 'r1'}, running=False)]}] + for extra, said in [({'command': ['true']}, 'is not running'), ({'command': 'true'}, 'list of words')]: + done = self.make('exec', extra, rules=stopped) + self.assertEqual(done.returncode, 1) + self.assertIn(said, done.stderr) + done = self.run_verb('exec', {'kind': 'deployment', 'id': 'shop', 'command': ['true']}) + self.assertEqual(done.returncode, 1) + self.assertIn('an instance does', done.stderr) + def test_the_storage_actions_list_create_and_remove(self): rules = [{'prefix': ['volume', 'ls'], 'results': [[0, json.dumps({'Name': 'wiki-data', 'Labels': 'berth-host=studio'}) + '\n', '']]}, {'prefix': ['ps'], 'results': [[0, '', '']]}] @@ -632,7 +702,7 @@ def test_create_brings_the_project_up_with_the_spec_on_stdin(self): 'labels': {'berth-resource': 'shop', 'berth-token': 'tok-9'}}, rules=self.create_rules()) self.assertEqual(done.returncode, 0, done.stderr) - self.assertEqual(json.loads(done.stdout), {'id': 'shop', 'state': 'running'}) + self.assertEqual(json.loads(done.stdout), {'id': 'shop', 'state': 'running', 'endpoints': []}) calls = [c for c in self.calls() if c['args'][:1] == ['compose']] self.assertEqual(calls[0]['args'], [*COMPOSE, '-f', '-', 'config', '--services']) up = calls[1]['args'] @@ -719,7 +789,7 @@ def test_create_without_a_spec_is_refused(self): def test_change_applies_the_new_spec_and_keeps_the_labels(self): done = self.deploy('change', {'spec': SPEC}, rules=self.create_rules()) self.assertEqual(done.returncode, 0, done.stderr) - self.assertEqual(json.loads(done.stdout), {'state': 'running'}) + self.assertEqual(json.loads(done.stdout), {'state': 'running', 'endpoints': []}) [up] = [c for c in self.calls() if 'up' in c['args']] self.assertEqual(up['args'][-3:], ['up', '-d', '--remove-orphans']) self.assertEqual(up['stdin'], SPEC) @@ -727,10 +797,10 @@ def test_change_applies_the_new_spec_and_keeps_the_labels(self): self.assertEqual(override['services']['web']['labels'], {'berth-resource': 'shop', 'berth-token': 'tok-9'}) def test_status_reads_both_forms_of_ps(self): - cases = [(ps(('web', 'running'), ('db', 'running')), {'state': 'running'}), - (ps(('web', 'running'), ('db', 'running'), lines=True), {'state': 'running'}), - (ps(('web', 'exited'), ('db', 'exited'), lines=True), {'state': 'stopped'}), - (ps(('web', 'running'), ('db', 'exited')), {'state': 'running', 'message': 'db not running'}), + cases = [(ps(('web', 'running'), ('db', 'running')), {'state': 'running', 'endpoints': []}), + (ps(('web', 'running'), ('db', 'running'), lines=True), {'state': 'running', 'endpoints': []}), + (ps(('web', 'exited'), ('db', 'exited'), lines=True), {'state': 'stopped', 'endpoints': []}), + (ps(('web', 'running'), ('db', 'exited')), {'state': 'running', 'message': 'db not running', 'endpoints': []}), ([0, '[]', ''], {'state': 'gone'}), ([0, '', ''], {'state': 'gone'})] for result, expected in cases: diff --git a/tests/test_environment.py b/tests/test_environment.py index 899f0ed..c772119 100644 --- a/tests/test_environment.py +++ b/tests/test_environment.py @@ -428,8 +428,8 @@ def test_the_keys_of_berth_have_a_block_no_apply_removes(self): self.assertEqual(check['loop'], '{{ devbox_berth_ssh_public_key.splitlines() }}') self.assertNotIn('failed_when', check) # Both come before any change to the SSH server. - main = (ROLE / 'tasks/main.yml').read_text() - self.assertLess(main.index('accounts.yml'), main.index('host.yml')) + workstation = (ROLE / 'tasks/workstation.yml').read_text() + self.assertLess(workstation.index('accounts.yml'), workstation.index('host.yml')) if __name__ == '__main__': diff --git a/tests/test_incus_plugin.py b/tests/test_incus_plugin.py index 3c204f2..2373565 100644 --- a/tests/test_incus_plugin.py +++ b/tests/test_incus_plugin.py @@ -50,7 +50,7 @@ def handle_error(self, request, client_address): def public(instance): - return {k: v for k, v in instance.items() if k not in ('files', 'request', 'console')} + return {k: v for k, v in instance.items() if k not in ('files', 'request', 'console', 'logs')} class Handler(BaseHTTPRequestHandler): @@ -185,6 +185,20 @@ def route(self): server.events.append((action['action'], instance['name'], action.get('force', False))) instance['status'] = {'start': 'Running', 'stop': 'Stopped'}[action['action']] return self.answer(202, server.operation(), 'async') + if rest == ['exec'] and self.command == 'POST': + server.events.append(('exec', instance['name'], json.loads(body))) + base = f'/1.0/instances/{instance["name"]}/logs' + instance['logs'] = {'exec_1.stdout': b'hi from web\n', 'exec_1.stderr': b'one warning\n'} + key = server.operation() + server.operations[key]['metadata'] = {'return': 2, 'output': {'1': f'{base}/exec_1.stdout', + '2': f'{base}/exec_1.stderr'}} + return self.answer(202, key, 'async') + if rest[:1] == ['logs'] and len(rest) == 2: + if self.command == 'DELETE': + server.events.append(('log-delete', rest[1])) + instance.get('logs', {}).pop(rest[1], None) + return self.answer(200, {}) + return self.raw(instance.get('logs', {}).get(rest[1], b'')) if rest == ['console'] and self.command == 'GET': server.events.append(('console', query.get('type'))) return self.raw(instance.get('console', b'')) @@ -268,7 +282,7 @@ def actions(self): def test_container_takes_the_release_before_its_first_start(self): code, answer, err = self.create() self.assertEqual(code, 0, err) - self.assertEqual(answer, {'id': 'berth-studio-2', 'address': '192.0.2.50'}) + self.assertEqual(answer, {'id': 'berth-studio-2', 'endpoints': [], 'address': '192.0.2.50'}) self.assertEqual(self.actions(), [ ('create', 'berth-studio-2', 'container'), ('push', '/opt/devbox-bootstrap', 'directory', 'Stopped', '0700'), @@ -327,10 +341,10 @@ def test_a_missing_image_is_named(self): def test_status_follows_the_instance(self): self.create() - self.assertEqual(self.run_verb('status', id='berth-studio-2')[1], {'state': 'running', 'address': '192.0.2.50'}) + self.assertEqual(self.run_verb('status', id='berth-studio-2')[1], {'state': 'running', 'endpoints': [], 'address': '192.0.2.50'}) self.assertEqual(self.run_verb('stop', id='berth-studio-2')[:2], (0, {})) self.assertIn(('stop', 'berth-studio-2', True), self.server.events) - self.assertEqual(self.run_verb('status', id='berth-studio-2')[1], {'state': 'stopped'}) + self.assertEqual(self.run_verb('status', id='berth-studio-2')[1], {'state': 'stopped', 'endpoints': []}) self.assertEqual(self.run_verb('start', id='berth-studio-2')[:2], (0, {})) self.assertEqual(self.run_verb('status', id='berth-studio-2')[1]['state'], 'running') self.assertEqual(self.run_verb('status', id='berth-gone')[1], {'state': 'gone'}) @@ -406,7 +420,7 @@ def make(name='web_1', token='7f3a', **arguments): **arguments}) code, answer, err = make() self.assertEqual(code, 0, err) - self.assertEqual(answer, {'id': 'web-1', 'state': 'running', 'address': '192.0.2.50'}) + self.assertEqual(answer, {'id': 'web-1', 'state': 'running', 'endpoints': [], 'address': '192.0.2.50'}) made = self.server.instances['web-1']['request'] self.assertTrue(made['start']) self.assertEqual(made['source']['alias'], 'debian/12') @@ -493,7 +507,7 @@ def test_change_sets_new_limits_and_grows_the_disk(self): code, answer, err = self.run_verb('change', id='berth-studio-2', arguments={'cpus': '2', 'memory': '4GiB', 'disk': '80GiB'}) self.assertEqual(code, 0, err) - self.assertEqual(answer, {'state': 'running'}) + self.assertEqual(answer, {'state': 'running', 'endpoints': []}) [patch] = [e for e in self.server.events if e[0] == 'patch'] self.assertEqual(patch[2]['config'], {'limits.cpu': '2', 'limits.memory': '4GiB'}) self.assertEqual(patch[2]['devices']['root']['size'], '80GiB') @@ -503,6 +517,37 @@ def test_change_sets_new_limits_and_grows_the_disk(self): self.assertEqual(code, 1) self.assertIn('not made by berth', err) + def test_proxy_devices_are_endpoints_on_the_server(self): + self.create() + self.server.instances['berth-studio-2']['expanded_devices'] = { + 'root': {'type': 'disk', 'path': '/', 'pool': 'default'}, + 'web': {'type': 'proxy', 'listen': 'tcp:0.0.0.0:8080', 'connect': 'tcp:127.0.0.1:80'}, + 'dns': {'type': 'proxy', 'listen': 'udp:192.0.2.1:5353', 'connect': 'udp:127.0.0.1:53'}, + 'Socket': {'type': 'proxy', 'listen': 'unix:/run/web.sock', 'connect': 'tcp:127.0.0.1:80'}, + } + code, answer, err = self.run_verb('status', id='berth-studio-2') + self.assertEqual(code, 0, err) + # The fake answers on this machine's loopback, so its addresses are this machine's. + self.assertEqual(answer['endpoints'], [ + {'name': 'dns', 'address': '192.0.2.1', 'port': 5353, 'protocol': 'udp', 'local': True}, + {'name': 'web', 'address': '127.0.0.1', 'port': 8080, 'protocol': 'tcp', 'hint': 'http', 'local': True}, + ]) + + def test_a_command_runs_in_the_instance_with_its_output_and_exit_status(self): + self.create() + code, answer, err = self.run_verb('exec', id='berth-studio-2', command=['sh', '-c', 'echo hi; exit 2']) + self.assertEqual(code, 0, err) + self.assertEqual(answer, {'exit_code': 2}) + self.assertIn('hi from web\none warning\n', err) + [exec_] = [e for e in self.server.events if e[0] == 'exec'] + self.assertEqual(exec_[2]['command'], ['sh', '-c', 'echo hi; exit 2']) + self.assertEqual((exec_[2]['record-output'], exec_[2]['interactive']), (True, False)) + self.assertEqual({e[1] for e in self.server.events if e[0] == 'log-delete'}, {'exec_1.stdout', 'exec_1.stderr'}) + for fields, said in [({'stdin': 'x'}, 'without standard input'), ({'command': 'ls'}, 'list of words')]: + code, _, err = self.run_verb('exec', id='berth-studio-2', **{'command': ['ls'], **fields}) + self.assertEqual(code, 1) + self.assertIn(said, err) + def test_logs_are_the_last_lines_of_the_console(self): self.create() self.server.instances['berth-studio-2']['console'] = ''.join(f'line {n}\n' for n in range(1, 6)).encode() diff --git a/tests/test_kept_profile.py b/tests/test_kept_profile.py index 48cb12f..ae66067 100644 --- a/tests/test_kept_profile.py +++ b/tests/test_kept_profile.py @@ -43,13 +43,11 @@ def playbook(): class SeedTests(unittest.TestCase): def test_the_seed_builder_hands_the_kept_profile_to_the_first_boot(self): builder = load('builder', 'provision/bin/devbox.py') - c = {'profile': 'cloud', 'tailscale_auth_key': 'tskey-once', 'berth_profile': DOCUMENT} + c = {'target': 'cloud', 'berth_profile': DOCUMENT} variables = builder.deployment(c, 'a' * 64)['ansible_vars'] self.assertEqual(variables['devbox_berth_profile'], DOCUMENT) # A profile written without berth keeps nothing. - self.assertNotIn('devbox_berth_profile', builder.deployment({'profile': 'cloud'}, 'a' * 64)['ansible_vars']) - # The first boot reads these variables from host.json and strips only the key. - self.assertNotIn('tskey', json.dumps(variables['devbox_berth_profile'])) + self.assertNotIn('devbox_berth_profile', builder.deployment({'target': 'cloud'}, 'a' * 64)['ansible_vars']) @unittest.skipUnless(yaml and playbook(), 'Run with the Ansible venv Python for the role tasks') diff --git a/tests/test_node.py b/tests/test_node.py new file mode 100644 index 0000000..48fd8b6 --- /dev/null +++ b/tests/test_node.py @@ -0,0 +1,626 @@ +"""A node: the role's path without the desktop, its firewall helper against a +fake ufw, the helper that upgrades a berth server's container against a fake +docker, and the features of the Incus and Docker plugins that a node takes.""" +import json +import os +import shutil +import subprocess +import sys +import tempfile +import unittest +from pathlib import Path + +try: + import yaml +except ImportError: + yaml = None + +SRC = Path(__file__).resolve().parents[1] +ROLE = SRC / 'provision/ansible/roles/devbox' +FIREWALL = ROLE / 'files/node-firewall.sh' +UPGRADE = ROLE / 'files/berth-server-upgrade.py' +PLAYBOOK = shutil.which('ansible-playbook') + + +def tasks(path): + """The tasks of a task file, with those inside blocks.""" + found = [] + + def take(task): + found.append(task) + for key in ('block', 'rescue', 'always'): + for child in task.get(key, []): + take(child) + + for task in yaml.safe_load(Path(path).read_text()): + take(task) + return found + + +def named(name, path): + return next(task for task in tasks(path) if task['name'] == name) + + +def node_tasks(): + """The tasks of the node's path in the order they run, with those of the + node's own task files it imports in their place.""" + found = [] + for task in tasks(ROLE / 'tasks/node.yml'): + imported = task.get('ansible.builtin.import_tasks', '') + found.append(task) + if imported.startswith('node_'): + found.extend(tasks(ROLE / 'tasks' / imported)) + return found + + +def node_named(name): + return next(task for task in node_tasks() if task['name'] == name) + + +def node_text(): + return ''.join((ROLE / 'tasks' / name).read_text() + for name in ('node.yml', 'node_facts.yml', 'node_ssh.yml', 'node_berth.yml')) + + +def write_program(path, text): + path.write_text(text) + path.chmod(0o755) + + +@unittest.skipUnless(yaml, 'Run with the Ansible venv Python for YAML checks') +class RolePath(unittest.TestCase): + def test_the_role_takes_a_node_apart_from_a_workstation(self): + main = yaml.safe_load((ROLE / 'tasks/main.yml').read_text()) + self.assertEqual([(task['ansible.builtin.include_tasks'], task['when']) for task in main], [ + ('node.yml', "devbox_level == 'node'"), ('workstation.yml', "devbox_level != 'node'")]) + + def test_a_node_has_nothing_of_the_desktop_or_its_accounts(self): + node = node_text() + included = [task.get('ansible.builtin.import_tasks') or task.get('ansible.builtin.include_tasks') + for task in node_tasks()] + included = sorted(name for name in included if name and not name.startswith('node_')) + self.assertEqual(included, ['applied.yml', 'berth.yml', 'plugins.yml', 'system.yml', + 'tailnet.yml', 'tailnet_check.yml', 'tailscale.yml']) + for absent in ('desktop.yml', 'accounts.yml', 'applications.yml', 'toolchains.yml', 'laptop.yml', + 'host.yml', 'devbox_packages', 'novnc', 'cockpit', 't3code'): + self.assertNotIn(absent, node, absent) + packages = yaml.safe_load((ROLE / 'defaults/main.yml').read_text())['devbox_node_packages'] + for absent in ('xfce4', 'tigervnc-standalone-server', 'novnc', 'nodejs'): + self.assertNotIn(absent, packages) + self.assertIn('openssh-server', packages) + self.assertIn('unattended-upgrades', packages) + + def test_root_keeps_a_login_with_keys_and_others_none(self): + task = node_named('Require SSH keys, and keep root\'s login as berth uses it') + content = task['ansible.builtin.copy']['content'] + self.assertIn('PasswordAuthentication no', content) + self.assertIn("PermitRootLogin {{ 'prohibit-password' if devbox_node_user == 'root' or", content) + self.assertNotIn("'yes' if", content) + # Only when the SSH user has a key, which berth's own is, so no apply locks berth out. + self.assertEqual(task['when'], 'devbox_harden_ssh and devbox_node_keys.rc == 0') + given = node_named('Give the SSH user the keys of the berth that applies this') + self.assertEqual(given['ansible.builtin.blockinfile']['path'], '{{ devbox_node_home }}/.ssh/authorized_keys') + order = [task['name'] for task in node_tasks()] + self.assertLess(order.index('Give the SSH user the keys of the berth that applies this'), + order.index('Require SSH keys, and keep root\'s login as berth uses it')) + + def test_the_firewall_keeps_ssh_and_comes_last(self): + task = node_named("Set the node's firewall") + rules = task['vars']['devbox_node_firewall'] + self.assertIn("'allow 22/tcp' if devbox_ssh_from == 'any'", rules) + self.assertIn("'allow from ' ~ devbox_apply_from ~ ' to any port 22 proto tcp'", rules) + self.assertIn('devbox_firewall_rules', rules) + order = [task['name'] for task in node_tasks()] + self.assertGreater(order.index("Set the node's firewall"), order.index('Apply the features of the node')) + + def test_a_node_is_not_applied_over_a_workstation(self): + task = node_named('Refuse a host that berth set up as a workstation') + self.assertIn("level | default('workstation') == 'workstation'", task['when'][1]) + + def test_the_actions_of_a_node_run_what_it_installs(self): + actions = {task['name']: task for task in yaml.safe_load((ROLE / 'vars/main.yml').read_text())['devbox_node_tasks']} + self.assertEqual(list(actions), ['status', 'update', 'reboot', 'secrets', 'upgrade-server']) + self.assertTrue(actions['status']['run'].endswith('/status.sh node')) + self.assertIn('if [[ ${1:-} == node ]]', (ROLE / 'files/tasks/status.sh').read_text()) + upgrade = actions['upgrade-server'] + self.assertTrue(upgrade['run'].startswith('/usr/local/lib/berth/berth-server-upgrade ')) + self.assertEqual(upgrade['arguments'][0]['name'], 'version') + installed = node_named("Install the helper that upgrades a berth server's container") + self.assertEqual(installed['ansible.builtin.copy']['dest'], '/usr/local/lib/berth/berth-server-upgrade') + + def test_a_feature_says_what_it_offers_and_for_which_hosts(self): + plugin = tasks(ROLE / 'tasks/plugin.yml') + names = [task['name'] for task in plugin] + self.assertLess(names.index('Require the feature to be for this type of host'), names.index("Run the feature's tasks")) + self.assertGreater(names.index('Record the provisioners the feature offers'), names.index("Run the feature's tasks")) + applied = named('Record which berth and plugins this configuration came from', ROLE / 'tasks/applied.yml') + self.assertIn("{'offers': devbox_plugin_offers}", applied['ansible.builtin.copy']['content']) + + @unittest.skipUnless(PLAYBOOK, 'ansible-playbook is not installed') + def test_the_node_path_and_the_features_parse(self): + # A role's task files that are included at run time are checked only when imported. + with tempfile.TemporaryDirectory() as scratch: + playbook = Path(scratch) / 'check.yml' + files = [ROLE / 'tasks/node.yml', SRC / 'plugins/incus/tasks/server.yml', SRC / 'plugins/docker/tasks/engine.yml'] + playbook.write_text('- hosts: all\n tasks:\n' + ''.join( + f' - ansible.builtin.import_tasks: {path}\n' for path in files)) + env = dict(os.environ, ANSIBLE_ROLES_PATH=str(ROLE.parent), ANSIBLE_HOME=str(Path(scratch) / 'home'), + ANSIBLE_LOCAL_TEMP=str(Path(scratch) / 'tmp'), ANSIBLE_NOCOLOR='1') + done = subprocess.run([PLAYBOOK, '--syntax-check', '-i', 'big,', str(playbook)], + capture_output=True, text=True, env=env, timeout=120) + self.assertEqual(done.returncode, 0, done.stdout + done.stderr) + + +@unittest.skipUnless(yaml, 'Run with the Ansible venv Python for YAML checks') +class Features(unittest.TestCase): + def feature(self, plugin, feature): + manifest = json.loads((SRC / 'plugins' / plugin / 'package.json').read_text()) + found = next(item for item in manifest['berth']['features'] if item['id'] == feature) + return manifest, found + + def test_the_engines_are_features_of_a_node_that_offer_their_provisioners(self): + for plugin, feature, offers in (('incus', 'server', ['vm', 'container']), ('docker', 'engine', ['container'])): + with self.subTest(plugin=plugin): + manifest, found = self.feature(plugin, feature) + self.assertEqual(found['types'], ['node']) + self.assertEqual(found['offers'], offers) + provisioners = [item['id'] for item in manifest['berth']['provisioners']] + self.assertTrue(set(offers) <= set(provisioners)) + self.assertIn('tasks', manifest['files']) + steps = tasks(SRC / 'plugins' / plugin / found['tasks']) + offered = [step for step in steps if 'devbox_offer' in step.get('ansible.builtin.set_fact', {})] + self.assertEqual(len(offered), 1) + # The status of a node goes on with the feature's own. + self.assertEqual(found['actions'][0]['extends'], 'status') + # Actions are templated by the role, so they carry no Jinja of their own. + self.assertNotIn('{{', json.dumps(found['actions'])) + + def test_incus_trusts_berths_certificate_and_pins_its_own(self): + manifest, found = self.feature('incus', 'server') + self.assertEqual(found['trusts'], {'certificate': 'incus_client_cert', 'key': 'incus_client_key'}) + for provisioner in manifest['berth']['provisioners']: + names = [secret if isinstance(secret, str) else secret['name'] for secret in provisioner['secrets']] + self.assertTrue({'incus_client_cert', 'incus_client_key', 'incus_server_cert'} <= set(names)) + offer = named('Open the firewall to the guests and to the API, and offer the provisioners', + SRC / 'plugins/incus/tasks/server.yml')['ansible.builtin.set_fact']['devbox_offer'] + self.assertEqual(offer['settings'], {'server': 'https://{address}:{{ devbox_incus_port }}'}) + self.assertIn('incus_server_cert', offer['secrets']) + self.assertEqual(offer['aliases']['server'], ['https://incus-host:{{ devbox_incus_port }}']) + + def test_incus_never_makes_a_pool_on_a_disk(self): + steps = tasks(SRC / 'plugins/incus/tasks/server.yml') + check = next(step for step in steps if step['name'] == 'Check the Incus settings') + self.assertIn("not devbox_incus.pool_source.startswith('/dev/')", check['ansible.builtin.assert']['that']) + init = next(step for step in steps if 'incus admin init' in str(step.get('ansible.builtin.command', ''))) + # Only an Incus without any storage pool is set up; one that has a pool is kept. + self.assertEqual(init['when'], 'devbox_incus_pools | length == 0') + defaults = next(step for step in steps if step['name'] == 'Take the Incus settings of the profile') + self.assertIn("default('dir')", defaults['ansible.builtin.set_fact']['devbox_incus']['pool_driver']) + + def test_docker_keeps_an_engine_that_is_installed(self): + steps = tasks(SRC / 'plugins/docker/tasks/engine.yml') + install = next(step for step in steps if step['name'] == "Install Docker from Ubuntu's packages") + self.assertEqual(install['when'], 'devbox_docker_found.rc != 0') + offer = next(step for step in steps if step['name'] == 'Offer the provisioner of this engine') + self.assertIn("'ssh://{user}@{address}:{port}'", offer['ansible.builtin.set_fact']['devbox_offer']['settings']['host']) + + +FAKE_UFW = r'''#!/usr/bin/env python3 +"""A ufw that keeps its rules in a file, as `ufw show added` lists them.""" +import json, os, sys +state = os.environ['FAKE_UFW'] +data = json.load(open(state)) if os.path.exists(state) else {'rules': [], 'active': False, 'calls': []} +args = sys.argv[1:] +data['calls'].append(args) +words = ' '.join(args) +if args[:2] == ['show', 'added']: + print("Added user rules (see 'ufw status' for running firewall):") + for rule in data['rules']: + print('ufw ' + rule) +elif args[:1] == ['status']: + print('Status: active' if data['active'] else 'Status: inactive') + if data['active']: + print('Default: deny (incoming), allow (outgoing), deny (routed)') +elif args[:2] == ['--force', 'enable']: + data['active'] = True +elif args[:1] == ['default']: + pass +elif args[:2] == ['route', 'delete']: + rule = 'route ' + ' '.join(args[2:]) + data['rules'] = [known for known in data['rules'] if not known.startswith(rule + ' comment') and known != rule] +elif args[:1] == ['delete']: + if args[1] == 'route': + print('ERROR: Invalid syntax', file=sys.stderr) + sys.exit(1) + rule = ' '.join(args[1:]) + data['rules'] = [known for known in data['rules'] if not known.startswith(rule + ' comment') and known != rule] +else: + # Real ufw lists a comment in quotes. + if 'comment' in args: + at = args.index('comment') + words = ' '.join(args[:at]) + " comment '" + ' '.join(args[at + 1:]) + "'" + if words not in data['rules']: + data['rules'].append(words) +json.dump(data, open(state, 'w')) +''' + + +class Firewall(unittest.TestCase): + def setUp(self): + self.scratch = Path(tempfile.mkdtemp()) + self.addCleanup(shutil.rmtree, self.scratch) + write_program(self.scratch / 'ufw', FAKE_UFW) + self.state = self.scratch / 'ufw.json' + + def run_firewall(self, *rules): + env = dict(os.environ, DEVBOX_UFW=str(self.scratch / 'ufw'), FAKE_UFW=str(self.state)) + done = subprocess.run(['bash', str(FIREWALL)], input='\n'.join(rules) + '\n', capture_output=True, + text=True, env=env, timeout=60) + self.assertEqual(done.returncode, 0, done.stderr) + return done.stdout + + def kept(self): + return json.loads(self.state.read_text()) + + def test_rules_go_in_before_the_firewall_is_switched_on_and_once(self): + said = self.run_firewall('allow 22/tcp', 'allow from 192.0.2.7 to any port 22 proto tcp', 'allow in on incusbr0') + self.assertIn('added: allow 22/tcp', said) + self.assertIn('switched the firewall on', said) + calls = self.kept()['calls'] + enabled = calls.index(['--force', 'enable']) + added = calls.index(['allow', '22/tcp', 'comment', 'berth']) + self.assertLess(added, enabled) + self.assertEqual(self.run_firewall('allow 22/tcp', 'allow from 192.0.2.7 to any port 22 proto tcp', + 'allow in on incusbr0'), '') + + def test_a_rule_no_longer_asked_for_goes_and_one_by_hand_stays(self): + self.run_firewall('allow 22/tcp', 'allow in on incusbr0', 'route allow in on incusbr0', 'route allow out on incusbr0') + data = self.kept() + data['rules'].append('allow 80/tcp') + self.state.write_text(json.dumps(data)) + said = self.run_firewall('allow 22/tcp') + self.assertIn('removed: route allow in on incusbr0', said) + self.assertEqual(self.kept()['rules'], ["allow 22/tcp comment 'berth'", 'allow 80/tcp']) + + def test_ssh_stays_whatever_ufw_makes_of_a_rule(self): + # One ufw lists otherwise than it was given is deleted as stale, and goes back in. + self.state.write_text(json.dumps({'rules': ["allow 22 comment 'berth'"], 'active': True, 'calls': []})) + self.run_firewall('allow 22/tcp') + self.assertIn("allow 22/tcp comment 'berth'", self.kept()['rules']) + + +@unittest.skipUnless(yaml and PLAYBOOK, 'Run with the Ansible venv Python for the playbook') +class TakingIn(unittest.TestCase): + """What a node keeps of a machine set up before: the firewall's rules for + SSH and root's login with keys, as the role's own expressions make them.""" + + def render(self, facts, wanted): + """`wanted` as the role's tasks make it from these facts, run by ansible-playbook.""" + limited = node_named('Keep SSH as limited as it was') + firewall = node_named("Set the node's firewall") + sshd = node_named("Require SSH keys, and keep root's login as berth uses it") + scratch = Path(tempfile.mkdtemp()) + self.addCleanup(shutil.rmtree, scratch) + tasks = [{'name': 'Facts', 'ansible.builtin.set_fact': facts}, + {'name': limited['name'], 'ansible.builtin.set_fact': limited['ansible.builtin.set_fact']}, + {'name': 'Show', 'ansible.builtin.copy': {'dest': str(scratch / 'shown.json'), 'content': ( + '{{ {"firewall": devbox_node_firewall, "sshd": sshd} | to_json }}')}, + 'vars': dict(firewall['vars'], sshd=sshd['ansible.builtin.copy']['content'])}] + playbook = scratch / 'play.yml' + playbook.write_text(yaml.safe_dump([{'hosts': 'all', 'gather_facts': False, 'vars': { + 'devbox_manage_firewall': True, 'devbox_ssh_from': 'any', 'devbox_apply_from': '192.0.2.5', + 'devbox_bootstrap_ssh_cidr': '', 'devbox_install_tailscale': False, 'devbox_firewall_rules': [], + 'devbox_node_user': 'berth'}, 'tasks': tasks}])) + done = subprocess.run([PLAYBOOK, '-i', 'localhost,', '--connection', 'local', str(playbook)], + capture_output=True, text=True, timeout=120, + env=dict(os.environ, ANSIBLE_LOCALHOST_WARNING='0')) + self.assertEqual(done.returncode, 0, done.stdout + done.stderr) + return json.loads((scratch / 'shown.json').read_text())[wanted] + + def test_ssh_that_was_limited_by_hand_is_not_opened(self): + added = {'stdout_lines': ["Added user rules (see 'ufw status' for running firewall):", + 'ufw allow from 198.51.100.7 to any port 22 proto tcp', + 'ufw allow in on incusbr0']} + rules = self.render({'devbox_node_ufw_added': added}, 'firewall') + self.assertNotIn('allow 22/tcp', rules) + self.assertIn('allow from 192.0.2.5 to any port 22 proto tcp', rules) + # One rule that opens SSH to all is no limit to keep. + added['stdout_lines'].append('ufw allow 22/tcp') + self.assertIn('allow 22/tcp', self.render({'devbox_node_ufw_added': added}, 'firewall')) + # A machine with no rules gets the profile's. + self.assertIn('allow 22/tcp', self.render({'devbox_node_ufw_added': {'stdout_lines': []}}, 'firewall')) + + def test_root_keeps_a_login_with_keys_it_had(self): + for found, kept in (('prohibit-password', 'prohibit-password'), ('yes', 'prohibit-password'), + ('without-password', 'prohibit-password'), ('no', 'no'), ('', 'no')): + facts = {'devbox_node_ufw_added': {'stdout_lines': []}, 'devbox_node_root_login': {'stdout': found}} + self.assertIn(f'PermitRootLogin {kept}\n', self.render(facts, 'sshd'), found) + facts = {'devbox_node_ufw_added': {'stdout_lines': []}, 'devbox_node_root_login': {'stdout': 'no'}, + 'devbox_node_user': 'root'} + self.assertIn('PermitRootLogin prohibit-password\n', self.render(facts, 'sshd')) + + def test_the_facts_are_read_before_they_are_used(self): + names = [task['name'] for task in node_tasks()] + self.assertLess(names.index('See whether root logs in with keys here already'), + names.index("Require SSH keys, and keep root's login as berth uses it")) + self.assertLess(names.index("Read the firewall's rules"), names.index('Keep SSH as limited as it was')) + self.assertLess(names.index('Keep SSH as limited as it was'), names.index("Set the node's firewall")) + + +FAKE_DOCKER = r'''#!/usr/bin/env python3 +import json, os, sys +state = os.environ['FAKE_DOCKER'] +data = json.load(open(state)) +args = sys.argv[1:] +data.setdefault('calls', []).append(args) +if '--env-file' in args: + data['env_file'] = open(args[args.index('--env-file') + 1]).read() + data['env_mode'] = oct(os.stat(args[args.index('--env-file') + 1]).st_mode & 0o777) +def save(): + json.dump(data, open(state, 'w')) +containers, images = data['containers'], data['images'] +VALUED = {'--name', '--restart', '--network', '--hostname', '--domainname', '--publish', '--volume', + '--group-add', '--add-host', '--memory', '--cpus', '--shm-size', '--log-driver', '--log-opt', + '--stop-signal', '--stop-timeout', '--user', '--label', '--env-file'} +def made(args): + # A container as docker inspect shows one that `docker run ARGS` made. + given, rest = {}, list(args) + while rest and rest[0].startswith('--'): + flag = rest.pop(0) + given.setdefault(flag, []).append(rest.pop(0) if flag in VALUED else True) + ref, command = rest[0], rest[1:] + image = images[ref] + one = lambda flag, default=None: given.get(flag, [default])[-1] + env = list(image['Config'].get('Env') or []) + if '--env-file' in given: + env += [line for line in open(one('--env-file')).read().splitlines() if line] + ports = {} + for published in given.get('--publish', []): + parts = published.rsplit(':', 2) + ports.setdefault(parts[-1] + '/tcp', []).append({'HostIp': parts[0] if len(parts) == 3 else '', + 'HostPort': parts[-2] if len(parts) > 1 else ''}) + mounts = [] + for volume in given.get('--volume', []): + source, target, *mode = volume.split(':') + mounts.append({'Type': 'bind' if source.startswith('/') else 'volume', 'Source': source, 'Name': source, + 'Destination': target, 'Mode': ','.join(mode), 'RW': 'ro' not in mode}) + restart = (one('--restart', 'no') or 'no').split(':') + return { + 'Id': 'f00d' + str(len(data['calls'])).rjust(12, '0'), 'Name': '/' + one('--name'), 'Image': image['Id'], + 'State': {'Running': data.get('new_runs', True)}, + 'Config': {'Image': ref, 'Hostname': 'f00d' + str(len(data['calls'])).rjust(8, '0'), 'Env': env, + 'Cmd': command or image['Config'].get('Cmd'), 'Entrypoint': image['Config'].get('Entrypoint'), + 'User': one('--user', image['Config'].get('User') or ''), + 'WorkingDir': image['Config'].get('WorkingDir') or '', 'Labels': {}}, + 'HostConfig': {'RestartPolicy': {'Name': restart[0], 'MaximumRetryCount': int(restart[1]) if len(restart) > 1 else 0}, + 'NetworkMode': one('--network', 'bridge'), 'PortBindings': ports, + 'GroupAdd': given.get('--group-add') or None, 'ExtraHosts': given.get('--add-host') or None, + 'LogConfig': {'Type': one('--log-driver', 'json-file'), + 'Config': dict(opt.split('=', 1) for opt in given.get('--log-opt', []))}, + 'Runtime': 'runc', 'IpcMode': 'private', 'ShmSize': 67108864, 'CgroupnsMode': 'private', + 'MaskedPaths': ['/proc/acpi', '/proc/kcore']}, + 'NetworkSettings': {'Networks': {one('--network', 'bridge'): {}}}, + 'Mounts': mounts, + } +if args[:2] == ['container', 'inspect'] and args[2] == '--format': + found = containers.get(args[-1]) + save() + print('true' if found and found['State']['Running'] else 'false') + sys.exit(0 if found else 1) +if args[:2] == ['container', 'inspect']: + found = containers.get(args[2]) + save() + if not found: + print('Error: No such container', file=sys.stderr); sys.exit(1) + print(json.dumps([found])); sys.exit(0) +if args[:2] == ['image', 'inspect']: + save() + if args[2] not in images: + print('Error: No such image', file=sys.stderr); sys.exit(1) + print(json.dumps([images[args[2]]])); sys.exit(0) +if args[:1] == ['pull']: + images[args[1]] = images[images['new']['Id']] = images['new'] + save(); sys.exit(0) +if args[:2] == ['container', 'rename']: + containers[args[3]] = containers.pop(args[2]) +if args[:2] == ['container', 'stop']: + containers[args[2]]['State']['Running'] = False +if args[:2] in (['container', 'start'], ['start']): + containers[args[-1]]['State']['Running'] = True +if args[:2] == ['container', 'rm']: + containers.pop(args[-1], None) +if args[:1] == ['run']: + container = made(args[1:]) + containers[container['Name'][1:]] = container +if args[:1] == ['exec'] and args[-3:] == ['berth', 'server', 'setup-key']: + print('berth-setup-made-up-key') +if args[:1] == ['exec'] and args[-3:] == ['berth', 'key', 'show']: + print('ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMadeUpMadeUpMadeUpMadeUpMadeUpMadeUpMadeUp berth') +save() +''' + +# The berth server's container that the incus-host recipe of berth 0.5.2 ran, +# as docker inspect shows it on a daemon with log options in daemon.json. +FOUND_052 = json.loads((SRC / 'tests/fixtures/berth-server-0.5.2.json').read_text()) + + +def container(**host): + """The berth server's container as the recipe ran it, with what a real one has besides.""" + host_config = { + 'RestartPolicy': {'Name': 'unless-stopped', 'MaximumRetryCount': 0}, + 'NetworkMode': 'bridge', + 'PortBindings': {'7777/tcp': [{'HostIp': '127.0.0.1', 'HostPort': '7777'}]}, + 'GroupAdd': ['989'], + 'ExtraHosts': ['incus-host:host-gateway'], + 'LogConfig': {'Type': 'json-file', 'Config': {}}, + 'Runtime': 'runc', + 'IpcMode': 'private', + } + host_config.update(host) + return { + 'Id': 'abc123def4567890', 'Name': '/berth', 'Image': 'sha256:old', + 'State': {'Running': True}, + 'Config': {'Image': 'ghcr.io/helightdev/berth:0.5.2', 'Hostname': 'abc123def456', + 'Env': ['XDG_CONFIG_HOME=/config', 'BERTH_TOKEN_HINT=s3cret'], + 'Cmd': ['serve', '--listen', '0.0.0.0:7777', '--expose'], 'Entrypoint': ['tini', '--', 'berth'], + 'User': 'berth', 'WorkingDir': '/src', 'Labels': {}}, + 'HostConfig': host_config, + 'NetworkSettings': {'Networks': {'bridge': {}}}, + 'Mounts': [ + {'Type': 'bind', 'Source': '/srv/berth/config', 'Destination': '/config', 'RW': True}, + {'Type': 'bind', 'Source': '/srv/berth/data', 'Destination': '/data', 'RW': True}, + {'Type': 'bind', 'Source': '/srv/berth/ssh', 'Destination': '/home/berth/.ssh', 'RW': True}, + {'Type': 'bind', 'Source': '/var/run/docker.sock', 'Destination': '/var/run/docker.sock', 'RW': True}, + ], + } + + +def image(version, image_id): + return {'Id': image_id, 'Config': { + 'Env': ['XDG_CONFIG_HOME=/config'], 'Cmd': ['serve', '--listen', '0.0.0.0:7777', '--expose'], + 'Entrypoint': ['tini', '--', 'berth'], 'User': 'berth', 'WorkingDir': '/src', + 'Labels': {'org.opencontainers.image.version': version}}} + + +class ServerUpgrade(unittest.TestCase): + def setUp(self): + self.scratch = Path(tempfile.mkdtemp()) + self.addCleanup(shutil.rmtree, self.scratch) + write_program(self.scratch / 'docker', FAKE_DOCKER) + self.state = self.scratch / 'docker.json' + + def helper(self, *args, found=None, old_image=None, new='0.6.0', new_runs=True, images=None, keep=False): + if not keep: + found = found or container() + old_image = old_image or image('0.5.2', 'sha256:old') + self.state.write_text(json.dumps({ + 'containers': {'berth': found}, + 'images': dict({found['Image']: old_image, found['Config']['Image']: old_image, + 'new': image(new, 'sha256:new')}, **(images or {})), + 'new_runs': new_runs})) + else: + data = json.loads(self.state.read_text()) + data['calls'] = [] + self.state.write_text(json.dumps(data)) + env = dict(os.environ, DEVBOX_DOCKER=str(self.scratch / 'docker'), FAKE_DOCKER=str(self.state), + DEVBOX_UPGRADE_SETTLE='0', BERTH_ARG_VERSION=new) + done = subprocess.run([sys.executable, str(UPGRADE), 'berth', *args], capture_output=True, text=True, + env=env, timeout=60) + return done, json.loads(self.state.read_text()) + + def upgrade(self, version, found=None, new_runs=True): + return self.helper(found=found, new=version, new_runs=new_runs) + + @staticmethod + def run_call(data): + return next(call for call in data['calls'] if call[0] == 'run') + + def test_the_new_container_keeps_the_data_and_the_options(self): + done, data = self.upgrade('0.6.0') + self.assertEqual(done.returncode, 0, done.stderr) + self.assertIn('berth runs ghcr.io/helightdev/berth:0.6.0 now', done.stdout) + run = self.run_call(data) + words = ' '.join(run) + for kept in ('--name berth', '--restart unless-stopped', '--publish 127.0.0.1:7777:7777', + '--volume /srv/berth/data:/data', '--volume /srv/berth/ssh:/home/berth/.ssh', + '--volume /var/run/docker.sock:/var/run/docker.sock', '--group-add 989', + '--add-host incus-host:host-gateway'): + self.assertIn(kept, words) + self.assertEqual(run[-1], 'ghcr.io/helightdev/berth:0.6.0') + # The environment the image does not set travels in a private file, not in argv. + self.assertNotIn('s3cret', words) + self.assertEqual((data['env_file'], data['env_mode']), ('BERTH_TOKEN_HINT=s3cret\n', '0o600')) + self.assertEqual(sorted(data['containers']), ['berth']) + self.assertIn(['pull', 'ghcr.io/helightdev/berth:0.6.0'], data['calls']) + + def test_a_container_of_0_5_2_on_a_daemon_with_log_options_is_carried_over(self): + found, old = FOUND_052['container'], FOUND_052['image'] + new = dict(old, Id='sha256:new', Config=dict(old['Config'], Cmd=['serve', '--listen', '0.0.0.0:7777', '--expose'])) + done, data = self.helper(found=found, old_image=old, images={'new': new}) + self.assertEqual(done.returncode, 0, done.stderr) + run = self.run_call(data) + words = ' '.join(run) + for kept in ('--log-driver json-file', '--log-opt max-file=5', '--log-opt max-size=10m', + '--add-host incus-host:host-gateway', '--group-add 989', + '--volume /var/run/docker.sock:/var/run/docker.sock', '--volume /srv/berth/config:/config', + '--publish 127.0.0.1:7777:7777', '--restart unless-stopped'): + self.assertIn(kept, words) + # What docker fills in for every container is not an option of this one. + for left in ('--shm-size', '--hostname', '--user', '--privileged', 'MaskedPaths'): + self.assertNotIn(left, words) + # Its command was its image's, so the new image's own runs: no --web of 0.5.2. + self.assertEqual(run[-1], 'ghcr.io/helightdev/berth:0.6.0') + self.assertEqual(data['env_file'], '') + + def test_an_option_it_cannot_carry_over_changes_nothing(self): + done, data = self.upgrade('0.6.0', found=container(Privileged=True, Devices=[{'PathOnHost': '/dev/kvm'}])) + self.assertNotEqual(done.returncode, 0) + self.assertIn('it runs privileged; it has devices', done.stderr) + self.assertFalse([call for call in data['calls'] if call[0] in ('pull', 'run', 'container') and call[1:2] != ['inspect']]) + for host in ({'ReadonlyRootfs': True}, {'DeviceRequests': [{'Count': -1}]}, {'Runtime': 'sysbox-runc'}, + {'AutoRemove': True}): + done, _ = self.upgrade('0.6.0', found=container(**host)) + self.assertNotEqual(done.returncode, 0, host) + self.assertIn('Upgrade it by hand', done.stderr) + + def test_an_older_version_is_refused(self): + done, data = self.upgrade('0.5.1') + self.assertNotEqual(done.returncode, 0) + self.assertIn('runs berth 0.5.2, which is newer than 0.5.1', done.stderr) + self.assertNotIn('pull', [call[0] for call in data['calls']]) + done, _ = self.upgrade('latest') + self.assertIn("'latest' is not a version of berth", done.stderr) + + def test_the_old_container_runs_again_when_the_new_one_stops(self): + done, data = self.upgrade('0.6.0', new_runs=False) + self.assertNotEqual(done.returncode, 0) + self.assertIn('so berth runs ghcr.io/helightdev/berth:0.5.2 again', done.stderr) + self.assertEqual(sorted(data['containers']), ['berth']) + self.assertTrue(data['containers']['berth']['State']['Running']) + self.assertIn(['container', 'rename', 'berth-before-0.6.0', 'berth'], data['calls']) + + def test_the_recipe_sets_its_options_over_the_containers_and_keeps_the_others(self): + asked = ['--publish', '127.0.0.1:7878:7777', '--volume', '/srv/berth/data:/data', + '--add-host', 'node:host-gateway', '--add-host', 'incus-host:host-gateway', '--group-add', '989'] + done, data = self.helper('--image', 'ghcr.io/helightdev/berth', '--pull', *asked) + self.assertEqual(done.returncode, 0, done.stderr) + run = self.run_call(data) + words = ' '.join(run) + self.assertIn('--publish 127.0.0.1:7878:7777', words) + self.assertNotIn('127.0.0.1:7777:7777', words) + self.assertEqual(words.count('--add-host incus-host:host-gateway'), 1) + self.assertEqual(words.count('--group-add 989'), 1) + for kept in ('--add-host node:host-gateway', '--volume /var/run/docker.sock:/var/run/docker.sock'): + self.assertIn(kept, words) + self.assertEqual(run[-1], 'ghcr.io/helightdev/berth') + # Run again with the same, it changes nothing. + done, data = self.helper('--image', 'ghcr.io/helightdev/berth', '--pull', *asked, keep=True) + self.assertEqual(done.returncode, 0, done.stderr) + self.assertIn('berth is unchanged', done.stdout) + self.assertNotIn('run', [call[0] for call in data['calls']]) + + def test_without_an_update_the_recipe_keeps_the_image(self): + done, data = self.helper('--image', 'ghcr.io/helightdev/berth', '--keep-image', + '--publish', '127.0.0.1:7777:7777') + self.assertEqual(done.returncode, 0, done.stderr) + self.assertIn('berth is unchanged', done.stdout) + self.assertNotIn('pull', [call[0] for call in data['calls']]) + done, data = self.helper('--image', 'ghcr.io/helightdev/berth', '--keep-image', + '--add-host', 'node:host-gateway') + self.assertEqual(done.returncode, 0, done.stderr) + self.assertEqual(self.run_call(data)[-1], 'ghcr.io/helightdev/berth:0.5.2') + # A tag pulled again since is not taken up without an update. + done, data = self.helper('--image', 'x', '--keep-image', '--add-host', 'node:host-gateway', + images={'ghcr.io/helightdev/berth:0.5.2': image('0.6.0', 'sha256:new')}) + self.assertEqual(done.returncode, 0, done.stderr) + self.assertEqual(self.run_call(data)[-1], 'sha256:old') + + def test_the_recipe_never_goes_back_to_an_older_server(self): + done, data = self.helper('--image', 'ghcr.io/helightdev/berth:0.5.1', '--pull', new='0.5.1') + self.assertNotEqual(done.returncode, 0) + self.assertIn('which is newer than 0.5.1', done.stderr) + self.assertNotIn('run', [call[0] for call in data['calls']]) + + +if __name__ == '__main__': + unittest.main() diff --git a/tests/test_server_recipe.py b/tests/test_server_recipe.py new file mode 100644 index 0000000..44c40ac --- /dev/null +++ b/tests/test_server_recipe.py @@ -0,0 +1,161 @@ +"""The berth server recipe's container, run end to end by ansible-playbook +against a fake docker: a first run, a run over a container that the +incus-host recipe of berth 0.5.2 made, and a second run that changes nothing.""" +import json +import os +import shutil +import socket +import subprocess +import tempfile +import unittest +from pathlib import Path + +from test_node import FAKE_DOCKER, FOUND_052, PLAYBOOK, UPGRADE, image, write_program, yaml + +SRC = Path(__file__).resolve().parents[1] +RECIPE = SRC / 'recipes/berth-server' +SHARED = SRC / 'recipes/shared/berth-server.yml' + +# The tasks of site.yml that run the server, in their order. +SERVER_TASKS = ['See whether this machine runs Incus', "Find the group of Docker's socket", 'Run the berth server'] + +# A listener on the server's port for the recipe to wait for, as the server in a new container is. +LISTENER = ''' +import socket, sys, time +server = socket.socket(); server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) +server.bind(("127.0.0.1", int(sys.argv[1]))); server.listen(); server.settimeout(1) +end = time.time() + 30 +while time.time() < end: + try: + server.accept()[0].close() + except OSError: + pass +''' + + +def free_port(): + with socket.socket() as probe: + probe.bind(('127.0.0.1', 0)) + return probe.getsockname()[1] + + +@unittest.skipUnless(yaml and PLAYBOOK, 'Run with the Ansible venv Python for the playbook') +class ServerRecipe(unittest.TestCase): + def setUp(self): + self.scratch = Path(tempfile.mkdtemp()) + self.addCleanup(shutil.rmtree, self.scratch) + bin_dir = self.scratch / 'bin' + bin_dir.mkdir() + # Each `docker run` also opens the port, so the recipe finds the server answering. + write_program(bin_dir / 'docker', FAKE_DOCKER + f''' +if args[:1] == ['run']: + import subprocess + subprocess.Popen([sys.executable, '-c', {LISTENER!r}, os.environ['FAKE_PORT']], start_new_session=True, + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) +''') + self.state = self.scratch / 'docker.json' + self.port = free_port() + self.incus = self.scratch / 'incus-server.crt' + (self.scratch / 'docker.sock').write_text('') + self.env = dict(os.environ, PATH=f'{bin_dir}:{os.environ["PATH"]}', FAKE_DOCKER=str(self.state), + FAKE_PORT=str(self.port), DEVBOX_DOCKER=str(bin_dir / 'docker'), + DEVBOX_UPGRADE_SETTLE='0', ANSIBLE_NOCOLOR='1', ANSIBLE_LOCALHOST_WARNING='0') + self.playbook = self.write_playbook() + + def write_playbook(self): + """The recipe's play with only the tasks that run the server, the + paths of this machine put in scratch.""" + play = yaml.safe_load((RECIPE / 'site.yml').read_text())[0] + tasks = [task for task in play['tasks'] if task['name'] in SERVER_TASKS] + self.assertEqual([task['name'] for task in tasks], SERVER_TASKS) + tasks[0]['ansible.builtin.stat']['path'] = str(self.incus) + tasks[1]['ansible.builtin.command'] = f'stat --format %g {self.scratch / "docker.sock"}' + shared = yaml.safe_load(SHARED.read_text()) + for task in shared: + # The folders belong to the user 1000, which only root or that user can make them. + if 'ansible.builtin.file' in task and os.getuid() not in (0, 1000): + task['ansible.builtin.file'].pop('owner') + task['ansible.builtin.file'].pop('group') + (self.scratch / 'shared.yml').write_text(yaml.safe_dump(shared, sort_keys=False)) + tasks[2]['ansible.builtin.include_tasks'] = str(self.scratch / 'shared.yml') + tasks[2]['vars']['berth_server_helper'] = str(UPGRADE) + variables = yaml.safe_load((RECIPE / 'vars.yml').read_text()) + variables.update(berth_server_dir=str(self.scratch / 'srv'), berth_server_port=self.port, + berth_server_tailscale_serve=False, tailscale_hostname='pier') + playbook = self.scratch / 'site.yml' + playbook.write_text(yaml.safe_dump([{ + 'name': play['name'], 'hosts': 'all', 'gather_facts': False, 'vars': variables, 'tasks': tasks}], + sort_keys=False)) + return playbook + + def docker(self, found=None): + new = image('0.6.0', 'sha256:new') + images = {'new': new, 'sha256:new': new, 'ghcr.io/helightdev/berth': new} + containers = {} + if found: + images.update({found['Image']: FOUND_052['image'], found['Config']['Image']: FOUND_052['image']}) + containers['berth'] = found + self.state.write_text(json.dumps({'containers': containers, 'images': images})) + + def apply(self, *extra): + data = json.loads(self.state.read_text()) + data['calls'] = [] + self.state.write_text(json.dumps(data)) + done = subprocess.run( + [PLAYBOOK, '--connection', 'local', '--inventory', 'localhost,', str(self.playbook), *extra], + capture_output=True, text=True, env=self.env, timeout=300) + self.assertEqual(done.returncode, 0, done.stdout + done.stderr) + return done, json.loads(self.state.read_text()) + + @staticmethod + def runs(data): + return [call for call in data['calls'] if call[0] == 'run'] + + def test_an_update_over_the_incus_host_recipes_container_keeps_what_it_reached(self): + self.docker(found=FOUND_052['container']) + done, data = self.apply('-e', 'berth_server_node=true', '-e', 'berth_server_update=true') + [run] = self.runs(data) + words = ' '.join(run) + for kept in ('--add-host incus-host:host-gateway', '--add-host node:host-gateway', + '--volume /var/run/docker.sock:/var/run/docker.sock', '--group-add 989', + '--log-opt max-size=10m', '--restart unless-stopped', + f'--publish 127.0.0.1:{self.port}:7777', f'--volume {self.scratch}/srv/data:/data'): + self.assertIn(kept, words) + self.assertNotIn('/srv/berth/data:/data', words) + self.assertIn(['pull', 'ghcr.io/helightdev/berth'], data['calls']) + self.assertEqual(run[-1], 'ghcr.io/helightdev/berth') + # The server keeps its token, so no setup key is made for a first user. + self.assertNotIn(['exec', 'berth', 'berth', 'server', 'setup-key'], data['calls']) + + # The same run again finds everything in place. + done, data = self.apply('-e', 'berth_server_node=true', '-e', 'berth_server_update=true') + self.assertEqual(self.runs(data), []) + self.assertRegex(done.stdout, r'localhost\s*: ok=\d+\s+changed=0\b') + + def test_a_run_without_an_update_keeps_the_image(self): + self.docker(found=FOUND_052['container']) + done, data = self.apply() + self.assertNotIn('pull', [call[0] for call in data['calls']]) + [run] = self.runs(data) + self.assertEqual(run[-1], 'ghcr.io/helightdev/berth:0.5.2') + self.assertIn('--add-host incus-host:host-gateway', ' '.join(run)) + + def test_a_first_run_on_a_machine_with_incus_gives_the_name_incus_host(self): + self.incus.write_text('made-up certificate') + self.docker() + done, data = self.apply() + [run] = self.runs(data) + words = ' '.join(run) + for given in ('--name berth', '--add-host incus-host:host-gateway', + '--volume /var/run/docker.sock:/var/run/docker.sock', f'--publish 127.0.0.1:{self.port}:7777'): + self.assertIn(given, words) + self.assertNotIn('node:host-gateway', words) + self.assertIn(['exec', 'berth', 'berth', 'server', 'setup-key'], data['calls']) + # Then it is in place. + done, data = self.apply() + self.assertEqual(self.runs(data), []) + self.assertRegex(done.stdout, r'localhost\s*: ok=\d+\s+changed=0\b') + + +if __name__ == '__main__': + unittest.main() diff --git a/tests/test_tailnet_role.py b/tests/test_tailnet_role.py new file mode 100644 index 0000000..7933edb --- /dev/null +++ b/tests/test_tailnet_role.py @@ -0,0 +1,142 @@ +"""How the role joins a host to the tailnet: only with a key berth made, +never by hand, and never again for a host that is on it. The check and the +join run with Ansible against a stand-in for the tailscale client.""" +import json +import os +from pathlib import Path +import shutil +import subprocess +import sys +import tempfile +import unittest + +try: + import yaml +except ImportError: + yaml = None + +ROOT = Path(__file__).resolve().parents[1] +ROLE = ROOT / 'provision/ansible/roles/devbox' +RUNNING = '{"BackendState": "Running", "Self": {"DNSName": "studio.tailnet.test."}}' +LOGGED_OUT = '{"BackendState": "NeedsLogin"}' + + +def playbook(): + """ansible-playbook of the Python that runs the tests, or on the PATH.""" + beside = Path(sys.executable).parent / 'ansible-playbook' + return str(beside) if beside.is_file() else shutil.which('ansible-playbook') + + +@unittest.skipUnless(yaml, 'needs PyYAML: install provision/requirements.txt') +class RoleTextTests(unittest.TestCase): + def tasks(self, name): + return yaml.safe_load((ROLE / 'tasks' / name).read_text()) + + def test_there_is_no_way_to_join_by_hand(self): + text = ''.join((ROLE / 'tasks' / name).read_text() for name in ('tailnet.yml', 'tailnet_check.yml')) + for gone in ('tailscale logout', 'tailscale login', 'needs-enrollment', 'TS_API_KEY'): + self.assertNotIn(gone, text) + defaults = yaml.safe_load((ROLE / 'defaults/main.yml').read_text()) + for name in ('devbox_tailscale_auth_key', 'devbox_tailscale_login_server', 'devbox_tailscale_key_command', + 'devbox_tailnet_problem'): + self.assertIn(name, defaults) + + def test_the_key_never_reaches_argv_or_the_log(self): + tasks = self.tasks('tailnet.yml') + named = {task['name']: task for task in tasks} + self.assertTrue(named['Create a one-use enrollment key with berth']['no_log']) + self.assertTrue(named['Choose the enrollment key']['no_log']) + enroll = named['Enroll Tailscale only when not already enrolled']['block'] + stored = next(task for task in enroll if 'ansible.builtin.copy' in task) + self.assertTrue(stored['no_log']) + self.assertEqual(stored['ansible.builtin.copy']['mode'], '0600') + up = next(task for task in enroll if task['name'] == 'Consume enrollment key') + argv = up['ansible.builtin.command']['argv'] + self.assertIn('--auth-key=file:/run/devbox-tailscale-key', argv) + self.assertNotIn('devbox_tailnet_key', argv) + cleanup = named['Enroll Tailscale only when not already enrolled']['always'][0] + self.assertEqual(cleanup['ansible.builtin.file']['state'], 'absent') + + def test_the_check_comes_before_any_change(self): + main = self.tasks('workstation.yml') + self.assertIn('ansible.builtin.assert', main[0]) + self.assertEqual(main[1]['ansible.builtin.include_tasks'], 'tailnet_check.yml') + self.assertIn('devbox_install_tailscale', main[1]['when']) + # The play of a workstation: a node's comes first, and ends the host there. + e2e = yaml.safe_load((ROOT / 'tests/e2e/site.yml').read_text())[1]['tasks'] + self.assertEqual(e2e[2]['ansible.builtin.include_role']['tasks_from'], 'tailnet_check.yml') + + +@unittest.skipUnless(yaml and playbook(), 'Run with the Ansible venv Python for the role tasks') +class TailnetTaskTests(unittest.TestCase): + def setUp(self): + self.dir = Path(tempfile.mkdtemp()) + self.addCleanup(shutil.rmtree, self.dir) + bin_dir = self.dir / 'bin' + bin_dir.mkdir() + self.calls = self.dir / 'calls' + self.status = self.dir / 'status.json' + # The client says what status.json holds and notes every call. + tailscale = bin_dir / 'tailscale' + tailscale.write_text(f'#!/bin/sh\necho "$*" >> {self.calls}\n' + f'[ "$1" = status ] && cat {self.status}\nexit 0\n') + tailscale.chmod(0o755) + self.path = f'{bin_dir}:{os.environ["PATH"]}' + self.made = self.dir / 'made' + + def run_tasks(self, name, status, **variables): + self.status.write_text(status) + (self.dir / 'play.yml').write_text(yaml.safe_dump([{ + 'hosts': 'localhost', 'gather_facts': False, + 'vars': {'devbox_tailscale_auth_key': '', 'devbox_tailscale_login_server': '', + 'devbox_tailscale_key_command': [], 'devbox_tailnet_problem': '', **variables}, + 'tasks': [{'ansible.builtin.include_tasks': str(ROLE / 'tasks' / name)}]}])) + return subprocess.run( + [playbook(), '-i', 'localhost,', '-c', 'local', str(self.dir / 'play.yml')], + capture_output=True, text=True, env=dict(os.environ, PATH=self.path, ANSIBLE_NOCOLOR='1')) + + def calls_made(self): + return self.calls.read_text().splitlines() if self.calls.exists() else [] + + def test_a_host_off_the_tailnet_without_a_way_to_a_key_stops_with_berths_words(self): + problem = 'studio joins the tailnet at Tailscale, and berth cannot make its key. berth secret set tailnet_api_key' + done = self.run_tasks('tailnet_check.yml', LOGGED_OUT, devbox_tailnet_problem=problem) + self.assertNotEqual(done.returncode, 0) + self.assertIn(problem, done.stdout) + done = self.run_tasks('tailnet_check.yml', '') + self.assertNotEqual(done.returncode, 0) + self.assertIn('secret tailnet_api_key', done.stdout) + self.assertIn('devbox_install_tailscale to false', done.stdout) + + def test_a_host_of_berth_0_5_that_was_never_joined_goes_on_without_the_tailnet(self): + problem = 'studio joins the tailnet at Tailscale, and berth cannot make its key. berth secret set tailnet_api_key' + done = self.run_tasks('tailnet.yml', LOGGED_OUT, devbox_tailnet_problem=problem, + devbox_tailnet_optional=True) + self.assertEqual(done.returncode, 0, done.stdout + done.stderr) + self.assertIn('The apply goes on without the tailnet', done.stdout) + self.assertIn(problem, done.stdout) + # Nothing tried to join it. + self.assertEqual(set(self.calls_made()), {'status --json'}) + + def test_a_way_to_a_key_lets_the_play_go_on(self): + command = ['touch', str(self.made)] + self.assertEqual(self.run_tasks('tailnet_check.yml', LOGGED_OUT, + devbox_tailscale_key_command=command).returncode, 0) + self.assertEqual(self.run_tasks('tailnet_check.yml', LOGGED_OUT, + devbox_tailscale_auth_key='tskey-auth-kFAKE-0000').returncode, 0) + # The check makes no key. + self.assertFalse(self.made.exists()) + + def test_a_host_on_the_tailnet_is_left_alone_even_without_a_key(self): + problem = 'berth cannot make its key' + done = self.run_tasks('tailnet.yml', RUNNING, devbox_tailnet_problem=problem) + self.assertEqual(done.returncode, 0, done.stdout + done.stderr) + done = self.run_tasks('tailnet.yml', RUNNING, devbox_tailscale_key_command=['touch', str(self.made)]) + self.assertEqual(done.returncode, 0, done.stdout + done.stderr) + # No key is made, and the client is only asked how it is. + self.assertFalse(self.made.exists()) + self.assertEqual(set(self.calls_made()), {'status --json'}) + + +if __name__ == '__main__': + unittest.main()