| title | ctrl-exec - Security Model |
|---|---|
| subtitle | Trust boundaries, threat model, file permissions, and operational guidance |
| brand | odcc |
ctrl-exec is built around one guarantee:
A holder of a valid dispatcher certificate can invoke the approved actions and nothing else, and cannot alter the controls (allowlist, auth hook, config, trust) or the audit trail - even with full control of the dispatcher. Within that, an action runs with whatever privilege the operator gave its profile.
Security is enforced at ctrl-exec's own layer - the allowlist, mTLS identity, the auth hook policy engine, per-script profiles, and the audit - not by crippling the agent's ability to do its job. A managed script runs with the privilege its work needs; what is controlled is which script, invoked by whom, and recorded. What arguments it may be given is constrained by the auth hooks, where a site chooses to write that policy - core itself never validates argument values (see Argument validation scope).
This is what makes ctrl-exec a sound replacement for ssh + sudo: it does the
same operational work, but the surface is a fixed set of named, reviewed,
cryptographically-authenticated, audited operations (with argument policy
available through the auth hooks) - and there is no shell.
Think of two boundaries an attacker would have to cross, and what each enforces.
First boundary - the dispatcher / API. The dispatcher (and its optional HTTP
API) is where requests originate. It is authenticated, authorised by the auth
hook, rate-limited, and exposes only run/ping/discovery. Security here does
not depend on the protocol being secret - it is assumed known.
Second boundary - the agent. This is the real enforcement point. Even an
attacker who bypasses the dispatcher entirely and speaks the mTLS protocol
directly to an agent still needs a CA-signed dispatcher certificate, and even
with one is bound by the agent to exactly the approved action set: the
allowlist (only named scripts), per-script profiles (the privilege each runs
with), and the auth hook. Arguments are constrained only if the agent's
auth_hook checks them: the agent passes the arguments to that hook, and core
itself does not validate argument values. It cannot
extend the allowlist, edit the hook/config, change trust, or erase audit - the
executor runs every action with the agent's control and state directories
read-only (see Allowlist and Execution Security).
The headline: the dispatcher is a control and convenience layer, not a trusted one. The agent is the enforcement boundary, and its guarantees hold under full dispatcher compromise.
Suppose an advanced AI drives the dispatcher's API and is taken over by a bad actor - with API access, and even read access to the source.
- Reading the source gains nothing: security is not by obscurity; the protocol is assumed public.
- Through the API, the attacker can do only what the dispatcher is authorised to do - bounded by the auth hook and rate limits, and every call is attributed and logged.
- If it bypasses the API and speaks mTLS protocol straight to agents, it is held to the same bound by each agent: the approved, profiled and audited action set, at the declared privilege - and it cannot touch the agents' controls or audit.
So compromising the dispatcher (or the AI driving it) does not escalate beyond the action set the operator already approved. The blast radius is "everything the dispatcher was allowed to do" - fully recorded - and no more.
ctrl-exec uses a private CA for all mTLS trust. The CA is created once on the dispatcher host. All agent certificates are signed by this CA. Neither the dispatcher nor any agent trusts certs from public CAs or other sources for the operational port - only certs from the private CA are accepted.
The CA private key never leaves the dispatcher host. Loss of the CA key means all agent certs must be reissued via re-pairing.
Several distinct dispatchers each keep their own CA (0.14). An agent trusts a bundle of their CAs; each dispatcher verifies the agents it paired normally against its own CA, and an agent another dispatcher's CA certified against that CA, pinned in that agent's registry entry at pairing (Pairing with another dispatcher, below). Only redundant instances of one dispatcher (HA) share a CA.
In a container the same split holds: the API runs as the unprivileged
ctrl-exec user and cannot read the CA key, which only root operations
(setup-ca, pairing-mode, approve, rotate-cert, maintain) can. See
DOCKER.md, Container privilege model.
Port 7443 - operational (mTLS)
: All run, ping, capabilities, and cert renewal traffic. Both dispatcher
and agent must present a valid cert signed by the private CA.
SSL_verify_mode => SSL_VERIFY_PEER is set on both sides - there is no
fallback to unauthenticated. An agent with no cert, an expired cert, or a
cert from a different CA cannot connect.
Port 7444 - pairing
: Bootstrap port. The agent has no cert yet when it first connects, so client
cert is not required. This is a deliberate bootstrap exception. Mitigations:
the operator reviews the displayed hostname and IP before approving; the port
is only open while pairing-mode is actively running; a random nonce
prevents misrouted or replayed approvals (see below). Close the pairing port
promptly after completing the pairing session.
Port 7445 - API
: No mTLS. All endpoints pass through the auth hook. The default behaviour
when no hook is configured is controlled by api_auth_default in
ctrl-exec.conf. The shipped default is deny - all requests are
rejected until a hook is configured. Set to allow only on isolated
networks where no credential checking is needed.
The API binds to 127.0.0.1 by default (api_bind in ctrl-exec.conf).
It is not reachable from the network unless api_bind is explicitly changed.
Same-host callers can use a unix socket instead (api_socket; api_port = off closes the TCP port). The socket is created mode 0660, so only its
owner and members of api_socket_group can connect at all; the auth hook
still decides every request, and sees the caller's uid as reported by the
kernel (ENVEXEC_PEER_UID), which, unlike a token, a caller cannot forge. The uid of a run's starter is
stored with the run and handed to the hook on a later status request
(ENVEXEC_OWNER_PEER_UID), so a hook can return a run's output only to the
uid that started it.
Loopback TCP, by contrast, is open to every local user up to the hook.
For internet-facing deployments, place the API behind a reverse proxy that
handles authentication and TLS termination rather than binding directly to
an external interface.
Pairing code verification
: Each pairing request has a 6-digit confirmation code. The agent displays it
once it has connected to the pairing port; the dispatcher displays it
alongside the hostname and IP in the approval prompt, list-requests and
approve. The operator verifies both sides match before approving. Each side
computes the code independently - nothing extra crosses the network.
From 0.13 the code (v2) is the first 4 bytes of
SHA-256(CSR || F), as an integer modulo 1,000,000, where F is the
SHA-256 fingerprint of the DER of the TLS certificate on the pairing
connection: the agent hashes the certificate it was shown, the dispatcher
the certificate its pairing listener presents. A matching code shows that
the dispatcher received this agent's CSR and that the agent was talking
to this dispatcher's pairing listener. Without --dispatcher-fingerprint
(see Verified pairing) the agent connects without verifying the dispatcher
(it has no CA yet), so a man in the middle can terminate its TLS - but it
then presents its own certificate, the agent's code is computed over that,
and the operator sees two different codes. Before 0.13 the code
was SHA-256(CSR) alone, which a relay forwarding the CSR unchanged
reproduced exactly.
The dispatcher only accepts a CSR in exactly the form openssl writes it. openssl ignores text around the PEM block, so without this a relay could append text to the agent's CSR - changing the hashed bytes but not the request that is signed - and try variants until the codes matched.
Limits. The code is six digits. An attacker in the path who substitutes a CSR of its own (its own key) can still try one CSR after another until the dispatcher's code for it equals the code the agent shows - about a million CSR signatures on average, within the 10-minute pairing window. The code makes a relay visible; it does not make a determined, well-resourced attacker in the path impossible. Treat a mismatch as an attack.
The code alone is adequate when the path between agent and dispatcher is one an attacker cannot sit in: a closed network you control, or an authenticated path such as an SSH tunnel to the dispatcher's pairing port. On any other network - shared, routed over the internet, or simply not yours - use verified pairing.
Verified pairing
: ctrl-exec-agent request-pairing --dispatcher-fingerprint SHA256:<hex>
gives the agent the SHA-256 fingerprint of the dispatcher's CA certificate,
taken out of band from the dispatcher (ctrl-exec-dispatcher ca-fingerprint, or the pairing-mode banner, which prints the whole agent
command). The pairing listener (0.14 and later) sends its CA certificate
after its own. Once the TLS handshake is done and before anything is sent,
the agent finds the certificate with that fingerprint in the presented
chain and runs a full X.509 verification of the dispatcher's certificate
with that CA as the only trust anchor: signature, validity dates and CA
constraints. The TLS handshake has already proved the peer holds the key of
the certificate it presented. If the CA is not in the chain, or the
presented certificate was not issued by it, the agent stops with an error
naming the expected and the seen fingerprints, and its CSR is never sent. A
man in the middle cannot pass by presenting the real CA certificate (it is
public); it needs a certificate that CA issued, and the key for it.
The fingerprint is of the CA, not of the dispatcher certificate, so it stays
the same across rotate-cert. The host name is not checked: the pinned CA is
the trust. With the channel verified, the six-digit code remains the
operator's check that the pending request is the one the agent just made; it
can no longer be ground, because a man in the middle never sees the CSR.
The dispatcher cannot require verification - an agent's claim to have verified would be unauthenticated - so it advises: the banner says to use the command on any untrusted network. Without the flag pairing runs as before. A dispatcher older than 0.14 does not send its CA certificate, so an agent given the flag refuses it; upgrade the dispatcher first.
Certificate roles. The dispatcher's CA also issues the agents'
certificates. From 0.14 it marks every certificate it issues with its role
(see Certificate Roles below), and verified pairing refuses a presented
certificate marked agent - Pairing refused: saying it is an agent
certificate, ACTION=pair-verify-failed, and the CSR is not sent. It
accepts one marked dispatcher, and one with no mark.
What verified pairing accepts. Before the role marks: any certificate the pinned CA issued (and that verifies), the dispatcher's or an agent's. From 0.14, of the certificates the pinned CA issued:
- marked
dispatcher: accepted; - unmarked (issued before 0.14), the dispatcher's or an agent's: accepted, as before - the check cannot tell them apart;
- marked
agent: refused; - carrying the role extension with any other value: refused.
A certificate the pinned CA did not issue is refused, as before.
Limit. Certificates issued before 0.14 carry no mark, and nothing is
reissued on upgrade: an agent's certificate is marked at its next automatic
renewal (past half its life), the dispatcher's own at its next rotation
(rotate-cert, or automatically before expiry). Until then a host holding
a paired agent's unmarked certificate and key still passes the check; in
the path, it is back to the grinding window the code alone leaves (above).
ctrl-exec-dispatcher rotate-cert and the agents' renewal close the gap
sooner. Protect agent keys as the credentials they are, and revoke a
compromised agent promptly.
Pairing with another dispatcher : An agent already paired (its certificate verifies against a CA in its bundle and matches its key) pairing with another dispatcher B keeps its key and certificate. Its request carries a CSR made from its existing key, its certificate and the CA that issued it; the pairing code is computed over all three, so a relay cannot swap the certificate or CA under an unchanged CSR. What each side accepts:
- B, at the request: the certificate and CA must each be exactly one
canonical PEM certificate, the CSR's self-signature must verify (the
sender holds the key), the CSR and the certificate must carry the same
public key, and the certificate must verify (signature, dates, CA
constraints) with that CA as the only trust anchor. Otherwise the request
is refused before it is queued (
pair-reject REASON=kept-cert-invalid). The operator sees both fingerprints and the code before approving. - B, after approval: nothing is signed. The CA is pinned in that agent's
registry entry, and B's TLS client verifies that agent against that CA
alone - not B's own CA, not a shared bundle - so a certificate from any
other CA, B's own included, is refused for that agent, and the pinned CA
vouches for no other agent. Agents without a pin are verified against B's
caas before. B does not renew a pinned agent's certificate. If the pinned CA is B's own, nothing is pinned. - The agent: B's acknowledgement must confirm it keeps the certificate, and
its approval must carry no certificate; otherwise the agent stores
nothing (a dispatcher before 0.14 would sign a new certificate, which
would replace the one the other dispatchers verify). With
--dispatcher-fingerprint, the CA delivered at approval must be that CA and only it. It then appends B's CA to its bundle and B to its map;agent.crtandagent.keyare untouched. Its listener takes the new bundle without a restart.
Pinning the CA rather than the certificate keeps renewal by the issuing
dispatcher working for B. The cost: B accepts, at that agent's address, any
certificate the pinned CA issued - the same property an agent paired
normally has under its dispatcher's CA. --rekey makes a new key and a
certificate from the dispatcher pairing, as an unpaired agent does; the
other dispatchers then refuse the agent until re-paired.
Pairing code versions
: The agent states the code version it computed in its pairing request.
v2 is shown as 123456 (v2). An agent older than 0.13 sends no version
and gets the old code, shown as 123456 (v1, pre-0.13 agent) with a note
that it is not bound to the dispatcher: a matching v1 code does not rule out
a relay. Upgrade agents to 0.13 before pairing them where this matters. A
0.13 agent paired with a dispatcher older than 0.13 shows a v2 code the
dispatcher's v1 code will not match (the agent warns when the dispatcher does
not confirm v2) - upgrade dispatchers first. Existing pairings are
unaffected: the code is used only while pairing.
Nonce verification : Each pairing request includes a 32-hex-character random nonce generated by the agent. The dispatcher stores it with the pending request and echoes it in the approval response. The agent verifies the nonce matches before storing any certs. This prevents a race condition where an approval for a different concurrent pairing request is accepted by the wrong agent.
Preflight writability check
: ctrl-exec-agent request-pairing checks that /etc/ctrl-exec-agent is
writable before making any network connection. If not writable, it dies
immediately rather than leaving a stale pending request in the dispatcher's
pairing queue.
Stale request cleanup : Pending requests older than 10 minutes with no approval or denial are automatically deleted from the pairing queue.
Hostname validation
: The agent-supplied hostname becomes the agent's registry key (a filename).
It is validated against ^[A-Za-z0-9][A-Za-z0-9._-]*$ before the request is
queued and again before the registry entry is written, so a hostile pairing
client cannot use a crafted hostname (e.g. ../../...) to write outside the
registry directory on approval. The same validation guards edit-agent
renames.
Queue depth limit
: The pairing queue is capped at 10 pending requests by default (configurable
via pairing_max_queue in ctrl-exec.conf). Once the cap is reached,
further connection attempts are rejected immediately with a structured error
response. Stale expiry runs before the count is checked, so aged-out entries
do not consume quota. This prevents a flood of pairing requests from an
attacker across multiple source IPs from burying a legitimate request in
list-requests output.
Operator review
: The operator is the last line of defence for pairing. Always verify the
hostname and source IP displayed in list-requests or the interactive prompt
before approving.
Renewal uses the already-authenticated mTLS connection on port 7443. No new trust is established - the renewal exchange is carried entirely within a session that both sides have already authenticated.
The agent reuses its existing private key across renewals. Only the cert is replaced. This preserves key continuity and means a renewal does not require generating or protecting new key material.
The dispatcher only renews certs for hosts in its registry. An agent that has been unpaired cannot receive a renewal.
From 0.14 the dispatcher CA marks every certificate it issues with its role:
dispatcher for the dispatcher's own certificate (setup-ctrl-exec,
rotate-cert, automatic rotation) and agent for every certificate signed
from a CSR (pairing approval, renewal). The CA sets the role from which of the
two it is doing; a role or any other extension a CSR asks for is never copied.
Verified pairing refuses a pairing listener presenting an agent certificate
(see Verified pairing).
The mark is a non-critical private X.509 extension, OID
2.25.266442206991433179228746487034237142618 (self-assigned under the UUID
arc 2.25 from a random UUID; the project holds no private enterprise number),
whose value is the UTF8String dispatcher or agent. TLS implementations
ignore an unknown non-critical extension, so marked certificates work with
every peer that took unmarked ones. Extended key usage cannot carry the role:
agents are TLS servers on 7443 and dispatchers clients there, the reverse on
the pairing port. openssl x509 -noout -text -in <cert> shows the mark as
that OID followed by the role.
Certificates issued before 0.14 have no mark and keep working; agents gain it at their next renewal, the dispatcher at its next rotation.
Script name validation
: All script names are validated against /^[\w-]+$/ before allowlist lookup.
This pattern excludes /, ., spaces, and shell metacharacters. Path
traversal is impossible at the name level.
No shell execution
: Scripts are executed via exec { $path } $path, @args - the two-argument
form that bypasses PATH lookup and invokes execve() directly. No shell is
involved. Shell metacharacters in arguments have no effect.
Script directory restriction
: With script_dirs set in agent.conf, any allowlist entry pointing outside
an approved directory is rejected at load time. The check is repeated at
execution time, guarding against allowlist modifications between agent startup
and a run request. At execution time the path is resolved with abs_path
before the comparison, so an entry that uses .. or a symlink to escape an
approved directory is rejected on the real, canonical path. When script_dirs
is not set, any absolute path is accepted.
Script ownership (checked; enforced with script_ownership = strict)
: The rule: an allowlisted script, and every directory from / down to it, is
owned by root (uid 0) and writable by neither group nor other, and the
script is a regular file. Otherwise anyone able to write the script, or any
directory on its path, could change what root (or a profile's run_as user)
runs. Group-writable is refused too, including the sticky world-writable
/tmp and /var/tmp; a group that may read and execute (root:ctrl-exec-agent 0750, as installed) is fine. A symbolic link on the path is followed only
after the directory holding it has passed (so only root can have made or
changed it), and its target is checked by the same rule. The rule holds
whatever the profile's run_as: scripts.conf is root's policy, so its
scripts are root's files.
What a breaking script does is set by script_ownership in agent.conf
(REFERENCE.md):
warn(the default): the script runs. The agent logs the reason each time it loadsscripts.conf(ACTION=script-ownership, WARNING) andcea self-checkshows it next to the script; nothing is added to a run's output. This is the trade-off:warnaccepts that a user who can write the script, or a directory on its path (a group-writable/srv/scripts, say), can change what a profile runs - as root, for a root profile. It exists so that an existing layout keeps working on upgrade.strict: the script is refused, as described below. Recommended onceself-checkshows no ownership warnings: fix the layout (chown root,chmod go-w), then setscript_ownership = strictand reload.
With strict, the executor enforces it before every run (it is the authority). In the
action's own process, after the profile is applied, it walks the path with
openat(O_PATH|O_NOFOLLOW) from each checked directory, checks each component
with fstat, and refuses with exit 126 and executor: script not permitted: '<path>' is group-writable (or is world-writable, is owned by uid N, not root, is not a regular file, is not a directory, has too many levels of symbolic links). It then execs the script by its path, after confirming the
path still names the file it checked (same device and inode), or refuses
with '<path>' changed while it was being checked. Between that check and
execve only root can change what the path names, since every directory on
it is root-owned and not writable by others. The script is exec'd by path,
not by file descriptor (fexecve), on purpose: for a #! script the kernel
would give the interpreter /dev/fd/N as the script's name, so $0,
dirname "$0" and FindBin would stop finding the script's own files.
ctrl-exec-executor check-path <path> prints the verdict for one path.
With warn the executor still runs the check but runs the script whatever
the verdict, binding the exec to the file the path names at that moment
(the same device and inode check); it writes nothing into the run.
The front end applies the same rule (the parity is tested) at allowlist
load and, with strict, again before each run. With strict a breaking
entry is skipped with the reason,
so a run answers script not permitted: <reason>, the reason is logged
(ACTION=allowlist-skip) and cea self-check lists it before any run.
Without an executor (direct mode) the front end is the enforcement. A script
that does not exist yet, or that the agent user cannot reach, gets no
front-end verdict; the executor decides when it runs.
Allowlist is server-enforced : The allowlist is validated on the agent, not trusted from the dispatcher request. A dispatcher cannot request a script that is not in the agent's allowlist regardless of what it sends.
Privilege separation and per-script profiles (optional, recommended)
: With executor_socket set in agent.conf, the network-facing agent runs
unprivileged and hands each authorised run to a small, root, no-network
executor (ctrl-exec-executor) over a unix socket. The socket is owned by the
agent user, mode 0600, and the executor rejects any connection whose
SO_PEERCRED uid is not the agent (a defence-in-depth check on top of the
permissions). The executor re-derives the script path and its security
profile from its own root-owned config - it trusts nothing in the request - so
even a compromised front-end can only run an allowlisted script with its pinned
profile, never an arbitrary command. It then applies the profile before
exec: a mount namespace in which /etc/ctrl-exec-agent and
/var/lib/ctrl-exec-agent (the control and state directories) are read-only
- so even a
run_as=rootaction cannot tamper with the allowlist, hook, config, trust map, or audit - the capability set the profile lists,run_as, andno_new_privileges. Every profile must be defined inagent.conf, includingdefault(the profile an unannotated script resolves to) - there is no built-in fallback, so a script whose profile is undefined is a fatal config error (fail-closed) rather than running under an implicit context. The shippedagent.conf.exampledefines[profile default]asrun_as=nobody; the rule is that nothing runs as root unless a profile explicitly setsrun_as=root. Withoutexecutor_socket, the agent runs scripts directly in its own unprivileged process (no profiles). Profiles are defined inagent.conf([profile <name>]) and referenced fromscripts.conf(profile=<name>); seeagent.conf.example.
Profiles must never be dropped silently. When executor_socket is set and the
executor is absent or refusing, a run fails with an error naming the
executor (it does not fall back to running unprofiled), and self-check
reports it. The one place the setting is changed for the operator is
install.sh on a host that cannot build the executor (no gcc or libcap
headers): it writes the new agent.conf with executor_socket commented out
and states at the end of the install that profiles are not applied; the agent
also warns at start when scripts.conf names profiles with no
executor_socket. An existing agent.conf is never edited.
This split is what bounds a front-end compromise (e.g. an RCE in the network-facing code): the only thing that can act with privilege is the small, audited executor, and it can run only allowlisted scripts under their declared profiles. The privileged surface is kept deliberately tiny.
The action's environment under the executor
: The executor never passes its own environment to an action, and nothing in
the request enters it (request context reaches the script on stdin), except
for a script declared input=raw, raw:<name> or fields: it gets the
context JSON, without input and without the caller's token, in
CTRL_EXEC_CONTEXT, and for fields the input directory's path in
CTRL_EXEC_INPUT (see "Confidential input at the executor"). The
action gets a fixed PATH
(/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin). Under a
profile with run_as, it also gets HOME, USER and LOGNAME from the
agent host's own passwd entry for that user (pw_dir, pw_name), as su -
and systemd's User= set them, so the user's tools (docker, git, ssh) find
their configuration. A numeric run_as with no passwd entry gets none of the
three, and a profile without run_as gets PATH alone. Loader and shell
variables that are attack surface for script interpreters (LD_*,
BASH_ENV, IFS, PERL5LIB, and the rest) are never passed.
Setting HOME does not create it: a run_as user needs a real home
directory if its tools write there. A profile with writable runs on a
read-only filesystem, so it must also list that home (or the parts of it the
tools write) in writable. The executor unit sets no ProtectHome, so home
directories are visible to actions.
The action's file descriptors under the executor
: The action inherits stdin, stdout and stderr and no other file descriptor:
0, 1 and 2 are the pipes the executor feeds and drains, and it holds nothing
of the executor's. The executor's connection to the agent is accepted
close-on-exec, as are its listening socket, the input directory fd and the
files and directories it opens; in the action's process, just before
execve, every fd from 3 up is closed (close_range, or each fd listed in
/proc/self/fd on a kernel without it), which also covers any fd the
executor itself inherited from whatever started it. Before 0.13.0 the
connection socket stayed open in the action, so a script could write its
own result frame to the agent (any exit status and output) or corrupt the
real one. An input=fields action reads its files by the path in
CTRL_EXEC_INPUT, never through an inherited descriptor. If the executor
itself is started with any of fd 0-2 closed, it fills them with /dev/null
before opening anything, so none of its sockets, pipes or directories can
take those numbers.
Schema sidecars are advertised, never executed
: A script's optional <script>.schema.json sidecar (see REFERENCE.md) is read
only for allowlisted scripts, capped at 64 KiB, and treated as opaque data:
the agent advertises it on /capabilities but never parses its arguments /
argv, never validates against it, and never lets it influence execution. The
allowlist remains the sole gate on what runs - a sidecar cannot widen the
callable surface, and a sidecar beside a non-allowlisted script is never read.
This is what lets the schema be safely consumed by an LLM via the MCP bridge:
the model can only select an operator-approved script and fill
operator-defined argument fields, never invent operations.
JSON context on stdin
: Scripts receive full request context as JSON on stdin (script name, args,
reqid, peer IP, username, token, timestamp). The agent writes this context
with a non-blocking write loop and a configurable timeout (stdin_timeout
in agent.conf, default 10 seconds). If the script does not read stdin and
the pipe buffer fills, the agent logs ACTION=stdin-timeout and closes the
write end, delivering EOF to the script. The script continues to execute.
Scripts that do not use stdin context do not need any special handling.
Confidential input
: A request may carry input (named string fields) for values that must not
appear in the process table: keys, passwords. The script declares in
scripts.conf how it receives them (input=context|raw|raw:<name>|fields,
see REFERENCE.md): inside the stdin JSON, as stdin bytes, or as 0600 files in
a private 0700 directory removed after the run. Input is never placed in
argv or in the environment (CTRL_EXEC_INPUT is only a path), never given
to either auth hook (they see field names and the total size), never logged
and never written to the run store. A script that echoes its input to
stdout or stderr returns it in the result - that is the script's choice.
On the way in, input should travel only over the API's unix socket
(api_socket) or TLS (api_tls_cert/api_tls_key), never plain HTTP across a
network: the API body carries the values. The dispatcher-to-agent hop is
always mTLS. ctrl-exec-cli warns on stderr when --input or
--input-fields would be sent over plain HTTP to a non-loopback address.
Confidential input at the executor
: With executor_socket set, the agent hands the input to the executor in the
run request (frame v2) and the executor delivers it; the executor never
parses JSON. What root does with it:
- It takes the input mode from its own root-owned
scripts.conf, as it does the path and profile, never from the request. A request that does not fit the mode (rawwithout exactly one field,raw:<name>without that field) is refused before anything is created or forked, naming the field. - It bounds the input itself before using any of it: at most 32 fields, names
[A-Za-z0-9_.-]{1,64}other than.and.., no name twice, at most 4 MiB of values, and nothing after the last field. Field values are read in place from the request buffer and never logged; that buffer and the copy of the context-mode stdin are zeroed before they are freed. raw: the field's bytes are written to the action's stdin by the same non-blocking loop that feeds the context JSON.fields: before the fork, root makes/run/ctrl-exec-agent/input/<reqid>. <16 random hex>withmkdiratmode 0700 in the root-owned, 0711inputdirectory (reachable by name, not listable), opens itO_NOFOLLOWand checks it withfstat, creates each file withopenatrelative to that directory fd (O_CREAT|O_EXCL|O_NOFOLLOW, mode 0600), writes it, andfchowns each file and then the directory to the profile'srun_asuser and group. No path that another user could swap is used. The action is then forked and the profile applied (mount namespace,writable, capabilities,run_as,no_new_privileges) exactly as before; just beforeexecvethe child checks thatCTRL_EXEC_INPUTstill names that directory (same device and inode) and that it is owned by the uid it now runs as. Because the directory exists before the namespace is made, awritableprofile sees it read-only. Only therun_asuser (and root) can read the files; the agent user cannot.- Removal: once the action has been reaped - a normal exit, a crash, or a
kill from outside - the handler removes the directory, walking it by
directory fds opened
O_NOFOLLOWand unlinking withunlinkat, so anything the action placed there (links, subdirectories, FIFOs) is removed and never followed. If the handler itself dies, the executor removes every entry in theinputdirectory at its next start-up. A process the action left running keeps its open files until it exits. - The environment gains only
CTRL_EXEC_CONTEXT(the agent's context JSON without input and without the caller's token) forrawandfields, andCTRL_EXEC_INPUT(the path) forfields. Executor messages name fields and request ids, never values.
Confidential output (secrets at rest)
: By default the dispatcher stores every run's result, stdout and stderr
included, in /var/lib/ctrl-exec/runs/<reqid>.json (0640, group
ctrl-exec) for 24 hours, and an async result also waits on the agent in
/var/lib/ctrl-exec-agent/runs/ until fetched. A script that prints a
secret (a new private key) is declared output=confidential in the agent's
scripts.conf (see REFERENCE.md). Its output then reaches the caller in the
synchronous response and is stored nowhere: the agent marks the result
output_confidential, the dispatcher's run store writes such a result with
the exit status and metadata only (sync records, and any async result
fetched with the mark), and the agent refuses async runs of the script, so
nothing is held agent-side. No log line on the agent, executor or dispatcher
carries a run's output, for any script. The executor holds the output in
memory only, to hand it back to the agent.
The protection rests on the agent's declaration. The dispatcher cannot tell that output is secret unless the result is marked, so an unmarked result - any script not declared, or any result from an agent older than 0.13.0 - is stored whole. What the caller does with the output it receives is outside ctrl-exec.
Scripts limited per dispatcher (dispatchers=)
: A scripts.conf entry with dispatchers=<id>[,<id>...] runs, and is listed
in /capabilities, only for those dispatchers (see REFERENCE.md). The id is
the one the agent resolved from its own trusted map for the connection's
certificate, keyed by issuing CA and serial, so one dispatcher's CA cannot
mint a certificate that passes as another dispatcher; nothing the request
says is used.
The executor checks the limit again from its own read of the root-owned
scripts.conf, as it does input=. It has no TLS view, so the dispatcher id
it checks is the agent front end's statement in the run frame: the executor
cannot verify which dispatcher connected. What its check does hold: a
front end that is out of step with scripts.conf (not yet reloaded, or a
bug in its check) cannot run a limited script for a dispatcher the file does
not list, and a request with no valid dispatcher id never runs a limited
script. A compromised front end can name any listed id, as it can choose any
allowlisted script; the limit is not a boundary against the agent user, the
allowlist and profiles are.
Two separate hooks exist with different scopes.
The dispatcher-side hook (configured via auth_hook in ctrl-exec.conf)
is called before every run, ping, capabilities, and API request. It
is the sole access control policy engine for the dispatcher — the dispatcher
has no built-in ACLs.
The agent-side hook (configured via auth_hook in agent.conf) is called
after allowlist validation on the agent, before script execution. It covers
run requests only — ping and capabilities requests do not invoke the
agent hook. The hooks are independent: both can be configured simultaneously,
or only one, or neither.
Default auth mode
: When no hook is configured, behaviour depends on the caller. CLI invocations
(ced run, ced ping) unconditionally pass - CLI access is
already gated by system user and group permissions. API callers are governed
by api_auth_default in ctrl-exec.conf. The default is deny - all API
requests are rejected without a hook. Set to allow for isolated networks
where credential checking is not needed. This setting has no effect when a
hook is configured.
username is advisory
: The username field is a caller-supplied string. The dispatcher does not
authenticate it or verify it matches any local or remote account. It is
forwarded unchanged to the hook and to the agent. Its intended purpose is
to carry an identity assertion that the hook can forward to an external
authentication service alongside the token - the service validates whether
the claimed identity is consistent with the token's authority. A hook that
grants elevated permissions based solely on username without validating
it via the token or an external mechanism can be bypassed by any caller
that sets the field to a privileged value. See SECURITY-OPERATIONS.md for
the recommended pattern.
Argument inspection
: Always use ENVEXEC_ARGS_JSON in hook scripts to inspect script
arguments. This is a reliable JSON array. ENVEXEC_ARGS (space-joined)
is set for backward compatibility but is deprecated - it is lossy for
arguments containing spaces or newlines, and naive pattern-matching on it
can be bypassed by crafted argument values.
Per-dispatcher identity
: Auth hooks receive ENVEXEC_DISPATCHER (the stable dispatcher id) and
ENVEXEC_DISPATCHER_SERIAL (the connecting cert serial). Per-dispatcher
policy must key on ENVEXEC_DISPATCHER, never on the serial, because
serials rotate while the id is stable.
Token forwarding : Tokens are included in the JSON payload sent from the dispatcher to the agent, and in the JSON context piped to scripts on stdin. This supports token validation at every stage of an execution pipeline. Each hop can independently verify the token is still valid and still authorised for the stated purpose, without trusting the previous hop.
Hook isolation
: The hook executable is run via fork/exec. local $SIG{CHLD} = 'DEFAULT' is
set before forking to prevent the API server's SIGCHLD reaper from collecting
the hook process before waitpid can. stdout and stderr of the hook are
redirected to /dev/null - output from the hook does not reach the caller.
Token logging
: Tokens are never logged by the dispatcher or the agent. They appear in the
hook's environment and in JSON stdin. Do not log environment variables within
the hook; log only specific fields from stdin. A hook that logs env output
exposes the token in syslog.
Token in CLI
: Pass tokens via ENVEXEC_TOKEN environment variable rather than --token
to prevent the value appearing in ps output.
Hook must not produce output : The hook's stdout and stderr are discarded. Audit logging within the hook should use syslog.
Agent-side hook
: The agent-side hook (configured via auth_hook in agent.conf) runs
after allowlist validation, before script execution. It covers run
requests only — ping and capabilities do not invoke it. The hook
receives the same request context including the forwarded username and
token. It does not receive a hosts field; the agent is unaware of
which other agents are targeted in the same invocation. For
source-based restriction on the agent, use allowed_ips in agent.conf
or ENVEXEC_SOURCE_IP in the hook. Behind a reverse proxy this is the proxy's address, not the client's, so hooks must not use it for client identity there. If no agent hook is configured,
the agent authorises unconditionally at the agent level, relying on
mTLS and the allowlist as its primary controls.
/etc/ctrl-exec-dispatcher/ca.key 0600 root CA private key
/etc/ctrl-exec-dispatcher/ca.crt 0644 root CA certificate
/etc/ctrl-exec-dispatcher/dispatcher.key 0600 ctrl-exec dispatcher private key (API service user reads it; root also can)
/etc/ctrl-exec-dispatcher/dispatcher.crt 0644 root dispatcher certificate
/etc/ctrl-exec-dispatcher/dispatcher.key.prev 0600 ctrl-exec previous key, kept from a rotation until the old serial is retired
/etc/ctrl-exec-dispatcher/dispatcher.crt.prev 0644 root previous certificate, likewise
/etc/ctrl-exec-dispatcher/auth-hook 0755 root Auth hook executable
/etc/ctrl-exec-dispatcher/ 0750 root:ctrl-exec
/etc/ctrl-exec-agent/agent.key 0640 root:ctrl-exec-agent
/etc/ctrl-exec-agent/agent.crt 0640 root:ctrl-exec-agent
/etc/ctrl-exec-agent/ca.crt 0644 root
/etc/ctrl-exec-agent/agent.conf 0640 root:ctrl-exec-agent
/etc/ctrl-exec-agent/scripts.conf 0640 root:ctrl-exec-agent
/etc/ctrl-exec-agent/ 0750 root:ctrl-exec-agent
/var/lib/ctrl-exec-agent/ 0750 ctrl-exec-agent:ctrl-exec-agent
/var/lib/ctrl-exec-agent/ctrl-exec-dispatchers 0644 ctrl-exec-agent trusted-dispatcher map (public serials + ids; agent-writable for rotation)
/opt/ctrl-exec-scripts/ 0755 root:root
/opt/ctrl-exec-scripts/*.sh 0755 root:root
/opt/ctrl-exec-scripts/*.schema.json 0640 root:ctrl-exec-agent schema sidecar (data, not executed)
scripts and every directory above them: owned by root,
not group- or other-writable (enforced; see above)
The run_as user must also reach and execute the script.
The default profile runs as nobody, so the directory and
scripts are 0755 root:root; 0750 root:<group> suits a
profile whose run_as user is in that group.
/var/lib/ctrl-exec/ 0770 root:ctrl-exec
/var/lib/ctrl-exec/pairing/ 0770 root:ctrl-exec
/var/lib/ctrl-exec/agents/ 2770 root:ctrl-exec setgid: files inherit the ctrl-exec group
/var/lib/ctrl-exec/locks/ 2770 root:ctrl-exec setgid; lock files 0660 so root CLI + ctrl-exec API can both flock
/var/lib/ctrl-exec/runs/ 2770 root:ctrl-exec setgid; dispatcher run/status records
/var/lib/ctrl-exec/runs/*.json 0640 root:ctrl-exec
/var/lib/ctrl-exec-agent/runs/ 0750 ctrl-exec-agent async result store (agent side)
/var/lib/ctrl-exec-agent/runs/<dispatcher-id>/ 0750 ctrl-exec-agent per-owner partition
/var/lib/ctrl-exec-agent/runs/<dispatcher-id>/*.json 0640 ctrl-exec-agent
The ctrl-exec-agent system user has no login shell and no home directory.
The ctrl-exec group grants non-root operators read access to the registry and
run records, so the monitoring commands (list-agents, status, list-locks)
run without sudo. Operations that use a private key still require root: run and
ping read the dispatcher key (0600 ctrl-exec - readable only by the API
service user and root, never the ctrl-exec group), and maintain,
pairing-mode, rotate-cert, approve/deny and setup-* use the CA key
(0600 root). The API server runs as the unprivileged ctrl-exec user, which
owns the dispatcher key; it never needs the CA key (cert renewal is driven by
ced maintain, which runs as root), so an RCE in the network-facing API is not
root and cannot sign new certificates.
Both run stores hold script stdout/stderr at rest, the same sensitivity as a
live run's output. They are not world-readable (records 0640, directories
0750) and are purged 24 hours after a run completes. The agent-side store is
owned by the unprivileged ctrl-exec-agent user because the agent process
writes it directly; the dispatcher store is owned by root:ctrl-exec.
The network-facing units - the agent front-end (ctrl-exec-agent) and the API -
are heavily sandboxed:
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
PrivateDevices=yes
The API unit additionally sets ReadWritePaths=/var/lib/ctrl-exec to
restrict filesystem write access to the runtime directory only.
Under privilege separation the agent front-end does not execute privileged scripts - it does network, mTLS, allowlist, schema, and auth-hook work, then hands the run to the executor - so this tight sandbox costs nothing and shrinks the blast radius of an RCE in the network-facing code. Write-bearing or privileged scripts run via the executor, not the front-end (see below).
The executor unit (ctrl-exec-executor) is deliberately not sandboxed: it is
root and needs CAP_SYS_ADMIN (to build each action's mount namespace),
setuid, and full filesystem reach to do its job. Its protection is its small,
audited surface, the peer-cred check on its socket, and the fact that it runs
only allowlisted scripts under their profiles - not an OS sandbox. Enable it only
alongside executor_socket in agent.conf.
Legacy mode (no
executor_socket): the front-end runs scripts directly, so they inherit the front-end's sandbox above - suitable for read-only/diagnostic scripts. For scripts that must write or hold privilege, enable the executor and give them a profile; that is the supported path, and it keeps the front-end locked down.
The agent unit sets StateDirectory=ctrl-exec-agent, which creates and owns
/var/lib/ctrl-exec-agent (mode 0750) and is the only path the agent may
write under ProtectSystem=strict — the async result store lives there. It
also sets KillMode=process so that stopping or restarting the agent signals
only the main process: detached async jobs keep running to completion rather
than being killed with the control group, and the restarted agent serves their
results from the store. This widens nothing — detached jobs were already
children of the agent and run with the same unprivileged identity and the same
allowlist constraints; KillMode=process only changes which processes receive
the stop signal.
The agent unit applies additional containment:
SystemCallFilter=@system-service
SystemCallFilter=~@privileged @resources
CapabilityBoundingSet=
AmbientCapabilities=
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
RestrictNamespaces=yes
LockPersonality=yes
MemoryDenyWriteExecute=yes
CapabilityBoundingSet= (empty) drops all capabilities from the bounding
set, preventing any allowlisted script from acquiring capabilities regardless
of file capability bits. MemoryDenyWriteExecute=yes is safe for the current
bash-only script inventory but must be removed if a JIT-compiled runtime
(Java, Node.js, Python with JIT) is added to the allowlist - there is no
detection mechanism for this conflict at load time; the operator must review
this directive when adding new allowlist entries. AF_UNIX is required
in RestrictAddressFamilies because the agent connects to /dev/log via a
Unix domain socket to deliver syslog messages - omitting it silently blocks
all logging.
The agent no longer stores a single trusted dispatcher serial. Instead it
stores a trusted-dispatcher map at
/var/lib/ctrl-exec-agent/ctrl-exec-dispatchers (path configurable via
trusted_dispatchers_path in agent.conf, default
/var/lib/ctrl-exec-agent/ctrl-exec-dispatchers). The map lives in the
agent-writable state directory — not under /etc/ctrl-exec-agent, which
holds only secrets (the agent key, cert, and CA) — so the agent, running as
the unprivileged service user, can rewrite trusted serials in place during
cert rotation. Holding the map in the agent-writable state directory does not
weaken the lateral-reconnaissance protection: a serial is inert without a
CA-signed cert bearing it (the CA key is operator-held), and an attacker on a
compromised peer host cannot write this host's map. The map holds one line per
trusted dispatcher cert in the format <ca-fingerprint> <hex-serial> <dispatcher-id>, <ca-fingerprint> being the SHA-256 fingerprint of the CA
in the agent's CA bundle that issued that cert; lines beginning with # are
comments. A pre-0.14 two-field line (<hex-serial> <dispatcher-id>) is
upgraded in place on first load with the fingerprint of the CA the agent held
then. The map is loaded at startup and reloaded on
SIGHUP. This key replaces the former dispatcher_serial_path /
ctrl-exec-serial single-serial file.
On every /run, /ping, /result, and /capabilities request, the agent
checks whether the connecting client cert's issuing CA and serial match an
entry in the map. The issuing CA is the CA in the bundle whose name is the
cert's issuer and whose key verifies the cert's signature, so a cert from
another trusted CA that carries a trusted serial matches nothing: holding one
dispatcher CA's key does not let anyone act as a dispatcher of another CA. A
request that matches no entry is rejected with a 403. When it matches, the dispatcher identity recorded against that key is used for
permission decisions and attribution. If the map is empty (an unpaired or
legacy agent that has not yet been re-paired), /capabilities logs
capabilities-no-serial and skips the restriction; /run, /ping, and
/result still hard-deny.
Each trusted dispatcher has a stable dispatcher_id, set via dispatcher_id
in ctrl-exec.conf (defaulting to the dispatcher's hostname) and delivered to
the agent at pairing. Per-dispatcher policy must key on this id, never on the
serial: serials rotate, the id is stable.
Native multi-dispatcher support
: An agent can serve more than one dispatcher. Pairing appends a dispatcher
to the map rather than replacing the existing entry, so re-pairing to enrol
an additional dispatcher leaves existing dispatchers trusted. Each dispatcher
presents its own cert. The agent's ca.crt is a bundle: pairing appends the
dispatcher's CA (once per fingerprint) instead of replacing the CAs already
trusted, and map entries are keyed by issuing CA, so dispatchers with
different CAs cannot impersonate each other at the agent. Re-pairing a
dispatcher id under a different CA ends that id's entries under its old CA.
A renewed agent cert must come from the CA that issued the current one. A dedicated single-operator agent instance remains available as
optional defence-in-depth, but it is not the multi-dispatcher mechanism.
Async result ownership
: Runs are stored partitioned by owner at
/var/lib/ctrl-exec-agent/runs/<dispatcher-id>/<reqid>.json. A
GET /result/<reqid> returns a run only to the dispatcher that submitted it.
Another dispatcher receives 404 unknown with no disclosure of the run's
existence or output. The agent logs result-deny (REASON=not-owner) when a
dispatcher requests another dispatcher's result.
/renew and /renew-complete are intentionally exempt from the map check.
After a dispatcher cert rotation, an agent the serial broadcast has not
reached yet (offline at the time, still pending) sees the dispatcher's new
cert before its map holds the new serial; the map check on the renewal
endpoints would stop that agent's certificate renewal until the broadcast
reaches it. (The broadcast itself, /rotate-serial, is map-checked: it is
sent with the previous cert, which the agent trusts.) The renewal endpoints require a valid CA-signed cert (mTLS still
applies) but do not require the peer serial to be a key in the map. See Cert
Rotation below.
The window during which these endpoints are reachable by any CA-signed cert is
the duration of the serial broadcast - on a well-connected fleet this is
seconds; on a slow fleet it is bounded by the dispatcher's connection timeout.
A host with a valid CA-signed cert calling /renew-complete during this window
could attempt to deliver a replacement cert, but only if it already holds a
cert signed by the private CA. If an attacker has a CA-signed cert, /run is
the more direct path; /renew abuse does not represent a meaningful
escalation.
The dispatcher cert is renewed automatically before expiry. The renewal process and the serial tracking work together to rotate credentials without disrupting the fleet:
Renewal trigger
: The dispatcher checks its own cert expiry at startup and every 4 hours
(configurable via cert_check_interval). When fewer than cert_renewal_days
remain (default: 90), it generates a new cert automatically.
Broadcast
: The new cert and key are first written beside the served pair
(<cert>.next, <key>.next) and the served pair is copied to <cert>.prev
and <key>.prev. The dispatcher then POSTs to each registered agent's
built-in /rotate-serial operation in parallel, carrying the new serial and
authenticated with the previous cert - the one the agent already trusts -
and only after that installs the new pair as the served one. (Before 0.14
the new cert was installed first, so the broadcast presented a serial no
agent trusted, every agent refused it, and a rotation cut the dispatcher off
from its fleet.) The agent's rotate handler adds the new serial to its
trusted-dispatcher map and sends itself SIGHUP. The dispatcher identity is
derived from the caller's authenticated cert serial — it is never sent in the
request — so an agent only ever adds a serial under the calling dispatcher's
own identity. Agents that respond successfully are marked current in the
registry.
Trust chain : Each new serial is authorised by the currently-trusted (previous) serial, which chains back to the original human-supervised pairing approval. Any number of rotations therefore stays rooted in that one human approval, and no rotation can introduce a serial that was not vouched for by an already-trusted one.
Seamless rotation (0.9.0)
: Cert rotation updates each agent's trusted-dispatcher map automatically
over the run channel, with no re-pairing, via add-then-remove. On rotation
the dispatcher broadcasts the new serial together with its stable
dispatcher_id; each reachable agent adds it to the map against that
identity. The old serial stays trusted through the overlap window, so a
dispatcher instance presenting either cert is accepted throughout. After the overlap window the dispatcher
broadcasts removal of the old serial (retire_previous_serial, logged as
serial-retire), so the retired cert stops being trusted. Because
dispatcher_id is stable across rotation, trust and attribution carry over;
the agent can rewrite its own map because the map lives in the
agent-writable state directory. An agent that was offline during the
broadcast misses the new serial and, once the overlap window expires, is
marked stale and does need re-pairing — rotation is seamless only for
agents reachable during the broadcast.
Overlap window
: Agents that were offline during the broadcast are marked pending. The
dispatcher retries them on each subsequent check interval (and each
maintain run), authenticated with the kept previous cert, which they still
trust. After
cert_overlap_days (default: 30, configurable) the overlap expires and
any remaining pending agents are marked stale. A stale agent needs
re-pairing - it has missed the rotation window. /run, /ping, and
/result deny with 403 once the new serial is not a key in the map;
/capabilities warns but allows.
ced serial-status
: Shows the current and previous dispatcher serial, rotation timestamp,
overlap expiry, and per-agent serial state (current/pending/stale/unknown).
Use this to identify agents that need attention after a rotation.
ced rotate-cert
: Manual rotation trigger. Runs the same logic as the automatic check,
broadcasts immediately, and reports per-agent results. Use after a suspected
compromise or to test the rotation path.
dispatcher re-keying
: Running setup-ctrl-exec again generates a new cert with a new serial
and marks all agents as pending. The dispatcher binary checks the registry
before proceeding and displays the number of agents that will require
re-pairing if the overlap window is missed.
Built-in /rotate-serial operation
: Cert rotation is handled by the agent front-end (ctrl-exec-agent) as a
first-class control-plane operation — no scripts.conf allowlist entry is
required, and the work runs in the front-end rather than through the executor.
This is deliberate: with privilege separation enabled the executor mounts the
agent's control/state directories read-only for every action, so a script
could not write the trusted-dispatcher map; the front-end can, which is why
rotation works identically whether or not the executor is enabled. The handler
adds the broadcast serial to the trusted-dispatcher map (and, at retirement,
removes the old one) and sends SIGHUP. Access is gated by the same
trusted-dispatcher serial check as /run, plus the auth hook (which sees
ENVEXEC_ACTION=rotate).
The certificate serial is the trust identity: the agent's trusted-dispatcher
map, the revocation list and the registry all key on it. From 0.13.0 the CA
gives every certificate it issues (its own, the dispatcher's, and each agent
cert at pairing and renewal) a random serial - 16 bytes from /dev/urandom,
top bit clear so the integer is positive, 126 bits of entropy, always 32
lowercase hex digits - so a serial cannot be guessed or predicted from an
earlier one. Earlier sequential serials (01, 02, ...) remain valid and
keep working; they are not re-issued. ca.serial is no longer a counter.
The agent maintains a revocation list at /etc/ctrl-exec-agent/revoked-serials
(path configurable via revoked_serials in agent.conf). On every incoming
mTLS connection, after the handshake verifies the CA signature, the peer cert
serial is checked against this list. A revoked cert is rejected with a 403
and a syslog warning before any request is processed.
A line <serial> revokes that serial in a cert from any CA (the pre-0.14 form,
still honoured, so an existing revocation never loosens). A line
<ca-fingerprint> <serial> revokes it only in certs issued by that CA (the
SHA-256 fingerprint of the CA certificate, 64 hex characters), so the same
serial issued by another dispatcher's CA keeps working.
The serial is given in any of the following formats, all normalised to lowercase hex on load:
- Plain hex:
deadbeef - Colon-separated:
DE:AD:BE:EF(as returned by some tools andIO::Socket::SSL) 0x-prefixed:0xdeadbeefserial=-prefixed:serial=DEADBEEF(direct output ofopenssl x509 -serial)- Decimal integer:
3735928559
Lines beginning with # are treated as comments. A missing or empty file means
no certs are revoked - the normal state for a new installation.
The revocation list is loaded at agent startup and reloaded on SIGHUP without restarting the agent or dropping active connections:
systemctl reload ctrl-exec-agentTo revoke a cert:
- Obtain the serial:
openssl x509 -noout -serial -in /etc/ctrl-exec-dispatcher/dispatcher.crt - Append the output directly to
/etc/ctrl-exec-agent/revoked-serialson each affected agent - no format conversion needed,serial=DEADBEEFis accepted as-is - Reload:
systemctl reload ctrl-exec-agent
For fleet-wide revocation, run a ctrl-exec script that appends the serial and
sends SIGHUP on each agent. A revoke-cert script is a natural entry in the
agent allowlist for this purpose.
Unpairing and revocation
: ced unpair <hostname> removes the agent from the registry and
prevents further cert renewal. The agent cert remains technically valid until
natural expiry. To immediately close this window, add the agent cert serial
to the dispatcher's own revocation check (if implemented) or decommission
the host promptly. The revocation list on the agent only covers certs
presented to the agent - it does not prevent a stolen agent cert from
connecting to the dispatcher.
What revocation closes : A compromised dispatcher cert that has been revoked cannot connect to any agent that has been updated, even though it was signed by the CA and has not expired. A compromised agent cert cannot be revoked via this mechanism on the dispatcher side - that requires the dispatcher-side equivalent (serial tracking work, next phase).
What revocation does not close : A compromised cert that reaches an agent before the revocation list is updated. The list is only as current as the last SIGHUP. For time-critical revocation, restart the agent service rather than reloading - the connection is still rejected on reconnect but any in-flight connection from a revoked cert may complete if it was established before the reload.
The CA key is the root of trust for the entire deployment. If it is compromised, an attacker can issue valid agent certs and connect to any agent.
Recommended practices:
- Back up
/etc/ctrl-exec-dispatcher/ca.keyto encrypted offline storage immediately aftersetup-ca - Restrict access to the dispatcher host itself; the CA key should not be accessible over the network
- For redundant instances of one dispatcher (HA), transfer the CA key over an
encrypted channel with host key verification (
scpwith known_hosts, notStrictHostKeyChecking=no). A second, distinct dispatcher never needs it: it has its own CA, and agents pair with it as usual (HIGH-AVAILABILITY.md) - Audit access to the dispatcher host via system auth logs
Connection rate limiting
: The agent tracks connection attempts per source IP in memory. A source
that establishes more than 10 connections within 60 seconds is blocked for
5 minutes (volume threshold). A source that causes more than 3 TLS handshake
failures within 600 seconds is blocked for 1 hour (probe threshold). Blocks
are held in the agent process memory and cleared on SIGHUP. The block state
is logged at the point the threshold is crossed; repeat checks are silent.
Rate state is not persisted across reloads - the built-in /rotate-serial
handler sends SIGHUP as part of normal rotation and clears all rate blocks as
a side effect. A legitimate dispatcher that triggers the probe block due to a
cert misconfiguration can be unblocked immediately with
systemctl reload ctrl-exec-agent. Thresholds are configurable via
rate_limit_volume and rate_limit_probe in agent.conf.
IP allowlist
: If allowed_ips is set in agent.conf, the agent enforces an IP allowlist
before the rate check and before any request is processed. Any source IP not
in the list is rejected immediately with the connection closed and an
ACTION=ip-block syslog entry. The allowlist supports exact IPs and /8,
/16, /24 CIDR prefixes. Invalid entries are filtered at load time with a
warning; the agent starts normally with the remaining valid entries. When
allowed_ips is absent, all source IPs are permitted.
TLS version and cipher restriction
: TLS 1.2 and TLS 1.3 only - TLS 1.0/1.1 (and SSLv3) are disabled explicitly
via SSL_version, not left to the system OpenSSL policy. For TLS 1.2, only
ECDHE cipher suites using AES-GCM are permitted; CBC mode, RC4, export-grade,
and anonymous ciphers are excluded. TLS 1.3 suites are AEAD-only. This policy
is defined once in Exec::TLS::hardening and applied to every TLS endpoint -
the agent operational server, the dispatcher client, the pairing listener,
and the API server (when TLS is enabled) - so it cannot drift between them.
Request body size limit
: The agent and the API limit incoming request bodies to 1 MiB plus
max_input_bytes (default 1 MiB, so 2 MiB). Any request declaring a
Content-Length above this threshold is rejected with HTTP 413 before the
body is read. Without confidential run input no legitimate request
approaches 1 MiB - the largest (a /renew-complete cert delivery) is under
10 KB.
HTTP header count limit : The agent limits incoming requests to 32 header lines. Any request exceeding this is rejected with HTTP 431 before the body is read or any handler is invoked. This applies to all connections including those from peers with a valid CA-signed cert.
Argument validation scope
: The dispatcher does not validate script argument values. Validation of arguments
is the responsibility of the allowlisted script itself and the auth hooks.
This includes the agent: it does not validate arguments against a script's
schema sidecar (the sidecar is advertised data only). To constrain arguments
on an agent, set auth_hook there; the agent calls it after allowlist
validation with the arguments, and a non-zero exit refuses the run. The
dispatcher's own hook can do the same before any connection is made. A hook
receives arguments as a JSON array via ENVEXEC_ARGS_JSON and
on stdin; the script receives them directly as @ARGV. Neither path involves
shell interpretation.
Sites that want arguments narrowed per script (permitted values, patterns,
or path prefixes) can opt in to the shipped agent hook auth-hook-args.example
(installed beside agent.conf by install.sh; under
/usr/share/ctrl-exec/examples/ in the Debian agent package). It validates
ENVEXEC_ARGS_JSON against the script's schema sidecar (see
SCHEMA-SIDECAR.md), resolves path arguments through symlinks before
comparing them with the permitted prefixes, and denies with exit 3 before the
script starts. It is off by default: nothing changes until auth_hook in
agent.conf names it. Without it, every argument the allowlisted script
accepts is in scope, host-wide - a valid design where who may run the script
is the control. The hook sees only what the agent passes it, so the input
proposed in R-010 would not be constrained.
The API binds to 127.0.0.1 by default and is not reachable from the network
without an explicit api_bind change in ctrl-exec.conf. For any deployment
where the API is reachable from outside the dispatcher host:
- Configure an auth hook. Without a hook,
api_auth_default = denyblocks all requests. Setapi_auth_default = allowonly on isolated networks. - Place the API behind a reverse proxy (nginx, caddy) that handles TLS termination and request rate limiting. The API has no built-in rate limiting and no host count cap - a reverse proxy is the appropriate layer for both.
- Enable TLS on the API by setting
api_tls_certandapi_tls_keyinctrl-exec.conf, with a certificate from a CA the clients trust whose subject alternative name matches the API's host name, and have clients verify it (ctrl-exec-cli --ca FILE). The API refuses to start rather than fall back to plain HTTP when the pair is incomplete or unreadable. Do not sign API certificates with the ctrl-exec CA: agents trust it for dispatcher identities. - Send confidential run input only over the unix socket or TLS.
GET /health bypasses auth and returns the API version string. This endpoint
is publicly accessible on any externally-bound deployment. Version disclosure
is low risk in a private deployment but should be considered for
internet-facing deployments - place the API behind a proxy that strips or
restricts the /health path if version disclosure is a concern.
For operational security guidance (monitoring, incident response, known limitations, Docker-specific notes), see SECURITY-OPERATIONS.md.
| Threat | Mitigation |
|---|---|
| Unauthenticated agent connection | mTLS on port 7443, both sides verify CA |
| Rogue dispatcher connecting to agent | Agent verifies dispatcher cert against CA |
| Pairing replay or misrouting | Nonce verified before cert storage |
| Pairing CSR injection / social engineering | 6-digit pairing code verified by operator on both sides |
| Pairing relay (man in the middle on the pairing port) | Verified pairing (--dispatcher-fingerprint): the agent refuses a listener whose certificate does not chain to the dispatcher's CA, or (0.14) is marked by that CA as an agent certificate, before sending its CSR. Without it, the v2 pairing code bound to the dispatcher certificate the agent saw; a relay shows mismatching codes (limits under Pairing code verification) |
| Lateral reconnaissance via capabilities | /capabilities restricted to the trusted-dispatcher map (per-dispatcher serial); hard deny on unknown serial; re-pair to activate |
| Script execution by non-current dispatcher cert | /run, /ping, and /result restricted to the trusted-dispatcher map (per-dispatcher serial); hard deny on unknown serial |
| Cross-dispatcher result disclosure | Runs partitioned per owner; GET /result/<reqid> owner-gated, 404 unknown to other dispatchers (result-deny, REASON=not-owner) |
| Compromised dispatcher (or AI driving its API) | Bounded by each agent to the allowlisted, profiled and audited action set; cannot extend the allowlist, edit hook/config/trust, or erase audit |
| RCE in the agent front-end (network-facing code) | Privilege separation: the front-end is unprivileged; only the small executor can act, and it runs only allowlisted scripts under their pinned profiles (re-derived from its own root-owned config) |
| Action tampering with agent controls or audit | Executor runs every action with /etc/ctrl-exec-agent and /var/lib/ctrl-exec-agent read-only - including run_as=root profiles |
| Arbitrary script execution via run | Agent-side allowlist, name pattern validation |
| Path traversal in script name | /^[\w-]+$/ excludes / and . |
| Shell injection via arguments | exec without shell, args passed as list |
| Argument bypass via ENVEXEC_ARGS | Use ENVEXEC_ARGS_JSON; ENVEXEC_ARGS is deprecated |
| Script outside approved directories | script_dirs check at load and exec time |
| Connection flood from valid cert | Rate limiting: volume block at 10 conn/60s (5 min), probe block at 3 failures/600s (1 hr) |
| Pairing queue flood | Queue depth capped at 10 (configurable via pairing_max_queue); stale expiry runs first |
| Header flood from valid cert | HTTP 431 after 32 header lines; connection closed before body is read |
| API host count exhaustion | 500-host ceiling per request (fixed Engine limit; not user-configurable) |
| Port scan or TLS probe from unexpected host | IP allowlist (allowed_ips in agent.conf); connection closed before any request |
| TLS downgrade or weak cipher negotiation | TLS 1.2 minimum; ECDHE+AEAD ciphers only; CBC, RC4, export-grade excluded |
| Memory exhaustion via large request body | Body size limit: 1 MiB plus max_input_bytes checked before read; 413 returned on excess |
| Privilege escalation via allowlisted script | Per-script profile bounds run_as + capabilities (executor); on the front-end, empty CapabilityBoundingSet, MemoryDenyWriteExecute, restricted syscall filter |
| Namespace escape from agent process | RestrictNamespaces=yes in agent systemd unit |
| Unauthorised API access (no hook) | api_auth_default = deny blocks all requests by default |
| Unauthorised API access (network) | API binds to 127.0.0.1 by default; external bind is opt-in |
| Unauthorised API access (local users) | Optional unix socket (api_socket), mode 0660 in api_socket_group; hook sees the caller's kernel-reported uid |
| Confidential input read on the wire to the API | Unix socket or TLS (api_tls_cert/api_tls_key); incomplete TLS settings refuse start-up instead of falling back to plain HTTP; CLI warns on plain HTTP to a non-loopback address |
| API script inventory exposure | All endpoints including /openapi-live.json pass through auth |
| CA key theft | 0600 root-only, offline backup, host access controls |
| Cert remaining valid after unpair | Revocation list on agent; add serial and reload |
| Compromised dispatcher cert | Add serial to revoked-serials on all agents; reload |
| Secret script output at rest on the dispatcher | output=confidential in the agent's scripts.conf: output returned to the caller only, run store keeps exit status; async refused (depends on the agent's declaration; older agents send no mark) |
| Token leaking via logs | Tokens never logged by dispatcher or agent |
| Token leaking via ps | Use ENVEXEC_TOKEN env var not --token flag |
SECURITY-OPERATIONS.md covers the operational side of a running deployment: monitoring and alerting recommendations, dispatcher host security requirements, token and credential lifecycle, auth hook operational guidance, CA compromise recovery procedure, known limitations, and Docker-specific security notes.