feat(aegisctl): cluster preflight for 10 cluster dependencies by Lincyaw · Pull Request #65 · OperationsPAI/AegisLab

Lincyaw · 2026-04-18T12:09:54Z

Closes OperationsPAI/aegis#17

Implements OperationsPAI/aegis#17. Adds aegisctl cluster preflight that enumerates 10 cluster/DB/redis dependencies and reports missing/unhealthy ones with fix suggestions. Supports --check <id> for single-check runs and --fix for idempotent remediation of SA + leaked token_bucket entries. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Adds a new aegisctl cluster preflight command that runs a catalog of dependency checks against the Kubernetes cluster and backing services referenced by config.$ENV_MODE.toml, with optional idempotent remediation for a subset of checks.

Changes:

Introduces aegisctl cluster preflight Cobra subcommand with --check, --fix, --config, and --check-timeout flags.
Adds a new cluster package implementing a check registry/runner, live environment probes (K8s/Net/ClickHouse/MySQL/Redis), and 10 default checks.
Documents the new CLI behavior and check catalog in docs/aegisctl-cli-spec.md.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
`src/cmd/aegisctl/cmd/cluster.go`	Registers the `cluster` and `cluster preflight` commands and wires flags into the cluster preflight runner.
`src/cmd/aegisctl/cluster/live_env.go`	Implements live probes/clients and TOML config loading for preflight checks.
`src/cmd/aegisctl/cluster/env.go`	Defines `CheckEnv` and probe interfaces used by checks and unit tests.
`src/cmd/aegisctl/cluster/cluster.go`	Implements the check registry, runner execution, and table rendering.
`src/cmd/aegisctl/cluster/checks.go`	Adds the 10 default preflight checks plus optional fixers for SA creation and Redis token bucket cleanup.
`src/cmd/aegisctl/cluster/checks_test.go`	Adds unit tests for selected checks and validates the default registry catalog.
`docs/aegisctl-cli-spec.md`	Documents the new `aegisctl cluster preflight` command, flags, and check catalog.

Copilot · 2026-04-18T12:13:52Z

+type mysqlCreds struct{ user, pass, db string }
+
+var loadedMySQLCreds mysqlCreds
+
+func firstNonEmpty(a, b string) string {
+	if a != "" {
+		return a
+	}
+	return b
+}
+
+// LoadConfig reads config.dev.toml (or the env-specific equivalent) and
+// returns a populated Config. A missing file is tolerated — individual
+// checks flag missing values themselves.
+func LoadConfig(explicitPath string) (Config, error) {
+	v := viper.New()
+	v.SetConfigType("toml")
+	if explicitPath != "" {
+		v.SetConfigFile(explicitPath)
+	} else {
+		env := os.Getenv("ENV_MODE")
+		if env == "" {
+			env = "dev"
+		}
+		v.SetConfigName("config." + env)
+		cwd, _ := os.Getwd()
+		v.AddConfigPath(cwd)
+		v.AddConfigPath(filepath.Join(cwd, "src"))
+		v.AddConfigPath(".")
+	}
+	_ = v.ReadInConfig()
+
+	cfg := Config{
+		K8sNamespace:   v.GetString("k8s.namespace"),
+		MySQLHost:      v.GetString("database.mysql.host"),
+		MySQLPort:      v.GetString("database.mysql.port"),
+		ClickHouseHost: v.GetString("database.clickhouse.host"),
+		ClickHousePort: v.GetString("database.clickhouse.port"),
+		ClickHouseDB:   v.GetString("database.clickhouse.database"),
+		ClickHouseUser: v.GetString("database.clickhouse.user"),
+		ClickHousePass: v.GetString("database.clickhouse.password"),
+		RedisAddr:      v.GetString("redis.host"),
+		EtcdEndpoints:  v.GetStringSlice("etcd.endpoints"),
+		ServiceAccount: v.GetString("k8s.job.service_account.name"),
+		DatasetPVC:     v.GetString("k8s.job.volume_mount.dataset.claim_name"),
+	}
+	loadedMySQLCreds = mysqlCreds{
+		user: firstNonEmpty(v.GetString("database.mysql.user"), "root"),
+		pass: v.GetString("database.mysql.password"),
+		db:   firstNonEmpty(v.GetString("database.mysql.db"), "rcabench"),
+	}
+	return cfg, nil
+}
+
+func NewLiveEnv(cfg Config) *LiveEnv {
+	return &LiveEnv{cfg: cfg, dialTimeout: 3 * time.Second, mysqlCreds: loadedMySQLCreds}
+}


loadedMySQLCreds is a package-level mutable global that LoadConfig() writes and NewLiveEnv() reads. This makes configuration stateful and not safe if aegisctl ever loads multiple configs in-process (tests, future subcommands, or concurrent runs), and it’s easy to accidentally call NewLiveEnv() without having called LoadConfig() first. Prefer returning MySQL creds as part of Config (or a separate struct) and storing them on LiveEnv directly, without global state.

Copilot · 2026-04-18T12:13:52Z

+	dsn := fmt.Sprintf("%s:%s@tcp(%s:%s)/%s?parseTime=true&timeout=3s&readTimeout=3s",
+		m.creds.user, m.creds.pass, host, port, m.creds.db)
+	conn, err := sql.Open("mysql", dsn)


MySQL DSN is assembled with fmt.Sprintf("%s:%s@tcp(...)", user, pass, ...). If the password (or user) contains special characters (e.g., '@', ':', '/'), the DSN can become invalid and connections will fail. Consider using mysql.Config (from go-sql-driver/mysql) or url.QueryEscape-equivalent encoding for credentials when building the DSN.

Copilot · 2026-04-18T12:13:53Z

+func joinHostPort(host, port string) string {
+	host = strings.TrimSpace(host)
+	port = strings.TrimSpace(port)
+	if host == "" {
+		return ""
+	}
+	if strings.Contains(host, ":") {
+		return host
+	}
+	if port == "" {
+		return host
+	}
+	return net.JoinHostPort(host, port)


joinHostPort() treats any host containing ":" as already having a port and returns it unchanged. This breaks IPv6 literals (which contain ":") and results in invalid dial addresses when the port is configured separately. Consider using net.SplitHostPort to detect an existing host:port, and otherwise net.JoinHostPort (including adding brackets for IPv6).

Copilot · 2026-04-18T12:13:53Z

+func checkMySQL(ctx context.Context, env CheckEnv) Result {
+	cfg := env.Config()
+	addr := joinHostPort(cfg.MySQLHost, cfg.MySQLPort)
+	if addr == "" {
+		return Result{Status: StatusFail, Detail: "database.mysql.host not configured",
+			Fix: "set [database.mysql] host/port in config.dev.toml"}
+	}
+	if err := env.Net().DialTimeout(ctx, addr); err != nil {
+		return Result{Status: StatusFail, Detail: fmt.Sprintf("cannot dial %s: %v", addr, err),


checkMySQL() relies on joinHostPort(cfg.MySQLHost, cfg.MySQLPort). If host is set but port is omitted (common when users expect default 3306), joinHostPort returns just the host and DialTimeout will fail with a "missing port" error. Consider defaulting the port to 3306 in this check (or updating joinHostPort semantics) so the preflight doesn’t false-fail when port is not explicitly set.

Copilot · 2026-04-18T12:13:53Z

+func checkClickHouseTCP(ctx context.Context, env CheckEnv) Result {
+	cfg := env.Config()
+	addr := joinHostPort(cfg.ClickHouseHost, cfg.ClickHousePort)
+	if addr == "" {
+		return Result{Status: StatusFail, Detail: "[database.clickhouse] host/port missing in config",
+			Fix: "add a [database.clickhouse] section with host/port/database/user/password to config.dev.toml"}
+	}
+	if err := env.Net().DialTimeout(ctx, addr); err != nil {
+		return Result{Status: StatusFail, Detail: fmt.Sprintf("cannot dial %s: %v", addr, err),


checkClickHouseTCP() has the same default-port issue as checkMySQL(): if ClickHouseHost is set but ClickHousePort is empty, joinHostPort returns only the host, and DialTimeout will fail. Consider defaulting to ClickHouse’s native TCP port (9000) when the config omits the port, to match liveClickHouse.TablesIn() which already defaults to 9000.

Copilot · 2026-04-18T12:13:53Z

+		defer cancel()
+		allOK, _ := runner.Run(ctx, env, opts, os.Stdout)
+		if !allOK {
+			os.Exit(1)


clusterPreflightCmd.RunE calls os.Exit(1) on failure. This bypasses cobra’s normal error handling and skips defers in the current goroutine (including the deferred cancel), and it makes it harder to unit test this command’s behavior. Consider returning a non-nil error instead and letting cmd.Execute() map that to exit code 1 (reserving os.Exit only for non-1 custom exit codes like wait).

Suggested change

os.Exit(1)

return fmt.Errorf("one or more preflight checks failed")

Copilot · 2026-04-18T12:13:54Z

+func joinHostPort(host, port string) string {
+	host = strings.TrimSpace(host)
+	port = strings.TrimSpace(port)
+	if host == "" {
+		return ""
+	}
+	if strings.Contains(host, ":") {
+		return host
+	}
+	if port == "" {
+		return host
+	}
+	return net.JoinHostPort(host, port)
+}
+
+func checkMySQL(ctx context.Context, env CheckEnv) Result {
+	cfg := env.Config()
+	addr := joinHostPort(cfg.MySQLHost, cfg.MySQLPort)
+	if addr == "" {
+		return Result{Status: StatusFail, Detail: "database.mysql.host not configured",
+			Fix: "set [database.mysql] host/port in config.dev.toml"}
+	}
+	if err := env.Net().DialTimeout(ctx, addr); err != nil {
+		return Result{Status: StatusFail, Detail: fmt.Sprintf("cannot dial %s: %v", addr, err),
+			Fix: "start MySQL (docker compose up mysql) and verify [database.mysql].host/port"}
+	}
+	return Result{Status: StatusOK, Detail: "dialed " + addr}
+}
+
+func checkClickHouseTCP(ctx context.Context, env CheckEnv) Result {
+	cfg := env.Config()
+	addr := joinHostPort(cfg.ClickHouseHost, cfg.ClickHousePort)
+	if addr == "" {
+		return Result{Status: StatusFail, Detail: "[database.clickhouse] host/port missing in config",
+			Fix: "add a [database.clickhouse] section with host/port/database/user/password to config.dev.toml"}
+	}
+	if err := env.Net().DialTimeout(ctx, addr); err != nil {
+		return Result{Status: StatusFail, Detail: fmt.Sprintf("cannot dial %s: %v", addr, err),
+			Fix: "ensure ClickHouse is running and the [database.clickhouse] host/port match"}
+	}
+	return Result{Status: StatusOK, Detail: "dialed " + addr}
+}


Test coverage: joinHostPort(), checkMySQL(), and checkClickHouseTCP() contain edge-case logic (default ports, IPv6/host:port parsing) that can regress easily. Consider adding unit tests that cover: (1) host only + empty port uses the expected default; (2) IPv6 literal with separate port is joined correctly; (3) already-host:port inputs are preserved.

Copilot · 2026-04-18T12:13:54Z

+func LoadConfig(explicitPath string) (Config, error) {
+	v := viper.New()
+	v.SetConfigType("toml")
+	if explicitPath != "" {
+		v.SetConfigFile(explicitPath)
+	} else {
+		env := os.Getenv("ENV_MODE")
+		if env == "" {
+			env = "dev"
+		}
+		v.SetConfigName("config." + env)
+		cwd, _ := os.Getwd()
+		v.AddConfigPath(cwd)
+		v.AddConfigPath(filepath.Join(cwd, "src"))
+		v.AddConfigPath(".")
+	}
+	_ = v.ReadInConfig()
+


LoadConfig() ignores v.ReadInConfig() errors unconditionally. When --config is provided and the path is wrong/unreadable, the command will silently proceed with empty values and report confusing failures. Consider: (1) if explicitPath != "" then return an error from ReadInConfig; (2) only tolerate missing config when using the default resolution paths (and optionally distinguish not-found vs parse errors).

Copilot AI review requested due to automatic review settings April 18, 2026 12:09

Lincyaw had a problem deploying to test-server April 18, 2026 12:09 — with GitHub Actions Failure

Copilot started reviewing on behalf of Lincyaw April 18, 2026 12:10 View session

Copilot AI reviewed Apr 18, 2026

View reviewed changes

Lincyaw closed this Apr 19, 2026

Lincyaw deleted the issue-17/cluster-preflight branch April 19, 2026 03:47

Navigation Menu

feat(aegisctl): cluster preflight for 10 cluster dependencies - #65

feat(aegisctl): cluster preflight for 10 cluster dependencies#65
Lincyaw wants to merge 1 commit into
mainfrom
issue-17/cluster-preflight

Lincyaw commented Apr 18, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 18, 2026

Uh oh!

Copilot AI Apr 18, 2026

Uh oh!

Copilot AI Apr 18, 2026

Uh oh!

Copilot AI Apr 18, 2026

Uh oh!

Copilot AI Apr 18, 2026

Uh oh!

Copilot AI Apr 18, 2026

Uh oh!

Copilot AI Apr 18, 2026

Uh oh!

Copilot AI Apr 18, 2026

Uh oh!

This branch had an error being deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	os.Exit(1)
	return fmt.Errorf("one or more preflight checks failed")

Conversation

Lincyaw commented Apr 18, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

Uh oh!

This branch had an error being deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants