From 250cb98c26777a7202f3e8dc886ba190cabebeb2 Mon Sep 17 00:00:00 2001 From: overtrue Date: Mon, 18 May 2026 22:28:35 +0800 Subject: [PATCH] docs: remove obsolete spec and test matrix --- README.md | 2 + docs/SPEC.md | 603 ------------------------------------ docs/TEST_MATRIX.md | 252 --------------- docs/reference/rc/README.md | 2 + 4 files changed, 4 insertions(+), 855 deletions(-) delete mode 100644 docs/SPEC.md delete mode 100644 docs/TEST_MATRIX.md diff --git a/README.md b/README.md index d726658..6f31919 100644 --- a/README.md +++ b/README.md @@ -267,6 +267,8 @@ rc admin rebalance status local --json ## Command Overview +For full command documentation, see the [`rc` command reference](docs/reference/rc/README.md). + | Command | Description | |---------------|------------------------------------------------------------------------------| | `alias` | Manage storage service aliases | diff --git a/docs/SPEC.md b/docs/SPEC.md deleted file mode 100644 index 803a3dc..0000000 --- a/docs/SPEC.md +++ /dev/null @@ -1,603 +0,0 @@ -# rc CLI Specification v2 - -> **PROTECTED FILE**: Changes to this specification require the Breaking Change process. -> See AGENTS.md for the required workflow. - -## Overview - -`rc` is a command-line interface for S3-compatible object storage services. It is designed -to work with RustFS, AWS S3, and other S3-compatible backends. - -## General Conventions - -### Path Syntax - -- **Remote path**: `/[/]` - - `local/mybucket` - refers to a bucket - - `local/mybucket/path/to/file.txt` - refers to an object - - `local/mybucket/path/to/dir/` - trailing slash indicates directory semantics - -- **Local path**: Standard filesystem path - - `/home/user/file.txt` - absolute path - - `./file.txt` - relative path - - `../file.txt` - relative path - -### Trailing Slash Semantics - -The trailing `/` is significant: - -| Path | Meaning | -|------|---------| -| `alias/bucket/dir` | Could be a file named "dir" or a directory | -| `alias/bucket/dir/` | Explicitly a directory (prefix) | - -For `ls`: A path without trailing slash lists the item itself if it's a file, or contents if it's a directory. -For `cp`: A path with trailing slash preserves the source filename. - -### Overwrite Strategy - -Default behavior depends on the mode: - -- **Interactive mode** (TTY detected): Prompt for confirmation -- **Non-interactive mode**: Error unless flag specified - -| Flag | Behavior | -|------|----------| -| `--overwrite` | Force overwrite existing files | -| `--no-clobber` | Skip existing files silently | -| `--if-match ` | Only overwrite if ETag matches (conditional write) | - -### Output Modes - -| Flag | Behavior | -|------|----------| -| (none) | Human-readable output with colors and progress bars | -| `--json` | Strict JSON output (no colors, no progress, no logs) | -| `--quiet` | Suppress non-error output | -| `--no-color` | Disable colored output | -| `--no-progress` | Disable progress bars | - -### JSON Output Contract - -When `--json` is specified: - -1. Output is valid JSON following `schemas/output_v2.json` -2. No ANSI color codes or escape sequences -3. No progress bars or spinners -4. Timestamps are ISO8601 UTC: `2024-01-15T10:30:00Z` -5. Sizes include both `size_bytes` (integer) and `size_human` (string) -6. Paths use `/` as separator regardless of platform - ---- - -## Exit Codes - -| Code | Name | Description | -|------|------|-------------| -| 0 | SUCCESS | Operation completed successfully | -| 1 | GENERAL_ERROR | Unspecified error | -| 2 | USAGE_ERROR | Invalid arguments or path format | -| 3 | NETWORK_ERROR | Retryable network error (timeout, 503, etc.) | -| 4 | AUTH_ERROR | Authentication or permission failure | -| 5 | NOT_FOUND | Bucket or object does not exist | -| 6 | CONFLICT | Precondition failed, version conflict | -| 7 | UNSUPPORTED_FEATURE | Backend does not support this operation | -| 130 | INTERRUPTED | Operation interrupted (Ctrl+C) | - ---- - -## Commands - -### alias - Manage Storage Aliases - -#### alias set - -Add or update a storage alias. - -``` -rc alias set [OPTIONS] -``` - -**Arguments:** -| Argument | Description | -|----------|-------------| -| NAME | Unique alias name (alphanumeric, underscore, hyphen) | -| ENDPOINT | S3 endpoint URL | -| ACCESS_KEY | Access key ID | -| SECRET_KEY | Secret access key | - -**Options:** -| Option | Default | Description | -|--------|---------|-------------| -| --region | us-east-1 | AWS region | -| --signature | v4 | Signature version: v4, v2 | -| --bucket-lookup | auto | Bucket lookup: auto, path, dns | -| --insecure | false | Allow insecure TLS | - -**Exit Codes:** 0 (success), 2 (invalid input) - -**Example:** -```bash -rc alias set local http://localhost:9000 accesskey secretkey -rc alias set s3 https://s3.amazonaws.com AKIAIOSFODNN7EXAMPLE wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY -``` - -#### alias list - -List configured aliases. - -``` -rc alias list [OPTIONS] -``` - -**Options:** -| Option | Description | -|--------|-------------| -| -l, --long | Show full details including endpoints | - -**Output (human):** -``` -local http://localhost:9000 -s3 https://s3.amazonaws.com -``` - -**Output (--json):** -```json -{ - "aliases": [ - {"name": "local", "endpoint": "http://localhost:9000", "region": "us-east-1"}, - {"name": "s3", "endpoint": "https://s3.amazonaws.com", "region": "us-east-1"} - ] -} -``` - -**Exit Codes:** 0 (success) - -#### alias remove - -Remove an alias. - -``` -rc alias remove -``` - -**Exit Codes:** 0 (success), 5 (alias not found) - ---- - -### admin - Administrative Operations - -Administrative commands for cluster management. - -#### admin info - -Display cluster, server, or disk information. - -``` -rc admin info cluster -rc admin info server -rc admin info disk [OPTIONS] -``` - -**Arguments:** -| Argument | Description | -|----------|-------------| -| ALIAS | Alias name of the server | - -**Options (disk):** -| Option | Description | -|--------|-------------| -| --offline | Show only offline disks | -| --healing | Show only healing disks | - -**Output (--json):** -- `admin info cluster`: See `schemas/output_v2.json#admin-info-cluster` -- `admin info server`: See `schemas/output_v2.json#admin-info-server` -- `admin info disk`: See `schemas/output_v2.json#admin-info-disk` - -**Exit Codes:** 0, 4 (auth error), 5 (alias not found) - -#### admin heal - -Manage cluster healing operations. - -``` -rc admin heal status -rc admin heal start [OPTIONS] -rc admin heal stop -``` - -**Arguments:** -| Argument | Description | -|----------|-------------| -| ALIAS | Alias name of the server | - -**Options (start):** -| Option | Default | Description | -|--------|---------|-------------| -| -b, --bucket | (all) | Specific bucket to heal | -| -p, --prefix | (none) | Object prefix to heal | -| --scan-mode | normal | Scan mode: normal, deep | -| --remove | false | Remove dangling objects/parts | -| --recreate | false | Recreate missing data | -| --dry-run | false | Show what would be healed without healing | - -**Output (--json):** -- `admin heal status`: See `schemas/output_v2.json#admin-heal-status` -- `admin heal start/stop`: See `schemas/output_v2.json#admin-heal-operation` - -**Exit Codes:** 0, 2 (invalid input), 4 (auth error), 5 (alias not found) - -#### admin pool - -List server pools and inspect pool status. This is the CLI entry point for -post-expansion verification: RustFS expansion is performed by starting the -deployment with additional pools, then using these commands to verify that the -new pool is visible. - -``` -rc admin pool list -rc admin pool status [POOL] [OPTIONS] -``` - -**Arguments:** -| Argument | Description | -|----------|-------------| -| ALIAS | Alias name of the server | -| POOL | Pool command line, or zero-based pool ID with `--by-id` | - -**Options (status):** -| Option | Default | Description | -|--------|---------|-------------| -| --by-id | false | Interpret `POOL` as a zero-based pool ID | - -**Output (--json):** -- `admin pool list`: See `schemas/output_v2.json#admin-pool-list` -- `admin pool status `: See `schemas/output_v2.json#admin-pool-status` -- `admin pool status `: See `schemas/output_v2.json#admin-pool-list` - -**Exit Codes:** 0, 4 (auth error), 5 (alias not found) - -#### admin decommission - -Manage server pool decommissioning. - -``` -rc admin decommission start [OPTIONS] -rc admin decommission status [POOL] [OPTIONS] -rc admin decommission cancel [OPTIONS] -``` - -Alias: `rc admin decom` - -**Arguments:** -| Argument | Description | -|----------|-------------| -| ALIAS | Alias name of the server | -| POOL | Pool command line, comma-separated pool command lines for `start`, or zero-based pool ID with `--by-id` | - -**Options:** -| Option | Default | Description | -|--------|---------|-------------| -| --by-id | false | Interpret `POOL` as zero-based pool ID values | - -**Output (--json):** -- `admin decommission status`: See `schemas/output_v2.json#admin-pool-status` or `schemas/output_v2.json#admin-pool-list` -- `admin decommission start/cancel`: See `schemas/output_v2.json#admin-decommission-operation` - -**Exit Codes:** 0, 4 (auth error), 5 (alias not found) - -#### admin rebalance - -Manage data rebalancing after server pool expansion. - -``` -rc admin rebalance start -rc admin rebalance status -rc admin rebalance stop -``` - -**Arguments:** -| Argument | Description | -|----------|-------------| -| ALIAS | Alias name of the server | - -**Output (--json):** -- `admin rebalance status`: See `schemas/output_v2.json#admin-rebalance-status` -- `admin rebalance start/stop`: See `schemas/output_v2.json#admin-rebalance-operation` - -**Exit Codes:** 0, 4 (auth error), 5 (alias not found) - ---- - -### ls - List Objects - -List buckets or objects. - -``` -rc ls [OPTIONS] -``` - -**Arguments:** -| Argument | Description | -|----------|-------------| -| PATH | Remote path: `alias/` (list buckets) or `alias/bucket[/prefix]` | - -**Options:** -| Option | Default | Description | -|--------|---------|-------------| -| -l, --long | false | Show detailed information | -| -r, --recursive | false | List recursively | -| --max-keys | 1000 | Maximum keys per request | - -**Output (human):** -``` -[2024-01-15 10:30:00] 0B dir/ -[2024-01-15 10:30:00] 1.2MiB file.txt -``` - -**Output (--json):** See `schemas/output_v2.json#ls` - -**Exit Codes:** 0, 2 (invalid path), 4 (auth error), 5 (bucket not found) - ---- - -### mb - Make Bucket - -Create a new bucket. - -``` -rc mb -``` - -**Arguments:** -| Argument | Description | -|----------|-------------| -| PATH | Remote path: `alias/bucket` | - -**Exit Codes:** 0, 2 (invalid path), 4 (auth error), 6 (bucket exists) - ---- - -### rb - Remove Bucket - -Delete a bucket. - -``` -rc rb [OPTIONS] -``` - -**Options:** -| Option | Description | -|--------|-------------| -| --force | Delete bucket even if not empty (deletes all objects first) | - -**Exit Codes:** 0, 4 (auth error), 5 (bucket not found), 6 (bucket not empty) - ---- - -### cat - Display Object Contents - -Output object contents to stdout. - -``` -rc cat -``` - -**Exit Codes:** 0, 4 (auth error), 5 (object not found) - ---- - -### head - Display First Lines - -Output first N lines of an object. - -``` -rc head [OPTIONS] -``` - -**Options:** -| Option | Default | Description | -|--------|---------|-------------| -| -n, --lines | 10 | Number of lines to display | - -**Exit Codes:** 0, 4 (auth error), 5 (object not found) - ---- - -### stat - Show Object Metadata - -Display object or bucket metadata. - -``` -rc stat -``` - -**Output (human):** -``` -Name : file.txt -Size : 1.2 MiB (1258291 bytes) -Type : application/octet-stream -ETag : d41d8cd98f00b204e9800998ecf8427e -Modified : 2024-01-15T10:30:00Z -``` - -**Output (--json):** See `schemas/output_v2.json#stat` - -**Exit Codes:** 0, 4 (auth error), 5 (not found) - ---- - -### cp - Copy Objects - -Copy objects between locations. - -``` -rc cp [OPTIONS] -``` - -**Options:** -| Option | Description | -|--------|-------------| -| -r, --recursive | Copy directories recursively | -| --overwrite | Overwrite existing objects | -| --no-clobber | Skip existing objects | - -**Supported Transfers:** -- Local → Remote: `rc cp ./file.txt local/bucket/` -- Remote → Local: `rc cp local/bucket/file.txt ./` -- Remote → Remote: `rc cp local/bucket1/file.txt local/bucket2/` - -**Exit Codes:** 0, 2 (invalid path), 4 (auth error), 5 (source not found) - ---- - -### mv - Move Objects - -Move (rename) objects. - -``` -rc mv [OPTIONS] -``` - -Same options and behavior as `cp`, but deletes source after successful copy. - ---- - -### rm - Remove Objects - -Delete objects. - -``` -rc rm [OPTIONS] -``` - -**Options:** -| Option | Description | -|--------|-------------| -| -r, --recursive | Delete recursively | -| --force | Don't prompt for confirmation | - -**Exit Codes:** 0, 4 (auth error), 5 (not found) - ---- - -### find - Find Objects - -Search for objects matching criteria. - -``` -rc find [OPTIONS] -``` - -**Options:** -| Option | Description | -|--------|-------------| -| --name | Glob pattern for object names | -| --larger | Minimum size (e.g., "10MB") | -| --smaller | Maximum size | -| --newer-than | Modified after (e.g., "1d", "2024-01-01") | -| --older-than | Modified before | - ---- - -### diff - Compare Locations - -Show differences between two locations. - -``` -rc diff -``` - ---- - -### mirror - Synchronize Locations - -Mirror objects from source to target. - -``` -rc mirror [OPTIONS] -``` - -**Options:** -| Option | Description | -|--------|-------------| -| --delete | Delete objects in target not in source | -| --dry-run | Show what would be done | - ---- - -### tree - Display Tree Structure - -Display objects in tree format. - -``` -rc tree [OPTIONS] -``` - ---- - -### share - Generate Presigned URL - -Generate a presigned URL for temporary access. - -``` -rc share download [OPTIONS] -``` - -**Options:** -| Option | Default | Description | -|--------|---------|-------------| -| --expire | 7d | URL expiration time | - ---- - -## Optional Commands (Capability-Dependent) - -These commands require specific backend support. If the backend doesn't support -the feature, the command returns exit code 7 (UNSUPPORTED_FEATURE). - -Use `--force` to attempt the operation anyway. - -### version - Bucket Versioning - -``` -rc version enable -rc version suspend -rc version list -``` - -### retention - Object Retention - -``` -rc retention set [OPTIONS] -rc retention clear -``` - -### tag - Object Tags - -``` -rc tag set -rc tag list -rc tag remove -``` - -### watch - Event Notifications - -``` -rc watch -``` - -### sql - S3 Select Queries - -``` -rc sql [OPTIONS] -``` - ---- - -## Configuration - -Configuration is stored in `~/.config/rc/config.toml`. - -See the plan document for full configuration schema. diff --git a/docs/TEST_MATRIX.md b/docs/TEST_MATRIX.md deleted file mode 100644 index 1f20ed8..0000000 --- a/docs/TEST_MATRIX.md +++ /dev/null @@ -1,252 +0,0 @@ -# rc 测试矩阵 - -本文档定义了 rc CLI 的测试策略和兼容性矩阵。 - -## 兼容性矩阵 - -### 后端服务 - -| 级别 | 服务 | 版本 | CI 状态 | 说明 | -|------|------|------|---------|------| -| Tier 1 | RustFS | latest | 每次 PR | 主要目标，必须 100% 通过 | -| Tier 2 | AWS S3 | - | 每次 PR | 完全支持，必须 100% 通过 | -| Tier 3 | AWS S3 | - | 每周 | 尽力支持，定期测试 | -| Best Effort | 其他 | - | 手动 | 不保证，欢迎 PR | - -### 命令分组 - -| 分组 | 命令 | 级别 | -|------|------|------| -| basic | ls, mb, rb, cat, head, stat | core | -| transfer | cp, mv, rm, pipe | core | -| advanced | find, diff, mirror, tree, share | core | -| optional | version, quota, retention, tag, watch, sql | optional | - -## 测试层级 - -### 1. 单元测试 (Unit Tests) - -**运行时机**: 每次提交 - -**覆盖范围**: -- 路径解析 (`core::path`) -- 配置管理 (`core::config`) -- 别名管理 (`core::alias`) -- 错误映射 (`core::error`) -- 退出码 (`cli::exit_code`) -- 分页合并逻辑 -- 重试策略 - -**命令**: -```bash -cargo test --workspace -``` - -### 2. 集成测试 (Integration Tests) - -**运行时机**: 每次 PR - -**覆盖范围**: -- 命令行解析和验证 -- 退出码验证（每个命令至少 2 个场景） -- JSON 输出格式验证 -- 配置文件读写 - -**命令**: -```bash -cargo test --workspace --features integration -``` - -### 3. 端到端测试 (E2E Tests) - -**运行时机**: 每日 + Release - -**环境**: Docker Compose (RustFS + S3-compatible) - -**覆盖范围**: -- 完整命令流程 -- 多后端兼容性 -- 大文件传输 -- 断点续传 -- 并发操作 - -**命令**: -```bash -docker compose -f docker/docker-compose.yml up -d -cargo test --workspace --features e2e -docker compose -f docker/docker-compose.yml down -``` - -## Golden Tests - -Golden tests 用于验证 JSON 输出格式不发生意外变化。 - -### 目录结构 - -``` -tests/ -├── golden/ -│ ├── ls_bucket.json -│ ├── ls_objects.json -│ ├── stat_object.json -│ ├── alias_list.json -│ └── ... -└── golden.rs -``` - -### 更新 Golden 文件 - -当输出格式有意变更时： - -```bash -# 更新所有 golden 文件 -UPDATE_GOLDEN=1 cargo test --features golden - -# 更新特定 golden 文件 -UPDATE_GOLDEN=1 cargo test golden::test_ls_output -``` - -### CI 验证 - -```bash -cargo test --features golden -``` - -## 退出码测试 - -每个命令必须至少测试 2 个退出码场景： - -```rust -#[test] -fn test_ls_success() { - let result = run_command(&["ls", "local/bucket"]); - assert_eq!(result.exit_code, 0); -} - -#[test] -fn test_ls_bucket_not_found() { - let result = run_command(&["ls", "local/nonexistent"]); - assert_eq!(result.exit_code, 5); // NOT_FOUND -} - -#[test] -fn test_ls_invalid_path() { - let result = run_command(&["ls", "invalid"]); - assert_eq!(result.exit_code, 2); // USAGE_ERROR -} -``` - -## CI 配置 - -### 基础 CI (ci.yml) - -```yaml -on: [push, pull_request] -jobs: - test: - - cargo fmt --check - - cargo clippy -- -D warnings - - cargo test --workspace - - cargo test --features integration -``` - -### 兼容性矩阵 (compat-matrix.yml) - -```yaml -on: - schedule: - - cron: '0 2 * * *' # 每天凌晨 2 点 - workflow_dispatch: - -jobs: - e2e: - strategy: - matrix: - backend: [rustfs, local] - command_group: [basic, transfer, advanced] -``` - -## 本地测试 - -### 快速测试 - -```bash -# 格式检查 -cargo fmt --all --check - -# Lint -cargo clippy --workspace -- -D warnings - -# 单元测试 -cargo test --workspace - -# 特定测试 -cargo test --package rc-core test_path_parsing -``` - -### 完整测试 - -```bash -# 启动测试环境 -docker compose -f docker/docker-compose.yml up -d - -# 运行所有测试 -cargo test --workspace --all-features - -# 清理 -docker compose -f docker/docker-compose.yml down -v -``` - -## 测试工具 - -### 模拟 S3 后端 - -对于单元测试和集成测试，使用 `mockall` 创建模拟的 `ObjectStore`: - -```rust -use mockall::mock; - -mock! { - pub ObjectStore {} - - #[async_trait] - impl ObjectStore for ObjectStore { - async fn list_buckets(&self) -> Result>; - // ... - } -} -``` - -### 测试辅助函数 - -```rust -// tests/common/mod.rs - -pub fn setup_test_alias() -> Alias { - Alias::new("test", "http://localhost:9000", "accesskey", "secretkey") -} - -pub async fn setup_test_bucket(client: &S3Client, name: &str) { - client.create_bucket(name).await.unwrap(); -} - -pub fn run_command(args: &[&str]) -> CommandResult { - // ... -} -``` - -## 覆盖率 - -目标覆盖率： - -| 模块 | 目标 | -|------|------| -| core | >= 80% | -| cli/commands | >= 70% | -| s3 | >= 60% | - -生成覆盖率报告： - -```bash -cargo tarpaulin --workspace --out html -``` diff --git a/docs/reference/rc/README.md b/docs/reference/rc/README.md index d3551cb..b549d5c 100644 --- a/docs/reference/rc/README.md +++ b/docs/reference/rc/README.md @@ -2,6 +2,8 @@ This reference documents the operations exposed by `rc`, the RustFS S3-compatible command-line client. The structure follows the command-reference style used by MinIO `mc`: each operation describes its purpose, syntax, parameters, examples, and behavior, while keeping the examples specific to `rc`. +For installation and quick-start usage, see the [project README](../../../README.md). + `rc` supports both noun-first command groups and legacy command names. Prefer the noun-first groups for new scripts: | Operation | Preferred form | Legacy-compatible form |