docs: cloud data uploads

[spomichter]

Usage page for dimos data (upload/ls/pull/status/quota): codec selection with measured ratios, the full GlobalConfig table (flag/env/.env forms), staging and quiet-window behavior. cli.md gains dimos login and dimos data command entries linking to it.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

@@            Coverage Diff             @@
##             main    #3865      +/-   ##
==========================================
+ Coverage   77.64%   77.73%   +0.08%     
==========================================
  Files        1323     1325       +2     
  Lines      125419   125918     +499     
  Branches    10911    10992      +81     
==========================================
+ Hits        97384    97884     +500     
+ Misses      24890    24889       -1     
  Partials     3145     3145              

Flag	Coverage Δ
OS-ubuntu-24.04-arm	74.42% <ø> (+0.10%)	⬆️
OS-ubuntu-latest	74.96% <ø> (+0.10%)	⬆️
Py-3.10	74.95% <ø> (+0.09%)	⬆️
Py-3.11	74.95% <ø> (+0.09%)	⬆️
Py-3.12	74.95% <ø> (+0.09%)	⬆️
Py-3.13	74.95% <ø> (+0.09%)	⬆️
Py-3.14	74.96% <ø> (+0.10%)	⬆️
Py-3.14t	74.95% <ø> (+0.09%)	⬆️
SelfHosted-Large	30.19% <ø> (-0.03%)	⬇️
SelfHosted-Linux	34.67% <ø> (-0.05%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 11 files with indirect coverage changes

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[greptile-apps]

Greptile Summary

This change documents hosted cloud-data workflows and adds related CLI reference entries. Three documented workflows do not match the current command behavior: combining a named path with --since uploads different discovered recordings, recent recordings skipped by discovery cause a nonzero exit, and raw files named with the configured codec suffix cannot be pulled back byte-identically. These failures were reproduced with focused executions of the cloud upload and pull paths.

Confidence Score: 2/5

Not safe to merge as documented until the upload-selection, quiet-window exit behavior, and codec-suffix round-trip failures are addressed.

Focused executable reproductions confirmed three independent user-facing failures in the documented cloud-data workflows.

Files Needing Attention: docs/usage/cli.md and docs/usage/cloud_data.md describe the affected behavior; the corresponding implementation paths are in dimos/cloud/cli.py and dimos/cloud/data.py.

T-Rex Logs

What T-Rex did

• Reproduced the P1 finding using a narrow PATH reproduction harness and captured the PATH-specific command output.
• Reproduced quiet-window discovery using the provided reproduction source and documented the command output.
• Explored an executable invalid artifact.lz4 round-trip harness and logged the upload/pull outputs.
• Viewed the path-since-priority-02-after.py harness results, including which files were uploaded or skipped and how the explicit-path setting affected behavior.
• Ran the quiet-window discovery repro with uv run and noted the observed DISCOVERED files, QUIET_SECONDS, and the CLI exit behavior.

_{Ran code and verified through T-Rex}

Comments Outside Diff (3)

1. General comment

   PATH is silently ignored when combined with --since

   • Bug
     • The CLI accepts the documented combined form dimos data upload [PATH|latest] [--since 1h], but supplying both a named path and --since uploads every discovered recording within the time window instead of the named path. The executable harness equivalent to dimos data upload named-path.db --since 1h called discovery with 3600.0 and uploaded two unrelated fixtures, not named-path.db.
   • Cause
     • dimos/cloud/cli.py:62 uses recordings(since_s) if since_s else [path] if path else ..., so a truthy since_s takes precedence over path without rejecting or documenting the conflict.
   • Fix
     • Reject the ambiguous combination at CLI validation time (preferred), or explicitly define and document its precedence. If both behaviors are required, use separate options/subcommands so a named path cannot be silently discarded.

   _{Ran code and verified through T-Rex}

2. General comment

   Discovery quiet-window skip is surfaced as command failure

   • Bug
     • For a recording discovered without an explicit path and modified within the 30-second quiet window, the CLI reports file still being written — skipped and exits 1 instead of completing successfully after skipping it.
   • Cause
     • CloudData.upload() raises RuntimeError for a recent discovered file (dimos/cloud/data.py:223-224), and dimos.cloud.cli.upload() catches that exception, sets failed=True, then raises typer.Exit(1) (dimos/cloud/cli.py:81-85).
   • Fix
     • Represent a quiet-window discovery result as a non-error skip (or handle this specific condition without setting failed) so no-argument and --since discovery modes return success when recent recordings are omitted, matching docs/usage/cloud_data.md:25.

   _{Ran code and verified through T-Rex}

3. General comment

   Raw files ending in .lz4 are incorrectly stamped as LZ4-compressed

   • Bug
     • With default LZ4 configuration, a non-LZ4 file named artifact.lz4 skips compression, is transferred unchanged, and is recorded as LZ4 encoded. Pull uses that encoding stamp, attempts to decode raw invalid bytes, and fails rather than restoring the source.
   • Cause
     • MultipartBackend.upload uses suffix equality to bypass compression (dimos/cloud/data.py:123-130) but independently unconditionally sets content_encoding=self.codec_id (dimos/cloud/data.py:132-138).
   • Fix
     • Set content_encoding only when compression was performed, or always stage/compress configured-codec uploads regardless of their input suffix. Add a regression test that uploads invalid bytes named artifact.lz4 and requires byte-identical pull.

   _{Ran code and verified through T-Rex}

_{Reviews (1): Last reviewed commit: "docs: cloud data uploads" | Re-trigger Greptile}

[greptile-apps]

Upload selectors silently conflict

The synopsis permits combining a path with --since, but dimos data upload named-path.db --since 1h ignores named-path.db and uploads every recording discovered in the time window instead. Reject this ambiguous combination, or document and implement an explicit precedence that cannot silently discard the user-selected file.

Artifacts

Narrow PATH plus --since reproduction harness

• Authored and executed a minimal in-process harness that feeds the upload handler the parsed equivalent of `dimos data upload named-path.db --since 1h` and records targets, proving the named path is not selected.

Observed PATH plus --since command output

• Captured output of `python trex-artifacts/path-since-priority-02-after.py` from `/home/user/repo`; it shows discovery received 3600 seconds and uploaded unrelated recordings instead of the named path, confirming the claim.

_{Ran code and verified through T-Rex}

[greptile-apps]

Quiet-window skips fail discovery uploads

A fresh recording found by no-argument or --since discovery is reported as file still being written — skipped, but the command exits with status 1. Handle this expected quiet-window condition as a successful skip so the documented discovery behavior does not make scheduled uploads fail.

Artifacts

Quiet-window discovery reproduction source

• Authored Python harness creates one fresh recording and executes the repository’s data discovery and CLI upload sources with offline-only dependency stubs; the takeaway is that the tested path is narrow and reproducible.

Quiet-window discovery command output

• Captured stdout, stderr, command, working directory, and exit behavior from the `uv run --no-sync` reproduction; the takeaway is that discovery selects the fresh recording and the CLI exits 1.

_{Ran code and verified through T-Rex}

[greptile-apps]

Codec suffix breaks arbitrary-file round trips

An arbitrary non-LZ4 file named artifact.lz4 bypasses compression because of its filename, but is still stored with content_encoding=lz4. Pull then tries to decompress the raw bytes and fails instead of restoring the file byte-identically. Only stamp an encoding when compression occurred, or always stage and compress uploads using the configured codec.

Artifacts

Executable invalid artifact.lz4 round-trip harness

• This isolated in-memory transport invokes the real upload and pull implementation with invalid bytes named artifact.lz4, confirming the target path is exercised.

Captured invalid artifact.lz4 harness source

• This command capture includes the harness SHA-256 and complete executed source so the reproduction inputs and calls are auditable, confirming the exact test setup.

Observed invalid artifact.lz4 upload and pull output

• This successful command capture shows identical raw source and wire hashes, an lz4 encoding stamp, and the LZ4 decode failure with no output file, confirming the documented round trip fails.

_{Ran code and verified through T-Rex}

[spomichter]

Confirmed; fixing in a draft PR against main (stamp content_encoding only when compression ran, regression test included) — branch fix/codec-stamp.