Add dittosh server group — Ditto Server (portal) HTTP API client - #4
Open
Aaron LaBeau (biozal) wants to merge 2 commits into
Open
Add dittosh server group — Ditto Server (portal) HTTP API client#4Aaron LaBeau (biozal) wants to merge 2 commits into
Aaron LaBeau (biozal) wants to merge 2 commits into
Conversation
New command group covering the portal HTTP API surface (public docs + the portal's own rpcClient wire shapes): - server execute (alias exec): DQL over HTTP — positional/-e/-f/stdin batch, -p/--args, --txn-id, --api-version v4|v5, all output formats, -o export, pager, --max-rows, --time, --continue-on-error - server remote-execute: DQL on connected peers (SYNC CONTEXT) - server attachment upload/get (multipart; binary-safe download) - server roles list/create/delete, users list/set-roles/delete (RBAC) - server webhook-secrets list/create/rotate/delete - server doctor: config -> connection -> auth probe with source labels Config: --url/--api-key > shell env (DITTOSH_SERVER_URL/_API_KEY, aliases DITTO_CLOUD_URL/DITTO_API_KEY) > cwd .env. Key never printed; redacted from errors. Legacy pre-DQL store API (find/findbyid/count/ write) deliberately omitted — execute covers it. 802 tests green (unit: mock FetchLike, hermetic env scrubbing; e2e: node:http mock server, execa extendEnv:false). Coverage 90.8/86.2/ 96.2/92.1 vs 85 gate. docs/testing-server.md manual checklist. Verified live against the portal (retail app). Adversarial review: 4 rounds, 2 reviewers, CONVERGED.
Agreed majors: - Fail closed on non-contract 2xx bodies: execute/remote-execute require DQL hallmark keys (no more silent [] on proxy/SSO HTML 200s; no raw TypeError on items:'string'); roles/users require their envelope keys (portal parity); webhook list keeps portal's [] fallbacks + 404 -> []. - Reject non-loopback http:// — the cloud's 308 sent the key cleartext on hop 1, then stripped it on redirect -> misleading 401. - --timeout flag (default 120s) + PortalTimeoutError: honest 'may still be running' message, exit 1 (a timeout is not proof of failure; the portal leaves DQL unbounded — 30s stays only for admin endpoints). - roles create --help example was a shell syntax error in zsh AND bash (unescaped single quotes) — now uses " " inside JSON. - docs/testing-server.md webhook section rewritten: the provider must already exist (verified live: 400 Provider not found). Agreed minors: .env UTF-8 BOM strip; per-key .env hints; batch checks stdoutBroken() (| head); users-list 404 unsupported-endpoint hint; --permissions JSON strings checked against the blanket whitelist; remote-execute sets DITTOSH_JSON_OUT + accepts leading comments; printWarnings never prints 'undefined'; -o help includes ADVISE; usage-first --api-version ordering; stripEq comment corrected; No-statements error is red; dead ?? removed. Process: e2e coverage gaps closed per AGENTS.md (doctor, users, webhook-secrets, roles create/delete, attachment upload, remote-execute happy path). README gained the full dittosh server feature section; exit-code table updated. Gates by exit code: lint 0, tsc 0, 765+ tests 0, coverage 90.7/86.3/ 96.3/92.1 (gate 85), build 0. Live-verified against the portal.
Aaron LaBeau (biozal)
marked this pull request as ready for review
September 4, 2026 13:00
Aaron LaBeau (biozal)
requested review from
Chris Draper (draper-ditto) and
Skyler Jokiel (skylerjokiel)
September 5, 2026 00:44
Erik Everson (ErikEverson)
approved these changes
Sep 7, 2026
Erik Everson (ErikEverson)
left a comment
Erik Everson (ErikEverson)
left a comment
There was a problem hiding this comment.
Looked at the auth stuff
| - [x] `server remote-execute` — POST `/api/v5/sync/remote_execute`; client-side SYNC CONTEXT check (exit 2); per-peer JSON envelope out. | ||
| - [x] ~~Legacy store API~~ — **pulled post-review**: find/findbyid/count/write are the legacy pre-DQL API; `server execute` covers all of it with full DQL. (Verified live that legacy `:param` placeholders are rejected by the current deployment despite the OpenAPI text — another reason not to ship them.) | ||
| - [x] `server attachment upload|get` — multipart POST / byte GET; get refuses binary on a TTY without `-o`, pipes raw bytes otherwise. | ||
| - [x] `server roles list|create|delete` + `server users list|set-roles|delete` — RBAC endpoints the portal uses (undocumented publicly); both GET /roles wire shapes (bucketed + cursor-paged) normalized. Destructive deletes confirm (`-y` / TTY prompt / exit 2 piped). |
There was a problem hiding this comment.
One thing to note here. There are two different levels of user management that are possible.
- Control plane user management (The user data that the overall big peer and better auth stores)
- AppId/DatabaseId user management (the one currently implemented)
Since a user can be signed in to a big peer and be part of multiple databaseid's or be added and then removed from a databaseId's we have a concept of the user at the big peer level and a user at the databaseId level.
The Control plane user management is handled through BetterAuth API's today (we may wrap them eventually but it was not worth it given our time constraints for the initial launch of Auth v2)
Better Auth has both the Admin plugin, the organizations plugin and the SSO plugins enabled and the API's related to those if you wanted to also have this able to handle those as well
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a
dittosh servercommand group: a full HTTPS client for the Ditto Server (Big Peer) HTTP RPC API — the same API the portal's DQL editor uses. Endpoint inventory from the public docs (incl. the OpenAPI spec) and the portal's own client in cloud-services (portal/core/src/api/rpcClient.ts), which covers undocumented endpoints (RBAC, webhook secrets).Commands
server execute(aliasexec)POST /api/v5/store/execute(--api-version v4= strict mode)server remote-executePOST /api/v5/sync/remote_execute(peers, SYNC CONTEXT)server attachment upload/get/api/v4/attachments(multipart up, raw bytes down)server roles list/create/delete,server users list/set-roles/delete/api/v4/auth/*(portal RBAC)server webhook-secrets list/create/rotate/delete/api/v4/auth/webhook/secretserver doctorThe legacy pre-DQL store API (find/findbyid/count/write) is deliberately omitted —
executecovers it with full DQL (and its:paramplaceholders are rejected by the current deployment despite the OpenAPI text).Configuration
--url/--api-keyflags > shell env (DITTOSH_SERVER_URL/DITTOSH_SERVER_API_KEY, aliasesDITTO_CLOUD_URL/DITTO_API_KEY) >.envin the current directory. The API key is never printed and is redacted from error messages. Exit codes follow the house contract (2 usage · 3 config/auth/connection · 1 query/API).Every command's
--helpcarries request body shapes, wire formats, and examples — most of these APIs aren't publicly documented, so the help text is the documentation.Testing
FetchLikemock (no network), hermetic env scrubbing + tmpdir chdir (repo.envmay hold real portal creds). e2e: real subprocess against a localnode:httpmock Ditto Server, execaextendEnv: false.--args -TTY hang, mid-body timeout mapping, roles-list pagination truncation, …). Ledger inplans/SDKS-4855-implementation-plan.mdM9.docs/testing-server.md: copy-pasteable manual checklist for the portal.Notes
.env.sampledocuments the two new variables.http_loginexchange, CDC/Kafka streams, server-side REPL.