diff --git a/.github/CICD.md b/.github/CICD.md index fa616c528..2c59bb533 100644 --- a/.github/CICD.md +++ b/.github/CICD.md @@ -187,6 +187,29 @@ This ensures: - Each APK is built against the exact assets produced in the same run - Formplayer build outputs do not pollute git history +### Documentation Site + +**Workflow**: `.github/workflows/docs.yml` + +Builds and publishes the Docusaurus site in [`docs/`](../docs/) to [opendataensemble.org](https://opendataensemble.org/) via GitHub Pages. + +#### Triggers + +| Event | Result | +|-------|--------| +| Pull request to `main` or `dev` touching `docs/**` | Validate and build only | +| Push to `main` touching `docs/**` | Validate, build, and deploy | +| `workflow_dispatch` | Validate and build | + +#### Notes + +- The deployed branch is main, so the published docs track the released version of ODE. Changes on dev and pull requests are validated by CI but are not deployed. +- The site is an independent **npm** project inside the pnpm monorepo: `cache-dependency-path` points at `docs/package-lock.json`, which is force-included in the root `.gitignore` so `npm ci` is reproducible. +- `docs/docs/` is the Docusaurus content root. URLs are unchanged from the previous standalone `OpenDataEnsemble/docs` repository — `routeBasePath` is still `/docs`. +- `actions/upload-pages-artifact@v4` is used deliberately: v5 excludes dotfiles by default, which would drop `static/.nojekyll` and `static/CNAME` from the artifact. +- The upload path is `docs/build` (repo-root relative). `defaults.run.working-directory` does not apply to `uses:` steps. +- No secrets are required. The deploy job needs `contents: read`, `pages: write`, and `id-token: write`. + ### SBOM (CycloneDX) on releases **Workflow**: `.github/workflows/sbom-release.yml` diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml new file mode 100644 index 000000000..4d1bc8ed1 --- /dev/null +++ b/.github/workflows/docs.yml @@ -0,0 +1,102 @@ +name: Docs + +on: + pull_request: + branches: [main, dev] + paths: + - 'docs/**' + - '.github/workflows/docs.yml' + push: + branches: [main, dev] + paths: + - 'docs/**' + - '.github/workflows/docs.yml' + workflow_dispatch: + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + # Do not cancel an in-flight production deploy when another push lands. + cancel-in-progress: ${{ github.event_name != 'push' }} + +permissions: + contents: read + +env: + NODE_VERSION: '24' + +jobs: + validate: + name: Validate + runs-on: ubuntu-latest + defaults: + run: + working-directory: docs + steps: + - name: Checkout repository + uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + fetch-depth: 0 + + - name: Set up Node.js + uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 + with: + node-version: ${{ env.NODE_VERSION }} + cache: npm + cache-dependency-path: docs/package-lock.json + + - name: Install dependencies + run: npm ci + + - name: Check links and doc IDs + run: npm run test + + - name: Build site + env: + CI: true + run: npm run build + + deploy: + name: Deploy to GitHub Pages + needs: validate + if: github.event_name == 'push' && github.ref == 'refs/heads/main' + runs-on: ubuntu-latest + + permissions: + contents: read # checkout (job permissions replace the workflow defaults) + pages: write # to deploy to Pages + id-token: write # to verify the deployment originates from an appropriate source + + environment: + name: github-pages + url: ${{ steps.deployment.outputs.page_url }} + + defaults: + run: + working-directory: docs + + steps: + - name: Checkout repository + uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + + - name: Set up Node.js + uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 + with: + node-version: ${{ env.NODE_VERSION }} + cache: npm + cache-dependency-path: docs/package-lock.json + + - name: Install dependencies + run: npm ci + + - name: Build site + run: npm run build + + - name: Upload Pages artifact + uses: actions/upload-pages-artifact@7b1f4a764d45c48632c6b24a0339c27f5614fb0b # v4 + with: + # Action paths are relative to the repo root, not defaults.run.working-directory. + path: docs/build + + - name: Deploy to GitHub Pages + id: deployment + uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4 diff --git a/.gitignore b/.gitignore index c241ba558..923b5b760 100644 --- a/.gitignore +++ b/.gitignore @@ -11,6 +11,8 @@ package-lock.json **/package-lock.json # F-Droid builds use npm ci / install --build-from-source in formulus/ !formulus/package-lock.json +# Documentation site deploys with npm ci, so its lockfile must be committed +!docs/package-lock.json # Generated credentials **/credentials.txt diff --git a/AGENTS.md b/AGENTS.md index 419a3eda4..546d3fbb4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -57,6 +57,7 @@ Do not assume custom app authors have local checkouts of **ODE** or internal exa | [packages/tokens](packages/tokens/) | Design tokens (`@ode/tokens`) | Style Dictionary | [packages/tokens/AGENTS.md](packages/tokens/AGENTS.md) | | [packages/components](packages/components/) | Shared UI (`@ode/components`) | React | [packages/components/AGENTS.md](packages/components/AGENTS.md) | | [desktop](desktop/) | Data management + Forms / app workbench (Tauri) | React, Rust | [desktop/AGENTS.md](desktop/AGENTS.md) | +| [docs](docs/) | Documentation site (published at [opendataensemble.org](https://opendataensemble.org/)) | Docusaurus, MDX | [docs/README.md](docs/README.md) | --- diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0821317fe..ccd3d7190 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -64,6 +64,7 @@ This is a monorepo, so the setup depends on the project you're touching. The ful - **synkronus-cli** - Go command-line client. See `synkronus-cli/README.md`; pre-flight is `cd synkronus-cli && go build ./cmd/synkronus`. - **synkronus-portal** - web admin UI (React). See `synkronus-portal/README.md` for both Docker and Dockerless setups. - **desktop** - Tauri app. See `desktop/README.md` (Quick start). +- **docs** - public Docusaurus site under `docs/`. It is an independent **npm** project (`cd docs && npm install && npm start`); do not use pnpm there. See `docs/README.md`. - **packages/tokens** and **packages/components** - see `packages/tokens/CONTRIBUTING.md` and `packages/components/CONTRIBUTING.md`. The quickest cheat sheet for any package is its `AGENTS.md` file - it lists the day-to-day commands and exactly what to run before a pull request. diff --git a/FORM_LOCALIZATION_GUIDE.md b/FORM_LOCALIZATION_GUIDE.md index 00acaa419..bbd7f9921 100644 --- a/FORM_LOCALIZATION_GUIDE.md +++ b/FORM_LOCALIZATION_GUIDE.md @@ -233,7 +233,7 @@ Behavioral config (`maxStars`, filters, …) stays in **`schema.json`** (`config | UI locale preference (host) | `formulus/src/lib/locale.ts`, Settings → Language | | Linked child specs for sub-obs columns | `FormInitData.linkedFormSpecs` (built in Formulus / ODE Desktop) | -When changing merge rules or label resolution, update **`applyFormUiTranslations.test.ts`**, affected renderers, and the [published form translations guide](https://opendataensemble.org/docs/guides/form-translations) in **ode-docs**. +When changing merge rules or label resolution, update **`applyFormUiTranslations.test.ts`**, affected renderers, and the [published form translations guide](https://opendataensemble.org/docs/guides/form-translations) in **`docs/docs/guides/form-translations.md`**. --- diff --git a/README.md b/README.md index a2bbb508c..d671cf1e6 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,7 @@ [![Formulus Android](https://github.com/OpenDataEnsemble/ode/actions/workflows/formulus-android.yml/badge.svg?branch=main)](https://github.com/OpenDataEnsemble/ode/actions/workflows/formulus-android.yml) [![ODE Desktop](https://github.com/OpenDataEnsemble/ode/actions/workflows/ode-desktop.yml/badge.svg?branch=main)](https://github.com/OpenDataEnsemble/ode/actions/workflows/ode-desktop.yml) [![E2E attachments](https://github.com/OpenDataEnsemble/ode/actions/workflows/e2e-attachments.yml/badge.svg?branch=main)](https://github.com/OpenDataEnsemble/ode/actions/workflows/e2e-attachments.yml) +[![Docs](https://github.com/OpenDataEnsemble/ode/actions/workflows/docs.yml/badge.svg?branch=dev)](https://github.com/OpenDataEnsemble/ode/actions/workflows/docs.yml) [![Latest release](https://img.shields.io/github/v/release/OpenDataEnsemble/ode?include_prereleases&sort=semver)](https://github.com/OpenDataEnsemble/ode/releases) @@ -26,7 +27,7 @@ ODE is a monorepo containing all the core components in the ODE universe - the e ## Architecture -This repository houses four main components: +This repository houses the core components plus the public documentation site: ### **formulus** A React Native project containing the code for Android and iOS apps. This is your mobile data collection companion, designed for field work and offline-first data gathering. @@ -58,6 +59,9 @@ curl -fsSL https://raw.githubusercontent.com/OpenDataEnsemble/ode/main/scripts/i ### **synkronus-portal** A web-based version of the the **synkronus-cli** +### **docs** +The public documentation site (Docusaurus), published at [opendataensemble.org](https://opendataensemble.org/). It is an independent npm project inside this monorepo — see [docs/README.md](docs/README.md). + ## We're Young & Fresh! 🌱🌱🌱 ODE is a **young and vibrant open-source project**, and we're incredibly welcoming to contributors of all experience levels and interests! Whether you're passionate about: @@ -89,6 +93,7 @@ This monorepo uses GitHub Actions for CI/CD. For details (trigger conditions, ta - Synkronus Docker build & publish: `.github/workflows/synkronus-docker.yml` - Formulus Android build (includes Formplayer asset build): `.github/workflows/formulus-android.yml` +- Documentation site: `.github/workflows/docs.yml` (validates PRs; deploys from `dev`) - Synkronus deployment docs: `synkronus/DOCKER.md`, `synkronus/DEPLOYMENT.md` ## Code Quality: Linting & Formatting diff --git a/docs/.gitignore b/docs/.gitignore new file mode 100644 index 000000000..b23aad10b --- /dev/null +++ b/docs/.gitignore @@ -0,0 +1,44 @@ +# Dependencies +node_modules/ + +# Production build +build/ +.docusaurus/ +.cache-loader/ + +# Generated files +.docusaurus/ +.cache/ + +# Misc +.DS_Store +.env +.env.local +.env.development.local +.env.test.local +.env.production.local + +# Logs +npm-debug.log* +yarn-debug.log* +yarn-error.log* +lerna-debug.log* +devserver.log +devserver.err.log +devserver.pid + +# Editor directories and files +.idea/ +.vscode/ +*.swp +*.swo +*~ +.project +.classpath +.settings/ +*.sublime-workspace + +# OS +.DS_Store +Thumbs.db + diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 000000000..c7693fe2c --- /dev/null +++ b/docs/README.md @@ -0,0 +1,54 @@ +# ODE Documentation Site + +The ODE documentation site, built with [Docusaurus](https://docusaurus.io/) and published at [opendataensemble.org](https://opendataensemble.org/). + +This site lives in the [`OpenDataEnsemble/ode`](https://github.com/OpenDataEnsemble/ode) monorepo under `docs/`. It is an independent **npm** project — the rest of the monorepo uses **pnpm**, so run npm commands from this directory. + +## Layout + +| Path | Purpose | +|------|---------| +| `docs/docs/` | Documentation content (Markdown/MDX). One folder per section: `getting-started/`, `guides/`, `using/`, `reference/`, `community/`, … | +| `docs/static/` | Assets copied verbatim to the site root (`img/`, `fonts/`, `diagrams/`), plus `CNAME` and `.nojekyll` | +| `docusaurus.config.ts` | Site config: navbar, footer, routing, plugins | +| `sidebars.ts` | Sidebar structure (persona-based: For Data Collectors / For Implementers / For Developers) | +| `plugins/` | Remark/rehype link fixups | +| `scripts/validate-docs.ts` | Fast validation run by `npm run test` | +| `src/` | Custom React components, theme overrides, and the `/downloads` page | + +## Local development + +```bash +npm install +npm start +``` + +Starts a dev server with hot reload. The site is served at `/`, documentation under `/docs`. + +## Validation + +Run these before opening a PR — the CI workflow runs the same commands. + +```bash +npm run test # fast: docId references, internal links, config paths +npm run build # full: compiles every page, fails on broken links +npm run validate # both of the above +``` + +`npm run build` is the real gate: the site is configured with `onBrokenLinks: 'throw'`, so a broken internal link fails the build rather than shipping. + +## Deployment + +[`docs.yml`](../.github/workflows/docs.yml) publishes the site to GitHub Pages: + +| Trigger | Result | +|---------|--------| +| Pull request to `main` or `dev` touching `docs/**` | Validate and build only | +| Push to `dev` touching `docs/**` | Validate, build, and deploy | +| Manual (`workflow_dispatch`) | Validate and build | + +The deployed branch is **`dev`**, so the site reflects the latest merged work rather than the last release. Deploys run with `actions/deploy-pages` and require no secrets. + +:::note Versioning +Docusaurus versioning is currently **disabled** (`disableVersioning: true` in `docusaurus.config.ts`, and `versions.json` is empty). There is no version dropdown: every visitor sees the current docs. +::: diff --git a/docs/babel.config.js b/docs/babel.config.js new file mode 100644 index 000000000..c0097fcba --- /dev/null +++ b/docs/babel.config.js @@ -0,0 +1,4 @@ +module.exports = { + presets: [require.resolve('@docusaurus/core/lib/babel/preset')], +}; + diff --git a/docs/custom-question-types.md b/docs/custom-question-types.md deleted file mode 100644 index d5181c820..000000000 --- a/docs/custom-question-types.md +++ /dev/null @@ -1,182 +0,0 @@ -# Custom Question Types in Formplayer - -Formplayer supports a plugin system that allows you to render custom UI components for specific questions in your forms. This is useful for complex interactions that standard JSON Forms inputs cannot handle (e.g., drag-and-drop ranking, signature pads, or complex search interfaces). - -## How it Works - -1. **Folder Structure**: You place your custom component inside the `/forms/question_types//` directory of your `custom_app`. -2. **Implementation**: You create a JavaScript file (`renderer.js` or `index.js`) that exports a React component. -3. **Usage in Schema**: You set the `"format"` property in your `schema.json` to match the `` folder. -4. **Loading Payload**: When Formulus opens a form, it reads all custom question types in the `question_types` folder and injects their source code into the Formplayer WebView. -5. **Execution**: Formplayer dynamically evaluates and registers your component. It wraps it in an error boundary and adapts it to the JSONForms architecture. - -### Formulus API: WebView file URLs (v1.2.0+) - -Inside the WebView, `` cannot load legacy relative paths like `/default/data/tables/...`. Use the injected `getFormulus()` API (see `FormulusInterfaceDefinition.ts` in Formulus / Formplayer): - -- **`getAttachmentUri(fileName)`** — returns a `file://` URL if that basename exists under the app attachments directory (or `pending_upload`), else `null`. Use the observation media `filename` / `photo.filename` basename. -- **`getAttachmentsUri()`** — base `file://` URL for the attachments folder (trailing slash). -- **`getCustomAppUri()`** — base `file://` URL for `DocumentDirectory/app/`. -- **`getFormSpecsUri()`** — base `file://` URL for `DocumentDirectory/forms/`. - ---- - -## Creating a Custom Question Type - -### 1. File Location - -Create a folder named after your custom format (e.g., `ranking`). Inside it, create `renderer.js`. - -``` -custom_app/ - forms/ - question_types/ - ranking/ - renderer.js <-- Your component -``` - -### 2. The Component Interface - -Your `renderer.js` must **default export** a React component (CommonJS style `module.exports = { default: Component }`). It receives the following props: - -```ts -interface CustomQuestionTypeProps { - // 1. Current field data - value: any; - - // 2. The callback to update the form's data state. Must be called when user changes value. - onChange: (newValue: any) => void; - - // 3. Schema parameters. Any non-standard JSON Schema keys are passed here. - config: Record; - - // 3b. Display strings from ui.json Control.options (after locale preprocess). - options?: Record; - - // 4. Validation state for styling error cases - validation: { - error: boolean; - message: string; - }; - - // 5. Context from JSON Forms to access the whole form data/schema - jsonFormsContext: { - core: { - data: any; // The entire form's current data - schema: any; // The root schema - errors: any[]; // All validation errors - }; - // ...other JSONForms state - }; - - // 6. UI properties - enabled: boolean; - label: string; - description?: string; - fieldPath: string; // The dot-notation path in the data (e.g. "team.ranking") -} -``` - -### 3. Example Implementation: `renderer.js` - -Because the Formplayer evaluates this at runtime within a browser without a bundler, you cannot rely on ES modules (`import/export`). Instead, use CommonJS (`module.exports`) and rely on injected globals like `React` and `MaterialUI`. - -```javascript -const { useState, useEffect } = React; -const { Button, Typography, Box } = MaterialUI; - -function MyCustomRenderer(props) { - const { value, onChange, config, validation, label } = props; - - // "config" contains any extra properties from your schema.json - const maxItems = config.maxItems || 5; - - return ( - - {label} - Current Value: {JSON.stringify(value)} - - - - {validation.error && ( - {validation.message} - )} - - ); -} - -// Emulate an ES Module default export for the loader -module.exports = { - default: MyCustomRenderer, -}; -``` - ---- - -## Internationalization - -Custom question types use the **same** `ui.json` `translations` pattern as built-in controls. Put user-visible strings in `label`, `description`, and `Control.options` — not hardcoded in `renderer.js`: - -```json -{ - "type": "Control", - "scope": "#/properties/rating", - "label": "Rate this", - "options": { "hint": "Tap a star" }, - "translations": { - "pt": { - "label": "Avalie", - "options": { "hint": "Toque numa estrela" } - } - } -} -``` - -The renderer reads `props.options.hint`. Behavioral settings (`maxStars`, filters) stay in `schema.json` as `config`. See [Form translations](https://opendataensemble.org/docs/guides/form-translations). - ---- - -## Using it in your Form - -Once your custom question type is defined, use it in your form's `schema.json`. - -By default, standard JSON Schema properties (`type`, `title`, `description`, `required`, etc.) are consumed by the core engine. **Any other custom properties** inside the field definition will be passed to your component inside the `config` prop! - -```json -{ - "type": "string", - "title": "Select a Person", - "format": "select-person", <-- Matches the folder name - - "endpoint": "/api/v1/people", <-- Passed to props.config.endpoint - "showSearch": true, <-- Passed to props.config.showSearch - "theme": "dark" <-- Passed to props.config.theme -} -``` - -### Data Storage - -Your component can return complex objects, arrays, or primitive values back to `onChange()`. Just ensure the `"type"` property in your schema matches what you are returning (e.g. `"type": "object"` if returning an object) so that AJV validation passes. - ---- - -## Custom validators (related) - -Custom **question types** render UI; custom **validators** (`validators//index.js` in the app bundle) run on `ui.json` `options.customValidators` and return errors. Validators may also **mutate** the full form `data` object in place (for example assigning sequence numbers on embedded sub-observation arrays). Formplayer detects those mutations and refreshes state so tables and dependent fields update without extra custom question types. - -**Per-session scope:** Validators run only in the **active** Formplayer session. Nested sub-observation child forms need their own validators (or parent snapshot init fields) for numbering and cross-row rules — root-only validators are not enough for deep embedded trees. - -See [Custom Extensions](https://opendataensemble.org/docs/guides/custom-extensions) on opendataensemble.org for validator packaging, [nested sessions](https://opendataensemble.org/docs/guides/custom-extensions#nested-sessions-and-custom-validators), [parent context](https://opendataensemble.org/docs/guides/custom-extensions#parent-context-across-nesting-levels), and sub-observation configuration (`linkedForm` required; `parentKey` optional). - ---- - -## Error Handling - -If your custom component throws an exception or crashes while rendering, Formplayer will catch it and display a red fallback UI in place of your question. This ensures that a bug in one custom question does not break the entire form or block the user from answering other questions. diff --git a/docs/docs/.gitkeep b/docs/docs/.gitkeep new file mode 100644 index 000000000..e69de29bb diff --git a/docs/docs/collector/collector-getting-started.md b/docs/docs/collector/collector-getting-started.md new file mode 100644 index 000000000..841deb485 --- /dev/null +++ b/docs/docs/collector/collector-getting-started.md @@ -0,0 +1,360 @@ +--- +sidebar_position: 1 +title: Getting Started with Formulus +--- + +# Getting Started: Data Collection with Formulus + +This guide walks you through installing Formulus and submitting your first form. It should take about **10-15 minutes**. + +## What You'll Need + +Before starting, make sure you have: + +- ✅ A smartphone (Android 8.0+ or iOS 13.0+) +- ✅ Internet access (for downloading and initial setup) +- ✅ Instructions from your project manager (server URL, project code, or app link) + +## Step 1: Install Formulus + +### Option A: F-Droid (Android) + +1. Open the [Formulus page on F-Droid](https://f-droid.org/en/packages/org.opendataensemble.formulus/) +2. Install the F-Droid client if prompted +3. Tap **Install** and wait for the download to complete + +### Option B: App Store (iPhone and iPad) + +1. Open the [Formulus App Store page](https://apps.apple.com/dk/app/formulus/id6798318215) +2. Tap **Get** +3. Authenticate with Face ID, Touch ID, or Apple ID +4. Wait for installation to complete + +### Option C: Obtainium or direct APK (Android) + +Use [Obtainium](https://github.com/ImranR98/Obtainium) with `https://github.com/OpenDataEnsemble/ode` for updates from GitHub Releases, or download the APK directly from [Downloads](/downloads). + +:::note +The [Downloads](/downloads) page has the current Android APK and all desktop/CLI downloads. +::: + +## Step 2: Open Formulus & Connect to Your Project + +1. **Open Formulus** - Tap the app icon on your home screen +2. **See the welcome screen** - You'll see the Formulus logo and connection options +3. **Enter Server Details** - Ask your project manager for: + - Server URL (e.g., `https://forms.myorganization.org`) + - Project Code or QR code +4. **Authenticate** - Enter credentials provided by your project manager +5. **Download Forms** - Formulus will download available forms for your project + +:::tip Getting Your Server Details +Your project manager should provide you with: +- The server web address +- Your username and password (or a QR code) +- The project or team you're assigned to + +If you don't have these, contact your project manager. +::: + +## Step 3: Submit Your First Form + +### Opening a Form + +1. **Tap the "+" button** or **"New Submission"** in the app +2. **Select a form** from the list +3. **Read the form title** - This tells you what you're collecting + +### Filling Out the Form + +Forms contain different types of fields: + +| Type | How to Fill | Example | +|------|-----------|---------| +| **Text** | Type your answer | Name, comments | +| **Number** | Enter digits | Age, count | +| **Date** | Tap to pick a date | Birthday, visit date | +| **Multiple Choice** | Select one option | Gender, yes/no | +| **Photo** | Tap camera icon | Pictures of situation | +| **GPS** | Tap location button | Geographic coordinates | + +:::info Required Fields +Fields marked with a **red asterisk (*)** must be filled out before submitting. +::: + +### Example: Simple Survey + +Let's say you're filling out a health survey: + +``` +Question: What is your name? +↓ Type: "John Smith" + +Question: What is your age? +↓ Tap number field and type: "32" + +Question: Are you feeling healthy today? +↓ Select: "Yes" or "No" + +Question: Date of visit +↓ Tap calendar icon and select today's date +``` + +### Validating Your Answers + +**As you type**, Formulus checks if your answers are valid: + +- ✅ **Green checkmark** = Answer is correct +- ❌ **Red error** = Something is wrong (e.g., age over 150) +- ⚠️ **Yellow warning** = Be careful with this answer + +Fix any errors before submitting. + +## Step 4: Submit the Form + +### When You're Done + +1. **Review your answers** - Scroll up to check everything +2. **Look for the Submit button** - Usually at the bottom +3. **Tap Submit** - Formulus will validate your form +4. **Confirm** - You'll see a message saying the form was submitted + +### What Happens After Submit? + +``` +Submitted Locally + ↓ +Form saved on your phone + ↓ +Waiting to sync with server + ↓ +[When you connect to internet] + ↓ +Synced to Server ✅ +``` + +You'll see a **"Submitted"** section in the app showing all your completed forms. + +## Step 5: Sync Your Data + +Syncing sends your completed forms from your phone to the server. + +### Automatic Sync + +When your phone has internet: +- Formulus automatically syncs in the background +- You'll see a sync icon (🔄) at the top of the app +- Submissions appear as "Synced" + +### Manual Sync + +To sync manually: + +1. **Tap the sync icon** (usually at top-right of app) +2. **Wait for the sync to complete** +3. **Check the status** - Green checkmark means success + +:::tip Sync Status +- 🟢 **Synced** - Successfully sent to server +- 🟡 **Pending** - Waiting to sync +- 🔄 **Syncing** - Currently sending +- ❌ **Error** - Check your connection + +If sync fails, check: +1. Do you have internet? +2. Is the server accessible? +3. Are you still logged in? + +Contact your project manager if problems persist. +::: + +## Working Offline + +One of ODE's superpowers is **offline capability**. + +### Can I Work Without Internet? + +**Yes!** You can: +- ✅ Create new submissions +- ✅ Fill out forms completely +- ✅ Take photos and record audio +- ✅ Save your work + +### How Offline Works + +``` +No Internet Available + ↓ +Forms stored on phone + ↓ +You create & save submissions + ↓ +Data saved locally (not on server yet) + ↓ +When you connect to internet + ↓ +Formulus syncs automatically + ↓ +Data is now on server +``` + +### Tips for Offline Work + +1. **Download forms before going offline** - Do this at the office +2. **Check you have battery** - Offline sync uses battery +3. **Keep phone storage available** - Each form needs storage space +4. **Sync when you can** - Try syncing at least daily + +:::warning Data Safety +Your data is safe on your phone until synced. To prevent data loss: +- Don't delete the Formulus app +- Don't clear app data +- Back up your phone regularly +- Sync at least once per day +::: + +## Common Tasks + +### Viewing Submitted Forms + +1. **Tap the "Submitted" tab** (or similar view in your app) +2. **See all your submissions** - Shows dates and status +3. **Tap a submission to view details** if needed + +### Editing a Draft + +If you saved a form but didn't submit: + +1. **Tap the "Draft" or "Saved" section** +2. **Tap the form to edit it** +3. **Make changes** +4. **Submit when ready** + +### Logging Out + +1. **Tap Settings** (gear icon) +2. **Tap Logout** +3. **Confirm** - You'll return to login screen + +:::note After Logging Out +You won't be able to see forms until you log back in. +::: + +### Updating Forms + +Your project manager may release new forms: + +1. **Open Formulus** +2. **Tap the refresh/sync icon** +3. **New forms will appear** - They'll be available to use + +## Troubleshooting Common Issues + +### "I can't connect to the server" + +**Causes:** +- No internet connection +- Wrong server URL +- Server is down + +**Fix:** +1. Check you have WiFi or mobile data +2. Ask your project manager for correct server URL +3. Try again in 5 minutes +4. Contact your project manager if still failing + +### "The app says my answer is invalid" + +**Causes:** +- Value is outside expected range (e.g., age > 150) +- Wrong format (e.g., text when number expected) +- Required field is empty + +**Fix:** +1. Check the error message +2. Review the field requirements +3. Enter the correct format +4. Try again + +### "I lost my saved form" + +**Causes:** +- App was force-closed +- Phone ran out of battery +- Storage space ran out + +**Prevent:** +1. Submit forms as soon as complete +2. Keep phone battery charged (> 50%) +3. Delete old submissions to free space +4. Sync regularly + +:::warning Data Recovery +Once submitted and synced, your data is safe on the server. Contact your project manager if data was lost before syncing. +::: + +### "Sync keeps failing" + +**Causes:** +- Poor internet connection +- Server temporarily unavailable +- Authentication expired + +**Fix:** +1. Check internet connection (try browsing web) +2. Wait a few minutes +3. Log out and log back in +4. Restart the app +5. Contact your project manager if still failing + +## Tips for Success + +### Before You Start +- 📋 **Read form instructions** - Each form may have specific guidance +- 📱 **Make sure you're online** - Download forms while connected +- 🔋 **Charge your phone** - Ensure good battery level + +### While Collecting +- ✏️ **Complete forms immediately** - Don't wait to fill them later +- 📸 **Use good lighting** - For photos (if required) +- 🔊 **Find quiet places** - If recording audio +- 🗺️ **Enable location** - If the form requests GPS data + +### Before Submitting +- ✅ **Review your answers** - Make sure everything is correct +- 🔴 **Check for required fields** - Fields with * must be filled +- 📍 **Verify dates are correct** - Easy to enter wrong date by mistake + +### After Submitting +- 🔄 **Sync as soon as possible** - When you have internet +- 📊 **Check if your data appears** - Ask your project manager to confirm +- 💾 **Keep backups** - Periodically back up your phone + +## Next Steps + +Congratulations! You've successfully submitted your first form. Now: + +1. **[Learn about the app features](/docs/using/formulus-features)** - Explore what Formulus can do +2. **[Understand working offline](/docs/using/working-offline)** - Maximize battery and storage +3. **[Get help with issues](/docs/using/troubleshooting)** - Find solutions to common problems +4. **[Check the FAQ](/docs/getting-started/faq)** - Common questions answered + +## Need Help? + +- **App issue?** → [Troubleshooting Guide](/docs/using/troubleshooting) +- **Question about syncing?** → [Syncing Data](/docs/using/synchronization) +- **Need community help?** → [Get Help](/docs/community/getting-help) +- **Contact your project manager** → They know your specific project setup + +--- + +:::tip Pro Tip +Most issues can be resolved by: +1. Checking your internet connection +2. Restarting the app +3. Syncing manually + +If problems persist, contact your project manager. +::: + +**Happy collecting!** 📱📊 diff --git a/docs/docs/collector/collector-index.md b/docs/docs/collector/collector-index.md new file mode 100644 index 000000000..2996b27fd --- /dev/null +++ b/docs/docs/collector/collector-index.md @@ -0,0 +1,196 @@ +--- +sidebar_position: 2 +title: For Data Collectors +--- + +# Data Collection with ODE + +Welcome to the **Data Collector** section! Whether you're a field researcher, community health worker, or survey enumerator, this guide covers everything you need to collect data using Formulus. + +## Who This Guide Is For + +This section is designed for **non-technical users** who: + +- Are collecting data in the field +- Use Formulus (the mobile app) +- Want to understand how to complete forms, sync data, and work offline +- Need troubleshooting help + +:::info Not designing forms? +If you're creating or customizing forms, see the [Implementer Guide](/docs/implementer/implementer-index). +If you're developing or extending ODE, see the [Developer Guide](/docs/developer/developer-index). +::: + +## Quick Start + +Get up and running in 5 minutes: + +1. **[Install Formulus](/docs/getting-started/installation/installing-formulus)** - Download and set up the app +2. **[Submit Your First Form](/docs/using/your-first-form)** - Get familiar with the interface +3. **[Sync Your Data](/docs/using/synchronization)** - Send data to the server + +## What You'll Learn + +
+
+
+
+

Getting Started

+
+
+

Install Formulus and set up your first project.

+ Get Started → +
+
+
+ +
+
+
+

Using the App

+
+
+

Learn form controls, data entry, and app features.

+ Learn More → +
+
+
+ +
+
+
+

Syncing Data

+
+
+

Upload forms and sync with the server.

+ Learn More → +
+
+
+ +
+
+
+

Troubleshooting

+
+
+

Fix common issues and get help.

+ Get Help → +
+
+
+ +
+
+
+

Offline Mode

+
+
+

Understand how Formulus works without internet.

+ Learn More → +
+
+
+ +
+
+
+

FAQ

+
+
+

Answers to common questions.

+ Read FAQ → +
+
+
+
+ +## Documentation Roadmap + +### Learn the Basics +- [Installation & Setup](/docs/getting-started/installation/installing-formulus) +- [Your First Submission](/docs/using/your-first-form) +- [Basic Form Navigation](/docs/using/formulus-features) + +### Practical Tasks +- [Collecting Different Data Types](/docs/using/formulus-features#form-field-types) +- [Managing Media (Photos, Audio)](/docs/using/formulus-features#form-field-types) +- [Working in Areas Without Internet](/docs/using/working-offline) + +### Data Management +- [Syncing Data to the Server](/docs/using/synchronization) +- [Understanding Sync Status](/docs/using/synchronization#sync-status) +- [Managing Local Storage](/docs/using/data-management) + +### Troubleshooting +- [Sync Issues](/docs/using/troubleshooting#synchronization-issues) +- [App Crashes or Freezes](/docs/using/troubleshooting#form-issues) +- [Device Not Connecting](/docs/using/troubleshooting#connection-issues) +- [Data Loss Prevention](/docs/using/troubleshooting#data-issues) + +## Key Concepts + +### What is Formulus? +Formulus is a mobile application available for Android and iOS. It's designed to: +- Work in areas with poor or no internet connectivity +- Securely store data on your device +- Automatically sync when you connect to the internet +- Display forms customized by your project manager + +### What is Sync? +Sync is the process of sending completed forms from your phone to the server. Your data is: +- Encrypted during transmission +- Stored securely on the server +- Available for your project manager to review and analyze + +### How Does Offline Work? +Formulus keeps a copy of all data on your phone. You can: +- Create new forms even without internet +- Edit and save responses locally +- Sync when you're back online +- See sync status in the app + +## Before You Start + +Make sure you have: + +- ✅ A smartphone (Android 8.0+ or iOS 13.0+) +- ✅ Access to internet (at least for initial setup) +- ✅ Instructions from your project manager (server URL, project code) +- ✅ Basic phone experience + +## Terminology + +| Term | Meaning | +|------|---------| +| **Form** | A survey or questionnaire you fill out | +| **Submission** | A completed form with your responses | +| **Sync** | Sending your data to the server | +| **Offline** | Working without internet connection | +| **Formulus** | The mobile app you use to collect data | +| **Synkronus** | The server that stores your data | + +## Getting Help + +If you get stuck: + +1. **Check the troubleshooting guide** → [Troubleshooting](/docs/using/troubleshooting) +2. **Search the FAQ** → [FAQ](/docs/getting-started/faq) +3. **Contact your project manager** → They can help with project-specific issues +4. **Reach out to the community** → [Get Help](/docs/community/getting-help) + +## Next Steps + +Ready to start collecting data? Let's go! + +→ **[Install Formulus](/docs/getting-started/installation/installing-formulus)** + +Or if you're already installed: + +→ **[Submit Your First Form](/docs/using/your-first-form)** + +--- + +:::note Questions? +For questions about designing forms or running a project, see the [Implementer Guide](/docs/implementer/implementer-index). +::: diff --git a/docs/docs/community/.gitkeep b/docs/docs/community/.gitkeep new file mode 100644 index 000000000..e69de29bb diff --git a/docs/docs/community/_category_.json b/docs/docs/community/_category_.json new file mode 100644 index 000000000..d41a22d47 --- /dev/null +++ b/docs/docs/community/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Community", + "position": 8 +} + diff --git a/docs/docs/community/contribute/about.md b/docs/docs/community/contribute/about.md new file mode 100644 index 000000000..61d93acd1 --- /dev/null +++ b/docs/docs/community/contribute/about.md @@ -0,0 +1,183 @@ +--- +sidebar_position: 1 +--- + +# About Contributions + +How to contribute to ODE. + +## Overview + +The OpenDataEnsemble (ODE) project welcomes contributions from everyone—whether you're interested in code, documentation, design, testing, or community building. We believe that diverse perspectives make our project stronger, and we're committed to creating an inclusive environment where contributors of all backgrounds and experience levels feel welcome. + +You don't need to be an expert to contribute. Many of our most valuable contributions come from people who: +- Found a bug while using ODE +- Wanted to improve our documentation +- Had an idea for a new feature +- Wanted to help other users + +## Ways to Contribute + +### 1. Report Bugs + +**Found an issue?** Help us improve ODE by reporting it: + +- Check [existing issues](https://github.com/OpenDataEnsemble/ode/issues) first to avoid duplicates +- Create a [new issue](https://github.com/OpenDataEnsemble/ode/issues/new) with: + - Clear title describing the bug + - Steps to reproduce the issue + - Expected behavior vs. actual behavior + - Your environment (OS, browser/app version, etc.) + - Screenshots or error messages if applicable + - Relevant logs or stack traces + +### 2. Suggest Features + +**Have an idea?** We'd love to hear it: + +- Check [existing issues](https://github.com/OpenDataEnsemble/ode/issues) and [GitHub Discussions](https://github.com/OpenDataEnsemble/ode/discussions) to prevent duplicates +- Start a discussion about your idea +- Describe the feature and why it would be useful +- Include examples of how you'd use it +- Wait for community feedback before implementing + +### 3. Improve Documentation + +**Love writing?** Documentation is critical to ODE's success: + +- Fix typos and grammar errors +- Clarify confusing sections +- Add missing examples or code snippets +- Create how-to guides for common tasks +- Translate documentation to other languages +- Add FAQs based on common questions + +Documentation lives in this monorepo under `docs/` (Markdown + Docusaurus). Content pages are in `docs/docs/`. + +### 4. Write Code + +**Ready to code?** We welcome code contributions: + +- **Bug fixes** - Fix reported issues +- **Features** - Implement new functionality +- **Refactoring** - Improve code quality +- **Tests** - Add missing test coverage +- **Performance** - Optimize bottlenecks +- **Dependencies** - Update libraries and frameworks + +ODE is built with: +- **Frontend:** TypeScript, React, React Native +- **Backend:** Go, PostgreSQL +- **DevOps:** Docker, GitHub Actions + +### 5. Review Code + +**Experienced with ODE?** Help review PRs: + +- Review pull requests from other contributors +- Test changes locally +- Provide constructive feedback +- Help document design decisions +- Verify that solutions address the root cause + +### 6. Participate in Community + +**Like helping others?** Build our community: + +- Answer questions in [GitHub Discussions](https://github.com/OpenDataEnsemble/ode/discussions) +- Help troubleshoot issues +- Share your ODE projects and use cases +- Contribute to design discussions +- Mentor new contributors +- Organize community events or webinars + +### 7. Improve Accessibility + +**Care about inclusion?** Help make ODE accessible: + +- Report accessibility issues +- Test with screen readers +- Improve color contrast +- Add missing alt-text to images +- Ensure keyboard navigation works +- Test with different browsers/devices + +## Contribution Process + +### Phase 1: Planning + +1. **Check existing work** - Browse [issues](https://github.com/OpenDataEnsemble/ode/issues) and [discussions](https://github.com/OpenDataEnsemble/ode/discussions) to understand scope +2. **Discuss first** - For major features, start a discussion to get feedback +3. **Claim the work** - Comment on the issue: "I'd like to work on this" +4. **Understand requirements** - Talk to maintainers about expectations + +### Phase 2: Development + +1. **Set up environment** - Follow setup guide for your component +2. **Create a branch** - Use `git checkout -b descriptive-branch-name` +3. **Make changes** - Write code or documentation +4. **Test thoroughly** - Run tests and verify your changes work +5. **Commit with clear messages** - Use conventional commit format +6. **Keep commits logical** - One feature/fix per commit + +For detailed instructions, see the [First Time Contributors](/docs/community/contribute/first-time) guide. + +### Phase 3: Pull Request + +1. **Create PR** - Push your branch and open a pull request on GitHub +2. **Fill out template** - Describe your changes clearly +3. **Link the issue** - Reference the issue you're fixing +4. **Wait for review** - Maintainers will review your code +5. **Respond to feedback** - Address comments and requests +6. **Iterate** - Push new commits to update your PR + +### Phase 4: Merge + +1. **Get approval** - Usually 1-2 approvals required +2. **Pass checks** - All automated tests must pass +3. **Merge!** - Maintainers will merge your PR +4. **Celebrate!** - Your code is now part of ODE! + +## Additional Resources + +### Getting Started +- [First Time Contributors Guide](/docs/community/contribute/first-time) - Step-by-step walkthrough +- [Architecture Overview](/docs/getting-started/architecture-overview) - Understand how ODE works +- [Development Setup](/docs/development/setup) - Set up your development environment + +### Development Guides +- [Formulus Development](/docs/development/formulus-development) - Mobile app +- [Formplayer Development](/docs/development/formplayer-development) - Web app +- [Synkronus Development](/docs/development/synkronus-development) - Backend server + +### Community +- [GitHub Issues](https://github.com/OpenDataEnsemble/ode/issues) - Bug reports and feature requests +- [GitHub Discussions](https://github.com/OpenDataEnsemble/ode/discussions) - Community discussions +- [Code of Conduct](/docs/community/contribute/code-of-conduct) - Our community standards + +## Recognition + +Contributors are the backbone of ODE. We recognize contributions by: + +- **Mentioning you in release notes** - Your work is highlighted in each release +- **Adding you to contributors list** - Your name appears in the README and documentation +- **Inviting you to join the team** - Major contributors may be invited to join maintainers +- **Sharing your work** - We highlight great contributions on social media and in announcements + +## Questions? + +Not sure where to start? Here are some options: + +1. **Read this page** - You might find your answer here +2. **Check existing discussions** - Search [GitHub Discussions](https://github.com/OpenDataEnsemble/ode/discussions) +3. **Start a new discussion** - Ask your question in GitHub Discussions +4. **Comment on an issue** - Ask questions on related issues +5. **Email maintainers** - Contact the project maintainers directly + +We're here to help and want to make contributing as easy as possible! + +## Related Content + +- [First Time Contributors](/docs/community/contribute/first-time) +- [Code of Conduct](/docs/community/contribute/code-of-conduct) + diff --git a/docs/docs/community/contribute/code-of-conduct.md b/docs/docs/community/contribute/code-of-conduct.md new file mode 100644 index 000000000..5b150daa6 --- /dev/null +++ b/docs/docs/community/contribute/code-of-conduct.md @@ -0,0 +1,139 @@ +--- +sidebar_position: 3 +--- + +# Code of Conduct + +Code of conduct for the ODE community. + +## Overview + +The OpenDataEnsemble (ODE) project is committed to building an inclusive, respectful, and harassment-free community. We welcome contributors from all backgrounds and experiences, and we are dedicated to providing a safe and supportive environment for everyone. + +This Code of Conduct applies to all community spaces, including: +- GitHub Issues and Pull Requests +- GitHub Discussions +- Code repositories +- Community meetings and events +- Official communications from maintainers + +All community members, contributors, and maintainers are expected to uphold this Code of Conduct. By participating in the ODE community, you agree to abide by these principles. + +## Our Standards + +### Examples of Behavior That Contribute to a Positive Environment + +- **Be respectful** - Treat all community members with courtesy and respect, regardless of their background, experience level, or viewpoint +- **Be inclusive** - Welcome newcomers warmly and help them feel comfortable participating +- **Listen actively** - Take time to understand others' perspectives before responding +- **Be constructive** - Provide helpful feedback and criticism that is focused on ideas, not individuals +- **Be patient** - Remember that contributors volunteer their time and may not respond immediately +- **Be honest** - Be truthful in your communications and acknowledge when you're wrong +- **Assume good intent** - Interpret others' words and actions positively unless clearly malicious +- **Give credit** - Acknowledge others' contributions and ideas +- **Stay on topic** - Keep discussions focused and relevant to the project + +### Examples of Unacceptable Behavior + +- **Harassment** - Any unwelcome comments or conduct based on protected characteristics (race, ethnicity, religion, gender, sexual orientation, disability, etc.) +- **Discrimination** - Treating individuals unfairly based on their identity or background +- **Intimidation** - Threatening or hostile behavior intended to make others feel unsafe +- **Bullying** - Repeated unwelcome behavior targeting specific individuals +- **Sexual harassment** - Unwelcome sexual advances, requests for sexual favors, or sexually suggestive comments +- **Trolling** - Deliberately provoking conflict or disrupting constructive discussions +- **Spam** - Repetitive, off-topic, or self-promotional content +- **Exclusion** - Deliberately excluding individuals from discussions or opportunities +- **Doxxing** - Sharing others' personal information without consent +- **Plagiarism** - Presenting others' work as your own +- **Abuse of power** - Maintainers or moderators using their position to silence or harm others + +## Expected Behavior + +### For All Community Members + +1. **Participate authentically** - Bring your genuine self and perspective to discussions +2. **Respect boundaries** - Honor others' privacy and personal limits +3. **Accept responsibility** - If you make a mistake, acknowledge it and learn from it +4. **Communicate clearly** - Be explicit and kind in your communication; avoid sarcasm that might be misunderstood +5. **Respect decisions** - Accept maintainers' decisions on project direction and issues + +### For Maintainers and Moderators + +1. **Model the behavior you want** - Demonstrate the standards outlined in this Code of Conduct +2. **Be transparent** - Explain decisions clearly and publicly when appropriate +3. **Show empathy** - Understand that most conflicts arise from misunderstandings +4. **Be fair** - Apply standards consistently across all community members +5. **Protect community** - Take swift action when someone violates the Code of Conduct + +## Reporting and Enforcement + +### Reporting Violations + +If you witness or experience a violation of this Code of Conduct, please report it by emailing: + +**conduct@opendata.ensemble.org** (or project maintainers' email) + +When reporting, please include: + +1. **Description** - What happened? +2. **When** - When did this occur? +3. **Where** - Which platform or space? (GitHub issue, discussion, email, etc.) +4. **Who** - Who was involved? (usernames or names) +5. **Evidence** - Links, screenshots, or other documentation +6. **Impact** - How did this affect you or others? +7. **Contact** - How should we reach you for follow-up? + +**All reports are confidential.** We will not share your identity without your permission, except as necessary to address the violation. + +### Investigation and Resolution + +When we receive a report, we will: + +1. **Acknowledge** - Send you confirmation that we received your report within 24 hours +2. **Investigate** - Gather information and talk to involved parties +3. **Determine** - Assess whether the Code of Conduct was violated +4. **Decide** - Determine appropriate consequences +5. **Communicate** - Inform you of the outcome (while respecting confidentiality) + +### Consequences + +Violations of the Code of Conduct may result in: + +- **Warning** - First-time minor violations may receive a warning +- **Temporary suspension** - Removal from community spaces for a set period +- **Permanent ban** - Serious or repeated violations result in permanent removal +- **Legal action** - In cases of illegal activity, we may report to appropriate authorities + +Consequences are applied consistently and proportionally to the violation. + +### Appeals + +If you disagree with a decision, you may appeal by: + +1. Sending a detailed written explanation to the maintainers +2. Clearly stating why you believe the decision was unfair +3. Providing new information that wasn't available in the original investigation + +Your appeal will be reviewed by a different group of maintainers to ensure fairness. + +## Attribution and References + +This Code of Conduct is adapted from: +- [Contributor Covenant](https://www.contributor-covenant.org/version/2/1/code_of_conduct/) +- [Mozilla Community Participation Guidelines](https://www.mozilla.org/en-US/about/governance/policies/participation/) +- [Django Code of Conduct](https://www.djangoproject.com/conduct/) +- [Kubernetes Code of Conduct](https://www.kubernetes.io/community/code-of-conduct/) + +## Questions? + +If you have questions about this Code of Conduct or how to report a violation, please contact the maintainers at: + +**conduct@opendata.ensemble.org** + +We're here to help create a welcoming community for everyone. + +## Related Content + +- [About Contributions](/docs/community/contribute/about) +- [First Time Contributors](/docs/community/contribute/first-time) + diff --git a/docs/docs/community/contribute/first-time.md b/docs/docs/community/contribute/first-time.md new file mode 100644 index 000000000..55c67e631 --- /dev/null +++ b/docs/docs/community/contribute/first-time.md @@ -0,0 +1,265 @@ +--- +sidebar_position: 2 +--- + +# First Time Contributors + +Guide for first-time contributors to ODE. + +## Overview + +Welcome! We're excited to have you contribute to the OpenDataEnsemble (ODE) project. Whether you want to report bugs, submit features, improve documentation, or contribute code, there's a place for you in our community. + +ODE is an open source data collection platform built for offline-first mobile applications. Our codebase is written in TypeScript, Go, and React, and we welcome contributors at all skill levels. You don't need to be an expert—we're here to help new contributors learn and grow. + +## Getting Started + +### 1. Fork and Clone the Repository + +Start by creating your own fork of the ODE repository on GitHub: + +```bash +# Clone your fork +git clone https://github.com/YOUR_USERNAME/ode.git +cd ode + +# Add upstream remote +git remote add upstream https://github.com/OpenDataEnsemble/ode.git +``` + +### 2. Set Up Your Development Environment + +Depending on which component you want to contribute to. + +ODE JavaScript packages use **pnpm** (not npm). Enable the pinned version once per machine: + +```bash +corepack enable +corepack prepare pnpm@10.33.2 --activate +``` + +Build `@ode/tokens` before Formulus or Formplayer. See [Development Setup](/docs/development/setup#package-manager-pnpm). + +**For mobile app (Formulus):** +```bash +cd packages/tokens && pnpm install && pnpm run build && cd ../.. +cd formulus +pnpm install +pnpm start +``` + +**For web app (Formplayer):** +```bash +cd packages/tokens && pnpm install && pnpm run build && cd ../.. +cd formulus-formplayer +pnpm install +pnpm start +``` + +**For backend server (Synkronus):** +```bash +cd synkronus +go mod download +go run ./cmd/synkronus +``` + +For detailed setup instructions, see the relevant component's development guide: +- [Formulus Development](/docs/development/formulus-development) +- [Formplayer Development](/docs/development/formplayer-development) +- [Synkronus Development](/docs/development/synkronus-development) + +### 3. Pick an Issue to Work On + +Start by browsing our [GitHub Issues](https://github.com/OpenDataEnsemble/ode/issues). Look for issues labeled: +- `good-first-issue` - Perfect for newcomers +- `help-wanted` - We're explicitly looking for community help +- `documentation` - Great for improving docs + +Comment on the issue to let maintainers know you'd like to work on it. This helps prevent duplicate efforts. + +### 4. Create a Feature Branch + +```bash +# Update from the latest default branch +git fetch upstream +git switch -c fix/bug-description upstream/dev +``` + +Use descriptive branch names that clearly indicate what you're working on. + +### 5. Make Your Changes + +Make your code changes, following these guidelines: + +**Code Style:** +- TypeScript/JavaScript: Follow ESLint configuration in the project +- Go: Use `gofmt` and `golint` +- React: Follow component patterns in existing code + +**Commits:** +```bash +# Make logical commits with clear messages +git commit -m "fix: resolve issue with sync conflict detection" +git commit -m "docs: add example for dynamic choice lists" +``` + +Use conventional commit format: +- `fix:` for bug fixes +- `feat:` for new features +- `docs:` for documentation changes +- `refactor:` for code restructuring +- `test:` for adding/updating tests + +### 6. Write or Update Tests + +If your change affects functionality, write tests: + +```bash +# Run tests for your component (from repo root) +cd formulus && pnpm run test --ci --coverage --watchAll=false +cd formulus-formplayer && pnpm run test run +cd synkronus && go test ./... +``` + +Aim for ~80% code coverage on new code. + +### 7. Update Documentation + +If you're adding a feature: +- Update relevant docs in `/docs` +- Add code examples if applicable +- Update API documentation if you changed endpoints + +See [Form Design Guide](/docs/guides/form-design) if adding form features, or [API Reference](/docs/reference/rest-api/overview) if adding endpoints. + +### 8. Submit a Pull Request + +```bash +# Push your branch to your fork +git push origin fix/bug-description +``` + +Go to GitHub and create a Pull Request with: + +**Title:** Clear, descriptive title +- Example: "fix: prevent sync conflicts when offline changes diverge" + +**Description:** Include: +- What problem does this solve? +- How does your solution work? +- Any relevant issue numbers (e.g., "Closes #123") +- Screenshots or examples if applicable + +**Template:** +```markdown +## Description +Briefly describe your changes + +## Related Issue +Closes # + +## Type of Change +- [ ] Bug fix +- [ ] New feature +- [ ] Documentation +- [ ] Breaking change + +## Testing +How did you test this? + +## Checklist +- [ ] Tests pass locally +- [ ] Documentation updated +- [ ] No breaking changes +``` + +### 9. Respond to Feedback + +Maintainers will review your PR. They may: +- Ask clarifying questions +- Request changes +- Suggest improvements + +This is normal and helpful! Push new commits to your branch to update the PR: + +```bash +# Make requested changes +git add . +git commit -m "refactor: address review feedback" +git push origin fix/bug-description +``` + +### 10. Celebrate! + +Once your PR is approved and merged, you're officially a contributor! Your code will be included in the next release. + +## Next Steps + +### After Your First Contribution + +**Now that you've contributed:** + +1. **Join our community** - Post on the [forum](https://forum.opendataensemble.org) to connect with other contributors +2. **Take on more issues** - Look for the next `good-first-issue` or a `help-wanted` item +3. **Consider becoming a reviewer** - Help review other contributors' PRs +4. **Contribute to discussions** - Help shape the future of ODE by participating in design discussions + +### Learning Resources + +**Understanding ODE Architecture:** +- [Architecture Overview](/docs/getting-started/architecture-overview) +- [Key Concepts](/docs/getting-started/key-concepts) +- [Data Synchronization](/docs/using/synchronization) + +**Technology-Specific Guides:** +- [TypeScript/React Development](/docs/development/contributing) +- [Go Backend Development](/docs/development/synkronus-development) +- [Mobile Development](/docs/development/formulus-development) + +**Contributing to Documentation:** +- Documentation lives in this monorepo under [`docs/`](https://github.com/OpenDataEnsemble/ode/tree/dev/docs) (Markdown + Docusaurus). Doc pages are in `docs/docs/` +- That site uses **npm** for its own tooling (`npm install`, `npm start` in the `docs` directory); ODE application code uses **pnpm** (see above) +- The site publishes automatically when a change to `docs/` is merged to `dev` +- Follow the same PR process for doc changes + +### Common Tasks for First-Time Contributors + +**Documentation:** +- Fix typos and grammar +- Add missing examples +- Clarify unclear sections +- Add FAQ entries + +**Code:** +- Fix `good-first-issue` bugs +- Add unit tests for untested code +- Improve error messages +- Optimize performance + +**Community:** +- Help answer questions in Discussions +- Improve setup guides +- Create how-to guides for common tasks +- Test release candidates + +## Getting Help + +**Stuck? Here's how to get help:** + +1. **Check existing documentation** - Many answers are in our docs +2. **Search GitHub Issues** - Your question might already be answered +3. **Ask on the forum** - Post on the [community forum](https://forum.opendataensemble.org) +4. **Ask in your PR** - Comment on your PR with questions +5. **Reach out to maintainers** - We're here to help! + +**Please be patient:** Maintainers volunteer their time. We'll respond as soon as we can, but it may take a few days. + +## Code of Conduct + +All contributors are expected to follow our [Code of Conduct](/docs/community/contribute/code-of-conduct). We're committed to providing a welcoming and respectful environment for everyone. + +## Related Content + +- [About Contributions](/docs/community/contribute/about) +- [Code of Conduct](/docs/community/contribute/code-of-conduct) + diff --git a/docs/docs/community/examples.md b/docs/docs/community/examples.md new file mode 100644 index 000000000..46819f4c5 --- /dev/null +++ b/docs/docs/community/examples.md @@ -0,0 +1,344 @@ +--- +sidebar_position: 2 +--- + +# Examples + +Example ODE applications, forms, and configurations to help you get started. + +## Form Examples + +### Basic Survey Form + +A simple form collecting name and age: + +```json +{ + "type": "object", + "properties": { + "name": { + "type": "string", + "title": "Full Name" + }, + "age": { + "type": "integer", + "title": "Age", + "minimum": 0, + "maximum": 120 + } + }, + "required": ["name", "age"] +} +``` + +### Health Survey Form + +Example health survey form collecting patient information: + +```json +{ + "type": "object", + "properties": { + "patientId": { + "type": "string", + "title": "Patient ID" + }, + "visitDate": { + "type": "string", + "format": "date", + "title": "Visit Date" + }, + "symptoms": { + "type": "array", + "title": "Symptoms", + "items": { + "type": "string", + "enum": ["fever", "cough", "headache", "fatigue"] + } + }, + "temperature": { + "type": "number", + "title": "Temperature (°C)", + "minimum": 35, + "maximum": 42 + }, + "notes": { + "type": "string", + "title": "Clinical Notes" + } + }, + "required": ["patientId", "visitDate"] +} +``` + +### Research Data Collection Form + +Example research form for field data collection: + +```json +{ + "type": "object", + "properties": { + "siteId": { + "type": "string", + "title": "Site ID" + }, + "location": { + "type": "string", + "format": "gps", + "title": "Site Location" + }, + "sitePhoto": { + "type": "object", + "format": "photo", + "title": "Site Photo" + }, + "observations": { + "type": "string", + "title": "Field Observations" + }, + "timestamp": { + "type": "string", + "format": "date-time", + "title": "Observation Time" + } + }, + "required": ["siteId", "location"] +} +``` + +## Custom Application Examples + +### Basic Custom App + +Simple custom application that lists available forms: + +```html + + + + Form Selector + + + + +

Available Forms

+
    + + + + +``` + +### Advanced Custom App + +Advanced custom applications can include complex workflows, custom navigation, and integration with external systems. Examples of advanced patterns will be added as they become available. For now, see the [Custom Applications guide](/guides/custom-applications) for detailed implementation guidance. + +## Configuration Examples + +### Server Configuration + +Example server configuration for development: + +```bash +PORT=8080 +DB_CONNECTION=postgres://synkronus:password@localhost:5432/synkronus?sslmode=disable +JWT_SECRET=your-secret-key-change-this-in-production +LOG_LEVEL=debug +APP_BUNDLE_PATH=./data/app-bundles +MAX_VERSIONS_KEPT=5 +``` + +### Docker Compose Configuration + +Example Docker Compose configuration for production: + +```yaml +version: '3.8' + +services: + postgres: + image: postgres:15 + environment: + POSTGRES_PASSWORD: ${DB_ROOT_PASSWORD} + volumes: + - postgres-data:/var/lib/postgresql/data + + synkronus: + image: ghcr.io/opendataensemble/synkronus:latest + environment: + DB_CONNECTION: postgres://synkronus:${DB_PASSWORD}@postgres:5432/synkronus?sslmode=disable + JWT_SECRET: ${JWT_SECRET} + LOG_LEVEL: info + depends_on: + - postgres + volumes: + - app-bundles:/app/data/app-bundles + + nginx: + image: nginx:alpine + ports: + - "80:80" + volumes: + - ./nginx.conf:/etc/nginx/nginx.conf + depends_on: + - synkronus + +volumes: + postgres-data: + app-bundles: +``` + +## Integration Examples + +### API Integration + +Example API integration using different methods: + + + + +```bash +# Authenticate +TOKEN=$(curl -X POST http://your-server:8080/auth/login \ + -H "Content-Type: application/json" \ + -d '{"username":"user","password":"pass"}' \ + | jq -r '.token') + +# Pull data +curl -X POST http://your-server:8080/sync/pull \ + -H "Authorization: Bearer $TOKEN" \ + -H "Content-Type: application/json" \ + -d '{"client_id":"my-client","since_change_id":0}' + +# Push data +curl -X POST http://your-server:8080/sync/push \ + -H "Authorization: Bearer $TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "transmission_id": "550e8400-e29b-41d4-a716-446655440000", + "client_id": "my-client", + "records": [...] + }' +``` + + + + +```bash +# Login (stores token automatically) +synk login --username user + +# Pull data +synk sync pull data.json --client-id my-client + +# Push data +synk sync push data.json +``` + + + + +```python +import requests + +# Authenticate +response = requests.post( + 'http://your-server:8080/auth/login', + json={'username': 'user', 'password': 'pass'} +) +token = response.json()['token'] + +# Pull data +response = requests.post( + 'http://your-server:8080/sync/pull', + headers={'Authorization': f'Bearer {token}'}, + json={'client_id': 'my-client', 'since_change_id': 0} +) +data = response.json() +``` + + + + +### Data Export Examples + +Export observations using different methods: + + + + +```bash +# Export all observations +synk data export observations.zip + +# Export to specific directory +synk data export ./backups/observations_$(date +%Y%m%d).zip + +# Export to JSON +synk data export observations.json --format json +``` + + + + +```bash +# Export to Parquet ZIP +curl -X GET http://your-server:8080/api/dataexport/parquet \ + -H "Authorization: Bearer YOUR_TOKEN" \ + -o observations.zip +``` + + + + +1. Navigate to the Portal +2. Go to "Data Export" +3. Select format (Parquet, JSON, CSV) +4. Click "Export" to download + + + + +The exported ZIP contains Parquet files organized by schema type, suitable for analysis in tools like Python pandas, R, or data analysis platforms. + +## Community Projects + +Projects built by the ODE community will be showcased here as they become available. + +## Contributing Examples + +If you have examples you'd like to share, please: + +1. Create a pull request with your example +2. Include clear documentation +3. Follow the existing example format +4. Ensure examples are tested and working + +## Related Resources + +- [Form Design Guide](/guides/form-design) +- [Custom Applications Guide](/guides/custom-applications) +- [API Reference](/reference/api) +- [Getting Help](/community/getting-help) diff --git a/docs/docs/community/getting-help.md b/docs/docs/community/getting-help.md new file mode 100644 index 000000000..20f08b9c6 --- /dev/null +++ b/docs/docs/community/getting-help.md @@ -0,0 +1,74 @@ +--- +sidebar_position: 1 +--- + +# Getting Help + +Resources and channels for getting help with ODE. + +## Support Channels + +### Documentation + +The first place to look for help is this documentation site. It covers: + +- Installation and setup guides +- Usage instructions +- API reference +- Troubleshooting guides +- FAQ section + +### GitHub + +- **Issues**: Report bugs, request features, or ask questions on [GitHub Issues](https://github.com/OpenDataEnsemble/ode/issues) +- **Discussions**: Join discussions on [GitHub Discussions](https://github.com/OpenDataEnsemble/ode/discussions) +- **Repository**: Browse the source code at [GitHub Repository](https://github.com/OpenDataEnsemble/ode) + +### Email + +For general inquiries or to get involved: + +- **Email**: [hello@opendataensemble.org](mailto:hello@opendataensemble.org) + +## Before Asking for Help + +To get the most effective help, please: + +1. **Search existing resources**: Check the documentation, FAQ, and existing GitHub issues +2. **Provide context**: Include relevant information about your setup, error messages, and what you've tried +3. **Be specific**: Describe the problem clearly and include steps to reproduce if applicable + +## Common Issues + +Many common issues are covered in the [Troubleshooting guide](/using/troubleshooting). Check there first for: + +- Connection issues +- Synchronization problems +- Form-related errors +- Data management issues +- Performance problems + +## Reporting Issues + +When reporting issues on GitHub, please include: + +- **Description**: Clear description of the problem +- **Steps to reproduce**: If applicable, steps to reproduce the issue +- **Expected behavior**: What you expected to happen +- **Actual behavior**: What actually happened +- **Environment**: Operating system, ODE version, component versions +- **Error messages**: Any error messages or logs +- **Screenshots**: If applicable, screenshots showing the issue + +See [Reporting Issues on GitHub](https://github.com/OpenDataEnsemble/ode/issues/new) to create a new issue. + +## Contributing + +If you'd like to contribute to ODE, see the [Contributing guide](/development/contributing) for information on how to get started. + +## Related Resources + +- [FAQ](/getting-started/faq) +- [Troubleshooting Guide](/using/troubleshooting) +- [GitHub Issues](https://github.com/OpenDataEnsemble/ode/issues) +- [Examples](/community/examples) diff --git a/docs/docs/community/index.md b/docs/docs/community/index.md new file mode 100644 index 000000000..e59036366 --- /dev/null +++ b/docs/docs/community/index.md @@ -0,0 +1,40 @@ +--- +sidebar_position: 0 +--- + +# Community + +Connect with the ODE community, get help, and share your experiences. + +## Get Help & Support + +
    +
    +
    +
    +

    Getting Help

    +
    +
    +

    Find resources and channels to get help with ODE.

    + Get Help → +
    +
    +
    +
    +
    +
    +

    Examples

    +
    +
    +

    Explore example projects, use cases, and implementations.

    + View Examples → +
    +
    +
    +
    + +## Join the Community + +- **Forum**: [forum.opendataensemble.org](https://forum.opendataensemble.org) - Ask questions and share knowledge +- **GitHub**: [github.com/OpenDataEnsemble/ode](https://github.com/OpenDataEnsemble/ode) - Contribute code and report issues +- **Email**: [hello@opendataensemble.org](mailto:hello@opendataensemble.org) - Contact the team directly diff --git a/docs/docs/community/resources/examples.md b/docs/docs/community/resources/examples.md new file mode 100644 index 000000000..70d700681 --- /dev/null +++ b/docs/docs/community/resources/examples.md @@ -0,0 +1,25 @@ +--- +sidebar_position: 1 +--- + +# Examples + +Example ODE applications and configurations. + +## Overview + +[Description placeholder] + +## Example Forms + +[Description placeholder] + +## Example Applications + +[Description placeholder] + +## Related Content + +- [Community Projects](/docs/community/resources/projects) +- [Tutorials](/guides/quick-start-custom-app) + diff --git a/docs/docs/community/resources/projects.md b/docs/docs/community/resources/projects.md new file mode 100644 index 000000000..e87494410 --- /dev/null +++ b/docs/docs/community/resources/projects.md @@ -0,0 +1,25 @@ +--- +sidebar_position: 2 +--- + +# Community Projects + +Projects built by the ODE community. + +## Overview + +[Description placeholder] + +## Featured Projects + +[Description placeholder] + +## Submit Your Project + +[Description placeholder] + +## Related Content + +- [Examples](/docs/community/resources/examples) +- [Contribute](/docs/community/contribute/about) + diff --git a/docs/docs/community/support/getting-help.md b/docs/docs/community/support/getting-help.md new file mode 100644 index 000000000..4250fcaa3 --- /dev/null +++ b/docs/docs/community/support/getting-help.md @@ -0,0 +1,25 @@ +--- +sidebar_position: 1 +--- + +# Getting Help + +Where to get help with ODE. + +## Overview + +[Description placeholder] + +## Support Channels + +[Description placeholder] + +## Before Asking + +[Description placeholder] + +## Related Content + +- [Reporting Issues](/docs/community/support/reporting-issues) +- [FAQ](/getting-started/faq) + diff --git a/docs/docs/community/support/reporting-issues.md b/docs/docs/community/support/reporting-issues.md new file mode 100644 index 000000000..f6e32fb48 --- /dev/null +++ b/docs/docs/community/support/reporting-issues.md @@ -0,0 +1,29 @@ +--- +sidebar_position: 2 +--- + +# Reporting Issues + +How to report bugs and issues. + +## Overview + +[Description placeholder] + +## Issue Reporting Process + +[Description placeholder] + +## Bug Reports + +[Description placeholder] + +## Feature Requests + +[Description placeholder] + +## Related Content + +- [Getting Help](/docs/community/support/getting-help) +- [GitHub Issues](https://github.com/OpenDataEnsemble/ode/issues) + diff --git a/docs/docs/developer/developer-getting-started.md b/docs/docs/developer/developer-getting-started.md new file mode 100644 index 000000000..4f09e3648 --- /dev/null +++ b/docs/docs/developer/developer-getting-started.md @@ -0,0 +1,408 @@ +--- +sidebar_position: 1 +title: Getting Started with ODE Development +--- + +# Getting Started: ODE Development + +Welcome to the ODE development guide! This page will help you get started contributing to or extending ODE. + +## Choose Your Path + +### Path 1: Contributing to ODE Core + +You want to help improve ODE itself (Formulus, Synkronus, Formplayer, or CLI). + +**Requirements:** +- Comfortable with Git and GitHub +- Familiar with your tech stack (React Native, Go, React, TypeScript) +- Willing to follow project standards + +**Time to first contribution:** 2-4 hours + +**Next Steps:** +1. [Set up your environment](/docs/development/setup) +2. [Read architecture overview](/docs/development/architecture) +3. [Choose a component](/docs/development) +4. [Follow contributing guide](/docs/development/contributing) + +### Path 2: Building Custom Applications + +You want to build custom data collection apps using ODE APIs. + +**Requirements:** +- Knowledge of REST APIs +- Understanding of authentication (JWT) +- Ability to build web/mobile applications + +**Time to first app:** 4-8 hours + +**Next Steps:** +1. [Understand ODE architecture](/docs/development/architecture) +2. [Learn the REST API](/docs/reference/rest-api/overview) +3. [Review API examples](/docs/reference/rest-api/authentication) +4. [Read extending guide](/docs/development/extending) + +### Path 3: System Administration & Deployment + +You want to deploy and manage ODE in your infrastructure. + +**Requirements:** +- Linux/Docker experience +- Understanding of databases (PostgreSQL) +- Basic networking knowledge +- Experience with deployment platforms (cloud or on-premise) + +**Time to first deployment:** 1-2 hours + +**Next Steps:** +1. [Server Architecture for IT](/docs/guides/server-architecture-for-it) +2. [Understand system architecture](/docs/development/architecture) +3. [Learn server configuration](/docs/reference/configuration/server) +4. [Follow deployment guide](/docs/guides/deployment) +5. [Security reference](/docs/reference/security) + +### Path 4: Integration & APIs + +You want to integrate ODE with external systems (database, analytics, etc.). + +**Requirements:** +- Experience with API integrations +- Understanding of data formats (JSON, Parquet, CSV) +- Knowledge of your target system + +**Time to first integration:** 2-4 hours + +**Next Steps:** +1. [Learn REST API](/docs/reference/rest-api/overview) +2. [Understand data formats](/docs/reference/app-bundle-format) +3. [Review sync protocol](/docs/reference/rest-api/sync) +4. [Build custom integrations](/docs/development/extending) + +## Quick Prerequisites + +Before diving in, make sure you have: + +### For Core Development + +**macOS:** +```bash +# Install Homebrew if not present +/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" + +# Install dependencies +brew install node go postgresql git +``` + +**Linux (Ubuntu/Debian):** +```bash +sudo apt-get update +sudo apt-get install nodejs npm golang postgresql git +# Enable pnpm for the ODE monorepo (pinned in package.json files): +# corepack enable && corepack prepare pnpm@10.33.2 --activate +``` + +**Windows:** +- [Node.js LTS](https://nodejs.org/) +- [Go 1.22+](https://golang.org/doc/install) +- [PostgreSQL](https://www.postgresql.org/download/) +- [Git](https://git-scm.com/) + +### For All Developers + +- Git account & credentials configured +- GitHub account (for issues, PRs, discussions) +- Code editor (VS Code recommended) +- Terminal/command line experience + +## 15-Minute Quick Start + +### 1. Clone the Repository + +```bash +git clone https://github.com/OpenDataEnsemble/ode.git +cd ode +``` + +### 2. Explore the Structure + +```bash +# See the main components +ls -la + +# You'll see: +# formulus/ - React Native app +# formulus-formplayer/ - React form renderer +# synkronus/ - Go server +# docs/ - Documentation +``` + +### 3. Choose a Component to Explore + +#### Option A: Explore Backend (Go) + +```bash +cd synkronus +cat README.md # Read the overview +ls -la cmd/synkronus/ # See the entry point +``` + +#### Option B: Explore Frontend (React/React Native) + +```bash +cd formulus +cat README.md # Read the overview +cat package.json | grep scripts # See build commands +``` + +#### Option C: Explore Form Renderer + +```bash +cd formulus-formplayer +cat README.md # Read the overview +cat package.json # See scripts +``` + +### 4. Understand the Architecture + +Read the [Architecture Overview](/docs/development/architecture) - it explains: +- How components communicate +- The sync protocol +- Authentication flow +- Data models + +### 5. Set Up Development Environment + +Follow the detailed [Environment Setup](/docs/development/setup) guide for your component. + +## Understanding ODE's Tech Stack + +### Frontend Components + +| Component | Language | Framework | Purpose | +|-----------|----------|-----------|---------| +| **Formulus** | TypeScript | React Native | Mobile app (Android/iOS) | +| **Formplayer** | TypeScript | React | Form rendering in WebView | + +**Frontend Skills Needed:** +- TypeScript/JavaScript +- React or React Native +- State management (Redux/MobX) +- REST API integration + +### Backend Components + +| Component | Language | Purpose | +|-----------|----------|---------| +| **Synkronus** | Go | Server, API, sync protocol | +| **Synkronus CLI** | Go | Command-line administration | +| **Synkronus Portal** | TypeScript/React | Web-based admin dashboard | + +**Backend Skills Needed:** +- Go (for server development) +- REST API design +- Database design (PostgreSQL) +- Authentication/security + +### Databases + +| System | Purpose | Skill Level | +|--------|---------|------------| +| **PostgreSQL** | Server data storage | Intermediate | +| **WatermelonDB** | Mobile local storage | Beginner | + +## Common Development Tasks + +### Task 1: Add a New Form Control + +**Skills:** React/TypeScript +**Time:** 2-3 hours +**Impact:** Enable new data types for form designers + +→ [Extending ODE - Form Controls](/docs/development/extending#form-controls) + +### Task 2: Fix a Bug in Formulus + +**Skills:** React Native, TypeScript +**Time:** 1-2 hours +**Impact:** Improve stability for field workers + +→ [Formulus Development](/docs/development/formulus-development) + +### Task 3: Add a New API Endpoint + +**Skills:** Go, REST APIs, PostgreSQL +**Time:** 2-4 hours +**Impact:** Enable custom integrations + +→ [Synkronus Development](/docs/development/synkronus-development) + +### Task 4: Improve Documentation + +**Skills:** Technical writing +**Time:** 1-2 hours +**Impact:** Help other developers + +→ [Contributing - Documentation](/docs/development/contributing#documentation) + +### Task 5: Set Up Local Development + +**Skills:** Docker, Docker Compose, or manual setup +**Time:** 30-60 minutes +**Impact:** Run ODE locally for testing + +→ [Setup Environment](/docs/development/setup) + +## Development Workflow + +### Step 1: Plan Your Work + +1. **Check existing issues** - Is this already being worked on? +2. **Open an issue** - Describe what you want to do +3. **Discuss approach** - Get feedback from maintainers +4. **Get approval** - Make sure the direction is right + +### Step 2: Set Up Your Environment + +1. **Clone the repo** +2. **Install dependencies** +3. **Run tests** to verify setup +4. **Create a feature branch** + +### Step 3: Make Your Changes + +1. **Write code** following project standards +2. **Run tests** to check your work +3. **Lint your code** (ESLint, Go fmt) +4. **Test manually** if needed + +### Step 4: Submit Your Work + +1. **Commit with clear messages** +2. **Push to your fork** +3. **Create a pull request** with description +4. **Address feedback** from reviewers +5. **Celebrate when merged** + +## Code Quality Standards + +ODE maintains high standards: + +### JavaScript/TypeScript +```bash +# Run linter +pnpm run lint + +# Auto-fix issues +pnpm run lint:fix + +# Format code +pnpm run format +``` + +### Go +```bash +# Format code +go fmt ./... + +# Run tests +go test ./... + +# Check coverage +go test -cover ./... +``` + +### Testing +- Write tests for new features +- Maintain test coverage > 80% +- Run full test suite before submitting PR + +### Git Commits +- Use meaningful messages +- Reference issues (`Fixes #123`) +- Keep commits focused +- Squash related commits + +Example: +``` +git commit -m "Add GPS field control for forms (Fixes #456)" +``` + +## Learning Resources + +### ODE Specific +- [Architecture Deep Dive](/docs/development/architecture) +- [Component Documentation](/docs/development) +- [Contributing Guide](/docs/development/contributing) +- [API Reference](/docs/reference/rest-api/overview) + +### React Native +- [Official Docs](https://reactnative.dev/) +- [React Native Community](https://github.com/react-native-community) + +### Go +- [Official Docs](https://golang.org/doc/) +- [Go by Example](https://gobyexample.com/) + +### React +- [Official Docs](https://react.dev/) +- [Learn React](https://react.dev/learn) + +### PostgreSQL +- [Official Docs](https://www.postgresql.org/docs/) +- [SQL Tutorial](https://sqlzoo.net/) + +## Getting Help + +**Stuck on something?** + +1. **Check the docs** - Most answers are in [Architecture](/docs/development/architecture) or [Component Guides](/docs/development) +2. **Search existing issues** - Your question might be answered +3. **Ask in discussions** - [GitHub Discussions](https://github.com/OpenDataEnsemble/ode/discussions) +4. **Contact maintainers** - hello@opendataensemble.org + +## What to Expect + +### First Week +- ✅ Understand project structure +- ✅ Set up development environment +- ✅ Make first small contribution (docs, tiny fix) +- ✅ Get familiar with contribution process + +### First Month +- ✅ Make several contributions +- ✅ Understand a component deeply +- ✅ Have code merged to main +- ✅ Help answer other developers' questions + +### First Quarter +- ✅ Be recognized as contributor +- ✅ Have meaningful PRs merged +- ✅ Potentially become code reviewer +- ✅ Mentor new contributors + +## Next Steps + +### Ready to Code? + +Choose your path and dive in: + +1. **[Core Contributor Path](/docs/development/setup)** - Set up environment +2. **[Custom App Path](/docs/reference/rest-api/overview)** - Learn the APIs +3. **[Deployment Path](/docs/guides/server-architecture-for-it)** - IT overview, then [deployment guide](/docs/guides/deployment) +4. **[Integration Path](/docs/development/extending)** - Build integrations + +### Read More + +- [Architecture Overview](/docs/development/architecture) +- [Component Development](/docs/reference/components) +- [Contributing Guide](/docs/development/contributing) +- [Community Support](/docs/community/getting-help) + +--- + +:::tip Welcome to ODE Development! +We're excited to have you contribute. Don't hesitate to ask questions—the community is friendly and helpful. + +→ **[Set Up Your Environment](/docs/development/setup)** +::: diff --git a/docs/docs/developer/developer-index.md b/docs/docs/developer/developer-index.md new file mode 100644 index 000000000..fe8ed6ae7 --- /dev/null +++ b/docs/docs/developer/developer-index.md @@ -0,0 +1,315 @@ +--- +sidebar_position: 4 +title: ⚙️ For Developers +--- + +# Development & Engineering + +Welcome to the **Developer** section! Whether you're contributing to ODE, building custom extensions, or deploying the platform, this guide covers development, architecture, and contribution workflows. + +:::note Documentation paths +Persona pages live under `/docs/developer/`; technical guides under `/docs/development/`. Deploying for production? Start with [Server Architecture for IT](/docs/guides/server-architecture-for-it). +::: + +## Who This Guide Is For + +This section is designed for: + +- Software engineers contributing to ODE +- Developers building custom applications with ODE +- System administrators deploying ODE +- Architects evaluating ODE architecture +- Contributors wanting to improve the codebase + +:::tip AI coding assistants +Building **custom applications** (HTML, JS, CSS bundles and JSON forms) without cloning the ODE monorepo? Use the **[custom_app](https://github.com/OpenDataEnsemble/custom_app)** repository on GitHub (`AGENTS.md` and `CONTEXT_*.md` for assistants and authors), together with the main **[documentation](https://opendataensemble.org/docs/)** site. +::: + +:::info Not developing? +If you're collecting data, see the [Data Collector Guide](/docs/collector/collector-index). +If you're designing forms, see the [Implementer Guide](/docs/implementer/implementer-index). +::: + +## Quick Start + +Set up your development environment in 20 minutes: + +1. **[Getting Started](/docs/developer/developer-getting-started)** - Choose your path +2. **[Architecture Overview](/docs/development/architecture)** - Understand the system +3. **[Set Up Environment](/docs/development/setup)** - Install dependencies +4. **[Run Tests](/docs/development/building-testing)** - Verify your setup + +## What You'll Learn + +
    +
    +
    +
    +

    🚀 Getting Started

    +
    +
    +

    Choose your development path and get started.

    + Start → +
    +
    +
    + +
    +
    +
    +

    🏗️ Architecture

    +
    +
    +

    Deep dive into ODE system design and components.

    + Learn More → +
    +
    +
    + +
    +
    +
    +

    🔧 Setup Environment

    +
    +
    +

    Configure your dev machine and install dependencies.

    + Setup → +
    +
    +
    + +
    +
    +
    +

    🏗️ Build & Test

    +
    +
    +

    Build projects and run test suites.

    + Learn More → +
    +
    +
    + +
    +
    +
    +

    🤝 Contributing

    +
    +
    +

    How to contribute to ODE development.

    + Contribute → +
    +
    +
    + +
    +
    +
    +

    ⚡ Extending ODE

    +
    +
    +

    Build custom extensions and integrations.

    + Learn More → +
    +
    +
    +
    + +## Documentation Roadmap + +### 🚀 Get Started +- [Development Paths](/docs/developer/developer-getting-started#paths) +- [Prerequisites & Requirements](/docs/developer/developer-getting-started#prerequisites) +- [Quick Start Guide](/docs/developer/developer-getting-started#quick-start) + +### 🏗️ Understand Architecture +- [System Overview](/docs/development/architecture) +- [Component Architecture](/docs/development/architecture#components) +- [Data Flow & Sync Protocol](/docs/development/architecture#data-flow) +- [Authentication & Security](/docs/reference/security) + +### 🔧 Set Up Environment +- [macOS Setup](/docs/development/setup#macos) +- [Linux Setup](/docs/development/setup#linux) +- [Windows Setup](/docs/development/setup#windows) +- [Docker Setup](/docs/development/setup#docker) + +### 📚 Component Development +- [Formulus (React Native)](/docs/development/formulus-development) +- [Synkronus Server (Go)](/docs/development/synkronus-development) +- [Formplayer (React)](/docs/development/formplayer-development) +- [Synkronus CLI (Go)](/docs/development/setup) + +### 🏗️ Building & Testing +- [Building Projects](/docs/development/building-testing#building) +- [Running Tests](/docs/development/building-testing#testing) +- [Code Quality & Linting](/docs/development/building-testing#quality) +- [CI/CD Pipeline](/docs/development/building-testing#ci-cd) + +### 🤝 Contributing +- [Contributing Workflow](/docs/development/contributing#workflow) +- [Code Standards](/docs/development/contributing#standards) +- [Commit Messages](/docs/development/contributing#commits) +- [Pull Request Process](/docs/development/contributing#pull-requests) +- [Code of Conduct](/docs/community/contribute/code-of-conduct) + +### ⚡ Extending ODE +- [Custom Applications](/docs/development/extending#custom-apps) +- [Custom Form Controls](/docs/development/extending#form-controls) +- [Server Plugins](/docs/development/extending#plugins) +- [Integration Patterns](/docs/development/extending#integrations) + +### 📖 API Reference +- [REST API Overview](/docs/reference/rest-api/overview) +- [Authentication](/docs/reference/rest-api/authentication) +- [Sync Protocol](/docs/reference/rest-api/sync) +- [App Bundle Format](/docs/reference/app-bundle-format) +- [Attachments](/docs/reference/rest-api/attachments) + +## Development Paths + +### Path 1: Contributing to ODE Core + +You want to help improve Formulus, Synkronus, or Formplayer. + +1. [Clone the monorepo](https://github.com/OpenDataEnsemble/ode) +2. [Set up environment](/docs/development/setup) +3. [Choose a component](/docs/development) +4. [Follow contributing guide](/docs/development/contributing) + +→ **Best for:** Passionate developers improving the platform + +### Path 2: Building Custom Applications + +You want to build custom data collection applications on ODE. + +1. [Understand architecture](/docs/development/architecture) +2. [Learn the REST API](/docs/reference/rest-api/overview) +3. [Set up development environment](/docs/development/setup) +4. [Follow extending guide](/docs/development/extending) + +→ **Best for:** Building organization-specific solutions + +### Path 3: System Administration & Deployment + +You want to deploy and manage ODE in your infrastructure. + +1. [Understand system architecture](/docs/development/architecture) +2. [Read deployment guide](/docs/guides/deployment) +3. [Learn configuration options](/docs/reference/configuration/server) +4. [Set up monitoring](/docs/development/setup#monitoring) + +→ **Best for:** SysAdmins and DevOps engineers + +### Path 4: Integration & APIs + +You want to integrate ODE with external systems. + +1. [Learn REST API](/docs/reference/rest-api/overview) +2. [Understand app bundle format](/docs/reference/app-bundle-format) +3. [Review sync protocol](/docs/reference/rest-api/sync) +4. [Build custom integrations](/docs/development/extending#integrations) + +→ **Best for:** Backend developers and integrations engineers + +## Tech Stack Overview + +### Formulus (Mobile App) +- **Language:** TypeScript/JavaScript +- **Framework:** React Native +- **Database:** WatermelonDB +- **Platforms:** Android, iOS + +### Synkronus (Server) +- **Language:** Go 1.22+ +- **Database:** PostgreSQL +- **API:** REST + OpenAPI +- **Deployment:** Docker, Kubernetes + +### Formplayer (Form Renderer) +- **Language:** TypeScript/JavaScript +- **Framework:** React +- **Rendering:** JSON Forms +- **Execution:** WebView (Formulus) or Browser + +### Synkronus CLI +- **Language:** Go +- **Distribution:** Prebuilt binaries +- **Package Manager:** Homebrew, curl + +## Key Architecture Concepts + +### Offline-First Design +``` +Device Server +├─ Local DB ├─ PostgreSQL +│ (WatermelonDB) │ +├─ Forms Cache ├─ REST API +└─ Sync Queue ├─ Sync Engine + (offline buffer) └─ Authentication +``` + +### Client-Server Sync +``` +1. Pull: Client requests new forms from server +2. Push: Client sends completed submissions +3. Merge: Server resolves conflicts +4. Acknowledge: Client marks submissions as synced +``` + +### Authentication & Security +- JWT-based authentication +- Role-based access control (RBAC) +- End-to-end encryption options +- Secure offline token storage + +## Getting Help + +- **Architecture questions?** → [Read Architecture Guide](/docs/development/architecture) +- **Setup issues?** → [Setup Environment](/docs/development/setup) +- **Contributing questions?** → [Contributing Guide](/docs/development/contributing) +- **API documentation?** → [REST API](/docs/reference/rest-api/overview) +- **Need community help?** → [Community Support](/docs/community/getting-help) + +## Contribution Ideas + +Here are some ways to contribute: + +| Contribution | Difficulty | Impact | +|--------------|------------|--------| +| Fix typos in docs | Easy | High | +| Report bugs | Easy | High | +| Fix UI bugs | Medium | High | +| Add tests | Medium | High | +| Implement features | Medium-Hard | High | +| Add new form control | Hard | High | +| Database optimization | Hard | Medium | +| Mobile performance | Hard | High | + +## Code Quality Standards + +ODE maintains high code quality: + +- ✅ ESLint + Prettier for JavaScript/TypeScript +- ✅ Go fmt for Go code +- ✅ Comprehensive test coverage +- ✅ Pre-commit hooks +- ✅ CI/CD validation on all PRs + +## Next Steps + +Ready to contribute or develop? + +→ **[Getting Started](/docs/developer/developer-getting-started)** + +Or dive into a specific area: + +→ **[Architecture Deep Dive](/docs/development/architecture)** +→ **[Set Up Environment](/docs/development/setup)** +→ **[Contributing Guide](/docs/development/contributing)** + +--- + +:::tip Welcome to ODE! +We're excited to have you contribute to ODE. Whether it's code, documentation, bug reports, or ideas—all contributions are valued! +::: diff --git a/docs/docs/development/.gitkeep b/docs/docs/development/.gitkeep new file mode 100644 index 000000000..e69de29bb diff --git a/docs/docs/development/architecture.md b/docs/docs/development/architecture.md new file mode 100644 index 000000000..5fdf2d044 --- /dev/null +++ b/docs/docs/development/architecture.md @@ -0,0 +1,273 @@ +--- +sidebar_position: 2 +--- + +# Architecture + +Complete architecture documentation for ODE, including sync protocol, database design, and extension points. + +:::tip Audience +**Hosting / IT?** See [Server Architecture for IT](/docs/guides/server-architecture-for-it). +**Product overview?** See [Architecture Overview](/docs/getting-started/architecture-overview). +::: + +## System Overview + +ODE follows a client-server architecture designed for offline-first data collection. High-level component relationships are documented in [Architecture Overview](/docs/getting-started/architecture-overview). Server installation layout is in [Server Architecture for IT](/docs/guides/server-architecture-for-it). + +## Components + +### Formulus + +React Native mobile application providing: + +- **Local Storage**: WatermelonDB for offline data storage +- **Sync Module**: Handles synchronization with server +- **WebView Hosts**: Runs Formplayer and custom applications +- **Native Features**: Camera, GPS, file system access + +**Technology Stack:** +- React Native +- WatermelonDB +- TypeScript + +### Synkronus + +Go backend server providing: + +- **REST API**: Comprehensive API for all operations +- **Synchronization**: Bidirectional sync protocol +- **Storage**: PostgreSQL database +- **Authentication**: JWT-based authentication +- **App Bundle Management**: Versioning and distribution + +**Technology Stack:** +- Go 1.24+ +- PostgreSQL +- Chi router +- JWT authentication + +### Formplayer + +React web application providing: + +- **Form Rendering**: JSON Forms-based form rendering +- **Question Types**: Various input types and renderers +- **Validation**: Client-side and schema validation +- **Integration**: JavaScript interface for custom apps + +**Technology Stack:** +- React +- JSON Forms +- Material-UI +- TypeScript + +### Synkronus CLI + +Go command-line utility providing: + +- **Server Management**: Administrative operations +- **Data Operations**: Sync, export, import +- **App Bundle Management**: Upload, download, version management + +**Technology Stack:** +- Go 1.24+ +- Cobra CLI framework + +### ODE Desktop + +Tauri desktop application providing: + +- **Local observation store**: SQLite per profile workspace +- **Sync console**: Pull, push, conflict visibility, index rebuild +- **Workbench**: Bundle download, form preview, custom app embed +- **Developer mode**: Local custom app mirror for iteration + +**Technology Stack:** +- Tauri 2 (Rust) +- React, TypeScript, Vite +- Embedded formplayer + +See [ODE Desktop Development](/development/ode-desktop-development). + +## Data Flow + +### Observation Creation + +1. User fills out form in Formplayer (WebView) +2. Formplayer validates and submits to Formulus +3. Formulus creates observation in local database (WatermelonDB) +4. Observation is marked for synchronization + +### Synchronization + +1. **Pull Phase**: Formulus requests changes from server + - Server returns records with `change_id > client_last_seen` + - Client applies changes to local database + +2. **Push Phase**: Formulus sends local changes to server + - Client sends records with transmission ID for idempotency + - Server validates and stores records + - Server returns success/failure for each record + +3. **Attachment Sync**: Separate phase for binary files + - Upload: Client uploads attachments referenced in observations + - Download: Client downloads attachments referenced in new observations + +### Conflict Resolution + +Conflicts are detected using hash comparison: + +- Each record has a hash computed from data, schemaType, and schemaVersion +- If server hash ≠ client's last seen hash, conflict detected +- Server accepts overwrite with warning +- Previous version stored in conflicts table + +## Sync Protocol + +### Change Detection + +Uses cursor-based approach with `change_id`: + +- Each record has a strictly increasing `change_id` (server-assigned) +- Client stores last seen `change_id` per schemaType +- Pull returns all records where `change_id > last_seen` + +**Advantages:** +- No dependence on system clocks +- No ambiguity about ordering +- Enables clean pagination and deduplication + +### Record Model + +Each form submission is an entity: + +- `id`: Unique identifier +- `schemaType`: Form type identifier +- `schemaVersion`: Form version +- `data`: JSON data (form responses) +- `hash`: Computed hash for conflict detection +- `change_id`: Strictly increasing change identifier +- `last_modified`: Server-assigned timestamp +- `last_modified_by`: Username from JWT +- `deleted`: Soft delete flag +- `origin_client_id`: Client that created the record + +### Attachment Handling + +Attachments are managed separately: + +- **Immutable**: Once uploaded, cannot be modified +- **Referenced**: Observations reference attachments by ID +- **Separate Sync**: Uploaded/downloaded in separate phase +- **Manifest-based**: Server provides manifest of changes + +See the [Synchronization guide](/docs/using/synchronization) for more details. + +## Database Design + +### Observations Table + +| Column | Type | Description | +|--------|------|-------------| +| `id` | UUID | Primary key | +| `schema_type` | String | Form type identifier | +| `schema_version` | String | Form version | +| `data` | JSONB | Form data | +| `hash` | String | Computed hash | +| `change_id` | Integer | Strictly increasing change ID | +| `last_modified` | Timestamp | Last modification time | +| `last_modified_by` | String | Username | +| `deleted` | Boolean | Soft delete flag | +| `origin_client_id` | String | Creating client ID | +| `created_at` | Timestamp | Creation time | + +### Attachments Table + +| Column | Type | Description | +|--------|------|-------------| +| `id` | String | Attachment ID (UUID) | +| `hash` | String | SHA-256 hash | +| `size` | Integer | File size in bytes | +| `mime_type` | String | MIME type | +| `change_id` | Integer | Change ID for sync | +| `last_modified` | Timestamp | Last modification time | +| `sync_state` | String | Sync state | + +## Security Architecture + +### Authentication + +- **JWT Tokens**: JSON Web Tokens for authentication +- **Token Refresh**: Refresh tokens for long-lived sessions +- **Role-Based Access**: `read-only`, `read-write`, `admin` roles + +### Authorization + +- **Endpoint Protection**: Middleware validates JWT on protected routes +- **Role Checks**: Endpoints check required roles +- **Resource Access**: Users can only access their own data (unless admin) + +### Data Security + +- **Transport**: HTTPS enforced in production (TLS terminated at your reverse proxy) +- **Storage at rest**: Encryption is a **host platform** responsibility (volume/disk encryption, managed DB TDE)—not a separate Synkronus feature +- **Secrets**: Environment variables for sensitive data +- **Validation**: Input validation on all endpoints + +See [Security reference](/docs/reference/security) for deployment checklist and mobile storage details. + +## Performance Considerations + +### Client-Side + +- **Local Database**: Fast queries using WatermelonDB +- **Incremental Sync**: Only sync changes since last sync +- **Adaptive pages**: Formulus starts at 32 pull / 4 push, grows toward 500 / 100, floor 1 +- **Lazy Loading**: Load attachments on demand +- **Caching**: Cache app bundles and form specifications + +### Server-Side + +- **Database Indexing**: Indexes on `change_id`, `schema_type`, `hash` +- **Connection Pooling**: Efficient database connection management +- **Pagination**: Cursor-based pagination for large datasets +- **Caching**: ETag support for efficient caching + +## Extension Points + +### Custom Renderers + +Create custom question type renderers: + +1. Define result interface in `FormulusInterfaceDefinition.ts` +2. Add interface method +3. Implement React component +4. Register renderer +5. Add mock implementation + +### Custom Applications + +Build custom web applications: + +1. Create HTML/CSS/JavaScript files +2. Include Formulus load script +3. Use Formulus JavaScript interface +4. Package as app bundle +5. Upload to server + +### Plugins + +ODE's plugin system allows for extending functionality without modifying core code. The plugin architecture is under active development and will be documented as it evolves. + +Current extension points: +- Custom renderers for question types +- Custom applications +- Server-side handlers (for advanced use cases) + +## Related Documentation + +- [Synchronization Details](/docs/using/synchronization) +- [Server Architecture for IT](/docs/guides/server-architecture-for-it) +- [Security reference](/docs/reference/security) +- [API Reference](/docs/reference/api) diff --git a/docs/docs/development/building-testing.md b/docs/docs/development/building-testing.md new file mode 100644 index 000000000..b1dc7be05 --- /dev/null +++ b/docs/docs/development/building-testing.md @@ -0,0 +1,364 @@ +--- +sidebar_position: 4 +--- + +# Building & Testing + +Complete guide to building ODE components from source and running tests. + +## Building from Source + +### Formulus + +Build the React Native mobile application: + + + + +```bash +cd formulus +cd ../packages/tokens && pnpm install && pnpm run build && cd ../formulus +pnpm install +pnpm run android # For Android +pnpm run ios # For iOS (macOS only) +``` + + + + +```bash +cd formulus/android +./gradlew assembleRelease +``` + +APK will be in `android/app/build/outputs/apk/release/` + + + + +```bash +cd formulus/ios +xcodebuild -workspace Formulus.xcworkspace -scheme Formulus -configuration Release +``` + +Or build from Xcode: +1. Open `Formulus.xcworkspace` in Xcode +2. Select "Any iOS Device" or specific device +3. Product → Archive +4. Distribute App + + + + +### Formplayer + +Build the React web form renderer: + +```bash +cd ../packages/tokens && pnpm install && pnpm run build && cd ../formulus-formplayer +pnpm install +pnpm run build +``` + +**Build and copy into Formulus (and Desktop):** + +```bash +pnpm run build:copy +``` + +This builds and copies the output to the Formulus app (and ODE Desktop when using the full pipeline). + +### Synkronus + +Build the Go server: + +```bash +cd synkronus +go build -o bin/synkronus cmd/synkronus/main.go +``` + +**Cross-platform builds:** + + + + +```bash +GOOS=linux GOARCH=amd64 go build -o bin/synkronus-linux cmd/synkronus/main.go +``` + + + + +```powershell +$env:GOOS="windows"; $env:GOARCH="amd64"; go build -o bin/synkronus.exe cmd/synkronus/main.go +``` + +Or using bash (Git Bash/WSL): + +```bash +GOOS=windows GOARCH=amd64 go build -o bin/synkronus.exe cmd/synkronus/main.go +``` + + + + +```bash +GOOS=darwin GOARCH=amd64 go build -o bin/synkronus-macos cmd/synkronus/main.go +``` + + + + +### Synkronus CLI + +Build the CLI: + +```bash +cd synkronus-cli +go build -o bin/synk ./cmd/synkronus +``` + +## Testing + +### Frontend Testing + +#### Formulus + +```bash +cd formulus +pnpm run test --ci --coverage --watchAll=false +``` + +Runs Jest tests with React Native Testing Library. + +#### Formplayer + +```bash +cd formulus-formplayer +pnpm run test run +``` + +Runs Vitest for React components. + +### Backend Testing + +#### Synkronus + +```bash +cd synkronus +go test ./... +``` + +Run all tests: + +```bash +# With coverage +go test -cover ./... + +# Verbose output +go test -v ./... + +# Specific package +go test ./internal/handlers +``` + +#### Integration Tests + +```bash +# Run integration tests (requires database) +go test -tags=integration ./... +``` + +### End-to-End Testing + +End-to-end testing infrastructure is under development. Current testing focuses on: + +- Unit tests for individual components +- Integration tests for API endpoints +- Component tests for React components + +E2E testing will be added as the testing infrastructure evolves. + +## Code Quality Checks + +### Linting + +**Frontend:** + +```bash +# Formulus +cd formulus +pnpm run lint +pnpm run lint:fix + +# Formplayer +cd formulus-formplayer +pnpm run lint +pnpm run lint:fix +``` + +**Backend:** + +```bash +# Synkronus +cd synkronus +golangci-lint run # If configured +``` + +### Formatting + +**Frontend:** + +```bash +# Format code +pnpm run format + +# Check formatting +pnpm run format:check +``` + +**Backend:** + +```bash +# Format Go code +go fmt ./... + +# Check with goimports +goimports -w . +``` + +## CI/CD Pipeline + +The project uses GitHub Actions for continuous integration: + +### Workflows + +**Synkronus Docker Build:** +- Builds on relevant pushes to `main` or `dev`, pull requests, published GitHub Releases, and manual dispatches +- Publishes multi-platform images to GitHub Container Registry (pull requests build without publishing) +- Stable releases publish `v{version}`, major/minor pointers, and `latest` +- Pre-releases publish `v{version}-{pre}` and `latest-pre-release` +- Branch pushes publish `main` or `dev` plus an immutable `sha-{short}` tag +- Manual dispatches publish only `sha-{short}`; feature-branch images are not published automatically + +See the [Deployment guide](/docs/guides/deployment) for the image-tag channels and recommended uses. + +**Frontend Quality Checks:** +- Runs on all PRs +- Checks linting and formatting +- Runs tests +- Builds components + +### Local CI Simulation + +Run CI checks locally: + +```bash +# Frontend +cd formulus && pnpm run lint && pnpm run format:check && pnpm run test --ci --coverage --watchAll=false +cd formulus-formplayer && pnpm run lint && pnpm run format:check && pnpm run test run + +# Backend +cd synkronus && go test ./... && go fmt ./... +``` + +## Docker Builds + +### Synkronus Docker Image + +Build locally: + +```bash +cd synkronus +docker build -t synkronus:local . +``` + +**Multi-platform build:** + +```bash +docker buildx create --name multiplatform --use +docker buildx build --platform linux/amd64,linux/arm64 -t synkronus:local . +``` + +## Release Process + +### Versioning + +ODE follows semantic versioning (MAJOR.MINOR.PATCH): + +- **MAJOR**: Breaking changes +- **MINOR**: New features (backward compatible) +- **PATCH**: Bug fixes (backward compatible) + +### Creating a Release + +1. Update version numbers in: + - `package.json` (frontend projects) + - `versioninfo.json` (Go projects) + - Documentation + +2. Create release branch: + +```bash +git checkout -b release/v1.0.0 +``` + +3. Update changelog + +4. Create tag: + +```bash +git tag -a v1.0.0 -m "Release version 1.0.0" +git push origin v1.0.0 +``` + +5. CI/CD will automatically: + - Build Docker images + - Publish to container registry + - Create GitHub release + +## Troubleshooting Build Issues + +### Node Modules Issues + +```bash +# Clear and reinstall +rm -rf node_modules +pnpm install +``` + +### Go Module Issues + +```bash +# Clean module cache +go clean -modcache +go mod download +go mod tidy +``` + +### Android Build Issues + +```bash +# Clean Android build +cd android +./gradlew clean +cd .. +pnpm run android +``` + +### iOS Build Issues + +```bash +# Clean pods +cd ios +rm -rf Pods Podfile.lock +bundle exec pod install +cd .. +pnpm run ios +``` + +## Related Documentation + +- [Development Setup](/development/setup) +- [Contributing Guide](/development/contributing) +- [Architecture Overview](/development/architecture) diff --git a/docs/docs/development/contributing.md b/docs/docs/development/contributing.md new file mode 100644 index 000000000..d967bc08b --- /dev/null +++ b/docs/docs/development/contributing.md @@ -0,0 +1,258 @@ +--- +sidebar_position: 3 +--- + +# Contributing + +Guide to contributing to ODE, including contribution process, coding standards, and community guidelines. + +## Overview + +ODE is an open-source project and welcomes contributions from the community. We believe that diverse perspectives and varied skill sets make our project stronger. + +## Documentation URL convention + +- **`/docs/developer/*`** — Persona landing pages (overview, getting started paths) +- **`/docs/development/*`** — Substantive developer content (architecture, setup, component guides) + +When adding links in docs, use the full path prefix `/docs/...`. + +## Ways to Contribute + +You can contribute to ODE in many ways: + +- **Code Contributions**: Bug fixes, new features, improvements +- **Documentation**: Improve existing docs, add examples, fix typos +- **Testing**: Test the platform, report bugs, verify fixes +- **Community**: Help others, answer questions, share use cases +- **Design**: UI/UX improvements, design system contributions +- **Translation**: Help translate documentation and interfaces + +## Contribution Process + +### 1. Find Something to Work On + +- Browse [GitHub Issues](https://github.com/OpenDataEnsemble/ode/issues) for open issues +- Look for issues labeled `good first issue` for beginners +- Check discussions for ideas and feature requests +- Review documentation for gaps or improvements + +### 2. Set Up Development Environment + +Follow the [Development Setup guide](/development/setup) to set up your local environment. + +### 3. Create a Branch + +```bash +git checkout -b feature/your-feature-name +# or +git checkout -b fix/your-bug-fix +``` + +Use descriptive branch names: +- `feature/` for new features +- `fix/` for bug fixes +- `docs/` for documentation +- `refactor/` for code refactoring + +### 4. Make Your Changes + +- Write clean, well-documented code +- Follow coding standards (see below) +- Add tests for new functionality +- Update documentation as needed + +### 5. Test Your Changes + +```bash +# Frontend projects (from each package directory; see Development Setup for pnpm) +cd formulus && pnpm run lint && pnpm run format:check && pnpm run test --ci --coverage --watchAll=false +cd formulus-formplayer && pnpm run lint && pnpm run format:check && pnpm run test run + +# Go projects +cd synkronus && go test ./... +go fmt ./... +``` + +### 6. Commit Your Changes + +Write clear, descriptive commit messages: + +``` +feat: Add support for custom question types + +- Add interface for custom renderers +- Implement renderer registration +- Add documentation and examples +``` + +**Commit Message Format:** +- `feat:` for new features +- `fix:` for bug fixes +- `docs:` for documentation +- `style:` for formatting +- `refactor:` for code refactoring +- `test:` for tests +- `chore:` for maintenance + +### 7. Push and Create Pull Request + +```bash +git push origin feature/your-feature-name +``` + +Create a pull request on GitHub with: + +- Clear description of changes +- Reference to related issues +- Screenshots (if UI changes) +- Testing notes + +## Coding Standards + +### Frontend (React/React Native) + +**TypeScript:** +- Use TypeScript for all new code +- Enable strict type checking +- Define interfaces for data structures +- Avoid `any` type + +**Code Style:** +- Follow ESLint rules +- Use Prettier for formatting +- Use functional components with hooks +- Keep components small and focused + +**Example:** + +```typescript +interface User { + id: string; + username: string; + role: 'read-only' | 'read-write' | 'admin'; +} + +const UserProfile: React.FC<{user: User}> = ({user}) => { + return
    {user.username}
    ; +}; +``` + +### Backend (Go) + +**Code Style:** +- Follow `gofmt` formatting +- Use `golint` recommendations +- Write clear, descriptive function names +- Add godoc comments for exported functions + +**Example:** + +```go +// GetUser retrieves a user by username. +// Returns an error if the user is not found. +func (s *Service) GetUser(username string) (*User, error) { + // Implementation +} +``` + +**Error Handling:** +- Always handle errors explicitly +- Return descriptive error messages +- Use error wrapping for context + +### General Guidelines + +- **Write clear code**: Code should be self-documenting +- **Add comments**: Explain why, not what +- **Keep functions small**: Single responsibility principle +- **Use meaningful names**: Variables and functions should be descriptive +- **Avoid duplication**: DRY (Don't Repeat Yourself) +- **Test your code**: Write tests for new functionality + +## Code Quality Checks + +### Before Submitting + +Ensure your code passes all quality checks: + +```bash +# Frontend (pnpm, per package directory) +pnpm run lint +pnpm run format:check +pnpm run test + +# Backend +go test ./... +go fmt ./... +go vet ./... +``` + +### CI/CD Checks + +The CI pipeline automatically checks: + +- Linting (ESLint for frontend) +- Formatting (Prettier for frontend, gofmt for backend) +- Tests (Jest for frontend, go test for backend) +- Build (ensures code compiles) + +## Pull Request Guidelines + +### PR Description + +Include: + +- **Summary**: Brief description of changes +- **Motivation**: Why this change is needed +- **Changes**: What was changed +- **Testing**: How it was tested +- **Screenshots**: If UI changes +- **Related Issues**: Link to related issues + +### Review Process + +- Maintainers will review your PR +- Address feedback promptly +- Be open to suggestions +- Keep discussions constructive + +### After Approval + +- Maintainers will merge your PR +- Your contribution will be included in the next release +- Thank you for contributing! + +## Code of Conduct + +We are committed to providing a welcoming and inclusive environment. Please: + +- Be respectful and considerate +- Welcome newcomers and help them learn +- Focus on constructive feedback +- Respect different viewpoints and experiences + +## Getting Help + +If you need help: + +- Check the [documentation](/) +- Search [GitHub Issues](https://github.com/OpenDataEnsemble/ode/issues) +- Ask in [GitHub Discussions](https://github.com/OpenDataEnsemble/ode/discussions) +- Contact maintainers + +## Recognition + +Contributors are recognized in: + +- GitHub contributors list +- Release notes +- Project documentation + +Thank you for contributing to ODE! + +## Related Documentation + +- [Development Setup](/development/setup) +- [Building & Testing](/development/building-testing) +- [Architecture Overview](/development/architecture) diff --git a/docs/docs/development/extending.md b/docs/docs/development/extending.md new file mode 100644 index 000000000..e223bf08a --- /dev/null +++ b/docs/docs/development/extending.md @@ -0,0 +1,242 @@ +--- +sidebar_position: 5 +--- + +# Extending ODE + +Guide to extending ODE functionality through custom renderers, plugins, and internal APIs. + +## Overview + +ODE is designed to be extensible. You can extend functionality through: + +- **Custom Renderers**: Add new question types +- **Custom Applications**: Build specialized web applications +- **Internal APIs**: Extend server functionality (advanced) + +## Custom Renderers + +Custom renderers allow you to add new question types to the Formplayer. + +### Implementation Steps + +1. **Define Result Interface** + +In `FormulusInterfaceDefinition.ts`: + +```typescript +export interface MyCustomResultData { + type: 'mycustom'; + value: string; + timestamp: string; +} + +export type MyCustomResult = ActionResult; +``` + +2. **Add Interface Method** + +```typescript +export interface FormulusInterface { + requestMyCustom(fieldId: string): Promise; +} +``` + +3. **Create React Component** + +Create `MyCustomQuestionRenderer.tsx`: + +```typescript +import React, {useState, useCallback} from 'react'; +import {withJsonFormsControlProps} from '@jsonforms/react'; +import {FormulusClient} from './FormulusInterface'; + +const MyCustomQuestionRenderer: React.FC = ({ + data, + handleChange, + path, +}) => { + const [isLoading, setIsLoading] = useState(false); + const formulusClient = useRef(new FormulusClient()); + + const handleAction = useCallback(async () => { + setIsLoading(true); + try { + const result = await formulusClient.current.requestMyCustom(path); + if (result.status === 'success') { + handleChange(path, result.data.value); + } + } finally { + setIsLoading(false); + } + }, [path, handleChange]); + + return ( + + ); +}; + +export default withJsonFormsControlProps(MyCustomQuestionRenderer); +``` + +4. **Register Renderer** + +In `App.tsx`: + +```typescript +import MyCustomQuestionRenderer, { + myCustomQuestionTester, +} from './MyCustomQuestionRenderer'; + +const customRenderers = [ + ...materialRenderers, + {tester: myCustomQuestionTester, renderer: MyCustomQuestionRenderer}, +]; +``` + +5. **Add Mock Implementation** + +For development testing, add mock support in `webview-mock.ts`. + +6. **Implement Native Handler** + +In Formulus React Native code, implement the native handler in `FormulusMessageHandlers.ts`. + +See the [Form Design guide](/guides/form-design) for complete examples. + +## Custom Applications + +Custom applications are web-based interfaces that run within Formulus. + +### Creating a Custom App + +1. **Create HTML Structure** + +```html + + + + My Custom App + + + +

    My Custom App

    + + + +``` + +2. **Use Formulus API** + +```javascript +async function init() { + const api = await getFormulus(); + + // Use API methods + const forms = await api.getAvailableForms(); + // ... +} +``` + +3. **Package as App Bundle** + +Create a ZIP file with: +- `index.html` +- `manifest.json` +- Assets (CSS, JS, images) + +4. **Upload to Server** + +```bash +synk app-bundle upload bundle.zip --activate +``` + +See the [Custom Applications guide](/guides/custom-applications) for details. + +## Internal APIs + +For advanced extensions, you can extend the server functionality. + +### Adding New Endpoints + +1. **Define Handler** + +In `internal/handlers/`: + +```go +func (h *Handler) MyNewEndpoint(w http.ResponseWriter, r *http.Request) { + // Implementation +} +``` + +2. **Register Route** + +In `internal/api/api.go`: + +```go +r.Route("/my-endpoint", func(r chi.Router) { + r.Get("/", h.MyNewEndpoint) +}) +``` + +3. **Add Tests** + +Create tests in `internal/handlers/`: + +```go +func TestMyNewEndpoint(t *testing.T) { + // Test implementation +} +``` + +### Adding New Services + +1. **Create Service** + +In `internal/services/`: + +```go +type MyService struct { + // Dependencies +} + +func NewMyService(config *Config) (*MyService, error) { + // Initialization +} +``` + +2. **Integrate with Handlers** + +Pass service to handlers and use in endpoints. + +## Best Practices + +### Custom Renderers + +- Follow existing renderer patterns +- Handle loading and error states +- Provide fallback options (manual input) +- Test thoroughly in mock and native environments + +### Custom Applications + +- Always use `getFormulus()` before accessing API +- Handle offline scenarios gracefully +- Optimize for mobile devices +- Test on actual devices + +### Internal APIs + +- Follow Go best practices +- Add comprehensive tests +- Document endpoints +- Consider backward compatibility + +## Related Documentation + +- [Form Design Guide](/guides/form-design) +- [Custom Applications Guide](/guides/custom-applications) +- [API Reference](/reference/api) +- [Architecture Overview](/development/architecture) diff --git a/docs/docs/development/formplayer-development.md b/docs/docs/development/formplayer-development.md new file mode 100644 index 000000000..341cd7fbe --- /dev/null +++ b/docs/docs/development/formplayer-development.md @@ -0,0 +1,103 @@ +--- +sidebar_position: 4 +--- + +# Formplayer Development + +Complete guide for developing the Formplayer form rendering component. + +## Overview + +Formplayer is a React web application that renders JSON Forms. It runs within WebViews in the Formulus mobile app. + +## Prerequisites + +- **Node.js** 20+ and **pnpm** 10.33.2 +- **React** development experience +- **`@ode/tokens` built** — from `packages/tokens`: `pnpm install && pnpm run build` (see [Development Setup](/docs/development/setup#package-manager-pnpm)) + +## Local Development + +### Setup + +```bash +cd packages/tokens && pnpm install && pnpm run build && cd ../.. +cd formulus-formplayer +pnpm install +``` + +### Development Server + +```bash +pnpm start +``` + +Opens at http://localhost:3000 (or the port Vite prints). + +### Development Features + +- **Hot Reload**: Changes reflect immediately +- **Source Maps**: Debug in browser DevTools +- **Error Overlay**: Errors shown in browser + +## Building + +### Build and copy to Formulus (and ODE Desktop) + +```bash +pnpm run build:copy +``` + +This: + +1. Syncs the Formulus interface definition +2. Builds the React app (`build/`) +3. Copies assets into Formulus Android/iOS formplayer directories +4. Copies assets into `desktop/public/formplayer_dist/` when building the full pipeline + +From **ODE Desktop** only (requires an existing `formulus-formplayer/build/`): + +```bash +cd desktop +pnpm copy:formplayer +``` + +### Build for Web only + +```bash +pnpm run build +``` + +Output in `build/` directory. + +## Project Structure + +- `src/`: React source code +- `public/`: Static assets +- `build/`: Production build output + +## Adding Question Types + +1. **Create Renderer Component:** + ```typescript + // src/NewQuestionRenderer.tsx + export function NewQuestionRenderer(props) { + return + } + ``` + +2. **Register in Formplayer:** + Add to Formplayer configuration when initialized by Formulus. + +When you change `formulus/src/webview/FormulusInterfaceDefinition.ts`, run `pnpm run sync-interface` (or `pnpm run build`) in formulus-formplayer. + +## Testing + +```bash +pnpm run test run +``` + +## Related Documentation + +- [Formplayer Reference](/reference/formplayer) - Component reference +- [Form Design Guide](/guides/form-design) - Creating forms diff --git a/docs/docs/development/formulus-development.md b/docs/docs/development/formulus-development.md new file mode 100644 index 000000000..49ba525ef --- /dev/null +++ b/docs/docs/development/formulus-development.md @@ -0,0 +1,367 @@ +--- +sidebar_position: 3 +--- + +# Formulus Development + +Complete guide for developing the Formulus mobile application. + +## Overview + +This guide covers local development and production building for the Formulus React Native application. + +## Prerequisites + +### Required Tools + +- **Node.js** 20+ and **pnpm** 10.33.2 (see [Development Setup](/docs/development/setup#package-manager-pnpm)) +- **React Native CLI** or Expo CLI +- **Java Development Kit (JDK)** 11 or higher +- **Android Studio** (for Android development) +- **Xcode** (for iOS development, macOS only) +- **Android SDK** (via Android Studio) +- **CocoaPods** (for iOS, macOS only) + +### Platform-Specific Requirements + +#### Android + +- Android SDK Platform 33+ +- Android SDK Build Tools +- Android Emulator or physical device + +#### iOS (macOS only) + +- Xcode 14+ +- CocoaPods +- iOS Simulator or physical device + +## Local Development Setup + +### Step 1: Clone Repository + +```bash +git clone https://github.com/OpenDataEnsemble/ode.git +cd ode/formulus +``` + +### Step 2: Install Dependencies + +```bash +cd ../packages/tokens && pnpm install && pnpm run build && cd ../formulus +pnpm install +``` + +### Step 3: Install iOS Dependencies + + + + +```bash +cd ios +bundle install +bundle exec pod install +cd .. +``` + + + + +iOS development is only available on macOS. Skip this step if you're developing for Android only. + + + + +### Step 4: Generate API Client + +Generate the Synkronus API client from OpenAPI spec: + +```bash +pnpm run generate:api +``` + +### Step 5: Generate WebView Injection Script + +Generate the JavaScript injection script: + +```bash +pnpm run generate +``` + +### Step 6: Start Metro Bundler + +```bash +pnpm start +``` + +Keep this terminal open. Metro is the JavaScript bundler. + +### Step 7: Run on Device/Emulator + + + + +```bash +pnpm run android +``` + +This will: +1. Build the Android app +2. Install it on your connected device/emulator +3. Start the app +4. Connect to Metro bundler + + + + +```bash +pnpm run ios +``` + +This will: +1. Build the iOS app +2. Install it on your iOS Simulator or connected device +3. Start the app +4. Connect to Metro bundler + + + + +## Development Workflow + +### Hot Reload + +Changes to JavaScript/TypeScript files automatically reload: + +- **Fast Refresh**: React components update without losing state +- **Live Reload**: Full app reload on some changes +- **Error Overlay**: Errors displayed in app + +### Debugging + +#### React Native Debugger + +1. Shake device or press `Ctrl+M` (Windows/Linux) or `Cmd+M` (macOS) +2. Select "Debug" +3. Chrome DevTools opens +4. Set breakpoints and inspect code + +#### Logging + +```typescript +import { console } from 'react-native' + +console.log('Debug message') +console.warn('Warning message') +console.error('Error message') +``` + +View logs: + + + + +```bash +adb logcat | grep -i formulus +``` + + + + +View logs in Xcode console: +1. Open Xcode +2. Run the app +3. View logs in the bottom panel + +Or use Console.app on macOS: +```bash +# Filter for your app +log stream --predicate 'processImagePath contains "Formulus"' +``` + + + + +For Android development on Windows: + +```powershell +adb logcat | Select-String formulus +``` + +Or using Git Bash/WSL: + +```bash +adb logcat | grep -i formulus +``` + +For iOS development, Windows is not supported. Use macOS with Xcode. + + + + +### Testing + +```bash +# Run tests (do not use `pnpm test -- --watch`; use pnpm run test) +pnpm run test + +# CI-style run +pnpm run test --ci --coverage --watchAll=false +``` + +## Building for Production + +### Android Production Build + +#### Step 1: Generate Signing Key + +```bash +keytool -genkeypair -v -storetype PKCS12 -keystore formulus-release.keystore -alias formulus -keyalg RSA -keysize 2048 -validity 10000 +``` + +#### Step 2: Configure Signing + +Edit `android/gradle.properties`: + +```properties +FORMULUS_RELEASE_STORE_FILE=formulus-release.keystore +FORMULUS_RELEASE_KEY_ALIAS=formulus +FORMULUS_RELEASE_STORE_PASSWORD=your-store-password +FORMULUS_RELEASE_KEY_PASSWORD=your-key-password +``` + +#### Step 3: Build APK + +```bash +cd android +./gradlew assembleRelease +``` + +APK location: `android/app/build/outputs/apk/release/app-release.apk` + +#### Step 4: Build AAB (for Play Store) + +```bash +cd android +./gradlew bundleRelease +``` + +AAB location: `android/app/build/outputs/bundle/release/app-release.aab` + +### iOS Production Build (macOS only) + +#### Step 1: Configure Signing + +1. Open `ios/Formulus.xcworkspace` in Xcode +2. Select project in navigator +3. Go to "Signing & Capabilities" +4. Select team and provisioning profile + +#### Step 2: Build Archive + +1. In Xcode: Product → Archive +2. Wait for build to complete +3. Distribute App in Organizer + +#### Step 3: Export IPA + +1. Select archive in Organizer +2. Click "Distribute App" +3. Choose distribution method (App Store, Ad Hoc, Enterprise) +4. Follow export wizard + +## Project Structure + +### Key Directories + +- `src/`: TypeScript source code +- `android/`: Android native code +- `ios/`: iOS native code +- `assets/`: Static assets (images, fonts, etc.) + +### Important Files + +- `App.tsx`: Main application component +- `package.json`: Dependencies and scripts +- `tsconfig.json`: TypeScript configuration +- `metro.config.js`: Metro bundler configuration +- `babel.config.js`: Babel configuration + +## Common Tasks + +### Adding Dependencies + +```bash +pnpm add package-name +``` + +For native dependencies, may need to: + +```bash +# Android +cd android && ./gradlew clean && cd .. + +# iOS +cd ios && pod install && cd .. +``` + +### Updating API Client + +When Synkronus API changes: + +```bash +pnpm run generate:api +``` + +### Updating WebView Interface + +When Formulus interface changes: + +```bash +pnpm run generate +``` + +### Cleaning Build + +```bash +# Android +cd android && ./gradlew clean && cd .. + +# iOS +cd ios && xcodebuild clean && cd .. + +# Remove node_modules +rm -rf node_modules +pnpm install +``` + +## Troubleshooting + +### Common Issues + +**Metro Bundler Won't Start:** +- Clear Metro cache: `pnpm start -- --reset-cache` +- Android: if Gradle reports missing `:notifee_core`, run `pnpm run vendor:notifee` (or use `pnpm run android`, which runs it via `preandroid`) +- Delete `node_modules` and reinstall + +**Build Fails:** +- Clean build: `cd android && ./gradlew clean` +- Check Java version: `java -version` (should be 11+) +- Verify Android SDK is installed + +**iOS Build Fails:** +- Run `pod install` in `ios/` directory +- Clean build folder in Xcode +- Check signing configuration + +**App Crashes on Launch:** +- Check logs: `adb logcat` or Xcode console +- Verify API client is generated +- Check WebView injection script is generated + +## Related Documentation + +- [Installing Formulus for Development](/docs/development/installing-formulus-dev) - ADB/emulator setup +- [Formulus Reference](/reference/formulus) - Component reference +- [Building and Testing](/development/building-testing) - Build procedures + diff --git a/docs/docs/development/index.md b/docs/docs/development/index.md new file mode 100644 index 000000000..6b011a502 --- /dev/null +++ b/docs/docs/development/index.md @@ -0,0 +1,132 @@ +--- +sidebar_position: 0 +--- + +# Development + +Resources for developers who want to contribute to ODE or extend its functionality. + +## Getting Started with Development + +
    +
    +
    +
    +

    Setup

    +
    +
    +

    Set up your development environment and get the codebase running locally.

    + Setup Guide → +
    +
    +
    +
    +
    +
    +

    Architecture

    +
    +
    +

    Understand the architecture and design decisions behind ODE.

    + View Architecture → +
    +
    +
    +
    + +## Component Development + +
    +
    +
    +
    +

    Formulus Development

    +
    +
    +

    Develop and contribute to the Formulus mobile app (React Native).

    + View Docs → +
    +
    +
    +
    +
    +
    +

    Formplayer Development

    +
    +
    +

    Develop and contribute to the Formplayer form engine (React).

    + View Docs → +
    +
    +
    +
    +
    +
    +

    Synkronus Development

    +
    +
    +

    Develop and contribute to the Synkronus server backend (Go).

    + View Docs → +
    +
    +
    +
    +
    +
    +

    Synkronus Portal Development

    +
    +
    +

    Develop and contribute to the Synkronus web portal.

    + View Docs → +
    +
    +
    +
    + +## Contributing & Extending + +
    +
    +
    +
    +

    Contributing

    +
    +
    +

    Learn how to contribute to ODE, including code style and best practices.

    + Contribute → +
    +
    +
    +
    +
    +
    +

    Building & Testing

    +
    +
    +

    Build ODE components from source and run the test suite.

    + Build & Test → +
    +
    +
    +
    +
    +
    +

    Extending ODE

    +
    +
    +

    Extend ODE functionality with custom renderers and integrations.

    + Extend → +
    +
    +
    +
    +
    +
    +

    Installing Formulus Dev

    +
    +
    +

    Install and run the Formulus development build on your device.

    + Install → +
    +
    +
    +
    diff --git a/docs/docs/development/installation.md b/docs/docs/development/installation.md new file mode 100644 index 000000000..fffd9c26d --- /dev/null +++ b/docs/docs/development/installation.md @@ -0,0 +1,432 @@ +--- +sidebar_position: 4 +--- + +# Installation + +Complete installation guide for all ODE components. This guide covers prerequisites, server setup, and mobile app installation. + +## Prerequisites + +Before installing ODE components, ensure your system meets the following requirements. + +### System Requirements + +#### For Mobile Development (Formulus) + +| Requirement | Minimum | Recommended | +|-------------|---------|-------------| +| **Operating System** | macOS 10.15, Windows 10, or Linux | Latest stable version | +| **Node.js** | 20.0 or higher | Latest stable | +| **pnpm** | 10.33.2 (via Corepack or global install) | Match `packageManager` in ODE `package.json` files | +| **Android Studio** | Latest stable | Latest stable | +| **Xcode** | 14.0 or higher (macOS only) | Latest stable | +| **Java Development Kit** | JDK 17 | JDK 17 or higher | + +#### For Server Deployment (Synkronus) + +| Requirement | Minimum | Recommended | +|-------------|---------|-------------| +| **Operating System** | Linux, macOS, or Windows | Linux | +| **Go** | 1.24 or higher | Latest stable | +| **PostgreSQL** | 13.0 or higher | 15.0 or higher | +| **Docker** | 20.10 or higher (optional) | Latest stable | +| **Memory** | 2 GB RAM | 4 GB RAM or more | +| **Storage** | 10 GB free space | 50 GB or more | + +### Development Tools + +**Required Tools:** +- Git: Version control system +- Code Editor: Visual Studio Code, IntelliJ IDEA, or similar +- Terminal: Command-line interface for running commands + +**Recommended Tools:** +- Postman or curl: For testing API endpoints +- pgAdmin or DBeaver: For database management +- Docker Desktop: For containerized development + +### Verification + +Verify your installation by running: + + + + +```bash +# Check Node.js version +node --version + +# Check pnpm version +pnpm --version + +# Check Go version +go version + +# Check PostgreSQL version +psql --version + +# Check Docker version (if using) +docker --version +``` + + + + +Using PowerShell: + +```powershell +# Check Node.js version +node --version + +# Check pnpm version +pnpm --version + +# Check Go version +go version + +# Check PostgreSQL version +psql --version + +# Check Docker version (if using) +docker --version +``` + +Or using Git Bash/WSL (same commands as Linux/macOS): + +```bash +node --version +pnpm --version +go version +psql --version +docker --version +``` + + + + +## Installing Synkronus Server + +Synkronus is the backend server component of ODE, responsible for data synchronization, storage, and API services. + + + + +The easiest way to run Synkronus is using Docker: + +```bash +# Pull the latest image +docker pull ghcr.io/opendataensemble/synkronus:latest + +# Run the container +docker run -d \ + --name synkronus \ + -p 8080:8080 \ + -e DB_CONNECTION="postgres://user:password@host:5432/synkronus" \ + -e JWT_SECRET="your-secret-key-here" \ + -v synkronus-bundles:/app/data/app-bundles \ + ghcr.io/opendataensemble/synkronus:latest +``` + + + + +For a complete setup with PostgreSQL: + +```bash +# Clone the repository +git clone https://github.com/OpenDataEnsemble/ode.git +cd ode + +# Start with Docker Compose (includes PostgreSQL) +docker compose up -d +``` + + + + +Build and run Synkronus from source: + + + + +```bash +# Clone the repository +git clone https://github.com/OpenDataEnsemble/ode.git +cd ode + +# Build the Synkronus portal first +cd packages/tokens && pnpm install --frozen-lockfile && pnpm run build && cd ../.. +cd packages/components && pnpm install --frozen-lockfile && cd .. +cd synkronus-portal && pnpm install --frozen-lockfile && pnpm run build && cd .. + +# Build and run Synkronus +cd synkronus +go build -o synkronus ./cmd/synkronus +./synkronus +``` + + + + +Using PowerShell: + +```powershell +# Clone the repository +git clone https://github.com/OpenDataEnsemble/ode.git +cd ode + +# Build the Synkronus portal first +cd packages/tokens && pnpm install --frozen-lockfile && pnpm run build && cd ../.. +cd packages/components && pnpm install --frozen-lockfile && cd .. +cd synkronus-portal && pnpm install --frozen-lockfile && pnpm run build && cd .. + +# Build and run Synkronus +cd synkronus +go build -o synkronus.exe ./cmd/synkronus +.\synkronus.exe +``` + +Or using Git Bash/WSL (same as Linux/macOS): + +```bash +git clone https://github.com/OpenDataEnsemble/ode.git +cd ode + +# Build the Synkronus portal first +cd packages/tokens && pnpm install --frozen-lockfile && pnpm run build && cd ../.. +cd packages/components && pnpm install --frozen-lockfile && cd .. +cd synkronus-portal && pnpm install --frozen-lockfile && pnpm run build && cd .. + +# Build and run Synkronus +cd synkronus +go build -o bin/synkronus ./cmd/synkronus +./bin/synkronus +``` + + + + + + + +### Configuration + +Synkronus is configured using environment variables. Create a `.env` file or set environment variables: + +| Variable | Description | Default | Required | +|----------|-------------|---------|----------| +| `PORT` | HTTP server port | `8080` | No | +| `DB_CONNECTION` | PostgreSQL connection string | - | Yes | +| `JWT_SECRET` | Secret key for JWT signing | - | Yes | +| `LOG_LEVEL` | Logging level (debug, info, warn, error) | `info` | No | +| `APP_BUNDLE_PATH` | Directory for app bundles | `./data/app-bundles` | No | +| `MAX_VERSIONS_KEPT` | Maximum app bundle versions to keep | `5` | No | + +**Example Configuration:** + +```bash +PORT=8080 +DB_CONNECTION=postgres://synkronus:password@localhost:5432/synkronus?sslmode=disable +JWT_SECRET=your-secret-key-change-this-in-production +LOG_LEVEL=info +APP_BUNDLE_PATH=./data/app-bundles +MAX_VERSIONS_KEPT=5 +``` + +### Database Setup + +Before running Synkronus, set up a PostgreSQL database: + + + + +```bash +# Connect to PostgreSQL +psql -U postgres +``` + +Then run SQL commands: + +```sql +-- Create database +CREATE DATABASE synkronus; + +-- Create user (optional) +CREATE USER synkronus WITH PASSWORD 'your-password'; +GRANT ALL PRIVILEGES ON DATABASE synkronus TO synkronus; +``` + + + + +Using PowerShell or Command Prompt: + +```powershell +# Connect to PostgreSQL +psql -U postgres +``` + +Then run SQL commands: + +```sql +-- Create database +CREATE DATABASE synkronus; + +-- Create user (optional) +CREATE USER synkronus WITH PASSWORD 'your-password'; +GRANT ALL PRIVILEGES ON DATABASE synkronus TO synkronus; +``` + +Or using Git Bash/WSL (same as Linux/macOS): + +```bash +psql -U postgres +``` + + + + +```bash +# Connect to PostgreSQL container +docker compose exec postgres psql -U postgres +``` + +Then run SQL commands: + +```sql +CREATE DATABASE synkronus; +CREATE USER synkronus WITH PASSWORD 'your-password'; +GRANT ALL PRIVILEGES ON DATABASE synkronus TO synkronus; +``` + + + + +The database schema will be created automatically on first run. + +### Verification + +Verify the installation: + +1. Check that the server is running: + ```bash + curl http://localhost:8080/health + ``` + +2. Check the API documentation: + ```bash + curl http://localhost:8080/api/docs + ``` + +## Installing Formulus App + +Formulus is the mobile application component of ODE, available for Android and iOS devices. + +For detailed installation instructions, see the [Installing Formulus guide](/docs/getting-started/installation/installing-formulus) which covers: + +- F-Droid installation (recommended for end users) +- Direct APK installation +- System requirements +- Post-installation setup + +For developers who want to install via ADB or emulator, see the [Development Installation guide](/docs/development/installing-formulus-dev). + +### Quick Installation Summary + +#### For End Users + +1. **F-Droid** (Recommended): Install via F-Droid app store +2. **Direct APK**: Download and install APK file directly + +See [Installing Formulus](/docs/getting-started/installation/installing-formulus) for complete instructions. + +#### For Developers + +1. **ADB Installation**: Install via Android Debug Bridge +2. **Emulator**: Run on Android emulator +3. **Development Build**: Build from source with hot reload + +See [Installing Formulus for Development](/docs/development/installing-formulus-dev) for complete instructions. + +### App Configuration + +After installation, configure the app to connect to your Synkronus server: + +1. Open the Formulus app +2. Navigate to Settings +3. Enter your server URL (e.g., `https://your-server.com`) +4. Enter your authentication credentials +5. Save the configuration + +## Installing Synkronus CLI + +The Synkronus CLI is a command-line utility for interacting with the Synkronus server. + +### Installation + +```bash +go install github.com/OpenDataEnsemble/ode/synkronus-cli/cmd/synkronus@latest +``` + +Or build from source: + +```bash +git clone https://github.com/OpenDataEnsemble/ode/synkronus-cli.git +cd synkronus-cli +go build -o bin/synk ./cmd/synkronus +``` + +### Configuration + +By default, the CLI uses a configuration file located at `$HOME/.synkronus.yaml`. + +**Example configuration file:** + +```yaml +api: + url: http://localhost:8080 + version: 1.0.0 +``` + +You can override this per-command with `--config ` or use `synk config use ` to set a persistent default. + +## Troubleshooting + +### Server Not Accessible + +If the mobile app cannot connect to the server: + +- Verify the server is running: `curl http://localhost:8080/health` +- Check firewall settings +- For Android emulator, use `10.0.2.2` instead of `localhost` +- For iOS simulator, use `localhost` or your machine's IP address + +### Database Connection Issues + +**Problem**: Cannot connect to database + +**Solution**: Verify the `DB_CONNECTION` string format and ensure PostgreSQL is running and accessible. + +### Port Already in Use + +**Problem**: Port 8080 is already in use + +**Solution**: Change the `PORT` environment variable to use a different port. + +### Build Errors + +**Problem**: Build fails with errors + +**Solution**: +- Ensure all prerequisites are installed +- Check that dependencies are up to date +- Review error messages for specific issues +- See component-specific documentation for build instructions + +## Next Steps + +- Follow the [Quick Start guide](/docs/getting-started/index) for a complete setup +- Learn how to [use the app](/docs/using/your-first-form) for data collection +- Review the [Deployment guide](/docs/guides/deployment) for production setup + diff --git a/docs/docs/development/installing-formulus-dev.md b/docs/docs/development/installing-formulus-dev.md new file mode 100644 index 000000000..8119c5272 --- /dev/null +++ b/docs/docs/development/installing-formulus-dev.md @@ -0,0 +1,513 @@ +--- +sidebar_position: 2 +--- + +# Installing Formulus for Development + +Complete guide for developers to install Formulus on physical devices or emulators using ADB or development builds. + +## Overview + +Developers can install Formulus on Android devices or emulators using several methods: + +- **ADB Installation** - Install APK directly via Android Debug Bridge +- **Android Emulator** - Run and test on virtual devices +- **Development Build** - Build and run from source with hot reload + +This guide covers cross-platform commands for Linux, macOS, and Windows. + +## Prerequisites + +Before installing, ensure you have: + +| Requirement | Description | +|-------------|-------------| +| **Android SDK** | Android SDK Platform Tools (includes ADB) | +| **ADB** | Android Debug Bridge (part of SDK Platform Tools) | +| **USB Drivers** | Device-specific USB drivers (for physical devices) | +| **Developer Options** | Enabled on Android device (for physical devices) | +| **USB Debugging** | Enabled on Android device (for physical devices) | + +### Installing Android SDK Platform Tools + + + + +```bash +# Ubuntu/Debian +sudo apt-get update +sudo apt-get install android-tools-adb android-tools-fastboot + +# Fedora +sudo dnf install android-tools + +# Arch Linux +sudo pacman -S android-tools + +# Verify installation +adb version +``` + + + + +```bash +# Using Homebrew +brew install android-platform-tools + +# Verify installation +adb version +``` + + + + +1. **Download Android SDK Platform Tools** from [developer.android.com](https://developer.android.com/studio/releases/platform-tools) +2. **Extract the ZIP file** to a location like `C:\platform-tools` +3. **Add to PATH**: + - Open System Properties → Environment Variables + - Add `C:\platform-tools` to the PATH variable +4. **Verify installation**: + ```powershell + adb version + ``` + + + + +## Method 1: ADB Installation on Physical Device + +### Step 1: Enable Developer Options + +1. **Open Settings** on your Android device +2. **Navigate to About Phone** (or About Device) +3. **Find "Build Number"** (usually at the bottom) +4. **Tap "Build Number" 7 times** until you see "You are now a developer!" + +### Step 2: Enable USB Debugging + +1. **Go back to Settings** +2. **Open Developer Options** (now visible in Settings) +3. **Enable "USB Debugging"** +4. **Accept the warning** about USB debugging + +### Step 3: Connect Device + +1. **Connect your device** to your computer via USB cable +2. **On your device**, you may see a prompt: "Allow USB debugging?" +3. **Check "Always allow from this computer"** (optional but recommended) +4. **Tap "Allow"** + +### Step 4: Verify Device Connection + + + + +```bash +adb devices +``` + + + + +```powershell +adb devices +``` + + + + +**Expected output:** +``` +List of devices attached +ABC123XYZ456 device +``` + +If you see "unauthorized", check your device and accept the USB debugging prompt. + +### Step 5: Install Formulus APK + +#### Option A: Install from Local APK File + + + + +```bash +# Navigate to directory containing APK +cd /path/to/formulus/android/app/build/outputs/apk/debug + +# Install APK +adb install app-debug.apk +``` + + + + +```powershell +# Navigate to directory containing APK +cd C:\path\to\formulus\android\app\build\outputs\apk\debug + +# Install APK +adb install app-debug.apk +``` + + + + +#### Option B: Install from Remote URL + + + + +```bash +# Browse the release and download the arm64-v8a APK for most phones: +# https://github.com/OpenDataEnsemble/ode/releases/tag/v1.3.2 +# Asset names look like: formulus-v1.3.2-64-universal-release-YYYYMMDD.apk +adb install /path/to/formulus-v1.3.2-*-universal-release-*.apk +``` + + + + +```powershell +# Browse the release and download the arm64-v8a APK for most phones: +# https://github.com/OpenDataEnsemble/ode/releases/tag/v1.3.2 +# Asset names look like: formulus-v1.3.2-64-universal-release-YYYYMMDD.apk +adb install "C:\path\to\formulus-v1.3.2-*-universal-release-*.apk" +``` + + + + +### Step 6: Verify Installation + + + + +```bash +# Check if app is installed +adb shell pm list packages | grep formulus + +# Launch the app +adb shell am start -n com.opendataensemble.formulus/.MainActivity +``` + + + + +```powershell +# Check if app is installed +adb shell pm list packages | Select-String formulus + +# Launch the app +adb shell am start -n com.opendataensemble.formulus/.MainActivity +``` + + + + +## Method 2: Android Emulator Installation + +### Step 1: Set Up Android Emulator + +#### Using Android Studio + +1. **Install Android Studio** from [developer.android.com](https://developer.android.com/studio) +2. **Open Android Studio** → **Tools** → **Device Manager** +3. **Create Virtual Device** → Select a device (e.g., Pixel 5) +4. **Select System Image** (e.g., Android 11, API 30) +5. **Finish** and start the emulator + +#### Using Command Line (Linux/macOS) + +```bash +# Install emulator via SDK Manager +sdkmanager "emulator" "platform-tools" "platforms;android-30" + +# List available system images +sdkmanager --list | grep system-images + +# Create AVD (Android Virtual Device) +avdmanager create avd -n formulus_emulator -k "system-images;android-30;google_apis;x86_64" + +# Start emulator +emulator -avd formulus_emulator & +``` + +### Step 2: Verify Emulator Connection + + + + +```bash +# Wait for emulator to boot (may take 1-2 minutes) +adb devices + +# Should show emulator +List of devices attached +emulator-5554 device +``` + + + + +```powershell +adb devices +``` + + + + +### Step 3: Install Formulus on Emulator + + + + +```bash +# Build APK first (if not already built) +cd formulus +pnpm run android + +# Install on emulator +adb -s emulator-5554 install android/app/build/outputs/apk/debug/app-debug.apk +``` + + + + +```powershell +# Build APK first +cd formulus +pnpm run android + +# Install on emulator +adb -s emulator-5554 install android\app\build\outputs\apk\debug\app-debug.apk +``` + + + + +### Step 4: Important Notes for Emulator + +When configuring Formulus on an emulator to connect to a local server: + +- **Use `10.0.2.2` instead of `localhost`** - This is the special IP that the emulator uses to access the host machine +- **Example server URL**: `http://10.0.2.2:8080` +- **For network servers**: Use the actual IP address or domain name + +## Method 3: Development Build with Hot Reload + +### Prerequisites + +- **Node.js** 20+ and **pnpm** 10.33.2 +- **React Native CLI** or **Expo CLI** +- **Java Development Kit (JDK)** 11 or higher +- **Android Studio** with Android SDK + +### Step 1: Install Dependencies + + + + +```bash +cd packages/tokens && pnpm install && pnpm run build && cd ../.. +cd formulus +pnpm install +``` + + + + +### Step 2: Start Metro Bundler + + + + +```bash +# In formulus directory +pnpm start +``` + +Keep this terminal open. Metro is the JavaScript bundler for React Native. + + + + +### Step 3: Build and Run on Device/Emulator + + + + +```bash +# For Android device (connected via USB) +pnpm run android + +# For Android emulator (must be running) +pnpm run android +``` + + + + +This command will: +1. Build the Android app +2. Install it on your device/emulator +3. Start the app +4. Connect to Metro bundler for hot reload + +### Step 4: Development Features + +With development build, you get: + +- **Hot Reload** - Changes reflect immediately +- **Fast Refresh** - React components update without losing state +- **Debug Menu** - Shake device or press `Ctrl+M` (Windows/Linux) or `Cmd+M` (macOS) +- **React Native Debugger** - Debug JavaScript code in Chrome DevTools + +## Common ADB Commands + +### Useful Commands for Development + +**List connected devices:** +```bash +adb devices +``` + +**Install APK:** +```bash +adb install path/to/app.apk +``` + +**Uninstall app:** +```bash +adb uninstall com.opendataensemble.formulus +``` + +**Reinstall app (uninstall + install):** +```bash +adb install -r path/to/app.apk +``` + +**View app logs:** +```bash +# All logs +adb logcat + +# Filter for Formulus only +adb logcat | grep -i formulus + +# Clear logs +adb logcat -c +``` + +**Pull file from device:** +```bash +adb pull /path/on/device /path/on/computer +``` + +**Push file to device:** +```bash +adb push /path/on/computer /path/on/device +``` + +**Open app:** +```bash +adb shell am start -n com.opendataensemble.formulus/.MainActivity +``` + +**Stop app:** +```bash +adb shell am force-stop com.opendataensemble.formulus +``` + +**Clear app data:** +```bash +adb shell pm clear com.opendataensemble.formulus +``` + +## Troubleshooting + +### Device Not Detected + +**Problem**: `adb devices` shows no devices. + +**Solutions**: +- Check USB cable connection +- Try a different USB port +- Install device-specific USB drivers (Windows) +- Enable USB debugging on device +- Accept USB debugging prompt on device +- Restart ADB server: `adb kill-server && adb start-server` + +### Installation Fails + +**Problem**: `adb install` fails with error. + +**Solutions**: +- Uninstall existing version first: `adb uninstall com.opendataensemble.formulus` +- Use `-r` flag to reinstall: `adb install -r app.apk` +- Check device has enough storage +- Verify APK is not corrupted +- Check device is in file transfer mode (not charging only) + +### Emulator Connection Issues + +**Problem**: Cannot connect to local server from emulator. + +**Solutions**: +- Use `10.0.2.2` instead of `localhost` for server URL +- Check firewall isn't blocking connections +- Verify server is running and accessible +- Use actual IP address instead of localhost + +### Permission Denied Errors + +**Problem**: ADB commands fail with permission errors. + + + + +```bash +# Add user to plugdev group +sudo usermod -a -G plugdev $USER + +# Create udev rules +sudo nano /etc/udev/rules.d/51-android.rules +# Add: SUBSYSTEM=="usb", ATTR{idVendor}=="####", MODE="0664", GROUP="plugdev" + +# Reload udev rules +sudo udevadm control --reload-rules +sudo udevadm trigger + +# Log out and log back in +``` + + + + +macOS typically doesn't require special permissions for ADB. If you encounter permission issues: + +1. Check USB cable connection +2. Try a different USB port +3. Restart ADB: `adb kill-server && adb start-server` +4. Check System Preferences → Security & Privacy for blocked apps + + + + +Windows typically handles USB device permissions automatically. If you encounter issues: + +1. Install device-specific USB drivers from manufacturer +2. Check Device Manager for unrecognized devices +3. Try different USB port or cable +4. Restart ADB: `adb kill-server && adb start-server` + + + + +## Related Documentation + +- [Formulus Development Setup](/docs/development/formulus-development) - Complete development environment setup +- [Formulus Component Reference](/docs/reference/formulus) - Detailed component documentation +- [Building and Testing](/docs/development/building-testing) - Build and test procedures + diff --git a/docs/docs/development/ode-desktop-development.md b/docs/docs/development/ode-desktop-development.md new file mode 100644 index 000000000..d0be21726 --- /dev/null +++ b/docs/docs/development/ode-desktop-development.md @@ -0,0 +1,118 @@ +--- +sidebar_position: 7 +--- + +# ODE Desktop Development + +Complete guide for developing **ODE Desktop** from the ODE monorepo. + +## Prerequisites + +- **Node.js** 20+ and **pnpm** 10+ +- **Rust** toolchain ([rustup](https://rustup.rs/)) +- Platform build tools per [Tauri prerequisites](https://v2.tauri.app/start/prerequisites/) + +Install shared design tokens before the desktop package: + +```bash +cd packages/tokens && pnpm install && pnpm run build && cd ../.. +``` + +## Local development setup + +```bash +cd desktop +pnpm install +pnpm tauri dev +``` + +:::warning Use the Tauri window + +`pnpm tauri dev` opens the **Tauri desktop window** with the Rust backend. Do not use a regular browser at `http://localhost:1420` — IPC commands such as `invoke` only work inside the Tauri shell. + +::: + +For frontend-only work without Tauri IPC: + +```bash +pnpm dev +``` + +This starts the Vite dev server only; most Data management and Workbench features require `pnpm tauri dev`. + +## Scripts + +| Script | Purpose | +|--------|---------| +| `pnpm dev` | Vite dev server (frontend only) | +| `pnpm build` | Typecheck + Vite production build | +| `pnpm build:formplayer` | Build `formulus-formplayer` and copy into `public/formplayer_dist/` | +| `pnpm build:tauri` | Prepare Formplayer assets, then build the desktop frontend | +| `pnpm tauri build` | Full desktop bundle (runs `build:tauri` first) | +| `pnpm tauri dev` | Development with hot reload in the Tauri shell | +| `pnpm lint` / `pnpm lint:fix` | ESLint | +| `pnpm format` / `pnpm format:check` | Prettier | +| `pnpm test` | Vitest unit and component tests | +| `pnpm typecheck` | `tsc --noEmit` | +| `pnpm copy:formplayer` | Copy existing `formulus-formplayer/build/` into `public/formplayer_dist/` | + +From `formulus-formplayer/`, `pnpm run build:copy` builds formplayer and copies assets to Formulus and ODE Desktop in one step. + +### Rust backend + +```bash +cd desktop/src-tauri +cargo test +cargo fmt +cargo clippy +``` + +## Formplayer integration + +ODE Desktop embeds the same formplayer bundle as Formulus. Production builds copy formplayer into `public/formplayer_dist/`. + +After changing formplayer or the Formulus bridge contract: + +1. Update `formulus/src/webview/FormulusInterfaceDefinition.ts` (source of truth). +2. Run `pnpm run sync-interface` in `formulus-formplayer`. +3. Rebuild and copy: `pnpm run build:copy` from `formulus-formplayer/`, or `pnpm build:formplayer` from `desktop/`. + +## OpenAPI client + +The desktop app regenerates a TypeScript Synkronus client from `synkronus/openapi/synkronus.yaml`: + +```bash +cd desktop +pnpm codegen:synk-client +``` + +CI fails if the generated client does not match the OpenAPI spec. + +## Developer mode (user-facing) + +To test a local custom app build against a profile workspace, use **Workbench developer mode**. See: + +- [ODE Desktop developer mode](/docs/guides/ode-desktop-developer-mode) — user guide +- [ODE Desktop reference](/docs/reference/ode-desktop) — `bundles/dev-local/` paths and bridge behavior + +## Project layout + +| Path | Role | +|------|------| +| `desktop/src/` | React UI (pages, components, store) | +| `desktop/src-tauri/` | Rust backend (SQLite, sync, bundle apply, dev mirror) | +| `desktop/public/formplayer_dist/` | Embedded formplayer bundle | +| `desktop/public/formulus-injection.js` | Bridge injection for custom app and form preview | + +## Contributing + +- Follow [Conventional Commits](https://www.conventionalcommits.org/) +- Run `pnpm lint`, `pnpm format:check`, and `pnpm test` before pushing +- PRs touching `desktop/**` trigger the ODE Desktop GitHub Actions workflow + +## Related documentation + +- [ODE Desktop reference](/docs/reference/ode-desktop) — component overview +- [Installing ODE Desktop](/docs/getting-started/installation/installing-ode-desktop) +- [Formplayer development](/development/formplayer-development) +- [Building and testing](/development/building-testing) diff --git a/docs/docs/development/quick-start.md b/docs/docs/development/quick-start.md new file mode 100644 index 000000000..4d6d5a607 --- /dev/null +++ b/docs/docs/development/quick-start.md @@ -0,0 +1,197 @@ +--- +sidebar_position: 5 +--- + +# Quick Start + +Get up and running with ODE in approximately 15 minutes. + +## Overview + +This guide walks you through setting up a complete ODE environment and creating your first form. + +## Step 1: Start Synkronus Server + +Using Docker Compose is the fastest method: + +```bash +# Clone the repository +git clone https://github.com/OpenDataEnsemble/ode.git +cd ode/synkronus + +# Start the server with Docker Compose +docker compose up -d +``` + +The server will be available at `http://localhost:8080`. + +## Step 2: Install Formulus App + + + + +Download and install the APK from the [releases page](https://github.com/OpenDataEnsemble/ode/releases), or build from source: + +```bash +cd ode/packages/tokens && pnpm install && pnpm run build && cd ../.. +cd ode/formulus +pnpm install +pnpm run android +``` + + + + +Build from source (requires macOS and Xcode): + +```bash +cd ode/packages/tokens && pnpm install && pnpm run build && cd ../.. +cd ode/formulus +pnpm install +cd ios && bundle install && bundle exec pod install && cd .. +pnpm run ios +``` + + + + +## Step 3: Configure the App + +1. Open the Formulus app +2. Navigate to Settings +3. Enter server URL: `http://your-server-ip:8080` (or `http://localhost:8080` for emulator) +4. Enter your credentials (create a user account first if needed) +5. Save the configuration + +## Step 4: Create Your First Form + +Forms are defined using JSON schema. Create a simple form: + +```json +{ + "type": "object", + "properties": { + "name": { + "type": "string", + "title": "Name" + }, + "age": { + "type": "integer", + "title": "Age", + "minimum": 0, + "maximum": 120 + } + }, + "required": ["name", "age"] +} +``` + +Upload this form to your Synkronus server using the API or CLI tool. + +## Step 5: Collect Data + +1. Open the Formulus app +2. Navigate to your form +3. Fill out the form fields +4. Submit the observation +5. The data will be stored locally and synchronized to the server + +## Step 6: Verify Data Collection + +Check that your observation was created: + + + + +```bash +curl http://localhost:8080/api/observations \ + -H "Authorization: Bearer YOUR_TOKEN" +``` + + + + +```bash +synk observations list +``` + + + + +1. Navigate to the Portal +2. Go to "Observations" +3. View your submitted observations + + + + +## Next Steps + +Now that you have a working setup: + +- Learn about [form design](/guides/form-design) to create more complex forms +- Explore [custom applications](/guides/custom-applications) for specialized workflows +- Review the [API reference](/reference/api) for integration options +- Read about [synchronization](/using/synchronization) to understand data flow + +## Troubleshooting + +### Server Not Accessible + +If the mobile app cannot connect to the server: + + + + +Verify the server is running: `curl http://localhost:8080/health` + +Check firewall settings + +Use your machine's IP address: `http://192.168.1.100:8080` + +Ensure device and server are on the same network + + + + +Use `10.0.2.2` instead of `localhost`: `http://10.0.2.2:8080` + +Verify the server is running on the host machine + +Check that port 8080 is accessible + + + + +Use `localhost` or your machine's IP address: `http://localhost:8080` + +Verify the server is running + +Check firewall settings + + + + +### Forms Not Appearing + +If forms don't appear in the app: + +- Verify the form was uploaded correctly +- Check that the app has synchronized with the server +- Review server logs for errors + +### Synchronization Issues + +If data is not synchronizing: + +- Check network connectivity +- Verify authentication credentials +- Review server logs for sync errors +- Ensure the observation was saved locally before sync + +## Related Documentation + +- [Installation Guide](/docs/getting-started/installation) +- [Your First Form](/using/your-first-form) +- [Form Design Guide](/guides/form-design) + diff --git a/docs/docs/development/setup.md b/docs/docs/development/setup.md new file mode 100644 index 000000000..2b5940e65 --- /dev/null +++ b/docs/docs/development/setup.md @@ -0,0 +1,331 @@ +--- +sidebar_position: 1 +--- + +# Development Setup + +Complete guide to setting up a development environment for ODE. + +## Prerequisites + +Before setting up the development environment, ensure you have: + +- **Node.js** 20.0 or higher +- **pnpm** 10.33.2 (ODE pins this via `packageManager` in each package; enable with [Corepack](https://nodejs.org/api/corepack.html): `corepack enable && corepack prepare pnpm@10.33.2 --activate`) +- **Go** 1.24 or higher (for server and CLI development) +- **PostgreSQL** 13.0 or higher (for server development) +- **Git** for version control +- **Docker** and **Docker Compose** (optional, for containerized development) + +## Repository Structure + +ODE is a monorepo containing multiple components: + +``` +ode/ +├── formulus/ # React Native mobile app +├── formulus-formplayer/ # React web form renderer +├── synkronus/ # Go backend server +├── synkronus-cli/ # Go command-line utility +├── synkronus-portal/ # React web portal +└── packages/ + ├── tokens/ # Design tokens package + └── components/ # Shared UI components +``` + +## Package manager (pnpm) + +ODE JavaScript/TypeScript packages use **pnpm** (`pnpm@10.33.2` via Corepack) with a **per-package** `pnpm-lock.yaml` (there is no root workspace install). Run `pnpm install` inside each component directory you work on. + +**Recommended install order** when setting up several components: + +1. `packages/tokens` — `pnpm install` then `pnpm run build` +2. Consumers — `pnpm install` in `formulus-formplayer`, `formulus`, `packages/components`, `synkronus-portal`, or `desktop` as needed + +CI and Docker use `pnpm install --frozen-lockfile` for reproducible installs. + +## Clone the Repository + +```bash +git clone https://github.com/OpenDataEnsemble/ode.git +cd ode +``` + +## Formulus Development + +### Setup + +```bash +cd packages/tokens && pnpm install && pnpm run build && cd ../.. +cd formulus +pnpm install +``` + +Android builds require the Notifee native core (gitignored). `pnpm run android` runs `preandroid` to vendor it automatically; or run `pnpm run vendor:notifee` before `./gradlew` directly. + +### Running + +```bash +# Start Metro bundler +pnpm start + +# Run on Android +pnpm run android + +# Run on iOS (macOS only) +pnpm run ios +``` + +### Code Quality + +ODE enforces consistent formatting and linting: + +```bash +# Run linting +pnpm run lint + +# Run linting with auto-fix +pnpm run lint:fix + +# Format code +pnpm run format + +# Check formatting (no writes) +pnpm run format:check +``` + +### Generating Files + +```bash +# Generate WebView injection script +pnpm run generate + +# Generate API client from OpenAPI spec +pnpm run generate:api +``` + +## Formplayer Development + +### Setup + +```bash +cd packages/tokens && pnpm install && pnpm run build && cd ../.. +cd formulus-formplayer +pnpm install +``` + +### Running + +```bash +# Development server +pnpm start + +# Build and copy assets into Formulus (and ODE Desktop) +pnpm run build:copy +``` + +### Code Quality + +```bash +# Run linting +pnpm run lint + +# Run linting with auto-fix +pnpm run lint:fix + +# Format code +pnpm run format + +# Check formatting (no writes) +pnpm run format:check +``` + +## ODE Desktop Development + +See [ODE Desktop Development](/docs/development/ode-desktop-development) for setup, scripts, and Formplayer integration. + +For testing a local custom app build in the Workbench, see [ODE Desktop developer mode](/docs/guides/ode-desktop-developer-mode). + +## Synkronus Development + +### Setup + +```bash +cd synkronus +go mod download +``` + +### Configuration + +Create a `.env` file: + +```bash +PORT=8080 +DB_CONNECTION=postgres://synkronus:password@localhost:5432/synkronus?sslmode=disable +JWT_SECRET=your-secret-key-for-development +LOG_LEVEL=debug +APP_BUNDLE_PATH=./data/app-bundles +``` + +### Running + +```bash +# Build +go build -o bin/synkronus cmd/synkronus/main.go + +# Run +./bin/synkronus + +# Or run directly +go run cmd/synkronus/main.go +``` + +### Database Setup + +Ensure PostgreSQL is running and create a database: + +```sql +CREATE DATABASE synkronus; +``` + +The schema will be created automatically on first run. + +## Synkronus CLI Development + +### Setup + +```bash +cd synkronus-cli +go mod download +``` + +### Building + +```bash +# Build +go build -o bin/synk ./cmd/synkronus + +# Run +./bin/synk +``` + +## Development Workflow + +### 1. Create a Feature Branch + +```bash +git checkout -b feature/your-feature-name +``` + +### 2. Make Changes + +Make your code changes following the coding standards. + +### 3. Test Locally + +```bash +# Run tests (from each package directory) +cd formulus && pnpm run test --ci --coverage --watchAll=false +cd formulus-formplayer && pnpm run test run +go test ./... # For Go projects (from synkronus/, etc.) + +# Check code quality +pnpm run lint +pnpm run format:check +``` + +### 4. Commit Changes + +```bash +git add . +git commit -m "Description of changes" +``` + +### 5. Push and Create Pull Request + +```bash +git push origin feature/your-feature-name +``` + +Create a pull request on GitHub. + +## Code Quality Standards + +### Frontend (React/React Native) + +- **Linting**: ESLint with project-specific rules +- **Formatting**: Prettier with consistent configuration +- **TypeScript**: Strict type checking enabled +- **Testing**: Jest for unit tests + +### Backend (Go) + +- **Formatting**: `gofmt` or `goimports` +- **Linting**: `golangci-lint` (if configured) +- **Testing**: Standard Go testing package +- **Documentation**: Godoc comments for exported functions + +### CI/CD + +The CI pipeline automatically: + +- Runs linting and formatting checks +- Runs tests +- Builds components +- Publishes Docker images (for synkronus) + +## Development Tools + +### Recommended IDE Setup + +- **VS Code**: With extensions for TypeScript, Go, and React +- **IntelliJ IDEA**: With Go and JavaScript plugins +- **Android Studio**: For Android development +- **Xcode**: For iOS development (macOS only) + +### Useful Commands + +```bash +# Check all components +cd formulus && pnpm run lint && cd .. +cd formulus-formplayer && pnpm run lint && cd .. +cd synkronus && go test ./... && cd .. + +# Format all code +cd formulus && pnpm run format && cd .. +cd formulus-formplayer && pnpm run format && cd .. +cd synkronus && go fmt ./... && cd .. +``` + +## Troubleshooting + +### Node Modules Issues + +```bash +# Clear and reinstall (run inside the package directory, e.g. formulus/) +rm -rf node_modules +pnpm install +``` + +If dependencies are missing after clone, ensure you ran `pnpm install` in that package directory (and built `packages/tokens` first when using `@ode/tokens` or `@ode/components`). + +### Go Module Issues + +```bash +# Clean module cache +go clean -modcache +go mod download +``` + +### Database Connection Issues + +- Verify PostgreSQL is running +- Check connection string format +- Ensure database exists +- Verify user permissions + +## Related Documentation + +- [Architecture Overview](/development/architecture) +- [Contributing Guide](/development/contributing) +- [Building & Testing](/development/building-testing) diff --git a/docs/docs/development/synkronus-development.md b/docs/docs/development/synkronus-development.md new file mode 100644 index 000000000..ce542d47f --- /dev/null +++ b/docs/docs/development/synkronus-development.md @@ -0,0 +1,183 @@ +--- +sidebar_position: 5 +--- + +# Synkronus Server Development + +Complete guide for developing the Synkronus server component. + +## Prerequisites + +- **Go** 1.22+ +- **PostgreSQL** 12+ +- **Git** + +## Local Development Setup + +### Step 1: Clone Repository + +```bash +git clone https://github.com/OpenDataEnsemble/ode.git +cd ode/synkronus +``` + +### Step 2: Install Dependencies + +```bash +go mod download +``` + +### Step 3: Set Up Database + + + + +```bash +# Create database +createdb synkronus + +# Or using psql +psql -U postgres -c "CREATE DATABASE synkronus;" +``` + + + + +Using PowerShell or Command Prompt: + +```powershell +# Using psql +psql -U postgres -c "CREATE DATABASE synkronus;" +``` + +Or using Git Bash/WSL: + +```bash +# Create database +createdb synkronus + +# Or using psql +psql -U postgres -c "CREATE DATABASE synkronus;" +``` + + + + +### Step 4: Configure Environment + +Create `.env` file: + +```bash +PORT=8080 +DB_CONNECTION=postgres://user:password@localhost:5432/synkronus?sslmode=disable +JWT_SECRET=dev-secret-change-in-production +LOG_LEVEL=debug +APP_BUNDLE_PATH=./data/app-bundles +MAX_VERSIONS_KEPT=5 +ADMIN_USERNAME=admin +ADMIN_PASSWORD=admin +``` + +### Step 5: Create Directories + +```bash +mkdir -p data/app-bundles +``` + +### Step 6: Run Server + +```bash +go run cmd/synkronus/main.go +``` + +Or build and run: + +```bash +go build -o bin/synkronus cmd/synkronus/main.go +./bin/synkronus +``` + +## Development Workflow + +### Hot Reload + +Use tools like `air` for hot reload: + +```bash +go install github.com/cosmtrek/air@latest +air +``` + +### Testing + +```bash +go test ./... +``` + +### API Documentation + +View OpenAPI docs: + +```bash +# Server must be running +open http://localhost:8080/openapi/swagger-ui.html +``` + +## Building for Production + +### Build Binary + +```bash +go build -o bin/synkronus cmd/synkronus/main.go +``` + +### Cross-Platform Builds + + + + +```bash +GOOS=linux GOARCH=amd64 go build -o bin/synkronus-linux cmd/synkronus/main.go +``` + + + + +```powershell +$env:GOOS="windows"; $env:GOARCH="amd64"; go build -o bin/synkronus.exe cmd/synkronus/main.go +``` + +Or using bash (Git Bash/WSL): + +```bash +GOOS=windows GOARCH=amd64 go build -o bin/synkronus.exe cmd/synkronus/main.go +``` + + + + +```bash +GOOS=darwin GOARCH=amd64 go build -o bin/synkronus-macos cmd/synkronus/main.go +``` + + + + +## Docker Development + +### Using Docker Compose + +```bash +docker compose up -d +``` + +### Development with Hot Reload + +Mount source code for live updates. + +## Related Documentation + +- [Synkronus Server Reference](/reference/synkronus-server) - Component reference +- [Deployment Guide](/guides/deployment) - Production deployment +- [Configuration Guide](/guides/configuration) - Configuration options + diff --git a/docs/docs/development/synkronus-portal-development.md b/docs/docs/development/synkronus-portal-development.md new file mode 100644 index 000000000..d06308851 --- /dev/null +++ b/docs/docs/development/synkronus-portal-development.md @@ -0,0 +1,106 @@ +--- +sidebar_position: 6 +--- + +# Synkronus Portal Development + +Complete guide for developing the Synkronus Portal web interface. + +## Prerequisites + +- **Node.js** 20+ and **pnpm** 10.33.2 +- **Go** 1.22+ (for backend) +- **PostgreSQL** 17+ (for backend) + +Install shared packages before the portal: + +```bash +cd packages/tokens && pnpm install && pnpm run build && cd ../.. +cd packages/components && pnpm install && cd ../.. +``` + +## Local Development Setup + + + + +#### Step 1: Set Up Backend + +See [Synkronus Development](/development/synkronus-development) for backend setup. + +#### Step 2: Set Up Frontend + +```bash +cd synkronus-portal +pnpm install +``` + +#### Step 3: Start Development Server + +```bash +pnpm run dev +``` + +Portal available at http://localhost:5174 + + + + +#### Start Backend Services + +```bash +docker compose up -d postgres synkronus +``` + +#### Start Frontend + +```bash +cd synkronus-portal +pnpm install +pnpm run dev +``` + +Portal available at http://localhost:5174 + + + + +## Development Features + +- **Hot Module Replacement**: Instant code updates +- **Fast Refresh**: React components update without losing state +- **Source Maps**: Debug in browser DevTools +- **Error Overlay**: Errors shown in browser + +## Building for Production + +### Build + +```bash +pnpm run build +``` + +`prebuild` runs OpenAPI client generation from `../synkronus/openapi/synkronus.yaml`. Output is in `dist/`. + +### Docker Production Build + +```bash +docker compose up -d --build +``` + +## Project Structure + +- `src/`: React source code +- `src/components/`: Reusable components +- `src/pages/`: Page components +- `src/services/`: API service +- `src/contexts/`: React contexts + +## Adding Features + +See [Synkronus Portal Reference](/reference/synkronus-portal) for detailed patterns. + +## Related Documentation + +- [Synkronus Portal Reference](/reference/synkronus-portal) - Component reference +- [Deployment Guide](/guides/deployment) - Production deployment diff --git a/docs/docs/getting-started/.gitkeep b/docs/docs/getting-started/.gitkeep new file mode 100644 index 000000000..e69de29bb diff --git a/docs/docs/getting-started/architecture-overview.md b/docs/docs/getting-started/architecture-overview.md new file mode 100644 index 000000000..944787ec7 --- /dev/null +++ b/docs/docs/getting-started/architecture-overview.md @@ -0,0 +1,196 @@ +--- +sidebar_position: 1 +--- + +# Architecture Overview + +ODE (Open Data Ensemble) is a comprehensive platform for mobile data collection and synchronization. This guide explains the core architecture and components. + +> **Current ODE release:** [v1.3.2](https://github.com/OpenDataEnsemble/ode/releases/tag/v1.3.2) · [Downloads](/downloads) + +## Core Components + +ODE consists of these main components that work together to provide a complete data collection solution: + +### Synkronus Server +The backend server responsible for: +- **API Services**: RESTful APIs for data management +- **Data Synchronization**: Offline-first sync capabilities +- **User Authentication**: JWT-based authentication +- **App Bundle Management**: Upload and version management of custom applications +- **Embedded Portal**: Web interface built directly into the Go binary + +### Formulus Mobile App +React Native mobile application for: +- **Data Collection**: Field data capture with rich form support +- **Offline Storage**: Local database with WatermelonDB +- **Camera Integration**: Photo and document capture +- **GPS Location**: Geotagging and location services +- **QR Code Scanning**: Workflow automation + +### Formulus Formplayer +React web application that: +- **Renders Forms**: Displays JSON Schema forms in web browsers +- **Form Validation**: Client-side validation with JSON Schema +- **Embedded in Mobile**: Used within the React Native app via WebView + +### Synkronus CLI +Command-line utility for: +- **App Management**: Upload and manage custom applications +- **Data Export**: Export data to Parquet format +- **User Administration**: Manage users and permissions +- **Server Administration**: Maintenance and monitoring tasks + +### ODE Desktop + +Desktop application (Tauri + React + Rust) for data stewardship and app development: + +- **Data management**: Pull, inspect, edit, import, and sync observations against Synkronus +- **Forms / app workbench**: Download app bundles, preview forms, and test custom apps +- **Developer mode**: Mirror a local custom app build without replacing the Synk-downloaded bundle +- **Same public API**: Uses Synkronus REST API — no privileged desktop channel + +Introduced in **ODE v1.1.0**. See [ODE Desktop reference](/docs/reference/ode-desktop) and [install guide](/docs/getting-started/installation/installing-ode-desktop). + +### Synkronus Portal + +Web-based administrative interface (also embedded in the Synkronus binary at `/portal`): + +- **User and bundle management** +- **Observation viewing and export** +- **Same API** as Formulus, CLI, and ODE Desktop + +## Architecture Patterns + +### Embedded Portal Architecture + +The Synkronus Portal is a React application that gets **embedded directly into the Go binary** during the build process: + +``` +┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐ +│ React Portal │ -> │ Go Build │ -> │ Single Binary │ +│ (Source) │ │ (Embed) │ │ (API + Portal) │ +└─────────────────┘ └──────────────────┘ └─────────────────┘ +``` + +**Build Process:** +1. Portal built with `pnpm run build` (in `synkronus-portal/`) +2. Build output copied to `synkronus/portal/dist/` +3. Go binary embeds portal using `//go:embed` +4. Portal served at `/portal` route + +**Benefits:** +- Single deployment artifact +- No separate web server needed +- Version alignment between API and UI +- Simplified operations + +### Component Library System + +ODE includes a shared component library for consistent UI across applications: + +``` +packages/ +├── tokens/ # Design tokens (colors, typography, spacing) +└── components/ # React components (web and native) +``` + +**Features:** +- **Cross-platform**: Works in React Native and React Web +- **Design Tokens**: Centralized design system +- **TypeScript**: Full type safety +- **Theme Support**: Light/dark mode capabilities + +### Custom Application Architecture + +Custom applications are React apps that run inside the Formulus container: + +``` +Custom App Structure: +├── app.config.json # App configuration and theme +├── forms/ # Form definitions (JSON Schema + UI Schema) +├── src/ +│ ├── components/ # Reusable components +│ ├── screens/ # Main screens +│ └── theme.js # Theme generation from config +└── scripts/ # Build and validation scripts +``` + +**Key Features:** +- **Configuration-driven**: `app.config.json` controls app behavior +- **Form Integration**: Seamless integration with Formulus form system +- **Local observation indexes**: Device-only index table for fast queries (never synced to Synkronus) +- **Theme System**: Material Design 3 based theming +- **Build Process**: Optimized bundle creation for deployment + +Study-specific apps (for example AnthroCollect) are **app bundles** uploaded to Synkronus—not separate backend services. IT hosts only Synkronus, PostgreSQL, and a TLS reverse proxy. + +## Server deployment view + +![Installation view: reverse proxy, Synkronus container, PostgreSQL, and volumes](/img/it-architecture-installation.svg) + +For IT departments: see [Server Architecture for IT](/docs/guides/server-architecture-for-it) (full diagrams, security model, backups, sizing). + +## Data Flow + +### Form Submission Flow +``` +Mobile App -> Local Storage -> Sync Server -> PostgreSQL -> Portal Display +``` + +### App Bundle Deployment +``` +Development -> Build -> ZIP Upload -> Server Storage -> Mobile Download +``` + +## Technology Stack + +### Backend (Synkronus) +- **Language**: Go 1.24+ +- **Database**: PostgreSQL 13+ +- **Authentication**: JWT tokens +- **API**: REST with OpenAPI specification + +### Frontend Components +- **Mobile**: React Native 0.83+ with React 19.2+ +- **Web**: React 19.2+ with TypeScript +- **UI Framework**: Material Design 3 components +- **Build Tools**: Vite for web apps + +### Development Tools +- **Package Manager**: pnpm (per-package lockfiles in the ODE monorepo) +- **Code Quality**: ESLint, Prettier, TypeScript +- **Testing**: Jest for unit tests +- **Containerization**: Docker with multi-stage builds + +## Deployment Models + +### Single Container Deployment +- **Synkronus**: Go server with embedded portal (`ghcr.io/opendataensemble/synkronus`) +- **Database**: PostgreSQL (container or managed service) +- **Reverse proxy**: TLS termination (Caddy in [synkronus-quickstart](https://github.com/OpenDataEnsemble/synkronus-quickstart); or your institutional proxy) +- **Mobile apps**: Formulus on field devices via Obtainium or F-Droid + +See [Server Architecture for IT](/docs/guides/server-architecture-for-it) and [Deployment guide](/docs/guides/deployment). + +### Development Setup +- **Local Development**: Docker Compose with hot reload +- **Component Development**: Shared library development +- **Form Development**: JSON Schema with validation tools + +## Security Considerations + +- **Authentication**: JWT-based with configurable expiration +- **Data encryption in transit**: TLS at your reverse proxy (required in production) +- **Data encryption at rest**: Host platform responsibility (see [Security reference](/docs/reference/security)) +- **Input Validation**: JSON Schema validation for all forms +- **Access Control**: Role-based permissions for API endpoints + +## Performance Optimizations + +- **Offline-First**: Local storage with sync capabilities +- **Bundle Optimization**: Minimized custom app bundles +- **Database Indexing**: Optimized queries for large datasets +- **Caching**: Built-in caching for frequently accessed data + +This architecture enables scalable, maintainable data collection applications that work seamlessly across mobile and web platforms. diff --git a/docs/docs/getting-started/faq.md b/docs/docs/getting-started/faq.md new file mode 100644 index 000000000..0e7bba6be --- /dev/null +++ b/docs/docs/getting-started/faq.md @@ -0,0 +1,116 @@ +--- +sidebar_position: 5 +--- + +# Frequently Asked Questions + +Common questions about ODE installation, usage, and development. + +> **Current ODE release:** [v1.3.2](https://github.com/OpenDataEnsemble/ode/releases/tag/v1.3.2) (Synkronus container, Formulus, Desktop, CLI, Portal) · [Downloads](/downloads) + +## General Questions + +### How do you pronounce ODE? + +ODE is pronounced like "code", without the "C." + +### What is ODE? + +Open Data Ensemble (ODE) is a platform for mobile data collection and synchronization. It provides tools for creating forms, collecting data offline, and synchronizing data across devices and servers. + +### Is ODE free to use? + +Yes, ODE is open source and free to use. The code is available under the MIT license. + +### What platforms does ODE support? + +ODE supports Android and iOS mobile devices. The server component runs on Linux, macOS, and Windows. + +### Do I need internet connectivity to use ODE? + +No, ODE is designed to work offline. Data is stored locally and synchronized when connectivity is available. + +## Installation Questions + +### What are the system requirements? + +See the [Installation](/docs/getting-started/installation) page for detailed system requirements. + +### Can I run ODE in the cloud? + +Yes, ODE can be deployed to cloud platforms such as AWS, Google Cloud, or Azure. See the [Deployment guide](/docs/guides/deployment) and [Server Architecture for IT](/docs/guides/server-architecture-for-it). + +### Do I need a database? + +Yes, ODE requires PostgreSQL for data storage. The database schema is created automatically on first run. + +## Usage Questions + +### How do I create forms? + +Forms are defined using JSON schema. See the [Form Design guide](/docs/guides/form-design) for details. + +### Can I customize the user interface? + +Yes, ODE supports custom applications and renderers. See [Custom Applications](/guides/custom-apps/overview) for details. + +### How does synchronization work? + +ODE uses a bidirectional sync protocol that pushes local data to the server and pulls new data from the server. See [Synchronization](/using/synchronization) for details. + +### What happens if two devices modify the same data? + +ODE automatically resolves conflicts using a version-based approach. The most recent version takes precedence. + +## Development Questions + +### How do I contribute to ODE? + +See the [Contributing guide](/development/contributing/guide) for information on how to contribute. + +### Can I extend ODE functionality? + +Yes, ODE is designed to be extensible. See [Extending ODE](/development/extending/overview) for details. + +### What programming languages are used? + +ODE uses React Native for mobile apps, React for web components, and Go for the server. + +## Troubleshooting + +### The app won't connect to the server + +- Verify the server is running and accessible +- Check the server URL in app settings +- Verify firewall and network settings +- For Android emulator, use `10.0.2.2` instead of `localhost` + +### Forms are not appearing + +- Verify forms were uploaded to the server +- Check that the app has synchronized +- Review server logs for errors + +### Data is not synchronizing + +- Check network connectivity +- Verify authentication credentials +- Review server logs for sync errors +- Ensure observations were saved locally + +### Build errors + +- Ensure all prerequisites are installed +- Check that dependencies are up to date +- Review error messages for specific issues +- See component-specific documentation for build instructions + +## Getting Help + +If you cannot find an answer to your question: + +- Check the [Troubleshooting guide](/using/troubleshooting) +- Review the [API Reference](/reference/api/overview) +- Search [GitHub Issues](https://github.com/OpenDataEnsemble/ode/issues) +- Ask questions in the [Community section](/community/getting-help) + diff --git a/docs/docs/getting-started/index.md b/docs/docs/getting-started/index.md new file mode 100644 index 000000000..d0e736a37 --- /dev/null +++ b/docs/docs/getting-started/index.md @@ -0,0 +1,85 @@ +--- +sidebar_position: 0 +--- + +# Getting Started + +Welcome to ODE! This section will help you understand what ODE is, why you should use it, and how to get up and running quickly. + +## What You'll Learn + +Whether you're a researcher, developer, or data practitioner, these guides will help you get started with ODE and start collecting data efficiently. + +
    +
    +
    +
    +

    What is ODE?

    +
    +
    +

    Learn about the Open Data Ensemble platform and its architecture.

    + Get Started → +
    +
    +
    +
    +
    +
    +

    Why ODE?

    +
    +
    +

    Discover the benefits and advantages of using ODE.

    + Learn More → +
    +
    +
    +
    +
    +
    +

    Key Concepts

    +
    +
    +

    Understand fundamental concepts and terminology.

    + Read Guide → +
    +
    +
    +
    +
    +
    +

    Installation

    +
    +
    +

    Step-by-step instructions to install ODE components.

    + Install Now → +
    +
    +
    +
    +
    +
    +

    Quick Start

    +
    +
    +

    Get up and running quickly with a simple example.

    + Start Here → +
    +
    +
    +
    +
    +
    +

    FAQ

    +
    +
    +

    Find answers to frequently asked questions.

    + View FAQ → +
    +
    +
    +
    + +## Next Steps + +Once you've completed the getting started guides, explore the [Using ODE](/docs/using/your-first-form) section to learn how to create forms and manage data. + diff --git a/docs/docs/getting-started/installation.md b/docs/docs/getting-started/installation.md new file mode 100644 index 000000000..d9fd4bae0 --- /dev/null +++ b/docs/docs/getting-started/installation.md @@ -0,0 +1,26 @@ +--- +sidebar_position: 4 +--- + +# Installation + +To run ODE you need two things: a **server** (Synkronus) that stores and syncs data, and a **client** on each device (Formulus or another app) that collects data and talks to the server. + +## What to install + +| Component | What it is | Guide | +|-----------|------------|--------| +| **Server (Synkronus)** | Backend that hosts the API, portal, and database. Runs on a Linux server or VPS. | [Install Synkronus](installation/installing-synkronus) | +| **Client (Formulus)** | Mobile app for Android and iOS that field workers use to fill forms and sync data. | [Install Formulus](installation/installing-formulus) | + +Install the server first so that the client has something to connect to. Then install Formulus (or your client app) on each device and point it at your Synkronus server. + +## For IT / infrastructure teams + +Hosting Synkronus for a study? See **[Server Architecture for IT](/docs/guides/server-architecture-for-it)** for a one-page overview: container layout, TLS, backups, and how custom apps (app bundles) relate to the server. Current platform release: **v1.3.2**. + +## Next steps + +- **[Install Synkronus](installation/installing-synkronus)** — Set up the server on a Linux machine or VPS. +- **[Downloads](/downloads)** — Get Formulus, ODE Desktop, or the Synkronus CLI for your platform. +- **[Install Formulus](installation/installing-formulus)** — Put the Formulus app on Android or iOS devices and connect it to your server. diff --git a/docs/docs/getting-started/installation/.gitkeep b/docs/docs/getting-started/installation/.gitkeep new file mode 100644 index 000000000..e69de29bb diff --git a/docs/docs/getting-started/installation/installing-formulus.md b/docs/docs/getting-started/installation/installing-formulus.md new file mode 100644 index 000000000..be08f4569 --- /dev/null +++ b/docs/docs/getting-started/installation/installing-formulus.md @@ -0,0 +1,322 @@ +--- +sidebar_position: 2 +--- + +# Installing Formulus App + +Complete guide for installing the Formulus mobile application on Android and iOS devices. For the current links, see [Downloads](/downloads). + +## Overview + +Formulus is available for Android and iOS. Choose the method that best fits your device: + +- **F-Droid** (recommended for Android) - Install Formulus directly from [F-Droid](https://f-droid.org/en/packages/org.opendataensemble.formulus/) +- **Obtainium** (Android) - Installs Formulus from GitHub releases with automatic updates. +- **Direct APK** (Android) - Download the current APK from [Downloads](/downloads) or [GitHub releases](https://github.com/OpenDataEnsemble/ode/releases). +- **App Store** (iPhone/iPad) - Install Formulus from the [Apple App Store](https://apps.apple.com/dk/app/formulus/id6798318215). +- **Development Build** - For developers who want to build from source + +## System Requirements + +Before installing, ensure your device meets these requirements: + +| Requirement | Minimum | +|-------------|---------| +| **Android Version** | Android 7.0 (API level 24) or higher | +| **iOS Version** | iOS 15.1 or higher | +| **Storage Space** | 50 MB free space | +| **Internet Connection** | Required for initial setup and synchronization | +| **Permissions** | Camera, Storage, Location (for form features) | + +## Installation Methods + +### Method 1: Obtainium (Recommended) + +Obtainium is the recommended method for installing Formulus. It allows you to install and update Formulus directly from GitHub releases, including pre-release versions. You can install Obtainium via F-Droid or by downloading it directly. + +#### Step 1: Install Obtainium + +You have two options to install Obtainium: + +##### Option A: Install Obtainium via F-Droid (Recommended) + +1. **Open your web browser** and navigate to [f-droid.org](https://f-droid.org/) +2. **Locate the "DOWNLOAD F-DROID" button** on the main page +3. **Tap the "DOWNLOAD F-DROID" button** to download the F-Droid APK file +4. **Enable installation from unknown sources**: + - Go to **Settings** → **Security** → **Unknown Sources** (or **Apps** → **Special app access** → **Install unknown apps** on newer Android versions) + - Select your browser (Chrome, Firefox, etc.) from the list + - **Enable "Allow from this source"** toggle switch + - Read and acknowledge the security warning about installing apps from unknown sources +5. **Return to your browser** and open the downloaded F-Droid APK file +6. **Tap "Install"** when prompted +7. **Wait for installation** to complete, then tap **"Open"** to launch F-Droid + +![F-Droid Download Page](/img/installation/f-droid-download.png) + +8. **Open F-Droid** and search for "Obtainium" +9. **Tap on Obtainium** in the search results +10. **Tap "Install"** to install Obtainium +11. **Wait for installation** to complete + +##### Option B: Install Obtainium Directly from GitHub + +1. **Download Obtainium** from the [Obtainium releases page](https://github.com/ImranR98/Obtainium/releases) +2. **Enable installation from unknown sources**: + - Go to **Settings** → **Security** → **Unknown Sources** (or **Apps** → **Special app access** → **Install unknown apps** on newer Android versions) + - Select your browser from the list + - **Enable "Allow from this source"** toggle switch + - Acknowledge the security warning +3. **Install Obtainium** by opening the downloaded APK file +4. **Tap "Install"** when prompted in the installation confirmation dialog +5. **Wait for installation** to complete + +![Obtainium Installation](/img/installation/obtainium-install.png) + +#### Step 2: Add ODE Repository to Obtainium + +1. **Open Obtainium** on your device +2. **Navigate to the "Add app" tab** in the bottom navigation bar (indicated by a "+" icon) +3. **Enter the GitHub repository URL** in the "App source URL" field: + - The field should contain: `https://github.com/OpenDataEnsemble/ode` + - Ensure the full URL is entered correctly +4. **Configure GitHub options** (for pre-release testing only): + - **Enable "Include prereleases"** if you need alpha/beta builds + - **Enable "Fallback to older releases"** if newer releases are unavailable +5. **Tap the "Add" button** to save the repository +6. **Wait for Obtainium to fetch** the repository information and available releases + +![Obtainium Add App Screen](/img/installation/obtainium-add-app.png) + +**Stable release:** Install **v1.3.2** (or the latest [GitHub release](https://github.com/OpenDataEnsemble/ode/releases)). Pre-release toggles are only needed for alpha/beta testing. + +#### Step 3: Install Formulus + +1. **Open Obtainium** and navigate to the **"Apps"** tab +2. **Find "ode"** in your apps list (it should appear after adding the repository) +3. **Tap on "ode"** to open the app details page +4. **Review the app information**: + - App name: **ode** + - Developer: **OpenDataEnsemble** + - Package: `org.opendataensemble.formulus` + - Latest version: **v1.3.2** (or current [release](https://github.com/OpenDataEnsemble/ode/releases)) + - Status: **Not installed** +5. **Tap the "Install" button** at the bottom of the screen +6. **Confirm installation** when prompted: + - A dialog will appear showing the **Formulus** app icon and asking "Do you want to install this app?" + - **Tap "Install"** in the confirmation dialog (the "Cancel" button is on the left) +7. **Wait for download and installation** to complete +8. **Find Formulus** in your app drawer or search for it: + - Open your app drawer or launcher + - Search for "Formulus" or "formulu" + - The app icon shows a yellow giraffe head with a green background and white clipboard + - **Tap on Formulus** to launch the app + +![ODE App Details in Obtainium](/img/installation/obtainium-ode-details.png) + +![Formulus Installation Dialog](/img/installation/formulus-install-dialog.png) + +![Formulus App Search](/img/installation/formulus-app-search.png) + +#### Automatic Updates + +Obtainium will automatically check for updates: + +1. **Open Obtainium** and navigate to the **"Apps"** tab +2. **Apps with available updates** will be marked +3. **Tap on "ode"** to see update details +4. **Tap "Update"** or **"Install"** to install the new version +5. **Confirm the update** when prompted +6. **App data is preserved** during update + +### Method 2: F-Droid (recommended for Android) + +Install Formulus directly from F-Droid (no Obtainium required): + +1. Install the [F-Droid](https://f-droid.org/) client if you do not already have it +2. Open F-Droid and search for **Formulus** (package `org.opendataensemble.formulus`) +3. Tap **Install** and wait for the download to complete +4. Updates are available through F-Droid when a new version is published + +### Method 3: Direct APK Installation (Android) + +If Obtainium is not available or you prefer direct installation: + +#### Step 1: Download the APK + +1. **Download the latest APK** from [Downloads](/downloads) or the [releases page](https://github.com/OpenDataEnsemble/ode/releases) +2. **Save the file** to your device's Downloads folder + +#### Step 2: Enable Unknown Sources + +1. **Go to Settings** → **Security** (or **Apps** → **Special app access** → **Install unknown apps** on newer Android versions) +2. **Select your browser or file manager** from the list (e.g., Chrome, Firefox, Files) +3. **Enable "Allow from this source"** toggle switch +4. **Read and acknowledge** the security warning: + > "Your phone and personal data are more vulnerable to attack by unknown apps. By installing apps from this source, you agree that you are responsible for any damage to your phone or loss of data that may result from their use." + +![Enable Unknown Sources](/img/installation/enable-unknown-sources.png) + +#### Step 3: Install the APK + +1. **Open your file manager** or Downloads app +2. **Navigate to the Downloads folder** +3. **Tap on the Formulus APK file** +4. **Review the permissions** requested by the app +5. **Tap "Install"** to begin installation +6. **Wait for installation** to complete +7. **Tap "Open"** to launch the app + +### Method 4: App Store (iPhone and iPad) + +1. Open the [Formulus App Store page](https://apps.apple.com/dk/app/formulus/id6798318215) on your iPhone or iPad. +2. Tap **Get**, then authenticate with Face ID, Touch ID, or your Apple ID. +3. Wait for Formulus to install, then open it from your home screen. + +### Method 5: Development Build + +For developers who want to build and install from source, see the [Development Installation Guide](/docs/development/formulus-development). + +## Post-Installation Setup + +After installing Formulus, you need to configure it to connect to your Synkronus server: + +### Initial Configuration + +1. **Open Formulus** on your device +2. **You'll see the welcome screen** with configuration options +3. **Choose your configuration method**: + - **QR Code Scan** (Recommended) - Scan a QR code with server details + - **Manual Entry** - Enter server URL and credentials manually + +### QR Code Configuration + +1. **Tap "Scan QR Code"** on the welcome screen +2. **Grant camera permission** if prompted +3. **Point the camera** at the QR code provided by your administrator +4. **Settings auto-populate** with server URL, username, and password +5. **Tap "Connect"** to verify and save the configuration + +### Manual Configuration + +1. **Tap "Manual Configuration"** on the welcome screen +2. **Enter Server URL**: `http://your-server-ip:8080` or `https://your-server-domain` +3. **Enter Username**: Your username provided by your administrator +4. **Enter Password**: Your password +5. **Tap "Test Connection"** to verify connectivity +6. **Tap "Save"** to store the configuration + +### First Login + +1. **After configuration**, you'll be prompted to log in +2. **Credentials should be pre-filled** (if using QR code) +3. **Tap "Login"** to authenticate +4. **Wait for authentication** - A token is stored locally for future sessions +5. **You'll be redirected** to the main app interface + +## Verification + +To verify that Formulus is installed correctly: + +1. **Check app icon** appears in your app drawer +2. **Open the app** and verify it launches without errors +3. **Check Settings** to confirm server configuration is saved +4. **Test connection** by tapping "Test Connection" in Settings +5. **Verify login** by logging in with your credentials + +## Troubleshooting Installation + +### Installation Fails + +**Problem**: APK installation fails with "App not installed" error. + +**Solutions**: +- Ensure you have enough storage space (at least 50 MB free) +- Check that "Unknown Sources" is enabled for your file manager +- Try downloading the APK again (file may be corrupted) +- Ensure your device meets minimum Android version requirements (7.0+) + +### App Crashes on Launch + +**Problem**: Formulus crashes immediately after opening. + +**Solutions**: +- Restart your device +- Clear app data: Settings → Apps → Formulus → Storage → Clear Data +- Uninstall and reinstall the app +- Check that your device has sufficient RAM available + +### Cannot Connect to Server + +**Problem**: App cannot connect to the Synkronus server. + +**Solutions**: +- Verify server URL is correct (check for typos) +- Ensure device has internet connection +- Check that server is running and accessible +- Verify firewall settings aren't blocking the connection +- For local development, use `10.0.2.2` instead of `localhost` on Android emulator + +### Obtainium Not Finding Updates + +**Problem**: Obtainium doesn't show Formulus updates. + +**Solutions**: +- Ensure the ODE repository is added correctly in Obtainium +- Check that "Include prereleases" is enabled if you want pre-release versions +- Refresh Obtainium: Open the app and wait for it to check for updates +- Check that Obtainium has internet connection +- Verify repository URL is correct: `https://github.com/OpenDataEnsemble/ode` + +## Updating Formulus + +### Via Obtainium + +1. **Open Obtainium** and navigate to the **"Apps"** tab +2. **Find "ode"** in your apps list +3. **Tap on "ode"** to see update information +4. **Tap "Install"** or **"Update"** if a newer version is available +5. **Confirm the installation** when prompted +6. **App data is preserved** during update + +### Via Direct APK + +1. **Download the latest APK** from [Downloads](/downloads) or the [releases page](https://github.com/OpenDataEnsemble/ode/releases) +2. **Install over existing installation** (no need to uninstall) +3. **App data is preserved** during update + +## Uninstalling Formulus + +To uninstall Formulus: + +1. **Go to Settings** → **Apps** (or **Application Manager**) +2. **Find Formulus** in the app list +3. **Tap on Formulus** +4. **Tap "Uninstall"** +5. **Confirm uninstallation** + +**Note**: Uninstalling will remove all local data, including: +- Saved observations (not yet synced) +- App configuration +- Cached app bundles +- Local database + +**Important**: Ensure all data is synced to the server before uninstalling. + +## Finding Formulus After Installation + +After installation, you can find and launch Formulus: + +1. **Open your app drawer** or launcher +2. **Search for "Formulus"** or "formulu" using your device's search function +3. **Look for the Formulus icon**: A yellow giraffe head with a green background and white clipboard with checkmarks +4. **Tap on the Formulus icon** to launch the app + +![Finding Formulus App](/img/installation/formulus-app-search.png) + +## Related Documentation + +- [Formulus Features](/docs/using/formulus-features) - Learn about app features and usage +- [Your First Form](/docs/using/your-first-form) - Get started with data collection +- [Synchronization](/docs/using/synchronization) - Understand how data syncs work +- [Development Installation](/docs/development/formulus-development) - For developers building from source diff --git a/docs/docs/getting-started/installation/installing-ode-desktop.md b/docs/docs/getting-started/installation/installing-ode-desktop.md new file mode 100644 index 000000000..869eaef6c --- /dev/null +++ b/docs/docs/getting-started/installation/installing-ode-desktop.md @@ -0,0 +1,118 @@ +--- +sidebar_position: 3 +--- + +# Installing ODE Desktop + +Complete guide for installing **ODE Desktop** on Windows, macOS, and Linux. + +:::info ODE v1.3.2 + +ODE Desktop is part of the **ODE v1.3.2** release. Use the [Downloads](/downloads) page for direct, platform-matched installers or browse [GitHub Releases](https://github.com/OpenDataEnsemble/ode/releases). + +::: + +## Overview + +ODE Desktop is a native desktop application (Tauri) for: + +- **Data management** — inspect, edit, import, and sync observations with Synkronus +- **Forms / app workbench** — download app bundles, preview forms, and test custom apps + +Choose the installation method that fits your role: + +| Method | Best for | +|--------|----------| +| **GitHub Release** | Data stewards and app authors who want a ready-to-run installer | +| **Build from source** | Contributors and early adopters working from the ODE monorepo | + +## System requirements + +| Platform | Requirements | +|----------|--------------| +| **Windows** | Windows 10 or 11; [WebView2](https://developer.microsoft.com/en-us/microsoft-edge/webview2/) (usually pre-installed) | +| **macOS** | Recent macOS; Xcode command-line tools for source builds | +| **Linux** | WebKitGTK and related packages per [Tauri prerequisites](https://v2.tauri.app/start/prerequisites/) | + +## Method 1: GitHub Releases (recommended) + +1. Open [Downloads](/downloads) to download the installer matched to your platform, or open [OpenDataEnsemble/ode releases](https://github.com/OpenDataEnsemble/ode/releases). +2. Select the **v1.3.2** release (or the latest stable tag). +3. Download the artifact for your platform: + + | Platform | Typical artifact | + |----------|------------------| + | Windows | `.msi` installer | + | macOS | `.dmg` or `.app` bundle | + | Linux | AppImage, `.deb`, or similar | + +4. Run the installer or extract the bundle and launch **ODE Desktop**. + +:::note Installer script + +A curl-style installer script (`scripts/install-ode-desktop.sh`) exists in the monorepo as a placeholder for future one-line installs. Until it is wired to release asset URLs, use GitHub Releases directly. + +::: + +## Method 2: Build from source + +For development or when no pre-built artifact is available for your platform: + +### Prerequisites + +- **Node.js** 20+ and **pnpm** 10+ +- **Rust** toolchain ([rustup](https://rustup.rs/)) +- Platform build tools (see [Tauri prerequisites](https://v2.tauri.app/start/prerequisites/)) + +### Build steps + +```bash +git clone https://github.com/OpenDataEnsemble/ode.git +cd ode/desktop +pnpm install +pnpm tauri build +``` + +`pnpm tauri build` runs the full pipeline: Formplayer assets are prepared, the frontend is built, and Tauri packages the native app. Installers or bundles appear under `desktop/src-tauri/target/release/bundle/`. + +### Development run + +To run with hot reload during development: + +```bash +cd ode/desktop +pnpm install +pnpm tauri dev +``` + +:::warning Use the Tauri window + +`pnpm tauri dev` opens the **Tauri desktop window**. Do not use a regular browser at `http://localhost:1420` — IPC commands such as `invoke` only work inside the Tauri shell. + +::: + +## First launch + +1. Open **ODE Desktop**. +2. Switch to **Data management** mode (if not already selected). +3. On **Profiles**, add or select a profile with your Synkronus server URL. +4. **Authenticate** with your Synkronus credentials. +5. On **Sync**, run **Pull** to download observations, or **Download & apply** an app bundle from **Workbench → Bundles**. + +## Next steps + +- [ODE Desktop user guide](/docs/using/ode-desktop/) — screen-by-screen usage +- [ODE Desktop reference](/docs/reference/ode-desktop) — architecture and workspace layout +- [ODE Desktop developer mode](/docs/guides/ode-desktop-developer-mode) — test a local custom app build +- [ODE Desktop development](/docs/development/ode-desktop-development) — contributor setup + +## Troubleshooting + +| Issue | Suggestion | +|-------|------------| +| App won't start on Linux | Install WebKitGTK and dependencies from Tauri docs | +| WebView2 missing on Windows | Install the [WebView2 runtime](https://developer.microsoft.com/en-us/microsoft-edge/webview2/) | +| Build fails on `pnpm tauri build` | Ensure Rust and platform prerequisites are installed; run `pnpm build:formplayer` first if Formplayer assets are missing | +| Sync authentication fails | Verify server URL and credentials on **Profiles**; check server reachability on **Sync** | + +For more help, see [Getting Help](/docs/community/getting-help) or the [forum](https://forum.opendataensemble.org). diff --git a/docs/docs/getting-started/installation/installing-synkronus.md b/docs/docs/getting-started/installation/installing-synkronus.md new file mode 100644 index 000000000..8949b1e65 --- /dev/null +++ b/docs/docs/getting-started/installation/installing-synkronus.md @@ -0,0 +1,183 @@ +--- +sidebar_position: 1 +--- + +# Install Synkronus Server + +Setting up the server is often the hardest part of getting started with +Synkronus. Once the server is running, the rest of the system becomes +much easier to work with. + +**We don't recommend any particular hosting provider.** Any VPS or VM +that can run Linux and containers is fine. This guide is written for a +generic Linux server. If you need to create a new server and want a +concrete example, we show **DigitalOcean** in the next section—use it if +you like, or skip it and follow the rest of the steps on your own +machine or provider. + +This setup is **not intended for production use**. Later guides cover +custom domains, proper TLS, backups, logging, and production patterns. +Here the goal is simply to **get a working server running quickly**. + +You will need: + +- A machine running **Linux** (we use **Ubuntu 24.04 LTS** in the examples) +- At least **1–2 GB RAM** (enough for PostgreSQL and Synkronus) +- Root or sudo access to install packages and run containers + +--- + +## Example: Create a VPS on DigitalOcean + +If you already have a server, skip to [Install required tools](#install-required-tools). + +If you want to create a new VPS and are happy to use DigitalOcean: + +1. Create a DigitalOcean account and go to **Droplets → Create Droplet**. +2. Choose: + - A datacenter location close to you + - **Ubuntu 24.04 LTS** + - **Shared CPU / 1 GB RAM** + +At the time of writing this costs about **$6 USD per month**. + +When the droplet is created, note the **public IPv4 address**. + +DigitalOcean droplet with IPv4 address + +Click **Console** to open the server terminal, then continue with the steps below. + +--- + +## Install required tools + +On your Linux server, install the required packages. + +We use **Podman instead of Docker** to keep the stack fully open source. +Podman is daemonless, OCI-compatible, maintained by Red Hat, and largely +CLI-compatible with Docker. For workloads like Synkronus it works very well. + +```bash +sudo apt update +sudo apt install -y podman podman-compose git +``` + +--- + +## Clone the quickstart repository + +Clone the Synkronus quickstart repository and run the installer. + +Install script running in shell + +```bash +git clone --depth 1 https://github.com/OpenDataEnsemble/synkronus-quickstart.git server +cd server +chmod +x ./install.sh +./install.sh +``` + +During installation you will be asked: + +**Do you have a domain name pointing to this server? (y/n)** + +If you **do not have a domain**, answer **n**, then enter the server's +public IP when prompted. + +The installer will configure access using: + +**https://``.sslip.io** + +This hostname resolves to your IP and works with automatic TLS. + +--- + +## Bring the server online + +Start the containers: + +```bash +podman compose up -d +``` + +The first startup may take a minute while images are downloaded. + +Check that everything is running: + +```bash +podman ps +``` + +--- + +## Access the Synkronus Portal + +Once the containers are running, open your browser and go to: + +**https://``.sslip.io** + +You should see the **Synkronus Portal login screen**. + +Synkronus Portal login + +Use the **admin username and password printed by the installer**. + +--- + +## TLS and HTTPS + +Synkronus uses **Caddy** as a reverse proxy. Caddy automatically +provisions TLS certificates via Let's Encrypt, handles HTTPS, and +forwards requests to the Synkronus server. With `sslip.io`, certificates +are issued automatically. + +For this to work, the server must be reachable from the internet on +**port 80** and **port 443**. + +:::caution If the certificate fails initially + +Sometimes Let's Encrypt validation fails on first boot if the server +isn't yet reachable. If you see a certificate error in your browser, +restart Caddy: + +```bash +podman restart synkronus_caddy +``` + +After a short moment, HTTPS should work. + +::: + +--- + +## Create a user + +After logging in with the admin account, create a user for your client +applications (e.g. Formulus): + +1. Open the **Users** tab. +2. Click **+ Create User**. +3. Assign **Read/Write** permissions. + +Create user in portal + +This user can now be used by Synkronus clients such as **Formulus**. + +--- + +## Alternative networking setups + +If you already run a **tunnel service** (e.g. **Cloudflare Zero Trust** +tunnels), you can skip automatic TLS. When prompted for the public IP +during installation, enter **localhost**. Caddy will run locally without +provisioning certificates, and your tunnel can handle HTTPS externally. + +--- + +Your Synkronus server is now running. + +For production planning (TLS, backups, sizing, security), see **[Server Architecture for IT](/docs/guides/server-architecture-for-it)**. + +

    + +

    diff --git a/docs/docs/getting-started/key-concepts.md b/docs/docs/getting-started/key-concepts.md new file mode 100644 index 000000000..4a965644b --- /dev/null +++ b/docs/docs/getting-started/key-concepts.md @@ -0,0 +1,102 @@ +--- +sidebar_position: 3 +--- + +# Key Concepts + +Understanding these core concepts will help you work effectively with ODE. + +## Core Concepts + +### Forms + +Forms are the primary mechanism for data collection in ODE. A form consists of: + +- **Schema**: Defines the data structure and validation rules +- **UI Schema**: Defines how the form is presented to users +- **Question Types**: Define the input methods available (text, number, date, etc.) + +Forms are defined using JSON and follow the JSON Forms specification. See the [Form Design guide](/guides/forms/overview) for details. + +### Observations + +An observation is a single data record collected through a form. Each observation contains: + +- A unique identifier +- The form type used to collect it +- The data values entered by the user +- Metadata such as creation time and last modification time +- A sync status indicating whether it has been synchronized with the server + +### Synchronization + +Synchronization is the process of exchanging data between mobile devices and the server. ODE uses a bidirectional sync protocol that: + +- Pushes local observations to the server +- Pulls new or updated observations from the server +- Resolves conflicts when the same observation is modified on multiple devices +- Handles attachments separately from observation metadata + +See [Synchronization](/using/synchronization) for more details. + +### App Bundles + +An app bundle is a collection of resources that define a custom application. It includes: + +- Custom HTML, CSS, and JavaScript files +- Form specifications +- Custom renderers for question types +- Configuration files + +App bundles are uploaded to the server and downloaded by mobile devices during synchronization. See [Custom Applications](/guides/custom-apps/overview) for details. + +### Custom Applications + +Custom applications are web-based interfaces that run within the Formulus mobile app. They provide: + +- Custom navigation and user interfaces +- Integration with the ODE form system +- Access to observation data through the Formulus JavaScript interface + +Custom applications are defined in app bundles and can be tailored to specific use cases. + +## Data Flow + +The following diagram illustrates how data flows through the ODE system: + +``` +User Input → Form → Observation (Local) → Sync → Server → Database + ↓ + Sync → Other Devices +``` + +1. User fills out a form on a mobile device +2. An observation is created and stored locally +3. When connectivity is available, the observation is synchronized to the server +4. The server stores the observation in the database +5. Other devices can pull the observation during their sync operations + +## Terminology + +| Term | Definition | +|------|------------| +| **Form** | A data collection interface defined by schema and UI schema | +| **Observation** | A single data record collected through a form | +| **Schema** | JSON schema defining the structure and validation rules for a form | +| **UI Schema** | JSON schema defining the presentation of form fields | +| **Question Type** | A component that handles a specific type of input (text, number, etc.) | +| **Renderer** | A component that renders a question type in the form | +| **Sync** | The process of exchanging data between devices and server | +| **App Bundle** | A collection of resources defining a custom application | +| **Custom App** | A web-based application that runs within Formulus | +| **Formulus** | The mobile application component of ODE | +| **Synkronus** | The server component of ODE | +| **Formplayer** | The web-based form rendering component | + +## Related Documentation + +- [Form Design Guide](/guides/forms/overview) +- [Synchronization Details](/using/synchronization) +- [Custom Applications](/guides/custom-apps/overview) +- [Architecture Overview](/development/architecture/overview) + diff --git a/docs/docs/getting-started/quickstart/_category_.json b/docs/docs/getting-started/quickstart/_category_.json new file mode 100644 index 000000000..d1ffbef93 --- /dev/null +++ b/docs/docs/getting-started/quickstart/_category_.json @@ -0,0 +1,4 @@ +{ + "label": "Quickstart", + "position": 1 +} diff --git a/docs/docs/getting-started/synkronus-quickstart.md b/docs/docs/getting-started/synkronus-quickstart.md new file mode 100644 index 000000000..44ddb7124 --- /dev/null +++ b/docs/docs/getting-started/synkronus-quickstart.md @@ -0,0 +1,285 @@ +--- +sidebar_position: 6 +title: Synkronus Quickstart +--- + +# Synkronus Quickstart — Deploy in Minutes + +Get a full **Synkronus server** running in minutes with Docker or Podman, including database, automated HTTPS, and portal UI. + +> **TL;DR:** Clone, run installer, done. See [Easiest: Run the Installer](#easiest-run-the-installer). + +--- + +## What you get + +- ✅ **Synkronus server** — REST API for mobile app sync +- ✅ **PostgreSQL database** — Data persistence +- ✅ **Synkronus Portal** — Web UI for admin tasks +- ✅ **Caddy reverse proxy** — Auto TLS/HTTPS (Let's Encrypt) +- ✅ **Single Docker volume** — All data in one place +- ✅ **Works locally or in cloud** — GitHub Codespaces ready + +--- + +## Prerequisites + +**Docker or Podman** + +```bash +# macOS (Homebrew) +brew install podman podman-compose + +# Ubuntu/Debian +sudo apt update && sudo apt install -y podman podman-compose git +``` + +**Git** — Clone the repository + +--- + +## Easiest: Run the Installer ⭐ + +The installer generates strong passwords, configures TLS, and handles everything. + +```bash +# 1. Clone the repository +git clone --depth 1 https://github.com/OpenDataEnsemble/synkronus-quickstart.git server +cd server + +# 2. Run the installer +chmod +x ./install.sh +./install.sh + +# 3. Start the stack +podman compose up -d +``` + +The installer will ask: + +**Do you have a domain name?** + +- **Yes** → Enter your domain (e.g., `myserver.com`) + - Caddy obtains a real TLS certificate from Let's Encrypt + - Access: `https://myserver.com` + +- **No, use public IP** → Enter your server's IP + - Caddy uses `.sslip.io` (real certificate) + - Access: `https://.sslip.io` + +- **No, localhost** → Local testing only + - HTTP on `localhost:80` + - Access: `http://localhost` + +**Output:** +``` +Admin username: admin +Admin password: < strong password saved > +``` + +Save these credentials! + +--- + +## After setup + +### Access the portal + +- **Domain:** `https://myserver.com` +- **Public IP:** `https://.sslip.io` +- **Localhost:** `http://localhost` + +Login with the credentials from the installer. + +### Create users + +In Synkronus Portal: +1. **Settings** → **Users** +2. Click **+ Add User** +3. Enter username, password, and permissions + +### Verify the server + +```bash +# Check if server is running +curl https://myserver.com/health +# Response: "OK" +``` + +--- + +## Data storage + +Data is stored in a single Docker volume at `/app/data`: + +``` +/app/data/ +├── app-bundle/active/ # Currently deployed app bundle +├── app-bundle/versions/ # Version history +├── attachments/ # Photos, documents, etc. +└── database/ # PostgreSQL data (in named volume) +``` + +To find the volume path: + +```bash +podman volume inspect --format '{{ .Mountpoint }}' +``` + +--- + +## Utilities + +The repo includes helpful scripts: + +| Script | Purpose | +|--------|---------| +| `backup-db.sh` | Backup PostgreSQL database to `.sql` file | +| `backup-attachments.sh` | Copy attachment blobs from container | +| `migrate-synkronus-data.sh` | Used when upgrading from older versions | + +**Example: Backup the database** + +```bash +chmod +x ./utilities/backup-db.sh +./utilities/backup-db.sh +# Creates: synkronus-db-backup-.sql +``` + +--- + +## Manual setup (advanced) + +If you prefer to configure manually: + +### 1. Clone and prepare + +```bash +git clone https://github.com/OpenDataEnsemble/synkronus-quickstart.git +cd synkronus-quickstart +``` + +### 2. Edit docker-compose.yml + +Set environment variables: + +```yaml +postgres: + environment: + POSTGRES_PASSWORD: + +synkronus: + environment: + DB_CONNECTION: "user=synkronus password= host=db dbname=synkronus" + JWT_SECRET: + ADMIN_USERNAME: admin + ADMIN_PASSWORD: +``` + +### 3. Initialize the database + +```bash +# Terminal 1: Start database only +podman compose up db + +# Terminal 2: Create the Synkronus database +chmod +x ./create_sync_db.sh +./create_sync_db.sh +``` + +### 4. Start the full stack + +```bash +podman compose up -d +``` + +### 5. Verify + +```bash +curl http://localhost:8080/health +# Response: "OK" +``` + +--- + +## Using GitHub Codespaces + +Perfect for trying out Synkronus in 30 seconds — no local setup needed. + +1. Go to [synkronus-quickstart repo](https://github.com/OpenDataEnsemble/synkronus-quickstart) +2. Click **"Open in Codespaces"** +3. Wait for startup (containers will auto-start) +4. Check **Ports** tab to find the forwarded URL +5. Test: + ```bash + curl /health + ``` + +--- + +## Upgrading from older versions + +If you previously used older paths for app bundles: + +1. **Stop the stack:** + ```bash + podman compose down + ``` + +2. **Backup the data volume:** + ```bash + docker run --rm \ + -v :/data \ + -v "$(pwd)":/backup alpine \ + tar czf /backup/backup.tgz -C /data . + ``` + +3. **Run the migration script:** + ```bash + chmod +x ./utilities/migrate-synkronus-data.sh + podman run --rm \ + -v :/data:Z \ + -v "$PWD/utilities/migrate-synkronus-data.sh:/migrate.sh:ro,Z" \ + docker.io/library/alpine:3.21 \ + sh /migrate.sh /data + ``` + +4. **Start the new version:** + ```bash + podman compose up -d + ``` + +See [upgrade-path.md](https://github.com/OpenDataEnsemble/synkronus-quickstart/blob/main/upgrade-path.md) for full details. + +--- + +## Troubleshooting + +| Problem | Solution | +|---------|----------| +| Certificate error on first boot | Wait 1-2 minutes for Let's Encrypt validation. Restart Caddy: `podman restart synkronus_caddy` | +| Can't connect to server | Verify stack is running: `podman compose ps` | +| Database won't start | Check disk space and container logs: `podman logs db` | +| Lost attachments | Use `backup-attachments.sh` before recreating containers | + +--- + +## Next steps + +1. **[Build a custom app](../guides/building-custom-apps.md)** — Create your first data collection app +2. **[Install Formulus](./installation/installing-formulus.md)** — Get the mobile app +3. **Configure mobile nodes** — Set up offline sync +4. **Join the community** — [forum.opendataensemble.org](https://forum.opendataensemble.org) + +--- + +## Reference + +- **[Repository](https://github.com/OpenDataEnsemble/synkronus-quickstart)** — Source code and utilities +- **[Synkronus Server API](../reference/synkronus-server.md)** — Full API reference +- **[Configuration](../reference/configuration/server.md)** — Server settings +- **[Forum](https://forum.opendataensemble.org)** — Community support + +--- + +**Ready to sync data?** 🚀 diff --git a/docs/docs/getting-started/what-is-ode.md b/docs/docs/getting-started/what-is-ode.md new file mode 100644 index 000000000..8948f75f6 --- /dev/null +++ b/docs/docs/getting-started/what-is-ode.md @@ -0,0 +1,96 @@ +--- +sidebar_position: 1 +--- + +# What is ODE? + +:::tip Pronunciation +ODE is pronounced like "code", without the "C." +::: + +Open Data Ensemble (ODE) is a platform designed to simplify mobile data collection and management. It provides tools and infrastructure for creating forms, collecting data in the field, and synchronizing information across devices and servers. + +## Overview + +ODE addresses the challenges of data collection in environments where connectivity is unreliable or intermittent. The platform is built with an offline-first architecture, ensuring that data collection continues regardless of network availability. + +## Problem Statement + +Traditional data collection solutions often require constant internet connectivity, making them unsuitable for field work in remote areas. ODE solves this by: + +- Storing data locally on mobile devices +- Synchronizing data when connectivity is available +- Resolving conflicts automatically when multiple devices modify the same data +- Providing a flexible form design system that adapts to various use cases + +## Key Features + +### Offline-First Design + +All data is stored locally on the device using WatermelonDB, a reactive database optimized for React Native. This ensures that data collection continues even when the device is offline. + +### Flexible Form System + +Forms are defined using JSON schema, following the JSON Forms specification. This allows for complex validation rules, conditional logic, and custom question types. + +### Reliable Synchronization + +The synchronization protocol handles conflicts, ensures data integrity, and supports incremental updates to minimize bandwidth usage. + +### Cross-Platform Support + +ODE applications run on Android and iOS devices, with a web-based form player for preview and testing. + +### Extensible Architecture + +The platform supports custom applications and renderers, allowing organizations to tailor the user experience to their specific needs. + +## Use Cases + +ODE is suitable for various data collection scenarios: + +| Use Case | Description | +|----------|-------------| +| **Health Surveys** | Collect patient data, medical records, and health indicators | +| **Research Studies** | Gather research data in field conditions | +| **Monitoring & Evaluation** | Track program outcomes and indicators | +| **Asset Management** | Inventory and track assets in the field | +| **Quality Assurance** | Conduct inspections and quality checks | + +## Architecture Overview + +ODE follows a client-server architecture: + +``` +┌─────────────┐ ┌──────────────┐ ┌─────────────┐ +│ Formulus │◄───────►│ Synkronus │◄───────►│ Formulus │ +│ (Mobile) │ Sync │ (Server) │ Sync │ (Mobile) │ +└─────────────┘ └──────────────┘ └─────────────┘ + │ │ │ + │ │ │ + ▼ ▼ ▼ +┌─────────────┐ ┌──────────────┐ ┌─────────────┐ +│ Formplayer │ │ Database │ │ Formplayer │ +│ (WebView) │ │ (PostgreSQL) │ │ (WebView) │ +└─────────────┘ └──────────────┘ └─────────────┘ +``` + +The mobile application (Formulus) communicates with the server (Synkronus) to synchronize data. Forms are rendered using the Formplayer component, which can be embedded in custom applications. + +## Technology Stack + +ODE is built using modern, open-source technologies: + +- **React Native** for mobile applications +- **React** for web-based form rendering +- **Go** for the backend server +- **PostgreSQL** for data storage +- **WatermelonDB** for local data storage on mobile devices +- **JSON Forms** for form rendering and validation + +## Next Steps + +- Learn about [Why ODE?](/getting-started/why-ode) to understand the benefits +- Review [Key Concepts](/getting-started/key-concepts) to understand the terminology +- Follow the [Installation guide](/docs/getting-started/installation) to set up your environment + diff --git a/docs/docs/getting-started/why-ode.md b/docs/docs/getting-started/why-ode.md new file mode 100644 index 000000000..a30591e13 --- /dev/null +++ b/docs/docs/getting-started/why-ode.md @@ -0,0 +1,70 @@ +--- +sidebar_position: 2 +--- + +# Why ODE? + +ODE provides several advantages over traditional data collection solutions, particularly for organizations working in challenging environments with unreliable connectivity. + +## Advantages + +### Offline-First Architecture + +Unlike many data collection platforms that require constant connectivity, ODE is designed to work offline. Data is stored locally and synchronized when connectivity is available, ensuring that field work is never interrupted by network issues. + +### Conflict Resolution + +When multiple devices modify the same data, ODE automatically resolves conflicts using a version-based approach. This ensures data integrity without requiring manual intervention. + +### Flexible Form Design + +The JSON-based form system allows for complex validation rules, conditional logic, and custom question types. Forms can be updated without requiring application updates, enabling rapid iteration and adaptation. + +### Open Source + +ODE is fully open source, allowing organizations to audit the code, customize the platform, and contribute improvements. This transparency builds trust and enables community-driven development. + +### Cross-Platform Support + +Applications built with ODE run on both Android and iOS devices, reducing the need to maintain separate codebases for different platforms. + +### Extensible + +The platform supports custom applications and renderers, allowing organizations to create specialized workflows and user interfaces that match their specific needs. + +## Comparison with Alternatives + +| Feature | ODE | Traditional Solutions | +|--------|-----|----------------------| +| Offline Support | Full offline functionality | Limited or none | +| Conflict Resolution | Automatic | Manual or none | +| Form Updates | Without app updates | Requires app updates | +| Customization | High flexibility | Limited | +| Open Source | Yes | Often proprietary | +| Cost | Free and open source | Licensing fees | + +## When to Use ODE + +ODE is particularly well-suited for: + +- Organizations conducting field research or data collection in remote areas +- Projects requiring complex forms with validation and conditional logic +- Teams needing to customize the user experience +- Organizations prioritizing data privacy and security +- Projects requiring offline functionality + +## When to Consider Alternatives + +ODE may not be the best fit if: + +- Your use case requires real-time collaboration features +- You need extensive third-party integrations out of the box +- Your team lacks technical resources for deployment and maintenance +- Your data collection needs are very simple and don't require offline support + +## Next Steps + +- Review [Key Concepts](/getting-started/key-concepts) to understand ODE terminology +- Check the [Installation](/docs/getting-started/installation) guide for system requirements +- Follow the [Installation guide](/docs/getting-started/installation) to get started + diff --git a/docs/docs/guides/.gitkeep b/docs/docs/guides/.gitkeep new file mode 100644 index 000000000..e69de29bb diff --git a/docs/docs/guides/building-custom-apps-v1.md b/docs/docs/guides/building-custom-apps-v1.md new file mode 100644 index 000000000..461483b8a --- /dev/null +++ b/docs/docs/guides/building-custom-apps-v1.md @@ -0,0 +1,347 @@ +--- +sidebar_position: 6 +title: Your First Custom App (v1) +--- + +# Your First Custom App: Coffee Tracker v1 + +## Overview + +In this guide, we'll build **Coffee Tracker v1** — our first custom_app for registering roasted coffee beans. + +You'll learn: +- How forms (schema.json + ui.json) work together +- How to structure a custom_app bundle +- How to upload and test your app in Formulus + +**Time to complete:** 30–45 minutes +**Prerequisites:** +- Comfortable editing JSON and HTML +- Access to a Synkronus server and Formulus app +- (Recommended) [Synkronus Quickstart](../getting-started/synkronus-quickstart.md) for a test server + +--- + +## What is a custom_app? + +A `custom_app` is a zipped archive with two folders: + +``` +coffee_tracker-v1.0.0.zip +├── app/ # HTML + JavaScript for the UI +│ ├── index.html +│ └── style.css +└── forms/ # Questionnaire specifications + └── register_coffee/ + ├── schema.json # Data shape + └── ui.json # Form layout +``` + +--- + +## Part 1: Create the forms + +### The form specification + +A form is defined by **two files**: +- **schema.json** — What data is collected (data shape, types, required fields) +- **ui.json** — How the form looks and flows (screens, layout, field order) + +### Coffee registration form + +We'll collect: +- Photo of the beans +- Bean variety name +- Country of origin +- Roaster name +- Roasting date +- Roast level + +**Folder structure:** +``` +forms/ +└── register_coffee/ + ├── schema.json + └── ui.json +``` + +### schema.json + +Create `forms/register_coffee/schema.json`: + +```json +{ + "type": "object", + "properties": { + "photo": { + "type": "object", + "format": "photo", + "title": "Bean photo", + "description": "Take a picture of the beans in portrait mode" + }, + "name": { + "type": "string", + "title": "Bean variety", + "description": "Name of the coffee variety" + }, + "origin": { + "type": "string", + "title": "Country of Origin" + }, + "roaster": { + "type": "string", + "title": "Roaster", + "description": "Who roasted the coffee" + }, + "roast_date": { + "type": "string", + "format": "date-time", + "title": "Roasting date", + "description": "Date and time when this batch was roasted" + }, + "roast_profile": { + "type": "string", + "title": "Roast Profile", + "enum": ["dark", "medium", "medium-light", "light"] + } + }, + "required": ["name", "roast_date"] +} +``` + +### ui.json + +Create `forms/register_coffee/ui.json`: + +```json +{ + "type": "SwipeLayout", + "options": { + "headerTitle": "Register Coffee", + "headerFields": ["name"] + }, + "elements": [ + { + "type": "Label", + "text": "

    Mmmm... Coffee...

    ", + "options": { + "html": true + } + }, + { + "type": "Control", + "scope": "#/properties/photo" + }, + { + "type": "Control", + "scope": "#/properties/name" + }, + { + "type": "Control", + "scope": "#/properties/origin" + }, + { + "type": "Control", + "scope": "#/properties/roaster" + }, + { + "type": "Control", + "scope": "#/properties/roast_profile" + }, + { + "type": "Control", + "scope": "#/properties/roast_date" + } + ] +} +``` + +> **Note:** ODE extends JSONForms with types like `photo`, `gps`, `qr-code`, and more. For details, see [Form Specifications](../reference/form-specifications.md). + +--- + +## Part 2: Create the app + +For v1, we'll keep the app simple — just a static HTML page. Imagine you'd welcome users or show instructions here. + +**Folder structure:** +``` +app/ +├── index.html +├── style.css +└── coffee_cup.png +``` + +### index.html + +Create `app/index.html`: + +```html + + + + Coffee Tracker v1.0 + + + + +
    +

    Coffee Tracker

    +

    v1.0

    +

    A simple app for registering roasted coffee beans.

    + Coffee cup +

    Start by registering a new coffee to begin tracking!

    +
    + + +``` + +### style.css + +Create `app/style.css`: + +```css +body { + font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif; + background: linear-gradient(135deg, #6B4423 0%, #9C2C07 100%); + color: #333; + margin: 0; + padding: 16px; +} + +.container { + max-width: 600px; + margin: 0 auto; + background: white; + padding: 32px; + border-radius: 8px; + box-shadow: 0 4px 12px rgba(0, 0, 0, 0.1); + text-align: center; +} + +h1 { + color: #9C2C07; + margin: 0 0 16px; +} + +h2 { + color: #666; + margin: 0 0 24px; + font-size: 18px; +} + +p { + color: #555; + line-height: 1.6; + margin: 12px 0; +} + +.image { + max-width: 200px; + margin: 24px 0; +} + +em { + color: #9C2C07; +} +``` + +--- + +## Part 3: Bundle and upload + +### Create the zip archive + +Ensure your folder structure is: + +``` +coffee_tracker-v1.0.0/ +├── app/ +│ ├── index.html +│ ├── style.css +│ └── coffee_cup.png +└── forms/ + └── register_coffee/ + ├── schema.json + └── ui.json +``` + +**On macOS/Linux:** +```bash +zip -r coffee_tracker-v1.0.0.zip app/ forms/ +``` + +**On Windows (PowerShell):** +```powershell +Compress-Archive -Path app/, forms/ -DestinationPath coffee_tracker-v1.0.0.zip +``` + +### Upload to Synkronus + +**Option 1: Web Portal** +1. Log in to Synkronus Portal +2. Go to **App Bundle** page +3. Upload `coffee_tracker-v1.0.0.zip` + +**Option 2: CLI** +```bash +./synk app-bundle upload ./coffee_tracker-v1.0.0.zip -a +``` + +Download the CLI from [ODE releases](https://github.com/OpenDataEnsemble/ode/releases). + +--- + +## Part 4: Test in Formulus + +1. Install Formulus on your device +2. Configure the server (point to your Synkronus instance) +3. Log in +4. Go to **Sync** page → **Update App Bundle** +5. Open the **Coffee Tracker** app + +You should see your HTML page with the welcome message. + +--- + +## Troubleshooting + +**App doesn't appear after sync:** +- Check Synkronus logs: `podman logs synkronus` +- Verify the upload succeeded in the Portal +- Ensure `app/` and `forms/` are at the root of your zip file + +**Form validation errors:** +- Check JSON syntax (use an online validator) +- Verify required fields in schema match the form + +**Photos not saving:** +- Ensure device has camera permissions +- Check Synkronus attachments directory + +--- + +## Next steps + +Congratulations! Your first custom_app is live. Ready to add more power? + +**In v2**, we'll add: +- Follow-up form for "shots pulled" +- Dashboard showing registered coffees +- Data injection to link observations +- Formulus API integration + +→ [Go to v2: Longitudinal Data](./building-custom-apps-v2.md) + +--- + +## Reference + +- **[JSONForms Spec](../reference/form-specifications.md)** — Full form configuration +- **[App Bundle Format](../reference/app-bundle-format.md)** — Technical bundle spec +- **[Synkronus Quickstart](../getting-started/synkronus-quickstart.md)** — Set up a test server +- **[Forum](https://forum.opendataensemble.org)** — Get help from the community + +--- + +**Coffee status:** ☕ Registered. Ready to brew! diff --git a/docs/docs/guides/building-custom-apps-v2.md b/docs/docs/guides/building-custom-apps-v2.md new file mode 100644 index 000000000..7502ae19f --- /dev/null +++ b/docs/docs/guides/building-custom-apps-v2.md @@ -0,0 +1,644 @@ +--- +sidebar_position: 7 +title: Longitudinal Data & Formulus API (v2) +--- + +# Longitudinal Data Collection: Coffee Tracker v2 + +## Overview + +In this guide, we extend Coffee Tracker with **longitudinal data** — the ability to follow up on initial registrations with related observations. + +You'll learn: +- How to link forms (one-to-many relationships) +- How to use the Formulus API to query and open forms +- How to inject data across linked forms +- How to build dashboards around collected data + +**Time to complete:** 60–90 minutes +**Prerequisites:** +- Completed [v1 guide](./building-custom-apps-v1.md) +- Basic JavaScript knowledge +- Access to Synkronus and Formulus + +--- + +## What is longitudinal data? + +**Longitudinal data** means following up on an entity over time: + +``` +Register Coffee (v1.0) + ↓ + ├→ Pull Shot (first follow-up) + ├→ Pull Shot (second follow-up) + └→ Pull Shot (third follow-up) +``` + +In ODE terms: +- **One-to-many relationship** between `register_coffee` and `pull_shot` observations +- Follow-ups include a reference to the original (e.g., `bean: "Prainema"`) +- The app queries and displays related data + +Some projects instead embed related **child payloads** **inside** the parent observation using **`format: sub-observation`** (a JSON array on one observation). That pattern is documented under [Custom Extensions](./custom-extensions.md#sub-observations-format-sub-observation); this guide focuses on **separate** follow-up observations linked by queries and IDs. + +--- + +## Part 1: Create the follow-up form + +### Add pull_shot form + +Create a new form for "shots pulled" (espresso shots). Folder structure: + +``` +forms/ +├── register_coffee/ +│ ├── schema.json +│ └── ui.json +└── pull_shot/ # NEW + ├── schema.json + └── ui.json +``` + +### pull_shot/schema.json + +```json +{ + "type": "object", + "properties": { + "bean": { + "type": "string", + "title": "Bean name", + "description": "Which coffee bean is this shot from?" + }, + "yield": { + "type": "number", + "title": "Yield (grams)", + "description": "Output weight in grams" + }, + "time": { + "type": "number", + "title": "Time (seconds)", + "description": "How long the shot took" + }, + "rating": { + "type": "string", + "title": "Rating", + "enum": ["poor", "fair", "good", "excellent"] + }, + "notes": { + "type": "string", + "title": "Notes", + "description": "Optional tasting notes" + } + }, + "required": ["bean", "yield", "time"] +} +``` + +### pull_shot/ui.json + +```json +{ + "type": "SwipeLayout", + "options": { + "headerTitle": "Pull Shot", + "headerFields": ["bean"] + }, + "elements": [ + { + "type": "Control", + "scope": "#/properties/bean", + "options": { + "readOnly": true + } + }, + { + "type": "Control", + "scope": "#/properties/yield" + }, + { + "type": "Control", + "scope": "#/properties/time" + }, + { + "type": "Control", + "scope": "#/properties/rating" + }, + { + "type": "Control", + "scope": "#/properties/notes" + } + ] +} +``` + +**Key detail:** The `bean` field is `readOnly` — we'll inject the value from the app, so users don't enter it manually. + +--- + +## Part 2: Build the app with data access + +### New app structure + +``` +app/ +├── index.html +├── index.js +├── details.html +├── details.js +├── style.css +├── formulus-load.js # Helper to access Formulus API +└── coffee_cup.png +``` + +### Get the Formulus API helper + +Download `formulus-load.js` from [ODE repo](https://github.com/OpenDataEnsemble/ode/blob/dev/formulus/android/app/src/main/assets/formplayer_dist/formulus-load.js) and save it in your `app/` folder. + +### index.html — Coffee list + +Create `app/index.html`: + +```html + + + + Coffee Tracker v2.0 + + + + +
    +

    Coffee Tracker

    +

    v2.0 — Longitudinal Data

    + + + + + + + + + + + +
    BeanCountry
    +

    Loading registered coffees...

    +
    + + + + + +``` + +### index.js — Query and display coffees + +Create `app/index.js`: + +```javascript +/** + * Display registered coffees in a table + * @param {Array} beans - Array of coffee observations + */ +async function updateBeans(beans) { + const beansTable = document.getElementById('beans-table').querySelector('tbody'); + const loading = document.getElementById('loading'); + + loading.style.display = 'none'; + + if (beans.length === 0) { + beansTable.innerHTML = 'No coffees registered yet.'; + return; + } + + beans.forEach(bean => { + const data = bean.data || {}; + beansTable.innerHTML += ` + + ${data.name || 'Unknown'} + ${data.origin || '—'} + + + + + `; + }); +} + +/** + * Navigate to details page + */ +function goToDetails(beanName) { + window.location.href = `details.html?bean=${encodeURIComponent(beanName)}`; +} + +/** + * Initialize: fetch coffees and display + */ +async function init() { + try { + const api = await getFormulus(); + const observations = await api.getObservations('register_coffee'); + updateBeans(observations); + } catch (err) { + console.error('Error loading coffees:', err); + document.getElementById('loading').textContent = 'Error loading coffees. See console.'; + } +} + +// Start when page loads +document.addEventListener('DOMContentLoaded', init); +``` + +### details.html — Coffee details + shots + +Create `app/details.html`: + +```html + + + + Coffee Details - Coffee Tracker v2 + + + + +
    + + +
    + +
    + +
    +

    Shots Pulled

    + + + + + + + + + + + +
    YieldTimeRating
    +
    +
    + + + + + +``` + +### details.js — Display linked data + +Create `app/details.js`: + +```javascript +/** + * Get query parameter from URL + */ +function getQueryParam(name) { + const url = new URL(window.location); + return url.searchParams.get(name); +} + +/** + * Display coffee details + */ +async function displayCoffee(beanName) { + try { + const api = await getFormulus(); + + // Fetch all registered coffees + const coffees = await api.getObservations('register_coffee'); + const match = coffees.find(obs => (obs.data || {}).name === beanName); + + if (!match) { + document.getElementById('coffee-details').innerHTML = '

    Coffee not found.

    '; + return; + } + + const data = match.data || {}; + const detailsDiv = document.getElementById('coffee-details'); + + detailsDiv.innerHTML = ` +

    ${data.name || 'Unknown'}

    +
    +
    Origin
    +
    ${data.origin || '—'}
    +
    Roaster
    +
    ${data.roaster || '—'}
    +
    Roast Level
    +
    ${data.roast_profile || '—'}
    +
    Roasted
    +
    ${data.roast_date ? new Date(data.roast_date).toLocaleDateString() : '—'}
    +
    + `; + + // Fetch and display shots pulled for this coffee + displayShots(api, beanName); + + // Store bean name for adding shots + window.currentBeanName = beanName; + + } catch (err) { + console.error('Error loading coffee details:', err); + } +} + +/** + * Display shots pulled for this coffee + */ +async function displayShots(api, beanName) { + const shots = await api.getObservations('pull_shot'); + const relatedShots = shots.filter(obs => (obs.data || {}).bean === beanName); + + const table = document.getElementById('shots-table').querySelector('tbody'); + + if (relatedShots.length === 0) { + table.innerHTML = 'No shots pulled yet.'; + return; + } + + relatedShots.forEach(shot => { + const data = shot.data || {}; + table.innerHTML += ` + + ${data.yield || '—'} g + ${data.time || '—'} s + ${data.rating || '—'} + + `; + }); +} + +/** + * Open form to pull a new shot (with data injection) + */ +async function addShot() { + const beanName = window.currentBeanName; + if (!beanName) { + alert('Please select a coffee first.'); + return; + } + + try { + const api = await getFormulus(); + + // Open pull_shot form and inject the bean name + await api.openFormplayer( + 'pull_shot', + { bean: beanName }, // Injected values + {} // Additional options + ); + + // Refresh the page after user closes the form + location.reload(); + + } catch (err) { + console.error('Error opening form:', err); + alert('Failed to open form. See console.'); + } +} + +/** + * Initialize: get bean name from URL and load details + */ +document.addEventListener('DOMContentLoaded', () => { + const beanName = getQueryParam('bean'); + if (beanName) { + displayCoffee(decodeURIComponent(beanName)); + } +}); +``` + +### Updated style.css + +Update `app/style.css`: + +```css +body { + font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif; + background: linear-gradient(135deg, #6B4423 0%, #9C2C07 100%); + color: #333; + margin: 0; + padding: 16px; +} + +.container { + max-width: 800px; + margin: 0 auto; + background: white; + padding: 32px; + border-radius: 8px; + box-shadow: 0 4px 12px rgba(0, 0, 0, 0.1); +} + +h1 { + color: #9C2C07; + margin: 0 0 8px; +} + +h2 { + color: #333; + margin: 0 0 24px; + font-size: 24px; +} + +h3 { + color: #9C2C07; + margin-top: 32px; +} + +table { + width: 100%; + border-collapse: collapse; + margin: 16px 0; +} + +th, td { + padding: 12px; + text-align: left; + border-bottom: 1px solid #ddd; +} + +th { + background: #f5f5f5; + font-weight: bold; + color: #9C2C07; +} + +button { + background: #9C2C07; + color: white; + padding: 8px 16px; + border: none; + border-radius: 4px; + cursor: pointer; + font-size: 14px; +} + +button:hover { + background: #7a2306; +} + +.btn-primary { + margin: 16px 0; + padding: 12px 24px; + font-size: 16px; +} + +.back-btn { + margin-bottom: 16px; + background: #666; + padding: 8px 12px; +} + +.back-btn:hover { + background: #444; +} + +dl { + display: grid; + grid-template-columns: 100px 1fr; + gap: 12px; + margin: 16px 0; +} + +dt { + font-weight: bold; + color: #9C2C07; +} + +dd { + margin: 0; + color: #555; +} + +#loading { + text-align: center; + color: #666; + padding: 32px; +} +``` + +--- + +## Part 3: Bundle and upload + +Same as v1, but now with more files: + +```bash +zip -r coffee_tracker-v2.0.0.zip app/ forms/ +``` + +Upload to Synkronus: +```bash +./synk app-bundle upload ./coffee_tracker-v2.0.0.zip -a +``` + +--- + +## Part 4: Test in Formulus + +1. Sync the new app in Formulus +2. Open Coffee Tracker +3. Register a few coffees +4. Click "Details" to see the details page +5. Click "+ Pull Shot" to open the follow-up form + - Notice the bean name is pre-filled +6. Complete multiple shots for the same coffee +7. Return to details — see all related shots listed + +--- + +## How it works: Data injection + +Prefer `defaultData` (keys must match schema root properties): + +```javascript +await api.openFormplayer( + 'pull_shot', + { defaultData: { bean: beanName } }, + {} +) +``` + +Legacy flat keys on the params object (other than reserved bridge keys) are still accepted when `defaultData` is omitted. + +The Formulus API: +1. Opens the `pull_shot` form +2. Prefills matching schema fields from `defaultData` +3. User fills in the rest (yield, time, rating) +4. When submitted, the observation includes the injected value + +**Important — visibility vs injection:** Formplayer **clears** a field when its Control is hidden by a `SHOW`/`HIDE` rule. Injected stamps must live in the **schema** (and `defaultData`) and must **not** be bound to a Control that starts hidden. To show the value read-only, use SwipeLayout `headerFields` or a separate computed / `lbl_*` display field—not a hidden Control on the real property. See [Form design — Conditional Logic](./form-design.md#conditional-logic-in-ode-forms). + +--- + +## API reference + +The bridge also exposes synchronous `api.getProfileId()` and `api.getLocalStorageRef()` for profile-aware custom-app state. Use the storage reference instead of raw `localStorage` under the shared `file://` origin; see [profile-aware browser storage](./custom-applications.md#profile-aware-browser-storage) for examples and limitations. + +### getObservations(formName) + +```javascript +const observations = await api.getObservations('register_coffee'); +// Returns array of observation objects: [{ id, data, ... }, ...] +``` + +### openFormplayer(formName, injection, options) + +```javascript +await api.openFormplayer( + 'pull_shot', + { bean: 'Prainema', roaster: 'Local' }, + { readOnly: ['roaster'] } // Optional: make fields read-only +); +``` + +--- + +## Troubleshooting + +**API not available:** +- Ensure `formulus-load.js` is included +- Check browser console for errors + +**Data not injecting:** +- Verify field name matches schema +- Check `readOnly` in ui.json matches intended read-only fields + +**Shots not appearing:** +- Verify `bean` field value matches exactly (case-sensitive) +- Check Synkronus logs for form submission errors + +--- + +## Next steps + +You've built a complete longitudinal data collection app! Next: + +1. **Add dashboards** — Visualize shot results with charts +2. **Use a framework** — Upgrade from vanilla JS to React/SolidJS +3. **Set up CI/CD** — Auto-deploy on code changes +4. **Scale it** — Add more forms, more data relationships + +--- + +## Reference + +- **[Formulus API Docs](../reference/formulus.md)** — Full API reference +- **[JSONForms Spec](../reference/form-specifications.md)** — Form details +- **[App Bundle Format](../reference/app-bundle-format.md)** — Bundle structure +- **[Forum](https://forum.opendataensemble.org)** — Get help + +--- + +**Congratulations!** You've mastered longitudinal data. Time to pour that espresso! ☕ diff --git a/docs/docs/guides/building-custom-apps.md b/docs/docs/guides/building-custom-apps.md new file mode 100644 index 000000000..d64ea3be6 --- /dev/null +++ b/docs/docs/guides/building-custom-apps.md @@ -0,0 +1,86 @@ +--- +sidebar_position: 5 +title: Building Custom Apps +--- + +# Building Custom Apps + +A **custom_app** is a zipped archive containing two folders: + +``` +custom_app.zip +├── forms/ # Questionnaires (JSONForms specifications) +└── app/ # Application layer (HTML/JavaScript) +``` + +## What is a custom_app? + +In ODE, forms alone are just questionnaires. But custom_apps turn them into rich, interactive data collection experiences. You can: + +- **Look up existing observations** (filled-out form responses) +- **Display dashboards and summaries** of collected data +- **Open follow-up forms** with pre-populated values +- **Build longitudinal workflows** (registration → follow-ups) +- **Add helper text, validation, and guidance** for data collectors + +The app layer is built with **HTML and JavaScript** — if you can build a web page, you can build a custom_app. Support for frameworks like React, SolidJS, Angular is fully supported. + +## When to use custom apps + +Use custom_apps when you need: + +- **Longitudinal data collection** — follow-up questionnaires linked to initial registrations +- **Context-aware forms** — show relevant forms based on previous answers +- **Dashboards** — visualize or summarize collected data in real-time +- **Offline support** — access and display data on field devices +- **Smart workflows** — complex data collection journeys beyond simple forms + +## Key concepts + +### Forms vs. Observations + +- **Form** — The specification (schema.json + ui.json) that defines what data to collect +- **Observation** — A single filled-out instance of a form (the actual data) + +### JSONForms + +Custom apps use **JSONForms** extended with ODE-specific question types: +- Standard types: text, number, date, dropdown, radio, checkbox +- ODE extensions: photo, GPS, QR code, signature, **sub-observations** (embedded child payloads as JSON arrays — see [Custom Extensions](./custom-extensions.md#sub-observations-format-sub-observation)), and more + +### Data relationships + +In longitudinal workflows, observations are linked: +- **One-to-many**: One registered entity → many follow-up observations + - Example: One coffee bean registration → multiple shots pulled + +## Getting started + +Ready to build your first custom_app? See the step-by-step guides: + +- **[Your First Custom App (v1)](./building-custom-apps-v1.md)** — Learn the fundamentals with the Coffee Tracker example +- **[Longitudinal Data (v2)](./building-custom-apps-v2.md)** — Extend your app with follow-up observations + +## Example: Coffee Tracker + +Throughout these guides, we'll build **Coffee Tracker** — a simple but complete app for collecting information about roasted coffee beans and tracking espresso shots. + +**Version 1.0:** Basic registration form for coffee beans (forms + static landing page) +**Version 2.0:** Add longitudinal tracking of shots pulled, dashboards, and data injection + +## Upload to Synkronus + +Once built, upload via: +- **Web UI** — Synkronus Portal → App Bundle page +- **CLI** — `synk app-bundle upload path/to/app.zip -a` + +Then users sync by opening Formulus → Sync page → Update App Bundle. + +## Next steps + +1. Follow the [v1 guide](./building-custom-apps-v1.md) for a complete walkthrough +2. Extend with [v2 guide](./building-custom-apps-v2.md) for longitudinal features +3. Check the [Reference → App Bundle Format](../reference/app-bundle-format.md) for technical details + +**Need help?** +Join our [community forum](https://forum.opendataensemble.org) — we're here to help! diff --git a/docs/docs/guides/choice-lists.md b/docs/docs/guides/choice-lists.md new file mode 100644 index 000000000..7962c83db --- /dev/null +++ b/docs/docs/guides/choice-lists.md @@ -0,0 +1,583 @@ +--- +sidebar_position: 4 +--- + +# Choice lists + +This guide explains how dropdown menus work in ODE custom app forms. You do **not** need to be a programmer to follow the walkthroughs — you only need to edit JSON files in your app bundle and test on a device. + +--- + +## Two kinds of dropdowns + +In ODE forms, a “choice list” is any field where the user picks one option from a list. + +| Kind | Plain English | When to use it | +|------|---------------|----------------| +| **Shared choice list** | A **fixed menu** you define once (Yes/No, job roles, regions, …) and reuse in many forms | The options are **known in advance** and do not depend on data already collected | +| **Dynamic choice list** | A menu **filled from observations** already saved on the device (sites, participants, visits, …) | The options **come from earlier forms** or other records on the tablet | + +**Simple rule:** If the list could be printed on a paper protocol, use a **shared** list. If the list only makes sense after people have entered data in the field, use a **dynamic** list. + +**Files you will touch:** + +| Kind | Main files | +|------|------------| +| Shared | `forms/shared-choice-defs.schema.json` + each form’s `schema.json` | +| Dynamic | Only the form’s `schema.json` (and `ui.json` for layout) | + +**Related guides:** [Observation queries and local indexes](./observation-queries) (optional performance tuning for dynamic lists), [Form design](./form-design), [Custom extensions](./custom-extensions). + +--- + +## Part 1 — Shared choice lists + +### What you are building + +Imagine a single spreadsheet tab named **“All our dropdown menus”**. Every form can point at a row on that tab instead of copying the same options again and again. + +In ODE, that “spreadsheet tab” is one file: + +**`forms/shared-choice-defs.schema.json`** + +Each menu is a named block inside `$defs`. Forms connect to it with a **`$ref`** link. + +### How a shared list is stored + +Each option has: + +- **`const`** — the value saved in the database (short code, e.g. `yes`) +- **`title`** — the label the user sees (e.g. `Yes`) + +Example structure: + +```json +{ + "$id": "forms/shared-choice-defs.schema.json", + "$schema": "http://json-schema.org/draft-07/schema#", + "$defs": { + "yes_no": { + "oneOf": [ + { "const": "yes", "title": "Yes" }, + { "const": "no", "title": "No" } + ] + } + } +} +``` + +Use **snake_case** names for lists (`yes_no`, `region_list`, `priority_level`). + +### How shared lists render on device + +By default, a plain `oneOf` / `$ref` field renders as a **native HTML ` + + + ); +} +``` + +### React Web (Portal, Custom Apps) + +```javascript +// import components +import { Button, Card, Input } from '@ode/components/react-web'; + +function MyComponent() { + return ( + + + + + ); +} +``` + +### Custom App Integration + +Custom applications can use the component library for consistent styling: + +```javascript +// In custom app +import { Button, Card } from '@ode/components/react-web'; +import { buildTheme } from './theme'; + +function App() { + const theme = buildTheme('light'); + + return ( + + + + + + ); +} +``` + +## Best Practices + +### Token Usage +- **Use tokens, not hard-coded values**: Always reference design tokens +- **Semantic naming**: Use semantic token names (e.g., `primary` not `blue-500`) +- **Consistent spacing**: Use spacing tokens for all margins/padding + +### Component Development +- **Cross-platform compatibility**: Ensure components work on both platforms +- **Accessibility**: Include proper accessibility attributes +- **Performance**: Optimize for mobile and web performance +- **TypeScript**: Use TypeScript for all components + +### Theming +- **Theme agnostic**: Components should work with any theme +- **Dark mode support**: Ensure components work in dark mode +- **Custom themes**: Allow applications to override tokens + +## Migration Guide + +### From Hard-coded Styles + +**Before:** +```javascript +const styles = StyleSheet.create({ + button: { + backgroundColor: '#1976d2', + padding: 16, + borderRadius: 8, + }, +}); +``` + +**After:** +```javascript +const styles = StyleSheet.create({ + button: { + backgroundColor: tokens.colors.primary.main, + padding: tokens.spacing.md, + borderRadius: tokens.borderRadius.md, + }, +}); +``` + +### From Custom Components + +**Before:** +```javascript +// Custom button implementation +export const MyButton = ({ children, onPress }) => ( + + {children} + +); +``` + +**After:** +```javascript +// Use library component +import { Button } from '@ode/components/react-native'; + +export const MyButton = ({ children, onPress }) => ( + +); +``` + +## Troubleshooting + +### Common Issues + +**Components not found:** +- Ensure proper import path: `@ode/components/react-native` or `@ode/components/react-web` +- Check that packages are installed and linked correctly + +**Tokens not working:** +- Verify tokens are built: `pnpm run build` in tokens package +- Check import path: `@ode/tokens` + +**Theme not applying:** +- Ensure ThemeProvider wrapper for web components +- Check that tokens are properly integrated + +**Platform-specific issues:** +- Verify you're importing from the correct platform package +- Check platform-specific file extensions (.native.js, .web.js) + +This component library provides a solid foundation for building consistent, maintainable applications across the ODE ecosystem. diff --git a/docs/docs/guides/configuration.md b/docs/docs/guides/configuration.md new file mode 100644 index 000000000..4229cb862 --- /dev/null +++ b/docs/docs/guides/configuration.md @@ -0,0 +1,360 @@ +--- +sidebar_position: 4 +--- + +# Configuration + +Complete configuration guide for ODE components including server settings, client settings, and environment variables. + +## Server Configuration (Synkronus) + +Synkronus is configured using environment variables. You can use either environment variables directly or a `.env` file for local development. + +### Configuration File Locations + +For local development, create a `.env` file in one of these locations (searched in order): + +1. Current working directory (where you run the command from) +2. Same directory as the executable +3. Parent directory of the executable + +### Required Configuration + +| Variable | Description | Example | +|----------|-------------|---------| +| `DB_CONNECTION` | PostgreSQL connection string | `postgres://user:password@localhost:5432/synkronus?sslmode=disable` | +| `JWT_SECRET` | Secret key for JWT token signing (minimum 32 characters) | Generate with `openssl rand -base64 32` | + +### Optional Configuration + +| Variable | Default | Description | +|----------|---------|-------------| +| `PORT` | `8080` | HTTP server port | +| `LOG_LEVEL` | `info` | Logging level: `debug`, `info`, `warn`, or `error` | +| `APP_BUNDLE_PATH` | `./data/app-bundles` | Directory path for app bundle storage | +| `MAX_VERSIONS_KEPT` | `5` | Maximum number of app bundle versions to retain | +| `ADMIN_USERNAME` | `admin` | Initial admin username | +| `ADMIN_PASSWORD` | `admin` | Initial admin password (must be changed in production) | + +### Example Configuration File + +```bash +# Server Configuration +PORT=8080 +DB_CONNECTION=postgres://synkronus:password@localhost:5432/synkronus?sslmode=disable +JWT_SECRET=your-secret-key-change-this-in-production +LOG_LEVEL=info +APP_BUNDLE_PATH=./data/app-bundles +MAX_VERSIONS_KEPT=5 + +# Admin Configuration +ADMIN_USERNAME=admin +ADMIN_PASSWORD=change-this-password +``` + +### Database Connection String Format + +The `DB_CONNECTION` string follows PostgreSQL connection URI format: + +``` +postgres://[user[:password]@][host][:port][/database][?param1=value1&...] +``` + +**Components:** +- `user`: Database username +- `password`: Database password +- `host`: Database hostname or IP address +- `port`: Database port (default: 5432) +- `database`: Database name +- `sslmode`: SSL mode (`disable`, `require`, `verify-full`, etc.) + +**Examples:** + +```bash +# Local development +DB_CONNECTION=postgres://synkronus:password@localhost:5432/synkronus?sslmode=disable + +# Remote database +DB_CONNECTION=postgres://synkronus:password@db.example.com:5432/synkronus?sslmode=require + +# Docker Compose +DB_CONNECTION=postgres://synkronus:password@postgres:5432/synkronus?sslmode=disable +``` + +## Client Configuration (Formulus) + +The Formulus mobile app is configured through the app settings interface. + +### Server URL + +Enter the URL of your Synkronus server: + + + + +**Physical Device**: `http://192.168.1.100:8080` (your machine's IP address) + +**Android Emulator**: `http://10.0.2.2:8080` (special IP for emulator) + +**iOS Simulator**: `http://localhost:8080` or your machine's IP address + + + + +**HTTPS URL**: `https://synkronus.your-domain.com` + +**Custom Port**: `https://synkronus.your-domain.com:8443` (if using non-standard port) + + + + +### Authentication + +Configure authentication credentials: + +1. **Username**: Your user account username +2. **Password**: Your user account password +3. **Server URL**: As described above + +The app stores credentials securely and uses them for API authentication. + +### Sync Settings + +Configure synchronization behavior: + +- **Auto-sync**: Enable automatic synchronization when connectivity is available +- **Sync interval**: How often to check for sync (default: every 15 minutes) +- **Sync on app start**: Automatically sync when the app is opened +- **Sync on observation save**: Sync immediately after saving an observation + +## Synkronus CLI Configuration + +The Synkronus CLI uses a configuration file located at `$HOME/.synkronus.yaml` by default. + +### Configuration File Format + +```yaml +api: + url: http://localhost:8080 + version: 1.0.0 +``` + +### Multiple Endpoints + +You can manage multiple endpoint configurations: + +```bash +# Create separate config files +synk config init -o ~/.synkronus-dev.yaml +synk config init -o ~/.synkronus-prod.yaml + +# Point CLI at dev by default +synk config use ~/.synkronus-dev.yaml + +# Point CLI at prod by default +synk config use ~/.synkronus-prod.yaml + +# Temporarily override for a single command +synk --config ~/.synkronus-dev.yaml status +``` + +### Authentication + +The CLI stores authentication tokens in the configuration file after login: + +```bash +# Login to the API +synk login --username your-username + +# Check authentication status +synk status + +# Logout +synk logout +``` + +## Docker Configuration + +### Docker Compose Configuration + +When using Docker Compose, configuration is set in the `docker-compose.yml` file: + +```yaml +services: + synkronus: + image: ghcr.io/opendataensemble/synkronus:latest + environment: + PORT: 8080 + DB_CONNECTION: postgres://synkronus:password@postgres:5432/synkronus?sslmode=disable + JWT_SECRET: your-secret-key + LOG_LEVEL: info + APP_BUNDLE_PATH: /app/data/app-bundles + MAX_VERSIONS_KEPT: 5 + volumes: + - app-bundles:/app/data/app-bundles + depends_on: + - postgres +``` + +### Environment File + +You can also use a `.env` file with Docker Compose: + +```bash +# .env file +DB_CONNECTION=postgres://synkronus:password@postgres:5432/synkronus?sslmode=disable +JWT_SECRET=your-secret-key +LOG_LEVEL=info +``` + +Reference in `docker-compose.yml`: + +```yaml +services: + synkronus: + env_file: + - .env +``` + +## Security Configuration + +### JWT Secret + +Generate a strong JWT secret: + +```bash +# Using OpenSSL +openssl rand -base64 32 + +# Using PowerShell (Windows) +[Convert]::ToBase64String((1..32 | ForEach-Object { Get-Random -Minimum 0 -Maximum 256 })) +``` + +**Important:** Use a different secret for each environment (development, staging, production). + +### Database Passwords + +Generate strong database passwords: + +```bash +# Generate random password +openssl rand -base64 24 +``` + +### Admin Password + +Change the default admin password immediately after deployment: + +```bash +# Via API +curl -X POST https://your-server.com/users/change-password \ + -H "Authorization: Bearer YOUR_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{"current_password":"admin","new_password":"new-secure-password"}' +``` + +## Logging Configuration + +### Log Levels + +| Level | Description | Use Case | +|-------|-------------|----------| +| `debug` | Detailed diagnostic information | Development, troubleshooting | +| `info` | General informational messages | Production (default) | +| `warn` | Warning messages | Production | +| `error` | Error messages only | Production (minimal logging) | + +### Log Output + +Logs are written to standard output (stdout) and can be captured by: + +- Docker logging drivers +- Systemd journal +- Log aggregation services (e.g., Fluentd, Logstash) + +### Docker Logging + +Configure Docker logging in `docker-compose.yml`: + +```yaml +services: + synkronus: + logging: + driver: "json-file" + options: + max-size: "10m" + max-file: "3" +``` + +## Performance Configuration + +### PostgreSQL Settings + +Optimize PostgreSQL for your workload: + +```yaml +services: + postgres: + command: + - "postgres" + - "-c" + - "max_connections=100" + - "-c" + - "shared_buffers=256MB" + - "-c" + - "effective_cache_size=1GB" +``` + +### Resource Limits + +Set resource limits in Docker Compose: + +```yaml +services: + synkronus: + deploy: + resources: + limits: + cpus: '1.0' + memory: 512M + reservations: + cpus: '0.5' + memory: 256M +``` + +## Configuration Validation + +### Verify Configuration + +Test your configuration: + +```bash +# Check environment variables are set +docker compose config + +# Test database connection +docker compose exec synkronus sh -c 'apk add postgresql-client && psql "$DB_CONNECTION"' + +# Check server health +curl http://localhost:8080/health +``` + +### Common Configuration Issues + +**Problem**: Database connection fails + +**Solution**: Verify `DB_CONNECTION` string format and database accessibility. + +**Problem**: JWT authentication fails + +**Solution**: Ensure `JWT_SECRET` is at least 32 characters and matches across all instances. + +**Problem**: App bundles not persisting + +**Solution**: Verify `APP_BUNDLE_PATH` is set and volume is properly mounted. + +## Related Documentation + +- [Installation Guide](/docs/getting-started/installation) +- [Deployment Guide](/guides/deployment) +- [API Reference](/reference/api) diff --git a/docs/docs/guides/custom-applications.md b/docs/docs/guides/custom-applications.md new file mode 100644 index 000000000..35216b3ce --- /dev/null +++ b/docs/docs/guides/custom-applications.md @@ -0,0 +1,394 @@ +--- +sidebar_position: 2 +--- + +# Custom Applications + +Complete guide to building and deploying custom applications that integrate with ODE. + +> **Getting started?** Start with our hands-on [Building Custom Apps](../guides/building-custom-apps.md) tutorial, which walks you through a complete example (Coffee Tracker) from scratch. This reference covers more advanced patterns. + +## Overview + +Custom applications are **web applications** (HTML, CSS, and JavaScript) that run inside the Formulus mobile app’s WebView. + +:::note ODE monorepo vs your custom app +The **ODE repository** (Formulus, Formplayer, Synkronus Portal, design packages) uses **pnpm** — see [Development Setup](/docs/development/setup#package-manager-pnpm). **Your** custom app project can use **npm**, **pnpm**, or **yarn**; the examples below use common **npm** script names from the [custom_app](https://github.com/OpenDataEnsemble/custom_app) template. +::: You may author them with **any** stack—plain static files, **Vite**, **React**, **Vue**, **Svelte**, or another bundler—**as long as the build output** can be packaged as described in the [app bundle format](/docs/reference/app-bundle-format) (entry HTML, assets, and `forms/` layout). They provide specialized workflows, custom navigation, integration with the ODE form system, and interfaces tailored to your use case. + +## Profile-aware browser storage + +Formulus hosts each custom app for the active **profile**. Users add and switch profiles in the in-app **Profiles** screen; the host remounts the WebView when switching. The host scopes observations, attachments, bundle files, and Formplayer drafts to the active profile. Custom apps should use the synchronous profile-aware browser storage reference after the bridge is ready: + +```javascript +const api = await getFormulus(); +const profileId = api.getProfileId(); // stable for this WebView; not the display name +const storage = api.getLocalStorageRef(); +storage.setItem('lastTab', 'home'); +const lastTab = storage.getItem('lastTab'); +storage.removeItem('lastTab'); +// storage.clear() removes only this profile's custom-app keys. +``` + +The reference uses physical keys `ode:{profileId}:app:{key}`. Both helpers are synchronous, and storage errors propagate. Do **not** use raw `localStorage` for profile-specific state: custom apps and dependencies loaded under a shared `file://` origin may read or write raw storage outside the namespace. This is organizational/storage namespacing, **not a sandbox or confidentiality boundary**; deleting a profile cannot guarantee removal of unrelated third-party raw keys. Depending on WebView file-access settings and device behavior, code in a custom app or form extension may also read other profiles' data or attachments via file access. A connected Synkronus server can distribute app bundles containing code; not every server necessarily does so. Connect only to trusted servers and install only trusted bundles. Do not store secrets in browser storage. See the [Formulus bridge reference](../reference/formulus.md#getprofileid-and-getlocalstorageref). + +## Scaffolding + +ODE does **not** require a special installer: start from a **standard** project scaffold (for example **`npm create vite@latest`** with React, Svelte, or Solid templates) and then align the **folder layout** with the app bundle spec. Copy-paste commands, a **Vite `outDir` example**, and a post-scaffold checklist are maintained in the **[custom_app](https://github.com/OpenDataEnsemble/custom_app)** repository README on GitHub (AI and author context for the Formulus API and forms live in that repo as well). + +## Application Structure + +There is **no single mandatory** project layout. The tree below is **one** common pattern (React + Vite + optional `app.config.json` for theming). You can use a simpler folder tree if you prefer hand-written HTML/JS or a different framework, provided the **zip** you upload matches the [bundle format](/docs/reference/app-bundle-format). + +``` +my-app/ +├── app.config.json # Optional: app metadata and theme (if your template uses it) +├── forms/ # Form definitions (see bundle format) +│ ├── survey/ # One folder per form type (form name) +│ │ ├── schema.json # JSON Schema (draft-07) +│ │ └── ui.json # JSON Forms UI schema (ODE rules) +│ └── forms-manifest.json # Form registry (if used by your project) +├── src/ # Optional: only if you use a bundler (e.g. React) +│ ├── components/ +│ ├── screens/ +│ ├── utils/ +│ └── theme.js +├── scripts/ +├── package.json # Optional: if you use npm tooling +└── vite.config.js # Optional: example bundler config +``` + +## Configuration System + +### app.config.json + +If your template uses **`app.config.json`** (common in React-based examples), it can hold application metadata and theme. Plain HTML apps may omit it and configure styling in CSS/JS instead. When present, it is typically the single place for those settings: + +```json +{ + "name": "My Application", + "version": "1.0.0", + "navigation": { + "tabs": ["Home", "Forms", "Sync", "More"] + }, + "theme": { + "light": { + "primary": "#1976d2", + "primaryLight": "#42a5f5", + "primaryDark": "#1565c0", + "onPrimary": "#ffffff", + "background": "#fafafa", + "surface": "#ffffff", + "onBackground": "#212121", + "onSurface": "#424242" + }, + "dark": { + "primary": "#42a5f5", + "primaryLight": "#90caf9", + "primaryDark": "#1976d2", + "onPrimary": "#000000", + "background": "#121212", + "surface": "#1e1e1e", + "onBackground": "#ffffff", + "onSurface": "#e0e0e0" + } + }, + "observationIndexes": [ + { "key": "patient_id", "path": "$.patient_id" }, + { "key": "site_code", "path": "$.site_code", "formTypes": ["visit_*"] } + ] +} +``` + +`observationIndexes` declares **local-only** SQLite indexes for fast `getObservationsByQuery` filters on `data.*` fields. They are maintained on the device and are **not synced**. See [Observation queries](./observation-queries.md). + +### Theme Integration + +Themes are automatically generated from your configuration: + +```javascript +// src/theme.js +import { createTheme } from '@mui/material/styles'; +import appConfig from '../public/app.config.json'; + +export function buildTheme(mode = 'light') { + const colors = appConfig.theme[mode] ?? appConfig.theme.light; + + return createTheme({ + palette: { + mode, + primary: { + main: colors.primary, + light: colors.primaryLight, + dark: colors.primaryDark, + contrastText: colors.onPrimary, + }, + background: { + default: colors.background, + paper: colors.surface, + }, + // ... complete theme configuration + }, + }); +} +``` + +## Form Development + +### Form Structure + +Each form is a **directory** named with the **form type** (for example `survey/`). Inside it, two files are required: + +1. **`schema.json`**: [JSON Schema](https://json-schema.org/) (draft-07) defining data shape, validation, and question types (including ODE `format` values). See [Form specifications](/docs/reference/form-specifications). +2. **`ui.json`**: [JSON Forms](https://jsonforms.io/) **UI schema** defining layout (`VerticalLayout`, `Control`, `scope`, rules). ODE follows JSON Forms with project-specific rules—see [Form specifications](/docs/reference/form-specifications). The [app bundle format](/docs/reference/app-bundle-format) describes how these files sit inside the zip. + +Synkronus accepts **`forms//schema.json`** and **`ui.json`** at the **bundle root** (with **`forms/`** as a **sibling** of **`app/`**), or the alternate path **`app/forms//...`** where **`forms`** sits **inside** **`app`**. See [App bundle format](/docs/reference/app-bundle-format). + +### Example Form + +**schema.json:** +```json +{ + "type": "object", + "properties": { + "name": { + "type": "string", + "title": "Full Name" + }, + "age": { + "type": "integer", + "title": "Age", + "minimum": 0, + "maximum": 120 + }, + "email": { + "type": "string", + "title": "Email", + "format": "email" + } + }, + "required": ["name", "age"] +} +``` + +**ui.json:** +```json +{ + "type": "VerticalLayout", + "elements": [ + { + "type": "Control", + "scope": "#/properties/name" + }, + { + "type": "Control", + "scope": "#/properties/age" + }, + { + "type": "Control", + "scope": "#/properties/email" + } + ] +} +``` + +## Form Integration + +### Opening Forms + +Use the Formulus API to open forms: + +```javascript +import { openForm } from './utils/formulusApi'; + +async function handleFormSubmit() { + try { + const result = await openForm('survey', { + mode: 'create', + initialData: {} + }); + + if (result.status === 'completed') { + console.log('Form data:', result.data); + // Handle successful submission + } + } catch (error) { + console.error('Form error:', error); + } +} +``` + +### Mock API for Development + +During development, a mock API provides realistic behavior without needing the mobile app: + +```javascript +// src/utils/mockFormulusApi.js +export class MockFormulusAPI { + async openForm(formId, options = {}) { + // Simulate form opening with mock data + return { + status: 'completed', + data: { /* mock form data */ } + }; + } +} +``` + +## Build Process + +### Package.json Scripts + +```json +{ + "scripts": { + "copy-forms": "node scripts/copy-forms.js", + "dev": "npm run copy-forms && vite", + "build": "npm run copy-forms && vite build", + "zip": "node scripts/build-zip.js", + "validate:forms": "node scripts/validate-forms.js" + } +} +``` + +### Build Scripts + +**scripts/copy-forms.js:** +```javascript +import fs from 'fs-extra'; +import path from 'path'; + +const sourceDir = path.join(process.cwd(), '..', 'forms'); +const targetDir = path.join(process.cwd(), 'public', 'forms'); + +fs.copy(sourceDir, targetDir) + .then(() => console.log('Forms copied successfully')) + .catch(err => console.error('Error copying forms:', err)); +``` + +**scripts/build-zip.js:** +```javascript +import AdmZip from 'adm-zip'; +import path from 'path'; + +const zip = new AdmZip(); +const buildDir = path.join(process.cwd(), '..', 'app-bundles', 'app'); +const formsDir = path.join(process.cwd(), '..', 'forms'); + +zip.addLocalFolder(buildDir, 'app'); +zip.addLocalFolder(formsDir, 'forms'); + +zip.writeZip(path.join(process.cwd(), '..', 'bundle-v1.0.0.zip')); +``` + +## Form Validation + +### Validation Script + +**scripts/validate-forms.js:** +```javascript +import Ajv from 'ajv'; +import fs from 'fs-extra'; + +const ajv = new Ajv(); + +function validateForm(formPath) { + const schema = fs.readJsonSync(path.join(formPath, 'schema.json')); + const uiSchema = fs.readJsonSync(path.join(formPath, 'ui.json')); + + // Validate JSON Schema + const validate = ajv.compile(schema); + + // Validate UI schema references + // ... validation logic + + return { valid: true, errors: [] }; +} +``` + +## Development Workflow + +### Local Development + +1. **Setup**: `npm install` +2. **Development**: `npm run dev` (includes form copying) +3. **Validation**: `npm run validate:forms` +4. **Build**: `npm run build && npm run zip` + +### Testing Forms + +Use the built-in form explorer to test forms locally: + +```javascript +// Navigate to http://localhost:5173/#/forms +// Browse and test all forms with mock data +``` + +## Deployment + +### Bundle Creation + +1. Build the application: `npm run build` +2. Create deployment bundle: `npm run zip` +3. Upload bundle to Synkronus server +4. Deploy to mobile devices via sync + +### Version Management + +Update the version in `app.config.json` and `package.json`: + +```json +{ + "version": "1.1.0" +} +``` + +Bundle filename will automatically reflect the version: `bundle-v1.1.0.zip` + +## Best Practices + +### Form Design +- Use clear, descriptive field titles +- Implement proper validation rules +- Provide helpful error messages +- Consider offline usage scenarios + +### Performance +- Optimize bundle size (target < 200KB) +- Use lazy loading for large forms +- Implement efficient data structures +- Test on target devices + +### User Experience +- Follow Material Design 3 guidelines +- Implement consistent navigation +- Provide clear feedback for actions +- Handle errors gracefully + +### Code Organization +- Separate concerns (UI, logic, data) +- Use TypeScript for type safety +- Implement proper error handling +- Write maintainable, documented code + +## Troubleshooting + +### Common Issues + +**Forms not appearing:** +- Verify forms are copied to `public/forms/` +- Check `forms-manifest.json` format +- Validate form schemas + +**Build failures:** +- Ensure all dependencies are installed +- Check form validation errors +- Verify configuration syntax + +**Deployment issues:** +- Confirm bundle size limits +- Validate app configuration +- Test bundle extraction + +This guide provides the foundation for building robust custom applications that integrate seamlessly with the ODE ecosystem. + diff --git a/docs/docs/guides/custom-extensions.md b/docs/docs/guides/custom-extensions.md new file mode 100644 index 000000000..c158072bf --- /dev/null +++ b/docs/docs/guides/custom-extensions.md @@ -0,0 +1,701 @@ +--- +sidebar_position: 5 +--- + +# Custom Extensions + +Create custom question types and extend the Formulus Formplayer with specialized input fields. + +:::info Production Ready +The ODE extension system allows developers to package custom question types that work seamlessly with the form system. Extensions are deployed via app bundles and automatically available to all users. +::: + +## Overview + +The extension system enables you to: + +- **Custom Question Types** - Add specialized input components beyond built-in types +- **Business Logic** - Implement domain-specific validation and processing +- **Reusable Components** - Package extensions for distribution to other implementations +- **Automatic Distribution** - Deploy via app bundles; users get updates automatically + +## Profile-aware extensions + +Custom renderers and validators run in a Formplayer WebView for the **active host profile**. Formulus owns the profile's observation database, attachments, app bundle cache, and Formplayer draft storage. Switch profiles from the Formulus **Profiles** screen; switching remounts the WebView rather than changing its profile in place. + +For extension-owned browser preferences, use the synchronous bridge helpers `formulus.getProfileId()` and `formulus.getLocalStorageRef()` (after the bridge is ready). The latter supports `getItem`, `setItem`, `removeItem`, and `clear` for keys in `ode:{profileId}:app:{key}`; `clear` affects only this profile's app namespace. Avoid raw `localStorage`: a shared `file://` origin may expose raw keys to other profiles or third-party code. Profiles provide organizational/storage namespacing, **not a sandbox or confidentiality boundary**, nor a guarantee that third-party raw keys will be erased on profile deletion. Depending on WebView file-access settings and device behavior, code in a form extension or custom app may read other profiles' data or attachments via file access. A connected Synkronus server can supply bundles with such code, though connecting does not mean every server executes arbitrary code. Connect only to trusted servers and install only trusted app bundles; do not store secrets in browser storage. See [Formulus JavaScript interface](../reference/formulus.md#getprofileid-and-getlocalstorageref) and [custom-app storage guidance](./custom-applications.md#profile-aware-browser-storage). + +## Sub-observations (`format: sub-observation`) + +:::tip Built-in Formplayer control +Sub-observations are rendered by **Formplayer itself**. You declare them on the parent form’s JSON Schema. +::: + +Use sub-observations when related answers should live **inside the parent observation** as an **embedded JSON array** of child payloads, instead of separate top-level observations per child. + +### Behavior + +- The parent schema defines one property (often `type: "array"`) with `"format": "sub-observation"` plus the configuration keys below. +- Each completed child payload is plain JSON appended or updated in that array when the enumerator finishes the nested **child form**. +- **Add / Edit** opens the child form through the Formulus API [`openFormplayer`](../reference/formulus.md) with `{ subObservationMode: true }`. The nested session returns child JSON **without** creating a separate synced observation for each completion. +- **Remove** deletes one embedded payload from the parent array only (within the current parent draft or saved observation). + +### Schema configuration + +| Property | Required | Description | +|----------|----------|-------------| +| `format` | yes | Must be `"sub-observation"`. | +| `linkedForm` | yes | Child **form type** opened for add/edit (non-empty string). | +| `parentKey` | optional | When set, field name written on **new** child payloads linking back to the parent (for example a foreign key). When omitted, embedded repeats rely on data already nested in the parent JSON — typical when the child form does not need an injected parent id. | +| `parentValuePath` | recommended when `parentKey` is set | Dot path into **current parent form data** for that key’s value (falls back to parent `observationId` when absent). | +| `columns` | optional | `{ key, label }[]` entries for the on-screen summary list; if omitted, `displayField` drives a single summary column. | +| `displayField` | optional | Fallback field key for the summary column (default `observationId`). | +| `itemLabel` | optional | Singular name for each embedded item (for example `"room"`). When set, the add button shows `+ Add {itemLabel}`, the empty table shows `No {itemLabel}`, and delete confirmations fall back to `this {itemLabel}`. When omitted, legacy copy is unchanged (`+ Add observation`, etc.). | +| `orderBy` | optional | Sort embedded items by field: string field name or `{ key, direction }` (`asc` / `desc`). Without `key`, sorts by `createdAt` descending when present on payloads. | +| `allowDelete` | optional | Default `true`. | +| `subObservationInitValues` | optional | Map merged into **initial params** when **adding** a new embedded child. Values support templates `{{parentValue}}`, `{{currentInstanceId}}`, or `{{dot.path}}` into parent data. | +| `subObservationEditInitValues` | optional | Map merged **on top of** the saved child payload when **opening an existing** embedded item for edit—useful when parent-derived fields must be refreshed each time (often omitted). | +| `skipFinalize` | optional | When `true`, the nested child form **omits the Finalize page**; **Done** on the last content page runs the same submit path as Finalize. The child is still validated against **its own** `schema.json` (AJV + that form's custom validators) before `formData` is returned to the parent. Formulus also skips GPS `beginObservationSession()` and suppresses the success modal for this fast path. Can be set on the schema property or passed via `openFormplayer(..., { skipFinalize: true })`. | + +**`openFormplayer` options (custom apps):** `{ subObservationMode?, skipFinalize?, skipDraftSelection? }`. Use `skipDraftSelection: true` on **root** forms when the custom app orchestrates the session and must not show the draft picker (for example headless follow-up after `persistObservation`). Sub-observation sessions never offer the draft picker. + +**UI schema override:** On the parent `ui.json` Control, `options.addButtonLabel` sets the **full** add-button text (JSON Forms array convention). It takes precedence over `itemLabel` when both are set — useful for localized phrasing (for example `"+ Adicionar quarto"`). + +### Validation and `skipFinalize` + +`skipFinalize` does **not** defer validation to the root form. Each nested session is a separate Formplayer instance with its own `ui.json` and schema: + +| When | What validates | +|------|----------------| +| Child **Done** / submit (`skipFinalize` or Finalize page) | Child form only — required fields, AJV, `options.customValidators` on **that** form | +| Parent data change / parent Finalize | Parent form — including validators on embedded arrays at the parent level | + +On success, `SubObservationQuestionRenderer` merges `result.formData` into the parent array and closes the child modal immediately. Parent-level logic (denormalized indexes, cross-row rules, global sequence numbers) does **not** run inside the child session unless you duplicate it there or pass context in (see below). + +### Nested sessions and custom validators + +Custom validators run in the **active Formplayer session only**. For a multi-level embedded tree (for example `household → rooms[] → beds[] → persons[]`): + +- A validator on the **root** form's `rooms` control runs when **root** `data` changes — not when the enumerator adds a bed inside an open **room** sub-form. +- Put validators on **each form** where rows are added if numbering or summary columns must update as soon as the child returns (typical with `skipFinalize`). +- Use **config** (for example `scope: "household" | "quarto" | "cama"`) so one validator module can serve multiple form types. + +**Authoring checklist for auto-numbering embedded rows:** + +1. Root form — validator on the top-level sub-observation array; rebuild parent-only indexes (for example a flattened lookup array). +2. Each nested child form — validator on its own sub-observation array for local sequence fields (`bed_num`, `person_num`, …). +3. **Global** sequences across the whole tree — pass a read-only snapshot from the parent via flat `subObservationInitValues` / `subObservationEditInitValues` (single-token templates preserve JSON types), or wait for platform **parent context** (below). Strip ephemeral snapshot fields on root finalize so they are not persisted. + +See [Custom validators](#custom-validators-validators) and [Parent context across nesting levels](#parent-context-across-nesting-levels). + +### Parent context across nesting levels + +Nested `openFormplayer` sessions receive only the **current row** as `core.data`, not the full parent observation. That limits cross-sibling validation, global numbering, and extension helpers that need ancestor fields. + +**Today (workaround):** Copy needed parent slices into child **data** with flat init templates, for example `"household_rooms": "{{rooms}}"` on `subObservationInitValues`. Formplayer resolves **top-level string templates only** — not nested object maps. Mark snapshot fields `readOnly` and remove them in a root-level validator before persist. Distinct from `format: "form_context"` / `params.context`, which are better for session metadata than large tree copies. + +**Proposed (not yet in ODE):** `subObservationContext` — read-only parent snapshot resolved at open time, exposed to validators/extensions, **not** validated against the child schema and **not** merged into persisted child JSON. Until then, use init templates or duplicate validators per level. + +Example property on the parent schema: + +```json +{ + "linked_visits": { + "type": "array", + "format": "sub-observation", + "title": "Visits", + "linkedForm": "visit", + "parentKey": "household_id", + "parentValuePath": "hh_id", + "displayField": "visit_date", + "allowDelete": true, + "subObservationInitValues": { + "household_id": "{{parentValue}}" + } + } +} +``` + +:::note Types in legacy forms +Some forms use `type: ["array", "string"]` with `"format": "sub-observation"` for migration compatibility; Formplayer activates the control whenever `format` matches. +::: + +## Custom validators (`validators/`) + +Bundle **custom validators** alongside custom question types. Register them in the app manifest (`validators//index.js`); reference them from `ui.json` control `options.customValidators`. + +```json +{ + "type": "Control", + "scope": "#/properties/quartos", + "options": { + "customValidators": [ + { "name": "assignRepeatPositions", "config": { "quartosField": "quartos" } } + ] + } +} +``` + +**Mutating validators:** A validator may update `data` in place (for example auto-numbering embedded sub-observation rows or rebuilding a denormalized index array). Formplayer detects mutations after each change and before finalize, then refreshes form state so summary tables and dependent fields update immediately. Return validation errors in the usual way; returning patches is not required. + +**Per-session scope:** Mutations apply to the **current** form session. Nested sub-observations need validators on each level where rows are added, or a parent snapshot field (see [Parent context across nesting levels](#parent-context-across-nesting-levels)). Root-only validators are not enough for deep embedded trees. + +```json +{ + "type": "Control", + "scope": "#/properties/beds", + "options": { + "customValidators": [ + { + "name": "assignRepeatPositions", + "config": { "scope": "room", "bedsField": "beds" } + } + ] + } +} +``` + +See also [Form specifications](../reference/form-specifications.md) and [Formplayer](../reference/formplayer.md). + +## Quick Start + +### Creating a Custom Question Type + +A custom question type consists of: + +1. **TypeScript/JavaScript Component** - React component for rendering +2. **Type Definition** - JSON schema for form configuration +3. **Registration** - Entry in the extension registry + +**Example: Custom Phone Number Input** + +```typescript +// phone-number-type.tsx + +import React from 'react'; +import { Control } from 'react-hook-form'; + +interface PhoneNumberProps { + value: string; + onChange: (value: string) => void; + format: 'intl' | 'local'; // From UI Schema + country?: string; // From UI Schema + required?: boolean; + error?: string; +} + +export const PhoneNumberControl: React.FC = ({ + value, + onChange, + format, + country, + error, + required +}) => { + const handleChange = (e: React.ChangeEvent) => { + const input = e.target.value; + // Custom formatting logic + const cleaned = input.replace(/\D/g, ''); + const formatted = formatPhoneNumber(cleaned, format, country); + onChange(formatted); + }; + + return ( +
    + + {error && {error}} +
    + ); +}; + +// Helper functions +function formatPhoneNumber( + digits: string, + format: 'intl' | 'local', + country?: string +): string { + if (format === 'intl') { + return `+${digits}`; // International format + } + // Local format based on country + if (country === 'UG') { + return digits.replace(/(\d{3})(\d{2})(\d{6})/, '+256$2 $3'); + } + return digits; +} + +function getPlaceholder(format: 'intl' | 'local', country?: string): string { + if (format === 'intl') return '+256 701 234567'; + if (country === 'UG') return '0701 234567'; + return '(Enter phone number)'; +} +``` + +### Defining in Schema + +Define the custom type in your form schema: + +```json +{ + "schema": { + "type": "object", + "properties": { + "phone": { + "type": "string", + "title": "Phone Number", + "x-custom": { + "type": "phone-number", + "format": "intl", + "country": "UG" + } + } + } + }, + "uischema": { + "type": "VerticalLayout", + "elements": [ + { + "type": "Control", + "scope": "#/properties/phone" + } + ] + } +} +``` + +### Registering the Extension + +Extensions are registered when the app initializes using the extension system: + +```typescript +// In Formulus initialization or custom app setup + +import { registerExtension } from '@ode/formulus-extensions'; +import { PhoneNumberControl } from './phone-number-type'; + +registerExtension({ + name: 'phone-number', + renderer: PhoneNumberControl, + schema: { + type: 'string', + x-custom: { + type: 'phone-number' + } + }, + validation: { + pattern: '^\\+?\\d{6,15}$', + minLength: 6, + maxLength: 15 + }, + metadata: { + displayName: 'Phone Number', + description: 'International or local phone number input', + version: '1.0.0' + } +}); +``` + +## Extension Types + +### 1. Custom Question Type + +New input component for collecting specific data types: + +```typescript +interface QuestionTypeExtension { + name: string; // Unique type ID + renderer: React.ComponentType; // React component + supportedFormats?: string[]; // Optional format variants + validation?: ValidationSchema; // Validation rules + metadata: ExtensionMetadata; +} +``` + +**Examples:** +- Phone number with formatting +- GPS coordinate input with map +- Color picker +- Time range selector +- Signature capture with pressure + +### 2. Business Logic Extension + +Custom functions for validation, calculation, or data processing: + +```typescript +interface BusinessLogicExtension { + name: string; + description: string; + functions: { + [key: string]: (params: any) => any; + }; + metadata: ExtensionMetadata; +} + +// Example: Custom validation function +registerLogicExtension({ + name: 'advanced-validations', + functions: { + validateHouseholdStructure: (household) => { + // Validate household relationships + const adults = household.members.filter(m => m.age >= 18); + return adults.length > 0; + }, + calculateHouseholdSize: (household) => { + return household.members.length; + } + } +}); + +// Use in form +{ + "type": "object", + "properties": { + "members": { + "type": "array", + "x-validation": { + "function": "validateHouseholdStructure" + } + } + } +} +``` + +### 3. Data Enhancement Extension + +Augment observations with computed or retrieved data: + +```typescript +interface DataEnhancementExtension { + name: string; + enhancers: { + [key: string]: (obs: Observation) => Promise; + }; +} + +// Example: Fetch location name from coordinates +registerDataEnhancer({ + name: 'location-enrichment', + enhancers: { + reverseGeocode: async (observation) => { + const { lat, lng } = observation.data.location; + const response = await fetch( + `https://api.example.com/reverse?lat=${lat}&lng=${lng}` + ); + return { + location_name: response.locality, + location_admin: response.admin2, + location_country: response.country + }; + } + } +}); +``` + +## Packaging Extensions + +### App Bundle Structure + +Extensions are distributed as part of the app bundle: + +``` +app-bundle.zip +├── forms/ +│ └── *.json # Form definitions +├── question_types/ +│ ├── custom-phone-number.js +│ ├── custom-map-field.js +│ └── custom-signature.js +├── logic/ +│ ├── household-validations.js +│ └── calculations.js +├── styles/ +│ └── extensions.css # Custom CSS for extensions +└── metadata.json +``` + +### Metadata File + +Define extension metadata in `metadata.json`: + +```json +{ + "version": "1.2.0", + "description": "Custom question types for household surveys", + "forms": ["household", "hh_person", "hh_follow_up"], + "extensions": [ + { + "name": "phone-number", + "type": "question-type", + "description": "International phone number input", + "version": "1.0.0" + }, + { + "name": "location-detail", + "type": "question-type", + "description": "GPS with map preview", + "version": "1.1.0" + }, + { + "name": "household-validations", + "type": "business-logic", + "description": "Validation rules for household data", + "version": "1.0.0" + } + ], + "dependencies": [ + "formulus >= 1.0.0", + "formplayer >= 1.0.0" + ], + "author": "Your Organization", + "license": "MIT" +} +``` + +## Deployment + +### Upload App Bundle + +Use the Synkronus CLI or API to deploy: + +```bash +synk app-bundle upload path/to/bundle.zip +``` + +Or API: + +```bash +curl -X PUT https://synkronus.example.com/api/v1/app-bundle \ + -H "Authorization: Bearer $TOKEN" \ + -F "bundle=@bundle.zip" +``` + +### Versioning + +Maintain multiple versions: + +```bash +# List versions +synk app-bundle list + +# Activate specific version +synk app-bundle activate 1.2.0 + +# Previous version remains available for older clients +``` + +### Safe Rollback + +If issues occur: + +```bash +# Immediately activate previous version +synk app-bundle activate 1.1.0 + +# Clients will pull updated bundle on next sync +``` + +## Best Practices + +### Design + +✅ **Do:** +- Keep extensions focused and single-purpose +- Follow component composition patterns +- Reuse core Formulus components where possible +- Implement accessibility (ARIA labels, keyboard navigation) +- Support theme customization +- Validate input on every change + +❌ **Don't:** +- Create monolithic extensions doing too much +- Rely on external APIs without fallbacks +- Hard-code strings (use i18n) +- Break from standard form patterns +- Ignore edge cases +- Store sensitive data locally + +### Performance + +- Minimize bundle size (extensions increase app size) +- Lazy load if possible +- Cache expensive computations +- Avoid blocking operations +- Test on slow networks and low-end devices + +### Compatibility + +- Test across devices (Android 8+, iOS 13+) +- Support multiple screen sizes +- Handle orientation changes +- Verify offline functionality +- Test with real field data + +## Example Implementation + +### Complete Custom Time Range Picker + +```typescript +// time-range-picker.tsx + +import React, { useState } from 'react'; + +interface TimeRange { + start: string; // HH:MM + end: string; // HH:MM +} + +interface TimeRangePickerProps { + value: TimeRange; + onChange: (value: TimeRange) => void; + label?: string; +} + +export const TimeRangePicker: React.FC = ({ + value, + onChange, + label +}) => { + const [startTime, setStartTime] = useState(value?.start || ''); + const [endTime, setEndTime] = useState(value?.end || ''); + + const handleChange = (newStart: string, newEnd: string) => { + setStartTime(newStart); + setEndTime(newEnd); + onChange({ start: newStart, end: newEnd }); + }; + + const isValidRange = !startTime || !endTime || startTime <= endTime; + + return ( +
    + {label && } + +
    +
    + + handleChange(e.target.value, endTime)} + /> +
    + +
    to
    + +
    + + handleChange(startTime, e.target.value)} + /> +
    +
    + + {!isValidRange && ( +
    End time must be after start time
    + )} +
    + ); +}; +``` + +Register it: + +```typescript +registerExtension({ + name: 'time-range', + renderer: TimeRangePicker, + metadata: { + displayName: 'Time Range', + description: 'Select start and end times', + version: '1.0.0' + } +}); +``` + +Use in form: + +```json +{ + "working_hours": { + "type": "object", + "x-custom": { + "type": "time-range" + }, + "properties": { + "start": { "type": "string" }, + "end": { "type": "string" } + } + } +} +``` + +## Development Workflow + +1. **Setup** - Create React project for extension +2. **Develop** - Build and test component locally +3. **Test** - Verify in Formulus with test forms +4. **Package** - Bundle into app-bundle.zip +5. **Deploy** - Upload via CLI or API +6. **Verify** - Check deployment and monitor usage +7. **Update** - Continue improving based on feedback + +## Testing Extensions + +### Unit Tests + +```typescript +import { render, screen } from '@testing-library/react'; +import { TimeRangePicker } from './time-range-picker'; + +describe('TimeRangePicker', () => { + it('should accept time range', () => { + const onChange = jest.fn(); + render( + + ); + + const inputs = screen.getAllByRole('textbox'); + expect(inputs[0]).toHaveValue('09:00'); + expect(inputs[1]).toHaveValue('17:00'); + }); + + it('should validate time range', () => { + const { getByText } = render( + {}} + /> + ); + + expect(getByText(/End time must be after start time/)).toBeInTheDocument(); + }); +}); +``` + +### Integration Tests + +Test with actual forms and the Formulus environment. + +## Related Content + +- [Form Design](/docs/guides/form-design) - Learn about form structure and types +- [Custom Question Types](/docs/guides/custom-question-types) - Current renderer contract and packaging +- [Formplayer Reference](/docs/reference/formplayer) - Built-in question types +- [App Bundle Format](/docs/reference/app-bundle-format) - Full bundle specification +- [Deployment](/docs/guides/deployment) - Deploy to production \ No newline at end of file diff --git a/docs/docs/guides/custom-question-types.md b/docs/docs/guides/custom-question-types.md new file mode 100644 index 000000000..db4c7f678 --- /dev/null +++ b/docs/docs/guides/custom-question-types.md @@ -0,0 +1,207 @@ +# Custom Question Types + +Custom question types let you render your own UI for a single question — drag-and-drop ranking, a signature pad, a person picker with server-side search. They ship inside an [app bundle](/docs/guides/custom-applications), are loaded at runtime, and need no changes to Formulus or Formplayer. + +Custom question types are matched by the **`format`** property in your schema, not `type`. Built-in formats such as `photo`, `gps`, `signature`, `likert`, `duration`, and `sub-observation` already ship inside Formplayer — pick a new format name rather than shadowing one of those. + +## How it works + +1. **Package** — add a folder named after your format under `question_types/` in the app bundle, containing `renderer.js`. +2. **Declare** — set `"format": ""` on the field in `schema.json`. +3. **Scan** — when Formulus opens the form, it reads each renderer file and passes the **source text** to the Formplayer WebView in a manifest. +4. **Evaluate** — Formplayer evaluates the source in a sandbox where `React` and `MaterialUI` are provided as globals, then registers the component as a JSON Forms renderer for that format. +5. **Wrap** — every custom question type is wrapped in a shared question shell (label, description, required marker, validation message) and an error boundary, so your component only renders the input itself. + +## Where renderers live + +Formulus scans these directories, in order, and the **first** one to define a given folder name wins: + +| Order | Path | Notes | +|-------|------|-------| +| 1 | `app/question_types/` | **Current location.** Use this. | +| 2 | `app/forms/question_types/` | Legacy — still scanned, but shadowed by the path above | +| 3 | `/forms/question_types/` | Profile-local forms directory | + +Inside each format folder, Formulus looks for **`renderer.js`** first and falls back to **`index.js`**. A folder with neither is skipped with a warning and that format will not render. + +``` +app-bundle.zip +└── app/ + ├── manifest.json + ├── forms/ + │ └── household.json + └── question_types/ + ├── rating-stars/ + │ └── renderer.js <-- matched by "format": "rating-stars" + └── select-person/ + └── renderer.js <-- matched by "format": "select-person" +``` + +## Module format + +There is no bundler and no ES module support — the file is evaluated as a plain script with CommonJS shims. `React` and `MaterialUI` are injected as function parameters, so you can destructure them at the top of the file. + +The module must resolve to a **function**. Both of these work: + +```javascript +const { useState } = React; +const { Box, Typography } = MaterialUI; + +function RatingStars(props) { + return Box(null, String(props.value ?? '')); +} + +module.exports = { default: RatingStars }; +``` + +```javascript +module.exports = function RatingStars(props) { + return React.createElement('div', null, String(props.value ?? '')); +}; +``` + +If the module does not resolve to a function, that one format fails to load and is reported in the console; every other format in the bundle still loads. + +## Props reference + +| Prop | Type | Description | +|------|------|-------------| +| `value` | `unknown` | Current field value. The type follows the field's JSON Schema `type`. | +| `onChange` | `(newValue: unknown) => void` | Updates the field value. Pass a real JSON value, never a display string. | +| `config` | `Record` | Your custom schema properties (see below). | +| `options` | `Record` | Display settings from the `ui.json` `Control.options`, after locale preprocessing. | +| `validation.error` | `boolean` | Whether the field currently has a validation error. Use for styling only. | +| `validation.message` | `string` | **Always empty.** The shell renders the error copy — do not render this. | +| `enabled` | `boolean` | Whether the field is editable. | +| `visible` | `boolean` | JSON Forms relevance (SHOW/HIDE) result. The adapter already hides the component when this is `false`; it is exposed so renderers can react. | +| `fieldPath` | `string` | The field's JSON Pointer path, e.g. `#/properties/rating`. | +| `label` | `string` | The raw schema `title`. The shell renders the localized label — do not render it again. | +| `description` | `string` | The schema `description`, if present. Also rendered by the shell. | +| `jsonFormsContext` | `any` | JSON Forms context, including `core.data` (all form values), `core.schema` (root schema), and `core.errors`. | + +### What the shell already does + +Your component is rendered inside a shared shell, so **do not** re-implement any of this or it will appear twice: + +- the localized **label** and **description** +- the **required** marker +- the **validation error message** +- hiding the question when `visible` is `false` + +Use `validation.error` for a red border or similar affordance, and leave the text to the shell. + +### `config`: your schema properties + +Every schema property that is **not** reserved JSON Schema is passed through in `config`. Reserved keys are: + +`type`, `title`, `description`, `format`, `enum`, `const`, `default`, `required`, `properties`, `items`, `oneOf`, `anyOf`, `allOf`, `$ref`, `$schema`, `additionalProperties`, `pattern`, `minLength`, `maxLength`, `minimum`, `maximum`, `minItems`, `maxItems` + +Anything else — including keys starting with `_` or `x-` — arrives in `config`. + +### Numeric fields + +For numeric fields, **do not clamp to `minimum`/`maximum` on every keystroke.** Keep the in-progress text as local draft state while the field is focused, and commit the typed number (temporarily out of range values included) through `onChange`. AJV then reports the range violation through the normal error channel. Clamping as you type makes it impossible to clear or retype a value. + +## Example: `rating-stars` + +```javascript +const { useState, useEffect } = React; +const { Box, Typography, IconButton } = MaterialUI; + +function RatingStars(props) { + const { value, onChange, config, options, enabled } = props; + const maxStars = config.maxStars || 5; + const hint = options && options.hint ? options.hint : ''; + + const [draft, setDraft] = useState(null); + useEffect(function () { + setDraft(null); + }, [value]); + + const current = draft !== null ? draft : value; + const stars = []; + for (let n = 1; n <= maxStars; n++) { + stars.push( + IconButton( + { + key: n, + disabled: !enabled, + color: n <= (current || 0) ? 'primary' : 'default', + onClick: function () { + setDraft(null); + onChange(value === n ? null : n); + }, + }, + '*', + ), + ); + } + + return Box( + null, + Typography({ variant: 'body2', color: 'textSecondary' }, hint), + Box({ display: 'flex' }, stars), + ); +} + +module.exports = { default: RatingStars }; +``` + +## Using it in your form + +Set `format` to the folder name and add any parameters your component reads from `config`: + +```json +{ + "type": "object", + "title": "Rate this visit", + "format": "rating-stars", + "maxStars": 5 +} +``` + +With that schema, `props.config.maxStars === 5`. + +The field's JSON Schema `type` must match what you pass to `onChange()` — return an object for `"type": "object"`, an array for `"type": "array"` — so AJV validation passes. + +## Internationalization + +Custom question types use the **same** `ui.json` `translations` pattern as built-in controls. Put user-visible strings in `label`, `description`, and `Control.options` rather than hardcoding them in `renderer.js`: + +```json +{ + "type": "Control", + "scope": "#/properties/rating", + "label": "Rate this", + "options": { "hint": "Tap a star" }, + "translations": { + "pt": { + "label": "Avalie", + "options": { "hint": "Toque numa estrela" } + } + } +} +``` + +The renderer reads `props.options.hint`. Behavioural settings such as `maxStars` stay in `schema.json` and arrive in `config`. See [Form translations](/docs/guides/form-translations). + +## Loading media in the WebView + +Inside the WebView, `` cannot load legacy relative paths such as `/default/data/tables/...`. Use the injected `getFormulus()` API (see `FormulusInterfaceDefinition.ts` in Formulus / Formplayer): + +- **`getAttachmentUri(fileName)`** — returns a `file://` URL if that basename exists under the app attachments directory (or `pending_upload`), else `null`. Use the observation media `filename` / `photo.filename` basename. +- **`getAttachmentsUri()`** — base `file://` URL for the attachments folder (trailing slash). +- **`getCustomAppUri()`** — base `file://` URL for the app directory. +- **`getFormSpecsUri()`** — base `file://` URL for the form specs directory. + +## Error handling + +If your component throws while rendering, an error boundary catches it and shows a labelled fallback **in place of that question only**. The form keeps working and the remaining questions stay answerable, but the affected field cannot be edited until the component is fixed. The boundary's own text is translated by Formplayer. + +## Related: custom validators + +Custom **question types** render UI; custom **validators** (`validators//index.js` in the app bundle) run from `ui.json` `options.customValidators` and return errors. Validators may also **mutate** the full form `data` object in place — for example assigning sequence numbers on embedded sub-observation arrays — and Formplayer detects those mutations and refreshes state so tables and dependent fields update without extra custom question types. + +**Per-session scope:** Validators run only in the **active** Formplayer session. Nested sub-observation child forms need their own validators (or parent snapshot init fields) for numbering and cross-row rules; root-only validators are not enough for deep embedded trees. + +See [Custom Extensions](/docs/guides/custom-extensions) for validator packaging, [nested sessions](/docs/guides/custom-extensions#nested-sessions-and-custom-validators), [parent context](/docs/guides/custom-extensions#parent-context-across-nesting-levels), and sub-observation configuration (`linkedForm` required; `parentKey` optional). diff --git a/docs/docs/guides/deployment.md b/docs/docs/guides/deployment.md new file mode 100644 index 000000000..3f3ad05ba --- /dev/null +++ b/docs/docs/guides/deployment.md @@ -0,0 +1,595 @@ +--- +sidebar_position: 3 +--- + +# Deployment + +Complete guide to deploying ODE in production environments using containers (Docker or Podman). + +> **IT overview?** See [Server Architecture for IT](./server-architecture-for-it) for a one-page infrastructure summary. +> **Quick start?** For a fast setup with automated TLS, see the [Synkronus Quickstart](../getting-started/synkronus-quickstart.md) (Podman/Docker + Caddy + PostgreSQL 17). + +## Overview + +ODE production deployments center on the **Synkronus container image** (`ghcr.io/opendataensemble/synkronus`). The reference stack is [synkronus-quickstart](https://github.com/OpenDataEnsemble/synkronus-quickstart): Synkronus, PostgreSQL, and **Caddy** for TLS. Your IT team may use any hardened reverse proxy (Nginx, Apache, cloud load balancer) instead of Caddy—the requirement is **TLS termination** forwarding to Synkronus on port 8080. + +Pin the image tag in production (e.g. `ghcr.io/opendataensemble/synkronus:v1.3.2`), not `:latest`. + +## Recommended Production Setup + +For production deployment, we recommend: + +- **Clean Linux server** (Ubuntu 22.04 LTS or Debian 12) +- **Podman or Docker** with Compose +- **PostgreSQL** (container via quickstart, or managed service with `sslmode=require`) +- **TLS reverse proxy** (Caddy in quickstart; Nginx or institutional proxy equally valid) +- **Persistent volumes** for `pgdata` and Synkronus `appdata` +- **Backups** for database and `appdata` (see quickstart `utilities/`) +- **Security checklist** in [Security reference](/docs/reference/security) + +Optional: **Cloudflared tunnel** or similar if you prefer zero-trust ingress without opening inbound ports (not required when using a standard reverse proxy). + +## Quick Start + +### Server Preparation + + + + +```bash +# Update system +sudo apt update && sudo apt upgrade -y + +# Install Docker +curl -fsSL https://get.docker.com -o get-docker.sh +sudo sh get-docker.sh + +# Install Docker Compose +sudo apt install docker-compose-plugin -y + +# Verify installation +docker --version +docker compose version +``` + + + + +```bash +# Install Docker Desktop from https://www.docker.com/products/docker-desktop +# Or use Homebrew: +brew install --cask docker + +# Docker Compose is included with Docker Desktop +# Verify installation +docker --version +docker compose version +``` + + + + +1. **Install Docker Desktop** from [docker.com/products/docker-desktop](https://www.docker.com/products/docker-desktop) +2. **Docker Compose is included** with Docker Desktop +3. **Verify installation** (PowerShell): + ```powershell + docker --version + docker compose version + ``` + + + + +### Deploy Synkronus + +```bash +# Create deployment directory +mkdir -p ~/synkronus +cd ~/synkronus + +# Download configuration files +wget https://raw.githubusercontent.com/opendataensemble/ode/main/synkronus/docker-compose.example.yml -O docker-compose.yml +wget https://raw.githubusercontent.com/opendataensemble/ode/main/synkronus/nginx.conf + +# Generate secure secrets +JWT_SECRET=$(openssl rand -base64 32) +DB_ROOT_PASSWORD=$(openssl rand -base64 24) +ADMIN_PASSWORD=$(openssl rand -base64 16) + +# Update docker-compose.yml with secrets +sed -i "s/CHANGE_THIS_PASSWORD/$DB_ROOT_PASSWORD/g" docker-compose.yml +sed -i "s/CHANGE_THIS_TO_RANDOM_32_CHAR_STRING/$JWT_SECRET/g" docker-compose.yml +sed -i "s/CHANGE_THIS_ADMIN_PASSWORD/$ADMIN_PASSWORD/g" docker-compose.yml + +# Start the stack +docker compose up -d + +# Verify it's running +curl http://localhost/health +``` + +### Database Setup + +Create a database and user for Synkronus: + + + + +```bash +# Open a psql shell into the Postgres container +docker compose exec postgres psql -U postgres +``` + +From the `psql` prompt: + +```sql +-- Create role and database +CREATE ROLE synkronus_user LOGIN PASSWORD 'CHANGE_THIS_APP_PASSWORD'; +CREATE DATABASE synkronus OWNER synkronus_user; +``` + + + + +```bash +# Connect to local PostgreSQL +psql -U postgres +``` + +From the `psql` prompt: + +```sql +-- Create role and database +CREATE ROLE synkronus_user LOGIN PASSWORD 'CHANGE_THIS_APP_PASSWORD'; +CREATE DATABASE synkronus OWNER synkronus_user; +``` + + + + +Update the `synkronus` service `DB_CONNECTION` in `docker-compose.yml`: + +```yaml +services: + synkronus: + environment: + DB_CONNECTION: "postgres://synkronus_user:CHANGE_THIS_APP_PASSWORD@postgres:5432/synkronus?sslmode=disable" +``` + +## Using Pre-built Images + +Pre-built images are automatically published to GitHub Container Registry (GHCR) via CI/CD. + +### Choose an Image Tag + +Choose a deployment tag based on the update channel you want: + +| Tag | What it tracks | Recommended use | +|-----|----------------|-----------------| +| `latest` | Most recently published **stable** GitHub Release | Production deployments that intentionally auto-update between stable releases | +| `latest-pre-release` | Most recently published GitHub Release marked **pre-release** | Demo/staging deployments and Watchtower-managed pre-release testing | +| `dev` | Tip of the `dev` branch | Bleeding-edge integration testing; may contain unpublished work | +| `main` | Tip of the `main` branch | Testing current main between releases | +| `v1.2.3-alpha.4` | One specific pre-release | Reproducible pre-release deployment; does not auto-update | +| `v1.2.3` | One specific stable release | Reproducible production deployment; does not auto-update | +| `sha-abc1234` | One specific commit | Debugging or exact-build reproduction; does not auto-update | + +`dev` is a branch-head channel, **not** the published pre-release channel. To track published alpha or release-candidate images, use `latest-pre-release`: + +```bash +docker pull ghcr.io/opendataensemble/synkronus:latest-pre-release +``` + +Versioned and moving release tags are produced only when a GitHub Release is **published**. Merely pushing a Git tag is not enough, and the release must be marked as a pre-release for `latest-pre-release` to move. Publishing a stable release updates `latest` but does not update `latest-pre-release`; there is no single tag that tracks the newest release regardless of whether it is stable or pre-release. + +Feature-branch images are not published automatically. A manually dispatched workflow run publishes only an immutable `sha-{short}` tag and does not move `latest`, `latest-pre-release`, `main`, or `dev`. + +### Automatic Updates with Watchtower + +Watchtower follows the tag configured on the running container. For a demo server that should receive each published pre-release, configure the Synkronus service with the moving pre-release tag: + +```yaml +services: + synkronus: + image: ghcr.io/opendataensemble/synkronus:latest-pre-release +``` + +Use `latest` instead to follow stable releases. Do not use a versioned tag such as `v1.2.3-alpha.4` if you expect automatic upgrades; versioned and `sha-*` tags identify fixed builds. + +For production, pin a tested version tag and perform controlled upgrades rather than relying on an automatically moving tag. + +### Run Pre-built Image + +```bash +docker run -d \ + --name synkronus \ + -p 8080:8080 \ + -e DB_CONNECTION="postgres://user:password@host:5432/synkronus" \ + -e JWT_SECRET="your-secret-key" \ + -e APP_BUNDLE_PATH="/app/data/app-bundles" \ + -v synkronus-bundles:/app/data/app-bundles \ + ghcr.io/opendataensemble/synkronus:latest +``` + +## Cloudflared Tunnel Setup + +Cloudflared provides secure external access without exposing ports or managing SSL certificates. + +### Install Cloudflared + +```bash +# Download and install +wget https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-amd64.deb +sudo dpkg -i cloudflared-linux-amd64.deb + +# Verify installation +cloudflared --version +``` + +### Create Tunnel + +```bash +# Login to Cloudflare +cloudflared tunnel login + +# Create tunnel +cloudflared tunnel create synkronus + +# Note the tunnel ID from the output +``` + +### Configure Tunnel + +Create `~/.cloudflared/config.yml`: + +```yaml +tunnel: +credentials-file: /root/.cloudflared/.json + +ingress: + - hostname: synkronus.your-domain.com + service: http://localhost:80 + - service: http_status:404 +``` + +### Route DNS + +```bash +# Route your domain to the tunnel +cloudflared tunnel route dns synkronus synkronus.your-domain.com +``` + +### Run Tunnel as Service + +```bash +# Install as systemd service +sudo cloudflared service install + +# Start service +sudo systemctl start cloudflared +sudo systemctl enable cloudflared + +# Check status +sudo systemctl status cloudflared +``` + +Your Synkronus instance is now accessible at `https://synkronus.your-domain.com` with automatic SSL. + +## Environment Variables + +### Required Variables + +| Variable | Description | Example | +|----------|-------------|---------| +| `DB_CONNECTION` | PostgreSQL connection string | `postgres://user:pass@postgres:5432/synkronus` | +| `JWT_SECRET` | Secret key for JWT token signing | Generate with `openssl rand -base64 32` | + +### Optional Variables + +| Variable | Default | Description | +|----------|---------|-------------| +| `PORT` | `8080` | HTTP server port | +| `LOG_LEVEL` | `info` | Logging level (`debug`, `info`, `warn`, `error`) | +| `APP_BUNDLE_PATH` | `/app/data/app-bundles` | Path for app bundle storage | +| `MAX_VERSIONS_KEPT` | `5` | Number of app bundle versions to retain | +| `ADMIN_USERNAME` | `admin` | Initial admin username | +| `ADMIN_PASSWORD` | `admin` | Initial admin password (CHANGE THIS!) | + +## Volume Management + +### Persistent Volumes + +The docker-compose setup creates persistent volumes: + +1. **postgres-data**: PostgreSQL database files +2. **app-bundles**: Uploaded application bundles + +```bash +# List volumes +docker volume ls + +# Inspect volume +docker volume inspect synkronus_postgres-data + +# Backup volume +docker run --rm -v synkronus_postgres-data:/data -v $(pwd):/backup alpine tar czf /backup/postgres-backup.tar.gz /data + +# Restore volume +docker run --rm -v synkronus_postgres-data:/data -v $(pwd):/backup alpine tar xzf /backup/postgres-backup.tar.gz -C / +``` + +### App Bundle Directory Permissions + +When bind-mounting a host directory for `app-bundles`, ensure proper permissions. The container runs as user `synkronus` with `uid=1000` and `gid=1000`: + +```bash +# Fix permissions on host directory +sudo chown -R 1000:1000 ~/server/app-bundles + +# Restart after fixing permissions +docker compose restart synkronus +``` + +## Monitoring and Maintenance + +### View Logs + +```bash +# All services +docker compose logs -f + +# Specific service +docker compose logs -f synkronus +docker compose logs -f postgres +docker compose logs -f nginx + +# Last 100 lines +docker compose logs --tail=100 synkronus +``` + +### Health Checks + +```bash +# Check service status +docker compose ps + +# Test health endpoint +curl http://localhost/health + +# Via cloudflared tunnel +curl https://synkronus.your-domain.com/health +``` + +### Restart Services + +```bash +# Restart all services +docker compose restart + +# Restart specific service +docker compose restart synkronus + +# Reload nginx configuration +docker compose exec nginx nginx -s reload +``` + +### Update to Latest Version + +```bash +# Pull latest image +docker compose pull + +# Recreate containers with new image +docker compose up -d + +# Remove old images +docker image prune -f +``` + +## Backup and Restore + +### Database Backup + +```bash +# Create backup +docker compose exec postgres pg_dump -U synkronus_user synkronus > backup-$(date +%Y%m%d).sql + +# Automated daily backups (add to crontab) +0 2 * * * cd ~/synkronus && docker compose exec -T postgres pg_dump -U synkronus_user synkronus > /backups/synkronus-$(date +\%Y\%m\%d).sql +``` + +### Database Restore + +```bash +# Restore from backup +docker compose exec -T postgres psql -U synkronus_user synkronus < backup-20250114.sql +``` + +### Full System Backup + +```bash +# Backup everything +tar czf synkronus-full-backup-$(date +%Y%m%d).tar.gz \ + docker-compose.yml \ + nginx.conf \ + $(docker volume inspect synkronus_postgres-data --format '{{ .Mountpoint }}') \ + $(docker volume inspect synkronus_app-bundles --format '{{ .Mountpoint }}') +``` + +## Security Best Practices + +### 1. Use Strong Secrets + +```bash +# Generate strong JWT secret +openssl rand -base64 32 + +# Generate strong passwords +openssl rand -base64 24 +``` + +### 2. Change Default Admin Password + +After first deployment, change the admin password via API or CLI. + +### 3. Regular Updates + +```bash +# Update system packages +sudo apt update && sudo apt upgrade -y + +# Update Docker images +docker compose pull +docker compose up -d +``` + +### 4. Firewall Configuration + +If not using Cloudflared, configure firewall: + +```bash +sudo ufw allow 80/tcp +sudo ufw allow 443/tcp +sudo ufw enable +``` + +## Performance Tuning + +### Reverse proxy timeouts + +Field sync, photo upload, and app-bundle download can run for minutes on slow radio. The bundled [`nginx.conf`](https://github.com/OpenDataEnsemble/ode/blob/main/synkronus/nginx.conf) sets `proxy_send_timeout` and `proxy_read_timeout` to **600s**. If you use Caddy, Apache, or an institutional load balancer, set equivalent send/read (or idle) timeouts to at least 10 minutes. Leave login/refresh on the default short path — Synkronus already bounds `/api/auth/*` at 25s. + +### PostgreSQL Optimization + +Add to `docker-compose.yml` under postgres service: + +```yaml +command: + - "postgres" + - "-c" + - "max_connections=100" + - "-c" + - "shared_buffers=256MB" + - "-c" + - "effective_cache_size=1GB" +``` + +### Resource Limits + +Add to `docker-compose.yml` under each service: + +```yaml +deploy: + resources: + limits: + cpus: '1.0' + memory: 512M + reservations: + cpus: '0.5' + memory: 256M +``` + +## Architecture + +The deployment architecture includes: + +``` +┌─────────────────────────────────────────┐ +│ Cloudflared Tunnel │ +│ (Optional - Cloudflare) │ +│ Automatic SSL/TLS │ +└──────────────┬──────────────────────────┘ + │ HTTPS + ▼ +┌─────────────────────────────────────────┐ +│ Nginx Reverse Proxy │ +│ Port 80/443 │ +│ - Load balancing │ +│ - Request routing │ +│ - Compression │ +└──────────────┬──────────────────────────┘ + │ HTTP + ▼ +┌─────────────────────────────────────────┐ +│ Synkronus Container │ +│ Port 8080 (internal) │ +│ - API endpoints │ +│ - Business logic │ +│ - File storage │ +└──────────────┬──────────────────────────┘ + │ PostgreSQL protocol + ▼ +┌─────────────────────────────────────────┐ +│ PostgreSQL Database │ +│ Port 5432 (internal) │ +│ - Data persistence │ +│ - Transactions │ +└─────────────────────────────────────────┘ +``` + +## Troubleshooting + +### Service Won't Start + +```bash +# Check logs +docker compose logs synkronus + +# Check environment variables +docker compose config + +# Verify database connection +docker compose exec synkronus sh +# Inside container: +apk add postgresql-client +psql "$DB_CONNECTION" +``` + +### Database Connection Issues + +```bash +# Check PostgreSQL is running +docker compose ps postgres + +# Check PostgreSQL logs +docker compose logs postgres + +# Test connection from synkronus container +docker compose exec synkronus sh -c 'apk add postgresql-client && psql "$DB_CONNECTION"' +``` + +### Nginx Issues + +```bash +# Test nginx configuration +docker compose exec nginx nginx -t + +# Reload nginx +docker compose exec nginx nginx -s reload + +# Check nginx logs +docker compose logs nginx +``` + +## Production Checklist + +Before going live: + +- [ ] Strong JWT secret generated +- [ ] Strong database password set +- [ ] Admin password changed from default +- [ ] Cloudflared tunnel configured (or SSL certificates installed) +- [ ] Backup strategy implemented +- [ ] Monitoring configured +- [ ] Health checks passing +- [ ] Firewall configured (if not using Cloudflared) +- [ ] Resource limits set +- [ ] Log rotation configured +- [ ] Documentation reviewed +- [ ] Test deployment verified + +## Related Documentation + +- [Installation Guide](/docs/getting-started/installation) +- [Configuration Guide](/guides/configuration) +- [API Reference](/reference/api) diff --git a/docs/docs/guides/dynamic-choice-lists.md b/docs/docs/guides/dynamic-choice-lists.md new file mode 100644 index 000000000..6409d9571 --- /dev/null +++ b/docs/docs/guides/dynamic-choice-lists.md @@ -0,0 +1,18 @@ +--- +sidebar_position: 5 +slug: /guides/dynamic-choice-lists +--- + +# Dynamic choice lists + +This page has been **consolidated** into the full form-author guide: + +**[Choice lists →](./choice-lists)** (shared + dynamic choice lists) + +That guide includes step-by-step walkthroughs for both list types: + +- **Shared lists** — catalog file, `$ref`, Yes/No and “Other” patterns +- **Dynamic lists** — participant picker, distinct values, cascading dropdowns, filters +- **Optional performance** — `observationIndexes` in `app.config.json` + +If you bookmarked this URL, use [Choice lists](./choice-lists) going forward. diff --git a/docs/docs/guides/form-design.md b/docs/docs/guides/form-design.md new file mode 100644 index 000000000..4e77bd26a --- /dev/null +++ b/docs/docs/guides/form-design.md @@ -0,0 +1,1490 @@ +--- +sidebar_position: 1 +--- + +# Form Design + +Complete guide to designing forms in ODE using JSON schema and JSON Forms. + +## Overview + +ODE forms support optional **embedded translations** in `ui.json` (form-owned copy) separate from **ODE platform locales** (Formulus Settings → Language). For multi-locale forms, put display strings on `Control.label` / `Label.text` with a `translations` block — see [Form translations](/guides/form-translations). + +Forms in ODE are defined using JSON schema, following the JSON Forms specification. A form consists of two main components: + +1. **Schema**: Defines the data structure and validation rules +2. **UI Schema**: Defines how the form is presented to users + +## How Forms Work in ODE + +Understanding the mental model of forms in ODE is essential for effective form design. + +### What is a "Form" in ODE? + +A form in ODE is a structured data collection interface that consists of: + +- **JSON Schema**: Defines what data can be collected, its structure, types, and validation rules +- **UI Schema**: Defines how the form is presented to users, including layout, field ordering, and conditional visibility +- **Formplayer**: The rendering engine that interprets these schemas and creates the interactive form interface + +### The Role of JSON Schema + +JSON Schema serves as the **data contract** for your form: + +- **Structure Definition**: Defines the shape of data (properties, types, nesting) +- **Validation Rules**: Enforces data quality (required fields, ranges, formats) +- **Type Safety**: Ensures data types match expectations (string, number, boolean, etc.) + +JSON Schema follows the [JSON Schema Draft 7](https://json-schema.org/specification-links.html#draft-7) specification, but ODE Formplayer intentionally supports a **safe, predictable subset** of JSON Schema features. + +### The Role of UI Schema + +UI Schema (JSON Forms UI Schema) controls the **presentation layer**: + +- **Layout**: How fields are arranged (vertical, horizontal, grouped, paginated) +- **Ordering**: The sequence in which fields appear +- **Conditional Logic**: When fields are shown or hidden +- **Field Configuration**: Labels, placeholders, and display options + +For **multi-locale** forms, user-visible labels belong on `Control.label` in `ui.json` (with optional `translations`), not on `schema.json` `title`. See [Schema `title` vs UI `label`](/guides/form-translations#schema-title-vs-ui-label) in the form translations guide. + +### The Role of Formplayer + +Formplayer is the React-based rendering engine that: + +- **Interprets Schemas**: Reads JSON Schema and UI Schema to understand form structure +- **Renders Components**: Creates interactive form elements (inputs, selects, media capture, etc.) +- **Validates Input**: Enforces schema validation rules in real-time +- **Manages State**: Tracks form data, validation errors, and user interactions + +### Why ODE Supports a Subset of JSON Schema + +**Key Message**: ODE Formplayer intentionally supports a safe, predictable subset of JSON Schema and JSON Forms to ensure: + +1. **Reliability**: Forms work consistently across all devices and scenarios +2. **Performance**: Complex schema features don't slow down form rendering +3. **Predictability**: Form behavior is deterministic and easy to reason about +4. **Mobile Optimization**: Features work well in resource-constrained mobile environments + +**Important**: Forms that use unsupported JSON Schema features may load but are **not guaranteed to work**. Always refer to the [Formplayer Supported Schema & UI Profile](/reference/formplayer#supported-schema--ui-profile) for the definitive list of supported features. + +## Basic Form Structure + +Here's a simple form example: + +```json +{ + "schema": { + "type": "object", + "properties": { + "name": { + "type": "string", + "title": "Name" + }, + "age": { + "type": "integer", + "title": "Age", + "minimum": 0, + "maximum": 120 + } + }, + "required": ["name", "age"] + }, + "uischema": { + "type": "VerticalLayout", + "elements": [ + { + "type": "Control", + "scope": "#/properties/name" + }, + { + "type": "Control", + "scope": "#/properties/age" + } + ] + } +} +``` + +## Schema Definition + +The schema defines the data structure and validation rules for your form. It follows the JSON Schema specification. + +### Property Types + +ODE supports various property types: + +| Type | Description | Example | +|------|-------------|---------| +| `string` | Text input | Name, description | +| `integer` | Whole number | Age, count | +| `number` | Decimal number | Weight, temperature | +| `boolean` | True/false value | Consent, agreement | +| `array` | List of items | Multiple selections | +| `object` | Nested object | Complex data structures | + +### Validation Rules + +You can add validation rules to properties: + +```json +{ + "type": "string", + "title": "Email", + "format": "email", + "minLength": 5, + "maxLength": 100 +} +``` + +Common validation rules: + +- `minimum` / `maximum`: For numbers +- `minLength` / `maxLength`: For strings +- `pattern`: Regular expression pattern +- `format`: Predefined formats (email, date, etc.) +- `enum`: List of allowed values + +## Designing UI Schemas for ODE Formplayer + +The UI schema defines how form fields are presented to users. It controls layout, ordering, and presentation. ODE Formplayer has specific requirements and best practices for UI schema design. + +### Required Layout Structure + +**All ODE forms must use `SwipeLayout` as the root element.** This enables pagination and swipe navigation between form sections. + +#### SwipeLayout (Required Root) + +`SwipeLayout` is the root layout type that enables multi-page forms with swipe navigation. It automatically wraps other layout types if not explicitly specified. + +**Required Structure:** +```json +{ + "type": "SwipeLayout", + "elements": [ + // Each element becomes a swipeable page + ] +} +``` + +**Key Characteristics:** +- **Root Element**: Must be the top-level element in your UI schema +- **Pagination**: Each element in `elements[]` becomes a separate page +- **Swipe Navigation**: Users can swipe left/right to navigate between pages +- **Progress Tracking**: Shows progress bar indicating current page +- **Auto-wrapping**: If root is not SwipeLayout, Formplayer automatically wraps it + +**SwipeLayout `options`:** + +| Option | Values | Description | +|--------|--------|-------------| +| `labelLayout` | `"inline"` (default) \| `"stacked"` | Compact two-column layout (title left, input right) vs classic stacked fields | +| `headerFields` | string[] (max 2) | Read-only field keys shown under the progress bar on every page | +| `showInnerTitle` | boolean (default `false`) | Show the form `schema.title` in the inner header (off by default to avoid duplicating the Formulus chrome) | +| `autoFocusFirstInput` | boolean (default `true`) | Focus the first text input when a page opens (keeps the keyboard open across swipes) | +| `nextButtonLabel` | string | Override the Next button label | +| `finalizeButtonLabel` | string | Override the label on the last content page before Finalize | + +Per-field override: set `"options": { "labelLayout": "stacked" }` on a `Control` to force a full-width stacked row (useful for photo, signature, GPS, or wide button groups). + +**Safe Example:** +```json +{ + "type": "SwipeLayout", + "elements": [ + { + "type": "VerticalLayout", + "elements": [ + { + "type": "Control", + "scope": "#/properties/name" + }, + { + "type": "Control", + "scope": "#/properties/age" + } + ] + }, + { + "type": "VerticalLayout", + "elements": [ + { + "type": "Control", + "scope": "#/properties/email" + } + ] + } + ] +} +``` + +**Unsafe Example:** +```json +{ + "type": "VerticalLayout", // ❌ Not SwipeLayout - will be auto-wrapped + "elements": [...] +} +``` + +### Layout Types + +#### VerticalLayout + +Fields arranged vertically in a single column. + +**Required Fields:** +- `type`: `"VerticalLayout"` +- `elements`: Array of UI schema elements + +**Usage:** +```json +{ + "type": "VerticalLayout", + "elements": [ + { + "type": "Control", + "scope": "#/properties/field1" + }, + { + "type": "Control", + "scope": "#/properties/field2" + } + ] +} +``` + +#### HorizontalLayout + +Fields arranged horizontally in a row. + +**Required Fields:** +- `type`: `"HorizontalLayout"` +- `elements`: Array of UI schema elements + +**Usage:** +```json +{ + "type": "HorizontalLayout", + "elements": [ + { + "type": "Control", + "scope": "#/properties/firstName" + }, + { + "type": "Control", + "scope": "#/properties/lastName" + } + ] +} +``` + +#### Group + +Groups related fields together with a label. + +**Required Fields:** +- `type`: `"Group"` +- `label`: Group title (required) +- `elements`: Array of UI schema elements + +**Usage:** +```json +{ + "type": "Group", + "label": "Personal Information", + "elements": [ + { + "type": "Control", + "scope": "#/properties/name" + }, + { + "type": "Control", + "scope": "#/properties/email" + } + ] +} +``` + +**Note**: Groups can be used as pages within SwipeLayout. Each Group element in a SwipeLayout's `elements[]` becomes a separate page. + +### Control Configuration + +Controls bind UI elements to schema properties. + +**Required Fields:** +- `type`: `"Control"` +- `scope`: JSON pointer to schema property (must exist in schema) + +**Optional Fields:** +- `label`: Override field label +- `options`: Additional configuration + +**Safe Example:** +```json +{ + "type": "Control", + "scope": "#/properties/name", // ✅ Scope exists in schema + "label": "Full Name", + "options": { + "placeholder": "Enter your name" + } +} +``` + +**Unsafe Example:** +```json +{ + "type": "Control", + "scope": "#/properties/nonexistent" // ❌ Scope doesn't exist - will cause error +} +``` + +### Label Element + +Displays text labels within forms. + +**Required Fields:** +- `type`: `"Label"` +- `text`: Label text content + +**Usage:** +```json +{ + "type": "Label", + "text": "Section Introduction" +} +``` + +### UI Schema Best Practices + +#### Pagination Patterns + +**Recommended**: Use SwipeLayout with VerticalLayout pages for multi-step forms: + +```json +{ + "type": "SwipeLayout", + "elements": [ + { + "type": "VerticalLayout", + "elements": [ + { "type": "Label", "text": "Step 1: Basic Info" }, + { "type": "Control", "scope": "#/properties/name" }, + { "type": "Control", "scope": "#/properties/age" } + ] + }, + { + "type": "VerticalLayout", + "elements": [ + { "type": "Label", "text": "Step 2: Contact" }, + { "type": "Control", "scope": "#/properties/email" } + ] + } + ] +} +``` + +#### Grouping Best Practices + +**Use Groups for logical organization:** +```json +{ + "type": "SwipeLayout", + "elements": [ + { + "type": "Group", + "label": "Demographics", + "elements": [ + { "type": "Control", "scope": "#/properties/age" }, + { "type": "Control", "scope": "#/properties/gender" } + ] + }, + { + "type": "Group", + "label": "Health Information", + "elements": [ + { "type": "Control", "scope": "#/properties/height" }, + { "type": "Control", "scope": "#/properties/weight" } + ] + } + ] +} +``` + +#### Common Pitfalls + +**❌ Missing elements array:** +```json +{ + "type": "SwipeLayout" + // ❌ Missing elements - will cause error +} +``` + +**✅ Always include elements:** +```json +{ + "type": "SwipeLayout", + "elements": [] // ✅ Empty array is safe +} +``` + +**❌ Invalid scope paths:** +```json +{ + "type": "Control", + "scope": "#/properties/missingField" // ❌ Field doesn't exist in schema +} +``` + +**✅ Verify scope exists:** +```json +{ + "type": "Control", + "scope": "#/properties/existingField" // ✅ Field exists in schema +} +``` + +**❌ Nested SwipeLayout:** +```json +{ + "type": "SwipeLayout", + "elements": [ + { + "type": "SwipeLayout", // ❌ Nested SwipeLayout not supported + "elements": [...] + } + ] +} +``` + +**✅ Use VerticalLayout or Group inside SwipeLayout:** +```json +{ + "type": "SwipeLayout", + "elements": [ + { + "type": "VerticalLayout", // ✅ Use VerticalLayout for pages + "elements": [...] + } + ] +} +``` + +## Question Types + +ODE supports various question types through the Formplayer component. Question types are specified using the `format` property in the schema. + +### Basic Input Types + +**Text Input:** +```json +{ + "type": "string", + "title": "Name", + "format": "text" +} +``` + +**Number Input:** +```json +{ + "type": "integer", + "title": "Age", + "minimum": 0, + "maximum": 120 +} +``` + +**Date and Time:** +```json +{ + "type": "string", + "title": "Date", + "format": "date" +} +``` + +**Selection:** +```json +{ + "type": "string", + "title": "Choice", + "enum": ["option1", "option2"], + "enumNames": ["Option 1", "Option 2"] +} +``` + +**Boolean:** +```json +{ + "type": "boolean", + "title": "Consent" +} +``` + +### Sub-observations + +Use **`format: sub-observation`** on an array property for **embedded repeats**: each nested completion stores JSON on the parent observation. Adding or editing opens the linked child form in **sub-observation mode**, so Synkronus still receives **one** parent observation payload. + +See [Custom Extensions](./custom-extensions.md#sub-observations-format-sub-observation) for schema keys (`linkedForm`, optional `parentKey`, `parentValuePath`, `subObservationInitValues`, `skipFinalize`, templates, etc.). + +**`skipFinalize`:** Skips only the Finalize **page**; the child form still validates on **Done** before returning `formData` to the parent. Validation is **not** deferred to the root form. + +**Custom validators** (for example auto-numbering embedded rows) are declared in `ui.json` via `options.customValidators`. Validators run in the **active** form session — for nested trees, attach validators on **each** level where rows are added, not only on the root array. See [Custom Extensions — nested sessions](./custom-extensions.md#nested-sessions-and-custom-validators) and [parent context](./custom-extensions.md#parent-context-across-nesting-levels). + +## Control options + +Form authors can tune presentation and behaviour per field via the `options` object on a `Control` in `ui.json`. + +### Choice display + +Plain `oneOf` / shared `$ref` single-select fields default to a **native HTML ``) | +| `placeholder` | `string` | Native select placeholder | +| `multi` | `true` | Multi-line text input | + +**Scope Format:** +- Must start with `#/properties/` +- Examples: + - `"#/properties/name"` - Root property + - `"#/properties/person/properties/age"` - Nested property (requires explicit UI schema) + +#### Label + +**Structure:** +```json +{ + "type": "Label", + "text": "Instructions or information" +} +``` + +**Required:** +- `type`: `"Label"` +- `text`: String to display + +**Behavior:** +- Static text display +- No `elements` array needed +- Useful for instructions or section headers + +### Critical Requirements + +> ⚠️ **HARD RUNTIME ASSUMPTION** +> +> Formplayer assumes every layout element has an `elements` array. +> Missing this will cause runtime crashes in some render paths (e.g., "Cannot read properties of undefined (reading 'find')"). +> Always include `elements: []` even if the layout is empty. + +**All Layouts Must Have `elements` Array:** +```json +// ❌ UNSAFE - Missing elements +{ + "type": "Group", + "label": "Section" +} + +// ✅ SAFE - elements array present (even if empty) +{ + "type": "Group", + "label": "Section", + "elements": [] +} +``` + +> ⚠️ **HARD RUNTIME ASSUMPTION** +> +> Formplayer assumes all `Control.scope` paths exist in the schema. +> Invalid scopes may cause runtime crashes or silent rendering failures. +> Always validate scope paths exist before deployment. + +**All Scopes Must Exist in Schema:** +- Every `Control.scope` must reference a property in `schema.properties` +- Validation script checks this pre-deployment +- Runtime crashes if scope doesn't exist + +**Nested Objects Require Explicit UI Schema:** +- Object-typed properties don't auto-render nested fields +- Must provide UI schema for each nested property +- See [Object Handling](#object-handling) section + +--- + +## Validation Layer + +### Pre-Deployment Validation + +**Script:** `validate-forms.js` (found in app directories) + +**What's Enforced:** + +#### Schema Validation + +1. **Structure Checks:** + - `$schema` must be `"http://json-schema.org/draft-07/schema#"` + - Root `type` must be `"object"` + - Must have `properties` object + +2. **Compilation Check:** + - Schema must compile with Ajv (structural validity) + - Catches syntax errors, invalid keywords + +#### UI Schema Validation + +1. **Root Type:** + - Must be `SwipeLayout`, `VerticalLayout`, or `HorizontalLayout` + - (Note: Auto-wrapping happens at runtime, but validation checks input) + +2. **Element Types:** + - Valid types: `Control`, `Label`, `VerticalLayout`, `HorizontalLayout`, `SwipeLayout` + - Invalid types are flagged + +3. **Control Elements:** + - Must have `scope` property + - `scope` must start with `#/properties/` + +4. **Layout Elements:** + - `VerticalLayout`, `HorizontalLayout`, `SwipeLayout` must have `elements` array + - Missing `elements` is flagged as error + +5. **Rules:** + - `rule.effect` must be `SHOW`, `HIDE`, `ENABLE`, or `DISABLE` + - `rule.condition.scope` must start with `#/properties/` + +#### Field Reference Validation + +- All `Control.scope` paths must exist in schema properties +- All `rule.condition.scope` paths must exist in schema properties +- Prevents runtime crashes from invalid references + +### Runtime Validation + +**When:** During form interaction (`validationMode="ValidateAndShow"`) + +**What's Validated:** +- Data values against schema constraints +- Required fields +- Type constraints (string, number, etc.) +- Format validators (date, email, etc.) +- Custom format validators (photo, gps, etc.) + +**What's NOT Validated:** +- Schema structure (assumed valid from pre-deployment) +- UI schema structure (assumed valid from pre-deployment) +- Field existence (assumed valid from pre-deployment) + +### What's NOT Enforced (Runtime Only) + +These issues are **not caught** by validation scripts but **will cause runtime errors**: + +1. **`$data` References:** + - Validation script doesn't check for `$data` + - Runtime error: `"minimum value must be ['number']"` + +2. **Missing `elements` on Nested Layouts:** + - Validation only checks root and direct children + - Deeply nested missing `elements` may not be caught + - Runtime error: `"Cannot read properties of undefined (reading 'find')"` + +3. **Invalid Rule Scopes:** + - Validation checks scope format, not existence in all cases + - May cause unexpected rule behavior + +--- + +## Custom Formats & Renderers + +### Supported Custom Formats + +All custom formats are registered in Ajv and have corresponding custom renderers: + +| Format | Schema Type | Renderer | Description | +|--------|-------------|----------|-------------| +| `photo` | `object` | `PhotoQuestionRenderer` | Camera capture | +| `gps` | `string` or `object` | `GPSQuestionRenderer` | GPS coordinates | +| `signature` | `object` | `SignatureQuestionRenderer` | Signature pad | +| `qrcode` | `string` or `object` | `QrcodeQuestionRenderer` | Barcode scanner | +| `audio` | `object` | `AudioQuestionRenderer` | Audio recording | +| `video` | `object` | `VideoQuestionRenderer` | Video recording | +| `select_file` | `object` | `FileQuestionRenderer` | Generic file attachment (picker) | + +### Format Registration + +**Ajv Registration** (App.tsx): +```typescript +ajv.addFormat('photo', () => true); // Accepts any value +ajv.addFormat('qrcode', () => true); +ajv.addFormat('signature', () => true); +ajv.addFormat('select_file', () => true); +ajv.addFormat('audio', () => true); +ajv.addFormat('gps', () => true); +ajv.addFormat('video', () => true); +``` + +**Renderer Registration** (App.tsx:164-175): +```typescript +export const customRenderers = [ + { tester: photoQuestionTester, renderer: PhotoQuestionRenderer }, + { tester: qrcodeQuestionTester, renderer: QrcodeQuestionRenderer }, + { tester: signatureQuestionTester, renderer: SignatureQuestionRenderer }, + { tester: fileQuestionTester, renderer: FileQuestionRenderer }, + { tester: audioQuestionTester, renderer: AudioQuestionRenderer }, + { tester: gpsQuestionTester, renderer: GPSQuestionRenderer }, + { tester: videoQuestionTester, renderer: VideoQuestionRenderer }, +]; +``` + +### Format Requirements + +**Photo:** +```json +// Schema +{ + "patient_photo": { + "type": "object", + "format": "photo", + "title": "Patient Photo" + } +} + +// UI Schema +{ + "type": "Control", + "scope": "#/properties/patient_photo" +} +``` + +**File (`select_file`):** + +Use `type: object` with `format: select_file`. The Formplayer stores **basename-only** attachment keys and portable metadata (mime type, size, extension, optional original picker display name). The native app copies the picked file into **`attachments/draft/`** (same layout as photos). The UI shows the **filename only**—there is no embedded preview. + +```json +// Schema +{ + "supporting_doc": { + "type": "object", + "format": "select_file", + "title": "Supporting document" + } +} + +// UI Schema +{ + "type": "Control", + "scope": "#/properties/supporting_doc" +} +``` + +**GPS:** +```json +// Schema (string format) +{ + "location": { + "type": "string", + "format": "gps", + "title": "Location" + } +} + +// OR (object format) +{ + "location": { + "type": "object", + "format": "gps", + "title": "Location" + } +} + +// UI Schema +{ + "type": "Control", + "scope": "#/properties/location" +} +``` + +**Key Points:** +- ✅ No special UI schema required (standard `Control` is sufficient) +- ✅ Format must be specified in schema +- ✅ Attachment-backed builtins (**`photo`**, **`audio`**, **`video`**, **`select_file`**) use **`type: object`** in current Formplayer (match JSON Forms testers in `App.tsx`). Other formats may accept `string` or `object` (e.g. **`gps`**, **`qrcode`**). +- ✅ Custom renderer handles all UI and interaction + +--- + +## Rules & Conditional Logic + +### Rule Structure + +```json +{ + "type": "Control", + "scope": "#/properties/targetField", + "rule": { + "effect": "SHOW|HIDE|ENABLE|DISABLE", + "condition": { + "scope": "#/properties/sourceField", + "schema": { + // Condition schema (const, enum, etc.) + } + } + } +} +``` + +### Rule Effects + +| Effect | Behavior | +|--------|----------| +| `SHOW` | Element is hidden until condition is true | +| `HIDE` | Element is shown until condition is true | +| `ENABLE` | Element is disabled until condition is true | +| `DISABLE` | Element is enabled until condition is true | + +> **Clear-on-hide:** For `SHOW` and `HIDE`, when a Control becomes not visible Formplayer **deletes that field’s value** from form data (see `useClearOnHide`). Use visibility rules only for answers that should reset when irrelevant. Keep injected / stamp fields in the schema and `defaultData` **without** a hidden Control—display via `headerFields` or a separate computed/`lbl_*` field if needed. Details: [Form design — Conditional Logic](../guides/form-design.md#conditional-logic-in-ode-forms). + +### Condition Schema + +**Supported Condition Types:** + +1. **Constant Match:** +```json +{ + "condition": { + "scope": "#/properties/field", + "schema": { "const": "value" } + } +} +``` + +2. **Enum Match:** +```json +{ + "condition": { + "scope": "#/properties/field", + "schema": { "enum": ["value1", "value2"] } + } +} +``` + +3. **Boolean Match:** +```json +{ + "condition": { + "scope": "#/properties/field", + "schema": { "const": true } + } +} +``` + +### Rule Evaluation + +**How Rules Work:** +- Rules are evaluated by JSON Forms core (not custom Formplayer code) +- Condition is evaluated by validating the referenced field's value against `condition.schema` +- If validation passes, effect is applied +- Evaluation happens on every data change + +**Undefined Scope Behavior:** +- JSON Forms treats undefined scopes as condition success (by default) +- Formplayer doesn't add defensive checks +- **Best Practice:** Ensure all rule condition scopes exist in schema + +> ⚠️ **HARD RUNTIME ASSUMPTION** +> +> Formplayer assumes all rule condition scopes exist in the schema. +> Invalid scopes may cause rules to fail silently or exhibit undefined behavior. +> JSON Forms treats undefined scopes as condition success by default, which may not match intended behavior. + +### Critical Requirements + +**Rule Condition Scope Must Exist:** +```json +// ❌ UNSAFE - Scope doesn't exist +{ + "rule": { + "condition": { + "scope": "#/properties/nonexistent", // Field doesn't exist + "schema": { "const": "value" } + } + } +} + +// ✅ SAFE - Scope exists in schema +{ + "schema": { + "properties": { + "hasConsent": { "type": "boolean" } + } + }, + "uischema": { + "rule": { + "condition": { + "scope": "#/properties/hasConsent", // Field exists + "schema": { "const": true } + } + } + } +} +``` + +**Rule Condition Schema Must Be Simple:** +```json +// ❌ UNSAFE - Complex schema not supported +{ + "condition": { + "schema": { + "if": { "type": "string" }, + "then": { "const": "value" } + } + } +} + +// ✅ SAFE - Simple const or enum +{ + "condition": { + "schema": { "const": "value" } + } +} +``` + +### Rule Examples + +**Show When Value Equals:** +```json +{ + "type": "Control", + "scope": "#/properties/detailField", + "rule": { + "effect": "SHOW", + "condition": { + "scope": "#/properties/showDetail", + "schema": { "const": true } + } + } +} +``` + +**Hide When Value Equals:** +```json +{ + "type": "Control", + "scope": "#/properties/skipReason", + "rule": { + "effect": "HIDE", + "condition": { + "scope": "#/properties/completed", + "schema": { "const": true } + } + } +} +``` + +**Show When One of Multiple Values:** +```json +{ + "type": "Control", + "scope": "#/properties/referralForm", + "rule": { + "effect": "SHOW", + "condition": { + "scope": "#/properties/testResult", + "schema": { "enum": ["positive", "inconclusive"] } + } + } +} +``` + +**Group-Level Rules:** +```json +{ + "type": "Group", + "label": "TB Screening", + "elements": [...], + "rule": { + "effect": "SHOW", + "condition": { + "scope": "#/properties/screeningType", + "schema": { "const": "tb" } + } + } +} +``` + +--- + +## Renderer Boundaries + +### SwipeLayoutRenderer + +**File:** `SwipeLayoutRenderer.tsx` + +**Critical Assumptions:** +- `uischema.elements` exists (defensive: `|| []` at line 53) +- `layouts[currentPage]` exists (uses optional chaining at line 89) +- `layouts.length > 0` checked before rendering (line 126) + +**What Must Exist:** +- `uischema.type` (checked at line 48) +- `uischema.elements` (defensive fallback to `[]`) + +**Where Undefined Causes Crashes:** +- If `uischema` is `null` or `undefined` (shouldn't happen after normalization) +- If `layouts[currentPage]` is accessed when `currentPage >= layouts.length` (guarded by length check) + +### FinalizeRenderer + +**File:** `FinalizeRenderer.tsx` + +**Critical Assumptions:** +- `fullUISchema.elements` exists (guarded at line 139) +- `screen.elements` exists before iteration (guarded at line 153) +- `fullSchema.properties` exists (guarded at line 177) + +**What Must Exist:** +- `fullUISchema` and `fullUISchema.elements` (guarded) +- `screen.elements` for each screen (guarded with `'elements' in screen`) +- `fullSchema.properties` (guarded) + +**Where Undefined Causes Crashes:** +- `screen.elements.find()` if `screen.elements` is undefined (guarded at line 153) +- `fullUISchema.elements.forEach()` if `elements` is missing (guarded at line 139) +- `schema.properties[key]` if `properties` is missing (guarded at line 177) + +### Structural Assumptions + +**Guaranteed by Normalization:** +1. Root is always `SwipeLayout` (via `ensureSwipeLayoutRoot()`) +2. `Finalize` element always present (via `processUISchemaWithFinalize()`) +3. Root has `elements` array (created if missing) + +**Not Guaranteed (Must Be Enforced):** +1. Nested layouts have `elements` arrays (validation script checks, but deep nesting may be missed) +2. All `Control.scope` paths exist (validation script checks) +3. All `rule.condition.scope` paths exist (validation script checks format, not always existence) + +**Defensive Checks in Code:** +- Most renderers use optional chaining (`?.`) and fallbacks (`|| []`) +- Some code paths assume structure exists (crashes if assumption fails) +- **Best Practice:** Always include `elements: []` even if empty + +--- + +## Error Patterns & Solutions + +### Error: "minimum value must be ['number']" + +**Cause:** +- Using `$data` reference or non-numeric value for `minimum`/`maximum` +- Ajv expects literal numbers, not references + +**Example:** +```json +// ❌ CAUSES ERROR +{ + "type": "integer", + "minimum": { "$data": "/otherField" } +} + +// ✅ FIX +{ + "type": "integer", + "minimum": 0 +} +``` + +**Solution:** +- Use literal numbers only for `minimum`, `maximum`, `exclusiveMinimum`, `exclusiveMaximum`, `multipleOf` +- If dynamic constraints needed, handle at application level, not schema level + +### Error: "Cannot read properties of undefined (reading 'find')" + +**Cause:** +- Accessing `.find()` or other array methods on undefined `elements` array +- Layout element missing `elements` property + +**Example:** +```json +// ❌ CAUSES ERROR +{ + "type": "Group", + "label": "Section" + // Missing "elements" array +} + +// ✅ FIX +{ + "type": "Group", + "label": "Section", + "elements": [] // Always include, even if empty +} +``` + +**Solution:** +- Always include `elements: []` for all layout types +- Validation script should catch this, but check nested layouts manually +- Defensive code exists in some renderers, but not all paths are protected + +### Error: Rule Condition Fails Unexpectedly + +**Cause:** +- Rule condition scope doesn't exist in schema +- Rule condition scope points to undefined data +- Complex condition schema not supported + +**Example:** +```json +// ❌ CAUSES ISSUES +{ + "rule": { + "condition": { + "scope": "#/properties/nonexistent", // Field doesn't exist + "schema": { "const": "value" } + } + } +} + +// ✅ FIX +// Ensure field exists in schema: +{ + "schema": { + "properties": { + "hasConsent": { "type": "boolean" } + } + }, + "uischema": { + "rule": { + "condition": { + "scope": "#/properties/hasConsent", // Field exists + "schema": { "const": true } + } + } + } +} +``` + +**Solution:** +- Validate all rule condition scopes exist in schema +- Use simple condition schemas (`const` or `enum`) +- Test rules with missing data to verify behavior + +### Error: Object Properties Not Rendering + +**Cause:** +- Object-typed property without explicit UI schema for nested fields +- JSON Forms doesn't auto-render nested object properties + +**Example:** +```json +// ❌ NESTED FIELDS WON'T RENDER +{ + "schema": { + "properties": { + "gps_location": { + "type": "object", + "properties": { + "latitude": { "type": "number" }, + "longitude": { "type": "number" } + } + } + } + }, + "uischema": { + "type": "Control", + "scope": "#/properties/gps_location" + // Missing UI schema for nested fields + } +} + +// ✅ FIX - Explicit UI Schema +{ + "uischema": { + "type": "Group", + "label": "GPS Location", + "elements": [ + { "type": "Control", "scope": "#/properties/gps_location/properties/latitude" }, + { "type": "Control", "scope": "#/properties/gps_location/properties/longitude" } + ] + } +} +``` + +**Solution:** +- Provide explicit UI schema for nested object properties +- Use `Group` or `VerticalLayout` to organize nested fields +- Exception: Custom format objects (photo, gps, etc.) don't need nested UI schema + +--- + +## Safe vs Unsafe Patterns + +### Schema Patterns + +#### ✅ Safe: Literal Validation Values +```json +{ + "type": "integer", + "minimum": 0, + "maximum": 100 +} +``` + +#### ❌ Unsafe: $data References +```json +{ + "type": "integer", + "minimum": { "$data": "/otherField" } +} +``` + +#### ✅ Safe: oneOf with const + title +```json +{ + "type": "string", + "oneOf": [ + { "const": "value1", "title": "Display 1" }, + { "const": "value2", "title": "Display 2" } + ] +} +``` + +#### ⚠️ Works but Less Flexible: enum +```json +{ + "type": "string", + "enum": ["value1", "value2"] +} +``` + +### UI Schema Patterns + +#### ✅ Safe: Complete Layout Structure +```json +{ + "type": "SwipeLayout", + "elements": [ + { + "type": "Group", + "label": "Section", + "elements": [ + { "type": "Control", "scope": "#/properties/field1" } + ] + } + ] +} +``` + +#### ❌ Unsafe: Missing elements Array +```json +{ + "type": "Group", + "label": "Section" + // Missing "elements" +} +``` + +#### ✅ Safe: Valid Scope References +```json +{ + "type": "Control", + "scope": "#/properties/existingField" +} +``` + +#### ❌ Unsafe: Invalid Scope References +```json +{ + "type": "Control", + "scope": "#/properties/nonexistentField" +} +``` + +### Rule Patterns + +#### ✅ Safe: Rule with Existing Scope +```json +{ + "schema": { + "properties": { + "hasConsent": { "type": "boolean" } + } + }, + "uischema": { + "rule": { + "condition": { + "scope": "#/properties/hasConsent", + "schema": { "const": true } + } + } + } +} +``` + +#### ❌ Unsafe: Rule with Missing Scope +```json +{ + "rule": { + "condition": { + "scope": "#/properties/nonexistent", + "schema": { "const": "value" } + } + } +} +``` + +#### ✅ Safe: Simple Condition Schema +```json +{ + "condition": { + "schema": { "const": "value" } + } +} +``` + +#### ❌ Unsafe: Complex Condition Schema +```json +{ + "condition": { + "schema": { + "if": { "type": "string" }, + "then": { "const": "value" } + } + } +} +``` + +### Object Handling Patterns + +#### ✅ Safe: Custom Format Object (No Nested UI Needed) +```json +{ + "schema": { + "patient_photo": { + "type": "object", + "format": "photo" + } + }, + "uischema": { + "type": "Control", + "scope": "#/properties/patient_photo" + } +} +``` + +#### ✅ Safe: Regular Object with Explicit UI Schema +```json +{ + "schema": { + "gps_location": { + "type": "object", + "properties": { + "latitude": { "type": "number" }, + "longitude": { "type": "number" } + } + } + }, + "uischema": { + "type": "Group", + "label": "GPS Location", + "elements": [ + { "type": "Control", "scope": "#/properties/gps_location/properties/latitude" }, + { "type": "Control", "scope": "#/properties/gps_location/properties/longitude" } + ] + } +} +``` + +#### ❌ Unsafe: Regular Object without UI Schema +```json +{ + "schema": { + "gps_location": { + "type": "object", + "properties": { + "latitude": { "type": "number" } + } + } + }, + "uischema": { + "type": "Control", + "scope": "#/properties/gps_location" + // Nested fields won't render + } +} +``` + +--- + +## Examples + +### Example 1: Complete Working Form + +**File:** `demos/demo_malaria_screening/forms/registration/` + +**Schema (`schema.json`):** +```json +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "title": "Patient Registration", + "type": "object", + "properties": { + "full_name": { + "type": "string", + "title": "Full Name" + }, + "gender": { + "type": "integer", + "title": "Gender", + "oneOf": [ + { "const": 1, "title": "Male" }, + { "const": 2, "title": "Female" } + ] + }, + "age_years": { + "type": "integer", + "title": "Age in years", + "minimum": 0 + }, + "temperature_c": { + "type": "number", + "title": "Temperature (°C)", + "exclusiveMinimum": 25, + "exclusiveMaximum": 46 + } + }, + "required": ["full_name", "gender"] +} +``` + +**UI Schema (`ui.json`):** +```json +{ + "type": "SwipeLayout", + "elements": [ + { + "type": "VerticalLayout", + "elements": [ + { + "type": "Control", + "scope": "#/properties/full_name" + }, + { + "type": "Control", + "scope": "#/properties/gender" + }, + { + "type": "Control", + "scope": "#/properties/age_years" + } + ] + }, + { + "type": "VerticalLayout", + "elements": [ + { + "type": "Control", + "scope": "#/properties/temperature_c" + } + ] + } + ] +} +``` + +**Why This Works:** +- ✅ Proper `$schema` declaration +- ✅ Root `type: "object"` with `properties` +- ✅ Literal validation values (`minimum`, `exclusiveMinimum`) +- ✅ `oneOf` with `const` + `title` for choices +- ✅ `SwipeLayout` root with `elements` arrays +- ✅ All `Control.scope` paths exist in schema +- ✅ All layouts have `elements` arrays + +### Example 2: Form with Conditional Logic + +**Schema:** +```json +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "title": "Health Screening", + "type": "object", + "properties": { + "has_cough": { + "type": "string", + "title": "Does patient have cough?", + "oneOf": [ + { "const": "yes", "title": "Yes" }, + { "const": "no", "title": "No" } + ] + }, + "cough_duration": { + "type": "string", + "title": "Cough duration", + "oneOf": [ + { "const": "less_2_weeks", "title": "Less than 2 weeks" }, + { "const": "2_weeks_plus", "title": "2 weeks or more" } + ] + } + }, + "required": ["has_cough"] +} +``` + +**UI Schema:** +```json +{ + "type": "SwipeLayout", + "elements": [ + { + "type": "VerticalLayout", + "elements": [ + { + "type": "Control", + "scope": "#/properties/has_cough" + }, + { + "type": "Control", + "scope": "#/properties/cough_duration", + "rule": { + "effect": "SHOW", + "condition": { + "scope": "#/properties/has_cough", + "schema": { "const": "yes" } + } + } + } + ] + } + ] +} +``` + +**Why This Works:** +- ✅ Rule condition scope (`has_cough`) exists in schema +- ✅ Simple condition schema (`const`) +- ✅ Proper rule structure with `effect` and `condition` + +### Example 3: Form with Custom Format + +**Schema:** +```json +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "title": "Patient Intake", + "type": "object", + "properties": { + "patient_photo": { + "type": "object", + "format": "photo", + "title": "Patient Photo" + }, + "location": { + "type": "string", + "format": "gps", + "title": "Location" + } + } +} +``` + +**UI Schema:** +```json +{ + "type": "SwipeLayout", + "elements": [ + { + "type": "VerticalLayout", + "elements": [ + { + "type": "Control", + "scope": "#/properties/patient_photo" + }, + { + "type": "Control", + "scope": "#/properties/location" + } + ] + } + ] +} +``` + +**Why This Works:** +- ✅ Custom formats registered in Ajv +- ✅ Custom renderers registered in renderer chain +- ✅ Standard `Control` elements (no special UI schema needed) +- ✅ Format specified in schema + +--- + +## Summary Checklist + +When creating a form, ensure: + +### Schema Checklist +- [ ] `$schema` is `"http://json-schema.org/draft-07/schema#"` +- [ ] Root `type` is `"object"` +- [ ] Has `properties` object +- [ ] All validation values are literals (no `$data`) +- [ ] Use `oneOf` with `const` + `title` for choices (recommended) +- [ ] Nested objects have explicit UI schema (unless custom format) + +### UI Schema Checklist +- [ ] Root is `SwipeLayout` (or will be auto-wrapped) +- [ ] All layouts have `elements` arrays (even if empty) +- [ ] All `Control.scope` paths exist in schema +- [ ] All `Group` elements have `label` +- [ ] Rules reference existing fields +- [ ] Rule condition schemas are simple (`const` or `enum`) + +### Validation Checklist +- [ ] Run `validate-forms.js` script before deployment +- [ ] All field references validated +- [ ] No `$data` references +- [ ] All `elements` arrays present +- [ ] All scopes valid + +--- + +## Future Evolution (Non-Contractual) + +**This section describes potential future improvements that are not currently guaranteed and should not be relied upon.** + +These are areas where Formplayer might evolve as the project matures: + +### Potential Enhancements + +1. **Schema Pre-Validation for Unsupported Keywords** + - Early detection of `$data`, `$ref` to externals, and other unsupported features + - Compile-time rejection of out-of-contract schemas + - Clearer error messages pointing to contract violations + +2. **Stronger UI Schema Structural Validation** + - Deep validation of nested layouts (not just root level) + - Guaranteed `elements` array presence at all nesting levels + - Validation of rule condition scope existence + +3. **Defensive Guards in Renderers** + - More comprehensive optional chaining and fallbacks + - Graceful degradation instead of crashes + - Better error messages when assumptions fail + +4. **Formal Form Compilation Step** + - Pre-compilation of forms to validate contract compliance + - Optimization of renderer selection + - Static analysis of rule conditions + +5. **Extended JSON Schema Support** + - Potential support for `$data` references (with performance trade-offs) + - Support for external `$ref` resolution (with network requirements) + - More complex conditional schemas in rules + +**Important:** These are not commitments. Forms should be designed to work with the current contract. Future enhancements may expand the contract, but backward compatibility with the current contract will be maintained. + +**For maintainers:** When considering new features, evaluate them against: +- Performance impact on mobile devices +- Complexity for form authors +- Offline-first requirements +- Backward compatibility with existing forms + +--- + +## Version History + +- **v1.0** (2024): Initial contract document based on Formplayer codebase analysis + +--- + +## References + +- **JSON Forms Documentation:** https://jsonforms.io +- **JSON Schema Draft-07:** https://json-schema.org/draft-07/schema# +- **Ajv Validator:** https://ajv.js.org +- **Formplayer Source:** `ode/formulus-formplayer/src/` + +--- + +**End of Contract** + diff --git a/docs/docs/reference/formplayer.md b/docs/docs/reference/formplayer.md new file mode 100644 index 000000000..0c4f1d41f --- /dev/null +++ b/docs/docs/reference/formplayer.md @@ -0,0 +1,525 @@ +--- +sidebar_position: 6 +--- + +# Formplayer Component Reference + +Complete technical reference for the Formplayer form rendering component. + +:::info[Authoritative Reference Available] + +For the complete, authoritative contract defining all supported JSON Schema features, UI schema structure, validation rules, and behavioral guarantees, see the **[Formplayer Contract](/reference/formplayer-contract)**. + +::: + +## Overview + +Formplayer is a React web application that renders JSON Forms and provides the dynamic form interface for data collection. It runs within WebViews in the Formulus mobile app and communicates with the native app through a JavaScript bridge. + +## Supported Schema & UI Profile + +**Critical**: ODE Formplayer intentionally supports a safe, predictable subset of JSON Schema and JSON Forms. Forms outside this profile may load but are **not guaranteed to work**. + +### Supported JSON Schema Features + +| Feature | Supported | Notes | +|---------|-----------|-------| +| `type` | ✅ | `string`, `number`, `integer`, `boolean`, `object`, `array` | +| `properties` | ✅ | Object property definitions | +| `required` | ✅ | Array of required property names | +| `minimum` / `maximum` | ✅ | Literal numbers only (e.g., `"minimum": 0`) | +| `minLength` / `maxLength` | ✅ | String length constraints | +| `pattern` | ✅ | Regular expression patterns | +| `format` | ✅ | `email`, `date`, `date-time`, `uri`, `uuid` | +| `enum` | ✅ | Array of allowed values | +| `enumNames` | ✅ | Display names for enum values | +| `oneOf` | ✅ | Recommended for single-choice selections | +| `title` | ✅ | Field display title | +| `description` | ✅ | Field description/help text | +| `default` | ✅ | Default values | +| `const` | ✅ | Constant values (used in rules) | +| `$data` | ❌ | **Will crash** - not supported | +| `if` / `then` / `else` | ❌ | **Not supported** in rule conditions | +| `$ref` | ⚠️ | Limited support - use with caution | +| `allOf` / `anyOf` | ⚠️ | Limited support - use with caution | + +### Supported UI Schema Elements + +| Element | Required Fields | Notes | +|---------|----------------|-------| +| **SwipeLayout** | `type`, `elements[]` | Root layout (required or auto-wrapped) | +| **VerticalLayout** | `type`, `elements[]` | Vertical field arrangement | +| **HorizontalLayout** | `type`, `elements[]` | Horizontal field arrangement | +| **Group** | `type`, `label`, `elements[]` | Grouped fields with label | +| **Control** | `type`, `scope` | Field control (scope must exist in schema) | +| **Label** | `type`, `text` | Text label element | + +### Unsupported / Unsafe Features + +**❌ JSON Schema:** +- `$data` references (dynamic values) +- `if`/`then`/`else` conditional schemas +- Complex `$ref` resolution +- `allOf`/`anyOf` (limited support) + +**❌ UI Schema:** +- Missing `elements` array in layouts +- Invalid `scope` paths (referencing non-existent schema properties) +- Rules referencing missing fields +- Nested SwipeLayout +- `Categorization` layout (not recommended) + +**⚠️ Common Pitfalls:** +- Using `$data` in `minimum`/`maximum` (use literal numbers) +- Rule conditions with scopes that don't exist in schema +- Controls with scopes pointing to non-existent properties +- Missing `elements` array in SwipeLayout or other layouts + +### Safe Form Patterns + +**✅ Recommended Structure:** +```json +{ + "schema": { + "type": "object", + "properties": { + "field1": { "type": "string", "title": "Field 1" }, + "field2": { "type": "integer", "minimum": 0, "maximum": 100 } + }, + "required": ["field1"] + }, + "uischema": { + "type": "SwipeLayout", + "elements": [ + { + "type": "VerticalLayout", + "elements": [ + { + "type": "Control", + "scope": "#/properties/field1" + }, + { + "type": "Control", + "scope": "#/properties/field2" + } + ] + } + ] + } +} +``` + +**❌ Unsafe Patterns:** +```json +{ + "schema": { + "properties": { + "field1": { "type": "string" } + } + }, + "uischema": { + "type": "Control", + "scope": "#/properties/missingField" // ❌ Field doesn't exist + } +} +``` + +```json +{ + "schema": { + "properties": { + "value": { + "type": "number", + "minimum": { "$data": "#/minValue" } // ❌ $data not supported + } + } + } +} +``` + +## Architecture + +### Technology Stack + +- **Framework**: React +- **Language**: TypeScript +- **Form Library**: JSON Forms +- **UI Framework**: Material-UI (via JSON Forms) +- **Build Tool**: Webpack/Vite + +### Component Structure + +``` +formulus-formplayer/ +├── src/ +│ ├── App.tsx # Main application component +│ ├── FormLayout.tsx # Form layout renderer +│ ├── QuestionShell.tsx # Question wrapper component +│ ├── *QuestionRenderer.tsx # Question type renderers +│ ├── FormulusInterface.ts # Bridge interface definition +│ └── theme.ts # Theming configuration +├── public/ +│ └── formulus-load.js # API loading script +└── build/ # Production build output +``` + +## Core Responsibilities + +Formplayer is responsible for: + +1. **Form Rendering**: Render forms based on JSON schema and UI schema +2. **Data Collection**: Capture user input through various question types +3. **Validation**: Validate form responses against schema rules +4. **Observation Management**: Create, edit, and delete observations +5. **Draft Management**: Save and load draft observations + +## Integration with Formulus + +### Initialization + +Formplayer is initialized by the Formulus app with: + +- **Renderers**: Container components for form layout +- **Cells**: Question type components (text, date, photo, etc.) +- **Form Specs**: JSON form specifications from server +- **Formulus API**: JavaScript interface to native app + +### Communication Model + +``` +┌─────────────────┐ ┌──────────────────┐ +│ Formulus │ │ Formplayer │ +│ (Native) │◄───────►│ (WebView) │ +│ │ │ │ +│ • Database │ │ • Form Render │ +│ • Sync Engine │ │ • Validation │ +│ • API Bridge │ │ • User Input │ +└─────────────────┘ └──────────────────┘ +``` + +## JavaScript Interface + +Formplayer exposes methods to custom applications and receives configuration from Formulus. + +### Available Methods + +#### addObservation(formType, initializationData) + +Open a form to create a new observation. + +```javascript +window.formulus.formplayer.addObservation('survey', { + participantId: '123', + location: 'Field Site A' +}); +``` + +**Parameters:** +- `formType` (string): Form type identifier +- `initializationData` (object): Optional pre-population data + +#### editObservation(formType, observationId) + +Open a form to edit an existing observation. + +```javascript +window.formulus.formplayer.editObservation('survey', 'obs-123'); +``` + +**Parameters:** +- `formType` (string): Form type identifier +- `observationId` (string): Observation ID to edit + +#### deleteObservation(formType, observationId) + +Delete an observation. + +```javascript +window.formulus.formplayer.deleteObservation('survey', 'obs-123'); +``` + +**Parameters:** +- `formType` (string): Form type identifier +- `observationId` (string): Observation ID to delete + +## Question Types + +Formplayer supports various question types through custom renderers: + +### Text Input + +- **Single-line text**: Standard text input +- **Multi-line text**: Textarea for longer responses +- **Email**: Email format validation +- **Phone**: Phone number format validation +- **URL**: URL format validation + +### Number Input + +- **Integer**: Whole numbers only +- **Decimal**: Floating-point numbers +- **Range**: Min/max value constraints + +### Date and Time + +- **Date**: Date picker +- **Time**: Time picker +- **DateTime**: Combined date and time picker + +### Selection + +- **Single Select**: Native ` + + + +

    + We respect your privacy. Unsubscribe at any time. +

    + + + )} + + + +
    + Giraffe reading a newsletter +
    + + + + + ); +} + diff --git a/docs/src/components/Newsletter/styles.module.css b/docs/src/components/Newsletter/styles.module.css new file mode 100644 index 000000000..e3e73d1f7 --- /dev/null +++ b/docs/src/components/Newsletter/styles.module.css @@ -0,0 +1,275 @@ +/** + * Newsletter Component Styles + */ + +.newsletter { + padding: 4rem 0; + background: var(--ifm-color-emphasis-100); + text-align: center; +} + +.newsletterContent { + max-width: 1000px; + margin: 0 auto; +} + +.newsletterGrid { + display: grid; + grid-template-columns: 1fr 1fr; + gap: 3rem; + align-items: center; +} + +.newsletterLeft { + text-align: left; +} + +.newsletterRight { + display: flex; + justify-content: center; + align-items: center; +} + +.giraffeImage { + max-width: 100%; + height: auto; + border-radius: 12px; + box-shadow: 0 8px 24px rgba(0, 0, 0, 0.1); +} + +.newsletterTitle { + font-size: 2.5rem; + margin-bottom: 1rem; + color: var(--ifm-color-primary); +} + +.newsletterSubtitle { + font-size: 1.2rem; + color: var(--ifm-color-emphasis-700); + margin-bottom: 2rem; + line-height: 1.6; +} + +.newsletterForm { + margin-top: 2rem; +} + +.form { + background: white; + border-radius: 8px; + box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1); + max-width: 500px; + margin: 0 auto; + overflow: hidden; +} + +.formContent { + padding: 2rem; +} + +.formFields { + display: flex; + flex-direction: column; + gap: 1rem; + margin-bottom: 1rem; +} + +.emailInput { + padding: 0.75rem 1rem; + border: 2px solid var(--ifm-color-emphasis-300); + border-radius: 6px; + font-size: 1rem; + transition: border-color 0.2s ease; +} + +.emailInput:focus { + outline: none; + border-color: var(--ifm-color-primary); +} + +.submitButton { + background: var(--ifm-color-primary); + color: white; + border: none; + padding: 0.75rem 2rem; + border-radius: 6px; + font-size: 1rem; + font-weight: 600; + cursor: pointer; + transition: background-color 0.2s ease; +} + +.submitButton:hover { + background: var(--ifm-color-primary-dark); +} + +.privacy { + font-size: 0.875rem; + color: var(--ifm-color-emphasis-600); + margin: 0; + text-align: center; +} + +[data-theme='dark'] .privacy { + color: var(--ode-neutral-800); +} + +.thankYouMessage { + background: white; + border-radius: 8px; + box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1); + max-width: 500px; + margin: 0 auto; + overflow: hidden; +} + +.thankYouContent { + padding: 2rem; + text-align: center; +} + +.thankYouTitle { + color: var(--ifm-color-success); + font-size: 1.5rem; + font-weight: 700; + margin: 0 0 1rem 0; +} + +.thankYouText { + color: var(--ifm-color-emphasis-800); + font-size: 1rem; + margin: 0 0 1rem 0; + line-height: 1.6; +} + +.thankYouHint { + color: var(--ifm-color-emphasis-700); + font-size: 0.9rem; + margin: 0; + line-height: 1.5; +} + +@media screen and (max-width: 996px) { + .newsletter { + padding: 2.5rem 0; + } + + .newsletterGrid { + grid-template-columns: 1fr; + gap: 2rem; + } + + .newsletterLeft { + text-align: center; + } + + .newsletterTitle { + font-size: 2rem; + } + + .newsletterSubtitle { + font-size: 1.1rem; + } + + .giraffeImage { + max-width: 300px; + } + + .formContent { + padding: 1.5rem; + } +} + +@media screen and (max-width: 768px) { + .newsletter { + padding: 2rem 0; + } + + .newsletterGrid { + gap: 1.5rem; + } + + .newsletterTitle { + font-size: 1.75rem; + margin-bottom: 0.75rem; + } + + .newsletterSubtitle { + font-size: 1rem; + margin-bottom: 1.5rem; + } + + .newsletterForm { + margin-top: 1.5rem; + } + + .form { + max-width: 100%; + } + + .formContent { + padding: 1.25rem; + } + + .giraffeImage { + max-width: 250px; + } + + .thankYouContent { + padding: 1.5rem; + } + + .thankYouTitle { + font-size: 1.25rem; + } +} + +@media screen and (max-width: 576px) { + .newsletter { + padding: 1.5rem 0; + } + + .newsletterTitle { + font-size: 1.5rem; + } + + .newsletterSubtitle { + font-size: 0.95rem; + } + + .formContent { + padding: 1rem; + } + + .emailInput { + font-size: 0.9rem; + padding: 0.625rem 0.875rem; + } + + .submitButton { + font-size: 0.9rem; + padding: 0.625rem 1.5rem; + width: 100%; + } + + .formFields { + gap: 0.75rem; + } + + .giraffeImage { + max-width: 200px; + } + + .thankYouContent { + padding: 1.25rem; + } + + .thankYouTitle { + font-size: 1.125rem; + } + + .thankYouText { + font-size: 0.9rem; + } +} + diff --git a/docs/src/components/README.md b/docs/src/components/README.md new file mode 100644 index 000000000..7d9f1d0c0 --- /dev/null +++ b/docs/src/components/README.md @@ -0,0 +1,251 @@ +# ODE Design System Components + +A collection of reusable React components built with ODE design tokens. All components support dark mode, are fully accessible, and follow the ODE design system specifications. + +## Components + +### Alert +Display contextual feedback messages with semantic variants. + +```tsx +import { Alert } from '@site/src/components'; + + + This is an informational message. + + +Operation completed successfully! +Please review your input. +Something went wrong. +``` + +**Props:** +- `variant`: `'info' | 'success' | 'warning' | 'error'` (default: `'info'`) +- `size`: `'sm' | 'md'` (default: `'md'`) +- `title`: Optional title text +- `icon`: Optional custom icon element +- `children`: Alert message content + +--- + +### Badge +Small status indicators and labels. + +```tsx +import { Badge } from '@site/src/components'; + +New +Active +Pending +``` + +**Props:** +- `variant`: `'primary' | 'secondary' | 'success' | 'warning' | 'error' | 'info' | 'neutral'` (default: `'primary'`) +- `size`: `'sm' | 'md'` (default: `'md'`) + +--- + +### Button +Interactive button component with multiple variants and sizes. + +```tsx +import { Button, ButtonGroup } from '@site/src/components'; + + + + + + +{/* Pill buttons with fading borders - perfect for pairs */} + + + + + + +``` + +**Props:** +- `variant`: `'primary' | 'secondary' | 'outline' | 'ghost' | 'pill-light' | 'pill-dark'` (default: `'primary'`) + - `pill-light`: Light style with transparent background, dark border/text, fade on left side + - `pill-dark`: Dark style with filled background, light text, fade on right side +- `size`: `'sm' | 'md' | 'lg'` (default: `'md'`) +- `disabled`: Boolean +- `href` or `to`: Renders as a link instead of button +- `onClick`: Click handler (when used as button) + +**Pill Button Behavior:** +- `pill-light`: Default state has transparent background with dark border/text. On hover, fills with light background. +- `pill-dark`: Default state has dark filled background with light text. On hover, darkens further. +- When paired together, they create opposite visual states perfect for toggle or action pairs. + +--- + +### CodeBlock +Enhanced code block with optional title and line numbers. + +```tsx +import { CodeBlock } from '@site/src/components'; + + +{`function greet(name: string) { + return \`Hello, \${name}!\`; +}`} + +``` + +**Props:** +- `language`: Code language for syntax highlighting +- `title`: Optional title shown in header +- `showLineNumbers`: Boolean (default: `false`) + +--- + +### Container +Responsive container with max-width constraints. + +```tsx +import { Container } from '@site/src/components'; + + +

    Content

    +     +``` + +**Props:** +- `size`: `'sm' | 'md' | 'lg' | 'xl' | '2xl' | 'full'` (default: `'lg'`) + +--- + +### Divider +Visual separator with optional text. + +```tsx +import { Divider } from '@site/src/components'; + + + +Or + +``` + +**Props:** +- `variant`: `'solid' | 'dashed' | 'dotted'` (default: `'solid'`) +- `orientation`: `'horizontal' | 'vertical'` (default: `'horizontal'`) +- `spacing`: `'none' | 'sm' | 'md' | 'lg'` (default: `'md'`) +- `children`: Optional text to display in center + +--- + +### Spacer +Consistent spacing utility component. + +```tsx +import { Spacer } from '@site/src/components'; + + + + +``` + +**Props:** +- `size`: `0 | 1 | 2 | 3 | 4 | 5 | 6 | 8 | 10 | 12 | 16 | 20 | 24` +- `axis`: `'x' | 'y' | 'both'` (default: `'both'`) + +--- + +### Tag +Categorization tags with optional remove functionality. + +```tsx +import { Tag } from '@site/src/components'; + +React +TypeScript + console.log('removed')}> + Removable + +``` + +**Props:** +- `variant`: `'primary' | 'secondary' | 'neutral'` (default: `'neutral'`) +- `size`: `'sm' | 'md'` (default: `'md'`) +- `removable`: Boolean (default: `false`) +- `onRemove`: Callback when remove button is clicked + +--- + +### Tooltip +Contextual information on hover/focus. + +```tsx +import { Tooltip } from '@site/src/components'; + + + + +``` + +**Props:** +- `content`: Tooltip content +- `position`: `'top' | 'bottom' | 'left' | 'right'` (default: `'top'`) +- `delay`: Delay in milliseconds before showing (default: `200`) + +--- + +### ButtonGroup +Container for grouping buttons together, especially useful for pill button pairs. + +```tsx +import { ButtonGroup, Button } from '@site/src/components'; + + + + + +``` + +**Props:** +- `children`: Button elements to group together +- `className`: Optional additional CSS classes + +--- + +## Design Tokens + +All components use ODE design tokens defined in `src/css/ode-tokens.css`. These include: + +- **Colors**: Brand (primary/secondary), semantic (success/error/warning/info), neutral +- **Spacing**: 4px base unit scale (0-24) +- **Typography**: Font families, sizes, weights, line heights +- **Borders**: Radius and width variants +- **Shadows**: Elevation levels +- **Motion**: Duration and easing functions +- **Layout**: Breakpoints and container max-widths + +## Dark Mode + +All components automatically support dark mode through the `[data-theme='dark']` selector. No additional configuration needed. + +## Accessibility + +Components follow accessibility best practices: +- Proper ARIA attributes +- Keyboard navigation support +- Focus management +- Semantic HTML elements +- Color contrast compliance + +## Usage in MDX + +Components can be imported and used directly in MDX files: + +```mdx +import { Alert, Button, Badge } from '@site/src/components'; + + + This works in MDX! + + + +``` diff --git a/docs/src/components/Spacer/index.tsx b/docs/src/components/Spacer/index.tsx new file mode 100644 index 000000000..4407aaf96 --- /dev/null +++ b/docs/src/components/Spacer/index.tsx @@ -0,0 +1,31 @@ +import type { ReactElement } from 'react'; +import clsx from 'clsx'; +import styles from './styles.module.css'; + +type SpacerSize = 0 | 1 | 2 | 3 | 4 | 5 | 6 | 8 | 10 | 12 | 16 | 20 | 24; +type SpacerAxis = 'x' | 'y' | 'both'; + +interface SpacerProps { + size: SpacerSize; + axis?: SpacerAxis; + className?: string; +} + +export default function Spacer({ + size, + axis = 'both', + className, +}: SpacerProps): React.ReactElement { + return ( +