Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -516,7 +516,7 @@ Some modules have conditional loading based on dependencies:
* 5-minute refresh buffer before expiry
* OAuth 2.0 device flow

### SDK Metadata System (ENHANCED - Run v0.0.4, Item v0.0.3)
### SDK Metadata System (ENHANCED - Run v0.0.6, Item v0.0.3)

**Automatic Run & Item Tracking**: Every application run and item submitted through the SDK automatically includes comprehensive metadata about the execution context, with support for tags and timestamps.

Expand All @@ -526,7 +526,7 @@ Some modules have conditional loading based on dependencies:
* **Environment Detection**: Automatically detects script/CLI/GUI and user/test/bridge contexts
* **CI/CD Integration**: Captures GitHub Actions workflow information and pytest test context
* **User Information**: Includes authenticated user and organization details
* **Schema Validation**: Pydantic-based validation with JSON Schema (Run: v0.0.4, Item: v0.0.3)
* **Schema Validation**: Pydantic-based validation with JSON Schema (Run: v0.0.6, Item: v0.0.3)
* **Versioned Schema**: Published JSON Schema at `docs/source/_static/sdk_{run|item}_custom_metadata_schema_*.json`
* **Tags Support** (NEW): Associate runs and items with searchable tags
* **Timestamps** (NEW): Track creation and update times (`created_at`, `updated_at`)
Expand Down Expand Up @@ -577,7 +577,7 @@ aignostics application run list --tags experiment-1,batch-A
* Integration: Automatic in `platform.resources.runs.submit()`
* User Agent: Enhanced `utils.user_agent()` with CI/CD context
* Tests: Comprehensive test suite in `tests/aignostics/platform/sdk_metadata_test.py`
* **Schema Files**: `sdk_run_custom_metadata_schema_v0.0.4.json` and `sdk_item_custom_metadata_schema_v0.0.3.json`
* **Schema Files**: `sdk_run_custom_metadata_schema_v0.0.6.json` and `sdk_item_custom_metadata_schema_v0.0.3.json`

See `platform/CLAUDE.md` for detailed documentation.

Expand Down
758 changes: 132 additions & 626 deletions README.md

Large diffs are not rendered by default.

6 changes: 3 additions & 3 deletions docs/partials/README_glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Software Development Kit providing multiple pathways to interact with the Aignos
Fully automated advanced machine learning workflow composed of specific tasks (e.g., Tissue Quality Control, Tissue Segmentation, Cell Detection, Cell Classification) designed for particular analysis purposes.

**Application Run**
The execution instance of an application on submitted whole slide images, which can be in various states: received, scheduled, running, completed, rejected, cancelled by system, or cancelled by user.
The execution instance of an application on submitted whole slide images. A run moves through three states — pending, processing, and terminated — and a terminated run records a termination reason (all items processed, canceled by user, or canceled by system).

**Application Version**
Specific version of an application with defined input requirements, processing tasks, and output formats. Each application can have multiple versions.
Expand Down Expand Up @@ -153,8 +153,8 @@ Limit on the number of whole slide images an organization or user can process pe

### R

**Reference**
Unique identifier string for each input item in an application run, used to match results with original inputs.
**Reference** (`external_id`)
Unique identifier string you assign to each input item in an application run (the `external_id` field), used to match results with original inputs.

**Results**
Output data from application processing, including measurements, statistics, heatmaps, and annotations, automatically deleted after 30 days.
Expand Down
643 changes: 31 additions & 612 deletions docs/partials/README_main.md

Large diffs are not rendered by default.

108 changes: 97 additions & 11 deletions docs/partials/README_platform.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,23 +51,24 @@ Once registered to the Platform, your organization will automatically gain acces

To trigger the application run, users can use the Aignostics Launchpad, Aignostics CLI, Example Notebooks, our Client Library, or directly call the REST API. The platform expects the user payload, containing the metadata and the signed URLs to the whole slide images (WSIs). The detailed requirements of the payload depend on the application and are described in the documentation, and accessible via the Info button in the Launchpad, as well as via the CLI and `/v1/applications` endpoint in the API.

When the application run is created, it can be in one of the following states:
When the application run is created, it progresses through three states:

1. **received**: the application run received from the client
2. **scheduled**: the application run request is valid and is scheduled for execution
3. **running**: the application run execution started
4. **completed**: the application run execution is done and all outputs are available for download
5. **completed**: the application run execution is done, but some items end up in the failed state
6. **rejected**: the application run request is rejected before it is scheduled
7. **cancelled by the system**: the application run failed during the execution with the number of errors higher than the threshold
9. **cancelled by the user**: the application run is cancelled by the user before it is finished
1. **pending**: the application run has been received and is waiting to start processing
2. **processing**: the application run is being executed; results for individual slides become available as they complete
3. **terminated**: the application run has finished

A terminated run carries a *termination reason* explaining the outcome:

- **all items processed**: every slide was processed (individual slides may still have failed — check the per-slide results)
- **canceled by the user**: the run was canceled by the user before it finished
- **canceled by the system**: the run was stopped by the platform, for example when the number of failed slides exceeded the allowed threshold
Comment thread
omid-aignostics marked this conversation as resolved.
Comment thread
omid-aignostics marked this conversation as resolved.

The status and operations of an application run are private to the user who triggered the run.

### Results
When the processing of whole slide image is successfully completed, the resulting outputs become available for download. To assess specifics of application outputs please consult our application specific documentation, which you can find in the **Console**. Please note that you access to documentation is restricted to those applications your organisation subscribed to.

Application run outputs are automatically deleted 30 days after the application run has completed. However, the owner of the application run (the user who initiated it) can use the API to manually delete outputs earlier, once the run has reached a final state - completed, cancelled by the system or cancelled by the user. The Launchpad and CLI provide enable to delete results with one click resp. command.
Application run outputs are automatically deleted 30 days after the application run has terminated. However, the owner of the application run (the user who initiated it) can use the API to manually delete outputs earlier, once the run has terminated. The Launchpad and CLI let you delete results with one click or one command, respectively.

### Quotas
Every organization has a limit on how many WSIs it can process in a calendar month. The following quotas exist:
Expand Down Expand Up @@ -100,6 +101,91 @@ For integration with programming languages other than Python, you can use the RE

### Cost

Every WSI processed by the Platform generates a cost. Usage of the "Test Application" is free of charge for any registered user. The cost for other applications is defined in your business agreement with Aignostics. The cost is calculated based on the number of slides processed. When an application run is cancelled, either by the system or by the user, only processed images incur a cost.
Every WSI processed by the Platform generates a cost. Usage of the "Test Application" is free of charge for any registered user. The cost for other applications is defined in your business agreement with Aignostics. The cost is calculated based on the number of slides processed. When an application run is canceled, either by the system or by the user, only processed images incur a cost.

**[Read the API reference documentation](https://aignostics.readthedocs.io/en/latest/api_reference_v1.html)** or use our **[Interactive API Explorer](https://platform.aignostics.com/explore-api)** to dive into details of all operations and parameters.

### Platform workflow

The Aignostics Platform delivers enterprise-grade computational pathology through a secure, scalable cloud architecture. Organizations subscribe to the platform, and their users interact through multiple interfaces - all part of the Python SDK - to leverage advanced AI/ML models running on dedicated NVIDIA® GPU infrastructure.

**Key architectural components:**

- **Python SDK**: Provides multiple interfaces (Launchpad desktop app, CLI, Python Library, and MCP server) with unified functionality
- **Enterprise authentication**: Powered by Auth0, supporting Single Sign-On (SSO) and existing identity management systems
- **Organization storage**: Dedicated Google Cloud Storage bucket per organization with automatic 30-day cleanup
- **Aignostics Platform API**: Orchestrates application discovery, run submission, status monitoring, and results delivery
- **NVIDIA® GPU clusters**: Dedicated compute provisioned per application run for maximum security and compliance

```mermaid
%%{init: {'theme':'dark', 'themeVariables': { 'fontSize':'18px', 'fontFamily':'arial', 'darkMode':'true', 'background':'#1e1e1e', 'primaryColor':'#4a4a4a', 'primaryTextColor':'#ffffff', 'primaryBorderColor':'#ffffff', 'lineColor':'#ffffff', 'secondaryColor':'#3a3a3a', 'tertiaryColor':'#2a2a2a', 'actorBkg':'#4a4a4a', 'actorBorder':'#ffffff', 'actorTextColor':'#ffffff', 'actorLineColor':'#ffffff', 'signalColor':'#ffffff', 'signalTextColor':'#ffffff', 'labelBoxBkgColor':'#3a3a3a', 'labelBoxBorderColor':'#ffffff', 'labelTextColor':'#ffffff', 'noteBkgColor':'#4a4a4a', 'noteTextColor':'#ffffff', 'noteBorderColor':'#ffffff', 'sequenceNumberColor':'#000000'}}}%%
sequenceDiagram
autonumber
actor User as User<br/>(Organization Member)
participant SDK as Python SDK<br/>(Launchpad/CLI/Library/MCP)
participant Auth0 as Auth0<br/>(Enterprise Identity)
participant Bucket as Organization Bucket<br/>(Google Cloud Storage)
participant API as Aignostics Platform API
participant GPU as NVIDIA® GPU Cluster<br/>(per-run isolation)

Note over User,GPU: Authentication & Authorization
User->>SDK: Launch interface
SDK->>Auth0: Authenticate user
Auth0-->>SDK: Access token
SDK->>API: Validate token
API-->>SDK: User authorized

Note over User,GPU: Application Selection
User->>SDK: Browse applications
SDK->>API: List applications & versions
API-->>SDK: Application catalog
SDK-->>User: Display options

Note over User,GPU: Data Upload
User->>SDK: Select WSIs + metadata
SDK->>Bucket: Upload files
Note over Bucket: 30-day auto-cleanup
Bucket-->>SDK: Upload complete
SDK->>SDK: Generate signed download URLs

Note over User,GPU: Run Submission
SDK->>API: Submit run (app, metadata, signed URLs)
API-->>SDK: Run ID + queue position
SDK-->>User: Confirm submission

Note over User,GPU: GPU Processing
API->>GPU: Provision dedicated NVIDIA® cluster
GPU->>Bucket: Download WSIs via signed URLs
GPU->>GPU: Process slides incrementally
GPU->>API: Upload results per slide
Note over GPU: Deprovision after completion

Note over User,GPU: Status Monitoring & Results
User->>SDK: Check status
SDK->>API: Poll run status
API-->>SDK: Progress (e.g., "3 of 10 complete")
SDK-->>User: Display progress

User->>SDK: Download results
SDK->>API: Request result URLs
API-->>SDK: Signed download URLs
SDK->>API: Download files (GeoJSON, CSV, TIFF)
SDK-->>User: Results ready for inspection
```

**How it works:**

Organizations subscribe to the Aignostics Platform and receive dedicated infrastructure including a Google Cloud Storage bucket and API access. Users within the organization authenticate through Auth0, which integrates with enterprise identity management systems for seamless Single Sign-On (SSO).

The Python SDK - available as a desktop application (Launchpad), command-line interface (CLI), Python library, and MCP server - handles all complexity of authentication, data upload, run orchestration, and results delivery. Users simply select an application, provide whole slide images with metadata, and submit.

Behind the scenes, the Aignostics Platform API provisions dedicated NVIDIA® GPU clusters for each application run, ensuring data isolation and compliance with healthcare regulations. Processing occurs incrementally (slide-by-slide), allowing users to monitor progress and download results as they become available rather than waiting for entire cohorts.

The organization's Google Cloud Storage bucket stores uploaded files with automatic 30-day cleanup, optimizing costs while maintaining data availability throughout processing. All data transfers use time-limited signed URLs, eliminating credential management complexity and security risks.

**Enterprise benefits:**

- **Security & compliance**: Per-run GPU isolation, enterprise SSO integration, zero-trust architecture with signed URLs
- **Scalability**: Handles single exploratory slides through thousand-slide clinical studies with identical user experience
- **Cost efficiency**: Pay-per-use GPU provisioning, automatic storage cleanup, no idle infrastructure costs
- **Operational simplicity**: Python SDK abstracts all cloud complexity; IT teams manage access through existing identity systems
16 changes: 16 additions & 0 deletions docs/partials/_get_started_signup.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
## Sign up for the Aignostics Platform

Before you can run an analysis, you need an account on the Aignostics Platform. These account steps are the same whichever interface you use.

> 💡 **Already have an account?** Skip to the installation steps below.

1. **Find your invitation email.** Look in your inbox for a message from `support@aignostics.com`, with a subject like "You've been invited to join your organization's Aignostics account". If you can't find it, check your spam folder. If it isn't there either, ask your organization's administrator or email `support@aignostics.com`.

2. **Accept the invitation.** Open the email and click **Accept Invitation**. A page opens in your browser where you enter your full name and set a password.

3. **Set up two-factor authentication.** Next, you are asked to set up two-factor authentication. This is a second login step that uses a code from your phone. Install one of these free authenticator apps, then scan the code shown in your browser and enter the six-digit code it gives you:

- **Android:** [Google Authenticator](https://play.google.com/store/apps/details?id=com.google.android.apps.authenticator2)
- **iPhone:** [Google Authenticator](https://apps.apple.com/app/google-authenticator/id388497605)

You are done when you see "Welcome to the Console of the Aignostics Platform". From now on, each time you log in you enter your password and then a fresh six-digit code from the app.
119 changes: 119 additions & 0 deletions docs/partials/get_started_cli.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
# Get started with the CLI

The **Aignostics CLI** is a command-line tool for interacting with the Aignostics Platform directly from your terminal or shell scripts. It is well suited to processing larger cohorts, automating repetitive analyses, and integrating with computational pipelines.

Common workflows:

- Download public datasets from the NCI Image Data Commons
- Submit batch processing runs for many slides
- Monitor run status and download results incrementally
- Automate repetitive tasks with shell scripts

```{include} ../partials/_get_started_signup.md
```

## Install the CLI

The CLI runs through [uv](https://docs.astral.sh/uv/). The command below installs uv if it isn't already present, updates it if it's out of date (including Homebrew-managed installs), and makes it available in your current shell. Afterwards every command is available through `uvx aignostics`.

**On macOS or Linux:**

```bash
if ! command -v uv &> /dev/null; then
echo "uv not found, installing..."
curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env
else
UV_VERSION=$(uv --version | cut -d' ' -f2)
if [ "$(printf '%s\n' "0.6.17" "$UV_VERSION" | sort -V | head -n1)" != "0.6.17" ]; then
Comment thread
omid-aignostics marked this conversation as resolved.
echo "Updating uv to the latest version..."
UV_PATH=$(which uv)
if [[ "$UV_PATH" == *"brew"* ]]; then
echo "Updating uv using Homebrew..."
brew upgrade uv
else
echo "Updating uv using the installer..."
uv self update
fi
else
echo "uv is up to date"
fi
fi
```

**On Windows (PowerShell):**

```powershell
winget install --id=Microsoft.VCRedist.2015+.x64 -e
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
```

Verify the install:

```bash
uvx aignostics --help
```

You should see the list of available command groups (`application`, `dataset`, `bucket`, `system`, and more). If the command isn't found, open a new terminal and try again.

## Run Atlas H&E-TME from the command line

This example downloads a public lung cancer dataset, submits an [Atlas H&E-TME](https://www.aignostics.com/products/he-tme-profiling-product) run, and downloads the results:

```shell
# Download a sample dataset from the NCI Image Data Commons (IDC) portal.
# This dataset id refers to the TCGA LUAD collection and creates a tcga_luad directory of DICOM files.
uvx aignostics dataset idc download 1.3.6.1.4.1.5962.99.1.1069745200.1645485340.1637452317744.2.0 data/
# Prepare a run.csv, extracting the required metadata from the DICOM files.
uvx aignostics application run prepare he-tme data/tcga_luad/run.csv data/
# Edit the run.csv to add the required staining method, tissue type, and disease (use any editor).
nano data/tcga_luad/run.csv
# Upload the run.csv and referenced whole slide images to the Aignostics Platform.
uvx aignostics application run upload he-tme data/tcga_luad/run.csv
# Submit the application run and print the run id.
uvx aignostics application run submit he-tme data/tcga_luad/run.csv
# Check the status of your run.
uvx aignostics application run list
# Incrementally download results as they become available (use the id from the previous step).
uvx aignostics application run result download APPLICATION_RUN_ID
```

For convenience, `application run execute` combines preparation, upload, submission, and download. The command below is equivalent to the above, supplying the required metadata with a mapping:

```shell
uvx aignostics dataset idc download 1.3.6.1.4.1.5962.99.1.1069745200.1645485340.1637452317744.2.0 data/
uvx aignostics application run execute he-tme data/tcga_luad/run.csv data/ --mapping ".*\.dcm:staining_method=H&E,tissue=LUNG,disease=LUNG_CANCER"
```

The CLI provides extensive help at every level:

```shell
uvx aignostics --help # list all command groups
uvx aignostics application --help # subcommands in the application group
uvx aignostics application run --help # subcommands in the application run group
uvx aignostics application run list --help # help for a specific command
```

See the [CLI reference](https://aignostics.readthedocs.io/en/latest/cli_reference.html) for all commands and options.

## System health checks

The CLI checks system health before uploading slides or submitting runs. If the system is unhealthy, the operation is blocked:

```
Error: Platform is not healthy: <reason>. Aborting.
```

To override this (not recommended for production), add `--force`:

```shell
uvx aignostics application run upload he-tme data/tcga_luad/run.csv --force
uvx aignostics application run submit he-tme data/tcga_luad/run.csv --force
uvx aignostics application run execute he-tme data/tcga_luad/run.csv data/ --force
```

To check system health manually:

```shell
uvx aignostics system health
```
Loading
Loading