Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 8 additions & 12 deletions mkdocs/docs/concepts/tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -223,10 +223,10 @@ the index of the node within that group.

Currently, only `resources`, `commands`, and `ports` can be configured per node group. [`groups`](../reference/dstack.yml/task.md#groups) and top-level `nodes` are mutually exclusive.Support for other properties is coming soon.

??? info "Prefill/decode example"
Node groups can mix CPU and GPU roles. This SGLang prefill/decode split uses a CPU
??? info "PD disaggregation"
While PD disaggregaton is mostly used with [services](services.md#pd-disaggregation), it also possible to run it as tasks. The example below runs a CPU
router (`groups[0]`, the master) and GPU workers. `startup_order: workers-first`
starts prefill and decode before the router.
instructs `dstack` to start prefill and decode workers before the router.

<div editor-title=".dstack.yml">

Expand All @@ -241,8 +241,7 @@ Currently, only `resources`, `commands`, and `ports` can be configured per node
startup_order: workers-first
groups:
# Router (CPU) — master node; wires prefill + decode by IP
- name: router
nodes: 1
- nodes: 1
commands:
- pip install smg
- |
Expand All @@ -259,8 +258,7 @@ Currently, only `resources`, `commands`, and `ports` can be configured per node
resources:
cpu: 4
- name: prefill
nodes: 1
- nodes: 1
commands:
- |
python -m sglang.launch_server \
Expand All @@ -272,8 +270,7 @@ Currently, only `resources`, `commands`, and `ports` can be configured per node
resources:
gpu: H200
- name: decode
nodes: 1
- nodes: 1
commands:
- |
python -m sglang.launch_server \
Expand All @@ -287,9 +284,8 @@ Currently, only `resources`, `commands`, and `ports` can be configured per node

</div>

!!! info "Examples"
See the [Ray+RAGEN](../examples/training/ray-ragen.md) example for running a Ray cluster,
and the [NCCL/RCCL tests](../examples/clusters/nccl-rccl-tests.md) example for running `mpirun` with node groups.
> See the [Ray+RAGEN](../examples/training/ray-ragen.md) example for running a Ray cluster,
and the [NCCL/RCCL tests](../examples/clusters/nccl-rccl-tests.md) example for running `mpirun` with node groups.

### Resources

Expand Down
12 changes: 4 additions & 8 deletions mkdocs/docs/examples/clusters/nccl-rccl-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,7 @@ Here's an example of a task that runs AllReduce test on 2 nodes, each with 4 GPU
- NCCL_DEBUG=INFO

groups:
- name: master # The name property is optional
nodes: 1
- nodes: 1
commands:
- |
mpirun \
Expand All @@ -44,8 +43,7 @@ Here's an example of a task that runs AllReduce test on 2 nodes, each with 4 GPU
gpu: nvidia:1..8
shm_size: 16GB

- name: workers
nodes: 1
- nodes: 1
commands:
- sleep infinity
resources:
Expand Down Expand Up @@ -83,8 +81,7 @@ Here's an example of a task that runs AllReduce test on 2 nodes, each with 4 GPU
- OPEN_MPI_HOME=/usr/lib/x86_64-linux-gnu/openmpi

groups:
- name: master # The name property is optional
nodes: 1
- nodes: 1
commands:
# Setup MPI and build RCCL tests
- apt-get install -y git libopenmpi-dev openmpi-bin
Expand All @@ -110,8 +107,7 @@ Here's an example of a task that runs AllReduce test on 2 nodes, each with 4 GPU
resources:
gpu: MI300X:8

- name: workers
nodes: 1
- nodes: 1
commands:
# Setup MPI and build RCCL tests
- apt-get install -y git libopenmpi-dev openmpi-bin
Expand Down
6 changes: 2 additions & 4 deletions mkdocs/docs/examples/training/ray-ragen.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,7 @@ env:
- WANDB_API_KEY

groups:
- name: head # The name property is optional
nodes: 1
- nodes: 1
commands:
- wget -O miniconda.sh https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
- bash miniconda.sh -b -p /workflow/miniconda
Expand All @@ -52,8 +51,7 @@ groups:
gpu: 80GB:8
shm_size: 128GB

- name: workers
nodes: 1
- nodes: 1
commands:
- wget -O miniconda.sh https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
- bash miniconda.sh -b -p /workflow/miniconda
Expand Down
Loading