Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions documentation/architecture/query-engine.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,8 +59,13 @@ to process data in table page frames for better CPU use.
multi-core fashion. Some queries, for example those involving an index, are executed on a single
thread. Other queries, like those involving `GROUP BY` and `SAMPLE BY`, execute a pipeline with some single-threaded stages and some multi-threaded stages to avoid slow downs when groups are unbalanced.

- **Worker pools:** QuestDB allows to configure different pools for specialized functions, like
parsing incoming data, applying WAL file changes, handling PostgreSQL-Wire protocol, or responding to HTTP connections. By default, most tasks are handled by a shared worker pool.
- **Worker pools:** QuestDB splits work across three thread pools that are sized
independently: a network pool for HTTP, PostgreSQL and ILP server I/O, a query pool
for parallel query execution, and a write pool for WAL apply jobs, table writes and
materialized view refresh. Because query execution has its own pool, a burst of
ingestion does not starve readers of CPU. Individual subsystems can still be given
dedicated threads on top of this. See
[shared workers](/docs/configuration/shared-workers/) for sizing.

- **Query plan caching:**
The system caches query plans for reuse within the same connection. (Query results are not
Expand Down
3 changes: 2 additions & 1 deletion documentation/deployment/hetzner.md
Original file line number Diff line number Diff line change
Expand Up @@ -337,7 +337,8 @@ questdb01$ docker restart questdb
For comprehensive configuration options, see the [Configuration reference](/docs/configuration/overview/) documentation. Common production settings include:

- **Connection limits**: `pg.connection.pool.size`
- **Memory settings**: `shared.worker.count`
- **Worker threads**: `shared.network.worker.count`, `shared.query.worker.count`,
`shared.write.worker.count`
- **Security**: [TLS configuration](/docs/security/tls/)
- **Authentication**: [RBAC setup](/docs/security/rbac/)

Expand Down
34 changes: 29 additions & 5 deletions documentation/getting-started/capacity-planning.md
Original file line number Diff line number Diff line change
Expand Up @@ -188,12 +188,36 @@ for other database processes to use.

### CPU cores

By default, QuestDB tries to use all available CPU cores.
By default, QuestDB tries to use all available CPU cores. Assuming that the disk
is not bottlenecked on IOPS, the throughput of read-only queries scales
proportionally with the number of available cores, so a machine with more cores
will provide better query performance.

Work is divided across three independently sized thread pools, which is what you
tune when one class of work needs more CPU than another:

| Pool | Handles | Setting |
| ---- | ------- | ------- |
| Network | HTTP, PostgreSQL and ILP server I/O | `shared.network.worker.count` |
| Query | Parallel query execution — filters, `GROUP BY` | `shared.query.worker.count` |
| Write | WAL apply, table writes, materialized view refresh, housekeeping | `shared.write.worker.count` |

Each pool defaults to roughly the CPU count, so the pools oversubscribe the
machine by design and the OS scheduler arbitrates between them. That is a
reasonable starting point. Move capacity deliberately when a workload is
lopsided:

- **Read-heavy** — raise `shared.query.worker.count` and lower
`shared.network.worker.count`, provided connection counts are modest.
- **Ingestion-heavy** — raise `shared.write.worker.count` so WAL apply keeps up
with incoming data. A growing WAL apply backlog is the signal to watch.
- **Many concurrent clients** — raise `shared.network.worker.count`, since I/O
rather than query execution is the constraint.

On small machines, cutting a pool too far can leave a subsystem unable to make
progress; keep at least two threads in each.
[The guide on shared worker configuration](/docs/configuration/shared-workers/)
explains how to change the default settings. Assuming that the disk is not
bottlenecked on IOPS, the throughput of read-only queries scales proportionally
with the number of available cores. As a result, a machine with more cores will
provide better query performance.
covers the defaults and the per-pool affinity settings.

### Writer page size

Expand Down