Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions documentation/changelog.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,17 @@ This page tracks significant updates to the QuestDB documentation.
### New

- [Migrate QuestDB onto the Kubernetes Operator](/docs/enterprise-kubernetes-operator/getting-started/migrate/) - Move an existing Enterprise deployment onto the Operator with a replica-first cutover: restore the source backup, consume replication WAL, then promote after a controlled source drain
- [ALTER LIVE VIEW](/docs/query/sql/alter-live-view/) - New reference page for managing a live view's disk tier: `SET TTL`, `DROP PARTITION`, `CONVERT PARTITION` between native and Parquet, `RESUME WAL` and `SUSPEND WAL`, with the asynchronous apply, why a manual `DROP PARTITION` can be undone by recovery, the rejected clauses, the per-clause permissions, and how replicas apply the statements locally

### Updated

- [Kafka connector](/docs/connect/message-brokers/kafka/) - Updated for QWP with a quick start, guidance on preventing duplicates and recovering from outages, and migration steps for existing HTTP pipelines
- [Kubernetes Operator](/docs/enterprise-kubernetes-operator/) - Refreshed for Operator 0.2.1 across installation, configuration, high availability, backup and restore, known limitations, troubleshooting, and the generated [API reference](/docs/enterprise-kubernetes-operator/reference/api/)
- [Rust](/docs/connect/clients/rust/) and [C and C++](/docs/connect/clients/c-and-cpp/) clients - Documented the UUID byte order, which was previously unstated. `Buffer::column_uuid` takes `(lo, hi)` where `hi` is the most significant half, matching `java.util.UUID`, and the chunk setter takes the 16 bytes in canonical RFC-4122 order
- [CREATE LIVE VIEW](/docs/query/sql/create-live-view/#ttl) - Documented the `TTL` clause, its partition-size granularity rule, and how the window is enforced during the initial seed
- [Live views](/docs/concepts/live-views/#retention-and-storage-format) - New retention and storage format section, replacing the "no TTL on the view" limitation, plus the replication behavior of the durable-tier statements
- [TTL](/docs/concepts/ttl/#on-live-views) - Live views take TTL at creation and afterwards, evaluated when the view's own table commits
- [Parquet](/docs/concepts/parquet/#bloom-filters-for-in-place-conversion) - Corrected the in-place conversion example: `ALTER TABLE ... CONVERT PARTITION` names the option `fpp` and requires a quoted value, while `bloom_filter_fpp` is the `COPY ... TO` spelling

## August 2026

Expand Down
49 changes: 46 additions & 3 deletions documentation/concepts/live-views.md
Original file line number Diff line number Diff line change
Expand Up @@ -262,6 +262,37 @@ supported. A keep-last `UPSERT` replacement at an earlier timestamp is reflected
in the view. A view over a deduplicated base is one `FLUSH EVERY` cycle behind
rather than sub-cycle fresh, because its refresh is coupled to base apply.

## Retention and storage format

A live view's disk tier is a WAL-backed table, and
[`ALTER LIVE VIEW`](/docs/query/sql/alter-live-view/) manages it with the same
verbs a table has:

- [`SET TTL`](/docs/query/sql/alter-live-view/#set-ttl), or the `TTL` clause of
[`CREATE LIVE VIEW`](/docs/query/sql/create-live-view/#ttl), drops partitions
older than a retention window. The view enforces it on its own commits, so the
window holds without operator action.
- [`DROP PARTITION`](/docs/query/sql/alter-live-view/#drop-partition) removes
partitions on demand.
- [`CONVERT PARTITION`](/docs/query/sql/alter-live-view/#convert-partition)
moves partitions between the native and Parquet storage formats.

A view's retention is independent of its base table's. The view stores computed
rows, so a wide projection can outgrow the base table even when both keep the
same window.

A live view is derived rather than ingested, which gives its retention two
properties a table's does not have. The
[`ALTER LIVE VIEW`](/docs/query/sql/alter-live-view/) page covers both in detail:

- **A manual `DROP PARTITION` is not permanent.** An out-of-order base commit, or
a rebuild from the view's `START FROM` boundary, recomputes output over the
dropped period and writes those rows back. TTL survives recovery, because the
rule is re-applied to the recomputed rows.
- **The two tiers evict independently.** Rows removed from disk can still be
served from the in-memory tier until it is rebuilt, so neither statement makes
data unreadable at a known point in time.

## Monitoring

The [`live_views()`](/docs/query/functions/meta/#live_views) function exposes
Expand Down Expand Up @@ -343,14 +374,16 @@ in [Base table lifecycle](#base-table-lifecycle).
- **Deterministic queries only.** Non-deterministic functions such as `now()`,
`sysdate()`, `systimestamp()`, and `rnd_*()` are rejected in the projection,
the `WHERE` filter, and window-function arguments.
- **No TTL on the view.** Live-view disk growth is unbounded in this version.
Size retention on the base table instead.
- **Retention is partition-granular.** A live view drops whole partitions, as a
table does. Choose the view's `PARTITION BY` with the retention window in
mind, and see [Retention and storage format](#retention-and-storage-format).

## Tradeoffs

- **Storage grows with output.** The computed rows are stored on the live view's
disk tier in addition to the base table's rows. For wide projections or long
retention the view's footprint can exceed the base table.
retention the view's footprint can exceed the base table. Bound it with a
[TTL](#retention-and-storage-format) on the view.
- **No admission control.** A view that cannot keep up with ingestion stays
correct but stale, with no automatic throttle or drop.
- **Per-partition state for partitioned windows grows with distinct partition
Expand Down Expand Up @@ -391,6 +424,14 @@ A role switch continues the local refresh state; it does not reconstruct or
transfer the former primary's live-view rows. Replica freshness therefore also
depends on base-table replication and apply lag.

[`ALTER LIVE VIEW`](/docs/query/sql/alter-live-view/) statements that manage the
disk tier do replicate. They travel over a replicated control table and each node
applies them to its own copy of the view, holding a partition change until its own
refresh has reached the base-table progress the primary had when the change was
taken. A
node with live views or refresh disabled applies them through the ordinary WAL
apply job.

### Backup and restore

A live view is captured by the object-store backup like a materialized view: its
Expand All @@ -402,6 +443,8 @@ base table.

- **SQL commands**
- [`CREATE LIVE VIEW`](/docs/query/sql/create-live-view/): Create a live view
- [`ALTER LIVE VIEW`](/docs/query/sql/alter-live-view/): Manage a live view's
retention, storage format and WAL
- [`DROP LIVE VIEW`](/docs/query/sql/drop-live-view/): Remove a live view

- **Related concepts**
Expand Down
5 changes: 3 additions & 2 deletions documentation/concepts/parquet.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,14 +131,15 @@ optionally set the false positive probability (FPP) using `WITH`:
```questdb-sql title="Convert with explicit bloom filter columns"
ALTER TABLE trades CONVERT PARTITION TO PARQUET
WHERE timestamp < '2025-08-31'
WITH (bloom_filter_columns = 'symbol,side', bloom_filter_fpp = 0.01);
WITH (bloom_filter_columns = 'symbol,side', fpp = '0.01');
```

:::note

When an explicit `bloom_filter_columns` list is provided, it overrides any
per-column `PARQUET(BLOOM_FILTER)` metadata on the table. If the option is
omitted, per-column metadata is used.
omitted, per-column metadata is used. The option is named `fpp` here, and its
value must be quoted. `COPY ... TO` spells the same setting `bloom_filter_fpp`.

:::

Expand Down
27 changes: 27 additions & 0 deletions documentation/concepts/ttl.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,33 @@ A view's TTL is independent of its base table's TTL. For full syntax, see
and
[ALTER MATERIALIZED VIEW SET TTL](/docs/query/sql/alter-mat-view-set-ttl/).

### On live views

[Live views](/docs/concepts/live-views/) take TTL the same way, bounding the
disk tier that holds their computed rows:

```questdb-sql
-- At view creation
CREATE LIVE VIEW trades_ma
FLUSH EVERY 1s
PARTITION BY DAY
TTL 4 WEEKS
START FROM NOW
AS
SELECT timestamp, symbol,
avg(price) OVER (PARTITION BY symbol ORDER BY timestamp ROWS 300 PRECEDING)
AS moving_avg
FROM trades;

-- On an existing view
ALTER LIVE VIEW trades_ma SET TTL 4 WEEKS;
```

A live view evaluates its TTL when its own table commits, which happens on the
`FLUSH EVERY` cadence while the view is producing rows. For full syntax, see
[CREATE LIVE VIEW](/docs/query/sql/create-live-view/#ttl) and
[ALTER LIVE VIEW SET TTL](/docs/query/sql/alter-live-view/#set-ttl).

## How TTL works

TTL drops partitions based on the **partition's time range**, not individual row
Expand Down
Loading
Loading