From efecba645ef2a3222866088caaf1c667bf1a506b Mon Sep 17 00:00:00 2001 From: Andrei Pechkurov Date: Wed, 9 Sep 2026 17:37:38 +0300 Subject: [PATCH 1/2] docs: document live view TTL, retention and Parquet conversion Add an ALTER LIVE VIEW reference page for the statements that manage a live view's disk tier: SET TTL, DROP PARTITION, CONVERT PARTITION between native and Parquet, and the pre-existing RESUME WAL and SUSPEND WAL. The page states what a live view's derived nature costs an operator: the change is applied asynchronously by the refresh worker, rows removed from disk can still be served from the in-memory tier until it is rebuilt, a manual DROP PARTITION is undone by any recovery that recomputes output over the dropped period, and an out-of-order base commit rewrites every Parquet partition its correction range covers. It also lists the clauses a live view rejects, including FORCE DROP PARTITION, the permission each clause is authorized with, and how a replica applies a relayed statement to its own copy of the view. Document the CREATE LIVE VIEW TTL clause alongside FLUSH EVERY, IN MEMORY and PARTITION BY, with the partition-size granularity rule and the behaviour of a retention window during the initial seed. Give the live views concept page a retention and storage format section and drop its "no TTL on the view" limitation, which this work makes false. Cover live views on the TTL concept page next to materialized views. Correct the in-place Parquet conversion example, which named the option bloom_filter_fpp and left the value unquoted. ALTER TABLE ... CONVERT PARTITION accepts fpp with a quoted value; bloom_filter_fpp is the COPY ... TO spelling. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01PbJoRNP8MurxdLzmSuRYzY --- documentation/changelog.mdx | 5 + documentation/concepts/live-views.md | 48 ++- documentation/concepts/parquet.md | 5 +- documentation/concepts/ttl.md | 27 ++ documentation/query/sql/alter-live-view.md | 346 ++++++++++++++++++++ documentation/query/sql/create-live-view.md | 42 ++- documentation/sidebars.js | 1 + 7 files changed, 466 insertions(+), 8 deletions(-) create mode 100644 documentation/query/sql/alter-live-view.md diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index fd08cbb65..001de1b61 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -17,12 +17,17 @@ This page tracks significant updates to the QuestDB documentation. ### New - [Migrate QuestDB onto the Kubernetes Operator](/docs/enterprise-kubernetes-operator/getting-started/migrate/) - Move an existing Enterprise deployment onto the Operator with a replica-first cutover: restore the source backup, consume replication WAL, then promote after a controlled source drain +- [ALTER LIVE VIEW](/docs/query/sql/alter-live-view/) - New reference page for managing a live view's disk tier: `SET TTL`, `DROP PARTITION`, `CONVERT PARTITION` between native and Parquet, `RESUME WAL` and `SUSPEND WAL`, with the asynchronous apply, why a manual `DROP PARTITION` can be undone by recovery, the rejected clauses, the per-clause permissions, and how replicas apply the statements locally ### Updated - [Kafka connector](/docs/connect/message-brokers/kafka/) - Updated for QWP with a quick start, guidance on preventing duplicates and recovering from outages, and migration steps for existing HTTP pipelines - [Kubernetes Operator](/docs/enterprise-kubernetes-operator/) - Refreshed for Operator 0.2.1 across installation, configuration, high availability, backup and restore, known limitations, troubleshooting, and the generated [API reference](/docs/enterprise-kubernetes-operator/reference/api/) - [Rust](/docs/connect/clients/rust/) and [C and C++](/docs/connect/clients/c-and-cpp/) clients - Documented the UUID byte order, which was previously unstated. `Buffer::column_uuid` takes `(lo, hi)` where `hi` is the most significant half, matching `java.util.UUID`, and the chunk setter takes the 16 bytes in canonical RFC-4122 order +- [CREATE LIVE VIEW](/docs/query/sql/create-live-view/#ttl) - Documented the `TTL` clause, its partition-size granularity rule, and how the window is enforced during the initial seed +- [Live views](/docs/concepts/live-views/#retention-and-storage-format) - New retention and storage format section, replacing the "no TTL on the view" limitation, plus the replication behavior of the durable-tier statements +- [TTL](/docs/concepts/ttl/#on-live-views) - Live views take TTL at creation and afterwards, evaluated when the view's own table commits +- [Parquet](/docs/concepts/parquet/#bloom-filters-for-in-place-conversion) - Corrected the in-place conversion example: `ALTER TABLE ... CONVERT PARTITION` names the option `fpp` and requires a quoted value, while `bloom_filter_fpp` is the `COPY ... TO` spelling ## August 2026 diff --git a/documentation/concepts/live-views.md b/documentation/concepts/live-views.md index 3271bd516..69c554078 100644 --- a/documentation/concepts/live-views.md +++ b/documentation/concepts/live-views.md @@ -262,6 +262,37 @@ supported. A keep-last `UPSERT` replacement at an earlier timestamp is reflected in the view. A view over a deduplicated base is one `FLUSH EVERY` cycle behind rather than sub-cycle fresh, because its refresh is coupled to base apply. +## Retention and storage format + +A live view's disk tier is a WAL-backed table, and +[`ALTER LIVE VIEW`](/docs/query/sql/alter-live-view/) manages it with the same +verbs a table has: + +- [`SET TTL`](/docs/query/sql/alter-live-view/#set-ttl), or the `TTL` clause of + [`CREATE LIVE VIEW`](/docs/query/sql/create-live-view/#ttl), drops partitions + older than a retention window. The view enforces it on its own commits, so the + window holds without operator action. +- [`DROP PARTITION`](/docs/query/sql/alter-live-view/#drop-partition) removes + partitions on demand. +- [`CONVERT PARTITION`](/docs/query/sql/alter-live-view/#convert-partition) + moves partitions between the native and Parquet storage formats. + +A view's retention is independent of its base table's. The view stores computed +rows, so a wide projection can outgrow the base table even when both keep the +same window. + +A live view is derived rather than ingested, which gives its retention two +properties a table's does not have. The +[`ALTER LIVE VIEW`](/docs/query/sql/alter-live-view/) page covers both in detail: + +- **A manual `DROP PARTITION` is not permanent.** An out-of-order base commit, or + a rebuild from the view's `START FROM` boundary, recomputes output over the + dropped period and writes those rows back. TTL survives recovery, because the + rule is re-applied to the recomputed rows. +- **The two tiers evict independently.** Rows removed from disk can still be + served from the in-memory tier until it is rebuilt, so neither statement makes + data unreadable at a known point in time. + ## Monitoring The [`live_views()`](/docs/query/functions/meta/#live_views) function exposes @@ -343,14 +374,16 @@ in [Base table lifecycle](#base-table-lifecycle). - **Deterministic queries only.** Non-deterministic functions such as `now()`, `sysdate()`, `systimestamp()`, and `rnd_*()` are rejected in the projection, the `WHERE` filter, and window-function arguments. -- **No TTL on the view.** Live-view disk growth is unbounded in this version. - Size retention on the base table instead. +- **Retention is partition-granular.** A live view drops whole partitions, as a + table does. Choose the view's `PARTITION BY` with the retention window in + mind, and see [Retention and storage format](#retention-and-storage-format). ## Tradeoffs - **Storage grows with output.** The computed rows are stored on the live view's disk tier in addition to the base table's rows. For wide projections or long - retention the view's footprint can exceed the base table. + retention the view's footprint can exceed the base table. Bound it with a + [TTL](#retention-and-storage-format) on the view. - **No admission control.** A view that cannot keep up with ingestion stays correct but stale, with no automatic throttle or drop. - **Per-partition state for partitioned windows grows with distinct partition @@ -391,6 +424,13 @@ A role switch continues the local refresh state; it does not reconstruct or transfer the former primary's live-view rows. Replica freshness therefore also depends on base-table replication and apply lag. +[`ALTER LIVE VIEW`](/docs/query/sql/alter-live-view/) statements that manage the +disk tier do replicate. They travel over a replicated control table and each node +applies them to its own copy of the view, holding a change until its own refresh +has reached the base-table progress the primary had when the change was taken. A +node with live views or refresh disabled applies them through the ordinary WAL +apply job. + ### Backup and restore A live view is captured by the object-store backup like a materialized view: its @@ -402,6 +442,8 @@ base table. - **SQL commands** - [`CREATE LIVE VIEW`](/docs/query/sql/create-live-view/): Create a live view + - [`ALTER LIVE VIEW`](/docs/query/sql/alter-live-view/): Manage a live view's + retention, storage format and WAL - [`DROP LIVE VIEW`](/docs/query/sql/drop-live-view/): Remove a live view - **Related concepts** diff --git a/documentation/concepts/parquet.md b/documentation/concepts/parquet.md index fd48b3a0d..ace53a461 100644 --- a/documentation/concepts/parquet.md +++ b/documentation/concepts/parquet.md @@ -131,14 +131,15 @@ optionally set the false positive probability (FPP) using `WITH`: ```questdb-sql title="Convert with explicit bloom filter columns" ALTER TABLE trades CONVERT PARTITION TO PARQUET WHERE timestamp < '2025-08-31' -WITH (bloom_filter_columns = 'symbol,side', bloom_filter_fpp = 0.01); +WITH (bloom_filter_columns = 'symbol,side', fpp = '0.01'); ``` :::note When an explicit `bloom_filter_columns` list is provided, it overrides any per-column `PARQUET(BLOOM_FILTER)` metadata on the table. If the option is -omitted, per-column metadata is used. +omitted, per-column metadata is used. The option is named `fpp` here, and its +value must be quoted. `COPY ... TO` spells the same setting `bloom_filter_fpp`. ::: diff --git a/documentation/concepts/ttl.md b/documentation/concepts/ttl.md index 9c1ee9f40..81cd713f6 100644 --- a/documentation/concepts/ttl.md +++ b/documentation/concepts/ttl.md @@ -90,6 +90,33 @@ A view's TTL is independent of its base table's TTL. For full syntax, see and [ALTER MATERIALIZED VIEW SET TTL](/docs/query/sql/alter-mat-view-set-ttl/). +### On live views + +[Live views](/docs/concepts/live-views/) take TTL the same way, bounding the +disk tier that holds their computed rows: + +```questdb-sql +-- At view creation +CREATE LIVE VIEW trades_ma +FLUSH EVERY 1s +PARTITION BY DAY +TTL 4 WEEKS +START FROM NOW +AS +SELECT timestamp, symbol, + avg(price) OVER (PARTITION BY symbol ORDER BY timestamp ROWS 300 PRECEDING) + AS moving_avg +FROM trades; + +-- On an existing view +ALTER LIVE VIEW trades_ma SET TTL 4 WEEKS; +``` + +A live view evaluates its TTL when its own table commits, which happens on the +`FLUSH EVERY` cadence while the view is producing rows. For full syntax, see +[CREATE LIVE VIEW](/docs/query/sql/create-live-view/#ttl) and +[ALTER LIVE VIEW SET TTL](/docs/query/sql/alter-live-view/#set-ttl). + ## How TTL works TTL drops partitions based on the **partition's time range**, not individual row diff --git a/documentation/query/sql/alter-live-view.md b/documentation/query/sql/alter-live-view.md new file mode 100644 index 000000000..03ef61050 --- /dev/null +++ b/documentation/query/sql/alter-live-view.md @@ -0,0 +1,346 @@ +--- +title: ALTER LIVE VIEW +sidebar_label: ALTER LIVE VIEW +description: + ALTER LIVE VIEW SQL keyword reference documentation, covering TTL, partition + retention, Parquet conversion and WAL recovery on a live view. +--- + +Manages the disk tier of a [live view](/docs/concepts/live-views/): how long its +computed rows are retained, the storage format of its partitions, and the +recovery of its WAL writer. Retention keeps a view's footprint bounded, and +Parquet conversion compresses the older partitions a view no longer appends to. + +A live view's query, output schema and refresh cadence are fixed at creation and +cannot be altered. `ALTER LIVE VIEW` accepts five clauses: + +- [`CONVERT PARTITION`](#convert-partition) switches partitions between the + native and Parquet storage formats. +- [`DROP PARTITION`](#drop-partition) removes partitions from disk immediately. +- [`RESUME WAL`](#resume-wal) restarts a suspended view. +- [`SET TTL`](#set-ttl) sets a retention period the view enforces itself. +- [`SUSPEND WAL`](#suspend-wal) stops the view's apply on purpose. + +## Syntax + +```questdb-sql title="CONVERT PARTITION" +ALTER LIVE VIEW viewName CONVERT PARTITION TO PARQUET + { LIST partitionName [, partitionName ...] | WHERE booleanExpression } + [ WITH ( parquetOption [, parquetOption ...] ) ]; + +ALTER LIVE VIEW viewName CONVERT PARTITION TO NATIVE + { LIST partitionName [, partitionName ...] | WHERE booleanExpression }; +``` + +```questdb-sql title="DROP PARTITION" +ALTER LIVE VIEW viewName DROP PARTITION LIST partitionName [, partitionName ...]; + +ALTER LIVE VIEW viewName DROP PARTITION WHERE booleanExpression; +``` + +```questdb-sql title="RESUME WAL" +ALTER LIVE VIEW viewName RESUME WAL [ FROM { TRANSACTION | TXN } sequencerTxn ]; +``` + +```questdb-sql title="SET TTL" +ALTER LIVE VIEW viewName SET TTL + n { HOUR[S] | DAY[S] | WEEK[S] | MONTH[S] | YEAR[S] }; +``` + +```questdb-sql title="SUSPEND WAL" +ALTER LIVE VIEW viewName SUSPEND WAL; +``` + +Where [`parquetOption`](#parquet-options) is a bloom filter setting for the +conversion, and `booleanExpression` filters on the view's designated timestamp. + +## Parameters + +| Parameter | Description | +| --------- | ----------- | +| `viewName` | Name of the live view to modify | +| `partitionName` | Partition directory name, following the [partition naming convention](/docs/concepts/partitions/) | +| `booleanExpression` | Predicate on the view's designated timestamp, selecting the partitions to act on | +| `n` | Number of time units to retain | +| `sequencerTxn` | Transaction to resume from. Defaults to the failed transaction | + +## How the change is applied + +A live view holds its computed rows in two tiers: an in-memory tier that serves +fresh reads, and a WAL-backed disk tier written on the `FLUSH EVERY` cadence. +`ALTER LIVE VIEW` changes the disk tier, and it does so asynchronously, the way +a WAL table does. + +The statement returns once the change is committed to the view's sequencer. The +refresh worker applies it alongside the next flush, or on its next scan if the +view is idle. `FLUSH EVERY` sets the cadence, not a deadline: writer contention, +apply backoff and an in-flight out-of-order repair can all delay the change. + +Rows the change removes from disk can still be served from the in-memory tier +until that tier is rebuilt, so neither `DROP PARTITION` nor `SET TTL` makes data +unreadable at a known point in time. + +A suspended view applies nothing until [`RESUME WAL`](#resume-wal), including +these statements. Suspension is visible in +[`wal_tables()`](/docs/query/functions/meta/#wal_tables). + +## CONVERT PARTITION + +Converts partitions of the view's disk tier between QuestDB's native format and +[Parquet](/docs/concepts/parquet/). This changes the storage format only: the +rows, the view's output and every query over it are unaffected. + +```questdb-sql title="Convert older partitions to Parquet" +ALTER LIVE VIEW trades_ma CONVERT PARTITION TO PARQUET +WHERE timestamp < '2026-08-01'; +``` + +```questdb-sql title="Convert one partition back to native" +ALTER LIVE VIEW trades_ma CONVERT PARTITION TO NATIVE LIST '2026-07-15'; +``` + +Convert partitions the view has moved past. The newest partition takes every +flush, and a write into a Parquet partition is a merge that rewrites the whole +file, so leaving it native keeps the flush cheap. + +### Parquet options + +`WITH` accepts the same bloom filter options as +[`ALTER TABLE CONVERT PARTITION`](/docs/concepts/parquet/#bloom-filters-for-in-place-conversion), +and only when converting to Parquet: + +- `bloom_filter_columns = 'col[,col ...]'` builds bloom filters for the listed + columns, enabling row group pruning for equality and `IN` queries. +- `fpp = 'probability'` sets the false positive probability, exclusive between 0 + and 1. Quote the value. + +```questdb-sql title="Convert with bloom filters" +ALTER LIVE VIEW trades_ma CONVERT PARTITION TO PARQUET +LIST '2026-07-15' +WITH (bloom_filter_columns = 'symbol', fpp = '0.01'); +``` + +:::caution + +`WITH` options are not stored on the view. Anything that rewrites a converted +partition later re-encodes it from the server's Parquet configuration, without +the bloom filters. A live view has no per-column `PARQUET()` metadata to fall +back on, so bloom filters set here survive only until the partition is next +rewritten. + +::: + +### Out-of-order base commits over a Parquet partition + +Rows cannot be removed from a Parquet file in place. When a late base-table +commit forces the view to correct output it has already written, the writer +converts every Parquet partition the correction covers back to native, applies +the correction, and converts them back. Reads of those partitions are unaffected +and the partitions are Parquet again when the apply finishes. + +The cost is a full decode and re-encode of each covered partition, per +correction. A correction that cannot anchor on a checkpoint covers the view's +whole range, and then pays it for every Parquet partition the view holds. A view +over a base table that takes frequent out-of-order writes is a poor candidate +for Parquet conversion. + +If the process dies during that rewrite, the partition is left native and the +next writer to open the view finishes the re-encode. Nothing is lost but the +compaction, and `CONVERT PARTITION TO PARQUET` restores it. + +## DROP PARTITION + +Removes whole partitions from the view's disk tier. Both selectors of +[`ALTER TABLE DROP PARTITION`](/docs/query/sql/alter-table-drop-partition/) are +accepted, and partition names follow the same convention. Inspect the view's +partitions with +[`table_partitions()`](/docs/query/functions/meta/#table_partitions). + +```questdb-sql title="Drop a partition by name" +ALTER LIVE VIEW trades_ma DROP PARTITION LIST '2026-07-15'; +``` + +```questdb-sql title="Drop everything before a date" +ALTER LIVE VIEW trades_ma DROP PARTITION WHERE timestamp < '2026-08-01'; +``` + +The newest partition cannot be dropped. The refresh pipeline appends to it and +an out-of-order correction rewrites it, so it is rejected with +`cannot drop the active partition of a live view [partition=...]`, both when the +statement is compiled and again when it is applied. + +:::caution + +`DROP PARTITION` removes durable rows now, and only now. A live view is derived +from its base table, so any later recovery that recomputes output over the +dropped period brings those rows back: + +- an out-of-order base commit whose correction range covers the dropped period + re-emits the overlapping rows; +- a restart that cannot resume from the view's checkpoint rebuilds the view from + its `START FROM` boundary, re-materializing every dropped partition the base + table still holds. + +For retention that survives recovery, use [`SET TTL`](#set-ttl), which the view +re-enforces on every commit. + +::: + +## RESUME WAL + +Restarts WAL transactions on a live view after the error that suspended it has +been resolved. It behaves as +[`ALTER TABLE RESUME WAL`](/docs/query/sql/alter-table-resume-wal/) does, and +recovers only a suspended WAL writer. It does not revalidate a view that was +invalidated by a base-table schema change. + +```questdb-sql title="Resume from the failed transaction" +ALTER LIVE VIEW trades_ma RESUME WAL; +``` + +```questdb-sql title="Skip past a transaction" +ALTER LIVE VIEW trades_ma RESUME WAL FROM TRANSACTION 5; +``` + +## SET TTL + +Sets a [time-to-live](/docs/concepts/ttl/) period on the view's disk tier, +dropping partitions whose entire time range falls outside the window. A view's +TTL is independent of its base table's TTL. + +```questdb-sql title="Keep four weeks of computed rows" +ALTER LIVE VIEW trades_ma SET TTL 4 WEEKS; +``` + +```questdb-sql title="Shorthand form" +ALTER LIVE VIEW trades_ma SET TTL 12h; +``` + +Accepted units are `HOUR[S]`, `DAY[S]`, `WEEK[S]`, `MONTH[S]` and `YEAR[S]`, with +the `h`, `d`, `w`, `M` and `y` shorthands. The period must be a whole number +multiple of the view's partition size, which is the view's `PARTITION BY` if it +declared one, and the base table's scheme otherwise. Reference-time and +partition-boundary rules are the table rules, described in +[TTL](/docs/concepts/ttl/). + +Clearing a TTL needs a unit as well: + +```questdb-sql title="Clear the retention period" +ALTER LIVE VIEW trades_ma SET TTL 0h; +``` + +The view evaluates its TTL whenever its own table commits, which is the flush +cadence while the view is producing rows, plus the commit that applies this +statement. A view that has stopped producing output stops evicting. + +Unlike [`DROP PARTITION`](#drop-partition), a TTL survives recovery: a rebuilt +view re-applies the same rule to the recomputed rows, so the retention window +converges again without operator action. + +Read the current setting from [`tables()`](/docs/query/functions/meta/#tables), +where a `ttlValue` of `0` means no TTL: + +```questdb-sql title="Check the retention period" +SELECT table_name, ttlValue, ttlUnit FROM tables() +WHERE table_name = 'trades_ma'; +``` + +`SHOW CREATE LIVE VIEW` re-emits a non-zero TTL, so a view altered here +round-trips through its own DDL. + +:::caution + +On QuestDB Enterprise, TTL is superseded by +[storage policy](/docs/concepts/storage-policy/) for tables, and a non-zero +`SET TTL` on a live view is currently rejected with +`TTL is not supported on Enterprise tables; use a storage policy instead`. +`SET TTL 0` is accepted, and the `TTL` clause of +[`CREATE LIVE VIEW`](/docs/query/sql/create-live-view/#ttl) is accepted. + +::: + +## SUSPEND WAL + +Stops the apply of the view's WAL, leaving the view quiescent. It behaves as +[`ALTER TABLE SUSPEND WAL`](/docs/query/sql/alter-table-suspend-wal/) does: +refresh keeps computing and committing to the sequencer, nothing is applied, and +queries stop seeing new rows until `RESUME WAL` drains the queued transactions in +order. Suspending and resuming share a single authorization. + +```questdb-sql title="Suspend a live view" +ALTER LIVE VIEW trades_ma SUSPEND WAL; +``` + +## Unsupported clauses + +`ALTER TABLE` never reaches a live view. It fails with `cannot modify live view`, +whichever clause follows. + +Every `ALTER LIVE VIEW` clause outside the five above is rejected, because a live +view's schema is a function of its `SELECT`. That covers `ADD COLUMN`, +`ALTER COLUMN`, `RENAME`, `ATTACH PARTITION`, `DETACH PARTITION`, +`SQUASH PARTITIONS`, `DEDUP`, `SET PARAM`, `SET TYPE` and `SET FORMAT`. + +`FORCE DROP PARTITION` is rejected separately, with +`FORCE DROP PARTITION is not supported on live views`. On a table it bypasses the +WAL and writes through a directly acquired writer, which on a live view is owned +by the refresh worker. The recovery a live view has is `SUSPEND WAL`, +`RESUME WAL` and the ordinary sequenced `DROP PARTITION`. + +## Permissions (Enterprise) + +Each clause is authorized with the same permission its `ALTER TABLE` counterpart +uses, checked against the live view: + +| Clause | Permission | +| ------ | ---------- | +| `CONVERT PARTITION TO NATIVE` | `CONVERT PARTITION TO NATIVE` | +| `CONVERT PARTITION TO PARQUET` | `CONVERT PARTITION TO PARQUET` | +| `DROP PARTITION` | `DROP PARTITION` | +| `RESUME WAL`, `SUSPEND WAL` | `RESUME WAL` | +| `SET TTL` | `SET TABLE PARAM` | + +```questdb-sql title="Grant retention management on one view" +GRANT DROP PARTITION, SET TABLE PARAM ON trades_ma TO user1; +``` + +See [Role-based access control](/docs/security/rbac/) for the full model. + +## Replication (Enterprise) + +A live view's rows are node-local: every node with live views enabled computes +and flushes its own copy, and live-view WAL is never transferred between nodes. +These statements are relayed to replicas over a replicated control table and +applied by each node to its own copy of the view, so retention and storage format +converge without shipping rows. + +A replica holds a relayed change until its own refresh has reached the base-table +progress the primary had when it took the change, so both nodes remove or convert +the same rows. A `WHERE` selector is resolved to a concrete partition list on the +primary and travels as that list, which keeps the two nodes from resolving the +same predicate against different data. A node with live views or refresh disabled +applies the change through the ordinary WAL apply job. + +## Errors + +| Error | Cause | +| ----- | ----- | +| `cannot modify live view` | `ALTER TABLE` was used on a live view. Use `ALTER LIVE VIEW` | +| `'set', 'drop', 'convert', 'resume' or 'suspend' expected` | The clause is not part of the `ALTER LIVE VIEW` grammar | +| `'ttl' expected` | `SET` was followed by something other than `TTL` | +| `FORCE DROP PARTITION is not supported on live views` | `FORCE DROP PARTITION` was used on a live view | +| `cannot drop the active partition of a live view [partition=...]` | The dropped partition is the one the view is appending to | +| `TTL value must be an integer multiple of the partition size` | The TTL period is not a whole multiple of the view's partition size | +| `missing unit, 'HOUR(S)', 'DAY(S)', 'WEEK(S)', 'MONTH(S)' or 'YEAR(S)' expected` | `SET TTL 0` was written without a unit | +| `no partitions matched WHERE clause` | The `WHERE` selector matched no partition when the statement was compiled | +| `bloom_filter_columns or fpp expected` | An unknown option was passed to `WITH` | +| `permission denied` | Missing permission (Enterprise) | + +## See also + +- [Live views concept](/docs/concepts/live-views/) +- [CREATE LIVE VIEW](/docs/query/sql/create-live-view/) +- [DROP LIVE VIEW](/docs/query/sql/drop-live-view/) +- [TTL concept](/docs/concepts/ttl/) +- [Parquet](/docs/concepts/parquet/) +- [table_partitions()](/docs/query/functions/meta/#table_partitions) diff --git a/documentation/query/sql/create-live-view.md b/documentation/query/sql/create-live-view.md index 6c6fd579d..d5b7266ec 100644 --- a/documentation/query/sql/create-live-view.md +++ b/documentation/query/sql/create-live-view.md @@ -25,6 +25,7 @@ CREATE LIVE VIEW [ IF NOT EXISTS ] viewName FLUSH EVERY duration [ IN MEMORY duration ] [ PARTITION BY ( YEAR | MONTH | WEEK | DAY | HOUR ) ] +[ TTL n { HOUR[S] | DAY[S] | WEEK[S] | MONTH[S] | YEAR[S] } ] START FROM ( NOW | BEGINNING | 'timestamp' ) AS [ ( ] query [ ) ] [ OWNED BY ownerName ] @@ -38,9 +39,9 @@ Where: [window functions](/docs/query/functions/window-functions/overview/). `FLUSH EVERY` is required and must come first. `START FROM` is also required and -may appear in any order with the optional `IN MEMORY` and `PARTITION BY` -clauses. These clauses all precede `AS`; the optional `OWNED BY` clause follows -the query. +may appear in any order with the optional `IN MEMORY`, `PARTITION BY` and `TTL` +clauses. Each may appear at most once. These clauses all precede `AS`; the +optional `OWNED BY` clause follows the query. ## Parameters @@ -51,6 +52,7 @@ the query. | `FLUSH EVERY` | How often computed rows are persisted to disk. Required | | `IN MEMORY` | Window of recent rows kept in RAM for fresh reads. Defaults to `FLUSH EVERY` | | `PARTITION BY` | Partitioning unit for the view's disk tier. Defaults to the base table's scheme | +| `TTL` | Retention period for the view's disk tier. No retention by default | | `START FROM` | Inclusive event-time boundary: `NOW`, `BEGINNING`, or a timestamp literal. Required | | `query` | A window-function `SELECT` over a single WAL-backed base table | | `OWNED BY` | Assign ownership (Enterprise) | @@ -122,6 +124,37 @@ SELECT timestamp, symbol, FROM trades; ``` +### TTL + +`TTL` sets a [time-to-live](/docs/concepts/ttl/) period on the view's disk tier. +QuestDB drops partitions of the view whose entire time range falls outside the +window, which bounds the view's footprint without touching the base table. A +view's TTL is independent of its base table's TTL. + +The period must be a whole number multiple of the view's partition size, whether +that comes from `PARTITION BY` or from the base table. Accepted units are +`HOUR[S]`, `DAY[S]`, `WEEK[S]`, `MONTH[S]` and `YEAR[S]`, with the `h`, `d`, `w`, +`M` and `y` shorthands. + +```questdb-sql +CREATE LIVE VIEW trades_ma +FLUSH EVERY 1s +PARTITION BY DAY +TTL 4 WEEKS +START FROM NOW +AS +SELECT timestamp, symbol, + avg(price) OVER (PARTITION BY symbol ORDER BY timestamp ROWS 300 PRECEDING) + AS moving_avg +FROM trades; +``` + +The window is enforced against the view's own rows as the refresh worker writes +them, so a view created with `START FROM BEGINNING` and a short `TTL` still +recomputes the whole history during its initial seed, evicting as it goes. Change +or clear the period afterwards with +[`ALTER LIVE VIEW SET TTL`](/docs/query/sql/alter-live-view/#set-ttl). + ### START FROM `START FROM` defines the inclusive event-time boundary for rows in the live @@ -336,11 +369,14 @@ OWNED BY analysts; | `wildcard column select is not allowed in live view queries` | The top-level projection contains `*` | | `live view unbounded window must have an ANCHOR clause` | A stateful partitioned window uses the default unbounded frame without an anchor | | `non-deterministic function cannot be used in live view` | The query uses `now()`, `rnd_*()`, or a similar non-deterministic function | +| `TTL value must be an integer multiple of the partition size` | The `TTL` period is not a whole multiple of the view's partition size | +| `live view TTL clause specified more than once` | The `TTL` clause appears twice | | `permission denied` | Missing required permission (Enterprise) | ## See also - [Live views concept](/docs/concepts/live-views/) +- [ALTER LIVE VIEW](/docs/query/sql/alter-live-view/) - [DROP LIVE VIEW](/docs/query/sql/drop-live-view/) - [Window functions](/docs/query/functions/window-functions/overview/) - [live_views()](/docs/query/functions/meta/#live_views) diff --git a/documentation/sidebars.js b/documentation/sidebars.js index f0b4f587f..9dc4d761c 100644 --- a/documentation/sidebars.js +++ b/documentation/sidebars.js @@ -329,6 +329,7 @@ module.exports = { "query/sql/alter-table-change-symbol-capacity", // SYMBOL CAPACITY ], }, + "query/sql/alter-live-view", { type: "category", label: "ALTER MATERIALIZED VIEW", From a9d82c76e5f8bd73b0618f227fb5ca29ac5807f1 Mon Sep 17 00:00:00 2001 From: Andrei Pechkurov Date: Wed, 9 Sep 2026 17:41:08 +0300 Subject: [PATCH 2/2] docs: scope the live view relay fence to partition changes The relay fences a received command on the view's own base progress so that a partition removal or conversion affects the same rows on both nodes. SET TTL carries no fence, because the rule applies to whatever the view holds when it commits, so calling every relayed statement fenced overstated it. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01PbJoRNP8MurxdLzmSuRYzY --- documentation/concepts/live-views.md | 5 +++-- documentation/query/sql/alter-live-view.md | 7 ++++--- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/documentation/concepts/live-views.md b/documentation/concepts/live-views.md index 69c554078..174d46a26 100644 --- a/documentation/concepts/live-views.md +++ b/documentation/concepts/live-views.md @@ -426,8 +426,9 @@ depends on base-table replication and apply lag. [`ALTER LIVE VIEW`](/docs/query/sql/alter-live-view/) statements that manage the disk tier do replicate. They travel over a replicated control table and each node -applies them to its own copy of the view, holding a change until its own refresh -has reached the base-table progress the primary had when the change was taken. A +applies them to its own copy of the view, holding a partition change until its own +refresh has reached the base-table progress the primary had when the change was +taken. A node with live views or refresh disabled applies them through the ordinary WAL apply job. diff --git a/documentation/query/sql/alter-live-view.md b/documentation/query/sql/alter-live-view.md index 03ef61050..5ae5b2ee6 100644 --- a/documentation/query/sql/alter-live-view.md +++ b/documentation/query/sql/alter-live-view.md @@ -314,9 +314,10 @@ These statements are relayed to replicas over a replicated control table and applied by each node to its own copy of the view, so retention and storage format converge without shipping rows. -A replica holds a relayed change until its own refresh has reached the base-table -progress the primary had when it took the change, so both nodes remove or convert -the same rows. A `WHERE` selector is resolved to a concrete partition list on the +A replica holds a relayed partition change until its own refresh has reached the +base-table progress the primary had when it took the change, so both nodes remove +or convert the same rows. `SET TTL` carries no such fence, since the view applies +the rule to whatever it holds. A `WHERE` selector is resolved to a concrete partition list on the primary and travels as that list, which keeps the two nodes from resolving the same predicate against different data. A node with live views or refresh disabled applies the change through the ordinary WAL apply job.