Skip to content

docs: document live view TTL, retention and Parquet conversion - #549

Open
puzpuzpuz wants to merge 2 commits into
mainfrom
puzpuzpuz_live_view_ttl
Open

docs: document live view TTL, retention and Parquet conversion#549
puzpuzpuz wants to merge 2 commits into
mainfrom
puzpuzpuz_live_view_ttl

Conversation

@puzpuzpuz

Copy link
Copy Markdown
Contributor

Summary

Documents the durable-tier DDL that questdb/questdb#7612 adds to live views.

New page: ALTER LIVE VIEW. The statement had no reference page at all, so it covers all five clauses: SET TTL, DROP PARTITION, CONVERT PARTITION between native and Parquet, and the pre-existing RESUME WAL and SUSPEND WAL. Beyond the grammar it states what a live view being a derived object costs an operator:

  • the change is applied asynchronously by the refresh worker, so FLUSH EVERY is a cadence and not a deadline;
  • rows removed from disk can still be served from the in-memory tier until it is rebuilt, so neither DROP PARTITION nor SET TTL makes data unreadable at a known point in time;
  • a manual DROP PARTITION is undone by any recovery that recomputes output over the dropped period, which is why SET TTL is the retention that survives recovery;
  • an out-of-order base commit decodes and re-encodes every Parquet partition its correction range covers, so a view over a base table with frequent late writes is a poor Parquet candidate;
  • WITH (bloom_filter_columns = ...) is not stored, and a live view has no per-column PARQUET() metadata to fall back on, so bloom filters last only until the partition is next rewritten.

It also lists the clauses a live view rejects (including FORCE DROP PARTITION), the permission each clause is authorized with, and how a replica applies a relayed statement to its own copy of the view.

CREATE LIVE VIEW: the TTL clause, its partition-size granularity rule, and what a retention window does during the initial seed.

Concept pages: a retention and storage format section on live views, replacing the now-false "no TTL on the view" limitation, plus the replication behaviour of these statements; and a live views subsection on the TTL page next to materialized views.

Unrelated correction: the in-place Parquet conversion example named the option bloom_filter_fpp and left the value unquoted, so it fails as written. ALTER TABLE ... CONVERT PARTITION parses fpp and needs a quoted value; bloom_filter_fpp is the COPY ... TO spelling.

Open question

The SET TTL section documents Enterprise as it behaves today: EntSqlCompilerImpl.alterTableOrMatViewSetTtl lets only a materialized view or a system table through, so a non-zero ALTER LIVE VIEW ... SET TTL is rejected with TTL is not supported on Enterprise tables; use a storage policy instead, while the CREATE LIVE VIEW ... TTL clause is accepted. Whether live views follow the materialized-view rule (TTL allowed) or the table rule (storage policy) is still open on the feature branch. That caution block needs a revisit before this merges.

Dependencies

🤖 Generated with Claude Code

https://claude.ai/code/session_01PbJoRNP8MurxdLzmSuRYzY

Add an ALTER LIVE VIEW reference page for the statements that manage a
live view's disk tier: SET TTL, DROP PARTITION, CONVERT PARTITION between
native and Parquet, and the pre-existing RESUME WAL and SUSPEND WAL. The
page states what a live view's derived nature costs an operator: the
change is applied asynchronously by the refresh worker, rows removed from
disk can still be served from the in-memory tier until it is rebuilt, a
manual DROP PARTITION is undone by any recovery that recomputes output
over the dropped period, and an out-of-order base commit rewrites every
Parquet partition its correction range covers. It also lists the clauses
a live view rejects, including FORCE DROP PARTITION, the permission each
clause is authorized with, and how a replica applies a relayed statement
to its own copy of the view.

Document the CREATE LIVE VIEW TTL clause alongside FLUSH EVERY, IN MEMORY
and PARTITION BY, with the partition-size granularity rule and the
behaviour of a retention window during the initial seed.

Give the live views concept page a retention and storage format section
and drop its "no TTL on the view" limitation, which this work makes
false. Cover live views on the TTL concept page next to materialized
views.

Correct the in-place Parquet conversion example, which named the option
bloom_filter_fpp and left the value unquoted. ALTER TABLE ... CONVERT
PARTITION accepts fpp with a quoted value; bloom_filter_fpp is the
COPY ... TO spelling.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PbJoRNP8MurxdLzmSuRYzY
The relay fences a received command on the view's own base progress so
that a partition removal or conversion affects the same rows on both
nodes. SET TTL carries no fence, because the rule applies to whatever
the view holds when it commits, so calling every relayed statement
fenced overstated it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PbJoRNP8MurxdLzmSuRYzY
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

🚀 Build success!

Latest successful preview: https://preview-549--questdb-documentation.netlify.app/docs/

Commit SHA: a9d82c7

📦 Build generates a preview & updates the link on each commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant