Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/static-gates.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,3 +55,5 @@ jobs:
run: |
python3 scripts/ci/check_advisory_ignores.py --self-test
python3 scripts/ci/check_advisory_ignores.py
- name: Sequence-plane gate
run: python3 scripts/ci/check_sequence_plane.py
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -111,3 +111,6 @@ dmypy.json
.cargo/
.claude/prep/
.pi-subagents

# local search tool index
.tgrep/
2 changes: 2 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@ Welcome to the NodeDB docs. These guides explain what each engine does, when to

## Cross-Engine Features

- [Sequences](sequences.md) — CP-side counters, DEFAULT nextval on every engine, constant-context accessors, typed row-scope errors

- [Bitemporal Queries](bitemporal.md) — System time and valid time, audit trails, corrections, compliance
- [Cross-Engine Identity](architecture.md#cross-engine-identity) — Surrogate bitmaps for fused multi-engine queries

Expand Down
89 changes: 89 additions & 0 deletions docs/sequences.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# Sequences

NodeDB sequences are CP-side, cross-core counters with PostgreSQL-style accessors: `nextval`, `currval`, and `setval`. Every value is allocated through a replicated registry, so two cores (or two statements racing on the same sequence) never hand out the same number.

## When to Use

- Surrogate keys and auto-increment columns
- Per-tenant order numbers, invoice IDs, and shard-safe counters
- Deterministic test fixtures that need reproducible value streams

## Creating and Dropping Sequences

```sql
CREATE SEQUENCE order_ids;
DROP SEQUENCE order_ids;
SHOW SEQUENCES;
```

Sequences are database-scoped. Names follow the same case rules as collections. A sequence advances per call and is never rewound by a restart: the registry is loaded from the catalog on startup and its state is replicated with it. `DROP SEQUENCE` participates in transactional DDL visibility like other catalog objects — a sequence created and dropped inside one transaction leaves nothing usable behind.

## DEFAULT Accessors — Every Engine

The primary use is filling a column per row at insert time. `DEFAULT nextval('seq')` works on every table engine (kv, columnar, document, strict) and advances once per inserted row:

```sql
CREATE SEQUENCE order_ids;

CREATE COLLECTION orders (
id BIGINT DEFAULT nextval('order_ids') PRIMARY KEY,
sku TEXT
) WITH (engine = 'kv');

INSERT INTO orders (sku) VALUES ('A-100'), ('A-101'), ('A-102');
-- id: 1, 2, 3
```

Rules:

- `nextval('seq')` as a DEFAULT is evaluated by the control plane per row, in insertion order.
- `currval('seq')` and `setval('seq', n)` as a DEFAULT are rejected loudly (a stateful DEFAULT must name the value source, not query it). Malformed defaults — `nextval('')`, extra arguments — raise a plan error instead of silently NULLing.
- A DEFAULT that names a missing sequence fails the insert with a plan error naming the sequence.

## Constant Contexts — Real Evaluation

`nextval`/`currval`/`setval` are real expressions wherever a statement has no row scope: FROM-less `SELECT`, and explicit `VALUES` cells.

```sql
SELECT nextval('order_ids'); -- 1, then 2 on the next execution
SELECT currval('order_ids'); -- the value handed out last in this session
SELECT setval('order_ids', 41); -- sets and returns 41; the next nextval returns 42

INSERT INTO orders (id) VALUES (nextval('order_ids')), (nextval('order_ids'));
-- advances per VALUES cell, in order
```

Semantics:

- Multiple accessors in one statement evaluate in expression order.
- `setval` takes `(name, value)`; the following `nextval` returns `value + 1`.
- `EXPLAIN SELECT nextval('seq')` plans without advancing — planning is side-effect-free, matching PostgreSQL.
- A statement that folds an accessor at plan time is never admitted to the physical-plan cache; every execution re-plans and advances.

## Row-Scope Contexts — Loud, Typed Errors

Evaluating a stateful accessor once per row would require a control-plane round-trip per row, which the executor deliberately does not have. Anywhere a row scope exists, accessors raise SQLSTATE `0A000` (`feature_not_supported`) instead of silently evaluating to NULL:

```sql
SELECT nextval('order_ids') FROM orders; -- 0A000
SELECT id FROM orders WHERE nextval('order_ids') > 0; -- 0A000
SELECT id FROM orders ORDER BY nextval('order_ids'); -- 0A000
UPDATE orders SET sku = nextval('order_ids'); -- 0A000
```

This is a deliberate boundary, not an accident: a missing sequence or a malformed accessor in a row-scope context would otherwise surface as a silent NULL per row. The error text names the boundary (`sequence accessors are supported as column DEFAULTs; SELECT-time evaluation is not yet wired`) so the failure mode is self-describing.

## Error Classes

| Situation | SQLSTATE |
|---|---|
| DEFAULT `nextval` per row | — (works) |
| DEFAULT `currval`/`setval`/malformed | plan error (42601) |
| Row-scope accessor (SELECT list, WHERE, ORDER BY, UPDATE SET, JOIN ON, HAVING, GROUP BY) | `0A000` |
| Missing sequence in a constant context | plan error naming the sequence |
| Missing sequence in a DEFAULT | plan error naming the sequence |

## Notes

- The registry is per-database, not per-collection: two collections can share one sequence.
- Sequence values are `BIGINT`. `currval` returns the value this session's registry last produced for that sequence; the registry overlay is connection-scoped, so a session that has produced no value yet behaves per-session rather than reading another session's last value.
4 changes: 4 additions & 0 deletions nodedb-cluster/src/rpc_codec/data_plane_error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -112,4 +112,8 @@ pub enum DataPlaneErrorCode {
limit: u64,
},
DivisionByZero,
/// Registered sequence accessor reached expression evaluation (0A000).
FeatureNotSupported {
name: String,
},
}
8 changes: 8 additions & 0 deletions nodedb-physical/src/physical_plan/kv/op.rs
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,14 @@ pub enum KvOp {
/// See `Get::surrogate_ceiling`; drops entries above the ceiling.
#[serde(default)]
surrogate_ceiling: Option<u32>,
/// Output column names (same format as DocumentOp::Scan). Empty =
/// return the whole row document.
#[serde(default)]
projection: Vec<String>,
/// Serialized `Vec<ComputedColumn>` applied per row after the scan
/// (same format as DocumentOp::Scan). Empty = none.
#[serde(default)]
computed_columns: Vec<u8>,
},

/// Set or update TTL on an existing key.
Expand Down
10 changes: 10 additions & 0 deletions nodedb-query/src/expr/eval.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,16 @@ use super::types::SqlExpr;
pub enum EvalError {
#[error("division by zero")]
DivisionByZero,
/// Sequence accessors (`nextval`/`currval`/`setval`) are stateful and
/// CP-side only. They are evaluated as column DEFAULTs by the plan
/// converter, never by the row-scope scalar evaluator. Reaching this
/// error means an accessor escaped the DEFAULT path (e.g. a bare
/// `SELECT nextval('s')`), which must surface loudly as 0A000 — never as
/// a silent `Null`.
#[error(
"sequence accessors are supported as column DEFAULTs (DEFAULT nextval('s')); SELECT-time evaluation is not yet wired"
)]
FeatureNotSupported { name: &'static str },
}

/// Row scope for `SqlExpr::eval_scope`: how `Column(..)` and `OldColumn(..)`
Expand Down
36 changes: 36 additions & 0 deletions nodedb-query/src/functions/eval.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,19 @@ use super::{array, conditional, datetime, fts, id, json, math, string, system, t
/// fallible arm doesn't force every scalar-function module to carry a
/// `Result` it can never actually produce.
pub fn eval_function(name: &str, args: &[Value]) -> Result<Value, EvalError> {
// Sequence accessors are stateful (CP-side, DEFAULT-scoped). They are
// never scalar-evaluable: if one reaches this dispatcher, it escaped the
// DEFAULT path and must raise 0A000 instead of falling through to the
// geo fallback's silent `Null`.
let canonical = match name.to_ascii_lowercase().as_str() {
"nextval" => Some("nextval"),
"currval" => Some("currval"),
"setval" => Some("setval"),
_ => None,
};
if let Some(cname) = canonical {
return Err(EvalError::FeatureNotSupported { name: cname });
}
if let Some(v) = string::try_eval(name, args) {
return Ok(v);
}
Expand Down Expand Up @@ -63,6 +76,29 @@ mod tests {
eval_function(name, &args).unwrap()
}

#[test]
fn sequence_accessors_are_loud_not_null() {
// Regression: accessors used to fall through to the geo fallback and
// return Ok(Null). They must error as FeatureNotSupported (0A000).
for name in ["nextval", "currval", "setval", "NEXTVAL"] {
let err = eval_function(name, &[Value::String("s".into())]).unwrap_err();
assert!(
matches!(err, crate::expr::EvalError::FeatureNotSupported { .. }),
"{name} must raise FeatureNotSupported, got {err:?}"
);
}
}

#[test]
fn non_sequence_unknown_still_nulls() {
// The guard is scoped: unknown non-sequence names keep the legacy
// geo-fallback behaviour (Ok(Null)), not an error.
assert_eq!(
eval_function("definitely_not_a_fn", &[]).unwrap(),
Value::Null
);
}

#[test]
fn mod_by_zero_errors() {
let err = eval_function("mod", &[Value::Integer(5), Value::Integer(0)]).unwrap_err();
Expand Down
9 changes: 9 additions & 0 deletions nodedb-sql/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,15 @@ pub enum SqlError {
#[error("function {name}(...) does not exist")]
UndefinedFunction { name: String },

/// A registered sequence accessor (`nextval`/`currval`/`setval`) was
/// const-folded or row-evaluated. Accessors are stateful and CP-side
/// only — valid as column DEFAULTs, invalid in any SQL expression
/// context. Maps to SQLSTATE 0A000 (feature not supported).
#[error(
"sequence accessors are supported as column DEFAULTs (DEFAULT nextval('s')); SELECT-time evaluation is not yet wired"
)]
FeatureNotSupported { name: String },

#[error("unknown column '{column}' in table '{table}'")]
UnknownColumn { table: String, column: String },

Expand Down
2 changes: 2 additions & 0 deletions nodedb-sql/src/functions/builtins/scalars.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ mod math;
mod misc;
mod pg_fts;
mod pg_json;
mod sequence;
mod spatial;
mod string;
mod vector;
Expand All @@ -22,6 +23,7 @@ pub(super) fn scalar_functions() -> Vec<FunctionMeta> {
let mut fns = Vec::new();
fns.extend(vector::vector_functions());
fns.extend(spatial::spatial_functions());
fns.extend(sequence::sequence_functions());
fns.extend(datetime::datetime_functions());
fns.extend(doc::doc_functions());
fns.extend(string::string_functions());
Expand Down
85 changes: 85 additions & 0 deletions nodedb-sql/src/functions/builtins/scalars/sequence.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
// SPDX-License-Identifier: Apache-2.0

//! Sequence accessor registrations (`nextval`/`currval`/`setval`).
//!
//! These names are stateful and CP-side only: they evaluate as column
//! DEFAULTs via the plan converter, never as SQL-expression scalars. The
//! plan-time existence gate still needs them registered so that a bare
//! expression use is planned (typed, arity-checked) and then fails LOUDLY
//! at fold/row-eval time with 0A000 — instead of being rejected as
//! "function does not exist" (42883) or silently NULLing at runtime.
//!
//! This list must stay in sync with the sequence guard arm in
//! `nodedb_query::functions::eval_function`.

use nodedb_types::columnar::ColumnType;

use crate::functions::arg_types;
use crate::functions::registry::{ArgTypeSpec, FunctionCategory::Scalar, FunctionMeta};

use super::super::helpers::{m, no_trigger};

static SEQ_1_ARGS: &[ArgTypeSpec] = &[arg_types::any("seq")];
static SEQ_2_ARGS: &[ArgTypeSpec] = &[arg_types::any("seq"), arg_types::any("value")];

pub(super) fn sequence_functions() -> Vec<FunctionMeta> {
vec![
m(
"nextval",
Scalar,
1,
1,
no_trigger(),
Some(ColumnType::Int64),
SEQ_1_ARGS,
),
m(
"currval",
Scalar,
1,
1,
no_trigger(),
Some(ColumnType::Int64),
SEQ_1_ARGS,
),
m(
"setval",
Scalar,
2,
2,
no_trigger(),
Some(ColumnType::Int64),
SEQ_2_ARGS,
),
]
}

#[cfg(test)]
mod tests {
use crate::functions::registry::FunctionRegistry;

#[test]
fn accessors_registered_for_plan_gate() {
let reg = FunctionRegistry::new();
for name in ["nextval", "currval", "setval"] {
assert!(reg.lookup(name).is_some(), "{name} must be registered");
}
}

#[test]
fn accessors_error_loudly_in_expression_eval() {
// A1+A3 contract: registered, arity-checked, then loud 0A000 at
// fold/row-eval — never a silent Null (parity invariant seed).
for name in ["nextval", "currval"] {
let err = nodedb_query::functions::eval_function(
name,
&[nodedb_types::Value::String("s".into())],
)
.unwrap_err();
assert!(
matches!(err, nodedb_query::EvalError::FeatureNotSupported { .. }),
"{name} must raise FeatureNotSupported, got {err:?}"
);
}
}
}
Loading
Loading