Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions docs/data-annotations/continuous-aggregates.md
Original file line number Diff line number Diff line change
Expand Up @@ -207,15 +207,20 @@ By default, continuous aggregates are populated with data when created. Set to `

### CreateGroupIndexes

Controls whether indexes are automatically created on GROUP BY columns. Enabled by default:
Controls whether TimescaleDB creates indexes on the GROUP BY columns when the aggregate is created. Maps to the `timescaledb.create_group_indexes` option:

- **Unconfigured** — the `CreateGroupIndexes` property is never set — the option is omitted from `CREATE MATERIALIZED VIEW`, so TimescaleDB's server default applies.
- Setting **`CreateGroupIndexes = true`** or **`CreateGroupIndexes = false`** emits the option explicitly.

```csharp
[ContinuousAggregate(
MaterializedViewName = "trade_stats",
ParentName = nameof(Trade),
CreateGroupIndexes = true)]
CreateGroupIndexes = false)]
```

> :warning: **Note:** `timescaledb.create_group_indexes` is a create-only option in TimescaleDB — `ALTER MATERIALIZED VIEW` rejects it. Only transitions to or from `false` are structural: because the server default is `true`, unconfigured and explicit `true` produce identical databases, so adding or removing a redundant `CreateGroupIndexes = true` is a no-op that never touches an existing aggregate. Changing the value to or from `false` on an **existing** aggregate drops and recreates it (see [Migration Ordering](../fluent-api/continuous-aggregates#migration-ordering)), which rematerializes it and, in a hierarchy, cascades the drop to every descendant. If the source hypertable's retention policy has already dropped the raw data covered by the aggregate, that history is lost on recreate. The flag is not recoverable from an existing database, so scaffolded aggregates never carry an explicit `CreateGroupIndexes` value.

### MaterializedOnly

By default, queries combine materialized data with recent unmaterialized data from the source hypertable. Set to `true` to return only pre-computed materialized data:
Expand Down
13 changes: 10 additions & 3 deletions docs/fluent-api/continuous-aggregates.md
Original file line number Diff line number Diff line change
Expand Up @@ -285,7 +285,7 @@ public class TradeDaily
Ordering across the chain is handled automatically:

- Parents are created before their children; children are dropped before their parents.
- A structural change to a parent (bucket width, bucket column name, aggregate functions, GROUP BY, or WHERE) drops and recreates all of its descendants as well, and their refresh policies are re-added afterwards.
- A structural change to a parent (bucket width, bucket column name, aggregate functions, GROUP BY, WHERE, or the create-only create_group_indexes option) drops and recreates all of its descendants as well, and their refresh policies are re-added afterwards.

### Scaffolding

Expand All @@ -310,7 +310,10 @@ builder.IsContinuousAggregate<TradeAggregate, Trade>(

### CreateGroupIndexes

Control whether indexes are automatically created on GROUP BY columns. Enabled by default:
Controls whether TimescaleDB creates indexes on the GROUP BY columns when the aggregate is created. This maps to the `timescaledb.create_group_indexes` option:

- **Unconfigured** — no `.CreateGroupIndexes(...)` call — the option is omitted from `CREATE MATERIALIZED VIEW`, so TimescaleDB's server default applies.
- **`.CreateGroupIndexes(true)`** and **`.CreateGroupIndexes(false)`** emit the option explicitly.

```csharp
builder.IsContinuousAggregate<TradeAggregate, Trade>(
Expand All @@ -319,9 +322,13 @@ builder.IsContinuousAggregate<TradeAggregate, Trade>(
x => x.Timestamp)
.AddAggregateFunction(x => x.AveragePrice, x => x.Price, EAggregateFunction.Avg)
.AddGroupByColumn(x => x.Ticker)
.CreateGroupIndexes(true);
.CreateGroupIndexes(false);
```

> :warning: **Note:** In version <= `v10.3.0` an aggregate configured without a `.CreateGroupIndexes(...)` call was created with `timescaledb.create_group_indexes = false`, contradicting the server default. Unconfigured aggregates now omit the option and inherit the server default (`true`). Aggregates that need indexes suppressed must call `.CreateGroupIndexes(false)` explicitly.

> :warning: **Note:** `timescaledb.create_group_indexes` is a create-only option in TimescaleDB — `ALTER MATERIALIZED VIEW` rejects it. Only transitions to or from `false` are structural: because the server default is `true`, unconfigured and `.CreateGroupIndexes(true)` produce identical databases, so adding or removing a redundant `.CreateGroupIndexes(true)` call is a no-op that never touches an existing aggregate. Changing the value to or from `false` on an **existing** aggregate drops and recreates it (see [Migration Ordering](#migration-ordering)), which rematerializes it and, in a hierarchy, cascades the drop to every descendant. If the source hypertable's retention policy has already dropped the raw data covered by the aggregate, that history is lost on recreate. The flag is not recoverable from an existing database — it leaves only the indexes behind — so scaffolded aggregates never carry an explicit `.CreateGroupIndexes(...)` call.

### MaterializedOnly

By default, queries to a continuous aggregate combine materialized data with recent unmaterialized data from the source hypertable. Use `.MaterializedOnly()` to return only the pre-computed materialized data:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ public void Configure(EntityTypeBuilder<TradeAggregate> builder)
.AddGroupByColumn(x => x.Exchange)
.AddGroupByColumn("1, 2")
.Where("\"ticker\" = 'MCRS'")
.CreateGroupIndexes(false)
.MaterializedOnly()
.WithRefreshPolicy(startOffset: "7 days", endOffset: "1 hour", scheduleInterval: "1 hour")
.WithRefreshNewestFirst(true);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ public IReadOnlyList<MethodCallCodeFragment> GenerateFluentApiCalls(
string? chunkInterval = GetString(annotations, ContinuousAggregateAnnotations.ChunkInterval);
bool materializedOnly = Find(annotations, ContinuousAggregateAnnotations.MaterializedOnly)?.Value is true;
bool withNoData = Find(annotations, ContinuousAggregateAnnotations.WithNoData)?.Value is true;
bool createGroupIndexes = Find(annotations, ContinuousAggregateAnnotations.CreateGroupIndexes)?.Value as bool? ?? true;
bool? createGroupIndexes = Find(annotations, ContinuousAggregateAnnotations.CreateGroupIndexes)?.Value as bool?;

ViewDefinitionParser.ParsedViewDefinition? parsed = viewDefinition is not null
? ViewDefinitionParser.Parse(viewDefinition)
Expand Down Expand Up @@ -100,7 +100,7 @@ public IReadOnlyList<MethodCallCodeFragment> GenerateFluentApiCalls(
call = call.Chain(WithNoDataMethod, true);
}

if (!createGroupIndexes)
if (createGroupIndexes is false)
{
call = call.Chain(CreateGroupIndexesMethod, false);
}
Expand Down Expand Up @@ -172,7 +172,7 @@ public IReadOnlyList<AttributeCodeFragment> GenerateDataAnnotationAttributes(
string? chunkInterval = GetString(annotations, ContinuousAggregateAnnotations.ChunkInterval);
bool materializedOnly = Find(annotations, ContinuousAggregateAnnotations.MaterializedOnly)?.Value is true;
bool withNoData = Find(annotations, ContinuousAggregateAnnotations.WithNoData)?.Value is true;
bool createGroupIndexes = Find(annotations, ContinuousAggregateAnnotations.CreateGroupIndexes)?.Value as bool? ?? true;
bool? createGroupIndexes = Find(annotations, ContinuousAggregateAnnotations.CreateGroupIndexes)?.Value as bool?;

ViewDefinitionParser.ParsedViewDefinition? parsed = viewDefinition is not null
? ViewDefinitionParser.Parse(viewDefinition)
Expand Down Expand Up @@ -223,7 +223,7 @@ public IReadOnlyList<AttributeCodeFragment> GenerateDataAnnotationAttributes(
caNamedArgs[nameof(ContinuousAggregateAttribute.WithNoData)] = true;
}

if (!createGroupIndexes)
if (createGroupIndexes is false)
{
caNamedArgs[nameof(ContinuousAggregateAttribute.CreateGroupIndexes)] = false;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ public void Generate(CreateContinuousAggregateOperation operation, IndentedStrin
if (operation.WithNoData)
call.Arg("withNoData", code.Literal(true));

if (operation.CreateGroupIndexes)
call.Arg("createGroupIndexes", code.Literal(true));
if (operation.CreateGroupIndexes.HasValue)
call.Arg("createGroupIndexes", code.Literal(operation.CreateGroupIndexes.Value));

if (operation.MaterializedOnly)
call.Arg("materializedOnly", code.Literal(true));
Expand Down Expand Up @@ -81,8 +81,6 @@ public void Generate(AlterContinuousAggregateOperation operation, IndentedString
if (!string.IsNullOrEmpty(operation.ChunkInterval))
call.Arg("chunkInterval", code.Literal(operation.ChunkInterval));

if (operation.CreateGroupIndexes)
call.Arg("createGroupIndexes", code.Literal(true));

if (operation.MaterializedOnly)
call.Arg("materializedOnly", code.Literal(true));
Expand All @@ -91,9 +89,6 @@ public void Generate(AlterContinuousAggregateOperation operation, IndentedString
if (!string.IsNullOrEmpty(operation.OldChunkInterval))
call.Arg("oldChunkInterval", code.Literal(operation.OldChunkInterval));

if (operation.OldCreateGroupIndexes)
call.Arg("oldCreateGroupIndexes", code.Literal(true));

if (operation.OldMaterializedOnly)
call.Arg("oldMaterializedOnly", code.Literal(true));

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,20 @@ public class ContinuousAggregateAttribute : Attribute

/// <summary>
/// Gets or sets a value indicating whether to automatically create indexes on the GROUP BY columns.
/// Defaults to true. Corresponds to the 'timescaledb.create_group_indexes' option.
/// Defaults to true. Corresponds to the create-only 'timescaledb.create_group_indexes' option.
public bool CreateGroupIndexes
{
get => _createGroupIndexes ?? true;
set => _createGroupIndexes = value;
}

private bool? _createGroupIndexes;

/// <summary>
/// The configured <see cref="CreateGroupIndexes"/> value, or <see langword="null"/> when the property
/// was never set. The convention writes the annotation only when this has a value.
/// </summary>
public bool CreateGroupIndexes { get; set; } = true;
internal bool? CreateGroupIndexesConfigured => _createGroupIndexes;

/// <summary>
/// Gets or sets a value indicating whether queries to the view should only return materialized data.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,10 @@ public void ProcessEntityTypeAdded(IConventionEntityTypeBuilder entityTypeBuilde
entityTypeBuilder.HasAnnotation(ContinuousAggregateAnnotations.ParentName, continuousAggregateAttribute.ParentName);
entityTypeBuilder.HasAnnotation(ContinuousAggregateAnnotations.ChunkInterval, continuousAggregateAttribute.ChunkInterval);
entityTypeBuilder.HasAnnotation(ContinuousAggregateAnnotations.WithNoData, continuousAggregateAttribute.WithNoData);
entityTypeBuilder.HasAnnotation(ContinuousAggregateAnnotations.CreateGroupIndexes, continuousAggregateAttribute.CreateGroupIndexes);
if (continuousAggregateAttribute.CreateGroupIndexesConfigured is bool createGroupIndexes)
{
entityTypeBuilder.HasAnnotation(ContinuousAggregateAnnotations.CreateGroupIndexes, createGroupIndexes);
}
entityTypeBuilder.HasAnnotation(ContinuousAggregateAnnotations.MaterializedOnly, continuousAggregateAttribute.MaterializedOnly);
entityTypeBuilder.HasAnnotation(ContinuousAggregateAnnotations.WhereClause, continuousAggregateAttribute.Where);

Expand Down
20 changes: 8 additions & 12 deletions src/Eftdb/Generators/ContinuousAggregateSqlGenerator.cs
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,14 @@ public static List<string> Generate(CreateContinuousAggregateOperation operation
List<string> statements = [];

// Build WITH options
List<string> withOptions =
[
"timescaledb.continuous",
$"timescaledb.create_group_indexes = {operation.CreateGroupIndexes.ToString().ToLower()}",
$"timescaledb.materialized_only = {operation.MaterializedOnly.ToString().ToLower()}"
];
List<string> withOptions = ["timescaledb.continuous"];

if (operation.CreateGroupIndexes.HasValue)
{
withOptions.Add($"timescaledb.create_group_indexes = {operation.CreateGroupIndexes.Value.ToString().ToLower()}");
}

withOptions.Add($"timescaledb.materialized_only = {operation.MaterializedOnly.ToString().ToLower()}");

// Add optional chunk_interval if specified
if (!string.IsNullOrEmpty(operation.ChunkInterval))
Expand Down Expand Up @@ -226,12 +228,6 @@ public static List<string> Generate(AlterContinuousAggregateOperation operation,
}
}

// Check for CreateGroupIndexes change
if (operation.CreateGroupIndexes != operation.OldCreateGroupIndexes)
{
string createGroupIndexesValue = operation.CreateGroupIndexes.ToString().ToLower();
statements.Add($"ALTER MATERIALIZED VIEW {qualifiedIdentifier} SET (timescaledb.create_group_indexes = {createGroupIndexesValue});");
}

// Check for MaterializedOnly change
if (operation.MaterializedOnly != operation.OldMaterializedOnly)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -87,9 +87,6 @@ public IReadOnlyList<MigrationOperation> GetDifferences(IRelationalModel? source

/// <summary>
/// Find structural changes that require drop and recreate
/// Note: Only certain properties can be altered (ChunkInterval, CreateGroupIndexes,
/// MaterializedOnly, and compression settings).
/// For structural changes (time bucket, aggregates, group by, where), drop and recreate is required.
/// </summary>
private static void FindStructuralChanges(
List<CreateContinuousAggregateOperation> sourceAggregates,
Expand All @@ -113,6 +110,7 @@ private static void FindStructuralChanges(
x.Target.TimeBucketColumnName != x.Source.TimeBucketColumnName ||
x.Target.TimeBucketGroupBy != x.Source.TimeBucketGroupBy ||
x.Target.WithNoData != x.Source.WithNoData ||
(x.Target.CreateGroupIndexes ?? true) != (x.Source.CreateGroupIndexes ?? true) ||
!AreAggregateFunctionsEqual(x.Target.AggregateFunctions, x.Source.AggregateFunctions) ||
!AreGroupByColumnsEqual(x.Target.GroupByColumns, x.Source.GroupByColumns) ||
x.Target.WhereClause != x.Source.WhereClause ||
Expand All @@ -134,9 +132,8 @@ private static void FindStructuralChanges(

/// <summary>
/// Find changes limited to properties that can be applied in place (ChunkInterval,
/// CreateGroupIndexes, MaterializedOnly, and compression settings) and emit alter operations.
/// Aggregates already marked for drop and recreate are skipped; their recreated definition
/// carries the new settings.
/// MaterializedOnly, and compression settings) and emit alter operations. Aggregates already
/// marked for drop and recreate are skipped; their recreated definition carries the new settings.
/// </summary>
private static void FindAlterableChanges(
List<CreateContinuousAggregateOperation> sourceAggregates,
Expand All @@ -155,7 +152,6 @@ private static void FindAlterableChanges(
!droppedNames.Contains(x.Target.MaterializedViewName) &&
(
x.Target.ChunkInterval != x.Source.ChunkInterval ||
x.Target.CreateGroupIndexes != x.Source.CreateGroupIndexes ||
x.Target.MaterializedOnly != x.Source.MaterializedOnly ||
x.Target.EnableCompression != x.Source.EnableCompression ||
!CompressionDiffHelper.AreStringListsEqual(x.Target.CompressionSegmentBy, x.Source.CompressionSegmentBy) ||
Expand All @@ -170,13 +166,11 @@ private static void FindAlterableChanges(
Schema = aggregate.Target.Schema,
MaterializedViewName = aggregate.Target.MaterializedViewName,
ChunkInterval = aggregate.Target.ChunkInterval,
CreateGroupIndexes = aggregate.Target.CreateGroupIndexes,
MaterializedOnly = aggregate.Target.MaterializedOnly,
EnableCompression = aggregate.Target.EnableCompression,
CompressionSegmentBy = aggregate.Target.CompressionSegmentBy,
CompressionOrderBy = aggregate.Target.CompressionOrderBy,
OldChunkInterval = aggregate.Source.ChunkInterval,
OldCreateGroupIndexes = aggregate.Source.CreateGroupIndexes,
OldMaterializedOnly = aggregate.Source.MaterializedOnly,
OldEnableCompression = aggregate.Source.EnableCompression,
OldCompressionSegmentBy = aggregate.Source.CompressionSegmentBy,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ private static IEnumerable<CreateContinuousAggregateOperation> ExtractContinuous
bool timeBucketGroupBy = entityType.FindAnnotation(ContinuousAggregateAnnotations.TimeBucketGroupBy)?.Value as bool? ?? true;
string? chunkInterval = entityType.FindAnnotation(ContinuousAggregateAnnotations.ChunkInterval)?.Value as string;
bool withNoData = entityType.FindAnnotation(ContinuousAggregateAnnotations.WithNoData)?.Value as bool? ?? false;
bool createGroupIndexes = entityType.FindAnnotation(ContinuousAggregateAnnotations.CreateGroupIndexes)?.Value as bool? ?? false;
bool? createGroupIndexes = entityType.FindAnnotation(ContinuousAggregateAnnotations.CreateGroupIndexes)?.Value as bool?;
bool materializedOnly = entityType.FindAnnotation(ContinuousAggregateAnnotations.MaterializedOnly)?.Value as bool? ?? false;
string? whereClause = entityType.FindAnnotation(ContinuousAggregateAnnotations.WhereClause)?.Value as string;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ public static OperationBuilder<CreateContinuousAggregateOperation> CreateContinu
string? schema = null,
string? chunkInterval = null,
bool withNoData = false,
bool createGroupIndexes = false,
bool? createGroupIndexes = null,
bool materializedOnly = false,
string? timeBucketWidth = null,
string? timeBucketSourceColumn = null,
Expand Down Expand Up @@ -75,13 +75,17 @@ public static OperationBuilder<AlterContinuousAggregateOperation> AlterContinuou
MaterializedViewName = materializedViewName,
Schema = schema ?? string.Empty,
ChunkInterval = chunkInterval,
#pragma warning disable CS0618 // Create-only option retained for source compatibility; no longer emitted as SQL.
CreateGroupIndexes = createGroupIndexes,
#pragma warning restore CS0618
MaterializedOnly = materializedOnly,
EnableCompression = enableCompression,
CompressionSegmentBy = compressionSegmentBy,
CompressionOrderBy = compressionOrderBy,
OldChunkInterval = oldChunkInterval,
#pragma warning disable CS0618 // Create-only option retained for source compatibility; no longer emitted as SQL.
OldCreateGroupIndexes = oldCreateGroupIndexes,
#pragma warning restore CS0618
OldMaterializedOnly = oldMaterializedOnly,
OldEnableCompression = oldEnableCompression,
OldCompressionSegmentBy = oldCompressionSegmentBy,
Expand Down
Loading
Loading