Feature/hypercore

.claude/reference/file-organization.md

@@ -23,6 +23,9 @@ Quick reference for locating key files in the CmdScale.EntityFrameworkCore.Times
 | `Configuration/Hypertable/HypertableAttribute.cs` | Data annotation |
 | `Configuration/Hypertable/DimensionAttribute.cs` | Data annotation for additional partitioning dimensions |
 | `Configuration/Hypertable/HypertableConvention.cs` | Convention processing |
+| `Configuration/Hypertable/SparseIndex.cs` | `SparseIndex` value type and `SparseIndexSelector<TEntity>` typed fluent builder |
+| `Configuration/Hypertable/SparseIndexAttribute.cs` | `[SparseIndex]` data annotation (AllowMultiple); also `DisableAutoSparseIndexes` on `[Hypertable]` |
+| `Configuration/Hypertable/SparseIndexValidationConvention.cs` | IModelFinalizedConvention that validates sparse index entries (bloom/minmax arity, segmentby/orderby prerequisites, duplicates) |
 | `Internals/Features/Hypertables/HypertableDiffer.cs` | Diffing logic |
 | `Internals/Features/Hypertables/HypertableModelExtractor.cs` | Model extraction |
 | `Generators/HypertableSqlGenerator.cs` | Runtime SQL generation |
@@ -64,6 +67,28 @@ Quick reference for locating key files in the CmdScale.EntityFrameworkCore.Times
 | `Operations/AlterRetentionPolicyOperation.cs` | Migration operation |
 | `Operations/DropRetentionPolicyOperation.cs` | Migration operation |
 
+### Compression Policy
+
+| File | Purpose |
+|------|---------|
+| `Configuration/CompressionPolicy/CompressionPolicyTypeBuilder.cs` | Fluent API (including scaffold-targeting overload) |
+| `Configuration/CompressionPolicy/CompressionPolicyStringBuilder.cs` | String-based builder used in scaffolded code |
+| `Configuration/CompressionPolicy/CompressionPolicyAnnotations.cs` | Annotation constants |
+| `Configuration/CompressionPolicy/CompressionPolicyAttribute.cs` | Data annotation |
+| `Configuration/CompressionPolicy/CompressionPolicyConvention.cs` | Convention processing |
+| `Configuration/CompressionPolicy/CompressionPolicyPrerequisiteValidationConvention.cs` | IModelFinalizedConvention that validates compression is enabled on any continuous aggregate that has a compression policy |
+| `Internals/Features/CompressionPolicies/CompressionPolicyDiffer.cs` | Diffing logic |
+| `Internals/Features/CompressionPolicies/CompressionPolicyModelExtractor.cs` | Model extraction |
+| `Internals/Features/CompressionPolicies/CompressionPolicyDefaultHelper.cs` | Dynamic schedule_interval default (12h when chunk interval >= 1 day, else half the chunk interval) |
+| `Internals/CompressionAnnotationExtractor.cs` | Shared helper for extracting segment-by and order-by column lists from entity annotations; used by both hypertable and continuous-aggregate model extractors |
+| `Internals/Features/CompressionDiffHelper.cs` | Shared comparison and rewrite helpers for compression differ logic; used by both hypertable and continuous-aggregate differs |
+| `Generators/CompressionPolicySqlGenerator.cs` | Runtime SQL generation |
+| `Generators/CompressionSettingsSqlHelper.cs` | Shared SQL-building helpers for compression settings (compress SET clause, alter diff, enable-state check); used by hypertable and continuous-aggregate SQL generators |
+| `MigrationExtensions/CompressionPolicyMigrationExtensions.cs` | Typed migrationBuilder methods |
+| `Operations/AddCompressionPolicyOperation.cs` | Migration operation |
+| `Operations/AlterCompressionPolicyOperation.cs` | Migration operation |
+| `Operations/DropCompressionPolicyOperation.cs` | Migration operation |
+
 ### Continuous Aggregate
 
 | File | Purpose |
@@ -123,13 +148,17 @@ Quick reference for locating key files in the CmdScale.EntityFrameworkCore.Times
 | `Internals/TimescaleMigrationsModelDiffer.cs` | Differ orchestration, context building, operation prioritization |
 | `Internals/Features/IFeatureDiffer.cs` | Differ interface |
 | `Internals/Features/FeatureDiffContext.cs` | Cross-cutting diff state (renames, recreated aggregates) |
-| `Generators/SqlBuilderHelper.cs` | Identifier quoting, regclass, command grouping, SELECT→PERFORM |
+| `Generators/SqlBuilderHelper.cs` | Identifier quoting, regclass, command grouping, SELECT→PERFORM, `EscapeStringLiteral`, `FormatTimestamp` |
 | `Generators/PolicyJobSqlBuilder.cs` | Shared `alter_job` clause builder for policies |
+| `Configuration/ConventionValidationHelper.cs` | Shared validation helpers for conventions: `ValidateExclusiveFields` (XOR guard) and `ParseInitialStart` (DateTime parse with error context) |
 | `Configuration/TimeColumnStoreTypeValidationConvention.cs` | Model-finalized validation of hypertable & continuous-aggregate time-column store types |
 | `Internals/TimeColumnStoreTypeValidator.cs` | Allowed PostgreSQL store types for a TimescaleDB time dimension |
+| `Internals/ExpressionHelper.cs` | Shared helper consolidating CLR property-name extraction from lambda expressions |
 | `DefaultValues.cs` | Centralized defaults |
+| `TimescaleDbOptions.cs` | Provider options: `UseLegacyCompressionSql()` for pre-2.18 compatibility |
 | `Abstractions/Dimension.cs` | Range/hash partitioning |
 | `Abstractions/EAggregateFunction.cs` | Aggregate function enum |
+| `Abstractions/ESparseIndexType.cs` | Sparse index kind enum (`Bloom`, `MinMax`) |
 | `Abstractions/ContinuousAggregateFunction.cs` | Typed aggregate-function value |
 
 ## Design Library Key Files
@@ -143,6 +172,7 @@ Quick reference for locating key files in the CmdScale.EntityFrameworkCore.Times
 | `Generators/HypertableCSharpGenerator.cs` | Emits `CreateHypertable`/`AlterHypertable` calls |
 | `Generators/ReorderPolicyCSharpGenerator.cs` | Emits reorder-policy calls |
 | `Generators/RetentionPolicyCSharpGenerator.cs` | Emits retention-policy calls |
+| `Generators/CompressionPolicyCSharpGenerator.cs` | Emits compression-policy calls |
 | `Generators/ContinuousAggregateCSharpGenerator.cs` | Emits continuous-aggregate calls |
 | `Generators/ContinuousAggregatePolicyCSharpGenerator.cs` | Emits CA-policy calls |
 | `Generators/MigrationCallWriter.cs` | Writes a `.Method(arg: value, …)` call |
@@ -162,6 +192,7 @@ Quick reference for locating key files in the CmdScale.EntityFrameworkCore.Times
 | `Generators/AnnotationRenderers/ContinuousAggregatePolicyAnnotationRenderer.cs` | Renders continuous aggregate policy annotations to `WithRefreshPolicy(...)` fluent API or `[ContinuousAggregatePolicy]` attribute |
 | `Generators/AnnotationRenderers/RetentionPolicyAnnotationRenderer.cs` | Renders retention policy annotations to `WithRetentionPolicy(...)` fluent API or `[RetentionPolicy]` attribute; registered after parent renderers (hypertable and continuous aggregate) |
 | `Generators/AnnotationRenderers/ReorderPolicyAnnotationRenderer.cs` | Renders reorder policy annotations to `WithReorderPolicy(...)` fluent API or `[ReorderPolicy]` attribute; registered after the hypertable renderer |
+| `Generators/AnnotationRenderers/CompressionPolicyAnnotationRenderer.cs` | Renders compression policy annotations to `WithCompressionPolicy(...)` fluent API or `[CompressionPolicy]` attribute; registered after the hypertable renderer |
 | `Generators/AnnotationRenderers/PolicyJobRendererHelper.cs` | Shared helpers for emitting policy-job optional arguments (`InitialStart`, `WithScheduleInterval`, etc.) |
 | `Generators/AnnotationRenderers/AnnotationRendererHelper.cs` | Static helpers: `Find`, `GetString`, `SplitColumns`, `Consume`, `ResolvePropertyName`, `TryResolvePropertyName` |
 | `Generators/AnnotationRenderers/NameOfCodeFragment.cs` | Custom `CodeFragment` producing `nameof(X)` or `$"{nameof(X)} DESC"` |
@@ -183,6 +214,10 @@ Quick reference for locating key files in the CmdScale.EntityFrameworkCore.Times
 | `Scaffolding/ContinuousAggregateAnnotationApplier.cs` | Apply continuous aggregate annotations |
 | `Scaffolding/ContinuousAggregatePolicyScaffoldingExtractor.cs` | Query continuous aggregate refresh policies from database |
 | `Scaffolding/ContinuousAggregatePolicyAnnotationApplier.cs` | Apply continuous aggregate policy annotations |
+| `Scaffolding/CompressionPolicyScaffoldingExtractor.cs` | Query compression policies from `timescaledb_information.jobs` joined with `_timescaledb_config.bgw_job` for timezone |
+| `Scaffolding/CompressionPolicyAnnotationApplier.cs` | Apply compression policy annotations; suppresses default schedule intervals to avoid phantom migrations |
+| `Scaffolding/CompressionSettingsScaffoldingHelper.cs` | Shared helper that reads `timescaledb_information.hypertable_columnstore_settings` (2.18+) with fallback to `compression_settings` (pre-2.18); used by both the hypertable and continuous-aggregate scaffolding extractors |
+| `Scaffolding/ScaffoldingExtractorHelper.cs` | Shared infrastructure for extractors: `UsingConnection` (execute-around connection-state management), `ViewExists` (parameterized view lookup), and `TimescaleInternalSchemaExclusion` constant |
 | `Scaffolding/IntervalParsingHelper.cs` | Parses and normalizes PostgreSQL interval strings (e.g. `"01:00:00"` → `"1 hour"`) and integer offsets |
 | `Scaffolding/ViewDefinitionParser.cs` | Parses continuous aggregate view SQL to extract structured configuration for code generation |
 | `build/CmdScale.EntityFrameworkCore.TimescaleDB.Design.targets` | MSBuild integration |
@@ -209,6 +244,7 @@ src/
 ├── Eftdb/                  # Core runtime library (CmdScale.EntityFrameworkCore.TimescaleDB)
 │   ├── Abstractions/       # Domain objects (Dimension, enums)
 │   ├── Configuration/      # Fluent API, attributes, conventions
+│   │   ├── CompressionPolicy/
 │   │   ├── ContinuousAggregate/
 │   │   ├── ContinuousAggregatePolicy/
 │   │   ├── Hypertable/
@@ -218,6 +254,7 @@ src/
 │   ├── MigrationExtensions/ # Typed migrationBuilder.* methods
 │   ├── Internals/          # Core diffing logic
 │   │   └── Features/
+│   │       ├── CompressionPolicies/
 │   │       ├── ContinuousAggregates/
 │   │       ├── ContinuousAggregatePolicies/
 │   │       ├── Hypertables/

.claude/reference/patterns.md

@@ -141,13 +141,11 @@ new ContinuousAggregateFunction("average_price", EAggregateFunction.Avg, "price"
 
 `ToAnnotationValue()` serializes it to the colon-delimited wire format stored on `CreateContinuousAggregateOperation.AggregateFunctions`:
 
-**Format:**
-- Basic: `"alias:Function:sourceColumn"`
-- First/Last: `"alias:Function:sourceColumn:timeColumn"`
+**Format:** `"alias:Function:sourceColumn"` (always three parts — First/Last take no time column in the wire format; the SQL generator supplies the time-bucket column as their second argument: `last("price", "timestamp")`).
 
-**Examples:** `"average_price:Avg:price"`, `"last_price:Last:price:timestamp"`
+**Examples:** `"average_price:Avg:price"`, `"last_price:Last:price"`
 
-**Parsing:** Split by `:` and validate array length (3 or 4 elements).
+**Parsing:** Split by `:` and validate array length (exactly 3 elements; malformed entries are skipped).
 
 **Location:** `Abstractions/ContinuousAggregateFunction.cs`, `ContinuousAggregateModelExtractor.cs`, `Generators/ContinuousAggregateSqlGenerator.cs`
 
@@ -166,12 +164,12 @@ builder.AddAggregateFunction(
     function: EAggregateFunction.Avg
 )
 
-// First/Last with time column
+// First/Last — the time argument is always the continuous aggregate's
+// time-bucket column; there is no timeColumn parameter
 builder.AddAggregateFunction(
     aggregateProperty: x => x.LastPrice,
     sourceProperty: x => x.Price,
-    function: EAggregateFunction.Last,
-    timeColumn: x => x.Timestamp
+    function: EAggregateFunction.Last
 )
 
 // Group by columns

.claude/rules/testing.md

@@ -34,14 +34,21 @@ public async Task Should_Detect_New_Hypertable() { /* ... */ }
 
 - `InitialContext` — represents the "before" model state
 - `ModifiedContext` — represents the "after" model state with changes
-- Comment changed values with `// <-- Changed from <original value>`
 
 ## Test Structure (AAA)
 
 1. **Arrange:** Create initial migration, apply to database
 2. **Act:** Diff `InitialContext` vs `ModifiedContext` to produce operations
 3. **Assert:** Verify generated operations match expected changes
 
+## Comments
+
+Comments are for navigation, never explanation. If a test needs an explanatory comment, it is not a good test — make the test plainer instead.
+
+- Allowed: XML doc `<summary>` on test classes, bare AAA markers (`// Arrange`, `// Act`, `// Assert`, bare combinations like `// Act & Assert`), thin `// ── Title ──` dividers, `#region`/`#endregion`
+- Forbidden: explanatory text on or under AAA markers, comments in test-data setup (model classes, `OnModelCreating`), trailing explanations after code lines (including `// <-- Changed from <value>` delta markers), `//`-comment banners above test classes or methods
+- Shared test infrastructure (base classes, fixtures) may carry a comment only for a genuine non-obvious constraint the code cannot express
+
 ## Integration Tests
 
 - Use `Testcontainers` for real TimescaleDB instances (requires Docker)

README.md

@@ -36,11 +36,17 @@ Seamlessly define and manage **TimescaleDB hypertables** using standard EF Core
 - **Chunk Skipping**: Enable chunk skipping to improve query performance on specific columns.
 - **Compression Segment By**: Define columns to group compressed data by, allowing efficient access to specific segments without decompressing entire chunks.
 - **Compression Order By**: Specify the sort order within compressed segments, with support for ascending/descending direction and NULLS FIRST/LAST positioning.
+- **Sparse Indexes**: Configure bloom-filter or min/max sparse indexes on the hypertable's columnstore via a type-safe fluent API (`.WithSparseIndex(s => s.Bloom(x => x.Col), s => s.MinMax(x => x.Col))`) or the `[SparseIndex]` attribute (allowMultiple). Use `.WithoutAutoSparseIndexes()` or `DisableAutoSparseIndexes = true` on `[Hypertable]` to suppress auto-generated indexes.
+- **Compress Chunk Time Interval**: Set `compress_chunk_time_interval` via `.WithCompressChunkTimeInterval("7 days")` or the `CompressChunkTimeInterval` property on `[Hypertable]` to control the minimum age of a chunk before the compression policy will compress it.
 
 ### Reorder Policies
 
 Take full control over how your hypertable data is organized on disk with **TimescaleDB's** reorder policies. By defining a reorder policy, you can automatically re-sort chunks of data by a specified index, significantly improving the performance of queries that scan large time ranges or specific index values.
 
+### Compression Policies
+
+Automate when TimescaleDB compresses chunks on a hypertable.
+
 ### Retention Policies
 
 Automatically drop old chunks from hypertables and continuous aggregates so storage stays bounded as your time-series data grows.
@@ -54,6 +60,8 @@ Create and manage **TimescaleDB continuous aggregates** — automatically refres
 - **Group By Columns**: Add additional grouping dimensions beyond time.
 - **Filtering**: Apply WHERE clauses to filter source data.
 - **Refresh Policies**: Configure automatic refresh with customizable time windows, schedule intervals, and batching options.
+- **Compression**: Enable compression on the continuous aggregate's materialized view, with segment-by and order-by column control, using the same fluent API as hypertables (`.WithCompression()`, `.WithCompressionSegmentBy()`, `.WithCompressionOrderBy()`).
+- **Compression Policies**: Schedule automatic compression of the continuous aggregate's chunks via `.WithCompressionPolicy()` or `[CompressionPolicy]`, independent of the hypertable's own compression policy.
 
 ### Query Functions