Add observable and native change feed support to document stores by aritchie · Pull Request #4 · shinyorg/DocumentDb

aritchie · 2026-05-31T20:11:48Z

Description of Change

This PR adds comprehensive change notification capabilities to the document store system:

Observable Document Store (IObservableDocumentStore): In-process change notifications for documents modified through a store instance. Changes are emitted for Insert, Update, Remove, and Clear operations, with transaction support (changes are buffered until commit).
Native Change Feed (IChangeFeedDocumentStore): Server-side change notifications that observe all writers, not just the current store instance. Implemented for:
- PostgreSQL: Uses LISTEN/NOTIFY with row-level triggers
- SQL Server: Uses Change Tracking with optional Query Notifications for low-latency polling
- Azure Cosmos DB: Uses the native Change Feed API
Core Types:
- DocumentChange<T>: Represents a single change with type, Id, and optional document body
- DocumentChangeType: Enum for Inserted, Updated, Removed, Cleared
- ChangeBroadcaster: Internal fan-out mechanism for in-process notifications
- ChangeFeedSubscription: Reusable wrapper for background change feed workers
Extensions: ObservableDocumentStoreExtensions provides convenience helpers for consuming change streams without System.Reactive dependency.
Implementation Updates:
- DocumentStore (SQL-based): Publishes changes on all CRUD operations; buffers during transactions
- LiteDbDocumentStore: Publishes in-process changes with transaction buffering
- CosmosDbDocumentStore: Implements native change feed via Cosmos Change Feed API
- Database providers: Added SupportsChangeFeed property and SubscribeChangesAsync method
Comprehensive Tests:
- ObservableTests: 15 tests covering in-process notifications, filtering, transactions, and unsubscribe behavior
- ChangeFeedTests: Integration tests for native change feeds (external writer detection, disposal)

Issues Resolved

None

API Changes

New Public Types:

IObservableDocumentStore interface
IChangeFeedDocumentStore interface
DocumentChange<T> class
DocumentChangeType enum
ObservableDocumentStoreExtensions static class
SqlServerChangeFeedOptions class

New Methods:

IObservableDocumentStore.WhenChanged<T>() → IObservable<DocumentChange<T>>
IChangeFeedDocumentStore.SubscribeChanges<T>(...) → Task<IAsyncDisposable>
IDatabaseProvider.SupportsChangeFeed property
IDatabaseProvider.SubscribeChangesAsync(...) method

Modified Implementations:

DocumentStore now implements IObservableDocumentStore and IChangeFeedDocumentStore
LiteDbDocumentStore now implements IObservableDocumentStore
CosmosDbDocumentStore now implements IChangeFeedDocumentStore

Behavioral Changes

All CRUD operations (Insert, Update, Remove, Clear, SetProperty, RemoveProperty, BatchInsert) now emit change notifications to in-process observers
Changes within transactions are buffered and only emitted after successful commit; rollbacks discard pending changes
SQL Server and PostgreSQL providers now support native change feeds that observe all writers to the database
Cosmos DB now exposes its native change feed capability

Testing Procedure

Run the new ObservableTests suite (15 tests) to verify in-process change notifications work correctly across all providers
Run the new ChangeFeedTests suite (3 integration tests) on PostgreSQL and SQL Server to verify native change feed functionality
Verify existing test suites pass (no breaking changes to existing APIs)
Manual verification: Subscribe to changes via store.WhenChanged<T>() and confirm notifications are delivered on CRUD operations

PR Checklist

Changes adhere to coding standard
Comprehensive test coverage added (18 new tests)
All store implementations updated consistently

https://claude.ai/code/session_01QXtq5v6bTDLgncKQowM6YW

Introduces an opt-in IObservableDocumentStore capability that emits in-process insert/update/remove/clear notifications so consumers can drive reactive UI from their own writes. - Core abstraction: IObservableDocumentStore, DocumentChange<T>, DocumentChangeType, plus a lightweight ChangeBroadcaster (no System.Reactive dependency) and Subscribe/Where/WhenChanged/ WhenDocumentChanged helper extensions. - Implemented on the relational DocumentStore (SQLite, SQLCipher, MySQL, SQL Server, PostgreSQL) and LiteDB. Changes made inside RunInTransaction are buffered and flushed only on commit; rollback discards them. Notifications are published outside the store lock so observer callbacks can safely re-enter the store. - Tests covering insert/update/remove/clear/set-property/batch, subscribe/unsubscribe semantics, per-id and per-type filtering, and transaction commit/rollback for both SQLite and LiteDB. https://claude.ai/code/session_01QXtq5v6bTDLgncKQowM6YW

Adds external-writer-aware, server-side change notifications backed by each database's own change mechanism, alongside the existing in-process IObservableDocumentStore. API: SubscribeChanges<T>(handler, ct) returns an IAsyncDisposable; dispose to stop. Backed by: - PostgreSQL: LISTEN/NOTIFY via idempotent row-level triggers (true push, full insert/update/delete fidelity). Verified end-to-end against a live PostgreSQL, including cross-connection (external writer) and dispose-stops-delivery. - SQL Server: Change Tracking polling with optional SqlDependency query notifications for low-latency wake-ups (configurable via SqlServerChangeFeedOptions; falls back to polling if notifications can't be armed). - CosmosDB: latest-version Change Feed scoped to the type's partition (delivers created/updated documents with body; no deletes). Core wiring lives in IDatabaseProvider (SupportsChangeFeed + SubscribeChangesAsync) and DocumentStore, which maps provider-level RawDocumentChange records to typed DocumentChange<T> and deserializes. Providers without a proper external-change mechanism (SQLite, LiteDB, IndexedDB, MySQL) throw NotSupportedException. Tests: a ChangeFeedTestsBase (insert/update/remove, external writer, dispose) wired for PostgreSQL (runs under Testcontainers), plus not-supported assertions for SQLite/LiteDB. https://claude.ai/code/session_01QXtq5v6bTDLgncKQowM6YW

Copilot

Pull request overview

This PR adds in-process observable change notifications and native provider-backed change feeds to DocumentDb, extending the store APIs and provider implementations for reactive and external-writer scenarios.

Changes:

Adds public change notification APIs and core change types.
Publishes observable changes from relational and LiteDB CRUD/transaction paths.
Adds native change-feed implementations for PostgreSQL, SQL Server, and Cosmos DB with new integration tests and README documentation.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
`src/Shiny.DocumentDb/DocumentChange.cs`	Adds typed change payload model.
`src/Shiny.DocumentDb/DocumentChangeType.cs`	Adds change kind enum.
`src/Shiny.DocumentDb/IObservableDocumentStore.cs`	Adds in-process observable store capability.
`src/Shiny.DocumentDb/IChangeFeedDocumentStore.cs`	Adds native change-feed store capability.
`src/Shiny.DocumentDb/ObservableDocumentStoreExtensions.cs`	Adds convenience observable/feed helpers.
`src/Shiny.DocumentDb/IDatabaseProvider.cs`	Extends providers with optional change-feed support.
`src/Shiny.DocumentDb/Internal/ChangeBroadcaster.cs`	Adds in-process observer fan-out.
`src/Shiny.DocumentDb/Internal/ChangeFeedSubscription.cs`	Adds reusable async subscription worker wrapper.
`src/Shiny.DocumentDb/Internal/RawDocumentChange.cs`	Adds provider-level raw change record.
`src/Shiny.DocumentDb/DocumentStore.cs`	Wires observable notifications and native feed subscription into relational stores.
`src/Shiny.DocumentDb.LiteDb/LiteDbDocumentStore.cs`	Adds observable notifications and transaction buffering for LiteDB.
`src/Shiny.DocumentDb.PostgreSql/PostgreSqlDatabaseProvider.cs`	Adds PostgreSQL LISTEN/NOTIFY trigger-backed feed.
`src/Shiny.DocumentDb.SqlServer/SqlServerDatabaseProvider.cs`	Adds SQL Server Change Tracking polling feed.
`src/Shiny.DocumentDb.SqlServer/SqlServerChangeFeedOptions.cs`	Adds SQL Server feed configuration options.
`src/Shiny.DocumentDb.CosmosDb/CosmosDbDocumentStore.cs`	Adds Cosmos DB Change Feed subscription support.
`tests/Shiny.DocumentDb.Tests/ObservableTests.cs`	Adds observable behavior tests.
`tests/Shiny.DocumentDb.Tests/ChangeFeedTests.cs`	Adds native change-feed integration tests.
`tests/Shiny.DocumentDb.Tests/SqliteProviderTests.cs`	Wires observable tests for SQLite.
`tests/Shiny.DocumentDb.Tests/LiteDbProviderTests.cs`	Wires observable tests for LiteDB.
`tests/Shiny.DocumentDb.Tests/PostgreSqlProviderTests.cs`	Wires change-feed tests for PostgreSQL.
`readme.md`	Documents observable stores and native change feeds.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+        return await this.provider.SubscribeChangesAsync(
+            tableName,
+            typeName,
+            async (raw, ct) =>


+        // Provision the trigger/function up front so connection or permission errors surface here.
+        await this.EnsureChangeFeedTriggerAsync(tableName, cancellationToken).ConfigureAwait(false);
+        var channel = ChannelName(tableName);
+        return new ChangeFeedSubscription(cancellationToken, token => this.RunListenerAsync(channel, typeName, onChange, token));


+        return new ChangeFeedSubscription(cancellationToken,
+            token => this.RunChangeFeedAsync(container, feedRange, typeInfo, onChange, token));


claude added 2 commits May 31, 2026 19:36

Copilot AI review requested due to automatic review settings May 31, 2026 20:11

Copilot started reviewing on behalf of aritchie May 31, 2026 20:11 View session

Copilot AI reviewed May 31, 2026

View reviewed changes

aritchie deleted the branch main June 1, 2026 03:08

aritchie closed this Jun 1, 2026

aritchie deleted the claude/query-monitor-providers-h0Dm0 branch June 1, 2026 03:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add observable and native change feed support to document stores#4

Add observable and native change feed support to document stores#4
aritchie wants to merge 2 commits into
mainfrom
claude/query-monitor-providers-h0Dm0

aritchie commented May 31, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		return new ChangeFeedSubscription(cancellationToken,
		token => this.RunChangeFeedAsync(container, feedRange, typeInfo, onChange, token));

Conversation

aritchie commented May 31, 2026

Description of Change

Issues Resolved

API Changes

Behavioral Changes

Testing Procedure

PR Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants