From 9ba076ab0e316d7d4c768d8b80e9d2455729c32c Mon Sep 17 00:00:00 2001 From: Tigran Babloyan Date: Thu, 16 Jul 2026 11:25:28 +0400 Subject: [PATCH] feat(AF-457): proxy result caching, batch writes & multi-replica LB - Opt-in Redis-backed SELECT result cache per datasource (TTL), keyed over the RLS-rewritten SQL + binds + mask/restriction scope, indexed per referenced table and invalidated on any proxied write (V115 adds result_cache_enabled / result_cache_ttl_seconds). - BEGIN..COMMIT envelopes fold homogeneous single-row INSERTs into one PreparedStatement executeLargeBatch() (PG strings bound as Types.OTHER to preserve literal type inference). - Multi-endpoint read replicas: datasource_read_replicas child table replaces the flat read_replica_* columns (V115 migrates + drops, breaking API change: read_replicas array), round-robin selection, per-node circuit breaker + async isValid prober, replica health on the admin dashboard. - Settings UI: dynamic replica endpoint list + Performance card; i18n x7; e2e spec rewritten; docs/website/README/CLAUDE.md synced. Closes #457 --- CLAUDE.md | 7 + README.md | 2 +- .../core/api/CreateDatasourceCommand.java | 37 ++- .../api/DatasourceConnectionDescriptor.java | 75 +++++- .../accessflow/core/api/DatasourceView.java | 38 ++- .../core/api/QueryExecutionRequest.java | 23 +- .../core/api/ReadReplicaEndpoint.java | 23 ++ .../core/api/ReplicaEndpointInput.java | 18 ++ .../core/api/TestReplicaCommand.java | 16 +- .../core/api/UpdateDatasourceCommand.java | 47 +++- .../DatasourceCacheConfigChangedEvent.java | 11 + .../internal/DatasourceAdminServiceImpl.java | 145 +++++++--- .../internal/DatasourceDescriptorMapper.java | 13 +- .../persistence/entity/DatasourceEntity.java | 19 +- .../entity/DatasourceReadReplicaEntity.java | 47 ++++ .../repo/DatasourceRepository.java | 4 +- .../internal/ErasureExecutionService.java | 5 +- .../RetentionPolicyExecutionService.java | 7 +- .../api/DatasourceConnectionPoolManager.java | 39 ++- .../proxy/api/DatasourceHealthSnapshot.java | 22 +- .../proxy/api/ReplicaEndpointHealth.java | 17 ++ .../proxy/api/ReplicaEndpointRef.java | 11 + .../proxy/internal/BatchInsertPlanner.java | 162 ++++++++++++ .../DatasourcePoolEvictionListener.java | 3 + .../proxy/internal/DatasourcePoolFactory.java | 23 +- ...efaultDatasourceConnectionPoolManager.java | 134 ++++++++-- .../DefaultDatasourceHealthService.java | 25 +- .../proxy/internal/DefaultQueryExecutor.java | 139 ++++++++-- .../proxy/internal/ProxyCacheProperties.java | 27 ++ .../proxy/internal/ProxyConfiguration.java | 3 +- .../proxy/internal/ProxyPoolProperties.java | 10 +- .../internal/ProxyReplicaProperties.java | 27 ++ .../proxy/internal/ReplicaHealthProber.java | 82 ++++++ .../proxy/internal/ReplicaHealthRegistry.java | 76 ++++++ .../internal/ResultCacheEvictionListener.java | 35 +++ .../internal/RoutingDataSourceResolver.java | 92 +++++-- .../proxy/internal/SelectResultCache.java | Bin 0 -> 8836 bytes .../internal/GroupExecutionService.java | 2 +- .../internal/web/DatasourceController.java | 30 ++- .../web/model/CreateDatasourceRequest.java | 15 +- .../web/model/DatasourceHealthResponse.java | 22 +- .../web/model/DatasourceResponse.java | 22 +- .../web/model/ReadReplicaRequest.java | 23 ++ .../web/model/TestReplicaRequest.java | 10 +- .../web/model/UpdateDatasourceRequest.java | 19 +- .../DefaultQueryLifecycleService.java | 2 +- backend/src/main/resources/application.yml | 14 + .../V115__multi_replica_and_result_cache.sql | 28 ++ .../main/resources/i18n/messages.properties | 2 + .../resources/i18n/messages_de.properties | 2 + .../resources/i18n/messages_es.properties | 2 + .../resources/i18n/messages_fr.properties | 2 + .../resources/i18n/messages_hy.properties | 2 + .../resources/i18n/messages_ru.properties | 2 + .../resources/i18n/messages_zh_CN.properties | 2 + .../DatasourceAdminServiceImplTest.java | 120 ++++++++- .../internal/BatchInsertPlannerTest.java | 173 ++++++++++++ .../internal/DatasourcePoolFactoryTest.java | 9 +- ...ltDatasourceConnectionPoolManagerTest.java | 103 +++++++- .../DefaultDatasourceHealthServiceTest.java | 31 ++- ...tQueryExecutorPostgresIntegrationTest.java | 25 ++ ...eryExecutorReadReplicaIntegrationTest.java | 83 +++++- .../internal/DefaultQueryExecutorTest.java | 198 ++++++++++++-- .../internal/ProxyCachePropertiesTest.java | 26 ++ .../internal/ProxyPoolPropertiesTest.java | 36 ++- .../internal/ProxyReplicaPropertiesTest.java | 27 ++ .../internal/ReplicaHealthProberTest.java | 104 ++++++++ .../internal/ReplicaHealthRegistryTest.java | 93 +++++++ .../ResultCacheEvictionListenerTest.java | 44 ++++ .../RoutingDataSourceResolverTest.java | 144 ++++++++-- .../SelectResultCacheIntegrationTest.java | 139 ++++++++++ .../proxy/internal/SelectResultCacheTest.java | 232 ++++++++++++++++ .../DatasourceControllerIntegrationTest.java | 35 ++- docs/03-data-model.md | 23 +- docs/04-api-spec.md | 38 ++- docs/05-backend.md | 41 ++- docs/07-security.md | 2 +- docs/09-deployment.md | 11 +- docs/12-roadmap.md | 1 + e2e/tests/datasource-read-replica.spec.ts | 101 +++++-- frontend/src/api/datasources.ts | 2 + .../editor/ReviewPlanPreview.test.tsx | 5 +- .../editor/useQueryAuthoring.test.tsx | 5 +- frontend/src/locales/de.json | 19 +- frontend/src/locales/en.json | 19 +- frontend/src/locales/es.json | 19 +- frontend/src/locales/fr.json | 19 +- frontend/src/locales/hy.json | 19 +- frontend/src/locales/ru.json | 19 +- frontend/src/locales/zh-CN.json | 19 +- frontend/src/mocks/data.ts | 16 +- .../pages/admin/DatasourceHealthPage.test.tsx | 1 + .../src/pages/admin/DatasourceHealthPage.tsx | 24 ++ .../datasources/DatasourceSettingsPage.tsx | 248 +++++++++++++----- .../DatasourceCreateWizardPage.test.tsx | 5 +- .../__tests__/DatasourceSettingsPage.test.tsx | 133 +++++++--- .../src/pages/editor/QueryEditorPage.test.tsx | 5 +- frontend/src/types/api.ts | 46 +++- .../internal/client/datasource.go | 2 +- website/docs/index.html | 59 +++-- website/index.html | 3 +- 101 files changed, 3657 insertions(+), 534 deletions(-) create mode 100644 backend/src/main/java/com/bablsoft/accessflow/core/api/ReadReplicaEndpoint.java create mode 100644 backend/src/main/java/com/bablsoft/accessflow/core/api/ReplicaEndpointInput.java create mode 100644 backend/src/main/java/com/bablsoft/accessflow/core/events/DatasourceCacheConfigChangedEvent.java create mode 100644 backend/src/main/java/com/bablsoft/accessflow/core/internal/persistence/entity/DatasourceReadReplicaEntity.java create mode 100644 backend/src/main/java/com/bablsoft/accessflow/proxy/api/ReplicaEndpointHealth.java create mode 100644 backend/src/main/java/com/bablsoft/accessflow/proxy/api/ReplicaEndpointRef.java create mode 100644 backend/src/main/java/com/bablsoft/accessflow/proxy/internal/BatchInsertPlanner.java create mode 100644 backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ProxyCacheProperties.java create mode 100644 backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ProxyReplicaProperties.java create mode 100644 backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ReplicaHealthProber.java create mode 100644 backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ReplicaHealthRegistry.java create mode 100644 backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ResultCacheEvictionListener.java create mode 100644 backend/src/main/java/com/bablsoft/accessflow/proxy/internal/SelectResultCache.java create mode 100644 backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/ReadReplicaRequest.java create mode 100644 backend/src/main/resources/db/migration/V115__multi_replica_and_result_cache.sql create mode 100644 backend/src/test/java/com/bablsoft/accessflow/proxy/internal/BatchInsertPlannerTest.java create mode 100644 backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ProxyCachePropertiesTest.java create mode 100644 backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ProxyReplicaPropertiesTest.java create mode 100644 backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ReplicaHealthProberTest.java create mode 100644 backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ReplicaHealthRegistryTest.java create mode 100644 backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ResultCacheEvictionListenerTest.java create mode 100644 backend/src/test/java/com/bablsoft/accessflow/proxy/internal/SelectResultCacheIntegrationTest.java create mode 100644 backend/src/test/java/com/bablsoft/accessflow/proxy/internal/SelectResultCacheTest.java diff --git a/CLAUDE.md b/CLAUDE.md index 7968298f..df3537c0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -287,6 +287,13 @@ com.bablsoft.accessflow/ | `ACCESSFLOW_PROXY_EXECUTION_MAX_ROWS` | Hard cap on rows returned by a single query (default `10000`). | | `ACCESSFLOW_PROXY_EXECUTION_STATEMENT_TIMEOUT` | Statement-level timeout applied to customer-DB JDBC statements (default `30s`). | | `ACCESSFLOW_PROXY_EXECUTION_DEFAULT_FETCH_SIZE` | Default JDBC fetch size (default `1000`). | +| `ACCESSFLOW_PROXY_EXECUTION_INSERT_BATCH_CHUNK_SIZE` | Rows per JDBC `executeBatch()` flush when a `BEGIN…COMMIT` envelope's homogeneous single-row INSERTs are batched (AF-457, default `1000`). | +| `ACCESSFLOW_PROXY_CACHE_ENABLED` | Deployment-wide kill-switch for the opt-in SELECT result cache (AF-457, default `true`). The real gate is each datasource's `result_cache_enabled` flag; entries are Redis-backed and invalidated on any proxied write to a referenced table. | +| `ACCESSFLOW_PROXY_CACHE_DEFAULT_TTL` | ISO-8601 duration. Result-cache TTL used when a datasource opts in without its own `result_cache_ttl_seconds` (default `PT60S`). | +| `ACCESSFLOW_PROXY_CACHE_MAX_ENTRY_BYTES` | Serialized-size cap per cached SELECT result; larger results are never cached (default `1000000`). | +| `ACCESSFLOW_PROXY_REPLICA_PROBE_INTERVAL` | ISO-8601 duration. Cadence of the per-node read-replica health prober (AF-457, default `PT30S`). Deliberately per-node (not ShedLock'd) — pools and breaker state are per JVM. | +| `ACCESSFLOW_PROXY_REPLICA_PROBE_TIMEOUT` | ISO-8601 duration. JDBC `isValid` timeout per replica endpoint probe (default `PT5S`). | +| `ACCESSFLOW_PROXY_REPLICA_COOLDOWN` | ISO-8601 duration. How long a failed replica endpoint sits out of the read rotation before a half-open retry (default `PT30S`). | | `ACCESSFLOW_PROXY_ENGINES__` | Generic per-engine plugin tuning (AF-418): binds `accessflow.proxy.engines..*`, passed verbatim into the engine's `QueryEngineContext` config map. Key names are each engine's contract (`_`/`.` normalize to `-`); generic env vars override `application.yml` defaults. See `docs/15-engine-sdk.md`. | | `ACCESSFLOW_PROXY_MONGO_CONNECT_TIMEOUT` | ISO-8601 duration. Connect timeout for the per-MongoDB-datasource `MongoClient` (default `PT10S`). MongoDB-only; the relational pools use the `ACCESSFLOW_PROXY_CONNECTION_TIMEOUT` HikariCP knob. Legacy alias for `accessflow.proxy.engines.mongodb.connect-timeout` — still fully supported. | | `ACCESSFLOW_PROXY_MONGO_SERVER_SELECTION_TIMEOUT` | ISO-8601 duration. MongoDB server-selection timeout (default `PT10S`). Legacy alias for `accessflow.proxy.engines.mongodb.server-selection-timeout`. | diff --git a/README.md b/README.md index 01fded69..825c7305 100644 --- a/README.md +++ b/README.md @@ -64,7 +64,7 @@ A glance at the day-to-day flows engineers and approvers actually use. ## Features -- **Proxy-first execution** — no user ever holds production credentials; the proxy holds them encrypted and opens connections only after approval. Single SQL statements run with autocommit; multi-statement INSERT/UPDATE/DELETE batches wrapped in `BEGIN; … COMMIT;` execute atomically inside one JDBC transaction (mixed SELECT/DML batches are rejected at parse time). Optional **read-replica routing**: attach a replica JDBC URL to a datasource and SELECT traffic is served from there, with automatic primary fall-back and an audit row on replica failure. +- **Proxy-first execution** — no user ever holds production credentials; the proxy holds them encrypted and opens connections only after approval. Single SQL statements run with autocommit; multi-statement INSERT/UPDATE/DELETE batches wrapped in `BEGIN; … COMMIT;` execute atomically inside one JDBC transaction (mixed SELECT/DML batches are rejected at parse time), with homogeneous INSERT runs collapsed into JDBC `executeBatch()` for bulk-load throughput. Optional **multi-replica read load balancing**: attach any number of replica endpoints to a datasource and SELECT traffic round-robins across the healthy ones — per-node health checks with circuit-breaker failover skip downed replicas, and only full replica-set exhaustion falls back to the primary (with an audit row). Optional **SELECT result caching**: opt a datasource into a Redis-backed result cache (per-datasource TTL) keyed over the security-rewritten query — masking and row-level security still apply — and invalidated on any proxied write to a referenced table. - **Configurable review workflows** — per-datasource review plans, multi-stage sequential approval chains, optional auto-approve for reads, approval timeouts with auto-reject. Reviewers can be scoped **per-datasource** (directly or via groups) so different teams see only the queues that belong to them. - **Policy-as-code routing** — ordered, attribute-based routing policies decide a query's path after AI analysis and before reviewers see it: **auto-approve**, **auto-reject**, **require N approvals**, or **escalate**. Conditions match on query type, referenced tables (glob), AI risk level / score, requester role or group, time-of-day / day-of-week, WHERE / LIMIT presence, the transactional flag, and the submission **client context** — source IP / CIDR, user-agent, time-since-last-approval, and CI/CD origin (API key or `X-AccessFlow-CI` header) — combined with AND / OR / NOT. Client-context conditions **fail closed** (missing context never auto-approves), so an off-network or stale-approval query escalates to stricter review instead. First match by priority wins; on no match the query falls through to the datasource's review plan. Every automated decision is recorded in the audit log. - **Just-in-time (JIT) access requests** — users self-request temporary, scoped access to a datasource (read/write/DDL, optional schema/table scope) or an API connection (read/write, optional operation allow-list) for an ISO-8601 duration. Requests flow through the same approval engine, a time-boxed permission is granted on approval, and a clustered scheduler auto-revokes it on expiry (admins can also revoke early). A grant can opt into **query pre-approval**: while it is active, queries it covers skip human review and are auto-approved with the grant recorded as the approval provenance — routing policies, high-risk AI verdicts, and behavioural anomalies still override. diff --git a/backend/src/main/java/com/bablsoft/accessflow/core/api/CreateDatasourceCommand.java b/backend/src/main/java/com/bablsoft/accessflow/core/api/CreateDatasourceCommand.java index 0ca4ba5d..059e9454 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/core/api/CreateDatasourceCommand.java +++ b/backend/src/main/java/com/bablsoft/accessflow/core/api/CreateDatasourceCommand.java @@ -1,5 +1,6 @@ package com.bablsoft.accessflow.core.api; +import java.util.List; import java.util.UUID; public record CreateDatasourceCommand( @@ -23,12 +24,32 @@ public record CreateDatasourceCommand( UUID customDriverId, String connectorId, String jdbcUrlOverride, - String readReplicaJdbcUrl, - String readReplicaUsername, - String readReplicaPassword, + List readReplicas, String localDatacenter, - String apiKey + String apiKey, + Boolean resultCacheEnabled, + Integer resultCacheTtlSeconds ) { + /** + * Backward-compatible constructor taking the pre-AF-457 single-replica triple; a non-blank + * {@code readReplicaJdbcUrl} maps to a one-endpoint replica list, result caching defaults off. + */ + public CreateDatasourceCommand( + UUID organizationId, String name, DbType dbType, String host, Integer port, + String databaseName, String username, String password, SslMode sslMode, + Integer connectionPoolSize, Integer maxRowsPerQuery, Boolean requireReviewReads, + Boolean requireReviewWrites, UUID reviewPlanId, Boolean aiAnalysisEnabled, + UUID aiConfigId, Boolean textToSqlEnabled, UUID customDriverId, String connectorId, + String jdbcUrlOverride, String readReplicaJdbcUrl, String readReplicaUsername, + String readReplicaPassword, String localDatacenter, String apiKey) { + this(organizationId, name, dbType, host, port, databaseName, username, password, sslMode, + connectionPoolSize, maxRowsPerQuery, requireReviewReads, requireReviewWrites, + reviewPlanId, aiAnalysisEnabled, aiConfigId, textToSqlEnabled, customDriverId, + connectorId, jdbcUrlOverride, + legacyReplicaList(readReplicaJdbcUrl, readReplicaUsername, readReplicaPassword), + localDatacenter, apiKey, null, null); + } + /** Backward-compatible constructor for the dialects with no {@code apiKey} (everything but the search engines). */ public CreateDatasourceCommand( UUID organizationId, String name, DbType dbType, String host, Integer port, @@ -60,4 +81,12 @@ public CreateDatasourceCommand( connectorId, jdbcUrlOverride, readReplicaJdbcUrl, readReplicaUsername, readReplicaPassword, null, null); } + + private static List legacyReplicaList(String jdbcUrl, String username, + String password) { + if (jdbcUrl == null || jdbcUrl.isBlank()) { + return List.of(); + } + return List.of(new ReplicaEndpointInput(null, jdbcUrl, username, password)); + } } diff --git a/backend/src/main/java/com/bablsoft/accessflow/core/api/DatasourceConnectionDescriptor.java b/backend/src/main/java/com/bablsoft/accessflow/core/api/DatasourceConnectionDescriptor.java index d473bac5..6de1bf2d 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/core/api/DatasourceConnectionDescriptor.java +++ b/backend/src/main/java/com/bablsoft/accessflow/core/api/DatasourceConnectionDescriptor.java @@ -1,5 +1,7 @@ package com.bablsoft.accessflow.core.api; +import java.nio.charset.StandardCharsets; +import java.util.List; import java.util.UUID; /** @@ -12,10 +14,13 @@ * datasources at least {@code customDriverId} is set; {@code jdbcUrlOverride} is set when the * datasource uses {@link DbType#CUSTOM} (free-form URL). * - *

The {@code readReplica*} fields are optional. When {@link #hasReadReplica()} is {@code true}, - * the proxy engine routes {@link QueryType#SELECT} queries to a sibling HikariCP pool built from - * the replica JDBC URL and credentials; non-SELECT queries always hit the primary regardless. - * Reuses the same driver class as the primary. + *

{@code readReplicas} lists the datasource's read-replica endpoints (AF-457). When + * {@link #hasReadReplica()} is {@code true}, the proxy engine load-balances + * {@link QueryType#SELECT} queries round-robin across the healthy replica pools; non-SELECT + * queries always hit the primary regardless. Replicas reuse the same driver class as the primary. + * + *

{@code resultCacheEnabled} / {@code resultCacheTtlSeconds} are the opt-in SELECT result-cache + * settings (AF-457). A {@code null} TTL falls back to the deployment-wide default. * *

{@code localDatacenter} is the Cassandra/ScyllaDB driver's load-balancing datacenter name * (the {@code withLocalDatacenter(...)} value). It is {@code null} for every other dialect. @@ -43,12 +48,54 @@ public record DatasourceConnectionDescriptor( UUID customDriverId, String connectorId, String jdbcUrlOverride, - String readReplicaJdbcUrl, - String readReplicaUsername, - String readReplicaPasswordEncrypted, + List readReplicas, boolean active, String localDatacenter, - String apiKeyEncrypted) { + String apiKeyEncrypted, + boolean resultCacheEnabled, + Integer resultCacheTtlSeconds) { + + public DatasourceConnectionDescriptor { + readReplicas = readReplicas == null ? List.of() : List.copyOf(readReplicas); + } + + /** + * Backward-compatible constructor taking the pre-AF-457 single-replica triple. A non-blank + * {@code readReplicaJdbcUrl} maps to a one-endpoint replica list (deterministic endpoint id + * derived from the URL); result caching defaults to off. Kept so engine plugins and tests + * compiled against the old shape keep working unchanged. + */ + public DatasourceConnectionDescriptor( + UUID id, + UUID organizationId, + DbType dbType, + String host, + Integer port, + String databaseName, + String username, + String passwordEncrypted, + SslMode sslMode, + int connectionPoolSize, + int maxRowsPerQuery, + boolean aiAnalysisEnabled, + UUID aiConfigId, + boolean textToSqlEnabled, + UUID customDriverId, + String connectorId, + String jdbcUrlOverride, + String readReplicaJdbcUrl, + String readReplicaUsername, + String readReplicaPasswordEncrypted, + boolean active, + String localDatacenter, + String apiKeyEncrypted) { + this(id, organizationId, dbType, host, port, databaseName, username, passwordEncrypted, + sslMode, connectionPoolSize, maxRowsPerQuery, aiAnalysisEnabled, aiConfigId, + textToSqlEnabled, customDriverId, connectorId, jdbcUrlOverride, + legacyReplicaList(readReplicaJdbcUrl, readReplicaUsername, + readReplicaPasswordEncrypted), + active, localDatacenter, apiKeyEncrypted, false, null); + } /** * Backward-compatible constructor for the dialects that have no {@code apiKeyEncrypted} (every @@ -117,6 +164,16 @@ public DatasourceConnectionDescriptor( } public boolean hasReadReplica() { - return readReplicaJdbcUrl != null && !readReplicaJdbcUrl.isBlank(); + return !readReplicas.isEmpty(); + } + + private static List legacyReplicaList(String jdbcUrl, String username, + String passwordEncrypted) { + if (jdbcUrl == null || jdbcUrl.isBlank()) { + return List.of(); + } + return List.of(new ReadReplicaEndpoint( + UUID.nameUUIDFromBytes(jdbcUrl.getBytes(StandardCharsets.UTF_8)), + jdbcUrl, username, passwordEncrypted)); } } diff --git a/backend/src/main/java/com/bablsoft/accessflow/core/api/DatasourceView.java b/backend/src/main/java/com/bablsoft/accessflow/core/api/DatasourceView.java index 7182a0f8..0c308d45 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/core/api/DatasourceView.java +++ b/backend/src/main/java/com/bablsoft/accessflow/core/api/DatasourceView.java @@ -1,6 +1,7 @@ package com.bablsoft.accessflow.core.api; import java.time.Instant; +import java.util.List; import java.util.UUID; public record DatasourceView( @@ -24,12 +25,43 @@ public record DatasourceView( UUID customDriverId, String connectorId, String jdbcUrlOverride, - String readReplicaJdbcUrl, - String readReplicaUsername, + List readReplicas, boolean active, Instant createdAt, - String localDatacenter + String localDatacenter, + boolean resultCacheEnabled, + Integer resultCacheTtlSeconds ) { + /** One read-replica endpoint as exposed to admins — never carries the password. */ + public record ReadReplicaView(UUID id, String jdbcUrl, String username) { + } + + public DatasourceView { + readReplicas = readReplicas == null ? List.of() : List.copyOf(readReplicas); + } + + /** + * Backward-compatible constructor taking the pre-AF-457 single-replica pair; a non-blank + * {@code readReplicaJdbcUrl} maps to a one-endpoint list (no endpoint id), caching defaults off. + */ + public DatasourceView( + UUID id, UUID organizationId, String name, DbType dbType, String host, Integer port, + String databaseName, String username, SslMode sslMode, int connectionPoolSize, + int maxRowsPerQuery, boolean requireReviewReads, boolean requireReviewWrites, + UUID reviewPlanId, boolean aiAnalysisEnabled, UUID aiConfigId, boolean textToSqlEnabled, + UUID customDriverId, String connectorId, String jdbcUrlOverride, + String readReplicaJdbcUrl, String readReplicaUsername, boolean active, + Instant createdAt, String localDatacenter) { + this(id, organizationId, name, dbType, host, port, databaseName, username, sslMode, + connectionPoolSize, maxRowsPerQuery, requireReviewReads, requireReviewWrites, + reviewPlanId, aiAnalysisEnabled, aiConfigId, textToSqlEnabled, customDriverId, + connectorId, jdbcUrlOverride, + readReplicaJdbcUrl == null || readReplicaJdbcUrl.isBlank() + ? List.of() + : List.of(new ReadReplicaView(null, readReplicaJdbcUrl, readReplicaUsername)), + active, createdAt, localDatacenter, false, null); + } + /** Backward-compatible constructor for non-Cassandra dialects (no {@code localDatacenter}). */ public DatasourceView( UUID id, UUID organizationId, String name, DbType dbType, String host, Integer port, diff --git a/backend/src/main/java/com/bablsoft/accessflow/core/api/QueryExecutionRequest.java b/backend/src/main/java/com/bablsoft/accessflow/core/api/QueryExecutionRequest.java index 68d0073c..2ef07918 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/core/api/QueryExecutionRequest.java +++ b/backend/src/main/java/com/bablsoft/accessflow/core/api/QueryExecutionRequest.java @@ -4,8 +4,15 @@ import java.time.Duration; import java.util.List; import java.util.Objects; +import java.util.Set; import java.util.UUID; +/** + * {@code referencedTables} (AF-457) carries the parser's normalized table set (lowercase + * {@code schema.table} or bare {@code table}) so the proxy can index cached SELECT results for + * write-invalidation. An empty set means "tables unknown": SELECTs are then never cached and + * writes purge the whole datasource cache (fail-safe). + */ public record QueryExecutionRequest( UUID datasourceId, String sql, @@ -17,7 +24,8 @@ public record QueryExecutionRequest( List rowSecurityPredicates, boolean transactional, List statements, - List softDeleteDirectives) { + List softDeleteDirectives, + Set referencedTables) { public QueryExecutionRequest { Objects.requireNonNull(datasourceId, "datasourceId"); @@ -41,6 +49,7 @@ public record QueryExecutionRequest( : List.copyOf(statements); softDeleteDirectives = softDeleteDirectives == null ? List.of() : List.copyOf(softDeleteDirectives); + referencedTables = referencedTables == null ? Set.of() : Set.copyOf(referencedTables); if (transactional) { if (queryType != QueryType.INSERT && queryType != QueryType.UPDATE @@ -57,6 +66,18 @@ public record QueryExecutionRequest( } } + /** Backward-compatible constructor without referenced tables (defaults to unknown). */ + public QueryExecutionRequest(UUID datasourceId, String sql, QueryType queryType, + Integer maxRowsOverride, Duration statementTimeoutOverride, + List restrictedColumns, List columnMasks, + List rowSecurityPredicates, + boolean transactional, List statements, + List softDeleteDirectives) { + this(datasourceId, sql, queryType, maxRowsOverride, statementTimeoutOverride, + restrictedColumns, columnMasks, rowSecurityPredicates, transactional, statements, + softDeleteDirectives, Set.of()); + } + public QueryExecutionRequest(UUID datasourceId, String sql, QueryType queryType, Integer maxRowsOverride, Duration statementTimeoutOverride) { this(datasourceId, sql, queryType, maxRowsOverride, statementTimeoutOverride, diff --git a/backend/src/main/java/com/bablsoft/accessflow/core/api/ReadReplicaEndpoint.java b/backend/src/main/java/com/bablsoft/accessflow/core/api/ReadReplicaEndpoint.java new file mode 100644 index 00000000..ab11e05b --- /dev/null +++ b/backend/src/main/java/com/bablsoft/accessflow/core/api/ReadReplicaEndpoint.java @@ -0,0 +1,23 @@ +package com.bablsoft.accessflow.core.api; + +import java.util.Objects; +import java.util.UUID; + +/** + * One read-replica endpoint of a datasource (AF-457). Carries only the encrypted password; never + * serialized to a public API response. {@code username} / {@code passwordEncrypted} may be + * {@code null}, in which case the primary datasource credentials are reused for the replica pool. + */ +public record ReadReplicaEndpoint( + UUID id, + String jdbcUrl, + String username, + String passwordEncrypted) { + + public ReadReplicaEndpoint { + Objects.requireNonNull(id, "id"); + if (jdbcUrl == null || jdbcUrl.isBlank()) { + throw new IllegalArgumentException("jdbcUrl must not be blank"); + } + } +} diff --git a/backend/src/main/java/com/bablsoft/accessflow/core/api/ReplicaEndpointInput.java b/backend/src/main/java/com/bablsoft/accessflow/core/api/ReplicaEndpointInput.java new file mode 100644 index 00000000..82b7aed7 --- /dev/null +++ b/backend/src/main/java/com/bablsoft/accessflow/core/api/ReplicaEndpointInput.java @@ -0,0 +1,18 @@ +package com.bablsoft.accessflow.core.api; + +import java.util.UUID; + +/** + * Admin input for one read-replica endpoint (AF-457), carried by + * {@link CreateDatasourceCommand} / {@link UpdateDatasourceCommand}. + * + *

{@code id} identifies an existing {@code datasource_read_replicas} row to update; {@code null} + * means "create a new endpoint". {@code password} semantics on update: {@code null} keeps the + * stored secret, empty string clears it (primary-credential fallback), non-blank re-encrypts. + */ +public record ReplicaEndpointInput( + UUID id, + String jdbcUrl, + String username, + String password) { +} diff --git a/backend/src/main/java/com/bablsoft/accessflow/core/api/TestReplicaCommand.java b/backend/src/main/java/com/bablsoft/accessflow/core/api/TestReplicaCommand.java index 0cd0d59c..f4ce8878 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/core/api/TestReplicaCommand.java +++ b/backend/src/main/java/com/bablsoft/accessflow/core/api/TestReplicaCommand.java @@ -1,13 +1,21 @@ package com.bablsoft.accessflow.core.api; +import java.util.UUID; + /** * Live-values request for the {@code POST /datasources/{id}/test-replica} endpoint. {@code jdbcUrl} * and {@code username} are required (the caller is exercising a candidate replica config); - * {@code password} may be {@code null} to fall back to the currently-persisted replica password — - * useful for changing URL/user without re-typing the secret. + * {@code password} may be {@code null} to fall back to the persisted password of the endpoint + * identified by {@code replicaId} — useful for changing URL/user without re-typing the secret. */ public record TestReplicaCommand( String jdbcUrl, String username, - String password -) {} + String password, + UUID replicaId +) { + /** Backward-compatible constructor without a {@code replicaId} (no persisted-password fallback). */ + public TestReplicaCommand(String jdbcUrl, String username, String password) { + this(jdbcUrl, username, password, null); + } +} diff --git a/backend/src/main/java/com/bablsoft/accessflow/core/api/UpdateDatasourceCommand.java b/backend/src/main/java/com/bablsoft/accessflow/core/api/UpdateDatasourceCommand.java index 20b698b2..cd2f1698 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/core/api/UpdateDatasourceCommand.java +++ b/backend/src/main/java/com/bablsoft/accessflow/core/api/UpdateDatasourceCommand.java @@ -1,7 +1,15 @@ package com.bablsoft.accessflow.core.api; +import java.util.List; import java.util.UUID; +/** + * Partial-update command: {@code null} means "leave unchanged" for every field. For + * {@code readReplicas} (AF-457), {@code null} keeps the current endpoint list, an empty list + * deletes all endpoints, and a non-empty list is a full replacement merged by endpoint id (items + * with a known {@code id} update that row; items without create new rows; stored rows absent from + * the list are removed). + */ public record UpdateDatasourceCommand( String name, String host, @@ -20,13 +28,33 @@ public record UpdateDatasourceCommand( Boolean textToSqlEnabled, Boolean clearAiConfig, String jdbcUrlOverride, - String readReplicaJdbcUrl, - String readReplicaUsername, - String readReplicaPassword, + List readReplicas, Boolean active, String localDatacenter, - String apiKey + String apiKey, + Boolean resultCacheEnabled, + Integer resultCacheTtlSeconds ) { + /** + * Backward-compatible constructor taking the pre-AF-457 single-replica triple. A {@code null} + * URL keeps the current replica list, a blank URL clears it, a non-blank URL replaces it with a + * single endpoint; cache settings stay unchanged. + */ + public UpdateDatasourceCommand( + String name, String host, Integer port, String databaseName, String username, + String password, SslMode sslMode, Integer connectionPoolSize, Integer maxRowsPerQuery, + Boolean requireReviewReads, Boolean requireReviewWrites, UUID reviewPlanId, + Boolean aiAnalysisEnabled, UUID aiConfigId, Boolean textToSqlEnabled, + Boolean clearAiConfig, String jdbcUrlOverride, String readReplicaJdbcUrl, + String readReplicaUsername, String readReplicaPassword, Boolean active, + String localDatacenter, String apiKey) { + this(name, host, port, databaseName, username, password, sslMode, connectionPoolSize, + maxRowsPerQuery, requireReviewReads, requireReviewWrites, reviewPlanId, + aiAnalysisEnabled, aiConfigId, textToSqlEnabled, clearAiConfig, jdbcUrlOverride, + legacyReplicaList(readReplicaJdbcUrl, readReplicaUsername, readReplicaPassword), + active, localDatacenter, apiKey, null, null); + } + /** Backward-compatible constructor for the dialects with no {@code apiKey} (everything but the search engines). */ public UpdateDatasourceCommand( String name, String host, Integer port, String databaseName, String username, @@ -56,4 +84,15 @@ public UpdateDatasourceCommand( aiAnalysisEnabled, aiConfigId, textToSqlEnabled, clearAiConfig, jdbcUrlOverride, readReplicaJdbcUrl, readReplicaUsername, readReplicaPassword, active, null, null); } + + private static List legacyReplicaList(String jdbcUrl, String username, + String password) { + if (jdbcUrl == null) { + return null; + } + if (jdbcUrl.isBlank()) { + return List.of(); + } + return List.of(new ReplicaEndpointInput(null, jdbcUrl, username, password)); + } } diff --git a/backend/src/main/java/com/bablsoft/accessflow/core/events/DatasourceCacheConfigChangedEvent.java b/backend/src/main/java/com/bablsoft/accessflow/core/events/DatasourceCacheConfigChangedEvent.java new file mode 100644 index 00000000..36aafd6f --- /dev/null +++ b/backend/src/main/java/com/bablsoft/accessflow/core/events/DatasourceCacheConfigChangedEvent.java @@ -0,0 +1,11 @@ +package com.bablsoft.accessflow.core.events; + +import java.util.UUID; + +/** + * Published when a datasource's SELECT result-cache settings change (enabled flag or TTL, AF-457). + * Consumers purge cached results for the datasource; unlike {@link DatasourceConfigChangedEvent} + * this must not evict connection pools. + */ +public record DatasourceCacheConfigChangedEvent(UUID datasourceId) { +} diff --git a/backend/src/main/java/com/bablsoft/accessflow/core/internal/DatasourceAdminServiceImpl.java b/backend/src/main/java/com/bablsoft/accessflow/core/internal/DatasourceAdminServiceImpl.java index e1b7b07f..7735e319 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/core/internal/DatasourceAdminServiceImpl.java +++ b/backend/src/main/java/com/bablsoft/accessflow/core/internal/DatasourceAdminServiceImpl.java @@ -30,9 +30,12 @@ import com.bablsoft.accessflow.core.api.UpdateDatasourceCommand; import com.bablsoft.accessflow.core.internal.persistence.repo.CustomJdbcDriverRepository; import com.bablsoft.accessflow.core.internal.persistence.entity.CustomJdbcDriverEntity; +import com.bablsoft.accessflow.core.api.ReplicaEndpointInput; +import com.bablsoft.accessflow.core.events.DatasourceCacheConfigChangedEvent; import com.bablsoft.accessflow.core.events.DatasourceConfigChangedEvent; import com.bablsoft.accessflow.core.events.DatasourceDeactivatedEvent; import com.bablsoft.accessflow.core.internal.persistence.entity.DatasourceEntity; +import com.bablsoft.accessflow.core.internal.persistence.entity.DatasourceReadReplicaEntity; import com.bablsoft.accessflow.core.internal.persistence.entity.DatasourceGroupPermissionEntity; import com.bablsoft.accessflow.core.internal.persistence.entity.DatasourceUserPermissionEntity; import com.bablsoft.accessflow.core.internal.persistence.entity.UserEntity; @@ -208,8 +211,13 @@ public DatasourceView create(CreateDatasourceCommand command) { if (command.reviewPlanId() != null) { entity.setReviewPlan(reviewPlanRepository.getReferenceById(command.reviewPlanId())); } - applyReplicaOnCreate(entity, command.readReplicaJdbcUrl(), - command.readReplicaUsername(), command.readReplicaPassword()); + applyReplicasOnCreate(entity, command.readReplicas()); + if (command.resultCacheEnabled() != null) { + entity.setResultCacheEnabled(command.resultCacheEnabled()); + } + if (command.resultCacheTtlSeconds() != null) { + entity.setResultCacheTtlSeconds(command.resultCacheTtlSeconds()); + } entity.setActive(true); return toView(datasourceRepository.save(entity)); } @@ -219,6 +227,7 @@ public DatasourceView create(CreateDatasourceCommand command) { public DatasourceView update(UUID id, UUID organizationId, UpdateDatasourceCommand command) { var entity = loadInOrganization(id, organizationId); var before = poolFingerprint(entity); + var cacheBefore = cacheFingerprint(entity); var wasActive = entity.isActive(); if (command.name() != null && !command.name().equals(entity.getName())) { if (datasourceRepository.existsByOrganization_IdAndNameIgnoreCaseAndIdNot( @@ -297,8 +306,13 @@ public DatasourceView update(UUID id, UUID organizationId, UpdateDatasourceComma if (command.reviewPlanId() != null) { entity.setReviewPlan(reviewPlanRepository.getReferenceById(command.reviewPlanId())); } - applyReplicaOnUpdate(entity, command.readReplicaJdbcUrl(), - command.readReplicaUsername(), command.readReplicaPassword()); + applyReplicasOnUpdate(entity, command.readReplicas()); + if (command.resultCacheEnabled() != null) { + entity.setResultCacheEnabled(command.resultCacheEnabled()); + } + if (command.resultCacheTtlSeconds() != null) { + entity.setResultCacheTtlSeconds(command.resultCacheTtlSeconds()); + } if (command.active() != null) { entity.setActive(command.active()); } @@ -306,6 +320,9 @@ public DatasourceView update(UUID id, UUID organizationId, UpdateDatasourceComma eventPublisher.publishEvent(new DatasourceDeactivatedEvent(entity.getId())); } else if (!Objects.equals(before, poolFingerprint(entity))) { eventPublisher.publishEvent(new DatasourceConfigChangedEvent(entity.getId())); + } else if (!Objects.equals(cacheBefore, cacheFingerprint(entity))) { + // Cache-setting changes purge cached results without evicting connection pools. + eventPublisher.publishEvent(new DatasourceCacheConfigChangedEvent(entity.getId())); } return toView(entity); } @@ -333,9 +350,11 @@ private static PoolFingerprint poolFingerprint(DatasourceEntity entity) { entity.getCustomDriver() != null ? entity.getCustomDriver().getId() : null, entity.getConnectorId(), entity.getJdbcUrlOverride(), - entity.getReadReplicaJdbcUrl(), - entity.getReadReplicaUsername(), - entity.getReadReplicaPasswordEncrypted(), + entity.getReadReplicas().stream() + .map(replica -> new ReplicaFingerprint(replica.getId(), + replica.getJdbcUrl(), replica.getUsername(), + replica.getPasswordEncrypted())) + .toList(), entity.getLocalDatacenter(), entity.getApiKeyEncrypted()); } @@ -344,49 +363,82 @@ private record PoolFingerprint(String host, Integer port, String databaseName, S String passwordEncrypted, SslMode sslMode, int connectionPoolSize, UUID customDriverId, String connectorId, String jdbcUrlOverride, - String readReplicaJdbcUrl, String readReplicaUsername, - String readReplicaPasswordEncrypted, String localDatacenter, + List readReplicas, String localDatacenter, String apiKeyEncrypted) { } + private record ReplicaFingerprint(UUID id, String jdbcUrl, String username, + String passwordEncrypted) { + } + + private record CacheFingerprint(boolean enabled, Integer ttlSeconds) { + } + + private static CacheFingerprint cacheFingerprint(DatasourceEntity entity) { + return new CacheFingerprint(entity.isResultCacheEnabled(), + entity.getResultCacheTtlSeconds()); + } + /** - * Replica clear-on-blank: when {@code jdbcUrl} is non-null but blank, all three replica fields - * are cleared. When {@code jdbcUrl} is non-null and non-blank, it is stored along with any - * provided username/password (re-encrypting the password). When {@code jdbcUrl} is null, the - * existing replica state is left intact and only username/password are updated if provided. + * Full-list replacement merged by endpoint id (AF-457): {@code null} keeps the current list, + * an empty list deletes every endpoint. Items with an {@code id} matching a stored row update + * it — a {@code null} password keeps the stored secret, an empty one clears it (primary + * credential fallback), a non-blank one is re-encrypted. Items without an {@code id} create + * new rows. Stored rows absent from the list are removed (orphanRemoval). */ - private void applyReplicaOnUpdate(DatasourceEntity entity, String jdbcUrl, - String username, String password) { - if (jdbcUrl != null) { - if (jdbcUrl.isBlank()) { - entity.setReadReplicaJdbcUrl(null); - entity.setReadReplicaUsername(null); - entity.setReadReplicaPasswordEncrypted(null); - return; - } - entity.setReadReplicaJdbcUrl(jdbcUrl); - } - if (username != null) { - entity.setReadReplicaUsername(username); + private void applyReplicasOnUpdate(DatasourceEntity entity, List inputs) { + if (inputs == null) { + return; } - if (password != null) { - entity.setReadReplicaPasswordEncrypted( - password.isEmpty() ? null : storeCredential(password)); + Map existing = new LinkedHashMap<>(); + for (var replica : entity.getReadReplicas()) { + existing.put(replica.getId(), replica); + } + List merged = new ArrayList<>(); + int position = 0; + for (var input : inputs) { + var current = input.id() != null ? existing.get(input.id()) : null; + if (current != null) { + current.setJdbcUrl(input.jdbcUrl()); + current.setUsername(blankToNull(input.username())); + if (input.password() != null) { + current.setPasswordEncrypted( + input.password().isEmpty() ? null : storeCredential(input.password())); + } + current.setPosition(position++); + merged.add(current); + } else { + merged.add(newReplica(entity, input, position++)); + } } + entity.getReadReplicas().clear(); + entity.getReadReplicas().addAll(merged); } - private void applyReplicaOnCreate(DatasourceEntity entity, String jdbcUrl, - String username, String password) { - if (jdbcUrl == null || jdbcUrl.isBlank()) { + private void applyReplicasOnCreate(DatasourceEntity entity, List inputs) { + if (inputs == null || inputs.isEmpty()) { return; } - entity.setReadReplicaJdbcUrl(jdbcUrl); - entity.setReadReplicaUsername(username); - if (password != null && !password.isEmpty()) { - entity.setReadReplicaPasswordEncrypted(storeCredential(password)); + int position = 0; + for (var input : inputs) { + entity.getReadReplicas().add(newReplica(entity, input, position++)); } } + private DatasourceReadReplicaEntity newReplica(DatasourceEntity entity, + ReplicaEndpointInput input, int position) { + var replica = new DatasourceReadReplicaEntity(); + replica.setId(UUID.randomUUID()); + replica.setDatasource(entity); + replica.setJdbcUrl(input.jdbcUrl()); + replica.setUsername(blankToNull(input.username())); + if (input.password() != null && !input.password().isEmpty()) { + replica.setPasswordEncrypted(storeCredential(input.password())); + } + replica.setPosition(position); + return replica; + } + @Override @Transactional(readOnly = true) public ConnectionTestResult test(UUID id, UUID organizationId) { @@ -422,11 +474,18 @@ public ConnectionTestResult testReplica(UUID id, UUID organizationId, TestReplic LocaleContextHolder.getLocale())); } String plaintextPassword; + String storedPassword = command.replicaId() == null ? null + : entity.getReadReplicas().stream() + .filter(replica -> command.replicaId().equals(replica.getId())) + .map(DatasourceReadReplicaEntity::getPasswordEncrypted) + .filter(Objects::nonNull) + .findFirst() + .orElse(null); if (command.password() != null && !command.password().isEmpty()) { plaintextPassword = command.password(); - } else if (entity.getReadReplicaPasswordEncrypted() != null) { + } else if (storedPassword != null) { plaintextPassword = secretResolutionService.resolve( - entity.getReadReplicaPasswordEncrypted(), entity.getId(), organizationId); + storedPassword, entity.getId(), organizationId); } else { throw new DatasourceConnectionTestException( messageSource.getMessage("error.replica_not_configured", null, @@ -646,11 +705,15 @@ private DatasourceView toView(DatasourceEntity entity) { entity.getCustomDriver() != null ? entity.getCustomDriver().getId() : null, entity.getConnectorId(), entity.getJdbcUrlOverride(), - entity.getReadReplicaJdbcUrl(), - entity.getReadReplicaUsername(), + entity.getReadReplicas().stream() + .map(replica -> new DatasourceView.ReadReplicaView(replica.getId(), + replica.getJdbcUrl(), replica.getUsername())) + .toList(), entity.isActive(), entity.getCreatedAt(), - entity.getLocalDatacenter()); + entity.getLocalDatacenter(), + entity.isResultCacheEnabled(), + entity.getResultCacheTtlSeconds()); } private CustomJdbcDriverEntity resolveCustomDriverForCreate(CreateDatasourceCommand command) { diff --git a/backend/src/main/java/com/bablsoft/accessflow/core/internal/DatasourceDescriptorMapper.java b/backend/src/main/java/com/bablsoft/accessflow/core/internal/DatasourceDescriptorMapper.java index a9e24a56..e08b7842 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/core/internal/DatasourceDescriptorMapper.java +++ b/backend/src/main/java/com/bablsoft/accessflow/core/internal/DatasourceDescriptorMapper.java @@ -1,6 +1,7 @@ package com.bablsoft.accessflow.core.internal; import com.bablsoft.accessflow.core.api.DatasourceConnectionDescriptor; +import com.bablsoft.accessflow.core.api.ReadReplicaEndpoint; import com.bablsoft.accessflow.core.internal.persistence.entity.DatasourceEntity; /** @@ -32,11 +33,15 @@ static DatasourceConnectionDescriptor from(DatasourceEntity entity) { entity.getCustomDriver() != null ? entity.getCustomDriver().getId() : null, entity.getConnectorId(), entity.getJdbcUrlOverride(), - entity.getReadReplicaJdbcUrl(), - entity.getReadReplicaUsername(), - entity.getReadReplicaPasswordEncrypted(), + entity.getReadReplicas().stream() + .map(replica -> new ReadReplicaEndpoint(replica.getId(), + replica.getJdbcUrl(), replica.getUsername(), + replica.getPasswordEncrypted())) + .toList(), entity.isActive(), entity.getLocalDatacenter(), - entity.getApiKeyEncrypted()); + entity.getApiKeyEncrypted(), + entity.isResultCacheEnabled(), + entity.getResultCacheTtlSeconds()); } } diff --git a/backend/src/main/java/com/bablsoft/accessflow/core/internal/persistence/entity/DatasourceEntity.java b/backend/src/main/java/com/bablsoft/accessflow/core/internal/persistence/entity/DatasourceEntity.java index 30e24950..dfc4d063 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/core/internal/persistence/entity/DatasourceEntity.java +++ b/backend/src/main/java/com/bablsoft/accessflow/core/internal/persistence/entity/DatasourceEntity.java @@ -3,6 +3,7 @@ import com.fasterxml.jackson.annotation.JsonIgnore; import com.bablsoft.accessflow.core.api.DbType; import com.bablsoft.accessflow.core.api.SslMode; +import jakarta.persistence.CascadeType; import jakarta.persistence.Column; import jakarta.persistence.Entity; import jakarta.persistence.EnumType; @@ -11,6 +12,8 @@ import jakarta.persistence.Id; import jakarta.persistence.JoinColumn; import jakarta.persistence.ManyToOne; +import jakarta.persistence.OneToMany; +import jakarta.persistence.OrderBy; import jakarta.persistence.Table; import lombok.Getter; import lombok.NoArgsConstructor; @@ -19,6 +22,8 @@ import org.hibernate.dialect.type.PostgreSQLEnumJdbcType; import java.time.Instant; +import java.util.ArrayList; +import java.util.List; import java.util.UUID; @Entity @@ -99,15 +104,15 @@ public class DatasourceEntity { @Column(name = "connector_id", length = 64) private String connectorId; - @Column(name = "read_replica_jdbc_url", columnDefinition = "TEXT") - private String readReplicaJdbcUrl; + @OneToMany(mappedBy = "datasource", cascade = CascadeType.ALL, orphanRemoval = true) + @OrderBy("position") + private List readReplicas = new ArrayList<>(); - @Column(name = "read_replica_username", length = 255) - private String readReplicaUsername; + @Column(name = "result_cache_enabled", nullable = false) + private boolean resultCacheEnabled = false; - @JsonIgnore - @Column(name = "read_replica_password_encrypted", columnDefinition = "TEXT") - private String readReplicaPasswordEncrypted; + @Column(name = "result_cache_ttl_seconds") + private Integer resultCacheTtlSeconds; @Column(name = "local_datacenter", length = 255) private String localDatacenter; diff --git a/backend/src/main/java/com/bablsoft/accessflow/core/internal/persistence/entity/DatasourceReadReplicaEntity.java b/backend/src/main/java/com/bablsoft/accessflow/core/internal/persistence/entity/DatasourceReadReplicaEntity.java new file mode 100644 index 00000000..4d91755d --- /dev/null +++ b/backend/src/main/java/com/bablsoft/accessflow/core/internal/persistence/entity/DatasourceReadReplicaEntity.java @@ -0,0 +1,47 @@ +package com.bablsoft.accessflow.core.internal.persistence.entity; + +import com.fasterxml.jackson.annotation.JsonIgnore; +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.FetchType; +import jakarta.persistence.Id; +import jakarta.persistence.JoinColumn; +import jakarta.persistence.ManyToOne; +import jakarta.persistence.Table; +import lombok.Getter; +import lombok.NoArgsConstructor; +import lombok.Setter; + +import java.time.Instant; +import java.util.UUID; + +@Entity +@Table(name = "datasource_read_replicas") +@Getter +@Setter +@NoArgsConstructor +public class DatasourceReadReplicaEntity { + + @Id + private UUID id; + + @ManyToOne(fetch = FetchType.LAZY, optional = false) + @JoinColumn(name = "datasource_id", nullable = false) + private DatasourceEntity datasource; + + @Column(name = "jdbc_url", nullable = false, columnDefinition = "TEXT") + private String jdbcUrl; + + @Column(length = 255) + private String username; + + @JsonIgnore + @Column(name = "password_encrypted", columnDefinition = "TEXT") + private String passwordEncrypted; + + @Column(nullable = false) + private int position = 0; + + @Column(name = "created_at", nullable = false, updatable = false) + private Instant createdAt = Instant.now(); +} diff --git a/backend/src/main/java/com/bablsoft/accessflow/core/internal/persistence/repo/DatasourceRepository.java b/backend/src/main/java/com/bablsoft/accessflow/core/internal/persistence/repo/DatasourceRepository.java index 15bad75c..93a9ea83 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/core/internal/persistence/repo/DatasourceRepository.java +++ b/backend/src/main/java/com/bablsoft/accessflow/core/internal/persistence/repo/DatasourceRepository.java @@ -47,8 +47,10 @@ select d.aiConfigId, count(d) from DatasourceEntity d @Query(""" select d from DatasourceEntity d where d.passwordEncrypted = :reference - or d.readReplicaPasswordEncrypted = :reference or d.apiKeyEncrypted = :reference + or exists ( + select 1 from DatasourceReadReplicaEntity r + where r.datasource = d and r.passwordEncrypted = :reference) """) List findAllByCredentialReference(@Param("reference") String reference); diff --git a/backend/src/main/java/com/bablsoft/accessflow/lifecycle/internal/ErasureExecutionService.java b/backend/src/main/java/com/bablsoft/accessflow/lifecycle/internal/ErasureExecutionService.java index 4d8f3ec7..e6d3e9e8 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/lifecycle/internal/ErasureExecutionService.java +++ b/backend/src/main/java/com/bablsoft/accessflow/lifecycle/internal/ErasureExecutionService.java @@ -29,6 +29,7 @@ import java.time.Clock; import java.util.ArrayList; +import java.util.Locale; import java.util.LinkedHashSet; import java.util.List; import java.util.Map; @@ -117,9 +118,11 @@ private long eraseTable(DeletionRequestEntity entity, String table, entity.getRawWhere(), null, null); String sql = "DELETE FROM " + table + (compiled.whereClause() == null ? "" : " WHERE " + compiled.whereClause()); + // referencedTables drives SELECT result-cache invalidation (AF-457) — erased rows must + // not survive in cached reads. var request = new QueryExecutionRequest(entity.getDatasourceId(), sql, QueryType.DELETE, null, null, List.of(), List.of(), compiled.directives(), - false, null, softDeletes); + false, null, softDeletes, Set.of(table.toLowerCase(Locale.ROOT))); var result = queryExecutor.execute(request); return result instanceof UpdateExecutionResult u ? u.rowsAffected() : 0; } diff --git a/backend/src/main/java/com/bablsoft/accessflow/lifecycle/internal/RetentionPolicyExecutionService.java b/backend/src/main/java/com/bablsoft/accessflow/lifecycle/internal/RetentionPolicyExecutionService.java index 63f02e7f..3830bd6a 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/lifecycle/internal/RetentionPolicyExecutionService.java +++ b/backend/src/main/java/com/bablsoft/accessflow/lifecycle/internal/RetentionPolicyExecutionService.java @@ -27,7 +27,9 @@ import java.time.Clock; import java.time.ZonedDateTime; import java.util.List; +import java.util.Locale; import java.util.Map; +import java.util.Set; import java.util.UUID; import java.util.regex.Pattern; @@ -123,8 +125,11 @@ private long eraseByPolicy(RetentionPolicyEntity policy) { ? directiveResolutionService.resolveSoftDeletes( policy.getOrganizationId(), policy.getDatasourceId()) : List.of(); + // referencedTables drives SELECT result-cache invalidation (AF-457) — retention-expired + // rows must not survive in cached reads. var request = new QueryExecutionRequest(policy.getDatasourceId(), sql, QueryType.DELETE, - null, null, List.of(), List.of(), compiled.directives(), false, null, softDeletes); + null, null, List.of(), List.of(), compiled.directives(), false, null, softDeletes, + Set.of(table.toLowerCase(Locale.ROOT))); var result = queryExecutor.execute(request); return result instanceof UpdateExecutionResult u ? u.rowsAffected() : 0; } diff --git a/backend/src/main/java/com/bablsoft/accessflow/proxy/api/DatasourceConnectionPoolManager.java b/backend/src/main/java/com/bablsoft/accessflow/proxy/api/DatasourceConnectionPoolManager.java index d9d486a9..7d683979 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/proxy/api/DatasourceConnectionPoolManager.java +++ b/backend/src/main/java/com/bablsoft/accessflow/proxy/api/DatasourceConnectionPoolManager.java @@ -1,5 +1,6 @@ package com.bablsoft.accessflow.proxy.api; +import java.util.List; import java.util.Optional; import java.util.UUID; import javax.sql.DataSource; @@ -9,11 +10,11 @@ * lazily on the first {@link #resolve(UUID)} call and live until either {@link #evict(UUID)} * is invoked (e.g. on a datasource update or deactivation event) or the application shuts down. * - *

When the datasource has a read replica configured, a sibling pool is built on demand by - * {@link #resolveReplica(UUID)}; both pools are evicted together. Implementations decrypt the - * persisted password no earlier than pool initialization and do not retain a reference to the - * plaintext beyond the call. The returned {@link DataSource} is the Hikari pool itself; callers - * obtain connections via the standard JDBC idiom: + *

When the datasource has read replicas configured (AF-457), a sibling pool per endpoint is + * built on demand by {@link #resolveReplica(UUID, UUID)}; all pools are evicted together. + * Implementations decrypt the persisted password no earlier than pool initialization and do not + * retain a reference to the plaintext beyond the call. The returned {@link DataSource} is the + * Hikari pool itself; callers obtain connections via the standard JDBC idiom: * 

{@code
  * try (var connection = manager.resolve(id).getConnection()) { ... }
  * }
@@ -30,19 +31,28 @@ public interface DatasourceConnectionPoolManager { DataSource resolve(UUID datasourceId); /** - * Return the cached read-replica pool for {@code datasourceId} if and only if the datasource - * has a replica configured. Empty when no replica is set; never falls back to the primary - * (the caller's routing layer owns that decision). + * Return the datasource's read-replica endpoints in configured order, without creating any + * pool. Empty when the datasource has no replicas. * * @throws DatasourceUnavailableException if the datasource is missing or inactive. + */ + List replicaEndpoints(UUID datasourceId); + + /** + * Return the cached pool for one read-replica endpoint of {@code datasourceId}, creating it + * on first use. Never falls back to the primary or a sibling endpoint (the caller's routing + * layer owns that decision). + * + * @throws DatasourceUnavailableException if the datasource is missing or inactive, or the + * endpoint id is not one of its replicas. * @throws PoolInitializationException if Hikari cannot establish the first connection to * the replica. */ - Optional resolveReplica(UUID datasourceId); + DataSource resolveReplica(UUID datasourceId, UUID endpointId); /** - * Close and remove both the primary and replica pools for {@code datasourceId}. No-op for - * either side when no pool is cached. + * Close and remove the primary and every replica pool for {@code datasourceId}. No-op for + * any side with no cached pool. */ void evict(UUID datasourceId); @@ -53,4 +63,11 @@ public interface DatasourceConnectionPoolManager { * customer database. */ Optional poolStats(UUID datasourceId); + + /** + * Return live gauges for the cached pool of one replica endpoint, or empty when that + * endpoint's pool is not currently cached. Never creates a pool (same guarantee as + * {@link #poolStats(UUID)}). + */ + Optional replicaPoolStats(UUID datasourceId, UUID endpointId); } diff --git a/backend/src/main/java/com/bablsoft/accessflow/proxy/api/DatasourceHealthSnapshot.java b/backend/src/main/java/com/bablsoft/accessflow/proxy/api/DatasourceHealthSnapshot.java index 288779cf..6180fde2 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/proxy/api/DatasourceHealthSnapshot.java +++ b/backend/src/main/java/com/bablsoft/accessflow/proxy/api/DatasourceHealthSnapshot.java @@ -2,13 +2,15 @@ import com.bablsoft.accessflow.core.api.DbType; +import java.util.List; import java.util.UUID; /** * One per-datasource row of the admin health dashboard. Pool gauges are {@code null} when no live * HikariCP pool is cached for the datasource (it has not been queried since the last restart or * pool eviction). Percentiles are {@code null} when no executed query carried a duration in the - * trailing 24h window. + * trailing 24h window. {@code replicas} carries per-endpoint read-replica health (AF-457) — + * empty when the datasource has no replicas. */ public record DatasourceHealthSnapshot( UUID datasourceId, @@ -23,5 +25,21 @@ public record DatasourceHealthSnapshot( long queriesLast24h, Double executionMsP50, Double executionMsP95, - long errorsLast24h) { + long errorsLast24h, + List replicas) { + + public DatasourceHealthSnapshot { + replicas = replicas == null ? List.of() : List.copyOf(replicas); + } + + /** Backward-compatible constructor without replica health (defaults to none). */ + public DatasourceHealthSnapshot( + UUID datasourceId, String datasourceName, DbType dbType, boolean active, + Integer poolActive, Integer poolIdle, Integer poolWaiting, Integer poolTotal, + Integer poolMax, long queriesLast24h, Double executionMsP50, Double executionMsP95, + long errorsLast24h) { + this(datasourceId, datasourceName, dbType, active, poolActive, poolIdle, poolWaiting, + poolTotal, poolMax, queriesLast24h, executionMsP50, executionMsP95, errorsLast24h, + List.of()); + } } diff --git a/backend/src/main/java/com/bablsoft/accessflow/proxy/api/ReplicaEndpointHealth.java b/backend/src/main/java/com/bablsoft/accessflow/proxy/api/ReplicaEndpointHealth.java new file mode 100644 index 00000000..c4aabdc3 --- /dev/null +++ b/backend/src/main/java/com/bablsoft/accessflow/proxy/api/ReplicaEndpointHealth.java @@ -0,0 +1,17 @@ +package com.bablsoft.accessflow.proxy.api; + +import java.util.UUID; + +/** + * Health of one read-replica endpoint as surfaced on the admin datasource-health snapshot + * (AF-457). {@code healthy} reflects this node's circuit-breaker state (a DOWN endpoint is being + * skipped by the read load-balancer until its cooldown elapses). Pool gauges are {@code null} + * when the endpoint's pool has not been created on this node yet. + */ +public record ReplicaEndpointHealth( + UUID endpointId, + String label, + boolean healthy, + Integer poolActive, + Integer poolTotal) { +} diff --git a/backend/src/main/java/com/bablsoft/accessflow/proxy/api/ReplicaEndpointRef.java b/backend/src/main/java/com/bablsoft/accessflow/proxy/api/ReplicaEndpointRef.java new file mode 100644 index 00000000..11686720 --- /dev/null +++ b/backend/src/main/java/com/bablsoft/accessflow/proxy/api/ReplicaEndpointRef.java @@ -0,0 +1,11 @@ +package com.bablsoft.accessflow.proxy.api; + +import java.util.UUID; + +/** + * Lightweight reference to one read-replica endpoint of a datasource (AF-457). {@code label} is a + * redacted display form of the endpoint's JDBC URL (host/port only — never credentials or query + * params), safe for logs, audit metadata, and admin health views. + */ +public record ReplicaEndpointRef(UUID endpointId, String label) { +} diff --git a/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/BatchInsertPlanner.java b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/BatchInsertPlanner.java new file mode 100644 index 00000000..9569b1e9 --- /dev/null +++ b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/BatchInsertPlanner.java @@ -0,0 +1,162 @@ +package com.bablsoft.accessflow.proxy.internal; + +import net.sf.jsqlparser.expression.BooleanValue; +import net.sf.jsqlparser.expression.DateValue; +import net.sf.jsqlparser.expression.DoubleValue; +import net.sf.jsqlparser.expression.Expression; +import net.sf.jsqlparser.expression.LongValue; +import net.sf.jsqlparser.expression.NullValue; +import net.sf.jsqlparser.expression.SignedExpression; +import net.sf.jsqlparser.expression.StringValue; +import net.sf.jsqlparser.expression.TimeValue; +import net.sf.jsqlparser.expression.TimestampValue; +import net.sf.jsqlparser.expression.operators.relational.ParenthesedExpressionList; +import net.sf.jsqlparser.parser.CCJSqlParserUtil; +import net.sf.jsqlparser.schema.Column; +import net.sf.jsqlparser.statement.insert.Insert; +import net.sf.jsqlparser.statement.select.Values; + +import java.util.ArrayList; +import java.util.List; + +/** + * Groups the statements of a {@code BEGIN…COMMIT} envelope into JDBC-batchable runs (AF-457). + * A statement is batch-eligible when its row-security rewrite was a no-op (no binds) and it is a + * simple single-row {@code INSERT INTO t [(cols)] VALUES (...)} whose values are all plain + * literals; consecutive eligible statements sharing the same table, column list, and arity form + * one {@link BatchStep} executed as a single {@code PreparedStatement} with + * {@code addBatch()}/{@code executeLargeBatch()}. Everything else stays a {@link SingleStep} on + * the existing per-statement path. (A single multi-row {@code VALUES (...),(...)} INSERT is + * already one statement/one round trip and is deliberately left untouched.) + */ +final class BatchInsertPlanner { + + private BatchInsertPlanner() { + } + + sealed interface Step permits SingleStep, BatchStep { + } + + /** Executed on the existing per-statement path with the original rewrite result. */ + record SingleStep(int statementIndex) implements Step { + } + + /** One parameterized INSERT template plus the literal bind row of each folded statement. */ + record BatchStep(String templateSql, List> rowBinds) implements Step { + } + + /** A statement's batch-relevant shape; {@code null} when not batch-eligible. */ + private record InsertShape(String table, String columnList, int arity, List values, + String templateSql) { + } + + /** + * Plans the execution steps for the envelope's statements. {@code batchable[i]} must be false + * for any statement whose row-security rewrite produced binds or changed the SQL — those are + * never folded. + */ + static List plan(List statements, boolean[] batchable) { + var steps = new ArrayList(); + var shapes = new InsertShape[statements.size()]; + for (int i = 0; i < statements.size(); i++) { + shapes[i] = batchable[i] ? shapeOf(statements.get(i)) : null; + } + int i = 0; + while (i < statements.size()) { + var head = shapes[i]; + if (head == null) { + steps.add(new SingleStep(i)); + i++; + continue; + } + int runEnd = i + 1; + while (runEnd < statements.size() && sameShape(head, shapes[runEnd])) { + runEnd++; + } + if (runEnd - i >= 2) { + var rows = new ArrayList>(runEnd - i); + for (int j = i; j < runEnd; j++) { + rows.add(shapes[j].values()); + } + steps.add(new BatchStep(head.templateSql(), rows)); + } else { + steps.add(new SingleStep(i)); + } + i = runEnd; + } + return steps; + } + + private static boolean sameShape(InsertShape head, InsertShape candidate) { + return candidate != null + && head.table().equals(candidate.table()) + && head.columnList().equals(candidate.columnList()) + && head.arity() == candidate.arity(); + } + + private static InsertShape shapeOf(String sql) { + Insert insert; + try { + var parsed = CCJSqlParserUtil.parse(sql); + if (!(parsed instanceof Insert candidate)) { + return null; + } + insert = candidate; + } catch (Exception ex) { + return null; + } + if (insert.getTable() == null || !(insert.getSelect() instanceof Values values)) { + return null; + } + // Single-row VALUES carry the value expressions directly; multi-row VALUES wrap each row + // in a ParenthesedExpressionList. Multi-row INSERTs are already one statement/one round + // trip and stay on the per-statement path. + var row = values.getExpressions(); + if (row == null || row.isEmpty() + || row.stream().anyMatch(e -> e instanceof ParenthesedExpressionList)) { + return null; + } + var literals = new ArrayList(row.size()); + for (Object expression : row) { + var literal = literalValue((Expression) expression); + if (literal == NOT_A_LITERAL) { + return null; + } + literals.add(literal); + } + String table = insert.getTable().getFullyQualifiedName(); + String columnList = insert.getColumns() == null ? "" + : insert.getColumns().stream() + .map(Column::getFullyQualifiedName) + .reduce((a, b) -> a + "," + b) + .orElse(""); + String template = "INSERT INTO " + table + + (columnList.isEmpty() ? "" : " (" + columnList + ")") + + " VALUES (" + "?, ".repeat(literals.size() - 1) + "?)"; + return new InsertShape(table, columnList, literals.size(), literals, template); + } + + private static final Object NOT_A_LITERAL = new Object(); + + private static Object literalValue(Expression expression) { + return switch (expression) { + case NullValue ignored -> null; + case LongValue value -> value.getValue(); + case DoubleValue value -> value.getValue(); + case StringValue value -> value.getValue(); + case BooleanValue value -> value.getValue(); + case DateValue value -> value.getValue(); + case TimeValue value -> value.getValue(); + case TimestampValue value -> value.getValue(); + case SignedExpression signed when signed.getSign() == '-' -> { + var inner = literalValue(signed.getExpression()); + yield switch (inner) { + case Long longValue -> -longValue; + case Double doubleValue -> -doubleValue; + case null, default -> NOT_A_LITERAL; + }; + } + default -> NOT_A_LITERAL; + }; + } +} diff --git a/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/DatasourcePoolEvictionListener.java b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/DatasourcePoolEvictionListener.java index 2287ceef..38e03638 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/DatasourcePoolEvictionListener.java +++ b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/DatasourcePoolEvictionListener.java @@ -12,14 +12,17 @@ class DatasourcePoolEvictionListener { private final DatasourceConnectionPoolManager connectionPoolManager; + private final ReplicaHealthRegistry replicaHealthRegistry; @ApplicationModuleListener void onConfigChanged(DatasourceConfigChangedEvent event) { connectionPoolManager.evict(event.datasourceId()); + replicaHealthRegistry.evict(event.datasourceId()); } @ApplicationModuleListener void onDeactivated(DatasourceDeactivatedEvent event) { connectionPoolManager.evict(event.datasourceId()); + replicaHealthRegistry.evict(event.datasourceId()); } } diff --git a/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/DatasourcePoolFactory.java b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/DatasourcePoolFactory.java index 23f67f55..e70695a1 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/DatasourcePoolFactory.java +++ b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/DatasourcePoolFactory.java @@ -6,6 +6,7 @@ import com.bablsoft.accessflow.core.api.DbType; import com.bablsoft.accessflow.core.api.DriverCatalogService; import com.bablsoft.accessflow.core.api.JdbcCoordinatesFactory; +import com.bablsoft.accessflow.core.api.ReadReplicaEndpoint; import com.bablsoft.accessflow.core.api.ResolvedDriver; import com.bablsoft.accessflow.core.api.SecretResolutionService; import com.zaxxer.hikari.HikariConfig; @@ -51,23 +52,25 @@ HikariDataSource createPool(DatasourceConnectionDescriptor descriptor) { } /** - * Build a Hikari pool against the read replica URL/credentials on the descriptor. Reuses - * the same driver class as the primary (so the replica must be the same engine). Caller is - * responsible for ensuring {@link DatasourceConnectionDescriptor#hasReadReplica()} is true. + * Build a Hikari pool against one read-replica endpoint of the descriptor (AF-457). Reuses + * the same driver class as the primary (so the replica must be the same engine); the + * endpoint's username/password fall back to the primary's when {@code null}. */ - HikariDataSource createReplicaPool(DatasourceConnectionDescriptor descriptor) { + HikariDataSource createReplicaPool(DatasourceConnectionDescriptor descriptor, + ReadReplicaEndpoint endpoint) { ResolvedDriver resolved = resolveDriver(descriptor); - String username = descriptor.readReplicaUsername() != null - ? descriptor.readReplicaUsername() + String username = endpoint.username() != null + ? endpoint.username() : descriptor.username(); - String encrypted = descriptor.readReplicaPasswordEncrypted() != null - ? descriptor.readReplicaPasswordEncrypted() + String encrypted = endpoint.passwordEncrypted() != null + ? endpoint.passwordEncrypted() : descriptor.passwordEncrypted(); + int position = descriptor.readReplicas().indexOf(endpoint); return buildPool( descriptor, resolved, - descriptor.id() + "-replica", - descriptor.readReplicaJdbcUrl(), + descriptor.id() + "-replica-" + (position >= 0 ? position : endpoint.id()), + endpoint.jdbcUrl(), username, encrypted, descriptor.connectionPoolSize()); diff --git a/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/DefaultDatasourceConnectionPoolManager.java b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/DefaultDatasourceConnectionPoolManager.java index 72afdf62..f689ce16 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/DefaultDatasourceConnectionPoolManager.java +++ b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/DefaultDatasourceConnectionPoolManager.java @@ -1,10 +1,12 @@ package com.bablsoft.accessflow.proxy.internal; +import com.bablsoft.accessflow.core.api.DatasourceConnectionDescriptor; import com.bablsoft.accessflow.core.api.DatasourceLookupService; import com.bablsoft.accessflow.proxy.api.DatasourceConnectionPoolManager; import com.bablsoft.accessflow.proxy.api.DatasourcePoolStats; import com.bablsoft.accessflow.proxy.api.DatasourceUnavailableException; import com.bablsoft.accessflow.proxy.api.PoolInitializationException; +import com.bablsoft.accessflow.proxy.api.ReplicaEndpointRef; import com.zaxxer.hikari.HikariDataSource; import jakarta.annotation.PreDestroy; import lombok.RequiredArgsConstructor; @@ -15,6 +17,7 @@ import org.springframework.stereotype.Service; import javax.sql.DataSource; +import java.util.List; import java.util.Optional; import java.util.UUID; import java.util.concurrent.ConcurrentHashMap; @@ -32,7 +35,9 @@ class DefaultDatasourceConnectionPoolManager implements DatasourceConnectionPool private final MessageSource messageSource; private final ConcurrentMap pools = new ConcurrentHashMap<>(); - private final ConcurrentMap replicaPools = new ConcurrentHashMap<>(); + /** Per-datasource map of replica pools keyed by endpoint id (AF-457). */ + private final ConcurrentMap> replicaPools = + new ConcurrentHashMap<>(); private String msg(String key) { return messageSource.getMessage(key, null, LocaleContextHolder.getLocale()); @@ -60,33 +65,46 @@ public DataSource resolve(UUID datasourceId) { } @Override - public Optional resolveReplica(UUID datasourceId) { + public List replicaEndpoints(UUID datasourceId) { + var descriptor = loadActiveDescriptor(datasourceId); + return descriptor.readReplicas().stream() + .map(endpoint -> new ReplicaEndpointRef(endpoint.id(), label(endpoint.jdbcUrl()))) + .toList(); + } + + @Override + public DataSource resolveReplica(UUID datasourceId, UUID endpointId) { var cached = replicaPools.get(datasourceId); - if (cached != null && !cached.isClosed()) { - return Optional.of(cached); + if (cached != null) { + var pool = cached.get(endpointId); + if (pool != null && !pool.isClosed()) { + return pool; + } } var descriptor = loadActiveDescriptor(datasourceId); - if (!descriptor.hasReadReplica()) { - return Optional.empty(); - } - var pool = replicaPools.compute(datasourceId, (id, current) -> { + var endpoint = descriptor.readReplicas().stream() + .filter(candidate -> candidate.id().equals(endpointId)) + .findFirst() + .orElseThrow(() -> new DatasourceUnavailableException( + msg("error.datasource_unavailable_not_found"))); + var byEndpoint = replicaPools.computeIfAbsent(datasourceId, + id -> new ConcurrentHashMap<>()); + return byEndpoint.compute(endpointId, (id, current) -> { if (current != null && !current.isClosed()) { return current; } try { - var created = poolFactory.createReplicaPool(descriptor); + var created = poolFactory.createReplicaPool(descriptor, endpoint); log.info("Created Hikari replica pool {} for datasource {}", - created.getPoolName(), id); + created.getPoolName(), datasourceId); return created; } catch (RuntimeException ex) { throw new PoolInitializationException(msg("error.pool_initialization_failed"), ex); } }); - return Optional.of(pool); } - private com.bablsoft.accessflow.core.api.DatasourceConnectionDescriptor loadActiveDescriptor( - UUID datasourceId) { + private DatasourceConnectionDescriptor loadActiveDescriptor(UUID datasourceId) { var descriptor = datasourceLookupService.findById(datasourceId) .orElseThrow(() -> new DatasourceUnavailableException( msg("error.datasource_unavailable_not_found"))); @@ -98,13 +116,35 @@ private com.bablsoft.accessflow.core.api.DatasourceConnectionDescriptor loadActi @Override public void evict(UUID datasourceId) { - closeAndRemove(pools, datasourceId, "primary"); - closeAndRemove(replicaPools, datasourceId, "replica"); + var removed = pools.remove(datasourceId); + if (removed != null && !removed.isClosed()) { + log.info("Evicting Hikari primary pool for datasource {}", datasourceId); + removed.close(); + } + var removedReplicas = replicaPools.remove(datasourceId); + if (removedReplicas != null) { + removedReplicas.forEach((endpointId, pool) -> { + if (!pool.isClosed()) { + log.info("Evicting Hikari replica pool for datasource {} endpoint {}", + datasourceId, endpointId); + pool.close(); + } + }); + } } @Override public Optional poolStats(UUID datasourceId) { - var pool = pools.get(datasourceId); + return toStats(pools.get(datasourceId)); + } + + @Override + public Optional replicaPoolStats(UUID datasourceId, UUID endpointId) { + var byEndpoint = replicaPools.get(datasourceId); + return toStats(byEndpoint == null ? null : byEndpoint.get(endpointId)); + } + + private static Optional toStats(HikariDataSource pool) { if (pool == null || pool.isClosed()) { return Optional.empty(); } @@ -117,27 +157,63 @@ public Optional poolStats(UUID datasourceId) { pool.getMaximumPoolSize())); } - private static void closeAndRemove(ConcurrentMap map, UUID id, - String label) { - var removed = map.remove(id); - if (removed != null && !removed.isClosed()) { - log.info("Evicting Hikari {} pool for datasource {}", label, id); - removed.close(); + /** + * Visits every currently-cached, open replica pool. Used by {@link ReplicaHealthProber} to + * probe only endpoints that have actually served traffic on this node — probing never creates + * pools (same rule as {@link #poolStats(UUID)}). + */ + void forEachCachedReplicaPool(ReplicaPoolVisitor visitor) { + replicaPools.forEach((datasourceId, byEndpoint) -> + byEndpoint.forEach((endpointId, pool) -> { + if (!pool.isClosed()) { + visitor.visit(datasourceId, endpointId, pool); + } + })); + } + + @FunctionalInterface + interface ReplicaPoolVisitor { + void visit(UUID datasourceId, UUID endpointId, DataSource pool); + } + + /** + * Redacted display form of a replica JDBC URL: everything between the authority separator and + * the first path/query delimiter (host[:port]), with any {@code user@} userinfo dropped. Never + * exposes credentials or query parameters. + */ + static String label(String url) { + int authorityStart = url.indexOf("//"); + if (authorityStart < 0) { + int colon = url.lastIndexOf(':'); + return colon > 0 ? url.substring(0, colon) : url; + } + int start = authorityStart + 2; + int end = url.length(); + for (int i = start; i < url.length(); i++) { + char c = url.charAt(i); + if (c == '/' || c == '?' || c == ';') { + end = i; + break; + } } + String authority = url.substring(start, end); + int at = authority.lastIndexOf('@'); + return at >= 0 ? authority.substring(at + 1) : authority; } @PreDestroy void shutdown() { - closeAll(pools); - closeAll(replicaPools); - } - - private static void closeAll(ConcurrentMap map) { - map.forEach((id, pool) -> { + pools.forEach((id, pool) -> { if (!pool.isClosed()) { pool.close(); } }); - map.clear(); + pools.clear(); + replicaPools.forEach((id, byEndpoint) -> byEndpoint.forEach((endpointId, pool) -> { + if (!pool.isClosed()) { + pool.close(); + } + })); + replicaPools.clear(); } } diff --git a/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/DefaultDatasourceHealthService.java b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/DefaultDatasourceHealthService.java index 80e0f2ca..fb4015dc 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/DefaultDatasourceHealthService.java +++ b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/DefaultDatasourceHealthService.java @@ -10,6 +10,7 @@ import com.bablsoft.accessflow.proxy.api.DatasourceHealthService; import com.bablsoft.accessflow.proxy.api.DatasourceHealthSnapshot; import com.bablsoft.accessflow.proxy.api.DatasourcePoolStats; +import com.bablsoft.accessflow.proxy.api.ReplicaEndpointHealth; import org.springframework.cache.CacheManager; import org.springframework.stereotype.Service; @@ -29,17 +30,20 @@ class DefaultDatasourceHealthService implements DatasourceHealthService { private final DatasourceAdminService datasourceAdminService; private final DatasourceConnectionPoolManager poolManager; private final DatasourceQueryStatsLookupService queryStatsLookupService; + private final ReplicaHealthRegistry replicaHealthRegistry; private final Clock clock; private final CacheManager cacheManager; DefaultDatasourceHealthService(DatasourceAdminService datasourceAdminService, DatasourceConnectionPoolManager poolManager, DatasourceQueryStatsLookupService queryStatsLookupService, + ReplicaHealthRegistry replicaHealthRegistry, Clock proxyClock, CacheManager cacheManager) { this.datasourceAdminService = datasourceAdminService; this.poolManager = poolManager; this.queryStatsLookupService = queryStatsLookupService; + this.replicaHealthRegistry = replicaHealthRegistry; this.clock = proxyClock; this.cacheManager = cacheManager; } @@ -84,8 +88,8 @@ public PageResponse snapshot(UUID organizationId, page.totalElements(), page.totalPages()); } - private static DatasourceHealthSnapshot toSnapshot(DatasourceView view, DatasourcePoolStats pool, - DatasourceQueryStats stats) { + private DatasourceHealthSnapshot toSnapshot(DatasourceView view, DatasourcePoolStats pool, + DatasourceQueryStats stats) { return new DatasourceHealthSnapshot( view.id(), view.name(), @@ -99,7 +103,22 @@ private static DatasourceHealthSnapshot toSnapshot(DatasourceView view, Datasour stats.queriesLast24h(), stats.executionMsP50(), stats.executionMsP95(), - stats.errorsLast24h()); + stats.errorsLast24h(), + replicaHealth(view)); + } + + private List replicaHealth(DatasourceView view) { + return view.readReplicas().stream() + .map(replica -> { + var pool = poolManager.replicaPoolStats(view.id(), replica.id()).orElse(null); + return new ReplicaEndpointHealth( + replica.id(), + DefaultDatasourceConnectionPoolManager.label(replica.jdbcUrl()), + replicaHealthRegistry.isHealthy(view.id(), replica.id()), + pool == null ? null : pool.active(), + pool == null ? null : pool.total()); + }) + .toList(); } private record HealthKey(UUID organizationId, UUID datasourceId) { diff --git a/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/DefaultQueryExecutor.java b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/DefaultQueryExecutor.java index a8984476..923f8090 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/DefaultQueryExecutor.java +++ b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/DefaultQueryExecutor.java @@ -51,6 +51,7 @@ class DefaultQueryExecutor implements QueryExecutor { private final RowSecurityRewriter rowSecurityRewriter; private final QueryEngineCatalog engineCatalog; private final DryRunPlannerRegistry dryRunPlannerRegistry; + private final SelectResultCache resultCache; private final Clock clock; private final MessageSource messageSource; private final ObservationRegistry observationRegistry; @@ -79,7 +80,7 @@ public QueryExecutionResult execute(QueryExecutionRequest request) { .start(); try (Observation.Scope ignored = observation.openScope()) { QueryExecutionResult result = executeInternal(request, descriptor, - effectiveMaxRows, effectiveTimeout, execProps); + effectiveMaxRows, effectiveTimeout, execProps, observation); observation.lowCardinalityKeyValue("outcome", "success"); return result; } catch (RuntimeException ex) { @@ -94,7 +95,8 @@ public QueryExecutionResult execute(QueryExecutionRequest request) { private QueryExecutionResult executeInternal(QueryExecutionRequest request, DatasourceConnectionDescriptor descriptor, int effectiveMaxRows, Duration effectiveTimeout, - ProxyPoolProperties.Execution execProps) { + ProxyPoolProperties.Execution execProps, + Observation observation) { if (engineCatalog.isEngineManaged(descriptor.dbType())) { return engineCatalog.engineFor(descriptor.dbType()) .execute(new QueryEngineExecutionRequest( @@ -103,10 +105,30 @@ private QueryExecutionResult executeInternal(QueryExecutionRequest request, Instant start = clock.instant(); if (request.transactional()) { - return executeTransactional(request, effectiveTimeout, start); + var result = executeTransactional(request, descriptor.dbType(), effectiveTimeout, + start); + resultCache.invalidateTables(request.datasourceId(), request.referencedTables()); + return result; } var rewrite = rowSecurityRewriter.rewrite(request.sql(), request.rowSecurityPredicates(), request.softDeleteDirectives()); + // SELECT result cache (AF-457): keyed over the RLS-rewritten SQL + binds + mask/restriction + // directives + row cap, so security scope is part of the key. SELECTs whose referenced + // tables are unknown are never cached (no write-invalidation coverage). + boolean cacheable = request.queryType() == QueryType.SELECT + && !request.referencedTables().isEmpty() + && resultCache.enabledFor(descriptor); + String cacheKey = null; + if (cacheable) { + cacheKey = SelectResultCache.cacheKey(rewrite.sql(), rewrite.binds(), + request.restrictedColumns(), request.columnMasks(), effectiveMaxRows); + var hit = resultCache.get(request.datasourceId(), cacheKey, durationSince(start)); + if (hit.isPresent()) { + observation.lowCardinalityKeyValue("cache", "hit"); + return hit.get(); + } + } + observation.lowCardinalityKeyValue("cache", cacheable ? "miss" : "off"); try (Connection connection = routingResolver.acquire(request.datasourceId(), request.queryType())) { connection.setReadOnly(request.queryType() == QueryType.SELECT); @@ -115,11 +137,20 @@ private QueryExecutionResult executeInternal(QueryExecutionRequest request, statement.setFetchSize(Math.min(effectiveMaxRows + 1, execProps.defaultFetchSize())); bind(statement, rewrite.binds()); if (request.queryType() == QueryType.SELECT) { - return runSelect(statement, effectiveMaxRows, descriptor.dbType(), start, + var result = runSelect(statement, effectiveMaxRows, descriptor.dbType(), start, request.restrictedColumns(), request.columnMasks(), rewrite.appliedPolicyIds()); + if (cacheable && result instanceof SelectExecutionResult select) { + resultCache.put(request.datasourceId(), cacheKey, + request.referencedTables(), resultCache.ttlFor(descriptor), select); + } + return result; } - return runUpdate(statement, start, rewrite.appliedPolicyIds()); + var result = runUpdate(statement, start, rewrite.appliedPolicyIds()); + // Any successful write drops cached SELECTs over the touched tables (unknown + // tables ⇒ full-datasource purge, fail-safe for DDL). + resultCache.invalidateTables(request.datasourceId(), request.referencedTables()); + return result; } } catch (SQLException ex) { log.debug("SQL execution failed for datasource {}: {}", @@ -199,24 +230,38 @@ public QueryDryRunResult dryRun(QueryExecutionRequest request) { } } - private QueryExecutionResult executeTransactional(QueryExecutionRequest request, + private QueryExecutionResult executeTransactional(QueryExecutionRequest request, DbType dbType, Duration effectiveTimeout, Instant start) { var appliedPolicyIds = new java.util.LinkedHashSet(); + // Rewrite every statement first; only rewrite-no-op statements (no RLS/soft-delete binds, + // SQL unchanged) are candidates for INSERT batching (AF-457). + var statements = request.statements(); + var rewrites = new RowSecurityRewriter.RewriteResult[statements.size()]; + var batchable = new boolean[statements.size()]; + for (int i = 0; i < statements.size(); i++) { + rewrites[i] = rowSecurityRewriter.rewrite(statements.get(i), + request.rowSecurityPredicates(), request.softDeleteDirectives()); + appliedPolicyIds.addAll(rewrites[i].appliedPolicyIds()); + batchable[i] = rewrites[i].binds().isEmpty() + && rewrites[i].sql().equals(statements.get(i)); + } + var steps = BatchInsertPlanner.plan(statements, batchable); + int chunkSize = properties.execution().insertBatchChunkSize(); try (Connection connection = routingResolver.acquire(request.datasourceId(), QueryType.OTHER)) { connection.setReadOnly(false); connection.setAutoCommit(false); long totalAffected = 0; try { - for (String stmtSql : request.statements()) { - var rewrite = rowSecurityRewriter.rewrite(stmtSql, - request.rowSecurityPredicates(), request.softDeleteDirectives()); - appliedPolicyIds.addAll(rewrite.appliedPolicyIds()); - try (PreparedStatement statement = connection.prepareStatement(rewrite.sql())) { - statement.setQueryTimeout(toTimeoutSeconds(effectiveTimeout)); - bind(statement, rewrite.binds()); - totalAffected += statement.executeLargeUpdate(); - } + for (var step : steps) { + totalAffected += switch (step) { + case BatchInsertPlanner.SingleStep single -> + runTransactionalStatement(connection, + rewrites[single.statementIndex()], effectiveTimeout); + case BatchInsertPlanner.BatchStep batch -> + runInsertBatch(connection, batch, dbType, effectiveTimeout, + chunkSize); + }; } connection.commit(); } catch (SQLException ex) { @@ -235,6 +280,70 @@ private QueryExecutionResult executeTransactional(QueryExecutionRequest request, } } + private long runTransactionalStatement(Connection connection, + RowSecurityRewriter.RewriteResult rewrite, + Duration effectiveTimeout) throws SQLException { + try (PreparedStatement statement = connection.prepareStatement(rewrite.sql())) { + statement.setQueryTimeout(toTimeoutSeconds(effectiveTimeout)); + bind(statement, rewrite.binds()); + return statement.executeLargeUpdate(); + } + } + + /** + * Executes one homogeneous INSERT run as a single {@code PreparedStatement} with + * {@code addBatch()} per row, flushing {@code executeLargeBatch()} every {@code chunkSize} + * rows. {@code SUCCESS_NO_INFO} counts as one affected row. + */ + private long runInsertBatch(Connection connection, BatchInsertPlanner.BatchStep batch, + DbType dbType, Duration effectiveTimeout, int chunkSize) + throws SQLException { + long affected = 0; + try (PreparedStatement statement = connection.prepareStatement(batch.templateSql())) { + statement.setQueryTimeout(toTimeoutSeconds(effectiveTimeout)); + int pending = 0; + for (var row : batch.rowBinds()) { + bindBatchRow(statement, row, dbType); + statement.addBatch(); + pending++; + if (pending >= chunkSize) { + affected += sumBatchCounts(statement.executeLargeBatch()); + pending = 0; + } + } + if (pending > 0) { + affected += sumBatchCounts(statement.executeLargeBatch()); + } + } + return affected; + } + + /** + * Binds one batched-INSERT row. String literals came out of SQL text where the server infers + * the column type; PostgreSQL types a plain {@code setString} bind as {@code varchar} and then + * rejects it against uuid/jsonb/enum columns (42804), so PG strings are sent with an + * unspecified type ({@code Types.OTHER}) to preserve literal semantics. + */ + private static void bindBatchRow(PreparedStatement statement, List row, DbType dbType) + throws SQLException { + for (int i = 0; i < row.size(); i++) { + Object value = row.get(i); + if (value instanceof String text && dbType == DbType.POSTGRESQL) { + statement.setObject(i + 1, text, java.sql.Types.OTHER); + } else { + statement.setObject(i + 1, value); + } + } + } + + private static long sumBatchCounts(long[] counts) { + long sum = 0; + for (long count : counts) { + sum += count == java.sql.Statement.SUCCESS_NO_INFO ? 1 : Math.max(count, 0); + } + return sum; + } + private QueryExecutionResult runSelect(PreparedStatement statement, int effectiveMaxRows, DbType dbType, Instant start, List restrictedColumns, diff --git a/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ProxyCacheProperties.java b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ProxyCacheProperties.java new file mode 100644 index 00000000..6d13f13a --- /dev/null +++ b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ProxyCacheProperties.java @@ -0,0 +1,27 @@ +package com.bablsoft.accessflow.proxy.internal; + +import org.springframework.boot.context.properties.ConfigurationProperties; + +import java.time.Duration; + +/** + * SELECT result-cache tuning (AF-457). {@code enabled} is the deployment-wide kill-switch (the + * real gate is each datasource's {@code result_cache_enabled} opt-in); {@code defaultTtl} applies + * when a datasource opts in without its own {@code result_cache_ttl_seconds}; entries whose + * serialized form exceeds {@code maxEntryBytes} are not cached. + */ +@ConfigurationProperties("accessflow.proxy.cache") +record ProxyCacheProperties(Boolean enabled, Duration defaultTtl, Long maxEntryBytes) { + + ProxyCacheProperties { + if (enabled == null) { + enabled = true; + } + if (defaultTtl == null) { + defaultTtl = Duration.ofSeconds(60); + } + if (maxEntryBytes == null) { + maxEntryBytes = 1_000_000L; + } + } +} diff --git a/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ProxyConfiguration.java b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ProxyConfiguration.java index 6434da07..b6c407ae 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ProxyConfiguration.java +++ b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ProxyConfiguration.java @@ -12,7 +12,8 @@ @Configuration(proxyBeanMethods = false) @EnableConfigurationProperties({ProxyPoolProperties.class, DriverProperties.class, - ProxyHealthProperties.class, EngineConfigProperties.class}) + ProxyHealthProperties.class, EngineConfigProperties.class, + ProxyReplicaProperties.class, ProxyCacheProperties.class}) class ProxyConfiguration { /** Spring cache holding per-{@code (organizationId, datasourceId)} health snapshots. */ diff --git a/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ProxyPoolProperties.java b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ProxyPoolProperties.java index fb8c3be6..8a2e3bfb 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ProxyPoolProperties.java +++ b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ProxyPoolProperties.java @@ -30,11 +30,12 @@ record ProxyPoolProperties( poolNamePrefix = "accessflow-ds-"; } if (execution == null) { - execution = new Execution(null, null, null); + execution = new Execution(null, null, null, null); } } - record Execution(Integer maxRows, Duration statementTimeout, Integer defaultFetchSize) { + record Execution(Integer maxRows, Duration statementTimeout, Integer defaultFetchSize, + Integer insertBatchChunkSize) { Execution { if (maxRows == null) { @@ -46,6 +47,11 @@ record Execution(Integer maxRows, Duration statementTimeout, Integer defaultFetc if (defaultFetchSize == null) { defaultFetchSize = 1_000; } + if (insertBatchChunkSize == null) { + insertBatchChunkSize = 1_000; + } } + // NOTE: no overloaded constructors here — @ConfigurationProperties record binding + // silently degrades to all-defaults when the record has more than one constructor. } } diff --git a/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ProxyReplicaProperties.java b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ProxyReplicaProperties.java new file mode 100644 index 00000000..e83cdda9 --- /dev/null +++ b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ProxyReplicaProperties.java @@ -0,0 +1,27 @@ +package com.bablsoft.accessflow.proxy.internal; + +import org.springframework.boot.context.properties.ConfigurationProperties; + +import java.time.Duration; + +/** + * Read-replica health-check tuning (AF-457). {@code probeInterval} is the cadence of the per-node + * background prober, {@code probeTimeout} the JDBC {@code isValid} timeout per endpoint, and + * {@code cooldown} how long a failed endpoint sits out of the read rotation before a half-open + * retry. + */ +@ConfigurationProperties("accessflow.proxy.replica") +record ProxyReplicaProperties(Duration probeInterval, Duration probeTimeout, Duration cooldown) { + + ProxyReplicaProperties { + if (probeInterval == null) { + probeInterval = Duration.ofSeconds(30); + } + if (probeTimeout == null) { + probeTimeout = Duration.ofSeconds(5); + } + if (cooldown == null) { + cooldown = Duration.ofSeconds(30); + } + } +} diff --git a/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ReplicaHealthProber.java b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ReplicaHealthProber.java new file mode 100644 index 00000000..0802f70e --- /dev/null +++ b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ReplicaHealthProber.java @@ -0,0 +1,82 @@ +package com.bablsoft.accessflow.proxy.internal; + +import jakarta.annotation.PostConstruct; +import jakarta.annotation.PreDestroy; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.scheduling.TaskScheduler; +import org.springframework.stereotype.Component; + +import java.sql.SQLException; +import java.util.concurrent.ScheduledFuture; + +/** + * Per-node async health prober for read-replica endpoints (AF-457). Every + * {@code accessflow.proxy.replica.probe-interval} it runs {@code Connection.isValid(...)} against + * each replica pool already cached on this node (probing never creates pools) and feeds + * the result into {@link ReplicaHealthRegistry} — so a recovered replica re-enters the read + * rotation without waiting for a half-open trial, and a silently-degraded one is taken out before + * a user query has to fail on it. + * + *

Deliberately NOT a {@code @Scheduled}/{@code @SchedulerLock} job: ShedLock makes a job a + * cluster singleton, but HikariCP pools and the breaker state are per JVM — every node must probe + * its own pools. The Boot-autoconfigured {@link TaskScheduler} runs the probe on virtual threads + * ({@code spring.threads.virtual.enabled=true}), so no platform thread is tied up. + */ +@Component +class ReplicaHealthProber { + + private static final Logger log = LoggerFactory.getLogger(ReplicaHealthProber.class); + + private final DefaultDatasourceConnectionPoolManager poolManager; + private final ReplicaHealthRegistry healthRegistry; + private final ProxyReplicaProperties properties; + private final TaskScheduler taskScheduler; + + private volatile ScheduledFuture scheduled; + + ReplicaHealthProber(DefaultDatasourceConnectionPoolManager poolManager, + ReplicaHealthRegistry healthRegistry, + ProxyReplicaProperties properties, + TaskScheduler taskScheduler) { + this.poolManager = poolManager; + this.healthRegistry = healthRegistry; + this.properties = properties; + this.taskScheduler = taskScheduler; + } + + @PostConstruct + void start() { + scheduled = taskScheduler.scheduleWithFixedDelay(this::probeAll, + properties.probeInterval()); + } + + @PreDestroy + void stop() { + var current = scheduled; + if (current != null) { + current.cancel(false); + } + } + + void probeAll() { + poolManager.forEachCachedReplicaPool((datasourceId, endpointId, pool) -> { + try (var connection = pool.getConnection()) { + if (connection.isValid((int) properties.probeTimeout().toSeconds())) { + healthRegistry.recordSuccess(datasourceId, endpointId); + } else { + markDown(datasourceId, endpointId, "isValid returned false"); + } + } catch (SQLException | RuntimeException ex) { + markDown(datasourceId, endpointId, + ex.getMessage() != null ? ex.getMessage() : ex.getClass().getName()); + } + }); + } + + private void markDown(java.util.UUID datasourceId, java.util.UUID endpointId, String reason) { + log.warn("Replica health probe failed for datasource {} endpoint {}: {}", + datasourceId, endpointId, reason); + healthRegistry.recordFailure(datasourceId, endpointId); + } +} diff --git a/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ReplicaHealthRegistry.java b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ReplicaHealthRegistry.java new file mode 100644 index 00000000..c718c1b7 --- /dev/null +++ b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ReplicaHealthRegistry.java @@ -0,0 +1,76 @@ +package com.bablsoft.accessflow.proxy.internal; + +import org.springframework.stereotype.Component; + +import java.time.Clock; +import java.time.Instant; +import java.util.UUID; +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.ConcurrentMap; + +/** + * Per-node circuit breaker over read-replica endpoints (AF-457). An endpoint with no recorded + * failure is HEALTHY; a failure marks it DOWN for {@code accessflow.proxy.replica.cooldown}, + * during which the read load-balancer skips it. Once the cooldown elapses, exactly one caller is + * admitted as a HALF_OPEN trial — success restores HEALTHY, failure re-arms the cooldown. + * + *

State is deliberately in-memory and per JVM: it mirrors this node's HikariCP pools, which + * are also per JVM. Each cluster node discovers replica outages (and recoveries) independently. + */ +@Component +class ReplicaHealthRegistry { + + private final Clock clock; + private final ProxyReplicaProperties properties; + private final ConcurrentMap states = new ConcurrentHashMap<>(); + + ReplicaHealthRegistry(Clock proxyClock, ProxyReplicaProperties properties) { + this.clock = proxyClock; + this.properties = properties; + } + + private record Key(UUID datasourceId, UUID endpointId) { + } + + private record State(boolean halfOpen, Instant downUntil) { + } + + /** + * Whether the load-balancer may try this endpoint now. HEALTHY endpoints always qualify; a + * DOWN endpoint qualifies only once its cooldown has elapsed, and then admits a single + * half-open trial (concurrent callers are turned away until the trial resolves). + */ + boolean isCandidate(UUID datasourceId, UUID endpointId) { + var key = new Key(datasourceId, endpointId); + var state = states.get(key); + if (state == null) { + return true; + } + if (state.halfOpen()) { + return false; + } + if (clock.instant().isBefore(state.downUntil())) { + return false; + } + return states.replace(key, state, new State(true, state.downUntil())); + } + + /** Whether the endpoint is currently in rotation (no open circuit), for health reporting. */ + boolean isHealthy(UUID datasourceId, UUID endpointId) { + return states.get(new Key(datasourceId, endpointId)) == null; + } + + void recordSuccess(UUID datasourceId, UUID endpointId) { + states.remove(new Key(datasourceId, endpointId)); + } + + void recordFailure(UUID datasourceId, UUID endpointId) { + states.put(new Key(datasourceId, endpointId), + new State(false, clock.instant().plus(properties.cooldown()))); + } + + /** Drops all endpoint state for a datasource (pool eviction / config change). */ + void evict(UUID datasourceId) { + states.keySet().removeIf(key -> key.datasourceId().equals(datasourceId)); + } +} diff --git a/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ResultCacheEvictionListener.java b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ResultCacheEvictionListener.java new file mode 100644 index 00000000..e1d85978 --- /dev/null +++ b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/ResultCacheEvictionListener.java @@ -0,0 +1,35 @@ +package com.bablsoft.accessflow.proxy.internal; + +import com.bablsoft.accessflow.core.events.DatasourceCacheConfigChangedEvent; +import com.bablsoft.accessflow.core.events.DatasourceConfigChangedEvent; +import com.bablsoft.accessflow.core.events.DatasourceDeactivatedEvent; +import lombok.RequiredArgsConstructor; +import org.springframework.modulith.events.ApplicationModuleListener; +import org.springframework.stereotype.Component; + +/** + * Purges the SELECT result cache for a datasource whenever its connection config changes, it is + * deactivated, or its cache settings change (AF-457). Sibling of + * {@link DatasourcePoolEvictionListener} — cache-setting changes deliberately do NOT evict pools. + */ +@Component +@RequiredArgsConstructor +class ResultCacheEvictionListener { + + private final SelectResultCache resultCache; + + @ApplicationModuleListener + void onConfigChanged(DatasourceConfigChangedEvent event) { + resultCache.invalidateAll(event.datasourceId()); + } + + @ApplicationModuleListener + void onDeactivated(DatasourceDeactivatedEvent event) { + resultCache.invalidateAll(event.datasourceId()); + } + + @ApplicationModuleListener + void onCacheConfigChanged(DatasourceCacheConfigChangedEvent event) { + resultCache.invalidateAll(event.datasourceId()); + } +} diff --git a/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/RoutingDataSourceResolver.java b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/RoutingDataSourceResolver.java index b015216d..945ffcbf 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/RoutingDataSourceResolver.java +++ b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/RoutingDataSourceResolver.java @@ -8,7 +8,7 @@ import com.bablsoft.accessflow.core.api.QueryType; import com.bablsoft.accessflow.proxy.api.DatasourceConnectionPoolManager; import com.bablsoft.accessflow.proxy.api.DatasourceUnavailableException; -import com.bablsoft.accessflow.proxy.api.PoolInitializationException; +import com.bablsoft.accessflow.proxy.api.ReplicaEndpointRef; import io.micrometer.observation.Observation; import io.micrometer.observation.ObservationRegistry; import lombok.RequiredArgsConstructor; @@ -20,15 +20,27 @@ import java.sql.Connection; import java.sql.SQLException; +import java.util.ArrayList; import java.util.HashMap; +import java.util.List; import java.util.UUID; +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.ConcurrentMap; +import java.util.concurrent.atomic.AtomicInteger; /** - * Picks the right HikariCP pool for a query: {@link QueryType#SELECT} goes to the read replica - * when one is configured, with fall-back to the primary on connection failure. All non-SELECT - * statements (DML/DDL and transactional batches) always hit the primary. Replica fall-back emits - * a {@link AuditAction#DATASOURCE_REPLICA_FALLBACK} audit row but never propagates the underlying - * failure to the caller — the query still runs against the primary. + * Picks the right HikariCP pool for a query. {@link QueryType#SELECT} load-balances round-robin + * across the datasource's healthy read-replica endpoints (AF-457): endpoints with an open circuit + * in {@link ReplicaHealthRegistry} are skipped, a connection failure marks the endpoint down and + * moves on to the next, and only when every endpoint is down or failed does the read fall back to + * the primary. All non-SELECT statements (DML/DDL and transactional batches) always hit the + * primary. + * + *

Replica fall-back emits one {@link AuditAction#DATASOURCE_REPLICA_FALLBACK} audit row per + * replica-set exhaustion with at least one live failure — not per endpoint, and not for + * SELECTs served by the primary while every circuit is already open (those were audited when they + * failed). The underlying failure never propagates to the caller — the query still runs against + * the primary. */ @Component @RequiredArgsConstructor @@ -37,11 +49,14 @@ class RoutingDataSourceResolver { private static final Logger log = LoggerFactory.getLogger(RoutingDataSourceResolver.class); private final DatasourceConnectionPoolManager poolManager; + private final ReplicaHealthRegistry healthRegistry; private final DatasourceLookupService datasourceLookupService; private final AuditLogService auditLogService; private final MessageSource messageSource; private final ObservationRegistry observationRegistry; + private final ConcurrentMap roundRobinOffsets = new ConcurrentHashMap<>(); + /** * Acquires a pooled connection, traced as {@code accessflow.datasource.acquire} (AF-454) so the * pool-acquire latency is a child span of the surrounding {@code accessflow.query.execute}. @@ -65,24 +80,61 @@ Connection acquire(UUID datasourceId, QueryType queryType) throws SQLException { private Connection doAcquire(UUID datasourceId, QueryType queryType) throws SQLException { if (queryType == QueryType.SELECT) { - try { - var replica = poolManager.resolveReplica(datasourceId); - if (replica.isPresent()) { - try { - return replica.get().getConnection(); - } catch (SQLException | RuntimeException ex) { - recordFallback(datasourceId, ex); - } - } - } catch (PoolInitializationException ex) { - recordFallback(datasourceId, ex); + var replica = tryReplicas(datasourceId); + if (replica != null) { + return replica; } } return poolManager.resolve(datasourceId).getConnection(); } - private void recordFallback(UUID datasourceId, Throwable ex) { - log.warn("Read-replica connection failed for datasource {}; falling back to primary: {}", + /** + * Round-robin over the healthy replica endpoints; {@code null} when the datasource has no + * replicas or none could serve the read (the caller then uses the primary). + */ + private Connection tryReplicas(UUID datasourceId) { + List endpoints = poolManager.replicaEndpoints(datasourceId); + if (endpoints.isEmpty()) { + return null; + } + int size = endpoints.size(); + int start = Math.floorMod( + roundRobinOffsets.computeIfAbsent(datasourceId, id -> new AtomicInteger()) + .getAndIncrement(), + size); + var triedLabels = new ArrayList(); + Throwable lastFailure = null; + for (int i = 0; i < size; i++) { + var endpoint = endpoints.get((start + i) % size); + if (!healthRegistry.isCandidate(datasourceId, endpoint.endpointId())) { + continue; + } + triedLabels.add(endpoint.label()); + try { + Connection connection = poolManager + .resolveReplica(datasourceId, endpoint.endpointId()) + .getConnection(); + healthRegistry.recordSuccess(datasourceId, endpoint.endpointId()); + return connection; + } catch (SQLException | RuntimeException ex) { + healthRegistry.recordFailure(datasourceId, endpoint.endpointId()); + log.warn("Read-replica {} failed for datasource {}; trying next candidate: {}", + endpoint.label(), datasourceId, ex.getMessage()); + lastFailure = ex; + } + } + if (lastFailure != null) { + recordFallback(datasourceId, lastFailure, size, triedLabels); + } else { + log.debug("All {} replica endpoints of datasource {} have open circuits; " + + "serving SELECT from primary", size, datasourceId); + } + return null; + } + + private void recordFallback(UUID datasourceId, Throwable ex, int replicaCount, + List triedLabels) { + log.warn("All read-replica endpoints failed for datasource {}; falling back to primary: {}", datasourceId, ex.getMessage()); try { var organizationId = datasourceLookupService.findById(datasourceId) @@ -93,6 +145,8 @@ private void recordFallback(UUID datasourceId, Throwable ex) { var metadata = new HashMap(); metadata.put("error", ex.getMessage() != null ? ex.getMessage() : ex.getClass().getName()); metadata.put("query_type", QueryType.SELECT.name()); + metadata.put("replica_count", replicaCount); + metadata.put("tried_endpoints", triedLabels); auditLogService.record(new AuditEntry( AuditAction.DATASOURCE_REPLICA_FALLBACK, AuditResourceType.DATASOURCE, diff --git a/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/SelectResultCache.java b/backend/src/main/java/com/bablsoft/accessflow/proxy/internal/SelectResultCache.java new file mode 100644 index 0000000000000000000000000000000000000000..4732bd32790d8301b595e8b3c7f87f3e87466488 GIT binary patch literal 8836 zcmcgyZFAek5$Wg|`Z z#cPp3*iC!mvdoJ4tP~5mR^`Vy#X;jzrkRe7=B5?Dzu*U#a`dQ>FsY#TQD!e$7i{n{_ zyF*K*tKk9{;^=EUW8#*A9?(a*0r~3<37Y;e8vfQ1Ea^Z0`ZsAs z)mjldT9PeAu4xU6fdE#Qa4+(9XZku(OB^;41r?Gd)^HUG4uqBECdL|GcCD}QyUoVRzlXQD{ZzycHeOl%#Scn-6nby8@pIPOXa*2j>7^1IJGkk9O$(Y-$K9=q9V2E5Se|%*3yUg{W{7b7FMS6cww*wJi03sk5gQbGXtqAV!Kp_l<~I z3ae)c?@_eA5fzQzO`xa@k8!-blp@{0q=`f{0DE|t^q0V#UKRO<*pCboS!Ph@KiTxk zh(xJc7vL?$f)Dxi!S^-j9VEHXnk;l*SlBlg^Zwv~@B??g1(-r|SSKNSBQ1bauIBWT z>=y^x@myB+L9|0MHmv%<2qxSjXyKcF9-X}(U%k5+zn=UG%DqP&wfEvdI+OF$@vl#% z!w&xltTkY*<6FsPW5A$SCd3kwuH9Z@h`~>m*t60sZGEqgUqwoIg749u*O=vaV z3YhSyR`?7hu$$g-Qe$oi{sIV886*dUxIe8y+I*UL2|VSn2{D^z@%1XpQ&~oR`&ZeuG(6cW_Mr&pc!H}sMROT5l0{0AVQYd7J28Ckp2e|pFOdU3bs)_0Yv7j8ukK7$L zk?a`hCTEijKEaP!XFlFue?TNVJ~nvQUX&|9H(>AtAXv3uIq;&YTonm~PTQ}M9vpEp za7-{bFf8R{7O?kDs;c~dm8TIl%!Wrl-2C{hPzPn3T=rKx< zyH?w54w4+I*pw(XRgk%(BW)MB8`ZJ7M-BFiSDQ*|WJT>xL;U{o_0`|{aW0EFiqGQ^ zT^HhR%*CbM*$)lkuO$0(Dz8`bjzgQYbk~(}BN8c74XC*rnl-~~7V8Be;3L^fd;2ZG z>_Y$V9J6unw(imboK8`q-pJxgE|TY z8fgB>+Ddx2cG);h%=8+MKp`T$n|zVp=SXt+RU1$S#|Q38<}9y|$G|8bT2 zg?sieX0*?WrUn|no>D-+xtM!5cq6$T9hJDYq0@l3-|oGrO>DACM9ff0uC{5l8@~#$ zcaqXq)I03q({JBvpSCbw@JEC(V7Et-R>SfENzC4s7-29Ad+eAw@I=XuLbxl8-lJ=% zU?P&sTY+f=MhA9G2dGD35YsEj@4MMNi@?kRojIBMcqX(Dfu@i9d!?)qwlA6?37VLi zJJ#5)G3`Y8!>#fXbU4iUQ3!^l=tvjOD#Ac-zChIVJU9>dScmq2AtM$W zQzgsDlQA=|m$uV@)=iIFIz8l`W~_-?;)G`awvlm;2`jo%aR1-g{dvW1(}tON+HDcT zyWLT9jPDpgfQ@yHXqRq`6OK{%MYhJHA++2*i+$D7P@p3F))=McR~v!F=ARg4SG3^i z9vJX%S5scdS#Wgo0y3k)IY-;Ys$_?jrAT~Rn*mJJ?O5k?`2Ir$#(y{%8 zILILQsk%CB!qmCZLnHb<{URPCu8pTm@?6eEKE+tA=`hu2LD7K!Q;_A{azU>Mji<&Y JK1JLf{1 select.rowCount(); case UpdateExecutionResult update -> update.rowsAffected(); diff --git a/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/DatasourceController.java b/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/DatasourceController.java index 9849f5f8..a9aae266 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/DatasourceController.java +++ b/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/DatasourceController.java @@ -16,6 +16,7 @@ import com.bablsoft.accessflow.core.api.DriverCatalogService; import com.bablsoft.accessflow.core.api.SecretResolutionService; import com.bablsoft.accessflow.core.api.IllegalDatasourceReviewerException; +import com.bablsoft.accessflow.core.api.ReplicaEndpointInput; import com.bablsoft.accessflow.core.api.TestReplicaCommand; import com.bablsoft.accessflow.core.api.UpdateDatasourceCommand; import com.bablsoft.accessflow.proxy.api.SampleDataService; @@ -36,6 +37,7 @@ import com.bablsoft.accessflow.security.internal.web.model.GroupPermissionResponse; import com.bablsoft.accessflow.security.internal.web.model.PermissionListResponse; import com.bablsoft.accessflow.security.internal.web.model.PermissionResponse; +import com.bablsoft.accessflow.security.internal.web.model.ReadReplicaRequest; import com.bablsoft.accessflow.security.internal.web.model.TestReplicaRequest; import com.bablsoft.accessflow.security.internal.web.model.SampleRowsResponse; import com.bablsoft.accessflow.security.internal.web.model.UpdateDatasourceRequest; @@ -65,6 +67,7 @@ import java.net.URI; import java.util.HashMap; +import java.util.List; import java.util.Map; import java.util.UUID; @@ -159,11 +162,11 @@ ResponseEntity createDatasource( request.customDriverId(), request.connectorId(), request.jdbcUrlOverride(), - request.readReplicaJdbcUrl(), - request.readReplicaUsername(), - request.readReplicaPassword(), + toReplicaInputs(request.readReplicas()), request.localDatacenter(), - request.apiKey()); + request.apiKey(), + request.resultCacheEnabled(), + request.resultCacheTtlSeconds()); var created = datasourceAdminService.create(command); recordAudit(AuditAction.DATASOURCE_CREATED, AuditResourceType.DATASOURCE, created.id(), caller, auditContext, Map.of("name", created.name(), "db_type", created.dbType().name())); @@ -216,12 +219,12 @@ DatasourceResponse updateDatasource(@PathVariable UUID id, request.textToSqlEnabled(), request.clearAiConfig(), request.jdbcUrlOverride(), - request.readReplicaJdbcUrl(), - request.readReplicaUsername(), - request.readReplicaPassword(), + toReplicaInputs(request.readReplicas()), request.active(), request.localDatacenter(), - request.apiKey()); + request.apiKey(), + request.resultCacheEnabled(), + request.resultCacheTtlSeconds()); var updated = datasourceAdminService.update(id, caller.organizationId(), command); recordAudit(AuditAction.DATASOURCE_UPDATED, AuditResourceType.DATASOURCE, id, caller, auditContext, Map.of("name", updated.name())); @@ -264,7 +267,7 @@ ConnectionTestResponse testReplicaConnection(@PathVariable UUID id, Authentication authentication) { var caller = currentClaims(authentication); var command = new TestReplicaCommand(request.jdbcUrl(), request.username(), - request.password()); + request.password(), request.replicaId()); return ConnectionTestResponse.from( datasourceAdminService.testReplica(id, caller.organizationId(), command)); } @@ -508,6 +511,15 @@ ResponseEntity removeReviewer(@PathVariable UUID id, @PathVariable UUID re return ResponseEntity.noContent().build(); } + private static List toReplicaInputs(List requests) { + if (requests == null) { + return null; + } + return requests.stream() + .map(r -> new ReplicaEndpointInput(r.id(), r.jdbcUrl(), r.username(), r.password())) + .toList(); + } + private JwtClaims currentClaims(Authentication authentication) { return (JwtClaims) authentication.getPrincipal(); } diff --git a/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/CreateDatasourceRequest.java b/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/CreateDatasourceRequest.java index 5985f85a..8e11dcc6 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/CreateDatasourceRequest.java +++ b/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/CreateDatasourceRequest.java @@ -2,6 +2,7 @@ import com.bablsoft.accessflow.core.api.DbType; import com.bablsoft.accessflow.core.api.SslMode; +import jakarta.validation.Valid; import jakarta.validation.constraints.Max; import jakarta.validation.constraints.Min; import jakarta.validation.constraints.NotBlank; @@ -9,6 +10,7 @@ import jakarta.validation.constraints.Pattern; import jakarta.validation.constraints.Size; +import java.util.List; import java.util.UUID; public record CreateDatasourceRequest( @@ -42,12 +44,13 @@ public record CreateDatasourceRequest( @Size(max = 2048, message = "{validation.jdbc_url.length}") @Pattern(regexp = "^jdbc:[a-zA-Z][a-zA-Z0-9+\\-.]*:.+$", message = "{validation.jdbc_url.format}") String jdbcUrlOverride, - @Size(max = 2048, message = "{validation.jdbc_url.length}") - @Pattern(regexp = "^jdbc:[a-zA-Z][a-zA-Z0-9+\\-.]*:.+$", - message = "{validation.jdbc_url.format}") String readReplicaJdbcUrl, - @Size(max = 255, message = "{validation.display_name.max}") String readReplicaUsername, - @Size(max = 4096) String readReplicaPassword, + @Size(max = 5, message = "{validation.read_replicas.max}") + List<@Valid ReadReplicaRequest> readReplicas, @Size(max = 255, message = "{validation.local_datacenter.max}") String localDatacenter, // API key for the search engines (Elasticsearch / OpenSearch); encrypted before persistence. - @Size(max = 4096, message = "{validation.api_key.max}") String apiKey + @Size(max = 4096, message = "{validation.api_key.max}") String apiKey, + Boolean resultCacheEnabled, + @Min(value = 1, message = "{validation.result_cache_ttl.range}") + @Max(value = 86_400, message = "{validation.result_cache_ttl.range}") + Integer resultCacheTtlSeconds ) {} diff --git a/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/DatasourceHealthResponse.java b/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/DatasourceHealthResponse.java index d04f586e..1fe509f0 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/DatasourceHealthResponse.java +++ b/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/DatasourceHealthResponse.java @@ -2,8 +2,10 @@ import com.bablsoft.accessflow.core.api.DbType; import com.bablsoft.accessflow.proxy.api.DatasourceHealthSnapshot; +import com.bablsoft.accessflow.proxy.api.ReplicaEndpointHealth; import com.fasterxml.jackson.annotation.JsonProperty; +import java.util.List; import java.util.UUID; /** @@ -24,7 +26,22 @@ public record DatasourceHealthResponse( @JsonProperty("queries_last_24h") long queriesLast24h, Double executionMsP50, Double executionMsP95, - @JsonProperty("errors_last_24h") long errorsLast24h) { + @JsonProperty("errors_last_24h") long errorsLast24h, + List replicas) { + + /** Health of one read-replica endpoint on this node (AF-457). */ + public record ReplicaHealthResponse( + UUID endpointId, + String label, + boolean healthy, + Integer poolActive, + Integer poolTotal) { + + static ReplicaHealthResponse from(ReplicaEndpointHealth health) { + return new ReplicaHealthResponse(health.endpointId(), health.label(), health.healthy(), + health.poolActive(), health.poolTotal()); + } + } public static DatasourceHealthResponse from(DatasourceHealthSnapshot snapshot) { return new DatasourceHealthResponse( @@ -40,6 +57,7 @@ public static DatasourceHealthResponse from(DatasourceHealthSnapshot snapshot) { snapshot.queriesLast24h(), snapshot.executionMsP50(), snapshot.executionMsP95(), - snapshot.errorsLast24h()); + snapshot.errorsLast24h(), + snapshot.replicas().stream().map(ReplicaHealthResponse::from).toList()); } } diff --git a/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/DatasourceResponse.java b/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/DatasourceResponse.java index aa28f5a8..8fff26db 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/DatasourceResponse.java +++ b/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/DatasourceResponse.java @@ -5,6 +5,7 @@ import com.bablsoft.accessflow.core.api.SslMode; import java.time.Instant; +import java.util.List; import java.util.UUID; public record DatasourceResponse( @@ -28,12 +29,20 @@ public record DatasourceResponse( UUID customDriverId, String connectorId, String jdbcUrlOverride, - String readReplicaJdbcUrl, - String readReplicaUsername, + List readReplicas, boolean active, Instant createdAt, - String localDatacenter + String localDatacenter, + boolean resultCacheEnabled, + Integer resultCacheTtlSeconds ) { + /** One read-replica endpoint — never carries the password. */ + public record ReadReplicaResponse(UUID id, String jdbcUrl, String username) { + static ReadReplicaResponse from(DatasourceView.ReadReplicaView view) { + return new ReadReplicaResponse(view.id(), view.jdbcUrl(), view.username()); + } + } + public static DatasourceResponse from(DatasourceView view) { return new DatasourceResponse( view.id(), @@ -56,10 +65,11 @@ public static DatasourceResponse from(DatasourceView view) { view.customDriverId(), view.connectorId(), view.jdbcUrlOverride(), - view.readReplicaJdbcUrl(), - view.readReplicaUsername(), + view.readReplicas().stream().map(ReadReplicaResponse::from).toList(), view.active(), view.createdAt(), - view.localDatacenter()); + view.localDatacenter(), + view.resultCacheEnabled(), + view.resultCacheTtlSeconds()); } } diff --git a/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/ReadReplicaRequest.java b/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/ReadReplicaRequest.java new file mode 100644 index 00000000..78cea02f --- /dev/null +++ b/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/ReadReplicaRequest.java @@ -0,0 +1,23 @@ +package com.bablsoft.accessflow.security.internal.web.model; + +import jakarta.validation.constraints.NotBlank; +import jakarta.validation.constraints.Pattern; +import jakarta.validation.constraints.Size; + +import java.util.UUID; + +/** + * One read-replica endpoint in a datasource create/update payload (AF-457). {@code id} references + * an existing endpoint on update ({@code null} creates a new one). {@code password} semantics on + * update: {@code null} keeps the stored secret, empty string clears it (primary-credential + * fallback), non-blank re-encrypts. + */ +public record ReadReplicaRequest( + UUID id, + @NotBlank(message = "{validation.jdbc_url.required}") + @Size(max = 2048, message = "{validation.jdbc_url.length}") + @Pattern(regexp = "^jdbc:[a-zA-Z][a-zA-Z0-9+\\-.]*:.+$", + message = "{validation.jdbc_url.format}") String jdbcUrl, + @Size(max = 255, message = "{validation.display_name.max}") String username, + @Size(max = 4096) String password +) {} diff --git a/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/TestReplicaRequest.java b/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/TestReplicaRequest.java index 1ad473b3..9b9b9bae 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/TestReplicaRequest.java +++ b/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/TestReplicaRequest.java @@ -4,10 +4,13 @@ import jakarta.validation.constraints.Pattern; import jakarta.validation.constraints.Size; +import java.util.UUID; + /** * Live-values request for {@code POST /api/v1/datasources/{id}/test-replica}. The {@code password} - * field is optional — when omitted, the test reuses the currently-persisted replica password (lets - * an admin re-test after changing only the URL or username without re-typing the secret). + * field is optional — when omitted, the test reuses the persisted password of the endpoint named + * by {@code replicaId} (lets an admin re-test after changing only the URL or username without + * re-typing the secret). */ public record TestReplicaRequest( @NotBlank(message = "{validation.jdbc_url.required}") @@ -16,5 +19,6 @@ public record TestReplicaRequest( message = "{validation.jdbc_url.format}") String jdbcUrl, @NotBlank(message = "{validation.username.required}") @Size(max = 255, message = "{validation.display_name.max}") String username, - @Size(max = 4096) String password + @Size(max = 4096) String password, + UUID replicaId ) {} diff --git a/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/UpdateDatasourceRequest.java b/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/UpdateDatasourceRequest.java index 6ac189aa..556d704e 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/UpdateDatasourceRequest.java +++ b/backend/src/main/java/com/bablsoft/accessflow/security/internal/web/model/UpdateDatasourceRequest.java @@ -1,11 +1,13 @@ package com.bablsoft.accessflow.security.internal.web.model; import com.bablsoft.accessflow.core.api.SslMode; +import jakarta.validation.Valid; import jakarta.validation.constraints.Max; import jakarta.validation.constraints.Min; import jakarta.validation.constraints.Pattern; import jakarta.validation.constraints.Size; +import java.util.List; import java.util.UUID; public record UpdateDatasourceRequest( @@ -31,15 +33,16 @@ public record UpdateDatasourceRequest( @Size(max = 2048, message = "{validation.jdbc_url.length}") @Pattern(regexp = "^jdbc:[a-zA-Z][a-zA-Z0-9+\\-.]*:.+$", message = "{validation.jdbc_url.format}") String jdbcUrlOverride, - // Replica URL accepts either a valid JDBC URL or an empty string (clear-on-blank). The - // pattern allows ^$ to permit the clear; the service maps blank → null on all 3 fields. - @Size(max = 2048, message = "{validation.jdbc_url.length}") - @Pattern(regexp = "^$|^jdbc:[a-zA-Z][a-zA-Z0-9+\\-.]*:.+$", - message = "{validation.jdbc_url.format}") String readReplicaJdbcUrl, - @Size(max = 255, message = "{validation.display_name.max}") String readReplicaUsername, - @Size(max = 4096) String readReplicaPassword, + // Full-list replacement merged by endpoint id: null keeps the current endpoints, an empty + // list deletes them all (AF-457). + @Size(max = 5, message = "{validation.read_replicas.max}") + List<@Valid ReadReplicaRequest> readReplicas, Boolean active, @Size(max = 255, message = "{validation.local_datacenter.max}") String localDatacenter, // API key for the search engines (Elasticsearch / OpenSearch); a blank value clears it. - @Size(max = 4096, message = "{validation.api_key.max}") String apiKey + @Size(max = 4096, message = "{validation.api_key.max}") String apiKey, + Boolean resultCacheEnabled, + @Min(value = 1, message = "{validation.result_cache_ttl.range}") + @Max(value = 86_400, message = "{validation.result_cache_ttl.range}") + Integer resultCacheTtlSeconds ) {} diff --git a/backend/src/main/java/com/bablsoft/accessflow/workflow/internal/DefaultQueryLifecycleService.java b/backend/src/main/java/com/bablsoft/accessflow/workflow/internal/DefaultQueryLifecycleService.java index 5a01ee35..953a0ed0 100644 --- a/backend/src/main/java/com/bablsoft/accessflow/workflow/internal/DefaultQueryLifecycleService.java +++ b/backend/src/main/java/com/bablsoft/accessflow/workflow/internal/DefaultQueryLifecycleService.java @@ -194,7 +194,7 @@ private ExecutionOutcome doExecute(QueryRequestSnapshot query, UUID actorUserId, var result = queryExecutor.execute(new QueryExecutionRequest( query.datasourceId(), query.sqlText(), query.queryType(), null, null, restrictedColumns, columnMasks, rowSecurityPredicates, parsed.transactional(), - parsed.statements(), softDeletes)); + parsed.statements(), softDeletes, parsed.referencedTables())); var completedAt = Instant.now(); var durationMs = (int) result.duration().toMillis(); Long rowsAffected; diff --git a/backend/src/main/resources/application.yml b/backend/src/main/resources/application.yml index 48940a2c..c083e86e 100644 --- a/backend/src/main/resources/application.yml +++ b/backend/src/main/resources/application.yml @@ -223,6 +223,20 @@ accessflow: max-rows: ${ACCESSFLOW_PROXY_EXECUTION_MAX_ROWS:10000} statement-timeout: ${ACCESSFLOW_PROXY_EXECUTION_STATEMENT_TIMEOUT:30s} default-fetch-size: ${ACCESSFLOW_PROXY_EXECUTION_DEFAULT_FETCH_SIZE:1000} + insert-batch-chunk-size: ${ACCESSFLOW_PROXY_EXECUTION_INSERT_BATCH_CHUNK_SIZE:1000} + # Opt-in SELECT result cache (AF-457). `enabled` is the deployment-wide kill-switch; the real + # gate is each datasource's result_cache_enabled flag. default-ttl applies when a datasource + # opts in without its own TTL; entries above max-entry-bytes are never cached. + cache: + enabled: ${ACCESSFLOW_PROXY_CACHE_ENABLED:true} + default-ttl: ${ACCESSFLOW_PROXY_CACHE_DEFAULT_TTL:PT60S} + max-entry-bytes: ${ACCESSFLOW_PROXY_CACHE_MAX_ENTRY_BYTES:1000000} + # Read-replica health checks (AF-457): per-node prober cadence, per-endpoint isValid timeout, + # and how long a failed endpoint sits out of the read rotation before a half-open retry. + replica: + probe-interval: ${ACCESSFLOW_PROXY_REPLICA_PROBE_INTERVAL:PT30S} + probe-timeout: ${ACCESSFLOW_PROXY_REPLICA_PROBE_TIMEOUT:PT5S} + cooldown: ${ACCESSFLOW_PROXY_REPLICA_COOLDOWN:PT30S} # Per-engine plugin tuning, passed verbatim (after key normalization) into the engine's # QueryEngineContext config map. Generic env form: ACCESSFLOW_PROXY_ENGINES__. # The ACCESSFLOW_PROXY_MONGO_* names are legacy aliases kept working via these placeholders. diff --git a/backend/src/main/resources/db/migration/V115__multi_replica_and_result_cache.sql b/backend/src/main/resources/db/migration/V115__multi_replica_and_result_cache.sql new file mode 100644 index 00000000..769809d5 --- /dev/null +++ b/backend/src/main/resources/db/migration/V115__multi_replica_and_result_cache.sql @@ -0,0 +1,28 @@ +-- AF-457: multi-replica read load balancing — replaces the single read_replica_* +-- columns (V46) with a datasource_read_replicas child table, and adds the opt-in +-- SELECT result-cache flags to datasources. + +CREATE TABLE datasource_read_replicas ( + id UUID PRIMARY KEY, + datasource_id UUID NOT NULL REFERENCES datasources(id) ON DELETE CASCADE, + jdbc_url TEXT NOT NULL, + username VARCHAR(255), + password_encrypted TEXT, + position INT NOT NULL DEFAULT 0, + created_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP +); + +CREATE INDEX idx_datasource_read_replicas_datasource + ON datasource_read_replicas (datasource_id); + +INSERT INTO datasource_read_replicas (id, datasource_id, jdbc_url, username, password_encrypted, position) +SELECT gen_random_uuid(), id, read_replica_jdbc_url, read_replica_username, read_replica_password_encrypted, 0 +FROM datasources +WHERE read_replica_jdbc_url IS NOT NULL AND read_replica_jdbc_url <> ''; + +ALTER TABLE datasources + DROP COLUMN read_replica_jdbc_url, + DROP COLUMN read_replica_username, + DROP COLUMN read_replica_password_encrypted, + ADD COLUMN result_cache_enabled BOOLEAN NOT NULL DEFAULT FALSE, + ADD COLUMN result_cache_ttl_seconds INT; diff --git a/backend/src/main/resources/i18n/messages.properties b/backend/src/main/resources/i18n/messages.properties index fe38af10..bbbf9b54 100644 --- a/backend/src/main/resources/i18n/messages.properties +++ b/backend/src/main/resources/i18n/messages.properties @@ -585,6 +585,8 @@ error.neo4j.procedure_forbidden=The procedure call ''{0}'' is not allowed error.neo4j.unbalanced=Unbalanced brackets or quotes in the Cypher query error.neo4j.label_required=A node label is required validation.api_key.max=API key must be at most 4,096 characters +validation.read_replicas.max=At most 5 read-replica endpoints are allowed +validation.result_cache_ttl.range=Result cache TTL must be between 1 and 86,400 seconds validation.local_datacenter.max=Local datacenter must be at most 255 characters validation.row_security_table.required=Target table is required validation.row_security_table.size=Target table must be at most 512 characters diff --git a/backend/src/main/resources/i18n/messages_de.properties b/backend/src/main/resources/i18n/messages_de.properties index 5ebd3256..e1ccf07c 100644 --- a/backend/src/main/resources/i18n/messages_de.properties +++ b/backend/src/main/resources/i18n/messages_de.properties @@ -604,6 +604,8 @@ error.neo4j.procedure_forbidden=Der Prozeduraufruf ''{0}'' ist nicht zulässig error.neo4j.unbalanced=Nicht ausbalancierte Klammern oder Anführungszeichen in der Cypher-Abfrage error.neo4j.label_required=Ein Knoten-Label ist erforderlich validation.api_key.max=Der API-Schlüssel darf höchstens 4.096 Zeichen lang sein +validation.read_replicas.max=Es sind höchstens 5 Lesereplikat-Endpunkte erlaubt +validation.result_cache_ttl.range=Die Ergebnis-Cache-TTL muss zwischen 1 und 86.400 Sekunden liegen validation.local_datacenter.max=Das lokale Rechenzentrum darf höchstens 255 Zeichen lang sein # AF-441 collaboration comments diff --git a/backend/src/main/resources/i18n/messages_es.properties b/backend/src/main/resources/i18n/messages_es.properties index 4fca3edb..5307e800 100644 --- a/backend/src/main/resources/i18n/messages_es.properties +++ b/backend/src/main/resources/i18n/messages_es.properties @@ -604,6 +604,8 @@ error.neo4j.procedure_forbidden=La llamada al procedimiento ''{0}'' no está per error.neo4j.unbalanced=Corchetes o comillas no balanceados en la consulta Cypher error.neo4j.label_required=Se requiere una etiqueta de nodo validation.api_key.max=La clave de API debe tener como máximo 4.096 caracteres +validation.read_replicas.max=Se permiten como máximo 5 puntos de conexión de réplicas de lectura +validation.result_cache_ttl.range=El TTL de la caché de resultados debe estar entre 1 y 86.400 segundos validation.local_datacenter.max=El centro de datos local debe tener como máximo 255 caracteres # AF-441 collaboration comments diff --git a/backend/src/main/resources/i18n/messages_fr.properties b/backend/src/main/resources/i18n/messages_fr.properties index 552f9b2b..a07a2772 100644 --- a/backend/src/main/resources/i18n/messages_fr.properties +++ b/backend/src/main/resources/i18n/messages_fr.properties @@ -607,6 +607,8 @@ error.neo4j.procedure_forbidden=L'appel de procédure ''{0}'' n'est pas autoris error.neo4j.unbalanced=Crochets ou guillemets non équilibrés dans la requête Cypher error.neo4j.label_required=Un label de nœud est requis validation.api_key.max=La clé d'API ne doit pas dépasser 4 096 caractères +validation.read_replicas.max=Au maximum 5 points de terminaison de réplicas de lecture sont autorisés +validation.result_cache_ttl.range=Le TTL du cache de résultats doit être compris entre 1 et 86 400 secondes validation.local_datacenter.max=Le centre de données local ne doit pas dépasser 255 caractères # AF-441 collaboration comments diff --git a/backend/src/main/resources/i18n/messages_hy.properties b/backend/src/main/resources/i18n/messages_hy.properties index 62f84567..1cfba07b 100644 --- a/backend/src/main/resources/i18n/messages_hy.properties +++ b/backend/src/main/resources/i18n/messages_hy.properties @@ -604,6 +604,8 @@ error.neo4j.procedure_forbidden=''{0}'' ընթացակարգի կանչը չի error.neo4j.unbalanced=Չհավասարակշռված փակագծեր կամ չակերտներ Cypher հարցումում error.neo4j.label_required=Հանգույցի պիտակը պարտադիր է validation.api_key.max=API բանալին պետք է լինի առավելագույնը 4096 նիշ +validation.read_replicas.max=Թույլատրվում է առավելագույնը 5 ընթերցման կրկնօրինակի վերջնակետ +validation.result_cache_ttl.range=Արդյունքների քեշի TTL-ը պետք է լինի 1-ից 86400 վայրկյանի միջակայքում validation.local_datacenter.max=Տեղական տվյալների կենտրոնը պետք է լինի առավելագույնը 255 նիշ # AF-441 collaboration comments diff --git a/backend/src/main/resources/i18n/messages_ru.properties b/backend/src/main/resources/i18n/messages_ru.properties index f757df4a..fedad596 100644 --- a/backend/src/main/resources/i18n/messages_ru.properties +++ b/backend/src/main/resources/i18n/messages_ru.properties @@ -604,6 +604,8 @@ error.neo4j.procedure_forbidden=Вызов процедуры ''{0}'' не до error.neo4j.unbalanced=Несбалансированные скобки или кавычки в запросе Cypher error.neo4j.label_required=Требуется метка узла validation.api_key.max=Ключ API не должен превышать 4 096 символов +validation.read_replicas.max=Допускается не более 5 конечных точек реплик чтения +validation.result_cache_ttl.range=TTL кэша результатов должен быть от 1 до 86 400 секунд validation.local_datacenter.max=Локальный центр обработки данных должен содержать не более 255 символов # AF-441 collaboration comments diff --git a/backend/src/main/resources/i18n/messages_zh_CN.properties b/backend/src/main/resources/i18n/messages_zh_CN.properties index cd23c656..44f37a26 100644 --- a/backend/src/main/resources/i18n/messages_zh_CN.properties +++ b/backend/src/main/resources/i18n/messages_zh_CN.properties @@ -604,6 +604,8 @@ error.neo4j.procedure_forbidden=不允许调用过程 ''{0}'' error.neo4j.unbalanced=Cypher 查询中的括号或引号不匹配 error.neo4j.label_required=需要节点标签 validation.api_key.max=API 密钥最多 4,096 个字符 +validation.read_replicas.max=最多允许 5 个读副本端点 +validation.result_cache_ttl.range=结果缓存 TTL 必须在 1 到 86,400 秒之间 validation.local_datacenter.max=本地数据中心最多 255 个字符 # AF-441 collaboration comments diff --git a/backend/src/test/java/com/bablsoft/accessflow/core/internal/DatasourceAdminServiceImplTest.java b/backend/src/test/java/com/bablsoft/accessflow/core/internal/DatasourceAdminServiceImplTest.java index b0ae02c2..99a4a008 100644 --- a/backend/src/test/java/com/bablsoft/accessflow/core/internal/DatasourceAdminServiceImplTest.java +++ b/backend/src/test/java/com/bablsoft/accessflow/core/internal/DatasourceAdminServiceImplTest.java @@ -12,11 +12,14 @@ import com.bablsoft.accessflow.core.api.IllegalDatasourcePermissionException; import com.bablsoft.accessflow.core.api.JdbcCoordinatesFactory; import com.bablsoft.accessflow.core.api.MissingAiConfigForDatasourceException; +import com.bablsoft.accessflow.core.api.ReplicaEndpointInput; import com.bablsoft.accessflow.core.api.SslMode; import com.bablsoft.accessflow.core.api.UpdateDatasourceCommand; +import com.bablsoft.accessflow.core.events.DatasourceCacheConfigChangedEvent; import com.bablsoft.accessflow.core.events.DatasourceConfigChangedEvent; import com.bablsoft.accessflow.core.events.DatasourceDeactivatedEvent; import com.bablsoft.accessflow.core.internal.persistence.entity.DatasourceEntity; +import com.bablsoft.accessflow.core.internal.persistence.entity.DatasourceReadReplicaEntity; import com.bablsoft.accessflow.core.internal.persistence.entity.DatasourceUserPermissionEntity; import com.bablsoft.accessflow.core.internal.persistence.entity.OrganizationEntity; import com.bablsoft.accessflow.core.internal.persistence.entity.UserEntity; @@ -493,7 +496,7 @@ void updateDoesNotPublishWhenOnlyNonPoolFieldsChange() { } @Test - void updateSetsReplicaFieldsAndEncryptsPassword() { + void updateAddsReplicaEndpointAndEncryptsPassword() { var entity = buildDatasource(datasourceId, orgId, "Prod"); when(datasourceRepository.findById(datasourceId)).thenReturn(Optional.of(entity)); when(encryptionService.encrypt("replica-pw")).thenReturn("ENC(replica-pw)"); @@ -503,19 +506,20 @@ void updateSetsReplicaFieldsAndEncryptsPassword() { null, null, null, null, null, "jdbc:postgresql://replica:5432/appdb", "replica-user", "replica-pw", null)); - assertThat(entity.getReadReplicaJdbcUrl()) - .isEqualTo("jdbc:postgresql://replica:5432/appdb"); - assertThat(entity.getReadReplicaUsername()).isEqualTo("replica-user"); - assertThat(entity.getReadReplicaPasswordEncrypted()).isEqualTo("ENC(replica-pw)"); + assertThat(entity.getReadReplicas()).hasSize(1); + var replica = entity.getReadReplicas().get(0); + assertThat(replica.getJdbcUrl()).isEqualTo("jdbc:postgresql://replica:5432/appdb"); + assertThat(replica.getUsername()).isEqualTo("replica-user"); + assertThat(replica.getPasswordEncrypted()).isEqualTo("ENC(replica-pw)"); + assertThat(replica.getId()).isNotNull(); + assertThat(replica.getPosition()).isZero(); verify(eventPublisher).publishEvent(new DatasourceConfigChangedEvent(datasourceId)); } @Test - void updateClearsReplicaWhenJdbcUrlBlank() { + void updateWithEmptyReplicaListRemovesAllEndpoints() { var entity = buildDatasource(datasourceId, orgId, "Prod"); - entity.setReadReplicaJdbcUrl("jdbc:postgresql://replica/appdb"); - entity.setReadReplicaUsername("ru"); - entity.setReadReplicaPasswordEncrypted("ENC(rpw)"); + addReplica(entity, "jdbc:postgresql://replica/appdb", "ru", "ENC(rpw)"); when(datasourceRepository.findById(datasourceId)).thenReturn(Optional.of(entity)); service.update(datasourceId, orgId, new UpdateDatasourceCommand( @@ -523,12 +527,77 @@ void updateClearsReplicaWhenJdbcUrlBlank() { null, null, null, null, null, "", null, null, null)); - assertThat(entity.getReadReplicaJdbcUrl()).isNull(); - assertThat(entity.getReadReplicaUsername()).isNull(); - assertThat(entity.getReadReplicaPasswordEncrypted()).isNull(); + assertThat(entity.getReadReplicas()).isEmpty(); verify(eventPublisher).publishEvent(new DatasourceConfigChangedEvent(datasourceId)); } + @Test + void updateMergesReplicaListByEndpointId() { + var entity = buildDatasource(datasourceId, orgId, "Prod"); + addReplica(entity, "jdbc:postgresql://replica-a/appdb", "ua", "ENC(a)"); + addReplica(entity, "jdbc:postgresql://replica-b/appdb", "ub", "ENC(b)"); + var keptId = entity.getReadReplicas().get(0).getId(); + when(datasourceRepository.findById(datasourceId)).thenReturn(Optional.of(entity)); + when(encryptionService.encrypt("new-pw")).thenReturn("ENC(new-pw)"); + + service.update(datasourceId, orgId, new UpdateDatasourceCommand( + null, null, null, null, null, null, null, null, null, null, null, null, + null, null, null, null, null, + List.of( + // Existing endpoint: URL changed, password null → stored secret kept. + new ReplicaEndpointInput(keptId, + "jdbc:postgresql://replica-a2/appdb", "ua2", null), + // New endpoint (no id) with a fresh password. + new ReplicaEndpointInput(null, + "jdbc:postgresql://replica-c/appdb", "uc", "new-pw")), + null, null, null, null, null)); + + assertThat(entity.getReadReplicas()).hasSize(2); + var kept = entity.getReadReplicas().get(0); + assertThat(kept.getId()).isEqualTo(keptId); + assertThat(kept.getJdbcUrl()).isEqualTo("jdbc:postgresql://replica-a2/appdb"); + assertThat(kept.getUsername()).isEqualTo("ua2"); + assertThat(kept.getPasswordEncrypted()).isEqualTo("ENC(a)"); + assertThat(kept.getPosition()).isZero(); + var added = entity.getReadReplicas().get(1); + assertThat(added.getJdbcUrl()).isEqualTo("jdbc:postgresql://replica-c/appdb"); + assertThat(added.getPasswordEncrypted()).isEqualTo("ENC(new-pw)"); + assertThat(added.getPosition()).isEqualTo(1); + verify(eventPublisher).publishEvent(new DatasourceConfigChangedEvent(datasourceId)); + } + + @Test + void updateReplicaEmptyPasswordClearsStoredSecret() { + var entity = buildDatasource(datasourceId, orgId, "Prod"); + addReplica(entity, "jdbc:postgresql://replica/appdb", "ru", "ENC(rpw)"); + var replicaId = entity.getReadReplicas().get(0).getId(); + when(datasourceRepository.findById(datasourceId)).thenReturn(Optional.of(entity)); + + service.update(datasourceId, orgId, new UpdateDatasourceCommand( + null, null, null, null, null, null, null, null, null, null, null, null, + null, null, null, null, null, + List.of(new ReplicaEndpointInput(replicaId, + "jdbc:postgresql://replica/appdb", "ru", "")), + null, null, null, null, null)); + + assertThat(entity.getReadReplicas().get(0).getPasswordEncrypted()).isNull(); + } + + @Test + void updateCacheSettingsPublishesCacheConfigChangedWithoutPoolEviction() { + var entity = buildDatasource(datasourceId, orgId, "Prod"); + when(datasourceRepository.findById(datasourceId)).thenReturn(Optional.of(entity)); + + service.update(datasourceId, orgId, new UpdateDatasourceCommand( + null, null, null, null, null, null, null, null, null, null, null, null, + null, null, null, null, null, null, null, null, null, true, 120)); + + assertThat(entity.isResultCacheEnabled()).isTrue(); + assertThat(entity.getResultCacheTtlSeconds()).isEqualTo(120); + verify(eventPublisher).publishEvent(new DatasourceCacheConfigChangedEvent(datasourceId)); + verify(eventPublisher, never()).publishEvent(any(DatasourceConfigChangedEvent.class)); + } + @Test void testReplicaWithBlankUrlThrowsDatasourceConnectionTestException() { var entity = buildDatasource(datasourceId, orgId, "Prod"); @@ -555,6 +624,21 @@ void testReplicaWithoutPasswordAndNoPersistedThrowsDatasourceConnectionTestExcep .isInstanceOf(com.bablsoft.accessflow.core.api.DatasourceConnectionTestException.class); } + @Test + void testReplicaWithReplicaIdButNoStoredSecretThrowsDatasourceConnectionTestException() { + var entity = buildDatasource(datasourceId, orgId, "Prod"); + addReplica(entity, "jdbc:postgresql://replica/appdb", "ru", null); + var replicaId = entity.getReadReplicas().get(0).getId(); + when(datasourceRepository.findById(datasourceId)).thenReturn(Optional.of(entity)); + when(messageSource.getMessage(eq("error.replica_not_configured"), any(), any())) + .thenReturn("not configured"); + + assertThatThrownBy(() -> service.testReplica(datasourceId, orgId, + new com.bablsoft.accessflow.core.api.TestReplicaCommand( + "jdbc:postgresql://replica/appdb", "ru", null, replicaId))) + .isInstanceOf(com.bablsoft.accessflow.core.api.DatasourceConnectionTestException.class); + } + @Test void updatePublishesDeactivatedWhenActiveFlipsToFalse() { var entity = buildDatasource(datasourceId, orgId, "Prod"); @@ -959,6 +1043,18 @@ private DatasourceEntity buildDatasource(UUID id, UUID organizationId, String na return entity; } + private static void addReplica(DatasourceEntity entity, String jdbcUrl, String username, + String passwordEncrypted) { + var replica = new DatasourceReadReplicaEntity(); + replica.setId(UUID.randomUUID()); + replica.setDatasource(entity); + replica.setJdbcUrl(jdbcUrl); + replica.setUsername(username); + replica.setPasswordEncrypted(passwordEncrypted); + replica.setPosition(entity.getReadReplicas().size()); + entity.getReadReplicas().add(replica); + } + @Test void createWithAiEnabledButNoConfigThrows() { when(datasourceRepository.existsByOrganization_IdAndNameIgnoreCase(orgId, "Prod")) diff --git a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/BatchInsertPlannerTest.java b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/BatchInsertPlannerTest.java new file mode 100644 index 00000000..36a7e897 --- /dev/null +++ b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/BatchInsertPlannerTest.java @@ -0,0 +1,173 @@ +package com.bablsoft.accessflow.proxy.internal; + +import org.junit.jupiter.api.Test; + +import java.util.List; + +import static org.assertj.core.api.Assertions.assertThat; + +class BatchInsertPlannerTest { + + private static boolean[] allBatchable(int size) { + var flags = new boolean[size]; + java.util.Arrays.fill(flags, true); + return flags; + } + + @Test + void foldsConsecutiveHomogeneousInsertsIntoOneBatch() { + var statements = List.of( + "INSERT INTO users (id, name) VALUES (1, 'a')", + "INSERT INTO users (id, name) VALUES (2, 'b')", + "INSERT INTO users (id, name) VALUES (3, 'c')"); + + var steps = BatchInsertPlanner.plan(statements, allBatchable(3)); + + assertThat(steps).hasSize(1); + var batch = (BatchInsertPlanner.BatchStep) steps.get(0); + assertThat(batch.templateSql()).isEqualTo("INSERT INTO users (id,name) VALUES (?, ?)"); + assertThat(batch.rowBinds()).containsExactly( + List.of(1L, "a"), List.of(2L, "b"), List.of(3L, "c")); + } + + @Test + void singleInsertStaysOnPerStatementPath() { + var steps = BatchInsertPlanner.plan( + List.of("INSERT INTO users (id) VALUES (1)"), allBatchable(1)); + + assertThat(steps).containsExactly(new BatchInsertPlanner.SingleStep(0)); + } + + @Test + void differentTablesBreakTheRun() { + var statements = List.of( + "INSERT INTO users (id) VALUES (1)", + "INSERT INTO orders (id) VALUES (2)", + "INSERT INTO orders (id) VALUES (3)"); + + var steps = BatchInsertPlanner.plan(statements, allBatchable(3)); + + assertThat(steps).hasSize(2); + assertThat(steps.get(0)).isEqualTo(new BatchInsertPlanner.SingleStep(0)); + var batch = (BatchInsertPlanner.BatchStep) steps.get(1); + assertThat(batch.templateSql()).isEqualTo("INSERT INTO orders (id) VALUES (?)"); + } + + @Test + void differentColumnListsBreakTheRun() { + var statements = List.of( + "INSERT INTO users (id, name) VALUES (1, 'a')", + "INSERT INTO users (id) VALUES (2)"); + + var steps = BatchInsertPlanner.plan(statements, allBatchable(2)); + + assertThat(steps).containsExactly( + new BatchInsertPlanner.SingleStep(0), new BatchInsertPlanner.SingleStep(1)); + } + + @Test + void insertWithoutColumnListBatchesByArity() { + var statements = List.of( + "INSERT INTO users VALUES (1, 'a')", + "INSERT INTO users VALUES (2, 'b')"); + + var steps = BatchInsertPlanner.plan(statements, allBatchable(2)); + + assertThat(steps).hasSize(1); + var batch = (BatchInsertPlanner.BatchStep) steps.get(0); + assertThat(batch.templateSql()).isEqualTo("INSERT INTO users VALUES (?, ?)"); + } + + @Test + void literalTypeMatrixBindsExpectedValues() { + var statements = List.of( + "INSERT INTO t (a, b, c, d, e) VALUES (1, 2.5, 'x', NULL, -7)", + "INSERT INTO t (a, b, c, d, e) VALUES (2, 3.5, 'y', NULL, -8)"); + + var steps = BatchInsertPlanner.plan(statements, allBatchable(2)); + + assertThat(steps).hasSize(1); + var batch = (BatchInsertPlanner.BatchStep) steps.get(0); + assertThat(batch.rowBinds().get(0)) + .containsExactly(1L, 2.5d, "x", null, -7L); + assertThat(batch.rowBinds().get(1)) + .containsExactly(2L, 3.5d, "y", null, -8L); + } + + @Test + void nonLiteralValuesFallBackToSingleSteps() { + var statements = List.of( + "INSERT INTO t (a) VALUES (now())", + "INSERT INTO t (a) VALUES (now())"); + + var steps = BatchInsertPlanner.plan(statements, allBatchable(2)); + + assertThat(steps).containsExactly( + new BatchInsertPlanner.SingleStep(0), new BatchInsertPlanner.SingleStep(1)); + } + + @Test + void insertSelectAndMultiRowValuesFallBackToSingleSteps() { + var statements = List.of( + "INSERT INTO t (a) SELECT a FROM s", + "INSERT INTO t (a) VALUES (1), (2)"); + + var steps = BatchInsertPlanner.plan(statements, allBatchable(2)); + + assertThat(steps).containsExactly( + new BatchInsertPlanner.SingleStep(0), new BatchInsertPlanner.SingleStep(1)); + } + + @Test + void nonInsertStatementsFallBackToSingleSteps() { + var statements = List.of( + "UPDATE t SET a = 1 WHERE id = 1", + "DELETE FROM t WHERE id = 2"); + + var steps = BatchInsertPlanner.plan(statements, allBatchable(2)); + + assertThat(steps).containsExactly( + new BatchInsertPlanner.SingleStep(0), new BatchInsertPlanner.SingleStep(1)); + } + + @Test + void nonBatchableFlagExcludesRewrittenStatements() { + var statements = List.of( + "INSERT INTO users (id) VALUES (1)", + "INSERT INTO users (id) VALUES (2)"); + + var steps = BatchInsertPlanner.plan(statements, new boolean[]{true, false}); + + assertThat(steps).containsExactly( + new BatchInsertPlanner.SingleStep(0), new BatchInsertPlanner.SingleStep(1)); + } + + @Test + void runResumesAfterNonBatchableInterruption() { + var statements = List.of( + "INSERT INTO users (id) VALUES (1)", + "INSERT INTO users (id) VALUES (2)", + "UPDATE users SET name = 'x' WHERE id = 1", + "INSERT INTO users (id) VALUES (3)", + "INSERT INTO users (id) VALUES (4)"); + + var steps = BatchInsertPlanner.plan(statements, allBatchable(5)); + + assertThat(steps).hasSize(3); + assertThat(((BatchInsertPlanner.BatchStep) steps.get(0)).rowBinds()) + .containsExactly(List.of(1L), List.of(2L)); + assertThat(steps.get(1)).isEqualTo(new BatchInsertPlanner.SingleStep(2)); + assertThat(((BatchInsertPlanner.BatchStep) steps.get(2)).rowBinds()) + .containsExactly(List.of(3L), List.of(4L)); + } + + @Test + void unparseableStatementFallsBackToSingleStep() { + var statements = List.of("THIS IS NOT SQL", "INSERT INTO t (a) VALUES (1)"); + + var steps = BatchInsertPlanner.plan(statements, allBatchable(2)); + + assertThat(steps).containsExactly( + new BatchInsertPlanner.SingleStep(0), new BatchInsertPlanner.SingleStep(1)); + } +} diff --git a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DatasourcePoolFactoryTest.java b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DatasourcePoolFactoryTest.java index 39c6fbab..cf9a1e89 100644 --- a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DatasourcePoolFactoryTest.java +++ b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DatasourcePoolFactoryTest.java @@ -243,6 +243,7 @@ void createReplicaPoolUsesReplicaUrlAndCredentials() { datasourceId, organizationId, DbType.POSTGRESQL, "h", 5432, "appdb", "svc", "ENC(primary)", SslMode.DISABLE, 15, 1000, false, null, false, null, null, null, "jdbc:postgresql://replica:5432/appdb", "replica-user", "ENC(replica-pw)", true); + var endpoint = replicaDescriptor.readReplicas().get(0); when(secretResolutionService.resolve("ENC(replica-pw)", datasourceId, organizationId)) .thenReturn("replica-pw-plain"); @@ -251,14 +252,15 @@ void createReplicaPoolUsesReplicaUrlAndCredentials() { HikariDataSource.class, (mock, ctx) -> captured.set((HikariConfig) ctx.arguments().get(0)))) { - factory.createReplicaPool(replicaDescriptor); + factory.createReplicaPool(replicaDescriptor, endpoint); assertThat(mocked.constructed()).hasSize(1); var config = captured.get(); assertThat(config.getJdbcUrl()).isEqualTo("jdbc:postgresql://replica:5432/appdb"); assertThat(config.getUsername()).isEqualTo("replica-user"); assertThat(config.getPassword()).isEqualTo("replica-pw-plain"); - assertThat(config.getPoolName()).isEqualTo("accessflow-ds-" + datasourceId + "-replica"); + assertThat(config.getPoolName()) + .isEqualTo("accessflow-ds-" + datasourceId + "-replica-0"); // Primary password must not have been resolved on the replica path. verify(secretResolutionService, times(0)) .resolve("ENC(primary)", datasourceId, organizationId); @@ -271,13 +273,14 @@ void createReplicaPoolFallsBackToPrimaryUsernameAndPasswordWhenReplicaCredsAbsen datasourceId, organizationId, DbType.POSTGRESQL, "h", 5432, "appdb", "svc", "ENC(secret)", SslMode.DISABLE, 15, 1000, false, null, false, null, null, null, "jdbc:postgresql://replica:5432/appdb", null, null, true); + var endpoint = replicaDescriptor.readReplicas().get(0); var captured = new AtomicReference(); try (MockedConstruction ignored = Mockito.mockConstruction( HikariDataSource.class, (mock, ctx) -> captured.set((HikariConfig) ctx.arguments().get(0)))) { - factory.createReplicaPool(replicaDescriptor); + factory.createReplicaPool(replicaDescriptor, endpoint); var config = captured.get(); assertThat(config.getUsername()).isEqualTo("svc"); diff --git a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DefaultDatasourceConnectionPoolManagerTest.java b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DefaultDatasourceConnectionPoolManagerTest.java index 1e9107f7..185db7ae 100644 --- a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DefaultDatasourceConnectionPoolManagerTest.java +++ b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DefaultDatasourceConnectionPoolManagerTest.java @@ -183,29 +183,54 @@ void shutdownClosesAllPools() { } @Test - void resolveReplicaReturnsEmptyWhenNoReplicaConfigured() { + void replicaEndpointsReturnsEmptyWhenNoReplicaConfigured() { when(datasourceLookupService.findById(id)).thenReturn(Optional.of(activeDescriptor)); - assertThat(manager.resolveReplica(id)).isEmpty(); - verify(poolFactory, never()).createReplicaPool(any()); + assertThat(manager.replicaEndpoints(id)).isEmpty(); + verify(poolFactory, never()).createReplicaPool(any(), any()); } @Test - void resolveReplicaCreatesAndCachesPoolWhenReplicaConfigured() { + void replicaEndpointsCarryRedactedLabels() { + var replicaDescriptor = new DatasourceConnectionDescriptor( + id, organizationId, DbType.POSTGRESQL, "h", 5432, "db", "u", "ENC", SslMode.DISABLE, + 10, 1000, false, null, false, null, null, null, + "jdbc:postgresql://replica:5432/db?password=secret", "ru", "ENC(rpw)", true); + when(datasourceLookupService.findById(id)).thenReturn(Optional.of(replicaDescriptor)); + + var endpoints = manager.replicaEndpoints(id); + + assertThat(endpoints).hasSize(1); + assertThat(endpoints.get(0).endpointId()) + .isEqualTo(replicaDescriptor.readReplicas().get(0).id()); + assertThat(endpoints.get(0).label()).isEqualTo("replica:5432"); + } + + @Test + void resolveReplicaCreatesAndCachesPoolPerEndpoint() { var replicaDescriptor = new DatasourceConnectionDescriptor( id, organizationId, DbType.POSTGRESQL, "h", 5432, "db", "u", "ENC", SslMode.DISABLE, 10, 1000, false, null, false, null, null, null, "jdbc:postgresql://replica:5432/db", "ru", "ENC(rpw)", true); + var endpoint = replicaDescriptor.readReplicas().get(0); var replicaPool = mock(HikariDataSource.class); when(datasourceLookupService.findById(id)).thenReturn(Optional.of(replicaDescriptor)); - when(poolFactory.createReplicaPool(replicaDescriptor)).thenReturn(replicaPool); + when(poolFactory.createReplicaPool(replicaDescriptor, endpoint)).thenReturn(replicaPool); - var first = manager.resolveReplica(id); - var second = manager.resolveReplica(id); + var first = manager.resolveReplica(id, endpoint.id()); + var second = manager.resolveReplica(id, endpoint.id()); - assertThat(first).containsSame(replicaPool); - assertThat(second).containsSame(replicaPool); - verify(poolFactory, times(1)).createReplicaPool(replicaDescriptor); + assertThat(first).isSameAs(replicaPool); + assertThat(second).isSameAs(replicaPool); + verify(poolFactory, times(1)).createReplicaPool(replicaDescriptor, endpoint); + } + + @Test + void resolveReplicaWithUnknownEndpointThrowsDatasourceUnavailable() { + when(datasourceLookupService.findById(id)).thenReturn(Optional.of(activeDescriptor)); + + assertThatThrownBy(() -> manager.resolveReplica(id, UUID.randomUUID())) + .isInstanceOf(DatasourceUnavailableException.class); } @Test @@ -214,13 +239,14 @@ void evictClosesBothPrimaryAndReplicaPools() { id, organizationId, DbType.POSTGRESQL, "h", 5432, "db", "u", "ENC", SslMode.DISABLE, 10, 1000, false, null, false, null, null, null, "jdbc:postgresql://replica:5432/db", "ru", "ENC(rpw)", true); + var endpoint = replicaDescriptor.readReplicas().get(0); var primaryPool = mock(HikariDataSource.class); var replicaPool = mock(HikariDataSource.class); when(datasourceLookupService.findById(id)).thenReturn(Optional.of(replicaDescriptor)); when(poolFactory.createPool(replicaDescriptor)).thenReturn(primaryPool); - when(poolFactory.createReplicaPool(replicaDescriptor)).thenReturn(replicaPool); + when(poolFactory.createReplicaPool(replicaDescriptor, endpoint)).thenReturn(replicaPool); manager.resolve(id); - manager.resolveReplica(id); + manager.resolveReplica(id, endpoint.id()); manager.evict(id); @@ -276,10 +302,59 @@ void resolveReplicaWrapsFactoryFailureInPoolInitializationException() { id, organizationId, DbType.POSTGRESQL, "h", 5432, "db", "u", "ENC", SslMode.DISABLE, 10, 1000, false, null, false, null, null, null, "jdbc:postgresql://replica:5432/db", "ru", "ENC(rpw)", true); + var endpoint = replicaDescriptor.readReplicas().get(0); when(datasourceLookupService.findById(id)).thenReturn(Optional.of(replicaDescriptor)); - when(poolFactory.createReplicaPool(replicaDescriptor)).thenThrow(new RuntimeException("boom")); + when(poolFactory.createReplicaPool(replicaDescriptor, endpoint)) + .thenThrow(new RuntimeException("boom")); - assertThatThrownBy(() -> manager.resolveReplica(id)) + assertThatThrownBy(() -> manager.resolveReplica(id, endpoint.id())) .isInstanceOf(PoolInitializationException.class); } + + @Test + void replicaPoolStatsReturnsEmptyWithoutCachedPoolAndGaugesWithOne() { + var replicaDescriptor = new DatasourceConnectionDescriptor( + id, organizationId, DbType.POSTGRESQL, "h", 5432, "db", "u", "ENC", SslMode.DISABLE, + 10, 1000, false, null, false, null, null, null, + "jdbc:postgresql://replica:5432/db", "ru", "ENC(rpw)", true); + var endpoint = replicaDescriptor.readReplicas().get(0); + assertThat(manager.replicaPoolStats(id, endpoint.id())).isEmpty(); + + var replicaPool = mock(HikariDataSource.class); + var mxBean = mock(HikariPoolMXBean.class); + when(replicaPool.getHikariPoolMXBean()).thenReturn(mxBean); + when(mxBean.getActiveConnections()).thenReturn(1); + when(mxBean.getTotalConnections()).thenReturn(4); + when(replicaPool.getMaximumPoolSize()).thenReturn(10); + when(datasourceLookupService.findById(id)).thenReturn(Optional.of(replicaDescriptor)); + when(poolFactory.createReplicaPool(replicaDescriptor, endpoint)).thenReturn(replicaPool); + manager.resolveReplica(id, endpoint.id()); + + assertThat(manager.replicaPoolStats(id, endpoint.id())).hasValueSatisfying(s -> { + assertThat(s.active()).isEqualTo(1); + assertThat(s.total()).isEqualTo(4); + }); + } + + @Test + void forEachCachedReplicaPoolVisitsOnlyOpenPools() { + var replicaDescriptor = new DatasourceConnectionDescriptor( + id, organizationId, DbType.POSTGRESQL, "h", 5432, "db", "u", "ENC", SslMode.DISABLE, + 10, 1000, false, null, false, null, null, null, + "jdbc:postgresql://replica:5432/db", "ru", "ENC(rpw)", true); + var endpoint = replicaDescriptor.readReplicas().get(0); + var replicaPool = mock(HikariDataSource.class); + when(datasourceLookupService.findById(id)).thenReturn(Optional.of(replicaDescriptor)); + when(poolFactory.createReplicaPool(replicaDescriptor, endpoint)).thenReturn(replicaPool); + manager.resolveReplica(id, endpoint.id()); + + var visited = new java.util.ArrayList(); + manager.forEachCachedReplicaPool((dsId, endpointId, pool) -> visited.add(endpointId)); + assertThat(visited).containsExactly(endpoint.id()); + + when(replicaPool.isClosed()).thenReturn(true); + visited.clear(); + manager.forEachCachedReplicaPool((dsId, endpointId, pool) -> visited.add(endpointId)); + assertThat(visited).isEmpty(); + } } diff --git a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DefaultDatasourceHealthServiceTest.java b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DefaultDatasourceHealthServiceTest.java index 95ed7ebf..247b7b5a 100644 --- a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DefaultDatasourceHealthServiceTest.java +++ b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DefaultDatasourceHealthServiceTest.java @@ -49,7 +49,9 @@ class DefaultDatasourceHealthServiceTest { @BeforeEach void setUp() { service = new DefaultDatasourceHealthService(datasourceAdminService, poolManager, - queryStatsLookupService, clock, new ConcurrentMapCacheManager()); + queryStatsLookupService, + new ReplicaHealthRegistry(clock, new ProxyReplicaProperties(null, null, null)), + clock, new ConcurrentMapCacheManager()); } private static DatasourceView view(UUID id, UUID org, String name, DbType type, boolean active) { @@ -87,6 +89,33 @@ void emptyDatasourceYieldsZeroCountsNullPercentilesAndNullPool() { assertThat(row.poolMax()).isNull(); } + @Test + void replicaEndpointsSurfaceHealthAndPoolGauges() { + var dsId = UUID.randomUUID(); + var replicaId = UUID.randomUUID(); + var replicaView = new DatasourceView(dsId, orgId, "replicated-ds", DbType.POSTGRESQL, + "host", 5432, "db", "user", SslMode.DISABLE, 10, 1000, false, false, null, false, + null, false, null, null, null, + List.of(new DatasourceView.ReadReplicaView(replicaId, + "jdbc:postgresql://replica-a:5432/db", "ru")), + true, Instant.parse("2026-01-01T00:00:00Z"), null, false, null); + stubPage(replicaView); + when(queryStatsLookupService.statsFor(anyCollection(), any())).thenReturn(Map.of()); + when(poolManager.poolStats(dsId)).thenReturn(Optional.empty()); + when(poolManager.replicaPoolStats(dsId, replicaId)) + .thenReturn(Optional.of(new DatasourcePoolStats(2, 1, 0, 3, 10))); + + var row = service.snapshot(orgId, PageRequest.of(0, 50)).content().getFirst(); + + assertThat(row.replicas()).hasSize(1); + var replica = row.replicas().getFirst(); + assertThat(replica.endpointId()).isEqualTo(replicaId); + assertThat(replica.label()).isEqualTo("replica-a:5432"); + assertThat(replica.healthy()).isTrue(); + assertThat(replica.poolActive()).isEqualTo(2); + assertThat(replica.poolTotal()).isEqualTo(3); + } + @Test void failedOnlyDatasourceReportsErrorsAndLivePool() { var dsId = UUID.randomUUID(); diff --git a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DefaultQueryExecutorPostgresIntegrationTest.java b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DefaultQueryExecutorPostgresIntegrationTest.java index ee0c6781..9e95dcc0 100644 --- a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DefaultQueryExecutorPostgresIntegrationTest.java +++ b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DefaultQueryExecutorPostgresIntegrationTest.java @@ -158,6 +158,31 @@ void cleanup() { organizationRepository.deleteAll(); } + @Test + void transactionalEnvelopeBatchesHomogeneousInsertsAndPersistsAllRows() { + var request = new QueryExecutionRequest(datasource.getId(), + """ + BEGIN; + INSERT INTO items (id, name, qty) VALUES ('00000000-0000-0000-0000-000000000011', 'fig', 8); + INSERT INTO items (id, name, qty) VALUES ('00000000-0000-0000-0000-000000000012', 'grape', 9); + INSERT INTO items (id, name, qty) VALUES ('00000000-0000-0000-0000-000000000013', 'honeydew', 10); + COMMIT; + """, + QueryType.INSERT, null, null, List.of(), true, + List.of("INSERT INTO items (id, name, qty) VALUES ('00000000-0000-0000-0000-000000000011', 'fig', 8)", + "INSERT INTO items (id, name, qty) VALUES ('00000000-0000-0000-0000-000000000012', 'grape', 9)", + "INSERT INTO items (id, name, qty) VALUES ('00000000-0000-0000-0000-000000000013', 'honeydew', 10)")); + + var result = (UpdateExecutionResult) executor.execute(request); + + assertThat(result.rowsAffected()).isEqualTo(3L); + var select = new QueryExecutionRequest(datasource.getId(), + "SELECT name FROM items WHERE qty >= 8 ORDER BY qty", QueryType.SELECT, null, null); + var rows = (SelectExecutionResult) executor.execute(select); + assertThat(rows.rows()).extracting(row -> row.get(0)) + .containsExactly("fig", "grape", "honeydew"); + } + @Test void rowSecurityPredicateFiltersSelectedRows() { var policyId = UUID.randomUUID(); diff --git a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DefaultQueryExecutorReadReplicaIntegrationTest.java b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DefaultQueryExecutorReadReplicaIntegrationTest.java index a3449beb..6a13d869 100644 --- a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DefaultQueryExecutorReadReplicaIntegrationTest.java +++ b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DefaultQueryExecutorReadReplicaIntegrationTest.java @@ -39,10 +39,12 @@ import static org.assertj.core.api.Assertions.assertThat; /** - * End-to-end test of read-replica routing using two real PostgreSQL containers. The primary and - * replica are seeded with different rows so the test can prove the SELECT was answered by the - * replica; INSERT/UPDATE go to the primary; stopping the replica triggers a primary-fallback with - * a {@link AuditAction#DATASOURCE_REPLICA_FALLBACK} audit row. + * End-to-end test of multi-replica read routing (AF-457) using three real PostgreSQL containers + * (primary + two replicas). Each is seeded with a distinct row so the test can prove which + * database answered a SELECT: reads round-robin across the replicas, a downed replica is skipped + * in favour of its sibling, INSERT/UPDATE go to the primary, and only when every replica is down + * does the read fall back to the primary with a + * {@link AuditAction#DATASOURCE_REPLICA_FALLBACK} audit row. */ @SpringBootTest @ImportTestcontainers(TestcontainersConfig.class) @@ -60,6 +62,12 @@ class DefaultQueryExecutorReadReplicaIntegrationTest { .withUsername("r_user") .withPassword("r-pw"); + @SuppressWarnings({"rawtypes", "resource"}) + static PostgreSQLContainer replicaDb2 = new PostgreSQLContainer("postgres:18-alpine") + .withDatabaseName("replica2_db") + .withUsername("r2_user") + .withPassword("r2-pw"); + @Autowired QueryExecutor executor; @Autowired DatasourceConnectionPoolManager poolManager; @Autowired DatasourceRepository datasourceRepository; @@ -90,6 +98,7 @@ static void securityProperties(DynamicPropertyRegistry registry) throws Exceptio static void startContainers() { primaryDb.start(); replicaDb.start(); + replicaDb2.start(); } @AfterAll @@ -100,6 +109,9 @@ static void stopContainers() { if (replicaDb.isRunning()) { replicaDb.stop(); } + if (replicaDb2.isRunning()) { + replicaDb2.stop(); + } } @BeforeEach @@ -116,7 +128,8 @@ void setUp() throws Exception { datasource = saveDatasource(); seed(primaryDb, "primary-row"); - seed(replicaDb, "replica-row"); + seed(replicaDb, "replica-a-row"); + seed(replicaDb2, "replica-b-row"); } @SuppressWarnings("rawtypes") @@ -132,14 +145,40 @@ private void seed(PostgreSQLContainer container, String label) throws Exception } @Test - void selectIsServedByReplicaWhenConfigured() { + void selectsRoundRobinAcrossBothReplicas() { var request = new QueryExecutionRequest(datasource.getId(), "SELECT label FROM items", QueryType.SELECT, null, null); - var result = (SelectExecutionResult) executor.execute(request); + var first = (SelectExecutionResult) executor.execute(request); + var second = (SelectExecutionResult) executor.execute(request); - assertThat(result.rows()).hasSize(1); - assertThat(result.rows().get(0)).containsExactly("replica-row"); + var labels = java.util.Set.of( + (String) first.rows().get(0).get(0), + (String) second.rows().get(0).get(0)); + // Consecutive SELECTs alternate between the two replicas; neither hits the primary. + assertThat(labels).containsExactlyInAnyOrder("replica-a-row", "replica-b-row"); + } + + @Test + void downedReplicaIsSkippedInFavourOfSibling() { + // Warm both replica pools. + var request = new QueryExecutionRequest(datasource.getId(), + "SELECT label FROM items", QueryType.SELECT, null, null); + executor.execute(request); + executor.execute(request); + + replicaDb2.stop(); + // Evict cached pools so the downed endpoint fails fast at pool init instead of waiting + // out the Hikari connection timeout on a dead pooled connection. + poolManager.evict(datasource.getId()); + try { + for (int i = 0; i < 3; i++) { + var result = (SelectExecutionResult) executor.execute(request); + assertThat(result.rows().get(0)).containsExactly("replica-a-row"); + } + } finally { + replicaDb2.start(); + } } @Test @@ -160,16 +199,17 @@ void insertGoesToPrimaryEvenWhenReplicaConfigured() { } @Test - void selectFallsBackToPrimaryWhenReplicaUnreachable() { + void selectFallsBackToPrimaryWhenAllReplicasUnreachable() { // Warm the primary pool so the fallback path isn't constructing a pool from scratch. var warmup = new QueryExecutionRequest(datasource.getId(), "SELECT label FROM items", QueryType.SELECT, null, null); executor.execute(warmup); replicaDb.stop(); + replicaDb2.stop(); poolManager.evict(datasource.getId()); try { - // Reload primary pool only; replica resolve will fail after eviction. + // Reload primary pool only; every replica resolve fails after eviction. var fallback = new QueryExecutionRequest(datasource.getId(), "SELECT label FROM items", QueryType.SELECT, null, null); var result = (SelectExecutionResult) executor.execute(fallback); @@ -183,6 +223,7 @@ void selectFallsBackToPrimaryWhenReplicaUnreachable() { assertThat(audits.content()).isNotEmpty(); } finally { replicaDb.start(); + replicaDb2.start(); } } @@ -203,10 +244,24 @@ private DatasourceEntity saveDatasource() { ds.setRequireReviewReads(false); ds.setRequireReviewWrites(true); ds.setAiAnalysisEnabled(false); - ds.setReadReplicaJdbcUrl(replicaDb.getJdbcUrl()); - ds.setReadReplicaUsername(replicaDb.getUsername()); - ds.setReadReplicaPasswordEncrypted(encryptionService.encrypt(replicaDb.getPassword())); + addReplica(ds, replicaDb.getJdbcUrl(), replicaDb.getUsername(), + encryptionService.encrypt(replicaDb.getPassword())); + addReplica(ds, replicaDb2.getJdbcUrl(), replicaDb2.getUsername(), + encryptionService.encrypt(replicaDb2.getPassword())); ds.setActive(true); return datasourceRepository.save(ds); } + + private static void addReplica(DatasourceEntity ds, String jdbcUrl, String username, + String passwordEncrypted) { + var replica = new com.bablsoft.accessflow.core.internal.persistence.entity + .DatasourceReadReplicaEntity(); + replica.setId(UUID.randomUUID()); + replica.setDatasource(ds); + replica.setJdbcUrl(jdbcUrl); + replica.setUsername(username); + replica.setPasswordEncrypted(passwordEncrypted); + replica.setPosition(ds.getReadReplicas().size()); + ds.getReadReplicas().add(replica); + } } diff --git a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DefaultQueryExecutorTest.java b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DefaultQueryExecutorTest.java index 9d5e9219..3ec4e9ab 100644 --- a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DefaultQueryExecutorTest.java +++ b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/DefaultQueryExecutorTest.java @@ -51,6 +51,7 @@ import static org.assertj.core.api.Assertions.assertThat; import static org.assertj.core.api.Assertions.assertThatThrownBy; +import static org.mockito.ArgumentMatchers.any; import static org.mockito.ArgumentMatchers.anyString; import static org.mockito.ArgumentMatchers.contains; import static org.mockito.ArgumentMatchers.eq; @@ -74,13 +75,14 @@ class DefaultQueryExecutorTest { private final SqlExceptionTranslator translator = new SqlExceptionTranslator(messageSource); private final ProxyPoolProperties properties = new ProxyPoolProperties( null, null, null, null, null, - new ProxyPoolProperties.Execution(10_000, Duration.ofSeconds(30), 1_000)); + new ProxyPoolProperties.Execution(10_000, Duration.ofSeconds(30), 1_000, null)); private final AtomicLong nanos = new AtomicLong(); private final Clock clock = Clock.fixed(Instant.parse("2026-05-05T12:00:00Z"), ZoneOffset.UTC); private final com.bablsoft.accessflow.core.api.QueryEngineCatalog engineCatalog = mock(com.bablsoft.accessflow.core.api.QueryEngineCatalog.class); private final DryRunPlanner dryRunPlanner = mock(DryRunPlanner.class); + private final SelectResultCache resultCache = mock(SelectResultCache.class); private final SimpleMeterRegistry meterRegistry = new SimpleMeterRegistry(); private final ObservationRegistry observationRegistry = ObservationRegistry.create(); @@ -97,17 +99,20 @@ void setUp() throws SQLException { when(dataSource.getConnection()).thenReturn(connection); when(connection.prepareStatement(anyString())).thenReturn(statement); when(poolManager.resolve(datasourceId)).thenReturn(dataSource); - when(poolManager.resolveReplica(datasourceId)).thenReturn(Optional.empty()); + when(poolManager.replicaEndpoints(datasourceId)).thenReturn(List.of()); when(lookupService.findById(datasourceId)).thenReturn(Optional.of(descriptor(2_000))); observationRegistry.observationConfig() .observationHandler(new DefaultMeterObservationHandler(meterRegistry)); - var router = new RoutingDataSourceResolver(poolManager, lookupService, auditLogService, - messageSource, observationRegistry); + var healthRegistry = new ReplicaHealthRegistry(clock, + new ProxyReplicaProperties(null, null, null)); + var router = new RoutingDataSourceResolver(poolManager, healthRegistry, lookupService, + auditLogService, messageSource, observationRegistry); when(dryRunPlanner.supportedTypes()).thenReturn(java.util.Set.of(DbType.POSTGRESQL)); var dryRunRegistry = new DryRunPlannerRegistry(List.of(dryRunPlanner)); executor = new DefaultQueryExecutor(router, lookupService, properties, rowMapper, translator, new RowSecurityRewriter(messageSource), - engineCatalog, dryRunRegistry, clock, messageSource, observationRegistry); + engineCatalog, dryRunRegistry, resultCache, clock, messageSource, + observationRegistry); } @Test @@ -303,13 +308,10 @@ void overrideTimeoutBeatsConfiguredTimeout() throws SQLException { } @Test - void transactionalSumsRowsAffectedAndCommits() throws SQLException { - var stmt1 = mock(PreparedStatement.class); - var stmt2 = mock(PreparedStatement.class); - when(connection.prepareStatement("INSERT INTO t(id) VALUES (1)")).thenReturn(stmt1); - when(connection.prepareStatement("INSERT INTO t(id) VALUES (2)")).thenReturn(stmt2); - when(stmt1.executeLargeUpdate()).thenReturn(1L); - when(stmt2.executeLargeUpdate()).thenReturn(1L); + void transactionalBatchesHomogeneousInsertsAndCommits() throws SQLException { + var batchStmt = mock(PreparedStatement.class); + when(connection.prepareStatement("INSERT INTO t (id) VALUES (?)")).thenReturn(batchStmt); + when(batchStmt.executeLargeBatch()).thenReturn(new long[]{1L, 1L}); var request = new QueryExecutionRequest(datasourceId, "BEGIN; INSERT INTO t(id) VALUES (1); INSERT INTO t(id) VALUES (2); COMMIT;", @@ -319,6 +321,11 @@ void transactionalSumsRowsAffectedAndCommits() throws SQLException { var result = (UpdateExecutionResult) executor.execute(request); assertThat(result.rowsAffected()).isEqualTo(2L); + verify(batchStmt).setObject(1, 1L); + verify(batchStmt).setObject(1, 2L); + verify(batchStmt, times(2)).addBatch(); + verify(batchStmt, times(1)).executeLargeBatch(); + verify(batchStmt, never()).executeLargeUpdate(); verify(connection).setReadOnly(false); verify(connection).setAutoCommit(false); verify(connection).commit(); @@ -326,19 +333,83 @@ void transactionalSumsRowsAffectedAndCommits() throws SQLException { } @Test - void transactionalRollsBackOnFailure() throws SQLException { + void transactionalHeterogeneousStatementsUsePerStatementPath() throws SQLException { var stmt1 = mock(PreparedStatement.class); var stmt2 = mock(PreparedStatement.class); - when(connection.prepareStatement("INSERT INTO t(id) VALUES (1)")).thenReturn(stmt1); - when(connection.prepareStatement("INSERT INTO t(id) VALUES (1)/* dup */")).thenReturn(stmt2); + when(connection.prepareStatement("UPDATE t SET v = 1 WHERE id = 1")).thenReturn(stmt1); + when(connection.prepareStatement("DELETE FROM u WHERE id = 2")).thenReturn(stmt2); when(stmt1.executeLargeUpdate()).thenReturn(1L); - when(stmt2.executeLargeUpdate()) + when(stmt2.executeLargeUpdate()).thenReturn(1L); + + var request = new QueryExecutionRequest(datasourceId, + "BEGIN; UPDATE t SET v = 1 WHERE id = 1; DELETE FROM u WHERE id = 2; COMMIT;", + QueryType.UPDATE, null, null, List.of(), true, + List.of("UPDATE t SET v = 1 WHERE id = 1", "DELETE FROM u WHERE id = 2")); + + var result = (UpdateExecutionResult) executor.execute(request); + + assertThat(result.rowsAffected()).isEqualTo(2L); + verify(connection).commit(); + } + + @Test + void transactionalBatchFlushesEveryChunkSizeRows() throws SQLException { + var chunkedProperties = new ProxyPoolProperties( + null, null, null, null, null, + new ProxyPoolProperties.Execution(10_000, Duration.ofSeconds(30), 1_000, 2)); + var healthRegistry = new ReplicaHealthRegistry(clock, + new ProxyReplicaProperties(null, null, null)); + var router = new RoutingDataSourceResolver(poolManager, healthRegistry, lookupService, + auditLogService, messageSource, observationRegistry); + var chunkedExecutor = new DefaultQueryExecutor(router, lookupService, chunkedProperties, + rowMapper, translator, new RowSecurityRewriter(messageSource), + engineCatalog, new DryRunPlannerRegistry(List.of(dryRunPlanner)), resultCache, + clock, messageSource, observationRegistry); + var batchStmt = mock(PreparedStatement.class); + when(connection.prepareStatement("INSERT INTO t (id) VALUES (?)")).thenReturn(batchStmt); + when(batchStmt.executeLargeBatch()).thenReturn(new long[]{1L, 1L}, new long[]{1L}); + + var request = new QueryExecutionRequest(datasourceId, + "BEGIN; INSERT INTO t(id) VALUES (1); INSERT INTO t(id) VALUES (2);" + + " INSERT INTO t(id) VALUES (3); COMMIT;", + QueryType.INSERT, null, null, List.of(), true, + List.of("INSERT INTO t(id) VALUES (1)", "INSERT INTO t(id) VALUES (2)", + "INSERT INTO t(id) VALUES (3)")); + + var result = (UpdateExecutionResult) chunkedExecutor.execute(request); + + assertThat(result.rowsAffected()).isEqualTo(3L); + verify(batchStmt, times(2)).executeLargeBatch(); + } + + @Test + void transactionalBatchCountsSuccessNoInfoAsOneRow() throws SQLException { + var batchStmt = mock(PreparedStatement.class); + when(connection.prepareStatement("INSERT INTO t (id) VALUES (?)")).thenReturn(batchStmt); + when(batchStmt.executeLargeBatch()) + .thenReturn(new long[]{java.sql.Statement.SUCCESS_NO_INFO, 1L}); + + var request = new QueryExecutionRequest(datasourceId, + "BEGIN; INSERT INTO t(id) VALUES (1); INSERT INTO t(id) VALUES (2); COMMIT;", + QueryType.INSERT, null, null, List.of(), true, + List.of("INSERT INTO t(id) VALUES (1)", "INSERT INTO t(id) VALUES (2)")); + + var result = (UpdateExecutionResult) executor.execute(request); + + assertThat(result.rowsAffected()).isEqualTo(2L); + } + + @Test + void transactionalRollsBackOnFailure() throws SQLException { + var batchStmt = mock(PreparedStatement.class); + when(connection.prepareStatement("INSERT INTO t (id) VALUES (?)")).thenReturn(batchStmt); + when(batchStmt.executeLargeBatch()) .thenThrow(new SQLException("duplicate key", "23505", 7)); var request = new QueryExecutionRequest(datasourceId, - "BEGIN; INSERT INTO t(id) VALUES (1); INSERT INTO t(id) VALUES (1)/* dup */; COMMIT;", + "BEGIN; INSERT INTO t(id) VALUES (1); INSERT INTO t(id) VALUES (1); COMMIT;", QueryType.INSERT, null, null, List.of(), true, - List.of("INSERT INTO t(id) VALUES (1)", "INSERT INTO t(id) VALUES (1)/* dup */")); + List.of("INSERT INTO t(id) VALUES (1)", "INSERT INTO t(id) VALUES (1)")); assertThatThrownBy(() -> executor.execute(request)) .isInstanceOf(QueryExecutionFailedException.class); @@ -347,6 +418,97 @@ void transactionalRollsBackOnFailure() throws SQLException { verify(connection, never()).commit(); } + @Test + void selectCacheHitReturnsCachedResultWithoutTouchingJdbc() throws SQLException { + var cached = new SelectExecutionResult(List.of(), List.of(), 0, false, Duration.ZERO); + when(resultCache.enabledFor(any())).thenReturn(true); + when(resultCache.get(eq(datasourceId), anyString(), any(Duration.class))) + .thenReturn(Optional.of(cached)); + + var request = new QueryExecutionRequest(datasourceId, "SELECT v FROM t", + QueryType.SELECT, null, null, List.of(), List.of(), List.of(), false, null, + List.of(), java.util.Set.of("t")); + + var result = executor.execute(request); + + assertThat(result).isSameAs(cached); + verify(poolManager, never()).resolve(datasourceId); + verify(resultCache, never()).put(any(), anyString(), any(), any(), any()); + } + + @Test + void selectCacheMissExecutesAndStoresResult() throws SQLException { + var rs = emptyResultSet(); + when(statement.executeQuery()).thenReturn(rs); + when(resultCache.enabledFor(any())).thenReturn(true); + when(resultCache.get(eq(datasourceId), anyString(), any(Duration.class))) + .thenReturn(Optional.empty()); + when(resultCache.ttlFor(any())).thenReturn(Duration.ofSeconds(45)); + + var request = new QueryExecutionRequest(datasourceId, "SELECT v FROM t", + QueryType.SELECT, null, null, List.of(), List.of(), List.of(), false, null, + List.of(), java.util.Set.of("t")); + + var result = executor.execute(request); + + assertThat(result).isInstanceOf(SelectExecutionResult.class); + verify(resultCache).put(eq(datasourceId), anyString(), eq(java.util.Set.of("t")), + eq(Duration.ofSeconds(45)), any(SelectExecutionResult.class)); + } + + @Test + void selectWithUnknownReferencedTablesIsNeverCached() throws SQLException { + var rs = emptyResultSet(); + when(statement.executeQuery()).thenReturn(rs); + when(resultCache.enabledFor(any())).thenReturn(true); + + executor.execute(new QueryExecutionRequest(datasourceId, "SELECT 1", + QueryType.SELECT, null, null)); + + verify(resultCache, never()).get(any(), anyString(), any()); + verify(resultCache, never()).put(any(), anyString(), any(), any(), any()); + } + + @Test + void selectWithCacheDisabledForDatasourceSkipsCache() throws SQLException { + var rs = emptyResultSet(); + when(statement.executeQuery()).thenReturn(rs); + when(resultCache.enabledFor(any())).thenReturn(false); + + executor.execute(new QueryExecutionRequest(datasourceId, "SELECT v FROM t", + QueryType.SELECT, null, null, List.of(), List.of(), List.of(), false, null, + List.of(), java.util.Set.of("t"))); + + verify(resultCache, never()).get(any(), anyString(), any()); + verify(resultCache, never()).put(any(), anyString(), any(), any(), any()); + } + + @Test + void updateInvalidatesCachedEntriesForReferencedTables() throws SQLException { + when(statement.executeLargeUpdate()).thenReturn(3L); + + executor.execute(new QueryExecutionRequest(datasourceId, "UPDATE t SET v = 1", + QueryType.UPDATE, null, null, List.of(), List.of(), List.of(), false, null, + List.of(), java.util.Set.of("t"))); + + verify(resultCache).invalidateTables(datasourceId, java.util.Set.of("t")); + } + + @Test + void transactionalCommitInvalidatesCachedEntriesForReferencedTables() throws SQLException { + var batchStmt = mock(PreparedStatement.class); + when(connection.prepareStatement("INSERT INTO t (id) VALUES (?)")).thenReturn(batchStmt); + when(batchStmt.executeLargeBatch()).thenReturn(new long[]{1L, 1L}); + + executor.execute(new QueryExecutionRequest(datasourceId, + "BEGIN; INSERT INTO t(id) VALUES (1); INSERT INTO t(id) VALUES (2); COMMIT;", + QueryType.INSERT, null, null, List.of(), List.of(), List.of(), true, + List.of("INSERT INTO t(id) VALUES (1)", "INSERT INTO t(id) VALUES (2)"), + List.of(), java.util.Set.of("t"))); + + verify(resultCache).invalidateTables(datasourceId, java.util.Set.of("t")); + } + @Test void selectAppliesRowSecurityPredicateAndBindsParameter() throws SQLException { var rs = emptyResultSet(); diff --git a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ProxyCachePropertiesTest.java b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ProxyCachePropertiesTest.java new file mode 100644 index 00000000..913ea353 --- /dev/null +++ b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ProxyCachePropertiesTest.java @@ -0,0 +1,26 @@ +package com.bablsoft.accessflow.proxy.internal; + +import org.junit.jupiter.api.Test; + +import java.time.Duration; + +import static org.assertj.core.api.Assertions.assertThat; + +class ProxyCachePropertiesTest { + + @Test + void appliesDefaultsWhenUnset() { + var props = new ProxyCacheProperties(null, null, null); + assertThat(props.enabled()).isTrue(); + assertThat(props.defaultTtl()).isEqualTo(Duration.ofSeconds(60)); + assertThat(props.maxEntryBytes()).isEqualTo(1_000_000L); + } + + @Test + void keepsProvidedValues() { + var props = new ProxyCacheProperties(false, Duration.ofMinutes(5), 42L); + assertThat(props.enabled()).isFalse(); + assertThat(props.defaultTtl()).isEqualTo(Duration.ofMinutes(5)); + assertThat(props.maxEntryBytes()).isEqualTo(42L); + } +} diff --git a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ProxyPoolPropertiesTest.java b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ProxyPoolPropertiesTest.java index 91afed51..0fdd9944 100644 --- a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ProxyPoolPropertiesTest.java +++ b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ProxyPoolPropertiesTest.java @@ -2,8 +2,11 @@ import com.bablsoft.accessflow.proxy.internal.ProxyPoolProperties.Execution; import org.junit.jupiter.api.Test; +import org.springframework.boot.context.properties.bind.Binder; +import org.springframework.boot.context.properties.source.MapConfigurationPropertySource; import java.time.Duration; +import java.util.Map; import static org.assertj.core.api.Assertions.assertThat; @@ -21,6 +24,34 @@ void allNullArgumentsYieldDocumentedDefaults() { assertThat(props.execution().maxRows()).isEqualTo(10_000); assertThat(props.execution().statementTimeout()).isEqualTo(Duration.ofSeconds(30)); assertThat(props.execution().defaultFetchSize()).isEqualTo(1_000); + assertThat(props.execution().insertBatchChunkSize()).isEqualTo(1_000); + } + + /** + * Regression guard: record constructor-binding silently degrades to all-defaults when the + * record grows a second constructor — this binds through Spring's Binder exactly like + * application.yml does, so an override that stops binding fails here. + */ + @Test + void bindsNestedExecutionOverridesThroughTheBinder() { + var source = new MapConfigurationPropertySource(Map.of( + "accessflow.proxy.execution.statement-timeout", "3s", + "accessflow.proxy.execution.max-rows", "500", + "accessflow.proxy.execution.insert-batch-chunk-size", "42", + "accessflow.proxy.cache.default-ttl", "PT2M", + "accessflow.proxy.replica.cooldown", "PT10S")); + var binder = new Binder(source); + + var pool = binder.bind("accessflow.proxy", ProxyPoolProperties.class).get(); + assertThat(pool.execution().statementTimeout()).isEqualTo(Duration.ofSeconds(3)); + assertThat(pool.execution().maxRows()).isEqualTo(500); + assertThat(pool.execution().insertBatchChunkSize()).isEqualTo(42); + + var cache = binder.bind("accessflow.proxy.cache", ProxyCacheProperties.class).get(); + assertThat(cache.defaultTtl()).isEqualTo(Duration.ofMinutes(2)); + + var replica = binder.bind("accessflow.proxy.replica", ProxyReplicaProperties.class).get(); + assertThat(replica.cooldown()).isEqualTo(Duration.ofSeconds(10)); } @Test @@ -28,7 +59,7 @@ void explicitValuesPassThroughUnchanged() { var props = new ProxyPoolProperties( Duration.ofSeconds(5), Duration.ofSeconds(60), Duration.ofMinutes(15), Duration.ofSeconds(2), "custom-", - new Execution(500, Duration.ofSeconds(7), 250)); + new Execution(500, Duration.ofSeconds(7), 250, 100)); assertThat(props.connectionTimeout()).isEqualTo(Duration.ofSeconds(5)); assertThat(props.idleTimeout()).isEqualTo(Duration.ofSeconds(60)); @@ -42,10 +73,11 @@ void explicitValuesPassThroughUnchanged() { @Test void executionRecordYieldsDefaultsForNullFields() { - var execution = new Execution(null, null, null); + var execution = new Execution(null, null, null, null); assertThat(execution.maxRows()).isEqualTo(10_000); assertThat(execution.statementTimeout()).isEqualTo(Duration.ofSeconds(30)); assertThat(execution.defaultFetchSize()).isEqualTo(1_000); + assertThat(execution.insertBatchChunkSize()).isEqualTo(1_000); } } diff --git a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ProxyReplicaPropertiesTest.java b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ProxyReplicaPropertiesTest.java new file mode 100644 index 00000000..b19aa546 --- /dev/null +++ b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ProxyReplicaPropertiesTest.java @@ -0,0 +1,27 @@ +package com.bablsoft.accessflow.proxy.internal; + +import org.junit.jupiter.api.Test; + +import java.time.Duration; + +import static org.assertj.core.api.Assertions.assertThat; + +class ProxyReplicaPropertiesTest { + + @Test + void appliesDefaultsWhenUnset() { + var props = new ProxyReplicaProperties(null, null, null); + assertThat(props.probeInterval()).isEqualTo(Duration.ofSeconds(30)); + assertThat(props.probeTimeout()).isEqualTo(Duration.ofSeconds(5)); + assertThat(props.cooldown()).isEqualTo(Duration.ofSeconds(30)); + } + + @Test + void keepsProvidedValues() { + var props = new ProxyReplicaProperties( + Duration.ofMinutes(1), Duration.ofSeconds(2), Duration.ofSeconds(10)); + assertThat(props.probeInterval()).isEqualTo(Duration.ofMinutes(1)); + assertThat(props.probeTimeout()).isEqualTo(Duration.ofSeconds(2)); + assertThat(props.cooldown()).isEqualTo(Duration.ofSeconds(10)); + } +} diff --git a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ReplicaHealthProberTest.java b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ReplicaHealthProberTest.java new file mode 100644 index 00000000..bd0f262f --- /dev/null +++ b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ReplicaHealthProberTest.java @@ -0,0 +1,104 @@ +package com.bablsoft.accessflow.proxy.internal; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; +import org.mockito.junit.jupiter.MockitoSettings; +import org.mockito.quality.Strictness; +import org.springframework.scheduling.TaskScheduler; + +import javax.sql.DataSource; +import java.sql.Connection; +import java.sql.SQLException; +import java.time.Duration; +import java.util.UUID; +import java.util.concurrent.ScheduledFuture; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.anyInt; +import static org.mockito.Mockito.doAnswer; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +@ExtendWith(MockitoExtension.class) +@MockitoSettings(strictness = Strictness.LENIENT) +class ReplicaHealthProberTest { + + private final UUID datasourceId = UUID.randomUUID(); + private final UUID endpointId = UUID.randomUUID(); + + @Mock + private DefaultDatasourceConnectionPoolManager poolManager; + @Mock + private TaskScheduler taskScheduler; + @Mock + private DataSource pool; + @Mock + private Connection connection; + + private ReplicaHealthRegistry registry; + private ReplicaHealthProber prober; + + @BeforeEach + void setUp() throws SQLException { + registry = new ReplicaHealthRegistry(java.time.Clock.systemUTC(), + new ProxyReplicaProperties(null, Duration.ofSeconds(2), null)); + prober = new ReplicaHealthProber(poolManager, registry, + new ProxyReplicaProperties(null, Duration.ofSeconds(2), null), taskScheduler); + when(pool.getConnection()).thenReturn(connection); + doAnswer(invocation -> { + var visitor = invocation + .getArgument(0, DefaultDatasourceConnectionPoolManager.ReplicaPoolVisitor.class); + visitor.visit(datasourceId, endpointId, pool); + return null; + }).when(poolManager).forEachCachedReplicaPool(any()); + } + + @Test + void validConnectionRecordsSuccess() throws SQLException { + registry.recordFailure(datasourceId, endpointId); + when(connection.isValid(anyInt())).thenReturn(true); + + prober.probeAll(); + + assertThat(registry.isHealthy(datasourceId, endpointId)).isTrue(); + verify(connection).isValid(2); + verify(connection).close(); + } + + @Test + void invalidConnectionRecordsFailure() throws SQLException { + when(connection.isValid(anyInt())).thenReturn(false); + + prober.probeAll(); + + assertThat(registry.isHealthy(datasourceId, endpointId)).isFalse(); + } + + @Test + void connectionFailureRecordsFailure() throws SQLException { + when(pool.getConnection()).thenThrow(new SQLException("gone")); + + prober.probeAll(); + + assertThat(registry.isHealthy(datasourceId, endpointId)).isFalse(); + } + + @Test + void startSchedulesAndStopCancels() { + ScheduledFuture future = mock(ScheduledFuture.class); + org.mockito.Mockito.doReturn(future).when(taskScheduler) + .scheduleWithFixedDelay(any(Runnable.class), any(Duration.class)); + + prober.start(); + prober.stop(); + + verify(taskScheduler).scheduleWithFixedDelay(any(Runnable.class), + org.mockito.ArgumentMatchers.eq(Duration.ofSeconds(30))); + verify(future).cancel(false); + } +} diff --git a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ReplicaHealthRegistryTest.java b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ReplicaHealthRegistryTest.java new file mode 100644 index 00000000..0cd877c1 --- /dev/null +++ b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ReplicaHealthRegistryTest.java @@ -0,0 +1,93 @@ +package com.bablsoft.accessflow.proxy.internal; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; + +import java.time.Clock; +import java.time.Duration; +import java.time.Instant; +import java.time.ZoneOffset; +import java.util.UUID; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + +class ReplicaHealthRegistryTest { + + private final UUID datasourceId = UUID.randomUUID(); + private final UUID endpointId = UUID.randomUUID(); + private final Instant base = Instant.parse("2026-07-16T12:00:00Z"); + private Clock clock; + private ReplicaHealthRegistry registry; + + @BeforeEach + void setUp() { + clock = mock(Clock.class); + when(clock.instant()).thenReturn(base); + registry = new ReplicaHealthRegistry(clock, + new ProxyReplicaProperties(null, null, Duration.ofSeconds(30))); + } + + @Test + void unknownEndpointIsHealthyCandidate() { + assertThat(registry.isCandidate(datasourceId, endpointId)).isTrue(); + assertThat(registry.isHealthy(datasourceId, endpointId)).isTrue(); + } + + @Test + void failureOpensCircuitForCooldown() { + registry.recordFailure(datasourceId, endpointId); + + assertThat(registry.isCandidate(datasourceId, endpointId)).isFalse(); + assertThat(registry.isHealthy(datasourceId, endpointId)).isFalse(); + } + + @Test + void cooldownElapsedAdmitsSingleHalfOpenTrial() { + registry.recordFailure(datasourceId, endpointId); + when(clock.instant()).thenReturn(base.plusSeconds(31)); + + assertThat(registry.isCandidate(datasourceId, endpointId)).isTrue(); + // Second caller is turned away while the trial is outstanding. + assertThat(registry.isCandidate(datasourceId, endpointId)).isFalse(); + assertThat(registry.isHealthy(datasourceId, endpointId)).isFalse(); + } + + @Test + void successAfterHalfOpenTrialRestoresHealthy() { + registry.recordFailure(datasourceId, endpointId); + when(clock.instant()).thenReturn(base.plusSeconds(31)); + assertThat(registry.isCandidate(datasourceId, endpointId)).isTrue(); + + registry.recordSuccess(datasourceId, endpointId); + + assertThat(registry.isHealthy(datasourceId, endpointId)).isTrue(); + assertThat(registry.isCandidate(datasourceId, endpointId)).isTrue(); + } + + @Test + void failureAfterHalfOpenTrialRearmsCooldown() { + registry.recordFailure(datasourceId, endpointId); + when(clock.instant()).thenReturn(base.plusSeconds(31)); + assertThat(registry.isCandidate(datasourceId, endpointId)).isTrue(); + + registry.recordFailure(datasourceId, endpointId); + + assertThat(registry.isCandidate(datasourceId, endpointId)).isFalse(); + when(clock.instant()).thenReturn(base.plusSeconds(62)); + assertThat(registry.isCandidate(datasourceId, endpointId)).isTrue(); + } + + @Test + void evictDropsAllEndpointStateForDatasource() { + var otherDatasource = UUID.randomUUID(); + registry.recordFailure(datasourceId, endpointId); + registry.recordFailure(otherDatasource, endpointId); + + registry.evict(datasourceId); + + assertThat(registry.isHealthy(datasourceId, endpointId)).isTrue(); + assertThat(registry.isHealthy(otherDatasource, endpointId)).isFalse(); + } +} diff --git a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ResultCacheEvictionListenerTest.java b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ResultCacheEvictionListenerTest.java new file mode 100644 index 00000000..91f32aff --- /dev/null +++ b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/ResultCacheEvictionListenerTest.java @@ -0,0 +1,44 @@ +package com.bablsoft.accessflow.proxy.internal; + +import com.bablsoft.accessflow.core.events.DatasourceCacheConfigChangedEvent; +import com.bablsoft.accessflow.core.events.DatasourceConfigChangedEvent; +import com.bablsoft.accessflow.core.events.DatasourceDeactivatedEvent; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.mockito.InjectMocks; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; + +import java.util.UUID; + +import static org.mockito.Mockito.verify; + +@ExtendWith(MockitoExtension.class) +class ResultCacheEvictionListenerTest { + + private final UUID datasourceId = UUID.randomUUID(); + + @Mock + private SelectResultCache resultCache; + + @InjectMocks + private ResultCacheEvictionListener listener; + + @Test + void configChangePurgesDatasourceCache() { + listener.onConfigChanged(new DatasourceConfigChangedEvent(datasourceId)); + verify(resultCache).invalidateAll(datasourceId); + } + + @Test + void deactivationPurgesDatasourceCache() { + listener.onDeactivated(new DatasourceDeactivatedEvent(datasourceId)); + verify(resultCache).invalidateAll(datasourceId); + } + + @Test + void cacheConfigChangePurgesDatasourceCache() { + listener.onCacheConfigChanged(new DatasourceCacheConfigChangedEvent(datasourceId)); + verify(resultCache).invalidateAll(datasourceId); + } +} diff --git a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/RoutingDataSourceResolverTest.java b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/RoutingDataSourceResolverTest.java index ddabd8b0..c1008531 100644 --- a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/RoutingDataSourceResolverTest.java +++ b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/RoutingDataSourceResolverTest.java @@ -11,6 +11,7 @@ import com.bablsoft.accessflow.core.api.SslMode; import com.bablsoft.accessflow.proxy.api.DatasourceConnectionPoolManager; import com.bablsoft.accessflow.proxy.api.PoolInitializationException; +import com.bablsoft.accessflow.proxy.api.ReplicaEndpointRef; import io.micrometer.core.instrument.observation.DefaultMeterObservationHandler; import io.micrometer.core.instrument.simple.SimpleMeterRegistry; import io.micrometer.observation.ObservationRegistry; @@ -22,6 +23,11 @@ import javax.sql.DataSource; import java.sql.Connection; import java.sql.SQLException; +import java.time.Clock; +import java.time.Duration; +import java.time.Instant; +import java.time.ZoneOffset; +import java.util.List; import java.util.Optional; import java.util.UUID; @@ -37,38 +43,57 @@ class RoutingDataSourceResolverTest { private final UUID datasourceId = UUID.randomUUID(); private final UUID organizationId = UUID.randomUUID(); + private final UUID endpointA = UUID.randomUUID(); + private final UUID endpointB = UUID.randomUUID(); private final DatasourceConnectionPoolManager poolManager = mock(DatasourceConnectionPoolManager.class); private final DatasourceLookupService lookupService = mock(DatasourceLookupService.class); private final AuditLogService auditLogService = mock(AuditLogService.class); private final StaticMessageSource messageSource = new StaticMessageSource(); private final SimpleMeterRegistry meterRegistry = new SimpleMeterRegistry(); private final ObservationRegistry observationRegistry = ObservationRegistry.create(); + private ReplicaHealthRegistry healthRegistry; private RoutingDataSourceResolver resolver; private DataSource primaryPool; - private DataSource replicaPool; + private DataSource replicaPoolA; + private DataSource replicaPoolB; private Connection primaryConnection; - private Connection replicaConnection; + private Connection replicaConnectionA; + private Connection replicaConnectionB; @BeforeEach void setUp() throws SQLException { messageSource.addMessage("error.datasource_unavailable_not_found", java.util.Locale.ENGLISH, "Datasource not found"); primaryPool = mock(DataSource.class); - replicaPool = mock(DataSource.class); + replicaPoolA = mock(DataSource.class); + replicaPoolB = mock(DataSource.class); primaryConnection = mock(Connection.class); - replicaConnection = mock(Connection.class); + replicaConnectionA = mock(Connection.class); + replicaConnectionB = mock(Connection.class); when(primaryPool.getConnection()).thenReturn(primaryConnection); - when(replicaPool.getConnection()).thenReturn(replicaConnection); + when(replicaPoolA.getConnection()).thenReturn(replicaConnectionA); + when(replicaPoolB.getConnection()).thenReturn(replicaConnectionB); when(poolManager.resolve(datasourceId)).thenReturn(primaryPool); observationRegistry.observationConfig() .observationHandler(new DefaultMeterObservationHandler(meterRegistry)); - resolver = new RoutingDataSourceResolver(poolManager, lookupService, auditLogService, - messageSource, observationRegistry); + healthRegistry = new ReplicaHealthRegistry( + Clock.fixed(Instant.parse("2026-07-16T12:00:00Z"), ZoneOffset.UTC), + new ProxyReplicaProperties(null, null, Duration.ofSeconds(30))); + resolver = new RoutingDataSourceResolver(poolManager, healthRegistry, lookupService, + auditLogService, messageSource, observationRegistry); + } + + private void givenReplicas(UUID... endpointIds) { + var refs = new java.util.ArrayList(); + for (int i = 0; i < endpointIds.length; i++) { + refs.add(new ReplicaEndpointRef(endpointIds[i], "replica-" + i + ":5432")); + } + when(poolManager.replicaEndpoints(datasourceId)).thenReturn(List.copyOf(refs)); } @Test void recordsAcquireObservationWithQueryTypeAndSuccessOutcome() throws SQLException { - when(poolManager.resolveReplica(datasourceId)).thenReturn(Optional.empty()); + givenReplicas(); resolver.acquire(datasourceId, QueryType.SELECT); @@ -82,13 +107,13 @@ void nonSelectAlwaysUsesPrimary() throws SQLException { var connection = resolver.acquire(datasourceId, QueryType.INSERT); assertThat(connection).isSameAs(primaryConnection); - verify(poolManager, never()).resolveReplica(datasourceId); + verify(poolManager, never()).replicaEndpoints(datasourceId); verify(auditLogService, never()).record(any()); } @Test void selectWithoutReplicaUsesPrimary() throws SQLException { - when(poolManager.resolveReplica(datasourceId)).thenReturn(Optional.empty()); + givenReplicas(); var connection = resolver.acquire(datasourceId, QueryType.SELECT); @@ -97,20 +122,58 @@ void selectWithoutReplicaUsesPrimary() throws SQLException { } @Test - void selectWithReplicaUsesReplica() throws SQLException { - when(poolManager.resolveReplica(datasourceId)).thenReturn(Optional.of(replicaPool)); + void selectWithSingleReplicaUsesReplica() throws SQLException { + givenReplicas(endpointA); + when(poolManager.resolveReplica(datasourceId, endpointA)).thenReturn(replicaPoolA); var connection = resolver.acquire(datasourceId, QueryType.SELECT); - assertThat(connection).isSameAs(replicaConnection); + assertThat(connection).isSameAs(replicaConnectionA); verify(poolManager, never()).resolve(datasourceId); verify(auditLogService, never()).record(any()); } @Test - void selectFallsBackToPrimaryAndAuditsWhenReplicaConnectionFails() throws SQLException { - when(poolManager.resolveReplica(datasourceId)).thenReturn(Optional.of(replicaPool)); - when(replicaPool.getConnection()).thenThrow(new SQLException("conn refused")); + void selectsRoundRobinAcrossHealthyReplicas() throws SQLException { + givenReplicas(endpointA, endpointB); + when(poolManager.resolveReplica(datasourceId, endpointA)).thenReturn(replicaPoolA); + when(poolManager.resolveReplica(datasourceId, endpointB)).thenReturn(replicaPoolB); + + var first = resolver.acquire(datasourceId, QueryType.SELECT); + var second = resolver.acquire(datasourceId, QueryType.SELECT); + var third = resolver.acquire(datasourceId, QueryType.SELECT); + + assertThat(first).isSameAs(replicaConnectionA); + assertThat(second).isSameAs(replicaConnectionB); + assertThat(third).isSameAs(replicaConnectionA); + verify(auditLogService, never()).record(any()); + } + + @Test + void failedEndpointIsSkippedAndNextCandidateServes() throws SQLException { + givenReplicas(endpointA, endpointB); + when(poolManager.resolveReplica(datasourceId, endpointA)).thenReturn(replicaPoolA); + when(poolManager.resolveReplica(datasourceId, endpointB)).thenReturn(replicaPoolB); + when(replicaPoolA.getConnection()).thenThrow(new SQLException("conn refused")); + + var first = resolver.acquire(datasourceId, QueryType.SELECT); + // Endpoint A is now DOWN — the next SELECT must not try it again. + var second = resolver.acquire(datasourceId, QueryType.SELECT); + + assertThat(first).isSameAs(replicaConnectionB); + assertThat(second).isSameAs(replicaConnectionB); + verify(replicaPoolA, times(1)).getConnection(); + // A sibling replica served the read: no full exhaustion, no audit. + verify(auditLogService, never()).record(any()); + } + + @Test + void selectFallsBackToPrimaryAndAuditsOnceWhenAllReplicasFail() throws SQLException { + givenReplicas(endpointA, endpointB); + when(poolManager.resolveReplica(datasourceId, endpointA)).thenReturn(replicaPoolA); + when(poolManager.resolveReplica(datasourceId, endpointB)).thenReturn(replicaPoolB); + when(replicaPoolA.getConnection()).thenThrow(new SQLException("a gone")); + when(replicaPoolB.getConnection()).thenThrow(new SQLException("b gone")); when(lookupService.findById(datasourceId)).thenReturn(Optional.of(activeDescriptor())); var connection = resolver.acquire(datasourceId, QueryType.SELECT); @@ -125,13 +188,31 @@ void selectFallsBackToPrimaryAndAuditsWhenReplicaConnectionFails() throws SQLExc assertThat(entry.organizationId()).isEqualTo(organizationId); assertThat(entry.actorId()).isNull(); assertThat(entry.metadata()) - .containsEntry("error", "conn refused") - .containsEntry("query_type", "SELECT"); + .containsEntry("query_type", "SELECT") + .containsEntry("replica_count", 2) + .containsKey("error") + .containsKey("tried_endpoints"); + } + + @Test + void openCircuitsServePrimaryWithoutAdditionalAudit() throws SQLException { + givenReplicas(endpointA); + when(poolManager.resolveReplica(datasourceId, endpointA)).thenReturn(replicaPoolA); + when(replicaPoolA.getConnection()).thenThrow(new SQLException("gone")); + when(lookupService.findById(datasourceId)).thenReturn(Optional.of(activeDescriptor())); + + resolver.acquire(datasourceId, QueryType.SELECT); // fails + audits, opens circuit + var second = resolver.acquire(datasourceId, QueryType.SELECT); // circuit open: no attempt + + assertThat(second).isSameAs(primaryConnection); + verify(replicaPoolA, times(1)).getConnection(); + verify(auditLogService, times(1)).record(any()); } @Test void selectFallsBackToPrimaryAndAuditsWhenReplicaPoolInitFails() throws SQLException { - when(poolManager.resolveReplica(datasourceId)).thenThrow( + givenReplicas(endpointA); + when(poolManager.resolveReplica(datasourceId, endpointA)).thenThrow( new PoolInitializationException("pool init failed", new RuntimeException())); when(lookupService.findById(datasourceId)).thenReturn(Optional.of(activeDescriptor())); @@ -143,8 +224,9 @@ void selectFallsBackToPrimaryAndAuditsWhenReplicaPoolInitFails() throws SQLExcep @Test void primaryFailurePropagatesAfterReplicaFallback() throws SQLException { - when(poolManager.resolveReplica(datasourceId)).thenReturn(Optional.of(replicaPool)); - when(replicaPool.getConnection()).thenThrow(new SQLException("replica gone")); + givenReplicas(endpointA); + when(poolManager.resolveReplica(datasourceId, endpointA)).thenReturn(replicaPoolA); + when(replicaPoolA.getConnection()).thenThrow(new SQLException("replica gone")); when(primaryPool.getConnection()).thenThrow(new SQLException("primary gone")); when(lookupService.findById(datasourceId)).thenReturn(Optional.of(activeDescriptor())); @@ -156,8 +238,9 @@ void primaryFailurePropagatesAfterReplicaFallback() throws SQLException { @Test void auditFailureIsSwallowed() throws SQLException { - when(poolManager.resolveReplica(datasourceId)).thenReturn(Optional.of(replicaPool)); - when(replicaPool.getConnection()).thenThrow(new SQLException("replica gone")); + givenReplicas(endpointA); + when(poolManager.resolveReplica(datasourceId, endpointA)).thenReturn(replicaPoolA); + when(replicaPoolA.getConnection()).thenThrow(new SQLException("replica gone")); when(lookupService.findById(datasourceId)).thenReturn(Optional.of(activeDescriptor())); when(auditLogService.record(any())).thenThrow(new RuntimeException("audit broken")); @@ -166,6 +249,21 @@ void auditFailureIsSwallowed() throws SQLException { assertThat(connection).isSameAs(primaryConnection); } + @Test + void successfulConnectionRecordsSuccessAndClosesHalfOpenTrial() throws SQLException { + givenReplicas(endpointA); + when(poolManager.resolveReplica(datasourceId, endpointA)).thenReturn(replicaPoolA); + healthRegistry.recordFailure(datasourceId, endpointA); + // Cooldown is 30s on a fixed clock — not yet a candidate. + assertThat(healthRegistry.isCandidate(datasourceId, endpointA)).isFalse(); + healthRegistry.recordSuccess(datasourceId, endpointA); // simulate prober recovery + + var connection = resolver.acquire(datasourceId, QueryType.SELECT); + + assertThat(connection).isSameAs(replicaConnectionA); + assertThat(healthRegistry.isHealthy(datasourceId, endpointA)).isTrue(); + } + private DatasourceConnectionDescriptor activeDescriptor() { return new DatasourceConnectionDescriptor(datasourceId, organizationId, DbType.POSTGRESQL, "h", 5432, "db", "u", "ENC", SslMode.DISABLE, 10, 1000, diff --git a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/SelectResultCacheIntegrationTest.java b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/SelectResultCacheIntegrationTest.java new file mode 100644 index 00000000..fc136385 --- /dev/null +++ b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/SelectResultCacheIntegrationTest.java @@ -0,0 +1,139 @@ +package com.bablsoft.accessflow.proxy.internal; + +import com.bablsoft.accessflow.TestcontainersConfig; +import com.bablsoft.accessflow.core.api.DbType; +import com.bablsoft.accessflow.core.api.QueryExecutionRequest; +import com.bablsoft.accessflow.core.api.QueryType; +import com.bablsoft.accessflow.core.api.SelectExecutionResult; +import com.bablsoft.accessflow.core.api.SslMode; +import com.bablsoft.accessflow.core.api.CredentialEncryptionService; +import com.bablsoft.accessflow.core.internal.persistence.entity.DatasourceEntity; +import com.bablsoft.accessflow.core.internal.persistence.entity.OrganizationEntity; +import com.bablsoft.accessflow.core.internal.persistence.repo.DatasourceRepository; +import com.bablsoft.accessflow.core.internal.persistence.repo.OrganizationRepository; +import com.bablsoft.accessflow.proxy.api.QueryExecutor; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.testcontainers.context.ImportTestcontainers; +import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.test.context.DynamicPropertyRegistry; +import org.springframework.test.context.DynamicPropertySource; + +import java.security.KeyPairGenerator; +import java.security.interfaces.RSAPrivateCrtKey; +import java.util.Base64; +import java.util.Set; +import java.util.UUID; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * End-to-end test of the opt-in SELECT result cache (AF-457) against real PostgreSQL and Redis + * containers. A cache hit is proven by mutating the table behind the proxy's back (plain + * JDBC) and observing the stale cached rows; a write through the proxy then invalidates the + * cached entry and the next read sees fresh data. + */ +@SpringBootTest +@ImportTestcontainers(TestcontainersConfig.class) +class SelectResultCacheIntegrationTest { + + @Autowired QueryExecutor executor; + @Autowired DatasourceRepository datasourceRepository; + @Autowired OrganizationRepository organizationRepository; + @Autowired CredentialEncryptionService encryptionService; + @Autowired JdbcTemplate jdbcTemplate; + + private DatasourceEntity datasource; + + @DynamicPropertySource + static void securityProperties(DynamicPropertyRegistry registry) throws Exception { + var kpg = KeyPairGenerator.getInstance("RSA"); + kpg.initialize(2048); + var kp = kpg.generateKeyPair(); + var privateKey = (RSAPrivateCrtKey) kp.getPrivate(); + var pem = "-----BEGIN PRIVATE KEY-----\n" + + Base64.getMimeEncoder(64, new byte[]{'\n'}).encodeToString(privateKey.getEncoded()) + + "\n-----END PRIVATE KEY-----"; + registry.add("accessflow.jwt.private-key", () -> pem); + registry.add("accessflow.encryption-key", () -> + "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"); + } + + @BeforeEach + void setUp() { + jdbcTemplate.execute("DROP TABLE IF EXISTS cache_items"); + jdbcTemplate.execute( + "CREATE TABLE cache_items (id int PRIMARY KEY, label varchar(64) NOT NULL)"); + jdbcTemplate.update("INSERT INTO cache_items VALUES (1, 'v1')"); + + datasourceRepository.deleteAll(); + var org = new OrganizationEntity(); + org.setId(UUID.randomUUID()); + org.setName("CacheOrg"); + org.setSlug("cache-" + UUID.randomUUID()); + organizationRepository.save(org); + + var pg = TestcontainersConfig.postgres; + var ds = new DatasourceEntity(); + ds.setId(UUID.randomUUID()); + ds.setOrganization(org); + ds.setName("Cached-" + UUID.randomUUID()); + ds.setDbType(DbType.POSTGRESQL); + ds.setHost(pg.getHost()); + ds.setPort(pg.getMappedPort(5432)); + ds.setDatabaseName(pg.getDatabaseName()); + ds.setUsername(pg.getUsername()); + ds.setPasswordEncrypted(encryptionService.encrypt(pg.getPassword())); + ds.setSslMode(SslMode.DISABLE); + ds.setConnectionPoolSize(2); + ds.setMaxRowsPerQuery(1000); + ds.setRequireReviewReads(false); + ds.setRequireReviewWrites(true); + ds.setAiAnalysisEnabled(false); + ds.setResultCacheEnabled(true); + ds.setResultCacheTtlSeconds(300); + ds.setActive(true); + datasource = datasourceRepository.save(ds); + } + + private QueryExecutionRequest select() { + return new QueryExecutionRequest(datasource.getId(), + "SELECT label FROM cache_items ORDER BY id", QueryType.SELECT, null, null, + java.util.List.of(), java.util.List.of(), java.util.List.of(), false, null, + java.util.List.of(), Set.of("cache_items")); + } + + @Test + void cachedSelectServesStaleRowsUntilWriteInvalidates() { + var first = (SelectExecutionResult) executor.execute(select()); + assertThat(first.rows().get(0)).containsExactly("v1"); + + // Mutate behind the proxy's back — the cached entry must still serve the old value. + jdbcTemplate.update("UPDATE cache_items SET label = 'v2' WHERE id = 1"); + var cached = (SelectExecutionResult) executor.execute(select()); + assertThat(cached.rows().get(0)).containsExactly("v1"); + + // A write through the proxy to the referenced table invalidates the cached entry. + executor.execute(new QueryExecutionRequest(datasource.getId(), + "UPDATE cache_items SET label = 'v3' WHERE id = 1", QueryType.UPDATE, null, null, + java.util.List.of(), java.util.List.of(), java.util.List.of(), false, null, + java.util.List.of(), Set.of("cache_items"))); + var fresh = (SelectExecutionResult) executor.execute(select()); + assertThat(fresh.rows().get(0)).containsExactly("v3"); + } + + @Test + void selectWithoutOptInIsNeverCached() { + datasource.setResultCacheEnabled(false); + datasourceRepository.save(datasource); + + var first = (SelectExecutionResult) executor.execute(select()); + assertThat(first.rows().get(0)).containsExactly("v1"); + + jdbcTemplate.update("UPDATE cache_items SET label = 'v2' WHERE id = 1"); + var second = (SelectExecutionResult) executor.execute(select()); + assertThat(second.rows().get(0)).containsExactly("v2"); + } +} diff --git a/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/SelectResultCacheTest.java b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/SelectResultCacheTest.java new file mode 100644 index 00000000..393f90bc --- /dev/null +++ b/backend/src/test/java/com/bablsoft/accessflow/proxy/internal/SelectResultCacheTest.java @@ -0,0 +1,232 @@ +package com.bablsoft.accessflow.proxy.internal; + +import com.bablsoft.accessflow.core.api.ColumnMaskDirective; +import com.bablsoft.accessflow.core.api.DatasourceConnectionDescriptor; +import com.bablsoft.accessflow.core.api.DbType; +import com.bablsoft.accessflow.core.api.MaskingStrategy; +import com.bablsoft.accessflow.core.api.ReadReplicaEndpoint; +import com.bablsoft.accessflow.core.api.ResultColumn; +import com.bablsoft.accessflow.core.api.SelectExecutionResult; +import com.bablsoft.accessflow.core.api.SslMode; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.mockito.ArgumentCaptor; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; +import org.mockito.junit.jupiter.MockitoSettings; +import org.mockito.quality.Strictness; +import org.springframework.data.redis.core.SetOperations; +import org.springframework.data.redis.core.StringRedisTemplate; +import org.springframework.data.redis.core.ValueOperations; + +import java.sql.Types; +import java.time.Duration; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.UUID; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.anyString; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +@ExtendWith(MockitoExtension.class) +@MockitoSettings(strictness = Strictness.LENIENT) +class SelectResultCacheTest { + + private final UUID datasourceId = UUID.randomUUID(); + + @Mock + private StringRedisTemplate redisTemplate; + @Mock + private ValueOperations valueOps; + @Mock + private SetOperations setOps; + + private SelectResultCache cache; + + @BeforeEach + void setUp() { + when(redisTemplate.opsForValue()).thenReturn(valueOps); + when(redisTemplate.opsForSet()).thenReturn(setOps); + cache = new SelectResultCache(redisTemplate, + new ProxyCacheProperties(true, Duration.ofSeconds(60), 1_000_000L)); + } + + private static SelectExecutionResult sampleResult() { + return new SelectExecutionResult( + List.of(new ResultColumn("id", Types.BIGINT, "int8"), + new ResultColumn("email", Types.VARCHAR, "text", true)), + List.of(List.of(1L, "a@example.com"), List.of(2L, "b@example.com")), + 2, false, Duration.ofMillis(12), + Set.of(UUID.randomUUID()), Set.of(UUID.randomUUID())); + } + + private DatasourceConnectionDescriptor descriptor(boolean cacheEnabled, Integer ttlSeconds) { + return new DatasourceConnectionDescriptor(datasourceId, UUID.randomUUID(), + DbType.POSTGRESQL, "h", 5432, "db", "u", "ENC", SslMode.DISABLE, 10, 1000, + false, null, false, null, null, null, List.of(), true, null, + null, cacheEnabled, ttlSeconds); + } + + @Test + void enabledForRequiresGlobalSwitchAndDatasourceOptIn() { + assertThat(cache.enabledFor(descriptor(true, null))).isTrue(); + assertThat(cache.enabledFor(descriptor(false, null))).isFalse(); + + var disabled = new SelectResultCache(redisTemplate, + new ProxyCacheProperties(false, null, null)); + assertThat(disabled.enabledFor(descriptor(true, null))).isFalse(); + } + + @Test + void ttlFallsBackToDefaultWhenDatasourceHasNone() { + assertThat(cache.ttlFor(descriptor(true, null))).isEqualTo(Duration.ofSeconds(60)); + assertThat(cache.ttlFor(descriptor(true, 120))).isEqualTo(Duration.ofSeconds(120)); + } + + @Test + void putStoresJsonAndIndexesEveryReferencedTablePlusAllSet() { + cache.put(datasourceId, "abc", Set.of("public.users"), Duration.ofSeconds(30), + sampleResult()); + + var keyCaptor = ArgumentCaptor.forClass(String.class); + verify(valueOps).set(keyCaptor.capture(), anyString(), eq(Duration.ofSeconds(30))); + var valueKey = keyCaptor.getValue(); + assertThat(valueKey) + .isEqualTo(SelectResultCache.VALUE_PREFIX + datasourceId + ":abc"); + verify(setOps).add( + SelectResultCache.INDEX_PREFIX + datasourceId + ":public.users", valueKey); + verify(setOps).add( + SelectResultCache.INDEX_PREFIX + datasourceId + ":__all__", valueKey); + verify(redisTemplate).expire( + eq(SelectResultCache.INDEX_PREFIX + datasourceId + ":public.users"), + eq(Duration.ofSeconds(90))); + } + + @Test + void getRoundTripsStoredResultWithFreshDuration() { + var result = sampleResult(); + cache.put(datasourceId, "abc", Set.of("t"), Duration.ofSeconds(30), result); + var jsonCaptor = ArgumentCaptor.forClass(String.class); + verify(valueOps).set(anyString(), jsonCaptor.capture(), any(Duration.class)); + when(valueOps.get(SelectResultCache.VALUE_PREFIX + datasourceId + ":abc")) + .thenReturn(jsonCaptor.getValue()); + + var hit = cache.get(datasourceId, "abc", Duration.ofMillis(1)); + + assertThat(hit).isPresent(); + var cached = hit.get(); + assertThat(cached.columns()).isEqualTo(result.columns()); + assertThat(cached.rowCount()).isEqualTo(2); + assertThat(cached.truncated()).isFalse(); + assertThat(cached.duration()).isEqualTo(Duration.ofMillis(1)); + assertThat(cached.appliedMaskingPolicyIds()) + .isEqualTo(result.appliedMaskingPolicyIds()); + assertThat(cached.appliedRowSecurityPolicyIds()) + .isEqualTo(result.appliedRowSecurityPolicyIds()); + // JSON-natural types: numbers stay numbers, strings stay strings. + assertThat(cached.rows().get(0).get(1)).isEqualTo("a@example.com"); + assertThat(((Number) cached.rows().get(0).get(0)).longValue()).isEqualTo(1L); + } + + @Test + void getReturnsEmptyOnMissAndOnRedisFailure() { + when(valueOps.get(anyString())).thenReturn(null); + assertThat(cache.get(datasourceId, "abc", Duration.ZERO)).isEmpty(); + + when(valueOps.get(anyString())).thenThrow(new RuntimeException("redis down")); + assertThat(cache.get(datasourceId, "abc", Duration.ZERO)).isEmpty(); + } + + @Test + void putSkipsEntriesLargerThanMaxEntryBytes() { + var tiny = new SelectResultCache(redisTemplate, + new ProxyCacheProperties(true, Duration.ofSeconds(60), 10L)); + + tiny.put(datasourceId, "abc", Set.of("t"), Duration.ofSeconds(30), sampleResult()); + + verify(valueOps, never()).set(anyString(), anyString(), any(Duration.class)); + } + + @Test + void putSwallowsRedisFailures() { + org.mockito.Mockito.doThrow(new RuntimeException("redis down")) + .when(valueOps).set(anyString(), anyString(), any(Duration.class)); + + cache.put(datasourceId, "abc", Set.of("t"), Duration.ofSeconds(30), sampleResult()); + // no exception propagated + } + + @Test + void invalidateTablesDropsIndexedEntriesPerTable() { + var indexKey = SelectResultCache.INDEX_PREFIX + datasourceId + ":public.users"; + when(setOps.members(indexKey)).thenReturn(Set.of("k1", "k2")); + + cache.invalidateTables(datasourceId, Set.of("public.users")); + + verify(redisTemplate).delete(Set.of("k1", "k2")); + verify(redisTemplate).delete(indexKey); + } + + @Test + void invalidateTablesWithEmptySetPurgesWholeDatasource() { + var allKey = SelectResultCache.INDEX_PREFIX + datasourceId + ":__all__"; + when(setOps.members(allKey)).thenReturn(Set.of("k1")); + + cache.invalidateTables(datasourceId, Set.of()); + + verify(redisTemplate).delete(Set.of("k1")); + verify(redisTemplate).delete(allKey); + } + + @Test + void invalidateSwallowsRedisFailures() { + when(setOps.members(anyString())).thenThrow(new RuntimeException("redis down")); + + cache.invalidateTables(datasourceId, Set.of("t")); + cache.invalidateAll(datasourceId); + // no exception propagated + } + + @Test + void cacheKeyIsStableAndOrderInsensitiveForDirectives() { + var maskA = new ColumnMaskDirective("users.email", MaskingStrategy.FULL, Map.of(), + UUID.fromString("00000000-0000-0000-0000-000000000001")); + var maskB = new ColumnMaskDirective("users.ssn", MaskingStrategy.FULL, Map.of(), + UUID.fromString("00000000-0000-0000-0000-000000000002")); + + var key1 = SelectResultCache.cacheKey("SELECT 1", List.of(42), List.of("a", "b"), + List.of(maskA, maskB), 100); + var key2 = SelectResultCache.cacheKey("SELECT 1", List.of(42), List.of("b", "a"), + List.of(maskB, maskA), 100); + + assertThat(key1).isEqualTo(key2).hasSize(64); + } + + @Test + void cacheKeyDiscriminatesSecurityRelevantInputs() { + var base = SelectResultCache.cacheKey("SELECT 1", List.of(), List.of(), List.of(), 100); + + assertThat(SelectResultCache.cacheKey("SELECT 2", List.of(), List.of(), List.of(), 100)) + .isNotEqualTo(base); + assertThat(SelectResultCache.cacheKey("SELECT 1", List.of(7), List.of(), List.of(), 100)) + .isNotEqualTo(base); + assertThat(SelectResultCache.cacheKey("SELECT 1", List.of(), List.of("users.ssn"), + List.of(), 100)).isNotEqualTo(base); + assertThat(SelectResultCache.cacheKey("SELECT 1", List.of(), List.of(), + List.of(new ColumnMaskDirective("users.email", MaskingStrategy.FULL, Map.of(), + null)), 100)).isNotEqualTo(base); + assertThat(SelectResultCache.cacheKey("SELECT 1", List.of(), List.of(), List.of(), 50)) + .isNotEqualTo(base); + // Bind type participates in the key ("1" as String vs 1 as Long). + assertThat(SelectResultCache.cacheKey("SELECT 1", List.of("1"), List.of(), List.of(), 100)) + .isNotEqualTo( + SelectResultCache.cacheKey("SELECT 1", List.of(1L), List.of(), List.of(), 100)); + } +} diff --git a/backend/src/test/java/com/bablsoft/accessflow/security/internal/web/DatasourceControllerIntegrationTest.java b/backend/src/test/java/com/bablsoft/accessflow/security/internal/web/DatasourceControllerIntegrationTest.java index 37b7c894..767cdbd9 100644 --- a/backend/src/test/java/com/bablsoft/accessflow/security/internal/web/DatasourceControllerIntegrationTest.java +++ b/backend/src/test/java/com/bablsoft/accessflow/security/internal/web/DatasourceControllerIntegrationTest.java @@ -774,8 +774,27 @@ void testReplicaWithMissingFieldsFailsValidation() { } @Test - void testReplicaFallsBackToPersistedPasswordWhenOmitted() { + void testReplicaFallsBackToPersistedPasswordViaReplicaId() { var ds = saveContainerDatasourceWithReplica(primaryOrg, "DS-Replica-Pw"); + var replicaId = ds.getReadReplicas().get(0).getId(); + var body = String.format( + "{\"jdbc_url\":\"%s\",\"username\":\"%s\",\"replica_id\":\"%s\"}", + TestcontainersConfig.postgres.getJdbcUrl(), + TestcontainersConfig.postgres.getUsername(), + replicaId); + + var result = mvc.post().uri("/api/v1/datasources/" + ds.getId() + "/test-replica") + .header(HttpHeaders.AUTHORIZATION, "Bearer " + adminToken) + .contentType(MediaType.APPLICATION_JSON) + .content(body) + .exchange(); + + assertThat(result).hasStatus(200); + } + + @Test + void testReplicaWithoutPasswordAndWithoutReplicaIdReturns422() { + var ds = saveContainerDatasourceWithReplica(primaryOrg, "DS-Replica-NoId"); var body = String.format( "{\"jdbc_url\":\"%s\",\"username\":\"%s\"}", TestcontainersConfig.postgres.getJdbcUrl(), @@ -787,7 +806,7 @@ void testReplicaFallsBackToPersistedPasswordWhenOmitted() { .content(body) .exchange(); - assertThat(result).hasStatus(200); + assertThat(result).hasStatus(422); } @Test @@ -882,9 +901,15 @@ private DatasourceEntity saveContainerDatasource(OrganizationEntity org, String private DatasourceEntity saveContainerDatasourceWithReplica(OrganizationEntity org, String name) { var ds = saveContainerDatasource(org, name); var pg = TestcontainersConfig.postgres; - ds.setReadReplicaJdbcUrl(pg.getJdbcUrl()); - ds.setReadReplicaUsername(pg.getUsername()); - ds.setReadReplicaPasswordEncrypted(encryptionService.encrypt(pg.getPassword())); + var replica = new com.bablsoft.accessflow.core.internal.persistence.entity + .DatasourceReadReplicaEntity(); + replica.setId(UUID.randomUUID()); + replica.setDatasource(ds); + replica.setJdbcUrl(pg.getJdbcUrl()); + replica.setUsername(pg.getUsername()); + replica.setPasswordEncrypted(encryptionService.encrypt(pg.getPassword())); + replica.setPosition(0); + ds.getReadReplicas().add(replica); return datasourceRepository.save(ds); } diff --git a/docs/03-data-model.md b/docs/03-data-model.md index 970b2ef2..7dfa6f49 100644 --- a/docs/03-data-model.md +++ b/docs/03-data-model.md @@ -147,16 +147,33 @@ A customer database that AccessFlow proxies. Credentials are stored encrypted. | `custom_driver_id` | FK → `custom_jdbc_driver(id)` NULL, ON DELETE RESTRICT — when set, the proxy uses the uploaded driver's per-driver classloader instead of a catalog connector. A `CUSTOM` datasource sets exactly one of `custom_driver_id` or `connector_id`. | | `connector_id` | VARCHAR(64) NULL — references a catalog connector by its manifest id (e.g. `clickhouse`; see [14-connectors.md](./14-connectors.md)). Set for a `CUSTOM` datasource backed by an installed connector: the proxy loads the connector's cached driver into a per-connector classloader and builds the JDBC URL from the connector's template + host/port/database. Null for the five dialects and for uploaded-driver datasources. Only allowed when `db_type=CUSTOM`. | | `jdbc_url_override` | TEXT NULL — free-form JDBC connection string; used by an uploaded-driver `CUSTOM` datasource (required there, rejected for any bundled `db_type` and for connector-backed datasources, which build their URL from the connector template). | -| `read_replica_jdbc_url` | TEXT NULL — when set, SELECT queries are routed to a sibling HikariCP pool built from this URL. INSERT/UPDATE/DELETE/DDL always hit the primary. Reuses the primary's driver class. | -| `read_replica_username` | VARCHAR(255) NULL — username for the replica pool. When `NULL` the primary `username` is reused. | -| `read_replica_password_encrypted` | TEXT NULL — AES-256-GCM encrypted; same key (`ENCRYPTION_KEY`) as `password_encrypted`. When `NULL`, the primary `password_encrypted` is reused. `@JsonIgnore` on the entity. May also hold an external secret reference (AF-448), same semantics as `password_encrypted`. | | `local_datacenter` | VARCHAR(255) NULL (AF-421, migration `V77`) — the Cassandra / ScyllaDB driver's load-balancing datacenter (`withLocalDatacenter(...)`). NULL for every other dialect; the service layer **requires** it when `db_type` is `CASSANDRA` or `SCYLLADB`. | +| `result_cache_enabled` | BOOLEAN NOT NULL DEFAULT false (AF-457, migration `V115`) — opt-in SELECT result caching for this datasource. Entries live in Redis and are invalidated on any write to a referenced table; see [05-backend.md → SELECT result caching](./05-backend.md#select-result-caching). | +| `result_cache_ttl_seconds` | INTEGER NULL (AF-457, migration `V115`) — per-datasource cache TTL. NULL falls back to the deployment default (`ACCESSFLOW_PROXY_CACHE_DEFAULT_TTL`). | | `api_key_encrypted` | TEXT NULL (AF-420, migration `V80`) — AES-256-GCM-encrypted API key for the search engines (`ELASTICSEARCH` / `OPENSEARCH`), sent as `Authorization: ApiKey`. `@JsonIgnore`, never returned in an API response. NULL for basic-auth search datasources and every other dialect; the service layer requires **either** `username`+`password` **or** `api_key` for a search datasource. May also hold an external secret reference (AF-448), same semantics as `password_encrypted`. | | `is_active` | BOOLEAN DEFAULT true | | `created_at` | TIMESTAMPTZ | > **Constraint:** `UNIQUE (organization_id, name)` — added in `V10__datasource_unique_name_per_org.sql`. Attempting to create or rename a datasource into an existing name in the same organization returns HTTP 409 with `error: DATASOURCE_NAME_ALREADY_EXISTS`. +> **Migration note (AF-457, `V115`):** the pre-2.2 single-replica columns `read_replica_jdbc_url` / `read_replica_username` / `read_replica_password_encrypted` were **dropped** in favour of the `datasource_read_replicas` child table below. `V115` copies any existing single-replica config into the child table before dropping the columns, so no replica configuration is lost on upgrade. + +--- + +## datasource_read_replicas + +Read-replica endpoints of a datasource (AF-457, migration `V115`). A datasource may have any number of endpoints (the API caps a payload at 5); SELECT queries load-balance round-robin across the healthy ones. Replaces the pre-2.2 single `read_replica_*` columns on `datasources`. + +| Column | Type / Notes | +|--------|-------------| +| `id` | UUID PK | +| `datasource_id` | FK → `datasources` NOT NULL, ON DELETE CASCADE. Indexed (`idx_datasource_read_replicas_datasource`). | +| `jdbc_url` | TEXT NOT NULL — the endpoint's JDBC URL. Must run the same engine as the primary (the replica pool reuses the primary's driver class). | +| `username` | VARCHAR(255) NULL — username for this endpoint's pool. When `NULL` the primary `username` is reused. | +| `password_encrypted` | TEXT NULL — AES-256-GCM encrypted (same `ENCRYPTION_KEY` as the primary) or an external secret reference (AF-448). When `NULL`, the primary `password_encrypted` is reused. `@JsonIgnore` on the entity; never returned in an API response. | +| `position` | INT NOT NULL DEFAULT 0 — display / round-robin seed order. | +| `created_at` | TIMESTAMPTZ | + --- ## custom_jdbc_driver diff --git a/docs/04-api-spec.md b/docs/04-api-spec.md index c7af36cc..9f0ce4f0 100644 --- a/docs/04-api-spec.md +++ b/docs/04-api-spec.md @@ -291,7 +291,7 @@ fields. ``` When a provider is enabled, the datasource credential fields (`password`, -`read_replica_password`, `api_key`) accept an external secret **reference** instead of a raw +each `read_replicas[].password`, `api_key`) accept an external secret **reference** instead of a raw value — `vault:/#`, `aws:[#jsonField]`, or `azure:`. References are stored verbatim (not encrypted) and resolved through the store at credential-use time. Error responses on the create/update endpoints: @@ -364,9 +364,9 @@ Results are scoped to the caller's organization. ADMINs see all datasources in t "ai_analysis_enabled": true, "custom_driver_id": null, "jdbc_url_override": null, - "read_replica_jdbc_url": null, - "read_replica_username": null, - "read_replica_password": null + "read_replicas": [], + "result_cache_enabled": false, + "result_cache_ttl_seconds": null } ``` @@ -374,7 +374,11 @@ Results are scoped to the caller's organization. ADMINs see all datasources in t `api_key` (≤ 4096, AES-256-GCM encrypted, never returned) is the search-engine alternative to basic auth: for `db_type` `ELASTICSEARCH` / `OPENSEARCH` the request must carry **either** `username` + `password` **or** `api_key` (sent to the cluster as `Authorization: ApiKey`); supplying it is rejected for every other dialect. For these search datasources `database_name` is optional (it only scopes introspection — the index is named in the query). On update, sending a blank `api_key` clears it (reverting to basic auth). Null/unused for every non-search dialect. -When `read_replica_jdbc_url` is set, SELECT queries are routed to the replica pool while INSERT/UPDATE/DELETE/DDL always hit the primary. The replica reuses the primary's JDBC driver class — it must be the same engine. `read_replica_username` and `read_replica_password` are optional; when omitted, the primary's username/password is reused. `read_replica_password` is AES-256-GCM encrypted with the same `ENCRYPTION_KEY` as the primary password. +`read_replicas` (AF-457, ≤ 5 items) lists the datasource's read-replica endpoints as `{ "id": "uuid|null", "jdbc_url": "...", "username": "...", "password": "..." }` objects. SELECT queries load-balance round-robin across the healthy endpoints while INSERT/UPDATE/DELETE/DDL always hit the primary; each replica reuses the primary's JDBC driver class — it must be the same engine. `username` / `password` are optional per endpoint; when omitted, the primary's credentials are reused. Passwords are AES-256-GCM encrypted with the same `ENCRYPTION_KEY` as the primary password and never returned (responses carry only `id`, `jdbc_url`, `username` per endpoint). + +`result_cache_enabled` / `result_cache_ttl_seconds` (AF-457) opt this datasource into SELECT result caching; the TTL must be 1–86400 seconds and `null` falls back to the deployment default. See [05-backend.md → SELECT result caching](./05-backend.md#select-result-caching). + +> **Breaking change (2.2.0, AF-457):** the flat `read_replica_jdbc_url` / `read_replica_username` / `read_replica_password` fields were removed from datasource create/update payloads and responses in favour of the `read_replicas` array. Migration `V115` auto-converts any existing single-replica configuration. For datasources backed by an uploaded driver, set `custom_driver_id` to the `custom_jdbc_driver.id`. For fully dynamic datasources, set `db_type=CUSTOM`, @@ -422,14 +426,17 @@ All fields optional. Omitted fields are left unchanged. Providing `password` tri "password": "new-service-account-password", "local_datacenter": null, "connection_pool_size": 25, - "read_replica_jdbc_url": "jdbc:postgresql://replica.company.com:5432/app_prod", - "read_replica_username": "ro_svc", - "read_replica_password": "replica-secret", + "read_replicas": [ + { "id": "existing-endpoint-uuid", "jdbc_url": "jdbc:postgresql://replica-a.company.com:5432/app_prod", "username": "ro_svc" }, + { "jdbc_url": "jdbc:postgresql://replica-b.company.com:5432/app_prod", "username": "ro_svc", "password": "replica-secret" } + ], + "result_cache_enabled": true, + "result_cache_ttl_seconds": 120, "active": true } ``` -Setting `read_replica_jdbc_url` to an empty string clears all three replica fields. Providing `read_replica_password` triggers re-encryption with a fresh IV. +`read_replicas` is a **full-list replacement merged by endpoint id**: omitting the field keeps the current endpoints, an empty array deletes them all, and a non-empty array replaces the list — items whose `id` matches a stored endpoint update it (a `null`/omitted `password` keeps the stored secret, an empty string clears it back to the primary-credential fallback, a non-blank value re-encrypts with a fresh IV), items without an `id` create new endpoints, and stored endpoints absent from the array are removed. **Response 200:** Updated datasource object. **Response 404:** Datasource does not exist in the caller's organization. `error: DATASOURCE_NOT_FOUND`. @@ -466,11 +473,12 @@ Opens a transient JDBC connection to a candidate read-replica using the values s { "jdbc_url": "jdbc:postgresql://replica.company.com:5432/app_prod", "username": "ro_svc", - "password": "replica-secret" + "password": "replica-secret", + "replica_id": "existing-endpoint-uuid" } ``` -`password` is optional — when omitted, the test reuses the currently-persisted replica password (lets an admin retest after changing only the URL or username without re-typing the secret). If neither the body nor the stored datasource has a password, the call returns 400 `DATASOURCE_CONNECTION_TEST_FAILED` with a "no read replica configured" detail. +`password` is optional — when omitted, the test reuses the persisted password of the endpoint named by the optional `replica_id` (lets an admin retest a saved endpoint after changing only the URL or username without re-typing the secret). If the body has no password and `replica_id` names no endpoint with a stored password, the call returns 422 `DATASOURCE_CONNECTION_TEST_FAILED` with a "no read replica configured" detail. **Response 200:** ```json @@ -4026,7 +4034,11 @@ The snapshot is cached ~30 s per `(organization_id, datasource_id)` (Spring cach "queries_last_24h": 142, "execution_ms_p50": 12.5, "execution_ms_p95": 88.0, - "errors_last_24h": 3 + "errors_last_24h": 3, + "replicas": [ + { "endpoint_id": "9a2b…", "label": "replica-a.company.com:5432", "healthy": true, "pool_active": 1, "pool_total": 4 }, + { "endpoint_id": "b41c…", "label": "replica-b.company.com:5432", "healthy": false, "pool_active": null, "pool_total": null } + ] } ], "page": 0, @@ -4036,6 +4048,8 @@ The snapshot is cached ~30 s per `(organization_id, datasource_id)` (Spring cach } ``` +`replicas` (AF-457) carries per-endpoint read-replica health on the serving node: `label` is the redacted host:port of the endpoint's JDBC URL, `healthy` reflects the node's circuit-breaker state (an unhealthy endpoint is being skipped by the read load-balancer until its cooldown elapses), and the pool gauges are `null` when the endpoint's pool has not been created on that node yet. Empty for datasources without replicas. + **Response 401:** Not authenticated. **Response 403:** Caller is not an `ADMIN`. diff --git a/docs/05-backend.md b/docs/05-backend.md index c8a98458..b3d2062c 100644 --- a/docs/05-backend.md +++ b/docs/05-backend.md @@ -218,8 +218,8 @@ The proxy engine (`accessflow-proxy` module) is the heart of AccessFlow. It is t Implemented in `proxy/internal/`: -- `DatasourceConnectionPoolManager` (public API) — `DataSource resolve(UUID)`, `Optional resolveReplica(UUID)`, `void evict(UUID)`, and `Optional poolStats(UUID)`. Returns Hikari pools typed as `javax.sql.DataSource` so callers stay framework-agnostic and use the standard JDBC `try-with-resources` idiom. `resolveReplica` returns empty when the datasource has no replica configured; `evict` closes both the primary and replica pools. `poolStats` reads the live `HikariPoolMXBean` gauges (active / idle / waiting / total / max) for the **already-cached** primary pool and returns empty when none is cached — it never creates a pool, so reading health metrics can't trigger a connection attempt against an unreachable customer DB. -- `DefaultDatasourceConnectionPoolManager` — `ConcurrentHashMap` cache, atomic lazy creation via `compute`, `@PreDestroy` shutdown closes all pools. +- `DatasourceConnectionPoolManager` (public API) — `DataSource resolve(UUID)`, `List replicaEndpoints(UUID)`, `DataSource resolveReplica(UUID, UUID endpointId)`, `void evict(UUID)`, `Optional poolStats(UUID)`, and `Optional replicaPoolStats(UUID, UUID)` (AF-457). Returns Hikari pools typed as `javax.sql.DataSource` so callers stay framework-agnostic and use the standard JDBC `try-with-resources` idiom. `replicaEndpoints` lists the datasource's endpoints (id + redacted host:port label) without creating pools; `resolveReplica` lazily builds one pool per endpoint; `evict` closes the primary and every replica pool. The `*poolStats` methods read the live `HikariPoolMXBean` gauges (active / idle / waiting / total / max) for **already-cached** pools and return empty when none is cached — they never create a pool, so reading health metrics can't trigger a connection attempt against an unreachable customer DB. +- `DefaultDatasourceConnectionPoolManager` — `ConcurrentHashMap` cache (primary map + per-datasource endpoint-keyed replica map), atomic lazy creation via `compute`, `@PreDestroy` shutdown closes all pools. - `DatasourcePoolFactory` — owns the Hikari wiring; resolves the stored credential to plaintext only here (via `SecretResolutionService`) and drops the local reference before returning. - `DatasourcePoolEvictionListener` — `@ApplicationModuleListener` for `DatasourceConfigChangedEvent` and `DatasourceDeactivatedEvent` (both in `core/events/`); fires in a new transaction after the publisher's transaction commits. Annotation comes from `spring-modulith-events-api`. @@ -261,20 +261,41 @@ semantics on the execution path). Eviction events (in `core/events/`, published by `DatasourceAdminServiceImpl`): -- `DatasourceConfigChangedEvent(UUID datasourceId)` — fired from `update(...)` when any of `host`, `port`, `databaseName`, `username`, `passwordEncrypted`, `sslMode`, `connectionPoolSize`, `readReplicaJdbcUrl`, `readReplicaUsername`, or `readReplicaPasswordEncrypted` changed. Eviction closes both pools. +- `DatasourceConfigChangedEvent(UUID datasourceId)` — fired from `update(...)` when any of `host`, `port`, `databaseName`, `username`, `passwordEncrypted`, `sslMode`, `connectionPoolSize`, or the read-replica endpoint list changed. Eviction closes the primary and every replica pool (and purges the SELECT result cache). - `DatasourceDeactivatedEvent(UUID datasourceId)` — fired from `update(...)` when `active` flips `true → false`, and from `deactivate(...)` (idempotent — only when the entity was active before the call). +- `DatasourceCacheConfigChangedEvent(UUID datasourceId)` (AF-457) — fired from `update(...)` when only the result-cache settings (`result_cache_enabled` / `result_cache_ttl_seconds`) changed. Purges the datasource's cached SELECT results **without** evicting connection pools. -The proxy module reads the datasource state via `DatasourceLookupService` (`core/api/`) which returns a `DatasourceConnectionDescriptor` record — a Modulith-clean alternative to letting `proxy/internal/` reach into `core/internal/` JPA entities. The descriptor exposes `maxRowsPerQuery` so the executor can enforce per-datasource row caps without a second round trip. It also carries the optional `readReplicaJdbcUrl`/`readReplicaUsername`/`readReplicaPasswordEncrypted` fields plus a convenience `hasReadReplica()` method. +The proxy module reads the datasource state via `DatasourceLookupService` (`core/api/`) which returns a `DatasourceConnectionDescriptor` record — a Modulith-clean alternative to letting `proxy/internal/` reach into `core/internal/` JPA entities. The descriptor exposes `maxRowsPerQuery` so the executor can enforce per-datasource row caps without a second round trip. It also carries the ordered `readReplicas` endpoint list (`ReadReplicaEndpoint(id, jdbcUrl, username, passwordEncrypted)`), a convenience `hasReadReplica()` method, and the `resultCacheEnabled` / `resultCacheTtlSeconds` cache settings. -### Read-replica routing +### Read-replica routing & load balancing (AF-457) -When a datasource has a `read_replica_jdbc_url` set, `RoutingDataSourceResolver` (`proxy/internal/`) routes any query classified by `SqlParserService` as `QueryType.SELECT` to the sibling replica pool. INSERT/UPDATE/DELETE/DDL and transactional `BEGIN…COMMIT` batches always hit the primary, regardless of the replica configuration. Dry-runs (AF-445) route through the same resolver by their underlying `QueryType`, so a SELECT dry-run prefers the replica while a write dry-run plans on the primary. +When a datasource has endpoints in `datasource_read_replicas`, `RoutingDataSourceResolver` (`proxy/internal/`) load-balances any query classified by `SqlParserService` as `QueryType.SELECT` **round-robin across the healthy replica endpoints** (one HikariCP pool per endpoint). INSERT/UPDATE/DELETE/DDL and transactional `BEGIN…COMMIT` batches always hit the primary, regardless of the replica configuration. Dry-runs (AF-445) route through the same resolver by their underlying `QueryType`, so a SELECT dry-run prefers a replica while a write dry-run plans on the primary. -- Replica credentials are encrypted with the same `ENCRYPTION_KEY` as the primary, decrypted only inside `DatasourcePoolFactory.createReplicaPool(...)`, and surface a pool name suffixed `-replica`. When `read_replica_username` or `read_replica_password_encrypted` is `NULL`, the primary's credentials are reused — useful when the replica accepts the same service account. +- Replica credentials are encrypted with the same `ENCRYPTION_KEY` as the primary, decrypted only inside `DatasourcePoolFactory.createReplicaPool(...)`, and surface pool names suffixed `-replica-{n}`. When an endpoint's `username` or `password_encrypted` is `NULL`, the primary's credentials are reused — useful when the replicas accept the same service account. - The driver class is shared with the primary: replicas must use the same engine (you cannot point a PostgreSQL primary at a MySQL replica). -- On a connection failure against the replica (or on first-time pool init failure), the resolver records a `DATASOURCE_REPLICA_FALLBACK` audit row (action in `audit/api/AuditAction`; metadata includes the `error` message and `query_type=SELECT`), logs a `WARN`, and falls back to the primary so the query still runs. The fallback audit is recorded with `actorId=null` (system-initiated). Audit failures are swallowed. -- One audit row is written per failed SELECT. There is no rate-limiting — sustained replica downtime under load will produce one row per SELECT and the action filter on `/admin/audit-log` is the diagnostic. -- The Hikari pool tuning under `accessflow.proxy.*` (`connection-timeout`, `idle-timeout`, `max-lifetime`, `leak-detection-threshold`) applies to the replica pool too — no separate env vars. +- **Health / circuit breaker.** `ReplicaHealthRegistry` (`proxy/internal/`) keeps per-node, per-endpoint breaker state: a connection failure marks the endpoint DOWN for `accessflow.proxy.replica.cooldown` (default 30s), during which the load-balancer skips it; once the cooldown elapses a single half-open trial is admitted — success restores the endpoint to rotation, failure re-arms the cooldown. A failed endpoint never fails the read: the resolver moves on to the next candidate and only when **every** endpoint is down or failed does the SELECT fall back to the primary. +- **Async prober.** `ReplicaHealthProber` (`proxy/internal/`) runs `Connection.isValid(...)` against every replica pool already cached on the node every `accessflow.proxy.replica.probe-interval` (default 30s, per-endpoint timeout `probe-timeout`), feeding the registry — so recovered replicas re-enter rotation promptly and silently-degraded ones are removed before a user query fails on them. It is deliberately **not** a `@Scheduled`/`@SchedulerLock` job (the one sanctioned exception to the scheduled-job rule): ShedLock makes a job a cluster singleton, but Hikari pools and breaker state are per JVM — every node must probe its own pools. It runs on the Boot-autoconfigured virtual-thread `TaskScheduler` instead. +- On full replica-set exhaustion (every endpoint down or failed, with at least one live failure this attempt), the resolver records **one** `DATASOURCE_REPLICA_FALLBACK` audit row (action in `audit/api/AuditAction`; metadata includes the `error` message, `query_type=SELECT`, `replica_count`, and the redacted `tried_endpoints` labels), logs a `WARN`, and falls back to the primary so the query still runs. The fallback audit is recorded with `actorId=null` (system-initiated). Audit failures are swallowed. Note the changed cadence vs pre-2.2: SELECTs served by the primary while every circuit is already open do **not** re-audit — with the default cooldown, sustained downtime produces roughly one row per cooldown period instead of one per SELECT. +- Per-endpoint health (redacted host:port label, breaker state, pool gauges) is surfaced on the admin datasource-health snapshot (`replicas` array) and dashboard. +- The Hikari pool tuning under `accessflow.proxy.*` (`connection-timeout`, `idle-timeout`, `max-lifetime`, `leak-detection-threshold`) applies to every replica pool too — no separate env vars. + +### SELECT result caching (AF-457) + +`SelectResultCache` (`proxy/internal/`) is an opt-in, Redis-backed cache for relational SELECT results — repeated identical reads skip the customer database entirely. It is gated twice: the deployment-wide kill-switch `accessflow.proxy.cache.enabled` (default `true`) AND the per-datasource `result_cache_enabled` opt-in (default `false`). Engine-managed datasources (MongoDB & co.) are not cached. + +- **Security-safe keying.** Row-level security is spliced into the SQL *before* execution and masking is applied *post-fetch*, so the cache stores the **final post-mask** result keyed by a SHA-256 over: the RLS-rewritten SQL, its bind values, the sorted restricted-column list, the sorted mask directives, and the effective row cap. Two principals with different security scopes can never share an entry — different RLS means different SQL/binds, different masks mean a different key. Value keys are `accessflow:proxy:resultcache:{datasourceId}:{sha256}`. +- **Write invalidation.** Every cached key is indexed in one Redis SET per referenced table (`accessflow:proxy:resultcache:idx:{datasourceId}:{table}`, tables normalized exactly as `SqlParserServiceImpl` produces them) plus a per-datasource `__all__` set — deterministic, cluster-safe invalidation with no `SCAN`. Any successful write through the proxy (single DML/DDL or a committed `BEGIN…COMMIT` batch) drops every entry indexed under its referenced tables; a write whose tables are unknown (empty `referencedTables`) fails safe by purging the whole datasource cache. GDPR erasure and retention-policy deletes (the `lifecycle` module) pass their target table through the same path, so erased rows never survive in cached reads. `ResultCacheEvictionListener` additionally purges the datasource on `DatasourceConfigChangedEvent`, `DatasourceDeactivatedEvent`, and `DatasourceCacheConfigChangedEvent`. +- **Guards.** SELECTs whose referenced tables are unknown (e.g. `sampleTable` previews) are never cached — no invalidation coverage, no entry. Entries whose serialized form exceeds `accessflow.proxy.cache.max-entry-bytes` (default 1 MB) are skipped. Every Redis failure is swallowed with a WARN — cache degradation never fails a query. Dry-runs never touch the cache. +- **TTL.** Per-datasource `result_cache_ttl_seconds` (1–86400), falling back to `accessflow.proxy.cache.default-ttl` (default `PT60S`). Index sets outlive their members by 60s so invalidation never misses a live entry. +- **Semantics.** Cached rows round-trip through JSON, so a hit carries JSON-natural value types — identical to what API clients receive anyway, since results are only ever serialized to JSON downstream. `duration` reflects the (near-zero) cache-hit time, and the `accessflow.query.execute` observation is tagged `cache=hit|miss|off`. + +### Batch writes (AF-457) + +Inside a `BEGIN…COMMIT` envelope, `BatchInsertPlanner` (`proxy/internal/`) groups **consecutive homogeneous single-row INSERTs** — same table, same column list, same arity, literal-only VALUES, and a no-op row-security rewrite — into one `PreparedStatement` executed with `addBatch()` per row and `executeLargeBatch()`, flushed every `accessflow.proxy.execution.insert-batch-chunk-size` rows (default 1000). This collapses N network round trips into ~N/chunk for bulk-load envelopes while staying 100% `PreparedStatement` (literals become bound parameters). + +- Anything else in the envelope — expressions or subselects in VALUES, multi-row `VALUES (...),(...)` (already one statement/one round trip), INSERT…SELECT, UPDATE/DELETE, or a statement the RLS/soft-delete rewriter touched — stays on the existing per-statement path, in order, inside the same transaction (`autoCommit=false`, commit on success / rollback on `SQLException`). +- PostgreSQL string literals are bound with an unspecified JDBC type (`Types.OTHER`) so the server infers the column type exactly as it would for the inline literal — a plain `varchar` bind would be rejected against `uuid`/`jsonb`/enum columns (42804). Other dialects bind plainly. +- `SUCCESS_NO_INFO` batch counts are tallied as one affected row each. ### Datasource health dashboard (AF-365) diff --git a/docs/07-security.md b/docs/07-security.md index 328fc203..4192377b 100644 --- a/docs/07-security.md +++ b/docs/07-security.md @@ -605,7 +605,7 @@ transformed values; the raw data is never sent over the wire). High-security deployments can keep datasource credentials in an external secret manager instead of the local AES layer. When a provider is enabled (`accessflow.secrets.*` — see [docs/09-deployment.md → Secrets Manager](09-deployment.md)), the datasource credential fields -(`password`, `read_replica_password`, `api_key`) accept a **secret reference** that is stored +(`password`, each `read_replicas[].password`, `api_key`) accept a **secret reference** that is stored verbatim in the credential column and resolved through the store at credential-use time: | Provider | Reference syntax | Resolution | diff --git a/docs/09-deployment.md b/docs/09-deployment.md index 082db0ba..66c65501 100644 --- a/docs/09-deployment.md +++ b/docs/09-deployment.md @@ -877,7 +877,7 @@ Two layers exist: Optional external secret stores for datasource credentials. All providers are **off by default** — when a provider is enabled, a datasource credential field (`password`, -`read_replica_password`, `api_key`) may hold a secret reference +each `read_replicas[].password`, `api_key`) may hold a secret reference (`vault:/#` / `aws:[#jsonField]` / `azure:`) that is stored verbatim and resolved through the store at credential-use time instead of the local `ENCRYPTION_KEY` AES layer. Enabling a provider with missing required settings aborts @@ -976,6 +976,13 @@ Deployment-wide tuning for the `ai` module's `BehaviorAnomalyDetectionJob`, whic | `ACCESSFLOW_PROXY_EXECUTION_MAX_ROWS` | Optional | `10000` | Hard cap on rows returned by a single query execution | | `ACCESSFLOW_PROXY_EXECUTION_STATEMENT_TIMEOUT` | Optional | `30s` | Statement-level timeout applied to customer-DB JDBC statements | | `ACCESSFLOW_PROXY_EXECUTION_DEFAULT_FETCH_SIZE` | Optional | `1000` | Default JDBC fetch size | +| `ACCESSFLOW_PROXY_EXECUTION_INSERT_BATCH_CHUNK_SIZE` | Optional | `1000` | Rows per JDBC `executeBatch()` flush when a `BEGIN…COMMIT` envelope's homogeneous single-row INSERTs are batched (AF-457) | +| `ACCESSFLOW_PROXY_CACHE_ENABLED` | Optional | `true` | Deployment-wide kill-switch for the opt-in SELECT result cache (AF-457); the real gate is each datasource's `result_cache_enabled` flag | +| `ACCESSFLOW_PROXY_CACHE_DEFAULT_TTL` | Optional | `PT60S` | Result-cache TTL used when a datasource opts in without its own `result_cache_ttl_seconds` | +| `ACCESSFLOW_PROXY_CACHE_MAX_ENTRY_BYTES` | Optional | `1000000` | Serialized-size cap per cached SELECT result; larger results are never cached | +| `ACCESSFLOW_PROXY_REPLICA_PROBE_INTERVAL` | Optional | `PT30S` | Cadence of the per-node read-replica health prober (AF-457) | +| `ACCESSFLOW_PROXY_REPLICA_PROBE_TIMEOUT` | Optional | `PT5S` | JDBC `isValid` timeout per replica endpoint probe | +| `ACCESSFLOW_PROXY_REPLICA_COOLDOWN` | Optional | `PT30S` | How long a failed replica endpoint sits out of the read rotation before a half-open retry | | `ACCESSFLOW_PROXY_ENGINES__` | Optional | — | Generic per-engine plugin tuning (AF-418): binds `accessflow.proxy.engines..*` and is passed verbatim into the engine's `QueryEngineContext` config map. Key names are each engine's own contract (`_`/`.` in `` normalize to `-`, e.g. `ACCESSFLOW_PROXY_ENGINES_MONGODB_CONNECT_TIMEOUT` → `connect-timeout`); generic env vars override the `application.yml` defaults | | `ACCESSFLOW_PROXY_MONGO_CONNECT_TIMEOUT` | Optional | `PT10S` | Connect timeout for the per-MongoDB-datasource native `MongoClient` (MongoDB-only; relational pools use `ACCESSFLOW_PROXY_CONNECTION_TIMEOUT`). Legacy alias for `accessflow.proxy.engines.mongodb.connect-timeout` — still fully supported | | `ACCESSFLOW_PROXY_MONGO_SERVER_SELECTION_TIMEOUT` | Optional | `PT10S` | MongoDB server-selection timeout. Legacy alias for `accessflow.proxy.engines.mongodb.server-selection-timeout` | @@ -998,7 +1005,7 @@ Deployment-wide tuning for the `ai` module's `BehaviorAnomalyDetectionJob`, whic | `ACCESSFLOW_PROXY_ENGINES_NEO4J_MAX_CONNECTION_POOL_SIZE` | Optional | `100` | Max connections in the native Neo4j driver's internal Bolt connection pool for a datasource | | `ACCESSFLOW_PROXY_HEALTH_CACHE_TTL` | Optional | `PT30S` | Caffeine TTL for the admin datasource-health snapshot, cached per `(organizationId, datasourceId)` so the dashboard's 30s auto-refresh doesn't re-run the aggregate every poll. MongoDB, Redis, Cassandra, and ScyllaDB datasources report query stats but no JDBC pool counters | -> **Read-replica routing** (added in v1.2 — see [docs/05-backend.md → "Read-replica routing"](05-backend.md#read-replica-routing)) reuses the same `ACCESSFLOW_PROXY_*` HikariCP tunables above; there are no replica-specific env vars. Configure replicas per-datasource via the settings UI or `PUT /api/v1/datasources/{id}` with `read_replica_jdbc_url`/`read_replica_username`/`read_replica_password`. +> **Read-replica routing & load balancing** (multi-endpoint since v2.2/AF-457 — see [docs/05-backend.md → "Read-replica routing & load balancing"](05-backend.md#read-replica-routing--load-balancing-af-457)) reuses the same `ACCESSFLOW_PROXY_*` HikariCP tunables above for every replica pool; the `ACCESSFLOW_PROXY_REPLICA_*` vars tune only the health checks. Configure endpoints per-datasource via the settings UI or `PUT /api/v1/datasources/{id}` with the `read_replicas` array (the pre-2.2 flat `read_replica_*` fields were removed; migration `V115` auto-converts existing config). #### Custom JDBC Driver Cache diff --git a/docs/12-roadmap.md b/docs/12-roadmap.md index 81c05e80..5f7e6895 100644 --- a/docs/12-roadmap.md +++ b/docs/12-roadmap.md @@ -177,6 +177,7 @@ - **Native database wire-protocol gateway** — connect existing SQL clients (psql, DBeaver, DataGrip) and BI tools (Metabase, Tableau, Superset) through AccessFlow over the native PostgreSQL wire protocol, with every statement still flowing through the proxy's validation, masking, row-security, and audit path (AF-382) - **External secrets managers** — datasource credentials resolved from HashiCorp Vault, AWS Secrets Manager, or Azure Key Vault at connection time via a secret reference (`vault:…` / `aws:…` / `azure:…`) stored in place of the encrypted password, with local AES-256-GCM as the fallback, per-resolve audit, and SDK-managed token refresh (AF-448) +- **High-volume proxy performance** — opt-in Redis-backed SELECT result caching keyed over the security-rewritten query (invalidated on any proxied write to a referenced table), JDBC `executeBatch()` for homogeneous INSERT runs inside `BEGIN…COMMIT` envelopes, and multi-endpoint read-replica load balancing with per-node circuit-breaker health checks and primary failover (AF-457) --- diff --git a/e2e/tests/datasource-read-replica.spec.ts b/e2e/tests/datasource-read-replica.spec.ts index 6419bf95..eeaa1272 100644 --- a/e2e/tests/datasource-read-replica.spec.ts +++ b/e2e/tests/datasource-read-replica.spec.ts @@ -9,10 +9,11 @@ import { const ADMIN_EMAIL = 'e2e@accessflow.test'; const ADMIN_PASSWORD = 'E2ePassword!123'; -const UNIQUE_SUFFIX = `af360-${Date.now()}`; +const UNIQUE_SUFFIX = `af457-${Date.now()}`; const REPLICA_DS_NAME = `Postgres E2E ${UNIQUE_SUFFIX} REPLICA`; const REPLICA_JDBC_URL = 'jdbc:postgresql://postgres:5432/accessflow'; +const REPLICA_JDBC_URL_B = 'jdbc:postgresql://postgres:5432/accessflow?ApplicationName=replica-b'; const REPLICA_USER = 'accessflow'; const REPLICA_PASSWORD = 'accessflow'; @@ -35,15 +36,18 @@ async function waitForSettingsReady(page: Page, dsId: string): Promise { await expect(page.getByText('Loading datasource…')).toHaveCount(0, { timeout: 10_000 }); } -// AF-360 — configure, test, save, and clear a read-replica through the -// settings page UI. The "replica" here points at the same compose Postgres as -// the primary; the routing path is exercised by backend integration tests in -// DefaultQueryExecutorReadReplicaIntegrationTest. What this spec proves is -// that the settings page card surfaces the three fields, the Test replica +// AF-457 — configure, test, save, and remove read-replica endpoints through +// the settings page UI, plus the SELECT result-cache opt-in. The "replicas" +// here point at the same compose Postgres as the primary; routing, +// load-balancing, health failover, and cache correctness are exercised by +// backend integration tests (DefaultQueryExecutorReadReplicaIntegrationTest, +// SelectResultCacheIntegrationTest). What this spec proves is that the +// settings page renders the dynamic endpoint list, the per-row Test replica // button calls /test-replica with live values, the save persists the -// replica via the update endpoint, and a blank URL clears the replica. +// read_replicas array + cache settings via the update endpoint, and removing +// every endpoint saves an empty list. -test.describe.serial('datasource settings — read replica card', () => { +test.describe.serial('datasource settings — read replicas & performance', () => { let datasource: CreatedDatasource | null = null; test.beforeAll(async ({ request }) => { @@ -58,17 +62,18 @@ test.describe.serial('datasource settings — read replica card', () => { } }); - test('configures, tests, saves, and clears a read replica', async ({ page }) => { + test('adds, tests, saves, and removes replica endpoints', async ({ page }) => { if (!datasource) throw new Error('datasource fixture missing'); await loginViaUi(page, ADMIN_EMAIL, ADMIN_PASSWORD); await page.goto(`/datasources/${datasource.id}/settings`); await waitForSettingsReady(page, datasource.id); - // The replica fields are present and initially empty. - await expect(page.getByLabel('Replica JDBC URL')).toHaveValue(''); - await expect(page.getByLabel('Replica username')).toHaveValue(''); + // No endpoints yet — only the add button. + await expect(page.getByRole('button', { name: 'Add replica endpoint' })).toBeVisible(); + await expect(page.getByLabel('Replica JDBC URL')).toHaveCount(0); - // Fill the replica fields with live values. + // Add the first endpoint and fill it with live values. + await page.getByRole('button', { name: 'Add replica endpoint' }).click(); await page.getByLabel('Replica JDBC URL').fill(REPLICA_JDBC_URL); await page.getByLabel('Replica username').fill(REPLICA_USER); await page.getByLabel('Replica password').fill(REPLICA_PASSWORD); @@ -86,7 +91,18 @@ test.describe.serial('datasource settings — read replica card', () => { // .first(): on a fast response the inline alert and the success toast are visible at once. await expect(page.getByText(/Replica connected ·/).first()).toBeVisible({ timeout: 10_000 }); - // Save changes — the PUT body should include the replica fields. + // Add a second endpoint. + await page.getByRole('button', { name: 'Add replica endpoint' }).click(); + await expect(page.getByLabel('Replica JDBC URL')).toHaveCount(2); + await page.getByLabel('Replica JDBC URL').nth(1).fill(REPLICA_JDBC_URL_B); + await page.getByLabel('Replica username').nth(1).fill(REPLICA_USER); + await page.getByLabel('Replica password').nth(1).fill(REPLICA_PASSWORD); + + // Enable the SELECT result cache with a custom TTL. + await page.getByLabel('Cache SELECT results').click(); + await page.getByLabel('Cache TTL (seconds)').fill('120'); + + // Save changes — the PUT body should carry the endpoint list + cache settings. const [saveResponse] = await Promise.all([ page.waitForResponse( (r) => @@ -96,19 +112,47 @@ test.describe.serial('datasource settings — read replica card', () => { page.getByRole('button', { name: 'Save changes' }).click(), ]); expect(saveResponse.ok()).toBeTruthy(); - const savedBody = saveResponse.request().postDataJSON() as Record; - expect(savedBody.read_replica_jdbc_url).toBe(REPLICA_JDBC_URL); - expect(savedBody.read_replica_username).toBe(REPLICA_USER); - expect(savedBody.read_replica_password).toBe(REPLICA_PASSWORD); - - // Reload and verify the URL + username persisted (the password never round-trips). + const savedBody = saveResponse.request().postDataJSON() as { + read_replicas: { id?: string; jdbc_url: string; username?: string; password?: string }[]; + result_cache_enabled: boolean; + result_cache_ttl_seconds: number; + }; + expect(savedBody.read_replicas).toHaveLength(2); + expect(savedBody.read_replicas[0]!.jdbc_url).toBe(REPLICA_JDBC_URL); + expect(savedBody.read_replicas[0]!.username).toBe(REPLICA_USER); + expect(savedBody.read_replicas[0]!.password).toBe(REPLICA_PASSWORD); + expect(savedBody.read_replicas[1]!.jdbc_url).toBe(REPLICA_JDBC_URL_B); + expect(savedBody.result_cache_enabled).toBe(true); + expect(savedBody.result_cache_ttl_seconds).toBe(120); + + // Reload and verify both endpoints + cache settings persisted (passwords never round-trip). await page.reload(); await waitForSettingsReady(page, datasource.id); - await expect(page.getByLabel('Replica JDBC URL')).toHaveValue(REPLICA_JDBC_URL); - await expect(page.getByLabel('Replica username')).toHaveValue(REPLICA_USER); - - // Clear the URL and save — the backend clear-on-blank rule should null all three. - await page.getByLabel('Replica JDBC URL').fill(''); + await expect(page.getByLabel('Replica JDBC URL')).toHaveCount(2); + await expect(page.getByLabel('Replica JDBC URL').first()).toHaveValue(REPLICA_JDBC_URL); + await expect(page.getByLabel('Replica JDBC URL').nth(1)).toHaveValue(REPLICA_JDBC_URL_B); + await expect(page.getByLabel('Cache SELECT results')).toBeChecked(); + await expect(page.getByLabel('Cache TTL (seconds)')).toHaveValue('120'); + + // Re-test a persisted endpoint without re-typing the password — the request + // carries replica_id so the backend falls back to the stored secret. + const [retestResponse] = await Promise.all([ + page.waitForResponse( + (r) => + r.request().method() === 'POST' && + new RegExp(`/api/v1/datasources/${datasource!.id}/test-replica$`).test(r.url()), + ), + page.getByRole('button', { name: 'Test replica' }).first().click(), + ]); + expect(retestResponse.ok()).toBeTruthy(); + const retestBody = retestResponse.request().postDataJSON() as Record; + expect(typeof retestBody.replica_id).toBe('string'); + expect(retestBody.password).toBeUndefined(); + + // Remove both endpoints and save — the PUT body carries an empty list. + await page.getByRole('button', { name: 'Remove' }).first().click(); + await page.getByRole('button', { name: 'Remove' }).first().click(); + await expect(page.getByLabel('Replica JDBC URL')).toHaveCount(0); const [clearResponse] = await Promise.all([ page.waitForResponse( (r) => @@ -118,12 +162,11 @@ test.describe.serial('datasource settings — read replica card', () => { page.getByRole('button', { name: 'Save changes' }).click(), ]); expect(clearResponse.ok()).toBeTruthy(); - const clearedBody = clearResponse.request().postDataJSON() as Record; - expect(clearedBody.read_replica_jdbc_url).toBe(''); + const clearedBody = clearResponse.request().postDataJSON() as { read_replicas: unknown[] }; + expect(clearedBody.read_replicas).toEqual([]); await page.reload(); await waitForSettingsReady(page, datasource.id); - await expect(page.getByLabel('Replica JDBC URL')).toHaveValue(''); - await expect(page.getByLabel('Replica username')).toHaveValue(''); + await expect(page.getByLabel('Replica JDBC URL')).toHaveCount(0); }); }); diff --git a/frontend/src/api/datasources.ts b/frontend/src/api/datasources.ts index b6e1faec..07d2ad74 100644 --- a/frontend/src/api/datasources.ts +++ b/frontend/src/api/datasources.ts @@ -85,6 +85,8 @@ export interface TestReplicaInput { jdbc_url: string; username: string; password?: string; + // Persisted endpoint whose stored password backs the test when `password` is omitted (AF-457). + replica_id?: string; } export async function testReplicaConnection( diff --git a/frontend/src/components/editor/ReviewPlanPreview.test.tsx b/frontend/src/components/editor/ReviewPlanPreview.test.tsx index 0442fdca..8c8ff124 100644 --- a/frontend/src/components/editor/ReviewPlanPreview.test.tsx +++ b/frontend/src/components/editor/ReviewPlanPreview.test.tsx @@ -41,8 +41,9 @@ const baseDatasource: Datasource = { custom_driver_id: null, connector_id: null, jdbc_url_override: null, - read_replica_jdbc_url: null, - read_replica_username: null, + read_replicas: [], + result_cache_enabled: false, + result_cache_ttl_seconds: null, local_datacenter: null, active: true, created_at: '2026-05-01T00:00:00Z', diff --git a/frontend/src/components/editor/useQueryAuthoring.test.tsx b/frontend/src/components/editor/useQueryAuthoring.test.tsx index c0abd036..4e93480e 100644 --- a/frontend/src/components/editor/useQueryAuthoring.test.tsx +++ b/frontend/src/components/editor/useQueryAuthoring.test.tsx @@ -40,8 +40,9 @@ const baseDs: Datasource = { custom_driver_id: null, connector_id: null, jdbc_url_override: null, - read_replica_jdbc_url: null, - read_replica_username: null, + read_replicas: [], + result_cache_enabled: false, + result_cache_ttl_seconds: null, local_datacenter: null, active: true, created_at: '2026-05-01T00:00:00Z', diff --git a/frontend/src/locales/de.json b/frontend/src/locales/de.json index 2b2c674d..6ebdf115 100644 --- a/frontend/src/locales/de.json +++ b/frontend/src/locales/de.json @@ -890,7 +890,7 @@ "er_diagram_fk_label": "{{from}} → {{to}}", "er_diagram_help": "Klicken Sie auf eine Tabelle, um ihre Beziehungen hervorzuheben. Klicken Sie auf den Hintergrund, um die Auswahl aufzuheben.", "section_connection": "Verbindung", - "section_read_replica": "Lese-Replik", + "section_read_replica": "Lesereplikate", "section_limits": "Limits & Richtlinien", "label_name": "Name", "label_db_type": "Datenbanktyp", @@ -903,7 +903,7 @@ "label_replica_jdbc_url": "Replik-JDBC-URL", "label_replica_username": "Replik-Benutzername", "label_replica_password": "Replik-Passwort", - "replica_jdbc_url_help": "SELECT-Anfragen werden an diese Replik geleitet; INSERT/UPDATE/DDL gehen stets zur primären Datenbank. Leer lassen, um Replik-Routing zu deaktivieren.", + "replica_jdbc_url_help": "JDBC-URL dieses Replikat-Endpunkts. Er muss dieselbe Engine wie die Primärdatenbank verwenden.", "replica_password_placeholder": "Leer lassen, um das gespeicherte Replik-Passwort weiterzuverwenden", "test_replica_connection": "Replik testen", "replica_connection_ok": "Replik verbunden · {{ms}}ms", @@ -1162,7 +1162,17 @@ "sample_truncated": "{{count}} Zeilen (durch Proxy-Zeilenlimit begrenzt)", "sample_row_count": "{{count}} Zeilen", "sample_select_prompt": "Durchsuchen Sie das Schema und zeigen Sie eine begrenzte, kontrollierte Stichprobe der Zeilen einer Tabelle an.", - "sample_preview_aria": "Daten für {{table}} anzeigen" + "sample_preview_aria": "Daten für {{table}} anzeigen", + "replica_list_help": "SELECT-Abfragen werden im Round-Robin-Verfahren auf gesunde Replikate verteilt; INSERT/UPDATE/DDL treffen immer die Primärdatenbank. Ausgefallene Endpunkte werden übersprungen, bis sie sich erholen.", + "replica_jdbc_url_required": "Replikat-JDBC-URL ist erforderlich", + "replica_add": "Replikat-Endpunkt hinzufügen", + "replica_remove": "Entfernen", + "section_performance": "Leistung", + "label_cache_enabled": "SELECT-Ergebnisse zwischenspeichern", + "cache_enabled_help": "Wiederholte identische SELECTs aus einem Redis-Cache bedienen. Einträge werden bei jedem Schreibzugriff auf eine referenzierte Tabelle invalidiert; Maskierung und Zeilensicherheit gelten weiterhin.", + "label_cache_ttl": "Cache-TTL (Sekunden)", + "cache_ttl_help": "Wie lange ein zwischengespeichertes Ergebnis gültig bleibt (1–86.400 Sekunden). Leer lassen für den Bereitstellungsstandard.", + "cache_ttl_placeholder": "Bereitstellungsstandard" } }, "admin": { @@ -1819,7 +1829,8 @@ "errors_last_24h": "Fehler (24 Std.)", "execution_p50": "Ausf. p50", "execution_p95": "Ausf. p95", - "unit_ms": "ms" + "unit_ms": "ms", + "replicas_title": "{{healthy}}/{{total}} Replikate gesund" }, "slack": { "title": "Slack-App", diff --git a/frontend/src/locales/en.json b/frontend/src/locales/en.json index 36eaa9a3..690d79a4 100644 --- a/frontend/src/locales/en.json +++ b/frontend/src/locales/en.json @@ -932,7 +932,7 @@ "er_diagram_fk_label": "{{from}} → {{to}}", "er_diagram_help": "Click a table to highlight its relationships. Click the background to clear.", "section_connection": "Connection", - "section_read_replica": "Read replica", + "section_read_replica": "Read replicas", "section_limits": "Limits & policies", "label_name": "Name", "label_db_type": "Database type", @@ -945,7 +945,7 @@ "label_replica_jdbc_url": "Replica JDBC URL", "label_replica_username": "Replica username", "label_replica_password": "Replica password", - "replica_jdbc_url_help": "SELECT queries are routed to this replica; INSERT/UPDATE/DDL always hit the primary. Leave blank to disable replica routing.", + "replica_jdbc_url_help": "JDBC URL of this replica endpoint. It must run the same engine as the primary.", "replica_password_placeholder": "Leave blank to reuse the saved replica password", "test_replica_connection": "Test replica", "replica_connection_ok": "Replica connected · {{ms}}ms", @@ -1162,7 +1162,17 @@ "sample_truncated": "{{count}} rows (capped by the proxy row limit)", "sample_row_count": "{{count}} rows", "sample_select_prompt": "Browse the schema and preview a bounded, governed sample of any table's rows.", - "sample_preview_aria": "Preview data for {{table}}" + "sample_preview_aria": "Preview data for {{table}}", + "replica_list_help": "SELECT queries load-balance round-robin across healthy replicas; INSERT/UPDATE/DDL always hit the primary. Unhealthy endpoints are skipped until they recover.", + "replica_jdbc_url_required": "Replica JDBC URL is required", + "replica_add": "Add replica endpoint", + "replica_remove": "Remove", + "section_performance": "Performance", + "label_cache_enabled": "Cache SELECT results", + "cache_enabled_help": "Serve repeated identical SELECTs from a Redis cache. Entries are invalidated on any write to a referenced table; masking and row-security still apply.", + "label_cache_ttl": "Cache TTL (seconds)", + "cache_ttl_help": "How long a cached result stays valid (1–86,400 seconds). Leave blank for the deployment default.", + "cache_ttl_placeholder": "Deployment default" } }, "admin": { @@ -1819,7 +1829,8 @@ "errors_last_24h": "Errors (24h)", "execution_p50": "Exec p50", "execution_p95": "Exec p95", - "unit_ms": "ms" + "unit_ms": "ms", + "replicas_title": "{{healthy}}/{{total}} replicas healthy" }, "slack": { "title": "Slack app", diff --git a/frontend/src/locales/es.json b/frontend/src/locales/es.json index 44314a6d..104641c2 100644 --- a/frontend/src/locales/es.json +++ b/frontend/src/locales/es.json @@ -890,7 +890,7 @@ "er_diagram_fk_label": "{{from}} → {{to}}", "er_diagram_help": "Haz clic en una tabla para resaltar sus relaciones. Haz clic en el fondo para limpiar.", "section_connection": "Conexión", - "section_read_replica": "Réplica de lectura", + "section_read_replica": "Réplicas de lectura", "section_limits": "Límites y políticas", "label_name": "Nombre", "label_db_type": "Tipo de base de datos", @@ -903,7 +903,7 @@ "label_replica_jdbc_url": "URL JDBC de la réplica", "label_replica_username": "Usuario de la réplica", "label_replica_password": "Contraseña de la réplica", - "replica_jdbc_url_help": "Las consultas SELECT se enrutan a esta réplica; INSERT/UPDATE/DDL siempre van a la primaria. Deja en blanco para desactivar el enrutamiento a la réplica.", + "replica_jdbc_url_help": "URL JDBC de este punto de conexión de réplica. Debe ejecutar el mismo motor que la base primaria.", "replica_password_placeholder": "Deja en blanco para reutilizar la contraseña guardada de la réplica", "test_replica_connection": "Probar réplica", "replica_connection_ok": "Réplica conectada · {{ms}}ms", @@ -1162,7 +1162,17 @@ "sample_truncated": "{{count}} filas (limitado por el límite de filas del proxy)", "sample_row_count": "{{count}} filas", "sample_select_prompt": "Explore el esquema y previsualice una muestra limitada y controlada de las filas de cualquier tabla.", - "sample_preview_aria": "Previsualizar datos de {{table}}" + "sample_preview_aria": "Previsualizar datos de {{table}}", + "replica_list_help": "Las consultas SELECT se balancean en round-robin entre las réplicas sanas; INSERT/UPDATE/DDL siempre van a la base primaria. Los puntos de conexión no disponibles se omiten hasta que se recuperan.", + "replica_jdbc_url_required": "La URL JDBC de la réplica es obligatoria", + "replica_add": "Añadir punto de conexión de réplica", + "replica_remove": "Eliminar", + "section_performance": "Rendimiento", + "label_cache_enabled": "Cachear resultados SELECT", + "cache_enabled_help": "Servir SELECTs idénticos repetidos desde una caché Redis. Las entradas se invalidan con cada escritura en una tabla referenciada; el enmascaramiento y la seguridad de filas siguen aplicándose.", + "label_cache_ttl": "TTL de caché (segundos)", + "cache_ttl_help": "Cuánto tiempo permanece válido un resultado en caché (1–86.400 segundos). Déjelo en blanco para el valor por defecto del despliegue.", + "cache_ttl_placeholder": "Valor por defecto del despliegue" } }, "admin": { @@ -1819,7 +1829,8 @@ "errors_last_24h": "Errores (24 h)", "execution_p50": "Ejec p50", "execution_p95": "Ejec p95", - "unit_ms": "ms" + "unit_ms": "ms", + "replicas_title": "{{healthy}}/{{total}} réplicas sanas" }, "slack": { "title": "Aplicación de Slack", diff --git a/frontend/src/locales/fr.json b/frontend/src/locales/fr.json index 45ea1ee1..e0a22a7c 100644 --- a/frontend/src/locales/fr.json +++ b/frontend/src/locales/fr.json @@ -890,7 +890,7 @@ "er_diagram_fk_label": "{{from}} → {{to}}", "er_diagram_help": "Cliquez sur une table pour mettre en évidence ses relations. Cliquez sur l'arrière-plan pour effacer.", "section_connection": "Connexion", - "section_read_replica": "Réplica de lecture", + "section_read_replica": "Réplicas de lecture", "section_limits": "Limites et politiques", "label_name": "Nom", "label_db_type": "Type de base", @@ -903,7 +903,7 @@ "label_replica_jdbc_url": "URL JDBC de la réplica", "label_replica_username": "Utilisateur de la réplica", "label_replica_password": "Mot de passe de la réplica", - "replica_jdbc_url_help": "Les requêtes SELECT sont routées vers cette réplica ; INSERT/UPDATE/DDL vont toujours sur la primaire. Laissez vide pour désactiver le routage vers la réplica.", + "replica_jdbc_url_help": "URL JDBC de ce point de terminaison de réplica. Il doit exécuter le même moteur que la base primaire.", "replica_password_placeholder": "Laissez vide pour réutiliser le mot de passe enregistré de la réplica", "test_replica_connection": "Tester la réplica", "replica_connection_ok": "Réplica connectée · {{ms}}ms", @@ -1162,7 +1162,17 @@ "sample_truncated": "{{count}} lignes (limité par la limite de lignes du proxy)", "sample_row_count": "{{count}} lignes", "sample_select_prompt": "Parcourez le schéma et prévisualisez un échantillon limité et contrôlé des lignes de n'importe quelle table.", - "sample_preview_aria": "Aperçu des données de {{table}}" + "sample_preview_aria": "Aperçu des données de {{table}}", + "replica_list_help": "Les requêtes SELECT sont réparties en round-robin entre les réplicas sains ; INSERT/UPDATE/DDL vont toujours vers la base primaire. Les points de terminaison défaillants sont ignorés jusqu’à leur rétablissement.", + "replica_jdbc_url_required": "L’URL JDBC du réplica est requise", + "replica_add": "Ajouter un point de terminaison de réplica", + "replica_remove": "Supprimer", + "section_performance": "Performance", + "label_cache_enabled": "Mettre en cache les résultats SELECT", + "cache_enabled_help": "Servir les SELECT identiques répétés depuis un cache Redis. Les entrées sont invalidées à chaque écriture sur une table référencée ; le masquage et la sécurité des lignes s’appliquent toujours.", + "label_cache_ttl": "TTL du cache (secondes)", + "cache_ttl_help": "Durée de validité d’un résultat en cache (1–86 400 secondes). Laisser vide pour la valeur par défaut du déploiement.", + "cache_ttl_placeholder": "Valeur par défaut du déploiement" } }, "admin": { @@ -1819,7 +1829,8 @@ "errors_last_24h": "Erreurs (24 h)", "execution_p50": "Exéc p50", "execution_p95": "Exéc p95", - "unit_ms": "ms" + "unit_ms": "ms", + "replicas_title": "{{healthy}}/{{total}} réplicas sains" }, "slack": { "title": "Application Slack", diff --git a/frontend/src/locales/hy.json b/frontend/src/locales/hy.json index fe4e27af..3e8ebc73 100644 --- a/frontend/src/locales/hy.json +++ b/frontend/src/locales/hy.json @@ -890,7 +890,7 @@ "er_diagram_fk_label": "{{from}} → {{to}}", "er_diagram_help": "Սեղմեք աղյուսակի վրա՝ նրա կապերը ընդգծելու համար։ Սեղմեք ֆոնի վրա՝ ընտրությունը մաքրելու համար։", "section_connection": "Միացում", - "section_read_replica": "Ընթերցման ռեպլիկա", + "section_read_replica": "Ընթերցման կրկնօրինակներ", "section_limits": "Սահմանաչափեր և քաղաքականություններ", "label_name": "Անվանում", "label_db_type": "Տվյալների բազայի տեսակ", @@ -903,7 +903,7 @@ "label_replica_jdbc_url": "Ռեպլիկայի JDBC URL", "label_replica_username": "Ռեպլիկայի օգտանուն", "label_replica_password": "Ռեպլիկայի գաղտնաբառ", - "replica_jdbc_url_help": "SELECT հարցումները ուղղորդվում են այս ռեպլիկային; INSERT/UPDATE/DDL միշտ ուղղորդվում են առաջնային բազային։ Թողեք դատարկ՝ ռեպլիկայի երթուղավորումը անջատելու համար։", + "replica_jdbc_url_help": "Այս կրկնօրինակի վերջնակետի JDBC URL-ը: Այն պետք է աշխատի նույն շարժիչով, ինչ հիմնական բազան:", "replica_password_placeholder": "Թողեք դատարկ՝ պահպանված ռեպլիկայի գաղտնաբառը կրկին օգտագործելու համար", "test_replica_connection": "Փորձարկել ռեպլիկան", "replica_connection_ok": "Ռեպլիկան միացված է · {{ms}} մվ", @@ -1162,7 +1162,17 @@ "sample_truncated": "{{count}} տող (սահմանափակված է պրոքսիի տողերի սահմանով)", "sample_row_count": "{{count}} տող", "sample_select_prompt": "Դիտեք սխեման և նախադիտեք ցանկացած աղյուսակի տողերի սահմանափակ, վերահսկվող նմուշը։", - "sample_preview_aria": "Նախադիտել {{table}}-ի տվյալները" + "sample_preview_aria": "Նախադիտել {{table}}-ի տվյալները", + "replica_list_help": "SELECT հարցումները round-robin սկզբունքով բաշխվում են առողջ կրկնօրինակների միջև. INSERT/UPDATE/DDL-ը միշտ ուղղվում է հիմնական բազային: Անհասանելի վերջնակետերը բաց են թողնվում մինչև վերականգնվեն:", + "replica_jdbc_url_required": "Կրկնօրինակի JDBC URL-ը պարտադիր է", + "replica_add": "Ավելացնել կրկնօրինակի վերջնակետ", + "replica_remove": "Հեռացնել", + "section_performance": "Արտադրողականություն", + "label_cache_enabled": "Քեշավորել SELECT արդյունքները", + "cache_enabled_help": "Կրկնվող նույնական SELECT-ները սպասարկել Redis քեշից: Գրանցումներն անվավեր են ճանաչվում հղված աղյուսակի ցանկացած գրառման դեպքում. դիմակավորումը և տողերի անվտանգությունը շարունակում են գործել:", + "label_cache_ttl": "Քեշի TTL (վայրկյան)", + "cache_ttl_help": "Որքան ժամանակ է քեշավորված արդյունքը վավեր մնում (1–86400 վայրկյան): Թողեք դատարկ՝ տեղակայման լռելյայն արժեքի համար:", + "cache_ttl_placeholder": "Տեղակայման լռելյայն" } }, "admin": { @@ -1819,7 +1829,8 @@ "errors_last_24h": "Սխալներ (24ժ)", "execution_p50": "Կատ. p50", "execution_p95": "Կատ. p95", - "unit_ms": "մվ" + "unit_ms": "մվ", + "replicas_title": "{{healthy}}/{{total}} կրկնօրինակ առողջ է" }, "slack": { "title": "Slack հավելված", diff --git a/frontend/src/locales/ru.json b/frontend/src/locales/ru.json index a8a5df11..ada2a3b0 100644 --- a/frontend/src/locales/ru.json +++ b/frontend/src/locales/ru.json @@ -890,7 +890,7 @@ "er_diagram_fk_label": "{{from}} → {{to}}", "er_diagram_help": "Нажмите на таблицу, чтобы выделить её связи. Нажмите на фон, чтобы сбросить выделение.", "section_connection": "Подключение", - "section_read_replica": "Реплика чтения", + "section_read_replica": "Реплики чтения", "section_limits": "Лимиты и политики", "label_name": "Название", "label_db_type": "Тип базы", @@ -903,7 +903,7 @@ "label_replica_jdbc_url": "JDBC URL реплики", "label_replica_username": "Имя пользователя реплики", "label_replica_password": "Пароль реплики", - "replica_jdbc_url_help": "Запросы SELECT направляются на эту реплику; INSERT/UPDATE/DDL всегда выполняются на основной БД. Оставьте пустым, чтобы отключить маршрутизацию на реплику.", + "replica_jdbc_url_help": "JDBC URL этой конечной точки реплики. Она должна работать на том же движке, что и основная база.", "replica_password_placeholder": "Оставьте пустым, чтобы использовать сохранённый пароль реплики", "test_replica_connection": "Проверить реплику", "replica_connection_ok": "Реплика подключена · {{ms}} мс", @@ -1162,7 +1162,17 @@ "sample_truncated": "{{count}} строк (ограничено лимитом строк прокси)", "sample_row_count": "{{count}} строк", "sample_select_prompt": "Просматривайте схему и предварительно просматривайте ограниченную, контролируемую выборку строк любой таблицы.", - "sample_preview_aria": "Предпросмотр данных {{table}}" + "sample_preview_aria": "Предпросмотр данных {{table}}", + "replica_list_help": "Запросы SELECT балансируются по алгоритму round-robin между работоспособными репликами; INSERT/UPDATE/DDL всегда идут в основную базу. Недоступные конечные точки пропускаются до восстановления.", + "replica_jdbc_url_required": "JDBC URL реплики обязателен", + "replica_add": "Добавить конечную точку реплики", + "replica_remove": "Удалить", + "section_performance": "Производительность", + "label_cache_enabled": "Кэшировать результаты SELECT", + "cache_enabled_help": "Обслуживать повторяющиеся одинаковые SELECT из кэша Redis. Записи инвалидируются при любой записи в связанную таблицу; маскирование и защита строк продолжают действовать.", + "label_cache_ttl": "TTL кэша (секунды)", + "cache_ttl_help": "Как долго кэшированный результат остаётся действительным (1–86 400 секунд). Оставьте пустым для значения по умолчанию.", + "cache_ttl_placeholder": "Значение по умолчанию" } }, "admin": { @@ -1819,7 +1829,8 @@ "errors_last_24h": "Ошибки (24 ч)", "execution_p50": "Вып. p50", "execution_p95": "Вып. p95", - "unit_ms": "мс" + "unit_ms": "мс", + "replicas_title": "{{healthy}}/{{total}} реплик работоспособны" }, "slack": { "title": "Приложение Slack", diff --git a/frontend/src/locales/zh-CN.json b/frontend/src/locales/zh-CN.json index 5e0e0ef3..1da859c9 100644 --- a/frontend/src/locales/zh-CN.json +++ b/frontend/src/locales/zh-CN.json @@ -890,7 +890,7 @@ "er_diagram_fk_label": "{{from}} → {{to}}", "er_diagram_help": "点击表以突出显示其关系。点击背景以清除。", "section_connection": "连接", - "section_read_replica": "只读副本", + "section_read_replica": "读副本", "section_limits": "限制与策略", "label_name": "名称", "label_db_type": "数据库类型", @@ -903,7 +903,7 @@ "label_replica_jdbc_url": "副本 JDBC URL", "label_replica_username": "副本用户名", "label_replica_password": "副本密码", - "replica_jdbc_url_help": "SELECT 查询会路由到该副本；INSERT/UPDATE/DDL 始终在主库执行。留空可禁用副本路由。", + "replica_jdbc_url_help": "此副本端点的 JDBC URL。它必须与主库运行相同的引擎。", "replica_password_placeholder": "留空以沿用已保存的副本密码", "test_replica_connection": "测试副本", "replica_connection_ok": "副本已连接 · {{ms}}ms", @@ -1162,7 +1162,17 @@ "sample_truncated": "{{count}} 行（受代理行数限制）", "sample_row_count": "{{count}} 行", "sample_select_prompt": "浏览架构并预览任意表的受控、有限的行样本。", - "sample_preview_aria": "预览 {{table}} 的数据" + "sample_preview_aria": "预览 {{table}} 的数据", + "replica_list_help": "SELECT 查询在健康的副本之间轮询负载均衡；INSERT/UPDATE/DDL 始终发送到主库。不可用的端点会被跳过，直至恢复。", + "replica_jdbc_url_required": "副本 JDBC URL 为必填项", + "replica_add": "添加副本端点", + "replica_remove": "移除", + "section_performance": "性能", + "label_cache_enabled": "缓存 SELECT 结果", + "cache_enabled_help": "从 Redis 缓存中响应重复的相同 SELECT。对被引用表的任何写入都会使条目失效；脱敏和行级安全仍然生效。", + "label_cache_ttl": "缓存 TTL（秒）", + "cache_ttl_help": "缓存结果的有效时长（1–86,400 秒）。留空则使用部署默认值。", + "cache_ttl_placeholder": "部署默认值" } }, "admin": { @@ -1819,7 +1829,8 @@ "errors_last_24h": "错误（24 小时）", "execution_p50": "执行 p50", "execution_p95": "执行 p95", - "unit_ms": "毫秒" + "unit_ms": "毫秒", + "replicas_title": "{{healthy}}/{{total}} 个副本健康" }, "slack": { "title": "Slack 应用", diff --git a/frontend/src/mocks/data.ts b/frontend/src/mocks/data.ts index cda777af..dad37dc7 100644 --- a/frontend/src/mocks/data.ts +++ b/frontend/src/mocks/data.ts @@ -36,14 +36,14 @@ export const USERS: User[] = [ ]; export const DATASOURCES: Datasource[] = [ - { id: 'ds-01', organization_id: 'org-demo', name: 'Production PostgreSQL', db_type: 'POSTGRESQL', host: 'db-prod.acme.internal', port: 5432, database_name: 'app_prod', username: 'accessflow_svc', ssl_mode: 'VERIFY_FULL', max_rows_per_query: 1000, require_review_writes: true, require_review_reads: false, ai_analysis_enabled: true, ai_config_id: null, text_to_sql_enabled: false, active: true, connection_pool_size: 25, review_plan_id: 'rp-strict', custom_driver_id: null, connector_id: null, jdbc_url_override: null, read_replica_jdbc_url: null, read_replica_username: null, local_datacenter: null, created_at: '2026-01-15T10:00:00Z' }, - { id: 'ds-02', organization_id: 'org-demo', name: 'Production MySQL', db_type: 'MYSQL', host: 'mysql-prod.acme.internal', port: 3306, database_name: 'commerce', username: 'accessflow_svc', ssl_mode: 'VERIFY_FULL', max_rows_per_query: 1000, require_review_writes: true, require_review_reads: false, ai_analysis_enabled: true, ai_config_id: null, text_to_sql_enabled: false, active: true, connection_pool_size: 20, review_plan_id: 'rp-strict', custom_driver_id: null, connector_id: null, jdbc_url_override: null, read_replica_jdbc_url: null, read_replica_username: null, local_datacenter: null, created_at: '2026-01-15T10:00:00Z' }, - { id: 'ds-03', organization_id: 'org-demo', name: 'Analytics Replica', db_type: 'POSTGRESQL', host: 'replica-01.acme.internal', port: 5432, database_name: 'analytics', username: 'accessflow_svc', ssl_mode: 'REQUIRE', max_rows_per_query: 50000, require_review_writes: true, require_review_reads: false, ai_analysis_enabled: true, ai_config_id: null, text_to_sql_enabled: false, active: true, connection_pool_size: 50, review_plan_id: 'rp-readonly', custom_driver_id: null, connector_id: null, jdbc_url_override: null, read_replica_jdbc_url: null, read_replica_username: null, local_datacenter: null, created_at: '2026-01-20T10:00:00Z' }, - { id: 'ds-04', organization_id: 'org-demo', name: 'Staging PostgreSQL', db_type: 'POSTGRESQL', host: 'db-stage.acme.internal', port: 5432, database_name: 'app_stage', username: 'accessflow_svc', ssl_mode: 'REQUIRE', max_rows_per_query: 5000, require_review_writes: false, require_review_reads: false, ai_analysis_enabled: true, ai_config_id: null, text_to_sql_enabled: false, active: true, connection_pool_size: 10, review_plan_id: 'rp-light', custom_driver_id: null, connector_id: null, jdbc_url_override: null, read_replica_jdbc_url: null, read_replica_username: null, local_datacenter: null, created_at: '2026-01-22T10:00:00Z' }, - { id: 'ds-05', organization_id: 'org-demo', name: 'Billing PostgreSQL', db_type: 'POSTGRESQL', host: 'db-billing.acme.internal', port: 5432, database_name: 'billing', username: 'accessflow_svc', ssl_mode: 'VERIFY_FULL', max_rows_per_query: 500, require_review_writes: true, require_review_reads: true, ai_analysis_enabled: true, ai_config_id: null, text_to_sql_enabled: false, active: true, connection_pool_size: 15, review_plan_id: 'rp-strict', custom_driver_id: null, connector_id: null, jdbc_url_override: null, read_replica_jdbc_url: null, read_replica_username: null, local_datacenter: null, created_at: '2026-02-05T10:00:00Z' }, - { id: 'ds-06', organization_id: 'org-demo', name: 'Marketing Warehouse', db_type: 'POSTGRESQL', host: 'warehouse.acme.internal', port: 5432, database_name: 'mkt_warehouse', username: 'accessflow_svc', ssl_mode: 'REQUIRE', max_rows_per_query: 100000, require_review_writes: true, require_review_reads: false, ai_analysis_enabled: false, ai_config_id: null, text_to_sql_enabled: false, active: true, connection_pool_size: 30, review_plan_id: 'rp-readonly', custom_driver_id: null, connector_id: null, jdbc_url_override: null, read_replica_jdbc_url: null, read_replica_username: null, local_datacenter: null, created_at: '2026-02-18T10:00:00Z' }, - { id: 'ds-07', organization_id: 'org-demo', name: 'Legacy Reporting MySQL', db_type: 'MYSQL', host: 'legacy-mysql.acme.internal', port: 3306, database_name: 'reporting_v1', username: 'accessflow_svc', ssl_mode: 'DISABLE', max_rows_per_query: 10000, require_review_writes: true, require_review_reads: false, ai_analysis_enabled: true, ai_config_id: null, text_to_sql_enabled: false, active: true, connection_pool_size: 8, review_plan_id: 'rp-strict', custom_driver_id: null, connector_id: null, jdbc_url_override: null, read_replica_jdbc_url: null, read_replica_username: null, local_datacenter: null, created_at: '2026-03-04T10:00:00Z' }, - { id: 'ds-08', organization_id: 'org-demo', name: 'Sandbox PG', db_type: 'POSTGRESQL', host: 'sandbox.acme.internal', port: 5432, database_name: 'sandbox', username: 'accessflow_svc', ssl_mode: 'DISABLE', max_rows_per_query: 10000, require_review_writes: false, require_review_reads: false, ai_analysis_enabled: true, ai_config_id: null, text_to_sql_enabled: false, active: false, connection_pool_size: 5, review_plan_id: 'rp-light', custom_driver_id: null, connector_id: null, jdbc_url_override: null, read_replica_jdbc_url: null, read_replica_username: null, local_datacenter: null, created_at: '2026-03-15T10:00:00Z' }, + { id: 'ds-01', organization_id: 'org-demo', name: 'Production PostgreSQL', db_type: 'POSTGRESQL', host: 'db-prod.acme.internal', port: 5432, database_name: 'app_prod', username: 'accessflow_svc', ssl_mode: 'VERIFY_FULL', max_rows_per_query: 1000, require_review_writes: true, require_review_reads: false, ai_analysis_enabled: true, ai_config_id: null, text_to_sql_enabled: false, active: true, connection_pool_size: 25, review_plan_id: 'rp-strict', custom_driver_id: null, connector_id: null, jdbc_url_override: null, read_replicas: [], local_datacenter: null, result_cache_enabled: false, result_cache_ttl_seconds: null, created_at: '2026-01-15T10:00:00Z' }, + { id: 'ds-02', organization_id: 'org-demo', name: 'Production MySQL', db_type: 'MYSQL', host: 'mysql-prod.acme.internal', port: 3306, database_name: 'commerce', username: 'accessflow_svc', ssl_mode: 'VERIFY_FULL', max_rows_per_query: 1000, require_review_writes: true, require_review_reads: false, ai_analysis_enabled: true, ai_config_id: null, text_to_sql_enabled: false, active: true, connection_pool_size: 20, review_plan_id: 'rp-strict', custom_driver_id: null, connector_id: null, jdbc_url_override: null, read_replicas: [], local_datacenter: null, result_cache_enabled: false, result_cache_ttl_seconds: null, created_at: '2026-01-15T10:00:00Z' }, + { id: 'ds-03', organization_id: 'org-demo', name: 'Analytics Replica', db_type: 'POSTGRESQL', host: 'replica-01.acme.internal', port: 5432, database_name: 'analytics', username: 'accessflow_svc', ssl_mode: 'REQUIRE', max_rows_per_query: 50000, require_review_writes: true, require_review_reads: false, ai_analysis_enabled: true, ai_config_id: null, text_to_sql_enabled: false, active: true, connection_pool_size: 50, review_plan_id: 'rp-readonly', custom_driver_id: null, connector_id: null, jdbc_url_override: null, read_replicas: [], local_datacenter: null, result_cache_enabled: false, result_cache_ttl_seconds: null, created_at: '2026-01-20T10:00:00Z' }, + { id: 'ds-04', organization_id: 'org-demo', name: 'Staging PostgreSQL', db_type: 'POSTGRESQL', host: 'db-stage.acme.internal', port: 5432, database_name: 'app_stage', username: 'accessflow_svc', ssl_mode: 'REQUIRE', max_rows_per_query: 5000, require_review_writes: false, require_review_reads: false, ai_analysis_enabled: true, ai_config_id: null, text_to_sql_enabled: false, active: true, connection_pool_size: 10, review_plan_id: 'rp-light', custom_driver_id: null, connector_id: null, jdbc_url_override: null, read_replicas: [], local_datacenter: null, result_cache_enabled: false, result_cache_ttl_seconds: null, created_at: '2026-01-22T10:00:00Z' }, + { id: 'ds-05', organization_id: 'org-demo', name: 'Billing PostgreSQL', db_type: 'POSTGRESQL', host: 'db-billing.acme.internal', port: 5432, database_name: 'billing', username: 'accessflow_svc', ssl_mode: 'VERIFY_FULL', max_rows_per_query: 500, require_review_writes: true, require_review_reads: true, ai_analysis_enabled: true, ai_config_id: null, text_to_sql_enabled: false, active: true, connection_pool_size: 15, review_plan_id: 'rp-strict', custom_driver_id: null, connector_id: null, jdbc_url_override: null, read_replicas: [], local_datacenter: null, result_cache_enabled: false, result_cache_ttl_seconds: null, created_at: '2026-02-05T10:00:00Z' }, + { id: 'ds-06', organization_id: 'org-demo', name: 'Marketing Warehouse', db_type: 'POSTGRESQL', host: 'warehouse.acme.internal', port: 5432, database_name: 'mkt_warehouse', username: 'accessflow_svc', ssl_mode: 'REQUIRE', max_rows_per_query: 100000, require_review_writes: true, require_review_reads: false, ai_analysis_enabled: false, ai_config_id: null, text_to_sql_enabled: false, active: true, connection_pool_size: 30, review_plan_id: 'rp-readonly', custom_driver_id: null, connector_id: null, jdbc_url_override: null, read_replicas: [], local_datacenter: null, result_cache_enabled: false, result_cache_ttl_seconds: null, created_at: '2026-02-18T10:00:00Z' }, + { id: 'ds-07', organization_id: 'org-demo', name: 'Legacy Reporting MySQL', db_type: 'MYSQL', host: 'legacy-mysql.acme.internal', port: 3306, database_name: 'reporting_v1', username: 'accessflow_svc', ssl_mode: 'DISABLE', max_rows_per_query: 10000, require_review_writes: true, require_review_reads: false, ai_analysis_enabled: true, ai_config_id: null, text_to_sql_enabled: false, active: true, connection_pool_size: 8, review_plan_id: 'rp-strict', custom_driver_id: null, connector_id: null, jdbc_url_override: null, read_replicas: [], local_datacenter: null, result_cache_enabled: false, result_cache_ttl_seconds: null, created_at: '2026-03-04T10:00:00Z' }, + { id: 'ds-08', organization_id: 'org-demo', name: 'Sandbox PG', db_type: 'POSTGRESQL', host: 'sandbox.acme.internal', port: 5432, database_name: 'sandbox', username: 'accessflow_svc', ssl_mode: 'DISABLE', max_rows_per_query: 10000, require_review_writes: false, require_review_reads: false, ai_analysis_enabled: true, ai_config_id: null, text_to_sql_enabled: false, active: false, connection_pool_size: 5, review_plan_id: 'rp-light', custom_driver_id: null, connector_id: null, jdbc_url_override: null, read_replicas: [], local_datacenter: null, result_cache_enabled: false, result_cache_ttl_seconds: null, created_at: '2026-03-15T10:00:00Z' }, ]; const NOW = new Date('2026-05-04T10:30:00Z').getTime(); diff --git a/frontend/src/pages/admin/DatasourceHealthPage.test.tsx b/frontend/src/pages/admin/DatasourceHealthPage.test.tsx index edf5a2cb..3a411dd2 100644 --- a/frontend/src/pages/admin/DatasourceHealthPage.test.tsx +++ b/frontend/src/pages/admin/DatasourceHealthPage.test.tsx @@ -45,6 +45,7 @@ function row(overrides: Partial = {}): DatasourceHealth { execution_ms_p50: 12.5, execution_ms_p95: 88, errors_last_24h: 3, + replicas: [], ...overrides, }; } diff --git a/frontend/src/pages/admin/DatasourceHealthPage.tsx b/frontend/src/pages/admin/DatasourceHealthPage.tsx index a098099e..302e229f 100644 --- a/frontend/src/pages/admin/DatasourceHealthPage.tsx +++ b/frontend/src/pages/admin/DatasourceHealthPage.tsx @@ -132,6 +132,30 @@ function HealthCard({ row }: { row: DatasourceHealth }) { /> + {row.replicas.length > 0 && ( +

+
+ {t('admin.datasource_health.replicas_title', { + healthy: row.replicas.filter((r) => r.healthy).length, + total: row.replicas.length, + })} +
+ + {row.replicas.map((replica) => ( + + {replica.label} + {replica.pool_active != null && replica.pool_total != null + ? ` · ${replica.pool_active}/${replica.pool_total}` + : ''} + + ))} + +
+ )} ); } diff --git a/frontend/src/pages/datasources/DatasourceSettingsPage.tsx b/frontend/src/pages/datasources/DatasourceSettingsPage.tsx index b5911eac..0b33b016 100644 --- a/frontend/src/pages/datasources/DatasourceSettingsPage.tsx +++ b/frontend/src/pages/datasources/DatasourceSettingsPage.tsx @@ -282,11 +282,22 @@ interface ConfigTabProps { deletePending: boolean; } +interface ReplicaFormRow { + id?: string; + jdbc_url: string; + username?: string; + password?: string; +} + +type SettingsFormValues = Omit & { + read_replicas: ReplicaFormRow[]; +}; + function ConfigTab({ ds, onDelete, deletePending }: ConfigTabProps) { const { t } = useTranslation(); const { message } = App.useApp(); const queryClient = useQueryClient(); - const [form] = Form.useForm(); + const [form] = Form.useForm(); const reviewPlansQuery = useQuery({ queryKey: reviewPlanKeys.lists(), @@ -297,7 +308,7 @@ function ConfigTab({ ds, onDelete, deletePending }: ConfigTabProps) { const secretRefHelp = secretReferenceHelp(secretProviders, t); const secretRefRule = secretReferenceRule(secretProviders, t); - const initialValues: UpdateDatasourceInput = { + const initialValues: SettingsFormValues = { name: ds.name, host: ds.host ?? undefined, port: ds.port ?? undefined, @@ -312,51 +323,58 @@ function ConfigTab({ ds, onDelete, deletePending }: ConfigTabProps) { ai_analysis_enabled: ds.ai_analysis_enabled, ai_config_id: ds.ai_config_id ?? null, text_to_sql_enabled: ds.text_to_sql_enabled, - read_replica_jdbc_url: ds.read_replica_jdbc_url ?? '', - read_replica_username: ds.read_replica_username ?? '', + read_replicas: ds.read_replicas.map((r) => ({ + id: r.id, + jdbc_url: r.jdbc_url, + username: r.username ?? '', + password: '', + })), + result_cache_enabled: ds.result_cache_enabled, + result_cache_ttl_seconds: ds.result_cache_ttl_seconds ?? undefined, active: ds.active, }; - const [replicaResult, setReplicaResult] = useState(null); - const [replicaError, setReplicaError] = useState(null); + const [replicaResults, setReplicaResults] = useState< + Record + >({}); const testReplicaMutation = useMutation({ - mutationFn: () => { - const values = form.getFieldsValue([ - 'read_replica_jdbc_url', - 'read_replica_username', - 'read_replica_password', - ]) as { - read_replica_jdbc_url?: string; - read_replica_username?: string; - read_replica_password?: string; - }; - const password = values.read_replica_password?.trim() - ? values.read_replica_password - : undefined; + mutationFn: (index: number) => { + const rows = (form.getFieldValue('read_replicas') ?? []) as ReplicaFormRow[]; + const row = rows[index] ?? { jdbc_url: '' }; + const password = row.password?.trim() ? row.password : undefined; return testReplicaConnection(ds.id, { - jdbc_url: values.read_replica_jdbc_url ?? '', - username: values.read_replica_username ?? '', + jdbc_url: row.jdbc_url ?? '', + username: row.username ?? '', password, + // Fall back to this endpoint's stored password when none was typed. + replica_id: !password && row.id ? row.id : undefined, }); }, - onMutate: () => { - setReplicaResult(null); - setReplicaError(null); + onMutate: (index: number) => { + setReplicaResults((prev) => ({ ...prev, [index]: {} })); }, - onSuccess: (result) => { - setReplicaResult(result); + onSuccess: (result, index) => { + setReplicaResults((prev) => ({ ...prev, [index]: { result } })); if (result.ok) { message.success(t('datasources.settings.replica_connection_ok', { ms: result.latency_ms })); } else { - setReplicaError(result.message ?? t('datasources.settings.replica_connection_failed')); + setReplicaResults((prev) => ({ + ...prev, + [index]: { + error: result.message ?? t('datasources.settings.replica_connection_failed'), + }, + })); } }, - onError: (err: unknown) => { + onError: (err: unknown, index) => { const detail = isAxiosError(err) && typeof err.response?.data === 'object' && err.response?.data ? (err.response.data as { detail?: string }).detail : null; - setReplicaError(detail ?? t('datasources.settings.replica_connection_failed')); + setReplicaResults((prev) => ({ + ...prev, + [index]: { error: detail ?? t('datasources.settings.replica_connection_failed') }, + })); }, }); @@ -377,8 +395,9 @@ function ConfigTab({ ds, onDelete, deletePending }: ConfigTabProps) { }, }); - const onFinish = (values: UpdateDatasourceInput & { password?: string }) => { - const body: UpdateDatasourceInput = { ...values }; + const onFinish = (values: SettingsFormValues) => { + const { read_replicas: replicaRows, ...rest } = values; + const body: UpdateDatasourceInput = { ...rest }; if (!body.password || body.password.trim().length === 0) { delete body.password; } @@ -387,10 +406,17 @@ function ConfigTab({ ds, onDelete, deletePending }: ConfigTabProps) { body.clear_ai_config = true; body.ai_config_id = null; } - if (!body.read_replica_password || body.read_replica_password.trim().length === 0) { - delete body.read_replica_password; + // Full-list replacement: an empty list deletes every endpoint. A blank password on an + // existing row keeps the stored secret (undefined), never clears it. + body.read_replicas = (replicaRows ?? []).map((row) => ({ + id: row.id ?? undefined, + jdbc_url: row.jdbc_url, + username: row.username?.trim() ? row.username : undefined, + password: row.password?.trim() ? row.password : undefined, + })); + if (body.result_cache_enabled === false) { + delete body.result_cache_ttl_seconds; } - // Pass blank read_replica_jdbc_url straight through so the backend clear-on-blank rule fires. updateMutation.mutate(body); }; @@ -458,47 +484,135 @@ function ConfigTab({ ds, onDelete, deletePending }: ConfigTabProps) {
+

+ {t('datasources.settings.replica_list_help')} +

+ + {(fields, { add, remove }) => ( + <> + {fields.map(({ key, name }) => ( +
+ + + + + +
+ + + + + + + +
+ testReplicaMutation.mutate(name)} + buttonLabel={t('datasources.settings.test_replica_connection')} + successLabel={(ms) => + t('datasources.settings.replica_connection_ok', { ms }) + } + /> +
+ +
+
+ ))} + + + )} + +
+
- - -
- - + + prev.result_cache_enabled !== next.result_cache_enabled + } + noStyle > - + {({ getFieldValue }) => ( + + + + )} -
- testReplicaMutation.mutate()} - buttonLabel={t('datasources.settings.test_replica_connection')} - successLabel={(ms) => - t('datasources.settings.replica_connection_ok', { ms }) - } - /> -
diff --git a/frontend/src/pages/datasources/__tests__/DatasourceCreateWizardPage.test.tsx b/frontend/src/pages/datasources/__tests__/DatasourceCreateWizardPage.test.tsx index 5691d737..855558ee 100644 --- a/frontend/src/pages/datasources/__tests__/DatasourceCreateWizardPage.test.tsx +++ b/frontend/src/pages/datasources/__tests__/DatasourceCreateWizardPage.test.tsx @@ -123,8 +123,9 @@ const baseDatasource: Datasource = { custom_driver_id: null, connector_id: null, jdbc_url_override: null, - read_replica_jdbc_url: null, - read_replica_username: null, + read_replicas: [], + result_cache_enabled: false, + result_cache_ttl_seconds: null, local_datacenter: null, active: true, created_at: '2026-05-12T00:00:00Z', diff --git a/frontend/src/pages/datasources/__tests__/DatasourceSettingsPage.test.tsx b/frontend/src/pages/datasources/__tests__/DatasourceSettingsPage.test.tsx index 645b566b..675c74cc 100644 --- a/frontend/src/pages/datasources/__tests__/DatasourceSettingsPage.test.tsx +++ b/frontend/src/pages/datasources/__tests__/DatasourceSettingsPage.test.tsx @@ -100,11 +100,12 @@ const baseDs: Datasource = { custom_driver_id: null, connector_id: null, jdbc_url_override: null, - read_replica_jdbc_url: null, - read_replica_username: null, + read_replicas: [], local_datacenter: null, active: true, created_at: '2026-05-01T00:00:00Z', + result_cache_enabled: false, + result_cache_ttl_seconds: null, }; function wrap(node: ReactNode) { @@ -137,44 +138,48 @@ describe('DatasourceSettingsPage — read replica section', () => { listAllGroups.mockResolvedValue([]); }); - it('renders the read replica section with empty fields when datasource has no replica', async () => { + it('renders no endpoint rows and an add button when datasource has no replicas', async () => { getDatasource.mockResolvedValue(baseDs); render(wrap()); - await waitFor(() => expect(screen.getByLabelText('Replica JDBC URL')).toBeInTheDocument()); - expect((screen.getByLabelText('Replica JDBC URL') as HTMLInputElement).value).toBe(''); - expect((screen.getByLabelText('Replica username') as HTMLInputElement).value).toBe(''); - expect(screen.getByRole('button', { name: /Test replica/i })).toBeInTheDocument(); + await waitFor(() => + expect(screen.getByRole('button', { name: /Add replica endpoint/i })).toBeInTheDocument(), + ); + expect(screen.queryByLabelText('Replica JDBC URL')).not.toBeInTheDocument(); }); - it('pre-fills the form when datasource has a replica configured', async () => { + it('pre-fills one row per configured endpoint', async () => { getDatasource.mockResolvedValue({ ...baseDs, - read_replica_jdbc_url: 'jdbc:postgresql://replica:5432/app', - read_replica_username: 'replica-user', + read_replicas: [ + { id: 'rep-1', jdbc_url: 'jdbc:postgresql://replica-a:5432/app', username: 'ua' }, + { id: 'rep-2', jdbc_url: 'jdbc:postgresql://replica-b:5432/app', username: 'ub' }, + ], }); render(wrap()); await waitFor(() => - expect((screen.getByLabelText('Replica JDBC URL') as HTMLInputElement).value).toBe( - 'jdbc:postgresql://replica:5432/app', - ), - ); - expect((screen.getByLabelText('Replica username') as HTMLInputElement).value).toBe( - 'replica-user', + expect(screen.getAllByLabelText('Replica JDBC URL')).toHaveLength(2), ); + const urls = screen.getAllByLabelText('Replica JDBC URL') as HTMLInputElement[]; + expect(urls[0]!.value).toBe('jdbc:postgresql://replica-a:5432/app'); + expect(urls[1]!.value).toBe('jdbc:postgresql://replica-b:5432/app'); }); - it('calls testReplicaConnection with live form values when Test replica is clicked', async () => { + it('adds an endpoint row and tests it with live values', async () => { getDatasource.mockResolvedValue(baseDs); testReplicaConnection.mockResolvedValue({ ok: true, latency_ms: 8, message: null }); render(wrap()); - await waitFor(() => expect(screen.getByLabelText('Replica JDBC URL')).toBeInTheDocument()); + await waitFor(() => + expect(screen.getByRole('button', { name: /Add replica endpoint/i })).toBeInTheDocument(), + ); + fireEvent.click(screen.getByRole('button', { name: /Add replica endpoint/i })); + await waitFor(() => expect(screen.getByLabelText('Replica JDBC URL')).toBeInTheDocument()); fireEvent.change(screen.getByLabelText('Replica JDBC URL'), { target: { value: 'jdbc:postgresql://r:5432/db' }, }); @@ -192,15 +197,17 @@ describe('DatasourceSettingsPage — read replica section', () => { jdbc_url: 'jdbc:postgresql://r:5432/db', username: 'ru', password: 'rpw', + replica_id: undefined, }), ); }); - it('omits password when blank so the backend uses the persisted value', async () => { + it('falls back to the persisted password via replica_id when none is typed', async () => { getDatasource.mockResolvedValue({ ...baseDs, - read_replica_jdbc_url: 'jdbc:postgresql://replica:5432/app', - read_replica_username: 'replica-user', + read_replicas: [ + { id: 'rep-1', jdbc_url: 'jdbc:postgresql://replica:5432/app', username: 'replica-user' }, + ], }); testReplicaConnection.mockResolvedValue({ ok: true, latency_ms: 5, message: null }); @@ -215,55 +222,103 @@ describe('DatasourceSettingsPage — read replica section', () => { jdbc_url: 'jdbc:postgresql://replica:5432/app', username: 'replica-user', password: undefined, + replica_id: 'rep-1', }), ); }); - it('submits replica fields on save and strips blank password', async () => { - getDatasource.mockResolvedValue(baseDs); + it('submits the endpoint list on save, keeping the stored password when blank', async () => { + getDatasource.mockResolvedValue({ + ...baseDs, + read_replicas: [ + { id: 'rep-1', jdbc_url: 'jdbc:postgresql://replica:5432/app', username: 'ru' }, + ], + }); updateDatasource.mockResolvedValue(baseDs); render(wrap()); await waitFor(() => expect(screen.getByLabelText('Replica JDBC URL')).toBeInTheDocument()); - fireEvent.change(screen.getByLabelText('Replica JDBC URL'), { - target: { value: 'jdbc:postgresql://r:5432/db' }, - }); - fireEvent.change(screen.getByLabelText('Replica username'), { - target: { value: 'ru' }, + target: { value: 'jdbc:postgresql://replica-new:5432/app' }, }); - // Leave replica password blank. fireEvent.click(screen.getByRole('button', { name: /Save changes/i })); await waitFor(() => expect(updateDatasource).toHaveBeenCalled()); - const body = updateDatasource.mock.calls[0]![1] as Record; - expect(body.read_replica_jdbc_url).toBe('jdbc:postgresql://r:5432/db'); - expect(body.read_replica_username).toBe('ru'); - expect(body).not.toHaveProperty('read_replica_password'); + const body = updateDatasource.mock.calls[0]![1] as { + read_replicas: { id?: string; jdbc_url: string; username?: string; password?: string }[]; + }; + expect(body.read_replicas).toHaveLength(1); + expect(body.read_replicas[0]!.id).toBe('rep-1'); + expect(body.read_replicas[0]!.jdbc_url).toBe('jdbc:postgresql://replica-new:5432/app'); + expect(body.read_replicas[0]!.username).toBe('ru'); + expect(body.read_replicas[0]!.password).toBeUndefined(); }); - it('passes blank read_replica_jdbc_url through on save so the backend clear-on-blank rule fires', async () => { + it('submits an empty endpoint list after removing the last replica', async () => { getDatasource.mockResolvedValue({ ...baseDs, - read_replica_jdbc_url: 'jdbc:postgresql://replica:5432/app', - read_replica_username: 'ru', + read_replicas: [ + { id: 'rep-1', jdbc_url: 'jdbc:postgresql://replica:5432/app', username: 'ru' }, + ], }); updateDatasource.mockResolvedValue(baseDs); render(wrap()); await waitFor(() => expect(screen.getByLabelText('Replica JDBC URL')).toBeInTheDocument()); + fireEvent.click(screen.getByRole('button', { name: /Remove/i })); + await waitFor(() => + expect(screen.queryByLabelText('Replica JDBC URL')).not.toBeInTheDocument(), + ); - fireEvent.change(screen.getByLabelText('Replica JDBC URL'), { - target: { value: '' }, + fireEvent.click(screen.getByRole('button', { name: /Save changes/i })); + + await waitFor(() => expect(updateDatasource).toHaveBeenCalled()); + const body = updateDatasource.mock.calls[0]![1] as { read_replicas: unknown[] }; + expect(body.read_replicas).toEqual([]); + }); +}); + +describe('DatasourceSettingsPage — performance card', () => { + beforeEach(() => { + getDatasource.mockReset(); + updateDatasource.mockReset(); + listPermissions.mockReset(); + listPermissions.mockResolvedValue([]); + listGroupPermissions.mockReset(); + listGroupPermissions.mockResolvedValue([]); + listAllGroups.mockReset(); + listAllGroups.mockResolvedValue([]); + }); + + it('submits cache opt-in and TTL on save', async () => { + getDatasource.mockResolvedValue(baseDs); + updateDatasource.mockResolvedValue(baseDs); + + render(wrap()); + + await waitFor(() => + expect(screen.getByLabelText('Cache SELECT results')).toBeInTheDocument(), + ); + // TTL is disabled until the cache is enabled. + expect(screen.getByLabelText('Cache TTL (seconds)')).toBeDisabled(); + + fireEvent.click(screen.getByLabelText('Cache SELECT results')); + await waitFor(() => + expect(screen.getByLabelText('Cache TTL (seconds)')).not.toBeDisabled(), + ); + fireEvent.change(screen.getByLabelText('Cache TTL (seconds)'), { + target: { value: '120' }, }); + fireEvent.click(screen.getByRole('button', { name: /Save changes/i })); await waitFor(() => expect(updateDatasource).toHaveBeenCalled()); const body = updateDatasource.mock.calls[0]![1] as Record; - expect(body.read_replica_jdbc_url).toBe(''); + expect(body.result_cache_enabled).toBe(true); + expect(body.result_cache_ttl_seconds).toBe(120); }); }); diff --git a/frontend/src/pages/editor/QueryEditorPage.test.tsx b/frontend/src/pages/editor/QueryEditorPage.test.tsx index 8dc5a65f..e9d651cd 100644 --- a/frontend/src/pages/editor/QueryEditorPage.test.tsx +++ b/frontend/src/pages/editor/QueryEditorPage.test.tsx @@ -91,8 +91,9 @@ const baseDatasource: Datasource = { custom_driver_id: null, connector_id: null, jdbc_url_override: null, - read_replica_jdbc_url: null, - read_replica_username: null, + read_replicas: [], + result_cache_enabled: false, + result_cache_ttl_seconds: null, local_datacenter: null, active: true, created_at: '2026-05-01T00:00:00Z', diff --git a/frontend/src/types/api.ts b/frontend/src/types/api.ts index e5a883eb..5431acfc 100644 --- a/frontend/src/types/api.ts +++ b/frontend/src/types/api.ts @@ -585,11 +585,29 @@ export interface Datasource { custom_driver_id: string | null; connector_id: string | null; jdbc_url_override: string | null; - read_replica_jdbc_url: string | null; - read_replica_username: string | null; + read_replicas: ReadReplica[]; local_datacenter: string | null; active: boolean; created_at: string; + result_cache_enabled: boolean; + result_cache_ttl_seconds: number | null; +} + +// One read-replica endpoint of a datasource (AF-457); the password never round-trips. +export interface ReadReplica { + id: string; + jdbc_url: string; + username: string | null; +} + +// Replica endpoint in a create/update payload. `id` targets an existing endpoint on update +// (absent = create new). Password semantics on update: undefined keeps the stored secret, +// empty string clears it (primary-credential fallback), non-blank re-encrypts. +export interface ReadReplicaInput { + id?: string | null; + jdbc_url: string; + username?: string | null; + password?: string | null; } export interface ConnectionTestResult { @@ -618,11 +636,11 @@ export interface CreateDatasourceInput { custom_driver_id?: string | null; connector_id?: string | null; jdbc_url_override?: string | null; - read_replica_jdbc_url?: string | null; - read_replica_username?: string | null; - read_replica_password?: string | null; + read_replicas?: ReadReplicaInput[] | null; local_datacenter?: string | null; api_key?: string | null; + result_cache_enabled?: boolean; + result_cache_ttl_seconds?: number | null; } export interface UpdateDatasourceInput { @@ -643,12 +661,14 @@ export interface UpdateDatasourceInput { text_to_sql_enabled?: boolean; clear_ai_config?: boolean; jdbc_url_override?: string | null; - read_replica_jdbc_url?: string | null; - read_replica_username?: string | null; - read_replica_password?: string | null; + // Full-list replacement merged by endpoint id; undefined keeps the current endpoints, + // an empty array deletes them all (AF-457). + read_replicas?: ReadReplicaInput[] | null; local_datacenter?: string | null; api_key?: string | null; active?: boolean; + result_cache_enabled?: boolean; + result_cache_ttl_seconds?: number | null; } export interface CreatePermissionInput { @@ -1953,6 +1973,16 @@ export interface DatasourceHealth { execution_ms_p50: number | null; execution_ms_p95: number | null; errors_last_24h: number; + // Per-endpoint read-replica health on this node (AF-457); empty when no replicas configured. + replicas: ReplicaHealth[]; +} + +export interface ReplicaHealth { + endpoint_id: string; + label: string; + healthy: boolean; + pool_active: number | null; + pool_total: number | null; } export type DatasourceHealthPage = PageEnvelope; diff --git a/terraform-provider/internal/client/datasource.go b/terraform-provider/internal/client/datasource.go index 332bf6de..77d6b81f 100644 --- a/terraform-provider/internal/client/datasource.go +++ b/terraform-provider/internal/client/datasource.go @@ -3,7 +3,7 @@ package client import "context" // Datasource is the API response shape for a datasource. Write-only fields (password, -// read_replica_password, api_key) are never returned and are absent here. +// each read_replicas[].password, api_key) are never returned and are absent here. type Datasource struct { ID string `json:"id"` OrganizationID string `json:"organization_id"` diff --git a/website/docs/index.html b/website/docs/index.html index a2ebbe80..f1f16ce4 100644 --- a/website/docs/index.html +++ b/website/docs/index.html @@ -1057,21 +1057,39 @@

Datasources

  • Configuration. Pick the Review plan that gates this datasource, toggle Require review on reads / writes, and (optionally) enable AI analysis and/or text-to-query + pick an AI configuration. The AI configuration is shared by both features, so it is required whenever either toggle is on. With text-to-query on, users can draft a query from a natural-language prompt in the editor — in the engine's native query language (SQL or a NoSQL query) — and the draft still flows through the normal review pipeline. Pool size, max rows, and statement timeout default sensibly but can be tightened per datasource.

    • - Read-replica routing (optional). On the datasource settings page, - the Read replica card accepts an optional JDBC URL plus username and password. - When set, AccessFlow opens a separate connection pool and routes every query classified as - SELECT to the replica; INSERT / UPDATE / DELETE / DDL - and transactional BEGIN … COMMIT batches always hit the primary. The - replica must use the same database engine as the primary (it reuses the primary's - JDBC driver). Replica credentials are AES-256-GCM encrypted with the same - ENCRYPTION_KEY as the primary. Click Test replica to validate - the URL + credentials live without persisting; leaving the password field blank reuses - the saved replica password. If the replica is unreachable at query time, AccessFlow - falls back to the primary so the query still runs and writes a - DATASOURCE_REPLICA_FALLBACK audit row so the failure is visible at - /admin/audit-log. Clear the JDBC URL field to disable replica routing. - Replica pools reuse the same ACCESSFLOW_PROXY_* connection-pool tuning as the - primary — no separate environment variables. + Read replicas & load balancing (optional). On the datasource + settings page, the Read replicas card takes any number of replica endpoints + (JDBC URL plus optional username and password per endpoint — blank credentials reuse + the primary's). AccessFlow opens one connection pool per endpoint and load-balances + every query classified as SELECT round-robin across the healthy replicas; + INSERT / UPDATE / DELETE / DDL and transactional BEGIN … COMMIT batches + always hit the primary. Replicas must use the same database engine as the primary + (they reuse the primary's JDBC driver), and credentials are AES-256-GCM encrypted with + the same ENCRYPTION_KEY. Per-node health checks (a background prober plus + a circuit breaker) take a failed endpoint out of rotation for a cooldown + (ACCESSFLOW_PROXY_REPLICA_COOLDOWN, default 30s) and its health shows on + the Datasource health dashboard; only when every replica is down does the + read fall back to the primary, with one DATASOURCE_REPLICA_FALLBACK audit + row visible at /admin/audit-log. Click Test replica on any row + to validate its URL + credentials live without persisting; leaving the password blank + reuses that endpoint's saved password. Remove every endpoint to disable replica + routing. Replica pools reuse the same ACCESSFLOW_PROXY_* connection-pool + tuning as the primary; the health checks are tuned by the + ACCESSFLOW_PROXY_REPLICA_* variables. +

    +

    + SELECT result caching (optional). The settings page's + Performance card opts a datasource into a Redis-backed result cache for + repeated identical SELECTs, with a per-datasource TTL (1–86,400 seconds; + blank uses ACCESSFLOW_PROXY_CACHE_DEFAULT_TTL, default 60s). Caching is + security-safe by construction — entries are keyed over the row-security-rewritten + query and the caller's masking scope, so masking and row-level security always apply — + and any write executed through AccessFlow to a referenced table (including GDPR + erasure and retention deletes) immediately invalidates the affected entries. Note that + writes made outside AccessFlow are invisible to the cache and are served + stale until the TTL expires, so pick a TTL that matches how the datasource is written. + ACCESSFLOW_PROXY_CACHE_ENABLED=false switches the feature off + deployment-wide.

    Grant a user access. Open the datasource → Permissions tab and add a row per user — can read / can write / can DDL, allowed schemas, allowed tables, and restricted columns (masked as *** in SELECT results). Without a permission row, a user can't see or query the datasource at all. The allowed schemas / allowed tables lists are enforced when a query is submitted: every table it references — across joins, subqueries, CTEs, and BEGIN; …; COMMIT; batches — must appear in allowed tables or live in an allowed schema, or the query is rejected before it runs. Matching is case-insensitive, and an unqualified table name (FROM users) only matches an unqualified entry in allowed tables. Leave both fields empty to allow every table. @@ -1178,7 +1196,16 @@

    Data classification

  • Statement execution (all engines): ACCESSFLOW_PROXY_EXECUTION_MAX_ROWS (10000), ACCESSFLOW_PROXY_EXECUTION_STATEMENT_TIMEOUT (30s), - ACCESSFLOW_PROXY_EXECUTION_DEFAULT_FETCH_SIZE (1000).
    • + ACCESSFLOW_PROXY_EXECUTION_DEFAULT_FETCH_SIZE (1000), + ACCESSFLOW_PROXY_EXECUTION_INSERT_BATCH_CHUNK_SIZE (1000). +
  • SELECT result cache: + ACCESSFLOW_PROXY_CACHE_ENABLED (true), + ACCESSFLOW_PROXY_CACHE_DEFAULT_TTL (PT60S), + ACCESSFLOW_PROXY_CACHE_MAX_ENTRY_BYTES (1000000).
    • +
  • Read-replica health checks: + ACCESSFLOW_PROXY_REPLICA_PROBE_INTERVAL (PT30S), + ACCESSFLOW_PROXY_REPLICA_PROBE_TIMEOUT (PT5S), + ACCESSFLOW_PROXY_REPLICA_COOLDOWN (PT30S).
  • MongoDB: ACCESSFLOW_PROXY_MONGO_CONNECT_TIMEOUT (PT10S), …_SERVER_SELECTION_TIMEOUT (PT10S), …_MAX_POOL_SIZE (10).
    • diff --git a/website/index.html b/website/index.html index a96b06ba..fab99e92 100644 --- a/website/index.html +++ b/website/index.html @@ -329,7 +329,7 @@

    The middle ground between blanket access and a DBA ticket queue.

    Full query proxy — SQL and NoSQL

    Every query is inspected before it ever reaches the database — parsed, sorted by type (read, write, or schema change), checked against the tables a user is allowed to touch, and routed through your review policy. The same governance covers NoSQL too: MongoDB, Couchbase, Redis, Cassandra, Elasticsearch, DynamoDB, and Neo4j all flow through identical checks, with dangerous or server-side commands blocked up front. Layer on row-level security so users only see the rows they're cleared for, dynamic masking that hides sensitive columns per person, and data-classification tags (PII, PCI, PHI, GDPR) that auto-apply masking and raise a query's risk score. A dry-run preview shows a query's impact before review without changing any data, and every executed query is saved as an exact snapshot you can replay in a test environment.

    - PostgreSQL · MySQL · MariaDB · Oracle · MSSQL · ClickHouse · MongoDB · Couchbase · Redis · Cassandra · ScyllaDB · Elasticsearch · OpenSearch · DynamoDB · Neo4j · connector catalog · custom drivers · read-replica routing · dynamic masking · row-level security · data classification + PostgreSQL · MySQL · MariaDB · Oracle · MSSQL · ClickHouse · MongoDB · Couchbase · Redis · Cassandra · ScyllaDB · Elasticsearch · OpenSearch · DynamoDB · Neo4j · connector catalog · custom drivers · read-replica load balancing · result caching · dynamic masking · row-level security · data classification
    @@ -1264,6 +1264,7 @@

    Planned

    Connectors & access
    • Native wire-protocol gateway
      • +
    • Result caching · multi-replica load balancing
    • Column-level permissions