fix(db): preserve lexical source identity during optimization

.changeset/preserve-optimizer-source-identity.md

@@ -0,0 +1,5 @@
+---
+'@tanstack/db': patch
+---
+
+Preserve source identity during query optimization so a joined subquery and sibling include can reuse an alias without hiding parent rows.

docs/contributing/oracle-coverage.md

@@ -238,6 +238,7 @@ comment and the current API/architecture contract before extending its model.
 | --- | --- | --- |
 | Ordered relations and BTree | [top-K relation oracle](https://github.com/TanStack/db/blob/main/packages/db-ivm/tests/operators/topk-relation-oracle.test.ts), [fractional-index window moves](https://github.com/TanStack/db/blob/main/packages/db-ivm/tests/operators/topKWithFractionalIndex.test.ts), [BTree/Map](https://github.com/TanStack/db/blob/main/packages/db/tests/btree-map-oracle.test.ts), [incrementalization laws](https://github.com/TanStack/db/blob/main/packages/db-ivm/tests/incrementalization-law.property.test.ts) | Independent ordered relations and cumulative signed output. Top-K consolidation compares same-key values without hashing, including cyclic replacements and fresh transient cancellation. The fractional-index test checks an empty output message for a window move after all rows retract; it does not prove downstream lazy acquisition. Other hash-based operators retain hashing's declared domain. Algebra does not specify client readiness. The BTree/Map oracle checks point operations, neighbor pairs, and full, partial, boundary, exclusive-high, and empty range scans for integer keys and node sizes 4–8. It does not check balance or space, and it assumes consistent comparators ([code-weight BTree review](oracle-reviews/code-weight-btree-trim.md)). |
 | Includes and publication | [cross-formulation](https://github.com/TanStack/db/blob/main/packages/db/tests/query/includes-cross-formulation-oracle.property.test.ts), [temporal](https://github.com/TanStack/db/blob/main/packages/db/tests/query/includes-temporal-oracle.test.ts), [Collection includes](https://github.com/TanStack/db/blob/main/packages/db/tests/query/includes-collection-oracle.property.test.ts), [architecture and complete suite map](https://github.com/TanStack/db/blob/main/packages/db/src/query/live/ARCHITECTURE.md#executable-contracts) | Per-parent/flat-join/partition relations, callback-time rows, nested values, and route histories. Fragmented lazy-demand consolidation covers delta growth and successful, rejected/retried, and obsolete replacements at the Collection request boundary; it observes exact key unions and abort checkpoints. A compiled-includes fixture makes adapter unload evict owned child rows and proves a rejected union cannot remove established public child rows. Retry after that terminal live-query failure remains covered at the acquisition boundary; no automatic in-place recovery is claimed. Observe raw promised order; fresh queries do not establish continuous publication safety. |
+| Includes query shape and lexical source identity | `packages/db/tests/query/includes-query-shape-oracle.test.ts` | A full-recomputation model compares public parent and included rows after preload and each source update. Its bounded grammar crosses reused or distinct sibling aliases, separate or combined parent and joined predicates, a joined-only predicate, and joined or included issue updates. Eleven direct optimizer shapes assert that optimization retains each lexical source ID, alias, and collection without changing input IR; placement clones must create distinct source IDs. The original optimizer fails the public reused-alias initial checkpoint, and each of its four source-copy sites fails a direct identity witness when restored separately. The public grammar has one matching joined row per parent; the owner's older model covers join multiplicity separately. Arbitrary nested include graphs, unrelated predicate forms, asynchronous demand schedules, user mutation of public IR objects after construction, and direct compiler input maps remain outside this witness. |
 | Collection lifecycle | [mutation startup](https://github.com/TanStack/db/blob/main/packages/db/tests/collection-mutation-startup-oracle.test.ts), [change-event history](https://github.com/TanStack/db/blob/main/packages/db/tests/change-event-history-oracle.test.ts), [history](https://github.com/TanStack/db/blob/main/packages/db/tests/collection-subscription-lifecycle-history.property.test.ts), [publication](https://github.com/TanStack/db/blob/main/packages/db/tests/collection-subscription-lifecycle-publication.property.test.ts), [replay](https://github.com/TanStack/db/blob/main/packages/db/tests/collection-subscription-replay-oracle.property.test.ts), [effect disposal](https://github.com/TanStack/db/blob/main/packages/db/tests/effect-disposal-oracle.test.ts), [PR #1902 review](oracle-reviews/pr-1902-change-event-history.md), [batch-order decision review](oracle-reviews/change-event-batch-order.md) | Core Collection `insert`/`update`/`delete` admission while `startSync:false` is idle; complete public-row/change-message agreement across bounded one- and two-key histories plus replayable four-key campaigns; a two-key deferred sync history checks each key's causal insert/update/delete trace while allowing any cross-key interleaving. Reversing all deferred messages fails that check; reversing each sync transaction's distinct-key messages passes. Queued duplicate admission and cancellation, ownership and phase histories, exact caller/error/publication evidence, and late completion and restart have separate witnesses. Generated deferred histories with cancellation or reentrant callbacks need a witness in the change-event history owner before claiming general batch-shape coverage. Query write utilities and effect self-dependent disposal remain separate contracts. |
 | Paced mutations | [virtual-clock oracle](https://github.com/TanStack/db/blob/main/packages/db/tests/paced-mutations-oracle.test.ts) | `createPacedMutations` with queue front/back options, bounded waiting capacity at zero and one, cleanup draining in FIFO/LIFO order after a held write, cleanup timing at the configured wait for immediate writes, admitted writes after separate Collection cleanup, debounce and throttle schedules with explicit and omitted options, and first-leading throttle execution at epoch zero. Finite histories compare optimistic rows, returned transaction identity and receipt outcome, execution order and virtual-clock time. Held writes verify queue serialization. Leading-only throttle and debounce witnesses reject skipped calls immediately with their named dropped-call errors, including omitted edges, both edges disabled, and same-row rollback while a prior write is held. Capacity witnesses cover overflow rejection, distinct-key optimistic rollback, same-key admitted-write survival, and in-flight versus waiting admission. Cleanup witnesses check repeated queue cleanup, eventual admitted receipt settlement, direct queue strategy rejection of new callbacks, public post-cleanup rollback with `QueueDisposedError`, pending debounce transactions that wait until the last call's quiet edge after separate Collection cleanup, and a trailing throttle timer that runs at its regular edge after separate Collection cleanup. Deferred custom queue and batch callbacks check compatibility with `void` and `false` execute results, respectively. Frozen options verify factory non-mutation; a mutable option witness checks that debounce uses its construction-time trailing setting. New debounce/throttle admission after cleanup, failed persistence, broader capacities and schedules remain outside this owner. |
 | Optimistic state | [history model](https://github.com/TanStack/db/blob/main/packages/db/tests/optimistic-history-oracle.ts), [generated histories](https://github.com/TanStack/db/blob/main/packages/db/tests/optimistic-transaction-oracle.property.test.ts), [truncate capture ownership](https://github.com/TanStack/db/blob/main/packages/db/tests/collection-truncate-ownership-oracle.property.test.ts), [outcomes](https://github.com/TanStack/db/blob/main/packages/db/tests/optimistic-history-outcomes.test.ts), [publication](https://github.com/TanStack/db/blob/main/packages/db/tests/optimistic-history-publication.test.ts) | Independent whole-row snapshots, rollback dependencies, captured truncate ownership, metadata and prior-value events. Never rebase a pending snapshot merely to simplify the model. |

packages/db/src/query/optimizer.ts

@@ -124,7 +124,6 @@ import { deepEquals } from '../utils.js'
 import { CannotCombineEmptyExpressionListError } from '../errors.js'
 import { containsAggregate } from './compiler/group-by.js'
 import {
-  CollectionRef as CollectionRefClass,
   Func,
   PropRef,
   QueryRef as QueryRefClass,
@@ -135,7 +134,14 @@ import {
   getWhereExpression,
   isResidualWhere,
 } from './ir.js'
-import type { BasicExpression, From, QueryIR, Select, Where } from './ir.js'
+import type {
+  BasicExpression,
+  CollectionRef as CollectionRefClass,
+  From,
+  QueryIR,
+  Select,
+  Where,
+} from './ir.js'
 
 /**
  * Represents a WHERE clause after source analysis
@@ -497,7 +503,7 @@ function removeRedundantFromClause(from: From): From {
     // Return the inner query's FROM clause with this alias
     const innerFrom = removeRedundantFromClause(processedQuery.from)
     if (innerFrom.type === `collectionRef`) {
-      return new CollectionRefClass(innerFrom.collection, from.alias)
+      return innerFrom
     } else if (innerFrom.type === `queryRef`) {
       return new QueryRefClass(innerFrom.query, from.alias)
     }
@@ -814,13 +820,10 @@ function applyOptimizations(
 }
 
 /**
- * Helper function to create a deep copy of a QueryIR object for immutability.
- *
- * This ensures that all optimizations create new objects rather than modifying
- * existing ones, preventing infinite recursion and shared reference issues.
+ * Copy query containers and arrays while retaining lexical source references.
  *
  * @param query - QueryIR to deep copy
- * @returns New QueryIR object with all nested objects copied
+ * @returns New QueryIR containers with the same lexical collection sources
  */
 function deepCopyQuery(query: QueryIR): QueryIR {
   return {
@@ -845,7 +848,7 @@ function deepCopyQuery(query: QueryIR): QueryIR {
 
 function deepCopyFrom(from: From): From {
   if (from.type === `collectionRef`) {
-    return new CollectionRefClass(from.collection, from.alias)
+    return from
   }
 
   if (from.type === `queryRef`) {
@@ -930,9 +933,8 @@ function optimizeFromWithTracking(
   const whereClause = singleSourceClauses.get(from.alias)
 
   if (!whereClause) {
-    // No optimization needed, but return a copy to maintain immutability
     if (from.type === `collectionRef`) {
-      return new CollectionRefClass(from.collection, from.alias)
+      return from
     }
     // Must be queryRef due to type system
     return new QueryRefClass(deepCopyQuery(from.query), from.alias)
@@ -942,7 +944,7 @@ function optimizeFromWithTracking(
     // Create a new subquery with the WHERE clause for the collection
     // This is always safe since we're creating a new subquery
     const subQuery: QueryIR = {
-      from: new CollectionRefClass(from.collection, from.alias),
+      from,
       where: [whereClause],
     }
     actuallyOptimized.add(from.alias) // Mark as successfully optimized

packages/db/tests/oracle-config.ts

@@ -90,6 +90,7 @@ const staticOracleProperties = [
   `includes-optimistic.repeated-history`,
   `includes-optimistic.sibling-route-rollback`,
   `includes-query-shape.correlation`,
+  `includes-query-shape.identity`,
   `includes-query-shape.multiplicity`,
   `includes-query-shape.nullable`,
   `includes-work.correlated-links`,