feat: remap query virtual columns to clustered base table columns when equivalent

[clintropolis]

Description

This PR is a minor optimization to clustered segments to allow query virtual columns to be substituted with physical columns if the clustered segment was created with an 'equivalent' column (clustered segments preserve any virtual columns used during their creation in their metadata). For example, if the function lower(some_column) is used to create the column, after this patch running the same expression at query time will be swapped out to read the value from the column directly instead of recomputing the lower operation.

[FrankChen021]

Severity	Findings
P0	0
P1	0
P2	2
P3	0
Total	2

---

This is an automated review by Codex GPT-5.5

[FrankChen021]

[P2] Seed residual filter rewrites with identity mappings

When a remap exists, this rewrites the entire residual filter using only virtualColumnRemap. Filters such as EqualityFilter and RangeFilter require the rewrite map to contain their own required column and throw if it is absent. A query with v0 remapped plus an unrelated residual filter like region = 'us-east-1' will fold/remap v0, leave region as a residual filter, then pass only {v0 -> tenant_lower} here and fail during cursor construction. Build an identity map for rewritten.getRequiredColumns() before overlaying the remap.

[FrankChen021]

[P2] Only remap to readable clustered columns

findEquivalent searches all group virtual-column metadata, but not every group virtual-column output is a stored/readable column. For example, clustered base-table query-granularity carriers live in virtualColumns and are explicitly not stored in columns. If a query VC with a different name matches such a metadata-only VC, this records a remap, rebuildCursorBuildSpec drops the query VC, and the selector reads the missing target column, producing null or incorrect results. Gate candidates to clustering columns or names present in the clustered summary's stored columns.

[FrankChen021]

Severity	Findings
P0	0
P1	0
P2	2
P3	0
Total	2

Severity	Findings
P0	0
P1	0
P2	2
P3	0
Total	2

Reviewed 11 of 11 changed files.

Found 2 issues: residual filter rewriting can fail for unchanged columns, and remapping can target metadata-only virtual columns that group cursors cannot read.

---

This is an automated review by Codex GPT-5.5

[FrankChen021]

[P2] Seed filter rewrites with identity entries

When virtualColumnRemap is non-empty, rebuildCursorBuildSpec calls rewriteRequiredColumns with only the remapped VC entries. Existing filter implementations throw if their own required column is missing from the map, so a query that remaps one VC but still has an unchanged residual predicate, such as a materialized VC filter plus region = 'us-east-1', fails during cursor construction instead of evaluating the residual column. Build an identity map for the rewritten filter's required columns and overlay the remap, as the aggregate projection path does.

[FrankChen021]

[P2] Only remap to columns the group cursor can read

buildClusterVirtualColumnRemap treats any groupVcs.findEquivalent result as a readable materialized column, but clustered summaries also carry metadata-only virtual columns. For example, the query-granularity carrier __virtualGranularity is explicitly not a stored column; an equivalent query VC with another name is now dropped and remapped to __virtualGranularity, which the clustering/per-group selector cannot serve. That returns null/empty results instead of recomputing or reading __time. Restrict remap targets to clustering columns or stored group columns, or special-case the granularity carrier.

[capistrant]

naming seems backwards. aren't these actually whare are not being pruned out of the query?

[capistrant]

can we move to top or bottom of file to avoid interleaving with tests

[capistrant]

clustering-VC leaves were already folded to TRUE / FALSE by the per-group rewrite walk, so they won't reference the dropped virtual columns

This is a not entirely true. walkClusterGroupFilter only folds Equality, In, and Null. Everything else falls through unchanged. This will result in wrong results when something like a Range filter on a query vc with equivalent clustering column survives the walk.

Repro wrong empty results:

• tenant_lower := lower(tenant) clustered vc
• v0 := lower(tenant) ⇒ remap v0 → tenant_lower
• same test tenants as always
• RangeFilter("v0", ["a","z"])
• Both groups survive and you expect all rows but get empty result back

[FrankChen021]

Severity	Findings
P0	0
P1	1
P2	2
P3	0
Total	3

Reviewed 11 of 11 changed files.

Found three remapping correctness and integration issues: shadowing query virtual columns can replace stored targets, materialized remap targets are omitted from physicalColumns, and unsupported residual filters can be rewritten unconditionally.

---

This is an automated review by Codex GPT-5.6-Sol

[FrankChen021]

[P1] Avoid remapping through a shadowing query virtual column

The candidate target is checked only against stored columns, but query virtual columns may shadow physical columns. If the query also retains a VC named like the materialized target, the outer remap routes through the per-group virtualized delegate and reads that unrelated VC instead of the stored column. For example, with stored region_upper = upper(region), query VCs v1 = upper(region) and region_upper = lower(region) cause v1 to silently return lowercase values. Exclude targets shadowed by retained query VCs or bypass the virtual layer when reading remapped physical columns.

[FrankChen021]

[P2] Update physicalColumns for materialized remap targets

The rebuilt spec drops remapped VCs but copies the original non-null physicalColumns, which contains the VCs' raw inputs rather than their materialized targets. Native engines populate this set. On partial segments, prefetch therefore omits targets such as region_upper, and the later remapped selector triggers synchronous deep-storage I/O during cursor use; incremental selector capability snapshots can likewise reject the undeclared target. When physicalColumns is non-null, add all remap target columns to the rebuilt set.

[FrankChen021]

[P2] Do not unconditionally rewrite unsupported filters

Whenever any VC remap exists, this calls rewriteRequiredColumns on the residual filter even if none of its columns need remapping. Core filters including SpatialFilter, DimensionPredicateFilter, ColumnComparisonFilter, and JavaScriptFilter do not support required-column rewriting, so a query that groups or selects a remappable VC alongside one of these filters now throws UnsupportedOperationException. Preserve the filter when its required columns do not intersect the remap; if they do intersect and rewriting is unsupported, disable that substitution.

[FrankChen021]

Severity	Findings
P0	0
P1	1
P2	0
P3	0
Total	1

Severity	Findings
P0	0
P1	1
P2	0
P3	0
Total	1

Reviewed 13 of 13 changed files.

The update addresses the prior physical-column omission and guards the earlier shadowing and filter-rewrite cases, but their fallback to VC recomputation can still read omitted raw inputs and silently filter valid rows.

---

This is an automated review by Codex GPT-5.6-Sol

[FrankChen021]

[P1] Do not recompute retained VCs from omitted source columns

This exclusion assumes leaving the VC in the spec makes it safely recomputable, but clustered schemas may materialize a VC while omitting its raw inputs; this PR's tenant_lower example stores lower(tenant) without storing tenant. With a non-rewritable filter on v0 := lower(tenant), v0 is removed from the remap here, so the per-group filter evaluates it from the missing tenant column instead of tenant_lower and can silently discard valid rows. The same unsafe fallback is used when retained VCs depend on a materialized candidate. Remapped names need to be available inside the per-group filter and VC layer without requiring rewriteRequiredColumns, or recomputation must only be chosen when every source input is actually available.