Fix three Elasticsearch index-list bugs, and build the dated index names through one policy

[wu-sheng]

Fix three Elasticsearch index-list bugs, and build the dated index names through one policy

• Add a unit test to verify that the fix works.
• Explain briefly why the bug exists and how to fix it.

Follows #14139. That PR fixed five metric reads that picked the wrong day step. The root cause was that every caller chose the day step itself, through TimeSeriesUtils or TimeRangeIndexNameGenerator. This PR decides the step once per physical table and builds every dated index name in one place, so a caller can no longer pick the wrong one. Alias and management index names still come from IndexController, as before.

The policy

• IndexRolling (NONE, DAY_STEP, SUPER_DATASET_DAY_STEP) is decided once from the model.
• IndexController.LogicIndicesRegister.registerRelation records it per physical table, and rejects a table registered with two kinds. The catalog maps become concurrent, since runtime rules register models after boot.
• IndexFamily is a physical table plus its rolling, resolved from a model or a logical name. An unregistered name rolls by dayStep: only metric reads pass names that may be unregistered.
• IndexNames builds every dated name: writeIndex, pointIndex, rangeIndices, rangeIndicesBetweenTimestamps, latestIndex, indexStartDay. TimeSeriesUtils and TimeRangeIndexNameGenerator are removed.
• ElasticSearchClient.search takes an index array instead of a Supplier. The new search(String[], Search, SearchParams) keeps routing and scroll, and the scroller can take an array.

Apart from the three fixes below, index names are unchanged. Super dataset reads now take their step from the registered table, which every OAP registers at boot.

Bugs fixed on the way

• Zipkin trace search listed the daily indices of every day since 1970 when the query started at or before the epoch (a lookback equal to endTs). With the default day step of 1 that is over 20,000 names, which can exceed the request line Elasticsearch accepts. It now searches the zipkin_span family in that case, as its time filter already did.
• Registration without a schema change. Only isExists registered a model's table, and a node applying a model without a schema change skips it. One example is a peer reinstalling a bundled MAL rule after the runtime rule overriding it is deleted. With logicSharding on and the two rules using different functions, the metric's table changes, so reads kept the runtime rule's table. StorageEsInstaller now registers in populateLocalCacheOnly. I traced this path in code; I did not reproduce it on a cluster.
• OTLP trace by id (unreleased, from Store OTLP traces natively and bring the TraceQL module closer to the Tempo API #14096) joined the day indices into one comma-separated name. The namespace, sw by default, went on the first name only, and a missing day failed the scroll with index_not_found_exception. The day indices now go as an array, each namespaced, and missing days are skipped. No changelog line, since the feature hasn't been released.

Verification

• EsReadIndexSelectionTest was written first, against the old code. It pins the indices the migrated reads send. In the cases that list dated indices, one step rolls by 3 and the other by 1, so the names show which step the read used; the rest pin when a read falls back to the whole index family. The top N read is pinned by AggregationQueryEsDAOTest. The same assertions pass after the refactor.
• A temporary comparison of IndexNames with TimeSeriesUtils matched all 653,400 names it checked: steps 1 to 11 for both kinds, dates from 2000 to 2031, every downsampling, ranges up to 40 days either way, and latest indices. It is not committed, because the old classes are gone.
• ElasticSearchIT#scrollOverSeveralIndices runs on ES 6.3.2 and 7.8.0, with and without a namespace. All 4 runs pass. The old comma-joined form fails all 4 with index_not_found_exception.
• Unit tests of storage-elasticsearch-plugin and library-client, checkstyle, javadoc:javadoc and license-eye header check pass.
• E2E locally, on an image of master cf1c2be22c with this PR's library-client and storage-elasticsearch-plugin jars: Storage ES 8.18.8 (76 passed), Storage ES Sharding 8.18.8 (76 passed, which boots every model with the two-kinds rolling check), and Zipkin ES (7 passed).

Test changes

• TimeSeriesUtilsTest and TimeSeriesUtilsTestCase are ported to IndexNamesTest. Their queryIndexName(..., Step.DAY, true, true) case, which no production code reached, became a pointIndex on a super dataset table at second precision.
• AggregationQueryEsDAOTest captures an array instead of a Supplier.

Left for later

• TTL deletes a whole index once its first day passes the deadline. When dayStep or superDatasetDayStep is greater than 1, that is up to step − 1 days early; the default step of 1 is not affected. The fix is to compare the index's last day. It changes behavior, so it gets its own PR.

• Reads that receive a duration but search every index of a table (topology, records, alarm, event) could narrow to the range, each with its own rule.

• If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #.

• Update the CHANGES log.

[Copilot]

🟡 Changes recommended

Disabled record streams can resolve to the metrics rolling policy and silently miss existing trace data.

1 open finding

• Unregistered record queries use incorrect rolling index and lose traces · New

What changed in this PR

Centralizes Elasticsearch dated-index selection, extending the earlier metrics fix to a shared policy for reads, writes, and index management.

Changes:

• Introduces physical-table rolling policies and replaces the old naming utilities.
• Fixes Zipkin epoch-range searches, schema-free registration, and multi-index OTLP scrolling.
• Adds regression coverage and migrates existing tests.

File	Description
EsReadIndexSelectionTest.java	Tests index selection across query paths.
AggregationQueryEsDAOTest.java	Captures index arrays instead of suppliers.
TimeSeriesUtilsTestCase.java	Removes tests migrated to IndexNamesTest.
TimeSeriesUtilsTest.java	Removes superseded naming tests.
StorageEsInstallerTest.java	Tests registration without schema changes.
IndexNamesTest.java	Tests centralized naming and rolling.
StorageModuleElasticsearchProvider.java	Configures the shared naming policy.
ZipkinQueryEsDAO.java	Uses the alias for epoch-reaching searches.
TraceQueryEsDAO.java	Uses shared trace-index selection.
TagAutoCompleteQueryDAO.java	Uses shared tag-index selection.
OTLPTraceQueryEsDAO.java	Passes dated indices separately to scrolling.
MetricsQueryEsDAO.java	Centralizes metric point and range selection.
LogQueryEsDAO.java	Uses shared log-index selection.
GenAIEvaluationRecordQueryEsDAO.java	Uses shared evaluation-index selection.
AIAgentConversationQueryEsDAO.java	Migrates conversation and file searches.
AggregationQueryEsDAO.java	Uses shared top-N index selection.
TimeSeriesUtils.java	Removes the old naming utility.
TimeRangeIndexNameGenerator.java	Removes the supplier-based range generator.
StorageEsInstaller.java	Registers schema-free models and uses centralized naming.
RecordEsDAO.java	Migrates record write-index naming.
NoneStreamEsDAO.java	Migrates non-stream write-index naming.
MetricsEsDAO.java	Migrates metric storage-index naming.
IndexRolling.java	Defines model-based rolling policies.
IndexNames.java	Centralizes dated-index construction.
IndexFamily.java	Resolves physical tables and rolling policies.
IndexController.java	Adds concurrent catalogs and rolling validation.
HistoryDeleteEsDAO.java	Uses shared naming and date extraction.
EsDAO.java	Accepts index arrays for debug-enabled searches.
ElasticSearchIT.java	Tests scrolling across existing and missing indices.
ElasticSearchScroller.java	Supports index arrays.
ElasticSearchClient.java	Namespaces each index and preserves scroll parameters.
docs/​en/​changes/​changes.md	Documents fixes and naming consolidation.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.