feat(opensearch): extract vendor-neutral SearchAPI and phase-aware router

[fabrizzio-dotCMS]

Summary

Extracts a vendor-neutral SearchAPI interface to decouple the search layer from Elasticsearch-specific types (SearchResponse, ESSearchResults). A phase-aware router (SearchAPIImpl) delegates reads to ES (phases 0–1) or OS (phases 2–3) using the existing PhaseRouter<T> pattern. Addresses PR review point #1 by wiring search/searchRaw into ContentletAPIInterceptor and marking the legacy ES methods for removal.

Changes

New search layer (com.dotcms.content.index)

• SearchAPI — vendor-neutral interface replacing ESSeachAPI for search, searchRaw, searchRelated
• SearchAPIImpl — phase-aware router via PhaseRouter<SearchAPI>
• ESSearchAPIImpl — ES adapter: delegates to legacy ESSeachAPI, converts SearchResponse → neutral DTOs
• OSSearchAPIImpl — native OS implementation using SearchRequest._DESERIALIZER + VersionedIndicesAPI
• Neutral DTOs: ContentSearchResponse, ContentSearchResults, SearchHits, SearchHit, TotalHits, AggregationBucket

Interceptor & hook wiring (addresses review point #1)

• ContentletAPIInterceptor — added search() and searchRaw() overrides with full pre/post hook dispatch and community-license guard on searchRaw
• ContentletAPIPreHook / ContentletAPIPostHook — added search()/searchRaw() default no-op methods; marked esSearch/esSearchRaw hooks as @Deprecated

Deprecations

• ContentletAPI.esSearch / esSearchRaw — upgraded to @Deprecated(forRemoval = true), replaced by search() / searchRaw() default methods delegating to APILocator.getSearchAPI()
• ESContentletAPIImpl.esSearch / esSearchRaw — marked @Deprecated(forRemoval = true)

Caller migrations

• ContentTypeAPIImpl, BrowserAPIImpl, ESContentTool, WorkflowHelper, PageResource — migrated from getEsSearchAPI() to getSearchAPI()
• ESContentletAPIImpl — getRelatedChildren/getRelatedParents migrated to neutral SearchHit API

Tests

• Added OSSearchAPIImplIntegrationTest (registered in OpenSearchUpgradeSuite)
• Migrated ESMappingAPITest, ES6UpgradeTest, ContentletAPITest, PageResourceTest, ContentTypeDestroyAPIImplTest to neutral API

Deferred (separate PRs)

• ESContentResourcePortlet — relies on SearchResponse.toString() for ES wire-format JSON; marked with FIXME(OS-cutover)

Testing

# Compile
./mvnw compile -pl :dotcms-core -DskipTests

# OS integration tests (requires opensearch-upgrade container on http://localhost:9201)
./mvnw verify -pl :dotcms-integration \
    -Dcoreit.test.skip=false \
    -Dopensearch.upgrade.test=true \
    -Dit.test=OSSearchAPIImplIntegrationTest

# Spot-check migrated callers
./mvnw verify -pl :dotcms-integration \
    -Dcoreit.test.skip=false \
    -Dit.test=ESMappingAPITest,ContentTypeDestroyAPIImplTest,ContentletAPITest

Manual:

• Verify findRecentContent / getEntriesByContentTypes return correct results against ES
• Verify BrowserAPIImpl search still filters inodes correctly
• Verify Velocity $ESContent.search(...) and $ESContent.raw(...) work in templates
• Verify WorkflowHelper step aggregations display correctly in the workflow UI

Breaking Changes

None. All changes are backward-compatible: esSearch/esSearchRaw are deprecated but still functional until ESContentResourcePortlet is migrated and they are removed.

🤖 Generated with Claude Code

[claude]

Claude finished @fabrizzio-dotCMS's task in 6m 39s —— View job

---

PR review — vendor-neutral SearchAPI

Functional / behavioral risks

1. OS path silently drops SearchRequest body fields — OSSearchAPIImpl.executeSearch deserializes the user's JSON into a SearchRequest template, then manually copies 9 fields (query, aggregations, source, from, size, sort, highlight, postFilter, trackTotalHits) into a new builder. Anything else in the user's JSON — scroll, search_after, terminate_after, timeout, version, explain, min_score, indices_boost, runtime_mappings, stored_fields, suggest, docvalue_fields, script_fields, stats — is silently dropped. Legacy ES uses SearchSourceBuilderUtil.getSearchSourceBuilder(jsonObject.toString()) which round-trips the whole JSON. This is a real divergence the moment any caller (plugin, integration, future internal change) passes a query with one of those keys. (dotCMS/src/main/java/com/dotcms/content/index/opensearch/OSSearchAPIImpl.java:317-358)

2. No timeout on OS search — legacy ES path sets INDEX_OPERATIONS_TIMEOUT_IN_MS on every search. OS path doesn't. A bad query on a busy cluster can hang indefinitely. (OSSearchAPIImpl.java:317-365 vs enterprise/.../ESSearchAPIImpl.java:333-335)

3. Internal OS client symbol — SearchRequest._DESERIALIZER is the leading-underscore "internal use" path of opensearch-java. Locks the impl to a private contract that the OS client team can change without a semver bump. (OSSearchAPIImpl.java:324)

4. PhaseRouter.readChecked swallows non-OS errors as "OS unavailable" — in Phase 2 every Exception is caught and retried against ES with the log message "OS index may be stale or unavailable". A DotSecurityException from OSSearchAPIImpl.executeSearch (null user, frontend-roles=false) triggers a misleading error log and double-executes the security check on ES. Worth filtering by exception type so security/argument errors don't masquerade as OS health problems. (PhaseRouter.java:298-310)

API design

5. Overload collision on ContentletAPI.search — the new

default ContentSearchResults<Contentlet> search(String query, boolean live, User user, boolean respectFrontendRoles)

now sits next to the existing Lucene overloads:

List<Contentlet> search(String luceneQuery, int limit, int offset, String sortBy, User user, boolean respectFrontendRoles)

Different query language (JSON vs Lucene), different return type, same method name. The earlier searchJson/searchRawJson name (commit 3188eb5) avoided this — the rename back (320ccd8) doesn't have a rationale in the commit message and meaningfully raises the chance of caller confusion in IDE autocomplete. (Fix this →)

6. Default methods bypass the interceptor — ContentletAPI.search / searchRaw defaults call APILocator.getSearchAPI() directly. Any code path holding a non-interceptor ContentletAPI (a unit-test stub, a direct ESContentletAPIImpl reference) skips both the pre/post hooks and the community-license guard that the interceptor adds to searchRaw. Legacy esSearch/esSearchRaw were abstract, so subclasses had to opt in. Surface is wider now — worth a comment, or better, a guard inside the default itself.

7. SearchAPIImpl() constructor eagerly resolves CDI beans — APILocator.getSearchAPI() is lazy, but the first call constructs SearchAPIImpl which immediately does CDIUtils.getBeanThrows(ESSearchAPIImpl.class) and OSSearchAPIImpl.class. If anything in early startup (a BackgroundTask, a Quartz job, a CDI-eager bean) touches search before CDI is fully ready, you get a hard fail vs. the legacy lazy-resolved path. Verify the cold-start order or make the lookup lazy. (SearchAPIImpl.java:41-44)

Smaller items

8. ContentSearchResults declares serialVersionUID = 1L but does not implement Serializable. Pick one. (ContentSearchResults.java:24-26)

9. ESContentletAPIImpl.getRelatedChildren (and getRelatedParents) does if (response.hits().hits().isEmpty()) return result; and then iterates response.hits() anyway — the empty-check is dead weight, and the two response.hits() reads invite a future null mistake. Just iterate. (ESContentletAPIImpl.java:2402-2407)

10. OS searchRelated six-arg overload hard-codes live=false regardless of the caller-supplied live arg. Matches legacy ES (also looks like a bug there) but it's now contract on the new neutral API — at minimum add a Javadoc note. (OSSearchAPIImpl.java:162-174)

11. ESSearchAPIImpl wraps null returns from legacy esSearchRaw with "ES search returned null — unable to load index information". The real cause (a DotDataException from loadIndicies() that legacy logs as fatal and swallows) is gone by the time the adapter throws. Worth correlating these so an OS-cutover post-mortem isn't a forensic exercise. (ESSearchAPIImpl.java:71-77)

12. StringBuffer in OSSearchAPIImpl.executeSearch — copied from legacy; StringBuilder is fine for thread-local use. Nit.

Tests

OS integration tests (OSSearchAPIImplIntegrationTest) only exercise empty-index and one one-doc _source rewrite case. None cover: terms-aggregation buckets (the path ContentTypeAPIImpl.findRecentContent and getEntriesByContentTypes both rely on), the sort-by parameter, searchRelated with real related content, or the _source field-loss behavior flagged in #1. The aggregation path is the one most likely to regress silently and the one with the least coverage.

---

· branch: issue-34609/search-layer-neutral-api

[github-actions]

PRs linked to this issue

• refactor(search): convert ContentletSearch to immutable interface by @fabrizzio-dotCMS