feat(app): search across multiple sources, with one source as N=1 - #2885
feat(app): search across multiple sources, with one source as N=1#2885teeohhem wants to merge 1 commit into
Conversation
🦋 Changeset detectedLatest commit: f086064 The changes in this PR will be included in the next version bump. This PR includes changesets to release 4 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR exposes multi-source log and trace search, querying up to three independently configured sources and merging their rows into one normalized timeline.
Confidence Score: 4/5The PR is not yet safe to merge because sources with ascending or non-timestamp-first ordering can produce missing or incorrectly ordered rows in the merged timeline. The multi-source query plan still preserves each source’s explicit ORDER BY while SearchResultsTable merges every stream as timestamp-descending, so valid source configurations can violate the merger’s ordering and frontier assumptions. Files Needing Attention: packages/app/src/DBSearchPage.tsx and packages/app/src/components/SearchResultsTable.tsx
|
| Filename | Overview |
|---|---|
| packages/app/src/DBSearchPage.tsx | Wires multi-source selection and query specifications into the search page, but still allows per-source ordering that is incompatible with the descending timestamp merger. |
| packages/app/src/components/SearchResultsTable.tsx | Implements independent source querying, normalization, pagination, and merged-table rendering. |
| packages/app/src/hooks/useMultiSourceSearch.ts | Provides schema-union and per-source column-resolution helpers for multi-source queries. |
| packages/app/src/hooks/useResolvedSourcesParam.ts | Resolves and validates the source identifiers supplied through the multi-source URL parameter. |
| packages/app/src/hooks/useSourceSlots.ts | Supplies a fixed number of hook slots for the capped source selection. |
| .changeset/multi-source-search.md | Adds the release metadata for multi-source search. |
Sequence Diagram
sequenceDiagram
participant User
participant SearchPage as DBSearchPage
participant Sources as Source query streams
participant Merge as SearchResultsTable
participant Table as Results timeline
User->>SearchPage: Select up to three sources
SearchPage->>Sources: Build one query specification per source
par Independent source queries
Sources->>Sources: Query source A
Sources->>Sources: Query source B
Sources->>Sources: Query source C
end
Sources-->>Merge: Paginated row streams and status
Merge->>Merge: Normalize and merge by timestamp
Merge-->>Table: Interleaved rows with source badges
Table-->>User: Render combined timeline
Reviews (2): Last reviewed commit: "feat(app): search across multiple source..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| @@ -0,0 +1,618 @@ | |||
| import { useCallback, useEffect, useMemo, useState } from 'react'; | |||
There was a problem hiding this comment.
The new 618-line component combines query execution, pagination, stream merging, schema normalization, denoising, side-panel state, and rendering. This exceeds the repository's 300-line file limit by more than twofold, making these interdependent behaviors harder to review, test, and safely modify.
Context Used: CLAUDE.md (source)
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
| now expand into a multi-select (up to 3 log/trace sources): results interleave | ||
| into one timestamp-ordered timeline with a per-row source badge, normalized | ||
| columns (Timestamp, Source, Service, Level, Message, and Duration when traces | ||
| are included), a histogram stacked by source, and an add-column picker over the |
There was a problem hiding this comment.
The release note says this change includes a histogram stacked by source, but multi-source mode currently renders only the merged results table and explicitly defers the histogram to the next PR. Publishing this changeset would promise behavior that this release does not provide.
The search page had one results table bound to one source. It now takes a list: pick up to 3 log/trace sources and their rows interleave into a single timestamp-ordered timeline, each tagged with the source it came from. One source is not a separate path — it is N=1. Its spec carries the user's own SELECT and ORDER BY, so the table renders the authored columns with no source column, sorts, denoises, and reports errors to the page exactly as before. Several sources project the canonical aliases instead, and the client merges the streams behind the safe frontier. There is no UNION: sources can live on different ClickHouse connections, each keeps its own per-table machinery (Lucene serializer, text-index detection, materialized-column rewrites, query settings), and a source that fails degrades to a status chip instead of failing the search. Selections are shareable via ?sources=. Multi-source is Lucene-only; the histogram, total count, and filters sidebar stay single-source until the next change.
2348d2d to
f086064
Compare
E2E Test Results✅ All tests passed • 280 passed • 1 skipped • 1173s
Tests ran across 4 shards in parallel. |
1 participant
Stacked on #2884. This is where the feature becomes visible.
The search page had one results table bound to one source. It now takes a list: pick up to 3 log/trace sources and their rows interleave into a single timestamp-ordered timeline, each row tagged with the source it came from, with normalized columns (Timestamp, Source, Service, Level, Message, and Duration when traces are included) and an add-column picker over the union of the selected sources' columns.
One source is not a separate path — it is N=1. Its spec carries the user's own SELECT and ORDER BY, so the table renders the authored columns with no source column, sorts, denoises, and reports errors to the page exactly as before. Several sources project the canonical aliases from #2882 and the client merges the streams behind the safe frontier from #2883.
No
UNION: sources can live on different ClickHouse connections, each keeps its per-table machinery (Lucene serializer, text-index detection, materialized-column rewrites, query settings), and a failing source degrades to a status chip instead of failing the search. Fan-out cost is ~3 ClickHouse queries per source per refresh, which is why the selection is capped at 3.Scope: the histogram, total count, and filters sidebar stay single-source here and go N-ary in the next PR, so this diff is the table and its wiring. Multi-source is Lucene-only (raw SQL names one table's columns); saved searches and alerts remain single-source.