Search before asking
Motivation
Flink’s vector_search procedure currently performs vector candidate search locally in the procedure process. For large tables with many index splits, raw data ranges, or primary-key buckets, this limits scalability and prevents the search from utilizing cluster resources.
Solution
Add an optional distributed execution mode for Flink vector search. Large searches are dispatched to a bounded Flink job and merged into a global TopK result, while small searches retain the existing local execution path to avoid unnecessary job startup overhead.
Anything else?
No response
Are you willing to submit a PR?
Search before asking
Motivation
Flink’s vector_search procedure currently performs vector candidate search locally in the procedure process. For large tables with many index splits, raw data ranges, or primary-key buckets, this limits scalability and prevents the search from utilizing cluster resources.
Solution
Add an optional distributed execution mode for Flink vector search. Large searches are dispatched to a bounded Flink job and merged into a global TopK result, while small searches retain the existing local execution path to avoid unnecessary job startup overhead.
Anything else?
No response
Are you willing to submit a PR?