perf: batch execution AND/OR/NAND

[cheb0]

Description

Batch execution allows to iterate over inverted index batch by batch instead of lid by lid.

Batches are either of slice or roaring bitmap type. When batches are intersected/unioned they are converted to bitmaps.

Enabling

• query AST should suffice (AND/OR/AND NOT/range)
• range node can have at most 5 tids resolved
• iteration cost is evaluated and must be at least of configured number (freqs are required in fraction)

Iteration cost evaluation allows to enable batching when it's really worth it. In that case we know some good amount of LID blocks will be directly used as bitmaps and not be converted.

Results

• Searches - I'd say most queries are unaffected except those which iterate over millions of postings.

Query	env	Total	cold, ms		hot, ms		cold (branch), ms		hot (branch), ms		cold diff	hot diff
service:large-service	prod	220827	16.32	±3.45	0.54	±0.10	16.81	±6.27	0.7	±0.09	3%	29.6%
service:large-service AND level:6	prod	111809	24.9	±3.25	6.75	±0.12	18.42	±1.76	2.56	±0.49	-26%	-62.1%
service:small-service AND level:3	prod	349	35.75	±1.03	2.07	±0.13	38.12	±1.89	2.37	±0.12	6.6%	14.5%
service:srv* AND level:[0 to 3]	prod	214	51.54	±1.94	8.49	±0.15	51.67	±1.54	9.43	±0.19	0.3%	11.1%
request_host:large-host AND response_status:200	lb	1058473	47.01	±2.74	21.16	±0.36	22.23	±1.69	5.09	±0.25	-52.7%	-75.9%
request_host:large-host AND response_status:500	lb	2	14.91	±0.33	0.31	±0.03	16.62	±2.67	0.48	±0.15	11.5%	54.8%
request_host:large-host AND request_method:POST AND geoip_country:RU	lb	485486	66.71	±5.18	30.11	±0.17	25.66	±2.96	5.18	±0.70	-61.5%	-82.8%
(response_status:504 or response_status:502) and (cluster_name:cl1 or cluster_name:cl2)	lb	2996	49.42	±2.28	2.2	±0.06	44.76	±2.00	1.66	±0.03	-9.4%	-24.5%
k8s_service_name:medium-service and request_host:medium-host	lb	357675	21.1	±0.66	5.31	±0.13	17.55	±1.31	2.02	±0.10	-16.8%	-62%
(request_host:small-host or request_host:small-host2) and hostname:some-server	lb	25	19.72	±0.69	0.42	±0.03	18.66	±0.73	0.45	±0.04	-5.4%	7.1%
NOT response_status:200	lb	192352	32.85	±3.50	13.19	±0.20	45.88	±2.08	27.34	±0.72	39.7%	107.3%
request_host:large-host AND NOT response_status:200	lb	6747	69.38	±2.03	17.32	±0.15	22.18	±2.43	4.22	±1.12	-68%	-75.6%
request_host:small-host AND NOT (request_method:post or request_method:get)	lb	0	43.58	±2.40	17.62	±0.26	17.32	±0.75	2.77	±0.25	-60.3%	-84.3%

• Histograms - behave like searches but improvement is lower since we read MIDs

Query	env	Total	cold, ms		hot, ms		cold (branch), ms		hot (branch), ms		cold diff	hot diff
service:large-service AND level:[4 to 6] | hist 60s	prod	218423	95.5	±3.71	12.59	±0.41	87.72	±5.60	6.97	±1.43	-8.1%	-44.6%
request_host:large-host AND response_status:200 | hist 60s	lb	1058473	136.62	±5.49	27.33	±1.49	112.97	±4.36	11.1	±0.98	-17.3%	-59.4%

• Aggs - Mostly aggs do not use batching, but there is an overhead from allocating batches in LID cursors.

Query	env	Total	cold, ms		hot, ms		cold (branch), ms		hot (branch), ms		cold diff	hot diff
|service:marketing-actions-api | by k8s_pod	prod	112107	256.22	±4.80	173.08	±3.03	294.39	±29.50	216.32	±24.66	14.9%	25%
request_host:api.ozon.ru | by remote_addr	lb	1065220	2307.86	±107.43	2148.07	±127.33	2830.23	±473.64	2591.46	±388.58	22.6%	20.6%

Major problems

There are problems I found while working on batch execution. Can be partially addressed in future.

• LIDBatch - interface dispatch overhead is now present on inverted index which has partially affected hot queries performance.
• roaring bitmap doesn't have NextGEQ (single function)
• Batches can grow very large, i.e. level:[5 to 7] (nodeOrBatchedMulti) can yield a large batch of size more than LID block. Truncating a batch can cost CPU and increase query execution time.
• Batches needs to be allocated
• Batch truncation takes CPU time. Current execution model is same as for LIDs which means we need truncation when do OR/AND NOT. It's possible to redo the model with full materialization. It will sacrifice early exit and block skipping but can be more performant with current inverted index sort order.

---

• I have read and followed all requirements in CONTRIBUTING.md;
• I used LLM/AI assistance to make this pull request;