[improvement](inverted index) Shrink SNII indexes on mostly NULL columns and use per-field BM25 document counts by eldenmoon · Pull Request #68202 · apache/doris

eldenmoon · 2026-09-18T13:05:29Z

What problem does this PR solve?

Issue Number: None

Problem Summary:

A VARIANT column copies each inverted index definition to every materialized path, so one SNII segment holds one logical index per (path, definition). Most paths are sparse, and two SNII structures cost one entry per segment row regardless of how many rows hold a value:

BM25 norms: an analyzed index with positions stores one byte per row, NULL rows included.
NULL bitmaps: NullBitmapWriter serialized its CRoaring bitmap without runOptimize, so every 65536-row container of a mostly NULL path stayed an 8 KiB bitset (about N/8 bytes per index per segment). The definitions on one path also stored identical copies.

On top of that, BM25 used the segment row count as the document count, so on sparse paths avgdl came out far too small and idf differences were flattened.

This PR has two commits.

Commit 1: no on-disk format change

NullBitmapWriter run-optimizes and shrinks the bitmap before sizing and writing it. The inverted index, direct SNII index compaction and the BKD blob's bkd_nulls all go through this writer. Run containers are part of the portable CRoaring format, so older builds read them.
The compound writer stores one null bitmap per subcolumn. It remembers the last null bitmap it appended (suffix, XXH3-128 hash, length, region). When the next logical index on the same suffix produces an identical bitmap, it references that region instead of appending a copy. Region references are absolute offsets and no reader assumes a region belongs to one index, so the format is unchanged.
CollectionStatistics keeps a document count per field. SNII uses the field's indexed_doc_count (non-NULL rows) for both avgdl and idf N, which is what Lucene's docCount means. CLucene numbers are unchanged.

Commit 2: sparse BM25 norms behind a switch

New section type kNormsSparse (15). It stores norms only for the documents that carry one: present docids in 65536-docid blocks (ALL / ARRAY / BITSET with a rank table / RUNS), plus one byte per present document, or a single byte when all norms are equal.
The writer keeps the existing dense kNormsPod (14) section, byte-identical to before, when no document lacks a norm or when dense is not larger.
New mutable BE config enable_snii_sparse_norms, default true. The only decision point is LogicalIndexWriter::finalize_build, which load, compaction, schema change and BUILD INDEX all go through. Readers accept both layouts regardless of the config.
We also compared a CRoaring-based encoding of the present set on the same data:
- Section bytes differed by under 1%.
- Its per-document lookups were 2–34x slower on a RELEASE build. They were 1.7–2.4x slower even with a batch API at 5% density, the most common density we measured.
- Its zero-copy frozen view cannot be used safely on untrusted offsets.
- The hand-written block format was therefore kept.

End-to-end results. Each row compares master with this PR on the same data, measured as the bytes of one compacted segment file on an ASAN build.

Table	master	this PR	Detail
1.02M-row VARIANT table, 1000 paths at 0.8% density, 11 index definitions (3 analyzed with phrase)	601.2 MB	77.4 MB (7.8x)	norms 382.5 → 6.2 MB; inverted null bitmaps 114.8 → 16.4 MB; BKD null bitmaps 65.6 → 16.4 MB; dictionary / postings / BKD data unchanged
677k-row GitHub-events table, payload `parser=none` + `parser=english` (phrase)	913.1 MB	571.9 MB (1.60x)	norms 324.3 → 39.8 MB; inverted null bitmaps 86.4 → 31.9 MB; dictionary 351.2 MB unchanged
Same 1.02M-row data, `enable_snii_sparse_norms = false`, 10 load rowsets	647.1 MB	453.7 MB	null bitmaps only; norms stay in the dense layout

BM25 scores are bit-identical between the dense and sparse layouts: after an in-place upgrade, on tables mixing master-written and new segments, and after full compaction into the new layout. score() latency did not regress.

Compatibility (tested by swapping BE binaries over the same storage)

Case	Result
This PR reading segments written by master	MATCH, MATCH_PHRASE, equality, IS NULL and BKD range counts identical
Master reading segments written by this PR with the config off	Read natively, 0 fallbacks, `score()` works; commit 1 needs no switch
Master reading sparse norms sections	The index cannot be opened. Filters fall back to row-by-row evaluation with correct results; `score()` fails with E-6012. A full compaction by the older binary rewrites the index densely and restores `score()`. Turn the config off while older BEs may read newly written segments.

Release note

BM25 scores (score()) of SNII inverted indexes on columns or VARIANT paths with NULL rows change: avgdl and idf now use the field's non-NULL document count instead of the segment row count, as Lucene does. Filtering results do not change.
SNII NULL bitmaps are run-length encoded and stored once per subcolumn; the format is unchanged.
New mutable BE config enable_snii_sparse_norms (default true): SNII stores BM25 norms only for rows that carry one when that is smaller. BEs without this change cannot use indexes written this way (filters fall back to row-by-row evaluation and score() fails); set it to false while such BEs may read newly written segments, e.g. during a rolling upgrade or before a downgrade.

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Unit tests (both commits built and tested on their own):
- Commit 1: 1788 SNII / norms / NULL bitmap / BM25 tests, 1779 pass.
- Final head: 1803 tests, 1794 pass.
- In both runs the only failures are 8 environment-specific SniiBatchRangeFetcher.* / SniiLocalFile.* tests (their fixed /tmp files belong to another OS user on the test host), and SniiGoldenCorpus.WriteOrVerify is skipped without SNII_GOLDEN_DIR.
- New suites: SniiNormsSection, SniiSparseNormsTest, SniiSharedNullBitmap, plus extended SniiNullBitmap, SniiIndexCompactionTest (mixed dense/sparse sources, config on and off) and CollectionStatisticsTest.
Regression tests:
- New test_storage_format_snii_sparse_norms: config on / off / toggled per batch, full compaction, identical results and scores across layouts.
- test_storage_format_snii_norms: expectations regenerated for the new BM25 statistics and hand-checked.
- These pass unchanged: test_storage_format_snii, _utf8_wildcard, _custom_analyzer, test_variant_search_subcolumn_snii, regression_test_variant_var_index_snii, regression_test_variant_snii_compaction, test_variant_v2_snii_index, test_timestamp_ns_index.
Manual test: the end-to-end A/B, upgrade and rollback runs summarized above, on two clusters built from master and from this PR.
Behavior changed:
- No.
- Yes.
  - score() values change for SNII indexes on fields with NULL rows (see release note).
  - With enable_snii_sparse_norms on, new SNII segments may carry the new kNormsSparse section, which BEs without this change cannot use.
Does this need documentation?
- No.
- Yes.
  - An entry for enable_snii_sparse_norms in the BE configuration reference, and a note on the BM25 statistics in the scoring docs.
  - Doc PR to follow.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

🤖 Generated with Claude Code

…eld document counts for BM25 ### What problem does this PR solve? Issue Number: None Related PR: None Problem Summary: Three independent problems of SNII indexes on mostly NULL columns and VARIANT paths, fixed without any on-disk format change. 1. NullBitmapWriter serialized its CRoaring bitmap right after addMany, without runOptimize. NULL rows come in long runs (whole load batches, VARIANT paths absent from most rows), yet every container denser than 4096 values stayed an 8 KiB bitset, so each logical index on a mostly NULL path paid about N/8 bytes per segment no matter how few rows were non-NULL. NullBitmapWriter now run-optimizes and shrinks the bitmap once before its serialized size is computed and written. The writer is shared by the inverted logical index (also used by direct SNII index compaction) and the BKD blob index, so both benefit; build_memory_upper_bound also covers the one replacement container runOptimize/shrinkToFit hold next to the one they replace. Run containers are part of the portable CRoaring format, so every reader, including builds that predate this change, decodes the new sections. 2. A VARIANT column copies each index definition to every materialized subcolumn, so one SNII container holds one logical index per (definition, subcolumn), and every logical index wrote its own null bitmap section although the definitions on one subcolumn share its suffix and its NULL rows. The compound writer now remembers the last null bitmap it appended (the index suffix, an XXH3-128 hash and the length of the framed bytes, and the region). When the next logical index has the same suffix and an identical bitmap, its core metadata references that region and nothing is appended; the hash stands in for a byte comparison because each bitmap is still released as soon as it is written, so peak build memory does not change. Section references are absolute container offsets and no reader treats a region as owned by one logical index (null bitmaps are read as stateless byte ranges, the rewrite snapshot's physical prefix ends at the largest referenced region end, compaction decodes source bitmaps into docids), so builds without this change read the new containers. Loads, vertical compaction and SNII index compaction all write the definitions of one subcolumn back to back, so N definitions store one bitmap instead of N, saving (N - 1)/N of the null-bitmap bytes. 3. CollectionStatistics used the segment row count as the BM25 document count N (idf) and as the avgdl denominator, and required all SNII fields scored in one segment to share that count. For a field that is NULL in most rows this is wrong: avgdl comes out far too small and idf differences are flattened. Lucene's docCount is the number of documents that have the field. CollectionStatistics now keeps a document count per field. SNII segments add the field's indexed_doc_count (doc_count - null_count), which every SNII writer has stored since the format was introduced and the metadata decoder requires; the check that all scored SNII fields of a segment have one document count is gone. CLucene segments add their segment document count to every field, so their scores are unchanged. A field without documents yields finite statistics: avgdl divides by at least one document, and idf uses at least the term's document frequency, which a NULL ARRAY row that kept tokens under its NULL flag can raise above the indexed count. Results: - A 1.02M-row VARIANT table with 1000 paths at 0.8% density and 11 index definitions, one compacted segment, end-to-end A/B on the same data: null bitmaps 180.4 MB -> 45.2 MB. - Shared null bitmap, 1M-row container with 625,002 pseudo-random NULL rows: 262,978 -> 131,754 bytes with two definitions on one subcolumn, 920,313 -> 132,969 bytes with seven. - BM25, VARIANT path present in 2 of 3 rows, query "alpha": scores 0.311 and 0.5235 before, 0.1514 and 0.2292 after, i.e. idf = ln(1 + 0.5 / 2.5) with N = 2 and avgdl = 4 / 2. Columns without NULL rows score as before. ### Release note BM25 scores (score()) of SNII inverted indexes on columns or VARIANT paths with NULL rows change: avgdl and idf now use the field's non-NULL document count instead of the segment row count, as Lucene does. SNII null bitmaps take less space; the index format does not change. ### Check List (For Author) - Test: Unit Test, Regression test - SniiNullBitmap: a mostly NULL 1M-row bitmap is >100x smaller than the unoptimized serialization and round-trips; sections serialized without run optimization (as earlier builds wrote them) are still read; sizes follow later adds. - SniiSharedNullBitmap: definitions on one suffix reference one region and each reads its own NULL rows back; different NULL rows or document counts on one suffix, and identical NULL rows on different suffixes, keep separate regions; streamed (compaction) sessions share the region and still release their bitmap bytes; a rewrite that drops the index which wrote the region keeps the index that references it readable; saving measured for N = 2 and 7. - CollectionStatisticsTest: per-field counts from an SNII segment with NULL rows, fields of one segment with different counts, a field without documents, document frequency above the indexed count; query_v2 scoring tests set per-field counts. - Regression: test_storage_format_snii_norms expectations regenerated (the VARIANT path scores change as computed above); with this commit alone that suite, test_storage_format_snii, _utf8_wildcard, _custom_analyzer, test_variant_search_subcolumn_snii, regression_test_variant_var_index_snii, regression_test_variant_snii_compaction, test_variant_v2_snii_index and test_timestamp_ns_index pass. - Behavior changed: Yes. BM25 scores of SNII indexes on columns with NULL rows change as described in the release note; CLucene scores and SNII scores on columns without NULL rows do not change. Null bitmap sections are smaller and may be shared between index definitions of one VARIANT subcolumn; older BEs read them. - Does this need documentation: No Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…ly NULL indexes ### What problem does this PR solve? Issue Number: None Related PR: None Problem Summary: An analyzed SNII index with positions stores a one-byte BM25 norm for every row of the segment, NULL rows included, so each logical index costs one byte per segment row no matter how few rows have a value. VARIANT copies the index to every materialized path and most paths are sparse, so the norms dominate the index size. Lucene has stored norms sparsely since 7.0. A document now carries a norm when it is non-NULL, or when it is NULL but still produced tokens (a nullable ARRAY row can keep its nested payload under the NULL flag; its tokens are in the postings and scoring reads its norm). The writer picks the layout when a logical index is finished: - no document without a norm: the existing dense kNormsPod (14) section, byte-identical to what earlier writers produced; - otherwise the smaller of dense and a new sparse section kNormsSparse (15): [varint doc_count][varint present_count][u8 bytes_per_norm 0|1][varint block_count] [12-byte block headers][varint data_len][block payloads][constant norm | present_count bytes]. Present docids are split into 65536-docid blocks (ALL / ARRAY / BITSET with a rank table / RUNS with rank prefixes), so a lookup is a binary search over the block headers plus O(1) or a binary search inside the block, and the reader needs no memory beyond the section bytes. The layout is documented in norms_pod.h. Readers that predate this change reject a sparse section when they open the logical index (its length differs from the dense length they require and its type is not 14), so they never misread norms, but queries on such a segment fall back to evaluation without the index and score() fails. A new mutable BE config, enable_snii_sparse_norms (default true), therefore controls the writer: when it is off, every index written from then on uses the legacy dense layout. The decision is made in one place, LogicalIndexWriter::finalize_build, which every path goes through (load, compaction output including direct SNII index compaction, schema change and BUILD INDEX). Readers accept both layouts whatever the config says. The column writer no longer keeps a byte per NULL row: it keeps raw token counts of the rows that carry a norm (add_array_nulls drops NULL rows without tokens once their NULL flag is known) and encodes them at finish. Direct SNII index compaction still rebuilds norms from merged postings; it now keeps the documents that carry a norm and hands the NULL documents that still have postings to the session, so the destination gets the same layout as a fresh build under the same config, whatever the source layouts are. On the read side a norms region may not be longer than a dense section (the cache charge stays bounded by doc_count), sparse payloads are fully validated when loaded, a checked lookup of a docid without a norm returns a corruption error, and SniiStatsProvider rejects a norms section that covers fewer documents than the indexed count. Results, end-to-end A/B on the same data, one compacted segment each, before = master and after = this commit together with the previous one: - 1.02M-row VARIANT table, 1000 paths at 0.8% density, 11 index definitions: index 601.2 MB -> 89.7 MB; norms 382.5 MB -> 6.2 MB (this commit), null bitmaps 180.4 MB -> 45.2 MB (previous commit). - 677k-row GitHub-events table: index 913.1 MB -> 603.9 MB; norms 324.3 MB -> 39.8 MB. BM25 scores are bit-identical between the two layouts. ### Release note New mutable BE config enable_snii_sparse_norms (default true). SNII inverted indexes store BM25 norms only for the rows that carry one when that is smaller, which shrinks indexes on mostly NULL columns and VARIANT paths (for example norms 382.5 MB -> 6.2 MB on a 1M-row VARIANT table with 1000 sparse paths). BEs without this change cannot use segments written with sparse norms (their index is skipped and score() fails); set enable_snii_sparse_norms = false while such BEs may read newly written segments, for example during a rolling upgrade or before a downgrade. ### Check List (For Author) - Test: Unit Test, Regression test - SniiNormsSection: legacy dense bytes unchanged (compared with a hand-assembled section), forced dense with NULL rows equals the earlier bytes, sparse round trips (norm bytes / constant norm), every block kind, layout choice, invalid input, randomized lookups against a dense oracle, corrupt sparse payloads, sparse sections fail the checks older readers apply. - SniiSparseNormsTest: the production column writer writes the same NULL-heavy scalar and nullable ARRAY input (NULL rows with tokens) with enable_snii_sparse_norms on and off: sparse section vs a dense section byte-identical to the legacy one, identical norms and bit-identical BM25 scores; a segment without NULL rows is byte-identical in both modes; the config is on by default. - SniiIndexCompactionTest.NormsMergeOfMixedLayoutsMatchesRebuild: direct compaction over sparse, dense-with-NULLs and NULL-free sources, with the config on and off, equals a fresh build under the same config and scores like a dense build; existing compaction, streamed session, compound writer, writer and collection statistics tests updated to the new norms input (SniiWriterNorms.NullRowsKeepNormsOnlyWithTokens added). - Regression: new test_storage_format_snii_sparse_norms loads the same batches with the config on, off, and toggled between batches, then runs a full compaction (config off for the dense table, on for the others) and checks that MATCH / MATCH_PHRASE / IS NULL / score() results of all three tables are identical before and after, and that the sparse layout is smaller; test_storage_format_snii, _norms, _utf8_wildcard, _custom_analyzer, test_variant_search_subcolumn_snii, regression_test_variant_var_index_snii, regression_test_variant_snii_compaction, test_variant_v2_snii_index and test_timestamp_ns_index pass unchanged. - Behavior changed: Yes. New BE config enable_snii_sparse_norms. With it on (the default), new SNII segments of indexes with NULL rows may carry the new kNormsSparse section, which BEs without this change cannot read; with it off the written bytes are the same as before. Query results and BM25 scores do not depend on the layout. - Does this need documentation: Yes (an entry for enable_snii_sparse_norms in the BE configuration reference; doc PR not opened yet) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

hello-stephen · 2026-09-18T13:05:34Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

eldenmoon · 2026-09-18T13:05:39Z

run buildall

hello-stephen · 2026-09-18T14:35:13Z

TPC-H: Total hot run time: 27678 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c034934f9be38631b53353e6952e14146f717dfb, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17616	3882	3819	3819
q2	2305	347	300	300
q3	10029	1406	818	818
q4	4679	474	347	347
q5	7460	814	557	557
q6	180	168	136	136
q7	748	769	588	588
q8	9317	1537	1526	1526
q9	5556	4166	4114	4114
q10	6702	1613	1386	1386
q11	430	271	253	253
q12	637	413	306	306
q13	18080	2585	2002	2002
q14	269	262	243	243
q15	q16	726	713	666	666
q17	1744	1094	1034	1034
q18	6511	5603	5539	5539
q19	1154	1258	986	986
q20	493	408	266	266
q21	5507	2836	2489	2489
q22	432	388	303	303
Total cold run time: 100575 ms
Total hot run time: 27678 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4686	4605	4535	4535
q2	768	638	581	581
q3	4794	5370	4651	4651
q4	2265	2351	1441	1441
q5	4700	4496	4421	4421
q6	226	183	139	139
q7	1919	1714	1511	1511
q8	2293	2073	2313	2073
q9	7265	7065	6865	6865
q10	4246	4236	3838	3838
q11	509	386	349	349
q12	701	727	516	516
q13	2308	2626	2000	2000
q14	264	271	255	255
q15	q16	669	685	613	613
q17	7342	6781	6664	6664
q18	11878	11073	11771	11073
q19	1115	1014	1020	1014
q20	2227	2201	1918	1918
q21	5319	4493	4611	4493
q22	516	458	400	400
Total cold run time: 66010 ms
Total hot run time: 59350 ms

hello-stephen · 2026-09-18T14:45:12Z

TPC-DS: Total hot run time: 153258 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c034934f9be38631b53353e6952e14146f717dfb, data reload: false

query5	4305	598	458	458
query6	432	189	164	164
query7	4804	527	274	274
query8	314	167	167	167
query9	8782	3989	3984	3984
query10	443	306	252	252
query11	5830	2194	2002	2002
query12	159	99	92	92
query13	1269	557	409	409
query14	6500	4647	4325	4325
query14_1	4172	4083	4124	4083
query15	205	196	172	172
query16	996	452	448	448
query17	930	678	544	544
query18	2426	467	342	342
query19	203	181	138	138
query20	107	100	106	100
query21	220	139	120	120
query22	13107	13080	12775	12775
query23	15309	14692	14020	14020
query23_1	14211	14112	14083	14083
query24	7549	1694	1230	1230
query24_1	1249	1247	1238	1238
query25	529	401	344	344
query26	1268	288	165	165
query27	2688	542	321	321
query28	4532	1942	1939	1939
query29	1034	561	437	437
query30	323	226	197	197
query31	880	769	634	634
query32	148	96	90	90
query33	503	314	250	250
query34	1179	1089	632	632
query35	727	737	643	643
query36	805	802	704	704
query37	145	104	90	90
query38	1846	1780	1716	1716
query39	722	675	645	645
query39_1	670	646	650	646
query40	219	120	99	99
query41	67	67	65	65
query42	99	89	94	89
query43	352	352	317	317
query44	1390	691	707	691
query45	182	182	172	172
query46	1082	1226	698	698
query47	1480	1509	1371	1371
query48	405	398	268	268
query49	581	388	279	279
query50	925	345	250	250
query51	10570	10175	10218	10175
query52	83	85	73	73
query53	240	259	178	178
query54	249	206	206	206
query55	76	73	73	73
query56	243	241	211	211
query57	1360	1410	1325	1325
query58	243	211	212	211
query59	1970	2046	1811	1811
query60	279	235	217	217
query61	142	136	144	136
query62	398	320	269	269
query63	216	177	177	177
query64	2838	987	805	805
query65	4008	3979	3963	3963
query66	1795	426	302	302
query67	20033	20020	19695	19695
query68	3076	1413	925	925
query69	414	302	262	262
query70	957	905	879	879
query71	295	241	214	214
query72	3038	2468	2210	2210
query73	810	737	439	439
query74	4636	4458	4287	4287
query75	2308	2268	1927	1927
query76	2354	1166	711	711
query77	364	398	306	306
query78	8913	9017	8456	8456
query79	1181	1104	756	756
query80	533	470	374	374
query81	462	282	238	238
query82	280	168	126	126
query83	270	275	244	244
query84	286	141	110	110
query85	780	455	380	380
query86	283	239	224	224
query87	2011	1986	1831	1831
query88	3661	2734	2727	2727
query89	338	298	263	263
query90	2120	185	180	180
query91	160	153	125	125
query92	103	89	86	86
query93	1373	1437	797	797
query94	527	330	301	301
query95	667	382	422	382
query96	1125	755	331	331
query97	2437	2483	2317	2317
query98	198	188	196	188
query99	732	717	605	605
Total cold run time: 239937 ms
Total hot run time: 153258 ms

hello-stephen · 2026-09-18T14:50:04Z

ClickBench: Total hot run time: 24.16 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c034934f9be38631b53353e6952e14146f717dfb, data reload: false

query1	0.00	0.00	0.00
query2	0.09	0.05	0.05
query3	0.26	0.14	0.14
query4	1.60	0.14	0.14
query5	0.24	0.22	0.23
query6	1.16	0.95	0.94
query7	0.05	0.01	0.01
query8	0.05	0.04	0.04
query9	0.39	0.33	0.35
query10	0.53	0.55	0.57
query11	0.21	0.14	0.15
query12	0.18	0.15	0.15
query13	0.47	0.48	0.46
query14	0.94	0.94	0.94
query15	0.61	0.59	0.60
query16	0.30	0.33	0.33
query17	1.10	1.06	1.16
query18	0.22	0.20	0.19
query19	2.02	1.96	1.95
query20	0.02	0.01	0.01
query21	15.43	0.19	0.13
query22	4.98	0.05	0.05
query23	16.14	0.30	0.12
query24	2.96	0.41	0.33
query25	0.09	0.05	0.05
query26	0.74	0.20	0.15
query27	0.04	0.03	0.03
query28	3.49	0.75	0.36
query29	12.51	4.08	3.27
query30	0.28	0.15	0.16
query31	2.78	0.55	0.30
query32	3.22	0.58	0.49
query33	3.18	3.19	3.27
query34	15.57	3.92	3.32
query35	3.22	3.22	3.24
query36	0.56	0.42	0.41
query37	0.09	0.07	0.06
query38	0.05	0.04	0.03
query39	0.04	0.03	0.03
query40	0.18	0.15	0.14
query41	0.08	0.03	0.03
query42	0.03	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 96.15 s
Total hot run time: 24.16 s

hello-stephen · 2026-09-18T16:54:54Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (8/8) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	76.27% (34394/45097)
Line Coverage	61.18% (386309/631388)
Region Coverage	57.60% (325274/564675)
Branch Coverage	58.36% (148117/253807)

eldenmoon and others added 2 commits September 18, 2026 18:13

eldenmoon requested review from airborne12 and csun5285 as code owners September 18, 2026 13:05

Navigation Menu

[improvement](inverted index) Shrink SNII indexes on mostly NULL columns and use per-field BM25 document counts - #68202

[improvement](inverted index) Shrink SNII indexes on mostly NULL columns and use per-field BM25 document counts#68202
eldenmoon wants to merge 2 commits into
apache:masterfrom
eldenmoon:snii-sparse-norms

eldenmoon commented Sep 18, 2026

Uh oh!

hello-stephen commented Sep 18, 2026

Uh oh!

eldenmoon commented Sep 18, 2026

Uh oh!

hello-stephen commented Sep 18, 2026

Uh oh!

hello-stephen commented Sep 18, 2026

Uh oh!

hello-stephen commented Sep 18, 2026

Uh oh!

hello-stephen commented Sep 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

eldenmoon commented Sep 18, 2026

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Sep 18, 2026

Uh oh!

eldenmoon commented Sep 18, 2026

Uh oh!

hello-stephen commented Sep 18, 2026

Uh oh!

hello-stephen commented Sep 18, 2026

Uh oh!

hello-stephen commented Sep 18, 2026

Uh oh!

hello-stephen commented Sep 18, 2026

BE Regression && UT Coverage Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants