Skip to content

fix: handle list-level NULLs in NOT filters#6044

Open
fenfeng9 wants to merge 4 commits intolance-format:mainfrom
fenfeng9:fix/label-list-null-not
Open

fix: handle list-level NULLs in NOT filters#6044
fenfeng9 wants to merge 4 commits intolance-format:mainfrom
fenfeng9:fix/label-list-null-not

Conversation

@fenfeng9
Copy link
Contributor

closes #5904

LabelListIndex drops list-level NULL rows during unnest, so NOT filters can return NULL lists incorrectly.

Changed:

  • Persist list-level NULL rows in label_list_nulls.lance and merge into null sets.
  • Treat missing nulls file as empty for backward compatibility.
  • Warn when using an old/possibly incomplete LabelList index on nullable columns.

@github-actions github-actions bot added bug Something isn't working python labels Feb 27, 2026
@codecov
Copy link

codecov bot commented Feb 27, 2026

Codecov Report

❌ Patch coverage is 84.75610% with 25 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
rust/lance-index/src/scalar/label_list.rs 82.17% 8 Missing and 10 partials ⚠️
rust/lance/src/index/scalar.rs 53.84% 4 Missing and 2 partials ⚠️
rust/lance-index/src/scalar/lance_format.rs 98.00% 0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LabelListIndex: NOT filters mis-handle NULL lists (list-level NULLs)

1 participant