MDEV-39843: Prefix index causes incorrect range query result

[raghunandanbhat]

Fixes MDEV-39843

Problem:

When a prefix index is built on ignorable characters (under NON1TO1 collation), it has an empty weight, which sorts at the bottom of the index, even though full value's real weight is large.

The range scan for col >= c therefore positions past that empty weight entry and never reads it, even though full value actually satisfies col >= c. That's why the row is missing from the output.

Fix:

In Field::stored_field_make_mm_leaf do not derive an inequality range bound (<, <=, >, >=) from a prefix key segment(HA_PART_KEY_SEG) when the field's collation is MY_CS_NON1TO1. The prefix cannot be trusted as a range bound in that case and let the optimizer fall back to a scan with the full condition re-checked. Equality lookups and full-length (non-prefix) keys are unaffected.

[gemini-code-assist]

Code Review

This pull request addresses MDEV-39843, where a prefix index causes incorrect range query results under collations with complex, non one-to-one character-to-weight mappings. It introduces a helper function prefix_key_part_unsafe_for_range to identify these unsafe cases and returns early in Field::stored_field_make_mm_leaf for inequality comparison operators. The review feedback suggests consolidating the duplicated checks for inequality operators before the switch statement to improve code maintainability.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

The check prefix_key_part_unsafe_for_range(key_part, this) is duplicated across all four inequality comparison cases (SCALAR_CMP_LE, SCALAR_CMP_LT, SCALAR_CMP_GT, SCALAR_CMP_GE). Consolidating this check before the switch statement reduces code duplication and improves maintainability.

  if ((op == SCALAR_CMP_LE || op == SCALAR_CMP_LT ||
       op == SCALAR_CMP_GT || op == SCALAR_CMP_GE) &&
      prefix_key_part_unsafe_for_range(key_part, this))
    DBUG_RETURN(0);

  switch (op) {
  case SCALAR_CMP_LE:
    DBUG_RETURN(new (mem_root) SEL_ARG_LE(str, this));
  case SCALAR_CMP_LT:
    DBUG_RETURN(new (mem_root) SEL_ARG_LT(thd, str, key_part, this, value));
  case SCALAR_CMP_GT:
    DBUG_RETURN(new (mem_root) SEL_ARG_GT(thd, str, key_part, this, value));
  case SCALAR_CMP_GE:
    DBUG_RETURN(new (mem_root) SEL_ARG_GE(thd, str, key_part, this, value));