Skip to content

Negative-scale Decimals (e.g. 1E+2) are corrupted by DecimalLiteral.increment/decrement and truncate_decimal #4284

Open

Description

@AP24110011695

Apache Iceberg version

main (0.12.0)

Please describe the bug this is related to

Three decimal helpers still read a Decimal's scale as abs(int(value.as_tuple().exponent)). A Python Decimal carries the negated scale as its exponent, so a value with a non-negative exponent — e.g. Decimal("1E+2") (100 with scale −2) — is misread as having a positive scale of 2. This is the same root cause as #3997, which fixed three call sites; these remaining ones were not covered:

  • DecimalLiteral.increment() — pyiceberg/expressions/literals.py
  • DecimalLiteral.decrement() — pyiceberg/expressions/literals.py
  • truncate_decimal() — pyiceberg/utils/decimal.py

Describe how to reproduce the bug

from decimal import Decimal
from pyiceberg.expressions.literals import DecimalLiteral
from pyiceberg.utils.decimal import truncate_decimal

DecimalLiteral(Decimal("1E+2")).increment().value   # Decimal('0.02')  — expected 2E+2 (i.e. 200)
DecimalLiteral(Decimal("1E+2")).decrement().value   # Decimal('0.00')  — expected 0E+2 (i.e. 0)
truncate_decimal(Decimal("1.5E+3"), 4)              # Decimal('0.12')  — expected 1.2E+3 (i.e. 1200)

increment()/decrement() are used by the range-projection helpers in pyiceberg/transforms.py, so any future path that hands them a negative-scale Decimal produces bounds on the wrong order of magnitude. Schema-typed paths currently reject such decimals at bind time (fixed by #3997 in to(DecimalType)), which is why this went unnoticed — but the helpers themselves still return corrupted values for valid Decimal input.

Willingness to contribute

  • Yes. I will be willing to submit a PR for this bug.

Transparent disclosure: this issue was investigated and filed with AI assistance (reproduction verified locally on main @ e3564a5).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions