Skip to content

Improve IEEE decimal transcendental accuracy - #133921

Open
tannergooding wants to merge 3 commits into
dotnet:mainfrom
tannergooding:tannergooding-ulp-analysis
Open

tannergooding wants to merge 3 commits into
dotnet:mainfrom
tannergooding:tannergooding-ulp-analysis

Conversation

@tannergooding

@tannergooding tannergooding commented Sep 14, 2026

Copy link
Copy Markdown
Member

Fixes #133432.

Preserve decimal precision through cancellation-sensitive argument reduction for Decimal32, Decimal64, and Decimal128, without importing Intel DFP's large binary/decimal conversion tables. Remove redundant work in the affected transcendental paths.

  • Form small endpoint residuals in decimal before converting them to the existing binary engine. This covers inverse trig, inverse hyperbolic functions, logarithms, and the compensated Pow path used by RootN.
  • Verify and restore exact dyadic operands against their original decimal coefficients, including alternate representations of 1. Skip residual correction where it is unnecessary.
  • Keep both logarithms and their quotient in working precision for Log(x, base), rounding to decimal only once. Preserve the existing special-value handling.
  • Reduce Pi-scaled trig arguments in decimal, preserving distances from integers and half-integers. Evaluate the already-reduced angle directly; share polynomial evaluation for SinCosPi and TanPi, and bypass unnecessary computation for exact results.
  • Reuse the existing accurate ln(2) constant in Exp2 and Exp2M1; the previous reconstruction subtracted a low part that was already negative.
  • Select the final decimal quantum before rounding, avoiding a second rounding for subnormal results.
  • Reuse repeated power-of-ten chunk multipliers, classify Pow exponent parity only for negative bases, and convert RootN degrees directly to working precision. Handle RootN(x, 1) without log/exp evaluation while retaining the existing result cohort.

The existing approximation algorithms and compact tables are retained. No public API is added. Rounded numerical results intentionally change; source and binary compatibility are unchanged.

Accuracy: before / after

Observed maximum absolute error against a high-precision mathematical reference, measured in decimal ULPs. Bold after-values highlight improvements visible at two decimal places. 0.50 / 0.50 is deliberately not highlighted; rounding to two decimals can hide small differences on either side of the halfway point.

The baseline is 22b24296b6a2580cab3fcc89489bbda3eea79a23. Both builds ran the same inputs in two seeded sweeps, including full-significand random values, endpoint probes, and exponent windows. The sweeps evaluated 211,483 finite mathematical results. References used 220 decimal digits, with each of the 360 post-change window maxima per sweep rechecked at 440 digits. The subsequent wide-log change was evaluated over the same corpora; only log-base results changed. Later performance-only changes were checked for bit-identical results separately. These are sample maxima, not proven bounds or a universal correct-rounding guarantee.

The tables exclude exact-zero references and cases where the baseline returned a nonfinite result for a finite reference; those corrections are described separately below. ULP spacing is determined from the mathematical result and format precision, clamped to the minimum subnormal quantum. Original operands are parsed to the target format before computing the reference.

API Decimal32 before / after Decimal64 before / after Decimal128 before / after
Asin 0.50 / 0.50 0.50 / 0.50 5.18e+11 / 0.50
Acos 0.50 / 0.50 0.50 / 0.50 5.18e+28 / 0.50
AsinPi 0.50 / 0.50 0.55 / 0.50 1.65e+12 / 0.50
AcosPi 0.50 / 0.50 0.50 / 0.50 1.65e+29 / 0.50
Acosh 0.50 / 0.50 0.50 / 0.50 5.39e+27 / 0.50
Atanh 0.50 / 0.50 0.50 / 0.50 2.59e+27 / 0.50
Log 0.50 / 0.50 0.50 / 0.50 1.14e+29 / 0.50
Log2 0.50 / 0.50 0.50 / 0.50 1.50e+29 / 0.50
Log10 0.50 / 0.50 0.50 / 0.50 4.50e+29 / 0.50
LogP1 0.50 / 0.50 0.50 / 0.50 5.18e+27 / 0.50
Log2P1 0.50 / 0.50 0.50 / 0.50 7.48e+26 / 0.50
Log10P1 0.50 / 0.50 0.50 / 0.50 2.25e+27 / 0.50
Log(x, base) 2.86 / 0.50 3.83 / 0.50 5.29e+29 / 0.50
SinPi 0.50 / 0.50 0.50 / 0.50 1.63e+29 / 0.50
CosPi 0.50 / 0.50 0.50 / 0.50 1.63e+29 / 0.50
TanPi 0.50 / 0.50 0.50 / 0.50 2.20e+29 / 0.50
SinCosPi sine component 0.50 / 0.50 0.50 / 0.50 1.63e+29 / 0.50
SinCosPi cosine component 0.50 / 0.50 0.50 / 0.50 1.63e+29 / 0.50
Exp 0.53 / 0.50 0.50 / 0.50 1.19 / 1.19
ExpM1 0.50 / 0.50 0.50 / 0.50 1.31 / 1.31
Exp2 0.53 / 0.50 0.74 / 0.50 4.68e+18 / 2.84
Exp2M1 0.50 / 0.50 0.76 / 0.50 4.68e+18 / 2.84
Exp10 0.54 / 0.50 0.52 / 0.50 1.54 / 1.54
Exp10M1 0.50 / 0.50 0.50 / 0.50 1.54 / 1.54
Pow 0.54 / 0.50 0.52 / 0.50 8.43e+32 / 2.37
RootN 0.50 / 0.50 0.50 / 0.50 0.56 / 0.56

The extra accuracy headroom of the shared 128-bit working significand explains why most Decimal32/64 results were already good. Their improvements include subnormal rounding, the base-two exponential constant at large exponents, avoiding rounded log-base intermediates, and exact-result handling.

Cancellation-sensitive ranges: Decimal128

For endpoint probes, delta = m * 10^-k, with m in {1, 2, 5, 9} and k spanning the representable decimal precision. Interior/tiny/large-input windows not listed here remain approximately 0.50 / 0.50 for the inverse functions and single-argument logarithms.

API Sampled range Before After
Asin x = +/- (1 - delta) 5.18e+11 0.49
Acos x = +/- (1 - delta) 5.18e+28 0.49
AsinPi x = +/- (1 - delta) 1.65e+12 0.49
AcosPi x = +/- (1 - delta) 1.65e+29 0.50
Atanh x = +/- (1 - delta) 2.59e+27 0.50
Acosh x = 1 + delta 5.39e+27 0.50
Log x = 1 +/- delta 1.14e+29 0.50
Log2 x = 1 +/- delta 1.50e+29 0.50
Log10 x = 1 +/- delta 4.50e+29 0.50
LogP1 x = -1 + delta 5.18e+27 0.50
Log2P1 x = -1 + delta 7.48e+26 0.49
Log10P1 x = -1 + delta 2.25e+27 0.38
Pow Near-one bases; exponents targeting ordinary, subnormal, and large finite results 8.43e+32 2.37
Pow Positive bases in 0.1..10, same output-window sampling 8.45 1.47

Log(x, base) samples used bases 0.5, 2, 10, and the adjacent representable values below/above 1, over tiny, near-one, ordinary, and large positive arguments. Keeping both logarithms and the division wide reduces the sampled maxima to approximately 0.49999643, 0.50000000, and 0.50002430 ULP for Decimal32/64/128 respectively. The Decimal128 result is slightly above half an ULP; the table's 0.50 is not a correct-rounding claim.

Pi reduction ranges: Decimal128

API Sampled range Before After
SinPi Ordinary values in -1000..1000 1.65 0.50
CosPi Ordinary values in -1000..1000 1.26 0.50
TanPi Ordinary values in -1000..1000 7.51 0.50
SinPi, CosPi Beside integers/half-integers, down to the representable spacing 1.63e+29 0.48
TanPi Beside integers/half-integers, excluding exact poles 1.65e+29 0.49
SinPi, CosPi Full-precision fractional values with decimal exponents 3, 17, and 32 1.60e+29 0.50
TanPi Same large fractional inputs 2.20e+29 0.50

The corresponding SinCosPi components have the same errors. Tiny-input maxima remain approximately 0.50 / 0.50; the actual largest post-change sine/tangent error is slightly above half an ULP, so this is not a correct-rounding guarantee.

Base-two exponential ranges: Decimal128

Here u = x * log10(2), the decimal exponent of the mathematical 2^x before subtracting one. Each row takes the maximum across Exp2 and Exp2M1.

Sampled range Before After
Tiny inputs and abs(x) < 1 4.55e+14 0.50
Subnormal exponential outputs, u in -6177..-6143 3.62e+17 0.50
Negative normal exponential outputs, u in -6143..-1 4.28e+18 2.48
u in 0..100 6.95e+16 0.51
u in 100..1000 7.18e+17 0.69
u in 1000..6144 4.47e+18 2.57
Last finite decade, u in 6144..6145 4.68e+18 2.56
Additional probes immediately beside the overflow boundary 4.68e+18 2.84

The last row excludes cases where the baseline overflowed despite a finite mathematical result; the new build returns finite results for those probes.

Smaller-format and subnormal improvements

Format / API Sampled range Before After
Decimal32 Exp, Exp2 Subnormal exponential outputs 0.53 0.50
Decimal32 Exp10 Subnormal exponential outputs 0.54 0.50
Decimal64 AsinPi Tiny inputs spanning the negative exponent range 0.55 0.50
Decimal64 Exp2 Large negative inputs producing normal results 0.70 0.50
Decimal64 Exp2, Exp2M1 Last finite output decade 0.76 0.50
Decimal64 Exp10 Subnormal exponential outputs 0.52 0.50
Decimal128 Exp10 Subnormal exponential outputs 0.52 0.50

The remaining exponential families retain approximately the same overall maxima: Decimal128 Exp/ExpM1 reach 1.19/1.31, and Exp10/Exp10M1 reach 1.54 at extreme exponents. Fixing final subnormal rounding does not remove all conversion/approximation error.

Exact results and remaining limitations

  • Exact Pi zeros are now preserved for alternate decimal representations in all three formats. For example, Decimal32 SinPi(1000.000) previously returned approximately 9.45e-36; it now returns zero. These are reported as exact-result corrections, not enormous ULP figures computed using the minimum subnormal spacing at zero.
  • Alternate representations of 1 retain the correct inverse-function endpoints and poles. For example, Decimal64/128 Acosh(1.0000000) no longer returns NaN.
  • The original issue's Decimal128 Acos(1 - 1e-30) improves from 2.24e+24 to 0.07 ULP, and Asin(1 - 1e-30) from 2.24e+09 to 0.36 ULP.
  • This does not promise universal correct rounding or a universal two-ULP bound. Extreme-exponent errors remain, as shown above.
  • A separate pre-existing cohort-dependent overflow issue remains: Decimal128 Exp10 and Exp10M1 of 6145.000000000000000000000000000000 return MaxValue instead of infinity, whereas the representation 6145 returns infinity. This was reproduced on both the baseline and this change and is outside this PR.

Performance and size

Local BenchmarkDotNet measurements used Windows x64 / Ryzen 9 7950X. The initial accuracy fixes, compared with the original Release baseline, generally had roughly 0-5% overhead for exact inputs and 7-22% for inexact inverse/hyperbolic and LogP1 cases requiring decimal residual preservation. These are not claimed to be performance-neutral; the later optimizations recover work in specific paths rather than establishing a uniform end-to-end speedup.

Incremental measurements of the later optimizations showed approximately 8-15% faster ordinary SinCosPi and 10-23% faster TanPi from paired evaluation. Wide log-base evaluation improved sampled Decimal128 cases by roughly 11-23%. These comparisons use the preceding implementation, not the original pre-PR baseline.

The exact-result shortcuts showed the following incremental timings (ns/op, before / after), with two launches and fixed processor affinity:

Case Decimal32 Decimal64 Decimal128
RootN(0.7, 1) 384.36 / 6.01 389.04 / 6.41 388.54 / 8.74
TanPi(1000.000) 38.23 / 7.84 38.03 / 8.21 39.36 / 9.61

The measurements are mixed outside the targeted paths: ordinary RootN changes were small, and half-integer TanPi was about 1 ns slower for Decimal32/64. Earlier exact-result optimizations also showed small ordinary sine/cosine regressions. No universal performance improvement is claimed. No managed allocations were observed.

The final measured Release CoreLib file is 18,305,024 bytes, versus 18,300,928 bytes for the original baseline (+4,096 bytes). No large lookup tables were added.

Validation

  • Checked and Release CoreLib builds, including refreshed library testhosts.
  • Full System.Runtime.Tests passed after the product changes and test consolidation. After removing three redundant test rows, the final Decimal32/64/128 test run passed 7,407 tests, with 69 skipped and zero failures.
  • Strengthened the existing log-base accuracy tests with decimal oracles rather than adding parallel test methods. The existing Log(2.5, 3) case now detects the old two-ULP error in every format; all three fail on the pre-wide-log baseline and pass with this change.
  • Focused raw-bit tests cover signed-zero log-base results and the normal/subnormal result cohorts of the RootN(x, 1) shortcut. Existing endpoint, subnormal, Pi-cohort, and degree-boundary coverage is retained.
  • 211,483 finite-reference sampling evaluations across the two range sweeps described above; subsequent performance-only simplifications preserved all bits in a 224,518-evaluation comparison against the preceding optimized build.
  • 4,924 Intel DFP Update 3 vectors checked during the initial accuracy work; all 1,386 sampled exact-operation vectors remained bit-identical.

Execution and performance measurements were Windows x64 only. The changes are in shared CoreLib source.

Note

This PR description was drafted with GitHub Copilot.

Preserve decimal residuals and Pi reductions, correct the base-two exponential constant, and round subnormal results once. Skip residual conversion for verified exact dyadic inputs and unaffected logarithm intervals.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings September 14, 2026 23:36
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/area-system-numerics
See info in area-owners.md if you want to be subscribed.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Add regression coverage for Exp2M1, alternate π integer/half-integer cohorts, and high-precision RootN inputs.

Get a fresh assessment by requesting another Copilot review.

Review tier: Lite
Findings: 1 Low severity

Open (1)
What changed in this PR

Improves Decimal32/64/128 transcendental accuracy through decimal-preserving reductions, improved π handling, corrected exponential constants, and final rounding fixes.

Changes:

  • Preserves residual precision for inverse, logarithmic, hyperbolic, power, and root operations.
  • Adds decimal π reduction and cohort-aware exact results.
  • Corrects Exp2/Exp2M1 handling and subnormal rounding.
  • Expands accuracy and regression tests.
File Description
src/​libraries/​System.Runtime/​tests/​System.Runtime.Tests/​System/​Decimal64Tests.cs Adds Decimal64 accuracy and subnormal regressions.
src/​libraries/​System.Runtime/​tests/​System.Runtime.Tests/​System/​Decimal32Tests.cs Adds Decimal32 accuracy and rounding regressions.
src/​libraries/​System.Runtime/​tests/​System.Runtime.Tests/​System/​Decimal128Tests.cs Adds broad Decimal128 precision, boundary, and cohort coverage.
src/​libraries/​System.Private.CoreLib/​src/​System/​Number.DecimalIeee754.Transcendental.cs Integrates residual-preserving transcendental paths.
src/​libraries/​System.Private.CoreLib/​src/​System/​Number.DecimalIeee754.DiyFp128Pow.cs Improves near-one power reduction.
src/​libraries/​System.Private.CoreLib/​src/​System/​Number.DecimalIeee754.DiyFp128PiTrig.cs Implements decimal π reduction and exact-result handling.
src/​libraries/​System.Private.CoreLib/​src/​System/​Number.DecimalIeee754.DiyFp128InvTrig.cs Preserves inverse-trigonometric residuals.
src/​libraries/​System.Private.CoreLib/​src/​System/​Number.DecimalIeee754.DiyFp128InvHyper.cs Preserves inverse-hyperbolic residuals.
src/​libraries/​System.Private.CoreLib/​src/​System/​Number.DecimalIeee754.DiyFp128Exp.cs Reuses the accurate ln(2) constant.
src/​libraries/​System.Private.CoreLib/​src/​System/​Number.DecimalIeee754.DiyFp128Conversions.cs Adjusts final decimal rounding and subnormal handling.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 15, 2026 00:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

One or more issues must be addressed before approval.

Review tier: Lite
Findings: None

Resolved since last review (1)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Avoid evaluating the Pi sine and cosine independently

src/​libraries/​System.Private.CoreLib/​src/​System/​Number.DecimalIeee754.DiyFp128PiTrig.cs:137

DiyFp128SinCosPi calls DiyFp128SinPi and DiyFp128CosPi, so each call multiplies the reduced value by pi and runs a separate trig polynomial. This makes both SinCosPi and TanPi do the expensive reduction/evaluation twice despite the paired API and the existing DiyFp128SinCosPair helper; evaluate the angle once and map the paired sine/cosine to the octant instead.

…uracy

Keep log-base intermediates in working precision and share reduced Pi evaluation. Reuse scaling multipliers and avoid unnecessary exponent classification, degree conversion, and exact-result computation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 15, 2026 05:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The changes require final human review because they are too complex or risky for automated approval.

Review tier: Lite
Findings: None

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[.NET 11] Decimal128.Acos and Decimal128.Asin lose substantial accuracy near ±1

3 participants