fix: restore full precision to erf's interval constants - #412
Conversation
`erf_impl` reconstructs `erfc(z)` for `z >= 0.5` as
`exp(-z^2)/z * (b + r(z))`, where `r` is a minimax rational fit and `b` is a
per-interval constant. All 13 of those constants carried only 10 significant
decimal digits:
0.3440242112 // interval [0.5, 0.75)
0.419990927 // interval [0.75, 1.25)
Solving for the `b` each interval's fit implies, at 60 digits, gives a value
that is constant to ~19 digits across the whole interval - so the rational fits
are good to ~1e-19 and the truncated constants were the only error source. The
recovered values turn out to be exactly `f32`-representable, i.e. Boost declared
them as `float` literals and the port that statrs inherits transcribed the
printed decimal:
0.3440242112 -> 0.3440242111682891845703125
0.419990927 -> 0.4199909269809722900390625
Also compensates the squaring: `z * z` is rounded before `exp` sees it, which
costs roughly `z^2` ulps (~460 by z = 27). A Dekker product recovers the
rounding error of the square exactly and folds it back in as
`exp(-z^2) = exp(-sq) * (1 - err)`. (Dekker rather than `f64::mul_add`, which
falls back to a slow software FMA on targets without the instruction.)
Measured against mpmath at 45 dps over a 700-point sweep plus every interval
boundary:
before after
erf ~1e-10 rel 0.24 median / 1.1 max ulp
erfc ~1e-10 rel 0.52 median / 2.8 max ulp
Normal::cdf ~5e-11 rel 0.32 median
This propagated to everything built on `erfc` - `Normal` cdf/sf, `LogNormal`,
`Levy`. The test suite was masking it with `epsilon = 1e-9`/`1e-11`
tolerances, now tightened to 4 ulp.
Six reference literals in the tests were themselves wrong, having been fitted to
the old output; they are replaced with mpmath values. Three are `LogNormal::sf`
expectations (off by up to 5.1e-12) and three are `erfc_inv` expectations - one
of which, `erfc_inv(1e-10)`, was off by 8.8e-9 and had needed
`epsilon = 1e-7` to pass. `erf_inv`/`erfc_inv` themselves were already good to
<2 ulp; only the expectations were wrong.
|
isnt better keep erf fixes in https://github.com/rust-lang/compiler-builtins/tree/main/libm/src/math ? |
|
Fixing it there wouldn't change anything here, because statrs doesn't use libm's
The constants in this PR are interval boundaries in that implementation — the cutoffs selecting which rational approximation to use, e.g. Switching statrs to libm's |
erf_implreconstructserfc(z)forz >= 0.5asexp(-z²)/z · (b + r(z)), whereris a minimax rational fit andba per-interval constant. All 13bconstants carry only 10 significant decimal digits:Solving for the
beach interval's fit implies (at 60-digit precision) gives values constant to ~19 digits across each interval — so the rational fits are good to ~1e-19 and the truncated constants were the only error source. The recovered values are exactlyf32-representable: Boost declared them asfloatliterals and the port that statrs inherits transcribed the printed decimal. This PR restores the full values, e.g.0.3440242112→0.3440242111682891845703125.It also compensates the squaring:
z * zis rounded beforeexpsees it, costing ~z² ulps (~460 by z = 27). A Dekker product recovers the rounding error of the square exactly (Dekker rather thanf64::mul_add, which is a slow software FMA on targets without the instruction, e.g. baseline x86-64).Measured against mpmath at 45 dps over a 700-point sweep plus every interval boundary:
erferfcNormal::cdfThis propagates to everything built on
erfc—Normalcdf/sf,LogNormal,Levy. The suite masked it withepsilon = 1e-9/1e-11tolerances, now tightened to 4 ulp of each expected value.Six reference literals in the tests were themselves fitted to the old output and are replaced with mpmath values: three
LogNormal::sfexpectations (off by up to 5.1e-12) and threeerfc_invexpectations — one of which,erfc_inv(1e-10), was off by 8.8e-9 and had neededepsilon = 1e-7to pass.erf_inv/erfc_invthemselves were already ≤2 ulp; only the expectations were wrong.🤖 Generated with Claude Code