Fix directed powi OOM on primitive-float underflow

[cmpute]

Summary

DirectedPowI can drive fbig_rat to allocate gigabytes for a result whose magnitude is representable in dashu but astronomically beyond the primitive-float range. dashu returns that result promptly and correctly; the adapter then materializes its exact rational with an unbounded shift and exhausts memory.

• Reproducer: DirectedPowI::<f64, IBig>::eval(5.0, &IBig::from(-(1i64 << 31)), Direction::Up). 5.0^(-2³¹) ≈ 2^-5.0e9; f64::powi underflows this to 0.0 (finite), so the existing native guard does not fire; eval_adaptive calls dashu powi, which correctly returns the tiny representable FBig (53-bit significand, repr.exponent() ≈ -5.0e9); fbig_rat then runs UBig::from(1u8) << 5.0e9 → OOM (the fuzzer saw RSS climb to 600 MB+ before the kill). Reached via directed_unary / opendp_sequences.

Root cause

directed_powi!'s native branch catches overflow but not underflow (src/backend/dashu.rs):

if let Ok(small_exponent) = i32::try_from(exponent) {
    let native = base.powi(small_exponent);
    if !native.is_finite() {                       // catches ±∞ (overflow) only
        return extreme_power::<$ty>(native.is_sign_negative(), true, direction);
    }
} else if base.abs() == 1.0 { … }

For an i32 exponent whose exact result is below the smallest subnormal, f64::powi returns 0.0 — finite, so the guard does not fire and execution falls through to eval_adaptive → fbig_rat (which shifts by the huge exponent). The else arms only run when the exponent does not fit i32, so they never see this case.

fbig_rat materializes the exact rational with an unbounded shift:

RBig::from_parts(significand, UBig::from(1u8) << ((-exponent) as usize)) // unbounded

dashu is innocent here: FBig::<Up>::try_from(5.0).powi(IBig::from(-(1i64<<31))) returns in ~90 µs with a 53-bit significand and repr.exponent() ≈ -5.0e9 — a valid, representable value. The OOM is purely the adapter's unbounded materialization.

Fix

Resolve underflow the same way overflow is already resolved — a nonzero base whose power underflows to ±0 is outside the primitive-finite range, so return the structural directed endpoint from extreme_power instead of computing it exactly:

if native == 0.0 && base != 0.0 {
    let negative = base.is_sign_negative() && odd;
    return extreme_power::<$ty>(negative, /*overflow=*/ false, direction);
}

This mirrors the existing outside-i32 branch's magnitude_log2 outside [-1200, 1200] structural resolution, extended to the i32-exponent underflow case the native branch currently misses. (base == 0.0 is excluded: 0**n == 0 exactly is cheap.) extreme_power(neg, overflow=false, …) yields the directed underflow endpoints: smallest positive/negative subnormal under outward modes, ±0 under inward/nearest.

A defensive bound on fbig_rat's shift is deliberately not included — it's belt-and-suspenders for other hypothetical huge-exponent paths, and this fix makes the known path unreachable; the exact policy (clamp vs. sentinel vs. panic) is better settled separately.

Why the adapter, not dashu

dashu's FBig is arbitrary-precision with an isize exponent range (±~9.2e18 on 64-bit). A value like 2^-5e9 is genuinely representable and finite; returning it is correct. Capping dashu's notion of "representable" to what a primitive float can materialize would break arbitrary-precision users. The adapter, which bridges to fixed-width floats, is the right place to bound the materialization.

Verification

• The OOM reproducer now runs in 0 ms (no allocation, no OPENDP_NUM_VIOLATION) under a tight rss_limit_mb=512.
• A 126-case boundary sweep — bases {0.5, 0.999999, 1.0000001, 1.5, 2, 5, 10} × exponents {±2³¹, ±2⁴⁰, i32::MIN, i32::MAX, ±10000} × {Down, Up, Nearest} — agrees with the MPFR oracle on every case (0 mismatches), in ~635 µs total.
• Explicit endpoint check: powi(5, -2³¹) → Up = 0x0000000000000001 (min pos subnormal), Down/Nearest = 0.

🤖 Generated with Claude Code