Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ jobs:
components: clippy, rustfmt
- run: cargo fmt --all -- --check
- run: cargo clippy --workspace -- -D warnings
- run: cargo clippy --workspace --features seuil/bigdecimal -- -D warnings
- run: cargo build --workspace

test:
Expand All @@ -30,6 +31,7 @@ jobs:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- run: cargo test --workspace
- run: cargo test --workspace --features seuil/bigdecimal

vopr:
name: VOPR Campaign (10K seeds)
Expand Down
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,15 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Added

- Opt-in `bigdecimal` feature for decimal arithmetic, numeric functions, and
arbitrary-precision JSON numbers. Numeric functions also accept decimal strings
and return strings when given string operands (issue #1).
- Precision regression tests and CI coverage with the feature enabled and disabled.

## [0.1.0] - 2026-04-04

### Added
Expand Down
56 changes: 56 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,9 @@ let result = expr.evaluate(&data)?;
assert_eq!(result, serde_json::json!(300.0));
```

For exact decimal amounts such as `"100.30"`, see the optional
[BigDecimal example](#decimal-arithmetic-optional) below.

## Why seuil?

- **99.6% JSONata spec compliance** — 1,027 of 1,031 official test cases pass
Expand All @@ -53,6 +56,59 @@ seuil = "0.1"
serde_json = "1"
```

## Decimal arithmetic (optional)

Enable the `bigdecimal` feature to preserve decimal precision. Until this feature
is released on crates.io, use the Git revision containing the implementation:

```toml
[dependencies]
seuil = { git = "https://github.com/copyleftdev/seuil-rs", rev = "0d8d280d8da4384e2c1625c21d8512a1aa0ec7ff", features = ["bigdecimal"] }
serde_json = "1"
```

This also enables `serde_json/arbitrary_precision`. Parse JSON text with
`evaluate_str` or `serde_json::from_str` to retain every input digit; a Rust `f64`
literal passed to `json!` has already been rounded before Seuil receives it.

```rust
use seuil::Seuil;

fn main() -> seuil::Result<()> {
let expr = Seuil::compile("orders[status='paid'].amount ~> $sum()")?;
let data = serde_json::json!({
"orders": [
{"status": "paid", "amount": "100.30"},
{"status": "pending", "amount": "50.40"},
{"status": "paid", "amount": "200.25"}
]
});

let result = expr.evaluate(&data)?;
assert_eq!(result, serde_json::json!("300.55")); // 100.30 + 200.25
Ok(())
}
```

Numeric functions accept decimal strings as well as numbers. If an operand is a
string, the numeric result is a normalized decimal string; number-only inputs
produce JSON numbers. `$number` explicitly converts a string to a number.
Ordinary strings retain their type outside numeric functions. Arithmetic
operators still require numbers: use `$number` to convert a string first.

Addition, subtraction, multiplication, remainder, comparisons, and aggregates
use decimal arithmetic. Division, averages, square roots, and powers use
BigDecimal's precision rules (normally 100 significant digits for inexact
results). `$round` uses half-even rounding and accepts precision from -10,000 to
10,000. `$power` supports integer and half-integer exponents; other fractional
exponents return `D3061` instead of falling back to floating point. `$random`
retains the existing environment-provided random number behavior.

The feature intentionally extends JSONata's numeric semantics: decimal strings
are accepted, numbers can exceed the `f64` range, and `$string` preserves numeric
precision instead of truncating to 15 digits. The default feature set retains
the existing floating-point behavior.

## API

```rust
Expand Down
5 changes: 5 additions & 0 deletions crates/seuil/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,12 @@ categories = ["parser-implementations", "encoding", "data-structures", "compiler
documentation = "https://docs.rs/seuil"
rust-version = "1.77.0"

[features]
default = []
bigdecimal = ["dep:bigdecimal", "serde_json/arbitrary_precision"]

[dependencies]
bigdecimal = { version = "0.4", optional = true }
bumpalo = { workspace = true }
hashbrown = { workspace = true }
indexmap = { workspace = true }
Expand Down
56 changes: 56 additions & 0 deletions crates/seuil/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,9 @@ let result = expr.evaluate(&data)?;
assert_eq!(result, serde_json::json!(300.0));
```

For exact decimal amounts such as `"100.30"`, see the optional
[BigDecimal example](#decimal-arithmetic-optional) below.

## Why seuil?

- **99.6% JSONata spec compliance** — 1,027 of 1,031 official test cases pass
Expand All @@ -53,6 +56,59 @@ seuil = "0.1"
serde_json = "1"
```

## Decimal arithmetic (optional)

Enable the `bigdecimal` feature to preserve decimal precision. Until this feature
is released on crates.io, use the Git revision containing the implementation:

```toml
[dependencies]
seuil = { git = "https://github.com/copyleftdev/seuil-rs", rev = "0d8d280d8da4384e2c1625c21d8512a1aa0ec7ff", features = ["bigdecimal"] }
serde_json = "1"
```

This also enables `serde_json/arbitrary_precision`. Parse JSON text with
`evaluate_str` or `serde_json::from_str` to retain every input digit; a Rust `f64`
literal passed to `json!` has already been rounded before Seuil receives it.

```rust
use seuil::Seuil;

fn main() -> seuil::Result<()> {
let expr = Seuil::compile("orders[status='paid'].amount ~> $sum()")?;
let data = serde_json::json!({
"orders": [
{"status": "paid", "amount": "100.30"},
{"status": "pending", "amount": "50.40"},
{"status": "paid", "amount": "200.25"}
]
});

let result = expr.evaluate(&data)?;
assert_eq!(result, serde_json::json!("300.55")); // 100.30 + 200.25
Ok(())
}
```

Numeric functions accept decimal strings as well as numbers. If an operand is a
string, the numeric result is a normalized decimal string; number-only inputs
produce JSON numbers. `$number` explicitly converts a string to a number.
Ordinary strings retain their type outside numeric functions. Arithmetic
operators still require numbers: use `$number` to convert a string first.

Addition, subtraction, multiplication, remainder, comparisons, and aggregates
use decimal arithmetic. Division, averages, square roots, and powers use
BigDecimal's precision rules (normally 100 significant digits for inexact
results). `$round` uses half-even rounding and accepts precision from -10,000 to
10,000. `$power` supports integer and half-integer exponents; other fractional
exponents return `D3061` instead of falling back to floating point. `$random`
retains the existing environment-provided random number behavior.

The feature intentionally extends JSONata's numeric semantics: decimal strings
are accepted, numbers can exceed the `f64` range, and `$string` preserves numeric
precision instead of truncating to 15 digits. The default feature set retains
the existing floating-point behavior.

## API

```rust
Expand Down
27 changes: 27 additions & 0 deletions crates/seuil/src/decimal.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
//! Decimal conversion shared by the opt-in evaluator and numeric functions.
use bigdecimal::BigDecimal;

use crate::evaluator::value::Value;
use crate::{Error, Result};

/// Parse the decimal text directly; never pass arbitrary-precision input through f64.
pub(crate) fn parse(text: &str) -> Result<BigDecimal> {
text.parse()
.map_err(|_| Error::D3137Error("Invalid decimal number".into()))
}

pub(crate) fn number(value: &Value<'_>) -> Result<BigDecimal> {
match value {
Value::Number(n) if n.is_finite() => parse(&n.to_string()),
Value::Decimal(n) => parse(n),
_ => Err(Error::D3137Error("Expected a finite decimal number".into())),
}
}

/// Keep the existing small-number representation when its decimal text is exact.
/// Larger values remain arena-owned text, so BigDecimal's heap allocations are
/// dropped after each operation rather than leaked inside the bump arena.
pub(crate) fn lossless_f64(value: &BigDecimal) -> Option<f64> {
let n = value.to_string().parse::<f64>().ok()?;
(n.is_finite() && parse(&n.to_string()).ok().as_ref() == Some(value)).then_some(n)
}
47 changes: 43 additions & 4 deletions crates/seuil/src/evaluator/engine.rs
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,8 @@ impl<'a, 'env> Evaluator<'a, 'env> {
AstKind::Bool(b) => Ok(self.val_bool(b)),
AstKind::String(ref s) => Ok(Value::string(self.arena, s)),
AstKind::Number(n) => Ok(Value::number(self.arena, n)),
#[cfg(feature = "bigdecimal")]
AstKind::Decimal(ref n) => Ok(Value::decimal(self.arena, crate::decimal::parse(n)?)),
AstKind::Regex(ref r) => Ok(self.arena.alloc(Value::Regex(r.clone()))),

AstKind::Block(ref exprs) => self.evaluate_block(exprs, input),
Expand Down Expand Up @@ -299,6 +301,10 @@ impl<'a, 'env> Evaluator<'a, 'env> {
let result = self.evaluate(value, input)?;
match result {
Value::Undefined => Ok(self.undefined()),
#[cfg(feature = "bigdecimal")]
Value::Decimal(_) => {
Ok(Value::decimal(self.arena, -crate::decimal::number(result)?))
}
Value::Number(n) if result.is_valid_number()? => {
Ok(Value::number(self.arena, -*n))
}
Expand Down Expand Up @@ -372,19 +378,36 @@ impl<'a, 'env> Evaluator<'a, 'env> {
let l = if lhs.is_undefined() {
return Ok(self.undefined());
} else if lhs.is_valid_number()? {
lhs.as_f64()
lhs
} else {
return Err(Error::T2001LeftSideNotNumber(node.span, op.to_string()));
};

let r = if rhs.is_undefined() {
return Ok(self.undefined());
} else if rhs.is_valid_number()? {
rhs.as_f64()
rhs
} else {
return Err(Error::T2002RightSideNotNumber(node.span, op.to_string()));
};

#[cfg(feature = "bigdecimal")]
{
let l = crate::decimal::number(lhs)?;
let r = crate::decimal::number(rhs)?;
if !matches!(op, BinaryOp::Divide | BinaryOp::Modulus) || r != 0 {
let result = match op {
BinaryOp::Add => &l + &r,
BinaryOp::Subtract => &l - &r,
BinaryOp::Multiply => &l * &r,
BinaryOp::Divide => &l / &r,
BinaryOp::Modulus => &l % &r,
_ => return Err(Error::D3137Error("Invalid decimal operator".into())),
};
return Ok(Value::decimal(self.arena, result));
}
}
let (l, r) = (l.as_f64(), r.as_f64());
let result = match op {
BinaryOp::Add => l + r,
BinaryOp::Subtract => l - r,
Expand Down Expand Up @@ -421,8 +444,10 @@ impl<'a, 'env> Evaluator<'a, 'env> {
}

if lhs.is_number() && rhs.is_number() {
let l = lhs.as_f64();
let r = rhs.as_f64();
#[cfg(not(feature = "bigdecimal"))]
let (l, r) = (lhs.as_f64(), rhs.as_f64());
#[cfg(feature = "bigdecimal")]
let (l, r) = (crate::decimal::number(lhs)?, crate::decimal::number(rhs)?);
return Ok(self.val_bool(match op {
BinaryOp::LessThan => l < r,
BinaryOp::LessThanEqual => l <= r,
Expand Down Expand Up @@ -949,6 +974,16 @@ impl<'a, 'env> Evaluator<'a, 'env> {

let ord = match (aa, bb) {
(Value::String(ref sa), Value::String(ref sb)) => sa.cmp(sb),
#[cfg(feature = "bigdecimal")]
(a, b) if a.is_number() && b.is_number() => {
match (crate::decimal::number(a), crate::decimal::number(b)) {
(Ok(a), Ok(b)) => a.cmp(&b),
(Err(e), _) | (_, Err(e)) => {
sort_error = Some(e);
std::cmp::Ordering::Equal
}
}
}
(Value::Number(na), Value::Number(nb)) => na.total_cmp(nb),
_ => {
sort_error = Some(Error::T2007CompareTypeMismatch(
Expand Down Expand Up @@ -1761,6 +1796,8 @@ fn simple_stringify<'a>(value: &'a Value<'a>, buf: &mut String) {
Value::Undefined => {}
Value::Null => buf.push_str("null"),
Value::Bool(b) => buf.push_str(if *b { "true" } else { "false" }),
#[cfg(feature = "bigdecimal")]
Value::Decimal(n) => buf.push_str(n),
Value::Number(n) => {
if n.is_nan() || n.is_infinite() {
buf.push_str("null");
Expand Down Expand Up @@ -1818,6 +1855,8 @@ mod tests {
Value::Null => serde_json::Value::Null,
Value::Bool(b) => serde_json::Value::Bool(*b),
Value::Number(n) => serde_json::json!(*n),
#[cfg(feature = "bigdecimal")]
Value::Decimal(n) => serde_json::from_str(n).unwrap(),
Value::String(ref s) => serde_json::Value::String(s.as_str().to_string()),
Value::Array(ref a, _) => {
let arr: Vec<serde_json::Value> = a.iter().map(|v| value_to_json(v)).collect();
Expand Down
8 changes: 8 additions & 0 deletions crates/seuil/src/evaluator/functions/array.rs
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,10 @@ pub fn fn_sort<'a>(context: FnContext<'a, '_>, args: &[&'a Value<'a>]) -> Result
let cmp_result = (context.apply_fn)(span, context.input, comparator, &[a, b])?;
// Comparator should return a number. Negative = a < b, positive = a > b.
// Or a boolean: true = swap (a > b).
#[cfg(feature = "bigdecimal")]
if let Value::Decimal(_) = cmp_result {
return Ok(crate::decimal::number(cmp_result)? > 0);
}
if let Value::Number(n) = cmp_result {
Ok(*n > 0.0)
} else if let Value::Bool(b) = cmp_result {
Expand All @@ -130,6 +134,10 @@ pub fn fn_sort<'a>(context: FnContext<'a, '_>, args: &[&'a Value<'a>]) -> Result
let sorted = merge_sort(
unsorted,
&|a: &'a Value<'a>, b: &'a Value<'a>| match (a, b) {
#[cfg(feature = "bigdecimal")]
(a, b) if a.is_number() && b.is_number() => {
Ok(crate::decimal::number(a)? > crate::decimal::number(b)?)
}
(Value::Number(a), Value::Number(b)) => Ok(a > b),
(Value::String(a), Value::String(b)) => Ok(a > b),
_ => Err(Error::D3070InvalidDefaultSort(Span::at(context.char_index))),
Expand Down
4 changes: 4 additions & 0 deletions crates/seuil/src/evaluator/functions/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@
pub mod array;
pub mod datetime;
pub mod hof;
#[cfg(not(feature = "bigdecimal"))]
pub mod numeric;
#[cfg(feature = "bigdecimal")]
#[path = "numeric_decimal.rs"]
pub mod numeric;
pub mod object;
pub mod string;
Expand Down
Loading
Loading