Add interval mul/div support for Float64 - #10409
Conversation
|
Thanks - I'll try and find time for this but I need to focus on performance and bugfixes first |
TIA. If you need any help on other "performance and bugfixes" issue, please let me know, I would like to take a stab at them 😄 |
We are always looking for help reviewing other PRs (ensure test coverage, make sure you can unerstand the change, see if you can find any logic holes, etc) 🙏 |
| factor_scalar, | ||
| { | ||
| // Float scaling can create components that narrower interval units cannot store. | ||
| let interval = T::to_month_day_nano(interval); |
There was a problem hiding this comment.
i dont think we should have this conversion logic in arrow-rs; we should probably support only IntervalMonthDay explicitly and expect callers (e.g. datafusion) to coerce to this type themselves if they want to use this kernel. i think that has been the precedent for other kernels in arrow-rs
There was a problem hiding this comment.
Thanks. I agree with caller-side coercion for the numeric operand: callers can coerce multiplication and division factors to Int64 or Float64, so arrow-arith does not need to support every numeric type.
I’m less clear that the same precedent requires coercing the interval representation. #10336 directly supports Interval(YearMonth), Interval(DayTime), and Interval(MonthDayNano) for multiplication by Int64; this PR follows the same input coverage. For Float64, the result must be MonthDayNano because fractional months and days can cascade into smaller components, and widening YearMonth/DayTime to MonthDayNano is lossless.
If callers should also normalize the interval representation, should the same rule apply to #10336, or is the distinction that integer multiplication preserves the input representation while floating-point arithmetic widens it?
Could you also point me to a specific kernel or PR establishing this precedent for coercion between related Arrow types? The precedent discussed in #6906 appears to concern heterogeneous numeric operands; limiting factors to Int64 or Float64 already follows that boundary.
There was a problem hiding this comment.
since i64 multiplication preserves input type, its not doing any coercion so its fine as is. f64 requires converting to monthdaynano so this is something the caller should cast to instead of us casting inside an arithmetic kernel
Could you also point me to a specific kernel or PR establishing this precedent for coercion between related Arrow types? The precedent discussed in #6906 (comment) appears to concern heterogeneous numeric operands; limiting factors to
Int64orFloat64already follows that boundary.
the precedent is just the existing codebase as a whole; im not aware of any arithmetic kernels that cast the input types (apart from things like decimal widening, etc.) though i may be wrong
just in general, this type of input casting is best left to the caller to opt in to, and at the arrow-rs level its simpler to just support the exact types we actually support (in this cast f64 * monthdaynano)
There was a problem hiding this comment.
since i64 multiplication preserves input type, its not doing any coercion so its fine as is. f64 requires converting to monthdaynano so this is something the caller should cast to instead of us casting inside an arithmetic kernel
just in general, this type of input casting is best left to the caller to opt in to, and at the arrow-rs level its simpler to just support the exact types we actually support (in this cast f64 * monthdaynano)
sounds fair, let me remove the conversion and only keep the Interval(MonthDayNano) * f64 input. Thanks for the explaination!
Which issue does this PR close?
Rationale for this change
#10336 added interval multiplication by
Int64. Floating-point multiplication and division require preserving calendar components rather than flattening the entire interval into a duration.This PR adapts DuckDB's
INTERVAL * DOUBLEbehavior. Whole months and days remain calendar components, while fractional months cascade to days using 30 days per month and fractional days cascade to nanoseconds using 24 hours per day.DuckDB explicitly attributes its implementation to PostgreSQL's
interval_mul:What changes are included in this PR?
Add checked arithmetic for:
Interval(YearMonth | DayTime | MonthDayNano) * Float64Float64 * Interval(YearMonth | DayTime | MonthDayNano)Interval(YearMonth | DayTime | MonthDayNano) / Float64All results use
Interval(MonthDayNano).YearMonthandDayTimeinputs are widened first so fractional months and sub-millisecond results are not truncated.Are these changes tested?
Float64factors, preserving exact nanosecondsInterval / Int64,Float64 / Interval, wrapping multiplication, and additional numeric factor types are not added.Are these changes tested?
Yes. Tests cover all three interval units, both multiplication operand orders, division, array and scalar operands, null propagation, component cascading, integral-factor precision, negative values, ties-to-even rounding, non-finite values, division by zero, and overflow.
cargo fmt --all -- --check cargo test -p arrow-arith cargo clippy -p arrow-arith --all-targets --all-features -- -D warningsAre there any user-facing changes?
Yes. The checked
mulanddivkernels now accept the combinations listed above. Floating-point interval arithmetic always returnsInterval(MonthDayNano).There are no public API signature changes.