Skip to content

Implement Spark-compatible hypot function #23770

Description

@KarpagamKarthikeyan

Is your feature request related to a problem or challenge?

The datafusion-spark function library (tracked in #15914) is building out Spark SQL–compatible functions. Spark provides hypot(expr1, expr2), which returns sqrt(expr1² + expr2²) computed without intermediate overflow or underflow.

This function is not yet implemented in datafusion-spark. There is an auto-generated test stub at datafusion/sqllogictest/test_files/spark/math/hypot.slt, but its query is commented out and there is no implementation, so hypot is not available in the Spark dialect.

Describe the solution you'd like

Add a Spark-compatible hypot scalar function to datafusion-spark, following the existing math function pattern (e.g. datafusion/spark/src/function/math/pow.rs):

  • Add SparkHypot implementing ScalarUDFImpl in datafusion/spark/src/function/math/hypot.rs.
  • Signature: two Float64 arguments → Float64.
  • Back it with Rust's f64::hypot, which uses the same overflow-safe approach as Java/Spark's Math.hypot.
  • Register it in datafusion/spark/src/function/math/mod.rs.
  • Enable the existing hypot.slt sqllogictest (uncomment the query and add expected results).

Expected behavior (matches PySpark 3.5.5):

SELECT hypot(3, 4);
-- 5.0  (double)

### Describe alternatives you've considered

A naive `sqrt(a*a + b*b)` implementation was considered but rejected: it can overflow/underflow for large or small magnitudes, whereas `f64::hypot` is the standard overflow-safe implementation and matches Spark's semantics

### Additional context

Part of #15914 (Complete `datafusion-spark` Spark Compatible Functions). I'd like to implement this myself and will open a PR shortly.

Metadata

Metadata

Labels

enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions