test_interpolate_by.test_interpolate_vs_numpy fails often

[anath2]

Checks

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of Polars.

Reproducible example

pytest -vm "" tests/unit/operations/test_interpolate_by.py::test_interpolate_vs_numpy

Log output

____________________________________ test_interpolate_vs_numpy ____________________________________
tests/unit/operations/test_interpolate_by.py:166: in test_interpolate_vs_numpy
    def test_interpolate_vs_numpy(data: st.DataObject, x_dtype: pl.DataType) -> None:
tests/unit/operations/test_interpolate_by.py:230: in test_interpolate_vs_numpy
    assert_series_equal(result, expected)
polars/_utils/deprecation.py:119: in wrapper
    return function(*args, **kwargs)
E   AssertionError: Series are different (value mismatch)
E   [left]:  [164583760.0, 1.7881393432617188e-07, 0.0]
E   [right]: [164583760.0, 1.65617217337068e-07, 0.0]
E   Falsifying example: test_interpolate_vs_numpy(
E       data=data(...),
E       x_dtype=Float64,  # or any other generated value
E   )
E   Draw 1: shape: (4, 2)
E   ┌────────────┬─────────────┐
E   │ ts         ┆ value       │
E   │ ---        ┆ ---         │
E   │ f64        ┆ f64         │
E   ╞════════════╪═════════════╡
E   │ -9.9968e14 ┆ 1.6458376e8 │
E   │ 0.0        ┆ null        │
E   │ 0.0        ┆ null        │
E   │ 1.0        ┆ 0.0         │
E   └────────────┴─────────────┘
E   Explanation:
E       These lines were always and only run by failing examples:
E           /xxx/py-polars/polars/testing/asserts/series.py:327
E   
E   You can reproduce this example by temporarily adding @reproduce_failure('6.125.1', b'AXicc2R0ZGHQOMyTubKoZWuCBgMUIBj2HyAMR0YNx8XzNi0AMUGQEa6EAQCkrwpt') as a decorator on your test case
===================================== short test summary info =====================================
FAILED tests/unit/operations/test_interpolate_by.py::test_interpolate_vs_numpy - AssertionError: Series are different (value mismatch)

Issue description

The test test_interpolate_vs_numpy at tests/unit/operations/test_interpolate_by.pyfails sometimes. Likely due to fp precision issue.
Maybe more lenient tolerance is needed for comparison against np.interp for floating point values.

Expected behavior

Test result should be deterministic.

Installed versions

--------Version info---------
Polars:              1.27.1
Index type:          UInt32
Platform:            macOS-14.4.1-arm64-arm-64bit-Mach-O
Python:              3.13.1 (main, Dec  3 2024, 17:59:52) [Clang 16.0.0 (clang-1600.0.26.4)]
LTS CPU:             False

[mcrumiller]

In the CI, np.interp(x, xp, yp) is generating the result [297533270.0, -1.4901161193847656e-08, 0.0] whereas when I run locally it generates the result [297533270.0, 0.0, 0.0]. The np.interp function is a compiled numpy function so I think the key to solving this will be getting the correctly compiled version. I've been unsuccessful here--I've run on the same python 3.12.9 using numpy==2.1.2 but I am still getting the same local result, which does not trigger the pytest error.

[mcrumiller]

It looks like the CI uses pytorch as the index-url, and the exact numpy package appears to be https://download.pytorch.org/whl/numpy-2.1.2-cp312-cp312-macosx_14_0_arm64.whl.metadata (I'm using this failed example). I'm on WSL so I can't repro the macosx installation unfortunately.

I am not sure why we use pytorch instead of Pypi for these packages. There is a more recent version of numpy available (2.2.5) so it's possible that upgrading may resolve the issue.

[mcrumiller]

Ok, minor update: it looks like when --extra-index-url is provided it is given higher priority than the default index. So, since --extra-index-url is specified in requirements-ci.txt, then any package found at pytorch is used instead of PyPi, which includes numpy in this case.

[anath2]

In the CI, np.interp(x, xp, yp) is generating the result [297533270.0, -1.4901161193847656e-08, 0.0] whereas when I run locally it generates the result [297533270.0, 0.0, 0.0]. The np.interp function is a compiled numpy function so I think the key to solving this will be getting the correctly compiled version. I've been unsuccessful here--I've run on the same python 3.12.9 using numpy==2.1.2 but I am still getting the same local result, which does not trigger the pytest error.

FWIW, I can reproduce the same error values as CI on my Apple Silicon Mac, numpy==2.1.2 with:

@reproduce_failure('6.131.6', b'AXicc2R0ZGXQYIACLIwdAlDGAQ4Iw5ERLukIgYwajht3/w8DiQAAx1MHeg==')

Edit: I get the same values after upgrading to numpy==2.2.5

[mcrumiller]

@anath2 are these enough sig figs to reproduce? Here is the test with the inputs provided from the failed case I linked to:

import numpy as np
import polars as pl

dataframe = (
    pl.DataFrame({
        "ts": pl.Series([0.0, 0.0, 0.0, -1.1755e-38, -3.0], dtype=pl.Float64),
        "value": pl.Series([0.0, None, None, None, 2.9753327e8], dtype=pl.Float64),
    })
    .fill_nan(None)
    .unique("ts")
    .sort("ts")
)

result = dataframe.select(pl.col("value").interpolate_by("ts"))["value"]

mask = dataframe["value"].is_not_null()

np_dtype = "float64"
x = dataframe["ts"].to_numpy().astype(np_dtype)
xp = dataframe["ts"].filter(mask).to_numpy().astype(np_dtype)
yp = dataframe["value"].filter(mask).to_numpy().astype("float64")
interp = np.interp(x, xp, yp)
# Polars preserves nulls on boundaries, but NumPy doesn't.
first_non_null = dataframe["value"].is_not_null().arg_max()
last_non_null = len(dataframe) - dataframe["value"][::-1].is_not_null().arg_max()  # type: ignore[operator]
interp[:first_non_null] = float("nan")
interp[last_non_null:] = float("nan")
expected = dataframe.with_columns(value=pl.Series(interp, nan_to_null=True))["value"]

print(f"result: {result}")
print(f"expected: {expected}")

[anath2]

@mcrumiller Yep I can reproduce:
Output:

result: shape: (3,)
Series: 'value' [f64]
[
	2.9753327e8
	0.0
	0.0
]
expected: shape: (3,)
Series: 'value' [f64]
[
	2.9753327e8
	-1.4901e-8
	0.0
]

from polars.testing import assert_series_equal
assert_series_equal(result, expected)

Produces the exact error

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
Cell In[24], line 2
      1 from polars.testing import assert_series_equal
----> 2 assert_series_equal(result, expected)

File /polars/py-polars/polars/_utils/deprecation.py:119, in deprecate_renamed_parameter.<locals>.decorate.<locals>.wrapper(*args, **kwargs)
    114 @wraps(function)
    115 def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
    116     _rename_keyword_argument(
    117         old_name, new_name, kwargs, function.__qualname__, version
    118     )
--> 119     return function(*args, **kwargs)

    [... skipping hidden 3 frame]

File /polars/py-polars/polars/testing/asserts/utils.py:12, in raise_assertion_error(objects, detail, left, right, cause)
     10 __tracebackhide__ = True
     11 msg = f"{objects} are different ({detail})\n[left]:  {left}\n[right]: {right}"
---> 12 raise AssertionError(msg) from cause

AssertionError: Series are different (value mismatch)
[left]:  [297533270.0, 0.0, 0.0]
[right]: [297533270.0, -1.4901161193847656e-08, 0.0]

The assertion fails at assert_series_equal(result, expected, atol=1e-8) but passes at atol=1e-7

[mcrumiller]

@anath2 what's the result of sys.platform for you? Is it darwin?

[anath2]

@anath2 what's the result of sys.platform for you? Is it darwin?

Yes that's correct.