Provide testing macros/functions like assert_close?

[joshburkart]

In Python, the NumPy library provides nice array-comparison functions for unit tests etc., which I make heavy use of:

import numpy as np

a = np.arange(5, dtype=np.float32) + 1

np.testing.assert_allclose(a, 1e-10 + np.sqrt(a**2))  # Succeeds.
np.testing.assert_allclose(a, 1e-1 + np.sqrt(a**2))  # Fails.

Would it be useful/appropriate to add some of these to this library? I could be interested in working on it, but thought I'd check first.

(Seems like some such functions are used internally, but don't seem to be exported for external use.)

[bluss]

The reason to not export them publically is simple, then I'd need to ensure they have high quality.

Having these as macros sounds good, especially for the usual assertion as a macro benefits and possibly supporting different number of arguments or other options.

[bluss]

I think it would be important to support relative and absolute error, and weirdly enough that's not even a function anywhere. But it exists at this point in the code (again, of dubious quality)

https://github.com/bluss/rust-ndarray/blob/8218867ab15a9b05a7dc43b84165d24630bdbd16/numeric-tests/tests/accuracy.rs#L241-L250

[milibopp]

I'd like to point out, that there is a crate called approx dedicated to that purpose. One could depend on it or use it as a source of inspiration.

[joshburkart]

@bluss Cool. Yes definitely agree tolerances should be configurable.

@aepsil0n Thanks for pointing that out!

[joshburkart]

So I was thinking a nice option might be to integrate with the Rust port of Hamcrest: add some Hamcrest "matchers" for approximate equality of arrays in, say, a testing submodule of ndarray. Thoughts?

Below are some initial tests I wrote, which hopefully provide a general feel for the potential API. (Haven't done the Hamcrest integration yet.)

#[cfg(test)]
mod tests {
  use lib::ndarray;

  use lib::util::*;

  #[test]
  fn test_absolute_tolerance() {
    let a = ndarray::Array1::<f32>::range(1., 4., 1.);
    assert_that(&a, almost_equals(&a));
    assert_that(&a, almost_equals(&a + 1e-8));
    assert_that(&a, almost_equals(&a + 1e-6).with_atol(1e-5));
    assert_that(&a, not(almost_equals(&a + 1e-6)));
    assert_that(array![1., 2., 3.], almost_equals(a))
  }

  #[test]
  fn test_relative_tolerance() {
    let b = ndarray::Array1::<f32>::range(1., 4., 1.) * 1e6;
    assert_that(&b, almost_equals(&b + 1.).with_rtol(1e-4));
    assert_that(
      &b,
      not(almost_equals(&b + 1.).with_atol(1e-4).with_rtol(0.)),
    );
  }

  #[test]
  fn test_matrix() {
    let c = ndarray::Array2::<f32>::zeros((4, 5)) + 5.;
    assert_that(&c, almost_equals(&c));
    assert_that(&c, almost_equals(array![5.]));
    assert_that(c, almost_equals(ndarray::arr0(4.)).with_atol(1.));
  }
}

[bluss]

No hamcrest integration, we don't bring on major deps for minor features.

[joshburkart]

Ok. You haven't provided any guidance, though -- suggestions? Optional dep? Make a little framework sans hamcrest? Or not worth bothering with at all? I use the numpy.testing module quite frequently so thought something similar would be nice here.

[bluss]

I suggest for better allclose functionality in ndarray 1 macro for allclose and 1 for assert_allcose that can handle the absolute/relative and combination. It can also be a function for allclose if that's enough for a good API. If you want to develop bigger and more fluent API in the style of hamcrest that sounds awesome, but please then publish it as a separate crate.

[joshburkart]

Functions for allclose and assert_allclose sound good to me! What about trying to add a fluent testing API behind a feature in Cargo, though? Opt in to begin with? It's only a few hundred lines of code so I'm reluctant to make a whole new crate for this small feature...

[LukeMathWalker]

Isn't this handled by #581? @jturner314 @joshburkart

[joshburkart]

Yes that looks like the right way to go!

[jturner314]

Fixed by #581.