PolarBasis2D unit test

[garrettwrong]

Had a spurious failure that went away on restart (OSX). I think it is because the np.random.randn arrays in
https://github.com/ComputationalCryoEM/ASPIRE-Python/blob/develop/tests/test_PolarBasis2D.py are not controlled, so they can potentially be degenerate wrt the adjoint test expectations (particularly in singles).

Should cover this when circle back to the next round of random changes.

[garrettwrong]

Hrmm, this happened a second time, though I think someone restarted it for me while I was looking at it. I think conversion to singles is tickling this bug more. Should prioritize fixing it then.

[garrettwrong]

(2 out of 151 pipelines, fwiw)

[garrettwrong]

I was able to reproduce this in a loop on my machine.

It was mentioned in our team meeting that this issue may be due to comparing real vs float. I just ran a small test, and in some cases this did reduce the error, and might reduce the frequency of occurrences by about half in practice. Still that is too flaky, it didn't stop the degenerate cases that cause the test the fail... so I'm just going to set a seed like we talked about. I think the test will still serve its purpose.

[janden]

It's not clear to me that we should ever have inputs for which the adjoint test fails, though. I'm ok with fixing the seed, but I'm worried there's a bug hiding somewhere here.

[garrettwrong]

Sure, I will harvest a failing input for inspection; can take a closer look. Fair to leave this issue open until satisfied.

[garrettwrong]

I can't upload this file type on GitHub, so it will need to be renamed to .npz.

bad_polar_example.npz.txt

[janden]

Sounds good. Let's merge the PR but leave this open.

[garrettwrong]

Had few minutes to toy around with this. I can demonstrate for this case that using some stable floating point technique gets this under the prescribed tolerance. Specifically if I replace:

np.dot(y_t, x) ~~> np.sum(np.sort(y_t * x)) the error is reduced by a factor of ~5. (1.5E-5 to 3E-6).

FWIW the result of that dot product under four different computation implementations:

numpy dot in singles : 0.41462326
sorted sum of products in singles : 0.41462708
numpy dot in doubles : 0.4146291018253514
sorted sum of products in doubles : 0.4146291018253123

There could be more to it, but I would expect to hit these things from time to time when working in singles.

[janden]

All right, so this has to do with the fact that we have a large number of terms in the dot product? What's the size of y_t and x?

[garrettwrong]

>>> EX['y_t'].shape
(1, 128)
>>> print(np.min(EX['y_t']), np.max(EX['y_t']), np.min(np.abs(EX['y_t'])), np.max(np.abs(EX['y_t'])))
(-9.130599-1.86159j) (11.222629+11.935363j) 1.6673269 16.382927

>>> EX['x'].shape
(128,)
>>> print(np.min(EX['x']), np.max(EX['x']), np.min(np.abs(EX['x'])), np.max(np.abs(EX['x'])))
-2.2612197 2.7441943 0.073417395 2.7441943