zn_poly segfaults during tuning and tests on OS X and Cygwin when built on a busy system

[jpflori]

See #13137 for more info.
This is true with different versions of MPIR so seems to be because of zn_poly and not of MPIR.
No problems where spotted on Linuces.

---

New spkg: http://boxen.math.washington.edu/home/leif/Sage/spkgs/zn_poly-0.9.p11.spkg

md5sum: 012e63d181151c19ddc71bdfaeb14e03 zn_poly-0.9.p11.spkg

zn_poly-0.9.p11 (Leif Leonhardy, May 24th, 2013)

• zn_poly segfaults during tuning and tests on OS X and Cygwin when built on a busy system #13947: Fix nuss_mul() test failing especially if tuning happened
  under "heavy" load (at least on MacOS X and Cygwin)
  Add fix_fudge_factor_in_nuss-test.c.patch; fix suggested by David
  Harvey.

CC: @nexttime @jhpalmieri @jdemeyer @kcrisman @kwankyu

Component: packages: standard

Keywords: zn_poly spkg cygwin osx nuss_mul fail

Author: Leif Leonhardy

Reviewer: Jeroen Demeyer

Merged: sage-5.10.beta5

Issue created by migration from https://trac.sagemath.org/ticket/13947

[nexttime]

comment:1

Does it really segfault, and especially does the tuning segfault?

I thought zn_poly would just occasionally generate "unexpected^TM" values during tuning on MacOS X and Cygwin (presumably only under heavy system load), such that afterwards some tests (with zn_poly rebuilt, or more precisely, relinked with these paramaters) would deterministically fail.

(This might still depend on the compiler as well, at least the way it fails.)

[nexttime]

comment:2

P.S.: I was actually going to create a zn_poly spkg which simply saves the tuning parameters in case the tests fail, asking the user for submitting them to sage-devel or sage-release... (and probably doing a few more attempts to get working tuning parameters, and/or inform the user that he/she should reinstall the spkg when the sysload is lower). :-)

[nexttime]

comment:3

P.P.S.: John, is it always (just) nuss_mul() that fails the test?

[jdemeyer]

comment:4

I also reproduced it on bsd.math.

[jdemeyer]

comment:5

Also reproduced on hawk (OpenSolaris i386).

[nexttime]

comment:6

Replying to @jdemeyer:

I also reproduced it on bsd.math.

How? I tried hard yesterday, but didn't manage.

Even with John's tuning parameters that made the test(s) fail for him, still all tests (quick as well as extensive) pass for me on bsd.math (with Sage 5.6.beta3 [with GCC 4.6.3 built], and the included MPIR 2.4.0, FWIW).

I was actually hoping we could reproduce the test failures on e.g. Linux as well with such "failing" parameters, although probably depending on the GCC version, too.

[nexttime]

comment:7

Replying to @nexttime:

Replying to @jdemeyer:

I also reproduced it on bsd.math.

How? I tried hard yesterday, but didn't manage.

Hmmm, I probably forgot to run make test (which rebuilds test/test with a debug version of src/tuning.c) in addition to make (which apparently just rebuilds the static library, not used by test; IMHO a flaw in the Makefiles).

But still, I cannot reproduce the failure with John's parameters.

[nexttime]

comment:8

Replying to @nexttime:

Replying to @nexttime:

Replying to @jdemeyer:

I also reproduced it on bsd.math.

How? I tried hard yesterday, but didn't manage.

Hmmm, I probably forgot to run make test (which rebuilds test/test with a debug version of src/tuning.c) in addition to make (which apparently just rebuilds the static library, not used by test; IMHO a flaw in the Makefiles).

But still, I cannot reproduce the failure with John's parameters.

Ooops, not true. In the last attempt, I missed that nuss_mul() failed, but only when tested "extensively":

(sage-sh) leif@bsd:src$ test/test -quick all
mpn_smp_basecase()... ok
mpn_smp_kara()... ok
mpn_smp()... ok
mpn_mulmid()... ok
zn_array_recover_reduce()... ok
zn_array_pack()... ok
zn_array_unpack()... ok
zn_array_mul_KS1()... ok
zn_array_mul_KS2()... ok
zn_array_mul_KS3()... ok
zn_array_mul_KS4()... ok
zn_array_sqr_KS1()... ok
zn_array_sqr_KS2()... ok
zn_array_sqr_KS3()... ok
zn_array_sqr_KS4()... ok
zn_array_mulmid_KS1()... ok
zn_array_mulmid_KS2()... ok
zn_array_mulmid_KS3()... ok
zn_array_mulmid_KS4()... ok
nuss_mul()... ok
pmfvec_fft_dc()... ok
pmfvec_fft_huge()... ok
pmfvec_ifft_dc()... ok
pmfvec_ifft_huge()... ok
pmfvec_tpfft_dc()... ok
pmfvec_tpfft_huge()... ok
pmfvec_tpifft_dc()... ok
pmfvec_tpifft_huge()... ok
zn_array_mul_fft()... ok
zn_array_sqr_fft()... ok
zn_array_mulmid_fft()... ok
zn_array_mul_fft_dft()... ok
zn_array_invert()... ok

All tests passed.
(sage-sh) leif@bsd:src$ time test/test all
mpn_smp_basecase()... ok
mpn_smp_kara()... ok
mpn_smp()... ok
mpn_mulmid()... ok
zn_array_recover_reduce()... ok
zn_array_pack()... ok
zn_array_unpack()... ok
zn_array_mul_KS1()... ok
zn_array_mul_KS2()... ok
zn_array_mul_KS3()... ok
zn_array_mul_KS4()... ok
zn_array_sqr_KS1()... ok
zn_array_sqr_KS2()... ok
zn_array_sqr_KS3()... ok
zn_array_sqr_KS4()... ok
zn_array_mulmid_KS1()... ok
zn_array_mulmid_KS2()... ok
zn_array_mulmid_KS3()... ok
zn_array_mulmid_KS4()... ok
nuss_mul()... FAIL!

At least one test FAILED!

Presumably because those tests take a pretty long time... ;-)

[nexttime]

comment:9

The (extensive) tests that didn't get run because test exits upon the first failure (nuss_mul()) all pass for me:

(sage-sh) leif@bsd:src$ time test/test pmfvec_fft_dc pmfvec_fft_huge pmfvec_ifft_dc pmfvec_ifft_huge pmfvec_tpfft_dc pmfvec_tpfft_huge pmfvec_tpifft_dc pmfvec_tpifft_huge zn_array_mul_fft zn_array_sqr_fft zn_array_mulmid_fft zn_array_mul_fft_dft zn_array_invert && echo OK
pmfvec_fft_dc()... ok
pmfvec_fft_huge()... ok
pmfvec_ifft_dc()... ok
pmfvec_ifft_huge()... ok
pmfvec_tpfft_dc()... ok
pmfvec_tpfft_huge()... ok
pmfvec_tpifft_dc()... ok
pmfvec_tpifft_huge()... ok
zn_array_mul_fft()... ok
zn_array_sqr_fft()... ok
zn_array_mulmid_fft()... ok
zn_array_mul_fft_dft()... ok
zn_array_invert()... ok

All tests passed.

real	1m15.127s
user	1m15.054s
sys	0m0.027s
OK

(This is with John's "failing" tuning parameters, bsd.math.)

[jhpalmieri]

comment:10

Replying to @nexttime:

P.P.S.: John, is it always (just) nuss_mul() that fails the test?

That's my recollection. In my experiment yesterday, I only ran that test (using ./test nuss_mull).