`build/pkgs/mpmath` 1.4 #38565

mkoeppe · 2024-08-26T16:25:35Z

📝 Checklist

The title is concise and informative.
The description explains in detail what this PR is about.
I have linked a relevant issue or discussion.
I have created tests covering the changes.
I have updated the documentation and checked the documentation preview.

⌛ Dependencies

Depends on Remove outdated mpmath Sage backend #38113
Depends on build/pkgs/mpmath/version_requirements.txt: Reject 1.4 #38691 (split out from here)

… for speed'

…dle mpmath types without Sage backend

…ion_doctest.py: Update class name in doctest output

mkoeppe · 2024-08-26T16:43:59Z

CI shows numerous failures as reported and analyzed by @kiwifb in #38113

kiwifb · 2024-08-26T19:28:59Z

I did play with pdb yesterday. The code path in mpmath for sage integer inside mpmath is completely different. In fact the previous code path followed in 1.3.0 does not exist anymore. sage integer do not seem to be recognized as such in 1.4a1.

kiwifb · 2024-08-26T19:56:28Z

I am wrong, it still exist but sage integer are diverted to a new code path before reaching the right one. In 1.4a1 it seems to take this new path https://github.com/mpmath/mpmath/blob/7afe9ac70c430f7df7e4bc1c904f90bd1887832f/mpmath/ctx_mp_python.py#L82
In 1.3.0 it would take this path
https://github.com/mpmath/mpmath/blob/7afe9ac70c430f7df7e4bc1c904f90bd1887832f/mpmath/ctx_mp_python.py#L86

kiwifb · 2024-08-26T20:54:33Z

Re-ordering some code in ctx_mp_python.py made my particular example (sage/symbolic/function.pyx) pass. I will run a full doctest with that change now.

kiwifb · 2024-08-26T21:38:34Z

There is another internal mpmath path that seems to direct sage integers to be treated as rational. I had selected a simple test case to work with and it turns out it was exceptional. But at least I think we have a root cause.

kiwifb · 2024-08-26T22:19:32Z

The following patch (it is against mpmath master, it may need a tweak for 1.4a1)

diff --git a/mpmath/ctx_mp_python.py b/mpmath/ctx_mp_python.py
index 4f452f2..c61b7d6 100644
--- a/mpmath/ctx_mp_python.py
+++ b/mpmath/ctx_mp_python.py
@@ -79,9 +79,6 @@ def mpf_convert_arg(cls, x, prec, rounding):
         if isinstance(x, int_types): return from_int(x)
         if isinstance(x, float): return from_float(x)
         if isinstance(x, cls.context.constant): return x.func(prec, rounding)
-        if isinstance(x, numbers.Rational): return from_rational(x.numerator,
-                                                                 x.denominator,
-                                                                 prec, rounding)
         if hasattr(x, '_mpf_'): return x._mpf_
         if hasattr(x, '_mpmath_'):
             t = cls.context.convert(x._mpmath_(prec, rounding))
@@ -92,6 +89,9 @@ def mpf_convert_arg(cls, x, prec, rounding):
             if a == b:
                 return a
             raise ValueError("can only create mpf from zero-width interval")
+        if isinstance(x, numbers.Rational): return from_rational(x.numerator,
+                                                                 x.denominator,
+                                                                 prec, rounding)
         if type(x).__module__ == 'decimal':
             return from_Decimal(x, prec, rounding)
         raise TypeError("cannot create mpf from " + repr(x))
@@ -680,6 +680,10 @@ def convert(ctx, x, strings=True):
             return ctx.make_mpc((from_float(x.real), from_float(x.imag)))
         if type(x).__module__ == 'numpy': return ctx.npconvert(x)
         prec, rounding = ctx._prec_rounding
+        if hasattr(x, '_mpf_'): return ctx.make_mpf(x._mpf_)
+        if hasattr(x, '_mpc_'): return ctx.make_mpc(x._mpc_)
+        if hasattr(x, '_mpmath_'):
+            return ctx.convert(x._mpmath_(prec, rounding))
         if isinstance(x, numbers.Rational):
             p, q = x.numerator, x.denominator
             return ctx.make_mpf(from_rational(p, q, prec, rounding))
@@ -689,10 +693,6 @@ def convert(ctx, x, strings=True):
                 return ctx.make_mpf(_mpf_)
             except ValueError:
                 pass
-        if hasattr(x, '_mpf_'): return ctx.make_mpf(x._mpf_)
-        if hasattr(x, '_mpc_'): return ctx.make_mpc(x._mpc_)
-        if hasattr(x, '_mpmath_'):
-            return ctx.convert(x._mpmath_(prec, rounding))
         if type(x).__module__ == 'decimal':
             return ctx.make_mpf(from_Decimal(x, prec, rounding))
         return ctx._convert_fallback(x, strings)

fixes a number of conversion problem. As you can see from the structure of the patch, in current master of mpmath, sage integer are identified as numbers.Rational while they were previously identified as having hasattr(x, '_mpmath_'). The move around helps restore that behavior.

Unfortunately it is not all the problem with mpmath. Now, that I have taken care of this issue another one has gained greater visibility.

sage -t --long --random-seed=158424137200168225311908861487827046848 /usr/lib/python3.12/site-packages/sage/symbolic/constants.py
**********************************************************************
File "/usr/lib/python3.12/site-packages/sage/symbolic/constants.py", line 119, in sage.symbolic.constants
Failed example:
    R(khinchin)
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib/python3.12/site-packages/sage/doctest/forker.py", line 716, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/usr/lib/python3.12/site-packages/sage/doctest/forker.py", line 1146, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.symbolic.constants[28]>", line 1, in <module>
        R(khinchin)
      File "sage/structure/parent.pyx", line 901, in sage.structure.parent.Parent.__call__ (/home/portage/sci-mathematics/sagemath-standard-9999/work/sagemath-standard-9999-python3_12/build/cythonized/sage/structure/parent.c:12762)
        return mor._call_(x)
      File "sage/structure/coerce_maps.pyx", line 293, in sage.structure.coerce_maps.NamedConvertMap._call_ (/home/portage/sci-mathematics/sagemath-standard-9999/work/sagemath-standard-9999-python3_12/build/cythonized/sage/structure/coerce_maps.c:8515)
        cdef Element e = method(C)
      File "sage/symbolic/expression.pyx", line 1711, in sage.symbolic.expression.Expression._mpfr_ (/home/portage/sci-mathematics/sagemath-standard-9999/work/sagemath-standard-9999-python3_12/build/cythonized/sage/symbolic/expression.cpp:50259)
        return self._eval_self(R)
      File "sage/symbolic/expression.pyx", line 1605, in sage.symbolic.expression.Expression._eval_self (/home/portage/sci-mathematics/sagemath-standard-9999/work/sagemath-standard-9999-python3_12/build/cythonized/sage/symbolic/expression.cpp:49258)
        res = self._convert({'parent':R})
      File "sage/symbolic/expression.pyx", line 1690, in sage.symbolic.expression.Expression._convert (/home/portage/sci-mathematics/sagemath-standard-9999/work/sagemath-standard-9999-python3_12/build/cythonized/sage/symbolic/expression.cpp:49973)
        cdef GEx res = self._gobj.evalf(0, kwds)
      File "sage/symbolic/pynac_impl.pxi", line 2272, in sage.symbolic.expression.py_eval_constant (/home/portage/sci-mathematics/sagemath-standard-9999/work/sagemath-standard-9999-python3_12/build/cythonized/sage/symbolic/expression.cpp:39971)
        return kwds['parent'](constant)
      File "sage/structure/parent.pyx", line 901, in sage.structure.parent.Parent.__call__ (/home/portage/sci-mathematics/sagemath-standard-9999/work/sagemath-standard-9999-python3_12/build/cythonized/sage/structure/parent.c:12762)
        return mor._call_(x)
      File "sage/structure/coerce_maps.pyx", line 293, in sage.structure.coerce_maps.NamedConvertMap._call_ (/home/portage/sci-mathematics/sagemath-standard-9999/work/sagemath-standard-9999-python3_12/build/cythonized/sage/structure/coerce_maps.c:8515)
        cdef Element e = method(C)
      File "/usr/lib/python3.12/site-packages/sage/symbolic/constants.py", line 1125, in _mpfr_
        return a.eval_constant('khinchin', R)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.12/site-packages/sage/libs/mpmath/all.py", line 26, in eval_constant
        return ring(_constants_funcs[name](prec)) >> prec
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.12/site-packages/mpmath/libmp/libelefun.py", line 91, in g
        return f.memo_val >> (newprec-prec)
               ~~~~~~~~~~~^^~~~~~~~~~~~~~~~
    OverflowError: value could not be converted to C long long
**********************************************************************

again this one appears multiple times and across multiple files.

Taken from sagemath/sage#38565 (comment)

skirpichev · 2024-08-27T06:42:29Z

The following patch (it is against mpmath master, it may need a tweak for 1.4a1)

No need to backport this. I'll include sage fixes in next alpha release, see mpmath/mpmath#851

kiwifb · 2024-08-27T07:57:06Z

The following patch (it is against mpmath master, it may need a tweak for 1.4a1)

No need to backport this. I'll include sage fixes in next alpha release, see mpmath/mpmath#851

Thank you for that. I was going to open an issue on mpmath to discuss it and follow it with a PR if you guys approved the approach. I must say the second issue has me stomped so far. I'll take any clue.

skirpichev · 2024-08-27T08:23:51Z

The OverflowError coming from gmpy2. Could you run this test with MPMATH_NOGMPY=1 environment variable set?

kiwifb · 2024-08-27T08:57:48Z

Yes indeed

fbissey@tarazed ~ $ MPMATH_NOGMPY=1 sage -t --long /usr/lib/python3.12/site-packages/sage/symbolic/constants.pytoo many failed tests, not using stored timings
Running doctests with ID 2024-08-27-20-56-43-57d4cb37.
Running with SAGE_LOCAL='/usr' and SAGE_VENV='/usr'
Using --optional=pip,sage
Features to be detected: 4ti2,SAGE_SRC,benzene,bliss,buckygen,conway_polynomials,coxeter3,csdp,cvxopt,cvxopt,database_cremona_ellcurve,database_cremona_mini_ellcurve,database_cubic_hecke,database_ellcurves,database_graphs,database_jones_numfield,database_knotinfo,dvipng,ecm,fpylll,fricas,gap_package_atlasrep,gap_package_design,gap_package_grape,gap_package_guava,gap_package_hap,gap_package_polycyclic,gap_package_qpa,gap_package_quagroup,gfan,glucose,graphviz,imagemagick,ipython,jmol,jupymake,kenzo,kissat,latte_int,lrcalc_python,lrslib,mathics,matroid_database,mcqd,meataxe,mpmath,msolve,nauty,networkx,numpy,palp,pandoc,pdf2svg,pdftocairo,pexpect,phitigra,pillow,plantri,polytopes_db,polytopes_db_4d,pplpy,primecountpy,ptyprocess,pycosat,pycryptosat,pynormaliz,pyparsing,python_igraph,requests,rpy2,rubiks,sage.combinat,sage.geometry.polyhedron,sage.graphs,sage.groups,sage.libs.braiding,sage.libs.ecl,sage.libs.flint,sage.libs.gap,sage.libs.homfly,sage.libs.linbox,sage.libs.m4ri,sage.libs.ntl,sage.libs.pari,sage.libs.singular,sage.misc.cython,sage.modular,sage.modules,sage.numerical.mip,sage.plot,sage.rings.complex_double,sage.rings.finite_rings,sage.rings.function_field,sage.rings.number_field,sage.rings.padics,sage.rings.polynomial.pbori,sage.rings.real_double,sage.rings.real_mpfr,sage.sat,sage.schemes,sage.symbolic,sage_numerical_backends_coin,sagemath_doc_html,scipy,singular,sirocco,sphinx,symengine_py,sympy,tdlib,threejs,topcom
Doctesting 1 file.
sage -t --long --random-seed=211690163611789333881822304876232239576 /usr/lib/python3.12/site-packages/sage/symbolic/constants.py
    [241 tests, 0.77 s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------

I have gmpy2 2.2.1.

skirpichev · 2024-08-27T09:28:09Z

Hmm, could you try this workaround for L26 of all.py: ring(_constants_funcs[name](int(prec)))?

I have gmpy2 2.2.1.

Was it installed from wheel or built locally? Is this M$ Windows?

kiwifb · 2024-08-27T09:30:05Z

Hmm, could you try this workaround for L26 of all.py: ring(_constants_funcs[name](int(prec)))?

I have gmpy2 2.2.1.

Was it installed from wheel or built locally? Is this M$ Windows?

This is Gentoo Linux, my package management builds it locally :)

kiwifb · 2024-08-27T09:32:33Z

Hmm, could you try this workaround for L26 of all.py: ring(_constants_funcs[name](int(prec)))?

Can you give me a more complete path. There are plenty of all.py in sage.

kiwifb · 2024-08-27T09:32:59Z

Oh, wait mpmath.

kiwifb · 2024-08-27T09:34:40Z

Yes, that works.

skirpichev · 2024-08-27T10:04:19Z

This is Gentoo Linux, my package management builds it locally :)

Good news, this is something, well, sane... Looks like there is a bug in gmpy2 2.1.x, related with conversion to machine-size integers. Or something is wrong with integration of sage & gmpy2 (__mpz__() method).

@casevh, I would appreciate if you could take a look.

Yes, that works.

ok. Are there other test failures?

kiwifb · 2024-08-27T10:17:37Z

That fix took care of all the doctest that I am sure were caused by mpmath. I have number of doctest with other identified, or more likely sources. I have just that one left where I am not sure because I have not digged into it yet

sage -t --long --random-seed=158424137200168225311908861487827046848 /usr/lib/python3.12/site-packages/sage/functions/exp_integral.py
**********************************************************************
File "/usr/lib/python3.12/site-packages/sage/functions/exp_integral.py", line 995, in sage.functions.exp_integral.Function_cos_integral._evalf_
Failed example:
    cos_integral(ComplexField(100)(I))                                    # needs sage.symbolic
Expected:
    0.83786694098020824089467857943 + 1.5707963267948966192313216916*I
Got:
    0.83786694098020824089467857944 + 1.5707963267948966192313216916*I
**********************************************************************

but that's just numerical noise.

skirpichev · 2024-08-27T10:50:36Z

Well, I see no difference wrt 1.3.0:

>>> from mpmath import *
>>> mp.prec=100
>>> ci(1j)
mpc(real='0.83786694098020824089467857943576', imag='1.5707963267948966192313216916397')
>>> str(_)
'(0.83786694098020824089467857944 + 1.5707963267948966192313216916j)'
>>> import mpmath 
>>> mpmath.__version__
'1.3.0'

kiwifb · 2024-08-27T11:05:40Z

git blame says that value has been around for 13 years in sage and mpmath 1.3.0 has been in sage since July 2023. Not sure how this did not show up until now.

kiwifb · 2024-08-27T11:20:47Z

Actually one doctest timeout and crash that I thought was caused by singular appears to be linked to mpmath-1.4.0

sage -t --long --random-seed=158424137200168225311908861487827046848 /usr/lib/python3.12/site-packages/sage/functions/orthogonal_polys.py  # Timed out

it seems to get stuck somewhere in gmpy, I have huge backtrace that I am not sure is very useful, but some of the interesting bits of the logs below

sage: chebyshev_T(1234.5, I)                                                # needs sage.rings.real_mpfr sage.symbolic ## line 773 >
-1.21629397684152e472 - 1.21629397684152e472*I
sage: chebyshev_T._evalf_(10^6, 0.1)                                        # needs sage.rings.real_mpfr ## line 779 ##
------------------------------------------------------------------------
/usr/lib/python3.12/site-packages/cysignals/signals.cpython-312-x86_64-linux-gnu.so(+0xa0d4)[0x7f9ee7ade0d4]
/usr/lib/python3.12/site-packages/cysignals/signals.cpython-312-x86_64-linux-gnu.so(+0xa196)[0x7f9ee7ade196]
/usr/lib/python3.12/site-packages/cysignals/signals.cpython-312-x86_64-linux-gnu.so(+0xcba1)[0x7f9ee7ae0ba1]
/usr/lib64/libc.so.6(+0x3c2a0)[0x7f9ee885d2a0]
/usr/lib64/libgmp.so.10(__gmpn_mul_basecase_coreisbr+0x3ac)[0x7f9ee82e506c]
------------------------------------------------------------------------

skirpichev · 2024-08-28T01:41:19Z

git blame says that value has been around for 13 years in sage and mpmath 1.3.0 has been in sage since July 2023.

Anyway, new answer seems to be correct. Also, Mathematica:

In[4]:= N[Re[CosIntegral[I]], 29]

Out[4]= 0.83786694098020824089467857944

it seems to get stuck somewhere in gmpy

No, it's not gmpy2-related. But in 1.3.0 it quickly quits with NoConvergence exception, so there is a regression. Edit: mpmath/mpmath#852

Taken from sagemath/sage#38565 (comment)

orlitzky · 2024-09-17T22:52:41Z

Can we do the new version requirement separately? 1.4.0_alpha1 showed up in Gentoo and gets picked up by ./configure.

mkoeppe · 2024-09-18T00:14:57Z

Sure, feel free to cherry pick to a new PR

…

Sent from my phone

On Tue, Sep 17, 2024 at 14:53 Michael Orlitzky ***@***.***> wrote: Can we do the new version requirement separately? 1.4.0_alpha1 showed up in Gentoo and gets picked up by ./configure. — Reply to this email directly, view it on GitHub <#38565 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB7VNBKWDQYJOR3YAYFDYXDZXCXE5AVCNFSM6AAAAABNEK4VSKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNJXGA3DOMRUGM> . You are receiving this because you were assigned.Message ID: ***@***.***>

sagemathgh-38691: build/pkgs/mpmath/version_requirements.txt: Reject 1.4 Cherry-picked from sagemath#38565 Gentoo has mpmath-1.4.0_alpha1 in the tree now, and we don't want `./configure` to use it yet. URL: sagemath#38691 Reported by: Michael Orlitzky Reviewer(s):

Matthias Koeppe added 13 commits August 10, 2024 21:02

src/sage/env.py: Set MPMATH_NOSAGE

07c4f98

src/sage/libs/mpmath/utils.pyx: Remove 'patches some mpmath functions…

0f6e4b0

… for speed'

src/sage/libs/mpmath/ext_libmp.pyx: Remove

3ed5cca

src/sage/libs/mpmath/ext_libmp.p*: Remove

d52b7ec

src/sage/libs/mpmath/ext_main.p*: Remove

ec767ec

src/sage/libs/mpmath/utils.pyx: Use gmpy2

cf0a7a1

src/sage/rings/real_mpfr.pyx: Remove duplicate cimport

b57e0a4

src/sage/structure/coerce.pyx (is_mpmath_type, py_scalar_parent): Han…

125ae4e

…dle mpmath types without Sage backend

src/sage/tests/books/computational-mathematics-with-sagemath/integrat…

b713521

…ion_doctest.py: Update class name in doctest output

src/sage/libs/mpmath/utils.pyx: Update doctest output for 'ei'

84f280c

build/pkgs/mpmath/version_requirements.txt: Reject 1.4

ed6d0c2

Merge branch 'no_mpmath_sage' into mpmath-1.4

12c6b9f

build/pkgs/mpmath: Update to 1.4.0a1

81d3cce

skirpichev added a commit to skirpichev/mpmath that referenced this pull request Aug 27, 2024

Reorder hasattr() checks in mpf.mpf_convert_arg() and mp.convert()

0bc1e2f

Taken from sagemath/sage#38565 (comment)

skirpichev added a commit to skirpichev/mpmath that referenced this pull request Aug 29, 2024

Reorder hasattr() checks in mpf.mpf_convert_arg() and mp.convert()

4be9a02

Taken from sagemath/sage#38565 (comment)

orlitzky mentioned this pull request Sep 21, 2024

build/pkgs/mpmath/version_requirements.txt: Reject 1.4 #38691

Merged

skirpichev mentioned this pull request Oct 17, 2024

mpz(1)>>111111111111111111111 raises OverflowError aleaxit/gmpy#524

Closed

kiwifb mentioned this pull request Aug 23, 2025

Mpmath 1.4 alpha update, setup.py and meson compatible #40669

Open

5 tasks

orlitzky mentioned this pull request Oct 8, 2025

Update for mpmath-1.4 while remaining backwards-compatible #41014

Draft

Uh oh!

build/pkgs/mpmath 1.4 #38565

Are you sure you want to change the base?

build/pkgs/mpmath 1.4 #38565

Uh oh!

Conversation

mkoeppe commented Aug 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📝 Checklist

⌛ Dependencies

Uh oh!

mkoeppe commented Aug 26, 2024

Uh oh!

kiwifb commented Aug 26, 2024

Uh oh!

kiwifb commented Aug 26, 2024

Uh oh!

kiwifb commented Aug 26, 2024

Uh oh!

kiwifb commented Aug 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kiwifb commented Aug 26, 2024

Uh oh!

skirpichev commented Aug 27, 2024

Uh oh!

kiwifb commented Aug 27, 2024

Uh oh!

skirpichev commented Aug 27, 2024

Uh oh!

kiwifb commented Aug 27, 2024

Uh oh!

skirpichev commented Aug 27, 2024

Uh oh!

kiwifb commented Aug 27, 2024

Uh oh!

kiwifb commented Aug 27, 2024

Uh oh!

kiwifb commented Aug 27, 2024

Uh oh!

kiwifb commented Aug 27, 2024

Uh oh!

skirpichev commented Aug 27, 2024

Uh oh!

kiwifb commented Aug 27, 2024

Uh oh!

skirpichev commented Aug 27, 2024

Uh oh!

kiwifb commented Aug 27, 2024

Uh oh!

kiwifb commented Aug 27, 2024

Uh oh!

skirpichev commented Aug 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

orlitzky commented Sep 17, 2024

Uh oh!

mkoeppe commented Sep 18, 2024 via email

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

`build/pkgs/mpmath` 1.4 #38565

`build/pkgs/mpmath` 1.4 #38565

mkoeppe commented Aug 26, 2024 •

edited

Loading

kiwifb commented Aug 26, 2024 •

edited

Loading

skirpichev commented Aug 28, 2024 •

edited

Loading