sage.features.Feature.resolution: If SAGE_ROOT is available, recommend system packages

[mkoeppe]

Currently, if a Feature is keyed to an spkg, a FeatureNotPresentError recommends to use sage -i to install the package.

When SAGE_ROOT (or sage_bootstrap, see #29039) is available, we can do better:

1. We can recommend equivalent system packages for package systems available on the present system.

2. Moreover, if SAGE_ROOT/bin/sage-spkg is not available, then we remove the recommendation to use sage -i. This is for the benefit of distribution packaging.

   (Distribution packages would typically not install sage-spkg, so this information is automatically suppressed -- no more patching is necessary.)

3. Finally, for Python packages that have a requirements.txt (see Add build/pkgs/SPKG/requirements.txt for all Python SPKG #30024), we also show a pip invocation.

(1. and 3. depend on a few scripts from SAGE_ROOT/build/bin to be available in the PATH. Distribution packages could consider to install these scripts; perhaps, in some libexec-style directory. Also, SAGE_ROOT/build/pkgs/*/requirements.txt and SAGE_ROOT/build/pkgs/*/distros/ need to be made available at runtime.)

We also add the names of the detected package systems to the default optional tags in use by doctests.

CC: @kiwifb @jhpalmieri @slel @seblabbe

Component: build: configure

Author: Matthias Koeppe

Branch: 68a8e2a

Reviewer: Sébastien Labbé

Issue created by migration from https://trac.sagemath.org/ticket/30606

[mkoeppe]

Description changed:

--- 
+++ 
@@ -2,4 +2,7 @@
 
 When `SAGE_ROOT` (or `sage_bootstrap`) is available, we can do better: We can recommend equivalent system packages.
 
+Moreover, if `SAGE_ROOT/bin/sage-spkg` is not available, then we remove the recommendation to use `sage -i`. This is for the benefit of distribution packaging.
 
+
+

[mkoeppe]

Branch: u/mkoeppe/sage_features_feature_resolution__if_sage_root_is_available__recommend_system_packages

[mkoeppe]

Author: Matthias Koeppe

[mkoeppe]

New commits:

2a8e3b7

Feature.resolution: If spkg is given, try to give system package information; only show 'sage -i' if sage-spkg is available

[mkoeppe]

Commit: 2a8e3b7

[mkoeppe]

comment:4

resolution should be lazier

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

febf5e6	Feature.resolution: Use lazy strings to delay system package lookup
081e736	sage.features.package_systems: New, use it in Feature.resolution
36772a5	Feature.resolution: Fix up
04fa02d	build/bin/{sage-get-system-packages,sage-print-system-package-command}: Handle pip
3f66ab7	build/bin/sage-print-system-package-command: Handle --prompt=PROMPT, fix error reporting
d2055ac	sage.features: Add features PackageSystem, SagePackageSystem, PipPackageSystem, use them in Feature.resolution

[sagetrac-git]

Changed commit from 2a8e3b7 to d2055ac

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

fa10a90

src/sage/features/__init__.py: Add doctests

[sagetrac-git]

Changed commit from d2055ac to fa10a90

[mkoeppe]

Description changed:

--- 
+++ 
@@ -4,5 +4,6 @@
 
 Moreover, if `SAGE_ROOT/bin/sage-spkg` is not available, then we remove the recommendation to use `sage -i`. This is for the benefit of distribution packaging.
 
+Finally, for Python packages that have a `requirements.txt` (see #30024), we also show a `pip` invocation.
+ 
 
-

[kiwifb]

comment:8

I immediately thought that the doctests would fail in sage-on-gentoo and more generally in sage-on-distros. And I was right.

sage -t --long --warn-long 112.3 --random-seed=0 /usr/lib/python3.7/site-packages/sage/features/__init__.py
**********************************************************************
File "/usr/lib/python3.7/site-packages/sage/features/__init__.py", line 43, in sage.features
Failed example:
    Executable(name="random", executable="randomOochoz6x", spkg="random", url="http://rand.om").require()
Expected:
    Traceback (most recent call last):
    ...
    FeatureNotPresentError: random is not available.
    Executable 'randomOochoz6x' not found on PATH.
    ...try to run...sage -i random...
    Further installation instructions might be available at http://rand.om.
Got:
    <BLANKLINE>
    Traceback (most recent call last):
      File "/usr/lib/python3.7/site-packages/sage/doctest/forker.py", line 720, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/usr/lib/python3.7/site-packages/sage/doctest/forker.py", line 1145, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.features[5]>", line 1, in <module>
        Executable(name="random", executable="randomOochoz6x", spkg="random", url="http://rand.om").require()
      File "/usr/lib/python3.7/site-packages/sage/features/__init__.py", line 192, in require
        raise FeatureNotPresentError(self, presence.reason, presence.resolution)
    sage.features.FeatureNotPresentError: random is not available.
    Executable 'randomOochoz6x' not found on PATH.
    No equivalent system packages for pip are known to Sage.
    Further installation instructions might be available at http://rand.om.
**********************************************************************
File "/usr/lib/python3.7/site-packages/sage/features/__init__.py", line 219, in sage.features.Feature.resolution
Failed example:
    Executable(name="CSDP", spkg="csdp", executable="theta", url="http://github.org/dimpase/csdp").resolution()
Expected:
    l'...To install CSDP...you can try to run...sage -i csdp...Further installation instructions might be available at http://github.org/dimpase/csdp.'
Got:
    l'No equivalent system packages for pip are known to Sage.\nFurther installation instructions might be available at http://github.org/dimpase/csdp.'
**********************************************************************
File "/usr/lib/python3.7/site-packages/sage/features/__init__.py", line 307, in sage.features.FeatureTestResult
Failed example:
    str(FeatureTestResult(package, True).resolution)
Expected:
    '...To install GAP package NOT_A_PACKAGE...you can try to run...sage -i no_package...'
Got:
    'No equivalent system packages for pip are known to Sage.'
**********************************************************************
File "/usr/lib/python3.7/site-packages/sage/features/__init__.py", line 309, in sage.features.FeatureTestResult
Failed example:
    str(FeatureTestResult(package, False).resolution)
Expected:
    '...To install GAP package NOT_A_PACKAGE...you can try to run...sage -i no_package...'
Got:
    'No equivalent system packages for pip are known to Sage.'
**********************************************************************
File "/usr/lib/python3.7/site-packages/sage/features/__init__.py", line 623, in sage.features.StaticFile
Failed example:
    StaticFile(name="no_such_file", filename="KaT1aihu", search_path=("/",), spkg="some_spkg", url="http://rand.om").require()
Expected:
    Traceback (most recent call last):
    ...
    FeatureNotPresentError: no_such_file is not available.
    'KaT1aihu' not found in any of ['/']...
    To install no_such_file...you can try to run...sage -i some_spkg...
    Further installation instructions might be available at http://rand.om.
Got:
    <BLANKLINE>
    Traceback (most recent call last):
      File "/usr/lib/python3.7/site-packages/sage/doctest/forker.py", line 720, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/usr/lib/python3.7/site-packages/sage/doctest/forker.py", line 1145, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.features.StaticFile[1]>", line 1, in <module>
        StaticFile(name="no_such_file", filename="KaT1aihu", search_path=("/",), spkg="some_spkg", url="http://rand.om").require()
      File "/usr/lib/python3.7/site-packages/sage/features/__init__.py", line 192, in require
        raise FeatureNotPresentError(self, presence.reason, presence.resolution)
    sage.features.FeatureNotPresentError: no_such_file is not available.
    'KaT1aihu' not found in any of ['/']
    No equivalent system packages for pip are known to Sage.
    Further installation instructions might be available at http://rand.om.
**********************************************************************
File "/usr/lib/python3.7/site-packages/sage/features/__init__.py", line 673, in sage.features.StaticFile.absolute_path
Failed example:
    StaticFile(name="no_such_file", filename="KaT1aihu", search_path=(), spkg="some_spkg", url="http://rand.om").absolute_path()
Expected:
    Traceback (most recent call last):
    ...
    FeatureNotPresentError: no_such_file is not available.
    'KaT1aihu' not found in any of []...
    To install no_such_file...you can try to run...sage -i some_spkg...
    Further installation instructions might be available at http://rand.om.
Got:
    <BLANKLINE>
    Traceback (most recent call last):
      File "/usr/lib/python3.7/site-packages/sage/doctest/forker.py", line 720, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/usr/lib/python3.7/site-packages/sage/doctest/forker.py", line 1145, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.features.StaticFile.absolute_path[2]>", line 1, in <module>
        StaticFile(name="no_such_file", filename="KaT1aihu", search_path=(), spkg="some_spkg", url="http://rand.om").absolute_path()
      File "/usr/lib/python3.7/site-packages/sage/features/__init__.py", line 687, in absolute_path
        resolution=self.resolution())
    sage.features.FeatureNotPresentError: no_such_file is not available.
    'KaT1aihu' not found in any of []
    No equivalent system packages for pip are known to Sage.
    Further installation instructions might be available at http://rand.om.
**********************************************************************
5 items had failures:
   1 of   7 in sage.features
   1 of   3 in sage.features.Feature.resolution
   2 of  11 in sage.features.FeatureTestResult
   1 of   3 in sage.features.StaticFile
   1 of   4 in sage.features.StaticFile.absolute_path
    [118 tests, 6 failures, 9.26 s]
----------------------------------------------------------------------
sage -t --long --warn-long 112.3 --random-seed=0 /usr/lib/python3.7/site-packages/sage/features/__init__.py  # 6 doctests failed

At least that may tell you if things are working as you expected them.

[mkoeppe]

comment:9

Thanks for testing! This code is meant to work also on distributions without patching. I'll try to fix it.

[sagetrac-git]

Changed commit from fa10a90 to c01aa8f

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

14e9395	SagePackageSystem: Rename to sage_spkg (underscore) to make it an allowed optional tag
5c0da17	src/sage/doctest/control.py: Add available package systems to optional tags
c01aa8f	src/sage/features/__init__.py: Mark tests that check for 'sage -i' optional - sage_spkg