SPKG type: Make "normal/script/pip" orthogonal to "base/standard/optional/experimental"

[mkoeppe]

This ticket, inspired by a proposal by embray in #24586, removes the misuse of "package type" for the undocumented "script package" category (introduced in #19427). Likewise, it removes the same misuse of "package type" for the undocumented (as noted in #21033) "pip package" category (introduced in #19187).

With this ticket, type has to be one of base / standard / optional / experimental.

All type=pip packages are changed to optional.

All type=script packages are changed to optional; with the exception of sage_conf, which is a dependency of sagelib and is changed to standard.

Orthogonal to type is the new notion of a package "source", which is defined as follows:

• normal packages have a checksum.ini file
• pip packages have a requirements.txt file instead (which is used for pip install). (This allows us to change the pyopenssl package from script to pip.)
• script packages have neither of the two

The ticket also makes script packages more similar to normal packages: They can now optionally have a package-version.txt file; and their installation status is recorded.

Finally, this ticket adds documentation to the developer manual.

CC: @embray @dimpase @jhpalmieri @sagetrac-tmonteil @vbraun

Component: build

Author: Matthias Koeppe

Branch/Commit: 0919086

Reviewer: John Palmieri

Issue created by migration from https://trac.sagemath.org/ticket/29287

[mkoeppe]

Branch: u/mkoeppe/spkg_type__make__regular_script_pip__orthogonal_to__standard_optional_experimental_

[mkoeppe]

Commit: 90054f6

[mkoeppe]

Author: Matthias Koeppe

[mkoeppe]

New commits:

c5dcfd6	build/pkgs: Change all 'type=pip' packages to 'type=optional' and add requirements.txt
9a9b3c3	build/pkgs: Change all 'type=script' packages to 'type=optional'
90054f6	m4/sage_spkg_collect.m4: Make SPKG_SOURCE={normal|pip|script} orthogonal to SPKG_TYPE={base,standard,optional,experimental}

[sagetrac-git]

Changed commit from 90054f6 to f7a661c

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

f7a661c

sage.misc.package.list_packages: New argument pkg_sources, orthogonal to pkg_types

[jhpalmieri]

comment:5

This needs documentation.

[mkoeppe]

comment:6

Indeed, as noted already in #21033.

[jhpalmieri]

comment:7

Just putting the list in the description somewhere in the docs would help. What is a "script" package, anyway?

[jhpalmieri]

Dependencies: #21033

[mkoeppe]

comment:8

Yes, I'll work on it

[mkoeppe]

Changed dependencies from #21033 to #29096

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

09cc8ba	build/pkgs: Change all 'type=pip' packages to 'type=optional' and add requirements.txt
8a8100a	build/pkgs: Change all 'type=script' packages to 'type=optional'
ca47dc4	m4/sage_spkg_collect.m4: Make SPKG_SOURCE={normal|pip|script} orthogonal to SPKG_TYPE={base,standard,optional,experimental}
ff4db4b	sage.misc.package.list_packages: New argument pkg_sources, orthogonal to pkg_types
041c76d	Rename spkg-build, spkg-install etc. templates as spkg-build.in, spkg-install.in etc.
4524b04	Update developer manual
2724083	Undo rename of build/pkgs/*/spkg-install.py
ec540dd	Fix up sage-spkg
f819b51	Merge branch 't/29096/rename_spkg_build__spkg_install_etc__templates_as_spkg_build_in__spkg_install_in_etc_' into t/29287/spkg_type__make__regular_script_pip__orthogonal_to__standard_optional_experimental_