Install script and pip packages via sage-spkg

[mkoeppe]

Follow-up from #29287.

Script packages are installed by special rules in build/make/Makefile.

This ticket installs them by sage-spkg instead. This enables script packages to participate in the staged installs (SAGE_DESTDIR) and uninstallation mechanisms.

This is useful for #29387 to clean up #27870/#29003/#29082, which introduced files in SAGE_LOCAL that do not belong to any package and are never uninstalled.

We also make the decision whether spkg-install is generated from a template spkg-install.in or provided as is in the source tree orthogonal to the distinction source=script vs. source=pip.
We simply generate spkg-install if spkg-install.in is present.

Ideally, when done, changing a package's source between script and normal and pip will no longer require running bootstrap.

CC: @dimpase @jhpalmieri @embray @orlitzky

Component: build

Work Issues: Move "dummy script package" code to sage-spkg

Author: Matthias Koeppe

Branch/Commit: u/mkoeppe/install_script_packages_via_sage_spkg @ 7a25d1d

Issue created by migration from https://trac.sagemath.org/ticket/29386

[mkoeppe]

comment:1

pushing these forward to 9.2

[mkoeppe]

Dependencies: #29793

[mkoeppe]

Description changed:

--- 
+++ 
@@ -7,3 +7,6 @@
 This is useful for #29082... to clean up #27870/#29003, which introduced files in `SAGE_LOCAL` that do not belong to any package and are never uninstalled.
 
 
+We also make the decision whether `spkg-install` is generated from a template `spkg-install.in` or provided as is in the source tree orthogonal to the distinction `source=script` vs. `source=pip`.
+
+

[embray]

comment:6

+1 this would be a very nice improvement.

[mkoeppe]

Branch: u/mkoeppe/install_script_packages_via_sage_spkg

[mkoeppe]

Changed dependencies from #29793 to #29793, #30657

[mkoeppe]

Commit: fee5bc2

[mkoeppe]

New commits:

f0f03c9	build/bin/sage-spkg: Remove dead code for manufacturing an spkg-install script
46e5f68	build/bin/sage-spkg: Get rid of USE_LOCAL_SCRIPTS (it was always yes); prepare for script packages
fee5bc2	build/bin/sage-spkg: For script packages, do not poison the proxy variables

[sagetrac-git]

Changed commit from fee5bc2 to e4c27f6

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

9ee2110	build/bin/sage-dist-helpers: Also use $sudo for storing the wheel file
d7aac84	src/doc/en/developer/packaging.rst: Update sdh_... documentation
9b7c7a0	build/bin/sage-pip-{install,uninstall}: Fix typo in comment
4135e8b	build/bin/sage-pip-install: Remove an outdated comment
f2e7075	Merge tag '9.2.beta13' into t/29500/install_all_python_packages_via_pip_wheel__create_pep_503_simple_repository_for_wheels
bd0fd2f	Merge branch 't/29500/install_all_python_packages_via_pip_wheel__create_pep_503_simple_repository_for_wheels' into t/30657/fix_up__sage__p_
2e20f23	build/bin/sage-spkg: Move SAGE_SPKG_{INST,SCRIPTS,WHEELS} setting here from build/bin/install, make them relative to SAGE_INST_LOCAL
ac09b14	Merge branch 't/30657/fix_up__sage__p_' into t/29386/install_script_packages_via_sage_spkg
bf0c3dc	build/bin/sage-spkg: Exit with error for some old-style script invocations
e4c27f6	build/bin/sage-spkg: Remove error message when checksums.ini does not exist (script package)

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

3ceeaa7	build/bin/sage-spkg: Fix up for normal and script packages
8c4fb4d	build/bin/sage-spkg: Handle script packages without package-version.txt
137365a	build/make/Makefile.in: Replace 'SPKG-clean' by 'SPKG-uninstall', with deprecation
a9cbfda	build/make/Makefile.in: Install script packages using sage-spkg
81a63df	src/bin/sage: Use 'make SPKG-uninstall' instead of -clean

[sagetrac-git]

Changed commit from e4c27f6 to 81a63df

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

3d6fa59

build/make/Makefile.in: Replace 'SPKG-clean' by 'SPKG-uninstall', with deprecation, for pip packages too

[sagetrac-git]

Changed commit from 81a63df to 3d6fa59

[mkoeppe]

Description changed:

--- 
+++ 
@@ -8,5 +8,9 @@
 
 
 We also make the decision whether `spkg-install` is generated from a template `spkg-install.in` or provided as is in the source tree orthogonal to the distinction `source=script` vs. `source=pip`.
+We simply generate `spkg-install` if `spkg-install.in` is present.
 
 
+We also rename targets `SPKG-clean` to `SPKG-uninstall` (as originally discussed in #29097), leaving a deprecated target behind.  This is to unify the targets between normal and script packages (the latter used `-uninstall` already). 
+
+