Add a library of common helper functions for use in spkg-install

[embray]

This idea has come up before but not been implemented. My implementation simply adds a collection of shell functions with names prefixed by sdh_ for Sage distribution helpers (this is somewhat inspired by, but much simpler than, the dh_ programs from the debhelper package in Debian).

These functions are exported by sage-spkg, and so are immediately usable by any spkg-build, spkg-install, or similar scripts executed from sage-spkg (these are still not required to be shell scripts, but as most are having such a library is convenient).

One piece of boilerplate I replaced with a function is the check in most spkg-installs for $SAGE_LOCAL. A downside to this is that if the script is run outside the correct environment, then rather than getting a useful error message we just get that sdh_check_vars is undefined. I have two thoughts on this:

1. In general these scripts shouldn't be run directly in the first place. In the case that they are (such as testing) the person developing the package should have some awareness of the fact that these scripts are being run from sage-spkg and assume sage-env has been sourced.

2. Most of the time these scripts are run, it's after they've been copied to a temporary build directory for the package. If nothing else, sage-spkg could create a wrapper around these scripts that automatically sets some default assumptions (such as sourcing sage-env and the new sage-dist-helpers).

In general, though, I think having a library of helper functions will be a big improvement to the consistency and legibility of theses scripts.

This ticket also updates the spkg-install for patch for demonstration purposes only. I don't intend with this to go on a mass rewriting of install scripts, but having this will be helpful for some other work (e.g. #22509, #22510).

Depends on #23179

Component: build

Author: Erik Bray

Branch/Commit: 48fe6ac

Reviewer: Jeroen Demeyer

Issue created by migration from https://trac.sagemath.org/ticket/23160

[embray]

comment:1

This could also replace the $PIP_INSTALL variable that's used for Python packages with an sdh_pip_install helper, serving the same purpose, but that could be done separately.

[jdemeyer]

comment:2

The whole point of this

if [ -z "$SAGE_LOCAL" ]; then
    echo >&2 "Error: SAGE_LOCAL undefined - exiting..."
    echo >&2 "Maybe run 'sage -sh'?"
    exit 1
fi

is to abort when the variable is not defined. A plain sdh_check_common_vars does not do that. At least replace it by sdh_check_common_vars || exit $? or something like that.

[jdemeyer]

comment:3

I would also rename sdh_check_common_vars to something like sdh_guard (or sdh_begin or sdh_check). It's not important what sdh_check_common_vars does, only that it guards against accidental executing of spkg-install when the necessary variables are not defined.

[embray]

comment:4

I'm not sure I understand. It's already defined to exit the script immediately.

[embray]

comment:5

Replying to @embray:

I'm not sure I understand. It's already defined to exit the script immediately.

Oh, you mean if it's not defined in the first place.

[embray]

comment:6

One point I will make though, is that if more of the build functionality is moved into helper functions there's also less effect a script can have if it's run outside the sage environment. For example if sdh_make_install is not defined, then nothing will be installed. That's no a silver bullet though, so I can still see the usefulness of having this guard in place.

[embray]

comment:7

Okay, I'd be fine with something like sdh_guard || exit $?. It's still boilerplate but at least now it's only one line.

[embray]

comment:8

That said, this brings me back around to the idea I was mulling earlier, that maybe the spkg-* scripts that live in the package directories shouldn't be executable in the first place. That as, they would not have executable permissions, and not have a shebang line.

Only when the script is copied to the temporary install directory does it get marked executable, and gets a header automatically written which, among other things, ensures that the correct sage-env is sourced. So there's generally no danger in running it outside the correct sage environment.

This might limit it to being a shell script, but it could still be a wrapper for a non-shell executable if desired (there are only a couple cases of this currently, such as atlas).

[jdemeyer]

comment:9

Replying to @embray:

Only when the script is copied to the temporary install directory does it get marked executable, and gets a header automatically written which, among other things, ensures that the correct sage-env is sourced.

My gut feeling is that this is a bit too much magic. At least the sdh_guard || exit $? boilerplate is easy to understand with little surprises.

[embray]

comment:10

That was my feeling too at first, which is why I dropped the idea initially. But now that I've tried it I think it has a number of advantages, including the fact that it automatically makes these scripts more robust without their authors having to think about details that aren't immediately related to building/installing the package.

[embray]

comment:11

Have a look at #23179.

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

1b96a66	Make it a mere warning on Cygwin if any of the scripts in build/pkgs are executable, but a hard error on other platforms
b1155dd	Use a heredoc instead of a quoted variable, for clarity's sake.
d28f35d	Add an explicit variable indicating if this is an old-style package
4c4941f	Restore support old-style packages, for now, by not creating script wrappers
a6163ce	Updated the developer docs to reflect the changes in #23179
4f52331	Adds initial version of sage-dist-helpers
f07cb21	Use the handful of implemented helpers in 'patch' for demonstration purposes
55b68ab	Additional args shouldn't be quoted
a01a1bd	Add support for $SAGE_SUDO in all sdh_make_install calls
2b5e33d	As suggested, rename sdh_check_common_vars to sdh_guard