Allow using arb, flint and ntl from Homebrew

[mkoeppe]

• ntl/flint/singular: https://groups.google.com/d/msg/sage-devel/KXK_zxzfhIQ/0MaSLhvMAAAJ
• ecl: Reported in https://groups.google.com/d/msg/sage-devel/gFaUNw4v3aw/P-dTJ5IOCgAJ

/usr/local is leaking into our build, through wrong orders of include and/or library directives.

Critical for 9.3 because it affects a major supported platform. We should at least give a configure-time error if these installations are found on homebrew if we cannot fix this.

Depends on #29719

CC: @dimpase @jhpalmieri @vbraun @nbruin @kiwifb

Component: build: configure

Author: Dima Pasechnik

Branch/Commit: 2fd95cd

Reviewer: John Palmieri

Issue created by migration from https://trac.sagemath.org/ticket/30745

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,4 +1,4 @@
-- ntl/flint: https://groups.google.com/d/msg/sage-devel/KXK_zxzfhIQ/0MaSLhvMAAAJ
+- ntl/flint/singular: https://groups.google.com/d/msg/sage-devel/KXK_zxzfhIQ/0MaSLhvMAAAJ
 - ecl: Reported in https://groups.google.com/d/msg/sage-devel/gFaUNw4v3aw/P-dTJ5IOCgAJ
 
 `/usr/local` is leaking into our build, through wrong orders of include and/or library directives.

[dimpase]

comment:2

upstream ecl is indeed useless, as no known to me distro (gentoo, homebrew) provides sufficiently patched ecl-20.
So we can have a spkg-configure.m4 for it, erroring out if it's found, on any distro.

perhaps ntl/flint/singular combo on homebrew is good enough for Sage?

[vbraun]

comment:3

Traditionally build systems are supposed to pick up stuff in /usr/local, so its a bit of a feature. Gentoo doesn't install anything there afaik, it is intended for stuff that users install by hand (i.e. non-portage in Gentoo). I consider it the big homebrew WTF.

Realistically, if you have stuff installed in /usr/local then you don't have a supported platform any more. Sure we should try to accommodate homebrew but imho not a blocker.

[mkoeppe]

comment:4

Replying to @vbraun:

Realistically, if you have stuff installed in /usr/local then you don't have a supported platform any more. Sure we should try to accommodate homebrew but imho not a blocker.

Given that since Sage 9.1, we recommend homebrew packages that the user should install, I don't think the view that a system with homebrew installed in /usr/local is not a "supported platform" has much practical value.

Setting it to "critical" rather than "blocker" because there is, of course, an easy remedy - just uninstall the useless homebrew packages that cause the problem.

[mkoeppe]

Description changed:

--- 
+++ 
@@ -3,4 +3,4 @@
 
 `/usr/local` is leaking into our build, through wrong orders of include and/or library directives.
 
-Blocker for 9.2 because it affects a major supported platform. We should at least give a configure-time error if these installations are found on homebrew if we cannot fix this.
+Critical for 9.2 because it affects a major supported platform. We should at least give a configure-time error if these installations are found on homebrew if we cannot fix this.

[dimpase]

comment:5

Replying to @vbraun:

Traditionally build systems are supposed to pick up stuff in /usr/local, so its a bit of a feature. Gentoo doesn't install anything there afaik, it is intended for stuff that users install by hand (i.e. non-portage in Gentoo). I consider it the big homebrew WTF.

Realistically, if you have stuff installed in /usr/local then you don't have a supported platform any more. Sure we should try to accommodate homebrew but imho not a blocker.

Despite your personal misgivings about Homebrew, this is a non-starter, Volker, sorry. Homebrew is the most popular macOS package manager out there, and it goes without saying that we want to support as many packages there as possible, and hopefully have a proper Homebrew Sage package.

As well, many *BSD systems install their packages in /usr/local, e.g. FreeBSD (where we now have a semi-working native package, with all the deps neatly divided into native packages), OpenBSD, etc.

[mkoeppe]

comment:6

Trying this with tox -e local-homebrew-usrlocal-standard. Building the ecl extension uses the following command line:

gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -I/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -Isage/libs -I./sage/cpython -I/Users/mkoeppe/s/sage/sage-rebasing/.tox/local-homebrew-usrlocal-standard/local/lib/python3.8/site-packages/cysignals -I/Users/mkoeppe/s/sage/sage-rebasing/build/pkgs/sagelib/src -I/Users/mkoeppe/s/sage/sage-rebasing/build/pkgs/sagelib/src/sage/ext -I/usr/local/Cellar/python@3.8/3.8.6/Frameworks/Python.framework/Versions/3.8/include/python3.8 -I/Users/mkoeppe/s/sage/sage-rebasing/.tox/local-homebrew-usrlocal-standard/local/lib/python3.8/site-packages/numpy/core/include -Ibuild/cythonized -I/usr/local/include -I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/sqlite/include -I/Users/mkoeppe/s/sage/sage-rebasing/.tox/local-homebrew-usrlocal-standard/local/include -I/usr/local/Cellar/python@3.8/3.8.6/Frameworks/Python.framework/Versions/3.8/include/python3.8 -c build/cythonized/sage/libs/ecl.c -o build/temp.macosx-10.15-x86_64-3.8/build/cythonized/sage/libs/ecl.o -fno-strict-aliasing -DCYTHON_CLINE_IN_TRACEBACK=1 -std=c99

which indeed has /usr/local/include in front of $SAGE_LOCAL/include. (There is also CPATH, which is set to $SAGE_LOCAL/include; but CPATH is searched "after any paths given with -I options on the command line" - https://gcc.gnu.org/onlinedocs/gcc/Environment-Variables.html)

[mkoeppe]

comment:7

A related recent change was made in #29697 (sage_include_directories: Do not add another copy of SAGE_INC, SAGE_LOCAL/lib to include dirs, library dirs).

[mkoeppe]

comment:8

The correct fix for ecl is probably to call ecl-config to determine the correct flags. #30770

[dimpase]

comment:9

arb, ntl, and flint can and should just be enabled on Homebrew. ECL is a different story, as it lacks patches.

[mkoeppe]

comment:10

It's not quite that easy, I am afraid. There are still the threading issues such as #27764

[dimpase]

comment:11

Replying to @mkoeppe:

It's not quite that easy, I am afraid. There are still the threading issues such as #27764

these issues have solved themselves - I am able to build and doctest on macOS 10.15.7 with arb, ntl, flint from Homebrew just fine
(with #30805 and doctest patches from the Flint 2.6 upgrade ticket).

[dimpase]

comment:12

more precisely, I see 2 doctests failing in src/sage/rings/polynomial/plural.pyx - not sure if this is related. The rest of make ptest passes.