After #30053, sphinx 3.1.2 does not build on ubuntu-{trusty,xenial,bionic}, debian-jessie, centos-7 (again)

[kliem]

sage-system-python is the Python that we use to download and unpack upstream archives. We allow a very wide range of Python versions.

After #30053 undid the change in #29033, we are again running into locale / encoding problems with early (pre-3.7) versions of Python used as sage-system-python.

This is a blocker ticket for Sage 9.2 because it is a severe regression in platform support compared to Sage 9.1.

Originally reported on this ticket:

/sage/build/bin/sage-spkg: line 73: warning: setlocale: LC_ALL: cannot change locale (C.UTF-8): No such file or directory
/sage/build/bin/sage-spkg: line 73: warning: setlocale: LC_ALL: cannot change locale (C.UTF-8)
Found local metadata for sphinx-3.0.4.p0
bash: warning: setlocale: LC_ALL: cannot change locale (C.UTF-8)
Attempting to download package Sphinx-3.0.4.tar.gz from mirrors
http://mirrors.xmission.com/sage/spkg/upstream/sphinx/Sphinx-3.0.4.tar.gz
[......................................................................]
sphinx-3.0.4.p0
====================================================
Setting up build directory for sphinx-3.0.4.p0
bash: warning: setlocale: LC_ALL: cannot change locale (C.UTF-8)
Traceback (most recent call last):
  File "/sage/build/bin/sage-uncompress-spkg", line 23, in <module>
    run()
  File "/sage/build/bin/../sage_bootstrap/uncompress/cmdline.py", line 72, in run
    unpack_archive(archive, dirname)
  File "/sage/build/bin/../sage_bootstrap/uncompress/action.py", line 68, in unpack_archive
    archive.extractall(members=archive.names)
  File "/sage/build/bin/../sage_bootstrap/uncompress/tar_file.py", line 96, in extractall
    **kwargs)
  File "/usr/lib64/python3.6/tarfile.py", line 2010, in extractall
    numeric_owner=numeric_owner)
  File "/usr/lib64/python3.6/tarfile.py", line 2052, in extract
    numeric_owner=numeric_owner)
  File "/sage/build/bin/../sage_bootstrap/uncompress/tar_file.py", line 122, in _extract_member
    **kwargs)
  File "/usr/lib64/python3.6/tarfile.py", line 2122, in _extract_member
    self.makefile(tarinfo, targetpath)
  File "/usr/lib64/python3.6/tarfile.py", line 2163, in makefile
    with bltn_open(targetpath, "wb") as target:
UnicodeEncodeError: 'ascii' codec can't encode character '\xe4' in position 45: ordinal not in range(128)
************************************************************************
Error: failed to extract /sage/upstream/Sphinx-3.0.4.tar.gz
************************************************************************
Please email sage-devel (http://groups.google.com/group/sage-devel)
explaining the problem and including the log file
  /sage/logs/pkgs/sphinx-3.0.4.p0.log
Describe your computer, operating system, etc.
************************************************************************

See e.g. https://github.com/sagemath/sage/runs/811777719?check_suite_focus=true

But now (https://github.com/mkoeppe/sage/runs/1141520147) we see it for ubuntu-trusty-minimal (where sage-system-python is python 3.4).

We fix this problem by making locale fixes in the sage-system-python script. See e.g. https://github.com/mkoeppe/sage/runs/1142310012 for a run with the present ticket.

Depends on #30053

CC: @vbraun @dimpase @orlitzky @jhpalmieri @seblabbe

Component: packages: standard

Author: Matthias Koeppe

Branch/Commit: ff0dbc6

Reviewer: Jonathan Kliem, Sébastien Labbé

Issue created by migration from https://trac.sagemath.org/ticket/30008

[mkoeppe]

comment:1

That's a very strange error

[mkoeppe]

comment:2

https://bugs.python.org/issue17153

[mkoeppe]

comment:4

Perhaps #30001 happens to fix it?

[kliem]

comment:5

Testing here:

https://github.com/kliem/sage/pull/17/checks

[kliem]

comment:6

No, doesn't help. At least not as it is now.

[kliem]

comment:7

According to this

https://www.noreplied.com/how-to-fixed-cannot-change-locale-utf-8-error-in-centos-7/

one should add LC_CTYPE="en_US.UTF-8" to /etc/environment. Very unsatisfying fix for someone without root privileges.

This appears to be a bug with centos 7, which is merely exposed now.

[mkoeppe]

comment:8

Just exporting this environment variable setting in build/bin/sage-spkg should be enough. No need for a systemwide setting

[kliem]

comment:9

Ok. Testing this now. https://github.com/kliem/sage/pull/19/checks

[dimpase]

comment:10

apparently should be fixed by #30053

[kliem]

comment:11

It appears that things fixed itself: https://github.com/sagemath/sage/actions/runs/166309975