GH-139914: Handle stack growth direction on HPPA #140028

stefanor · 2025-10-13T08:49:45Z

Adapted from a patch for Python 3.14 submitted to the Debian BTS by John David Anglin https://bugs.debian.org/1105111#20

The backport to the 3.14 branch is non-trivial as some of the affected code was refactored in 7094f09 but the patch in the Debian BTS was targetted at 3.14 and can be referenced to backport.

Issue: Python 3.14 fails to build on hppa, error indicates stack overflow (false positive?) #139914

picnixz · 2025-10-13T09:48:20Z

No news, as this is a simple regression fix

Please do so. A regression fix needs to be announced so that users know that their bug has been indeed fixed.

picnixz · 2025-10-13T10:00:30Z

Python/ceval.c

    uintptr_t here_addr = _Py_get_machine_stack_pointer();
    _PyThreadStateImpl *_tstate = (_PyThreadStateImpl *)tstate;
+#ifdef __hppa__
+    if (here_addr <= _tstate->c_stack_soft_limit - margin_count * _PyOS_STACK_MARGIN_BYTES) {


Can't we pre-compute the bound?

The immediate answer is: No because margin_count is a function parameter.

However... This is always set to 1, so it could be removed...

Sorry, I meant to avoid computing twice _tstate->c_stack_soft_limit - margin_count * _PyOS_STACK_MARGIN_BYTES to avoid having a double ifdef in this function.

_Py_InitializeRecursionLimits() can change the limits if c_stack_hard_limit == 0 (I assume that's the uninitialized state)

We can pre-compute margin_count * _PyOS_STACK_MARGIN_BYTES (used 4 times in the source) potentially saving a multiply.

I assume that's the uninitialized state

That's also something I actually find weird. If we're in an uninitialized state, how come can we use the soft limit? anyway, let's leave the code as is as it could actually be less readable in the end and introduce a subtle issue if _Py_InitializeRecursionLimits could change the soft limit value at that point.

Come to think of it, we only hit that second if block if we pass the first one. Given that we're reversing the direction of the comparison, on HPPA we won't get here if c_stack_soft_limit is 0.

@markshannon: Why do we call _Py_InitializeRecursionLimits here?

Because this in an API function, and the stack limits might not have been initialized.
Maybe we are being overly conservative, but it is harmless to check.

Don't we risk never reaching never reaching the initialisation check on most platforms, if c_stack_soft_limit is 0? (And possibly on HPPA too, due to underflow of an unsigned int)

It seems like this whole first if block should just be removed.

Python/ceval.c

I meant to comment, not approve.

markshannon

The general implementation looks sound.

Rather than mysterious #ifdef __hppa__ could you #define STACK_GROWS_DOWN in the config and use #if STACK_GROWS_DOWN for clarity and to keep things tidy in case any other platform has a stack the grows up.

OOI what is hppa? Is this PA-RISC, or some new variant?

Include/internal/pycore_ceval.h

markshannon · 2025-10-14T14:09:45Z

Python/ceval.c

    uintptr_t here_addr = _Py_get_machine_stack_pointer();
    _PyThreadStateImpl *_tstate = (_PyThreadStateImpl *)tstate;
+#ifdef __hppa__
+    if (here_addr <= _tstate->c_stack_soft_limit - margin_count * _PyOS_STACK_MARGIN_BYTES) {


Because this in an API function, and the stack limits might not have been initialized.
Maybe we are being overly conservative, but it is harmless to check.

Python/ceval.c

bedevere-app · 2025-10-14T14:15:31Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

stefanor · 2025-10-14T15:46:04Z

OOI what is hppa? Is this PA-RISC, or some new variant?

Yes, it is. Still going as Debian port. The newest hardware is 20 years old at this point, but the port is still alive. It'll probably keep going as long as GCC and Linux support it.

Include/internal/pycore_ceval.h

encukou · 2025-10-17T12:35:05Z

Thank you for upstreaming the patch!

Do the tests pass on HPPA? I would think that TestRecursion in Lib/test/test_call.py needs an update too.

I've sent a PR to your branch to expose _Py_STACK_GROWS_DOWN to Python tests (as _testcapi._Py_STACK_GROWS_DOWN), and to the rest of CPython (through Autuconf).

This PR will conflict with #139668, which will take some time to approve as it adds new API. If this PR is not urgent, I think it's best to wait for #139668, then rebase this one and ask you to check the result. Does that sound OK?

python-cla-bot · 2025-10-19T18:11:50Z

All commit authors signed the Contributor License Agreement.

stefanor · 2025-10-20T07:07:28Z

Do the tests pass on HPPA?

I had run the tests and seen enough failures that I assumed HPPA was already in a bad state re tests. I'm afraid we don't run them automatically in Debian due to limited hardware capacity.

thesamesam · 2025-10-20T11:22:50Z

We have some skips on our end for HPPA but it's in a reasonable state overall. I haven't rechecked with this patch though.

(I just mention this in case someone stumbles upon it in future and thinks Python is broken there, as it isn't.)

Include/internal/pycore_ceval.h

Misc/NEWS.d/next/Core_and_Builtins/2025-10-13-13-54-19.gh-issue-139914.M-y_3E.rst

AA-Turner · 2025-10-20T12:06:53Z

configure.ac

 fi
 AC_SUBST([MULTIARCH_CPPFLAGS])

+# Guess C stack direction


Is ‘guess’ the right word here?

Yes. For an arbitrary platform we can't really be sure.

Include/pyport.h

Adapted from a patch for Python 3.14 submitted to the Debian BTS by John https://bugs.debian.org/1105111#20 Co-authored-by: John David Anglin <[email protected]>

AA-Turner · 2025-10-20T15:32:09Z

@stefanor please don't force-push, per guidance in devguide etc.

A

stefanor · 2025-10-20T15:36:56Z

I checked the test state more thoroughly. The 3.13 branch (08a2b2d) only has a single failure, but there's a lot in this branch. It looks like we have a GC bug to find.

https://gist.github.com/stefanor/393e358310b4dd791f6cbd2397fdcd58

stefanor · 2025-10-20T15:38:49Z

@stefanor please don't force-push, per guidance in devguide etc.

Sorry, I try to keep my changes as a readable series of commits. I forgot the local rules here :)
(I mostly work in GitLab where you can diff between pushes)

stefanor · 2025-10-21T20:36:51Z

It looks like we have a GC bug to find.

Bisection of test_gc failure takes me to 44e4c47 - where I found one more stack computation.

stefanor · 2025-10-22T05:13:07Z

Much better:

== Tests result: FAILURE then FAILURE ==

10 slowest tests:
- test.test_concurrent_futures.test_process_pool: 9 min 53 sec
- test_regrtest: 7 min 43 sec
- test_tarfile: 7 min 37 sec
- test_zipfile: 7 min 26 sec
- test_fstring: 7 min 23 sec
- test.test_multiprocessing_spawn.test_manager: 6 min 53 sec
- test.test_multiprocessing_spawn.test_misc: 6 min 14 sec
- test_multiprocessing_main_handling: 6 min 11 sec
- test_ssl: 5 min 51 sec
- test_compileall: 5 min 3 sec

24 tests skipped:
    test.test_asyncio.test_windows_events
    test.test_asyncio.test_windows_utils test.test_gdb.test_backtrace
    test.test_gdb.test_cfunction test.test_gdb.test_cfunction_full
    test.test_gdb.test_misc test.test_gdb.test_pretty_print
    test.test_os.test_windows test_android test_apple test_devpoll
    test_free_threading test_kqueue test_launcher test_msvcrt
    test_perf_profiler test_perfmaps test_samply_profiler
    test_startfile test_winapi test_winconsoleio test_winreg
    test_winsound test_wmi

4 tests skipped (resource denied):
    test_peg_generator test_tkinter test_ttk test_zipfile64

9 re-run tests:
    test.test_asyncio.test_ssl
    test.test_multiprocessing_forkserver.test_processes
    test.test_multiprocessing_spawn.test_processes test_capi
    test_ctypes test_profiling test_socket test_struct test_tools

8 tests failed:
    test.test_asyncio.test_ssl
    test.test_multiprocessing_spawn.test_processes test_capi
    test_ctypes test_profiling test_socket test_struct test_tools

459 tests OK.

Total duration: 1 hour 59 min
Total tests: run=46,818 failures=47 skipped=1,948
Total test files: run=500/495 failed=8 skipped=24 resource_denied=4 rerun=9

While looking at python#140028 I found some unrelated test regressions in the 3.14 cycle. These seem to all come from python#130317. From what I can tell, that made Python more correct than it was before. According to [0] HP PA RISC uses 1 for SNaN and thus a 0 for QNaN. Update tests to expect this. [0]: https://grouper.ieee.org/groups/1788/email/msg03272.html

While looking at python#140028 I found some test failures that are caused by new tests (from python#138122) running too slowly. This adds an arbitrary heuristic to double the sampling run time. We could do 10x instead? And/or move the heuristic into test_support. Thoughts?

While looking at python#140028 I found some test failures that are caused by new tests (from python#138122) running too slowly. This adds an arbitrary heuristic to 10x the sampling run time (to the default value of 10 seconds). Doubling the 1-second duration was sufficient for my HP PA tests, but Fedora reported one of the 2-second durations being too slow for a freethreaded build. This heuristic could move into test_support. Thoughts?

While looking at #140028, I found some unrelated test regressions in the 3.14 cycle. These seem to all come from #130317. From what I can tell, that made Python more correct than it was before. According to [0], HP PA RISC uses 1 for SNaN and thus a 0 for QNaN. [0]: https://grouper.ieee.org/groups/1788/email/msg03272.html

While looking at pythonGH-140028, I found some unrelated test regressions in the 3.14 cycle. These seem to all come from pythonGH-130317. From what I can tell, that made Python more correct than it was before. According to [0], HP PA RISC uses 1 for SNaN and thus a 0 for QNaN. [0]: https://grouper.ieee.org/groups/1788/email/msg03272.html (cherry picked from commit 76fea5596c235a7853cda8df87c3998d506e950c) Co-authored-by: Stefano Rivera <[email protected]>

…40467) gh-130317: Fix SNaN broken tests on HP PA RISC (GH-140452) While looking at GH-140028, I found some unrelated test regressions in the 3.14 cycle. These seem to all come from GH-130317. From what I can tell, that made Python more correct than it was before. According to [0], HP PA RISC uses 1 for SNaN and thus a 0 for QNaN. [0]: https://grouper.ieee.org/groups/1788/email/msg03272.html (cherry picked from commit 76fea55) Co-authored-by: Stefano Rivera <[email protected]>

stefanor requested a review from markshannon as a code owner October 13, 2025 08:49

bedevere-app bot mentioned this pull request Oct 13, 2025

Python 3.14 fails to build on hppa, error indicates stack overflow (false positive?) #139914

Open

bedevere-app bot added the awaiting review label Oct 13, 2025

picnixz changed the title ~~GH-139914: On HPPA, the stack grows up~~ GH-139914: Handle stack growth direction on HPPA Oct 13, 2025

picnixz previously approved these changes Oct 13, 2025

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting review labels Oct 13, 2025

bedevere-app bot added awaiting review and removed awaiting merge labels Oct 13, 2025

stefanor force-pushed the hppa-stack branch from b7962b3 to 5f1ed94 Compare October 13, 2025 11:59

markshannon requested changes Oct 14, 2025

View reviewed changes

bedevere-app bot added awaiting changes and removed awaiting review labels Oct 14, 2025

markshannon mentioned this pull request Oct 14, 2025

gh-139653: Add PyUnstable_ThreadState_SetStack() #139668

Open

stefanor requested a review from markshannon October 14, 2025 16:07

vstinner reviewed Oct 14, 2025

View reviewed changes

Include/internal/pycore_ceval.h Outdated Show resolved Hide resolved

stefanor force-pushed the hppa-stack branch from 6897b42 to 17fa052 Compare October 14, 2025 20:18

encukou added the OS-unsupported label Oct 17, 2025

github-project-automation bot added this to Unsupported platforms Oct 17, 2025

encukou mentioned this pull request Oct 17, 2025

_Py_STACK_GROWS_DOWN: use Autoconf & expose in _testcapi stefanor/cpython#1

Merged

stefanor requested review from AA-Turner, corona10, emmatyping and erlend-aasland as code owners October 19, 2025 18:11

stefanor force-pushed the hppa-stack branch from f2efafc to 695bbdc Compare October 19, 2025 21:23

stefanor force-pushed the hppa-stack branch from 5e6c2b5 to 5f794c0 Compare October 20, 2025 09:34

AA-Turner reviewed Oct 20, 2025

View reviewed changes

stefanor and others added 6 commits October 20, 2025 17:21

pythonGH-139914: On HPPA, the stack grows up

a373f4e

Adapted from a patch for Python 3.14 submitted to the Debian BTS by John https://bugs.debian.org/1105111#20 Co-authored-by: John David Anglin <[email protected]>

Define _Py_STACK_GROWS_DOWN to declare the stack direction

c0c9000

Use autoconf for _Py_STACK_GROWS_DOWN

7fec939

Expose _Py_STACK_GROWS_DOWN to tests

a096813

Include hppa64-* and hppaX.Y-* in the autoconf check

c9f0215

Support a stack that grows up in test_call

4434b24

stefanor force-pushed the hppa-stack branch from 5f794c0 to 4434b24 Compare October 20, 2025 15:23

Found another stack computation

b217e3f

stefanor requested review from ZeroIntensity and ericsnowcurrently as code owners October 21, 2025 20:36

stefanor mentioned this pull request Oct 22, 2025

gh-130317: Fix SNaN broken tests on HP PA RISC #140452

Merged

stefanor mentioned this pull request Oct 22, 2025

gh-140381: Handle slower machines in test_profiling #140460

Open

miss-islington mentioned this pull request Oct 22, 2025

[3.14] gh-130317: Fix SNaN broken tests on HP PA RISC (GH-140452) #140467

Merged

Uh oh!

GH-139914: Handle stack growth direction on HPPA #140028

Are you sure you want to change the base?

GH-139914: Handle stack growth direction on HPPA #140028

Uh oh!

Conversation

stefanor commented Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

picnixz commented Oct 13, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stefanor Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stefanor Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

markshannon left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

bedevere-app bot commented Oct 14, 2025

Uh oh!

stefanor commented Oct 14, 2025

Uh oh!

Uh oh!

encukou commented Oct 17, 2025

Uh oh!

python-cla-bot bot commented Oct 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stefanor commented Oct 20, 2025

Uh oh!

thesamesam commented Oct 20, 2025

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

AA-Turner commented Oct 20, 2025

Uh oh!

stefanor commented Oct 20, 2025

Uh oh!

stefanor commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stefanor commented Oct 21, 2025

Uh oh!

stefanor commented Oct 22, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

stefanor commented Oct 13, 2025 •

edited

Loading

stefanor Oct 13, 2025 •

edited

Loading

stefanor Oct 14, 2025 •

edited

Loading

python-cla-bot bot commented Oct 19, 2025 •

edited

Loading

stefanor commented Oct 20, 2025 •

edited

Loading