-
-
Notifications
You must be signed in to change notification settings - Fork 11.1k
DEP: Deprecate setting the strides attribute of a numpy array #28925
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
I'm having a bit of a déjà vu here 🤔 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, the deprecation needs to be moved, and it would be nice if you can fix the habit of putting spaces around kwarg=value
.
Otherwise, this mainly needs a release note now that explains the alternatives.
numpy/_core/src/multiarray/getset.c
Outdated
@@ -116,6 +116,10 @@ array_strides_get(PyArrayObject *self, void *NPY_UNUSED(ignored)) | |||
static int | |||
array_strides_set(PyArrayObject *self, PyObject *obj, void *NPY_UNUSED(ignored)) | |||
{ | |||
if( DEPRECATE("Setting the strides on a NumPy array has been deprecated in NumPy 2.3.") < 0 ) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It may be nice to point out the work-arounds, but considering that stride setting is a bit niche maybe we can also just do that in the release note.
Something like: See np.lib.stride_tricks.strided_window_view
and np.lib.stride_tricks.as_strided
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And here is a long line, which needs space fixes also.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you are at it, adding a /* Deprecated NumPy 2.3, 2025-05-11 */
above it would also be nice.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree the warning needs a suggestion for what to do instead and maybe a pointer to stride_tricks if someone really does need to mutate the strides of an existing array.
numpy/_core/tests/test_multiarray.py
Outdated
@@ -8290,7 +8297,8 @@ def test_padded_struct_array(self): | |||
def test_relaxed_strides(self, c=np.ones((1, 10, 10), dtype='i8')): | |||
# Note: c defined as parameter so that it is persistent and leak | |||
# checks will notice gh-16934 (buffer info cache leak). | |||
c.strides = (-1, 80, 8) # strides need to be fixed at export | |||
with pytest.warns(DeprecationWarning): | |||
c.strides = (-1, 80, 8) # strides need to be fixed at export |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The ones we shouldn't just delete when finalizing would be nice to just replace immediately, but also not a big thing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did not change this one because I was not sure about the test. If we use the stride_tricks
then we are creating a new array and might not catch the memory leak. Probably fine either way.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, the comment makes sense if a bit hard to follow. That just means you need to move the as_strided
call into the argument, though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you just change this? I.e. by moving the unwieldy or not, as_strided
or np.ndarray
call into the c=
kwarg.
(Needs a bit of line breaking, I suppose black style will do)
numpy/_core/tests/test_multiarray.py
Outdated
with suppress_warnings() as sup: | ||
sup.filter(DeprecationWarning, "Assigning the 'data' attribute") | ||
for s in attr: | ||
assert_raises(AttributeError, delattr, a, s) | ||
|
||
attr = ['strides'] | ||
with suppress_warnings() as sup: | ||
sup.filter(DeprecationWarning, "Assigning the 'data' attribute") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
N.B./nit: you should be fine with catch_warnings
now. Python fixed the bugs that made sup
necessary.
numpy/_core/tests/test_multiarray.py
Outdated
sup.filter(DeprecationWarning, "Assigning the 'data' attribute") | ||
for s in attr: | ||
with pytest.warns(DeprecationWarning): | ||
assert_raises(AttributeError, delattr, a, s) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is overly complicated (probably including that filter above), because you are giving the warning when it'll raise anyway.
You should just move the warning to after the error for deleting (value is NULL
).
(Or actually, move that error to the very beginning, since we are not bound by C89 style "declarations first" anymore.)
And I forgot to say, thanks splitting it out is much easier considering how unwieldy especially the |
This is not flagged by the linter. It is E251 (https://docs.astral.sh/ruff/rules/unexpected-spaces-around-keyword-parameter-equals/) should I add that to the configuration in a different PR? Runnnig it on the current codebase gives 52 changes. |
Yeah, that would be great. I am very confused why this isn't enabled. IIRC, long lines also don't seem to get flagged. |
PR created. The long lines is about 1500 changes which is maybe a bit too much? With |
Yeah, I hadn't realized that we don't have an lint on the newly added lines anymore, that used to be the setup. So, I think we should probably also just do it, and just before branching 2.3 is the right time (simplifies backports), but I'll let @charris make the decision. (Or it would be nice to fix this and re-introduce a more strict check only on changed code, that should not be hard!) |
Hmm. Ruff cannot automatically fix long lines (E501). So this would mean manually changing 1500 lines which is not impossible, but certainly not a quick job. There is also |
For fixing long line, I'd probably start with non-test files and do it in bits. One of the problems is data in tests, which we like to keep aligned for readability. But yeah, there are a lot of long lines. Some might be easy to fix, but others might be a pain, which why I suggest doing it in parts. For example, do |
BTW, please use hard line breaks when writing notes, don't depend on line wrap. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, a few nitpicks and things that would be nice to refactor now. But basically ready to go in, IMO, thanks!
numpy/_core/tests/test_multiarray.py
Outdated
@@ -8290,7 +8297,8 @@ def test_padded_struct_array(self): | |||
def test_relaxed_strides(self, c=np.ones((1, 10, 10), dtype='i8')): | |||
# Note: c defined as parameter so that it is persistent and leak | |||
# checks will notice gh-16934 (buffer info cache leak). | |||
c.strides = (-1, 80, 8) # strides need to be fixed at export | |||
with pytest.warns(DeprecationWarning): | |||
c.strides = (-1, 80, 8) # strides need to be fixed at export |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you just change this? I.e. by moving the unwieldy or not, as_strided
or np.ndarray
call into the c=
kwarg.
(Needs a bit of line breaking, I suppose black style will do)
numpy/_core/src/multiarray/getset.c
Outdated
@@ -116,6 +116,10 @@ array_strides_get(PyArrayObject *self, void *NPY_UNUSED(ignored)) | |||
static int | |||
array_strides_set(PyArrayObject *self, PyObject *obj, void *NPY_UNUSED(ignored)) | |||
{ | |||
if( DEPRECATE("Setting the strides on a NumPy array has been deprecated in NumPy 2.3.") < 0 ) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you are at it, adding a /* Deprecated NumPy 2.3, 2025-05-11 */
above it would also be nice.
numpy/_core/tests/test_regression.py
Outdated
assert np.lexsort((xs,), axis=0).shape[0] == 0 | ||
|
||
xs.shape = (2, 0) | ||
xs.strides = (16, 16) | ||
with pytest.warns(DeprecationWarning): | ||
xs.strides = (16, 16) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These ones would be nice to change, since the test is unrelated to the strides setting itself.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have you tried checking whether this impacts downstream projects? It might be polite to check e.g. the scipy, matplotlib, pandas, and scikit-learn to see how much noise this causes in their tests.
* `np.lib.stride_tricks.as_strided` for the general case, | ||
* or the `np.ndarray` constructor (``buffer`` is the original array) for a light-weight version. | ||
|
||
Manually setting strides can be useful but needs to be done with care as it can lead to unsafe memory use. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure what you mean by "manually setting strides" here. You mean passing strides
to the ndarray constructor or something else? Please clarify.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated the text: we mean setting the strides on a numpy array directly.
numpy/_core/src/multiarray/getset.c
Outdated
@@ -116,6 +116,10 @@ array_strides_get(PyArrayObject *self, void *NPY_UNUSED(ignored)) | |||
static int | |||
array_strides_set(PyArrayObject *self, PyObject *obj, void *NPY_UNUSED(ignored)) | |||
{ | |||
if( DEPRECATE("Setting the strides on a NumPy array has been deprecated in NumPy 2.3.") < 0 ) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree the warning needs a suggestion for what to do instead and maybe a pointer to stride_tricks if someone really does need to mutate the strides of an existing array.
needs a rebase |
Co-authored-by: Sebastian Berg <[email protected]>
44b4a93
to
288690d
Compare
This comment has been minimized.
This comment has been minimized.
scikit-learn tests pass locally, matplotlib has one failing test (lib/matplotlib/tests/test_backend_tk.py::test_canvas_focus) which seems unrelated to this PR. Searching for strides in the github repos of scipy and pandas gives no obvious places where the strides attribute is set (but I do see usage of https://github.com/search?q=repo%3Ascipy%2Fscipy+strides+language%3APython&type=code&l=Python |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good to me, thanks also from me for splitting it off!
My only nit is the last sentence of the what's-new entry.
One other thought, regarding impact on other projects: my sense would be simply to do this for the next release, so that other projects have a chance to test against -dev
(which most of the big ones will be doing).
* `np.lib.stride_tricks.as_strided` for the general case, | ||
* or the `np.ndarray` constructor (``buffer`` is the original array) for a light-weight version. | ||
|
||
Manually setting the strides attribute on a NumPy array can be useful, but needs to be done |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would just delete this -- since we now deprecate manually setting the strides, and as a warning for as_strided
it seems superfluous - that routine warns enough.
@@ -21,7 +21,7 @@ class TestHalf: | |||
def setup_method(self): | |||
# An array of all possible float16 values | |||
self.all_f16 = np.arange(0x10000, dtype=uint16) | |||
self.all_f16.dtype = float16 | |||
self.all_f16 = self.all_f16.view(float16) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These changes are not strictly related any more, though arguably improvements regardless.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indeed not related. The idea was to add them so the diff for the other (more complex) PR for the dtype will be small.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks. It seems we'll go with 2.4 (sorry about that), that makes me slightly hesitant to merge because we are in the RC stage where downstream tends to think that any breakage is critical to be fixed.
That said, happy if someone just merges it, I don't really expect downstream to use strides setting much.
numpy/_core/src/multiarray/getset.c
Outdated
} | ||
|
||
/* Deprecated NumPy 2.3, 2025-05-11 */ | ||
if( DEPRECATE("Setting the strides on a NumPy array has been deprecated in NumPy 2.3.") < 0 ) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if( DEPRECATE("Setting the strides on a NumPy array has been deprecated in NumPy 2.3.") < 0 ) { | |
if (DEPRECATE( | |
"Setting the strides on a NumPy array has been" | |
" deprecated in NumPy 2.3.") < 0 ) { |
Co-authored-by: Joren Hammudoglu <[email protected]>
A |
Sorry, this slipped, let's give it a shot, good time now for deprecations just after the release. Thanks @eendebakpt. |
* This patch allows the SciPy testsuite to pass against NumPy `main` branch, after numpy/numpy#28925 was merged in the last day or so.
* This patch allows the SciPy testsuite to pass against NumPy `main` branch, after numpy/numpy#28925 was merged in the last day or so.
* This patch allows the SciPy testsuite to pass against NumPy `main` branch, after numpy/numpy#28925 was merged in the last day or so.
) * MAINT: Update download-wheels for multiple pages The anaconda staging repositories have changed layout and now only 25 files are showing per page. This update uses a different URL to grab all of the names, thus avoiding the need for paging. There are also some cleanups and added comments to make future maintenance easier. [skip azp] [skip cirrus] [skip actions] * ENH: Disable the alloc cache under address and memory sanitizers We've been carrying a variant of this patch in our local copy of NumPy for some time. The alloc cache reduces the effectiveness of use-after-free or use-of-uninitialized memory warnings because caching of allocations means they may be reused for a different allocation, potentially hiding real problems. * ENH, SIMD: Initial implementation of Highway wrapper A thin wrapper over Google's Highway SIMD library to simplify its interface. This commit provides the implementation of that wrapper, consisting of: - simd.hpp: Main header defining the SIMD namespaces and configuration - simd.inc.hpp: Template header included multiple times with different namespaces The wrapper eliminates Highway's class tags by: - Using lane types directly which can be deduced from arguments - Leveraging namespaces (np::simd and np::simd128) for different register widths A README is included to guide usage and document design decisions. * SIMD: Update wrapper with improved docs and type support - Fix hardware/platform terminology in documentation for clarity - Add support for long double in template specializations - Add kMaxLanes constant to expose maximum vector width information - Follows clang formatting style for consistency with NumPy codebase. * SIMD: Improve isolation and constexpr handling in wrapper - Add anonymous namespace around implementation to ensure each translation unit gets its own constants based on local flags - Use HWY_LANES_CONSTEXPR for Lanes function to ensure proper constexpr evaluation across platforms * Update Highway submodule to latest master * SIMD: Fix compile error by using MaxLanes instead of Lanes for array size Replace hn::Lanes(f64) with hn::MaxLanes(f64) when defining the index array size to fix error C2131: "expression did not evaluate to a constant". This error occurs because Lanes() isn't always constexpr compatible, especially with scalable vector extensions. MaxLanes() provides a compile-time constant value suitable for static array allocation and should be used with non-scalable SIMD extensions when defining fixed-size arrays. * SIMD: Rename NPY_SIMDX to NPY_HWY Rename Highway wrapper macros for clarity: - NPY_SIMDX → NPY_HWY - NPY_SIMDX_F16 → NPY_HWY_F16 - NPY_SIMDX_F64 → NPY_HWY_F64 - NPY_SIMDX_FMA → NPY_HWY_FMA To avoids confusion with legacy SIMD macros. * Disable Highway EMU128 scalar emulation Skip Highway's EMU128 in favor of NumPy's scalar implementations for due to strict IEEE 754 floating-point compliance requirements * TYP: Type ``MaskedArray.__{iadd,isub,imul,itruediv,ifloordiv,ipow}__`` (numpy#28986) * MNT: fix linter issues on main * MNT: switch back to setup-python in CI * CI: update cibuildwheel to 3.0.0b1 and enable cp314 [wheel build] * MNT: respond to code review [wheel build] * MNT: move back to 'cd tools' hack [wheel build] * DOC: Fixes line numbers on classes decorated with set_module (numpy#28629) (numpy#28918) * MAINT: Bump larsoner/circleci-artifacts-redirector-action Bumps [larsoner/circleci-artifacts-redirector-action](https://github.com/larsoner/circleci-artifacts-redirector-action) from 1.0.0 to 1.1.0. - [Release notes](https://github.com/larsoner/circleci-artifacts-redirector-action/releases) - [Commits](scientific-python/circleci-artifacts-redirector-action@4e13a10...7eafdb6) --- updated-dependencies: - dependency-name: larsoner/circleci-artifacts-redirector-action dependency-version: 1.1.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * BEG, MAINT: Begin NumPy 2.4.0 development. - Create 2.4.0-notes.rst - Update release.rst - Update pavement.py - Update pyproject.toml - Update cversions.txt - Update numpyconfig.h - Delete release fragments * Fix workflow error * BUG: Avoid compile errors in f2py modules Some of the casts from cfuncs pass PyObject* to PyArrayObject*, which causes compile errors due to incompatible pointer types on at least GCC 14. * DOC: Expand/clean up extension module import error We get a lot of these reports and it is never us... But unfortunately, Python (currently) doesn't report *why* the module wasn't found. (I have opened an issue asking for that.) Until Python does, try to figure it out ourselves, i.e. list C modules (I guess its always one, but OK). If anything it'll give *us* an immediate thing to point out if an issue is reported... I also hid the "source import" thing to only occur if __config__ doesn't exist. Not sure that catches this fully, but I also feel like this isn't an actual problem anymore (i.e. we could just delete it also). Tested locally by renaming or deleting `_multiarray_umath`. * appease linter * Revert enable/CIBW_ENABLE changes [wheel build] * explicitly re-raise again (worked before because the next line raises...) * Update numpy/_core/__init__.py Co-authored-by: Melissa Weber Mendonça <[email protected]> * BUG: handle the case of modules with derived types * TST: tests for modules with derived types * fix W291 * BUG: Fix cache use regression numpygh-29006 got the branching wrong leaving the cache undefined on most GCC/clang, which means we wouldn't use it. Also move it up so that we can just remove the unused globals entirely. * ok, don't attempt to remove the statics (too ingrained) * ENH: add __array_function__ protocol in polynomial (numpy#28996) * TYP: annotate strings.slice * TYP: remove expired ``tostring`` methods * use pypy 3.11 nightly which has a fix for ctypeslib [skip azp][skip circleci] * BLD: allow targeting webassembly without emscripten * TYP: fix invalid overload definition in ``_core.defchararray.add`` * TYP: annotate the ``*args`` and ``**kwargs`` of the ``ufunc`` methods * TYP: annotate the return type of ``numpy.typing.__getattr__`` * TYP: annotate ``numpy.lib._format_impl`` * BLD: use sonoma image on Cirrus for [wheel build][skip actions][skip azp] * DOC: fix typo in documentation of vecmat The body of the summary uses the symbol v to reference the `x1` parameter, however in the displayed math, b is used instead. This commit changes b to v in the displayed math for concistency. * MAINT: Enforce ruff E501 * enforce more files * more enforcement * DOC: fix typo in documentation of matvec Same as for commit fba2e60, the vector was referred to as v in the body of the summary but b in the displayed math. This commit fixes the inconsistency. * Merge pull request numpy#29063 from MarcoGorelli/align-maskedarray-with-ndarray * BUG: add bounds-checking to in-place string multiply (numpy#29060) * BUG: add bounds-checking to in-place string multiply * MNT: check for overflow and raise OverflowError * MNT: respond to review suggestion * MNT: handle overflow in one more spot * MNT: make test behave the same on all architectures * MNT: reorder to avoid work in some cases * MAINT: Convert umath_linalg to multi-phase init (PEP 489) * MAINT: Convert dummymodule to multi-phase init (PEP 489) * MAINT: Convert lapack_lite to multi-phase init (PEP 489) * TYP: Type ``MaskedArray.__{add,radd,sub,rsub}__`` (numpy#29012) * review comments * Convert pocketfft_umath to multi-phase init (PEP 489) (numpy#29028) * MAINT: Convert multiarray to multi-phase init (PEP 489) (numpy#29022) * BLD: bump OpenBLAS version, use OpenBLAS for win-arm64 (numpy#29039) * BLD: bump OpenBLAS version, use OpenBLAS for win-arm64 [wheel build] * Update requirements/ci_requirements.txt Co-authored-by: Sebastian Berg <[email protected]> * use pip to install anaconda-client on win-arm64 [wheel build] * allow noblas in win32 wheels, use scipy-openblas32 on win-arm64 [wheel build] * improve runner arch detection logic [wheel build] * remove win_arm64 cibuildwheel override * remove 'strip' before calling delvewheel [wheel build] * use openblas 0.3.29.265 only on win-arm64 [wheel build] * add comment about lack of win-arm64 openblas64 wheels [wheel build] --------- Co-authored-by: Sebastian Berg <[email protected]> Co-authored-by: Joe Rickerby <[email protected]> * CI: bump to cibuildwheel 3.0.0b4 [wheel build] This bumps to cibuildwheel 3.0.0b4, which contains CPython 3.14.0b2, and removes the directory changing workaround. Signed-off-by: Henry Schreiner <[email protected]> * CI: clean up cibuildwheel config a bit [wheel build] This simplifies the configuration a bit: * Combine pyodide blocks * Use tables/lists for config-settings and skip * Remove a few repeated lines * Use a list for select Signed-off-by: Henry Schreiner <[email protected]> * MNT: Avoid use of deprecated _PyDict_GetItemStringWithError in f2py * Ignore all build-* directories * TYP: fix `NDArray[integer]` inplace operator mypy issue * TYP: regression tests for `NDArray[integer]` inplace ops * MAINT: Bump ossf/scorecard-action from 2.4.1 to 2.4.2 Bumps [ossf/scorecard-action](https://github.com/ossf/scorecard-action) from 2.4.1 to 2.4.2. - [Release notes](https://github.com/ossf/scorecard-action/releases) - [Changelog](https://github.com/ossf/scorecard-action/blob/main/RELEASE.md) - [Commits](ossf/scorecard-action@f49aabe...05b42c6) --- updated-dependencies: - dependency-name: ossf/scorecard-action dependency-version: 2.4.2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * MAINT: bump `mypy` to `1.16.0` * TYP: run mypy in strict mode * TYP: disable mypy's `no-untyped-call` errors in the `MaskedArray` type-tests * TYP: remove problematic runtime code from a `.pyi` test module * BUG: f2py: thread-safe forcomb (numpy#29091) * PERF: Use dict instead of list to make NpzFile member existence checks constant time (numpy#29098) Use dict instead of list to convert the passed key to the name used in the archive. * BENCH: Increase array sizes for ufunc and sort benchmarks (numpy#29084) * Fix some incorrect reST markups * MAINT: fix SPDX license expressions for LAPACK, GCC runtime libs See scipy/scipy#23061 for details. [skip ci] * MAINT: cleanup from finalized concatenate deprecation (numpy#29115) * TYP: minor ufunc alias fixes in ``__init__.pyi`` (numpy#29120) * update windows-2019 to windows-2022 and meson flag[wheel build] Co-authored-by: Charles Harris <[email protected]> * MAINT: Bump github/codeql-action from 3.28.18 to 3.28.19 Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.28.18 to 3.28.19. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@ff0a06e...fca7ace) --- updated-dependencies: - dependency-name: github/codeql-action dependency-version: 3.28.19 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * DOC: remove very outdated info on ATLAS (numpy#29119) * DOC: remove very outdated info on ATLAS ATLAS hasn't been developed for years, there is no reason to ever use it instead of OpenBLAS, BLIS, or MKL. So remove mentions of it. The troubleshooting instructions haven't been relevant in quite a while either. Addresses a comment on numpygh-29108 [skip cirrus] [skip github] [skip azp] * accept review suggestion [skip ci] Co-authored-by: Matti Picus <[email protected]> --------- Co-authored-by: Matti Picus <[email protected]> * DOC: fix typo in Numpy's module structure * update according to review * MAINT: Bump conda-incubator/setup-miniconda from 3.1.1 to 3.2.0 Bumps [conda-incubator/setup-miniconda](https://github.com/conda-incubator/setup-miniconda) from 3.1.1 to 3.2.0. - [Release notes](https://github.com/conda-incubator/setup-miniconda/releases) - [Changelog](https://github.com/conda-incubator/setup-miniconda/blob/main/CHANGELOG.md) - [Commits](conda-incubator/setup-miniconda@505e639...8352349) --- updated-dependencies: - dependency-name: conda-incubator/setup-miniconda dependency-version: 3.2.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * MAINT: Bump ``scipy-doctest`` to 1.8 (numpy#29085) * MAINT: check-docs: require scipy-doctest >= 1.8.0 See for the discussion https://discuss.scientific-python.org/t/scipy-doctest-select-only-doctests-or-both-doctests-and-unit-tests/1950 [docs only] * CI: bump scipy-doctest version on CI [skip azp] [skip actions] [skip cirrus] * MAINT: tweak doctests to not fail * MAINT: Bump mamba-org/setup-micromamba from 2.0.4 to 2.0.5 Bumps [mamba-org/setup-micromamba](https://github.com/mamba-org/setup-micromamba) from 2.0.4 to 2.0.5. - [Release notes](https://github.com/mamba-org/setup-micromamba/releases) - [Commits](mamba-org/setup-micromamba@0dea637...b09ef9b) --- updated-dependencies: - dependency-name: mamba-org/setup-micromamba dependency-version: 2.0.5 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * MAINT: Update main after 2.3.0 release. - Forwardport 2.3.0-changelog.rst - Forwardport 2.3.0-notes.rst - Forwardport .mailmap [skip azp] [skip cirrus] [skip actions] * TYP: update typing stubs for ``_pyinstaller/hook-numpy.py`` * DOC: Document assertion comparison behavior between scalar and empty array Noted in numpy#27457. The minimal thing we should do is document this behavior. While at it, I slightly homogenized the notes. * TYP: add missing ``numpy.lib`` exports * TYP: add ``containsderivedtypes`` to ``f2py.auxfuncs.__all__`` * TYP: remove ``run_command`` annotations in ``f2py.diagnose`` * BUG: Missing array-api ``capabilities()`` key * TST: migrating from pytz to zoneinfo + tzdata (where needed) For migration from pytz to zoneinfo function get_tzoffset_from_pytzinfo from numpy/_core/src/multiarray/datetime.c is modified to use astimezone instead of fromutc. As the object ZoneInfo is not directly compatible to be used with datetime object. Hence, something like this would result in an exception. from datetime import datetime from zoneinfo import ZoneInfo a = datetime(2025, 6, 7, 10, 0, 0) zoneInfo = ZoneInfo("US/Central") b = zoneInfo.fromutc(a) ValueError: fromutc: dt.tzinfo is not self The function astimezone can be used with both pytz.timezone object and zoneinfo.ZoneInfo object But, if we want to use the datetime object consistently we cannot let it be a naive type i.e. without a timezone. As the default behaviour of astimezone would take the system timezone if the datetime object is not timezone aware. Hence, I had to also change the call to create datetime object to take UTC timezone. See numpy#29064 * BUG: remove `NPY_ALIGNMENT_REQUIRED` * This machinery requires strict-aliasing UB and isn't needed anymore with any GCC from the last 15 years. This might also fix numpy#25004. Fixes: numpy#28991 * DOC: Document the removal of the NPY_ALIGNMENT_REQUIRED macro. * DOC: Remove version switcher colors [skip actions][skip azp][skip cirrus] * MAINT: Update comment for clarity * CI: Run mypy with Python 3.13 * DOC: tweak release walkthrough for numpy.org news blurb [skip actions][skip azp][skip cirrus] * Adding refactored tests . . * Removing additional references to pytz . . * TYP: Accept dispatcher function with optional returns in ``_core.overrides`` (numpy#29171) Co-authored-by: Sebastian Berg <[email protected]> * TYP: ``lib._iotools`` annotation improvements (numpy#29177) * TYP: ``any(None)`` and ``all(None)`` (numpy#29176) * TYP: Fix invalid inline annotations in ``lib._function_base_impl`` (numpy#29175) * TYP: Fix invalid inline annotations in ``lib._function_base_impl`` * TYP: ``ruff check --fix`` * TYP: prevent inline annotation from causing a circular import error * TYP: ``numpy._NoValue`` (numpy#29170) * TYP: ``out=...`` in ufuncs (numpy#29169) * TYP: Simplified ``dtype.__new__`` overloads (numpy#29168) * TYP: fix and improve ``numpy.lib._utils_impl`` (numpy#29181) * TYP: ``double = float64`` and ``cdouble = complex128`` (numpy#29155) * TYP: Fix missing ``_core.numeric`` (re-)exports (numpy#29166) * TYP: Sync `_core/numerictypes.pyi` with NumType's `numpy-stubs` * TYP: Fix missing ``_core.numeric`` (re-)exports * TYP: appease ``ruff`` * MAINT: Bump pypa/cibuildwheel from 3.0.0b4 to 3.0.0 (numpy#29180) Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 3.0.0b4 to 3.0.0. - [Release notes](https://github.com/pypa/cibuildwheel/releases) - [Changelog](https://github.com/pypa/cibuildwheel/blob/main/docs/changelog.md) - [Commits](pypa/cibuildwheel@cf078b0...5f22145) --- updated-dependencies: - dependency-name: pypa/cibuildwheel dependency-version: 3.0.0 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * CI: Bump `array-api-tests` to `v2025.05.23` (numpy#29149) * CI: Bump `array-api-tests` to `v2025.05.23` * TST: xfail new array-api test failures * TST: increase ``--max-examples`` from 100 to 500 for ``array-api-tests`` * CI: apply review suggestions Co-authored-by: Evgeni Burovski <[email protected]> --------- Co-authored-by: Evgeni Burovski <[email protected]> * TYP: fix ``ravel_multi_index`` false rejections (numpy#29184) * STY: ruff/isort config tweaks (numpy#29183) * DEV: import-related ruff config tweaks * STY: run ``ruff check --fix`` to fix the new ``I001`` errors * MAINT: bump ``ruff`` to ``0.11.13`` (numpy#29186) * BUG: fix matmul with transposed out arg (numpy#29179) * BUG: fix matmul with transposed out arg * DOC: add release note * fixes from review * TYP: add ``__all__`` in ``numpy._core.__init__`` (numpy#29187) Ported from numpy/numtype#156 * MAINT: Bump github/codeql-action from 3.28.19 to 3.29.0 Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.28.19 to 3.29.0. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@fca7ace...ce28f5b) --- updated-dependencies: - dependency-name: github/codeql-action dependency-version: 3.29.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * BUG: Revert `np.vectorize` casting to legacy behavior (numpy#29196) Revert use of no `dtype=object` to ensure correct cast behavior when the output dtype is discovered. Co-authored-by: Sebastian Berg <[email protected]> * DOC: Suppress distutils doc build warnings for python 3.12+ (numpy#29160) Enables an error free build of the docs for python 3.12+. Excludes related files, supresses all warnings for excluded files, and ignores case-by-case individual references. Closes numpy#29131. [skip azp][skip cirrus][skip actions] * TYP: fix ``ndarray.__array__`` annotation for ``copy`` (numpy#29204) * TST: additional tests for matmul with non-contiguous input and output (numpy#29197) * BUG: fix linting (numpy#29210) * CI: Add WoA validation setup to windows.yml * CI: Create action.yml for LLVM Win-ARM64 as reusable blocks * CI: Modify wheel.yml to use clang-cl for Win-ARM64 * CI: fix action.yml naming * CI: Fix reusable LLVM block * MAINT: Fix some undef warnings (numpy#29216) As noted by Chuck in numpygh-29138, there are some undef warnings that seem not nice this should fix them. The fact that `NPY_LONG`, etc. are not defined at macro expansion time is a bit of a trap, maybe it would be nice to have CI fail for this... * MAINT: bump `mypy` to `1.16.1` (numpy#29219) * TYP: Workaround for a mypy issue in ``ndarray.__iter__`` (numpy#29218) * ENH: improve Timsort with powersort merge-policy (numpy#29208) Implement the improved merge policy for Timsort, as developed by Munro and Wild. Benchmarks show a significant improvement in performance. * ENH: Detect CPU features on OpenBSD ARM and PowerPC64 Also while looking at this I noticed due to a compiler warning that npy__cpu_init_features_linux() was not enabled on FreeBSD with the original commit c47e962 thus the code was doing nothing on FreeBSD ARM. * CI: Add conditions to check hash of LLVM package * strides comparison performance fix, compare discussion numpy#29179 (numpy#29188) * TYP: Support iteration of ``StringDType`` arrays * BUG: Address interaction between SME and FPSR (numpy#29223) * BUG: Address interaction between SME and FPSR This is intended to resolve numpy#28687 The root cause is an interaction between Arm Scalable Matrix Extension (SME) and the floating point status register (FPSR). As noted in Arm docs for FPSR, "On entry to or exit from Streaming SVE mode, FPSR.{IOC, DZC, OFC, UFC, IXC, IDC, QC} are set to 1 and the remaining bits are set to 0". This means that floating point status flags are all raised when SME is used, regardless of values or operations performed. These are manifesting now because Apple Silicon M4 supports SME and macOS 15.4 enables SME codepaths for Accelerate BLAS / LAPACK. However, SME / FPSR behavior is not specific to Apple Silicon M4 and will occur on non-Apple chips using SME as well. Changes add compile and runtime checks to determine whether BLAS / LAPACK might use SME (macOS / Accelerate only at the moment). If so, special handling of floating-point error (FPE) is added, which includes: - clearing FPE after some BLAS calls - short-circuiting FPE read after some BLAS calls All tests pass Performance is similar Another approach would have been to wrap all BLAS / LAPACK calls with save / restore FPE. However, it added a lot of overhead for the inner loops that utilize BLAS / LAPACK. Some benchmarks were 8x slower. * add blas_supports_fpe and ifdef check Address the linker & linter failures * ENH: Detect CPU features on FreeBSD / OpenBSD RISC-V64. * BUG: avoid negating unsigned integers in resize implementation (numpy#29230) The negation of an unsigned int underflows and creates a large positive repeats, which leads to allocations failures and/or swapping. * TST: Fix test that uses unininitialized memory (numpy#29232) Tests should avoid this generally, this one is worse, since it can even fail due to warnings. * MAINT: bump ``ruff`` to ``0.12.0`` (numpy#29220) * BUG: Enforce integer limitation in concatenate (numpy#29231) * BUG: Enforce integer limitation in concatenate Concatenate internals only deal with integer many arrays, that should be fine in practice, but a SystemError (or in principle maybe also a harder crash?) is not really. * skip 32bit systems * DEP: Deprecate setting the strides attribute of a numpy array (numpy#28925) Deprecate setting strides (mutating) on an array. --------- Co-authored-by: Charles Harris <[email protected]> Co-authored-by: Sebastian Berg <[email protected]> Co-authored-by: Joren Hammudoglu <[email protected]> * ENH: np.unique: support hash based unique for string dtype (numpy#28767) * Support NPY_STRING, NPY_UNICODE * unique for NPY_STRING and NPY_UNICODE * fix construct array * remove unneccessary include * refactor * refactoring * comment * feature: unique for NPY_VSTRING * refactoring * remove unneccessary include * add test * add error message * linter * linter * reserve bucket * remove emoji from testcase * fix testcase * remove error * fix testcase * fix testcase name * use basic_string * fix testcase * add ValueError * fix testcase * fix memory error * remove multibyte char * refactoring * add multibyte char * refactoring * fix memory error * fix GIL * fix strlen * remove PyArray_GETPTR1 * refactoring * refactoring * use optional * refactoring * refactoring * refactoring * refactoring * fix comment * linter * add doc * DOC: fix * DOC: fix format * MNT: refactoring * MNT: refactoring * ENH: Store pointers to strings in the set instead of the strings themselves. * FIX: length in memcmp * ENH: refactoring * DOC: 49sec -> 34sec * Update numpy/lib/_arraysetops_impl.py Co-authored-by: Nathan Goldbaum <[email protected]> * DOC: Mention that hash-based np.unique returns unsorted strings * ENH: support medium and long vstrings * FIX: comment * ENH: use RAII wrapper * FIX: error handling of string packing * FIX: error handling of string packing * FIX: change default bucket size * FIX: include * FIX: cast * ENH: support equal_nan=False * FIX: function equal * FIX: check the case if pack_status douesn't return NULL * FIX: check the case if pack_status douesn't return NULL * FIX: stderr * ENH: METH_VARARGS -> METH_FASTCALL * FIX: log * FIX: release allocator * FIX: comment * FIX: delete log * ENH: implemented FNV-1a as hash function * bool -> npy_bool * FIX: cast * 34sec -> 35.1sec * fix: lint * fix: cast using const void * * fix: fix fnv1a hash * fix: lint * 35.1sec -> 33.5sec * enh: define macro HASH_TABLE_INITIAL_BUCKETS * enh: error handling of NpyString_load * enh: delete comments on GIL * fix: PyErr_SetString when NpyString_load failed * fix: PyErr_SetString -> npy_gil_error --------- Co-authored-by: Nathan Goldbaum <[email protected]> * DOC: Update CONTRIBUTING.rst (numpy#28158) * Update CONTRIBUTING.rst fixes numpy#19778 Updating the contibution section so that contributors avoid doing mkstakes while asking questions, instead they focus on doing contribution and work on project right after. * Update CONTRIBUTING.rst Shortened the length of the sentence. * Update CONTRIBUTING.rst * MAIN: Enable linting with E501 * MAIN: Enable linting with E501 * MAIN: Enable linting with E501 * DOC: Fix some markup errors * MAINT: Update main after 2.1.0 release. - Add 2.3.1-notes.rst - Add 2.3.1-changelog.rst - Update release.rst [skip cirrus] [skip azp] [skip actions] * DOC: Clarify dtype argument for __array__ in custom container guide (numpy#29254) * DOC: Clarify dtype argument for __array__ in custom container guide * Apply suggestions from code review Co-authored-by: Joren Hammudoglu <[email protected]> * TYP: Type ``MaskedArray.__{mul,rmul}__`` (numpy#29265) * BUG: fix fencepost error in StringDType internals (numpy#29269) This makes all the comparisons with NPY_MEDIUM_STRING_MAX_SIZE use <= consistently: * STY: ruff/isort config tweaks - episode 2 (numpy#29185) * DEV: enable ruff/isort ``combine-as-imports`` and ``split-on-trailing-comma`` * STY: run ``ruff check --fix`` to fix the new ``I001`` errors * MAINT: Fix ``I001`` ruff error on main (numpy#29272) * TYP: Work around a mypy issue with bool arrays (numpy#29248) * TYP: Add overloads for `MaskedArray.__{div,rdiv,floordiv,rfloordiv}__` (numpy#29271) * TYP: fix overloads where `out: _ArrayT` was typed as being the default (numpy#29278) * MAINT: Bump github/codeql-action from 3.29.0 to 3.29.1 (numpy#29285) Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.29.0 to 3.29.1. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@ce28f5b...39edc49) --- updated-dependencies: - dependency-name: github/codeql-action dependency-version: 3.29.1 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * BUG: handle case in mapiter where descriptors might get replaced (numpy#29286) Need to use the actual dtype, not the one passed to the array creation (because it can get replaced). Fixes numpy#29279. * BUG: Fix macro redefinition (numpy#29289) Also update .clang-format to indent preprocessor conditional blocks. * DOC: avoid searching some directories for doxygen-commented source code (numpy#29275) * DOC: avoid searching some directories for doxygen-commented source code * lint [skip azp][skip actions][skip cirrus] * use known paths [skip azp][skip actions][skip cirrus] * MAIN: Enforce ruff E501 rule * More linting * BUG: Fix version check in blas_utils.c Co-Authored-By: Daniel Bertalan <[email protected]> * MAINT: Bump github/codeql-action from 3.29.1 to 3.29.2 (numpy#29296) Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.29.1 to 3.29.2. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@39edc49...181d5ee) --- updated-dependencies: - dependency-name: github/codeql-action dependency-version: 3.29.2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * DOCS: Remove incorrect "Returns" section from `MaskedArray.sort` * DEP: Give a visible warning when `align=` to dtype is a non-bool This seems generally confusing. I would like to make it keyword only, but this already gives a warning when loading _very_ old pickles, meaning I am not quite sure we should change away from a warning quickly. We should fix things around pickling and start pickling in a way that makes it easier to move to keyword only arguments. (I suppose one could detect the case of `np.dtype(obj, False, True)` and assume it is via unpickling, but... I am assuming that it is OK to (eventually) break unpickling these 10+ year old files, but I am not in a rush to actually do so and go through with the deprecation. Signed-off-by: Sebastian Berg <[email protected]> * MAINT: Enable linting with ruff E501 (numpy#29300) * MAINT: Enforce ruff E501 * fix merge * TYP: Add shape typing to return values of `np.nonzero` and `ndarray.nonzero`, simplify `MaskedArray.nonzero` return type (numpy#29303) * chore: remove redundant words in comment (numpy#29306) * TYP: Add type annotations for `MaskedArray.__{pow,rpow}__` (numpy#29277) * test: add regression test for grammar in ufunc TypeError message * fix: correct singular/plural grammar in ufunc TypeError message * Update numpy/_core/tests/test_deprecations.py Co-authored-by: Matti Picus <[email protected]> * TYP: Type ``MaskedArray.{trace,round,cumsum,cumprod}`` (numpy#29307) * TYP: add explicit types for np.quantile (numpy#29305) * DOCS: Fix rendering of ``MaskedArray.anom`` ``dtype`` (numpy#29311) * BLD: remove unused github workflow (numpy#29312) * TYP: rename `_T` to `_ScalarT` in `matlib.pyi` for consistency (numpy#29310) * TYP: Type ``MaskedArray.dot`` and ``MaskedArray.anom`` (numpy#29309) * Convert unary into highway. --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Henry Schreiner <[email protected]> Signed-off-by: Sebastian Berg <[email protected]> Co-authored-by: Ralf Gommers <[email protected]> Co-authored-by: Charles Harris <[email protected]> Co-authored-by: Andrew Nelson <[email protected]> Co-authored-by: Peter Hawkins <[email protected]> Co-authored-by: Sayed Adel <[email protected]> Co-authored-by: Marco Edward Gorelli <[email protected]> Co-authored-by: Nathan Goldbaum <[email protected]> Co-authored-by: Marco Barbosa <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Stan Ulbrych <[email protected]> Co-authored-by: Angus Gibson <[email protected]> Co-authored-by: Sebastian Berg <[email protected]> Co-authored-by: Sebastian Berg <[email protected]> Co-authored-by: Melissa Weber Mendonça <[email protected]> Co-authored-by: Abdu Zoghbi <[email protected]> Co-authored-by: lvllvl <[email protected]> Co-authored-by: jorenham <[email protected]> Co-authored-by: mattip <[email protected]> Co-authored-by: Zebreus <[email protected]> Co-authored-by: Hamza Meel <[email protected]> Co-authored-by: Pieter Eendebak <[email protected]> Co-authored-by: Adam Turner <[email protected]> Co-authored-by: Joe Rickerby <[email protected]> Co-authored-by: Henry Schreiner <[email protected]> Co-authored-by: crusaderky <[email protected]> Co-authored-by: DWesl <[email protected]> Co-authored-by: Raghuveer Devulapalli <[email protected]> Co-authored-by: Yuki Kobayashi <[email protected]> Co-authored-by: Danis <[email protected]> Co-authored-by: abhishek-fujitsu <[email protected]> Co-authored-by: Marc Redemske <[email protected]> Co-authored-by: Evgeni Burovski <[email protected]> Co-authored-by: Christopher Sidebottom <[email protected]> Co-authored-by: Tim Hoffmann <[email protected]> Co-authored-by: V-R-S <[email protected]> Co-authored-by: David Seifert <[email protected]> Co-authored-by: François Rozet <[email protected]> Co-authored-by: Ben Woodruff <[email protected]> Co-authored-by: Michael <[email protected]> Co-authored-by: Mugundanmcw <[email protected]> Co-authored-by: Moritz Groß <[email protected]> Co-authored-by: Brad Smith <[email protected]> Co-authored-by: Developer-Ecosystem-Engineering <65677710+Developer-Ecosystem-Engineering@users.noreply.github.com> Co-authored-by: Koki Watanabe <[email protected]> Co-authored-by: specsy <[email protected]> Co-authored-by: Mohammed Abdul Rahman <[email protected]> Co-authored-by: Marco Edward Gorelli <[email protected]> Co-authored-by: Daniel Bertalan <[email protected]> Co-authored-by: ianlv <[email protected]> Co-authored-by: Shyok Mutsuddi <[email protected]> Co-authored-by: Sachin Shah <[email protected]>
Addresses part of #28800.
Changes are factored out of #28901.