Mergeback from main

.github/workflows/refresh-lockfiles.yml

@@ -91,7 +91,7 @@ jobs:
 
       - name: Create Pull Request
         id: cpr
-        uses: peter-evans/create-pull-request@18f90432bedd2afd6a825469ffd38aa24712a91d
+        uses: peter-evans/create-pull-request@671dc9c9e0c2d73f07fa45a3eb0220e1622f0c5f
         with:
           token: ${{ steps.generate-token.outputs.token }}
           commit-message: Updated environment lockfiles

.github/workflows/stale.yml

@@ -14,7 +14,7 @@ jobs:
     if: "github.repository == 'SciTools/iris'"
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/stale@v5
+      - uses: actions/stale@v6
         with:
           repo-token: ${{ secrets.GITHUB_TOKEN }}
 

docs/src/developers_guide/release.rst

@@ -209,6 +209,11 @@ branch, and then released by tagging ``v1.9.1``.
 
 New features shall not be included in a point release, these are for bug fixes.
 
+``whatsnew`` entries should be added to the existing 
+``docs/src/whatsnew/v1.9.rst`` file in a new ``v1.9.1`` section. A template for 
+this bugfix patches section can be found in the 
+``docs/src/whatsnew/latest.rst.template`` file.
+
 A point release does not require a release candidate, but the rest of the
 release process is to be followed, including the merge back of changes into
 ``main``.
@@ -224,17 +229,14 @@ These steps assume a release for ``1.9.0`` is to be created.
 Release Steps
 ~~~~~~~~~~~~~
 
-#. Create the release feature branch ``v1.9.x`` on `SciTools/iris`_.
-   The only exception is for a point/bugfix release, as it should already exist
-#. Update the ``iris.__init__.py`` version string e.g., to ``1.9.0``
 #. Update the ``whatsnew`` for the release:
 
    * Use ``git`` to rename ``docs/src/whatsnew/latest.rst`` to the release
      version file ``v1.9.rst``
-   * Update ``docs/src/whatsnews/index.rst`` to rename ``latest.rst`` in the
-     include statement and toctree.
    * Use ``git`` to delete the ``docs/src/whatsnew/latest.rst.template`` file
    * In ``v1.9.rst`` remove the ``[unreleased]`` caption from the page title.
+     Replace this with ``[release candidate]`` for the release candidate and
+     remove this for the actual release.
      Note that, the Iris version and release date are updated automatically
      when the documentation is built
    * Review the file for correctness
@@ -253,24 +255,28 @@ Release Steps
 #. Once all the above steps are complete, the release is cut, using
    the :guilabel:`Draft a new release` button on the
    `Iris release page <https://github.com/SciTools/iris/releases>`_
+   and targeting the release branch if it exists
+#. Create the release feature branch ``v1.9.x`` on `SciTools/iris`_ if it doesn't
+   already exist. For point/bugfix releases use the branch which already exists
 
 
 Post Release Steps
 ~~~~~~~~~~~~~~~~~~
 
 #. Check the documentation has built on `Read The Docs`_.  The build is
    triggered by any commit to ``main``.  Additionally check that the versions
-   available in the pop out menu in the bottom left corner include the new
+   available in the pop out menu in the bottom right corner include the new
    release version.  If it is not present you will need to configure the
    versions available in the **admin** dashboard in `Read The Docs`_.
 #. Review the `Active Versions`_ for the ``scitools-iris`` project on
    `Read The Docs`_ to ensure that the appropriate versions are ``Active``
    and/or ``Hidden``. To do this ``Edit`` the appropriate version e.g.,
    see `Editing v3.0.0rc0`_ (must be logged into Read the Docs).
-#. Make a new ``latest.rst`` from ``latest.rst.template`` and update the include
-   statement and the toctree in ``index.rst`` to point at the new
+#. Merge back to ``main``. This should be done after all releases, including
+   the release candidate, and also after major changes to the release branch.
+#. On main, make a new ``latest.rst`` from ``latest.rst.template`` and update
+   the include statement and the toctree in ``index.rst`` to point at the new
    ``latest.rst``.
-#. Merge back to ``main``
 
 
 .. _SciTools/iris: https://github.com/SciTools/iris
@@ -285,4 +291,4 @@ Post Release Steps
 .. _Generating Distribution Archives: https://packaging.python.org/tutorials/packaging-projects/#generating-distribution-archives
 .. _Packaging Your Project: https://packaging.python.org/guides/distributing-packages-using-setuptools/#packaging-your-project
 .. _latest CF standard names: http://cfconventions.org/standard-names.html
-.. _setuptools-scm: https://github.com/pypa/setuptools_scm
+.. _setuptools-scm: https://github.com/pypa/setuptools_scm

docs/src/whatsnew/latest.rst

@@ -50,7 +50,7 @@ This document explains the changes made to Iris for this release
 #. `@bjlittle`_ and `@lbdreyer`_ (reviewer) fixed the building of the CF
    Standard Names module ``iris.std_names`` for the ``setup.py`` commands
    ``develop`` and ``std_names``. (:issue:`4951`, :pull:`4952`)
-   
+
 #. `@lbdreyer`_ and `@pp-mo`_ (reviewer) fixed the cube print out such that
    scalar ancillary variables are displayed in a dedicated section rather than
    being added to the vector ancillary variables section. Further, ancillary
@@ -83,6 +83,13 @@ This document explains the changes made to Iris for this release
 
 #. `@rcomer`_ introduced the ``dask >=2.26`` minimum pin, so that Iris can benefit
    from Dask's support for `NEP13`_ and `NEP18`_. (:pull:`4905`)
+#. `@trexfeathers`_ advanced the Cartopy pin to ``>=0.21``, as Cartopy's
+   change to default Transverse Mercator projection affects an Iris test.
+   See `SciTools/cartopy@fcb784d`_ and `SciTools/cartopy@8860a81`_ for more
+   details.
+   (:pull:`4968`)
+#. `@trexfeathers`_ introduced the ``netcdf4!=1.6.1`` pin to avoid a problem
+   with segfaults. (:pull:`4968`)
 
 
 📚 Documentation
@@ -103,6 +110,10 @@ This document explains the changes made to Iris for this release
    package tests, see `pypa/setuptools#1684`_.  Also performed assorted
    ``setup.py`` script hygiene. (:pull:`4948`, :pull:`4949`, :pull:`4950`)
 
+#. `@pp-mo`_ split the module :mod:`iris.fileformats.netcdf` into separate
+   :mod:`~iris.fileformats.netcdf.loader` and :mod:`~iris.fileformats.netcdf.saver`
+   submodules, just to make the code easier to handle.
+
 
 .. comment
     Whatsnew author names (@github name) in alphabetical order. Note that,
@@ -117,4 +128,6 @@ This document explains the changes made to Iris for this release
 
 .. _NEP13: https://numpy.org/neps/nep-0013-ufunc-overrides.html
 .. _NEP18: https://numpy.org/neps/nep-0018-array-function-protocol.html
-.. _pypa/setuptools#1684: https://github.com/pypa/setuptools/issues/1684
+.. _pypa/setuptools#1684: https://github.com/pypa/setuptools/issues/1684
+.. _SciTools/cartopy@fcb784d: https://github.com/SciTools/cartopy/commit/fcb784daa65d95ed9a74b02ca292801c02bc4108
+.. _SciTools/cartopy@8860a81: https://github.com/SciTools/cartopy/commit/8860a8186d4dc62478e74c83f3b2b3e8f791372e

lib/iris/analysis/cartography.py

@@ -1008,8 +1008,8 @@ def _transform_distance_vectors_tolerance_mask(
     u_one_t, v_zero_t = _transform_distance_vectors(ones, zeros, ds, dx2, dy2)
     u_zero_t, v_one_t = _transform_distance_vectors(zeros, ones, ds, dx2, dy2)
     # Squared magnitudes should be equal to one within acceptable tolerance.
-    # A value of atol=2e-3 is used, which corresponds to a change in magnitude
-    # of approximately 0.1%.
+    # A value of atol=2e-3 is used, which masks any magnitude changes >0.5%
+    #  (approx percentage - based on experimenting).
     sqmag_1_0 = u_one_t**2 + v_zero_t**2
     sqmag_0_1 = u_zero_t**2 + v_one_t**2
     mask = np.logical_not(

lib/iris/experimental/ugrid/load.py

@@ -8,8 +8,7 @@
 Extensions to Iris' NetCDF loading to allow the construction of
 :class:`~iris.experimental.ugrid.mesh.Mesh`\\ es from UGRID data in the file.
 
-Eventual destination: :mod:`iris.fileformats.netcdf` (plan to split that module
-into ``load`` and ``save`` in future).
+Eventual destination: :mod:`iris.fileformats.netcdf`.
 
 """
 from contextlib import contextmanager
@@ -19,8 +18,8 @@
 
 from ...config import get_logger
 from ...coords import AuxCoord
-from ...fileformats import netcdf
 from ...fileformats._nc_load_rules.helpers import get_attr_units, get_names
+from ...fileformats.netcdf import loader as nc_loader
 from ...io import decode_uri, expand_filespecs
 from ...util import guess_coord_axis
 from .cf import (
@@ -202,7 +201,7 @@ def load_meshes(uris, var_name=None):
             else:
                 handling_format_spec = FORMAT_AGENT.get_spec(source, None)
 
-            if handling_format_spec.handler == netcdf.load_cubes:
+            if handling_format_spec.handler == nc_loader.load_cubes:
                 valid_sources.append(source)
             else:
                 message = f"Ignoring non-NetCDF file: {source}"
@@ -239,7 +238,7 @@ def _build_aux_coord(coord_var, file_path):
     assert isinstance(coord_var, CFUGridAuxiliaryCoordinateVariable)
     attributes = {}
     attr_units = get_attr_units(coord_var, attributes)
-    points_data = netcdf._get_cf_var_data(coord_var, file_path)
+    points_data = nc_loader._get_cf_var_data(coord_var, file_path)
 
     # Bounds will not be loaded:
     # Bounds may be present, but the UGRID conventions state this would
@@ -293,7 +292,7 @@ def _build_connectivity(connectivity_var, file_path, element_dims):
     assert isinstance(connectivity_var, CFUGridConnectivityVariable)
     attributes = {}
     attr_units = get_attr_units(connectivity_var, attributes)
-    indices_data = netcdf._get_cf_var_data(connectivity_var, file_path)
+    indices_data = nc_loader._get_cf_var_data(connectivity_var, file_path)
 
     cf_role = connectivity_var.cf_role
     start_index = connectivity_var.start_index
@@ -462,7 +461,7 @@ def _build_mesh(cf, mesh_var, file_path):
     )
     mesh_elements = filter(None, mesh_elements)
     for iris_object in mesh_elements:
-        netcdf._add_unused_attributes(
+        nc_loader._add_unused_attributes(
             iris_object, cf.cf_group[iris_object.var_name]
         )
 

lib/iris/experimental/ugrid/save.py

@@ -8,8 +8,7 @@
 Extensions to Iris' NetCDF saving to allow
 :class:`~iris.experimental.ugrid.mesh.Mesh` saving in UGRID format.
 
-Eventual destination: :mod:`iris.fileformats.netcdf` (plan to split that module
-into ``load`` and ``save`` in future).
+Eventual destination: :mod:`iris.fileformats.netcdf`.
 
 """
 from collections.abc import Iterable

lib/iris/fileformats/_nc_load_rules/helpers.py

@@ -31,9 +31,9 @@
 import iris.fileformats.netcdf
 from iris.fileformats.netcdf import (
     UnknownCellMethodWarning,
-    _get_cf_var_data,
     parse_cell_methods,
 )
+from iris.fileformats.netcdf.loader import _get_cf_var_data
 import iris.std_names
 import iris.util
 

lib/iris/fileformats/netcdf/__init__.py

@@ -0,0 +1,49 @@
+# Copyright Iris contributors
+#
+# This file is part of Iris and is released under the LGPL license.
+# See COPYING and COPYING.LESSER in the root of the repository for full
+# licensing details.
+"""
+Module to support the loading and saving of NetCDF files, also using the CF conventions
+for metadata interpretation.
+
+See : `NetCDF User's Guide <https://docs.unidata.ucar.edu/nug/current/>`_
+and `netCDF4 python module <https://github.com/Unidata/netcdf4-python>`_.
+
+Also : `CF Conventions <https://cfconventions.org/>`_.
+
+"""
+import iris.config
+
+# Note: *must* be done before importing from submodules, as they also use this !
+logger = iris.config.get_logger(__name__)
+
+from .loader import DEBUG, NetCDFDataProxy, load_cubes
+from .saver import (
+    CF_CONVENTIONS_VERSION,
+    MESH_ELEMENTS,
+    SPATIO_TEMPORAL_AXES,
+    CFNameCoordMap,
+    Saver,
+    UnknownCellMethodWarning,
+    parse_cell_methods,
+    save,
+)
+
+# Export all public elements from the loader and saver submodules.
+# NOTE: the separation is purely for neatness and developer convenience; from
+# the user point of view, it is still all one module.
+__all__ = (
+    "CFNameCoordMap",
+    "CF_CONVENTIONS_VERSION",
+    "DEBUG",
+    "MESH_ELEMENTS",
+    "NetCDFDataProxy",
+    "SPATIO_TEMPORAL_AXES",
+    "Saver",
+    "UnknownCellMethodWarning",
+    "load_cubes",
+    "logger",
+    "parse_cell_methods",
+    "save",
+)