diff --git a/docs/iris/src/whatsnew/2.3.rst b/docs/iris/src/whatsnew/2.3.rst new file mode 100644 index 0000000000..c5a6060146 --- /dev/null +++ b/docs/iris/src/whatsnew/2.3.rst @@ -0,0 +1,215 @@ +What's New in Iris 2.3.0 +************************ + +:Release: 2.3.0 +:Date: 2019-10-04 + +This document explains the new/changed features of Iris in version 2.3.0 +(:doc:`View all changes `.) + +Iris 2.3.0 Features +=================== +.. _showcase: + +.. admonition:: Increased Support for CF 1.7 + + We have introduced several changes that contribute to Iris's support for + the CF Conventions, including some CF 1.7 additions. We are now able to + support: + + * :ref:`Climatological Coordinates` + * :ref:`Standard name modifiers` + * :ref:`Geostationary projection` + + You can read more about each of these below. + + Additionally, the conventions attribute, added by Iris when saving to + NetCDF, has been updated to "CF-1.7", accordingly. + +.. _climatological: +.. admonition:: Climatological Coordinate Support + + Iris can now load, store and save `NetCDF climatological coordinates + `_. Any cube time + coordinate can be marked as a climatological time axis using the boolean + property: ``climatological``. The climatological bounds are stored in the + coordinate's ``bounds`` property. + + When an Iris climatological coordinate is saved in NetCDF, the NetCDF + coordinate variable will be given a 'climatology' attribute, and the + contents of the + coordinate's ``bounds`` property are written to a NetCDF boundary variable + called '_bounds'. These are in place of a standard + 'bounds' attribute and accompanying boundary variable. See below + for an `example adapted from CF conventions `_: + + .. code-block:: none + + dimensions: + time=4; + bnds=2; + variables: + float temperature(time,lat,lon); + temperature:long_name="surface air temperature"; + temperature:cell_methods="time: minimum within years time: mean over years"; + temperature:units="K"; + double time(time); + time:climatology="time_climatology"; + time:units="days since 1960-1-1"; + double time_climatology(time,bnds); + data: // time coordinates translated to date/time format + time="1960-4-16", "1960-7-16", "1960-10-16", "1961-1-16" ; + time_climatology="1960-3-1", "1990-6-1", + "1960-6-1", "1990-9-1", + "1960-9-1", "1990-12-1", + "1960-12-1", "1991-3-1" ; + + If a climatological time axis is detected when loading NetCDF - + indicated by the format described above - the ``climatological`` property + of the Iris coordinate will be set to ``True``. + +.. admonition:: New Chunking Strategy + + Iris now makes better choices of Dask chunk sizes when loading from NetCDF + files: If a file variable has small, specified chunks, Iris will now choose + Dask chunks which are a multiple of these up to a default target size. + + This is particularly relevant to files with an unlimited dimension, which + previously could produce a large number of small chunks. This had an adverse + effect on performance. + + In addition, Iris now takes its default chunksize from the default configured + in Dask itself, i.e. ``dask.config.get('array.chunk-size')``. + +.. admonition:: Lazy Statistics + + Several statistical operations can now be done lazily, taking advantage of the + performance improvements offered by Dask: + + * :meth:`~iris.cube.Cube.aggregated_by` + * :class:`~iris.analysis.RMS` (more detail below) + * :class:`~iris.analysis.MEAN` + +---- + +.. _geostationary: +.. _standard_name: +.. _conventions_1.7: + +* Cube data equality testing (and hence cube equality) now uses a more + relaxed + tolerance : This means that some cubes may now test 'equal' that previously + did not. + Previously, Iris compared cube data arrays using: + ``abs(a - b) < 1.e-8`` + + We now apply the default operation of :func:`numpy.allclose` instead, + which is equivalent to: + ``abs(a - b) < (1.e-8 + 1.e-5 * b)`` + +* Added support to render HTML for :class:`~iris.cube.CubeList` in Jupyter + Notebooks and JupyterLab. +* Loading CellMeasures with integer values is now supported. +* New coordinate system: :class:`iris.coord_systems.Geostationary`, + including load and save support, based on the `CF Geostationary projection + definition `_. +* :class:`iris.coord_systems.VerticalPerspective` can now be saved to and + loaded from NetCDF files. +* :class:`iris.experimental.regrid.PointInCell` moved to + :class:`iris.analysis.PointInCell` to make this regridding scheme public +* Iris now supports standard name modifiers. See `Appendix C, Standard Name Modifiers `_ for more information. +* :meth:`iris.cube.Cube.remove_cell_measure` now also allows removal of a cell + measure by its name (previously only accepted a CellMeasure object). +* The :data:`iris.analysis.RMS` aggregator now supports a lazy calculation. + However, the "weights" keyword is not currently supported by this, so a + *weighted* calculation will still return a realised result, *and* force + realisation of the original cube data. +* Iris now supports NetCDF Climate and Forecast (CF) Metadata Conventions 1.7 (see `CF 1.7 Conventions Document `_ for more information) + + +Iris 2.3.0 Dependency Updates +============================= +* Iris now supports Proj4 up to version 5, but not yet 6 or beyond, pending + `fixes to some cartopy tests `_. +* Iris now requires Dask >= 1.2 to allow for improved coordinate equality + checks. + + +Bugs Fixed +========== +* Cube equality of boolean data is now handled correctly. +* Fixed a bug where cell measures were incorrect after a cube + :meth:`~iris.cube.Cube.transpose` operation. Previously, this resulted in + cell-measures that were no longer correctly mapped to the cube dimensions. +* The :class:`~iris.coords.AuxCoord` disregarded masked points and bounds, as did the :class:`~iris.coords.DimCoord`. + Fix permits an :class:`~iris.coords.AuxCoord` to contain masked points/bounds, and a TypeError exception is now + raised when attempting to create or set the points/bounds of a + :class:`~iris.coords.DimCoord` with arrays with missing points. +* :class:`iris.coord_systems.VerticalPerspective` coordinate system now uses + the `CF Vertical perspective definition `_; had been + erroneously using Geostationary. +* :class:`~iris.coords.CellMethod` will now only use valid `NetCDF name tokens `_ to reference the coordinates involved in the statistical operation. +* The following var_name properties will now only allow valid `NetCDF name + tokens + `_ to + reference the said NetCDF variable name. Note that names with a leading + underscore are not permitted. + - :attr:`iris.aux_factory.AuxCoordFactory.var_name` + - :attr:`iris.coords.CellMeasure.var_name` + - :attr:`iris.coords.Coord.var_name` + - :attr:`iris.coords.AuxCoord.var_name` + - :attr:`iris.cube.Cube.var_name` +* Rendering a cube in Jupyter will no longer crash for a cube with + attributes containing ``\n``. +* NetCDF variables which reference themselves in their ``cell_measures`` + attribute can now be read. +* :func:`~iris.plot.quiver` now handles circular coordinates. +* The names of cubes loaded from abf/abl files have been corrected. +* Fixed a bug in UM file loading, where any landsea-mask-compressed fields + (i.e. with LBPACK=x2x) would cause an error later, when realising the data. +* :meth:`iris.cube.Cube.collapsed` now handles partial collapsing of + multidimensional coordinates that have bounds. +* Fixed a bug in the :data:`~iris.analysis.PROPORTION` aggregator, where cube + data in the form of a masked array with ``array.mask=False`` would cause an + error, but possibly only later when the values are actually realised. + ( Note: since netCDF4 version 1.4.0, this is now a common form for data + loaded from netCDF files ). +* Fixed a bug where plotting a cube with a + :class:`iris.coord_systems.LambertConformal` coordinate system would result + in an error. This would happen if the coordinate system was defined with one + standard parallel, rather than two. + In these cases, a call to + :meth:`~iris.coord_systems.LambertConformal.as_cartopy_crs` would fail. +* :meth:`iris.cube.Cube.aggregated_by` now gives correct values in points and + bounds when handling multidimensional coordinates. +* Fixed a bug in the :meth:`iris.cube.Cube.collapsed` operation, which caused + the unexpected realization of any attached auxiliary coordinates that were + *bounded*. It now correctly produces a lazy result and does not realise + the original attached AuxCoords. + + +Documentation Changes +===================== +* Added a gallery example showing `how to concatenate NEMO ocean model data + <../examples/Oceanography/load_nemo.html>`_. +* Added an example in the + `Loading Iris Cubes: Constraining on Time <../userguide/loading_iris_cubes + .html#constraining-on-time>`_ + Userguide section, demonstrating how to load data within a specified date + range. +* Added notes to the :func:`iris.load` documentation, and the userguide + `Loading Iris Cubes <../userguide/loading_iris_cubes.html>`_ + chapter, emphasizing that the *order* of the cubes returned by an iris load + operation is effectively random and unstable, and should not be relied on. +* Fixed references in the documentation of + :func:`iris.util.find_discontiguities` to a nonexistent + "mask_discontiguities" routine : these now refer to + :func:`~iris.util.mask_cube`. + diff --git a/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2018-Sep-17_aggregate_by_multdim_coords.txt b/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2018-Sep-17_aggregate_by_multdim_coords.txt deleted file mode 100644 index 4a2181ec2d..0000000000 --- a/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2018-Sep-17_aggregate_by_multdim_coords.txt +++ /dev/null @@ -1,2 +0,0 @@ -* :meth:`iris.cube.Cube.aggregated_by` now gives correct values in points and -bounds when handling multidimensional coordinates. \ No newline at end of file diff --git a/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Apr-03_partial_collapse_bounded_coords.txt b/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Apr-03_partial_collapse_bounded_coords.txt deleted file mode 100644 index 27f1aad5c6..0000000000 --- a/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Apr-03_partial_collapse_bounded_coords.txt +++ /dev/null @@ -1,2 +0,0 @@ -* :meth:`iris.cube.Cube.collapsed` now handles partial collapsing of -multidimensional coordinates that have bounds. diff --git a/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Aug-29_abf_abl_cube_names.txt b/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Aug-29_abf_abl_cube_names.txt deleted file mode 100644 index e324f625d2..0000000000 --- a/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Aug-29_abf_abl_cube_names.txt +++ /dev/null @@ -1 +0,0 @@ -* The name of cubes loaded from abf/abl files has been corrected. diff --git a/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Oct-01_cf_conventions_attribute.txt b/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Oct-01_cf_conventions_attribute.txt deleted file mode 100644 index 43c5065147..0000000000 --- a/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Oct-01_cf_conventions_attribute.txt +++ /dev/null @@ -1 +0,0 @@ -* Iris now supports NetCDF Climate and Forecast (CF) Metadata Conventions 1.7 (see `CF 1.7 Conventions Document `_ for more information) diff --git a/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Oct-01_cube_transpose_cell_measures.txt b/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Oct-01_cube_transpose_cell_measures.txt deleted file mode 100644 index 75dce7ff53..0000000000 --- a/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Oct-01_cube_transpose_cell_measures.txt +++ /dev/null @@ -1 +0,0 @@ -* Cell measures are now handled correctly when a cube is transposed. diff --git a/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Oct-01_masked_dimcoords_and_auxcoords.txt b/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Oct-01_masked_dimcoords_and_auxcoords.txt deleted file mode 100644 index 13b03cbeb2..0000000000 --- a/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Oct-01_masked_dimcoords_and_auxcoords.txt +++ /dev/null @@ -1,3 +0,0 @@ -* The :class:`~iris.coords.AuxCoord` disregarded masked points and bounds, as did the :class:`~iris.coords.DimCoord`. - Fix permits an :class:`~iris.coords.AuxCoord` to contain masked points/bounds, and an TypeError exception is now - raised when attempting to create or set the points/bounds of a :class:`~iris.coords.DimCoord` with masked arrays. diff --git a/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Oct-02_boolean_data_compare.txt b/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Oct-02_boolean_data_compare.txt deleted file mode 100644 index 332e60c863..0000000000 --- a/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Oct-02_boolean_data_compare.txt +++ /dev/null @@ -1 +0,0 @@ -* Cube equality of boolean data is now handled correctly. diff --git a/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Sep-03_self_referencing_vollcello_reading.txt b/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Sep-03_self_referencing_vollcello_reading.txt deleted file mode 100644 index b89d716930..0000000000 --- a/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Sep-03_self_referencing_vollcello_reading.txt +++ /dev/null @@ -1 +0,0 @@ -* Variables which reference themselves in ``cell_measures`` can now be read. diff --git a/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Sep-16_cellmethod_tokens.txt b/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Sep-16_cellmethod_tokens.txt deleted file mode 100644 index 6073b92fb2..0000000000 --- a/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Sep-16_cellmethod_tokens.txt +++ /dev/null @@ -1 +0,0 @@ -* :class:`~iris.coords.CellMethod` will now only use valid `NetCDF name tokens `_ to reference the coordinates involved in the statistical operation. diff --git a/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Sep-16_var_name_tokens.txt b/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Sep-16_var_name_tokens.txt deleted file mode 100644 index 59e85dbdcf..0000000000 --- a/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Sep-16_var_name_tokens.txt +++ /dev/null @@ -1 +0,0 @@ -* :attr:`~iris.aux_factory.AuxCoordFactory.var_name`, :attr:`~iris.coords.CellMeasure.var_name`, :attr:`~iris.coords.Coord.var_name`, :attr:`~iris.coords.AuxCoord.var_name` and :attr:`~iris.cube.Cube.var_name` will now only use valid `NetCDF name tokens `_ to reference the said NetCDF variable name. Note that, names with a leading inderscore are not permitted. diff --git a/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Sep-20_vp_coordsystem_definition.txt b/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Sep-20_vp_coordsystem_definition.txt deleted file mode 100644 index 1ed987782e..0000000000 --- a/docs/iris/src/whatsnew/contributions_2.3.0/bugfix_2019-Sep-20_vp_coordsystem_definition.txt +++ /dev/null @@ -1,4 +0,0 @@ -* :class:`iris.coord_systems.VerticalPerspective` coordinate system now uses -the `CF Vertical perspective definition `_; had been -erroneously using Geostationary. diff --git a/docs/iris/src/whatsnew/contributions_2.3.0/docchange_2019-Sep-13_nemo_gallery_example.txt b/docs/iris/src/whatsnew/contributions_2.3.0/docchange_2019-Sep-13_nemo_gallery_example.txt deleted file mode 100644 index b5f866c287..0000000000 --- a/docs/iris/src/whatsnew/contributions_2.3.0/docchange_2019-Sep-13_nemo_gallery_example.txt +++ /dev/null @@ -1 +0,0 @@ -* Added a gallery example showing how to concatenate NEMO ocean model data. diff --git a/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2018-Jul-25_dask_chunking.txt b/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2018-Jul-25_dask_chunking.txt deleted file mode 100644 index 0b710ed34d..0000000000 --- a/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2018-Jul-25_dask_chunking.txt +++ /dev/null @@ -1,7 +0,0 @@ -* Iris now makes better choices of dask chunk sizes, when loading from netcdf - files: If a file variable has specified, small chunks, Iris will now choose - dask chunks which are a multiple of these, up to a default target size. - This is particularly relevant to files with an unlimited dimension, which - previously could produce a large number of small chunks. - In addition, Iris now takes its default chunksize from the default configured - in dask itself, i.e. `dask.config.get('array.chunk-size')`. diff --git a/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Jul-11_lazy-aggregated_by.txt b/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Jul-11_lazy-aggregated_by.txt deleted file mode 100644 index 268b993f14..0000000000 --- a/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Jul-11_lazy-aggregated_by.txt +++ /dev/null @@ -1,2 +0,0 @@ -* :meth:`iris.cube.Cube.aggregated_by` now performs aggregation lazily where -possible. diff --git a/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Mar-08_remove_cell_measure_by_name.txt b/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Mar-08_remove_cell_measure_by_name.txt deleted file mode 100644 index a7c417c729..0000000000 --- a/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Mar-08_remove_cell_measure_by_name.txt +++ /dev/null @@ -1,2 +0,0 @@ -* :meth:`iris.cube.Cube.remove_cell_measure` now also allows removal of a cell -measure by its name (previously only accepted a CellMeasure object). diff --git a/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Oct-02_cube_equality_tolerance.txt b/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Oct-02_cube_equality_tolerance.txt deleted file mode 100644 index 7a0c15f421..0000000000 --- a/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Oct-02_cube_equality_tolerance.txt +++ /dev/null @@ -1,6 +0,0 @@ -* Cube data equality testing (and hence cube equality) now uses a more relaxed - tolerance : This means that some cubes may now test 'equal' that previously - did not. - Previously, Iris compared cube data arrays using "abs(a - b) < 1.e-8". - We now apply the default operation of :func:`numpy.allclose` instead, - which is equivalent to "abs(a - b) < (1.e-8 + 1.e-5 * b)". diff --git a/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Oct-02_cubelist_repr_html.txt b/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Oct-02_cubelist_repr_html.txt deleted file mode 100644 index 0bf2789be8..0000000000 --- a/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Oct-02_cubelist_repr_html.txt +++ /dev/null @@ -1 +0,0 @@ -* Added support to render HTML for :class:`~iris.cube.CubeList`s in Jupyter Notebooks and JupyterLab. diff --git a/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Sep-02_standard_name_modifiers.txt b/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Sep-02_standard_name_modifiers.txt deleted file mode 100644 index de8a7fc443..0000000000 --- a/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Sep-02_standard_name_modifiers.txt +++ /dev/null @@ -1 +0,0 @@ -* Iris now supports standard name modifiers. See `Appendix C, Standard Name Modifiers `_ for more information. diff --git a/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Sep-04_move_experimental_point-in-cell_regridder_to_analysis.txt b/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Sep-04_move_experimental_point-in-cell_regridder_to_analysis.txt deleted file mode 100644 index 65b3d8d568..0000000000 --- a/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Sep-04_move_experimental_point-in-cell_regridder_to_analysis.txt +++ /dev/null @@ -1,2 +0,0 @@ -* :class:`iris.experimental.regrid.PointInCell` moved to -:class:`iris.analysis.PointInCell` to make this regridding scheme public \ No newline at end of file diff --git a/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Sep-11_climatological_bounds.txt b/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Sep-11_climatological_bounds.txt deleted file mode 100644 index 7b4c62690e..0000000000 --- a/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Sep-11_climatological_bounds.txt +++ /dev/null @@ -1,3 +0,0 @@ -* Support for loading, storing and saving of NetCDF climatological coordinates. -Stored as a boolean property - ``climatological`` - of the relevant -cube coordinate. diff --git a/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Sep-20_geostationary_coordsystem.txt b/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Sep-20_geostationary_coordsystem.txt deleted file mode 100644 index 67900c288a..0000000000 --- a/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Sep-20_geostationary_coordsystem.txt +++ /dev/null @@ -1,5 +0,0 @@ -* New coordinate system: :class:`iris.coord_systems.Geostationary`, based on -the `CF Geostationary projection definition `_. -* :class:`iris.coord_systems.VerticalPerspective` can now be saved to and -loaded from NetCDF files. diff --git a/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Sep-27_cell_measure_integer_data.txt b/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Sep-27_cell_measure_integer_data.txt deleted file mode 100644 index 0a32b863dc..0000000000 --- a/docs/iris/src/whatsnew/contributions_2.3.0/newfeature_2019-Sep-27_cell_measure_integer_data.txt +++ /dev/null @@ -1 +0,0 @@ -* Loading CellMeasures with non-float values is now supported. diff --git a/lib/iris/coords.py b/lib/iris/coords.py index 67f41e911a..27c793241a 100644 --- a/lib/iris/coords.py +++ b/lib/iris/coords.py @@ -489,9 +489,10 @@ def __init__(self, points, standard_name=None, long_name=None, e.g. a :class:`~iris.coord_systems.GeogCS` for a longitude Coord. * climatological (bool): When True: the coordinate is a NetCDF climatological time axis. - When True: saving in NetCDF will label the time axis with - 'climatology' and 'climatology_bounds' in place of standard - bounds labels. + When True: saving in NetCDF will give the coordinate variable a + 'climatology' attribute and will create a boundary variable called + '_climatology' in place of a standard bounds + attribute and bounds variable. Will set to True when a climatological time axis is loaded from NetCDF. Always False if no bounds exist.