From bb503df349abf536bc8418fea0884925b04de6a9 Mon Sep 17 00:00:00 2001 From: Will Benfold Date: Thu, 24 Feb 2022 10:59:17 +0000 Subject: [PATCH 1/4] Developer documentation on available testing conveniences --- .../contributing_testing_index.rst | 1 + docs/src/developers_guide/testing_tools.rst | 70 +++++++++++++++++++ 2 files changed, 71 insertions(+) create mode 100755 docs/src/developers_guide/testing_tools.rst diff --git a/docs/src/developers_guide/contributing_testing_index.rst b/docs/src/developers_guide/contributing_testing_index.rst index 7c6eb1b3cc..8684a08f27 100644 --- a/docs/src/developers_guide/contributing_testing_index.rst +++ b/docs/src/developers_guide/contributing_testing_index.rst @@ -7,6 +7,7 @@ Testing :maxdepth: 3 contributing_testing + testing_tools contributing_graphics_tests imagehash_index contributing_running_tests diff --git a/docs/src/developers_guide/testing_tools.rst b/docs/src/developers_guide/testing_tools.rst new file mode 100755 index 0000000000..a0dc3ea376 --- /dev/null +++ b/docs/src/developers_guide/testing_tools.rst @@ -0,0 +1,70 @@ +.. include:: ../common_links.inc + +.. _testing_tools: + +Testing tools +************* + +Iris has various internal convenience functions and utilities available to +support writing tests. Using these makes tests quicker and easier to write, and +also consistent with the rest of Iris (which makes it easier to work with the +code). Most of these conveniences are accessed through the +:class:`iris.tests.IrisTest` class, from which Iris' test classes inherit. + +Custom assertions +================= + +:class:`iris.tests.IrisTest` supports a variety of custom unittest-style +assertions, such as :meth:`~iris.tests.IrisTest_nometa.assertStringEqual`, +:meth:`~iris.tests.IrisTest_nometa.assertArrayEqual`, +:meth:`~iris.tests.IrisTest_nometa.assertArrayAlmostEqual`. + +Saving results +-------------- + +Some tests compare the generated output to the expected result contained in a +file. Custom assertions for this include +:meth:`~iris.tests.IrisTest_nometa.assertCMLApproxData` +:meth:`~iris.tests.IrisTest_nometa.assertCDL` +:meth:`~iris.tests.IrisTest_nometa.assertCML` and +:meth:`~iris.tests.IrisTest_nometa.assertTextFile`. See docstrings for more +information. + +.. note:: + + Sometimes code changes alter the results expected from a test containing the + above methods. These can be updated by removing the existing result files + and then running the file containing the test with a ``--create-missing`` + command line argument, or setting the ``IRIS_TEST_CREATE_MISSING`` + environment variable to anything non-zero. This will create the files rather + than erroring, allowing you to commit the updated results. + +Context managers +================ + +Capturing exceptions and logging +-------------------------------- + +:class:`iris.tests.IrisTest` includes several context managers that can be used +to make test code tidier and easier to read. These include +:meth:`~iris.tests.IrisTest_nometa.assertWarnsRegexp` and +:meth:`~iris.tests.IrisTest_nometa.assertLogs`. + +Temporary files +--------------- + +It's also possible to generate temporary files in a concise fashion with +:meth:`~iris.tests.IrisTest_nometa.temp_filename`. + +Patching +======== + +:meth:`~iris.tests.IrisTest_nometa.patch` is a wrapper around ``unittest.patch`` +that will be automatically cleaned up at the end of the test. + +Graphic tests +============= + +As a package capable of generating graphical outputs, Iris has utilities for +creating and updating graphical tests - see :ref:`testing.graphics` for more +information. \ No newline at end of file From 4e9b52a10a9c070fa6c30c1b9282c4fbd7d38984 Mon Sep 17 00:00:00 2001 From: Will Benfold Date: Thu, 24 Feb 2022 11:09:06 +0000 Subject: [PATCH 2/4] what's new --- docs/src/whatsnew/latest.rst.template | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/src/whatsnew/latest.rst.template b/docs/src/whatsnew/latest.rst.template index 1b36d3f0b0..2c2e091395 100644 --- a/docs/src/whatsnew/latest.rst.template +++ b/docs/src/whatsnew/latest.rst.template @@ -92,6 +92,10 @@ NOTE: section above is a template for bugfix patches #. N/A +#. `@wjbenfold`_ added developer documentation to highlight some of the + utilities offered by :class:`iris.IrisTest` and how to update CML and other + output files. (:issue:`4544`, :pull:`4600`) + 💼 Internal =========== From 2764965444273ea7b18ff22488e75598ada2ef29 Mon Sep 17 00:00:00 2001 From: Will Benfold Date: Wed, 9 Mar 2022 13:40:55 +0000 Subject: [PATCH 3/4] Update to be easily usable without code links working --- docs/src/developers_guide/testing_tools.rst | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/docs/src/developers_guide/testing_tools.rst b/docs/src/developers_guide/testing_tools.rst index a0dc3ea376..537af8d795 100755 --- a/docs/src/developers_guide/testing_tools.rst +++ b/docs/src/developers_guide/testing_tools.rst @@ -9,7 +9,16 @@ Iris has various internal convenience functions and utilities available to support writing tests. Using these makes tests quicker and easier to write, and also consistent with the rest of Iris (which makes it easier to work with the code). Most of these conveniences are accessed through the -:class:`iris.tests.IrisTest` class, from which Iris' test classes inherit. +:class:`iris.tests.IrisTest` class, from +which Iris' test classes then inherit. + +.. tip:: + + All functions listed on this page are defined within + :mod:`iris.tests.__init__.py` as methods of + :class:`iris.tests.IrisTest_nometa` (which :class:`iris.tests.IrisTest` + inherits from). They can be accessed within a test using + ``self.exampleFunction``. Custom assertions ================= From fadc59667e4f7557309bacd0cdd6ec789136df53 Mon Sep 17 00:00:00 2001 From: Will Benfold Date: Mon, 9 May 2022 12:23:22 +0100 Subject: [PATCH 4/4] Rebase --- docs/src/whatsnew/latest.rst | 4 ++++ docs/src/whatsnew/latest.rst.template | 4 ---- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/src/whatsnew/latest.rst b/docs/src/whatsnew/latest.rst index 05d1e3fcdd..2e1262d91a 100644 --- a/docs/src/whatsnew/latest.rst +++ b/docs/src/whatsnew/latest.rst @@ -177,6 +177,10 @@ This document explains the changes made to Iris for this release #. `@rcomer`_ updated the "Load a Time Series of Data From the NEMO Model" gallery example. (:pull:`4741`) +#. `@wjbenfold`_ added developer documentation to highlight some of the + utilities offered by :class:`iris.IrisTest` and how to update CML and other + output files. (:issue:`4544`, :pull:`4600`) + 💼 Internal =========== diff --git a/docs/src/whatsnew/latest.rst.template b/docs/src/whatsnew/latest.rst.template index 2c2e091395..1b36d3f0b0 100644 --- a/docs/src/whatsnew/latest.rst.template +++ b/docs/src/whatsnew/latest.rst.template @@ -92,10 +92,6 @@ NOTE: section above is a template for bugfix patches #. N/A -#. `@wjbenfold`_ added developer documentation to highlight some of the - utilities offered by :class:`iris.IrisTest` and how to update CML and other - output files. (:issue:`4544`, :pull:`4600`) - 💼 Internal ===========