diff --git a/docs/src/common_links.inc b/docs/src/common_links.inc index 6f0ec83a3b..7ae2463ca9 100644 --- a/docs/src/common_links.inc +++ b/docs/src/common_links.inc @@ -3,7 +3,6 @@ .. _black: https://black.readthedocs.io/en/stable/ .. _cartopy: https://github.com/SciTools/cartopy -.. _.cirrus.yml: https://github.com/SciTools/iris/blob/main/.cirrus.yml .. _flake8: https://flake8.pycqa.org/en/stable/ .. _.flake8.yml: https://github.com/SciTools/iris/blob/main/.flake8 .. _cirrus-ci: https://cirrus-ci.com/github/SciTools/iris @@ -11,10 +10,12 @@ .. _contributor: https://github.com/SciTools/scitools.org.uk/blob/master/contributors.json .. _core developers: https://github.com/SciTools/scitools.org.uk/blob/master/contributors.json .. _generating sss keys for GitHub: https://docs.github.com/en/github/authenticating-to-github/adding-a-new-ssh-key-to-your-github-account +.. _GitHub Actions: https://docs.github.com/en/actions .. _GitHub Help Documentation: https://docs.github.com/en/github .. _GitHub Discussions: https://github.com/SciTools/iris/discussions .. _Iris: https://github.com/SciTools/iris .. _Iris GitHub: https://github.com/SciTools/iris +.. _Iris GitHub Actions: https://github.com/SciTools/iris/actions .. _iris-sample-data: https://github.com/SciTools/iris-sample-data .. _iris-test-data: https://github.com/SciTools/iris-test-data .. _isort: https://pycqa.github.io/isort/ diff --git a/docs/src/developers_guide/ci_checks.png b/docs/src/developers_guide/ci_checks.png old mode 100755 new mode 100644 index e088e03a66..54ab672b3c Binary files a/docs/src/developers_guide/ci_checks.png and b/docs/src/developers_guide/ci_checks.png differ diff --git a/docs/src/developers_guide/contributing_ci_tests.rst b/docs/src/developers_guide/contributing_ci_tests.rst index 87a3054605..1d06434843 100644 --- a/docs/src/developers_guide/contributing_ci_tests.rst +++ b/docs/src/developers_guide/contributing_ci_tests.rst @@ -13,51 +13,50 @@ The `Iris`_ GitHub repository is configured to run checks against all its branches automatically whenever a pull-request is created, updated or merged. The checks performed are: -* :ref:`testing_cirrus` +* :ref:`testing_gha` * :ref:`testing_cla` * :ref:`pre_commit_ci` -.. _testing_cirrus: +.. _testing_gha: -Cirrus-CI -********* +GitHub Actions +************** Iris unit and integration tests are an essential mechanism to ensure that the Iris code base is working as expected. :ref:`developer_running_tests` may be performed manually by a developer locally. However Iris is configured to -use the `cirrus-ci`_ service for automated Continuous Integration (CI) testing. +use `GitHub Actions`_ (GHA) for automated Continuous Integration (CI) testing. -The `cirrus-ci`_ configuration file `.cirrus.yml`_ in the root of the Iris repository -defines the tasks to be performed by `cirrus-ci`_. For further details -refer to the `Cirrus-CI Documentation`_. The tasks performed during CI include: +The Iris GHA YAML configuration files in the ``.github/workflows`` directory +defines the CI tasks to be performed. For further details +refer to the `GitHub Actions`_ documentation. The tasks performed during CI include: -* linting the code base and ensuring it adheres to the `black`_ format * running the system, integration and unit tests for Iris * ensuring the documentation gallery builds successfully * performing all doc-tests within the code base * checking all URL references within the code base and documentation are valid -The above `cirrus-ci`_ tasks are run automatically against all `Iris`_ branches +The above GHA tasks are run automatically against all `Iris`_ branches on GitHub whenever a pull-request is submitted, updated or merged. See the -`Cirrus-CI Dashboard`_ for details of recent past and active Iris jobs. +`Iris GitHub Actions`_ dashboard for details of recent past and active CI jobs. -.. _cirrus_test_env: +.. _gha_test_env: -Cirrus CI Test environment --------------------------- +GitHub Actions Test Environment +------------------------------- -The test environment on the Cirrus-CI service is determined from the requirement files -in ``requirements/ci/py**.yml``. These are conda environment files that list the entire -set of build, test and run requirements for Iris. +The CI test environments for our GHA is determined from the requirement files +in ``requirements/ci/pyXX.yml``. These are conda environment files list the top-level +package dependencies for running and testing Iris. For reproducible test results, these environments are resolved for all their dependencies -and stored as lock files in ``requirements/ci/nox.lock``. The test environments will not -resolve the dependencies each time, instead they will use the lock file to reproduce the -same exact environment each time. +and stored as conda lock files in the ``requirements/ci/nox.lock`` directory. The test environments +will not resolve the dependencies each time, instead they will use the lock files to reproduce the +exact same environment each time. -**If you have updated the requirement yaml files with new dependencies, you will need to +**If you have updated the requirement YAML files with new dependencies, you will need to generate new lock files.** To do this, run the command:: python tools/update_lockfiles.py -o requirements/ci/nox.lock requirements/ci/py*.yml @@ -86,37 +85,6 @@ and add the required fixes to this new branch. If the fixes are made to the Github Action is run. -.. _skipping Cirrus-CI tasks: - -Skipping Cirrus-CI Tasks ------------------------- - -As a developer you may wish to not run all the CI tasks when you are actively -developing e.g., you are writing documentation and there is no need for linting, -or long running compute intensive testing tasks to be executed. - -As a convenience, it is possible to easily skip one or more tasks by setting -the appropriate environment variable within the `.cirrus.yml`_ file to a -**non-empty** string: - -* ``SKIP_LINT_TASK`` to skip `flake8`_ linting and `black`_ formatting -* ``SKIP_TEST_MINIMAL_TASK`` to skip restricted unit and integration testing -* ``SKIP_TEST_FULL_TASK`` to skip full unit and integration testing -* ``SKIP_GALLERY_TASK`` to skip building the documentation gallery -* ``SKIP_DOCTEST_TASK`` to skip running the documentation doc-tests -* ``SKIP_LINKCHECK_TASK`` to skip checking for broken documentation URL references -* ``SKIP_ALL_TEST_TASKS`` which is equivalent to setting ``SKIP_TEST_MINIMAL_TASK`` and ``SKIP_TEST_FULL_TASK`` -* ``SKIP_ALL_DOC_TASKS`` which is equivalent to setting ``SKIP_GALLERY_TASK``, ``SKIP_DOCTEST_TASK``, and ``SKIP_LINKCHECK_TASK`` - -e.g., to skip the linting task, the following are all equivalent:: - - SKIP_LINT_TASK: "1" - SKIP_LINT_TASK: "true" - SKIP_LINT_TASK: "false" - SKIP_LINT_TASK: "skip" - SKIP_LINT_TASK: "unicorn" - - GitHub Checklist ---------------- @@ -150,9 +118,5 @@ pull-requests given the `Iris`_ GitHub repository `.pre-commit-config.yaml`_. See the `pre-commit.ci dashboard`_ for details of recent past and active Iris jobs. - -.. _Cirrus-CI Dashboard: https://cirrus-ci.com/github/SciTools/iris -.. _Cirrus-CI Documentation: https://cirrus-ci.org/guide/writing-tasks/ .. _.pre-commit-config.yaml: https://github.com/SciTools/iris/blob/main/.pre-commit-config.yaml .. _pre-commit.ci dashboard: https://results.pre-commit.ci/repo/github/5312648 -.. _GitHub Actions: https://github.com/SciTools/iris/actions/workflows/refresh-lockfiles.yml diff --git a/docs/src/developers_guide/contributing_documentation_full.rst b/docs/src/developers_guide/contributing_documentation_full.rst index 77b898c0f3..46f9c563d1 100755 --- a/docs/src/developers_guide/contributing_documentation_full.rst +++ b/docs/src/developers_guide/contributing_documentation_full.rst @@ -1,3 +1,4 @@ +.. include:: ../common_links.inc .. _contributing.documentation_full: @@ -31,7 +32,7 @@ The build can be run from the documentation directory ``docs/src``. The build output for the html is found in the ``_build/html`` sub directory. When updating the documentation ensure the html build has *no errors* or -*warnings* otherwise it may fail the automated `cirrus-ci`_ build. +*warnings* otherwise it may fail the automated `Iris GitHub Actions`_ build. Once the build is complete, if it is rerun it will only rebuild the impacted build artefacts so should take less time. @@ -66,8 +67,6 @@ This is useful for a final test before committing your changes. have been promoted to be **errors** to ensure they are addressed. This **only** applies when ``make html`` is run. -.. _cirrus-ci: https://cirrus-ci.com/github/SciTools/iris - .. _contributing.documentation.testing: Testing @@ -115,7 +114,7 @@ or ignore the url. ``spelling_word_list_filename``. -.. note:: In addition to the automated `cirrus-ci`_ build of all the +.. note:: In addition to the automated `Iris GitHub Actions`_ build of all the documentation build options above, the https://readthedocs.org/ service is also used. The configuration of this held in a file in the root of the diff --git a/docs/src/developers_guide/contributing_graphics_tests.rst b/docs/src/developers_guide/contributing_graphics_tests.rst index 07e2301141..d63221f48d 100644 --- a/docs/src/developers_guide/contributing_graphics_tests.rst +++ b/docs/src/developers_guide/contributing_graphics_tests.rst @@ -34,7 +34,7 @@ perceived as it may be a simple pixel shift. Testing Strategy ================ -The `Iris Cirrus-CI matrix`_ defines multiple test runs that use +Our `Iris GitHub Actions`_ define multiple test runs that use different versions of Python to ensure Iris is working as expected. To make this manageable, the ``iris.tests.IrisTest_nometa.check_graphic`` test @@ -157,12 +157,9 @@ To add your changes to Iris, you need to make two pull requests (PR). .. important:: - The Iris pull-request will not test successfully in Cirrus-CI until the - ``test-iris-imagehash`` pull request has been merged. This is because there - is an Iris_ test which ensures the existence of the reference images (uris) + The `Iris GitHub Actions`_ of a pull request will not pass successfully until the + associated ``test-iris-imagehash`` pull request has been merged. This is because there + is an Iris_ test which ensures the existence of the reference images (URIs) for all the targets in the image results database. It will also fail if you forgot to run ``recreate_v4_files_listing.py`` to update the image-listing file in ``test-iris-imagehash``. - - -.. _Iris Cirrus-CI matrix: https://github.com/scitools/iris/blob/main/.cirrus.yml diff --git a/docs/src/developers_guide/contributing_pull_request_checklist.rst b/docs/src/developers_guide/contributing_pull_request_checklist.rst index 3ae3784a1d..b8d8ad4b3e 100644 --- a/docs/src/developers_guide/contributing_pull_request_checklist.rst +++ b/docs/src/developers_guide/contributing_pull_request_checklist.rst @@ -31,7 +31,7 @@ is merged. Before submitting a pull request please consider this list. #. **Check all new dependencies added to the** `requirements/ci/`_ **yaml files.** If dependencies have been added then new nox testing lockfiles - should be generated too, see :ref:`cirrus_test_env`. + should be generated too, see :ref:`gha_test_env`. #. **Check the source documentation been updated to explain all new or changed features**. Note, we now use numpydoc strings. Any touched code should @@ -43,8 +43,6 @@ is merged. Before submitting a pull request please consider this list. #. **Check the documentation builds without warnings or errors**. See :ref:`contributing.documentation.building` -#. **Check for any new dependencies in the** `.cirrus.yml`_ **config file.** - #. **Check for any new dependencies in the** `readthedocs.yml`_ **file**. This file is used to build the documentation that is served from https://scitools-iris.readthedocs.io/en/latest/ diff --git a/docs/src/developers_guide/contributing_running_tests.rst b/docs/src/developers_guide/contributing_running_tests.rst index ab36172283..b9b89b3336 100644 --- a/docs/src/developers_guide/contributing_running_tests.rst +++ b/docs/src/developers_guide/contributing_running_tests.rst @@ -106,7 +106,7 @@ due to an experimental dependency not being present. Using Nox for Testing Iris ========================== -Iris has adopted the use of the `nox`_ tool for automated testing on `cirrus-ci`_ +The `nox`_ tool has for adopted for automated testing on `Iris GitHub Actions`_ and also locally on the command-line for developers. `nox`_ is similar to `tox`_, but instead leverages the expressiveness and power of a Python @@ -124,15 +124,12 @@ automates the process of: * building the documentation and executing the doc-tests * building the documentation gallery * running the documentation URL link check -* linting the code-base -* ensuring the code-base style conforms to the `black`_ standard - You can perform all of these tasks manually yourself, however the onus is on you to first ensure that all of the required package dependencies are installed and available in the testing environment. `Nox`_ has been configured to automatically do this for you, and provides a means to easily replicate -the remote testing behaviour of `cirrus-ci`_ locally for the developer. +the remote testing behaviour of `Iris GitHub Actions`_ locally for the developer. Installing Nox diff --git a/docs/src/developers_guide/documenting/whats_new_contributions.rst b/docs/src/developers_guide/documenting/whats_new_contributions.rst index 1a820ab24e..aa19722a69 100644 --- a/docs/src/developers_guide/documenting/whats_new_contributions.rst +++ b/docs/src/developers_guide/documenting/whats_new_contributions.rst @@ -1,3 +1,5 @@ +.. include:: ../../common_links.inc + .. _whats_new_contributions: ================================= @@ -38,7 +40,7 @@ situation is thought likely (large PR, high repo activity etc.): * PR author: create the "What's New" pull request * PR reviewer: once the "What's New" PR is created, **merge the main PR**. - (this will fix any `cirrus-ci`_ linkcheck errors where the links in the + (this will fix any `Iris GitHub Actions`_ linkcheck errors where the links in the "What's New" PR reference new features introduced in the main PR) * PR reviewer: review the "What's New" PR, merge once acceptable @@ -97,13 +99,11 @@ links to code. For more inspiration on possible content and references, please examine past what's :ref:`iris_whatsnew` entries. .. note:: The reStructuredText syntax will be checked as part of building - the documentation. Any warnings should be corrected. - `cirrus-ci`_ will automatically build the documentation when + the documentation. Any warnings should be corrected. The + `Iris GitHub Actions`_ will automatically build the documentation when creating a pull request, however you can also manually :ref:`build ` the documentation. -.. _cirrus-ci: https://cirrus-ci.com/github/SciTools/iris - Contribution Categories ======================= diff --git a/docs/src/whatsnew/3.0.rst b/docs/src/whatsnew/3.0.rst index 771a602954..223ef60011 100644 --- a/docs/src/whatsnew/3.0.rst +++ b/docs/src/whatsnew/3.0.rst @@ -97,9 +97,8 @@ v3.0.2 (27 May 2021) from collaborators targeting the Iris ``master`` branch. (:pull:`4007`) [``pre-v3.1.0``] - #. `@bjlittle`_ added conditional task execution to `.cirrus.yml`_ to allow - developers to easily disable `cirrus-ci`_ tasks. See - :ref:`skipping Cirrus-CI tasks`. (:pull:`4019`) [``pre-v3.1.0``] + #. `@bjlittle`_ added conditional task execution to ``.cirrus.yml`` to allow + developers to easily disable `cirrus-ci`_ tasks. (:pull:`4019`) [``pre-v3.1.0``] #. `@pp-mo`_ adjusted the use of :func:`dask.array.from_array` in :func:`iris._lazy_data.as_lazy_data`, to avoid the dask 'test access'. This makes loading of netcdf files with a diff --git a/docs/src/whatsnew/3.1.rst b/docs/src/whatsnew/3.1.rst index bd046a0a24..1f076572bc 100644 --- a/docs/src/whatsnew/3.1.rst +++ b/docs/src/whatsnew/3.1.rst @@ -227,9 +227,8 @@ This document explains the changes made to Iris for this release #. `@akuhnregnier`_ replaced `deprecated numpy 1.20 aliases for builtin types`_. (:pull:`3997`) -#. `@bjlittle`_ added conditional task execution to `.cirrus.yml`_ to allow - developers to easily disable `cirrus-ci`_ tasks. See - :ref:`skipping Cirrus-CI tasks`. (:pull:`4019`) +#. `@bjlittle`_ added conditional task execution to ``.cirrus.yml`` to allow + developers to easily disable `cirrus-ci`_ tasks. (:pull:`4019`) #. `@bjlittle`_ and `@jamesp`_ addressed a regression in behaviour when using `conda`_ 4.10.0 within `cirrus-ci`_. (:pull:`4084`) @@ -291,9 +290,8 @@ This document explains the changes made to Iris for this release #. `@bjlittle`_ enabled `cirrus-ci`_ compute credits for non-draft pull-requests from collaborators targeting the Iris ``master`` branch. (:pull:`4007`) -#. `@bjlittle`_ added conditional task execution to `.cirrus.yml`_ to allow - developers to easily disable `cirrus-ci`_ tasks. See - :ref:`skipping Cirrus-CI tasks`. (:pull:`4019`) +#. `@bjlittle`_ added conditional task execution to ``.cirrus.yml`` to allow + developers to easily disable `cirrus-ci`_ tasks. (:pull:`4019`)