From efdb499919148852422748c281ac93920180ca20 Mon Sep 17 00:00:00 2001
From: pciturri
Date: Tue, 17 Sep 2024 07:49:15 +0200
Subject: [PATCH 01/19] docs: finalized docstrings and API documentation
---
docs/examples/case_a.rst | 32 +++++++++++++++++---------------
docs/examples/case_b.rst | 8 ++++----
docs/examples/case_c.rst | 4 ++--
docs/examples/case_d.rst | 8 ++++----
docs/examples/case_e.rst | 8 ++++----
docs/examples/case_g.rst | 31 ++++++++++++++++++++-----------
docs/index.rst | 8 +-------
7 files changed, 52 insertions(+), 47 deletions(-)
diff --git a/docs/examples/case_a.rst b/docs/examples/case_a.rst
index d3b8fea..5e71c49 100644
--- a/docs/examples/case_a.rst
+++ b/docs/examples/case_a.rst
@@ -17,22 +17,21 @@ A - Simple Model and Catalog
After the calculation is complete, the results will be summarized in ``results/report.md``.
-Artifacts
----------
+Experiment Components
+---------------------
-The following example shows the definition of a testing experiment of a simple
-forecast against a simple catalog. The input structure of the experiment is:
+The following example shows the definition of a testing experiment of a single **time-independent** forecast against a catalog. The example files are found in ``floatcsep/examples/case_a``. The input structure of the experiment is:
::
case_a
+ ├── region.txt
├── catalog.csep
- ├── config.yml
├── best_model.dat
- └── region.txt
+ └── config.yml
-* The testing region consists of a grid with two 1ºx1º bins, whose bottom-left nodes are defined in the file `region.txt`. The grid spacing is obtained automatically. The nodes are:
+* The testing region ``region.txt`` consists of a grid with two 1ºx1º bins, defined by its bottom-left nodes. The grid spacing is obtained automatically. The nodes are:
.. literalinclude:: ../../examples/case_a/region.txt
@@ -40,18 +39,21 @@ forecast against a simple catalog. The input structure of the experiment is:
.. literalinclude:: ../../examples/case_a/catalog.csep
-* The forecast ``best_model.dat`` to be evaluated is written in the ``.dat`` format (see :doc:`pycsep:concepts/forecasts`). Forecast formats are detected automatically (see :class:`floatcsep.readers`)
+* The forecast ``best_model.dat`` to be evaluated is written in the ``.dat`` format (see :doc:`pycsep:concepts/forecasts`). Forecast formats are detected automatically (see :mod:`floatcsep.utils.readers.ForecastParsers`)
.. literalinclude:: ../../examples/case_a/best_model.dat
-.. important::
-
- Every file path should be relative to the ``config.yml`` file.
Configuration
-------------
-The experiment is defined by a time-, region-, model- and test-configurations. In this example, they are written together in the ``config.yml`` file.
+The experiment is defined by a time-, region-, model- and test-configurations, as well as a catalog and a region. In this example, they are written together in the ``config.yml`` file.
+
+
+.. important::
+
+ Every file path (e.g., of a catalog) specified in the ``config.yml`` file should be relative to the directory containing the configuration file.
+
Time
@@ -72,7 +74,7 @@ Time
Region
~~~~~~
- The region - a file path or :class:`csep` function (e.g. :obj:`csep.core.regions.italy_csep_region`) -, the depth limits and magnitude discretization are defined in the ``region_config`` inset.
+ The region - a file path or a :mod:`pycsep` function, such as :obj:`~csep.core.regions.italy_csep_region` (check the available regions in :mod:`csep.core.regions`) -, the depth limits and magnitude discretization are defined in the ``region_config`` inset.
.. literalinclude:: ../../examples/case_a/config.yml
:language: yaml
@@ -82,7 +84,7 @@ Region
Catalog
~~~~~~~
- It is defined in the ``catalog`` inset. This should only make reference to a catalog file or a catalog query function (e.g. ``query_comcat``). ``floatcsep`` will automatically filter the catalog to the experiment time, spatial and magnitude frames:
+ It is defined in the ``catalog`` inset. This should only make reference to a catalog file or a catalog query function (e.g. :func:`~csep.query_comcat`). ``floatcsep`` will automatically filter the catalog to the experiment time, spatial and magnitude frames:
.. literalinclude:: ../../examples/case_a/config.yml
:language: yaml
@@ -160,6 +162,6 @@ Results
Advanced
--------
-The experiment run logic can be seen in the file ``case_a.py``, which executes the same example but in python source code. The run logic of the terminal commands ``run``, ``plot`` and ``reproduce`` can be found in :class:`floatcsep.cmd.main`
+The experiment run logic can be seen in the file ``case_a.py``, which executes the same example but in python source code. The run logic of the terminal commands ``run``, ``plot`` and ``reproduce`` can be found in :mod:`floatcsep.commands.main`
diff --git a/docs/examples/case_b.rst b/docs/examples/case_b.rst
index 6e07d7b..8c30099 100644
--- a/docs/examples/case_b.rst
+++ b/docs/examples/case_b.rst
@@ -17,15 +17,15 @@ B - Multiple Models and Tests
After the calculation is complete, the results will be summarized in ``results/report.md``.
-Artifacts
-----------------------
+Experiment Components
+---------------------
-The following example is an experiment including multiple forecasts and evaluations. The input structure of the experiment is:
+The following example is an experiment including **multiple** time-independent forecasts and evaluations. The input structure of the experiment is:
::
case_b
- └── models
+ └── models
├── model_a.csv
├── model_b.csv
├── model_c.csv
diff --git a/docs/examples/case_c.rst b/docs/examples/case_c.rst
index e603646..e7d4a39 100644
--- a/docs/examples/case_c.rst
+++ b/docs/examples/case_c.rst
@@ -17,8 +17,8 @@ C - Multiple Time Windows
After the calculation is complete, the results will be summarized in ``results/report.md``.
-Artifacts
----------
+Experiment Components
+---------------------
The following example shows an experiment with multiple time windows. The input structure of the experiment is:
diff --git a/docs/examples/case_d.rst b/docs/examples/case_d.rst
index 01f270a..bbf7445 100644
--- a/docs/examples/case_d.rst
+++ b/docs/examples/case_d.rst
@@ -17,8 +17,8 @@ D - Catalog Queries and Model Repositories
After the calculation is complete, the results will be summarized in ``results/report.md``.
-Artifacts
----------
+Experiment Components
+---------------------
The following example shows an experiment whose forecasts are retrieved from a repository (Zenodo - https://zenodo.org) and the testing catalog from an authoritative source web service (namely the gCMT catalog from the ISC - http://www.isc.ac.uk). The initial structure is:
@@ -57,7 +57,7 @@ Configuration
Catalog
~~~~~~~
- The ``catalog`` inset from ``config.yml`` now makes reference to a catalog query function, in this case :func:`~floatcsep.accessors.query_isc_gcmt`.
+ The ``catalog`` inset from ``config.yml`` now makes reference to a catalog query function, in this case :func:`~pycsep.query_gcmt`.
.. literalinclude:: ../../examples/case_d/config.yml
:language: yaml
@@ -67,7 +67,7 @@ Catalog
.. note::
- Query functions exist in both ``pycsep`` and ``floatcsep`` (e.g. :func:`csep.query_comcat`, :func:`csep.query_bsi`, :func:`~csep.query_gcmt`, :func:`~csep.query_gns`). Only the name of the function is needed to retrieve the catalog. Refer to :obj:`csep` and :class:`floatcsep.accessors`.
+ Query functions are located in ``pycsep`` (e.g. :func:`csep.query_comcat`, :func:`csep.query_bsi`, :func:`csep.query_gcmt`, :func:`csep.query_gns`). Only the name of the function is needed to retrieve the catalog. Refer to :obj:`csep` API reference.
Models
~~~~~~
diff --git a/docs/examples/case_e.rst b/docs/examples/case_e.rst
index 92d3093..acb93ba 100644
--- a/docs/examples/case_e.rst
+++ b/docs/examples/case_e.rst
@@ -17,8 +17,8 @@ E - A Realistic Time-independent Experiment
After the calculation is complete, the results will be summarized in ``results/report.md``.
-Artifacts
----------
+Experiment Components
+---------------------
This example shows how to run a realistic testing experiment in Italy (based on https://doi.org/10.4401/ag-4844), summarizing the concepts of previous examples. The example has only a subset of the original models and evaluations. The input structure of the experiment is:
@@ -92,13 +92,13 @@ Post-Process
:language: yaml
:lines: 21-34
- See :func:`~csep.utils.plots.plot_spatial_dataset` for forecast plot options and :func:`~csep.utils.plots.plot_catalog` for the catalog placed on top.
+ The forecasts are plotted and stored in ``examples/case_e/results/{timewindow}/forecasts/``. See :func:`~csep.utils.plots.plot_spatial_dataset` for forecast plot options and :func:`~csep.utils.plots.plot_catalog` for the catalog placed on top.
Running the experiment
----------------------
- The experiment can be run by simply navigating to the ``examples/case_a`` folder in the terminal and typing.
+ The experiment can be run by simply navigating to the ``examples/case_e`` folder in the terminal and typing.
.. code-block:: console
diff --git a/docs/examples/case_g.rst b/docs/examples/case_g.rst
index 203f7b8..58de35b 100644
--- a/docs/examples/case_g.rst
+++ b/docs/examples/case_g.rst
@@ -17,8 +17,9 @@ G - Time-Dependent, Catalog-Based Model (from Source Code)
After the calculation is complete, the results will be summarized in ``results/report.md``.
-Artifacts
----------
+Experiment Components
+---------------------
+
This example shows how a time-dependent model should be set up for a time-dependent experiment
::
@@ -35,40 +36,48 @@ This example shows how a time-dependent model should be set up for a time-depend
├── models.yml
└── tests.yml
-* The model to be evaluated (``pymock``) is a source code that generates forecast for multiple time windows.
+* The model to be evaluated (``pymock``) is a source code that generates forecasts for multiple time windows.
-* The testing catalog `catalog.csv` works also as the input catalog, by being filtered until the starting test_date and allocated in `pymock/input` dynamically (before the model is run)
+* The testing catalog ``catalog.csv`` works also as the input catalog, by being filtered until the starting test_date and allocated in `pymock/input` dynamically (before the model is run)
Model
-----
-The experiment's complexity increases from time-independent to dependent, since we now need a **Model** (source code) that is able to generate forecast that changes every window. The model should be conceptualized as a **black-box**, whose only interface/interaction with the ``floatcsep`` system is to receive an input (i.e. input catalog) and generates an output (i.e. the forecasts).
+The experiment's complexity increases from time-independent to dependent, since we now need a **Model** (source code) that is able to generate forecast that changes every window. The model should be conceptualized as a **black-box**, whose only interface/interaction with the ``floatcsep`` system is to receive an input (i.e., input catalog and arguments) and generates an output (the forecasts).
-* Input: The input consists in input **data** and **arguments**.
+* **Input**: The input consists in input **data** and **arguments**.
- 1. The input data is, at the least, a catalog filtered until the forecast beginning, which is automatically allocated by ``fecsep`` in the `{model}/input` prior to each model's run. It is stored inside the model in ``csep.ascii`` format for simplicity's sake (see :doc:`pycsep:concepts/catalogs`).
+ 1. The **input data** is, at the least, a catalog filtered until the forecast beginning, which is automatically allocated by ``floatcsep`` in the `{model}/input` prior to each model's run. It is stored inside the model in ``csep.ascii`` format for simplicity's sake (see :doc:`pycsep:concepts/catalogs`).
.. literalinclude:: ../../examples/case_g/catalog.csv
:lines: 1-2
- 2. The input arguments controls how the source code works. The minimum arguments to run a model (which should be modified dynamically during an experiment) are the forecast ``start_date`` and ``end_date``. The experiment will read `{model}/input/args.txt` and change the values of ``start_date = {datetime}`` and ``end_date = {datetime}`' before the model is run. Additional arguments can be set by convenience, such as ``catalog`` (the input catalog name), ``n_sims`` (number of synthetic catalogs) and random ``seed`` for reproducibility.
+ 2. The **input arguments** controls how the model's source code works. The minimum arguments to run a model (which should be modified dynamically during an experiment) are the forecast ``start_date`` and ``end_date``. The experiment will read `{model}/input/args.txt` and change the values of ``start_date = {datetime}`` and ``end_date = {datetime}`` before the model is run. Additional arguments can be set by convenience, such as ``catalog`` (the input catalog name), ``n_sims`` (number of synthetic catalogs) and random ``seed`` for reproducibility.
+
+* **Output**: The model's output are the synthetic catalogs, which should be allocated in `{model}/forecasts/{filename}.csv`. The format is identically to ``csep_ascii``, but unlike in an input catalog, the ``catalog_id`` column should be modified for each synthetic catalog starting from 0. The file name follows the convention `{model_name}_{start}_{end}.csv`, where ``start`` and ``end`` folowws the `%Y-%m-%dT%H:%M:%S.%f` - ISO861 FORMAT
-* Output: The model's output are the synthetic catalogs, which should be allocated in `{model}/forecasts/{filename}.csv`. The format is identically to ``csep_ascii``, but unlike in an input catalog, the ``catalog_id`` column should be modified for each synthetic catalog starting from 0. The file name follows the convention `{model_name}_{start}_{end}.csv`, where ``start`` and ``end`` folowws the `%Y-%m-%dT%H:%M:%S.%f` - ISO861 FORMAT
+* **Model build**: Inside the model source code, there are multiple options to build it. A standard python ``setup.cfg`` is given, which can be built inside a python ``venv`` or ``conda`` managers. This is created and built automatically by ``floatCSEP``, as long as the the model build instructions are correctly set up.
-* Model run: The model should be run with a simple command to which only ``arguments`` should be passed. For this example, is
+* **Model run**: The model should be run with a simple command to which only ``arguments`` should be passed. For this example, is
.. code-block:: console
$ python pymock/run.py input/args.txt
-or using a binary (see ``pymock/setup.cfg:entry_point``)
+ or using a binary (see ``pymock/setup.cfg:entry_point``)
.. code-block:: console
$ pymock input/args.txt
+ It can also be run simply by
+
+ .. code-block:: console
+
+ $ pymock
+ as long as it internally reads the ``input/args.txt`` and ``input/catalog.csv`` files.
Configuration
diff --git a/docs/index.rst b/docs/index.rst
index d4db04a..82ab549 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -1,9 +1,3 @@
-.. floatCSEP documentation master file, created by
- sphinx-quickstart on Wed Nov 16 09:56:55 2022.
- You can adapt this file completely to your liking, but it should at least
- contain the root `toctree` directive.
-
-
floatCSEP: Floating Experiments
===============================
@@ -19,7 +13,7 @@ Preliminary documentation.
.. toctree::
:maxdepth: 1
- :caption: Example Experiments
+ :caption: Tutorial Experiments
examples/case_a.rst
examples/case_b.rst
From 7698f47d2443dd5fe64b90492a05bc7ff28b2476 Mon Sep 17 00:00:00 2001
From: pciturri
Date: Tue, 17 Sep 2024 18:54:27 +0200
Subject: [PATCH 02/19] docs: Added tutorial for case_f (time-dependent model
with only forecast files).
---
docs/examples/case_f.rst | 103 +++++++++++++++++++++++++++++++++++++++
1 file changed, 103 insertions(+)
create mode 100644 docs/examples/case_f.rst
diff --git a/docs/examples/case_f.rst b/docs/examples/case_f.rst
new file mode 100644
index 0000000..fad63fa
--- /dev/null
+++ b/docs/examples/case_f.rst
@@ -0,0 +1,103 @@
+G - Time-Dependent, Catalog-Based Model (from existing files)
+==========================================================
+
+.. currentmodule:: floatcsep
+
+.. contents::
+ :local:
+
+.. admonition:: **TL; DR**
+
+ In a terminal, navigate to ``floatcsep/examples/case_f`` and type:
+
+ .. code-block:: console
+
+ $ floatcsep run config.yml
+
+ After the calculation is complete, the results will be summarized in ``results/report.md``.
+
+
+Experiment Components
+---------------------
+
+
+This example shows how set up a time-dependent model, whose files are already existing, and experiment. The model structure is as follows:
+::
+
+ case_f
+ └── etas
+ ├── forecasts
+ ├── etas_2016-11-14_2016-11-15.csv (forecast files)
+ ...
+ └── etas_2016-11-20_2016-11-21.csv
+ ├── catalog.csv
+ ├── config.yml
+ ├── models.yml
+ └── tests.yml
+
+* The model to be evaluated (``etas``) is a collection of daily forecasts from ``2016-11-14`` until ``2016-11-21``.
+
+* The forecasts are located in a folder ``forecasts`` inside the model, to be consistent with models based on source codes (see the subsequent examples).
+
+
+Model
+-----
+
+The time-dependency of a model is manifested here by the provision of different forecasts, i.e., statistical descriptions of seismicity, for different time-windows. For this example, the forecasts were created from an external model https://github.com/lmizrahi/etas (https://doi.org/10.1785/0220200231
+), with which the experiment has no interface. This means that we only the forecast files are required. We leave the handling of a model source code for subsequent examples.
+
+
+
+Configuration
+-------------
+
+
+Time
+~~~~
+
+ The configuration is identical to time-independent models with multiple time-windows (e.g., case C), with the exception that now a ``horizon`` can be defined instead of ``intervals``, which is the forecast time-window length. The experiment's class should now be explicited as ``exp_class: td``
+
+ .. literalinclude:: ../../examples/case_f/config.yml
+ :language: yaml
+ :lines: 3-7
+
+Catalog
+~~~~~~~
+
+ The catalog was obtained ``previous to the experiment`` using ``query_geonet`` and it was filtered to the testing period.
+
+Models
+~~~~~~
+
+ Some additional arguments should be passed to a time-dependent model
+
+ .. literalinclude:: ../../examples/case_f/models.yml
+ :language: yaml
+ :lines: 1-4
+
+ For consistency with time-dependent models that will create forecasts from a source code, the ``path`` should point to the folder of the model. The path folder should contain a sub-folder named ``{path}/forecasts`` where the files are located. Note that fore catalog-based forecasts, the number of simulations should be explicit.
+
+Tests
+~~~~~
+
+ With time-dependent models, now catalog evaluations found in :obj:`csep.core.catalog_evaluations` can be used.
+
+
+ .. literalinclude:: ../../examples/case_f/tests.yml
+ :language: yaml
+
+ .. note::
+ It is possible to assign two plotting functions to a test, whose ``plot_args`` and ``plot_kwargs`` can be placed indented beneath
+
+
+Running the experiment
+----------------------
+
+ The experiment can be run by simply navigating to the ``examples/case_h`` folder in the terminal and typing.
+
+ .. code-block:: console
+
+ $ floatcsep run config.yml
+
+ This will automatically set all the calculation paths (testing catalogs, evaluation results, figures) and will create a summarized report in ``results/report.md``.
+
From 7e020458e9c326f52a1e1b08cc0e27d81c90ea88 Mon Sep 17 00:00:00 2001
From: pciturri
Date: Wed, 18 Sep 2024 18:18:42 +0200
Subject: [PATCH 03/19] docs: Finalized example tutoriales for cases a-f. fix:
the configuration for plot_forecasts now accept `catalog: True` instead of a
dictionary.
---
docs/examples/case_a.rst | 47 ++++----
docs/examples/case_b.rst | 26 +++--
docs/examples/case_c.rst | 41 ++++---
docs/examples/case_d.rst | 24 +++--
docs/examples/case_e.rst | 63 ++++++++---
docs/examples/case_f.rst | 49 ++++++---
docs/examples/case_g.rst | 28 ++++-
docs/examples/case_h.rst | 148 ++++++++++++++++++++++++++
docs/guide/postprocess_config.rst | 4 +
docs/index.rst | 4 +-
examples/case_a/best_model.dat | 3 +-
examples/case_a/config.yml | 4 +
examples/case_d/config.yml | 3 -
examples/case_e/config.yml | 1 +
examples/case_f/config.yml | 2 +-
examples/case_g/custom_plot_script.py | 6 +-
examples/case_h/config.yml | 4 +-
examples/case_h/custom_report.py | 11 ++
examples/case_h/models.yml | 2 +-
floatcsep/postprocess/plot_handler.py | 2 +
20 files changed, 365 insertions(+), 107 deletions(-)
create mode 100644 docs/examples/case_h.rst
create mode 100644 docs/guide/postprocess_config.rst
diff --git a/docs/examples/case_a.rst b/docs/examples/case_a.rst
index 5e71c49..93978dd 100644
--- a/docs/examples/case_a.rst
+++ b/docs/examples/case_a.rst
@@ -1,10 +1,9 @@
A - Simple Model and Catalog
============================
-.. currentmodule:: floatcsep
+The following example shows the definition of a testing experiment of a single **time-independent** forecast against a catalog.
-.. contents::
- :local:
+.. currentmodule:: floatcsep
.. admonition:: **TL; DR**
@@ -16,11 +15,15 @@ A - Simple Model and Catalog
After the calculation is complete, the results will be summarized in ``results/report.md``.
+.. contents::
+ :local:
+ :depth: 2
+
Experiment Components
---------------------
-The following example shows the definition of a testing experiment of a single **time-independent** forecast against a catalog. The example files are found in ``floatcsep/examples/case_a``. The input structure of the experiment is:
+The source code can be found in the ``examples/case_a`` folder or in `GitHub `_. The directory structure of the experiment is:
::
@@ -34,14 +37,17 @@ The following example shows the definition of a testing experiment of a single *
* The testing region ``region.txt`` consists of a grid with two 1ºx1º bins, defined by its bottom-left nodes. The grid spacing is obtained automatically. The nodes are:
.. literalinclude:: ../../examples/case_a/region.txt
+ :caption: examples/case_a/region.txt
* The testing catalog ``catalog.csep`` contains only one event and is formatted in the :meth:`~pycsep.utils.readers.csep_ascii` style (see :doc:`pycsep:concepts/catalogs`). Catalog formats are detected automatically
.. literalinclude:: ../../examples/case_a/catalog.csep
+ :caption: examples/case_a/catalog.csep
* The forecast ``best_model.dat`` to be evaluated is written in the ``.dat`` format (see :doc:`pycsep:concepts/forecasts`). Forecast formats are detected automatically (see :mod:`floatcsep.utils.readers.ForecastParsers`)
.. literalinclude:: ../../examples/case_a/best_model.dat
+ :caption: examples/case_a/best_model.dat
Configuration
@@ -62,6 +68,7 @@ Time
The time configuration is manifested in the ``time_config`` inset. The simplest definition is to set only the start and end dates of the experiment. These are always UTC date-times in isoformat (``%Y-%m-%dT%H:%M:%S.%f`` - ISO861):
.. literalinclude:: ../../examples/case_a/config.yml
+ :caption: examples/case_a/config.yml
:language: yaml
:lines: 3-5
@@ -77,6 +84,7 @@ Region
The region - a file path or a :mod:`pycsep` function, such as :obj:`~csep.core.regions.italy_csep_region` (check the available regions in :mod:`csep.core.regions`) -, the depth limits and magnitude discretization are defined in the ``region_config`` inset.
.. literalinclude:: ../../examples/case_a/config.yml
+ :caption: examples/case_a/config.yml
:language: yaml
:lines: 7-13
@@ -84,9 +92,10 @@ Region
Catalog
~~~~~~~
- It is defined in the ``catalog`` inset. This should only make reference to a catalog file or a catalog query function (e.g. :func:`~csep.query_comcat`). ``floatcsep`` will automatically filter the catalog to the experiment time, spatial and magnitude frames:
+ It is defined in the ``catalog`` inset. This should only make reference to a catalog **file** or a catalog **query function** (e.g. :func:`~csep.query_comcat`). **floatCSEP** will automatically filter the catalog to the experiment time, spatial and magnitude frames:
.. literalinclude:: ../../examples/case_a/config.yml
+ :caption: examples/case_a/config.yml
:language: yaml
:lines: 15-15
@@ -95,6 +104,7 @@ Models
The model configuration is set in the ``models`` inset with a list of model names, which specify their file paths (and other attributes). Here, we just set the path as ``best_model.dat``, whose format is automatically detected.
.. literalinclude:: ../../examples/case_a/config.yml
+ :caption: examples/case_a/config.yml
:language: yaml
:lines: 17-19
@@ -107,6 +117,7 @@ Evaluations
The experiment's evaluations are defined in the ``tests`` inset. It should be a list of test names, making reference to their function and plotting function. These can be either defined in ``pycsep`` (see :doc:`pycsep:concepts/evaluations`) or manually. In this example, we employ the consistency N-test: its function is :func:`csep.core.poisson_evaluations.number_test`, whereas its plotting function correspond to :func:`csep.utils.plots.plot_poisson_consistency_test`
.. literalinclude:: ../../examples/case_a/config.yml
+ :caption: examples/case_a/config.yml
:language: yaml
:lines: 21-24
@@ -127,26 +138,8 @@ Run command
.. note::
- The command ``floatcsep run `` can be called from any working directory, as long as the specified file paths (e.g. region, models) are relative to the ``config.yml`` file.
-
-
-Plot command
-~~~~~~~~~~~~
-
- If only the result plots are desired, when the calculation was already completed, you can type:
-
- .. code-block:: console
-
- $ floatcsep plot config.yml
-
- This can be used, for example, when an additional plot is desired. Try adding to ``config.yml`` the following lines
-
- .. code-block:: yaml
-
- postproc_config:
- plot_forecasts: True
+ The command ``floatcsep run {config_file}`` can be called from any working directory, as long as the specified file paths (e.g. region, models) are relative to the ``config.yml`` file.
- and re-run with the ``plot`` command. A forecast figure will appear in ``results/{window}/forecasts``
Results
~~~~~~~
@@ -155,13 +148,13 @@ Results
* The testing catalog of the window is stored in ``results/{window}/catalog`` in ``json`` format. This is a subset of the global testing catalog.
* Human-readable results are found in ``results/{window}/evaluations``
- * Catalog and evaluation results figures in ``results/{window}/figures``.
+ * Catalog, forecasts and evaluation results figures in ``results/{window}/figures``.
* The complete results are summarized in ``results/report.md``
Advanced
---------
+~~~~~~~~
-The experiment run logic can be seen in the file ``case_a.py``, which executes the same example but in python source code. The run logic of the terminal commands ``run``, ``plot`` and ``reproduce`` can be found in :mod:`floatcsep.commands.main`
+The experiment run logic can be seen in the file ``case_a.py``, which executes the same example but in python source code. The run logic of the terminal commands ``run``, ``plot`` and ``reproduce`` can be found in :mod:`floatcsep.commands.main`, and can be customized by creating a script similar to ``case_a.py``.
diff --git a/docs/examples/case_b.rst b/docs/examples/case_b.rst
index 8c30099..700019f 100644
--- a/docs/examples/case_b.rst
+++ b/docs/examples/case_b.rst
@@ -1,10 +1,9 @@
B - Multiple Models and Tests
=============================
-.. currentmodule:: floatcsep
+The following example is an experiment including **multiple** time-independent forecasts and **multiple** evaluations.
-.. contents::
- :local:
+.. currentmodule:: floatcsep
.. admonition:: **TL; DR**
@@ -16,11 +15,15 @@ B - Multiple Models and Tests
After the calculation is complete, the results will be summarized in ``results/report.md``.
+.. contents::
+ :local:
+ :depth: 2
+
Experiment Components
---------------------
-The following example is an experiment including **multiple** time-independent forecasts and evaluations. The input structure of the experiment is:
+The source code can be found in the ``examples/case_b`` folder or in `GitHub `_. The input structure of the experiment is:
::
@@ -36,11 +39,11 @@ The following example is an experiment including **multiple** time-independent f
├── tests.yml
└── region.txt
-
-The testing catalog is now defined in ``json`` format, which is the default catalog used by ``floatcsep``, as it allows the storage of metadata.
+.. important::
+ Although not necessary, the testing catalog is here defined in the ``.json`` format, which is the default catalog used by ``floatcsep``, as it allows the storage of metadata.
.. note::
- An user-defined catalog can be saved as ``json`` with :meth:`CSEPCatalog.write_json() ` using ``pycsep``
+ A catalog can be stored as ``.json`` with :meth:`CSEPCatalog.write_json() ` using ``pycsep``
Configuration
@@ -49,10 +52,11 @@ Configuration
In this example, the time, region and catalog specifications are written in the ``config.yml`` file.
.. literalinclude:: ../../examples/case_b/config.yml
+ :caption: examples/case_b/config.yml
:language: yaml
:lines: 3-15
-whereas the models' and tests' configurations are referred to external files for readability
+whereas the models' and tests' configurations are referred to external files for better readability
.. literalinclude:: ../../examples/case_b/config.yml
:language: yaml
@@ -64,6 +68,7 @@ Models
The model configuration is now set in the ``models.yml`` file, where a list of model names specify their file paths.
.. literalinclude:: ../../examples/case_b/models.yml
+ :caption: examples/case_b/models.yml
:language: yaml
Evaluations
@@ -72,11 +77,12 @@ Evaluations
.. literalinclude:: ../../examples/case_b/tests.yml
:language: yaml
+ :caption: examples/case_b/tests.yml
.. note::
Plotting keyword arguments can be set in the ``plot_kwargs`` option - see :func:`~csep.utils.plots.plot_poisson_consistency_test` and :func:`~csep.utils.plots.plot_comparison_test` -.
- .. note::
+ .. important::
Comparison tests (such as the ``paired_t_test``) requires a reference model, whose name should be set in ``ref_model`` at the given test configuration.
Running the experiment
@@ -87,7 +93,7 @@ The experiment can be run by simply navigating to the ``examples/case_b`` folder
.. code-block:: console
- floatcsep run config.yml
+ $ floatcsep run config.yml
This will automatically set all the file paths of the calculation (testing catalogs, evaluation results, figures) and will display a summarized report in ``results/report.md``.
diff --git a/docs/examples/case_c.rst b/docs/examples/case_c.rst
index e7d4a39..8368ca7 100644
--- a/docs/examples/case_c.rst
+++ b/docs/examples/case_c.rst
@@ -1,10 +1,9 @@
C - Multiple Time Windows
=========================
-.. currentmodule:: floatcsep
+The following example shows an experiment with **multiple time windows**.
-.. contents::
- :local:
+.. currentmodule:: floatcsep
.. admonition:: **TL; DR**
@@ -16,11 +15,15 @@ C - Multiple Time Windows
After the calculation is complete, the results will be summarized in ``results/report.md``.
+.. contents::
+ :local:
+
+
Experiment Components
---------------------
-The following example shows an experiment with multiple time windows. The input structure of the experiment is:
+The source code can be found in the ``examples/case_c`` folder or in `GitHub `_. The input structure of the experiment is:
::
@@ -42,11 +45,12 @@ Configuration
Time
~~~~
- The time configuration now set the time intervals between the start and end dates.
+ The time configuration now sets a sequence of time intervals between the start and end dates.
.. literalinclude:: ../../examples/case_c/config.yml
- :language: yaml
- :lines: 3-7
+ :caption: examples/case_c/config.yml
+ :language: yaml
+ :lines: 3-7
.. note::
@@ -58,25 +62,32 @@ Time
Evaluations
~~~~~~~~~~~
- The experiment's evaluations are defined in ``tests.yml``, which can now include temporal evaluations (see :func:`~floatcsep.extras.sequential_likelihood`, :func:`~floatcsep.extras.sequential_information_gain`, :func:`~floatcsep.utils.plot_sequential_likelihood`).
+ The experiment's evaluations are defined in ``tests.yml``, which can now include temporal evaluations (see :func:`~floatcsep.utils.helpers.sequential_likelihood`, :func:`~floatcsep.utils.helpers.sequential_information_gain`, :func:`~floatcsep.utils.helpers.plot_sequential_likelihood`).
.. literalinclude:: ../../examples/case_c/tests.yml
- :language: yaml
+ :language: yaml
+ :caption: examples/case_c/tests.yml
.. note::
- Plot arguments (title, labels, font sizes, axes limits, etc.) can be passed as a dictionary in ``plot_args`` (see details in :func:`~csep.utils.plots.plot_poisson_consistency_test`)
+ Plot arguments (title, labels, font sizes, axes limits, etc.) can be passed as a dictionary in ``plot_args`` (see the arguments details in :func:`~csep.utils.plots.plot_poisson_consistency_test`)
Results
-------
-The :obj:`~floatcsep.cmd.main.run` command creates the result path tree for all time windows.
+The :obj:`~floatcsep.cmd.main.run` command
+
+.. code-block:: console
+
+ $ floatcsep run config.yml
+
+now creates the result path tree for all time windows.
-* The testing catalog of the window is stored in ``results/{window}/catalog`` in ``json`` format. This is a subset of the global testing catalog.
-* Human-readable results are found in ``results/{window}/evaluations``
-* Catalog and evaluation results figures in ``results/{window}/figures``.
+* The testing catalog of the window is stored in ``results/{time_window}/catalog`` in ``json`` format. This is a subset of the global testing catalog.
+* Human-readable results are found in ``results/{time_window}/evaluations``
+* Catalog and evaluation results figures in ``results/{time_window}/figures``.
* The complete results are summarized in ``results/report.md``
-The report now shows the temporal evaluations for all time-windows, whereas the discrete evaluations are shown only for the last time window.
+The report shows the temporal evaluations for all time-windows, whereas the discrete evaluations are shown only for the last time window.
diff --git a/docs/examples/case_d.rst b/docs/examples/case_d.rst
index bbf7445..ef9c9c9 100644
--- a/docs/examples/case_d.rst
+++ b/docs/examples/case_d.rst
@@ -1,10 +1,11 @@
D - Catalog Queries and Model Repositories
==========================================
-.. currentmodule:: floatcsep
+The following example shows an experiment whose forecasts are **retrieved from a repository** (Zenodo - https://zenodo.org) and the testing **catalog** from an authoritative source **web service** (namely the gCMT catalog from the International Seismological Centre - http://www.isc.ac.uk).
+
-.. contents::
- :local:
+
+.. currentmodule:: floatcsep
.. admonition:: **TL; DR**
@@ -16,11 +17,14 @@ D - Catalog Queries and Model Repositories
After the calculation is complete, the results will be summarized in ``results/report.md``.
+.. contents::
+ :local:
+
Experiment Components
---------------------
-The following example shows an experiment whose forecasts are retrieved from a repository (Zenodo - https://zenodo.org) and the testing catalog from an authoritative source web service (namely the gCMT catalog from the ISC - http://www.isc.ac.uk). The initial structure is:
+The source code can be found in the ``examples/case_d`` folder or in `GitHub `_. The **initial** input structure of the experiment is:
::
@@ -60,8 +64,9 @@ Catalog
The ``catalog`` inset from ``config.yml`` now makes reference to a catalog query function, in this case :func:`~pycsep.query_gcmt`.
.. literalinclude:: ../../examples/case_d/config.yml
- :language: yaml
- :lines: 14-14
+ :caption: examples/case_d/config.yml
+ :language: yaml
+ :lines: 14-14
``floatcsep`` will automatically filter the catalog to the experiment time, spatial and magnitude windows of the experiment.
@@ -74,14 +79,15 @@ Models
The model configuration is set in ``models.yml``.
.. literalinclude:: ../../examples/case_d/models.yml
- :language: yaml
+ :caption: examples/case_d/models.yml
+ :language: yaml
* The option ``zenodo_id`` makes reference to the zenodo **record id**. The model ``team`` is found in https://zenodo.org/record/6289795, whereas the model ``wheel`` in https://zenodo.org/record/6255575.
- * The option ``flavours`` allows multiple model sub-classes to be quickly instantiated.
-
* The ``zenodo`` (or ``git``) repositories could contain multiple files, each of which can be assigned to a flavour.
+ * The option ``flavours`` allows multiple model sub-classes to be quickly instantiated.
+
* When multiple flavours are passed, ``path`` refers to the folder where the models would be downloaded.
* If a single file of the repository is needed (without specifying model flavours), ``path`` can reference to the file itself. For example, you can try replacing the whole WHEEL inset in ``models.yml`` to:
diff --git a/docs/examples/case_e.rst b/docs/examples/case_e.rst
index acb93ba..53c37be 100644
--- a/docs/examples/case_e.rst
+++ b/docs/examples/case_e.rst
@@ -1,10 +1,9 @@
E - A Realistic Time-independent Experiment
===========================================
-.. currentmodule:: floatcsep
+This example shows how to run a realistic testing experiment (based on https://doi.org/10.4401/ag-4844) while summarizing the concepts from the previous examples.
-.. contents::
- :local:
+.. currentmodule:: floatcsep
.. admonition:: **TL; DR**
@@ -16,11 +15,15 @@ E - A Realistic Time-independent Experiment
After the calculation is complete, the results will be summarized in ``results/report.md``.
+.. contents::
+ :local:
+
+
Experiment Components
---------------------
-This example shows how to run a realistic testing experiment in Italy (based on https://doi.org/10.4401/ag-4844), summarizing the concepts of previous examples. The example has only a subset of the original models and evaluations. The input structure of the experiment is:
+The source code can be found in the ``examples/case_e`` folder or in `GitHub `_. The input structure of the experiment is:
::
@@ -33,6 +36,9 @@ This example shows how to run a realistic testing experiment in Italy (based on
├── models.yml
└── tests.yml
+.. note::
+ This experiment has only a subset of the original models and evaluations.
+
Configuration
-------------
@@ -44,8 +50,9 @@ Time
The time configuration is manifested in the ``time-config`` inset.
.. literalinclude:: ../../examples/case_e/config.yml
- :language: yaml
- :lines: 3-7
+ :caption: examples/case_e/config.yml
+ :language: yaml
+ :lines: 3-7
Region
~~~~~~
@@ -53,8 +60,9 @@ Region
The testing region is the official Italy CSEP Region obtained from :obj:`csep.core.regions.italy_csep_region`.
.. literalinclude:: ../../examples/case_e/config.yml
- :language: yaml
- :lines: 9-15
+ :caption: examples/case_e/config.yml
+ :language: yaml
+ :lines: 9-15
Catalog
@@ -63,21 +71,23 @@ Catalog
The catalog is obtained from an authoritative source, namely the Bollettino Sismico Italiano (http://terremoti.ingv.it/en/bsi ), using the function :func:`csep.query_bsi`
.. literalinclude:: ../../examples/case_e/config.yml
- :language: yaml
- :lines: 17-17
+ :caption: examples/case_e/config.yml
+ :language: yaml
+ :lines: 17-17
Models
~~~~~~
The models are set in ``models.yml``. For simplicity, there are only three of the nineteen models originally submitted to the Italy Experiment.
.. literalinclude:: ../../examples/case_e/models.yml
- :language: yaml
+ :caption: examples/case_e/models.yml
+ :language: yaml
The ``.xml`` format is automatically detected and parsed by ``floatcsep`` readers.
- .. note::
+ .. important::
- The forecasts are defined in ``[Earthquakes / 10-years]``, specified with the ``forecast_unit`` option.
+ The forecasts are defined in ``[Earthquakes / 10-years]``, which is specified with the ``forecast_unit`` option (The default is `forecast_unit: 1`).
.. note::
@@ -86,23 +96,42 @@ Models
Post-Process
~~~~~~~~~~~~
- Additional options for post-processing can set using the ``postprocess`` option.
+ Additional options for post-processing can set using the ``postprocess`` option. Here, we customize the forecasts plotting with:
.. literalinclude:: ../../examples/case_e/config.yml
:language: yaml
:lines: 21-34
- The forecasts are plotted and stored in ``examples/case_e/results/{timewindow}/forecasts/``. See :func:`~csep.utils.plots.plot_spatial_dataset` for forecast plot options and :func:`~csep.utils.plots.plot_catalog` for the catalog placed on top.
+ The forecasts are plotted and stored in ``examples/case_e/results/{timewindow}/forecasts/``. See :func:`~csep.utils.plots.plot_spatial_dataset` for forecast plotting options and :func:`~csep.utils.plots.plot_catalog` for the catalog placed on top of those plots.
Running the experiment
----------------------
- The experiment can be run by simply navigating to the ``examples/case_e`` folder in the terminal and typing.
+ The experiment can be run by navigating to the ``examples/case_e`` folder in the terminal and typing.
.. code-block:: console
- floatcsep run config.yml
+ $ floatcsep run config.yml
This will automatically set all the calculation paths (testing catalogs, evaluation results, figures) and will create a summarized report in ``results/report.md``.
+
+Plot command
+~~~~~~~~~~~~
+
+ If only the result plots are desired when the calculation was already completed before, you can type:
+
+ .. code-block:: console
+
+ $ floatcsep plot config.yml
+
+ This can be used, for example, when an additional plot is desired without re-running the entire experiment. Try adding the following line to the ``postprocess`` inset of the ``config.yml`` file.
+
+ .. code-block:: yaml
+
+ postprocess:
+ plot_forecasts:
+ colormap: magma
+
+ and re-run with the ``plot`` command. A forecast figure will re-appear in ``results/{window}/forecasts`` with a different colormap. Additional forecast and catalog plotting options can be found in the :func:`csep.utils.plots.plot_spatial_dataset` and :func:`csep.utils.plots.plot_catalog` ``pycsep`` functions.
diff --git a/docs/examples/case_f.rst b/docs/examples/case_f.rst
index fad63fa..858d9ad 100644
--- a/docs/examples/case_f.rst
+++ b/docs/examples/case_f.rst
@@ -1,10 +1,9 @@
-G - Time-Dependent, Catalog-Based Model (from existing files)
-==========================================================
+F - Time-Dependent Catalog-Based Model (from existing files)
+=============================================================
-.. currentmodule:: floatcsep
+This example shows how set up an experiment with a **time-dependent** model, whose forecast files already exist.
-.. contents::
- :local:
+.. currentmodule:: floatcsep
.. admonition:: **TL; DR**
@@ -16,12 +15,16 @@ G - Time-Dependent, Catalog-Based Model (from existing files)
After the calculation is complete, the results will be summarized in ``results/report.md``.
+.. contents::
+ :local:
+
Experiment Components
---------------------
-This example shows how set up a time-dependent model, whose files are already existing, and experiment. The model structure is as follows:
+The source files can be found in the ``examples/case_e`` folder or in `GitHub `_. The experiment structure is as follows:
+
::
case_f
@@ -37,13 +40,14 @@ This example shows how set up a time-dependent model, whose files are already ex
* The model to be evaluated (``etas``) is a collection of daily forecasts from ``2016-11-14`` until ``2016-11-21``.
-* The forecasts are located in a folder ``forecasts`` inside the model, to be consistent with models based on source codes (see the subsequent examples).
+.. important::
+ The forecasts must be located in a folder ``forecasts`` inside the model folder. This is meant for consistency with models based on source codes (see subsequent examples).
Model
-----
-The time-dependency of a model is manifested here by the provision of different forecasts, i.e., statistical descriptions of seismicity, for different time-windows. For this example, the forecasts were created from an external model https://github.com/lmizrahi/etas (https://doi.org/10.1785/0220200231
+The time-dependency of a model is manifested here by the provision of different forecasts, i.e., statistical descriptions of seismicity, for different time-windows. In this example, the forecasts were created from an external model https://github.com/lmizrahi/etas (https://doi.org/10.1785/0220200231
), with which the experiment has no interface. This means that we only the forecast files are required. We leave the handling of a model source code for subsequent examples.
@@ -55,27 +59,40 @@ Configuration
Time
~~~~
- The configuration is identical to time-independent models with multiple time-windows (e.g., case C), with the exception that now a ``horizon`` can be defined instead of ``intervals``, which is the forecast time-window length. The experiment's class should now be explicited as ``exp_class: td``
+ The configuration is identical to time-independent models with multiple time-windows (e.g., case C) with the exception that a ``horizon`` could be defined instead of ``intervals``, which is the forecast time-window length. The experiment's class should now be explicited as ``exp_class: td``.
.. literalinclude:: ../../examples/case_f/config.yml
- :language: yaml
- :lines: 3-7
+ :caption: examples/case_f/config.yml
+ :language: yaml
+ :lines: 3-7
+
+.. note::
+ **floatCSEP** is flexible with the definition of time windows/deltas. Alternative string inputs for ``horizon`` can be ``1-day``, ``1 day``, ``1d``, etc.
Catalog
~~~~~~~
- The catalog was obtained ``previous to the experiment`` using ``query_geonet`` and it was filtered to the testing period.
+ The catalog ``catalog.json`` was obtained *previously* by using ``query_geonet`` and it was filtered to the testing period. However, it can be re-queried by changing its definition to:
+
+ .. code-block:: yaml
+
+ catalog: query_geonet
Models
~~~~~~
- Some additional arguments should be passed to a time-dependent model
+ Some additional arguments should be passed to a **time-dependent** model, such as its class ('td' for time-dependent) and the number of simulations.
.. literalinclude:: ../../examples/case_f/models.yml
- :language: yaml
- :lines: 1-4
+ :caption: examples/case_f/config.yml
+ :language: yaml
+ :lines: 1-4
+
+.. note::
+ For consistency with time-dependent models that will create forecasts from a source code, the ``path`` should point to the folder of the model, which itself should contain a sub-folder named ``{path}/forecasts`` where the files are located.
- For consistency with time-dependent models that will create forecasts from a source code, the ``path`` should point to the folder of the model. The path folder should contain a sub-folder named ``{path}/forecasts`` where the files are located. Note that fore catalog-based forecasts, the number of simulations should be explicit.
+.. important::
+ Note that for catalog-based forecasts, the model should explicit the number of simulations. This is meant for forecast files that contain synthetic catalogs with zero-event simulations, and therefore do not contain the total number of synthetic catalogs used.
Tests
~~~~~
diff --git a/docs/examples/case_g.rst b/docs/examples/case_g.rst
index 58de35b..f901df6 100644
--- a/docs/examples/case_g.rst
+++ b/docs/examples/case_g.rst
@@ -20,8 +20,8 @@ G - Time-Dependent, Catalog-Based Model (from Source Code)
Experiment Components
---------------------
-
This example shows how a time-dependent model should be set up for a time-dependent experiment
+
::
case_g
@@ -53,7 +53,7 @@ The experiment's complexity increases from time-independent to dependent, since
.. literalinclude:: ../../examples/case_g/catalog.csv
:lines: 1-2
- 2. The **input arguments** controls how the model's source code works. The minimum arguments to run a model (which should be modified dynamically during an experiment) are the forecast ``start_date`` and ``end_date``. The experiment will read `{model}/input/args.txt` and change the values of ``start_date = {datetime}`` and ``end_date = {datetime}`` before the model is run. Additional arguments can be set by convenience, such as ``catalog`` (the input catalog name), ``n_sims`` (number of synthetic catalogs) and random ``seed`` for reproducibility.
+ 2. The **input arguments** controls how the model's source code works. The minimum arguments to run a model (which should be modified dynamically during an experiment) are the forecast ``start_date`` and ``end_date``. The experiment will access `{model}/input/args.txt` and change the values of ``start_date = {datetime}`` and ``end_date = {datetime}`` before the model is run. Additional arguments can be set by convenience, such as ``catalog`` (the input catalog name), ``n_sims`` (number of synthetic catalogs) and random ``seed`` for reproducibility.
* **Output**: The model's output are the synthetic catalogs, which should be allocated in `{model}/forecasts/{filename}.csv`. The format is identically to ``csep_ascii``, but unlike in an input catalog, the ``catalog_id`` column should be modified for each synthetic catalog starting from 0. The file name follows the convention `{model_name}_{start}_{end}.csv`, where ``start`` and ``end`` folowws the `%Y-%m-%dT%H:%M:%S.%f` - ISO861 FORMAT
@@ -126,6 +126,30 @@ Tests
It is possible to assign two plotting functions to a test, whose ``plot_args`` and ``plot_kwargs`` can be placed indented beneath
+Custom Post-Process
+~~~~~~~~~~~~~~~~~~~
+
+ Additional to the default :func:`~floatcsep.postprocess.plot_handler.plot_results`, :func:`~floatcsep.postprocess.plot_handler.plot_catalogs`, :func:`~floatcsep.postprocess.plot_handler.plot_forecasts` functions, a custom plotting function(s) can be set within the ``postprocess`` configuration
+
+ .. literalinclude:: ../../examples/case_g/config.yml
+ :language: yaml
+ :lines: 22-23
+
+ This option provides `hook` for a python script and a function within as:
+
+ .. code-block:: console
+
+ {python_sript}:{function_name}
+
+ The requirements are that the script to be located within the same directory as the configuration file, whereas the function must receive a :class:`floatcsep.experiment.Experiment` as argument
+
+ .. literalinclude:: ../../examples/case_g/custom_plot_script.py
+ :language: yaml
+ :lines: 6-9
+
+ In this way, the plot function use all the :class:`~floatcsep.experiment.Experiment` attributes/methods to access catalogs, forecasts and test results. The script ``examples/case_g/custom_plot_script.py`` can also be viewed directly on `GitHub `_, where it is exemplified how to access the experiment artifacts.
+
+
Running the experiment
----------------------
diff --git a/docs/examples/case_h.rst b/docs/examples/case_h.rst
new file mode 100644
index 0000000..d0a5134
--- /dev/null
+++ b/docs/examples/case_h.rst
@@ -0,0 +1,148 @@
+H - Multiple Catalog-Based Models
+=================================
+
+.. currentmodule:: floatcsep
+
+.. contents::
+ :local:
+ :depth: 1
+
+.. admonition:: **TL; DR**
+
+ In a terminal, navigate to ``floatcsep/examples/case_h`` and type:
+
+ .. code-block:: console
+
+ $ floatcsep run config.yml
+
+ After the calculation is complete, the results will be summarized in ``results/report.md``.
+
+
+Experiment Components
+---------------------
+
+This example shows how to run an experiment that access, downloads, containerize and execute multiple time-dependent models. The experiment input files are:
+
+::
+
+ case_h
+ ├── catalog.csv
+ ├── config.yml
+ ├── models.yml
+ └── tests.yml
+
+* The ``models.yml`` contains the instructions to clone and build the source codes from software repositories (e.g., gitlab, Github)
+
+
+Models
+------
+
+As in Example G, the complexity increases because each **Model** requires to build and execute a source code to generate forecast for multilpe time-windows. The instructions for each model are located within ``models.yml``. The URL and specific version (e.g., commit hash, tag, release) are specified as:
+
+
+ .. literalinclude:: ../../examples/case_h/models.yml
+ :lines: 1-3
+
+The model source code requires to be synchronized with the experiment interface, in such a way that is able to read the inputs (catalog and argument file) from an ``{model_path}/input`` folder and store the forecast outputs from a folder ``{model_path}/forecasts``.
+
+ .. literalinclude:: ../../examples/case_h/models.yml
+ :lines: 4-6
+
+The ``func`` parameter indicates how the model should be invoked from a terminal (e.g, a python virtual environment, docker container, etc.). The type of container is set with the parameter ``build`` (``floatcsep`` supports ``conda``, ``venv`` and ``Dockerfile``). Note that we specify to ``floatcsep`` which arguments file will be read from the module. In this case, the ETAS model uses ``args.json`` file (``floatcsep`` suports ``.json`` and ``.txt`` files). ``floatcsep`` will dynamically modify the ``start_time`` and ``end_time`` for each time window run, and statically (i.e., for all time-windows) allocate the parameters set as:
+
+ .. literalinclude:: ../../examples/case_h/models.yml
+ :lines: 7-9
+
+Same as Example G, the repositories should follow the following:
+
+
+* **Input**: The input consists in input **data** and **arguments**.
+
+ 1. The **input data** is, at the least, a catalog filtered until the forecast beginning, which is automatically allocated by ``floatcsep`` in the `{model}/input` prior to each model's run. It is stored inside the model in ``csep.ascii`` format for simplicity's sake (see :doc:`pycsep:concepts/catalogs`).
+
+ .. literalinclude:: ../../examples/case_h/catalog.csv
+ :lines: 1-2
+
+ 2. The **input arguments** controls how the model's source code works. The minimum arguments to run a model (which should be modified dynamically during an experiment) are the forecast ``start_date`` and ``end_date``. The experiment will access `{model}/input/args.json` and change the values of ``"start_date": "{datetime}"`` and ``"end_date": "{datetime}"`` before the model is run. Additional arguments can be set by convenience, such as ``n_sims`` (number of synthetic catalogs), ``m_c`` for the cutoff magnitude and a random ``seed`` for reproducibility.
+
+* **Output**: The model's output are the synthetic catalogs, which should be allocated in `{model}/forecasts/{filename}.csv`. The format is identically to ``csep_ascii``, but unlike in an input catalog, the ``catalog_id`` column should be modified for each synthetic catalog starting from 0. The file name follows the convention `{model_name}_{start}_{end}.csv`, where ``start`` and ``end`` folowws the `%Y-%m-%dT%H:%M:%S.%f` - ISO861 FORMAT
+
+* **Model build**: Inside the model source code, there are multiple options to build it. A standard python ``setup.cfg`` is given, which can be built inside a python ``venv`` or ``conda`` managers. This is created and built automatically by ``floatCSEP``, as long as the the model build instructions are correctly set up.
+
+* **Model run**: The model should be run with a simple command to which only ``arguments`` should be passed. For this example, is
+
+ .. code-block:: console
+
+ $ etas-run
+
+
+ as long as it internally reads the ``input/args.json`` and ``input/catalog.csv`` files.
+
+
+Configuration
+-------------
+
+
+Time
+~~~~
+
+ The configuration is identical to time-independent models, with the exception that now a ``horizon`` can be defined instead of ``intervals``, which is the forecast time-window length. The experiment's class should now be explicited as ``exp_class: td``
+
+ .. literalinclude:: ../../examples/case_h/config.yml
+ :language: yaml
+ :lines: 3-7
+
+Catalog
+~~~~~~~
+
+ The catalog was obtained ``previous to the experiment`` using ``query_bsi``, but it was filtered from 2006 onwards, so it has enough data for the model calibration.
+
+
+Tests
+~~~~~
+
+ With time-dependent models, now catalog evaluations found in :obj:`csep.core.catalog_evaluations` can be used.
+
+
+ .. literalinclude:: ../../examples/case_h/tests.yml
+ :language: yaml
+
+ .. note::
+ It is possible to assign two plotting functions to a test, whose ``plot_args`` and ``plot_kwargs`` can be placed indented beneath
+
+
+Custom Post-Process
+~~~~~~~~~~~~~~~~~~~
+
+ A custom reporting function can be set within the ``postprocess`` configuration to replace the :func:`~floatcsep.postprocess.reporting.generate_report`:
+
+ .. literalinclude:: ../../examples/case_h/config.yml
+ :language: yaml
+ :lines: 22-23
+
+ This option provides `hook` for a python script and a function within as:
+
+ .. code-block:: console
+
+ {python_sript}:{function_name}
+
+ The requirements are that the script to be located within the same directory as the configuration file, whereas the function must receive a :class:`floatcsep.experiment.Experiment` as argument
+
+ .. literalinclude:: ../../examples/case_h/custom_report.py
+ :language: yaml
+ :lines: 5-11
+
+ In this way, the report function use all the :class:`~floatcsep.experiment.Experiment` attributes/methods to access catalogs, forecasts and test results. The script ``examples/case_h/custom_report.py`` can also be viewed directly on `GitHub `_, where it is exemplified how to access the experiment artifacts.
+
+
+Running the experiment
+----------------------
+
+ The experiment can be run by simply navigating to the ``examples/case_h`` folder in the terminal and typing.
+
+ .. code-block:: console
+
+ $ floatcsep run config.yml
+
+ This will automatically set all the calculation paths (testing catalogs, evaluation results, figures) and will create a summarized report in ``results/report.md``.
+
diff --git a/docs/guide/postprocess_config.rst b/docs/guide/postprocess_config.rst
new file mode 100644
index 0000000..025b590
--- /dev/null
+++ b/docs/guide/postprocess_config.rst
@@ -0,0 +1,4 @@
+Post-process options
+====================
+
+TBI
\ No newline at end of file
diff --git a/docs/index.rst b/docs/index.rst
index 82ab549..40a4956 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -20,8 +20,9 @@ Preliminary documentation.
examples/case_c.rst
examples/case_d.rst
examples/case_e.rst
+ examples/case_f.rst
examples/case_g.rst
-
+ examples/case_h.rst
.. toctree::
:maxdepth: 2
@@ -32,6 +33,7 @@ Preliminary documentation.
guide/region_config.rst
guide/model_config.rst
guide/tests_config.rst
+ guide/postprocess_config.rst
.. toctree::
:maxdepth: 2
diff --git a/examples/case_a/best_model.dat b/examples/case_a/best_model.dat
index a5b5003..55957d9 100644
--- a/examples/case_a/best_model.dat
+++ b/examples/case_a/best_model.dat
@@ -1,4 +1,3 @@
# lon_min lon_max lat_min lat_max depth_min depth_max mag_min mag_max rate mask
0 1 0 1 0 70 6 7 5e-1 1
-1 2 0 1 0 70 6 7 5e-1 0.1
-
+1 2 0 1 0 70 6 7 5e-1 0.1
\ No newline at end of file
diff --git a/examples/case_a/config.yml b/examples/case_a/config.yml
index 1835b0d..ceab9a3 100644
--- a/examples/case_a/config.yml
+++ b/examples/case_a/config.yml
@@ -23,3 +23,7 @@ tests:
func: poisson_evaluations.number_test
plot_func: plot_poisson_consistency_test
+postprocess:
+ plot_forecasts:
+ cmap: magma
+ catalog: True
diff --git a/examples/case_d/config.yml b/examples/case_d/config.yml
index dd2726b..a4d4917 100644
--- a/examples/case_d/config.yml
+++ b/examples/case_d/config.yml
@@ -14,6 +14,3 @@ region_config:
catalog: query_gcmt
models: models.yml
test_config: tests.yml
-
-postprocess:
- plot_forecasts: True
\ No newline at end of file
diff --git a/examples/case_e/config.yml b/examples/case_e/config.yml
index 1f235e5..5fc86c9 100644
--- a/examples/case_e/config.yml
+++ b/examples/case_e/config.yml
@@ -20,6 +20,7 @@ test_config: tests.yml
postprocess:
plot_forecasts:
+ cmap: magma
region_border: True
basemap: stock_img
clabel_fontsize: 14
diff --git a/examples/case_f/config.yml b/examples/case_f/config.yml
index 68adf11..d62af88 100644
--- a/examples/case_f/config.yml
+++ b/examples/case_f/config.yml
@@ -1,4 +1,4 @@
-name: case_g
+name: case_f
time_config:
start_date: 2016-11-14T00:00:00
diff --git a/examples/case_g/custom_plot_script.py b/examples/case_g/custom_plot_script.py
index 6f9d62f..c19c656 100644
--- a/examples/case_g/custom_plot_script.py
+++ b/examples/case_g/custom_plot_script.py
@@ -5,7 +5,11 @@
def main(experiment):
"""
- Example custom plot function
+ Example custom plot function (Observed vs. forecast rates in time)
+
+ Args:
+ experiment: a floatcsep.experiment.Experiment class
+
"""
# Get all the timewindows
diff --git a/examples/case_h/config.yml b/examples/case_h/config.yml
index ca5c7f9..3d93d70 100644
--- a/examples/case_h/config.yml
+++ b/examples/case_h/config.yml
@@ -1,4 +1,4 @@
-name: case_f
+name: case_h
time_config:
start_date: 2016-8-25T00:00:00
@@ -20,5 +20,5 @@ model_config: models.yml
test_config: tests.yml
postprocess:
+ report: custom_report.py:main
plot_catalog: False
- report: custom_report.py:main
\ No newline at end of file
diff --git a/examples/case_h/custom_report.py b/examples/case_h/custom_report.py
index 2d4d0cf..d705d5d 100644
--- a/examples/case_h/custom_report.py
+++ b/examples/case_h/custom_report.py
@@ -3,10 +3,19 @@
def main(experiment):
+ """
+ Args:
+ experiment: a floatcsep.experiment.Experiment class
+
+ """
+ # Access the last time-window
timewindow = experiment.timewindows[-1]
+
+ # Convert the time-window to a string
timestr = timewindow2str(timewindow)
+ # Instantiates a Report object and adds a title and objectives
report = MarkdownReport()
report.add_title(f"Experiment Report - {experiment.name}", "")
report.add_heading("Objectives", level=2)
@@ -17,6 +26,7 @@ def main(experiment):
]
report.add_list(objs)
+ # Adds an input figure
report.add_figure(
f"Input catalog",
[
@@ -44,4 +54,5 @@ def main(experiment):
width=200,
)
+ # Stores the report
report.save(experiment.registry.abs(experiment.registry.run_dir))
diff --git a/examples/case_h/models.yml b/examples/case_h/models.yml
index e2c99f7..fb3a607 100644
--- a/examples/case_h/models.yml
+++ b/examples/case_h/models.yml
@@ -1,8 +1,8 @@
- etas:
giturl: https://git.gfz-potsdam.de/csep/it_experiment/models/vetas.git
repo_hash: v3.2
- args_file: input/args.json
path: models/etas
+ args_file: input/args.json
func: etas-run
func_kwargs:
n_sims: 100
diff --git a/floatcsep/postprocess/plot_handler.py b/floatcsep/postprocess/plot_handler.py
index c8d0671..f6531c4 100644
--- a/floatcsep/postprocess/plot_handler.py
+++ b/floatcsep/postprocess/plot_handler.py
@@ -96,6 +96,8 @@ def plot_forecasts(experiment: "Experiment") -> None:
# If catalog option is passed, catalog is plotted on top of the forecast
if plot_forecast_config.get("catalog"):
cat_args = plot_forecast_config.get("catalog", {})
+ if cat_args is True:
+ cat_args = {}
experiment.catalog_repo.get_test_cat(window).plot(
ax=ax,
extent=ax.get_extent(),
From 91dad5c0792f60fc0048bbc498d995eb6843af8d Mon Sep 17 00:00:00 2001
From: pciturri
Date: Sun, 22 Sep 2024 23:39:07 +0200
Subject: [PATCH 04/19] docs: Finalized tutorials G and H. Reworked local TOC
for all examples.
---
docs/examples/case_a.rst | 2 +-
docs/examples/case_b.rst | 2 +-
docs/examples/case_c.rst | 2 +-
docs/examples/case_d.rst | 2 +-
docs/examples/case_e.rst | 2 +-
docs/examples/case_f.rst | 7 +-
docs/examples/case_g.rst | 111 ++++++++++++++++++++----------
docs/examples/case_h.rst | 137 +++++++++++++++++++++++++++----------
docs/index.rst | 11 +--
examples/case_h/models.yml | 6 +-
10 files changed, 194 insertions(+), 88 deletions(-)
diff --git a/docs/examples/case_a.rst b/docs/examples/case_a.rst
index 93978dd..f8d218c 100644
--- a/docs/examples/case_a.rst
+++ b/docs/examples/case_a.rst
@@ -15,7 +15,7 @@ The following example shows the definition of a testing experiment of a single *
After the calculation is complete, the results will be summarized in ``results/report.md``.
-.. contents::
+.. contents:: Contents
:local:
:depth: 2
diff --git a/docs/examples/case_b.rst b/docs/examples/case_b.rst
index 700019f..2a5897f 100644
--- a/docs/examples/case_b.rst
+++ b/docs/examples/case_b.rst
@@ -15,7 +15,7 @@ The following example is an experiment including **multiple** time-independent f
After the calculation is complete, the results will be summarized in ``results/report.md``.
-.. contents::
+.. contents:: Contents
:local:
:depth: 2
diff --git a/docs/examples/case_c.rst b/docs/examples/case_c.rst
index 8368ca7..5410ae0 100644
--- a/docs/examples/case_c.rst
+++ b/docs/examples/case_c.rst
@@ -15,7 +15,7 @@ The following example shows an experiment with **multiple time windows**.
After the calculation is complete, the results will be summarized in ``results/report.md``.
-.. contents::
+.. contents:: Contents
:local:
diff --git a/docs/examples/case_d.rst b/docs/examples/case_d.rst
index ef9c9c9..90ddff3 100644
--- a/docs/examples/case_d.rst
+++ b/docs/examples/case_d.rst
@@ -17,7 +17,7 @@ The following example shows an experiment whose forecasts are **retrieved from a
After the calculation is complete, the results will be summarized in ``results/report.md``.
-.. contents::
+.. contents:: Contents
:local:
diff --git a/docs/examples/case_e.rst b/docs/examples/case_e.rst
index 53c37be..dda7f5f 100644
--- a/docs/examples/case_e.rst
+++ b/docs/examples/case_e.rst
@@ -15,7 +15,7 @@ This example shows how to run a realistic testing experiment (based on https://d
After the calculation is complete, the results will be summarized in ``results/report.md``.
-.. contents::
+.. contents:: Contents
:local:
diff --git a/docs/examples/case_f.rst b/docs/examples/case_f.rst
index 858d9ad..a2856a4 100644
--- a/docs/examples/case_f.rst
+++ b/docs/examples/case_f.rst
@@ -15,7 +15,7 @@ This example shows how set up an experiment with a **time-dependent** model, who
After the calculation is complete, the results will be summarized in ``results/report.md``.
-.. contents::
+.. contents:: Contents
:local:
@@ -47,8 +47,7 @@ The source files can be found in the ``examples/case_e`` folder or in `GitHub <
Model
-----
-The time-dependency of a model is manifested here by the provision of different forecasts, i.e., statistical descriptions of seismicity, for different time-windows. In this example, the forecasts were created from an external model https://github.com/lmizrahi/etas (https://doi.org/10.1785/0220200231
-), with which the experiment has no interface. This means that we only the forecast files are required. We leave the handling of a model source code for subsequent examples.
+The time-dependency of a model is manifested here by the provision of different forecasts, i.e., statistical descriptions of seismicity, for different time-windows. In this example, the forecasts were created from an external model https://github.com/lmizrahi/etas (`doi:10.1785/0220200231 `_), with which the experiment has no interface. This means that we use **only the forecast files** and no source code. We leave the handling of a model source code for subsequent examples.
@@ -59,7 +58,7 @@ Configuration
Time
~~~~
- The configuration is identical to time-independent models with multiple time-windows (e.g., case C) with the exception that a ``horizon`` could be defined instead of ``intervals``, which is the forecast time-window length. The experiment's class should now be explicited as ``exp_class: td``.
+ The configuration is analogous to time-independent models with multiple time-windows (e.g., case C) with the exception that a ``horizon`` could be defined instead of ``intervals``, which is the forecast time-window length. The experiment's class should now be explicited as ``exp_class: td``.
.. literalinclude:: ../../examples/case_f/config.yml
:caption: examples/case_f/config.yml
diff --git a/docs/examples/case_g.rst b/docs/examples/case_g.rst
index f901df6..448e7ce 100644
--- a/docs/examples/case_g.rst
+++ b/docs/examples/case_g.rst
@@ -1,10 +1,9 @@
+.. _example_g:
+
G - Time-Dependent, Catalog-Based Model (from Source Code)
==========================================================
-.. currentmodule:: floatcsep
-
-.. contents::
- :local:
+Here, we set up a time-dependent model from its **source code** for an experiment.
.. admonition:: **TL; DR**
@@ -17,67 +16,95 @@ G - Time-Dependent, Catalog-Based Model (from Source Code)
After the calculation is complete, the results will be summarized in ``results/report.md``.
+.. currentmodule:: floatcsep
+
+.. contents:: Contents
+ :local:
+
+
+
Experiment Components
---------------------
-This example shows how a time-dependent model should be set up for a time-dependent experiment
+The example folder contains also, along with the already known components (configurations, catalog), a sub-folder for the **source code** of the model ``pymock``. The components of the experiment (and model) are:
::
case_g
- └── pymock
- ├── input
- ├── args.txt (model arguments)
- └── catalog.csv (dynamically allocated catalog)
- ├── pymock (model source code)
- └── forecasts (where forecasts will be stored)
+ └── pymock (Model's source code)
+ ├── input (input interface to floatcsep)
+ ├── args.txt (model arguments)
+ └── catalog.csv (dynamically allocated catalog)
+ ├── pymock
+ ├── libs.py (helper functions)
+ └── main.py (main routines)
+ └── forecasts (output interface to floatcsep)
+ ... (forecasts should be stored here when the model is run)
+ ├── run.py (One of the possibilities to run the model)
+ ├── pyproject.toml (Build instructions)
+ ├── setup.cfg (Build instructions)
+ ├── setup.py (Build instructions)
+ ├── requirements.txt(Build instructions)
+ ├── Dockerfile (Build instructions)
+ └── README.md (Information)
+
├── catalog.csv
├── config.yml
├── models.yml
+ ├── custom_plot_script.py
└── tests.yml
* The model to be evaluated (``pymock``) is a source code that generates forecasts for multiple time windows.
-* The testing catalog ``catalog.csv`` works also as the input catalog, by being filtered until the starting test_date and allocated in `pymock/input` dynamically (before the model is run)
+* The testing catalog ``catalog.csv`` works also as the input catalog, by being filtered until the testing ``start_date`` and allocated in `pymock/input` dynamically (before each time the model is run)
+.. _Model:
Model
-----
-The experiment's complexity increases from time-independent to dependent, since we now need a **Model** (source code) that is able to generate forecast that changes every window. The model should be conceptualized as a **black-box**, whose only interface/interaction with the ``floatcsep`` system is to receive an input (i.e., input catalog and arguments) and generates an output (the forecasts).
+The experiment's complexity increases from time-independent to dependent mostly because we now need a **Model** (source code) to generate forecasts that changes for every time-window. The model main components are:
+
* **Input**: The input consists in input **data** and **arguments**.
- 1. The **input data** is, at the least, a catalog filtered until the forecast beginning, which is automatically allocated by ``floatcsep`` in the `{model}/input` prior to each model's run. It is stored inside the model in ``csep.ascii`` format for simplicity's sake (see :doc:`pycsep:concepts/catalogs`).
+ 1. The **input data** is, at the very least, a catalog filtered until the forecast beginning. The catalog will be automatically allocated by ``floatcsep`` prior to each model's run (e.g., a single forecast run) in the `{model}/input` folder. It is stored in the ``csep.ascii`` format for simplicity's sake (see :doc:`pycsep:concepts/catalogs`).
.. literalinclude:: ../../examples/case_g/catalog.csv
+ :caption: examples/case_g/catalog.csv
:lines: 1-2
- 2. The **input arguments** controls how the model's source code works. The minimum arguments to run a model (which should be modified dynamically during an experiment) are the forecast ``start_date`` and ``end_date``. The experiment will access `{model}/input/args.txt` and change the values of ``start_date = {datetime}`` and ``end_date = {datetime}`` before the model is run. Additional arguments can be set by convenience, such as ``catalog`` (the input catalog name), ``n_sims`` (number of synthetic catalogs) and random ``seed`` for reproducibility.
+ 2. The **input arguments** controls how the model's source code works. The minimum arguments to run a model are the forecast ``start_date`` and ``end_date``, which will be modified dynamically during an experiment with multiple time-windows. The experiment system will access `{model}/input/args.txt` and change the values of ``start_date = {datetime}`` and ``end_date = {datetime}`` before the model is run. Additional arguments can be set by convenience, such as (not limited to) ``catalog`` (the input catalog name), ``n_sims`` (number of synthetic catalogs) and random ``seed`` for reproducibility.
-* **Output**: The model's output are the synthetic catalogs, which should be allocated in `{model}/forecasts/{filename}.csv`. The format is identically to ``csep_ascii``, but unlike in an input catalog, the ``catalog_id`` column should be modified for each synthetic catalog starting from 0. The file name follows the convention `{model_name}_{start}_{end}.csv`, where ``start`` and ``end`` folowws the `%Y-%m-%dT%H:%M:%S.%f` - ISO861 FORMAT
+* **Output**: The model's output are the synthetic catalogs, which should be allocated in `{model}/forecasts/{filename}.csv` by the source code after each rone. The format is identically to ``csep_ascii``, but unlike in an input catalog, the ``catalog_id`` column should be modified for each synthetic catalog starting from 0. The file name follows the convention `{model_name}_{start}_{end}.csv`, where ``start`` and ``end`` folows the `%Y-%m-%dT%H:%M:%S.%f` - ISO861 FORMAT
* **Model build**: Inside the model source code, there are multiple options to build it. A standard python ``setup.cfg`` is given, which can be built inside a python ``venv`` or ``conda`` managers. This is created and built automatically by ``floatCSEP``, as long as the the model build instructions are correctly set up.
-* **Model run**: The model should be run with a simple command to which only ``arguments`` should be passed. For this example, is
+* **Model run**: The model should be run with a simple command, e.g. **entrypoint**, to which only ``arguments`` could be passed if desired. The ``pymock`` model contains multiple example of entrypoints, but the modeler should use only one for clarity.
+
+ 1. A `python` call with arguments
.. code-block:: console
- $ python pymock/run.py input/args.txt
+ $ python run.py input/args.txt
- or using a binary (see ``pymock/setup.cfg:entry_point``)
+ 2. Using a binary entrypoint with arguments (for instance, defined in the python build instructions: ``pymock/setup.cfg:entry_point``)
.. code-block:: console
$ pymock input/args.txt
- It can also be run simply by
+ 3. A single binary entrypoint without arguments .
.. code-block:: console
$ pymock
- as long as it internally reads the ``input/args.txt`` and ``input/catalog.csv`` files.
+ This means that the source code should internally read the input data and arguments, ``input/catalog.csv`` and ``input/args.txt`` files respectively.
+
+.. important::
+
+ The model should be conceptualized as a **black-box**, whose only interface/interaction with the ``floatcsep`` system is to receive an input (i.e., input catalog and arguments) and generates an output (the forecasts).
Configuration
@@ -90,13 +117,14 @@ Time
The configuration is identical to time-independent models, with the exception that now a ``horizon`` can be defined instead of ``intervals``, which is the forecast time-window length. The experiment's class should now be explicited as ``exp_class: td``
.. literalinclude:: ../../examples/case_g/config.yml
- :language: yaml
- :lines: 3-7
+ :caption: examples/case_g/config.yml
+ :language: yaml
+ :lines: 3-7
Catalog
~~~~~~~
- The catalog was obtained ``previous to the experiment`` using ``query_bsi``, but it was filtered from 2006 onwards, so it has enough data for the model calibration.
+ The catalog was obtained `previous to the experiment` using ``query_bsi``, but it was filtered from 2006 onwards, so it has enough data for the model calibration.
Models
~~~~~~
@@ -104,14 +132,22 @@ Models
Additional arguments should be passed to time-independent models.
.. literalinclude:: ../../examples/case_g/models.yml
- :language: yaml
- :lines: 3-7
+ :caption: examples/case_g/models.yml
+ :language: yaml
+ :lines: 1-7
- Now ``path`` points to the folder where the source is installed. Input and forecasts should be in ``{path}/input`` and ``{path}/forecasts``. The ``func`` option is the shell command with which the model is run (the shell will be run from the model's directory). Note that ``python run.py`` can be changed to ``pymock`` (see entry_point in ``pymock/setup.cfg``). In practice, the model will be run as
+ 1. Now ``path`` points to the folder where the source is installed. Therefore, the input and the forecasts should be allocated ``{path}/input`` and ``{path}/forecasts``, respectively.
+ 2. The ``func`` option is the shell command with which the model is run. As seen in the `Model`_ section, this could be either ``pymock``, ``pymock input/args.txt`` or ``python run.py input/args``. We use the simplest option ``pymock``, but you are welcome to try different entrypoints.
- .. code-block:: console
+ .. note::
+ The ``func`` command will be run from the model's directory and a model containerization (e.g., ``Dockerfile``, ``conda``).
+
+ 3. The ``func_kwargs`` are extra arguments that will annotated to the ``input/args.txt`` file every time the model is run, or will be passed as extra arguments to the ``func`` call (Note that the two options are identical). This is useful to define sub-classes of models (or flavours) that uses the same source code, but a different instantiation.
+ 4. The ``build`` option defines the style of container within which the model will be placed. Currently in **floatCSEP**, only the python module ``venv``, the package manager ``conda`` and the containerization manager ``Docker`` are currently supported.
+
+ .. important::
+ For these examples, we use ``venv`` sub-environments, but we recommend ``Docker`` to set up real experiments.
- $ python {path}/run.py {path}/input/args.txt
Tests
~~~~~
@@ -120,7 +156,8 @@ Tests
.. literalinclude:: ../../examples/case_g/tests.yml
- :language: yaml
+ :caption: examples/case_g/tests.yml
+ :language: yaml
.. note::
It is possible to assign two plotting functions to a test, whose ``plot_args`` and ``plot_kwargs`` can be placed indented beneath
@@ -132,8 +169,9 @@ Custom Post-Process
Additional to the default :func:`~floatcsep.postprocess.plot_handler.plot_results`, :func:`~floatcsep.postprocess.plot_handler.plot_catalogs`, :func:`~floatcsep.postprocess.plot_handler.plot_forecasts` functions, a custom plotting function(s) can be set within the ``postprocess`` configuration
.. literalinclude:: ../../examples/case_g/config.yml
- :language: yaml
- :lines: 22-23
+ :caption: examples/case_g/config.yml
+ :language: yaml
+ :lines: 22-23
This option provides `hook` for a python script and a function within as:
@@ -144,10 +182,13 @@ Custom Post-Process
The requirements are that the script to be located within the same directory as the configuration file, whereas the function must receive a :class:`floatcsep.experiment.Experiment` as argument
.. literalinclude:: ../../examples/case_g/custom_plot_script.py
- :language: yaml
- :lines: 6-9
+ :caption: examples/case_g/custom_plot_script.py
+ :language: python
+ :lines: 6-13
+
+
- In this way, the plot function use all the :class:`~floatcsep.experiment.Experiment` attributes/methods to access catalogs, forecasts and test results. The script ``examples/case_g/custom_plot_script.py`` can also be viewed directly on `GitHub `_, where it is exemplified how to access the experiment artifacts.
+ In this way, the plot function can use all the :class:`~floatcsep.experiment.Experiment` attributes/methods to access catalogs, forecasts and test results. The script ``examples/case_g/custom_plot_script.py`` can also be viewed directly on `GitHub `_, where it is exemplified how to access the experiment data in runtime.
Running the experiment
diff --git a/docs/examples/case_h.rst b/docs/examples/case_h.rst
index d0a5134..4628c3b 100644
--- a/docs/examples/case_h.rst
+++ b/docs/examples/case_h.rst
@@ -1,11 +1,7 @@
H - Multiple Catalog-Based Models
=================================
-.. currentmodule:: floatcsep
-
-.. contents::
- :local:
- :depth: 1
+Here, we run an experiment that access, containerize and execute multiple **time-dependent models**, and then proceeds to evaluate the forecasts once they are created.
.. admonition:: **TL; DR**
@@ -17,70 +13,136 @@ H - Multiple Catalog-Based Models
After the calculation is complete, the results will be summarized in ``results/report.md``.
+.. currentmodule:: floatcsep
+
+.. contents:: Contents
+ :local:
+
+
Experiment Components
---------------------
-This example shows how to run an experiment that access, downloads, containerize and execute multiple time-dependent models. The experiment input files are:
+The experiment input files are:
::
case_h
├── catalog.csv
├── config.yml
- ├── models.yml
- └── tests.yml
+ ├── tests.yml
+ └── models.yml
-* The ``models.yml`` contains the instructions to clone and build the source codes from software repositories (e.g., gitlab, Github)
+* The ``models.yml`` contains the instructions to clone and build the source codes from software repositories (e.g., gitlab, Github), and how to interface them with **floatCSEP**. Once downloaded and built, the experiment structure should look like:
+::
+
+ case_h
+ ├── models
+ ├── etas
+ └── ... (ETAS source code)
+ ├── pymock_poisson
+ └── ... (Poisson-PyMock source code)
+ └── pymock_nb
+ └── ... (Negative-Binomial-PyMock source code)
+ ├── catalog.csv
+ ├── config.yml
+ ├── tests.yml
+ ├── custom_report.py
+ └── models.yml
+
+Configuration
+-------------
Models
-------
+~~~~~~
-As in Example G, the complexity increases because each **Model** requires to build and execute a source code to generate forecast for multilpe time-windows. The instructions for each model are located within ``models.yml``. The URL and specific version (e.g., commit hash, tag, release) are specified as:
+As in :ref:`Tutorial G`, each **Model** requires to build and execute a source code to generate forecasts. The instructions for each model are located within ``models.yml``, which we further explain here:
+.. note::
+ The ``models.yml`` will define how to interface **floatCSEP** to each Model, implying that a Model should be developed, or adapted to ensure the interface requirements specified below.
+
+1. The repository URL of each model and their specific versions (e.g., commit hash, tag, release) are specified as:
.. literalinclude:: ../../examples/case_h/models.yml
- :lines: 1-3
+ :caption: examples/case_h/models.yml
+ :language: yaml
+ :lines: 1-3, 11-13, 21-23
-The model source code requires to be synchronized with the experiment interface, in such a way that is able to read the inputs (catalog and argument file) from an ``{model_path}/input`` folder and store the forecast outputs from a folder ``{model_path}/forecasts``.
+2. A ``path`` needs to be indicated for each model, to both download the repository contents therein and from where the source code will be executed.
.. literalinclude:: ../../examples/case_h/models.yml
- :lines: 4-6
+ :caption: examples/case_h/models.yml
+ :language: yaml
+ :lines: 1-4
+ :emphasize-lines: 4
+ :lineno-match:
+
+ .. important::
+ The implies that the inputs (catalog and argument file) should be read by the model from a ``{path}/input`` folder, and the forecast outputs stored into a folder ``{path}/forecasts``.
-The ``func`` parameter indicates how the model should be invoked from a terminal (e.g, a python virtual environment, docker container, etc.). The type of container is set with the parameter ``build`` (``floatcsep`` supports ``conda``, ``venv`` and ``Dockerfile``). Note that we specify to ``floatcsep`` which arguments file will be read from the module. In this case, the ETAS model uses ``args.json`` file (``floatcsep`` suports ``.json`` and ``.txt`` files). ``floatcsep`` will dynamically modify the ``start_time`` and ``end_time`` for each time window run, and statically (i.e., for all time-windows) allocate the parameters set as:
+2. There is some flexibility to interface **floatCSEP** with a model. For instance, a different `filepath` can be set for the argument file:
.. literalinclude:: ../../examples/case_h/models.yml
- :lines: 7-9
+ :caption: examples/case_h/models.yml
+ :language: yaml
+ :lines: 5
+ :lineno-match:
-Same as Example G, the repositories should follow the following:
+ .. note::
+ **floatCSEP** can read `.txt`, `.json` and `.yml` format-styled argument files. In all cases, the minimum required arguments, should be formatted as:
+ .. code-block:: console
-* **Input**: The input consists in input **data** and **arguments**.
+ #.txt
+ start_date = {DATESTRING}
+ end_date = {DATESTRING}
- 1. The **input data** is, at the least, a catalog filtered until the forecast beginning, which is automatically allocated by ``floatcsep`` in the `{model}/input` prior to each model's run. It is stored inside the model in ``csep.ascii`` format for simplicity's sake (see :doc:`pycsep:concepts/catalogs`).
+ .. code-block:: yaml
- .. literalinclude:: ../../examples/case_h/catalog.csv
- :lines: 1-2
+ #.yml
+ start_date: {DATESTRING}
+ end_date: {DATESTRING}
- 2. The **input arguments** controls how the model's source code works. The minimum arguments to run a model (which should be modified dynamically during an experiment) are the forecast ``start_date`` and ``end_date``. The experiment will access `{model}/input/args.json` and change the values of ``"start_date": "{datetime}"`` and ``"end_date": "{datetime}"`` before the model is run. Additional arguments can be set by convenience, such as ``n_sims`` (number of synthetic catalogs), ``m_c`` for the cutoff magnitude and a random ``seed`` for reproducibility.
+ .. code-block:: json
-* **Output**: The model's output are the synthetic catalogs, which should be allocated in `{model}/forecasts/{filename}.csv`. The format is identically to ``csep_ascii``, but unlike in an input catalog, the ``catalog_id`` column should be modified for each synthetic catalog starting from 0. The file name follows the convention `{model_name}_{start}_{end}.csv`, where ``start`` and ``end`` folowws the `%Y-%m-%dT%H:%M:%S.%f` - ISO861 FORMAT
+ //.json
+ "start_date": "{DATESTRING}",
+ "end_date": "{DATESTRING}"
-* **Model build**: Inside the model source code, there are multiple options to build it. A standard python ``setup.cfg`` is given, which can be built inside a python ``venv`` or ``conda`` managers. This is created and built automatically by ``floatCSEP``, as long as the the model build instructions are correctly set up.
+ **floatcsep** will dynamically modify the ``start_date`` and ``start_date`` for each time window run, and any extra arguments will just be added for all time-windows.
-* **Model run**: The model should be run with a simple command to which only ``arguments`` should be passed. For this example, is
+4. The ``func`` entry indicates how the models are invoked from a shell terminal.
- .. code-block:: console
+ .. literalinclude:: ../../examples/case_h/models.yml
+ :caption: examples/case_h/models.yml
+ :language: yaml
+ :lines: 1,6,11,15,21,25
- $ etas-run
+ .. important::
+ Please refer to :ref:`Tutorial G` for example of how to set up ``func`` for the model and interface it to **floatCSEP**.
+5. A prefix for the resulting forecast filepaths can be specified beforehand for each model. Without specifying this, the default is ``{model_name}`` (e.g., `etas`, `Poisson Mock`, `Negbinom Mock`).
- as long as it internally reads the ``input/args.json`` and ``input/catalog.csv`` files.
+ .. literalinclude:: ../../examples/case_h/models.yml
+ :caption: examples/case_h/models.yml
+ :language: yaml
+ :lines: 21, 26
+ The experiment will read the forecasts as:
-Configuration
--------------
+ .. code-block::
+
+ {model_path}/{forecasts}/{prefix}_{start}_{end}.csv
+
+ where ``start`` and ``end`` follow either the ``%Y-%m-%dT%H:%M:%S.%f`` - ISO861 FORMAT, or the short date version ``%Y-%m-%d`` if the windows are set by midnight.
+
+6. Additional function arguments can be passed to the model with the entry ``func_kwargs``. We perhaps noted that both Poisson Mock and Negbinom Mock use the same source code. With ``func_kwargs`` a different subclass can be defined for the same source code (in this case, a Negative-Binomial number distribution instead of Poisson).
+
+ .. literalinclude:: ../../examples/case_h/models.yml
+ :caption: examples/case_h/models.yml
+ :language: yaml
+ :lines: 11,17-20,21,27-31
Time
@@ -89,13 +151,14 @@ Time
The configuration is identical to time-independent models, with the exception that now a ``horizon`` can be defined instead of ``intervals``, which is the forecast time-window length. The experiment's class should now be explicited as ``exp_class: td``
.. literalinclude:: ../../examples/case_h/config.yml
- :language: yaml
- :lines: 3-7
+ :caption: examples/case_h/config.yml
+ :language: yaml
+ :lines: 3-7
Catalog
~~~~~~~
- The catalog was obtained ``previous to the experiment`` using ``query_bsi``, but it was filtered from 2006 onwards, so it has enough data for the model calibration.
+ The catalog was obtained `previous to the experiment` using ``query_bsi``, but it was filtered from 2006 onwards, so it has enough data for the model calibration.
Tests
@@ -105,7 +168,8 @@ Tests
.. literalinclude:: ../../examples/case_h/tests.yml
- :language: yaml
+ :caption: examples/case_h/tests.yml
+ :language: yaml
.. note::
It is possible to assign two plotting functions to a test, whose ``plot_args`` and ``plot_kwargs`` can be placed indented beneath
@@ -117,8 +181,9 @@ Custom Post-Process
A custom reporting function can be set within the ``postprocess`` configuration to replace the :func:`~floatcsep.postprocess.reporting.generate_report`:
.. literalinclude:: ../../examples/case_h/config.yml
- :language: yaml
- :lines: 22-23
+ :caption: examples/case_h/config.yml
+ :language: yaml
+ :lines: 22-23
This option provides `hook` for a python script and a function within as:
diff --git a/docs/index.rst b/docs/index.rst
index 40a4956..dba327f 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -24,6 +24,12 @@ Preliminary documentation.
examples/case_g.rst
examples/case_h.rst
+.. toctree::
+ :maxdepth: 0
+ :caption: Help & Reference
+
+ reference/api_reference
+
.. toctree::
:maxdepth: 2
:caption: Defining an Experiment
@@ -41,11 +47,6 @@ Preliminary documentation.
deployment/intro.rst
-.. toctree::
- :maxdepth: 0
- :caption: Help & Reference
-
- reference/api_reference
diff --git a/examples/case_h/models.yml b/examples/case_h/models.yml
index fb3a607..dad36f1 100644
--- a/examples/case_h/models.yml
+++ b/examples/case_h/models.yml
@@ -23,10 +23,10 @@
repo_hash: v0.1
path: models/pymock_nb
func: pymock
+ prefix: pymock
func_kwargs:
n_sims: 100
mag_min: 3.5
- apply_mc_to_lambda: True
- distribution: negbinom
seed: 23
- prefix: pymock
+ distribution: negbinom
+ apply_mc_to_lambda: True
From 263269afc871a30a310925c8b3560e761df1e307 Mon Sep 17 00:00:00 2001
From: pciturri
Date: Mon, 23 Sep 2024 17:08:29 +0200
Subject: [PATCH 05/19] docs: Updated frontpage
---
docs/examples/case_a.rst | 6 ++-
docs/examples/case_b.rst | 2 +
docs/examples/case_c.rst | 2 +
docs/examples/case_d.rst | 2 +
docs/examples/case_e.rst | 2 +
docs/examples/case_f.rst | 6 ++-
docs/examples/case_g.rst | 4 +-
docs/examples/case_h.rst | 6 ++-
docs/index.rst | 77 ++++++++++++++++++++++++++++++++++---
docs/intro/installation.rst | 9 +++--
10 files changed, 99 insertions(+), 17 deletions(-)
diff --git a/docs/examples/case_a.rst b/docs/examples/case_a.rst
index f8d218c..4fe7f9a 100644
--- a/docs/examples/case_a.rst
+++ b/docs/examples/case_a.rst
@@ -1,5 +1,7 @@
-A - Simple Model and Catalog
-============================
+.. _example_a:
+
+A - Testing a Simple Model
+==========================
The following example shows the definition of a testing experiment of a single **time-independent** forecast against a catalog.
diff --git a/docs/examples/case_b.rst b/docs/examples/case_b.rst
index 2a5897f..d98dc42 100644
--- a/docs/examples/case_b.rst
+++ b/docs/examples/case_b.rst
@@ -1,3 +1,5 @@
+.. _example_b:
+
B - Multiple Models and Tests
=============================
diff --git a/docs/examples/case_c.rst b/docs/examples/case_c.rst
index 5410ae0..2236cf3 100644
--- a/docs/examples/case_c.rst
+++ b/docs/examples/case_c.rst
@@ -1,3 +1,5 @@
+.. _example_c:
+
C - Multiple Time Windows
=========================
diff --git a/docs/examples/case_d.rst b/docs/examples/case_d.rst
index 90ddff3..03c9611 100644
--- a/docs/examples/case_d.rst
+++ b/docs/examples/case_d.rst
@@ -1,3 +1,5 @@
+.. _example_d:
+
D - Catalog Queries and Model Repositories
==========================================
diff --git a/docs/examples/case_e.rst b/docs/examples/case_e.rst
index dda7f5f..c86d015 100644
--- a/docs/examples/case_e.rst
+++ b/docs/examples/case_e.rst
@@ -1,3 +1,5 @@
+.. _example_e:
+
E - A Realistic Time-independent Experiment
===========================================
diff --git a/docs/examples/case_f.rst b/docs/examples/case_f.rst
index a2856a4..c31dc24 100644
--- a/docs/examples/case_f.rst
+++ b/docs/examples/case_f.rst
@@ -1,5 +1,7 @@
-F - Time-Dependent Catalog-Based Model (from existing files)
-=============================================================
+.. _example_f:
+
+F - Testing Catalog-Based Forecasts
+===================================
This example shows how set up an experiment with a **time-dependent** model, whose forecast files already exist.
diff --git a/docs/examples/case_g.rst b/docs/examples/case_g.rst
index 448e7ce..d83cb03 100644
--- a/docs/examples/case_g.rst
+++ b/docs/examples/case_g.rst
@@ -1,7 +1,7 @@
.. _example_g:
-G - Time-Dependent, Catalog-Based Model (from Source Code)
-==========================================================
+G - A Time-Dependent Experiment
+===============================
Here, we set up a time-dependent model from its **source code** for an experiment.
diff --git a/docs/examples/case_h.rst b/docs/examples/case_h.rst
index 4628c3b..58f71d6 100644
--- a/docs/examples/case_h.rst
+++ b/docs/examples/case_h.rst
@@ -1,5 +1,7 @@
-H - Multiple Catalog-Based Models
-=================================
+.. _example_h:
+
+H - Multiple Time-Dependent Models
+==================================
Here, we run an experiment that access, containerize and execute multiple **time-dependent models**, and then proceeds to evaluate the forecasts once they are created.
diff --git a/docs/index.rst b/docs/index.rst
index dba327f..b9fe17f 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -1,9 +1,73 @@
+===============================
floatCSEP: Floating Experiments
===============================
-Preliminary documentation.
+*Testing earthquake forecasts made simple.*
+
+.. image:: https://img.shields.io/pypi/v/floatcsep.svg
+ :target: https://pypi.org/project/floatcsep/
+ :alt: PyPI version
+
+.. image:: https://img.shields.io/conda/vn/conda-forge/floatcsep.svg
+ :target: https://anaconda.org/conda-forge/floatcsep
+ :alt: Conda Version
+
+.. image:: https://github.com/cseptesting/floatcsep/actions/workflows/build-test.yml/badge.svg
+ :target: https://github.com/cseptesting/floatcsep/actions
+ :alt: Build Status
+
+.. image:: https://img.shields.io/github/license/cseptesting/floatcsep.svg
+ :target: https://github.com/cseptesting/floatcsep/blob/main/LICENSE
+ :alt: License
+
+
+.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.7953816.svg
+ :target: https://doi.org/10.5281/zenodo.7953816
+ :alt: Zenodo
+
+
+.. |start| image:: https://img.icons8.com/office/40/rocket.png
+ :target: intro/installation.html
+ :height: 48px
+
+.. |learn1| image:: https://img.icons8.com/nolan/64/literature.png
+ :target: https://docs.cseptesting.org/getting_started/core_concepts.html
+ :height: 48px
+
+.. |learn2| image:: https://i.postimg.cc/wMTtqBGB/icons8-literature-64.png
+ :target: https://docs.cseptesting.org/getting_started/theory.html
+ :height: 48px
+
+.. |experiment| image:: https://img.icons8.com/pulsar-color/48/science-application.png
+ :height: 48px
+
+.. |api| image:: https://img.icons8.com/nolan/64/code--v2.png
+ :target: reference/api_reference.html
+ :height: 48px
+
+.. |tutorials| image:: https://img.icons8.com/nolan/64/checklist.png
+ :target: examples/case_a.html
+ :height: 48px
+
+Quickstart
+----------
+
++--------------------------------------------------+-------------------------------------+
+| |start| **Get Started** | |tutorials| **Tutorials** |
+| | |
+| |learn1| **Forecasting Concepts** | - :ref:`example_a` |
+| | - :ref:`example_b` |
+| |learn2| **Testing Theory** | - :ref:`example_c` |
+| | - :ref:`example_d` |
+| |experiment| **Floating Experiments** | - :ref:`example_e` |
+| | - :ref:`example_f` |
+| |api| **API Reference** | - :ref:`example_g` |
+| | - :ref:`example_h` |
++--------------------------------------------------+-------------------------------------+
+
.. toctree::
+ :hidden:
:maxdepth: 1
:caption: Get Started
@@ -12,6 +76,7 @@ Preliminary documentation.
.. toctree::
+ :hidden:
:maxdepth: 1
:caption: Tutorial Experiments
@@ -25,6 +90,7 @@ Preliminary documentation.
examples/case_h.rst
.. toctree::
+ :hidden:
:maxdepth: 0
:caption: Help & Reference
@@ -32,6 +98,7 @@ Preliminary documentation.
.. toctree::
:maxdepth: 2
+ :hidden:
:caption: Defining an Experiment
guide/config.rst
@@ -42,6 +109,7 @@ Preliminary documentation.
guide/postprocess_config.rst
.. toctree::
+ :hidden:
:maxdepth: 2
:caption: Deploying an Experiment
@@ -50,9 +118,8 @@ Preliminary documentation.
-Indices and tables
-==================
-* :ref:`genindex`
-* :ref:`modindex`
+Search
+======
+
* :ref:`search`
diff --git a/docs/intro/installation.rst b/docs/intro/installation.rst
index 24b6eda..4cdbeca 100644
--- a/docs/intro/installation.rst
+++ b/docs/intro/installation.rst
@@ -1,13 +1,14 @@
Installation
============
- .. note::
-
- This application uses ``python >= 3.8``
-
Installing the latest version
-----------------------------
+ .. important::
+
+ This application uses ``python >= 3.9``
+
+
Using ``conda``
~~~~~~~~~~~~~~~
From b973be14b72385bfd9c4325637d502037e2d61c4 Mon Sep 17 00:00:00 2001
From: pciturri
Date: Mon, 23 Sep 2024 18:18:13 +0200
Subject: [PATCH 06/19] docs: Updated docs front page with useful links.
---
docs/conf.py | 17 ++++
docs/examples/case_d.rst | 4 +-
docs/examples/case_e.rst | 4 +-
docs/examples/case_g.rst | 4 +-
docs/examples/case_h.rst | 4 +-
docs/guide/config.rst | 2 +-
docs/guide/model_config.rst | 2 +-
docs/guide/postprocess_config.rst | 2 +-
docs/guide/region_config.rst | 2 +-
docs/guide/tests_config.rst | 2 +-
docs/guide/time_config.rst | 2 +-
docs/index.rst | 97 +++++++++++++++----
...{concepts.rst => floating_experiments.rst} | 31 ------
docs/intro/forecasting_experiments.rst | 5 +
requirements_dev.txt | 1 +
setup.cfg | 1 +
16 files changed, 117 insertions(+), 63 deletions(-)
rename docs/intro/{concepts.rst => floating_experiments.rst} (70%)
create mode 100644 docs/intro/forecasting_experiments.rst
diff --git a/docs/conf.py b/docs/conf.py
index 79f8e6c..64f56d3 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -29,6 +29,7 @@
"sphinx.ext.viewcode",
"sphinx.ext.napoleon",
"sphinx.ext.intersphinx",
+ "sphinx_copybutton",
]
# language = 'en'
@@ -65,3 +66,19 @@
}
html_logo = "_static/floatcsep_logo.svg"
todo_include_todos = False
+
+copybutton_prompt_text = "$ " # Text to ignore when copying (for shell commands)
+copybutton_only_copy_prompt_lines = False
+
+
+rst_epilog = """
+.. raw:: html
+
+
+
+"""
\ No newline at end of file
diff --git a/docs/examples/case_d.rst b/docs/examples/case_d.rst
index 03c9611..fbeb505 100644
--- a/docs/examples/case_d.rst
+++ b/docs/examples/case_d.rst
@@ -1,7 +1,7 @@
.. _example_d:
-D - Catalog Queries and Model Repositories
-==========================================
+D - Catalog and Model Queries
+=============================
The following example shows an experiment whose forecasts are **retrieved from a repository** (Zenodo - https://zenodo.org) and the testing **catalog** from an authoritative source **web service** (namely the gCMT catalog from the International Seismological Centre - http://www.isc.ac.uk).
diff --git a/docs/examples/case_e.rst b/docs/examples/case_e.rst
index c86d015..208246a 100644
--- a/docs/examples/case_e.rst
+++ b/docs/examples/case_e.rst
@@ -1,7 +1,7 @@
.. _example_e:
-E - A Realistic Time-independent Experiment
-===========================================
+E - A Time-Independent Experiment
+=================================
This example shows how to run a realistic testing experiment (based on https://doi.org/10.4401/ag-4844) while summarizing the concepts from the previous examples.
diff --git a/docs/examples/case_g.rst b/docs/examples/case_g.rst
index d83cb03..9529e40 100644
--- a/docs/examples/case_g.rst
+++ b/docs/examples/case_g.rst
@@ -1,7 +1,7 @@
.. _example_g:
-G - A Time-Dependent Experiment
-===============================
+G - Testing a Time-Dependent Model
+==================================
Here, we set up a time-dependent model from its **source code** for an experiment.
diff --git a/docs/examples/case_h.rst b/docs/examples/case_h.rst
index 58f71d6..6f3592e 100644
--- a/docs/examples/case_h.rst
+++ b/docs/examples/case_h.rst
@@ -1,7 +1,7 @@
.. _example_h:
-H - Multiple Time-Dependent Models
-==================================
+H - A time-dependent experiment
+===============================
Here, we run an experiment that access, containerize and execute multiple **time-dependent models**, and then proceeds to evaluate the forecasts once they are created.
diff --git a/docs/guide/config.rst b/docs/guide/config.rst
index 86a7a6e..2a6bd75 100644
--- a/docs/guide/config.rst
+++ b/docs/guide/config.rst
@@ -1,4 +1,4 @@
-Experiment configuration
+Experiment Configuration
========================
In progress
\ No newline at end of file
diff --git a/docs/guide/model_config.rst b/docs/guide/model_config.rst
index 165f615..18e29db 100644
--- a/docs/guide/model_config.rst
+++ b/docs/guide/model_config.rst
@@ -1,4 +1,4 @@
-Models configuration
+Models Configuration
====================
TBI
\ No newline at end of file
diff --git a/docs/guide/postprocess_config.rst b/docs/guide/postprocess_config.rst
index 025b590..d682785 100644
--- a/docs/guide/postprocess_config.rst
+++ b/docs/guide/postprocess_config.rst
@@ -1,4 +1,4 @@
-Post-process options
+Post-process Options
====================
TBI
\ No newline at end of file
diff --git a/docs/guide/region_config.rst b/docs/guide/region_config.rst
index d9c8cd4..add2089 100644
--- a/docs/guide/region_config.rst
+++ b/docs/guide/region_config.rst
@@ -1,4 +1,4 @@
-Region definition
+Region Definition
=================
TBI
\ No newline at end of file
diff --git a/docs/guide/tests_config.rst b/docs/guide/tests_config.rst
index fbca1bc..c83c7ff 100644
--- a/docs/guide/tests_config.rst
+++ b/docs/guide/tests_config.rst
@@ -1,4 +1,4 @@
-Evaluations definition
+Evaluations Definition
======================
TBI
\ No newline at end of file
diff --git a/docs/guide/time_config.rst b/docs/guide/time_config.rst
index d0642a8..a96c689 100644
--- a/docs/guide/time_config.rst
+++ b/docs/guide/time_config.rst
@@ -1,4 +1,4 @@
-Temporal definition
+Temporal Definition
===================
TBI
\ No newline at end of file
diff --git a/docs/index.rst b/docs/index.rst
index b9fe17f..0982c6b 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -4,6 +4,18 @@ floatCSEP: Floating Experiments
*Testing earthquake forecasts made simple.*
+.. image:: https://img.shields.io/badge/GitHub-Repository-blue?logo=github
+ :target: https://github.com/cseptesting/floatcsep
+ :alt: GitHub Repository
+
+.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.7953816.svg
+ :target: https://doi.org/10.5281/zenodo.7953816
+ :alt: Zenodo
+
+.. image:: https://img.shields.io/github/license/cseptesting/floatcsep.svg
+ :target: https://github.com/cseptesting/floatcsep/blob/main/LICENSE
+ :alt: License
+
.. image:: https://img.shields.io/pypi/v/floatcsep.svg
:target: https://pypi.org/project/floatcsep/
:alt: PyPI version
@@ -12,18 +24,6 @@ floatCSEP: Floating Experiments
:target: https://anaconda.org/conda-forge/floatcsep
:alt: Conda Version
-.. image:: https://github.com/cseptesting/floatcsep/actions/workflows/build-test.yml/badge.svg
- :target: https://github.com/cseptesting/floatcsep/actions
- :alt: Build Status
-
-.. image:: https://img.shields.io/github/license/cseptesting/floatcsep.svg
- :target: https://github.com/cseptesting/floatcsep/blob/main/LICENSE
- :alt: License
-
-
-.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.7953816.svg
- :target: https://doi.org/10.5281/zenodo.7953816
- :alt: Zenodo
.. |start| image:: https://img.icons8.com/office/40/rocket.png
@@ -49,6 +49,7 @@ floatCSEP: Floating Experiments
:target: examples/case_a.html
:height: 48px
+
Quickstart
----------
@@ -65,6 +66,70 @@ Quickstart
| | - :ref:`example_h` |
+--------------------------------------------------+-------------------------------------+
+What is floatCSEP
+-----------------
+
+The `Collaboratory for the Study of Earthquake Predictability `_ (CSEP) has organized Earthquake Forecast Testing Experiments during the last decades and is now consolidating its research into open-software initiatives.
+
+**floatCSEP** is an easy-to-use software application that contains the workflow to deploy Earthquake Forecasting Experiments. It is based on the code python library **pyCSEP** (`Github `_), which itself contains the core routines to test earthquake forecasts.
+
+Goals
+-----
+
+ * Set up a testing experiment for your forecasts using authoritative data sources/benchmarks.
+ * Encapsulate the complete experiment's definition and rules in a couple of lines.
+ * Produce human-readable results and figures.
+ * Reproduce, reuse, and share forecasting experiments.
+
+Running
+-------
+
+Start using **floatCSEP** by `installing `_ the latest version and running the ``examples`` with simply:
+
+.. code-block::
+
+ $ floatcsep run config.yml
+
+Useful Links
+------------
+
++---------------------------------------------------------+-----------------------------------------+
+| .. image:: https://img.icons8.com/nolan/64/github.png | **GitHub Repository** |
+| :height: 48px | |
+| :target: https://github.com/cseptesting/floatcsep | |
++---------------------------------------------------------+-----------------------------------------+
+| .. image:: https://i.postimg.cc/HW2Pssx1/logo-csep.png | **CSEP Website** |
+| :height: 48px | |
+| :target: https://cseptesting.org | |
++---------------------------------------------------------+-----------------------------------------+
+| .. image:: https://img.icons8.com/nolan/64/github.png | **pyCSEP GitHub** |
+| :height: 48px | |
+| :target: https://github.com/sceccode/pycsep | |
++---------------------------------------------------------+-----------------------------------------+
+| .. image:: https://img.icons8.com/nolan/64/europe.png | **European Testing Center** |
+| :height: 48px | |
+| :target: http://eqstats.efehr.org | |
++---------------------------------------------------------+-----------------------------------------+
+
+
+
+
+Collaborators
+-------------
+
+ * Pablo Iturrieta, GFZ Potsdam (pciturri@gfz-potsdam.de)
+ * William Savran, University of Nevada, Reno
+ * Jose Bayona, University of Bristol
+ * Francesco Serafini, University of Edinburgh
+ * Khawaja Asim, GFZ Potsdam
+ * Fabio Silva, Southern California Earthquake Center
+ * Marcus Hermann, University of Naples ‘Frederico II’
+ * Max Werner, University of Bristol
+ * Danijel Schorlemmner, GFZ Potsdam
+ * Philip Maechling, Southern California Earthquake Center
+
+
+
.. toctree::
:hidden:
@@ -72,7 +137,8 @@ Quickstart
:caption: Get Started
intro/installation.rst
- intro/concepts.rst
+ intro/forecasting_experiments.rst
+ intro/floating_experiments.rst
.. toctree::
@@ -118,8 +184,3 @@ Quickstart
-
-Search
-======
-
-* :ref:`search`
diff --git a/docs/intro/concepts.rst b/docs/intro/floating_experiments.rst
similarity index 70%
rename from docs/intro/concepts.rst
rename to docs/intro/floating_experiments.rst
index dde7229..85fe246 100644
--- a/docs/intro/concepts.rst
+++ b/docs/intro/floating_experiments.rst
@@ -22,34 +22,3 @@ This is an application to deploy reproducible and prospective experiments of ear
* Produce human-readable results and figures.
* Reproduce, reuse, and share forecasting experiments.
-Collaborators
--------------
-
- * Pablo Iturrieta, GFZ Potsdam (pciturri@gfz-potsdam.de)
-
- * William Savran, UNR
-
- * Fabio Silva, Southern California Earthquake Center
-
- * Khawaja Asim, GFZ Potsdam
-
- * Jose Bayona, University of Bristol
-
- * Leila Mizrahi, ETH
-
- * Kirsty Bayliss, GEM
-
- * Francesco Serafini, University of Edinburgh
-
- * Marcus Hermann, University of Naples ‘Frederico II’
-
- * Max Werner, University of Bristol
-
- * Danijel Schorlemmner, GFZ Potsdam
-
- * Philip Maechling, Southern California Earthquake Center
-
-
-
-
-
diff --git a/docs/intro/forecasting_experiments.rst b/docs/intro/forecasting_experiments.rst
new file mode 100644
index 0000000..ac0817a
--- /dev/null
+++ b/docs/intro/forecasting_experiments.rst
@@ -0,0 +1,5 @@
+Forecasting Experiments
+=======================
+
+
+TBI
\ No newline at end of file
diff --git a/requirements_dev.txt b/requirements_dev.txt
index 9c8799a..012c6c9 100644
--- a/requirements_dev.txt
+++ b/requirements_dev.txt
@@ -28,6 +28,7 @@ sphinx
sphinx-autoapi
sphinx-gallery
sphinx-rtd-theme
+sphinx_copybutton
tables
tox
vcrpy==4.3.1
diff --git a/setup.cfg b/setup.cfg
index 3528ec5..657f79b 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -76,6 +76,7 @@ dev =
sphinx-autoapi
sphinx-gallery
sphinx-rtd-theme
+ sphinx_copybutton
tables
tox
vcrpy==4.3.1
From 6e299a7669e0a0ce355c2896f546a68fcccf79ba Mon Sep 17 00:00:00 2001
From: pciturri
Date: Wed, 25 Sep 2024 00:17:22 +0200
Subject: [PATCH 07/19] refac: Renamed examples folder to tutorials
---
README.md | 45 +++++++++-----
docs/index.rst | 36 ++++++------
docs/intro/forecasting_experiments.rst | 2 +-
docs/intro/installation.rst | 58 +++++++++++++------
docs/{examples => tutorials}/case_a.rst | 40 ++++++-------
docs/{examples => tutorials}/case_b.rst | 22 +++----
docs/{examples => tutorials}/case_c.rst | 14 ++---
docs/{examples => tutorials}/case_d.rst | 16 ++---
docs/{examples => tutorials}/case_e.rst | 30 +++++-----
docs/{examples => tutorials}/case_f.rst | 22 +++----
docs/{examples => tutorials}/case_g.rst | 34 +++++------
docs/{examples => tutorials}/case_h.rst | 50 ++++++++--------
pyproject.toml | 1 +
tests/artifacts/models/template/Dockerfile | 4 +-
tests/integration/test_model_interface.py | 4 +-
tests/qa/test_data.py | 4 +-
tests/unit/test_readers.py | 2 +-
{examples => tutorials}/case_a/best_model.dat | 0
{examples => tutorials}/case_a/case_a.py | 0
{examples => tutorials}/case_a/catalog.csep | 0
{examples => tutorials}/case_a/config.yml | 0
{examples => tutorials}/case_a/region.txt | 0
{examples => tutorials}/case_b/case_b.py | 0
{examples => tutorials}/case_b/catalog.json | 0
{examples => tutorials}/case_b/config.yml | 0
{examples => tutorials}/case_b/models.yml | 0
.../case_b/models/model_a.csv | 0
.../case_b/models/model_b.csv | 0
.../case_b/models/model_c.csv | 0
.../case_b/models/model_d.csv | 0
{examples => tutorials}/case_b/region.txt | 0
{examples => tutorials}/case_b/tests.yml | 0
{examples => tutorials}/case_c/case_c.py | 0
{examples => tutorials}/case_c/catalog.json | 0
{examples => tutorials}/case_c/config.yml | 0
{examples => tutorials}/case_c/models.yml | 0
.../case_c/models/model_a.csv | 0
.../case_c/models/model_b.csv | 0
.../case_c/models/model_c.csv | 0
.../case_c/models/model_d.csv | 0
{examples => tutorials}/case_c/region.txt | 0
{examples => tutorials}/case_c/tests.yml | 0
{examples => tutorials}/case_d/config.yml | 0
{examples => tutorials}/case_d/models.yml | 0
{examples => tutorials}/case_d/tests.yml | 0
{examples => tutorials}/case_e/case_e.py | 0
{examples => tutorials}/case_e/catalog.json | 0
{examples => tutorials}/case_e/config.yml | 0
{examples => tutorials}/case_e/models.yml | 0
...gulia-wiemer.ALM.italy.10yr.2010-01-01.xml | 0
.../meletti.MPS04.italy.10yr.2010-01-01.xml | 0
...har.TripleS-CPTI.italy.10yr.2010-01-01.xml | 0
{examples => tutorials}/case_e/tests.yml | 0
{examples => tutorials}/case_f/catalog.json | 0
{examples => tutorials}/case_f/config.yml | 0
.../forecasts/etas_2016-11-14_2016-11-15.csv | 0
.../forecasts/etas_2016-11-15_2016-11-16.csv | 0
.../forecasts/etas_2016-11-16_2016-11-17.csv | 0
.../forecasts/etas_2016-11-17_2016-11-18.csv | 0
.../forecasts/etas_2016-11-18_2016-11-19.csv | 0
.../forecasts/etas_2016-11-19_2016-11-20.csv | 0
.../forecasts/etas_2016-11-20_2016-11-21.csv | 0
{examples => tutorials}/case_f/models.yml | 0
{examples => tutorials}/case_f/tests.yml | 0
{examples => tutorials}/case_g/catalog.csv | 0
{examples => tutorials}/case_g/config.yml | 0
.../case_g/custom_plot_script.py | 0
{examples => tutorials}/case_g/models.yml | 0
.../case_g/pymock/Dockerfile | 0
.../case_g/pymock/README.md | 0
.../case_g/pymock/input/args.txt | 0
.../case_g/pymock/pymock/__init__.py | 0
.../case_g/pymock/pymock/libs.py | 0
.../case_g/pymock/pymock/main.py | 0
.../case_g/pymock/pyproject.toml | 0
.../case_g/pymock/requirements.txt | 0
{examples => tutorials}/case_g/pymock/run.py | 0
.../case_g/pymock/setup.cfg | 0
.../case_g/pymock/setup.py | 0
{examples => tutorials}/case_g/tests.yml | 0
{examples => tutorials}/case_h/catalog.csv | 0
{examples => tutorials}/case_h/config.yml | 0
.../case_h/custom_report.py | 0
{examples => tutorials}/case_h/models.yml | 0
{examples => tutorials}/case_h/tests.yml | 0
85 files changed, 211 insertions(+), 173 deletions(-)
rename docs/{examples => tutorials}/case_a.rst (81%)
rename docs/{examples => tutorials}/case_b.rst (78%)
rename docs/{examples => tutorials}/case_c.rst (84%)
rename docs/{examples => tutorials}/case_d.rst (84%)
rename docs/{examples => tutorials}/case_e.rst (74%)
rename docs/{examples => tutorials}/case_f.rst (82%)
rename docs/{examples => tutorials}/case_g.rst (87%)
rename docs/{examples => tutorials}/case_h.rst (77%)
rename {examples => tutorials}/case_a/best_model.dat (100%)
rename {examples => tutorials}/case_a/case_a.py (100%)
rename {examples => tutorials}/case_a/catalog.csep (100%)
rename {examples => tutorials}/case_a/config.yml (100%)
rename {examples => tutorials}/case_a/region.txt (100%)
rename {examples => tutorials}/case_b/case_b.py (100%)
rename {examples => tutorials}/case_b/catalog.json (100%)
rename {examples => tutorials}/case_b/config.yml (100%)
rename {examples => tutorials}/case_b/models.yml (100%)
rename {examples => tutorials}/case_b/models/model_a.csv (100%)
rename {examples => tutorials}/case_b/models/model_b.csv (100%)
rename {examples => tutorials}/case_b/models/model_c.csv (100%)
rename {examples => tutorials}/case_b/models/model_d.csv (100%)
rename {examples => tutorials}/case_b/region.txt (100%)
rename {examples => tutorials}/case_b/tests.yml (100%)
rename {examples => tutorials}/case_c/case_c.py (100%)
rename {examples => tutorials}/case_c/catalog.json (100%)
rename {examples => tutorials}/case_c/config.yml (100%)
rename {examples => tutorials}/case_c/models.yml (100%)
rename {examples => tutorials}/case_c/models/model_a.csv (100%)
rename {examples => tutorials}/case_c/models/model_b.csv (100%)
rename {examples => tutorials}/case_c/models/model_c.csv (100%)
rename {examples => tutorials}/case_c/models/model_d.csv (100%)
rename {examples => tutorials}/case_c/region.txt (100%)
rename {examples => tutorials}/case_c/tests.yml (100%)
rename {examples => tutorials}/case_d/config.yml (100%)
rename {examples => tutorials}/case_d/models.yml (100%)
rename {examples => tutorials}/case_d/tests.yml (100%)
rename {examples => tutorials}/case_e/case_e.py (100%)
rename {examples => tutorials}/case_e/catalog.json (100%)
rename {examples => tutorials}/case_e/config.yml (100%)
rename {examples => tutorials}/case_e/models.yml (100%)
rename {examples => tutorials}/case_e/models/gulia-wiemer.ALM.italy.10yr.2010-01-01.xml (100%)
rename {examples => tutorials}/case_e/models/meletti.MPS04.italy.10yr.2010-01-01.xml (100%)
rename {examples => tutorials}/case_e/models/zechar.TripleS-CPTI.italy.10yr.2010-01-01.xml (100%)
rename {examples => tutorials}/case_e/tests.yml (100%)
rename {examples => tutorials}/case_f/catalog.json (100%)
rename {examples => tutorials}/case_f/config.yml (100%)
rename {examples => tutorials}/case_f/etas/forecasts/etas_2016-11-14_2016-11-15.csv (100%)
rename {examples => tutorials}/case_f/etas/forecasts/etas_2016-11-15_2016-11-16.csv (100%)
rename {examples => tutorials}/case_f/etas/forecasts/etas_2016-11-16_2016-11-17.csv (100%)
rename {examples => tutorials}/case_f/etas/forecasts/etas_2016-11-17_2016-11-18.csv (100%)
rename {examples => tutorials}/case_f/etas/forecasts/etas_2016-11-18_2016-11-19.csv (100%)
rename {examples => tutorials}/case_f/etas/forecasts/etas_2016-11-19_2016-11-20.csv (100%)
rename {examples => tutorials}/case_f/etas/forecasts/etas_2016-11-20_2016-11-21.csv (100%)
rename {examples => tutorials}/case_f/models.yml (100%)
rename {examples => tutorials}/case_f/tests.yml (100%)
rename {examples => tutorials}/case_g/catalog.csv (100%)
rename {examples => tutorials}/case_g/config.yml (100%)
rename {examples => tutorials}/case_g/custom_plot_script.py (100%)
rename {examples => tutorials}/case_g/models.yml (100%)
rename {examples => tutorials}/case_g/pymock/Dockerfile (100%)
rename {examples => tutorials}/case_g/pymock/README.md (100%)
rename {examples => tutorials}/case_g/pymock/input/args.txt (100%)
rename {examples => tutorials}/case_g/pymock/pymock/__init__.py (100%)
rename {examples => tutorials}/case_g/pymock/pymock/libs.py (100%)
rename {examples => tutorials}/case_g/pymock/pymock/main.py (100%)
rename {examples => tutorials}/case_g/pymock/pyproject.toml (100%)
rename {examples => tutorials}/case_g/pymock/requirements.txt (100%)
rename {examples => tutorials}/case_g/pymock/run.py (100%)
rename {examples => tutorials}/case_g/pymock/setup.cfg (100%)
rename {examples => tutorials}/case_g/pymock/setup.py (100%)
rename {examples => tutorials}/case_g/tests.yml (100%)
rename {examples => tutorials}/case_h/catalog.csv (100%)
rename {examples => tutorials}/case_h/config.yml (100%)
rename {examples => tutorials}/case_h/custom_report.py (100%)
rename {examples => tutorials}/case_h/models.yml (100%)
rename {examples => tutorials}/case_h/tests.yml (100%)
diff --git a/README.md b/README.md
index bb7e30e..a9bdf35 100644
--- a/README.md
+++ b/README.md
@@ -20,9 +20,11 @@
-* Set up a testing **experiment** for your earthquake forecasts using authoritative data sources and benchmarks.
+* Set up a testing **experiment** for your earthquake forecasts using authoritative data sources
+ and benchmarks.
* **Encapsulate** the complete experiment's definition and rules in a couple of lines.
-* **Reproduce**, **reuse**, and **share** forecasting experiments from you, other researchers and institutions.
+* **Reproduce**, **reuse**, and **share** forecasting experiments from you, other researchers
+ and institutions.
# Table of Contents
@@ -34,12 +36,14 @@
* [Contributing](#contributing)
* [License](#license)
-
# Installing floatCSEP
-The core of `floatCSEP` is built around the `pyCSEP` package (https://github.com/sceccode/pycsep), which itself contains the core dependencies.
+The core of `floatCSEP` is built around the `pyCSEP`
+package (https://github.com/sceccode/pycsep), which itself contains the core dependencies.
-The simplest way to install `floatCSEP`, is by creating a `conda` environment (https://conda.io - checkout Anaconda or Miniconda) and install `pyCSEP` from `conda-forge`
+The simplest way to install `floatCSEP`, is by creating a `conda`
+environment (https://conda.io - checkout Anaconda or Miniconda) and install `pyCSEP`
+from `conda-forge`
```
conda env create -n $NAME
@@ -48,21 +52,29 @@ conda install -c conda-forge pycsep
```
Clone and install the floatCSEP source code using `pip`
+
```
git clone https://github.com/cseptesting/floatcsep
cd floatcsep
pip install .
```
-Please read the [Installation](https://floatcsep.readthedocs.io/en/latest/intro/installation.html) documentation for detailed instructions and additional installation methods.
+Please read
+the [Installation](https://floatcsep.readthedocs.io/en/latest/intro/installation.html)
+documentation for detailed instructions and additional installation methods.
# Run an Experiment
-Using the command terminal, navigate to an example experiment in `floatcsep/examples/` and type
+Using the command terminal, navigate to an example experiment in ``floatcsep/tutorials/`` and
+type
+
```
floatcsep run config.yml
```
-A runtime directory will be created in a `results` folder. The experiment results can be visualized in `results/report.md`. **Check out the experiment, models and tests definition in the examples**!
+
+A runtime directory will be created in a `results` folder. The experiment results can be
+visualized in `results/report.md`. **Check out the experiment, models and tests definition in
+the tutorials**!
# Important Links
@@ -71,7 +83,6 @@ A runtime directory will be created in a `results` folder. The experiment result
* `pyCSEP` [Github](https://github.com/sceccode/pycsep)
* `pyCSEP` [Documentation](https://docs.cseptesting.org/)
-
# Roadmap and Known Issues
* Add report customization
@@ -80,21 +91,27 @@ A runtime directory will be created in a `results` folder. The experiment result
* Implement spatial database and HDF5 experiment storage feature
* Set up task paralellization
-
# Contributing
-We encourage all types of contributions, from reporting bugs, suggesting enhancements, adding new features and more. Please refer to the [Contribution Guidelines](https://github.com/cseptesting/floatcsep/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/cseptesting/floatcsep/blob/main/CODE_OF_CONDUCT.md) for more information
+We encourage all types of contributions, from reporting bugs, suggesting enhancements, adding
+new features and more. Please refer to
+the [Contribution Guidelines](https://github.com/cseptesting/floatcsep/blob/main/CONTRIBUTING.md)
+and the [Code of Conduct](https://github.com/cseptesting/floatcsep/blob/main/CODE_OF_CONDUCT.md)
+for more information
# License
-The `floatCSEP` (as well as `pyCSEP`) software is distributed under the BSD 3-Clause open-source license. Please see the [license file](https://github.com/cseptesting/floatcsep/blob/main/LICENSE) for more information.
+The `floatCSEP` (as well as `pyCSEP`) software is distributed under the BSD 3-Clause open-source
+license. Please see
+the [license file](https://github.com/cseptesting/floatcsep/blob/main/LICENSE) for more
+information.
## Support
-|

|

|
-|:---|:---|
+|

|

|
+|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
|
The work in this repository has received funding from the European Union’s Horizon research and innovation programme under grant agreements No.s 101058518 and 821115 of the projects
GeoInquire and
RISE.
|
diff --git a/docs/index.rst b/docs/index.rst
index 0982c6b..0410bc1 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -46,7 +46,7 @@ floatCSEP: Floating Experiments
:height: 48px
.. |tutorials| image:: https://img.icons8.com/nolan/64/checklist.png
- :target: examples/case_a.html
+ :target: tutorials/case_a.html
:height: 48px
@@ -56,14 +56,14 @@ Quickstart
+--------------------------------------------------+-------------------------------------+
| |start| **Get Started** | |tutorials| **Tutorials** |
| | |
-| |learn1| **Forecasting Concepts** | - :ref:`example_a` |
-| | - :ref:`example_b` |
-| |learn2| **Testing Theory** | - :ref:`example_c` |
-| | - :ref:`example_d` |
-| |experiment| **Floating Experiments** | - :ref:`example_e` |
-| | - :ref:`example_f` |
-| |api| **API Reference** | - :ref:`example_g` |
-| | - :ref:`example_h` |
+| |learn1| **Forecasting Concepts** | - :ref:`case_a` |
+| | - :ref:`case_b` |
+| |learn2| **Testing Theory** | - :ref:`case_c` |
+| | - :ref:`case_d` |
+| |experiment| **Floating Experiments** | - :ref:`case_e` |
+| | - :ref:`case_f` |
+| |api| **API Reference** | - :ref:`case_g` |
+| | - :ref:`case_h` |
+--------------------------------------------------+-------------------------------------+
What is floatCSEP
@@ -84,7 +84,7 @@ Goals
Running
-------
-Start using **floatCSEP** by `installing `_ the latest version and running the ``examples`` with simply:
+Start using **floatCSEP** by `installing `_ the latest version and running the ``tutorials`` with simply:
.. code-block::
@@ -146,14 +146,14 @@ Collaborators
:maxdepth: 1
:caption: Tutorial Experiments
- examples/case_a.rst
- examples/case_b.rst
- examples/case_c.rst
- examples/case_d.rst
- examples/case_e.rst
- examples/case_f.rst
- examples/case_g.rst
- examples/case_h.rst
+ tutorials/case_a.rst
+ tutorials/case_b.rst
+ tutorials/case_c.rst
+ tutorials/case_d.rst
+ tutorials/case_e.rst
+ tutorials/case_f.rst
+ tutorials/case_g.rst
+ tutorials/case_h.rst
.. toctree::
:hidden:
diff --git a/docs/intro/forecasting_experiments.rst b/docs/intro/forecasting_experiments.rst
index ac0817a..1b9e6ac 100644
--- a/docs/intro/forecasting_experiments.rst
+++ b/docs/intro/forecasting_experiments.rst
@@ -2,4 +2,4 @@ Forecasting Experiments
=======================
-TBI
\ No newline at end of file
+TBW
\ No newline at end of file
diff --git a/docs/intro/installation.rst b/docs/intro/installation.rst
index 4cdbeca..5d53517 100644
--- a/docs/intro/installation.rst
+++ b/docs/intro/installation.rst
@@ -12,15 +12,12 @@ Installing the latest version
Using ``conda``
~~~~~~~~~~~~~~~
-The core of `floatCSEP` is built around the `pyCSEP` package (https://github.com/sceccode/pycsep), which itself contains the core dependencies.
+To install **floatCSEP**, first a ``conda`` manager should be installed (https://conda.io). Checkout `Anaconda`, `Miniconda` or `Miniforge` (recommended). Once installed, create an environment with:
-The simplest way to install `floatCSEP`, is by creating a `conda` environment (https://conda.io - checkout Anaconda or Miniconda) and install `pyCSEP` from `conda-forge`
+ .. code-block:: sh
- .. code-block:: console
-
- $ conda env create -n $NAME
- $ conda activate $NAME
- $ conda install -c conda-forge pycsep
+ $ conda env create -n csep_env
+ $ conda activate csep_env
Then, clone and install the floatCSEP source code using ``pip``
@@ -30,27 +27,50 @@ Then, clone and install the floatCSEP source code using ``pip``
cd floatcsep
pip install .
-Using ``apt`` and ``pip``
-~~~~~~~~~~~~~~~~~~~~~~~~~
-To install from ``pip``, we require to install the binary dependencies of ``pyCSEP`` (see `Installing pyCSEP `_}
+Using ``pip``
+~~~~~~~~~~~~~
-Then, install the ``pycsep`` latest version
+To install using the ``pip`` manager, we require to install the binary dependencies of **pyCSEP** (see `Installing pyCSEP `_}. The **floatcsep** latest version then can be installed as:
.. code-block::
- git clone https://github.com/SCECcode/pycsep
- cd pycsep
- python -m virtualenv venv
- source venv/bin/activate
- pip install -e .[all]
+ cd ..
+ git clone https://github.com/cseptesting/floatcsep
+ cd floatcsep
+ pip install .
+
+Installing the stable version
+-----------------------------
+
+
+Using ``conda``
+~~~~~~~~~~~~~~~
+
+To install **floatCSEP**, first a ``conda`` manager should be installed (https://conda.io). Checkout `Anaconda`, `Miniconda` or `Miniforge` (recommended). Once installed, create an environment with:
-and the ``floatcsep`` latest version
+ .. code-block:: sh
+
+ $ conda env create -n csep_env
+ $ conda activate csep_env
+
+Then, clone and install the floatCSEP source code using ``pip``
+
+ .. code-block:: console
+
+ git clone https://github.com/cseptesting/floatcsep
+ cd floatcsep
+ pip install .
+
+
+Using ``pip``
+~~~~~~~~~~~~~
+
+To install using the ``pip`` manager, we require to install the binary dependencies of **pyCSEP** (see `Installing pyCSEP `_}. The **floatcsep** latest version can then be installed as:
.. code-block::
cd ..
git clone https://github.com/cseptesting/floatcsep
cd floatcsep
- pip install .[all]
-
+ pip install .
\ No newline at end of file
diff --git a/docs/examples/case_a.rst b/docs/tutorials/case_a.rst
similarity index 81%
rename from docs/examples/case_a.rst
rename to docs/tutorials/case_a.rst
index 4fe7f9a..45dd200 100644
--- a/docs/examples/case_a.rst
+++ b/docs/tutorials/case_a.rst
@@ -1,4 +1,4 @@
-.. _example_a:
+.. _case_a:
A - Testing a Simple Model
==========================
@@ -9,7 +9,7 @@ The following example shows the definition of a testing experiment of a single *
.. admonition:: **TL; DR**
- In a terminal, navigate to ``floatcsep/examples/case_a`` and type:
+ In a terminal, navigate to ``floatcsep/tutorials/case_a`` and type:
.. code-block:: console
@@ -25,7 +25,7 @@ The following example shows the definition of a testing experiment of a single *
Experiment Components
---------------------
-The source code can be found in the ``examples/case_a`` folder or in `GitHub `_. The directory structure of the experiment is:
+The source code can be found in the ``tutorials/case_a`` folder or in `GitHub `_. The directory structure of the experiment is:
::
@@ -38,18 +38,18 @@ The source code can be found in the ``examples/case_a`` folder or in `GitHub `_. The input structure of the experiment is:
+The source code can be found in the ``tutorials/case_b`` folder or in `GitHub `_. The input structure of the experiment is:
::
@@ -53,14 +53,14 @@ Configuration
In this example, the time, region and catalog specifications are written in the ``config.yml`` file.
-.. literalinclude:: ../../examples/case_b/config.yml
- :caption: examples/case_b/config.yml
+.. literalinclude:: ../../tutorials/case_b/config.yml
+ :caption: tutorials/case_b/config.yml
:language: yaml
:lines: 3-15
whereas the models' and tests' configurations are referred to external files for better readability
-.. literalinclude:: ../../examples/case_b/config.yml
+.. literalinclude:: ../../tutorials/case_b/config.yml
:language: yaml
:lines: 17-18
@@ -69,17 +69,17 @@ Models
~~~~~~
The model configuration is now set in the ``models.yml`` file, where a list of model names specify their file paths.
- .. literalinclude:: ../../examples/case_b/models.yml
- :caption: examples/case_b/models.yml
+ .. literalinclude:: ../../tutorials/case_b/models.yml
+ :caption: tutorials/case_b/models.yml
:language: yaml
Evaluations
~~~~~~~~~~~
The evaluations are defined in the ``tests.yml`` file as a list of evaluation names, with their functions and plots (see :doc:`pycsep:concepts/evaluations`). In this example, we use the N-, M-, S- and CL-consistency tests, along with the comparison T-test.
- .. literalinclude:: ../../examples/case_b/tests.yml
+ .. literalinclude:: ../../tutorials/case_b/tests.yml
:language: yaml
- :caption: examples/case_b/tests.yml
+ :caption: tutorials/case_b/tests.yml
.. note::
Plotting keyword arguments can be set in the ``plot_kwargs`` option - see :func:`~csep.utils.plots.plot_poisson_consistency_test` and :func:`~csep.utils.plots.plot_comparison_test` -.
@@ -91,7 +91,7 @@ Running the experiment
----------------------
-The experiment can be run by simply navigating to the ``examples/case_b`` folder in the terminal an type.
+The experiment can be run by simply navigating to the ``tutorials/case_b`` folder in the terminal an type.
.. code-block:: console
diff --git a/docs/examples/case_c.rst b/docs/tutorials/case_c.rst
similarity index 84%
rename from docs/examples/case_c.rst
rename to docs/tutorials/case_c.rst
index 2236cf3..d6b74d2 100644
--- a/docs/examples/case_c.rst
+++ b/docs/tutorials/case_c.rst
@@ -1,4 +1,4 @@
-.. _example_c:
+.. _case_c:
C - Multiple Time Windows
=========================
@@ -9,7 +9,7 @@ The following example shows an experiment with **multiple time windows**.
.. admonition:: **TL; DR**
- In a terminal, navigate to ``floatcsep/examples/case_c`` and type:
+ In a terminal, navigate to ``floatcsep/tutorials/case_c`` and type:
.. code-block:: console
@@ -25,7 +25,7 @@ The following example shows an experiment with **multiple time windows**.
Experiment Components
---------------------
-The source code can be found in the ``examples/case_c`` folder or in `GitHub `_. The input structure of the experiment is:
+The source code can be found in the ``tutorials/case_c`` folder or in `GitHub `_. The input structure of the experiment is:
::
@@ -49,8 +49,8 @@ Time
The time configuration now sets a sequence of time intervals between the start and end dates.
- .. literalinclude:: ../../examples/case_c/config.yml
- :caption: examples/case_c/config.yml
+ .. literalinclude:: ../../tutorials/case_c/config.yml
+ :caption: tutorials/case_c/config.yml
:language: yaml
:lines: 3-7
@@ -66,9 +66,9 @@ Evaluations
~~~~~~~~~~~
The experiment's evaluations are defined in ``tests.yml``, which can now include temporal evaluations (see :func:`~floatcsep.utils.helpers.sequential_likelihood`, :func:`~floatcsep.utils.helpers.sequential_information_gain`, :func:`~floatcsep.utils.helpers.plot_sequential_likelihood`).
- .. literalinclude:: ../../examples/case_c/tests.yml
+ .. literalinclude:: ../../tutorials/case_c/tests.yml
:language: yaml
- :caption: examples/case_c/tests.yml
+ :caption: tutorials/case_c/tests.yml
.. note::
diff --git a/docs/examples/case_d.rst b/docs/tutorials/case_d.rst
similarity index 84%
rename from docs/examples/case_d.rst
rename to docs/tutorials/case_d.rst
index fbeb505..3fdc51e 100644
--- a/docs/examples/case_d.rst
+++ b/docs/tutorials/case_d.rst
@@ -1,4 +1,4 @@
-.. _example_d:
+.. _case_d:
D - Catalog and Model Queries
=============================
@@ -11,7 +11,7 @@ The following example shows an experiment whose forecasts are **retrieved from a
.. admonition:: **TL; DR**
- In a terminal, navigate to ``floatcsep/examples/case_d`` and type:
+ In a terminal, navigate to ``floatcsep/tutorials/case_d`` and type:
.. code-block:: console
@@ -26,7 +26,7 @@ The following example shows an experiment whose forecasts are **retrieved from a
Experiment Components
---------------------
-The source code can be found in the ``examples/case_d`` folder or in `GitHub `_. The **initial** input structure of the experiment is:
+The source code can be found in the ``tutorials/case_d`` folder or in `GitHub `_. The **initial** input structure of the experiment is:
::
@@ -65,8 +65,8 @@ Catalog
The ``catalog`` inset from ``config.yml`` now makes reference to a catalog query function, in this case :func:`~pycsep.query_gcmt`.
- .. literalinclude:: ../../examples/case_d/config.yml
- :caption: examples/case_d/config.yml
+ .. literalinclude:: ../../tutorials/case_d/config.yml
+ :caption: tutorials/case_d/config.yml
:language: yaml
:lines: 14-14
@@ -80,8 +80,8 @@ Models
~~~~~~
The model configuration is set in ``models.yml``.
- .. literalinclude:: ../../examples/case_d/models.yml
- :caption: examples/case_d/models.yml
+ .. literalinclude:: ../../tutorials/case_d/models.yml
+ :caption: tutorials/case_d/models.yml
:language: yaml
* The option ``zenodo_id`` makes reference to the zenodo **record id**. The model ``team`` is found in https://zenodo.org/record/6289795, whereas the model ``wheel`` in https://zenodo.org/record/6255575.
@@ -104,7 +104,7 @@ Models
Running the experiment
----------------------
- The experiment can be run by simply navigating to the ``examples/case_d`` folder in the terminal and typing.
+ The experiment can be run by simply navigating to the ``tutorials/case_d`` folder in the terminal and typing.
.. code-block:: console
diff --git a/docs/examples/case_e.rst b/docs/tutorials/case_e.rst
similarity index 74%
rename from docs/examples/case_e.rst
rename to docs/tutorials/case_e.rst
index 208246a..061e6e0 100644
--- a/docs/examples/case_e.rst
+++ b/docs/tutorials/case_e.rst
@@ -1,15 +1,15 @@
-.. _example_e:
+.. _case_e:
E - A Time-Independent Experiment
=================================
-This example shows how to run a realistic testing experiment (based on https://doi.org/10.4401/ag-4844) while summarizing the concepts from the previous examples.
+This example shows how to run a realistic testing experiment (based on https://doi.org/10.4401/ag-4844) while summarizing the concepts from the previous tutorials.
.. currentmodule:: floatcsep
.. admonition:: **TL; DR**
- In a terminal, navigate to ``floatcsep/examples/case_e`` and type:
+ In a terminal, navigate to ``floatcsep/tutorials/case_e`` and type:
.. code-block:: console
@@ -25,7 +25,7 @@ This example shows how to run a realistic testing experiment (based on https://d
Experiment Components
---------------------
-The source code can be found in the ``examples/case_e`` folder or in `GitHub `_. The input structure of the experiment is:
+The source code can be found in the ``tutorials/case_e`` folder or in `GitHub `_. The input structure of the experiment is:
::
@@ -51,8 +51,8 @@ Time
The time configuration is manifested in the ``time-config`` inset.
- .. literalinclude:: ../../examples/case_e/config.yml
- :caption: examples/case_e/config.yml
+ .. literalinclude:: ../../tutorials/case_e/config.yml
+ :caption: tutorials/case_e/config.yml
:language: yaml
:lines: 3-7
@@ -61,8 +61,8 @@ Region
The testing region is the official Italy CSEP Region obtained from :obj:`csep.core.regions.italy_csep_region`.
- .. literalinclude:: ../../examples/case_e/config.yml
- :caption: examples/case_e/config.yml
+ .. literalinclude:: ../../tutorials/case_e/config.yml
+ :caption: tutorials/case_e/config.yml
:language: yaml
:lines: 9-15
@@ -72,8 +72,8 @@ Catalog
The catalog is obtained from an authoritative source, namely the Bollettino Sismico Italiano (http://terremoti.ingv.it/en/bsi ), using the function :func:`csep.query_bsi`
- .. literalinclude:: ../../examples/case_e/config.yml
- :caption: examples/case_e/config.yml
+ .. literalinclude:: ../../tutorials/case_e/config.yml
+ :caption: tutorials/case_e/config.yml
:language: yaml
:lines: 17-17
@@ -81,8 +81,8 @@ Models
~~~~~~
The models are set in ``models.yml``. For simplicity, there are only three of the nineteen models originally submitted to the Italy Experiment.
- .. literalinclude:: ../../examples/case_e/models.yml
- :caption: examples/case_e/models.yml
+ .. literalinclude:: ../../tutorials/case_e/models.yml
+ :caption: tutorials/case_e/models.yml
:language: yaml
The ``.xml`` format is automatically detected and parsed by ``floatcsep`` readers.
@@ -100,17 +100,17 @@ Post-Process
Additional options for post-processing can set using the ``postprocess`` option. Here, we customize the forecasts plotting with:
- .. literalinclude:: ../../examples/case_e/config.yml
+ .. literalinclude:: ../../tutorials/case_e/config.yml
:language: yaml
:lines: 21-34
- The forecasts are plotted and stored in ``examples/case_e/results/{timewindow}/forecasts/``. See :func:`~csep.utils.plots.plot_spatial_dataset` for forecast plotting options and :func:`~csep.utils.plots.plot_catalog` for the catalog placed on top of those plots.
+ The forecasts are plotted and stored in ``tutorials/case_e/results/{timewindow}/forecasts/``. See :func:`~csep.utils.plots.plot_spatial_dataset` for forecast plotting options and :func:`~csep.utils.plots.plot_catalog` for the catalog placed on top of those plots.
Running the experiment
----------------------
- The experiment can be run by navigating to the ``examples/case_e`` folder in the terminal and typing.
+ The experiment can be run by navigating to the ``tutorials/case_e`` folder in the terminal and typing.
.. code-block:: console
diff --git a/docs/examples/case_f.rst b/docs/tutorials/case_f.rst
similarity index 82%
rename from docs/examples/case_f.rst
rename to docs/tutorials/case_f.rst
index c31dc24..26438a3 100644
--- a/docs/examples/case_f.rst
+++ b/docs/tutorials/case_f.rst
@@ -1,4 +1,4 @@
-.. _example_f:
+.. _case_f:
F - Testing Catalog-Based Forecasts
===================================
@@ -9,7 +9,7 @@ This example shows how set up an experiment with a **time-dependent** model, who
.. admonition:: **TL; DR**
- In a terminal, navigate to ``floatcsep/examples/case_f`` and type:
+ In a terminal, navigate to ``floatcsep/tutorials/case_f`` and type:
.. code-block:: console
@@ -25,7 +25,7 @@ Experiment Components
---------------------
-The source files can be found in the ``examples/case_e`` folder or in `GitHub `_. The experiment structure is as follows:
+The source files can be found in the ``tutorials/case_e`` folder or in `GitHub `_. The experiment structure is as follows:
::
@@ -43,13 +43,13 @@ The source files can be found in the ``examples/case_e`` folder or in `GitHub <
* The model to be evaluated (``etas``) is a collection of daily forecasts from ``2016-11-14`` until ``2016-11-21``.
.. important::
- The forecasts must be located in a folder ``forecasts`` inside the model folder. This is meant for consistency with models based on source codes (see subsequent examples).
+ The forecasts must be located in a folder ``forecasts`` inside the model folder. This is meant for consistency with models based on source codes (see subsequent tutorials).
Model
-----
-The time-dependency of a model is manifested here by the provision of different forecasts, i.e., statistical descriptions of seismicity, for different time-windows. In this example, the forecasts were created from an external model https://github.com/lmizrahi/etas (`doi:10.1785/0220200231 `_), with which the experiment has no interface. This means that we use **only the forecast files** and no source code. We leave the handling of a model source code for subsequent examples.
+The time-dependency of a model is manifested here by the provision of different forecasts, i.e., statistical descriptions of seismicity, for different time-windows. In this example, the forecasts were created from an external model https://github.com/lmizrahi/etas (`doi:10.1785/0220200231 `_), with which the experiment has no interface. This means that we use **only the forecast files** and no source code. We leave the handling of a model source code for subsequent tutorials.
@@ -62,8 +62,8 @@ Time
The configuration is analogous to time-independent models with multiple time-windows (e.g., case C) with the exception that a ``horizon`` could be defined instead of ``intervals``, which is the forecast time-window length. The experiment's class should now be explicited as ``exp_class: td``.
- .. literalinclude:: ../../examples/case_f/config.yml
- :caption: examples/case_f/config.yml
+ .. literalinclude:: ../../tutorials/case_f/config.yml
+ :caption: tutorials/case_f/config.yml
:language: yaml
:lines: 3-7
@@ -84,8 +84,8 @@ Models
Some additional arguments should be passed to a **time-dependent** model, such as its class ('td' for time-dependent) and the number of simulations.
- .. literalinclude:: ../../examples/case_f/models.yml
- :caption: examples/case_f/config.yml
+ .. literalinclude:: ../../tutorials/case_f/models.yml
+ :caption: tutorials/case_f/config.yml
:language: yaml
:lines: 1-4
@@ -101,7 +101,7 @@ Tests
With time-dependent models, now catalog evaluations found in :obj:`csep.core.catalog_evaluations` can be used.
- .. literalinclude:: ../../examples/case_f/tests.yml
+ .. literalinclude:: ../../tutorials/case_f/tests.yml
:language: yaml
.. note::
@@ -111,7 +111,7 @@ Tests
Running the experiment
----------------------
- The experiment can be run by simply navigating to the ``examples/case_h`` folder in the terminal and typing.
+ The experiment can be run by simply navigating to the ``tutorials/case_h`` folder in the terminal and typing.
.. code-block:: console
diff --git a/docs/examples/case_g.rst b/docs/tutorials/case_g.rst
similarity index 87%
rename from docs/examples/case_g.rst
rename to docs/tutorials/case_g.rst
index 9529e40..1edfaa7 100644
--- a/docs/examples/case_g.rst
+++ b/docs/tutorials/case_g.rst
@@ -1,4 +1,4 @@
-.. _example_g:
+.. _case_g:
G - Testing a Time-Dependent Model
==================================
@@ -7,7 +7,7 @@ Here, we set up a time-dependent model from its **source code** for an experimen
.. admonition:: **TL; DR**
- In a terminal, navigate to ``floatcsep/examples/case_g`` and type:
+ In a terminal, navigate to ``floatcsep/tutorials/case_g`` and type:
.. code-block:: console
@@ -70,8 +70,8 @@ The experiment's complexity increases from time-independent to dependent mostly
1. The **input data** is, at the very least, a catalog filtered until the forecast beginning. The catalog will be automatically allocated by ``floatcsep`` prior to each model's run (e.g., a single forecast run) in the `{model}/input` folder. It is stored in the ``csep.ascii`` format for simplicity's sake (see :doc:`pycsep:concepts/catalogs`).
- .. literalinclude:: ../../examples/case_g/catalog.csv
- :caption: examples/case_g/catalog.csv
+ .. literalinclude:: ../../tutorials/case_g/catalog.csv
+ :caption: tutorials/case_g/catalog.csv
:lines: 1-2
2. The **input arguments** controls how the model's source code works. The minimum arguments to run a model are the forecast ``start_date`` and ``end_date``, which will be modified dynamically during an experiment with multiple time-windows. The experiment system will access `{model}/input/args.txt` and change the values of ``start_date = {datetime}`` and ``end_date = {datetime}`` before the model is run. Additional arguments can be set by convenience, such as (not limited to) ``catalog`` (the input catalog name), ``n_sims`` (number of synthetic catalogs) and random ``seed`` for reproducibility.
@@ -116,8 +116,8 @@ Time
The configuration is identical to time-independent models, with the exception that now a ``horizon`` can be defined instead of ``intervals``, which is the forecast time-window length. The experiment's class should now be explicited as ``exp_class: td``
- .. literalinclude:: ../../examples/case_g/config.yml
- :caption: examples/case_g/config.yml
+ .. literalinclude:: ../../tutorials/case_g/config.yml
+ :caption: tutorials/case_g/config.yml
:language: yaml
:lines: 3-7
@@ -131,8 +131,8 @@ Models
Additional arguments should be passed to time-independent models.
- .. literalinclude:: ../../examples/case_g/models.yml
- :caption: examples/case_g/models.yml
+ .. literalinclude:: ../../tutorials/case_g/models.yml
+ :caption: tutorials/case_g/models.yml
:language: yaml
:lines: 1-7
@@ -146,7 +146,7 @@ Models
4. The ``build`` option defines the style of container within which the model will be placed. Currently in **floatCSEP**, only the python module ``venv``, the package manager ``conda`` and the containerization manager ``Docker`` are currently supported.
.. important::
- For these examples, we use ``venv`` sub-environments, but we recommend ``Docker`` to set up real experiments.
+ For these tutorials, we use ``venv`` sub-environments, but we recommend ``Docker`` to set up real experiments.
Tests
@@ -155,8 +155,8 @@ Tests
With time-dependent models, now catalog evaluations found in :obj:`csep.core.catalog_evaluations` can be used.
- .. literalinclude:: ../../examples/case_g/tests.yml
- :caption: examples/case_g/tests.yml
+ .. literalinclude:: ../../tutorials/case_g/tests.yml
+ :caption: tutorials/case_g/tests.yml
:language: yaml
.. note::
@@ -168,8 +168,8 @@ Custom Post-Process
Additional to the default :func:`~floatcsep.postprocess.plot_handler.plot_results`, :func:`~floatcsep.postprocess.plot_handler.plot_catalogs`, :func:`~floatcsep.postprocess.plot_handler.plot_forecasts` functions, a custom plotting function(s) can be set within the ``postprocess`` configuration
- .. literalinclude:: ../../examples/case_g/config.yml
- :caption: examples/case_g/config.yml
+ .. literalinclude:: ../../tutorials/case_g/config.yml
+ :caption: tutorials/case_g/config.yml
:language: yaml
:lines: 22-23
@@ -181,20 +181,20 @@ Custom Post-Process
The requirements are that the script to be located within the same directory as the configuration file, whereas the function must receive a :class:`floatcsep.experiment.Experiment` as argument
- .. literalinclude:: ../../examples/case_g/custom_plot_script.py
- :caption: examples/case_g/custom_plot_script.py
+ .. literalinclude:: ../../tutorials/case_g/custom_plot_script.py
+ :caption: tutorials/case_g/custom_plot_script.py
:language: python
:lines: 6-13
- In this way, the plot function can use all the :class:`~floatcsep.experiment.Experiment` attributes/methods to access catalogs, forecasts and test results. The script ``examples/case_g/custom_plot_script.py`` can also be viewed directly on `GitHub `_, where it is exemplified how to access the experiment data in runtime.
+ In this way, the plot function can use all the :class:`~floatcsep.experiment.Experiment` attributes/methods to access catalogs, forecasts and test results. The script ``tutorials/case_g/custom_plot_script.py`` can also be viewed directly on `GitHub `_, where it is exemplified how to access the experiment data in runtime.
Running the experiment
----------------------
- The experiment can be run by simply navigating to the ``examples/case_g`` folder in the terminal and typing.
+ The experiment can be run by simply navigating to the ``tutorials/case_g`` folder in the terminal and typing.
.. code-block:: console
diff --git a/docs/examples/case_h.rst b/docs/tutorials/case_h.rst
similarity index 77%
rename from docs/examples/case_h.rst
rename to docs/tutorials/case_h.rst
index 6f3592e..c9e6637 100644
--- a/docs/examples/case_h.rst
+++ b/docs/tutorials/case_h.rst
@@ -1,4 +1,4 @@
-.. _example_h:
+.. _case_h:
H - A time-dependent experiment
===============================
@@ -7,7 +7,7 @@ Here, we run an experiment that access, containerize and execute multiple **time
.. admonition:: **TL; DR**
- In a terminal, navigate to ``floatcsep/examples/case_h`` and type:
+ In a terminal, navigate to ``floatcsep/tutorials/case_h`` and type:
.. code-block:: console
@@ -59,22 +59,22 @@ Configuration
Models
~~~~~~
-As in :ref:`Tutorial G`, each **Model** requires to build and execute a source code to generate forecasts. The instructions for each model are located within ``models.yml``, which we further explain here:
+As in :ref:`Tutorial G`, each **Model** requires to build and execute a source code to generate forecasts. The instructions for each model are located within ``models.yml``, which we further explain here:
.. note::
The ``models.yml`` will define how to interface **floatCSEP** to each Model, implying that a Model should be developed, or adapted to ensure the interface requirements specified below.
1. The repository URL of each model and their specific versions (e.g., commit hash, tag, release) are specified as:
- .. literalinclude:: ../../examples/case_h/models.yml
- :caption: examples/case_h/models.yml
+ .. literalinclude:: ../../tutorials/case_h/models.yml
+ :caption: tutorials/case_h/models.yml
:language: yaml
:lines: 1-3, 11-13, 21-23
2. A ``path`` needs to be indicated for each model, to both download the repository contents therein and from where the source code will be executed.
- .. literalinclude:: ../../examples/case_h/models.yml
- :caption: examples/case_h/models.yml
+ .. literalinclude:: ../../tutorials/case_h/models.yml
+ :caption: tutorials/case_h/models.yml
:language: yaml
:lines: 1-4
:emphasize-lines: 4
@@ -85,8 +85,8 @@ As in :ref:`Tutorial G`, each **Model** requires to build and execute
2. There is some flexibility to interface **floatCSEP** with a model. For instance, a different `filepath` can be set for the argument file:
- .. literalinclude:: ../../examples/case_h/models.yml
- :caption: examples/case_h/models.yml
+ .. literalinclude:: ../../tutorials/case_h/models.yml
+ :caption: tutorials/case_h/models.yml
:language: yaml
:lines: 5
:lineno-match:
@@ -116,18 +116,18 @@ As in :ref:`Tutorial G`, each **Model** requires to build and execute
4. The ``func`` entry indicates how the models are invoked from a shell terminal.
- .. literalinclude:: ../../examples/case_h/models.yml
- :caption: examples/case_h/models.yml
+ .. literalinclude:: ../../tutorials/case_h/models.yml
+ :caption: tutorials/case_h/models.yml
:language: yaml
:lines: 1,6,11,15,21,25
.. important::
- Please refer to :ref:`Tutorial G` for example of how to set up ``func`` for the model and interface it to **floatCSEP**.
+ Please refer to :ref:`Tutorial G` for example of how to set up ``func`` for the model and interface it to **floatCSEP**.
5. A prefix for the resulting forecast filepaths can be specified beforehand for each model. Without specifying this, the default is ``{model_name}`` (e.g., `etas`, `Poisson Mock`, `Negbinom Mock`).
- .. literalinclude:: ../../examples/case_h/models.yml
- :caption: examples/case_h/models.yml
+ .. literalinclude:: ../../tutorials/case_h/models.yml
+ :caption: tutorials/case_h/models.yml
:language: yaml
:lines: 21, 26
@@ -141,8 +141,8 @@ As in :ref:`Tutorial G`, each **Model** requires to build and execute
6. Additional function arguments can be passed to the model with the entry ``func_kwargs``. We perhaps noted that both Poisson Mock and Negbinom Mock use the same source code. With ``func_kwargs`` a different subclass can be defined for the same source code (in this case, a Negative-Binomial number distribution instead of Poisson).
- .. literalinclude:: ../../examples/case_h/models.yml
- :caption: examples/case_h/models.yml
+ .. literalinclude:: ../../tutorials/case_h/models.yml
+ :caption: tutorials/case_h/models.yml
:language: yaml
:lines: 11,17-20,21,27-31
@@ -152,8 +152,8 @@ Time
The configuration is identical to time-independent models, with the exception that now a ``horizon`` can be defined instead of ``intervals``, which is the forecast time-window length. The experiment's class should now be explicited as ``exp_class: td``
- .. literalinclude:: ../../examples/case_h/config.yml
- :caption: examples/case_h/config.yml
+ .. literalinclude:: ../../tutorials/case_h/config.yml
+ :caption: tutorials/case_h/config.yml
:language: yaml
:lines: 3-7
@@ -169,8 +169,8 @@ Tests
With time-dependent models, now catalog evaluations found in :obj:`csep.core.catalog_evaluations` can be used.
- .. literalinclude:: ../../examples/case_h/tests.yml
- :caption: examples/case_h/tests.yml
+ .. literalinclude:: ../../tutorials/case_h/tests.yml
+ :caption: tutorials/case_h/tests.yml
:language: yaml
.. note::
@@ -182,8 +182,8 @@ Custom Post-Process
A custom reporting function can be set within the ``postprocess`` configuration to replace the :func:`~floatcsep.postprocess.reporting.generate_report`:
- .. literalinclude:: ../../examples/case_h/config.yml
- :caption: examples/case_h/config.yml
+ .. literalinclude:: ../../tutorials/case_h/config.yml
+ :caption: tutorials/case_h/config.yml
:language: yaml
:lines: 22-23
@@ -195,17 +195,17 @@ Custom Post-Process
The requirements are that the script to be located within the same directory as the configuration file, whereas the function must receive a :class:`floatcsep.experiment.Experiment` as argument
- .. literalinclude:: ../../examples/case_h/custom_report.py
+ .. literalinclude:: ../../tutorials/case_h/custom_report.py
:language: yaml
:lines: 5-11
- In this way, the report function use all the :class:`~floatcsep.experiment.Experiment` attributes/methods to access catalogs, forecasts and test results. The script ``examples/case_h/custom_report.py`` can also be viewed directly on `GitHub `_, where it is exemplified how to access the experiment artifacts.
+ In this way, the report function use all the :class:`~floatcsep.experiment.Experiment` attributes/methods to access catalogs, forecasts and test results. The script ``tutorials/case_h/custom_report.py`` can also be viewed directly on `GitHub `_, where it is exemplified how to access the experiment artifacts.
Running the experiment
----------------------
- The experiment can be run by simply navigating to the ``examples/case_h`` folder in the terminal and typing.
+ The experiment can be run by simply navigating to the ``tutorials/case_h`` folder in the terminal and typing.
.. code-block:: console
diff --git a/pyproject.toml b/pyproject.toml
index ef57c31..1c9d0f9 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -4,6 +4,7 @@ build-backend = "setuptools.build_meta"
[tool.pytest.ini_options]
addopts = "--cov=floatcsep"
+norecursedirs = ["tutorials", "tests/artifacts"]
testpaths = [
"tests",
]
diff --git a/tests/artifacts/models/template/Dockerfile b/tests/artifacts/models/template/Dockerfile
index dd5f778..b20e588 100644
--- a/tests/artifacts/models/template/Dockerfile
+++ b/tests/artifacts/models/template/Dockerfile
@@ -40,7 +40,7 @@ WORKDIR $MODEL_PATH
COPY --chown=$USER_UID:$USER_GID {relpaths_host} $MODEL_PATH
## Ideally, the folder/files to be copied should be explicited
-# COPY --chown=$USER_UID:$USER_GID source examples tests docs parameters setup $MODEL_PATH
+# COPY --chown=$USER_UID:$USER_GID source tutorials tests docs parameters setup $MODEL_PATH
# ______________________________
# 4- Build the model source code
@@ -49,7 +49,7 @@ COPY --chown=$USER_UID:$USER_GID {relpaths_host} $MODEL_PATH
## Provide a set of instructions to build the model inside the docker image, and run the unit tests
RUN setup && run_tests
-## See examples in
+## See tutorials in
#### Python (https://git.gfz-potsdam.de/csep-group/rise_italy_experiment/models/mockup_py)
#### R (https://git.gfz-potsdam.de/csep-group/rise_italy_experiment/models/mockup_R)
diff --git a/tests/integration/test_model_interface.py b/tests/integration/test_model_interface.py
index 318a708..ae2c0d8 100644
--- a/tests/integration/test_model_interface.py
+++ b/tests/integration/test_model_interface.py
@@ -139,7 +139,7 @@ def setUpClass(cls) -> None:
cls._dir = os.path.join(path, "../", "artifacts", "models")
cls._alm_fn = os.path.join(
path,
- "../../examples",
+ "../../tutorials",
"case_e",
"models",
"gulia-wiemer.ALM.italy.10yr.2010-01-01.xml",
@@ -201,7 +201,7 @@ def test_forecast_ti_from_hdf5(self):
def tearDownClass(cls) -> None:
alm_db = os.path.join(
cls._path,
- "../../examples",
+ "../../tutorials",
"case_e",
"models",
"gulia-wiemer.ALM.italy.10yr.2010-01-01.hdf5",
diff --git a/tests/qa/test_data.py b/tests/qa/test_data.py
index 8441d87..c2db68f 100644
--- a/tests/qa/test_data.py
+++ b/tests/qa/test_data.py
@@ -10,14 +10,14 @@ class DataTest(unittest.TestCase):
@staticmethod
def get_runpath(case):
return os.path.abspath(
- os.path.join(__file__, "../../..", "examples", f"case_{case}", f"config.yml")
+ os.path.join(__file__, "../../..", "tutorials", f"case_{case}", f"config.yml")
)
@staticmethod
def get_rerunpath(case):
return os.path.abspath(
os.path.join(
- __file__, "../../..", "examples", f"case_{case}", "results", f"repr_config.yml"
+ __file__, "../../..", "tutorials", f"case_{case}", "results", f"repr_config.yml"
)
)
diff --git a/tests/unit/test_readers.py b/tests/unit/test_readers.py
index a9e4bf8..f4935a1 100644
--- a/tests/unit/test_readers.py
+++ b/tests/unit/test_readers.py
@@ -71,7 +71,7 @@ def test_parse_csv_qtree(self):
def test_parse_xml(self):
fname = os.path.join(
self._path,
- "../../examples",
+ "../../tutorials",
"case_e",
"models",
"gulia-wiemer.ALM.italy.10yr.2010-01-01.xml",
diff --git a/examples/case_a/best_model.dat b/tutorials/case_a/best_model.dat
similarity index 100%
rename from examples/case_a/best_model.dat
rename to tutorials/case_a/best_model.dat
diff --git a/examples/case_a/case_a.py b/tutorials/case_a/case_a.py
similarity index 100%
rename from examples/case_a/case_a.py
rename to tutorials/case_a/case_a.py
diff --git a/examples/case_a/catalog.csep b/tutorials/case_a/catalog.csep
similarity index 100%
rename from examples/case_a/catalog.csep
rename to tutorials/case_a/catalog.csep
diff --git a/examples/case_a/config.yml b/tutorials/case_a/config.yml
similarity index 100%
rename from examples/case_a/config.yml
rename to tutorials/case_a/config.yml
diff --git a/examples/case_a/region.txt b/tutorials/case_a/region.txt
similarity index 100%
rename from examples/case_a/region.txt
rename to tutorials/case_a/region.txt
diff --git a/examples/case_b/case_b.py b/tutorials/case_b/case_b.py
similarity index 100%
rename from examples/case_b/case_b.py
rename to tutorials/case_b/case_b.py
diff --git a/examples/case_b/catalog.json b/tutorials/case_b/catalog.json
similarity index 100%
rename from examples/case_b/catalog.json
rename to tutorials/case_b/catalog.json
diff --git a/examples/case_b/config.yml b/tutorials/case_b/config.yml
similarity index 100%
rename from examples/case_b/config.yml
rename to tutorials/case_b/config.yml
diff --git a/examples/case_b/models.yml b/tutorials/case_b/models.yml
similarity index 100%
rename from examples/case_b/models.yml
rename to tutorials/case_b/models.yml
diff --git a/examples/case_b/models/model_a.csv b/tutorials/case_b/models/model_a.csv
similarity index 100%
rename from examples/case_b/models/model_a.csv
rename to tutorials/case_b/models/model_a.csv
diff --git a/examples/case_b/models/model_b.csv b/tutorials/case_b/models/model_b.csv
similarity index 100%
rename from examples/case_b/models/model_b.csv
rename to tutorials/case_b/models/model_b.csv
diff --git a/examples/case_b/models/model_c.csv b/tutorials/case_b/models/model_c.csv
similarity index 100%
rename from examples/case_b/models/model_c.csv
rename to tutorials/case_b/models/model_c.csv
diff --git a/examples/case_b/models/model_d.csv b/tutorials/case_b/models/model_d.csv
similarity index 100%
rename from examples/case_b/models/model_d.csv
rename to tutorials/case_b/models/model_d.csv
diff --git a/examples/case_b/region.txt b/tutorials/case_b/region.txt
similarity index 100%
rename from examples/case_b/region.txt
rename to tutorials/case_b/region.txt
diff --git a/examples/case_b/tests.yml b/tutorials/case_b/tests.yml
similarity index 100%
rename from examples/case_b/tests.yml
rename to tutorials/case_b/tests.yml
diff --git a/examples/case_c/case_c.py b/tutorials/case_c/case_c.py
similarity index 100%
rename from examples/case_c/case_c.py
rename to tutorials/case_c/case_c.py
diff --git a/examples/case_c/catalog.json b/tutorials/case_c/catalog.json
similarity index 100%
rename from examples/case_c/catalog.json
rename to tutorials/case_c/catalog.json
diff --git a/examples/case_c/config.yml b/tutorials/case_c/config.yml
similarity index 100%
rename from examples/case_c/config.yml
rename to tutorials/case_c/config.yml
diff --git a/examples/case_c/models.yml b/tutorials/case_c/models.yml
similarity index 100%
rename from examples/case_c/models.yml
rename to tutorials/case_c/models.yml
diff --git a/examples/case_c/models/model_a.csv b/tutorials/case_c/models/model_a.csv
similarity index 100%
rename from examples/case_c/models/model_a.csv
rename to tutorials/case_c/models/model_a.csv
diff --git a/examples/case_c/models/model_b.csv b/tutorials/case_c/models/model_b.csv
similarity index 100%
rename from examples/case_c/models/model_b.csv
rename to tutorials/case_c/models/model_b.csv
diff --git a/examples/case_c/models/model_c.csv b/tutorials/case_c/models/model_c.csv
similarity index 100%
rename from examples/case_c/models/model_c.csv
rename to tutorials/case_c/models/model_c.csv
diff --git a/examples/case_c/models/model_d.csv b/tutorials/case_c/models/model_d.csv
similarity index 100%
rename from examples/case_c/models/model_d.csv
rename to tutorials/case_c/models/model_d.csv
diff --git a/examples/case_c/region.txt b/tutorials/case_c/region.txt
similarity index 100%
rename from examples/case_c/region.txt
rename to tutorials/case_c/region.txt
diff --git a/examples/case_c/tests.yml b/tutorials/case_c/tests.yml
similarity index 100%
rename from examples/case_c/tests.yml
rename to tutorials/case_c/tests.yml
diff --git a/examples/case_d/config.yml b/tutorials/case_d/config.yml
similarity index 100%
rename from examples/case_d/config.yml
rename to tutorials/case_d/config.yml
diff --git a/examples/case_d/models.yml b/tutorials/case_d/models.yml
similarity index 100%
rename from examples/case_d/models.yml
rename to tutorials/case_d/models.yml
diff --git a/examples/case_d/tests.yml b/tutorials/case_d/tests.yml
similarity index 100%
rename from examples/case_d/tests.yml
rename to tutorials/case_d/tests.yml
diff --git a/examples/case_e/case_e.py b/tutorials/case_e/case_e.py
similarity index 100%
rename from examples/case_e/case_e.py
rename to tutorials/case_e/case_e.py
diff --git a/examples/case_e/catalog.json b/tutorials/case_e/catalog.json
similarity index 100%
rename from examples/case_e/catalog.json
rename to tutorials/case_e/catalog.json
diff --git a/examples/case_e/config.yml b/tutorials/case_e/config.yml
similarity index 100%
rename from examples/case_e/config.yml
rename to tutorials/case_e/config.yml
diff --git a/examples/case_e/models.yml b/tutorials/case_e/models.yml
similarity index 100%
rename from examples/case_e/models.yml
rename to tutorials/case_e/models.yml
diff --git a/examples/case_e/models/gulia-wiemer.ALM.italy.10yr.2010-01-01.xml b/tutorials/case_e/models/gulia-wiemer.ALM.italy.10yr.2010-01-01.xml
similarity index 100%
rename from examples/case_e/models/gulia-wiemer.ALM.italy.10yr.2010-01-01.xml
rename to tutorials/case_e/models/gulia-wiemer.ALM.italy.10yr.2010-01-01.xml
diff --git a/examples/case_e/models/meletti.MPS04.italy.10yr.2010-01-01.xml b/tutorials/case_e/models/meletti.MPS04.italy.10yr.2010-01-01.xml
similarity index 100%
rename from examples/case_e/models/meletti.MPS04.italy.10yr.2010-01-01.xml
rename to tutorials/case_e/models/meletti.MPS04.italy.10yr.2010-01-01.xml
diff --git a/examples/case_e/models/zechar.TripleS-CPTI.italy.10yr.2010-01-01.xml b/tutorials/case_e/models/zechar.TripleS-CPTI.italy.10yr.2010-01-01.xml
similarity index 100%
rename from examples/case_e/models/zechar.TripleS-CPTI.italy.10yr.2010-01-01.xml
rename to tutorials/case_e/models/zechar.TripleS-CPTI.italy.10yr.2010-01-01.xml
diff --git a/examples/case_e/tests.yml b/tutorials/case_e/tests.yml
similarity index 100%
rename from examples/case_e/tests.yml
rename to tutorials/case_e/tests.yml
diff --git a/examples/case_f/catalog.json b/tutorials/case_f/catalog.json
similarity index 100%
rename from examples/case_f/catalog.json
rename to tutorials/case_f/catalog.json
diff --git a/examples/case_f/config.yml b/tutorials/case_f/config.yml
similarity index 100%
rename from examples/case_f/config.yml
rename to tutorials/case_f/config.yml
diff --git a/examples/case_f/etas/forecasts/etas_2016-11-14_2016-11-15.csv b/tutorials/case_f/etas/forecasts/etas_2016-11-14_2016-11-15.csv
similarity index 100%
rename from examples/case_f/etas/forecasts/etas_2016-11-14_2016-11-15.csv
rename to tutorials/case_f/etas/forecasts/etas_2016-11-14_2016-11-15.csv
diff --git a/examples/case_f/etas/forecasts/etas_2016-11-15_2016-11-16.csv b/tutorials/case_f/etas/forecasts/etas_2016-11-15_2016-11-16.csv
similarity index 100%
rename from examples/case_f/etas/forecasts/etas_2016-11-15_2016-11-16.csv
rename to tutorials/case_f/etas/forecasts/etas_2016-11-15_2016-11-16.csv
diff --git a/examples/case_f/etas/forecasts/etas_2016-11-16_2016-11-17.csv b/tutorials/case_f/etas/forecasts/etas_2016-11-16_2016-11-17.csv
similarity index 100%
rename from examples/case_f/etas/forecasts/etas_2016-11-16_2016-11-17.csv
rename to tutorials/case_f/etas/forecasts/etas_2016-11-16_2016-11-17.csv
diff --git a/examples/case_f/etas/forecasts/etas_2016-11-17_2016-11-18.csv b/tutorials/case_f/etas/forecasts/etas_2016-11-17_2016-11-18.csv
similarity index 100%
rename from examples/case_f/etas/forecasts/etas_2016-11-17_2016-11-18.csv
rename to tutorials/case_f/etas/forecasts/etas_2016-11-17_2016-11-18.csv
diff --git a/examples/case_f/etas/forecasts/etas_2016-11-18_2016-11-19.csv b/tutorials/case_f/etas/forecasts/etas_2016-11-18_2016-11-19.csv
similarity index 100%
rename from examples/case_f/etas/forecasts/etas_2016-11-18_2016-11-19.csv
rename to tutorials/case_f/etas/forecasts/etas_2016-11-18_2016-11-19.csv
diff --git a/examples/case_f/etas/forecasts/etas_2016-11-19_2016-11-20.csv b/tutorials/case_f/etas/forecasts/etas_2016-11-19_2016-11-20.csv
similarity index 100%
rename from examples/case_f/etas/forecasts/etas_2016-11-19_2016-11-20.csv
rename to tutorials/case_f/etas/forecasts/etas_2016-11-19_2016-11-20.csv
diff --git a/examples/case_f/etas/forecasts/etas_2016-11-20_2016-11-21.csv b/tutorials/case_f/etas/forecasts/etas_2016-11-20_2016-11-21.csv
similarity index 100%
rename from examples/case_f/etas/forecasts/etas_2016-11-20_2016-11-21.csv
rename to tutorials/case_f/etas/forecasts/etas_2016-11-20_2016-11-21.csv
diff --git a/examples/case_f/models.yml b/tutorials/case_f/models.yml
similarity index 100%
rename from examples/case_f/models.yml
rename to tutorials/case_f/models.yml
diff --git a/examples/case_f/tests.yml b/tutorials/case_f/tests.yml
similarity index 100%
rename from examples/case_f/tests.yml
rename to tutorials/case_f/tests.yml
diff --git a/examples/case_g/catalog.csv b/tutorials/case_g/catalog.csv
similarity index 100%
rename from examples/case_g/catalog.csv
rename to tutorials/case_g/catalog.csv
diff --git a/examples/case_g/config.yml b/tutorials/case_g/config.yml
similarity index 100%
rename from examples/case_g/config.yml
rename to tutorials/case_g/config.yml
diff --git a/examples/case_g/custom_plot_script.py b/tutorials/case_g/custom_plot_script.py
similarity index 100%
rename from examples/case_g/custom_plot_script.py
rename to tutorials/case_g/custom_plot_script.py
diff --git a/examples/case_g/models.yml b/tutorials/case_g/models.yml
similarity index 100%
rename from examples/case_g/models.yml
rename to tutorials/case_g/models.yml
diff --git a/examples/case_g/pymock/Dockerfile b/tutorials/case_g/pymock/Dockerfile
similarity index 100%
rename from examples/case_g/pymock/Dockerfile
rename to tutorials/case_g/pymock/Dockerfile
diff --git a/examples/case_g/pymock/README.md b/tutorials/case_g/pymock/README.md
similarity index 100%
rename from examples/case_g/pymock/README.md
rename to tutorials/case_g/pymock/README.md
diff --git a/examples/case_g/pymock/input/args.txt b/tutorials/case_g/pymock/input/args.txt
similarity index 100%
rename from examples/case_g/pymock/input/args.txt
rename to tutorials/case_g/pymock/input/args.txt
diff --git a/examples/case_g/pymock/pymock/__init__.py b/tutorials/case_g/pymock/pymock/__init__.py
similarity index 100%
rename from examples/case_g/pymock/pymock/__init__.py
rename to tutorials/case_g/pymock/pymock/__init__.py
diff --git a/examples/case_g/pymock/pymock/libs.py b/tutorials/case_g/pymock/pymock/libs.py
similarity index 100%
rename from examples/case_g/pymock/pymock/libs.py
rename to tutorials/case_g/pymock/pymock/libs.py
diff --git a/examples/case_g/pymock/pymock/main.py b/tutorials/case_g/pymock/pymock/main.py
similarity index 100%
rename from examples/case_g/pymock/pymock/main.py
rename to tutorials/case_g/pymock/pymock/main.py
diff --git a/examples/case_g/pymock/pyproject.toml b/tutorials/case_g/pymock/pyproject.toml
similarity index 100%
rename from examples/case_g/pymock/pyproject.toml
rename to tutorials/case_g/pymock/pyproject.toml
diff --git a/examples/case_g/pymock/requirements.txt b/tutorials/case_g/pymock/requirements.txt
similarity index 100%
rename from examples/case_g/pymock/requirements.txt
rename to tutorials/case_g/pymock/requirements.txt
diff --git a/examples/case_g/pymock/run.py b/tutorials/case_g/pymock/run.py
similarity index 100%
rename from examples/case_g/pymock/run.py
rename to tutorials/case_g/pymock/run.py
diff --git a/examples/case_g/pymock/setup.cfg b/tutorials/case_g/pymock/setup.cfg
similarity index 100%
rename from examples/case_g/pymock/setup.cfg
rename to tutorials/case_g/pymock/setup.cfg
diff --git a/examples/case_g/pymock/setup.py b/tutorials/case_g/pymock/setup.py
similarity index 100%
rename from examples/case_g/pymock/setup.py
rename to tutorials/case_g/pymock/setup.py
diff --git a/examples/case_g/tests.yml b/tutorials/case_g/tests.yml
similarity index 100%
rename from examples/case_g/tests.yml
rename to tutorials/case_g/tests.yml
diff --git a/examples/case_h/catalog.csv b/tutorials/case_h/catalog.csv
similarity index 100%
rename from examples/case_h/catalog.csv
rename to tutorials/case_h/catalog.csv
diff --git a/examples/case_h/config.yml b/tutorials/case_h/config.yml
similarity index 100%
rename from examples/case_h/config.yml
rename to tutorials/case_h/config.yml
diff --git a/examples/case_h/custom_report.py b/tutorials/case_h/custom_report.py
similarity index 100%
rename from examples/case_h/custom_report.py
rename to tutorials/case_h/custom_report.py
diff --git a/examples/case_h/models.yml b/tutorials/case_h/models.yml
similarity index 100%
rename from examples/case_h/models.yml
rename to tutorials/case_h/models.yml
diff --git a/examples/case_h/tests.yml b/tutorials/case_h/tests.yml
similarity index 100%
rename from examples/case_h/tests.yml
rename to tutorials/case_h/tests.yml
From 4a8e76c39b2dee7ef061dba1ee009a8968610b78 Mon Sep 17 00:00:00 2001
From: pciturri
Date: Wed, 25 Sep 2024 02:35:12 +0200
Subject: [PATCH 08/19] docs: Updated installation section, modified rtd
options gh: added a workflow to automatically add a tutorial zip file into
any release files.
---
.github/workflows/release-tutorials.yml | 24 +++++++
MANIFEST.in | 2 +-
docs/conf.py | 7 +-
docs/intro/installation.rst | 96 +++++++++++++++----------
4 files changed, 89 insertions(+), 40 deletions(-)
create mode 100644 .github/workflows/release-tutorials.yml
diff --git a/.github/workflows/release-tutorials.yml b/.github/workflows/release-tutorials.yml
new file mode 100644
index 0000000..de87c73
--- /dev/null
+++ b/.github/workflows/release-tutorials.yml
@@ -0,0 +1,24 @@
+name: Release Tutorials
+
+on:
+ release:
+ types: [ published ]
+
+jobs:
+ upload-tutorials:
+ runs-on: ubuntu-latest
+
+ steps:
+ - name: Check out the code
+ uses: actions/checkout@v3
+
+ - name: Zip the tutorials folder
+ run: |
+ zip -r tutorials.zip tutorials/
+
+ - name: Upload the tutorials.zip to the release
+ uses: softprops/action-gh-release@v1
+ with:
+ files: tutorials.zip
+ env:
+ GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
diff --git a/MANIFEST.in b/MANIFEST.in
index 39e8be4..3ce7ff8 100644
--- a/MANIFEST.in
+++ b/MANIFEST.in
@@ -2,6 +2,7 @@ include README.md
include CODE_OF_CONDUCT.md
include requirements.txt
include requirements_dev.txt
+include environment.yml
include setup.py
include setup.cfg
include pyproject.toml
@@ -14,7 +15,6 @@ graft tests
graft docs
prune docs/_build
prune docs/referenced/generated
-prune docs/tutorials/
global-exclude .git*
global-exclude *.pyc
diff --git a/docs/conf.py b/docs/conf.py
index 64f56d3..32e462c 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -60,7 +60,8 @@
html_theme_options = {
"display_version": True,
"prev_next_buttons_location": "both",
- "collapse_navigation": False,
+ "sticky_navigation": True,
+ "collapse_navigation": True,
"style_nav_header_background": "#343131ff",
"logo_only": True,
}
@@ -79,6 +80,6 @@
GitHub |
CSEP Website |
pyCSEP |
- Download PDF
+ Download PDF
-"""
\ No newline at end of file
+"""
diff --git a/docs/intro/installation.rst b/docs/intro/installation.rst
index 5d53517..2d66424 100644
--- a/docs/intro/installation.rst
+++ b/docs/intro/installation.rst
@@ -1,20 +1,22 @@
Installation
============
-Installing the latest version
------------------------------
+.. important::
- .. important::
+ This application uses ``3.9 <= python <= 3.11``
- This application uses ``python >= 3.9``
+Latest Version
+--------------
-Using ``conda``
-~~~~~~~~~~~~~~~
+Recommended to learn the software, run the tutorials, and drafting Testing Experiments.
+
+1. Using ``conda``
+~~~~~~~~~~~~~~~~~~
To install **floatCSEP**, first a ``conda`` manager should be installed (https://conda.io). Checkout `Anaconda`, `Miniconda` or `Miniforge` (recommended). Once installed, create an environment with:
- .. code-block:: sh
+ .. code-block:: console
$ conda env create -n csep_env
$ conda activate csep_env
@@ -23,54 +25,76 @@ Then, clone and install the floatCSEP source code using ``pip``
.. code-block:: console
- git clone https://github.com/cseptesting/floatcsep
- cd floatcsep
- pip install .
+ $ git clone https://github.com/cseptesting/floatcsep
+ $ cd floatcsep
+ $ pip install .
-Using ``pip``
-~~~~~~~~~~~~~
+2. Using ``pip`` only
+~~~~~~~~~~~~~~~~~~~~~
-To install using the ``pip`` manager, we require to install the binary dependencies of **pyCSEP** (see `Installing pyCSEP `_}. The **floatcsep** latest version then can be installed as:
+To install using the ``pip`` manager only, we require to install the binary dependencies of **pyCSEP** (see `Installing pyCSEP `_}. The **floatCSEP** latest version can then be installed as:
- .. code-block::
+ .. code-block:: console
- cd ..
- git clone https://github.com/cseptesting/floatcsep
- cd floatcsep
- pip install .
+ $ git clone https://github.com/cseptesting/floatcsep
+ $ cd floatcsep
+ $ python -m venv venv
+ $ pip install .
-Installing the stable version
------------------------------
+Latest Stable Release
+---------------------
-Using ``conda``
-~~~~~~~~~~~~~~~
+Recommended for deploying live Floating Testing Experiments
-To install **floatCSEP**, first a ``conda`` manager should be installed (https://conda.io). Checkout `Anaconda`, `Miniconda` or `Miniforge` (recommended). Once installed, create an environment with:
+1. From the ``conda-forge`` channel
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Having a ``conda`` manager installed (https://conda.io), type in a console:
- .. code-block:: sh
+
+ .. code-block:: console
$ conda env create -n csep_env
$ conda activate csep_env
+ $ conda install -c conda-forge floatcsep
-Then, clone and install the floatCSEP source code using ``pip``
+
+2. From the ``PyPI`` repository
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Having installed the binary dependencies of **pyCSEP** (see `Installing pyCSEP `_}, install **floatCSEP** by:
+
+ .. code-block:: console
+
+ $ python -m venv venv
+ $ pip install floatcsep
+
+.. important::
+ If you want to run the tutorials from a **floatCSEP** installation obtained through ``conda-forge`` or ``PyPI``, the tutorials can be downloaded to your current directory as:
.. code-block:: console
- git clone https://github.com/cseptesting/floatcsep
- cd floatcsep
- pip install .
+ $ latest_version=$(curl --silent "https://api.github.com/repos/cseptesting/floatcsep/releases/latest" | grep '"tag_name":' | sed -E 's/.*"([^"]+)".*/\1/') && \
+ wget "https://github.com/cseptesting/floatcsep/releases/download/$latest_version/tutorials.zip"
+ $ unzip tutorials.zip -d ./ && rm tutorials.zip
+ Or downloaded manually from the `latest release `_.
-Using ``pip``
-~~~~~~~~~~~~~
-To install using the ``pip`` manager, we require to install the binary dependencies of **pyCSEP** (see `Installing pyCSEP `_}. The **floatcsep** latest version can then be installed as:
- .. code-block::
+For Developers
+--------------
+
+It is recommended (not obligatory) to use a ``conda`` environment to make sure your contributions do not depend on your system local libraries. For contributions to the **floatCSEP** codebase, please consider using a `fork `_ and creating pull-requests from there.
+
+ .. code-block:: console
+
+ $ conda env create -n csep_dev
+ $ conda activate csep_dev
+ $ git clone https://github.com/{your_fork}/floatcsep
+ $ cd floatcsep
+ $ pip install .[dev]
- cd ..
- git clone https://github.com/cseptesting/floatcsep
- cd floatcsep
- pip install .
\ No newline at end of file
+This will install (and configure) all the unit-testing, linting and documentation packages.
From ef32c94a2f726261651e07ad49331b34d00e2b3f Mon Sep 17 00:00:00 2001
From: pciturri
Date: Wed, 25 Sep 2024 19:43:43 +0200
Subject: [PATCH 09/19] docs: Re-wrote Concepts document.
---
docs/_static/float_scheme.png | Bin 0 -> 209933 bytes
docs/index.rst | 6 +--
docs/intro/concepts.rst | 71 +++++++++++++++++++++++++
docs/intro/floating_experiments.rst | 24 ---------
docs/intro/forecasting_experiments.rst | 5 --
5 files changed, 73 insertions(+), 33 deletions(-)
create mode 100644 docs/_static/float_scheme.png
create mode 100644 docs/intro/concepts.rst
delete mode 100644 docs/intro/floating_experiments.rst
delete mode 100644 docs/intro/forecasting_experiments.rst
diff --git a/docs/_static/float_scheme.png b/docs/_static/float_scheme.png
new file mode 100644
index 0000000000000000000000000000000000000000..ab527e78cc7d6affab3b67542a657105cfb97e1b
GIT binary patch
literal 209933
zcmd43XH-*R+b&24X;DCst{^t5BE4C#00N5AyEJK`_Y#_bpdczB(v>2;6CjjO6r_bB
zHIyJiz=R~U1PCE#>-)Vk>-?D?XV#iEdm-p1``J&u%XMA1uO1sb;9wPCrJ;dre-_$N7OypZWjY#pn9NChCKQaKWX=Y+otMfuTmG7sY_99y7=3LElW1eL|)R
z>sY5HCO!*hoh@#@Ss+;}I=0ePV{pr|K$mMq;_rvWR;rlSY5k~VycEjiyi!&&*O|1W
zyR@*MlLF^hgy{C@1tfx0{(X>(`5QbRZV3NK?c90aCU)kGjyce3ya$TGj%7?Vy*Vin
zxSjvVDayr
zKr3e-$fLqN9xj>{Tq&ynI_mp{9oE;zAQ@d{lkwis8LnwvXsNa_CZO6B-8^yucIoGE
zjB3-bdXs-6xcV~Y&TN~oFBE>HFBf!u*qUxPqbF{&HeBBd=gj5JRC`6nh5Jm}hmt7k
z=Z^HUc9XVx#rpEf0W^Df#t2f3g~BhW@_!oA+Rf}Sgus=!?e}VG<-2~C&a)hY4wS*k
ze;+T3vOZQ|?EJ24?_gBHloNR_C*r11i&D{MsrcV{Fe>wyWyeE6HIP$XBT`dWb!($}
zK;HuF#l+qj@o**8G{d2{U}oSC*4#20jMZ9#e`a)sd;T-OO~*Tz)%R4x_t`#t=nLO1
zjjkBuF+Y>0t3wMr*o=0XR@e!Rt#tEVRq-A%e{3mH^KaT=MOx0i3WjjRQ`IYsf;2ssHQ2gk@@sLrr{BYaXX8Qwm+D-o|Hb
z?INq9%VD%a(}>#n12K7gGG>{mTh7Kr8R`}INM+Kn`i*jKL5eqRF#=d
zi!N)NvmMRVLW${zwAzoHo|Bt=T7gB<<9QShMcnfkeGNeu1BYCj_n}!O4ch%C>(^iQ?kQ%PupmT
zC``ckf0Pq8sg7M3Y1XO`Xaut^s=>Q83L|rWS3lL(X0SVcro3l#<+TmrNI_0+TN1Df
z+K=I*Fn<(Zfniasz;vvu*X`Z=YYkVqxGY#B&qeS3C4ASC%qS(5%`p
zwifd97hhED>e-MjjPV^Fh*yEqE7i&E8kbM3nfq-xo}Lf`wYDs~)gu;wt-|xfiv5B0
zsbvc9{um_6;Ff(4A7I&hilcIao<@nJakl(+3?{DoTD-yf>gpCyCFBB=AhyZ(<#si`
zTB?O$xO6h!aVh_PeokqstH#OU<2Lv=THF4NkpG28IV+S-CQ(MOCCj5Gz`FGz%lF4N
z9bpB6-^}5buWSLs+V|ACp6~H(p}QaKtSPGAUhgjT*ibJX?>#skRr&+om{5#FzdtbR
z{LWvwyG6z+DR0&OuGIN#**tF}>h{c#sB1LYtUj>#yc=yMZMnrheMpR;s^zGjKiOsM$xDmh~4xZ$+;Y&B6jM+oTQ$xY$V+m$jdA?yK&uf;TK=GT_)~+SXX)SjNPyc
zvyf@b(J6Y0**^$ZW6P}C7_-Wm6yaIs_D3lAk2RT+({Xx-t-#x;-V|7eVg>)x?biiL
z9^^41`ISWuimZ75u1;DWmhl;lMijl6@K}hFGWz4+^{KZ|s&h?`{xcnam!vZq+P7F9
zKsfGK47fReKGu4Qyxd#ru=V_|4}bMO)Ri@?IS5ygDSW81jkaRrVrpB-8MCu?Qt*^%
zKPk5U{R5eWodSG=xsx+o&dXSnnu#g~{+V;tj&Jep3+9E;PL7<@Iz@Z_2YXK>_mNfn
zdw338EvL29VzAGDZ(dhkdn|Z!CCuRF=qX6{EBRcctE&mCBw)Ip2>YGi+u+%+jipt%
zefmt3wTg@K*2F0YVj~6CZVi3s{rV;og_kTfINU25lC6zqvE0Q{
zzy{5e>^T$l02^M+eHb{M+rHHU*lDwf$^4-G5%wr%CEIE%+L
zQ}aw!0z4)>hQ3cr%H|~{+w03z+JFQVk$8{ZP-1#oU%y5d&AN0G`dMT&2d!%ky}sZ@
zFMTmFSkeXhm1xs1B7rP&m$(0V`+6g|OuHgA;-$hyqObXbi7F(EsXF^T+DRcx#rOv`
zJH!i(wFBU;$oG(C5m>Jr#g^o#
zJn+lY@6XwRYRKrB&f%fi^9rb13Tl>qD=EO7u1QJV^>*dMQO)^J_cOoETLrKA9jqB<
zGYQ~Ao>pDk+jOQVA=IXuJFdwzFMq*9pyaHMF*@YE+ghdN#s$`TWQ|TV*KEhV1HY7f
zrqV^8!$^!%?=CbBC%S(&{ICH63wRpr_?3irnP1o-amejl3X5boqR0lLEy)_Wt?CNeD=+#GHKL?}*Y(C#PcgNN{;
zm6Pkz93~Oj0{nrOt-Ydo@~8FfxB0d^Qjzz2B5IUL8bNGULUk`@7f4BqOQjaZ2NZ>k
zr3-tr@?K3Kc=32$6C;y>TBSdfdJ@!feBoo)Pvkc9eiSre?rrJ|Tqtc`cz^wT`{2U_
z{Z(viHDcee=Zn_?6xI~WkB`@SU@n4qf7JUGCVkq6luX#tCY`sf{bcSW*tNzcf^k#?p8z-3f
zUI(uuD9HmfxqJ$ecN=O3mo(AqWarSf^_FcvHupepYt+!?&)5vxu$XE%*F3e!4LsLQ
zit<|?%Aad<*q_`jyd7SPBAG*xj4U-wA~d!d{A2;Z&$kKY$0R(sr|zgR#&RVk_;2)K
z&3MI3_fu!uiBIa{Q*kp>8(cd}XkZ`?25w15-n-!`v%r0`j<}?^5F22}4=z>+dZpjr
z_B(P&D*2l{z85ap!Hj{&h)2jqK{6g_1%XeGW
za#jj?Z+zih*el>OQzS{W^NFXQr3*j
zwjTUd$YS+CvgRPWifLjLl!W4TjA4W(u~VY6lrG0F*_kgtzN
zw`#4F8eluwL416eFPecs8WE%WycZ+1DGJ&&wi$ZJVSHodO;`cid>YMGd5md?t(2k{
z!i={D5*qN{k#buz;zTLHzhwV5>q^;QEPt{Xswi!>BEk@M;&2co9KQ1vDq`W4()@1G
zPFk?Hj@ca1n9F$LI}Nu({YvM(c)Zm#JLEGtA>~l&)=oBIR3KPemPP3W|`NrY5F6{#k`I&^BxY9b)^lH
zkNc5xx{=&vuJn?~rMAV$`?*1}iuzURP4@#ZA&T~Gj1%;pL$TdZb7u?JP^5x83rwM5
zdG+&iOqNp6E5-)y{jeO?@GhkJ6jn@|oIx~uR`oo!cfy#k=tVqcz3CJ0w73Kb*&`qL
z+z|3h56g1Jl+*@2fRy3N_cPAGe%m5N-oooAbEzGj%w5huhvMIJ*7aYyj4auywgEjw
z!~3#`B)k0TSC*NH1JyWq#}@wa2e!MpG@#5sz$VS&sb!OW#QAaCNYiA_e`zGN-1;yP
zvV2e`cP!lE?7guK)pF-o(F}0w#su|GCwAI`4t!h4G*O?QAM6{pP>m&L*!K>Dkq{(R
zEFVI=({d1r?yPhp%ubo$1lyOJUxkD@#+n@+YYg%g!LwX<`OKGsMQVU
zb=&tbw9Bdf`2GwicU}lW*CU(5li$pNos<@KZLzxgDTbiaJO9!U_l*#b~W1Bdexd)yOJaMyCMsvG1S<=G|}86+!tcCjX#5wgg`Gmj@pi|0_zgWwq8<=Gw%=MuE`6};$
zf2D>EkSv{3C`X17H`BsscY_Q^Bd@s
zu1OSul0yay(?Vo@apVw-7pK?DFw)`|d7Iqh^QW`Yz!ASYLj>|#FZkd1WsT^KaY{o#
zhFFG}cFNwl()Q>X`hcVYUD+J+*m8rFf5nciu%!L05Y{wf4m_FopmhJevBohW4#*uT
zb8^`FF$iqa3iEQRCL!$7%98LF`t)&2_;t;lRU1d~g2#Wtk
z9Lh}ByF0R`heKb-5L$AuYQs)Z30IxIq5gF0OoSL<#i~0YN@6#&al)A*K50J(lNJOD
zZd;Sb9-AwS)^&nc=$_djpn`}x6PoB|aH`Goh%Ij3c98!XKcRPEfZmkofsZ^Rk@x*k
zAUMd?y;oCAgEzn(3D0{$4-?GH5K={vsOxdhi#`!
zoJB_`j0=0Ob8a&B*6b_hs*T6czuMyVj=rs}ty}7V*+}@%djlBa>%YVD!MS`9lEg0<
zD|j&=maq#-A&TmRatc);wfA!rjbTH);t%=pYSHIeHqkcJi5un%?bqkC=8Q{utyb&0
zmdfalBff-3=B-Cml~slsD~nx8o*24C7xl&Ze$%iu*U7;XTNP+nKwW3Fja6upAFN<3
z)bShAhk4F%^_Kj4KQ;2*g_gKrPcXLkNnduku~ftLu^1M5_dN2I#+#H^?ndHf%!Ti>
zs5U&78Fhf`OGzkvct#5YY-=|lYqsu-$vp}fbmuR_H1^sNo{UD2IvGe^bg~hDmeE54
zlBxYt8p83MzBptWXtkGNBIdG;NK4S~NTcX-$9SSWEBLo<+SXZ~1FM!EJ&y7>}TI;C?YP)H_0W7UkuXWlxU
zQ@`CTHXvJj`%-iVx8M|;#;6jb#h~+0;6uZPs2aoGf_^nG=IUl}5T`dSMl?&;K#Lt0
z*}cq4D>W1fWS$4Qm@fdx^J8U&u95DqXO5x^#EG4psjzNn&v>G`kAd&5J&v(}HtW*}
zduN4ReO9VS?5bAm&jYPc01K$dMW$bs|50`IxTI=H
zw;MAZ$b@Z9IHfCP%iV@=TPeJ~{4j>z|J4yc1G7a>x_N1mzGW3+X@h1^&bV}HZsbG3(
z5XU$K*Zb};)fg>r2L(H9TnCeRt7%N)aHMNSA(r=6d24q+g&0GdPIA%CM6=ZuO4>}oB
zk``aX;5USN^v#65SEecIm;k*?-J01X1}u8FpcpVJWi)k(4-da?RtbE)gxUc=($+S5
z89LjVJUgvs{3)_DNx}Y-UnfE5u)6oMGMh*5`j~^SH8I_I$-paO=~_&^V_$HGr&uJ@
zO8wzrxh`W%V;YQ~vx5g0TIkD|`gH1}&KTL_z{Bb|p9IeOY(sX>C~dqc7`MIubgZe0
zd>y0M6I;`b>+fYG^|+;)#{=1QsEAhi2KSr9he;JQqIV_IJ5=U77uu;BLQPsZVIGIo
zSCzSkx2}D$T(_-}c?@NrgviUE9e(o?iCBAU^Y%&n5KC^5B*jy*9>9o5)ngY#>?D4X
zgA#b(R6?wm49vR5b>Bg9!SC%Q6q8MQt}4YdO;1b~8Y)TJR@wGP_A7Zd6-;4YDtf9c
z-wiB5`iVd3-UwFqg2KEfzwb>YB&Fh*9
z!FtIL;d|#ATMLUFB8co7)%V0-OpmqV6Lf&!FmAroeWMc(k);OoS(-GBMZA)6VdwN0
z@jH*L3ww@lkc!cpc-p!kKU;5HD~)Ky&o}%+qHBFP8$EFgN|A2iqr%6tnID2huW(nW
zHlflAgB<91G
z6L1(2*B=KI>=YwlKw7;#tH};m{{i*@0{H4j#+KY~
zPM1^2&4k_p@tl5+u4(?8SFXi3;8{i*4-k0`}YStIbp^
zp6KE^=y)$ZgJ~-^vW6>H<(&H0Rr2yvwQ~~}t|WF!gER%+=QQ<|-*XsX|MT>q8?@b(
zf6`%(AdW^XKZE_OLa%=6<{)?ppSo-ZtqF9B0sQ9
z1fmk}X+4PtqyVt&A)idAR(MUb?MWQTdazY541!VW-c;)S#8Ir}A|^b3+5e!s$#8ae
zaRh*Cj;s2dH$shJOnaEk1=3L_;kVkhXx{cPM`7yZs#o_)X766eikGPo-P4oinNmB2
z^4<#_IpoeGMv;i#j#8`J7s6E)8xxj-y1H`_VUm-%O>&K?u4EJ3WZS*JhrX|`PQSS*
z>8!gQX>}+tB@+*@GP_=AKKRSt9MKH`04Nm_I8+B&+s_#RDbCif=Pf9kkJ~cOs(yJ&
z`(>J`Y!!pxx5Mrtd5ML7`=JM`ULUt^Xw+F44-t~5SvnzJEt?Btm2HNR^)-qkcFP)4
z+(phA8k~aAo){-Nhfb&KKmuu+ySWvX_SiK4l*1*JShjE92tQ%GsI3L2Q2&^tfI^G2
z?r*#7<%g}=<@wB**2g2Nf}WQvqtCHL@$ejxI!+`27>ZF(U~LvZ@rQ%Z?-3S>T|Dy_
z#~}szLWUk$HP3h9k*~(q&6GWCHzua*?IY%1OUz=l+b^YlFZbaT3VX{K6K>)a6@iG>
zTr9xghzpC2XV;~gQP1qp__$*uhIC%A}kj)KY-}f(2y4Y89
z8EaqM!^V1YE(n5Q@c3hfBYQ4vK#9}P7L57kaSeNzdmdu~HHU3z27X%u8=yMNcsY(U
zxb;Nm+;;T-@o=>(qT#H&$BVIb&%o^1(1TQ}GoAXj;==A{r)&6`kr?j)x4B(^ppDo4
zh7HNry1U_Q28mmB{|M{3DF7SJQzsWjcoWNP&gS`CeaT@0ZQgHCqc~GLTlJ()`;DMx
zr)#A_oq4AzEuYK@KfdtNY(JkZHW3R%Bo|VfgNq~X)%!~S2AecLbS*CKevk@(i}dt$
zEco@D?bVdZyM>al4@5XzwK-!XNY~Wyqv=DE}L&6W2CSkHv*+
zHu6koo)I3cge;KqRcUMv`uP3h25N%3v&OrqYgB58-quhBh17Vf3BbgQGH1YDaG5)e?)=EDX|EV#8TyA4697sl?Y%y=
zAW@6!DywFESuFT0EyI~UsZ;Qb2HE`YiE}h9g$}2~anc1Y=!3k5+qdKipnfyguQhcM
zCJK_```4s7L^@c3fS{EcE9$Xv>mm@5WO_l3s+2lo?#axFX7QUiD<51@cDH+hxLjkP
z-lr>ROXcc3;Ey~K_fNcsBK@o`Uc~z_o2!^z?AH3yT+Gg{=(VA4*a#xP6hNv=tCZ(8-~5jFSbU^J~WPcc-k8Jy8)=7aN^Te(wgk
zyEr9j)-g1HV${Idbe6vMmmJb;)l%Pyce>cRZ|`BWOg9{xA{9oQeM0VSPhCkzY#l1N
znjG|lu-834(0hKG$Lr2C6FcH3%U|vKY|NH;U^KkFK9cLKPXmyFQw39n1!HJH{8J|Q
zri?_dw1CRYPt9qx_Uqen`LF$lxY|(+!Q0$J&yQU~ej|Sl*Y7vGI@g?{glcPQj%`!0
zXth6*k~7pz&`^Q@Rt9KW%Tz|vNUX%SCBHBX>B(=O-8_$Iw(*pdt>(R(;d#Sz-c6#sHM_8-PX_$+D>&6J
zwv9;Leg~Z#G@bEmEEG~Gw=4idor@(A%PrrIme;^_%h_#BQnj=2-itFacO+j~L&tAe
zL)Yf;K~(}oi@7V8iTglCyE$UZDr}GxYR36C``|Cj=4xD!3KPr7o%sCZZ~=e|$+dHZ
z#)k{}aplJJs`w17ao+~F0jtN`ghuaZ$+mWv44-uzso4`Z{qhq
z*Hvv=0IL1ZmB#Zf3(J_uAP~;8hs7>0b==#T{_p4Bv9r$V=&E$)NBp-@Z22=Tfa&|s
z_5c3WpQCdB+uMh%Pv8ds?d`>vj+X#!!FY@lf$-I_=FY$SeXiEvdeie_V%u*su#81t
z!=jMyJ6QgWB8_2ymS>*(jE?=aS7-k_vj3yW_}_Z;;}MbW+UMwqm00HY9jyOpx0G?H
zfL!pePlJH^#%SPSg22@O{o6{nA2!iIS{keS{9lCdx99&4bN|1+?thzrzr^ML2lM=Y
z_?oi|NeTccO_F8z9b
z&YS;H?&>m>$jrv_H-LTk^_cF!U&>>i^|BB<2Sy_79`Ks7z>NRQiCa%KV+VED!ds
zB13G+6a1C*W(|NMBW=*F`^LfV%Lf!$vqmmOa2@-SYEuE&b7PtlaFVp-TbPm`HdToa
z|6G-dImtXAuP+g$9*HnkeYzNA{kQq7zs+aVoLUUpUzJ<69vHr|sInW_V~t3_*}iA9
z)(z4O}lRC=p3v9SR9oql8cIOSlR34?&=J-Xyhsxi8g0j0n`XT7av!-
z0Sbb50s#Lkn3qHi0Rn4FL@G6e+jEt7U?6H4r)@ODVW^UiF`q(>S4hc2lZI?NK0p7s
zm+3SeK@t>dg&bBHYjf2Ea$uhD|0t=F3u*qFD{oAx?06D!X2d3Ao~eo+ubOUmTlloP
zazGR#0XWkaeg4hmM|;zX`2B+U3>PG*gNZK>)$!md&N+BpgBrLhG*rI3-*ins-rDE-
zF_G_4K6AzN-B5v>u6F$D8ftefEU;+yXJ(EaO~5KbX3Pr;eIjOqQV%+GY--uR>o@JJ
z9@7c7j~6NkEpWR%OGQM)JvDQb2nT4Kd8^qhFaF0P0(TNjYWs1IT
zMPedW_3v<+6wfD4w_nH!zp_A0@|3luuZg71%)xG1^)CBlv@b@SS|AibY^C@Ekvph5
zrIk65^ka01G50yufqI$>lkG@w?TrqYZQK*vp7;@5zdxSFlp`jlXLgi>x9IryxQOE0P5(%v|Pw4FOr0ECv>eY>MhWD&a)bJV3u*S^I+g+9W%D%
znB%gQV5ilL%jUHf;;G;h@dIDwMz6K4W3dPKEBA?eH*oC`GJz7cM`xvo2|5$w+Qo3
zZ@3x^*(zuqsMgT
zv__bMrEDGeSOk7H&5VhYZ)H0#O>@22ye25p=%3C(Gbmd-Q+zEe5C>R+0uda4r@Np1
zu%r8@d>G@O>3Zbe95->(f>`OCrfm)#KM|~9vkOt%>d;Sr^wm^wLekE}TSw=4e#0xx
zF2Lr6Jcb(>8R2-*2@5$MlHbm$vH<8XQTmW?=^O`Il<&|A(T)f0&@P1MF-H5=S1s^-
zW*&eKK4s?Q*op;&R*f$lMO#d(n?MDcyBT+u+3c{p?GnUb++-7$bQQ*Hn*~eFL4c{E=f4)dQXvpdR2FLX+9Trxiu=)8`R3=xMpDAhZELFUwaHJa%${QI1teb9Lj$D!
zNm0UvAfbPGNhSr@_ey#6Gx?rc2vGm*w@;^T&%KYfP>4>bsSKWsDE6Ii=Kur;nu0D6
zWCWfK2qf=DqG$<_Qp+&Tn#eq7N?5;s6G=;RbCXD3E_*{6QQtfbOMP)aoW}N8=wvy8
zJ+3oMg``0&>8RNbIsQYak~}d7^Qn{`b2RYIldFs|TNLYU2x2wIhIcP~XU+ZTCzP!;49nR8yMvb@egs0WB2@q+k~plsPyX$Qge2tA
z_AWO|4%vy)A=DI)jeiOs68vukT4y_ZucwXr-Pr@B0ibVzWo~zyGsjA|BuUox1d<9<%@klualkK=cD7-()M0Zf0=M6ITKN<^XLOyy8GY*(lF&
z1TkU${HzBdT|~DKyw|SQfzI(QU=ZHNLAg$Boq?GovadX(OhYl6L_L=FHlmI_%3l~E
zuETU<>laJ0p`U&NaNJ3zlYhE|-rh#%Xw)L9D~yd@KW7CrO)E4ply&N<5nZMR3nJpT
zr*McGH}!(wRH*Wa07jO7q|`w=n6TzO3qcV2^>gXJK=wQ1-&ms7T~L`?(otG|xA*?4
z7F-sd^9keH2gnhx`jCLrdt<0l!4RI5v~?&Kz^Q)7sUE4nSN0Pb1iwd8p++(RiXFSz
zU+IJqhOm)su3ED5o!=1rl*&yz(zjsrJt*@DLc@+UJp+KZ<1@AP4G|+&cAhs3glS}#
zobjOc&^5o7-Bv<6C>uIA(_T!Nl0Bh_d&TG(ZbxbX#_Gpo+V>kyrl)hJ(speFO*=f$
z{1|M}Qn8`bS=_=dZP8(fM>&S!dbCicQTXH+U+-JiiS-
z(b;%&=+ohms|&sZ+R3?GgQ&RvJ{JkKwYv?1*0+bw@qSg;om7>$&y3rWQjwO4fA3Yh
zGc=*vR^vxb&{+cWo%
zC1zsU1~aP0NyMzPtFD9Nza-p8aI$A)ICoFsgs}>qB;GUgN7kNNDsBE^}TGHz<$Vh91*4KUj)#u>Lg2$7gvMZ}|Sm`(zu3
zW*CZdU%a`eUZG@$LR=PK)cqABA3_)4!6CcM*>AJ8tn@3QWd%D{F@DIHAMXrB-JUADg=7VTvaouer(cl+#B}sOlBe%(FSacQgQX
zRv^AIGJX;Pk&V$Xh_xJr)`!+apu$!0&TC)TOlOJjAkvoN76d6}yPy}3;kcC<3N=5&
zdCe&^e_f<65BgJgsGaC&aj5?rKPer%1|caAEdr(}6VJkN(;$zdbZJ9&2vGV~2Qz3!
z_95p*IwvAIyXixKkU>(EBX=h|%Rb_PtlC+C1z)zc^5|}5OE8bmMdTu0R-0cO2Y}e6
zYzBeaRHw0k#l;(Lml4e*i3Yx-)Yk2**~6Vd{bPQehfPj_?FO*-tu=Y-dcBMPr_JiFg~wp-mG^>YQe#s)RckmPceE01@5X~`KNj~-==YpDD!os|^
zu_NONKMt%S%R3COf?PyBr8O7WVeH_L@7FlO0lD3gS-jxHb~k7mAv-2ID?4~%jF%#XZ=
zz;cA%wx1i7Gdy8DB!xUCE($yH{o6gRGg{`pVtX$50E)$zsdSHJqd8@9Vr`+|nK#qOVb9
zHdqV?zo;Dr6p6v^AN^2$p!Yek7sJ#UMoTfU@J`!;<7X}v>e*2cOJx}GDB@b`Vw%iS!E8x#xrY0@$$+idu
zqG(6AP-R2hF2UIapZ>BvOt>CF?+1MfoaFGq1}0J})n>}dU*3e%U8qbwZ}1Qn$=2CS}b&RT00
z{}QtXP2;I+b=L0%LxMz?|uquFI-PXa%uzYIQPm111VFm1jx;_6Bu%DI59)z-r$
zMdFzKz)L|+C~&^w3}Q{AE4p_ar-Nfz+`|@?t7bcYr3U
z^v|A`2DAf%4{6nObh^76u%8P188UBo?6fi-@z(f9zSVRuXxO{Bpt5)a+q|x_oH!F`
z{@`MFKN`%bap@o_q+M?{hCZg)*oY1p^kGZrjjt4Db2RzRkne7wMHY6+rRM@)LI!K{
z2b0*gX)o~T14ZEw!V{c%kd{Gw#AqO=9Lr6|0zYyGelZo<**n*TduxLey@GG;Yt>lu
z!mtJHZ~#FHy5OTWtBi_u(^9D0J#t$>T!V+E4S$TQ>uBLQFyGUdsj967;teOGiz46k
z1HjD8c|ER6W39rolN0F5f|(q&R(z~XW8V98eL~R&D*25jcZnL58HMaD=)-o=)0GQbIbd2?rCVUiJ1*o>-6ZlTA|^Em
z_D7^$w&wNiI=k-ZS9!S(?#-fKKk0orFf*>A3$k{kMA#ht9Q2CSW9v7FrXPi_(54A&
zC-;d-kHokD+Kkc~063U0nuZq~)=w!eh74wpO3Bt@$8f^OPV3w*Er!W}7n<)hXmc~b
zojkK9abTkK;;TMtRrcM(?zx;9ty9p~iIFf=R68>4?QY6TfI6UAMx$N5QH+2{B7G{)
z?=|_}ND@G9glYW_>-6H-Z3m6U;C@~z90F>ubgumkzZo<8;@M{R@t#?Moi_$cmjXIB
zHQH*zRzskEZTpXhhHF77ox?5dxwTF(()aHbu{R7BF6A+j>RCJW%^6+tgqA_zuJfIw
zCj$C^dOfKPA`YT&>Nazu&2{d5^B&<=Y$f&Geb@Y`tmuzZZxtH3*Ymz6h*+L;eYsvk
zR`cp!;9VwF@}15TiG`r7-H$bqfR(ebIEWISrOBlw!zDm*M2{mE6)PnnT)0TJdT$=I
z?os6aH6;Uzcye;p>m@xEyPI{9`Q|k
zXK5h1sCzVX;Q`WJ$5r-waS}Yr+Wuqov+i`(0Jm0A0GTx62av<7S1;P}t@O^0-S%Eu
zBvOZFl&M?u4L!>;2RSS82L3PBq!2lppd;K`q(`l*U+n0V>fT4Vh=N^oEH@`HFj6+q
zYh@v?^?W2cC1IFdnc__jrG}k5`buRD)j74Wodcj|%7E6FBwoc6Hm*xO0peNFT9{{1
zSJ$ebZFfiJLBk+ml4xNfxdrz=bY=oIT79OTk4FbUs94b)jkV^rIiA7st{#y1B0bgV
z%&|oXBB!&GhfH*1rvzJm&hK3W5FL;lgnKrztEMMi~qTJ$gfDHLj
zOD~zv#y%Ty*g%R}M6cdKoUOD41v`F#kiT4)!y38BX4cOJfzU+Id#PE}bg$%`f!Z*g
z&?568d5y9Us2D
zLqWL=OQ)y%nZy38$<&cLqoC@QovrpgV5}E7_U8EtW5pJvterYsfDmS7mi}(4+Si1J
zA_p8-FbTGSCY5{oZJc{MmkFB-rAX-7ypA}DX>#XR)$m%Iug`sY@l&Km4)W!_73OAk
zj!&u@JM>g4gl8{DYc+9pW}=Z6h-%g&9qDh-0VQC^*VXnEyG4o6;NUkqr*PeTIs8&$
zD+=Z4g{3qvNRR`?ZEluw-2F+RF@9ZjYdPuIj$W8>KF@{OLkgbJw`h-R{Bn_=7x+3djVwMjsD!q#t3@SL`y0Ft-(piHkB+VP+i|a^bTM_3
zM-DaEVN!c$`R+`2vz<@27Df~qao-GcI^1+I$E)_Kixi|K6nU6W9#~Eu8u>0Q_ore&
zbjeb|2dqV{BQr-sGLT&_hb(puH$jrf{jx0MZ9!MPKWwAdDl8;?Yg*u&-vD52I4m$F
zM2|vd7Us`yVszQ8@(GD#%#33s?Y=J3mls@GZE#XkKddGpld*#_mX6_yGX6WhVhV$&
zq@!~?1m+=#5~8vBb0R0TWsvQ%v#C#&q^HV
z{>!0SdW>WM`kptr;UvNWjEhh$d;tkX&6?|%l!ZnCB_DxA6sb5gT9qFFfj;56Il3C7H(
zd7<-ReRQYQfFed@O*Xj|gmdkFWNYWMt-Im#ONced_iCs^~#gqqhtoPwCW>V2?g5*fLzqiC-j
zY*cTkbLv=~0~|a^_bpmR`N9sRsI*ra-P;;8M=YrtwF&Ge#U2HS<*)wf_k{P_ljFUV
zwZbl_?|t&KO*JU&7w*j8jp?b#w}cEgr1%o62a&Gb?HWk61~5Y41%F2;+9{G&D1;FZ
zV-Mm!DSF#3FDHAHZ;jm^waF2Ky~`eL>RQwU3PDxoGR#Fk=Hg2V3e*iN
zsL?TQD;Jsn=H;PYVMN84LwCf6L3sA7{aryy
zg~li_*Js}VUeEW=4ea1mJ5!_QUMYU{y`qr*m$inbG**f-z;;^Zc2llHGx4h?O}LFHm_7HCcm-f(thDCcsi#8EFET3HD||v3G8Iw
zFTzk7_7SIgaHWML0YI^6{K>b=_d2&eamXEelFN;I>kW{KB#5$Y*C`-)xC&A8nkh_&
zh8C%ccY)Gm{W0)sRh;d%J?y1Z^)yn>}#oqCsC2NFQ^l)
zsULiDA~d=>I|bbPrkLw{VnnT@CpJBDOtT&b(A>Atqf0)Z4zPwaLTf2cP+9=FJb6s#
zE_glcLb&|+{H@BPX$@t?14TB~+`x~FC|Gk-*Q+f4z&g)y)`M_$^`rRRzy~A_4byDr
z(V0ZyLvO-2UXO)}*bakraNQ4VeoE7M$}Y5TlQyX#e&6Fc4lbFy)^8)r1Tcrsc>a=E
zu1hRSkg=}CO8Z|p%q#aCjj8qzHX^NE3K}Pypo1NkkJpz6RVF8}9C#byN_D7h>V$PJ
z*bEeJ>&_psOmQwQ8Zxo8D=g}(?n65wv#HG^(sC`f(1Gahw>bV-oZlw0RhllOzL);u
zwRnH%T%~dEk$u1t#}U>wOKn^v^N;aL6AZP!TS-gJ-^GUK&L5=L4OYL;@2^Q-)
zhB~jT3lja)o)hmdVjX9Q*pb>maiG?$%WGWMW!+sF&~4+VR9!O|Nhe4BWYxB&*oM?V
zD%3vHV!aYE*fd?zqhF*bQU+8a$+>=TQlY9#@6@@v)>_J&wS2b90R9^TuD=1+;Ex~i
zPrp102C8aZ9c+w$G~SwCA%6?fh!*(yQa-h@M*D>s;;5HHRqUwF20-jG+>qyNR(S4LG8cHOF!D4inR-Q6PHN_Th1p`@gx
zySux)n_C$+52XJYPCt&+|
zc5<@Rd{f#ytoKV|W?SQp*MW1Ygm!BnVuSf&+hs4Y^Jee)^h_%Q|0v6eCPxJQa8pjx
z<=Y**GF_dBa8=Dd%!OP|wT(3s4fDv|6CUSb->
zP@}c`AM-i1c76%i&2>}(w672@Ou!8IB6~JH*1h}w7Lb;dp#M3C{Hq=4+fMB_4K>(&
zBMH6_1Onn=65Ft^eWzRyZ$_=;!YHn~eCH*owW`J}{|fNNfl%<8T<$6XZ(bhTKO+*n
zVNclamz7GmHZRUP8x4U2RMh!t*_5p%2AaV9(nH#`Fi@(;(;8;5b}GL83DilFQ*e3>-PSh=eOF7p
z!@V|q4t)A8!I)=W-dtC~8>`}EqR_hEV#Fx_-+N!3Ff^S-u$`4$!JV_$Fq1LuX>m$Q
zX@lpZ+Ugz6w4DyH^V2hbfHkZ!n=Uk^aUmg3D|L+S+O=IamG)YuVUXkX4kjvuL$4|Fd)Ea;K90sW_oJJi+OiHocrb)FOEi
zy-DN0_AWtS%xI#IVCElNKq#Rc*S}n~7Jn%2F78{GnA*@6#*>TNOY*!9>0SZ9dRsLw
zt_$0l$6~rDyLy#AA(qu#5L-yGt|1OEM0rl>IBZ4LWIm<;c?VOpb~>=A;=g{rm^m5l
zGcK&IEi#$QwNTX@_bFe=6TLok(9%55E5w}2n|YV=SEEZ{RM6yN43=*BZ#p?{>)FW+
z5=NqtD~~+z-p1IzgWn@Yl-OA58i0k6U=LcBHpZ)z
zRxHo26yCU>+(drYG&gl_z)s(RtOvE%njWXAkBl*4fRrA4Afzdfd`JyO0vxpdM1JVhBJ9p+NuVTGt?Y$19YS%ci}p
zwrYL2cS}W0e<5Ifmmp_T>=}plFmIl=d}zwyub$NTvT?rVBf(z&{6oXj=~CRnK|oH5
zb^U4}iqc!}`(aD5vIbCziTfQl~!{M~%_Um5Ou<#;?`E(4ct
zbDY1ze@scRi7}2-PD`Nbk08Q478=qP#^8My$K$Eg6!g2+pP4$RMH^^k(yE9a<$K48
zFL@zH<;bRp2}acYE$6Cgs@IlAk{(5)VqbHA&qzg=$6swRrt9)(2F>cqtC~@lBs^$t
zeXC2$HscOzIq9)EU$-ahh6sGkuf-9SlUc8Q?Z1C=;lTv1gJV-}f~R#T*OGeg;L3*<
zD7U=2rG}i}Vli101bX)0IW1{5npPZBt}<$sngXN9?4xB~n6Y+1<4lfdB|RMtqE@^*izIDyErra^#M9LJ#r}9zm6vioa+R_7*$g2!$`o*
z9UItX44xn1tv{P`dgCD4V{H-|2|Wvev~vVCgi9HL;pJ
zwp$4);#5@)@>4P}uTQf8!JfO_cbR0&d9%{{k>^GvlHoV<`hUmKIZOC3KXBQ^qZg%Y
zrGlBt9#Jw#hV~)YKiWv4`4&fVmi&jW+oRF!!|h-qJmXiBS$9$P0
zkAV-ky*rhgV!(eeYYY$rVAiwoyZznNTW
zBtPJaY>lSR09U6vysRZFji!B?laWg8Xg4+q8AQn+azaRSJ1;>}ObLe4Cfv5W!QV9!
zn5S|~Gq4=Z-BkZUaa8?p09)9dR4+JfS9&eMvoT7Jwx9Kx%;w6N>9$L88HyL?dU^bW
zgommoz9fJ~w#iH!(XH>o9=M*7b_%=oRNogpr5$Y@d+a-bB)km03B
z@Un!g4(V1z?C;52jvO20G6QkT>&(nS6l39j=`tEGMNla^@Z9Y(XI*0N_9Mf`!&5PZ
zTxhG9JW@#>png?S_#|6gTC&ez8YN!Iql$r<%j|S!yC>1DThN>oMlCadf#iN^;F>7$
zNirY79}VmIDjoUswxis`>45QWX@IlpQ!F>W=D()L{%iU#vSl{clfwQ+hGD97a+ub6
zwI>8D^0bR;^pfypd|(z_SZ56!cfYj#%586Sth;d@k@A-^WsM&*vA94sRDPGcA~>6w
z310r4kV097D>?^`;5AH6f@{58ZOg1?7VPtzrGLkX{WIz|9u9&s+SU#~~
zkh7QB_idHUa3Q&av@SYhzJF(?p>fs>>n5h$zoQY-{X5{I^F$_3u+-yn;uW%(h=+;2
z@Va-^MPE|~ge>L3#>55eN%l8d@Dmi5?2}cwXPA`fMN)*9i5=8Qy?2Tki(~&vJy}ML
zy|dZVozwsvm171FDt*Dz5*_siC1>RwKUU5JB*>Bz&yrNgyTX+P4b9gpiql$N_l8XI
zGQIR|ZlXYM_3?fgM5}^{!q8&Qx`YNjJXx{;LE`6pT%SR>Y1_yaNaXAeUzPo@LLA1m
zL>U~0K&M3BdCf7uNRN2=vibfSflD?Rj%pc|*8q$E8uwYLP#8_b4}?LmiN<)q
z+zy9+ZYn-k4-0f114oc8w=Fsg?gZ8-A8Idxj9Pr9cRw)bbKY#QA%ENA2?W~jQ|_mI
z)XBU*Zma+7WpmBvBML7&RR*JPGo?JC5OpftBu~s-6ch6Ev#e@^1T2#pnT#7+0Uy-0
z17d$32{nIiIqjStfP)>B)o}l_$;{8arql#w
z8W4M(Bq%oRg6bWlYZotQ*G*?5>6G86db-H-@e6qK!Uj&MkW}!cV8z$4&8lskWvA
zr)w(7X7ko1HCc?`33R&>k9umF9WVNdTxYb(UA)oci~PRD!?E7{p~oX%jQB5%uz0r%
znnxkg0#x-|3DLI-ILF1AFXF;6-J9<$;kinPy>T#R{49x+&P9GvzDIW|=!{!++YZIs
zFB&y(tI!UaLW2+ZHpgSI_4x~*4M|CnS~;n%9UBeJ-JK8#>c?RJ7#lqOl*gw{Hdzt^
z!uEL;Q=&{bf;$@BrAHob2koc)$ItOAe(#-8(N3NaU<2#Y0>JM1?>er=3Y?
z=P7`N_i;r%aV5+lt>S|uov9p-it?~PMO$b*UDe(1??$@*c0%ry(NFgu*q>|!5*iGK
zo7c-Dv_4->`$Q{p%W8lRDasz~Q_a&BIh2GL{2)y>xbAR}Ywuwo=0orlwFhPPQ~2eP
zm?=*iZ9DuBr=R27-y4eIL}7!KDHRY)#zzWaRPnK!KWxcre9L{r1f1bK0(jMW9F$R8
z(cbgMlJY02i4Rxx%%PNFs`TxnBGblRf9#%<)kd
zA6wJ=ZSjVlP;bugTGZ`;I1?l9S$@1dH^G#QX0kqWf;FhnXScqmzbt=1$%dYd)rFAU
zyt)x-fe&TrJG9qh;pWE|qD(px?t0_^-~xc(_xiJRz|BfDT|r@S{<+wJU7w|o!mYrU
zsgcZY;)$TH}O3QB>CI>x$mH4vT(zn_P5E0v>?h@fO(9SF}Cg{OupU$~i
zIGTPJ(AfUW6gVZFQ64KFmo03ox?K4lg!v{%I)UhY4;{t3*683axg@3ZZsS9*3V#NTR%6<+F~YmgE7}G{X7r+b1LGVisZH3j5Vn{gAU<`DbjGN-nP8
z+7u0vDj@MB5%z7M3MCmVXOp)%9XGzGym0K0XWQE7>oSo;umV;!l}lO=#HJ6pjE~{I
zOo!v`Kfi%%75G_T`mD_0lxF;zc{O<`AGNm8B`5dZ)9oBi+?QcS=6-h~#qY)hFnm=Z
zL`Q4ei|<$rm~3j7G`y)86QLy9cc$xoq!Z5+d15Q_&m^N2b|&_S;l>{s1D)OkRDY4v
z#}DzBZ?02e2=>vbO>Wp8xkVAof
zKT!TS#;Utk53ww`vUk!Z`OH2p5wFcK>|6^^PUmqv$av-#DKO-Bu$r8w4iA9Tp4~pD
zo#@~Xy!tVcm*OR-_XnrmFS@}Uc3QFecYTZR#>;|=*Wq?*Z8d2Fk{JQmH%94MWBnTUasFpAKu
zl{U;sh;tbdYghhZpq%ehmrP#gZIGrE3)dP
zrIRNUoxL8^68cM0Orhp&5J7Lg*d8mFWZ1U}+gZp`?UQw!p26Ia=SOK*$7xu!FmGLY
zN#BR-w%~kTk28*3e3eHIx(-4FblN6Ve{M=V&Wp8N-K}LuMeG^LmI6c)}d5(
zW?<>^cc|SpMbb)sWl#hFQ{61gFLgPj)kI<6qqx%J@Uw(DG2k$DRq=r7Rs3%~b=VifOq^;Ni5I+L}JGRd+vC_s3ka^EZj
zg=JSzu{Mtg6{*-0)A2Rr{cgMric5ukKfYpqe)$@rldhzQWh}K(-tu|cYXB&xmSh;J
zue)zHw001UAPq9%JzAc2usA3$Z;|0dtkn6!6fRa>1+qjOsH+VflvapmZbTB8D0p~)
zQ$IbaD-2weocz4WMR^Otp=3(>uquTx+1_X4PVRKfe#9lUJ64&%rPi77VuQ+U
z_(QQXb+shTp7g4NNc}46#;kwbOotq!hL^?@
zQosXCkHm=~{H17mwL8>6Fv#3xgVA3=B;c%iNTtb+7!+*Cm{j+?XT!G){RW;V=f(cW
z!#kifu`dW?`x4P7yUO`Z$;H|RpJ+sIlt13%=x$XcwQrT(PpEhOmmbU5=qh?EeIS_-
z7vJMWd`{js4E6(0#G1fGOo|L9G$c*NW9!*%+D(^?yLaQy+pGF2+xMp5UdyQlRG($O
zc)ce1WNRdfjWBDI|&Qo)D87v4kE!F
zRdBXhKsb)*up7-AbMg8^-Qa%K^tpJD_iOa%@abe4s2_41@lbP2y33YNdtIVsSUQL9
zKc90G)iY*mMixl%EI>_-MjiT9h9i3(^ZvuRIW2uII6+)%s2-(lf<65$ZkWT)4|mNtP|8i*hlMZ2S%NHJ
zsRtL%Rn=daM(RXkDxUsp6(Cwwti=qb9=4JF!LG<=)are%h}8S;tJIoOG-%mpeXMW#
zU^{`Mm2tNcGj^T>Yg6KZl}p^nUkfZx96pBHhnLK>Du!>zCaWdNxAe|AG4PmfXeLRm
zL|DFeNav)<<3gmO&SP}NFj*zJsgKuhxnBOFzjZx9cH(&Q3*MLx4g;h3&s#hsD?i>;
zEK+W%o!=Q}<L+Qm_3PjOrx+4@o8}2c~kmW!c+(c3WRHwb2BpUiTf?N|d(^J#D`gZtqsRVZ
zM8M^6X1
zx8TT^n=nRvtp6snqN>zhw&gE|P*YsBd+`%im*RZbf^^|x;sv~Q
z0Iy1N4O)`U$j6OrMU8xTZyYhD*a2!_ZA!y0&m=|}fAm=}`eiS?bx}sOAW(qp2^CG_
z>$vejV9J9yfj*)RREw;y>1MG}-};-6ETwirpJFCIk1
zl2&8%=bq@(?H>B%=+N)5u4iWL$CN_^K2|{ez$a&`RPPyrq7-VqTq1){Nn~vKH1oFT
zHy(Xiw!5Q1YAOS)xnseez$#7%6QFy>laMB}WCJ6RLY;m6rJ5oyvPB6bit_(;BD~B
zk=H}xjq!4S!&lJ0kXsJvh#NR_tR~r{Fihd<{H}i1OK$QJM}HiP1CGrJcp+`(PhlfH=(k&DmU6{-b2V
zWj%xd0fi3aP*;S7nHtal
zff9I=dj*k0RS|XWt;f!B;obRYrQE%}BRL=x0RdIt*=s1nt(K(^D-?jgRPb{}&ztu&PEK^L104=drQk`xv`I0aVot
zie$@x6M%#jECgl$Az{KU8@#8B2u78V*-ls>wF25qoxN9;a|+yVR+mX!oiAEWMtG**
zzgm_32qX_^1z&MpsF!*pBCf!_r)05}GiK=etu_?fC$9xyfGm+ECt+k;Hap>@sETr2FvziCu{9qQ7h)RmHx^mH17DgkU4v
zVPKQkibB&>=UUCt3ADb-1Y5BD;|(WJ3A;m>B`29bD}F-wjK%Ygr
zwK+70Rf^cTM&`rBS&X
zUh1Eqf?-e)7N9Hw`l4yM-Kf|3C(qBB3j3F`^M*zw|5sIisp>OP34b#>I-BtWlm088MU{LZ%oj0Yh)}
z?>(VWBJ9phar!2l#+&|1B`uoa%@T#MxP)gs4~VM;2!qGhb7
zl^bxZJwm^(+gkALP0SzghaS+~=F8(-+jI^7eXf7Ga#mz#R;wr_Z2aU)YR-Pk$LF<%
z#9PRXpkRI^GunUa{+yF}AbzY-I5&94G^N~?WG-{{0V2j}6Krxy$R+AagHzhXO0;B?5USH6Su;gq+}KI>
z_Xa}HL7Y;
zdaOMkHNrfSto$!5Kl6SLz7?ZPy7*GSe`KQ!bnW5S~IvdUzy)oV+>>hpx+8
z6>NWxeiXM|tDWuSr;nt3)khggs!}l>kgQ#EFSP*r>{
ziM36b6bHTpOTCt1+S&QB4sDkLR-wB{^ak?ab__M%VmkuyXv!TjXO4XYhHTC@CdiGk
zWNyc$6F@1M%LoR6`d6OnZf8tz;@$f*qFP1Xv{mN4z^4TmQ9S@WGn_L8Wzxs2eP^|6
zl-W%a6iPyAxYT)(&_g`kx}Oz~Brz1E_81$G!xH{fiCwzGqog=Pa~ixCY7E`Pl@mD>
zlMq1@NC0UrB{(L}u$rtuU(8XM$|${|>e%;X-jf!&d^LhD)LIHa0shsfgq)^x-^zS}
zB>o5eGrgK?_!VYnAJG;8oj)2(8cvHqVfe2dp2X%zfby0s=RqL3pN)ML5!Ay
zZ?(CkYhjGOK2pKT5~sJ_0deMGUf}{rvfJ?EiV{=oW~;E`&oDjddqiV;@A_d|QYtE@
z#Hk4WuuH&Bz3lQRtqif_%!syR9Zip{CX(&ysEPFgjzcoV`vJF0m)wpQyZ~#c*z33-
z$X{NBsG7ZEkf`26P7e%$@NHCa+R>#q!Sp+smsD3;ff_2f?~-%WeE-_6sWibUey8B+
zRXp}@mW6g^HKOYtp=Xc$W@U@t`^WJk);2DDxPthPdDHOLXb;QTrbssQZW-^q6ky;o
zZF!hk{NjwP?@G&PAJ}W+-szdIGXSC%ZC#j(E&vk#Tr0Jz;N+1V(0WOF?pgj;IHqLs
zafRpA5EM4GQy*Zi$*ZopwYDkpcqNKwiLwJ*lO4ql!`WxLc
zQmlYigcR_KK&6kEdK*WuzM4h&q;Hg0S$|UlK@c#8%#A1l0nYWwa?FKAJER~Kf3o)J
z@l`V#$l#q@y^Ai>_WM|d|9T&q`!LyvQWKE?G;b&T^OW6%;McR35kuY;f5R7;2%5Qb
zWIcMgOORUO005jSMF-#4J<|-o@e_g67+L2MdSaS$qPTf9H`IP&oE?Iicd%7+|u=4>2;JgH2}$|6Qj
z{8r|hOx=p^AH`1yjeiBCGK|}#D+6Q<{3cEx)_9Y3oYk&6Mhsg&PfDBALvvYX?Y8Wt
z?r5z&u>AhGbPl8awpod6^f-C3P>tbcN&*C_|a2grMQCg_7pWFF^>JaDX50h(%<*w0a<#5zwxnT
zQI(NuZ5^3m%P-Y}in_{v=m6raE6}a^;{LPZ`Tbd5$Rd9B1by`nA?nN~Cw5(LJz9KE
zuayPwgxTwi4K-;-m*pydNhc1%gRWupfzBix^k|7iC^wRv0`m(;1YS|O7D%Sfl^W-Z{d
zq847blzeu!IyW^oKi1V>IVuK)DatWZN!7i@6fI*8-3u>Qcgy9~6Efx~F_KXt{JN2x
z(F-{gsdn@V?#;2bPVlFyTZQ*R_;u+V`!h%=n<^d?%cwoASKvMZ5L8?b2awhPdV)mq
z=F>61gPLg^1el)-$&V)gi2Tfg{zPMILcp)M;TQVyw$6ik$;FOHs>ija@
zZMMd1YXCXXvuWp)cSbq3lh+|{vZ6Skg`~iP>AA|kmlcp<-uaak?Hd^@^TN6vGIqF5
z{$8K4K&t9klOj8;
zpA292Ikwh9Xq7Sge}Kks!)r~5LS7(Bpu6kTXB@rQ6tP-m&P!L#u0d53_^%ja5^vhRo=wG{C2y3~{yti{=(eKpP|qWdgk{Kv6H$evW}fDgHb5gf?b
z?69KD)|j7lLzeYy%sxo=3$U2|-td<%qL(e{qI!o#G(s_U_SJXBsVa>i!e4iPV16~7
zZhOppZHy>6Is&jV@RT2cx`(Ng(u(9ep+I83dISk%_I4#E}1VM#pwQ!0K!zB3Z_KFwoNd`gvXS8=T~MJ3qxuPbDY_WPxDdMM@=xpnw2EoHzTT==7JW?+1I%
zQZ>If3kD0kbX;j2)`k8%+%N~bDQ
zQW143(Mp(f1AR?94}Q8HNqU!)e3sJpcQfsbn;tLeJCm~22sM4!f(>!D0v=Et()SP#
zu9VHj&KoRSg56J7Ts&0=YPiTx{j85*PJuDskaveOm$(&!T=#oEWZkn~hOC|T?);v@
z#Lt_8y7*W#+IuoPM7sBj*sdGng5kp%Ue6{|yOC4eD4V4(w`k*@CqgAsUu6ph^33;!W|Ij_w<
zl1H>bCB&+{jL|#TY)CZKn;h_n
z?IF^O5cVcd{~v85$G^xfqYIFJI8K--dZ(qC#5S%fH_T3sxZ5chB+N}TJS@%6%3(ku
zR(t$6d=+9LM>+*w!@GBkF`zS@#iumh;suwzgY)ua
z@Bp0t>27{y=*-XV6t_o@EQ+ibB8+kvKkf%LdU&Yty_baWZ*I6{xAb<5K1!cbo=Ncy
z@^jws2nF>pDun)tE}}!YXhvo`yK+a4QbGhF&lgj@Kf{){hGv0tjt|i9xY}#~gc*Fo
z??y_`pU6mBnlph3p}oP|>PTHRVKOV|Dn2{p{V0LR_-tJcYh5_##zyH?}d
z`OCowkLSzi@ZT{JR)Cu!8!%C-_u5D*eE(!}5yBwVC_zAuy)oSd=To0whSmNj|J
z5@+jk(BbApxxT`W_RyID9kU0n{^?#lGzr)3xMM6`PYwh612SoGLfpDosyN`@xC(1l
zs1@AZ6l`1`8?10R?!4pQ{LV1Y?gk=<8=sEwu8B3XT_lkS8M*27Ku2C_KQx$DB`H1+
z=={N3Sas}hqrev0}bR)ZU%3Cng{eY@tV%oi1d(UbX
z#rxtGspBSq*XPp9t@*vk2H%AVskhxr-%t^z-++(evd6x>E+l|(Nj76DY&s7m!uaJ$
znLS-xmz%qw>)6+m^8WT-umddL>FIv|5^so(@O;9R=>x1nyK+D=CF7ZNYmTOO2gNPq%)738wVWs1so@KGZ7toFNNap0JH&T^vu
zqhWD$tbnBh18;5HHm@qa96M$m(aRW&nqHsXuZ&0w*PdP9-(E-R?&x+tGn2uG
z<$_}4$%0oO`ou7V@39NAtI2o3_x-J-9{ZkGWjacM%-}AhwYB20G_Z+b{eZd6)fu+n
z@;I`$H!DZ!JMJE(x3hbB`gETDrQdFFf5uPJu~4!|)YPWiS~
zIj(Sk{m=F3%Js|693FthYFqip~oKqAK@Fr<+W7_M3lPouIJ0o-y$0B-#+%rt0(N(ClWWtFP8;
zomMAjJ=1zaC^c^`b!TLRIPwyIjEP6f%{{m7Wg>Ug1hI0iH>I6GCw(C2c>2=Q&ZOM~
z!1*yjr89iOr)O&AZE$4;$ttQLBm~rppG)k>bhW1`r=X0H(_H=&{U|vxQG5gT`Du%%
z<3$ruO@;x$`o_X`Y5
zHpzMufhF6g6!rl}L1Wd{w2u-D;^09AjXeFm=MFbbfROG
zcv6c&9JR&w=D=|gz(4%z&O+DWI$oPFJ_bYNPPSB!xq|76a|t&;XC0t{A8buj!M0IK
ztd-=Tg3mg0zWLj|B8tOX|%`Bg|+
zTn{rv*rGi8Q@RID*g8DxUYq5ZB4^)8T#Sb+7^I5PUalSZY5Dv%=OI)$J@xNQ!xs0a
z;`!QdNxnRtyBY?XlZzFUj%MxLi8`OuVkg=p-y8tZSkES
z6i;mO#-;+@{Hts^qDQ3f26{OWT4kp|&)dN+5+$F-L`fMLC8&%@3C8O~2xF+BqJISt
z3j7&)sL=KA)+58KCX%M9Df7!CO@9$jaN?+_z)|_5wOjXRCD9!cb`&(uyc-!dcS;;9
z0eXCZ)fwChLwVoR&~80v-TaicA&KevPTOIHchaxIFOO>?d|KiRuOjVr{%83B^uwIk
zoHNS0S74PXgv~*15R=*
zK4@o?@6eJT0G2z43_k;jnj$^hfhrp^;KbVbFps+CHBEJ#-`HBsMP)3zdMRxs#xHDp
z;5@xR!)1u42}NL%`>WP{z56slvWqh
zTZnJTsNMUuJl^}#Cw4GsKm{(jkR&O>;xaxp{y}n^uY_nRx$$x_SeCffkY?)QYy4`E
z$QkpDn1+iW?XP%$Np7wtlq@U%&AlbuaeTk*K~@drB!850J+@3j1&TV?C&|iSIp;a_
zVPxyGPP=ngZ~2vV?lC@Wlmwz+_LvjlWR#UY`2LDZXLWdCmEg%fW_{gI%snkT7Vx>d
z!w@AjjV66Hr2^1|Z)!q@$!sw>TfO!|z-usSdsB3P7M%B$&V(dd;w1u~kG;zW;;ED{7c53^wlH03E;(6D
zM~CmcI516slsVeoBZwzGIWZTb=otDDP1VMk`)y1f#36v&qXjdRx{;G!d6x1LS70p?
zHH6G=L3VeW{6CzG1IfEqbEEG)bR(y7c#EZQjQNV?KgYuU{VI^*6%m(%rD+G+A^u?dbU3y!~Kpo!Bm?0eoG%
z`Rf!S+jU#!llD!PMe^$Y2kRrNE55JQp3g5bCZ!@Mfhmlh*3_`#%G+nptWj-N?0-ZO
zdnq3Gx~g0VMclh~{U>p=_MfuC&RRk~2OZyJxiE^jPw5J`==*RJTIH{m!OZr6vM5aN
zXv<2O><%YqBR*Gcf{woM5YA}dYo6A+V8_bj%gZ2RGUT~P6Qm;H_bj+P!%<6O)6Sm9
z=qKFNDY!S_=L*U#nioA_{B5X@^(e$AiwZ+a5uzZbF4J|Vrc+nv|4Fu&s{)^CHlFL)
z#cUCP|Fb|Fbs%hAUMr|QN|$ayM6w1|lHluj%z;@jEfSwber*O)#FS4uQ{qy6gT0y_
z+}_inK-ICG9PznKw6!YbwS1eGYhSgBG+u5D2})hIPW7xV;J$i4SE<2MYR)6y?v*0w
z<*GfKu2;QFwpZZYJrieywH+0_6A{sV%~{UgT0U0*0d
zElZj4*X~`CMd4-jtA27P3`hllxoE^b1u*w>C{Z0+yg$Ea?RizJMj^k5h0#hvxj^-C3D_`r_m;a6ks|#gA~A@EPq*9&82
zgK;HJBIoe6{N-#^1D!VYE71WRh6PNGaQ>*xvWTxS@~w^!a)Ze|2pl^ordm8>%5D`oufG
zo78R%jtP|ZU6}j^t?6pDdvlDB0>`(@c+mp0e6-%z`qs|{fs4zo#dXd)J>i{~Q}3u1
zpJUumEIxK^af%&ds>?ALrM*s9y9X|P@$J^5fE~{+JU%=oE6u?0*dP9Wp_RslbZfZM
zL9r=vX9C7?^WgdTc$@iS{U|3T>)TtM{(j_Z+nnbX*0Kd}K1htlQ3(CLL$I}iJyceX
zG)s(t_p1C(>K0~tQ`z@-f23{Mg!OqTd56_?xVc9W3w(`myo4O6Q1aI$qzBGIa$+*E
zNiZ7FLbX54Z`d0ScpiJ*+0b^r6yKN0dVEy{DbrKiohD{>##X)@4ELcmv+J94O-flt
z3KmwQpRI=vySmLv)}eZxO*QUurK1A2D{fnH76J;<0TY22oawlyr|+*GGX@l&WrPEG
zdL!U87eYV94w@&`cL~Q;tdxfay{H(j(I>7jKc>*iPx61i^a-=YyRDLm%GGpiknclqS-y9w1
z8+F?TjnTNV?WD0iVPo62)wr=5+l_78wj0~YWa2yhefQq={W)vanziPA&U4N_d+&4J
zn`6kMQgDocRY{TTBJ<)B(m3S5jDjh&dMQPF<=K~Wo&g14DvW(-N=O}US-75{EC3En
zUn-$|N?IG5GqNwmypLZ_W*D`96c6nTeDsA&&QJIIc9FQ+5EHoRdS+WR^oI&g5-xoP
z$p?CA;;8v-u4ErUzsM({D(LyLP}633c#rZVV`J40oI&?s#v&q|5%Yi4PmAuX#p7}6
zd~@@Ahm-@6nE|s$vRwo^jp1lq@6q>l6!IzH{9dXOUqMMl>YG%hWB82PQgQ$o@Ox%>
zOJk|*>yva^=J{+D`4j*NQwg_rz#LuXb0FDPnI^%CA-*1FOZlh_U)oy;9e_t(5(+%_
zSi4@wA~SpxAQa{AR~-LUtzXY&?5zE?X?=-lH>dmgYuAH*_3Rm5XPyXkdTwi@`+_67
z>vS>ZtJmXb1zd;lQjG?hSi!2AqO7c3DcNQ`R$SDb%e?A8R(1}ob%la?c9=GSiJeQ{
zIyFENx4g1?U39MoCfMBeOh)L$&+@~StV@HtOXHBX;E9o7f{&;8vJ~ucT5%r!gXs;u
za#wz>EeTf+T%
znGPfKYAZXreLMS9QPhvepkv?^Wb*0bM5Y;YDS$n++XOPY=7Xun9~sLBO6_gx$+XzZ
zi?IEjTzAz?*c`mTwScZ)+WW(DWij_Bztx6O&%
zj~dz+K3Ak?6uoDy+JJI5kDu-VbQ3<6eVBsN)szzipZ4|iwzT`_d6Ut8Me(&F$hb&4n-1zw+G%G-1b#|;
zmnbHOEWuu?DK_7aIbFkn#|SBQw&}%G4AYtL;Uf&;r*JQ=c)9V
zg@@8UD5W||tu8TDpIX9E)9zhxwCjHsDsq!e0)}P#;8g=`xpWP`|GCp*O{)Z$zvbKf
z@d#HPv?m4dAaIR5-=_R3)&ci!|B5%3Mpk2OU*i<@utWOuw9E(#{G~x`d9bg&4Hh702WZ4qLOtd%$@V
z$E01lzQlYKM{U*$fG@%0ie$72nA=3HxVr@j6&~HbO(Yrd3GV0L;-afF&udh!4G;tr
zvip^{zra-{wr27hUmb@2$*>ApUYv0`<*?c$ZuY>5TtS}u*TxuW+^k3=_*iv#XrQqT
zH3%|%=rWCPEGZlUsk*l|$Fk^8E}(esZx$+WOVH`2!`k2gQtP?2-PgivD1xiGy03Nk
z?{Ta!QcuEk&ns_#mh-B*@LFO}KJPgIkJQNL
zEW*I5+2IWk5B^i$;RN2Fg*LGsXeK{{!aL?#*d<@8wp<+8v4zG}Qo(P52y!aUV`gpj
z34m%#lpNv3fhDO1f}dM=T4`6rJeoU)-?3ZT>f+6TlLu*?&W>$+3YCZyTP_cS;<_4g
z0jV@~M@mkbnGR;b#Bc-W_SoT1?!mA{r{^FB)By#Fxe?;x(5Z!QIN*H=sh^xvXPnFI
zUXi~S%o3^A;6kZoH@Q-Y7@r11NvKh*kj~~Vv
zunf6+%2%TdVX#n%luZsu+SA~r=SC`sVZcH`1PEmgDv)*MCUr(PJC*rD)>aJq*Z8f222s{WL19b>9(EsF!~B-Tg$2(QQ=RX3emuYdt$IaBEXST|-pR;}->G?7=N5
z#NzPh>5vOQ_6SCrX3zb65{g5a%Jf@1_4vy7b7v*tZYIy-FwRPi`_76{|G>Ehem4H!
zbXebE8>dBvze?#>ZGO3Nu^>STT7f6A1%Pg85b{i5h=nDktjeZfEhG0riHbjP|I+Nhz
z>>yW*xaDO4#5F2Pp6H4qP$xiZ68Nvg89djfNxpaU`9U&Pe=Vo+qF$6G8T}F$*2No<
zKerQ%g2Y$VPkO<)orZtMmh4}h*4pyT!IZa5UNfd&WK@_UZeXVtnxL@!h$~w#~kr$WoOs_1)Qv
zn!#_lzU^0kB>C1H;pq5d((B_C5N)uNoa7#3U2tTcO7=mwC!mOo!sJ(%vyr3Dro$w2
zL6EEgg%_vUYp3+`lX@l20OmEqb2`Vct1%>w`#w}3>M{Y)a{Kj@1l-fa7bHcM#U>0C
zZd{(r7R@r__<@|4cwQa%2yyK+&m4X6I1NXwK9oL=$=MJZDnut)#_oA;_#y6Hh~?d%
zi(x5?$jf8fn==oGRIxqpC!ssM$a92#7e5xnTfo)iHy}jzO6CtL=%pt{A4C_f@o_yH
zFML69=*Ufdjg%XyB2mvYTAa@@fd~a+ey%56#rPz?*d@wSQ`q0JZmvdqcjfFlm84T!Qc_8fys&G_d%Id+mWuM}V*llP|Oh6jlQ
zSfw>KSC)7TA-s0W%B97g^ZPe0tq2saH=N2VJ*IXIQI=w_1(K00A{yeUgPzK@OWA#9
z$j~yebEEWNiNrQg5wNMkJzG)pXTj2&vNv
zL9Z?agY56%Mu-aF{`#f`WhQ;q&=*@BAw^;uy#W-9PIp)8b%gT5w!JpgUkO(wJ_E
z`-G$Gy|!|rp)2ae{|&9Nk}QaBsK49Kf^Eu=$tx?0e^lBk0XkzbnBAd>7Ke`C7$>1H
zA@#VpnRBD5VjjAd%9Bj5hJmhWW+Ck}tyF?o{-Thb7X;lnYm1>TxR~cL#cWq!uLXS3
z^YO(N#C^L~2Im$i2$B^!0<@Af&YI>M4xip*pjSp|Vl;F&mEeiVf+wlsy7K5{8}G?u
ziCETf;hDMXmj1s=Q0&@`2S@!y>qX|zMd`lLH0!g^>~@L=%gM8*{xvm4=?j%%a3Ec@
zF!rXieQyR#FVWoYX*S;XTKVk0^wbx@;y2&F*w=sA*5r~67fD+z_s$)*q4~8m`ibFM
zwNA|0J2q#ql}5QbTsY8<-Vvt)OqH-ifNizWvljExDx%yUM|pmBrIDe5F!|A|o`%Pc
zLvypurDZdOq$?ByC*=4hKPrN1lR7D0UTFAr(d%J0ELzqLuGC3<(+*8!^>J4xJybQ>
z%*cF>6vjy@S3+529nMcU@6NYKh9eZKue-__9gY@De(YjUO;a2=_=$4VLG=chhIXqT
zX`lN%mQr-MUuEF%`jX^fkclT`$|c^3!@JJ%w`;c(J%4~^k9M5h<;RLi^Ot*xx{7S?
zYSsUm`!Cs1LfYtk=Co^h-E_|epw<;bXsVIyv*D^>{AT&3n0No6Jw(1gyz8UP)a-NO
zdAXw1hkdCf?WYDM;{;2Gb$D~kX?Nr-=o#8sf=k))MkLS{>05#qPZ7RAjv#Y*;PA2}
zt;U!h1VvrOQyfK;I2*gU6*gR;X%Rj2i82u-i18K$czSEYs$2mIZ6v(*2zq@wa0Ug!
zXYmu3oe|)3ZAzF|r+_!u=IQ+li5nK)Sm~{UdoS>-54<_$rdE=FiM%LoDZh3fAAr2Q
zT3!Jiy8gW1H*0w-V~1-W?If69M(eXkr`;mT4`0LW6c`)%F&DU#XV&!jsvwo^nFAo@
zt_mu1{?U`~qRZn7>`-kEH1nyrRP;Z1u`3Ge
zmiQ}w4BvWvsJA!+j=F#dagitadSJ9#DYXKc#Dq$17$exKO}(c5l94aqm|&CMDK
zHpqqhFUmAZ>lN)$f1~wxm@U;LWx=T#a4#~EbTe%)y(ofXG<*NW;|uNTgW8Cm;BzX3
zox^t&{wXa2#^lC}&sRM>t`Z|U{9~!2V}=jOAZcxHrrzx
z2=m{9@u#s;a@H{6iad>F9Q$_8nr9?Uj{%LGGYeINy&aGhM*JuEiubo6P4D*G(aG1d
zFSBQVXYGT8QDQro*)N3|IeMA0XExj=AZ;|t!r!&k`y*;ktJ#9*37*)@Stt75U4NE6
z$1jiwZ`v#rET3FI@2Q`w^cyrmYjKvapMxVPbExQx{XBnd7Mj%HPV_xc1wqX
zF~Tt$Qcu!@*z92QAz9_yAi7p8r$JG%_ux+lFDlC=FZje*MT=_EO)XwP0p^6+hMvdS
zAIdrY8{EutB8OSP8XPBLmII18j=DLROXm69EhG+?h+<1LC
z_FMce+rPg6#A}zA{Vb0CcJY~qc8m})pG*qW+<5iSCjHLg1TrsImPEv<3Xb<_0E~z&
ze~m#FyH;zPJ?`yyCblHm~&P#y@Ml;$k@jw+;t9WnTk#LEE&-Ue}zk9;v!w!3VO^IF~TrJL)fCO5j*>DzQgVx-@0CRxbz3e^AmUS
z{O2zPZ~I1$V3;I8uUe}gXD>VY0f9D?jqR?Dz-5jHuhEv{OCrs-(q$TQs)?xveZWH?
z5Ba^u!-p8#T%ZjV%jO`ahE*u4Rh`+2HTM^zW{G~Ib-(g1K4&_RanwL{ZPeD6=Ten-eGT?}T>#N`=PQ|%i&B~@gV}=O{Oi%uSyL
zTY{%vY1ApEl26Wj_MP*T`jAyMq&Umrv^fgaoEomF&e5Dspgex+T;G!t<
zT5%H%_kr8~lSY0o8^RDb*M{8Xo)5hS`H%zSiCNM3k=((G9f=&c|}tB6YH+lAFX
zZh_em+<%UaD8=cle{-@m*U0$b>fCa{^&bD@sxmjz4l5pg#h@NZZw>
zYs(GkOIC`{I@}WWD_psulhO|x$X+3e{Y1-q?8mt>Mi@{D
zk0LV+kp1=~VxDbvkpXi31C>5PYgiHTxehlYXT=M_C0C(}eRUzkh4D`*o
z$0L9}op*fMw(IR?>dtLAS~PpL!7iz$+Wa*BlDcaci7GKp*%~|yI~Qu4{bQp*@&FcY
zpY(ZX?DU2!Uv>Kt0b@j76hNRrO4CQ0<&o9?yw6ELsC2+K~+aLEl`-W=m9f-W8a8C;j
z4aV%xEXP5{@Z;g)lGPbYRWw+`BB8j|1}Wef;+>!W4&H&A1gf2ea&$phX^3%*)$QvK
zS#}^}ZdG%_TDG*#d%U{)-XL*1&Gg1#-@UF#1%j}mE4X4D7~C;HB^?u8vCHshL=qy?
znSMXfose0b?)d!%8EDDk8Y6AxO?wO+sCh>u>zxr5U$PddXh&qm{TuqQc2S~_d#d?(
ztHkI4`a0X~EJfBX-)0$Z(v(x`ux#bd
z9g!}l5pytGa3LCo3_8ie7+1Ek(L`wHVQ2br$szzXNJEn4NAu;;+-PPYg8P|xK{p&9
z$toNTI&~O