Merge branch 'main' into 690-ignore-sphinx-build-folders

Author: javier-alvarez

pull_request_template.md renamed to .github/pull_request_template.md

@@ -6,7 +6,5 @@ Please follow the guidelines for PRs contained [here](docs/pull_requests.md). Ch
 - [ ] Add unit tests for all functions that you introduced or modified.
 - [ ] Run PyCharm's code cleanup tools on your Python files.
 - [ ] Link the correct GitHub issue for tracking.
-- [ ] Update the [Changelog](CHANGELOG.md) file: Describe your change in terms of 
-Added/Changed/Removed/... in the "Upcoming" section.
 - [ ] When merging your PR, replace the default merge message with a description of your PR,
 and if needed a motivation why that change was required.

.github/workflows/check_changelog.yml

@@ -1,18 +1,17 @@
 # Changelog
 
-All notable changes to this project will be documented in this file.
+Early versions of this toolbox used a manually created changelog.
+As of March 2022, we have switched to using Github's auto-generated changelog.
+If you would like to view the changelog for a particular release, you can do so on the [Releases page](https://github.com/microsoft/InnerEye-DeepLearning/releases).
+Each release contains a link for "Full Changelog".
 
-The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres
-to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+## Changelog for Versions before March 2022
 
-For each Pull Request, the affected code parts should be briefly described and added here in the "Upcoming" section.
-Once a release is done, the "Upcoming" section becomes the release changelog, and a new empty "Upcoming" should be
-created.
+### Upcoming
 
-## Upcoming
-
-### Added
+#### Added
 
+- ([#709](https://github.com/microsoft/InnerEye-DeepLearning/pull/709)) Update hi-ml submodule.
 - ([#667](https://github.com/microsoft/InnerEye-DeepLearning/pull/667)) Automatically and linearly scale the learning rate of the SSL encoder to the number of GPUs.
 - ([#689](https://github.com/microsoft/InnerEye-DeepLearning/pull/689)) Show default argument values in help message.
 - ([#671](https://github.com/microsoft/InnerEye-DeepLearning/pull/671)) Remove sequence models and unused variables. Simplify README.
@@ -57,7 +56,7 @@ jobs that run in AzureML.
 - ([#656](https://github.com/microsoft/InnerEye-DeepLearning/pull/656)) Add subsampling transform and support for MIL mean pooling.
 - ([#679](https://github.com/microsoft/InnerEye-DeepLearning/pull/679)) Add FP and TN slides/tiles to DeepMIL outputs and extend outputs to multi-class problems.
 
-### Changed
+#### Changed
 
 - ([#677](https://github.com/microsoft/InnerEye-DeepLearning/pull/677)) Update TorchIO version to include the recent bug fix related to patch-based inference.
 - ([#666](https://github.com/microsoft/InnerEye-DeepLearning/pull/666)) Replace RadIO with TorchIO for patch-based inference.
@@ -97,9 +96,9 @@ gets uploaded to AzureML, by skipping all test folders.
   `autosave_every_n_val_epochs`.
 - ([#632](https://github.com/microsoft/InnerEye-DeepLearning/pull/632)) Nifti test data is no longer stored in Git LFS
 
-### Fixed
+#### Fixed
 
-- ([#698](https://github.com/microsoft/InnerEye-DeepLearning/pull/698)) Ignore Sphinx build folders.
+- ([#704](https://github.com/microsoft/InnerEye-DeepLearning/pull/704)) Add submodules to sys.path to fix autodoc's warning.
 - ([#699](https://github.com/microsoft/InnerEye-DeepLearning/pull/699)) Fix Sphinx warnings.
 - ([#682](https://github.com/microsoft/InnerEye-DeepLearning/pull/682)) Ensure the shape of input patches is compatible with model constraints.
 - ([#681](https://github.com/microsoft/InnerEye-DeepLearning/pull/681)) Pad model outputs if they are smaller than the inputs.
@@ -144,7 +143,8 @@ in inference-only runs when using lightning containers.
 - ([#655](https://github.com/microsoft/InnerEye-DeepLearning/pull/655)) Run pytest on Linux again, but with Ubuntu 20.04
 - ([#674](https://github.com/microsoft/InnerEye-DeepLearning/pull/674)) Fix DeepMIL metrics bug whereby hard labels were used instead of probabilities.
 
-### Removed
+#### Removed
+
 - ([#692](https://github.com/microsoft/InnerEye-DeepLearning/pull/692)) Replace InnerEye-DataQuality with a link to commit,
 - ([#577](https://github.com/microsoft/InnerEye-DeepLearning/pull/577)) Removing the monitoring of batch loading time,
   use the `BatchTimeCallback` from `hi-ml` instead
@@ -159,14 +159,14 @@ in inference-only runs when using lightning containers.
 - ([#596](https://github.com/microsoft/InnerEye-DeepLearning/pull/596)) Removed obsolete `TrainGlaucomaCV` from PR build.
 - ([#604](https://github.com/microsoft/InnerEye-DeepLearning/pull/604)) Removed all code that downloads datasets, this is now all handled by hi-ml
 
-### Deprecated
+#### Deprecated
 
 - ([#633](https://github.com/microsoft/InnerEye-DeepLearning/pull/633)) Model fields `recovery_checkpoint_save_interval` and `recovery_checkpoints_save_last_k` have been retired.
   Recovery checkpoint handling is now controlled by `autosave_every_n_val_epochs`.
 
-## 0.3 (2021-06-01)
+### 0.3 (2021-06-01)
 
-### Added
+#### Added
 
 - ([#483](https://github.com/microsoft/InnerEye-DeepLearning/pull/483)) Allow cross validation with 'bring your own' Lightning models (without ensemble building).
 - ([#489](https://github.com/microsoft/InnerEye-DeepLearning/pull/489)) Remove portal query for outliers.
@@ -246,7 +246,7 @@ console for easier diagnostics.
 - ([#463](https://github.com/microsoft/InnerEye-DeepLearning/pull/463)) Add arguments `dirs_recursive` and
   `dirs_non_recursive` to `mypy_runner.py` to let users specify a list of directories to run mypy on.
 
-### Changed
+#### Changed
 
 - ([#385](https://github.com/microsoft/InnerEye-DeepLearning/pull/385)) Starting an AzureML run now uses the
   `ScriptRunConfig` object, rather than the deprecated `Estimator` object.
@@ -279,7 +279,7 @@ console for easier diagnostics.
   `get_image_transform` (resp. `get_segmentation_transform`). These two functions replace the old
   `get_image_sample_transforms` method. See `docs/building_models.md` for more information on augmentations.
 
-### Fixed
+#### Fixed
 
 - ([#422](https://github.com/microsoft/InnerEye-DeepLearning/pull/422)) Documentation - clarified `setting_up_aml.md`
   datastore creation instructions and fixed small typos in `hello_world_model.md`
@@ -293,7 +293,7 @@ console for easier diagnostics.
 - ([#445](https://github.com/microsoft/InnerEye-DeepLearning/pull/445)) Fixed a bug when running inference for
  container models on machines with >1 GPU
 
-### Removed
+#### Removed
 
 - ([#439](https://github.com/microsoft/InnerEye-DeepLearning/pull/439)) Deprecated `start_epoch` config argument.
 - ([#450](https://github.com/microsoft/InnerEye-DeepLearning/pull/450)) Delete unused `classification_report.ipynb`.
@@ -303,11 +303,11 @@ console for easier diagnostics.
    RandAugment & Co. The user has now instead complete freedom to specify the set of augmentations to use.
 - ([#468](https://github.com/microsoft/InnerEye-DeepLearning/pull/468)) Removed the `KneeSinglecoil` example model
 
-### Deprecated
+#### Deprecated
 
-## 0.2 (2021-01-29)
+### 0.2 (2021-01-29)
 
-### Added
+#### Added
 
 - ([#323](https://github.com/microsoft/InnerEye-DeepLearning/pull/323)) There are new model configuration fields
   (and hence, commandline options), in particular for controlling PyTorch Lightning (PL) training:
@@ -326,7 +326,7 @@ console for easier diagnostics.
   config's `get_model_train_test_dataset_splits()` method. See the `InnerEye.ML.utils.split_dataset.DatasetSplits` class
   for details.
 
-### Changed
+#### Changed
 
 - ([#323](https://github.com/microsoft/InnerEye-DeepLearning/pull/323)) The codebase has undergone a massive
   refactoring, to use PyTorch Lightning as the foundation for all training. As a consequence of that:
@@ -351,13 +351,13 @@ console for easier diagnostics.
 - ([#427](https://github.com/microsoft/InnerEye-DeepLearning/pull/427)) Move dicom loading function from SimpleITK to
   pydicom. Loading time improved by 30x.
 
-### Fixed
+#### Fixed
 
 - When registering a model, it now has a consistent folder structured, described [here](docs/deploy_on_aml.md). This
   folder structure is present irrespective of using InnerEye as a submodule or not. In particular, exactly 1 Conda
   environment will be contained in the model.
 
-### Removed
+#### Removed
 
 - The commandline options to control which checkpoint is saved, and which is used for inference, have been removed:
   `save_start_epoch`, `save_step_epochs`, `epochs_to_test`, `test_diff_epochs`, `test_step_epochs`, `test_start_epoch`
@@ -367,8 +367,8 @@ console for easier diagnostics.
 - Removed ability to perform sub-fold cross validation. The parameters `number_of_cross_validation_splits_per_fold`
   and `cross_validation_sub_fold_split_index` have been removed from ScalarModelBase.
 
-### Deprecated
+#### Deprecated
 
-## 0.1 (2020-11-13)
+### 0.1 (2020-11-13)
 
 - This is the baseline release.

CHANGELOG.md

@@ -12,7 +12,6 @@ AzureML workspace. A member of the InnerEye team will be happy to assist then.
 by pressing Ctrl+Alt+L, or selecting "Reformat code" in the context menu of the file(s) in the project explorer window.
 Alternatively, you should tick all of "Reformat code", "Rearrange code", "Optimize imports", "Cleanup", "Scan with mypy"
 in the PyCharm version control check-in dialog.
-* Ensure that you modified [CHANGELOG.md](../CHANGELOG.md) and described your PR there.
 * Only publish your PR for review once you have a build that is passing. You can make use of the "Create as Draft"
 feature of GitHub.
 * Link the correct Github issue.

docs/pull_requests.md

@@ -19,6 +19,8 @@
 from pathlib import Path
 repo_dir = Path(__file__).absolute().parents[2]
 sys.path.insert(0, str(repo_dir))
+from InnerEye.Common import fixed_paths
+fixed_paths.add_submodules_to_path()
 
 
 # -- Imports -----------------------------------------------------------------