From 4fecb94b2f5c5c622645d4253efd4db7d90b57c6 Mon Sep 17 00:00:00 2001 From: vsalvatelli Date: Mon, 26 Jul 2021 10:26:19 +0100 Subject: [PATCH 1/7] Improving docs --- docs/WSL.md | 5 ----- docs/environment.md | 8 ++++---- docs/hello_world_model.md | 2 ++ 3 files changed, 6 insertions(+), 9 deletions(-) diff --git a/docs/WSL.md b/docs/WSL.md index 9a3ccc355..c0b581ec6 100644 --- a/docs/WSL.md +++ b/docs/WSL.md @@ -63,11 +63,6 @@ Start the Windows Terminal app, create an Ubuntu tab. In the shell, run the foll - Close your WSL shell and re-start it - Clone repo or access your repos via /mnt/c/... - Create conda environment: `conda env create --file environment.yml` -- Create a file in the top level directory of the repository, named `InnerEyeTestVariables.txt`, with one line: -``` -APPLICATION_KEY= -``` -This will enable you to run tests that require authentication to Azure. - Clean your pyc files (in case you have some left from Windows): ``` find * -name '*.pyc' | xargs -d'\n' rm` diff --git a/docs/environment.md b/docs/environment.md index be17b507b..e189f9bb4 100644 --- a/docs/environment.md +++ b/docs/environment.md @@ -16,7 +16,7 @@ submodule. We only recommended that if you are very handy with Python. More deta When developing on a Windows machine, we recommend using [the Windows Subsystem for Linux, WSL2](https://docs.microsoft.com/en-us/windows/wsl/about). That's because PyTorch has better support for Linux. -If you want to use WSL, please follow [these instructions](/docs/WSL.md) +If you want to use WSL2, please follow [these instructions](/docs/WSL.md) ## Installing Conda or Miniconda You can skip this step if you have installed WSL as per the previous item. @@ -125,9 +125,9 @@ point, if you want to dig deeper into the code. Our team uses [PyCharm](https://www.jetbrains.com/pycharm/) for development, but any good editor ([VSCode](https://code.visualstudio.com/) for example) will do as well. -This repository already contains a PyCharm configuration file in `.idea/InnerEye-DeepLearning.iml`. It will pick the -WSL Python interpreter (see [WSL.md](WSL.md)) as the default - based on your configuration, you will need to adjust -that as described [here](https://www.jetbrains.com/help/pycharm/configuring-python-interpreter.html). +This repository already contains a PyCharm configuration file in `.idea/InnerEye-DeepLearning.iml`. It should +automatically pick the WSL Python interpreter (see [WSL.md](WSL.md)) as the default (no need to import the settings file) +- if it doesn't happen you will need to adjust that as described [here](https://www.jetbrains.com/help/pycharm/configuring-python-interpreter.html). ## How to manually set up flake8 as a PyCharm external tool diff --git a/docs/hello_world_model.md b/docs/hello_world_model.md index 6b5c873cc..f19f36a80 100644 --- a/docs/hello_world_model.md +++ b/docs/hello_world_model.md @@ -9,6 +9,8 @@ We have created this file to demonstrate how to: * This model can be trained from the commandline, from the root of the repo: `python InnerEye/runner.py --model=HelloWorld` * If you want to test your AzureML workspace with the HelloWorld model: + * Make sure your AzureML workspace has been set up. You should have a settings.yml file inside the InnerEye folder + that specifies things like the datastore, the resource group and the workspace on which to run * Upload to datasets storage account for your AzureML workspace: `Tests/ML/test_data/dataset.csv` and `Test/ML/test_data/train_and_test_data` and name the folder "hello_world" * If you have set up AzureML then parameter search can be performed for this model by running: From e2274d0e00dc0c8ad5a2a5d2075b237dda2de28f Mon Sep 17 00:00:00 2001 From: vsalvatelli Date: Mon, 26 Jul 2021 14:31:24 +0100 Subject: [PATCH 2/7] adding workspace name to the info logging --- InnerEye/Azure/azure_config.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/InnerEye/Azure/azure_config.py b/InnerEye/Azure/azure_config.py index 336b29e56..3c32e2a04 100755 --- a/InnerEye/Azure/azure_config.py +++ b/InnerEye/Azure/azure_config.py @@ -251,8 +251,8 @@ def get_or_create_dataset(self, azure_dataset_id: str) -> FileDataset: raise ValueError("No value set for 'azureml_datastore' (name of the datastore in the AzureML workspace)") if not azure_dataset_id: raise ValueError("No dataset ID provided.") - logging.info(f"Retrieving datastore '{self.azureml_datastore}' from AzureML workspace") workspace = self.get_workspace() + logging.info(f"Retrieving datastore '{self.azureml_datastore}' from AzureML workspace {workspace.name}") datastore = Datastore.get(workspace, self.azureml_datastore) try: logging.info(f"Trying to retrieve AzureML Dataset '{azure_dataset_id}'") From c1611e9641f7b89834cfa650def164faa520f9ec Mon Sep 17 00:00:00 2001 From: vsalvatelli Date: Mon, 26 Jul 2021 16:10:31 +0100 Subject: [PATCH 3/7] ready for review --- CHANGELOG.md | 3 ++- docs/environment.md | 5 ++--- docs/hello_world_model.md | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 35f86f040..08bf8d995 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -34,7 +34,8 @@ gets uploaded to AzureML, by skipping all test folders. ### Fixed - ([#537](https://github.com/microsoft/InnerEye-DeepLearning/pull/537)) Print warning if inference is disabled but comparison requested. -- ([#525](https://github.com/microsoft/InnerEye-DeepLearning/pull/525)) Enable --store_dataset_sample +- ([#546](https://github.com/microsoft/InnerEye-DeepLearning/pull/537)) Environment and hello_world_model documentation updated +([#525](https://github.com/microsoft/InnerEye-DeepLearning/pull/525)) Enable --store_dataset_sample - ([#495](https://github.com/microsoft/InnerEye-DeepLearning/pull/495)) Fix model comparison. - ([#482](https://github.com/microsoft/InnerEye-DeepLearning/pull/482)) Check bool parameter is either true or false. - ([#475](https://github.com/microsoft/InnerEye-DeepLearning/pull/475)) Bug in AML SDK meant that we could not train diff --git a/docs/environment.md b/docs/environment.md index e189f9bb4..afce4c595 100644 --- a/docs/environment.md +++ b/docs/environment.md @@ -14,9 +14,8 @@ submodule. We only recommended that if you are very handy with Python. More deta ## Windows Subsystem for Linux Setup When developing on a Windows machine, we recommend using [the Windows Subsystem for Linux, WSL2](https://docs.microsoft.com/en-us/windows/wsl/about). -That's because PyTorch has better support for Linux. - -If you want to use WSL2, please follow [these instructions](/docs/WSL.md) +That's because PyTorch has better support for Linux. If you want to use WSL2, please follow +[these instructions](/docs/WSL.md) , that correspond to the manual installation in the official docs. ## Installing Conda or Miniconda You can skip this step if you have installed WSL as per the previous item. diff --git a/docs/hello_world_model.md b/docs/hello_world_model.md index f19f36a80..10cb56ab4 100644 --- a/docs/hello_world_model.md +++ b/docs/hello_world_model.md @@ -9,8 +9,8 @@ We have created this file to demonstrate how to: * This model can be trained from the commandline, from the root of the repo: `python InnerEye/runner.py --model=HelloWorld` * If you want to test your AzureML workspace with the HelloWorld model: - * Make sure your AzureML workspace has been set up. You should have a settings.yml file inside the InnerEye folder - that specifies things like the datastore, the resource group and the workspace on which to run + * Make sure your AzureML workspace has been set up. You should have inside the folder InnerEye a settings.yml file + that specifies the datastore, the resource group, and the workspace on which to run * Upload to datasets storage account for your AzureML workspace: `Tests/ML/test_data/dataset.csv` and `Test/ML/test_data/train_and_test_data` and name the folder "hello_world" * If you have set up AzureML then parameter search can be performed for this model by running: From 8b3bb2452643665515d70490398e5664d2000f59 Mon Sep 17 00:00:00 2001 From: vsalvatelli Date: Mon, 26 Jul 2021 16:23:31 +0100 Subject: [PATCH 4/7] fix typo in the changelog --- .idea/InnerEye-DeepLearning.iml | 2 +- CHANGELOG.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.idea/InnerEye-DeepLearning.iml b/.idea/InnerEye-DeepLearning.iml index b44301926..6a024f8f4 100644 --- a/.idea/InnerEye-DeepLearning.iml +++ b/.idea/InnerEye-DeepLearning.iml @@ -5,7 +5,7 @@ - + diff --git a/CHANGELOG.md b/CHANGELOG.md index 08bf8d995..0fb204854 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -35,7 +35,7 @@ gets uploaded to AzureML, by skipping all test folders. ### Fixed - ([#537](https://github.com/microsoft/InnerEye-DeepLearning/pull/537)) Print warning if inference is disabled but comparison requested. - ([#546](https://github.com/microsoft/InnerEye-DeepLearning/pull/537)) Environment and hello_world_model documentation updated -([#525](https://github.com/microsoft/InnerEye-DeepLearning/pull/525)) Enable --store_dataset_sample +- ([#525](https://github.com/microsoft/InnerEye-DeepLearning/pull/525)) Enable --store_dataset_sample - ([#495](https://github.com/microsoft/InnerEye-DeepLearning/pull/495)) Fix model comparison. - ([#482](https://github.com/microsoft/InnerEye-DeepLearning/pull/482)) Check bool parameter is either true or false. - ([#475](https://github.com/microsoft/InnerEye-DeepLearning/pull/475)) Bug in AML SDK meant that we could not train From 4cbabe054ad03cb02fd4bbc85134341eb6b2d791 Mon Sep 17 00:00:00 2001 From: vsalvatelli Date: Mon, 26 Jul 2021 16:29:10 +0100 Subject: [PATCH 5/7] fix wrong link in the changelog --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0fb204854..6f15d4523 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -34,7 +34,7 @@ gets uploaded to AzureML, by skipping all test folders. ### Fixed - ([#537](https://github.com/microsoft/InnerEye-DeepLearning/pull/537)) Print warning if inference is disabled but comparison requested. -- ([#546](https://github.com/microsoft/InnerEye-DeepLearning/pull/537)) Environment and hello_world_model documentation updated +- ([#546](https://github.com/microsoft/InnerEye-DeepLearning/pull/546)) Environment and hello_world_model documentation updated - ([#525](https://github.com/microsoft/InnerEye-DeepLearning/pull/525)) Enable --store_dataset_sample - ([#495](https://github.com/microsoft/InnerEye-DeepLearning/pull/495)) Fix model comparison. - ([#482](https://github.com/microsoft/InnerEye-DeepLearning/pull/482)) Check bool parameter is either true or false. From d79470c972b1722cbe219734b29ddb3e10612358 Mon Sep 17 00:00:00 2001 From: vsalvatelli Date: Mon, 26 Jul 2021 17:42:30 +0100 Subject: [PATCH 6/7] reverting dash in InnerEye-DeepLearning.iml --- .idea/InnerEye-DeepLearning.iml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.idea/InnerEye-DeepLearning.iml b/.idea/InnerEye-DeepLearning.iml index 6a024f8f4..72ba525ba 100644 --- a/.idea/InnerEye-DeepLearning.iml +++ b/.idea/InnerEye-DeepLearning.iml @@ -5,7 +5,7 @@ - + @@ -14,4 +14,4 @@ - \ No newline at end of file + From 3d28d2bab47e929d5e2ed8e3147f274d3a17152d Mon Sep 17 00:00:00 2001 From: vsalvatelli Date: Mon, 26 Jul 2021 17:46:55 +0100 Subject: [PATCH 7/7] restoring .idea/InnerEye-DeepLearning.iml from main --- .idea/InnerEye-DeepLearning.iml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.idea/InnerEye-DeepLearning.iml b/.idea/InnerEye-DeepLearning.iml index 72ba525ba..b44301926 100644 --- a/.idea/InnerEye-DeepLearning.iml +++ b/.idea/InnerEye-DeepLearning.iml @@ -14,4 +14,4 @@ - + \ No newline at end of file