From 8a09b69b32847f7e2065659357b976936ef509c3 Mon Sep 17 00:00:00 2001 From: Nicki Skafte Date: Tue, 27 Oct 2020 11:44:56 +0100 Subject: [PATCH 1/3] warning on states --- docs/source/metrics.rst | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/docs/source/metrics.rst b/docs/source/metrics.rst index dadf654b1ef67..79c74b9dcc177 100644 --- a/docs/source/metrics.rst +++ b/docs/source/metrics.rst @@ -78,7 +78,6 @@ If ``on_epoch`` is True, the logger automatically logs the end of epoch metric v self.valid_acc(logits, y) self.log('valid_acc', self.valid_acc, on_step=True, on_epoch=True) - This metrics API is independent of PyTorch Lightning. Metrics can directly be used in PyTorch as shown in the example: .. code-block:: python @@ -105,6 +104,13 @@ This metrics API is independent of PyTorch Lightning. Metrics can directly be us # total accuracy over all validation batches total_valid_accuracy = valid_accuracy.compute() +.. warning:: + + Metrics have internal states that keeps track of the data seen so far. Too + not mix metric states across training, validation and testing it is + highly recommended to re-initialize the metric per mode as shown in the + examples above. + ********************* Implementing a Metric ********************* From 609c0bb9970f2b702451bad7f7f2ba11e72279ce Mon Sep 17 00:00:00 2001 From: Nicki Skafte Date: Tue, 27 Oct 2020 12:52:54 +0100 Subject: [PATCH 2/3] suggestion --- docs/source/metrics.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/source/metrics.rst b/docs/source/metrics.rst index 79c74b9dcc177..73843c20e93cc 100644 --- a/docs/source/metrics.rst +++ b/docs/source/metrics.rst @@ -106,10 +106,10 @@ This metrics API is independent of PyTorch Lightning. Metrics can directly be us .. warning:: - Metrics have internal states that keeps track of the data seen so far. Too - not mix metric states across training, validation and testing it is - highly recommended to re-initialize the metric per mode as shown in the - examples above. + Metrics contain internal states that keep track of the data seen so far. + Do not mix metric states across training, validation and testing. + It is highly recommended to re-initialize the metric per mode as + shown in the examples above. ********************* Implementing a Metric From a7e009eb392e87bc7824386346f075e79281a829 Mon Sep 17 00:00:00 2001 From: Nicki Skafte Date: Wed, 28 Oct 2020 11:05:29 +0100 Subject: [PATCH 3/3] Update metrics.rst --- docs/source/metrics.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/metrics.rst b/docs/source/metrics.rst index 73843c20e93cc..cf7dcfbecc183 100644 --- a/docs/source/metrics.rst +++ b/docs/source/metrics.rst @@ -104,7 +104,7 @@ This metrics API is independent of PyTorch Lightning. Metrics can directly be us # total accuracy over all validation batches total_valid_accuracy = valid_accuracy.compute() -.. warning:: +.. note:: Metrics contain internal states that keep track of the data seen so far. Do not mix metric states across training, validation and testing.