PI-3483: Ensure collapsed/aggregated_by/rolling_window treat ancillary variables properly #3549

stephenworsley · 2019-11-18T16:47:12Z

Fixes a few issues with #3483. Rellies upon #3548 and #3546 being merged to work properly.

stephenworsley · 2019-11-20T13:15:43Z

This has now been rebased to include #3548 and #3546.

trexfeathers

Hi @stephenworsley. Sorry but I think the tests need more refactoring to fully account for your changes.

lib/iris/cube.py

lib/iris/tests/unit/cube/test_Cube.py

trexfeathers · 2019-11-21T10:07:14Z

Also, since this is a change in behaviour for cell measures, there will need to be a what's new entry

lib/iris/cube.py

trexfeathers

Tests look good! You might have missed it but I also added a comment that there should be a what's new for the changed cell measure behaviour. Thanks 👍

lib/iris/util.py

pp-mo · 2019-11-27T11:11:59Z

lib/iris/util.py

                del cube.attributes[key]
+
+
+def _remove_ancils_and_cms_mapping_dims(cube, dims):


After some consideration, I guess this is a reasonable place for it, as we already have other general-purpose private routines here.
I know it was my proposal, but can you think of a handier short name ?
(Anticipating complaint from @bjlittle as on this one !)

pp-mo · 2019-11-27T12:53:22Z

lib/iris/tests/unit/cube/test_Cube.py

            npts = cube.shape[i_dim]
            coord = DimCoord(np.arange(npts), long_name=name)
            cube.add_dim_coord(coord, i_dim)
+        self.ancillary_variable = AncillaryVariable([0, 1], long_name="foo")


I think adding this to a testclass called specifically Test_collapsed__lazy is a bit cheeky !
The actual (pre-existing) testcases show that this class is specifically to test operation with lazy data.

So I think we should instead add a new testclass to be the home for these new tests.
As we don't have a "Test_collapsed", and we're not even aiming to do that here+now, I would just call it something like 'Test_collapsed__cellmeasure_ancils` + keep it minimal for this purpose.

By contrast, I think the changes to rolling-window and aggregated-by classes below are fine as they are.

pp-mo · 2019-11-27T12:55:11Z

Hi @stephenworsley, reviewed what is left to do, so here is my list of what's outstanding :

pp-mo

Hi @stephenworsley
One more small point came up.
Otherwise all done, great !

docs/iris/src/whatsnew/contributions_3.0.0/bugfix_2019-Nov-27_cell_measure_statistics.txt

pp-mo · 2019-11-28T12:19:27Z

Nice work @stephenworsley, thanks for patience !!

stephenworsley added the Status: Work in Progress label Nov 18, 2019

stephenworsley added this to the v3.0.0 milestone Nov 18, 2019

stephenworsley force-pushed the anc_collapse branch from e111749 to efcaaee Compare November 18, 2019 17:03

This was referenced Nov 20, 2019

Fix copy to include Ancillary variables and Cell measures #3546

Merged

[PI] Cube operations support for Ancillary Variables #3483

Closed

stephenworsley added 2 commits November 20, 2019 13:13

ensure collapsed ancillary variables are removed

d676a0a

added tests

aba9844

stephenworsley force-pushed the anc_collapse branch from efcaaee to aba9844 Compare November 20, 2019 13:14

stephenworsley changed the title ~~Ensure collapsed/aggregated_by/rolling_window treat ancillary variables properly (DO NOT MERGE))~~ Ensure collapsed/aggregated_by/rolling_window treat ancillary variables properly Nov 20, 2019

fix bug

f478cbf

trexfeathers self-assigned this Nov 20, 2019

stephenworsley removed the Status: Work in Progress label Nov 20, 2019

trexfeathers requested changes Nov 20, 2019

View reviewed changes

lib/iris/cube.py Outdated Show resolved Hide resolved

lib/iris/tests/unit/cube/test_Cube.py Outdated Show resolved Hide resolved

lib/iris/tests/unit/cube/test_Cube.py Outdated Show resolved Hide resolved

lib/iris/tests/unit/cube/test_Cube.py Outdated Show resolved Hide resolved

address reviewer comments

902ead8

abooton added the T-Shirt: Small label Nov 21, 2019

pp-mo reviewed Nov 22, 2019

View reviewed changes

lib/iris/cube.py Outdated Show resolved Hide resolved

pp-mo reviewed Nov 22, 2019

View reviewed changes

lib/iris/cube.py Outdated Show resolved Hide resolved

stephenworsley self-assigned this Nov 25, 2019

stephenworsley added 4 commits November 25, 2019 11:08

address review comments

7722f30

fix bug in tests

f64b676

fix bug in tests

c5a72f2

move ancillary collapsing to iris.util

87d3236

trexfeathers self-requested a review November 25, 2019 13:37

trexfeathers requested changes Nov 25, 2019

View reviewed changes

stephenworsley assigned pp-mo Nov 26, 2019

pp-mo reviewed Nov 26, 2019

View reviewed changes

lib/iris/util.py Show resolved Hide resolved

abooton changed the title ~~Ensure collapsed/aggregated_by/rolling_window treat ancillary variables properly~~ PI:3483: Ensure collapsed/aggregated_by/rolling_window treat ancillary variables properly Nov 26, 2019

abooton changed the title ~~PI:3483: Ensure collapsed/aggregated_by/rolling_window treat ancillary variables properly~~ PI3483: Ensure collapsed/aggregated_by/rolling_window treat ancillary variables properly Nov 26, 2019

abooton changed the title ~~PI3483: Ensure collapsed/aggregated_by/rolling_window treat ancillary variables properly~~ PI-3483: Ensure collapsed/aggregated_by/rolling_window treat ancillary variables properly Nov 26, 2019

pp-mo reviewed Nov 27, 2019

View reviewed changes

stephenworsley added 3 commits November 27, 2019 13:29

separate tests

eff7d88

shorten function name

b6a64a1

add whatsnew

cf9de55

pp-mo reviewed Nov 27, 2019

View reviewed changes

docs/iris/src/whatsnew/contributions_3.0.0/bugfix_2019-Nov-27_cell_measure_statistics.txt Show resolved Hide resolved

stephenworsley added 2 commits November 27, 2019 16:19

blackify

5bb507c

change whatsnew file name

7c6f499

pp-mo merged commit 3c4dd16 into SciTools:master Nov 28, 2019

abooton changed the title ~~PI-3483: Ensure collapsed/aggregated_by/rolling_window treat ancillary variables properly~~ GH-3483: Ensure collapsed/aggregated_by/rolling_window treat ancillary variables properly Nov 30, 2019

abooton changed the title ~~GH-3483: Ensure collapsed/aggregated_by/rolling_window treat ancillary variables properly~~ #3483: Ensure collapsed/aggregated_by/rolling_window treat ancillary variables properly Nov 30, 2019

abooton changed the title ~~#3483: Ensure collapsed/aggregated_by/rolling_window treat ancillary variables properly~~ #3483 : Ensure collapsed/aggregated_by/rolling_window treat ancillary variables properly Nov 30, 2019

abooton changed the title ~~#3483 : Ensure collapsed/aggregated_by/rolling_window treat ancillary variables properly~~ PI-3483: Ensure collapsed/aggregated_by/rolling_window treat ancillary variables properly Nov 30, 2019

		del cube.attributes[key]


		def _remove_ancils_and_cms_mapping_dims(cube, dims):

PI-3483: Ensure collapsed/aggregated_by/rolling_window treat ancillary variables properly #3549

PI-3483: Ensure collapsed/aggregated_by/rolling_window treat ancillary variables properly #3549

Uh oh!

Conversation

stephenworsley commented Nov 18, 2019

Uh oh!

stephenworsley commented Nov 20, 2019

Uh oh!

trexfeathers left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

trexfeathers commented Nov 21, 2019

Uh oh!

Uh oh!

Uh oh!

trexfeathers left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pp-mo Nov 27, 2019

Choose a reason for hiding this comment

Uh oh!

pp-mo Nov 27, 2019

Choose a reason for hiding this comment

Uh oh!

pp-mo commented Nov 27, 2019 • edited by stephenworsley Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pp-mo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pp-mo commented Nov 28, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

pp-mo commented Nov 27, 2019 •

edited by stephenworsley

Loading