NetCDF global attributes vs data variable local attributes

[bjlittle]

Update Mon 3rd July:
this effort is now handled in it's own project
please see there for existing task breakdown + progress

---

At the moment iris takes a rational but naive approach to dealing with the local attributes of a NetCDF variable (a variable that becomes a cube) and the global attributes of the NetCDF file that the said variable comes from.

That is, the resultant cube.attributes will be a combination of both the local and global attributes, where the local attributes will take precedence, and overwrite, common global attributes.

From the inception of iris, and in the light of no use cases, this seemed like a reasonable thing to do. However, such an approach prevents preservation of the local and global attributes metadata. This is a major issue for many users, who require to preserve all attribute metadata.

We require to resolve this issue now in iris once and for all 😄

Note that, if a solution to this issue was implemented, then it would most likely be a breaking change - caution is needed here.

This is somewhat tangential related to #2352

[bjlittle]

Ping @zklaus 👍

[zklaus]

A non-breaking way to implement that may be to introduce two new class members global_attributes and local_attributes (or var_attributes) responsible for the respective attributes with a property replacing the current attributes that can mimic the current behaviour for reading, overwrite existing entries in the respective dictionary, and otherwise defaulting to the local one.

[bjlittle]

@zklaus I was thinking along the same lines. In summary:

• cube.local_attributes is a mutable dictionary for defining the attributes that are local to the associated NetCDF data variable of the cube when it's written to a NetCDF file
• cube.global_attributes is a mutable dictionary for defining the attributes that are global to the associated NetCDF file that the data variable of the cube is written to
• cube.attributes is a (stateless) combination of the cube.global_attributes and cube.local_attributes, akin to now, where the cube.local_attributes have priority over the cube.global_attributes

So for the case where there is a common attribute shared between cube.global_attributes and cube.local_attributes, the local value is shown in the cube.attributes. When saving such a cube to NetCDF, then the common local and global attribute is preserved (hoo-rah) i.e. the local on the NetCDF data variable of the cube, and the global in the global scope of the NetCDF file.

To ensure that there is a non-breaking behaviour here, then I think that I'm right in saying that if a user writes to the cube.attributes then this state is captured in the cube.global_attributes. Note that, cube.attributes is stateless, in the sense that it is simply derived on the fly at run-time from both cube.local_attributes and cube.global_attributes, with local having priority over global. This means that if a user wants to associate an attribute to the NetCDF data variable of the cube, then they must explicitly add the attribute to the cube.local_attributes, and not the cube.attributes.

Hmmm.... this make sense to me. Thoughts?

[zklaus]

Exactly what I was thinking!

[jonseddon]

We bumped into this ticket while looking at a related project. Can I check what would happen when you save a cubelist rather than a cube? If cubes in the cubelist had different values of the same global attribute, how would this be saved? If this saved netCDF file was loaded back into Iris would we get the same cubelist and if we didn't, would this matter?