Merge and concatenate don't stack up non-scalar data elements #3603
Description
It is rarer to merge cubes from different CF sources (files) than for other file formats, but it is sometimes needed. Most commonly, when each single timepoint in saved in a separate file.
In those cases, we may well have additional information, such as auxiliary coordinates, that "should" be stacked up in the new dimension along with the data.
See #3084
This was also noticed when adding support for ancillary variables, for which this could be a particular problem : Especially for 'quality flags', which would often be provided as 1-to-1 with data points.
Merge has no automatic way of doing this (#3084), and CF doesn't help to decide which elements "might" be intended to extend into the new dimension.
At present it will (fetch and) compare the data in the arrays -- itself maybe not that great an idea? -- and allow merging if they are all the same.
It "could" also choose to stack-up source elements with different data (but identical shapes).
All of that could also be done with cell-measures and auxiliary variables.
Or... we might decide that equailty-checking multidimensional data is a bad idea, and we will just stack these always.
Or ... we might take the view that as coordinate data is "about the sampling grid", we don't expect to have that information split across files, so we will not do this.
In that case, I think similar logic would apply to cell-measures, but ancillaries could still be stacked as just described.
Again, with ancillaries, we might decide to never check the data, but just always stack them
Concatenate can already combine this type of data (at least, once #3566 is done),
i.e. it can concatenate any dimensional-metadata items that map to the concatenated dimension.
However, it is not terribly easy to get your data into the required form.
Hence proposed improvements to iris.util.new_axis #3600, which should provide a possible solution.