[PI] Ensure flags are correctly loaded/saved to/from NetCDF

[lbdreyer]

Part of #3358

According to CF, flags contain flag_values, flag_masks and flag_meanings attributes.

This ticket is to check whether flags require any special handling to ensure that they are correctly loaded/saved.

Additional, 2019-12-12
(from @pp-mo)
Following the "CF-metadata" email list, it seems to me that some of the expected forms and vocabulary usages for flags still rather fluid + changing,
That is, maybe despite statements already accepted in the cf 1.7 document
See cf-convention/cf-conventions#216 in particular,
which is also the issue referenced by this comment from @stephenworsley below

---

Acceptance Criteria

• Decide criteria for identifying a flag variable which should be unitless (eg. standard name, standard name modifier)
• When loading such a flag variable, the units in iris should be "no_unit" (see [FB] [PI-3474] (WIP) Ensure flags load/save without units #3613)
• Loading then saving should result in a flag variable without units (see [PI] Aligning save behaviour of cube with units "unknown" and "no-unit" #3394)
• Added tests for loading a NetCDF file with a flag variable (see [FB] [PI-3474] (WIP) Ensure flags load/save without units #3613)
• Added tests for saving a cube with a flag Ancillary Variable to NetCDF file (see [FB] [PI-3474] Add tests for the loading and saving of ancillary variables #3799)

[alastair-gemmell]

This ticket is best tackled after #3473

[stephenworsley]

The convention with status_flags is to be unitless. An example of a status flag being described within CF conventions is given here: cf-convention/cf-conventions#216

variables:

    float sea_water_practical_salinity(time, z);
        sea_water_practical_salinity:units = "1";
        sea_water_practical_salinity:long_name = "Salinity";
        sea_water_practical_salinity:standard_name = "sea_water_practical_salinity";
        sea_water_practical_salinity:ancillary_variables = "sea_water_practical_salinity_qc_agg sea_water_practical_salinity_qc_flat_line_test";

    int sea_water_practical_salinity_qc_agg(time, z);
        sea_water_practical_salinity_qc_agg:long_name = "Salinity QARTOD Aggregate Flag";
        sea_water_practical_salinity_qc_agg:standard_name = "status_flag qartod_aggregate_quality_flag";
        sea_water_practical_salinity_qc_agg:missing_value = 2;
        sea_water_practical_salinity_qc_agg:flag_meanings = "PASS NOT_EVALUATED SUSPECT FAIL MISSING";
        sea_water_practical_salinity_qc_agg:flag_values = 1, 2, 3, 4, 9;

    int sea_water_practical_salinity_qc_flat_line_test(time, z);
        sea_water_practical_salinity_qc_flat_line_test:long_name = "Salinity QARTOD Flat Line Test Flag";
        sea_water_practical_salinity_qc_flat_line_test:standard_name = "status_flag qartod_flat_line_test_quality_flag";
        sea_water_practical_salinity_qc_flat_line_test:missing_value = 2;
        sea_water_practical_salinity_qc_flat_line_test:flag_meanings = "PASS NOT_EVALUATED SUSPECT FAIL MISSING";
        sea_water_practical_salinity_qc_flat_line_test:flag_values = 1, 2, 3, 4, 9;

When such a file is loaded by Iris, the current behaviour is to assign the unit to be "1" (In fact, this is the behaviour when using iris to directly instatiate an AncillaryVariable without specifying the unit). As such, a round trip (loading then saving) would result in a unit being assigned to the status_flag against current convention.
It seems as though the unit convention of status_flags in CF conventions is described by the standard name modifier. See #3408 for a discussion on handling of standard name modifiers.
Also, see #3394 with respect to saving without a unit.

Additional 2019-12-12
(@pp-mo)
Caution: the content of cf-convention/cf-conventions#216 has changed again since this was written. The ways in which such status flags + related info are stored and indicated are still evolving ...

[stephenworsley]

In order to preserve unitless cf variables, it may be worthwhile changing the general behaviour defined here:

iris/lib/iris/fileformats/_pyke_rules/fc_rules_cf.krb

Lines 1654 to 1656 in 242a62f

	attr_units = getattr(cf_var, CF_ATTR_UNITS, cf_units._UNIT_DIMENSIONLESS)
	if not attr_units:
	attr_units = '1'

Changing cf_units._UNIT_DIMENSIONLESS to "no_unit" or cf_units._NO_UNIT_STRING, combined with fixing #3394, ought to make round tripping consistent.

---

As an alternative to the general approach above, flag variables could be given special treatment, similar to that in the following line:

iris/lib/iris/fileformats/_pyke_rules/fc_rules_cf.krb

Lines 1673 to 1674 in 242a62f

	if np.issubdtype(cf_var.dtype, np.str_):
	attr_units = cf_units._NO_UNIT_STRING

The cases in which units would be set to "no_unit" could be decided by some combination of standard name, standard name modifier, and presence of flag values and meanings in the variable.