Not saving first dimension as grib and renaming sea_water_y_velocity to sea_water_x_velocity about iris-grib

[jweisbaum]

Friendly ping on this. Any ideas?

[jweisbaum]

I'm updating to 0.17.1 and 3.0.2 and I'll report back.

[jweisbaum]

Nope this is still an issue.

[jweisbaum]

Also when writing it is changing my fill values to 1+e20 is there a way to specify not to do this?

[pp-mo]

Sorry I should probably have replied to this, but I'm short on time at the moment.
Very briefly, though (if it helps) ..

(1) Regarding the time dimension
When you write data to GRIB each message is a 2D field. If you save multiple fields whcih differ only in time, then on loading back iris will convert each field into a 2D cube (what we call a "raw cube"), and then merge them to create a time dimension.
But in this case there is only 1 timepoint (= one message) so there is nothing to merge and Iris won't create a time dimension.

Netcdf is different because the file format itself represents multidimensional variables, so it can represent the difference between data with a scalar time coord and with a time dimension of only 1 point.

You can convert a scalar time coord to a dimension with iris.util.new_axis.
The converse, of course is just to take cube[0], turning the dim-coord into a scalar coord.

(2) Regarding the phenomenon
Your original data seems to have a conflict between the standard name of "sea_water_y_velocity" and the GRIB_PARAM attribute of "GRIB2:d010c001n002" -- which I think is 'x_velocity'
At least, in our code tables we have :

    CFName('sea_water_x_velocity', None, 'm s-1'): G2Param(2, 10, 1, 2),
    CFName('sea_water_y_velocity', None, 'm s-1'): G2Param(2, 10, 1, 3),

I guess that is correct, but I haven't checked against the latest GRIB spec.

Anyway, the key point is that a GRIB_PARAM attribute will define what discipline/category/parameter values are actually written into a saved GRIB message, which overrides any translation derived from the standard_name as noted here, so that is what has happened here.

(3) Regarding the fill-value
This is a thorny problem.
A GRIB message can encode missing data points, but it does not store any fill-value at all (it records missing points in a different way).
So, when Iris-grib writes a grib message with missing data, the fill value is simply lost.
Then, when Iris-grib reads a grib message with missing points, it constructs a numpy masked array with the usual default fill-value for its dtype.

In fact the fill-value of a numpy arrays doesn't actually mean much : you can just change it, and its only real use is to be the default value used by an "array.filled()" call. Numpy calculations don't preserve it either -- it goes back to the default whenever a new array is produced to store a result.
( If you know about dask, which is important to Iris, this also has problems : fill-value is not a stable property of lazy arrays; different sections of the same array will be masked or unmasked according to whether any masked data is contained, so of course a fill-value does not exist for unmasked portions).
For all those reasons, Iris doesn't even preserve the fill-values of numpy array in a netcdf save : you can control it, but the default is not taken from the data array. We have a documentation section about that.

Sorry this is such a brain-dump, I'm trying to be quick.
Hope some of this is of use ...

[jweisbaum]

The support for this repo is honestly fantastic. Thanks so much for this, it does give me quite a bit to chase down.

[pp-mo]

Many thanks for your appreciation.
Really pleased if it helps !