1
$\begingroup$

I have a 2d xarray dataset and I am having several nan values in the dataset. These nan values need to be filled with the nearest non-nan values in the 2d array. How do I do this?

I am trying to make my data consistent with the grid_spec file that the earth system model reads. And there are inconsistencies on what the land points are in the forcing data and grid_spec file leading to the blowing up of model.

$\endgroup$
  • 2
    $\begingroup$ Can you post the code you have so far? $\endgroup$ – Craeft Apr 2 at 0:31
  • 1
    $\begingroup$ @ManmeetSIngh I think this would get better tracking on SO rather than here. $\endgroup$ – gansub Apr 2 at 1:01
  • 1
    $\begingroup$ Look up Numpy .isnan(<your element or array>). $\endgroup$ – user14859 Apr 3 at 9:05
  • $\begingroup$ Or try using a masked array (mask where nan) $\endgroup$ – user14859 Apr 3 at 9:06
  • 1
    $\begingroup$ How do you define distance? $\endgroup$ – Spencer Apr 3 at 13:19
2
$\begingroup$

If I were you, I would exclude the NaN values and then perform gridding on the resulting irregularly spaced data.

There are already the tools to perform this in Python and using the library SciPy

https://scipy-cookbook.readthedocs.io/items/Matplotlib_Gridding_irregularly_spaced_data.html

But please document exactly what step you are doing and why. Excluding data is always ... interpreting the data.

| improve this answer | |
$\endgroup$
0
$\begingroup$

I was able to do this by the following functions

def fill_mismatch(ds_grid_spec, ds_data):
    ds_data_o = ds_data.copy()
    count = 0.0
    for i in range(ds_grid_spec.shape[0]):
        for j in range(ds_grid_spec.shape[1]):
            if ds_grid_spec[i,j].values > 0.0 and (not ds_data[i,j] > 0.0):
                count = count + 1.0
                fill_miss2d(ds_data_o, i, j)
            if ds_grid_spec[i,j].values == 0.0 and (not ds_data[i,j].values == 0.0):
                count = count + 1.0
                ds_data_o[i,j] = 0.0
    return ds_data_o

def fill_miss2d(ds, lat_i, lon_j):
# fist cycle
    cycle = 1

    while True:
        bl_i, bl_j = lat_i - cycle, lon_j - cycle
        tr_i, tr_j = lat_i + cycle, lon_j + cycle
        # Bottom left i
        if bl_i < 0:
            bl_i = 0
        # Bottom left j
        if bl_j < 0:
            bl_j = 0
        # Top right i
        if tr_i > ds.lat.shape[0]-1:
            tr_i = ds.lat.shape[0]-1
        # Top right j
        if tr_j > ds.lon.shape[0]-1:
            tr_j = ds.lon.shape[0]-1

        lats_i = np.arange(bl_i,tr_i)
        lons_j = np.arange(bl_j,tr_j)
        for lats_i_idx in lats_i:
            for lons_j_idx in lons_j:
                if ds[lats_i_idx, lons_j_idx]>0.0:
                    ds[lat_i, lon_j] = ds[lats_i_idx, lons_j_idx]
                    return
        cycle = cycle + 1

The grid_spec (which is the most commonly used grid exchange file for Modular Ocean Model MOM) had a variable AREA_LND and the file soiltype.nc had a variable soiltype. Both these variables had 0.0 at the grid points classified as ocean. Hence we dont need to use fill_miss2d where a mismatch occurs and ocean is encountered in grid_spec, but land is there in soiltype. Rather we directly fill the soiltype grid point with 0.0

The complete notebook can be found at https://github.com/manmeet3591/python_class/blob/master/xarray_tutorial/grid_soiltype.ipynb

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.