How to search for the nearest non nan value in 2d xarray dataset

I have a 2d xarray dataset and I am having several nan values in the dataset. These nan values need to be filled with the nearest non-nan values in the 2d array. How do I do this?

I am trying to make my data consistent with the grid_spec file that the earth system model reads. And there are inconsistencies on what the land points are in the forcing data and grid_spec file leading to the blowing up of model.

• Can you post the code you have so far? – Craeft Apr 2 '20 at 0:31
• @ManmeetSIngh I think this would get better tracking on SO rather than here. – gansub Apr 2 '20 at 1:01
• Look up Numpy .isnan(<your element or array>). – user14859 Apr 3 '20 at 9:05
• Or try using a masked array (mask where nan) – user14859 Apr 3 '20 at 9:06
• How do you define distance? – Spencer Apr 3 '20 at 13:19

If I were you, I would exclude the NaN values and then perform gridding on the resulting irregularly spaced data.

There are already the tools to perform this in Python and using the library SciPy

But please document exactly what step you are doing and why. Excluding data is always ... interpreting the data.

I was able to do this by the following functions

def fill_mismatch(ds_grid_spec, ds_data):
ds_data_o = ds_data.copy()
count = 0.0
for i in range(ds_grid_spec.shape[0]):
for j in range(ds_grid_spec.shape[1]):
if ds_grid_spec[i,j].values > 0.0 and (not ds_data[i,j] > 0.0):
count = count + 1.0
fill_miss2d(ds_data_o, i, j)
if ds_grid_spec[i,j].values == 0.0 and (not ds_data[i,j].values == 0.0):
count = count + 1.0
ds_data_o[i,j] = 0.0
return ds_data_o

def fill_miss2d(ds, lat_i, lon_j):
# fist cycle
cycle = 1

while True:
bl_i, bl_j = lat_i - cycle, lon_j - cycle
tr_i, tr_j = lat_i + cycle, lon_j + cycle
# Bottom left i
if bl_i < 0:
bl_i = 0
# Bottom left j
if bl_j < 0:
bl_j = 0
# Top right i
if tr_i > ds.lat.shape[0]-1:
tr_i = ds.lat.shape[0]-1
# Top right j
if tr_j > ds.lon.shape[0]-1:
tr_j = ds.lon.shape[0]-1

lats_i = np.arange(bl_i,tr_i)
lons_j = np.arange(bl_j,tr_j)
for lats_i_idx in lats_i:
for lons_j_idx in lons_j:
if ds[lats_i_idx, lons_j_idx]>0.0:
ds[lat_i, lon_j] = ds[lats_i_idx, lons_j_idx]
return
cycle = cycle + 1


The grid_spec (which is the most commonly used grid exchange file for Modular Ocean Model MOM) had a variable AREA_LND and the file soiltype.nc had a variable soiltype. Both these variables had 0.0 at the grid points classified as ocean. Hence we dont need to use fill_miss2d where a mismatch occurs and ocean is encountered in grid_spec, but land is there in soiltype. Rather we directly fill the soiltype grid point with 0.0

The complete notebook can be found at https://github.com/manmeet3591/python_class/blob/master/xarray_tutorial/grid_soiltype.ipynb