I have 250 or so soil sensors that I need to QC. They monitor various parameters such as soil temp, soil water content, soil co2, and soil heat flux. It isn't unusual for a sensor to behave poorly due to elk chewing up wires a little bit, electrical noise, and what not.

I already use some basic scripts to filter out based on moving averages, standard deviation, and physical constraints. Sometimes there are sensors that still have bad data, such as this soil water. It looks like there is still a viable signal in that noise that is traced out by a bunch of local minima, and I've confirmed this pattern with nearby sensors.

I've played around with some finite-impulse response filters in MATLAB to try to remove the noise, but I don't really know how to design one of these at all.

Any suggestions, resources, or other filters that could help with QCing this?

Soil water content at 5 cm

  • $\begingroup$ Maybe the simplest thing would be to do something like a moving average but use the minimum value rather than average. Do you understand the reason why the sources of error lead to unreasonably high values? That would help justify your filter. You might also check the literature on processing eddy covariance data - they have to filter aggressively to retrieve the usable signal. $\endgroup$
    – haresfur
    Sep 24, 2017 at 22:44
  • $\begingroup$ We were unable to figure out why there were months of high values. The probes are controlled by a CR23x datalogger, which had some program changes around the time when the data got noisy. This probe is alright in 2017. $\endgroup$
    – Lou-gazi
    Sep 25, 2017 at 17:02


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.