Turning ripples into waves

Removing spikes from CTD data — Part 2

There are two different types of spikes that may happen in CTD data:

  1. Bad data from electrical failures. Usually it manifest in all the measured variables (Temperature, Conductivity, Oxygen, etc);
  2. Salinity spikes due to a wrong temperature value used when computing it from conductivity. This can happen due to a a bad Alignment of the temperature and conductivity sensors, and/or poor Cell thermal mass correction (Lueck 1990).

How to detect the second case? For a start they do not manifest in the raw data. This kind of spike is a product of bad pre-processing parameters (either Align and/or Cell Thermal Mass). For more information on this check the SBE Software documentation and the Go-SHIP manual

In [2]:
import gsw
from ctd import DataFrame
In [3]:
before = DataFrame.from_cnv('./data/CTD-before.cnv.gz', compression='gzip').split()[0]
after = DataFrame.from_cnv('./data/CTD-after.cnv.gz', compression='gzip').split()[0]
In [4]:
# Compute salinity before.
p = before.index.values.astype(float)
before['SP'] = gsw.SP_from_C(before['c0S/m'].values * 10, before['t090C'].values, p)
In [5]:
# Compute salinity after.
p = after.index.values.astype(float)
after['SP'] = gsw.SP_from_C(after['c0S/m'].values * 10, after['t090C'].values, p)

The file CTD-before.cnv.gz is just the raw data acquired be the CTD. The file CTD-after.cnv.gz had both the Align (-0.01 s and 0.07 s for the primary and secondary conductivity sensors respectively as provided by the calibration lab) and the Cell Thermal Mass ($\alpha = 0.03$ for thermal anomaly amplitude and $\tau = 7$ s for the time constant) corrections applied.

Here is the result of those "corrections":

In [6]:
kw_before = dict(linestyle='-', color='#339933', linewidth=3, label=r"Practical Salinity Before Align and CellTM")
kw_after = dict(linestyle='-', color='#ff3333', alpha=0.65, linewidth=3, label=r"Practical Salinity After Align and CellTM")
fig, ax = before['SP'].plot(**kw_before)
ax.plot(after['SP'], after.index, **kw_after)
ax.set_xlabel("Salinity [g kg$^{-1}$]")
ax.set_ylabel("Pressure [dbar]")
l = ax.legend(loc="lower right")
fig.set_size_inches((5, 7))

Unfortunately most people apply those corrections using the defaults on the SBE Software and then "correct" the bad corrections with averaging, smoothing, and despiking techniques. A much better solution is to estimate these parameters for your CTD system and avoid them altogether.

In [7]:

This post was written as an IPython notebook. It is available for download or as a static html.

Creative Commons License
python4oceanographers by Filipe Fernandes is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Based on a work at