DATA EDITING AND FILTERING

Before any postflight data analysis for parameter estimation and related work is carried out, it would be desirable to preprocess the data to remove spurious spikes

and any high-frequency noise: (1) edit the data to remove wild points and replace the missing points (often an average from two samples is used), (2) filter the edited/raw data to reduce the effect of noise, and (3) decimate the data to obtain the data at the required sampling rate for further postprocessing; for this the data should have been filtered at a higher rate than required. Editing by using the finite – difference method and filtering using FFT (fast Fourier transform)/spectral analy­sis can be done. The editing is a process of removing the data spikes (wild points) and replacing them with suitable data points. Wild points occurring singly can be detected using the slope of the data set or first finite difference. Any data point exhibiting higher than the prefixed slope is considered as a wild point and eliminated. When wild points occur in groups, the surrounding points are con­sidered to detect them. A finite-difference array consisting of first-, second-, and third-difference (up to nth differences) is formed. The maximum allowable upper and lower limits for these differences are pre-specified and if the array indicates any value greater than the limits, the points are treated as wild points and eliminated. The points are replaced by suitable points by interpolation, considering surrounding points. If the editing limits are too high, the edited data will leave a large amount of wild points, and if the limit is too small, the edited data would appear distorted.

Filtering is the process of removing the noise components presented in the edited/raw data. These filters introduce the time lag effects in the flight data, thereby compounding the problem of parameter estimation. Discrete Fourier transform (DFT) method allows processing from time domain to frequency domain and does not introduce time lags. However, it is an offline procedure. Based on spectral analysis through DFT, one can know the frequency contained in the raw signal. Nowadays, filtering/editing can be easily carried out using certain functions from the MATLAB signal processing tool box. However, certain fundamental aspects are described briefly here.

The Fourier transform (FT) from time to frequency domain is given by

Here, h(t) is the time function of the signal, H(t) is the complex function in frequency domain, and N is the number of samples.

The inverse FT (from frequency to time domain) is given by

h(t)	(C4)

Here, H*(f) is the conjugate of H(f), and * is for the conjugate operation. By comparing Equations C2 and C4, it can be seen that the same transformation routine can be used. After the inverse transformation, the conjugate is not necessary because the data in time domain are real. The signal to be filtered is first transformed into frequency domain using DFT. Using cutoff frequency, the Fourier coefficients of the unwanted frequencies are set to zero. Inverse FT time domain yields the filtered signal. For proper use of the filtering method, selection of the cutoff frequency is crucial. In spectral analysis method, the power spectral density of the signal is plotted against frequency and the inspection of the plot helps distinguish the frequency contents of the signal. This information is used in selecting the proper cutoff frequency for the filter. The FT of the correlation functions are often used in analysis. The FT of the autocorrelation function (ACF) 1

fxx(T) exp(—jvT) dT

— 1

is called the power spectral density function of the random process x(t). The ‘‘power’’ term here is used in a generalized sense and indicates the expected squared value of the members of the ensemble. fxx(v) is indeed the spectral distribution of power density for x(t) in that integration of fxx(v) over frequencies in the band from v1 to v2 yields the mean-squared value of the process, whose ACF consists only of those harmonic components of fxx(T) that lie between these frequencies. The mean – squared value of x(t) is given by the integration of the power density spectrum for the random process over the full range of frequencies:

—1