Derivative Filter

Main functions

class SignalCreation.Treatment.DerivativeFilter.DerivativeFilter(filter_name, creation_mode=None, fcut=None, atten=None, **kwargs)

Class used to create, store and get data on a derivative filter. It is necessarily associated to a LidarSignal object

After its initialization, the class contain an attribute _filter_coeff, which is a pandas data frame storing having altitudes as index and the coefficients of the filter used at those altitudes. For optimisation purposes, only altitudes at which there is a change of filter are kept in the data frame.

After its init, the method ‘apply’ is the one to use to apply the filter to a xarray data array

Parameters:

  • filter_name (str)

  • creation_mode (str | None)

  • fcut (int | float | None)

  • atten (None)

static _create_filter_matrix(filter_coefficients, size_profile)

Create the matrix representing the derivative filter. This matrix is such that, let A be the matrix and B be the vector on which the filter must be applied, A.B is the vector once the filter has been applied. Rows that are dropped by the filter are set to NaN (for example, if a filter is an average on 5 points, then the 2 first and 2 last lines do not have enough neighbors for the filter to be applied and thus must be dropped). Other rows contain the filter’s coefficients at the correct position for the matrix product to give the series’ values once the filter has been applied.

Parameters:

  • filter_coefficients – list The list of the filter’s coefficient (the coe

  • size_profile – int Length of a line

Returns:

static _create_filter_vector(filter_coeff, position, profile_size)

Create filter vector that will be a row of filter matrix to apply to signal Vector will be creating as describe below : - vector will be a row of profile_size value - filter_coeff centered to the position given in parameters - zeros anywhere else If filter cannot fit entirely into the resulting vector, return a profile_size vector of 0

Examples : create_filter_vector([-0.5, 0, 0.5], 2, 7) –> [0 -0.5 0 0.5 0 0 0] create_filter_vector([-0.5, 0, 0.5], 0, 7) –> [0 0 0 0 0 0 0]

Parameters:

  • filter_coeff – array of filter coefficient

  • position – position at which filter_coeff will be centered in resulting vector

  • profile_size – size of the filter matrix (= signal size)

Returns:

filter vector = filter_matrix row (number position)

_filter_coeff_from_csv(altitude_unity='km', column_data_number=1)

Read the csv storing the number of coefficient to use at each altitude and add an attribute _filter_coeff. If is a pandas series with the altitude in km has index and the filter coefficients as values. If two consecutive line contain the same filter, the one with the highest altitude is dropped. In the end, only the altitudes at which the filter changes are kept. :param altitude_unity: File altitude unity :param column_data_number: File column to consider for filtering. File can have different column corresponding to different filter depending on signal type (far range / near range / Raman) :return:

_filter_coeff_from_fixed_number_points()

Compute filted according to the number of point given :return:

_filter_from_dial_poly(ican1, ican2, ncan1, ncan2, da)

This function returns the number of point of the filter in function of the altitude and the coefficients of the filter in function of the altitude. The calculation is based on the initial DIAL software (in Matlab) and is corresponded to a polynomial of order 3 between 2 indexes.

Parameters:

  • ican1 (str) – str parameter ican1 for the polynomial calculation

  • ican2 (str) – str parameter ican2 for the polynomial calculation

  • ncan1 (str) – str parameter ncan1 for the polynomial calculation

  • ncan2 (str) – str parameter ncan2 for the polynomial calculation

  • da (DataArray) – DataArray parameter ncan2 for the polynomial calculation

Return filter_coeff:

pd.Series The coefficients of the input filter

Return number_of_points_filter:

pd.Series The number of point of the filter

_first_below_error_threshold(altitude, error_df)

Return the first number of points that make the error go below the threshold for the input altitude

Parameters:

  • altitude (float) – float Altitude at which the number of points is calculated

  • error_df (DataFrame) – pd.DataFrame The error for each altitude and for each number of points in the filter. Is has to have at lest one valid number of points for the altitude

Returns:

Return type:

int

_from_kwargs(name_value, optional=False)

Check if key name_value is in self._kwargs and collect it if True. If a key is required but missing, raise Error :param name_value: Parameter to find :param optional: boolean, if False, parameter MUST be specified by user :return:

_get_specific_mode_attribute()

Collect attribute needed according to creation mode specified :return:

_no_filter()

Method created for Ozone object creation, if filter had already been computed on signal given :return:

apply(da, uncertainty_flag=False, resolution_unit='m')

Apply the filter to a xarray data array. If uncertainty_flag is True, the uncertainty will be calculated.

Parameters:

  • da (DataArray) – DataArray to fit

  • uncertainty_flag (bool) – True if the variable is an uncertainty (default: False)

  • resolution_unit (str)

Rtype da:

xr.Dataset

Rtype uncertainty_flag:

bool

Rtype resolution_unit:

str

copy_with_square_coeff()

Copy filter coefficient and squares them :return:

Return type:

object

static get_filter_coefficients(filter_name, nb_coef, fcut=None, atten=None)

This function returns a set of filter coefficients for use in a convolution process.

Parameters:

  • filter_name (str) – str String that must match a well known filter name

  • nb_coef (int) – int Number of coefficients to return

  • fcut – float The cutoff frequency. Not mandatory for every filters

Returns:

list The coefficients of the input filter

Return type:

list

get_number_of_points_filter(alt_array)

Get the number of points used in the derivative filter at each altitude

Parameters:

alt_array (DataArray) – xr.DataArray The altitude at which to return the number of points used in the filter in the form of an xarray data array. The data must be stored has pint object or the array must have a ‘Units’ attribute.

Return type:

DataArray

get_resol_init_matlab_software(initial_resolution)

Get the resolution of the filter according to the initial function in the Matlab software

Parameters:

initial_resolution – float initial resolution, before the filter

Return output_resolution:

pd.Series final resolution after the filter. Same unit as param “initial_resolution”

Return type:

ndarray

resolution_ndacc(method=None, init_resol=None)

Get the resolution of the filtered data by the methodology of the transfer function, validated by NDACC

Parameters:

  • method (str | None)

  • init_resol (float | Quantity | None)