multiple 2D wave filtering

Share icon

Thu, 04/09/2015 - 07:00 am

Hello Good People,
I have a few (~ 12) 2D waves that contain 30-40 columns and 3000 rows each row representing a measurement in time.
Let's say one column is a quality flag (and contains values of 0 or 1). Each wave will have its own set of flags. How should I go about filtering all the values with say quality flag 0 based on only wave1?
In other words I would like to get all readings from each wave that match/intersect with the times at which wave1 had a flag of 0.
This function takes in your multicolumn wave, the column number you want to examine, a test value and mode. The mode is -1 to extract all rows where the selected column is less than the value, 0 to extract rows equal to value, and +1 to extract rows greater than value. These modes may not be exactly what you need, but this should get you started toward something that does what you want.

The output wave is always called "ExtractedWave". It would be possible to add an input to use a name of your choosing.

Function extractByColumn(multicolumnWave, column, value, mode)
    Wave multicolumnWave
    Variable column, value
    Variable mode           // -1 less than, 0 equal, +1 greater than
   
    Duplicate/FREE/R=[][(column)] multicolumnwave, onecolumn
    if (mode < 0)
        Extract/INDX/FREE onecolumn, index,onecolumn<value
    elseif (mode == 0)
        Extract/INDX/FREE onecolumn, index,onecolumn==value
    else
        Extract/INDX/FREE onecolumn, index,onecolumn>value
    endif
    Make/O/Y=(wavetype(multicolumnWave))/N=(numpnts(index), DimSize(multicolumnWave, 1)) ExtractedWave
    ExtractedWave = multicolumnWave[index[p]][q]
end


John Weeks
WaveMetrics, Inc.
support@wavemetrics.com

April 9, 2015 at 09:57 am - Permalink

Thanks John.
However, the function only picks values that match the criteria from one wave only. What I am looking for is a way to use the conditions from one wave to filter data from all other waves.
The experiment is setup like this. I have the data from multiple instruments (a 2D wave for each instrument) with one instrument set up as reference (wave1). I would like to compare all other instrument's data against the reference instrument. However I only want to compare periods when the reference instrument is giving good data. So how do I go about matching the good period for the reference instrument across all instrument's data?

April 9, 2015 at 02:34 pm - Permalink

The index wave from the Extract operation can be used to pull data from any other wave as long as the waves have enough rows that the indices in the index wave aren't too large.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com

April 9, 2015 at 03:57 pm - Permalink

izishaw wrote:
Thanks John.
However, the function only picks values that match the criteria from one wave only. What I am looking for is a way to use the conditions from one wave to filter data from all other waves.
The experiment is setup like this. I have the data from multiple instruments (a 2D wave for each instrument) with one instrument set up as reference (wave1). I would like to compare all other instrument's data against the reference instrument. However I only want to compare periods when the reference instrument is giving good data. So how do I go about matching the good period for the reference instrument across all instrument's data?


I'd suggest getting acquainted with MatrixOP. Here is an example:

Suppose your data wave is waveD and your "quality" wave is waveQ. If you want to remove all entries for which waveQ==0 you can execute:
MatrixOP/o out=waveD*(waveQ/waveQ)

This will give you a copy of waveD where entries corresponding to waveQ==0 are set to NaN. Next execute:
WaveTransform zapNaNs out

Similarly, if you want to set all the values in waveD for which waveQ==0 it is simply:
MatrixOP/O out=waveD*waveQ


A.G.
WaveMetrics, Inc.

April 9, 2015 at 04:57 pm - Permalink