Sort multiple waves at the same time

Share icon

Fri, 03/31/2017 - 04:23 pm

Hi I have a wave that contains time stamps for my measurements, and 40 other waves of measurement data that correspond to each of the time stamps. Currently my time stamps are not sequential and I want to sort the time stamp wave and the 40 other data waves to make time series of all the measurements. My problem is that some of my time stamps are the same, i.e. I made duplicate measurements for those time stamps. My question is a) what would be a good approach to sort all data waves together? I tried referencing waves using strings of wave names from the wavelist command, but this only allows me to reference one wave at a time using a for loop and I think I need to kill the sorted time stamp every time and regenerate a new time wave to sort, but it doesn't feel very correct. b) when I sort the time stamps which have duplicates, would Igor be confused for the same time stamp to align my many other measurement data as they originally were but not mixed up?

My codes are

function sortwave(timestamp)

wave timestamp
string listwave
variable i

duplicate/o timestamp timestamp1
listwave =wavelist ("*pptv1",",","")

for (i=0;i<itemsinlist(listwave);i+=1)
string datastring = stringfromlist(i, listwave, ",")
wave compound= $datastring
duplicate/o timestamp timestamp1
sort timestamp1 timestamp1, compound
killwaves timestamp1
endfor

duplicate timestamp timestamp1
sort timestamp1 timestamp1

end



Thank you!
If you want to create a list of wave references instead of a string list you could use this instead of WaveList:
    //  Counts the number of waves in MyFolder
    Variable n=CountObjectsDFR(MyFolder, 1)

    //  Creates a list of all waves in MyFolder
    Make/FREE/O/WAVE/N=(n) AllWaves=WaveRefIndexedDFR(MyFolder, p)


But how do you know which wave corresponds to which time stamp? What is the format of the time stamps? date/time, milliseconds? Once that is sorted it's easy:

Wave TimeStampWave
Wave/WAVE MeasurementWaves
Sort TimeStampWave, TimeStampWave, MeasurementWaves


Your code seems to be sorting the values in each of the measurement waves instead of the waves themselves. Is that what you want to do?
Wave TimeStampWave
Wave/WAVE MeasurementWaves
Variable n=NumPnts(MeasurementWaves)

for (i=0; i<n; i+=1)
    Wave SingleWave=MeasurementWaves[i]
    Sort TimeStampWave, SingleWave     //    This will not change the TimeStampWave
endfor

April 1, 2017 at 02:25 am - Permalink

If you have one key wave and bunch of value waves that all need to be sorted according to the key wave, you have two choices:

1) Create a sort command in a string, adding each wave name (possibly with full data folder path) plus a comma to the Sort command. Then use Execute the run the command.

2) If you don't want to sort the key wave, just use Sort on a single value wave at a time in a loop. If you want to sort the key wave as well, you could use MakeIndex to create an index wave and use the index wave with IndexSort in a loop.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com

April 3, 2017 at 09:12 am - Permalink

Thank you olelytken and johnweeks! I do want to sort the values in each of the measurement wave, not the waves themselves. The timestamp is in date/time and the WaveRefIndexedDFR is indeed better than wavelist.

What is the reason for using execute and sort command in string? Is it because the command would be too long? Does it still have to observe the 400-character limit? "Help" for Sort says "The algorithm used does not maintain the relative position of items with the same key value." I don't quite understand what this means.

So it seems to use sort in a loop is a choice for the task. I can get around killing waves in loop by using sort. Are there differences in using Sort or IndexSort in the loop? I know IndexSort uses an extra wave from MakeIndex to sort the sortedwaves. Is one better than the other in loop? It seems I can use both whether I want to sort the sortkeywave or not.

When I try to operate on a wave that is in a different data folder do I switch to the other data folder, reference the wave and then switch back or can I directly reference the wave using a path without changing folders? How to use command to move waves from one datafolder to another?

Thanks!

April 3, 2017 at 02:34 pm - Permalink

mwpro wrote:
What is the reason for using execute and sort command in string? Is it because the command would be too long? Does it still have to observe the 400-character limit?

It's just that using a string variable that contains a list of wave names simply doesn't work. The parsing for Sort doesn't know how to look inside a string expression. It would be nice, but it doesn't.

Quote:
"Help" for Sort says "The algorithm used does not maintain the relative position of items with the same key value." I don't quite understand what this means.

if you have a key wave that has more than one instance of a value (say, {1, 3, 2, 3, 5}) there is no guarantee that the value waves will be sorted with point 1 before point 3. Igor is using a fast sort algorithm that doesn't maintain pre-existing ordering in case of ties.

Quote:
So it seems to use sort in a loop is a choice for the task. I can get around killing waves in loop by using sort. Are there differences in using Sort or IndexSort in the loop? I know IndexSort uses an extra wave from MakeIndex to sort the sortedwaves. Is one better than the other in loop? It seems I can use both whether I want to sort the sortkeywave or not.

That's all true. The complexity of Sort should be O(n log(n)); the complexity of IndexSort, once the index wave is made, should be O(n). Neglecting constant overhead, that means IndexSort will be faster in the loop. In fact, IndexSort in a loop should be almost as fast as Sort on a list since internally Sort works by making an index, then re-arranging the points in each wave.

Quote:
When I try to operate on a wave that is in a different data folder do I switch to the other data folder, reference the wave and then switch back or can I directly reference the wave using a path without changing folders?

Your loop will look something like
for (i = 0; i < nwaves; i++)
    WAVE w = $StringFromList(i, listofwaves)
    IndexSort indexWave, w
endfor

(I haven't actually compiled or run this "code").

Quote:
How to use command to move waves from one datafolder to another?

Have you found the command MoveWave?


John Weeks
WaveMetrics, Inc.
support@wavemetrics.com

April 3, 2017 at 04:16 pm - Permalink

Thank you so much John! I wonder if there are commands for browsing waves from other experiments and "drag" those waves to the current Igor experiment as can be achieved in the databrowser too. Can "Loadwave" load waves from igor experiment? Or the waves would have to be saved in an igor text from that experiment first before it can be loaded by other experiments?

Thanks again!

April 4, 2017 at 04:46 pm - Permalink

Have a look at LoadData. It can load waves, variables, strings, and even data folders from other experiments.
HJ

April 5, 2017 at 01:25 am - Permalink

mwpro wrote:
Thank you so much John! I wonder if there are commands for browsing waves from other experiments and "drag" those waves to the current Igor experiment as can be achieved in the databrowser too. Can "Loadwave" load waves from igor experiment? Or the waves would have to be saved in an igor text from that experiment first before it can be loaded by other experiments?

Thanks again!


These projects save a snapshot of open graphs, panels, etc. in an experiment as Igor notebooks. To some extent you can browse a saved experiment in as much as waves were displayed in graphs at the time the snapshot was created, but waves cannot be loaded using these snapshots.

http://www.igorexchange.com/project/OSXPreviewGenerator
http://www.igorexchange.com/project/JTG_ExperimentPreview

Maybe this helps?

April 5, 2017 at 05:35 am - Permalink

jtigor wrote:

These projects save a snapshot of open graphs, panels, etc. in an experiment as Igor notebooks. ...


Wow! There is some amazing stuff packed in those two experiments. Thanks.

mwpro wrote:
... I wonder if there are commands for browsing waves from other experiments and "drag" those waves to the current Igor experiment as can be achieved in the databrowser too. Can "Loadwave" load waves from igor experiment? Or the waves would have to be saved in an igor text from that experiment first before it can be loaded by other experiments?


In case you are interested in somewhat the inverse problem, this package will create a stand-alone snapshot experiment of all content from the frontmost graph.

http://www.igorexchange.com/project/SnapIt

--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH

April 5, 2017 at 07:03 pm - Permalink