matrix data sorting with many columns

JoshLorenz
Posts: 2
Joined: 2013-07-10
Location: United States

Greetings,

I am wanting to sort a 2D wave with 900+ columns. I want to sort the matrix wave such that the rows are rearranged with all values of column 1, for example, in ascending order (similar to an Excel data sort). I've been doing some Google searching and reviewing priors posts on this forum. I can work with either a single 2D wave or 900+ 1D waves. I've got the 'matrix' data available in either format. The wave lengths are about 25e3 rows, just for reference.

I have reviewed this topic: Sorting two waves simultaneously, a la Excel

In my case, the example method from Howard would require listing the 900+ explicit 1D waves in the Sort command (or so it seems to me). I've messed around a bit with trying to sort the 2D wave directly, but thus far no luck.

Thanks. Josh

Joshua Lorenz
Kato Engineering


JoshLorenz
Posts: 2
Joined: 2013-07-10
Location: United States

I searched in the code snippets information and found the function MDtextsort(). I converted the sort process to a numeric sort (per the instructions), and everything worked great. I just made a new function call MDsort(). My 2D wave was sorted by a particular column. Yahoo!

#pragma rtGlobals=1	//Use modern global access method
 
Function MDsort(w,keycol)
	Wave w
	variable keycol
 
	variable ii
 
 
	make/o/n=(dimsize(w,0)) key
	make/o/n=(dimsize(w,0)) valindex
 
	key[] = w[p][keycol]
	valindex=p
 
	sort key,key,valindex
 
	duplicate/o w, M_newtoInsert
 
	for(ii=0;ii<dimsize(w,0);ii+=1)
		M_newtoInsert[ii][] = w[valindex[ii]][q]
	endfor
 
	duplicate/o M_newtoInsert,w
	killwaves/z key,valindex,M_newtoInsert
End

Joshua Lorenz
Kato Engineering


Igor's picture
Posts: 426
Joined: 2007-06-29
Location: United States

It sounds as if you need to use SortColumns which is not available in IP6.

One way of accomplishing what you want is to split your data into columns and then apply the Sort operation with a list of wave names corresponding to the columns. The difficulty here is that with 900 columns you will quickly exceed the maximum length of the command line. The solution is then to run multiple Sort operations, say in groups of 50 columns or so, always providing a copy of the unsorted column 1 as the key wave.

A.G.
WaveMetrics, Inc.


[ last edited July 10, 2013 - 15:39 ]
ikonen
Posts: 45
Joined: 2008-04-23
Location: United States

There's also the operation, MakeIndex, which is meant to be paired with IndexSort, except that neither are MultiDimensional aware.

MakeIndex key, valindx

does the same thing as MDSort's
make/o/n=(dimsize(w,0)) valindex
valindex=p
sort key,key,valindex

, and
IndexSort valindex, wavetobesorted

is equivalent to
wavetobesorted = copyofwavetobesorted[valindex[p]]

except I think it does the sorting in place (Igor doesn't need to make a complete copy of the unsorted wave the way the second form does). It would be nice to have MD aware IndexSort precisely for large MD waves, but copy and assign works. Finally, I don't see why MDSort needs the for loop:
M_newtoInsert = w[valindex[p]][q]

should do the same thing as explicitly stepping through rows with the for loop and the ii variable.


andyfaff
Posts: 522
Joined: 2007-09-11
Location: Australia

I've updated the original snippet taking into account ikonen's suggestions, which were valid. It also uses free waves which makes the whole thing a bit cleaner.

http://www.igorexchange.com/node/599


andyfaff
Posts: 522
Joined: 2007-09-11
Location: Australia

Yet another edit to the original snippet allows it to be used to sort either text or numerical waves.

http://www.igorexchange.com/node/599


Back to top