Organizing Distinct Wave Elements

Share icon

Thu, 06/23/2011 - 11:57 am

Hi,

I am wondering if there is a way to obtain distinct elements in a wave.

For example, if I have a text wave called wave1 and it is 10 rows by 1 column, and its contents are {F,A,B,D,A,B,C,D,D,E}, I would like to obtain a string list {F;A;B;D;C;E}, a 6 by 1 wave with the same contents as that string list, or anything else that keeps track of the distinct elements in wave1 in a reasonable format.

Thanks.
This will be slow for large waves but fine for small waves.

Function/S TextWaveAsList(tw, distinctOnly)
    Wave/T tw
    Variable distinctOnly
   
    String list = ""
   
    Variable numPoints = numpnts(tw)
    Variable i
    for(i=0; i<numPoints; i+=1)
        String item = tw[i]
       
        if (distinctOnly)
            if (WhichListItem(item, list) < 0)      // Case-insensitive
                list += item + ";"
            endif      
        else
            list += item + ";"
        endif
    endfor
   
    return list
End

June 23, 2011 at 01:15 pm - Permalink

I think this will be faster for small and large waves (although I don't know the expense of removefromlist), but requires the installation of the SOCKIT xop (also on IGORexchange). However, it won't work with binary data as I don't think itemsinlist works when there are NUL's in the textwave.
BTW, this is another example of where it's useful to be able to extract wavedata into a string, there was another example on the mailing list earlier this week.

Function/s textwaveAsList(tw, distinctOnly)
    Wave/t tw
    variable distinctonly
    string alist = "", adistinct = "", item="", listsep = ";"
    variable ii

    //get the wave into a string (fast)
    duplicate/free/t tw, twcpy

    sockitwavetostring/txt=listsep twcpy, alist

    if(!distinctonly)
        return alist
    endif

    //now get a distinct list if you need it.
    for(; itemsinlist(alist, listsep) ;)
        item = stringfromlist(0, alist, listsep)
        adistinct +=  item + listsep
        alist = removefromlist(item, alist, listsep)
    endfor
    return adistinct

End

June 26, 2011 at 05:21 am - Permalink

andyfaff wrote:
I think this will be faster for small and large waves (although I don't know the expense of removefromlist)


I take it all back, Howards version is nearly 2 orders of magnitude faster when tested on a random textwave 1e6 elements long. It's quite expensive to rebuild the list using removefromlist, it's also more expensive to get an individual list item with itemfromlist than it is to get an individual element from a textwave (when there are 1e6 list items/1e6 textwave points). When one thinks about it itemfromlist has to search the string everytime it's called, whereas accessing an individual wavepoint is much faster (no searching).

June 26, 2011 at 05:55 am - Permalink

Thank you. I incorporated Howard's code and it works like a charm. Though since my waves are only 100 to 300 indexes long, either code should work fine. I have not yet tried the Sockit XOP, but I downloaded it and will give it a try in the future when the opportunity rises up.

July 6, 2011 at 10:06 am - Permalink