Shortening Loaded Wavenames using FreadLine

Share icon

Fri, 05/22/2015 - 03:52 pm

Hello,

I have been able to figure out how to label the waves with the date (I ended up appending it to the trace, not the wave itself) and now I am trying to load data from different instruments, and they all have different names depending on what instrument I load from. The main problem is that some names are so long that the first 31 characters (or however many Igor reads) are the same and so Igor throws a fit because there are 30 waves in each file that are the same. From what I understand, I am going to have to us Open FReadline Close, but I'm having trouble because I am unable to get the separations between points that I would like. Further, the waves aren't being named what I want them to be named. My data is structured such that I get a bunch of header info that I don't need, and then a row with the names of all the waves each separated with commas (e.g. UTC_Start, UTC_End, Stuff, Stuff2, etc) and then rows of the data itself, also separated by commas. I further have the problem (another reason LoadWave isn't working for me) is that sometimes, when the data is sent from the different instruments, there is a bug that repeats one of the data lines, so that two columns are exactly the same, and I have been trying to use the /O in loadwave, but that doesn't work.

My questions are: how do I get Igor to open the file, make waves such that each carriage return makes a new column (wave), and each comma makes a new row (point in a wave)? Further, if I get into the situation where the same data tries to load twice, how can I tell Igor to ignore those lines?

Thank you

This is what I have so far (mostly copied from the example)
Function ReadTextFile(pathName, filePath, outputWaveName, maxNumberOfLines)
    String pathName             // Symbolic path name or "" if filePath is full path. Can also be "" if filePath is "" to get Open File dialog.
    String filePath                 // File name, partial path or full path or "" to get Open File dialog.
    String outputWaveName           // NOTE: Overwrites existing wave, if any!
    Variable maxNumberOfLines       // Maximum expected number of lines in file.
 
    Variable refNum
    Variable err = 0
 
    if (strlen(filePath) == 0)          // Want Open File dialog?
        Open/D/R/P=$pathName refNum as filePath
        if (strlen(S_fileName) == 0)
            return -1               // User canceled.
        endif
        filePath = S_fileName       // This is a full path.
    endif
 
    Open/R/P=$pathName refNum as filePath
    if (GetRTError(1))
        return -2                   // Error. Probably file does not exist.
    endif
 
    Make/O/T/N=(maxNumberOfLines) $outputWaveName
    if (GetRTError(1))
        Close refNum
        return -3                   // Error. Probably there is already a numeric wave with this name.
    endif
 
    Wave/T tw = $outputWaveName
 
    Variable row, column
        for(row=0; row<maxNumberOfLines+1; row+=1)
        String data
        FReadLine refNum, data      // Terminates on CR, LF or CRLF.
       
        if (strlen(data) == 0)
            break                   // All done.
        endif
 
        if (row == maxNumberOfLines)
            err = -4                    // Error. File had more rows than specified maximum.
            break
        endif
 
        tw[row] = data

    endfor

Close refNum
Open/R/P=$pathName refNum as filePath
              //trying to get it to make different waves out of the data
        for(column=0; column<maxNumberOfLines+1;column+=1)
        String columns
        FReadLine/T=","/T=(num2char(13)) refnum, columns       
        if (strlen(columns) == 0)
            break                   // All done.
        endif
 
        if (column == maxNumberOfLines)
            err = -4                    // Error. File had more columns than specified maximum.
            break
        endif
 
        tw[column] =columns
        Redimension/N=(row, column) tw
    endfor
    Close refNum
 
    Redimension/N=(row, column) tw
 
    Print err                       // 0=success
End
<pre><code class="language-igor">
In order to help I would need a sample data file that illustrates the issues. It can be an abbreviated file so long as it illustrates all of the issues.

It would be best if you would also provide a concise list of what the issues are, and how you want to handle that issue. For example:
1. Column 3 (zero-based) has the same column label as column 4. In this case I want to ...
2. Row 37 has identical data to row 36. In this case I want to ...
...

May 22, 2015 at 06:06 pm - Permalink

Hello,

I attached a condensed sample excel file. In this one,

1) I would like to delete the first 497 (in this case) columns (up to UTC, JDAY, etc)
2) FReadLine is separating each row, but I would like each row to be separated into columns by the commas (it's delimited with commas). This case would have 457 columns (waves in this case)
3) Column JR(277) and KQ(302) are named TemperatureDifference-390nm-10800nm-MEAN_GOES13 and TemperatureDifference-390nm-10800nm-MEAN_GOES15. Loadwave only goes up to TemperatureDifference-390nm-108. This happens one other time in this specific file. I would like to keep the entire name, but if it is too long, I would like to either replace "temperaturedifference" with 'dT" or remove as little as I have to from the beginning of that name (The end of the name is more important)

I was unable to find the files that had repeated columns. I may have confused myself with that one.

Thank you very much
SEAC4RS-mrg10-dc8_merge_20130808_R2_Igor.csv

May 25, 2015 at 11:11 am - Permalink

I have written an Igor procedure that loads the file using LoadWave. It uses the LoadWave /B flag to specify the wave names. The procedures are attached to this post.

The procedure adds a "Load Aroob File" item to the Data->Load Waves menu.

The most convenient way to use the file is to put it in your "Igor Procedures" folder and restart Igor. If you are not familiar with "Igor Procedures", please execute this to read the help:
DisplayHelpTopic "Special Folders"


I have shortened some of the wave names to fit in Igor's 31 byte limit. See the procedure file for details.

You have a lot of what we call "liberal" names. To learn about liberal names, execute this:
DisplayHelpTopic "Object Names"
Load Aroob File.zip

May 25, 2015 at 01:27 pm - Permalink