multi general text loading

Share icon

Fri, 11/12/2010 - 07:49 am

hello
I want to write a proc by which I can load all the text files in a folder. the files are categorised by name, for example basename_001, basename_002 and so on.
for loadindg the files simultanously I have to have a flexible path, which changes automaticaly after it imports the first file.
I wrote a test proc in which it uses the loadwave function. after I execute it it ask for the path, I give the path by browsing and choosing the first file, so it store the full path to the s_path string, then I want to use a loop in which there is another loadwave function except the path is coming from the s_path+index.
index goes up one by one and it continues until loading all the waves. but in any form I used the path,the procedure doesnt accept it while compiling, and mention: ill formed path,
would any body help please
If you show us the code, we can help you better.

To include igor code in these forum postings, put <igor> and </igor> tags around the text, like this:

<igor>
Function/S ListFilesOfType(dataFolderStr,type,fileTypeStr)
String dataFolderStr
Variable type // popup,"Igor Binary;Igor Text;General Text;Delimited Text;"
String fileTypeStr // usually "TEXT" for *.txt files on Windows

String files=""
if( type == 1 )
files= IndexedFile($dataFolderStr,-1,fileTypeStr) // list of all files with matching type
else
if( type == 2 ) // Igor Text, either "TEXT" or "IGTX" (.txt or .itx)
// open each TEXT or IGTX file, and if the first line is "IGOR", add it to files
files= IgorOrTextFiles(dataFolderStr,fileTypeStr,1) // on Windows, *.txt (or user's file type)
if( CmpStr(fileTypeStr,"IGTX") != 0 ) // avoid listing a file twice
files += IgorOrTextFiles(dataFolderStr,"IGTX",1) // on Windows, *.itx,
endif
else
files= IgorOrTextFiles(dataFolderStr,fileTypeStr,0) // on Windows, *.txt
endif
endif
return files
End

</igor>

and it will be nicely formatted:

Function/S ListFilesOfType(dataFolderStr,type,fileTypeStr)
    String dataFolderStr
    Variable type           // popup,"Igor Binary;Igor Text;General Text;Delimited Text;"
    String fileTypeStr      // usually "TEXT" for *.txt files on Windows

    String files=""
    if( type == 1 )
        files= IndexedFile($dataFolderStr,-1,fileTypeStr)   // list of all files with matching type
    else
        if( type == 2 ) // Igor Text, either "TEXT" or "IGTX" (.txt or .itx)
            // open each TEXT or IGTX file, and if the first line is "IGOR", add it to files
            files= IgorOrTextFiles(dataFolderStr,fileTypeStr,1)     // on Windows, *.txt (or user's file type)
            if( CmpStr(fileTypeStr,"IGTX") != 0 )                   // avoid listing a file twice
                files +=  IgorOrTextFiles(dataFolderStr,"IGTX",1)   // on Windows, *.itx,
            endif
        else
            files= IgorOrTextFiles(dataFolderStr,fileTypeStr,0)     // on Windows, *.txt
        endif
    endif
    return files
End



--Jim Prouty
Software Engineer, WaveMetrics, Inc.

November 12, 2010 at 02:10 pm - Permalink

thank you for reply
I found two functions from Igor examples which do the thing that I want, but there still exists a problem
Function LoadAndGraph(fileName, pathName)
String fileName // Name of file to load or "" to get dialog
String pathName // Name of path or "" to get dialog
// Load the waves and set the variables.
LoadWave/g/D/O/P=$pathName fileName
if (V_flag==0) // No waves loaded. Perhaps user canceled.
return -1
endif
End
Function LoadAndGraphAll(pathName)
String pathName // Name of symbolic path or "" to get dialog
String fileName
Variable index=0
if (strlen(pathName)==0) // If no path specified, create one
NewPath/O temporaryPath // This will put up a dialog
if (V_flag != 0)
return -1 // User cancelled
endif
pathName = "temporaryPath"
endif
Variable result
do // Loop through each file in folder
fileName = IndexedFile($pathName, index, ".txt")
if (strlen(fileName) == 0) // No more files?
break // Break out of loop
endif
result = LoadAndGraph(fileName, pathName)
index += 1
while (1)
if (Exists("temporaryPath")) // Kill temp path if it exists
KillPath temporaryPath
endif
return 0 // Signifies success.
End

suppose that I have a folder with 100 text files in it, when I call this function the window of loading waves apears and I have to
click on load button 100 times. if I want to cancel now I have to click cancel button 100 times again. I am not satisfied with this, is there any way to load 100 text files at once.
thank you in advance

November 14, 2010 at 01:37 pm - Permalink

I would rewrite it this way, but you should note that if pathName and fileName are valid, there won't be any dialog other than the one initial NewPath dialog for the user to cancel out of, so you shouldn't ever get into the situation where there are many dialogs to cancel out of.

#pragma rtGlobals=1     // Use modern global access method.
Function LoadAndGraph(fileName, pathName)
    String fileName // Name of file to load or "" to get dialog
    String pathName // Name of path or "" to get dialog
    // Load the waves and set the variables.
    LoadWave/g/D/O/P=$pathName fileName     // no dialog if pathName and fileName are valid
    if (V_flag==0)
        return -1 // No waves loaded (if pathName and file name are right, user doesn't have opportunity to cancel).
    endif
    return 0 // 0 signifies success
End
Function LoadAndGraphAll(pathName)
    String pathName // Name of symbolic path or "" to get dialog
    String fileName
    Variable index=0
    if (strlen(pathName)==0) // If no path specified, create one
        NewPath/O temporaryPath // This will put up a dialog
        if (V_flag != 0)
            return -1 // User cancelled
        endif
        pathName = "temporaryPath"
    endif
    Variable result=0
    do // Loop through each file in folder
        fileName = IndexedFile($pathName, index, ".txt")
        if (strlen(fileName) == 0) // No more files?
            break // Break out of loop
        endif
        result = LoadAndGraph(fileName, pathName)
        if( result == -1 )
            break   // user cancelled or no waves in this file (which we take to be an error)
        endif
        index += 1
    while (1)
    if (Exists("temporaryPath")) // Kill temp path if it exists
        KillPath temporaryPath
    endif
    return result // 0 Signifies success.
End

--Jim Prouty
Software Engineer, WaveMetrics, Inc.

November 13, 2010 at 04:23 pm - Permalink

You need to add /A to the LoadWave command. Otherwise it will put up a dialog asking you to confirm or change the wave names.

November 13, 2010 at 06:47 pm - Permalink

thank you
I excecuted it and it worked properly. both advices were so helpful.
I learned a lot here. I will continue learning and discussing in this nice forum
thanks again

November 14, 2010 at 05:44 am - Permalink

hello again
I merged two functions and edited it a little. now it works nicely for me so I put it
here for whom it may be useful to import all the general text files in a folder at once.
function loadwaves(filename, pathname)
string filename
string pathname
if (strlen(pathname)==0)
newpath/o temporaryPath
if (v_flag!=0)
return -1
endif
pathname= "temporarypath"
endif
variable index = 0
do
filename= indexedfile ($pathname, index, ".txt")
if (strlen (filename)== 0)
break
endif
loadwave/a/g/o/d/p=$pathname filename
if (v_flag==0)
break
endif
index+=1
while(1)
end
menu "macros"
submenu "Load Waves"
"multi general text load", loadwaves("","")
end
end

November 14, 2010 at 01:27 pm - Permalink

Hi,

I've copied your program to Igor and tried to use it.
I got an error message that said "A non-existent data folder was referenced while accessing a child data folder".
My command line was: "LoadAndGraphAll(C:\Users\Aviv\Documents\Aviv\flex\Flex52 _Pani_90_30\)"

I'd appreciate any assistance on the matter, since you guys wrote that this program works for you.

Thanks you,
Aviv

JimProuty wrote:
I would rewrite it this way, but you should note that if pathName and fileName are valid, there won't be any dialog other than the one initial NewPath dialog for the user to cancel out of, so you shouldn't ever get into the situation where there are many dialogs to cancel out of.

#pragma rtGlobals=1     // Use modern global access method.
Function LoadAndGraph(fileName, pathName)
    String fileName // Name of file to load or "" to get dialog
    String pathName // Name of path or "" to get dialog
    // Load the waves and set the variables.
    LoadWave/g/D/O/P=$pathName fileName     // no dialog if pathName and fileName are valid
    if (V_flag==0)
        return -1 // No waves loaded (if pathName and file name are right, user doesn't have opportunity to cancel).
    endif
    return 0 // 0 signifies success
End
Function LoadAndGraphAll(pathName)
    String pathName // Name of symbolic path or "" to get dialog
    String fileName
    Variable index=0
    if (strlen(pathName)==0) // If no path specified, create one
        NewPath/O temporaryPath // This will put up a dialog
        if (V_flag != 0)
            return -1 // User cancelled
        endif
        pathName = "temporaryPath"
    endif
    Variable result=0
    do // Loop through each file in folder
        fileName = IndexedFile($pathName, index, ".txt")
        if (strlen(fileName) == 0) // No more files?
            break // Break out of loop
        endif
        result = LoadAndGraph(fileName, pathName)
        if( result == -1 )
            break   // user cancelled or no waves in this file (which we take to be an error)
        endif
        index += 1
    while (1)
    if (Exists("temporaryPath")) // Kill temp path if it exists
        KillPath temporaryPath
    endif
    return result // 0 Signifies success.
End

--Jim Prouty
Software Engineer, WaveMetrics, Inc.

December 22, 2011 at 02:04 am - Permalink

A literal string input to a function should be quoted:

 LoadAndGraphAll("C:\Users\Aviv\Documents\Aviv\flex\Flex52 _Pani_90_30\")

You also need to use double backslashes to have a backslash included in the path. Igor uses backslash to included special characters (such as tabs) in strings. You can also substitue a colon for a backslash in the path. Internally, Igor can use this Mac path separator convention. If you use double backslashes you command should look like this:

 LoadAndGraphAll("C:\\Users\\Aviv\\Documents\\Aviv\\flex\\Flex52 _Pani_90_30\\")

December 22, 2011 at 05:59 am - Permalink

Great, thank you very much.
I have another question on this part, if I want to use the filename variable and rename every wave loaded to it's filename + an extention, such as:
wave0 = filename_loaded
filename includes the file type "aviv.txt" instead of just "aviv".
I've tried using strcpy, strncpy, strcat, etc, but none of these commands work in Igor.
How can I shorten the filename?

Thank you again,
Aviv

December 29, 2011 at 02:50 am - Permalink

For example:
String str1 = "aviv.txt"
String str2 = RemoveEnding(str1, ".txt"); Print str2
String str3 = str1[0,strlen(str1)-5]; Print str3

December 29, 2011 at 05:30 am - Permalink

If the file extension can vary, then you might want to use ParseFilePath instead of RemoveEnding. To remove the extension use the form:

ParseFilePath(3, filename, ":", 0, 0)

where filename is your input file name with extension (with or without the path). If filename includes the path then the ":" above specifies the path separator. If you get the path from Igor, regardless of whether you are working on Mac or Windows, a colon will be the path separator. In effect this function removes everything after the final ".".

For example:

print ParseFilePath(3, "gabbagabba.hey", ":", 0, 0)

prints:

gabbagabba

December 29, 2011 at 01:44 pm - Permalink

Hello

I have also been attempting to write a macro which I can use to load all text files within a folder, unfortunately these are not categorised the same way as fairman's and are named as shown in the following example:

p100kimage545836.tif.dat
p100kimage545837.tif.dat
....etc. etc.......
p100kimage545936.tif.dat

Ideally I would also like the macro to append the new waves into seperate graph windows as they are loaded, as demonstrated using the following macro for an individual 2D intensity plot, unfortunately there are >100 scans make in each scan so it's not really feasible to do this individually!:

Macro Load2DImage(imagename)
    Variable imagename
    LoadWave/G/M/D/A=Image/E=0
    Display;AppendImage "Image0"
    ModifyImage "Image0" ctab= {0,50,Grays,1}
End


Initially I tried to accomplish this by using a do loop within a do loop, however, due to my lack of knowledge w.r.t. Igor syntax I have not found a means by which to read every file within a directory and allocate systematic wavenames to these files (wave0, wave1 etc.). Therefore after reading Fariman's original query I attempted to use the solutions that were suggested.

Unfortunately the following macro was only capable of loading a single wave and I'm unable to see how it differentiates from that suggested by JimProuty

macro Load2DImage(filename, pathname)
    string filename
    string pathname
    LoadWave/G/M/D/o/a/E=0/P=$pathname filename
    if (V_flag !=0)
        return -1
    endif
    return 0
end

macro Load2DImageAll(pathname)
    string pathname
    string filename
    variable index=0
   
    if (strlen(pathname)==0)
        newpath/o/m="Select a scan directory" temppath
        if (V_flag !=0)
            return -1
        endif
        pathname = "temppath"
    endif
    variable result=0
       
    do
        filename = indexedfile($pathname, index, ".tif.dat")
        if (strlen(filename) == 0)
            break
        endif
        result = Load2DImage(filename, pathname)
        if( result == -1 )
            break
        endif
        index += 1
    while (1)
    if (exists("temppath"))
        killpath temppath
    endif
    return result
end


Whereas the following macro failed to load any waves whatsoever and I'm also lost as to why.

Macro Load2DImage(filename, pathname)
string filename
string pathname
if (strlen(pathname)==0)
newpath/o/m="Select a scan directory" thepath
if (v_flag!=0)
return -1
endif
pathname= "thepath"
endif
variable index = 0
    do
        filename= indexedfile ($pathname, index, ".tif.dat")
        if (strlen (filename)== 0)
        break
        endif
        LoadWave/G/M/D/a=Image/E=0/P=$pathname filename
        Display;AppendImage $pathname filename
        ModifyImage $pathname filename ctab= {0,50,Grays,1}  
        if (V_flag==0)
        break
        endif
        index+=1
    while(1)
end


As you can probably establish I am fairly new to script writing as a whole so any suggestions, no matter how condescending you feel they may be, will be much appreciated.

Jon

February 18, 2012 at 09:58 am - Permalink

Igor considers the file name extension to be the text from the last dot to the end of the file name. So you need to use ".dat" with IndexedFile, not ".tif.dat".

Next, in Load2DImage, this statement:
    if (V_flag != 0)

should be:
    if (V_flag == 0)    // No waves loaded?


Finally, all new code should be written as functions, not as macros. To understand why, execute:
DisplayHelpTopic "Macros and Functions"


Here are the procedures rewritten as functions:

Function Load2DImage(filename, pathname)
    string filename
    string pathname
   
    LoadWave/G/M/D/o/a/E=0/P=$pathname filename
    if (V_flag == 0)        // No waves loaded?
        return -1       // Error
    endif
    return 0            // Success
End
 
Function Load2DImageAll(pathname)
    string pathname
 
    if (strlen(pathname)==0)
        newpath/o/m="Select a scan directory" temppath
        if (V_flag !=0)
            return -1
        endif
        pathname = "temppath"
    endif

    variable result=0
 
    Variable index=0
    do
        String filename = IndexedFile($pathname, index, ".dat")
        if (strlen(filename) == 0)
            break
        endif
        result = Load2DImage(filename, pathname)
        if (result != 0)
            Print "Error from Load2DImage, filename=", filename
            break
        endif
        index += 1
    while (1)
   
    if (exists("temppath"))
        killpath/Z temppath
    endif
   
    return result
End

February 18, 2012 at 11:22 am - Permalink

Hello Hrodstein,

Thank you very much for the quick reply, unfortunately when attempting to execute the functions posted I was met with "syntax error: expected string variable or string function".

Could this be effected in anyway by my using Igor 5.04B (on a mac)

Regards
Jon

February 19, 2012 at 09:47 am - Permalink

It works for me in Igor Pro 5.04B on Macintosh.

Quote:
when attempting to execute the functions posted I was met with "syntax error: expected string variable or string function".


You need to provide all of the parameters when calling a user-defined function, like this:
Load2DImageAll("")


If you omit the parameter (an empty string in this case), you will get the "expected string" error.

February 19, 2012 at 10:14 am - Permalink