multiple data(txt file) import

hskim
Posts: 3
Joined: 2010-02-23
Location: United States

Could someone explain how to import multiple data files? I just started to learn Igor instead of Origin so I think I need some time to get used to this. Tutorials were useful but it doesn't solve my question. I have bunch of data files but load them one by one currently...I've searched forum (and also Igor manual) and found some topic about 'Displaying a Multi-Selection Open File Dialog' but wasn't really helpful yet. Shoud I write a procedure file to do that? or any simple way I missed?


[ last edited March 12, 2010 - 17:14 ]
pczjrh
Posts: 35
Joined: 2009-04-20
Location: United Kingdom

I recently wrote some code to do exactly this.
I've trimmed out all of the stuff that is not important to you.
You will need to use the multiopenfiles XOP (the one you have been looking at already), pop it in the IgorExtensions folder as instructed in the installation file. Once there, the code below will work.
This will open each file, and store the filename in a wave called 'Results'.
Getting the LoadWave bit right for your text file files can be sticky sometime (I had to ask for help here).

I hope this is enough to get you going.
Jason.

#pragma rtGlobals=1		// Use modern global access method.
Function MultiLoad()
multiopenfiles
string text_string = S_filename
Make/O/T/N=(ItemsInList(text_string, ";")) text_wave
text_wave = StringFromList(p, text_string, ";")
Make/O/T/N=(numpnts(text_wave),1) Results 
For (nof=0;nof<=numpnts(text_wave)-1;nof+=1)
LoadWave/A/D/J/W/K=0/V={" "," $",0,0}/L={0,2,0,0,0}  text_wave[nof]
	Results[nof][1]=S_filename
	KillWaves/Z wave1
endfor
End Function 


[ last edited March 13, 2010 - 08:28 ]
hskim
Posts: 3
Joined: 2010-02-23
Location: United States

Thanks, Jason.
This is exactly what I wanted.
Now I feel like getting some ideas on Igor...


[ last edited March 13, 2010 - 04:05 ]
Posts: 497
Joined: 2007-03-01
Location: United States

If you're using Igor 6.1 or greater, we don't recommend using the XOP to do this. It's not that there is a problem with the XOP, but Igor 6.1 and greater has this functionality built in, and so it will simplify your life if you use the built in functionality.

Here is an example that's almost directly copied from the Igor help:

Function/S DoLoadMultipleFiles()
	Variable refNum
	String message = "Select one or more files"
	String outputPaths
	String fileFilters = "Data Files (*.txt,*.dat,*.csv):.txt,.dat,.csv;"
	fileFilters += "All Files:.*;"
 
	Open /D /R /MULT=1 /F=fileFilters /M=message refNum
	outputPaths = S_fileName
 
	if (strlen(outputPaths) == 0)
		Print "Cancelled"
	else
		Variable numFilesSelected = ItemsInList(outputPaths, "\r")
		Variable i
		for(i=0; i<numFilesSelected; i+=1)
			String path = StringFromList(i, outputPaths, "\r")
			Printf "%d: %s\r", i, path
			// Add commands here to load the actual waves.  An example command
			// is included below but you will need to modify it depending on how
			// the data you are loading is organized.
			//LoadWave/A/D/J/W/K=0/V={" "," $",0,0}/L={0,2,0,0,0} path
		endfor
	endif
 
	return outputPaths		// Will be empty if user canceled
End

You will have to work out a LoadWave command that works for the type of data you are trying to load. You should carefully read the command help for the LoadWave operation. I also recommend reading the help for how to load waves. You can access that by executing the following command in Igor:

DisplayHelpTopic "Loading Waves"

If you need additional help figuring out how to load your data, feel free to send an email to support@wavemetrics.com describing how you want the data to be imported into Igor and also send a sample data file. Or you can post those things here if you'd prefer.


Posts: 593
Joined: 2007-06-21
Location: United States

For the relevant help, execute this in Igor:

DisplayHelpTopic "Loading Waves Using Igor Procedures"


hskim
Posts: 3
Joined: 2010-02-23
Location: United States

I tried all the codes here and feel much better.
Now I can import several data files and things much easier. As you guys recommended, I read help topics and tried another code such as 'Loading All of the Files in a Folder', etc..BTW, I got an error message while compiling the sample code and couldn't figure out what was wrong. It says,
"unknown/inappropriate name or symbol

Error in Procedure:LoadAndGraphAll
result=LoadAndGraph(fileName,pathName"
with 'LoadAndGraph' highlighted.

Here's my problem. Unfortunately, I'm not 100% understanding all the codes. but for now just wanna try to combine and modify sample codes to make my work easier.

just one more quick question. To load multiple files, sample code from aclight works fine but I have more than 20 files in the folder so when the waves automatically named it's really hard to tell them. is it possible to name all the waves from file name autometically? If this issue also included in the help file let me know. I'll search them again.. Thanks again~

I kinda like plot quality and the way Igor handling plots...but the troubles I mentioned here are relatively difficult and sometimes annoying to me than Origin...but I'm keep trying to learn cause Igor definitely has some advantages for my work if I get used to it.


[ last edited March 13, 2010 - 18:10 ]
Posts: 593
Joined: 2007-06-21
Location: United States

>I got an error message while compiling the sample code

The LoadAndGraphAll function calls the LoadAndGraph function so you need to copy and paste both functions from the help into the procedure window.

>is it possible to name all the waves from file name autometically?

Yes. You can use the file name as an input to the LoadWave /A flag which sets the base name for the waves created by LoadWave. For example:

String baseName = CleanupName(fileName,0)
LoadWave /A=$baseName

Alternatively you can use the LoadWave /B flag but this gets complicated.

It is easier for me to write some starter code for you than to explain the details here. If you want me to do that, create a zip archive of some sample files and send them to support@wavemetrics.com along with an explanation of what you want to do with the columns and how you want to name them. If it is OK with you, I will write some starter code and post it on IgorExchange as an example.


uvempati
Posts: 1
Joined: 2010-04-13
Location: United States

hrodstein wrote:

Alternatively you can use the LoadWave /B flag but this gets complicated.

It is easier for me to write some starter code for you than to explain the details here. If you want me to do that, create a zip archive of some sample files and send them to support@wavemetrics.com along with an explanation of what you want to do with the columns and how you want to name them. If it is OK with you, I will write some starter code and post it on IgorExchange as an example.

I have a slightly different problem - I am trying to read a file that looks something like below:

1 5
1 0 0 0 0
2 0.1 0.1 0.1 0.1
3 0.2 0.2 0.2 0.2
4 0.3 0.3 0.3 0.3
5 0.4 0.4 0.4 0.4
2 5
1 0 0 0 0
2 0.1 0.1 0.1 0.1
3 0.2 0.2 0.2 0.2
4 0.3 0.3 0.3 0.3
5 0.4 0.4 0.4 0.4
3 5
1 0 0 0 0
2 0.1 0.1 0.1 0.1
3 0.2 0.2 0.2 0.2
4 0.3 0.3 0.3 0.3
5 0.4 0.4 0.4 0.4

So I have blocks of data in 5 columns separated by a two number row which correspond to a run ID and total number of rows in the subsequent block.

In each of these blocks, first column is my x-variable and columns 2 through 5 are the y-variables. Is there a way for me to write a program to read each block separately and append the wave names of the five columns with corresponding run ID like C1_1, C2_1, C3_1, C4_1, C5_1 for the columns in the first block where as C1_2, C2_2, C3_2, C4_2, C5_2 for columns in second block? I am trying to play with /B flag of LoadWave but I didn't know how to make the N flag of columnInfoStr (where columnInfoStr is input to /B just as in Igor Help) to read a variable.


Posts: 593
Joined: 2007-06-21
Location: United States

I would probably call LoadWave/J to load a single line containing the two numbers and then call it again to load the 5x5 block of numbers, then repeat till done.

Another approach is to use Open/FReadLine/Close to read each line one at a time and to use sscanf to convert the line into numbers, then store the numbers in waves.

If you want help, send a zipped sample file to support@wavemetrics.com with a description of what you want to do with the data, what OS you are running and what version of Igor.

By "what you want to do with the data" I mean, do you want to load each 5x5 block into a 5x5 wave? Do you want to into 5 separate 1D waves? Do you want to contatenate all of the blocks into one wave? And what do you want to do with the run IDs? And how do you want to name the waves?


dannycco
Posts: 2
Joined: 2010-04-20
Location: United States

I am having trouble reading the wave names from the file name. When using:

String baseName = CleanupName(fileName,0)
LoadWave /A=$baseName

with the Open command, I get an error (wave name too long) because fileName returns the entire path to the file, not the file name only. Any way to deal with that?

Thanks!


Posts: 593
Joined: 2007-06-21
Location: United States

Quote:
I get an error (wave name too long) because fileName returns the entire path to the file, not the file name only. Any way to deal with that?
Thanks!

Function/S FilePathToWaveName(path)
	String path		// Assumed to use colon separators
 
	// Get last element of path without extension
	String name = ParseFilePath(3, path, ":", 0, 0)	// e.g., MyFile.txt
 
	name = CleanupName(name, 0)
 
	return name
End


dannycco
Posts: 2
Joined: 2010-04-20
Location: United States

Ahh, ParseFilePath, thanks! I was thinking that it was a list of some kind.


nvhnag
Posts: 1
Joined: 2010-04-22
Location: Taiwan

hello guys, i have a slightly different problem..i have many text files each containing 2 columns..i have to load only the 2nd column from each file and assign the wave name with corresponding filename...

what i did was to put all files in a separate folder..create a text wave containing name of each file in each row (use "multiopenfiles" to get the string containing all the fullpaths/wavenames..then by writing a simple code i made the text wave which now contains one item in each row..since they are full paths, i replaced the unnecessary parts of the strings)...then i used the '"load all waves in a folder" with autonames and constructed a matrix.. then excute another code to construct another matrix only with the alternate columns (since the 1st column in each wave is not required) and extract them as 1d waves but with names from the text wave to each of the 1d waves in the same order..

i suppose there should be some smarter way..any advice ??

thanks in advance.. :)


[ last edited November 3, 2011 - 03:48 ]
Posts: 593
Joined: 2007-06-21
Location: United States

I don't follow what you have done entirely but . . .

You can get a list of file names in a folder using IndexedFile.

You can specify the name to be used for a wave loaded with the LoadWave operation using the /B flag. You can also specify that a column is to be skipped using /B.


Back to top