LoadData from a series of experiments

youpi
Posts: 11
Joined: 2008-05-23
Location: Switzerland

Hi,

I am using loaddata to import waves from one pack experiment to the other.
I use it 2 ways:
LoadData/O/L=1 path+file // when I want all the waves that were in the experiment
LoadData/O/J=objectlist path+file // when I want some waves whose name was defined

Now I would like to load the waves that for example only start with "TOP". So What I have down is that I use LoadData/O/L=1 path+file and then I kill all the waves that do not start with "TOP". The problem is that loading all the experiment for each file is long and I get pretty quickly out of memory.

*So maybe you could tell me if there is a way to load the list of wave in the experiment, simply a string that would contain the wave names as datafolderdir(2) but for the second experiment.
Then I could select the waves using this string and load only the one I need.
If you have any other suggestion I am more than happy to hear them!
Thank you in advance,
Celine


Posts: 1661
Joined: 2007-06-21
Location: United States

Quote:

So What I have down is that I use LoadData/O/L=1 path+file and then I kill all the waves that do not start with "TOP". The problem is that loading all the experiment for each file is long and I get pretty quickly out of memory.

Igor's memory management was improved in Igor Pro 6.00. If you are using an earlier version, upgrade to Igor Pro 6.1 and that should solve the memory problem.

You can download the Igor Pro 6.1 demo from the WaveMetrics web site. It is fully function for 30 days.


[ last edited July 29, 2010 - 09:14 ]
youpi
Posts: 11
Joined: 2008-05-23
Location: Switzerland

Thank you!


xufriedman
Posts: 23
Joined: 2008-05-09
Location: United States

Is this still the state of the art? I would like to load waves from another experiment that begin with some prefix, but I am not coming up with a nice way to do that. This would be straightforward if LoadData had wildcard capability, or WaveList could operate on a different experiment, which I think neither can. Any suggestions for strategy?

-Matthew


HJDrescher
Posts: 349
Joined: 2015-01-20
Location: Germany

Usually I parse the file header using freadline and load the waves of interest via an
LoadWave /O /A /B=TempString /J /L={0,NumberOfSkippedLines,0,0,0} /Q S_FileName
command.
'Tempstring' contains a pattern of skipped (e.g., "C=1,N='_skip_'; ") and parsed (e.g., "C=1,F=0,T=2,N=Frequency; ") parameters for each column.

However, this was on IP6.3x. (If you still use IP6.1, you might want to update...)

HJ

EDIT: I Just realized that the question was on LoadData and not on LoadWave.
Maybe programming the data browser might help?
displayhelptopic "Programming the Data Browser"


[ last edited January 11, 2018 - 15:30 ]
Posts: 1661
Joined: 2007-06-21
Location: United States

Quote:
Is this still the state of the art?

Yes for Igor7.

I am investigating the feasibility of providing more flexibility for Igor8. I should know more in a day or two.


Posts: 1661
Joined: 2007-06-21
Location: United States

For Igor Pro 8, I have added to the LoadData operation the ability to apply regular expressions to control which objects are loaded.

If you are interested in testing this feature, you first need to become an Igor8 beta tester - see https://www.igorpro.net/beta-signup. You will need a valid activation key for an Igor Pro 7 license with to become a beta tester.

Once you have signed up and installed the Igor8 beta, choose Help->Igor Pro Nightly Builds and install the latest nightly build.

Finally, to get documentation for the new LoadData feature, in Igor8 choose Help->Contact Support to create a new email addressed to WaveMetrics support. Enter "LoadData with Regular Expressions" for the subject, enter a brief explanation of what you are trying to do in the body, and send the request to support.


xufriedman
Posts: 23
Joined: 2008-05-09
Location: United States

I knew there was a reason I have been using Igor for decades and decades. Thanks for implementing that. I am certain it will be very useful. However, I'm afraid this is a production routine, so I have to stick with non-beta.
I figured out a workaround. I made people name their waves intelligently (i.e. with a sequential index, i.e. wave0, wave1, wave2, ...), so I could stop loading when LoadData failed to find the file.

function TryLoad (mywavename, sourceFile)
	string mywavename, sourceFile
 
	if (strlen (sourceFile) == 0 || strlen (mywavename) == 0)
		return 0
	endif
	if (exists (mywavename))
		return 1
	endif
	loaddata /q/j=mywavename sourcefile
	return (exists (mywavename))
end
function /t LoadMatches (prefixstr, sourceFile)
	string prefixstr, sourceFile
	variable i
	string resultstr = ""
 
	i = 0
	do
		if (tryload (prefixstr + num2str (i), sourceFile))
			resultstr = AddListItem (prefixstr + num2str (i), resultstr, ";", inf)
		else
			break
		endif
		i += 1
	while (1)
	return resultstr
end

That solution may only work for my particular situation, of course.

-mxf


Back to top