Igor not utilizing full CPU power

Share icon

Sun, 09/27/2015 - 05:29 pm

In my Lab we have a PC equipped with 3.6 GHz intel quad-core i7 processor, with 8 GB of RAM. I am trying to open a file that is ~650 MB large and it is basically taking Igor more than 5 minutes to open it.

I noticed that, in Task Manager, Igor is only utilizing 13% of the CPU power, and the rest of the processes are utilizing another 2%, while the remaining 85% is idle.

Is there anything to do so that Igor Pro can utilize more of the CPU power?
Your file opening won't be CPU-bound, it'll be IO bound. You could speed this up by going from a hard disk to a flash drive.

September 27, 2015 at 11:10 pm - Permalink

650MB is not very large, and I wouldn't expect Igor to take 5 minutes to open the file, unless there is some additional processing going on or Igor hooks being executed, for example when an experiment is opened.

andyfaff's response is correct. I wonder if you're loading this file from a network drive/file share. If so, try copying the file to a local disk first and then open from there.

September 28, 2015 at 08:42 am - Permalink

What types of files are you loading?
Are you using standard igor tools or some self written/third party code to load the files?

September 28, 2015 at 01:47 pm - Permalink

I am loading the file from a local disk (HDD), it is a standard .itx Igor file, no special tools / procedures.

September 30, 2015 at 07:26 pm - Permalink

I can confirm ali8's report.

The following code
Function DoStuff()
    variable passedTime = stopmstimer(-2)
    KillWaves/Z
    LoadWave/T "E:VT004.itx"
    printf "Elapsed time %g\r", (stopmstimer(-2) - passedTime)/1e6
End


outputs

Elapsed time 423.495

From reading the LoadWave docs it looks like loading igor text waves is not as highly tuned as loading delimited text data.
Your data holds 10 million rows and igor seems increase the wave's sizes in small steps.
Do you create the igor text waves yourself? I'm just asking because using igor binary waves would be more accurate for floating point data, smaller and much faster.

EDIT: I'm reading the data from an Intel SSD.

October 7, 2015 at 12:36 pm - Permalink

I am creating it myself. Specifically, the LabView program that interfaces the instruments outputs this format.

October 7, 2015 at 04:31 pm - Permalink

The Igor Text format was invented on computers with 1 MB of RAM (running at 8 MHz clock speed). It is not a good way to load 650 MB of data.

Three options that would be much faster:

1. General binary
Use the GBLoadWave operation to load the data. Execute this for details:
DisplayHelpTopic "GBLoadWave XOP"



2. HDF5
You need to activate the HDF5 XOP and use it to load the data. Execute this for details:
DisplayHelpTopic "HDF5XOP"


3. TDMS
You need to activate the TDM XOP (Windows only) to load the data. Execute this for details:
DisplayHelpTopic "TDM XOP"

October 7, 2015 at 07:08 pm - Permalink

You can also trick Igor to load the data as delimited text. You loose the wavenames and the scaling though.

Function DoStuff()
    variable refTimer

    KillWaves/Z/A
    refTimer = stopmstimer(-2)
    LoadWave/A/J/L={0, 3, 10e6, 0, 0} "E:VT004.itx"
    printf "Elapsed time: %g\r", (stopmstimer(-2) - refTimer) / 1e6
End


takes below 9 seconds here.

October 9, 2015 at 01:24 am - Permalink