speed up Igor loops

Share icon

Sun, 10/01/2017 - 04:43 pm

Iteration loops (do-while, for) in Igor programs (user functions) seem to execute very slowly on Igor compared to other compiled programming languages. I typically need to modify individual elements of waves, which seems to dramatically slow down execution. Aside from multithreading, are there any recommended methods to make iteration loops run faster on Igor?

I am using Igor Version 7.0.5.2 on Windows 7 Professional.
If you post the code in question, perhaps we can help you figure out a faster implementation.

Please enclose the code in <igor> and </igor> tags so that it gets properly syntax-colored:

Function MyFunction(Variable v)
... code here ...
End


--Jim Prouty
Software Engineer, WaveMetrics, Inc.

October 1, 2017 at 05:09 pm - Permalink

Modifying wave elements with for loops can be slow when you are dealing with extremely large waves and very fast operations, like 2+2 or similar.

If you are looking for speed use commands such as FastOP, MatrixOP and wave indexing (p, q, r), see DisplayHelpTopic "Waveform Arithmetic and Assignment". But depending on what you are doing that might not always be possible.

October 2, 2017 at 03:34 am - Permalink

markusic wrote:
Iteration loops (do-while, for) in Igor programs (user functions) seem to execute very slowly on Igor compared to other compiled programming languages. I typically need to modify individual elements of waves, which seems to dramatically slow down execution. Aside from multithreading, are there any recommended methods to make iteration loops run faster on Igor?

I am using Igor Version 7.0.5.2 on Windows 7 Professional.


I did tests and the results show that Igor programs run faster than Python,Matlab and Origin, greatly faster!
Your case may not be due to Igor. To find the reason it would help if you can show your code.

By the way, do-while loops run quite slowly in proc and macro. for loops can not be used under proc and macro. Please check whether you are using do-while loops in proc or macro!

October 3, 2017 at 05:09 am - Permalink

wings wrote:

I did tests and the results show that Igor programs run faster than Python,Matlab and Origin, greatly faster!
Your case may not be due to Igor. To find the reason it would help if you can show your code.


Can you share the code for the tests?

October 3, 2017 at 10:40 am - Permalink

thomas_braun wrote:
wings wrote:

I did tests and the results show that Igor programs run faster than Python,Matlab and Origin, greatly faster!
Your case may not be due to Igor. To find the reason it would help if you can show your code.


Can you share the code for the tests?


The test is to calculate fibonacci sequence using recursive algorithm.

1 In Igor:
function test2()
         variable ref=startmstimer
         variable v=fibonacci(34)
         variable t=stopmstimer(ref)
         printf "Time consumed is %f s,the fibonacci of 34 is %d", t/1000000,v
end
function fibonacci(i)
         variable i
         if(i<2)
                   return i
         endif
         return fibonacci(i-2)+fibonacci(i-1)
end

result is about 2.4 seconds

2 Program an XOP to calculate fibonacci sequence by C and call it from Igor:
XOP C Code:
<br />
struct FibonacciParams {<br />
         doublep1;<br />
         doubleresult;<br />
};<br />
typedef struct FibonacciParamsFibonacciParams; <br />
extern "C" int fibonacci2(int);<br />
extern "C" int<br />
Fibonacci2(FibonacciParams* p)<br />
{<br />
         p->result= fibonacci2((int)p->p1);<br />
 <br />
         return(0);                                              <br />
}<br />
 <br />
extern "C" int fibonacci2(int i)<br />
{<br />
         if(i < 2)<br />
         {<br />
                   return i;<br />
         }<br />
         else<br />
         {<br />
                   return fibonacci2(i - 2) + fibonacci2(i - 1);<br />
         }<br />
}<br />

Igor code:
function test()
         variable ref=startmstimer
         variable v=fibonacci2(34)
         variablet=stopmstimer(ref)
         printf "Time consumed is %f s,the fibonacci of 34 is %d", t/1000000,v
end

result is amazing, about 0.1 seconds, even faster than that in C++ (Visual Studio 2015)

3 In VS 2015, C++
<br />
#include "stdafx.h"<br />
#include "Windows.h" <br />
int fibonacci(int i);<br />
int _tmain(int argc, _TCHAR* argv[])<br />
{<br />
         DWORD start,stop;<br />
         start=GetTickCount();<br />
         fibonacci(34);<br />
         stop=GetTickCount()-start;<br />
         printf("timeconsumed is %lld",stop);<br />
         return 0;<br />
}<br />
 <br />
int fibonacci(int i)<br />
{<br />
         if(i<2)<br />
         {<br />
                   returni;<br />
         }<br />
         return fibonacci(i-2)+fibonacci(i-1);<br />
}<br />


result is about 0.5 seconds

4 In Matlab:

function y=fib(x)
if x<2
y=x;
return;
else
y=fib(x-2)+fib(x-1);
end

result is about 73 seconds

5 In OriginPro 2016
<br />
// Start your functions here.<br />
int fib(int i)<br />
{<br />
    if(i<2)<br />
    {<br />
        return i;<br />
    }<br />
    return fib(i-2)+fib(i-1);<br />
 }<br />
<br />
int test()          <br />
{<br />
       DWORD dw1 = GetTickCount();<br />
    int n=fib(34);<br />
    DWORD dw2 = GetTickCount();<br />
    printf("the 34th febonacci is %d, tome consumed is %d\n",n,(GetTickCount()-dw1)); <br />
    return 0; <br />
}<br />

result is about 138 seconds

BTW, the result depends the performance of the computer. The tests above were done on a notebook PC.
In my new desktop PC with higher performance, time required in Igor Pro 7 is about 1.2 s.

October 3, 2017 at 05:25 pm - Permalink

Interesting. I ran this code in Maple 2017.

ffib := proc (x) local y; if x < 2 then y := x else y := ffib(x-2)+ffib(x-1) end if; y end proc:

Time was ~12.6 s.

I ran your code in Igor64. Time was 1.2s

MacBook Pro Late 2013 with 16GB RAM.

(I am curious what you get with Matlab and Origin on the higher performance system where the Igor Pro / Igor64 times are comparable to my MBP).

--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH

October 3, 2017 at 09:13 pm - Permalink

jjweimer wrote:
Interesting. I ran this code in Maple 2017.

ffib := proc (x) local y; if x < 2 then y := x else y := ffib(x-2)+ffib(x-1) end if; y end proc:

Time was ~12.6 s.

I ran your code in Igor64. Time was 1.2s

MacBook Pro Late 2013 with 16GB RAM.

(I am curious what you get with Matlab and Origin on the higher performance system where the Igor Pro / Igor64 times are comparable to my MBP).

--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH


do you mean the time required in Matlab and Origin is comparable to Igor to your MBP?

October 3, 2017 at 10:37 pm - Permalink

I am not surprised that Igor is not as fast as C when using low level programing like that. Your code could also be optimized a lot, since you are recalculating the lower Fibonacci values over and over again for each higher value. I would make a wave with 34 elements and calculate the Fibonacci values in increasing order from low to high, taking advantage of the lower values already calculated when calculating the higher ones.

I get 10 ms for the code below

Function Test()
//  Test Function

    Make/O/N=35 FibonacciWave
   
    Variable t=StartMsTimer
   
    FibonacciWave[0]=0
    FibonacciWave[1]=1
    FibonacciWave[2,*]=FibonacciWave[p-2]+FibonacciWave[p-1]
   
    Print(StopMSTimer(t))
    Print(FibonacciWave[34])
end

October 4, 2017 at 12:59 am - Permalink

olelytken wrote:
I am not surprised that Igor is not as fast as C when using low level programing like that. Your code could also be optimized a lot, since you are recalculating the lower Fibonacci values over and over again for each higher value. I would make a wave with 34 elements and calculate the Fibonacci values in increasing order from low to high, taking advantage of the lower values already calculated when calculating the higher ones.


I take this low efficiency algorithm purposely because I want to know the performance of Igor. Recursive algorithm consumes memory and cpu resource greatly, so it is a good choice to test the performance of the Igor's compiler. There are of course better ways to calculate Fibonacci Sequence. The code follows only takes about 4 micro seconds (In my PC, your code takes about 6.6 micro seconds):
function test()
    Variable t=StartMsTimer
    variable i=1
    variable v1=1,v2=1,v3
    do
        v3=v1+v2
        v1=v2
        v2=v3
        i+=1
    while(i<=32)   
    Print(StopMSTimer(t))
end

October 4, 2017 at 01:45 am - Permalink

wings wrote:
... do you mean the time required in Matlab and Origin is comparable to Igor to your MBP?


No. I am curious what YOU get with these two apps on your faster machine. I have yet to update Matlab on my system.

--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH

October 4, 2017 at 06:34 am - Permalink

jjweimer wrote:
wings wrote:
... do you mean the time required in Matlab and Origin is comparable to Igor to your MBP?


No. I am curious what YOU get with these two apps on your faster machine. I have yet to update Matlab on my system.
--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH

I did not test that on my fster machine because I did not not install Matlab and Origin on it...

October 5, 2017 at 07:29 pm - Permalink

I profiled the code in MatLab 2017b. It took about 30 s. After this, it took far less time ... 9 s or less. Because I am ignorant of how MatLab works, I cannot comment on why successive iterations of the function call ran faster.

--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH

October 5, 2017 at 06:56 am - Permalink

Python 2.7.13 |Anaconda 4.4.0 (64-bit) 2.6 seconds
Igor Version: 7.0.5.1 (Build 29707) 1.84 seconds
PyPy2 V5.9.0-win32 140 ms
JavaScript (Chrome Version 61.0.3163.100 (Official Build) (64-bit)) 107 ms
Julia 0.6.0 53 ms
Assembly - who knows how to write assembly? not me!

October 9, 2017 at 10:34 am - Permalink

I'm not sure I agree with the testing procedure here. In most of the code bases I'm working with, the performance of executing functions recursively does not influence the overall performance. Most of the time is spent in shuffling data around and working on the data itself.

October 9, 2017 at 11:41 am - Permalink

Igor's function-call overhead, while not large, is non-trivial compared to a function body that has little to do.

Wow. That sounds like a politician, doesn't it?

You can time raw function-call overhead:
Function objectiveFunc(Variable nn)
end

Function testCall(Variable loops)

    Variable i
    Variable start = StopMSTimer(-2)
    for(i = 0; i < loops; i++)
    endfor
    Variable EmptyLoopTime = StopMSTimer(-2)-start
   
    start = StopMSTimer(-2)
    for(i = 0; i < loops; i++)
        objectiveFunc(i)
    endfor
    Variable CallAndLoopTime = StopMSTimer(-2)-start
   
    print "Function call time:", (CallAndLoopTime - EmptyLoopTime)/(1e6*loops)
end


On my 5k iMac:
•testCall(1e6)
Function call time: 3.30899e-08

Not too bad!

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com

October 9, 2017 at 03:27 pm - Permalink

johnweeks wrote:


.......
start = StopMSTimer(-2)
for(i = 0; i < loops; i++)
objectiveFunc(i)
endfor
Variable CallAndLoopTime = StopMSTimer(-2)-start

.........
Not too bad!



Agreed, especially when considering the for loop is inside the timer.

October 9, 2017 at 09:32 pm - Permalink

johnweeks wrote:
Note that the function runs a completely empty loop and then subtracts the empty loop time from the function-call loop.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com



Sorry I missed that.

Thanks,

- Tim

October 16, 2017 at 10:03 am - Permalink