Fit function slow to find wave references

I have a routine to fit the same function many times.

Because I do fits with
FuncFit customFunc W_coef waveToFit


I have to have a function which takes the arguments customFunc(W_coef,x), where W_coef is a wave... according to my understanding. An example:
function customFunc(W_coef,x)
    wave W_coef; variable x
    wave specialOtherWave = root:specialOtherWave // *** 38%
    return W_coef[0] *x * specialOtherWave[0]
end


In my real function I can't replace specialOtherWave by a constant, like in this example.

I used #include <FunctionProfiling> to test where my code is slow, and 62% of time is spent fitting with half of that (38%) spent simply accessing wave specialOtherWave!

Is there any way around this? It looks like a factor of 2 speedup would be possible. I guess I could pass specialOtherWave in as a part of W_coef, but that's a bit clunky. Is that the only way?
Perhaps you have lots and lots of waves in your root data folder? I can think of a couple of things to try:

1) Make sure specialOtherWave is the first wave in the list of waves in the data folder. The only way to re-order the list of waves is to move all the other waves to another data folder, then move them back.

2) Move specialOtherWave to its own data folder.

3) If specialOtherWave is the same wave every time, you could re-do your fit function as a structure-based fit function. It is cumbersome to use, as you must write a driver function that creates the structure instance and calls FuncFit but if you are already doing a batch fit, then you must already have such a function. Read about structure fit functions here:

DisplayHelpTopic "Structure Fit Functions"

Quote:
I guess I could pass specialOtherWave in as a part of W_coef, but that's a bit clunky. Is that the only way?

I presume you mean that you would copy the contents of specialOtherWave into the coefficient wave. That would work- you would have to hold all the coefficients that represent the contents of specialOtherWave.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
I experience a similar situation. My workaround was to create several fit functions with a systematic name system. The functions included hard coded references, e.g.:
Function StatLaserReferenceEven(wc,wy,wx):Fitfunc
Wave wc, wy, wx
Wave Avg=root:NAC:LaserReference:AverageEven
    MultiThread wy=wc[0]+wc[1]*Avg
End

and usage of the $ operator to select the fit function in the FuncFit-call
FuncFit /N=1 /W=2 /Q $("Stat"+Name+SelectString(Mod(i,2),"Even","Odd")), FitCoef, CurrentPeak


Maybe the all-at-once model I used here is already sufficiently faster, since the wave assignment is only done once for every fit iteration and not for every fit iteration and every x value.

The question is how many "specialOtherWaves" you have (I use about 40 and its ok, if you have ~1000 this approach might be a bad idea).
HJ
HJDrescher wrote:
Maybe the all-at-once model I used here is already sufficiently faster, since the wave assignment is only done once for every fit iteration and not for every fit iteration and every x value.

Yes, it must be the use of an all-at-once fit function that gave you the speed-up because a simple WAVE statement such as you show still has to look up the wave from the linked list of waves. Looking up a wave by name requires walking the list of waves from the start, regardless of how you do the look-up.

I forgot about suggesting an all-at-once fit function to the OP. That could be a way to get a pretty good speed-up without the cumbersome nature of a structure fit function.

HJDrescher- your situation might benefit from a structure fit function as well. It would allow you to put the wave lookup in the function that calls FuncFit, and you won't have to have 40 different fit functions.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
As far as I remember, hard-linking gained about 8% (on top to the ~90% from the all-at-once version). The name of the fit wave was coded as a fit constant and a case structure.
(The only thing on the to-do list for this program would be a test with Igor 7, hence I'm not going to test the structure fit now but I appreciate your suggestion)

Would using fit structures help to handle this kind of fits in a parallelized code? Copying the fitwaves between the threads and other overhead almost killed any speed up (~5% @ 8 HT-cores running 10e6 fits). Due to thesis writing I cannot test it in the near future but it will be good to know.

HJ
In Igor 7 a standard all-at-once function can get some benefit from automatic (internal) multithreading, structure fit functions do not. So for large problems or slow fit functions you might get a speed-up by marking the fit function as threadsafe, and running it on IP7.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
Finally, I've found the time to implement this. The Structure fit gave a 50% faster result, and with some other improvements, I have a 2.6x speedup!

Thanks John.