question about DoNewGlobalFit()

Share icon

Tue, 04/28/2015 - 08:56 am

I am working on incorporating the global fit package into my procedure file using the DoNewGlobalFit function.

I have a few questions regarding the Options parameter (bitwise flags for options). The help file provides descriptions such as "NewGFOptionAPPEND_RESULTS
Append Results to Top Graph (implies option NewGFOptionMAKE_FIT_WAVES)." but gives no indication as to which bit is associated with this option.

Could someone please provide a description of which bit is associated with each of the following??

NewGFOptionAPPEND_RESULTS
NewGFOptionCALC_RESIDS
NewGFOptionCOV_MATRIX
NewGFOptionFIT_GRAPH
NewGFOptionQUIET
NewGFOptionWTISSTD
NewGFOptionMAKE_FIT_WAVES
NewGFOptionCOR_MATRIX
NewGFOptionLOG_DEST_WAVE
Those are names of constants that you can use in your code. For instance, if you want to have both APPEND_RESULTS and CALC_RESIDS you would write:
Variable gfOptions = NewGFOptionAPPEND_RESULTS + NewGFOptionCALC_RESIDS
DoNewGlobalFit(..., gfOptions,...)

You can conditionally include certain options as well:
Variable gfOptions = NewGFOptionAPPEND_RESULTS + NewGFOptionCALC_RESIDS
ControlInfo QuietCheckbox
if (V_value)
    gfOptions += NewGFOptionQUIET
endif
DoNewGlobalFit(..., gfOptions,...)


John Weeks
WaveMetrics, Inc.
support@wavemetrics.com

April 28, 2015 at 09:35 am - Permalink

Follow up question.

My particular fit function requires access to a few global variables (the function actually fits a convolution of the test function with a system response function) whose location is a sub-folder named for the current data wave being fit (so it changes). Since the global fitting package changes the current data folder (to root:Packages:NewGlobalFit) the fit function no longer has access to those global variables.

question) what would be the best way to ensure the global variables are accessed? I was thinking of placing a string in the root: folder containing a path to the relevant global variables then editing the fit-function to access this string. Is there a better way of accomplishing this?

April 29, 2015 at 10:15 am - Permalink

I can't think of a better solution. Your original data wave is not available inside the fit function, so you can't compute the data folder that way.

What sort of data is in these globals? If they are single numeric global values, perhaps you could put them into the coefficient wave, and hold those coefficients. A bit clunky, and prone to error, but it packages up all the data you need for a given data set into a single unit.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com

April 30, 2015 at 09:27 am - Permalink

The function performs discrete convolution (using for loops) between a measured system response function and a test function (single or multi- exponential) then fits the result to the measured photoluminescence decay. It is the lifetime coefficients which are linked between decays.

The variables contain information such as (system response function wave name), number of exponentials, number of components per exponential (amplitude, lifetime and, optional beta component for a stretched exponential) and the data points over-which, to perform the convolution.

The code is currently running, although the fits seem to fail (see attached image). I'm not sure why it's failing in this manner (the fit baseline in this case was 24.2 so that parameter isn't the issue). I think the data sets may be accidentally including a few points from the next data set, so the first data set thinks it has some large values at the end.
failed global fit.jpg

April 30, 2015 at 10:43 am - Permalink

Quote:
I think the data sets may be accidentally including a few points from the next data set, so the first data set thinks it has some large values at the end.

It does look like that. It seems like a very large mismatch for so few bad points, though.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com

April 30, 2015 at 05:30 pm - Permalink

I wrote my previous reply very quickly as I was getting ready to leave for the day....

Can you test your basic fitting function against a single data set to make sure it works? Only then should you try it with Global Fit.

You might also test it with a single piece of the data set, and make sure it works fitting just one of those decays.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com

May 1, 2015 at 09:09 am - Permalink

The code works normally on a single wave . Testing the Global fit on a single wave, the fit fails (giving similar results to the previously attached image).

Question 1 ) During the global fitting attempt, the coefficient wave within the fit function (checked with a breakpoint) is '_free_' . Is this normal since the global fit package keeps track of all the fit coefficients?

question 2) The global fit package attempts to fit TempYW vs TempXW. If I graph this on the same plot as YCumData, the waves are not the same (in this instance, TempYW matches the previous images "fit" wave exactly (same data points). Why is the data being fit not equal to the data contained within YCumData?

May 4, 2015 at 09:44 am - Permalink

proland wrote:
Question 1 ) During the global fitting attempt, the coefficient wave within the fit function (checked with a breakpoint) is '_free_' . Is this normal since the global fit package keeps track of all the fit coefficients?

Yes. The driver fit function extracts the appropriate coefficients from the large coefficient wave it gets from the curve fit. It makes a free coefficient wave with the correct data for the particular data set it is working on.

Quote:
question 2) The global fit package attempts to fit TempYW vs TempXW. If I graph this on the same plot as YCumData, the waves are not the same (in this instance, TempYW matches the previous images "fit" wave exactly (same data points). Why is the data being fit not equal to the data contained within YCumData?

Hmm... I see that it is "helpfully" removing any non-fitted points from your data set. It removes NaNs and any points corresponding to zero weight or mask wave points. Do you have NaNs in your data?

To see where it is done, go to the function NewGF_CheckDSets_BuildCumWaves(). In the second big loop, you will see
            if (numtype(Ysetw[j]) != 0)
                continue
            endif
           
            if (doMasking)
                if ( (numtype(mw[j]) != 0) || (mw[j] == 0) )
                    continue
                endif
            endif
           
            if (doWeighting)
                if ( (numtype(ww[j]) != 0) || (ww[j] == 0) )
                    continue
                endif
            endif

Those continue statements skip adding a point to the cumulative data sets. I can't remember why I felt that was a good idea. You could make a copy of Global Fit 2.ipf and remove that code and try it out. I'd be interested in the result.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com

May 4, 2015 at 12:27 pm - Permalink

I discovered the NaN and Zero skipping a while ago as it was causing major problems in the fit attempts (since the x-offset between the system response function (SRF) and the data to be fit is a critical parameter). Having the code remove N points from the raw data and M points from the SRF made the fits far worse than they are now.

I currently replace any NaN or Zero with a 1 (practically no effect on the fit) and manually create a sub-set of the data based on the desired fit range. I've confirmed that both the SRF and data subset waves are the ones being used for the fit and that they are of the appropriate size.


I copied a couple data waves into the root: folder and made a simplified version of the fit function (only 1 test function available and most parameters replaced with constants) in order to try using the global fit GUI.

The GUI results were nowhere near as bad as my attempts at using the function directly, although it was far too eager to assign negative numbers to everything. Holding a few coefficients constant while narrowing down others seemed to help prevent absurd values (like an amplitude of 1e-30). The question now is, what's different between the GUI and my coding attempts.

So far, I have been unable to find any problems with the Global Fit waves (DataSets, CoefDataSetLinkage, etc.)

May 4, 2015 at 01:40 pm - Permalink

Ok, so I've tracked down my problem and, long story short, I'm an idiot. I was focused on the global fitting and waves related to it and neglected to notice a problem with the system response function (SRF). When doing a convolution fit, it is important to properly handle the baseline offset of the wave used in the convolution (which my normal code handles). Seems my routine for setting up the global fit was using the original SRF wave, with the baseline still present.

The global fitting results are now greatly improved, although tau2 (bi-exponential decay) is currently underestimated. I'll play around with epsilon values for the fit coefficients to try and get a better result.

Thanks for your help and patience.

May 5, 2015 at 12:05 pm - Permalink