Constrained Nonlinear Curve Fit VI: Termination Revisited (LabVIEW 8.5)

[DonRoth]

Hello:

In the Constrained Nonlinear Curve Fit VI, the Help for the VI states that if the number of iterations exceeds max iterations specified in the termination control, the fitting process terminates. Would this not indicate that if the above condition is met, the number of iterations recorded (an output of this VI) should be equal to the max iterations specified (since the VI is exits at this point).  I believe I have seen situations where this condition is met, yet the number of iterations recorded is greater than the max iterations I have specified in the termination control.  Has anyone else seen this?

Thanks,

Don

[DonRoth]

For example, the maximum iterations I specify is 1000.  The # of function calls that is returned after running the VI = 63000, and I get the message / error that the number of iterations was exceeded.  But I seem to get a reasonably good curve fit. Is the curve fit shown that for 1000 function calls (as the Help indicates since the routine is supposed to exit once maximum iterations is reached) or for 63000 calls?  (I am assuming # of function calls = iterations.)..Need to know what is going on here, thanks..Don


Message Edited by DonRoth on 04-11-2008 10:31 AM

[altenbach]

---

DonRoth wrote:
(I am assuming # of function calls = iterations.)..Need to know what is going on here, thanks..Don

---

The # of function calls incremented whenever the model function is called. If you use numerical derivates it would need to evaluate the function N+1 times if you have N parameters. (You can open the code and inspect it, it is not passworded.).

For example, if you have paramters A, B, C, it would need to calculate f(A,B,C), f(A+delta,B,C), f(A,B+delta,C), f(A,B,C+delta) for each iteration. How many parameters do you have?

If the function does not converge, but the result is satisfactory, it could mean that your tolerance is too low. How does the correlation matrix look like? Are any parameters strongly correlated?

[altenbach]

Actually, it seems it uses "Order 2" for the derivatives, meaning it uses 2N+1 function calls if you have N paramters. While the actual code contains provisions for order1 order2 and order4, order2 is hardwired in the code and not exposed to the user.

If you have paramters A, B, C, it would need to calculate f(A,B,C), f(A+delta,B,C), f(A,B+delta,C), f(A,B,C+delta),  f(A-delta,B,C), f(A,B-delta,C), f(A,B,C-delta)

[DonRoth]

It is a fairly complicated curve fit using a mixed Guassian-Laurentzian + polynomial model with 4 peaks (and weighting).  So I have about 30 parameters.

I can vary the termination criteria - all I am trying to figure out here is whether the '# of function calls' spit out by the VI is the same parameter as the "number of iterations" (specified in the termination criteria).  It sounds like you are telling me they are not the same thing.

Sincerely,

Don

[altenbach]

No, since you get 63 function calls per iteration, it means you have 31 parameters. (2x31+1=63). It needs to evaluate the model 63 times per iteration to get all the partial derivatives needed. It also means that you might be able to speed up your fitting by a factor of ~2 of you would customize the code to use order1.

That's the cost of Lev-Mar. 🙂

[DonRoth]

Thanks for the explanation.  So the routine does exit after the 1000 iterations.  But I will tell you - the fit looks awfully good.  Right now, I can adjust the tolerance or even increase the max iterations.  I'm not that concerned with time right now.  I do want to implement some other routines besides the Lev-Marq and TRDL at some point as well.

I'm still also interested in your 2d curve fitting.  I found your presentation at NIWeek a few years back quite interesting.

Sincerely,

Don


Message Edited by DonRoth on 04-11-2008 04:16 PM

Message Edited by DonRoth on 04-11-2008 04:16 PM

[DonRoth]

Is the 'tolerance' termination criteria evaluated during the run by comparing with the mean square error (MSE) / residue?

Thanks,

Don

[DSPGuy]

Don,

Just wanted to clarify the termination criteria for the nonlinear curve fitting routines.  There are two criteria used to terminate the fitting process:
"max iteration" and "tolerance".  The main while loop terminates when either of these conditions is met, so:
IF( (current_iteration > max_iteration) OR (current_tolerance <= tolerance) ) THEN (terminate loop)

"current iteration" is just the current loop counter(starts at 1 for the first iteration).  This term is mainly a guard against having the fitting process run too long.

tolerance is computed as the relative change between the current and previous weighted least squares values.
Let wls = SUM_OVER_ALL_X( weight(x)*(f(x,a)-data(x))^2 )
current_tolerance = ABS(current_wls - previous_wls)/(ABS(current_wls) + machine_epsilon)
where ABS indicates absolute value.  Adding machine_epsilon to the denominator is just a guard against division by zero.

For the bound nonlinear curve fit VIs, if the "method" input is chosen to be LAR or bi-square, then wls is defined as:
wls = SUM_OVER_ALL_X( weight(x)*reweight(x)*(f(x,a)-data(x))^2 )
where reweight(x) is a term reducing the weight of high-leverage data points.

For both the constrained and unconstrained fitting routines, if the weight input is unwired then all points are considered to have a weight of 1.

Christian has nicely covered the difference between the number of iterations and the number of function calls.  If your

When the fitting process terminates the best result obtained so far is returned.  As you noted, this may be a good result, although the algorithm may not have completely converged yet.

-Jim

[DonRoth]

Ok - I expect this clarifies also the difference between wls and mse as mse should be

wls/n

where n = number of data points.

Thanks,

Don