When performing nonlinear regression, Prism offers the option of identifying unstable parameters or identifying ambiguous fits. These options can be selected on the Confidence tab of the nonlinear regression dialog. The option to identify "Unstable" parameters was changed to be the default option beginning with Prism 9.0. Here is a brief explanation of how this feature works:

1.After completing the fit, Prism prepares the Hessian matrix. This is a square matrix with one row (and one column) for each parameter fit by the regression. The values in the matrix are the product of the partial derivative of the goodness-of-fit as you change the parameter that defines the row times the corresponding partial derivative for the parameter that defines the column. Goodness of fit is usually the  sum-of-squares (or weighted sum-of-squares), but is defined differently for Poisson regression. This matrix defines how sensitive goodness of fit is to changes in the parameters.

2.Prism computes the "condition" of the matrix. This basically quantifies how error-prone it will be to invert the matrix (which is needed as part of curve fitting). The condition is a number. If that number is low, there are no unstable parameters. If the number is high, the matrix is "ill conditioned", which means a small change in the Hessian matrix would result in a large change in the best-fit parameters.

3.Prism concludes there is one or more unstable parameter when the condition is greater than 10/sqrt(epsilon), where epsilon is the 64 bit machine epsilon, 2.22e-16, an upper bound on the relative error due to rounding in floating point arithmetic. So if the matrix condition is greater than about 670 million, Prism concludes that one (or more) of the parameters must be unstable. Of course, this is a somewhat arbitrary cutoff. The threshold of 10 divided by the square root of epsilon is not a typo; we find that using 10 (not 1) in the numerator provides a more useful threshold.

4.Prism identifies the parameter that causes the biggest increase in the Condition of the matrix and removes the corresponding row and column from the Hessian matrix and marks that parameter as "unstable". Then it repeats steps 2 and 3 to see if the Hessian matrix is still unstable, in which case another parameter needs to be removed as unstable.  Repeat until the condition of the matrix is low enough.

5.In the results table, Prism reports the unstable parameter(s), along with their standard error(s) and confidence interval(s), as "Unstable".