Why does Prism report "floating point error" or "did not converge" instead of the results of a nonlinear regression?
A "floating point" or "math" error means that Prism tried to divide by zero, take the log of a negative number, or do some other impossible computation. This means that curve fitting is impossible for your data, your model, your intial values and your constraints.
The #1 tool for diagnosing problems with nonlinear regression is to tell Prism not to try to fit a curve, but rather to show you the curve generated by the initial estimated values of the parameters. This is a choice at the top of the Diagnostics tab of the nonlinear regression tab in Prism 5, and on the first tab in Prism 4. If that initial curve doesn't go near the points and follow their shape, then you've either picked the wrong equation or have chosen a poor set of initial values.
Here is a short list of potential curve fitting problems with suggested solutions:
| Potential problem | Solution |
| The equation simply does not describe the data. | Try a different equation. |
| The initial values are too far from their correct values. | Enter different initial values. If you entered your own equation, check the rules for initial values. . |
| The range of X values is too narrow to define the curve completely. | If possible, collect more data. Otherwise, hold one of the variables to a constant value. |
| You have not collected enough data in a critical range of X values. | Collect more data in the important regions. |
| Your data are very scattered and don't really define a curve. | Try to collect less scattered data. If you are combining several experiments, normalize the data for each experiment to an internal control. |
| The equation includes more than one component, but your data don't follow a multicomponent model. | Use a simpler equation. |
|
|
If your X or Y values are huge, change the units. Avoid values greater than about 10000. |
|
|
If your X or Y values are tiny, change the units. Avoid values less than about 0.001. |
| You've set a parameter to an inappropriate constant value. | Either don't set the parameter to a constant value, or change that value. Perhaps you made a simple mistake like setting a maximum plateau to 1.0 when it should be 100, or a Hill slope to +1.0 when it should be -1.0. |
Keywords: math error troubleshoot