How does Prism compute and plot residuals from nonlinear regression?
If you choose (or accept the default) standard weighting, then the residuals are the difference between the actual Y value you entered and the Y value predicted by the model. If the data point is above the curve, the residual is positive. If the data point is below the curve, the residual is negative. Least-squares regression works to minimize the sum of the squares of these residuals.
If you choose another weighting scheme, Prism 5 adjusts the definition of the residuals accordingly. The residual that Prism tabulates and plots equals the residual defined in the prior paragraph, divided by the weighting factor. The most common common alternative weighting is "Weight by 1/Y2 (minimize relative distances squared)". In this case, the residual is defined to be the distance of the point from the curve divided by the Y value of the curve. Weighted nonlinear regression minimizes the sum of these residuals squared.
Note the ambiguity in defining weighting. The Prism dialog gives the choice to weight by 1/Y2. This means that the squared residual is divided by Y2. The weighted residual is defined as the residual divided by Y. Prism minimizes the sum of the squares of these weighted residuals.
Earlier versions of Prism (up to Prism 4) always plotted basic unweighted residuals, even if you chose to weight the points unequally.
When performing linear regression, Prism does not offer weighting so the residuals are always unweighted residuals as defined in the first paragraph above.