Remember that with linear regression, the prediction equation minimizes the squared residual values (meaning it picks the line through the data points that has the smallest sum of the squared vertical distance between the line and all of the points). With logistic regression, all of the observed values are coded as 0 (failure) or 1 (success). Similar to simple linear regression, simple logistic regression also has a single independent (X) variable. The goal of logistic regression is to predict the probability of observing a 0 or 1, and simply fitting a straight line to the data by minimizing the sum of the squared distance from the points to this line would result in a nonsensical model (discussed in the previous section on “How simple logistic regression differs from simple linear regression”).

So then how is the model fit? The model maximizes what statisticians call the likelihood. In very general terms, this process of maximizing the likelihood can be thought of as choosing the coefficient estimates for the model that would be most likely to generate the observed data. Something to keep in mind, therefore, is that logistic regression models will generally perform better at fitting (or classifying) the input data than they will at correctly predicting outcomes of new data.

Another key point to remember about logistic regression is that, unlike with linear regression, maximizing the likelihood for logistic regression requires an iterative solver, meaning that it picks some starting values and makes educated steps "up the mountain" towards the largest likelihood value in order to identify the maximum. This also means that – unlike linear regression, but similar to non-linear regression – there are times when the simple logistic regression model cannot be determined for a given set of data. In “statistician-speak”, we would say that the maximum likelihood cannot be determined, or simply that the model did not converge.

There are a few common reasons why a simple logistic regression model may fail to converge, and these are covered on a separate page discussing each case.