Logistic regression predicts the probability that an example belongs to a class, such as whether a customer will churn.
How It Works
Like linear regression, it computes a weighted sum of features. That sum is passed through the sigmoid (logistic) function, which squeezes any number into the range 0 to 1, giving a probability. A threshold — often 0.5 — turns the probability into a yes/no decision.
Interpreting It
Coefficients describe how each feature shifts the log-odds of the outcome. Exponentiating a coefficient gives an odds ratio: a value of 1.5 means a one-unit increase multiplies the odds by 1.5. This interpretability is a major reason logistic regression is used in regulated settings such as credit scoring.
Multiple Classes
For more than two classes, multinomial logistic regression predicts a probability for each class, and the probabilities sum to one.
Practical Tips
- Scale numeric features so the optimiser converges and regularisation treats them fairly.
- Use regularisation (the
Cparameter in scikit-learn) to avoid overfitting. - Choose the decision threshold based on the costs of false positives and false negatives, not by default.
- Check calibration: do predicted probabilities match observed frequencies?
When to Use It
It is a strong, fast, explainable baseline for classification, and often competitive on well-engineered tabular features.
from sklearn.linear_model import LogisticRegression
clf = LogisticRegression(max_iter=1000).fit(X_train, y_train)
proba = clf.predict_proba(X_test)[:, 1]