Some outcomes have only two states: a loan is approved or refused, a borrower defaults or keeps paying, a graduate is employed or not. When the outcome is a yes-or-no, ordinary regression on a 0/1 variable quietly misbehaves — and fixing it properly is one of the most-tested ideas in intermediate econometrics, which is why so many students bring it to a Warwick econometrics tutor. This page builds the fix: why a straight line fails, and how logit and probit repair it.
1 · Binary outcomes and the linear probability model
Write the outcome as y = 1 when the event happens, 0 when it does not. You model not y but the probability of the 1, P(y = 1 | x), against a driver x such as income. It must stay in [0, 1], and cannot respond to x at a constant rate.
Regress the 0/1 variable on x by least squares and you get the linear probability model:
P(y = 1 | x) = β₀ + β₁x.
Its slope reads off as a probability change, but it breaks three ways. The line leaves [0, 1] outside the incomes x = −β₀ ⁄ β₁ and x = (1 − β₀) ⁄ β₁, where it crosses 0 and 1. It forces one constant marginal effect, impossible near the boundaries. And its errors are heteroskedastic — variance P(1 − P) depends on x — so it needs robust standard errors from a separate lesson.
2 · The fix: send the index through a CDF
Keep the linear part — the index β₀ + β₁x — but pass it through a function F that squeezes any number into (0, 1):
P(y = 1 | x) = F(β₀ + β₁x).
A cumulative distribution function does exactly this, rising smoothly from 0 to 1 in an S-shape — flat near 0, steep in the middle, flattening toward 1. It cannot leave [0, 1], and its slope tapers on its own in the tails.
Two CDFs dominate. The logit uses the logistic, Λ(z) = 1 ⁄ (1 + e−z); the probit uses the standard normal, Φ(z). They behave almost identically: rescale one to the other — logit coefficients run about 1.6 times the probit coefficients, the standard rule of thumb — and the curves lie almost on top of each other, as the diagram shows against a straight linear fit.
3 · The trap: a coefficient is not a marginal effect
In the linear model the coefficient was the marginal effect; in logit and probit it is not, because the probability runs through a curve — the single most-tested point on this topic. The effect of x is
marginal effect = β₁ · f(β₀ + β₁x),
the coefficient times the density f at the applicant’s index — tallest at the centre, tiny in the tails. So a rise in income moves the probability most near a 50-50 chance, barely at all near certainty. The effect differs at every x; report it at a chosen point (the average applicant, or the sample average of effects), never as the raw coefficient.
The logit also has a clean odds reading: its log-odds are linear, so a unit rise in x multiplies the odds by eβ₁, a constant odds ratio. Both are fit by maximum likelihood, not least squares, in a lesson of its own.
Worked example — approving a loan by applicant income
A lender models approval (y = 1) or refusal (y = 0) against annual income x, in £10,000s, so x = 4 is £40,000. A fitted logit gives the index z = −2 + 0.5x; the numbers are stylised for clean arithmetic.
Step 1 — The model. P = Λ(−2 + 0.5x). The index is 0 at x = 4, so £40,000 is where approval is an even chance.
Step 2 — Logit predictions. At £10,000 (x = 1), z = −1.5 and P = 0.18; at £40,000, z = 0 and P = 0.50; at £80,000 (x = 8), z = 2 and P = 0.88 — all inside (0, 1).
Step 3 — A linear fit on the same data. Read instead the OLS line P = −0.3 + 0.2x. At £40,000 it agrees, P = 0.50, but at £10,000 it gives −0.10, a negative probability, and at £80,000 it gives 1.30. It crosses 0 at £15,000 and 1 at £65,000; beyond those it is nonsense, which the S-curve never is.
Step 4 — Marginal effect at the middle. The logit effect is 0.5 · Λ(1 − Λ). At £40,000, Λ = 0.5, so it is 0.5 × 0.5 × 0.5 = 0.125: a £10,000 rise lifts approval by about 12.5 points. The coefficient is 0.5, the effect is 0.125 — mistaking one for the other overstates it fourfold.
Step 5 — Marginal effect in the tail. At £80,000, Λ = 0.88, so the effect is 0.5 × 0.88 × 0.12 = 0.053, about 5.3 points — under half the middle value, because approval is already likely. Probit gives 12.5 and 5.7 points, a whisker away: the curve barely matters, the variation across income does.
Step 6 — Odds, and the lesson. Since e0.5 ≈ 1.65, each extra £10,000 multiplies the odds by about 1.65 — a constant 65% lift, while the probability lift fell from 12.5 to 5.3 points. The coefficient fixes the odds, the CDF the shape; the probability effect is computed at a point, never read off.
Given a fitted logit, could you turn its coefficient into a marginal effect at a chosen income — and say why that effect is largest at a 50-50 chance and tiny in the tails? Deriving the β·f(index) rule and reporting an effect at a point, rather than reading a coefficient as if it were the answer, is exactly what binary-choice questions reward. A one-on-one econometrics tutor works logit and probit through with you until the marginal effect is something you compute, not confuse with the coefficient. Book a trial session.
Practice
Q1. With the logit fit P = Λ(−2 + 0.5x), x in £10,000s: find P and the marginal effect at £60,000.
Q2. For that fit, at which income is the marginal effect largest, and how large?
Q3. For the linear fit P = −0.3 + 0.2x: below what income is P negative, and how does its slope compare with the logit effect at £40,000?
Answers. Q1: x = 6, z = 1, P = Λ(1) = 0.73; effect 0.5 × 0.73 × 0.27 = 0.098, ≈ 9.8 points. Q2: β₁·Λ(1 − Λ) peaks at Λ = 0.5, i.e. £40,000: 0.5 × 0.25 = 0.125, 12.5 points. Q3: P < 0 below £15,000; its flat 20 points per £10,000 overstates the logit's 12.5 at £40,000, more so in the tails.
Key takeaways
- Model the probability, not the value. P(y = 1 | x) must stay in (0, 1) and cannot respond to x at a constant rate.
- The linear probability model breaks twice — probabilities outside [0, 1], and a constant marginal effect — with heteroskedastic errors on top.
- Logit and probit pass the index through a CDF. The S-shape is bounded and steepest in the middle; the two agree under the 1.6 rule of thumb.
- A coefficient is not a marginal effect. The effect is β·f(index): largest near 50-50, tiny in the tails, different at every x — the most-tested point.
- Logit has a clean odds reading. A unit rise in x scales the odds by a constant eβ, though the probability effect varies.
Why Warwick econometrics students choose our econometrics tutoring
- Marginal effects done properly: sessions drill the β·f(index) formula and the average-versus-at-a-point distinction until you can report an effect correctly under exam pressure, not just quote a coefficient.
- Both models, on your own software: a tutor works logit and probit through in the package your module uses — Stata, R or Python — so your problem-set output matches the theory.
- Matched to your course: every one-on-one session runs on your own lecture notation and past papers, mirroring exactly what your examiners set.
FAQ
Q: Logit or probit — which should I use?
A: They tell the same story on almost any dataset; use whichever your course or software defaults to.
Q: Why not just run OLS on a 0/1 variable?
A: For a quick look you can, but it predicts probabilities outside [0, 1], forces a constant marginal effect, and has heteroskedastic errors. Logit and probit fix the first two.
Q: Why isn’t the logit coefficient the effect on the probability?
A: Because the probability runs through a curve: the effect is the coefficient times the curve’s slope, so it changes with x — largest in the middle, near zero in the tails.
Q: What does the 1.6 rule of thumb mean?
A: Logit coefficients run about 1.6 times the matching probit coefficients (a scale difference between the distributions); divide by 1.6 to compare the two.
Q: How are these models estimated?
A: By maximum likelihood — the coefficients making the observed ones and zeros most probable. Least squares is not used; the software does the maximisation.
Book a Warwick econometrics tutor online
Binary-choice models reward students who can derive a marginal effect, not just name a coefficient — the CDF, the β·f(index) rule, and the odds reading behind logit and probit. One-on-one sessions build that fluency on your own past papers and software. Tell us your module and exam date, and we will match you with the right tutor this week.
