Every estimator answers the same question: given the data, what value of the unknown parameter should you report? Maximum likelihood gives the most direct answer on offer — the value that makes the data you actually observed most probable. It powers probit, logit and most of what follows OLS, and students bring it to our econometrics tutors across the UK every exam season. This page builds it from a ten-observation example up.
1 · The likelihood principle
Start with a model: each observation is a draw from a distribution indexed by an unknown parameter θ. The joint probability of the whole sample — with the data held fixed and θ left free — is the likelihood function L(θ).
That reversal of roles is the whole trick. Probability asks: given θ, how likely is this data? Likelihood asks: given this data, which θ explains it best? The maximum likelihood estimate θ̂ is the value at which L(θ) peaks.
Make it concrete. A courier firm wants its next-day delivery rate p. You track 10 parcels: 6 arrive next day, 4 don’t. Each parcel is a Bernoulli draw, and independence lets you multiply:
L(p) = p⁶(1 − p)⁴
Try candidates. L(0.3) says six next-day arrivals were a fluke. L(0.9) says four failures were. Somewhere between sits the p under which this exact sample is least surprising.
2 · The log-likelihood and the score
Products of probabilities are miserable to differentiate, so take logs. The log is monotonic — it reorders nothing — so ℓ(θ) = ln L(θ) peaks at exactly the same θ̂. For the parcels:
ℓ(p) = 6 ln p + 4 ln(1 − p)
The derivative of ℓ with respect to the parameter is the score. Maximising is now calculus you know: set the score to zero and check the second derivative is negative. Because ℓ here is concave everywhere in (0, 1), the stationary point is the maximum.
That is the entire method — write the likelihood, log it, differentiate, solve — and the diagram below shows it: a concave hill, the estimate at the summit, the slope there exactly zero.
3 · How good is the estimator?
Three large-sample properties make MLE the default beyond OLS. Under standard regularity conditions the MLE is consistent (it converges to the true θ as n grows), asymptotically normal (so standard errors exist), and asymptotically efficient — no consistent rival beats its large-sample variance, the Cramér–Rao benchmark.
The precision comes from curvature. A sharply peaked log-likelihood pins θ down tightly; a flat one leaves it vague. Formally, the asymptotic variance is the inverse of the information — minus the expected second derivative of ℓ — so more data means more curvature and a smaller standard error.
What the list does not include is unbiasedness in small samples. The classic case: for normal data, the MLE of the variance divides the sum of squared deviations by n, not n − 1, so it is biased downward in any finite sample. The bias vanishes as n grows — consistency survives — but examiners love the distinction.
4 · Where OLS fits in
Assume the regression error is normal and write the sample’s log-likelihood. Maximising it over the coefficients is exactly minimising the sum of squared residuals: under normality, OLS is the maximum likelihood estimator of the coefficients. Two methods you learn separately are one method in that special case — and MLE is the one that still works for probit, logit and anything else with a likelihood you can write down. The wrinkle repeats section 3: the likelihood route divides the residual variance by n, biased low, where the regression formula uses degrees of freedom.
Worked example — the courier’s next-day rate
The courier’s sample again: 10 parcels, 6 next-day arrivals. Find the MLE of p.
Step 1 — Write the likelihood. Independent Bernoulli draws multiply: L(p) = p⁶(1 − p)⁴.
Step 2 — Log it and take bearings. ℓ(p) = 6 ln p + 4 ln(1 − p). Try values: ℓ(0.5) = −6.93, ℓ(0.6) = −6.73, ℓ(0.7) = −6.96. (Log-likelihoods of data are negative — closer to zero is better.) The hill crests near 0.6.
Step 3 — Set the score to zero. ℓ′(p) = 6/p − 4/(1 − p) = 0 gives 6(1 − p) = 4p, so p̂ = 6/10 = 0.6. The MLE is the sample frequency — the formula confirms the intuition.
Step 4 — Confirm it is a maximum. ℓ″(p) = −6/p² − 4/(1 − p)² is negative everywhere; at p̂ it equals −41.67. Concave, so 0.6 is the global peak, with ℓ(0.6) = −6.73.
Step 5 — Perturb: two more parcels arrive, both next-day. The counts become 8 of 12. The same score equation now gives p̂ = 8/12 ≈ 0.667. New data moves the peak; the method is unchanged.
Step 6 — Resolve the precision. The plug-in standard error is √(p̂(1 − p̂)/n): √(0.24/10) ≈ 0.155 with ten parcels, √(0.222/12) ≈ 0.136 with twelve. More observations, more curvature, tighter estimate.
Step 7 — Interpret. The MLE reports the delivery rate under which the observed sample is least surprising, and the sharpening standard error says how firmly the data insist on it. Nothing was assumed beyond the Bernoulli model itself.
Could you run the full derivation — likelihood, log, score, second-order check — without a wobble? That four-step routine, plus knowing where MLE is unbiased and where the small-sample bias bites, is exactly what likelihood questions test. Drilling it on your own past papers is what a one-on-one econometrics tutor does with you. Book a trial session.
Practice
Q1. A lender observes 20 borrowers; 15 repay. Write ℓ(p) for the repayment rate, find p̂, and compute ℓ(p̂) and ℓ(0.5) to 2 d.p. to confirm the peak beats the coin-toss value.
Q2. Four observations of a normally distributed return: 2, 4, 6, 8. Compute the MLE of the mean, the MLE of the variance, and the unbiased sample variance s². Which is smaller, and by what factor?
Q3. A café logs online orders on four days: 1, 2, 3, 2. Under a Poisson model, find the MLE of the arrival rate λ and verify the score Σxᵢ/λ − n is zero there.
Answers. Q1: ℓ(p) = 15 ln p + 5 ln(1 − p); p̂ = 15/20 = 0.75; ℓ(0.75) ≈ −11.25 > ℓ(0.5) ≈ −13.86. Q2: μ̂ = 5; σ̂² = 20/4 = 5; s² = 20/3 ≈ 6.67. The MLE is smaller by the factor (n − 1)/n = 0.75 — the downward bias of section 3. Q3: λ̂ = sample mean = 8/4 = 2; score at 2: 8/2 − 4 = 0.
Key takeaways
- The likelihood flips probability around: L(θ) is the probability of your fixed sample, read as a function of the free parameter. The MLE θ̂ sits at its peak.
- The recipe is fixed: log the likelihood, set the score to zero, check concavity. For Bernoulli data the answer is the sample frequency; for Poisson, the sample mean.
- Large samples are MLE’s home turf: consistent, asymptotically normal, efficient — but not always unbiased, and the n-divisor variance estimator is the standard counterexample.
- Under normal errors, OLS is MLE for the regression coefficients — one framework, of which least squares is the special case.
Why UK students choose our econometrics tutoring
- Degree-level specialists: tutors with postgraduate econometrics training teach exactly the likelihood material your module examines — score equations, information, the standard counterexamples.
- Your course’s notation: whether your lecturer writes ℓ(θ) or log L, sessions run on your own lecture notes and past papers.
- One-on-one and online: each session works through derivations live on a shared screen, at the pace the algebra actually needs.
FAQ
Q: What is maximum likelihood estimation in simple terms?
A: Pick the parameter value that makes the data you observed most probable: write the probability of your sample as a function of the unknown parameter, and find where that function peaks. That peak is the estimate.
Q: Why maximise the log-likelihood instead of the likelihood?
A: The log turns a product of probabilities into a sum, which is far easier to differentiate. The log is monotonic, so both functions peak at the same value — the estimate is identical either way.
Q: What is the score function?
A: The first derivative of the log-likelihood with respect to the parameter. The MLE sets it to zero, and the curvature of the log-likelihood around that point determines the estimator’s precision.
Q: Is the MLE always unbiased?
A: No. MLE guarantees are asymptotic — consistency, normality, efficiency as n grows. In small samples it can be biased: the normal-model MLE of the variance divides by n rather than n − 1, so it sits below the unbiased estimator.
Q: How is OLS related to maximum likelihood?
A: If the regression errors are normal, maximising the likelihood over the coefficients is the same problem as minimising squared residuals — the estimates coincide exactly. MLE then extends to models OLS cannot handle, like probit and logit.
Book a specialist econometrics tutor in the UK
Likelihood questions reward students who can run the full derivation — likelihood, log, score, second-order check — without a wobble. One-on-one sessions drill exactly that on your course’s own past papers. Tell us your module and exam date, and we’ll match you with the right tutor this week.
