Every hypothesis test is a bet. You bet the null hypothesis is false, and you accept a known chance of being wrong. Statistical inference quantifies that chance — the type I error rate, the type II error rate, and the power of the test. If you are studying introductory statistics in London, these concepts separate a passing grade from a confident one. They are also the first thing a stats tutor London students book for.
1 · The null and the alternative
Every test starts with two hypotheses. The null hypothesis H₀ is the default: no effect, no difference, no relationship. The alternative hypothesis H₁ is what you want to show: a treatment works, a mean differs, a correlation exists.
You never prove H₀. You either reject it or fail to reject it. That phrasing is deliberate. “Accepting” the null implies you have shown it is true, which you cannot do. You can only say the data are consistent with it. Examiners are strict about this.
The test statistic is a number computed from your sample. It has a known distribution under H₀. If the observed statistic is extreme enough — far into the tail of that distribution — you reject H₀. The question is how extreme is extreme enough.
2 · Type I error — the false positive
A type I error is rejecting H₀ when H₀ is actually true. You claim an effect exists when it does not.
The probability of a type I error is denoted α. It is the significance level of the test, and you set it before you look at the data. Common choices are α = 0.05, 0.01, or 0.10.
In the diagram below, α is the shaded area in the right tail of the null distribution. If the test statistic lands there, you reject H₀. But if H₀ is true, that area represents a mistake. You are willing to make that mistake α of the time.
The logic is simple. You pick α = 0.05. You run the test. If the p-value is below 0.05, you reject. If not, you do not. The p-value is the smallest α at which you would have rejected. It is the probability, under H₀, of observing a test statistic as extreme as the one you got.
3 · Type II error — the false negative
A type II error is failing to reject H₀ when H₀ is false. You miss a real effect.
The probability of a type II error is denoted β. It depends on three things: the true effect size, the sample size, and the significance level α. A small effect is harder to detect, so β is larger. A larger sample makes detection easier, so β is smaller.
In the diagram, β is the shaded area in the left tail of the alternative distribution — the part of the alternative distribution that falls below the critical value. Those are the samples where the true effect exists but the test fails to catch it.
Type II errors get less attention than type I errors, but they matter. A drug that works but fails a clinical trial is a type II error. So is a marketing campaign that lifts sales but the test says it did not.
4 · Power — the probability of detecting a real effect
Power is 1 − β. It is the probability of correctly rejecting H₀ when H₀ is false.
Power depends on the same three factors as β: effect size, sample size, and α. A test with power of 0.80 has an 80% chance of detecting a true effect of a given size. Researchers typically target 0.80 or higher.
You can increase power by raising α (accepting more type I errors), increasing the sample size, or studying a larger effect. The trade-off between α and β is the central tension in hypothesis testing. Lower α means a higher bar for rejection, which reduces type I errors but increases type II errors. The diagram makes this visible.
5 · The p-value — what it is and what it is not
The p-value is the probability, under H₀, of observing a test statistic as extreme as the one you got. It is not the probability that H₀ is true. It is not the probability that you made a mistake. It is a conditional probability: P(data | H₀), not P(H₀ | data).
This distinction is the most common misunderstanding in introductory statistics. A p-value of 0.03 does not mean there is a 3% chance the null is true. It means that if the null were true, you would see data this extreme only 3% of the time. That is a statement about the data, not about the hypothesis.
Examiners test this. They will ask: “Interpret the p-value.” The correct answer is: “If H₀ is true, the probability of observing a test statistic at least this extreme is p.” Anything about the probability of H₀ being true is wrong.
Worked example — testing a new textbook
A university adopts a new textbook for introductory statistics. Historically, the class mean on the final exam is 72 with a standard deviation of 10. The new class of 25 students scores a mean of 76. Is the new textbook effective? Test at α = 0.05.
Step 1 — State the hypotheses. H₀: μ = 72. The new textbook makes no difference. H₁: μ > 72. The new textbook raises scores. One-tailed test.
Step 2 — Choose the test statistic. You know the population standard deviation, so use a z-test. The standard error is σ/√n = 10/5 = 2. The test statistic is z = (76 − 72)/2 = 2.00.
Step 3 — Find the critical value. For α = 0.05, one-tailed, the critical z is 1.645. If z > 1.645, reject H₀.
Step 4 — Compare. z = 2.00 > 1.645. Reject H₀. The new textbook appears to raise exam scores.
Step 5 — Compute the p-value. P(Z > 2.00) = 0.0228. Interpretation: if the new textbook had no effect, you would see a sample mean of 76 or higher only 2.28% of the time.
Step 6 — Consider type II error and power. Suppose the true effect is a 4-point increase (μ = 76). What is the power of the test? The critical value in raw-score units is x_crit = 72 + 1.645 × 2 = 75.29. Under H₁ with μ = 76, the z-score of the critical value is (75.29 − 76)/2 = −0.355. P(Z > −0.355) = 0.64. Power is 0.64. The test has a 64% chance of detecting a 4-point increase.
Step 7 — Interpret the trade-off. With n = 25, α = 0.05, and a true effect of 4 points, the test misses the effect 36% of the time (β = 0.36). To increase power to 0.80, you would need a larger sample or a larger α.
Struggling to keep α, β, and power straight? That confusion is the most common exam trap in introductory statistics, and exactly what a one-on-one stats tutor London students use works through with you. Book a trial session.
Practice
Q1. A test of H₀: μ = 50 against H₁: μ > 50 uses n = 36, σ = 12, and α = 0.05.
(a) What is the critical value of the sample mean?
(b) If the true mean is 54, what is the power of the test?
(c) What is β?
Q2. A two-tailed test of H₀: μ = 100 against H₁: μ ≠ 100 uses n = 64, σ = 16, and α = 0.05.
(a) What are the critical values of the sample mean?
(b) If the true mean is 104, what is the power?
(c) What is β?
Q3. A researcher runs a test and gets a p-value of 0.04. She concludes there is a 4% chance the null hypothesis is true. Is she correct? Explain in one sentence.
Answers: Q1 (a) x̄_crit = 50 + 1.645 × (12/6) = 53.29; (b) z = (53.29 − 54)/2 = −0.355, power = P(Z > −0.355) = 0.64; (c) β = 0.36. Q2 (a) x̄_crit = 100 ± 1.96 × (16/8) = 96.08 and 103.92; (b) z = (96.08 − 104)/2 = −3.96, power = P(Z < −3.96) + P(Z > (103.92 − 104)/2 = −0.04) = 0.00004 + 0.516 = 0.516; (c) β = 0.484. Q3 No. The p-value is P(data | H₀), not P(H₀ | data).
Key takeaways
- A type I error (α) is rejecting a true null. A type II error (β) is failing to reject a false null.
- Power = 1 − β is the probability of detecting a real effect. It depends on effect size, sample size, and α.
- The p-value is the probability of the data under H₀, not the probability that H₀ is true.
- Lowering α reduces type I errors but raises β. The trade-off is unavoidable.
- You never “accept” H₀. You either reject it or fail to reject it.
Why London students choose our stats tutoring
- One-on-one format: every session is private and built around your course, from your lecture slides to your problem sets and your university’s notation for hypothesis tests.
- Intro-level specialists: our tutors teach type I/II error, power, and p-values as your department teaches them, with worked examples that match your exam style.
- Exam-first preparation: sessions work through past papers with marking schemes in view, because the “p-value is not P(H₀ | data)” distinction is where introductory marks are won and lost.
FAQ
Q: What is the difference between type I and type II error?
A: A type I error is rejecting a true null hypothesis — a false positive. A type II error is failing to reject a false null — a false negative. Type I error probability is α; type II is β.
Q: How do I calculate power?
A: Power = 1 − β. To calculate it, you need the true effect size, the sample size, the significance level α, and the standard deviation. Find the critical value under H₀, then find the probability of exceeding it under H₁.
Q: What does a p-value of 0.03 mean?
A: If the null hypothesis is true, the probability of observing a test statistic as extreme as the one you got is 0.03. It does not mean there is a 3% chance the null is true.
Q: Can I reduce both type I and type II errors at the same time?
A: Yes — by increasing the sample size. A larger sample gives a smaller standard error, which separates the null and alternative distributions more clearly. This reduces both α and β.
Q: Why do researchers use α = 0.05?
A: It is a convention from Ronald Fisher. It represents a 5% chance of a false positive, which Fisher considered an acceptable risk for most experiments. There is nothing mathematically special about 0.05.
Q: What is a one-tailed test vs a two-tailed test?
A: A one-tailed test checks for an effect in one direction (greater than or less than). A two-tailed test checks for an effect in either direction. Two-tailed tests are more conservative because the critical region is split between both tails.
Book a stats tutor in London
Hypothesis testing is a framework, not a formula. The diagram of two overlapping distributions — α in one tail, β in the other — is the single image that ties it all together. A one-on-one session turns that image into a tool you can use on any exam question. Tell us your university and module, and we will match you with the right tutor this week.
