Type I and II Error, Power, and Neyman-Pearson Lemma

Learning Outcomes

  • Type I Error
  • Type II Error
  • Power
  • Types of Hypothesis
  • Neyman-Pearson Lemma

Types of Error

Types of Error

Type I Error

A type I error occurs when \(H_0\) is rejected when \(H_0\) is true. The probability of a type I error is denoted as \(\alpha\).

Type II Error

A type I error occurs when \(H_0\) fails to be rejected when \(H_0\) is false. The probability of a type II error is denoted as \(\beta\).

Relationship Between \(\alpha\) and \(\beta\)

Type II Error for Z Tests

Type II Error for 2-sample Z Tests

Power

Power

The power of a test is the probability of rejecting \(H_0\) when the true parameter is \(\theta\).

\[ \mathrm{power}(\theta) = \mathrm P(\mathrm{Reject}\ H_0\ \mathrm{when}\ \theta\ \mathrm{is true}) \]

Power and Type II Error

\[ \beta = 1 - \mathrm{power}(\theta_a) \]

Types of Hypothesis

Simple Hypothesis

Composite Hypothesis

Neyman-Pearson Lemma

Neyman-Pearson Lemma

Suppose you test the simple null hypothesis (\(H_0: \theta=\theta_0\)) vs a simple alternative hypothesis (\(H_a: \theta=\theta_1\)), based on a random sample with parameter \(\theta\). Let \(L(\theta)\) denote the likelihood function of the sample with parameter \(\theta\). Then for a given \(\alpha\), the test that maximizes the power at \(\theta_1\) has a rejection region determined by \[ \frac{L(\theta_0)}{L(\theta_1)}<k \]The value of \(k\) is chosen so that the test has the desired value for \(\alpha\). such a test is most powerful \(\alpha\)-level test for \(H_0\) vs \(H_a\)