Week 8

This week, we will evaluate the goodness of point estimators and confidence intervals.

Published

October 13, 2024

Learning Outcomes

Monday

  • Consistency
  • Sufficiency
  • Information
  • Efficiency

Wednesday

  • Relative Efficiency
  • Consisteny

Important Concepts

Consistency

Let \(X_1,\ldots,X_n\) be a random sample from a distribution with parameter \(\theta\). The estimator \(\hat \theta\) is a consistent estimator of the \(\theta\) if

  1. \(E\{(\hat\theta-\theta)^2\}\rightarrow0\) as \(n\rightarrow \infty\)
  2. \(P(|\hat\theta-\theta|\ge \epsilon)\rightarrow0\) as \(n\rightarrow \infty\) for every \(\epsilon>0\)

Sufficiency

Sufficiency evaluates whether a statistic (or estimator) contains enough information of a parameter \(\theta\). In essence a statistic is considered sufficient to infer \(\theta\) if it provides enough information about \(\theta\).

Let \(X_1,\ldots,X_n\) be a random sample from a distribution with parameter \(\theta\). A statistic \(T=t(X_1,\ldots,X_n)\) is said to be sufficient for making inferences of a parameter \(\theta\) if condition joint distribution of \(X_1,\ldots,X_n\) given \(T=t\) does not depend on \(\theta\).

Factorization Theorem

Let \(f(x_1, \ldots, x_n;\theta)\) be the joint PMF of PDF of \(X_1, \ldots,X_n\). Then \(T=t(X_1,\ldots,X_n)\) is a sufficient statistic for \(\theta\) if and only if there exist 2 nonnegative functions, \(g\) and \(h\), such that

\[ f(x_1,\ldots,x_n) = g\{t(x_1,\ldots,x_n);\theta\}h(x_1,\ldots,x_n). \]

Minimum Sufficient Statistics

A minimum sufficient statistic is a sufficient statistic that has the smallest dimensionality, which represents the greatest possible reduction of the data without any information loss.

Information

In Statistics, information is thought of as how much does the data tell you about a parameter \(\theta\). In general, the more data is provided, the more information is provided to estimate \(\theta\).

Information can be quantified using Fisher’s Information \(I(\theta)\). For a single observation, Fisher’s Information is defined as

\[ I(\theta)=E\left[-\frac{\partial^2}{\partial\theta^2}\log\{f(X;\theta)\}\right], \]

where \(f(X;\theta)\) is either the PMF or PDF of the random variable \(X\). Furthermore, \(I(\theta)\) can be defined as

\[ I(\theta)=Var\left\{\frac{\partial}{\partial\theta}\log f(X;\theta)\right\}. \]

Additive Principle

Let \(X_1,\ldots,X_n\) be a random sample from a distribution with parameter \(\theta\), then Fisher’s Information for the random sample is defined as

\[ I_n(\theta)=nI(\theta) \]

Resources

You must log on to your CI Google account to access the video.

Press the “b” key to access hidden notes.

Lecture Slides Videos
Monday Slides Video
Wednesday Slides Video