Maximum Likelihood Estimators

Maximum Likelihood Estimators

• Maximum Likelihood Estimators

• MLE

• Example

Likelihood Function

Using the joint pdf or pmf of the sample \(\boldsymbol X\), the likelihood function is a function of \(\boldsymbol \theta\), given the observed data \(\boldsymbol X =\boldsymbol x\), defined as

\[ L(\boldsymbol \theta|\boldsymbol x)=f(\boldsymbol x|\boldsymbol \theta) \]

If the data is iid, then

\[ f(\boldsymbol x|\boldsymbol \theta) = \prod^n_{i=1}f(x_i|\boldsymbol\theta) \]

MLE

• Maximum Likelihood Estimators

• MLE

• Example

Likelihood Function

Using the joint pdf or pmf of the sample \(\boldsymbol X\), the likelihood function is a function of \(\boldsymbol \theta\), given the observed data \(\boldsymbol X =\boldsymbol x\), defined as

\[ L(\boldsymbol \theta|\boldsymbol x)=f(\boldsymbol x|\boldsymbol \theta) \]

If the data is iid, then

\[ f(\boldsymbol x|\boldsymbol \theta) = \prod^n_{i=1}f(x_i|\boldsymbol\theta) \]

Log-Likelihood Function

If \(\ln\{L(\boldsymbol \theta)\}\) is monotone of \(\boldsymbol \theta\), then maximizing \(\ell(\boldsymbol\theta) = \ln\{L(\boldsymbol \theta)\}\) will yield the maximum likelihood estimators.

Maximum log-Likelihood Estimator

The maximum likelihood estimator are the estimates of \(\boldsymbol \theta\) that maximize \(\ell(\boldsymbol\theta)\).

Example

• Maximum Likelihood Estimators

• MLE

• Example

Poisson Distribution

Let \(X_1,\ldots,X_n\overset{iid}{\sim}\mathrm{Pois}(\lambda)\), show that the MLE of \(\lambda\) is \(\bar x\).

Normal Distribution

Let \(X_1,\ldots,X_n\overset{iid}{\sim}N(\mu,\sigma^2)\). Show that the MLE’s of \(\mu\) and \(\sigma^2\) are \(\bar x\) and \(\frac{n-1}{n}s^2\), respectively.

Exponential Distribution

Let \(X_1,\ldots,X_n\overset{iid}{\sim}Exp(\lambda)\). Find the MLE of \(\lambda\)