Question

Let\({\rm{X}}\)represent the error in making a measurement of a physical characteristic or property (e.g., the boiling point of a particular liquid). It is often reasonable to assume that\({\rm{E(X) = 0}}\)and that\({\rm{X}}\)has a normal distribution. Thus, the pdf of any particular measurement error is

\({\rm{f(x;\theta ) = }}\frac{{\rm{1}}}{{\sqrt {{\rm{2\pi \theta }}} }}{{\rm{e}}^{{\rm{ - }}{{\rm{x}}^{\rm{2}}}{\rm{/2\theta }}}}\quad {\rm{ - \yen < x < \yen}}\)

(Where we have used\({\rm{\theta }}\)in place of\({{\rm{\sigma }}^{\rm{2}}}\)). Now suppose that\({\rm{n}}\)independent measurements are made, resulting in measurement errors\({{\rm{X}}_{\rm{1}}}{\rm{ = }}{{\rm{x}}_{\rm{1}}}{\rm{,}}{{\rm{X}}_{\rm{2}}}{\rm{ = }}{{\rm{x}}_{\rm{2}}}{\rm{, \ldots ,}}{{\rm{X}}_{\rm{n}}}{\rm{ = }}{{\rm{x}}_{\rm{n}}}{\rm{.}}\)Obtain the mle of\({\rm{\theta }}\).

Short answer

The solution is \({\rm{\hat \theta = }}\frac{{\rm{1}}}{{\rm{n}}}\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {{\rm{X}}_{\rm{i}}^{\rm{2}}} \)which the maximum likelihood estimator.

Step-by-step solution

Introduction

An estimator is a rule for computing an estimate of a given quantity based on observable data: the rule (estimator), the quantity of interest (estimate), and the output (estimate) are all distinct.

Explanation

Let random variables \({{\rm{X}}_{\rm{1}}}{\rm{,}}{{\rm{X}}_{\rm{2}}}{\rm{, \ldots ,}}{{\rm{X}}_{\rm{n}}}\)have joint pdf or pmb

\({\rm{f}}\left( {{{\rm{x}}_{\rm{1}}}{\rm{,}}{{\rm{x}}_{\rm{2}}}{\rm{, \ldots ,}}{{\rm{x}}_{\rm{n}}}{\rm{;}}{{\rm{\theta }}_{\rm{1}}}{\rm{,}}{{\rm{\theta }}_{\rm{2}}}{\rm{, \ldots ,}}{{\rm{\theta }}_{\rm{m}}}} \right){\rm{,}}\quad {\rm{n,m\hat I N}}\)

Where, the parameters \({{\rm{\theta }}_{\rm{i}}}{\rm{,i = 1,2, \ldots ,m}}\) are unknown. When function \({\rm{f}}\)is a function of parameters\({{\rm{\theta }}_{\rm{i}}}{\rm{,i = 1,2, \ldots ,m}}\), it is called the

likelihood function

Values \({{\rm{\hat \theta }}_{\rm{i}}}\)that maximize the likelihood function are the maximum likelihood estimates (mle's), or equally values \({{\rm{\hat \theta }}_{\rm{i}}}\)for which

for every\({{\rm{\theta }}_{\rm{i}}}{\rm{,i = 1,2, \ldots ,m}}\). By substituting \({{\rm{x}}_{\rm{i}}}\)with\({{\rm{x}}_{\rm{i}}}\), the

maximum likelihood estimators

are obtained.

The likelihood function, because of the independence, becomes

\(\begin{aligned}{\rm{f}}\left( {{{\rm{x}}_{\rm{1}}}{\rm{,}}{{\rm{x}}_{\rm{2}}}{\rm{, \ldots ,}}{{\rm{x}}_{\rm{n}}}{\rm{;\theta }}} \right)\\& = \frac{{\rm{1}}}{{\sqrt {{\rm{2\pi \times \theta }}} }}{\rm{ \times exp}}\left\{ {{\rm{ - }}\frac{{{\rm{x}}_{\rm{1}}^{\rm{2}}}}{{{\rm{2\theta }}}}} \right\}{\rm{ \times }}\frac{{\rm{1}}}{{\sqrt {{\rm{2\pi \times \theta }}} }}{\rm{ \times exp}}\left\{ {{\rm{ - }}\frac{{{\rm{x}}_{\rm{2}}^{\rm{2}}}}{{{\rm{2\theta }}}}} \right\}{\rm{ \times \ldots \times }}\frac{{\rm{1}}}{{\sqrt {{\rm{2\pi \times \theta }}} }}{\rm{ \times exp}}\left\{ {{\rm{ - }}\frac{{{\rm{x}}_{\rm{n}}^{\rm{2}}}}{{{\rm{2\theta }}}}} \right\}\\& = \prod\limits_{{\rm{i = 1}}}^{\rm{n}} {\frac{{\rm{1}}}{{\sqrt {{\rm{2\pi \times \theta }}} }}} {\rm{ \times exp}}\left\{ {{\rm{ - }}\frac{{{\rm{x}}_{\rm{i}}^{\rm{2}}}}{{{\rm{2\theta }}}}} \right\}\\&= (2\pi \sigma {{\rm{)}}^{{\rm{ - n/2}}}}{\rm{ \times exp}}\left\{ {{\rm{ - }}\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {\frac{{{\rm{x}}_{\rm{i}}^{\rm{2}}}}{{{\rm{2\sigma }}}}} } \right\}\end{aligned}\)

In order to find maximum, look at the log likelihood function

\(\begin{array}{l}{\rm{lnf}}\left( {{{\rm{x}}_{\rm{1}}}{\rm{,}}{{\rm{x}}_{\rm{2}}}{\rm{, \ldots ,}}{{\rm{x}}_{\rm{n}}}{\rm{;\theta }}} \right)\\{\rm{ = ln}}\left( {{{{\rm{(2\pi \sigma )}}}^{{\rm{ - n/2}}}}{\rm{ \times exp}}\left\{ {{\rm{ - }}\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {\frac{{{\rm{x}}_{\rm{i}}^{\rm{2}}}}{{{\rm{2\sigma }}}}} } \right\}} \right)\\{\rm{ = - }}\frac{{\rm{n}}}{{\rm{2}}}{\rm{ln(2\pi ) - }}\frac{{\rm{n}}}{{\rm{2}}}{\rm{ln\theta - }}\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {\frac{{{\rm{x}}_{\rm{i}}^{\rm{2}}}}{{{\rm{2\theta }}}}} \end{array}\)

By taking derivative of log likelihood function in respect to \({\rm{\theta }}\)and equating it to \({\rm{0}}\) the maximum likelihood estimator is obtained. Therefore, the derivative is

\(\begin{array}{l}\frac{{\rm{d}}}{{{\rm{d\theta }}}}{\rm{f}}\left( {{{\rm{x}}_{\rm{1}}}{\rm{,}}{{\rm{x}}_{\rm{2}}}{\rm{, \ldots ,}}{{\rm{x}}_{\rm{n}}}{\rm{;\theta }}} \right)\\ &= \frac{{\rm{d}}}{{{\rm{d\theta }}}}\left( {{\rm{ - }}\frac{{\rm{n}}}{{\rm{2}}}{\rm{ln(2\pi ) - }}\frac{{\rm{n}}}{{\rm{2}}}{\rm{ln\theta - }}\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {\frac{{{\rm{x}}_{\rm{i}}^{\rm{2}}}}{{{\rm{2\theta }}}}} } \right)\\&= 0 - \frac{{\rm{n}}}{{{\rm{2\theta }}}}{\rm{ + }}\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {\frac{{{\rm{x}}_{\rm{i}}^{\rm{2}}}}{{{\rm{2}}{{\rm{\theta }}^{\rm{2}}}}}} \\&= - \frac{{\rm{n}}}{{{\rm{2\theta }}}}{\rm{ + }}\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {\frac{{{\rm{x}}_{\rm{i}}^{\rm{2}}}}{{{\rm{2}}{{\rm{\theta }}^{\rm{2}}}}}} \end{array}\)

Therefore, the maximum likelihood estimator is obtained by solving equation

\(\begin{aligned}{\rm{ - }}\frac{{\rm{n}}}{{{\rm{2\hat \theta }}}}{\rm{ + }}\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {\frac{{{\rm{x}}_{\rm{i}}^{\rm{2}}}}{{{\rm{2}}\widehat {{{\rm{\theta }}^{\rm{2}}}}}}} \\&= 0 - n\theta + \sum\limits_{{\rm{i = 1}}}^{\rm{n}} {{\rm{x}}_{\rm{i}}^{\rm{2}}} \\ &= 0\end{aligned}\)
For\({\rm{\hat \theta }}\).

Therefore, the solution is\({\rm{\hat \theta = }}\frac{{\rm{1}}}{{\rm{n}}}\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {{\rm{X}}_{\rm{i}}^{\rm{2}}} \)which is the maximum likelihood estimator