Question

\({{\rm{X}}_{\rm{1}}}{\rm{,}}.....{\rm{,}}{{\rm{X}}_{\rm{n}}}\)be a random sample from a gamma distribution with parameters \({\rm{\alpha }}\) and \({\rm{\beta }}\). a. Derive the equations whose solutions yield the maximum likelihood estimators of \({\rm{\alpha }}\) and \({\rm{\beta }}\). Do you think they can be solved explicitly? b. Show that the mle of \({\rm{\mu = \alpha \beta }}\) is \(\widehat {\rm{\mu }}{\rm{ = }}\overline {\rm{X}} \).

Short answer

(a) Equations are,

\(\begin{array}{c}\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {{\rm{ln}}} {{\rm{x}}_{\rm{i}}}{\rm{ - nln\hat \beta - n}}\frac{{\rm{1}}}{{{\rm{\Gamma (\hat \alpha )}}}}{\rm{ \times }}\frac{{\rm{d}}}{{{\rm{d\hat \alpha }}}}{\rm{\Gamma (\hat \alpha ) = 0}}\\{\rm{n\hat \alpha }}\frac{{\rm{1}}}{{{\rm{\hat \beta }}}}{\rm{ + }}\frac{{\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {{{\rm{x}}_{\rm{i}}}} }}{{{{{\rm{\hat \beta }}}^{\rm{2}}}}}{\rm{ = 0}}\end{array}\)

(b) Shown, \(\overline {\rm{X}} {\rm{ = }}\widehat {\rm{\mu }}\).

Step-by-step solution

Define equations

A mathematical language that asserts that two algebraic expressions must be equal in nature is known as an equation.

Explanation

(a) The probability density function (pdf) of random variable \({\rm{X}}\) with gamma distribution is,

\({\rm{f(x;\alpha ,\beta ) = }}\frac{{\rm{1}}}{{{{\rm{\beta }}^{\rm{\alpha }}}{\rm{\Gamma (\alpha )}}}}{{\rm{x}}^{{\rm{\alpha - 1}}}}{{\rm{e}}^{{\rm{ - x/\beta }}}}{\rm{,x}} \ge {\rm{0}}\)

Where the parameters\({\rm{\alpha }}\)and\({\rm{\beta }}\)fulfil\({\rm{\alpha > 0}}\)and\({\rm{\beta > 0}}\), and is zero for\({\rm{x < 0}}\).

Allow joint pdf or pmb for random variables\({{\rm{X}}_{\rm{1}}}{\rm{,}}{{\rm{X}}_{\rm{2}}}{\rm{, \ldots ,}}{{\rm{X}}_{\rm{n}}}\).

\({\rm{f}}\left( {{{\rm{x}}_{\rm{1}}}{\rm{,}}{{\rm{x}}_{\rm{2}}}{\rm{, \ldots ,}}{{\rm{x}}_{\rm{n}}}{\rm{;}}{{\rm{\theta }}_{\rm{1}}}{\rm{,}}{{\rm{\theta }}_{\rm{2}}}{\rm{, \ldots ,}}{{\rm{\theta }}_{\rm{m}}}} \right){\rm{, n,m}} \in {\rm{N}}\)

where\({{\rm{\theta }}_{\rm{i}}}{\rm{,i = 1,2, \ldots ,m}}\)are unknown parameters. The likelihood function is defined as a function of parameters\({{\rm{\theta }}_{\rm{i}}}{\rm{,i = 1,2, \ldots ,m}}\)where function f is a function of parameter. The maximum likelihood estimates (mle's), or values\(\widehat {{{\rm{\theta }}_{\rm{i}}}}\)for which the likelihood function is maximised, are the maximum likelihood estimates,

\({\rm{f}}\left( {{{\rm{x}}_{\rm{1}}}{\rm{,}}{{\rm{x}}_{\rm{2}}}{\rm{, \ldots ,}}{{\rm{x}}_{\rm{n}}}{\rm{;}}{{{\rm{\hat \theta }}}_{\rm{1}}}{\rm{,}}{{{\rm{\hat \theta }}}_{\rm{2}}}{\rm{, \ldots ,}}{{{\rm{\hat \theta }}}_{\rm{m}}}} \right) \ge {\rm{f}}\left( {{{\rm{x}}_{\rm{1}}}{\rm{,}}{{\rm{x}}_{\rm{2}}}{\rm{, \ldots ,}}{{\rm{x}}_{\rm{n}}}{\rm{;}}{{\rm{\theta }}_{\rm{1}}}{\rm{,}}{{\rm{\theta }}_{\rm{2}}}{\rm{, \ldots ,}}{{\rm{\theta }}_{\rm{m}}}} \right)\)

As,\({\rm{i = 1,2, \ldots ,m}}\)for every\({{\rm{\theta }}_{\rm{i}}}\). Maximum likelihood estimators are derived by replacing\({{\rm{X}}_{\rm{i}}}\)with\({{\rm{x}}_{\rm{i}}}\).

Because of the independence, the likelihood function becomes,

\(\begin{array}{c}{\rm{f}}\left( {{{\rm{x}}_{\rm{1}}}{\rm{,}}{{\rm{x}}_{\rm{x}}}{\rm{, \ldots ,}}{{\rm{x}}_{\rm{n}}}{\rm{;\alpha ,\beta }}} \right){\rm{ = }}\frac{{\rm{1}}}{{{{\rm{\beta }}^{\rm{\alpha }}}{\rm{\Gamma (\alpha )}}}}{\rm{x}}_{\rm{1}}^{{\rm{\alpha - 1}}}{{\rm{e}}^{{\rm{ - }}{{\rm{x}}_{\rm{1}}}{\rm{/\beta }}}}{\rm{ \times }}\frac{{\rm{1}}}{{{{\rm{\beta }}^{\rm{\alpha }}}{\rm{\Gamma (\alpha )}}}}{\rm{x}}_{\rm{2}}^{{\rm{\alpha - 1}}}{{\rm{e}}^{{\rm{ - }}{{\rm{x}}_{\rm{2}}}{\rm{/\beta }}}}{\rm{ \times \ldots \times }}\frac{{\rm{1}}}{{{{\rm{\beta }}^{\rm{\alpha }}}{\rm{\Gamma (\alpha )}}}}{\rm{x}}_{\rm{n}}^{{\rm{\alpha - 1}}}{{\rm{e}}^{{\rm{ - }}{{\rm{x}}_{\rm{n}}}{\rm{/\beta }}}}\\{\rm{ = }}{\left( {{{\rm{x}}_{\rm{1}}}{{\rm{x}}_{\rm{2}}}{\rm{ \times \ldots \times }}{{\rm{x}}_{\rm{n}}}} \right)^{{\rm{\alpha - 1}}}}{\rm{ \times }}\frac{{\rm{1}}}{{{{\rm{\beta }}^{{\rm{n\alpha }}}}{{\rm{\Gamma }}^{\rm{n}}}{\rm{(\alpha )}}}}{\rm{ \times exp}}\left\{ {{\rm{ - }}\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {{{\rm{x}}_{\rm{i}}}} {\rm{/\beta }}} \right\}\end{array}\)

Evaluating the maximum likelihood estimators

Look at the log likelihood function to determine the maximum.

\(\begin{array}{c}{\rm{ln f}}\left( {{{\rm{x}}_{\rm{1}}}{\rm{,}}{{\rm{x}}_{\rm{x}}}{\rm{, \ldots ,}}{{\rm{x}}_{\rm{n}}}{\rm{;\alpha ,\beta }}} \right){\rm{ = ln}}\left( {{{\left( {{{\rm{x}}_{\rm{1}}}{{\rm{x}}_{\rm{2}}}{\rm{ \times \ldots \times }}{{\rm{x}}_{\rm{n}}}} \right)}^{{\rm{\alpha - 1}}}}{\rm{ \times }}\frac{{\rm{1}}}{{{{\rm{\beta }}^{{\rm{n\alpha }}}}{{\rm{\Gamma }}^{\rm{n}}}{\rm{(\alpha )}}}}{\rm{ \times exp}}\left\{ {{\rm{ - }}\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {{{\rm{x}}_{\rm{i}}}} {\rm{/\beta }}} \right\}} \right)\\{\rm{ = (\alpha - 1)}}\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {{\rm{ln}}} {{\rm{x}}_{\rm{i}}}{\rm{ - n\alpha ln\beta - nln\Gamma (\alpha ) - }}\frac{{\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {{{\rm{x}}_{\rm{i}}}} }}{{\rm{\beta }}}\end{array}\)

The maximum likelihood estimator is generated by taking the derivative of the log likelihood function in regard to\({\rm{\alpha }}\)and\({\rm{\beta }}\)equating it to\({\rm{0}}\).

As a result, the derivative,

\(\begin{array}{c}\frac{{\rm{d}}}{{{\rm{d\alpha }}}}{\rm{f}}\left( {{{\rm{x}}_{\rm{1}}}{\rm{,}}{{\rm{x}}_{\rm{x}}}{\rm{, \ldots ,}}{{\rm{x}}_{\rm{n}}}{\rm{;\alpha ,\beta }}} \right){\rm{ = }}\frac{{\rm{d}}}{{{\rm{d\alpha }}}}\left( {{\rm{(\alpha - 1)}}\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {{\rm{ln}}} {{\rm{x}}_{\rm{i}}}{\rm{ - n\alpha ln\beta - nln\Gamma (\alpha ) - }}\frac{{\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {{{\rm{x}}_{\rm{i}}}} }}{{\rm{\beta }}}} \right)\\{\rm{ = }}\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {{\rm{ln}}} {{\rm{x}}_{\rm{i}}}{\rm{ - nln\beta - n}}\frac{{\rm{1}}}{{{\rm{\Gamma (\alpha )}}}}{\rm{ \times }}\frac{{\rm{d}}}{{{\rm{d\alpha }}}}{\rm{\Gamma (\alpha )}}\end{array}\)

In the case of\({\rm{\beta }}\), the derivative is,

\(\begin{array}{c}\frac{{\rm{d}}}{{{\rm{d\beta }}}}{\rm{f}}\left( {{{\rm{x}}_{\rm{1}}}{\rm{,}}{{\rm{x}}_{\rm{x}}}{\rm{, \ldots ,}}{{\rm{x}}_{\rm{n}}}{\rm{;\alpha ,\beta }}} \right){\rm{ = }}\frac{{\rm{d}}}{{{\rm{d\beta }}}}\left( {{\rm{(\alpha - 1)}}\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {{\rm{ln}}} {{\rm{x}}_{\rm{i}}}{\rm{ - n\alpha ln\beta - nln\Gamma (\alpha ) - }}\frac{{\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {{{\rm{x}}_{\rm{i}}}} }}{{\rm{\beta }}}} \right)\\{\rm{ = - n\alpha }}\frac{{\rm{1}}}{{\rm{\beta }}}{\rm{ + }}\frac{{\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {{{\rm{x}}_{\rm{i}}}} }}{{{{\rm{\beta }}^{\rm{2}}}}}\end{array}\)

As a result, solving equation provides the maximum likelihood estimator.

\(\begin{array}{c}\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {{\rm{ln}}} {{\rm{x}}_{\rm{i}}}{\rm{ - nln\hat \beta - n}}\frac{{\rm{1}}}{{{\rm{\Gamma (\hat \alpha )}}}}{\rm{ \times }}\frac{{\rm{d}}}{{{\rm{d\hat \alpha }}}}{\rm{\Gamma (\hat \alpha ) = 0}}\\{\rm{n\hat \alpha }}\frac{{\rm{1}}}{{{\rm{\hat \beta }}}}{\rm{ + }}\frac{{\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {{{\rm{x}}_{\rm{i}}}} }}{{{{{\rm{\hat \beta }}}^{\rm{2}}}}}{\rm{ = 0}}\end{array}\)

For\({\rm{\hat \alpha }}\)and\({\rm{\hat \beta }}\).

The solution cannot be deduced explicitly since it is extremely complex and requires a great deal of background knowledge.

Explanation

(b) The following is true based on the second equation in (a).

\({\rm{ - n\hat \alpha }}\frac{{\rm{1}}}{{{\rm{\hat \beta }}}}{\rm{ + }}\frac{{\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {{{\rm{x}}_{\rm{i}}}} }}{{{{{\rm{\hat \beta }}}^{\rm{2}}}}}{\rm{ = 0}}\)

or, alternatively,

\(\frac{{\rm{1}}}{{\rm{n}}}\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {{{\rm{x}}_{\rm{i}}}} {\rm{ = \hat \alpha \times \hat \beta }}\)

The expected value of random variable X with gamma distribution, according to the proposition.

\({\rm{E(X) = \alpha \beta }}\)

As a result, the maximum likelihood estimator is,

\({\rm{\bar X = \hat mu}}\)

As, \({\rm{\mu = \alpha \beta }}\).