Question

Consider a random sample \({{\rm{X}}_{\rm{1}}}{\rm{,}}{{\rm{X}}_{\rm{2}}}.....{\rm{,}}{{\rm{X}}_{\rm{n}}}\) from the shifted exponential pdf

\({\rm{f(x;\lambda ,\theta ) = }}\left\{ {\begin{array}{*{20}{c}}{{\rm{\lambda }}{{\rm{e}}^{{\rm{ - \lambda (x - \theta )}}}}}&{{\rm{x}} \ge {\rm{\theta }}}\\{\rm{0}}&{{\rm{ otherwise }}}\end{array}} \right.\). Taking \({\rm{\theta = 0}}\) gives the pdf of the exponential distribution considered previously (with positive density to the right of zero). An example of the shifted exponential distribution appeared in Example \({\rm{4}}{\rm{.5}}\), in which the variable of interest was time headway in traffic flow and \({\rm{\theta = }}{\rm{.5}}\) was the minimum possible time headway. a. Obtain the maximum likelihood estimators of \({\rm{\theta }}\) and \({\rm{\lambda }}\). b. If \({\rm{n = 10}}\) time headway observations are made, resulting in the values \({\rm{3}}{\rm{.11,}}{\rm{.64,2}}{\rm{.55,2}}{\rm{.20,5}}{\rm{.44,3}}{\rm{.42,10}}{\rm{.39,8}}{\rm{.93,17}}{\rm{.82}}\), and \({\rm{1}}{\rm{.30}}\), calculate the estimates of \({\rm{\theta }}\) and \({\rm{\lambda }}\).

Short answer

(a) Maximum likelihood estimators are \({\rm{\hat \lambda = }}\frac{{\rm{n}}}{{\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {\left( {{{\rm{X}}_{\rm{i}}}{\rm{ - \hat \theta }}} \right)} }}\) and \({\rm{\hat \theta = min(}}{{\rm{X}}_{\rm{i}}}{\rm{)}}\).

(b) The estimates are \({\rm{\hat \theta = 0}}{\rm{.64}}\) and \({\rm{\hat \lambda = 0}}{\rm{.202}}\).

Step-by-step solution

Define exponential function

A function that increases or decays at a rate proportional to its present value is called an exponential function.

Explanation

(a) Allow joint pdf or pmb for random variables \({{\rm{X}}_{\rm{1}}}{\rm{,}}{{\rm{X}}_{\rm{2}}}{\rm{, \ldots ,}}{{\rm{X}}_{\rm{n}}}\).

\({\rm{f}}\left( {{{\rm{x}}_{\rm{1}}}{\rm{,}}{{\rm{x}}_{\rm{2}}}{\rm{, \ldots ,}}{{\rm{x}}_{\rm{n}}}{\rm{;}}{{\rm{\theta }}_{\rm{1}}}{\rm{,}}{{\rm{\theta }}_{\rm{2}}}{\rm{, \ldots ,}}{{\rm{\theta }}_{\rm{m}}}} \right){\rm{, n,m}} \in {\rm{N}}\)

where \({{\rm{\theta }}_{\rm{i}}}{\rm{,i = 1,2, \ldots ,m}}\) are unknown parameters. The likelihood function is defined as a function of parameters \({{\rm{\theta }}_{\rm{i}}}{\rm{,i = 1,2, \ldots ,m}}\) where function f is a function of parameter. The maximum likelihood estimates (mle's), or values \(\widehat {{{\rm{\theta }}_{\rm{i}}}}\) for which the likelihood function is maximised, are the maximum likelihood estimates,

\({\rm{f}}\left( {{{\rm{x}}_{\rm{1}}}{\rm{,}}{{\rm{x}}_{\rm{2}}}{\rm{, \ldots ,}}{{\rm{x}}_{\rm{n}}}{\rm{;}}{{{\rm{\hat \theta }}}_{\rm{1}}}{\rm{,}}{{{\rm{\hat \theta }}}_{\rm{2}}}{\rm{, \ldots ,}}{{{\rm{\hat \theta }}}_{\rm{m}}}} \right) \ge {\rm{f}}\left( {{{\rm{x}}_{\rm{1}}}{\rm{,}}{{\rm{x}}_{\rm{2}}}{\rm{, \ldots ,}}{{\rm{x}}_{\rm{n}}}{\rm{;}}{{\rm{\theta }}_{\rm{1}}}{\rm{,}}{{\rm{\theta }}_{\rm{2}}}{\rm{, \ldots ,}}{{\rm{\theta }}_{\rm{m}}}} \right)\)

As,\({\rm{i = 1,2, \ldots ,m}}\)for every\({{\rm{\theta }}_{\rm{i}}}\). Maximum likelihood estimators are derived by replacing\({{\rm{X}}_{\rm{i}}}\)with\({{\rm{x}}_{\rm{i}}}\).

Because of the independence, the likelihood function becomes,

\(\begin{array}{c}{\rm{f}}\left( {{{\rm{x}}_{\rm{1}}}{\rm{,}}{{\rm{x}}_{\rm{x}}}{\rm{, \ldots ,}}{{\rm{x}}_{\rm{n}}}{\rm{;\lambda ,\theta }}} \right){\rm{ = \lambda }}{{\rm{e}}^{{\rm{ - \lambda }}\left( {{{\rm{x}}_{\rm{1}}}{\rm{ - \theta }}} \right)}}{\rm{ \times \lambda }}{{\rm{e}}^{{\rm{ - \lambda }}\left( {{{\rm{x}}_{\rm{2}}}{\rm{ - \theta }}} \right)}}{\rm{ \times \ldots \times \lambda }}{{\rm{e}}^{{\rm{ - \lambda }}\left( {{{\rm{x}}_{\rm{n}}}{\rm{ - \theta }}} \right)}}\\{\rm{ = }}{{\rm{\lambda }}^{\rm{n}}}{{\rm{e}}^{{\rm{ - \lambda }}\sum\limits_{{\rm{i - 1}}}^{\rm{n}} {\left( {{{\rm{x}}_{\rm{i}}}{\rm{ - \theta }}} \right)} }}\end{array}\)

Look at the log likelihood function to determine the maximum.

\(\begin{array}{c}{\rm{lnf}}\left( {{{\rm{x}}_{\rm{1}}}{\rm{,}}{{\rm{x}}_{\rm{x}}}{\rm{, \ldots ,}}{{\rm{x}}_{\rm{n}}}{\rm{;\lambda ,\theta }}} \right){\rm{ = ln}}\left( {{{\rm{\lambda }}^{\rm{n}}}{{\rm{e}}^{{\rm{ - \lambda }}\sum\limits_{{\rm{i - 1}}}^{\rm{n}} {\left( {{{\rm{x}}_{\rm{i}}}{\rm{ - \theta }}} \right)} }}} \right)\\{\rm{ = nln\lambda - \lambda }}\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {\left( {{{\rm{x}}_{\rm{i}}}{\rm{ - \theta }}} \right)} \end{array}\)

Evaluating the maximum likelihood estimators

The maximum likelihood estimator is generated by taking the derivative of the log likelihood function in regard to\({\rm{\lambda }}\)and equating it to\({\rm{0}}\).

As a result, the derivative,

\(\begin{array}{c}\frac{{\rm{d}}}{{{\rm{d\theta }}}}{\rm{f}}\left( {{{\rm{x}}_{\rm{1}}}{\rm{,}}{{\rm{x}}_{\rm{x}}}{\rm{, \ldots ,}}{{\rm{x}}_{\rm{n}}}{\rm{;\lambda ,\theta }}} \right){\rm{ = }}\frac{{\rm{d}}}{{{\rm{d\lambda }}}}\left( {{\rm{nln\lambda - \lambda }}\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {\left( {{{\rm{x}}_{\rm{i}}}{\rm{ - \theta }}} \right)} } \right)\\{\rm{ = n}}\frac{{\rm{1}}}{{\rm{\lambda }}}{\rm{ - }}\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {\left( {{{\rm{x}}_{\rm{i}}}{\rm{ - \theta }}} \right)} \end{array}\)

As a result, solving equation provides the maximum likelihood estimator \({\rm{\hat \lambda }}\).

\(\begin{array}{c}{\rm{n}}\frac{{\rm{1}}}{{{\rm{\hat \lambda }}}}{\rm{ - }}\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {\left( {{{\rm{x}}_{\rm{i}}}{\rm{ - \hat \theta }}} \right)} {\rm{ = 0}}\\\frac{{\rm{1}}}{{{\rm{\hat \lambda }}}}{\rm{ = }}\frac{{\rm{1}}}{{\rm{n}}}\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {\left( {{{\rm{x}}_{\rm{i}}}{\rm{ - \hat \theta }}} \right)} \end{array}\)

For\({\rm{\hat \lambda }}\). Hence, the maximum likelihood estimator is,

\({\rm{\hat \lambda = }}\frac{{\rm{n}}}{{\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {\left( {{{\rm{X}}_{\rm{i}}}{\rm{ - \hat \theta }}} \right)} }}\)

The maximum likelihood estimator of parameter\({\rm{\theta }}\)is shown as\({\rm{\hat \theta }}\), with the estimator calculated as follows.

To find the maximum in terms\({\rm{\theta }}\)of the likelihood function,

\({{\rm{\lambda }}^{\rm{n}}}{{\rm{e}}^{{\rm{ - \lambda }}\sum\limits_{{\rm{i - 1}}}^{\rm{n}} {\left( {{{\rm{x}}_{\rm{i}}}{\rm{ - \theta }}} \right)} }}{\rm{ = }}{{\rm{\lambda }}^{\rm{n}}}{{\rm{e}}^{{\rm{ - \lambda }}\sum\limits_{{\rm{i - 1}}}^{\rm{n}} {{{\rm{x}}_{\rm{i}}}} }}{\rm{ \times }}{{\rm{e}}^{{\rm{ - n\lambda \theta }}}}{\rm{ }}\)

it's worth noting that\({\rm{\theta }}\)only appears in the term

\({{\rm{e}}^{{\rm{ - n\lambda \theta }}}}\)

Furthermore, the likelihood function is defined only for values where all\({{\rm{x}}_{\rm{i}}}\); are greater or equal to\({\rm{\theta }}\), and where the minimum value of all\({{\rm{x}}_{\rm{i}}}\); is greater or equal to\({\rm{\theta }}\).

Because\({\rm{\theta }}\)only exists in the specified term, and the exponent\({\rm{n\lambda \theta }}\)is positive, and the likelihood function is zero for\({\rm{\theta }}\)bigger than\({\rm{min}}\left( {{{\rm{x}}_{\rm{i}}}} \right)\), the highest value is attained when

\({\rm{\hat \theta = min(}}{{\rm{X}}_{\rm{i}}}{\rm{)}}\)

Explanation

(b) The minimum values - the maximum likelihood estimate of \({\rm{\theta }}\) is calculated using available data.

\({\rm{\hat \theta = 0}}{\rm{.64}}\)

The fact that\({\rm{n = 10}}\)and\({\rm{\hat \theta = 0}}{\rm{.64}}\),

\(\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {{{\rm{x}}_{\rm{i}}}} {\rm{ = 3}}{\rm{.11 + 0}}{\rm{.64 + \ldots + 1}}{\rm{.3 = 55}}{\rm{.8}}\)

The maximum likelihood estimates of\({\rm{\lambda }}\)is,

\(\begin{array}{c}{\rm{\hat \lambda = }}\frac{{\rm{n}}}{{\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {\left( {{{\rm{x}}_{\rm{i}}}{\rm{ - \hat \theta }}} \right)} }}\\{\rm{ = }}\frac{{\rm{n}}}{{\sum\limits_{{\rm{i = 1}}}^{\rm{n}} {{{\rm{x}}_{\rm{i}}}} {\rm{ - n\hat \theta }}}}\\{\rm{ = }}\frac{{{\rm{10}}}}{{{\rm{55}}{\rm{.8 - 6}}{\rm{.4}}}}\\{\rm{ = 0}}{\rm{.202}}\end{array}\)

Therefore, \({\rm{\hat \lambda = 0}}{\rm{.202}}\) and \({\rm{\hat \theta = 0}}{\rm{.64}}\).