Question

Suppose that \(Y_{1}, \ldots, Y_{n}\) are taken from an AR(1) process with innovation variance \(\sigma^{2}\) and correlation parameter \(\rho\) such that \(|\rho|<1\). Show that $$ \operatorname{var}(\bar{Y})=\frac{\sigma^{2}}{n^{2}\left(1-\rho^{2}\right)}\left\\{n+2 \sum_{j=1}^{n-1}(n-j) \rho^{j}\right\\} $$ and deduce that as \(n \rightarrow \infty\) for any fixed \(\rho, n \operatorname{var}(\bar{Y}) \rightarrow \sigma^{2} /(1-\rho)^{2}\). What happens when \(|\rho|=1 ?\) Discuss estimation of \(\operatorname{var}(\bar{Y})\) based on \((n-1)^{-1} \sum\left(Y_{j}-\bar{Y}\right)^{2}\) and an estimate \(\widehat{\rho}\).

Short answer

As \(n \to \infty\), \(n \operatorname{var}(\bar{Y}) \to \sigma^2/(1-\rho)^2\). If \(|\rho|=1\), variance is infinite due to non-stationarity.

Step-by-step solution

Understanding the AR(1) process

An AR(1) process is defined as \(Y_t = \rho Y_{t-1} + \epsilon_t\), where \(\epsilon_t\) are i.i.d. errors with variance \(\sigma^2\) and \(|\rho|<1\). Each \(Y_t\) is correlated with \(Y_{t-1}\) and has a variance that needs to be calculated over the mean of the process.

Calculating variance of the mean

The variance of the mean is given by \(\operatorname{var}(\bar{Y}) = \operatorname{var}\left(\frac{1}{n}\sum_{i=1}^{n} Y_i\right)\). Since the \(Y_i\) form an AR(1) process, we need to consider autocovariances: \(\gamma_j = \operatorname{cov}(Y_t, Y_{t-j}) = \sigma^2 \rho^j / (1-\rho^2)\). Using these, \(\operatorname{var}(\bar{Y})\) is derived by considering all pairs \((i,j)\).

Deriving the full expression

Substitute the autocovariances into the formula for variance of the sample mean: \[\operatorname{var}(\bar{Y}) = \frac{1}{n^2}\left(\sum_{i=1}^{n}\sum_{j=1}^{n} \operatorname{cov}(Y_i, Y_j)\right) = \frac{1}{n^2(1-\rho^2)}\left(n\sigma^2 + 2\sum_{j=1}^{n-1}(n-j)\sigma^2 \rho^j\right)\]. Simplifying this expression gives the required result.

Simplification for large n

As \(n \rightarrow \infty\), the term involving the summation of \(\rho^j\) behaves like a geometric series, which converges to \(\sigma^2 /(1-\rho)\) when \(n\) is large enough. Therefore, the variance of the mean tends to \(\sigma^2 /(1-\rho)^2\).

Case of \( |\rho| = 1 \)

When \(|\rho| = 1\), the AR(1) process becomes non-stationary, and the variance of the process does not exist in the usual way. The variance of the sample mean becomes infinite since the autocovariance does not decay, indicating instability.

Estimation of \(\operatorname{var}(\bar{Y})\)

The empirical variance is calculated as \((n-1)^{-1}\sum (Y_j - \bar{Y})^2\). This is adjusted for autocorrelation by incorporating an estimate of \(\widehat{\rho}\) into variance calculations. This helps to provide a more accurate measure of variability by acknowledging the correlated nature of the data.

Key concepts

AR(1) Process

The AR(1) process, short for the "Autoregressive process of order 1", is a foundational concept in time series analysis. It describes a series where each term is influenced by its immediate predecessor, along with a random error term. Formally, the process can be represented as: \[ Y_t = \rho Y_{t-1} + \epsilon_t \] where \(\rho\) is a parameter indicating the degree of influence from the previous term, \(|\rho| < 1\) ensures stability, and \(\epsilon_t\) represents the innovation or error term. This error is often assumed to be normally distributed with mean zero and a constant variance \(\sigma^2\).

• \(\rho > 0\) suggests a positive correlation between terms.
• \(\rho < 0\) suggests a negative correlation.
• \(|\rho| < 1\) ensures that the influence from the initial terms diminishes over time, preventing the accumulation of errors.

Understanding how \(\rho\) impacts the series is key to analyzing and forecasting time series data accurately. When studying AR(1) processes, we primarily focus on how the correlations between consecutive terms affect the overall behavior of the series.

Autocovariance

Autocovariance is crucial in evaluating how elements of a time series relate to each other over different time lags. For an AR(1) process, the autocovariance function helps understand the degree of linear dependence between the terms. Mathematically, the autocovariance \(\gamma_j\) for lag \(j\) can be expressed as:\[ \gamma_j = \frac{\sigma^2 \rho^j}{1-\rho^2} \] This equation tells us several important things:

• The term \(\rho^j\) reveals how the influence fades with increased lag (\(j\)).
• The presence of \(\sigma^2\) means that higher innovation variance leads to greater overall variability in the series.
• The denominator \((1-\rho^2)\) stabilizes the autocovariance, ensuring stationarity when \(|\rho| < 1\).

Autocovariance is not only fundamental in calculating the variance of the sample mean but also in assessing the stationarity of the series. When autocovariances decay to zero, the series is often stationary, ensuring consistent statistical properties over time.

Sample Mean Variance

In a time series, assessing the sample mean's variance can provide insights into the stability and predictability of the process. For AR(1) processes, this is slightly complicated by the autocorrelations between terms.The variance of the sample mean, \(\operatorname{var}(\bar{Y})\), reflects this interconnectedness as:\[ \operatorname{var}(\bar{Y}) = \frac{\sigma^2}{n^2(1-\rho^2)} \left\{ n + 2 \sum_{j=1}^{n-1}(n-j) \rho^j \right\} \] This expression illustrates how individual variances (\(\sigma^2\)) and their correlations (\(\rho^j\)) affect the overall variance of \(\bar{Y}\).

• As \(n\) becomes large, the variance term simplifies, indicating that with a sufficiently large sample size, \(\operatorname{var}(\bar{Y})\) approximates \(\sigma^2/(1-\rho)^2\).
• It underscores the diminishing impact of each additional term on the mean as \(n\) grows.

This relationship highlights how AR(1) processes can provide both stable and unstable forecasts, depending heavily on the estimated \(\rho\).

Stationarity

For any time series model, particularly AR(1) processes, stationarity is a critical characteristic that determines its reliability for forecasting and analysis. A time series is stationary if its statistical properties such as mean, variance, and autocovariance are constant over time.In the context of an AR(1) process, stationarity is assured when the absolute value of the correlation parameter alone, \(|\rho| < 1\). When this condition is satisfied:

• The series has consistent mean and variance over time, making it predictable.
• Autocovariances decrease to zero as the lag increases, implying a bounded relationship between past and future values.

If \(|\rho| = 1\), the process becomes non-stationary, leading to unbounded variance and making the series unpredictable at any future point. Thus, ensuring stationarity is integral to effective time series modeling, enabling the inference of meaningful insights and accurate forecasting from the model.

Parameter Estimation

Accurate parameter estimation in AR(1) processes is vital for reliable modeling. It involves estimating the correlation parameter \(\rho\) and the innovation variance \(\sigma^2\) using the available data.Techniques commonly employed include:

• Method of Moments: Utilizing empirical moments to infer parameter values, which can be straightforward but sometimes lacks precision.
• Maximum Likelihood Estimation (MLE): A more robust method assuming the data follows a specific distribution, typically normal, giving consistent estimates at the expense of computational complexity.

For practical estimation of \(\operatorname{var}(\bar{Y})\), the empirical variance \((n-1)^{-1}\sum (Y_j - \bar{Y})^2\) needs to be adjusted for autocorrelation by incorporating \(\widehat{\rho}\):\[ \operatorname{adj\_var}(\bar{Y}) = (n-1)^{-1}\sum (Y_j - \bar{Y})^2 + \frac{2\sigma^2}{1-\rho}\]Estimates in AR(1) modeling are crucial for accurate state predictions, signifying how past values influence the current state and providing an empirical basis for future decision-making.