text: In this paper we address the problem of predicting a time series using the ARMA (autoregressive moving average) model, under minimal assumptions on the noise terms. Using regret minimization techniques, we develop effective online learning algorithms for the prediction problem, without assuming that the noise terms are Gaussian, identically distributed or even independent. Furthermore, we show that our algorithm's performances asymptotically approaches the performance of the best ARMA model in hindsight.
  @cite_4: We consider AR(q) models in time series with non‐normal innovations represented by a member of a wide family of symmetric distributions (Student's t). Since the ML (maximum likelihood) estimators are intractable, we derive the MML (modified maximum likelihood) estimators of the parameters and show that they are remarkably efficient. We use these estimators for hypothesis testing, and show that the resulting tests are robust and powerful.
  @cite_8: Traditional econometric models assume a constant one-period forecast variance. To generalize this implausible assumption, a new class of stochastic processes called autoregressive conditional heteroscedastic (ARCH) processes are introduced in this paper. These are mean zero, serially uncorrelated processes with nonconstant variances conditional on the past, but constant unconditional variances. For such processes, the recent past gives information about the one-period forecast variance. A regression model is then introduced with disturbances following an ARCH process. Maximum likelihood estimators are described and a simple scoring iteration formulated. Ordinary least squares maintains its optimality properties in this set-up, but maximum likelihood is more efficient. The relative efficiency is calculated and can be infinite. To test whether the disturbances follow an ARCH process, the Lagrange multiplier procedure is employed. The test is based simply on the autocorrelation of the squared OLS residuals. This model is used to estimate the means and variances of inflation in the U.K. The ARCH effect is found to be significant and the estimated variances increase substantially during the chaotic seventies.
  @cite_9: 
  @cite_3: From the Publisher: This is a complete revision of a classic, seminal, and authoritative book that has been the model for most books on the topic written since 1970. It focuses on practical techniques throughout, rather than a rigorous mathematical treatment of the subject. It explores the building of stochastic (statistical) models for time series and their use in important areas of application forecasting, model specification, estimation, and checking, transfer function modeling of dynamic relationships, modeling the effects of intervention events, and process control. Features sections on: recently developed methods for model specification, such as canonical correlation analysis and the use of model selection criteria; results on testing for unit root nonstationarity in ARIMA processes; the state space representation of ARMA models and its use for likelihood estimation and forecasting; score test for model checking; and deterministic components and structural components in time series models and their estimation based on regression-time series model methods.
  @cite_12: Preface1Difference Equations12Lag Operators253Stationary ARMA Processes434Forecasting725Maximum Likelihood Estimation1176Spectral Analysis1527Asymptotic Distribution Theory1808Linear Regression Models2009Linear Systems of Simultaneous Equations23310Covariance-Stationary Vector Processes25711Vector Autoregressions29112Bayesian Analysis35113The Kalman Filter37214Generalized Method of Moments40915Models of Nonstationary Time Series43516Processes with Deterministic Time Trends45417Univariate Processes with Unit Roots47518Unit Roots in Multivariate Time Series54419Cointegration57120Full-Information Maximum Likelihood Analysis of Cointegrated Systems63021Time Series Models of Heteroskedasticity65722Modeling Time Series with Changes in Regime677A Mathematical Review704B Statistical Tables751C Answers to Selected Exercises769D Greek Letters and Mathematical Symbols Used in the Text786Author Index789Subject Index792
  @cite_11: SUMMARY The marginal and bivariate distributions of the observations generated from a standard autoregressive moving average scheme are derived, assuming the noise to have a double- exponential (Laplace) distribution. The distributions may differ substantially from their Gaussian counterparts. The AR(1) model with double-exponential noise is applied to a series of weekly measurements of sulphate concentration and is shown to give a significantly better fit when compared with the Gaussian model. Most of the theory behind the analysis of time series is based on the underlying assumption that the noise series which generates the process is normally distributed. If the model for the time series is linear, as in the autoregressive moving average (ARMA) case, then the assumed normality of the noise implies that the marginal distribution of the observed values will be normal as well. All the theory behind the analysis of ARMA models, as presented by Box and Jenkins (1974), is founded on the normality assumption, although most of the techniques that they advocate will work satisfactorily provided that the normality assumption is not too seriously violated. Martin and his colleagues (Martin and Yohai, 1985) have done a large amount of work on the robustness of the standard time series analysis procedures, under various deviations from the normality assumption, using mainly various forms of contami- nated normals as noise distribution. Their work also suggests how to analyse time series in a more robust way, but their techniques have not yet come into common use by most practical time series analysts. The double-exponential (Laplace) distribution is a symmetric alternative, which is rather far from the normal distribution, and lends itself to straightforward, though sometimes cumbersome, computations. Wolf and Gastwirth (1967) consider a model with marginally double-exponentially distributed observations, while we examine the case where the noise has a double-exponential distribution. As will be shown subse- quently, these two requirements cannot be simultaneously achieved within the class of linear time series models. The outline of the paper is as follows. In Section 2 we deduce the marginal distribution of observations generated by a standard ARMA model, with the exception that the noise series has a marginal double-exponential distribution. The distribution does not lend itself to further analysis in the general case, but when specialized
summary: In standard time series analysis, the squared loss is usually considered and the noise terms are assumed to be independent with bounded variance and zero-mean. In this specific setting, one can assume without loss of generality that the noise terms have identical Gaussian distribution (see @cite_12 @cite_3 @cite_9 for more information). This allows the use of statistical methods, such as least squares and maximum likelihood based methods, for the tasks of analysis and prediction. However, when different loss functions are considered these assumptions do not hold in general, and the aforementioned methods are not applicable. We are not aware of a previous approach that tries to relax these assumptions for general convex loss functions. We note that there has been previous work which tries to relax such assumptions for the squared loss, usually under additional modelling assumptions such as -distribution of the noise (e.g., @cite_11 @cite_4 ). We emphasize that the independence assumption is rather strict and previous works that relax this assumption usually offer specific dependency model, e.g., as proposed by @cite_8 for the ARCH model.