text: In this paper we address the problem of predicting a time series using the ARMA (autoregressive moving average) model, under minimal assumptions on the noise terms. Using regret minimization techniques, we develop effective online learning algorithms for the prediction problem, without assuming that the noise terms are Gaussian, identically distributed or even independent. Furthermore, we show that our algorithm's performances asymptotically approaches the performance of the best ARMA model in hindsight.
  @cite_2: The correlation analysis based methods are not suitable for identifying parameters of nonstationary autoregressive (AR), moving average (MA), and ARMA systems. By using estimation residuals in place of unmeasurable noise terms in information vector or matrix, we develop a least squares based and gradient based algorithms and establish the consistency of the proposed algorithms without assuming noise stationarity, ergodicity, or existence of higher order moments. Furthermore, we derive the conditions for convergence of the parameter estimation. The simulation results validate the convergence theorems proposed.
summary: Furthermore, an online approach that relies on regret minimization techniques was never considered for ARMA prediction, and hence regret bounds of the type we are interested simply do not exist. Yet, results on the convergence rate of the coefficient vectors do exist, and regret bounds can be derived from these results. E.g., in @cite_2 such results are presented, and a regret bound of @math can be derived for the squared loss. We are not familiar of these kind of results for general convex loss functions.