Paper Recurrent Neural Networks for Nonlinear Time Series

This study bridges classical time-series econometrics with modern machine learning by establishing theoretical performance guarantees for recurrent neural networks (RNNs) applied to complex time-series data. Specifically, the work considers data generated by nonlinear vector autoregressive moving-average models with exogenous variables — a broad and flexible class of processes commonly encountered in economic and financial forecasting. The researchers derive upper bounds on predictive risk, decomposing total error into two interpretable components: approximation error and estimation error. Approximation error — reflecting how well the network can represent the true underlying process — depends on the smoothness of the target function and its effective dimensionality. Estimation error — reflecting uncertainty introduced by finite data — depends on the network architecture. Importantly, both sources of error diminish as model complexity is allowed to grow alongside sample size, providing a principled foundation for scaling RNNs in practice. A key theoretical contribution concerns the role of recurrence itself. Under an invertibility condition on the data-generating process, recurrent architectures can capture temporal dependence in a parsimonious way — using compact internal representations rather than requiring long explicit lags. This leads to faster statistical convergence compared to traditional nonparametric regression approaches that approximate temporal dependence through high-order autoregressive truncations, which can require impractically large lag windows as the complexity of the underlying process increases. Together, these results provide rigorous theoretical support for the use of RNNs in econometric time-series forecasting, offering reassurance that their empirical success is not purely heuristic. The findings also clarify the conditions under which recurrent models are expected to outperform classical alternatives, contributing to a growing body of work aimed at grounding modern deep learning methods in formal statistical theory.

Get the paper