View source for Nonlinear time series analysis


{| class="wikitable"
!Copyright notice <!-- don't remove! --> 
|-
| This article ''Nonlinear time series analysis'' was adapted from an original article by Howell Tong, which appeared in ''StatProb: The Encyclopedia Sponsored by Statistics and Probability Societies''. The original article ([<nowiki>http://statprob.com/encyclopedia/NonlinearTimeSeriesAnalysis.html</nowiki> StatProb Source], Local Files: [[Media:nonlinear_time_series_analysis.pdf|pdf]] | [[Media:nonlinear_time_series_analysis.tex|tex]]) is copyrighted by the author(s), the article has been donated to ''Encyclopedia of Mathematics'', and its further issues are under ''Creative Commons Attribution Share-Alike License'. All pages from StatProb are contained in the [[:Category:Statprob|Category StatProb]].
|-
|}

{{MSC|60G10}}


<!-- \documentclass[10pt]{article} 
     \usepackage{amssymb} 
     \usepackage{amsmath} 
     \usepackage{amsfonts} 
     \begin{document} 
     \setlength{\begin{description}}{\end{description} -->}
$ \renewcommand{\endthebibliography}{\end{description}} $
$ \newcommand{\ds}{\displaystyle} $
$ \renewcommand{\endthebibliography}{\end{description}} $

<center>'''Nonlinear Time Series Analysis'''</center>

<center>  {Howell Tong} 

''London School of Economics and Political Science'' </center>

<!-- \maketitle 
     \noindent -->
'''1. Introduction.'''

A function $f$ from $R^p$ to $R$ is said to be ''linear'' if for
vectors $x, y\in R^p$ and any real scalar $\alpha$, $f(\alpha x +y)
= \alpha f(x) + f(y).$ Any function $f$ that is not linear is said
to be ''nonlinear.''

In the analysis of stationary time series, the spectral density
function, if it exists, is nonlinear under the above definition.
However, for reasons to be made clear later, a statistical analysis
that is based on it or its equivalents is ordinarily considered a
linear analysis. Often, a time series is observed at discrete time
intervals. For a discrete-time stationary time series $\{X_t: t=
\ldots, -1, 0, 1, \ldots\}$ with finite variance, $corr(X_t,
X_{t+s})$ is a function of $s$ only, say $\rho(s)$, and is called
the auto-correlation function. The spectral density function is the
Fourier transform of $\rho(s)$ if $\sum_{s=-\infty}^\infty |\rho(s)|
< \infty.$ Now, Yule (1927)
introduced the celebrated autoregressive model in time series.
Typically the model takes the form
\begin{eqnarray}
X_t = \alpha_0 + \alpha_1 X_{t-1} + \cdots + \alpha_p X_{t-p} +
\varepsilon_t,
\end{eqnarray} where the $\alpha$'s are parameters and $\{\varepsilon_t\}$
is a sequence of independent and identically distributed random
variables with zero mean and finite variance, or a white noise for
short. It is commonly denoted as an $AR(p)$ model. Clearly $X_t$ is
a linear function of $X_{t-1}, \ldots, X_{t-p}, \varepsilon_t$.
Under the assumption of normality, the distribution of the time
series is completely specified by its constant mean, constant
variance and $\rho(s)$'s. Perhaps for the close connection with the
analysis of linear models (of which the autoregressive model is
one), an analysis based on the autocorrelation function or
equivalently the spectral density function is loosely referred to as
a linear analysis of the time series. By the same token, an analysis
based on higher order moments or their Fourier transforms is
loosely called a nonlinear analysis. Broadly speaking, tools based
on the Fourier transforms of moments constitute what is called the
frequency-domain approach, while those based on the moments
constitute the time-domain approach, which often includes building
a time series model of the form (1) or its generalizations.

Similar discussion as the above can be extended to cover $\{X(t): t \in R \}$


<!-- \noindent -->
'''2. Can we do without nonlinearity?'''

A general answer is in the negative simply because the dynamical
laws governing Nature or human activities are seldom linear. In the
real world, we can see the footprints of nonlinearity everywhere we
look. Below are a few examples.

<!-- \noindent -->
(a) ''Phase Transition''

The melting of ice of a glacier will alter fundamentally the amount
of water flowed in a river near the glacier. Phase transition (from
solid to liquid in the above example) is an importance signature of
nonlinearity. Animals behave differently (e.g. hunting effort)
during time of short food supply versus time of abundant food
supply.

<!-- \noindent -->
(b) ''Saturation''

In economics, diminishing return is a well-known phenomenon:
doubling your effort does not necessarily double your reward.

<!-- \noindent -->
(c) ''Synchronization''

The celebrated Dutch scientist, Christiaan Huygens, observed that
clocks placed on the same piece of soft timber were synchronized!
Biological systems can also exhibit synchronization. It has been
noted that girls sharing the same dormitory have higher chance of
synchronizing their menstruation. Even female keepers of baboons
have been known to have similar experience.

<!-- \noindent -->
(d) ''Chaos''

When we toss a coin to randomize our choice, we are exploiting
nonlinearity, for the dynamical system underlying the tossing is a
set of (typically three) nonlinear ordinary differential equations,
the solution of which is generally very sensitive to the initial
spinning unless we `cheat'. The system is said to generate chaos in
a technical sense. When statisticians generate pseudo-random
numbers, they are also generating chaos. One of the most commonly
used pseudo-random generator is the linear congruential generator,
which is a piecewise linear (i.e. nonlinear) function that does
precisely this. It might surprise you that you are actually using
nonlinear devices almost daily because encrypting passwords is
closely related to pseudo-random number generation.

In the following sections, we focus on the time-domain approach
because at the current state of development, this approach tends to
admit simpler interpretations in practical applications.


<!-- \noindent -->
'''3. What is a nonlinear time series model?'''

A short answer is that it is not a linear time series model. This
raises the need to define a linear model. A fairly commonly adopted
definition is as follows. A stationary time series model is called
a linear time series model if it is equivalent (for example in the
mean-square sense) to
\begin{eqnarray}
X_t = \sum_{s=-\infty}^\infty \beta_s \varepsilon_{t-s},
\end{eqnarray}
where $\{\varepsilon_t\}$ is a white noise and the summation is
assumed to exist in some sense. An alternative definition due to
Hannan (1973) is one that requires that the minimizer of $E|X_{t}- h
(X_{t-1}, X_{t-2}, \ldots)|^2$ with respect to $h$ over the space of
all measurable functions is the linear function. Here the mean
square is assumed to exist.

<!-- \noindent -->
'''4. Are linear time series models fit for purpose?'''

Examples abound of the inability of linear time series models to
capture essential features of the underlying dynamics.

Yule (1927) introduced the autoregressive model to model the annual
sunspot numbers with a view to capturing the observed 11-year
sunspot cycle but noted the inadequacy of his model. He noted the
asymmetry of the cycle and attempted to model it with an $AR(4)$
model only to discover that it gave statistically a worse fit than a
simpler AR(2) model.

Moran (1953) fitted an $AR(2)$ model to the annual lynx data
corresponding to the MacKenzie River region in Canada, with a view
to capturing the observed 10-year cycle. He was quick to point out
that the fitted residuals were heteroscedastic.

Whittle (1954) analyzed a seiche record from Wellington Bay in New
Zealand. He noted that, besides the fundamental frequency of
oscillations and a frequency due to the reflection of an island at
the bay, there were sub-harmonics bearing an interesting arithmetic
relation with the above frequencies. Now, sub-harmonics are one of
the signatures of nonlinear oscillations, long known to the
physicists and
engineers. 


<!-- \noindent -->
'''5 Examples of nonlinear time series models.'''

First, we describe parametric models. Due to space limitation, we
describe the two most commonly used models. For other models, we
refer to Tong (1990). We shall describe (i) the threshold model and
(ii) the (generalized) autoregressive conditional heteroscedasticity
model, or in short the TAR model and the (G)ARCH model respectively.
The former was introduced by Tong in 1977 and developed
systematically in Tong and Lim (1980) and Tong (1983, 1990), and the
latter by Engle (1982), later generalized by Bollerslev (1986).

There are several different but equivalent ways to express a TAR
model. Here is a simple form. Let $\{Z_t\}$ denote an indicator time
series that takes positive integer values, say $\{1, 2, \ldots,
K\}$. Let $\{\eta_t\}$ denote a white noise with zero mean and unit
variance, $\alpha_0^{(j)}, \alpha_i^{(j)}, \beta^{(j)}$ be real
constants for $j =1,2, \ldots, K.$ Then a time series $\{X_t: t = 0,
\pm 1, \pm2, \ldots \}$ is said to follow a ''threshold
autoregressive model'' if it satisfies, when $Z_t = j, \;\; j=1,
\ldots, K,$
\begin{eqnarray}
X_t = \alpha_0^{(j)} + \sum_{i=1}^p \alpha_i^{(j)} X_{t-i} + \beta^{(j)}\eta_t.
\end{eqnarray}

For the case in which $Z_t = j$ if and only if $X_{t-d} \in R_j$ for
some positive integer $d$ and for some partition of $R$, i.e. $R =
\bigcup_{i=1}^K R_i$ say, the TAR model is called a ''self-exciting threshold autoregressive model'', or SETAR model for
short. In this case, given $X_{t-s}, s>0,$ the conditional mean of
$X_t$ is piecewise linear, and the conditional variance of $X_t$
piecewise constant.

For the case in which $Z_t = j$ if and only if $Y_{t-d} \in R_j$ for
some covariate time series $\{Y_t\}$, some positive integer $d$ and
some partition of $R$, i.e. $ R = \cup_{i=1}^K R_i $ say, then we
have a TAR model driven by (or excited by) $\{Y_t\}$. Note that the
covariate time series $\{Y_t\}$, and thus the indicator time series
$\{Z_t\}$, can be observable or hidden. If the indicator time
series, whether observable or hidden, forms a Markov chain, then we
call $\{X_t\}$ a Markov-chain driven TAR; this model was first
introduced by Tong (Tong and Lim, 1980, p.285 and Tong 1982, p.62).
In the econometric literature, the sub-class with a hidden Markov
chain is commonly called a ''Markov switching model''.

The TAR model, especially the SETAR model, has many practical
applications in diverse areas/disciplines, including earth sciences,
ecology, economics, engineering, environmental science, finance,
hydraulics, medical science, water resources and many others.

The nonlinear parametric model that is mostly and widely used in
econometrics and finance is the (G)ARCH model. The ARCH model is
given by
\begin{eqnarray}
X_t = \eta_t \sigma_t,
\end{eqnarray}
where $\{\eta_t\}$ is as defined previously but sometimes assumed to
be Gaussian, and $\sigma_t^2 = \alpha_0 + \sum_{i=1}^p \alpha_i
X_{t-i}^2, \;\; \alpha_0 >0, \alpha_i \ge 0, i = 1, \ldots, p.$ Note
that the ARCH model differs from the SETAR model in its $\sigma_t$
being a continuous function instead of a piecewise constant function
as in the latter. The GARCH model generalizes $\sigma_t^2$ to
$\sigma_t^2 = \alpha_0 + \sum_{i=1}^p \alpha_i X_{t-i}^2 +
\sum_{i=1}^q \beta_i \sigma_{t-i}^2,$ where the $\beta_i$s are
usually also assumed to be non-negative, although the non-negativity
assumption may be relaxed; see Cryer and Chan (2008, Chapter 12).

One of the limitations of any parametric modelling approach is the
subjectivity of selecting a family of possible parametric models. We
can sometimes mitigate the situation if a certain parametric family
is suggested by subject matter considerations. In the absence of the
above, mitigation is weaker even if we are assured that the family
is dense in some sufficiently large space of models. It is then
tempting to allow the data to suggest the form of $F$ where we are
contemplating a model of say
\begin{eqnarray}
X_t = F(X_{t-1}, \ldots, X_{t-p}, \varepsilon_t),
\end{eqnarray}
$F$ being unknown. This is one of the strengths of the nonparametric
modelling approach, which is a vast and rapidly expanding area. A
word of caution is the so-called curse of dimensionality, meaning
that when $p > 3$ the estimated $F$ is unlikely to be reliable
unless we have a huge sample size. One way to ameliorate the
situation is to replace $X_{t-1}, \ldots, X_{t-p}$ by $\xi_{t-1},
\ldots, \xi_{t-q}$ with $q$ much smaller than $p$, e.g. $q = 1$ or
$2$. The $\xi$'s are typically suitably chosen but unknown linear
functions of $X$'s, sometimes called indices. This is called the
semi-parametric modelling approach, which is also a rapidly
expanding field. For comprehensive accounts of the above
developments, see, e.g., Fan and Yao (2005) and Gao (2007). Another
way is to impose some simplifying structure on (5) such as zero
interaction as in Chen and Tsay (1993), who gave
\begin{eqnarray}
X_t = F(X_{t-1}) + \cdots + F(X_{t-p}) + \varepsilon_t.
\end{eqnarray}



<!-- \noindent -->
ACKNOWLEDGEMENTS

Reprinted with permission from Lovric, Miodrag (2011), International
Encyclopedia of Statistical Science. Heidelberg: Springer Science
+Business Media, LLC

<!-- \noindent -->

====References====
{|
|-
|valign="top"|{{Ref|1}}||valign="top"|  Bollerslev, T. (1986). Generalized Autoregressive Conditional Heteroskedasticity.  ''J. Econometrics''. vol. 31, 307-327.\ 
|-
|valign="top"|{{Ref|2}}||valign="top"|  Chen, R. and Tsay, R.S. (1993). Nonlinear Additive ARX Models.  ''J. Amer. Stat. Ass.'' vol. 88, pp. 955-967.\ 
|-
|valign="top"|{{Ref|3}}||valign="top"|  Cryer, J.D. and Chan, K.S. (2008)  ''Time Series Analysis: With Applications in R.'' New York: Springer Verlag.\ 
|-
|valign="top"|{{Ref|4}}||valign="top"|  Engle, R. (1982). Autoregressive Conditional Heteroscedasticity with Estimates of Variance of United Kingdom Inflation.  ''Econometrica.'' vol. 50, 987-1008.\ 
|-
|valign="top"|{{Ref|5}}||valign="top"|  Fan, J. & Yao, Q. (2005).  ''Nonlinear Time Series: Nonparametric and Parametric Methods.'' New York: Springer-Verlag.\ 
|-
|valign="top"|{{Ref|6}}||valign="top"|  Gao, J. (2007).  ''Nonlinear Time Series: Semiparametric and Nonparametric Methods.'' London: Chapman and Hall/CRC.\ 
|-
|valign="top"|{{Ref|7}}||valign="top"|  Hannan, E. J. (1973). The asymptotic theory of linear time-series models.  ''J. Applied Probability.'' vol. 10, 130-145.\ 
|-
|valign="top"|{{Ref|8}}||valign="top"|  Moran, P.A.P. (1953). The statistical analysis of the Canadian lynx cycle.  ''Aust. J. Zoology.'' vol. 1, 163-173.\ 
|-
|valign="top"|{{Ref|9}}||valign="top"|  Tong, H. (1983).  ''Threshold Models in Nonlinear Time Series Analysis.'' New York: Springer Lecture Notes in Statistics.\ 
|-
|valign="top"|{{Ref|10}}||valign="top"|  Tong, H. (1990).  ''Non-linear Time Series: A Dynamical System Approach.'' Oxford: Oxford University Press.\ 
|-
|valign="top"|{{Ref|11}}||valign="top"|  Tong, H. & Lim, K.S. (1980). Threshold autoregression, limit cycles and cyclical data.  ''J. Roy. Stat. Soc.'' series B, vol. 42, 245-292.\ 
|-
|valign="top"|{{Ref|12}}||valign="top"|  Whittle, P. (1954). The statistical analysis of a seiche record.  ''Sears Foundation J. Marine Research.'' vol. 13, 76-100.\ 
|-
|valign="top"|{{Ref|13}}||valign="top"|  Yule, U. (1927). On a method of investigating periodicities in disturbed series, with special reference to Wolfer's sunspot numbers.  ''Philos. Trans. Roy. Soc., London'', series A, vol. 226, 267-298.

|-
|}

<!-- \end{document} -->

<references />

[[Category:Statprob]]