As an alternative, we might change our framework and compute the sample average of our process across time, that means:. We want this quantity to converge towards the true mean of the time series, and we can achieve that under two conditions.
Plus, the autocovariance of two or more elements of the process has to depend only on the relative distance in time among them. In formula:. We can also strengthen this condition, introducing the so called strong stationarity, which occurs when the joint distribution of two or more elements depends only on their relative distance.
Note that strong stationarity implies weak stationarity, provided that the covariance exists. Here, if we compute the sample average across time, we will obtain something which overestimates the true values in the first periods and underestimate them in the last periods:. On the other hand, if we take the first difference of this series, we obtain something which looks way more stationary:.
And, computing the mean, we can see how this time it is a more accurate approximation of our series:. Now, is stationarity sufficient for our sample mean to be a good approximation of the real mean? The answer is no, since we need a further condition to reach this goal. With ergodicity, we are asking our process to move around the average, taking values all over its support. Imagine a stochastic process where each X is a Bernoulli binari R. Plus, our process is such that it takes a value 0 or 1 at the initial moment and stays fixed at this value for ever.
Hence, our time series will look like a straight line at either 1 or 0. We know that the true average of our process is something between 0 and 1, however, the average computed across time will return always 1 or 0. The idea behind ergodicity is that, while collecting more and more observations, we keep learning something new about the process.
If you think about the example above, you can now see how that process is clearly not ergodic: no matter how many observations we collect, there is no further information we are gathering, since everything is known since the beginning from the initial value taken by our process. Stationarity and Ergodicity are the basic assumptions to perform time series analysis, and it is important to have in mind how to achieve them and how to test whether they hold.
I'm a years-old student based in Milan, passionate about everything related to Statistics, Data Science and Machine Learning. The test needs quite strong hypotheses: the knowledge of the model producing the time series and the assumptions on the error term.
An important extension of the test was proposed by Said and Dickey where it is shown that the original Dickey-Fuller procedure is valid also for a more general ARIMA p ,1, q in which p and q are unknown. The extension allows using the test in the presence of a serial correlated error term.
Adding lags to the autocorrelation allows the elimination of the effect of serial correlation on the test statistics Phillips and Xiao An alternative procedure was proposed by Phillips using a semi-parametric test Phillips and Xiao allowing the error term to be weakly dependent and heterogeneously distributed Phillips ; Phillips and Perron Another test proposed as complementary to the unit root tests is the KPSS test Kwiatkowski, Phillips, Schmidt and Shin in which the null-hypothesis instead of being nonstationarity is stationarity.
Note that all the tests described above are parametric in the sense that they need assumptions about the stochastic process generating the tested time series. In a framework where the complexity of the model is such that an analytical form has been regarded as not able to represent the system, such assumptions over the data generator process may be too restrictive. Therefore, in addition to the parametric tests, it can be interesting to perform a nonparametric test that does not need any assumption over the data generator process.
As noted in the introduction, the problem with parametric tests is the power of the test. Fewer assumptions imply the need for more information thus more observations. Given that the test will be made on the artificial data, the power of the test is not a problem since the number of available observations can be increased at will with virtually no costs.
Our interest is to understand how a given set of moments of the simulated time series behaves. If we want to test the equilibrium properties of the model, if we want to compare observed and simulated moments or if we want to understand the effect of a given policy or change in the model, we are interested in the stationarity and the ergodicity of a given set of moments.
The test which is described in the next section will test whether a given moment is constant during the time series, using as the only information the agent-based model. The nonparametric test used is an application of the Wald-Wolfowitz test Wald and Wolfowitz As it will be shown the Wald-Wolfowitz test is suited for the type of nonparametric test needed and is easy to implement on any statistical or numerical software since the asymptotic distribution of its test statistic is a Normal distribution.
Stationarity Test 2. The Runs Test was developed by Wald and Wolfowitz to test the hypothesis that two samples come from the same population see paragraph about ergodicity below. Particularly the extension that uses the Runs Test to test the fitness of a given function will be employed Gibbons Given a time series and a function that is meant to explain the time series, the observations should be randomly distributed above and below the function if the function fits the time series, regardless of the distribution of errors.
The Runs Test tests whether the null hypothesis of randomness can be rejected or not. Given the estimated function, a 1 is assigned to the observations above the fitted line, and a 0 to the observations below the fitted line where 1 and 0 are considered as symbols. Supposing that the unknown probability distribution is continuous, probability will be 0 that a point lies exactly on the fitted line if this occurs that point should be disregarded.
The outcome of the described process is the sequence of ones and zeros that represents the sequence of observations above and below the fitted line. The statistics used to test the null hypothesis is the number of runs, where a run is defined as "a succession of one or more identical symbols which are followed and preceded by a different symbol or no symbol at all" Gibbons The number of runs, too many or too few runs, may reflect the existence of non-randomness in the sequence.
The Runs Test can be used with either one or two sided alternatives [3] Gibbons In the latter case the alternative is simply non-randomness, while the former with left tail alternative is more appropriate in the presence of trend alternatives and situations of clustered symbols, which are reflected by an unusually small number of runs.
Following Wald and Wolfowitz's notation , the U -statistic is defined as the number of runs, m as the number of points above the fitted function and n as the points below the fitted function. The mean and variance of the U -statistic under the null-hypothesis are 1 2 The asymptotic null-distribution of U , as m and n tend to infinity i.
In the implementation of the test, exact mean and variance 1 and 2 are used to achieve better results with few observations and equivalent results with many observations.
The derivation of the finite sample properties and of the asymptotic distribution of U is reported in the literature Wald and Wolfowitz ; Gibbons To conclude, the Runs Test tests the null-hypothesis that a given set of observations is randomly distributed around a given fitted function; it tests whether the fitted function gives a good explanation of the observations.
Defining the moment of order k as the non-centered moment of order k : 3 and supposing that we have an agent-based model, we want to test the stationarity of a given set of moments of the artificial time series. We may be interested in the behavior of the moments to compare them to some real data, or we may use the moments to analyze the behavior of the model under different conditions. In any case it is necessary to know whether the estimation of the artificial moment of order k is consistent i.
In order to check whether a moment is stationary, we have to check whether the moment is constant in time. The first step is to divide a time series produced with the model into w windows sub-time series. Then the moment of order k for each window is computed.
If the moment of order k is constant, the "window moments" are well explained by the moment of the same order computed over the whole time series "overall moment".
To test the hypothesis of stationarity the Runs Test is used: if the sample moments are fitted by the "overall moment" i. A strictly stationary process will have all stationary moments, while a stationary process of order k in this framework means that the first k non-centered moments are constant. Under the null hypothesis, longer windows imply a better estimation of the subsample moments, but at the same time they imply fewer windows given the length of the time series and a worse approximation of the distribution of runs toward the normal distribution.
The trade off can be solved by using long series and long windows. In the following, Monte Carlo experiments will be made to check the performance of the test; in particular a time series of observations will be used, and the performance of the test on processes will be checked.
The following window lengths will be used [4] : 1, 10, 50, , , , , By changing the length of the windows the number of samples is changed since the length of the time series is fixed.
The experiments have been carried out using the two-tail test. The different behavior of the first moment in the two different processes is clear. The test assigns a 0 to the window moments below the overall moment and a 1 to the window moments above the overall moment.
The different behavior of the two processes is detected by the test from the different number of Runs. In figure 1 the overall mean is a good estimator of the window means and the null-hypothesis cannot be rejected , in figure 2 the overall mean is not a good estimator of the window means the null-hypothesis is rejected.
Figure 1. The black line is the time series, the red line is the overall mean, the blue dots are the window means shown in the middle of the windows, the window length is In a stationary series the window moments are randomly distributed around the overall moment. Figure 2. The overall mean is not a good estimator of the window means, the test detects non-stationarity due to the small number of runs the number of runs defined on the window moments is only 2 in this example.
Figure 3. The process is stationary. Figure 4. Figure 5. The process is non stationary. The null hypothesis is that the first moment is constant, and in turn that the sub-time series of moments are fitted by the overall first moment. Since a type-I error [5] equal to 0. It is interesting to note that the length of the windows has no influence when the process is strictly stationary. Particularly, if every observation has the same distribution, the stationarity can be detected even when the window length is equal to 1.
Non-stationarity is also simple to detect; the test has full power it can always reject the null when the null is false for all the window lengths except the ones that reduce the number of windows under the threshold of good approximation of the normal distribution the test has power 1 as long as the number of samples is more than According to the experiments, the best option seems to be a window of length given the length of the whole time series that permits both the estimation of the subsample moments and at the same time the convergence of the distribution of the runs toward the normal distribution.
The test has to be repeated for every needed moment. The test outcome is correctly stationarity of the first moment and non-stationarity of the second moment. The "overall second moment" does in fact not fit the subsample second moments, and the test can detect this lack of fit caused by the limited number of runs only two in this case. Figure 6. The dots are the window moments, the line is the overall moment. The first moments are randomly distributed around the overall mean above.
The second moments are not randomly distributed around the overall moments below. If the length of the time series and the number of windows are properly set, the result stating stationarity for the tested moment is reliable, i. If non-stationarity is found, the traditional methods may be used to transform the series into stationary for example detrending or differentiating the series and the nonparametric test can then be used on the transformed series.
Ergodicity 3. If the process is stationary and ergodic the observation of a single sufficiently long sample provides information that can be used to infer about the true data generator process and the sample moments converge almost surely to the population moments as the number of observations tend to infinity see the Ergodic Theorem in Hayashi , p.
Ergodic processes with different initial conditions and in agent-based models, with different random seeds will thus have asymptotically convergent properties, since the process will eventually "forget" the past.
If the stationarity test reveals the convergence of the model toward a statistical equilibrium state, the ergodicity test can tell whether such equilibrium is unique. Supposing that we want to know the properties of a model with a given set of parameters, if the model is ergodic and stationary the properties can be analyzed by using just one long time series. If the model is non-ergodic it is necessary to analyze the properties over a set of time series produced by the same model with the same set of parameters but with different random seeds that is with a different sequence of random numbers.
It is even more important to take ergodicity into consideration if the model is compared with real data. If the data generator process is non-ergodic, the moments computed over real data cannot be used as a consistent estimation of the real moments, simply because they are just one realization of the data generator process, and the data generator process produces different results in different situations. A typical example of a stationary non-ergodic process is the constant series.
The process is stationary and non-ergodic. Any observation of a given realization of the process provides information only on that particular process and not on the data generator process. Despite the importance of the ergodicity hypothesis in the analysis of time series, the literature about ergodicity tests is scarce. Domowitz and El-Gamal ; describe a set of algorithms for testing ergodicity of a Markovian process.
The main intuition behind the test is the same as the one used in this paper: if a data generator process is ergodic it means that the properties of the produced time series as the number of observations goes to infinity is invariant with respect to the initial conditions. The test described below is different from the one described by Domowitz and El-Gamal since it involves different algorithms, uses a different nonparametric test and is intended to be used directly on any computational model an example of application to a simple agent-based model will be shown in section 4.
The algorithm basically creates two samples representing the behavior of the agent-based model with different random seeds and compares the two samples using the Wald-Wolfowitz test Wald and Wolfowitz There are a number of alternative nonparametric tests that could have been used such as the Kolmogorov-Smirnov test used by Domowitz and El-Gamal and the Cramer-Von Mises test for references about nonparametric tests see for example Darling , Gibbons and Gibbons and Chakraborti The Wald-Wolfowitz test was chosen due to its simplicity and to the fact that under the null-hypothesis the test statistic is asymptotically distributed as a Normal, which implies an easy implementation with most statistical and numerical software and libraries in this paper the Rpy library for Python was used.
The ergodicity test says if the first moments for example of a series can be used as the estimation of the true moment of the data generator process. It is necessary to replicate the test for every moment required. Ergodicity Test 3.
Eventually, the V set is created, i. In the event that null is true, the distribution of U is independent of f x and g x. A difference between g x and f x will tend to decrease U.
The mean and the variance of the U -statistics are 1 and 2. If m and n are large, the asymptotic distribution of U is a Normal distribution with the asymptotic mean and the asymptotic variance as in the stationarity test, the exact mean and variance to implement the test are used.
Given the actual number of runs, U , the null hypothesis is rejected if U is too low U is tested against its null distribution with the left one-tailed test. In this case the aim is to use this test as an ergodicity test supposing that the stationarity test has not rejected the null hypothesis of stationarity.
For an example of the opposite case i. The time average of of every sample function is equal to zero, as is the ensemble average over all time.
So the process is ergodic. However, the variance of any individual sample function shows the original square wave dependence on time, so the process is not stationary. This particular example is wide-sense stationary, but one can concoct related examples that are still ergodic but not even wide-sense stationary.
Sign up to join this community. The best answers are voted up and rise to the top. Stack Overflow for Teams — Collaborate and share knowledge with a private group. Create a free Team What is Teams? Learn more. What is the distinction between ergodic and stationary? Ask Question. Asked 9 years, 10 months ago. Active 3 months ago. Viewed 58k times. This is my understanding so far.
Now, it seems to me that a signal would have to be stationary, in order to be ergodic. And what kinds of signals could be stationary, but not ergodic? If a signal has the same variance for all time, for example, how could the time-averaged variance not converge to the true value? So, what is the real distinction between these two concepts? Can you give me an example of a process that is stationary without being ergodic, or ergodic without being stationary? Improve this question.
Gilles 3, 3 3 gold badges 18 18 silver badges 28 28 bronze badges. Matt Matt 1 1 gold badge 8 8 silver badges 9 9 bronze badges. I just cannot understand what Stationary Ergodic Process article is doing in Wikipedia? Does it mean that there is non-stationary ergodic process?
Add a comment. Active Oldest Votes. Stationarity refers to the distributions of the random variables. Ergodicity, on the other hand, doesn't look at statistical properties of the random variables but at the sample paths , i. Referring back to the random variables, recall that random variables are mappings from a sample space to the real numbers; each outcome is mapped onto a real number, and different random variables will typically map any given outcome to different numbers.
Ergodicity then deals with properties of the sample paths and how these properties relate to the properties of the random variables comprising the random process.
0コメント