Question about ARMA in new C# version of Suanshu

Hi Itay,

The constructor you have seen in other libraries expects you to provide a data series and the number of AR and MA coefficients respectively. Presumably it will then FIT a model to the data you have provided.

While this may be fine for many use cases, in truly object oriented programming it is considered good practice if each class will only have a single concern (i.e. being the model), rather than also fitting a model to data at the same time. That way you are not binding your code to any particular implementations of fitting etc.

So in our constructor you are only creating the model with EXISTING coefficients, rather than fitting it to data. Fitting of a model is performed via implementations of the ARMAFitting interface. We provide an implementation class called ConditionalSumOfSquares which can be used to fit a model and compute the AIC:

<br />
ARMAFitting fitting = new ConditionalSumOfSquares(data, AR, 0, MA);<br />
ARIMAModel model = fitting.getModel();<br />
double AIC = fitting.AIC();

I am not sure what residual series you would like to compute. Please let me know and I will try to help.

Otherwise you may find some helpful information at: http://www.numericalmethod.com/javadoc/suanshu/

Johannes Lehmann

Hello Johannes,

Thank you very much for the quick response, much appreciated 🙂

I am trying to use ARMA (or ARIMA) and GARCH to create a model on a log return time series.

I would like to find the optimal values of the AR & MA parameters by creating various combinations of AR & MA and grading them via the AIC (you can think of it as a brute force like grid search, so for example for a particular series ARMA(2,1) is the best model).

Afterwards, I would like to take the residual from the ARMA and apply it to GARCH (probably GARCH(1,1)) and get a prediction (for the variance and also the mean).

I did not see a way to get the residual series from ARMA or ARIMA – do you provide such a functionality.

In Matlab this can be done using a combined ARMA/GARCH model (I do not need to get the residuals from ARMA, only get the optimal AR & MA coefficients), here is what I do in Matlab :

I am using the ‘arima’ function to find the best arima parameters (I use the log likelyhood as the AIC) then I build a ‘garchset’ object :

spec = garchset(‘R’, p, ‘M’, q, ‘P’, 1, ‘Q’, 1);

where p & q – are the best AR & MA from ARIMA, and the rest are for GARCH(1,1), and then I use ‘garchfit’ and ‘garchpred’ to fit a GARCH and get a prediction for the variance and the mean

Could you please let me know if this be done using your library.

Also, I tried the following code (series is a double[] containing the log return series)

<br />
<br />
ARMAFitting fitting = new ConditionalSumOfSquares(series, 1, 0, 1);<br />
ARIMAModel arimaModel = fitting.getModel();<br />
double AIC = fitting.AIC();<br />
<br />
IntTimeTimeSeries xt = new SimpleTimeSeries(series);<br />
ARIMAForecast arimaForecast = new ARIMAForecast(xt, arimaModel);<br />
ARIMAForecast.Forecast forecast = arimaForecast.next();<br />
<br />

When I run the code it gets stuck on the last line – arimaForecast.next(), and finally after several minutes an exception pops us saying ‘order’ must be positive : 0′

Could you please advise what can be done.

Thank you, Itay

Hello Itay,

In SuanShu, you need to do the procedure step by step. In other words, you need to

2. fit an ARMA model to your time series

3. compute the residuals

4. fit a GARCH model to the residuals

There is no one command to do it as SuanShu is a programming library (of components).

Here is a sample code that I would do it:

<br />
    public static void fit_arima_residual_garch() {<br />
        // the time series<br />
        IntTimeTimeSeries x = new SimpleTimeSeries(new double[]{-1.1, 0.514, 0.116, -0.845, 0.872, -0.467, -0.977, -1.699, -1.228, -1.093});<br />
        double[] xt = x.toArray();<br />
<br />
        // fit an ARMA(1,1) model<br />
        ConditionalSumOfSquares fit1 = new ConditionalSumOfSquares(<br />
                xt,<br />
                1, 0, 1);<br />
        ARMAModel arma = fit1.getARMAModel();<br />
        System.out.println(String.format("the ARMA(1,1) model: %s", arma.toString()));<br />
<br />
        // residual time series     <br />
        ARMAForecastOneStep xt_hat = new ARMAForecastOneStep(x, arma);<br />
        double[] residuals = new double[xt.length];<br />
        for (int t = 0; t < xt.length; ++t) {<br />
            residuals[t] = xt[t] - xt_hat.xHat(t);<br />
        }<br />
<br />
        // fit a GARCH(1, 1) model<br />
        GARCH fit2 = new GARCH(residuals, 1, 1);<br />
        GARCHModel garch = fit2.getModel();<br />
        System.out.println(String.format("the GARCH(1,1) model: %s", garch.toString()));<br />
    }<br />

The same example is found here: http://www.numericalmethod.com/trac/numericalmethod/browser/public/Examples/src/com/numericalmethod/suanshu/examples/LinearTimeSeries.java

I got the following output:

Example fitting an ARMA and then GARCH on residuals the ARMA(1,1) model: xt = -0.36115827438444137 + (1.0*X_{t-1}+0.3885927300077173*X_{t-2}) + (1.0*e_{t-1}+-0.18262679464688403*e_{t-2}) + e_t; var(e_t) = 0.595852, d = 0 the GARCH(1,1) model: 0.01782245936465693+ 0.003111 * e_{t-1}+ 0.874147 * h_{t-1}

Note: there is bug in the C# version to prevent [tt:31vhlzuf]arma.toString()[/tt:31vhlzuf] from working properly. We fixed it for the future releases.

As for the exception, could you copy and paste the exact exception message here? I do not see where in the code it says “order must be positive”.

Also, could you print out the fitted model values (AR, MA) to see what they are?

Hello Mr. Li,

Thank you very much for your detailed answer, I appreciate your help very much.

I will split my answer to several points for clarity :

1.) Regarding the exception, I did copy/paste the exact message, I suspect that in the message

‘order’ must be positive : 0′

‘order’ is also a parameter, maybe you could look for the string ‘must be positive’ to find its origin in the code.

2.) Regarding the great code you sent, I ran it, the line

ARMAForecastOneStep xt_hat = new ARMAForecastOneStep(x, arma);

took 65 seconds to execute on my laptop (Intel core i3),

which seems very long, do you have any experience as to why it takes so long.

3.) Also regarding the code, after producing the garch model, could you please explain how to get a prediction/forecast for the next point/value in the series.

4.) This is a question I thought for quite a while –

When using garch on a log return series, you would be able to get a forecast for the next value of the volatility of the log return series.

If instead, you create a volatility series of the log return series (for example – running standard deviation of some length), and use the garch on it, you would get a forecast for the volatility of the volatility series (vol of vol).

Please correct me if I am wrong until here.

Now, I saw in matlab (and also in R – using ‘garchFit’), that there is a possibility of a mix/combined model of both ARMA & GARCH (I speculate that to use it properly to forecast the next value in a log return series, I should use the log return series in ARMA, use the residuals from ARMA in GARCH and then get a forecast.

The forecast (output) of the combined model includes 2 values (both in matlab & R), a SigmaForecast and MeanForecast. and I am using the mean forecast as a prediction for the next value in the log return series.

I have a little problem understanding what this value represents in this context when I combine ARMA with GARCH, maybe you could clarify that.

Thank you very much, Itay

The C# version may be slow due to the IKVM translation. We need to look into it before giving a definite answer.

This is simple. For a k-step-ahead prediction, we just need to do it one step at a time. E.g., see pages 51 – 52 of this tutorial: http://jhqian.org/ec310/06timeseries.pdf (You basically plug the residual values and past vol forecasts into the GARCH model iteratively for k steps.)

I am not exactly sure what you are asking. You seem to have already answered your own questions. Basically, GARCH predicts the variance of the residuals (hence the variance of the conditional mean/forecast); ARMA predicts the conditional mean (which you use for your forecast).

For the initial sigma_{t-1}, we can start with sigma_0 ^ 2 = a_0 / (1 – alpha_1 – beta_1) for GARCH(1, 1), i.e., the theoretical asymptotic value. The k-step-ahead prediction converges to this limit when k approaches infinity. Note that for GARCH(p, q), let r = max(p, q), then sum_{i = 1}^r (alpha_i + beta_i) < 1.

Hello Mr. Li.

thank you for your answers,

1.) Thank you for explaining the initial sigma generation, I am looking only for the next forecast (of 1 value) for GARCH, do you have a function/method for that in your API ?

2.) Sorry for not being clear about my last question, it is a little hard to explain, I think this will be clearer :

– My data series is a log return series – I use only ARMA(p,q) on the data and get a mean forecast – Then I use the combined ARMA(p,q)/GARCH(1,1) and get a different mean forecast (from the one I got when only using ARMA(p,q) ) – I am trying to imitate the functionality of the combined ARMA/GARCH by using ARMA and GARCH seperatly in the following procedure :

– using the data (log return series) in ARMA, – getting the residuals (I do not use ARMA to forecast, only to generate the model/residuals), – use the ARMA residuals in GARCH, and then use a forecast from GARCH

– But since GARCH only forecasts variance, I am asking myself how can the combined ARMA/GARCH forecast a mean which is different then the mean forecast by only ARMA.

Hope it is more clear now.

Thank you, Itay

#1. The function/method to do it is: [tt:1bz7t0a4]GARCHModel.sigma2[/tt:1bz7t0a4].

#2. You are talking two things here as there are two ways to do the fitting/forecast.

First, the matlab way: you assume a particular form of ARMA-GARCH model for the data set and the run MLE for the estimation.

Second, here is what we/SuanShu suggests:

2. etermine the lags (p and q) of the ARMA process and fit an ARMA(p, q) model. This is done by the usual ARMA fitting procedure., e.g., [tt:1bz7t0a4]ConditionalSumOfSquares[/tt:1bz7t0a4]

3. Select a suitable set of orders (P, Q) for the GARCH process. We can do this by looking at the PACF and ACF of the squared residuals and possibly use Ljung-Box test.

4. Fit a pure GARCH(P, Q) model to the residuals using conditional MLE.

5. Diagnostic checks.

You can do all steps 1 – 4 in SuanShu by calling the appropriate classes.

These two approaches make different assumptions and therefore will give different estimator values. On the other hand, in either case, we predict the conditional mean and conditional variance separately using first the ARMA equation for [tt:1bz7t0a4]MeanForecast[/tt:1bz7t0a4] and then the GARCH equation for [tt:1bz7t0a4]SigmaForecast[/tt:1bz7t0a4].

Below are the examples for both approaches.

One Step approach (in Matlab) http://www.mathworks.com/help/econ/garchpred.html

Two steps approach (in R) http://www.stat.berkeley.edu/~gido/Lab12

Hello Mr. Li,

Thank you for the details response,

I would like to take the research from the domain of Matlab and R to the domain of C#, so that is why I was trying to re-create the “one step approach” in C#.

I will look closely at your recommendations, Again, Thank you very much for the guidance,

Do you consider the C# version of your library as a test or do you intend to make it into a product (that I can buy) with appropriate speed optimization ?

Thank you, Itay

Hello Mr. Li,

I took some time to think about what you wrote, I think that the main issue is how to use GARCH in order to modify the ARMA/ARIMA forecasting and get a better result then when only using ARMA/ARIMA alone.

Could you please forward me (if you can) to any articles or documents that might shed some light on this issue.

Thank you, Itay

An ARMA model assumes constant conditional variance, but the ARMA-GARCH model assumes heteroskedasticity, which is the ‘H’ in GARCH. You may want to take a look at Sections 6.3 and 6.4 of Lai’s book: Statistical Models and Methods for Financial Markets.

http://www.amazon.com/Statistical-Methods-Financial-Springer-Statistics/dp/0387778268

Hello Mr. Li,

Thank you very much for the book reference, equations 6.34 and 6.35 describe exactly what I am looking for.

The book also mentions the Matlab garchFit function, I am trying to create/implement the same functionality in the .NET/C# domain :

1. Use ARMA to model a timeseries 2. Use Garch to model the ARMA residuals 3. Reestimate the ARMA parameters by using the result from step 2. 4. Continue the loop of steps 1 – 3 until the error is minimized (smaller then some epsilon threshold) or until the likelihood is maximized.

Your library provides the basic building blocks, which are the ARMA and the GARCH models separately, I would like to build a layer on top of them which uses them as I described in the steps above.

I would be very grateful if you could advise me or give some pointers on how this can be done using your library.

Thank you, Itay Dagani