Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GARCH model with exogenous regressors not functioning? #160

Open
KenRoytman opened this issue Feb 21, 2017 · 29 comments
Open

GARCH model with exogenous regressors not functioning? #160

KenRoytman opened this issue Feb 21, 2017 · 29 comments
Assignees

Comments

@KenRoytman
Copy link

Hello,

I'm struggling to figure out how to properly use this package to fit a GARCH(1,1) model with an exogenous variable. Here's an example Jupyter notebook to illustrate what I'm trying to do.

In short, using the canonical example of daily S&P 500 returns, I'm trying to add a dummy variable to a GARCH(1,1) model to examine the effect of Mondays. If I fit a model like so, arch_model(returns, x=mondays).fit().summary(), the summary output does not include any information for the dummy variable, making me think the variable was ignored altogether.

Looking through the ARCH documentation, I found a page specifying that I may need to specify a mean model for exogenous regressors. If I explicitly specify the mean model to be HARX, like so, arch_model(returns, x=mondays, mean='HARX').fit().summary(), I receive the following exception: ValueError: x must be nobs by n, where nobs is the same as the number of elements in y.

Am I missing something simple here? I'm trying to use this package for a graduate class in econometrics, and this question is causing some serious pain. Any help would be greatly appreciated. Thanks!

@dimab0
Copy link

dimab0 commented Feb 21, 2017

Unfortunately, I think the arch package seems to support only exogenous variables in calculating the mean, not in estimating the variance. I don't think the package supports what you are looking for, which is a regression for variance as a function of previous returns, previous variance, and this additional dummy value.

@KenRoytman
Copy link
Author

ARCH contributors, is the comment from @dimab0 true? I'm trying to fit the following model:

image

@bashtage
Copy link
Owner

What @dimab0 said is correct - there is not support for exogenous variance regressors in the current version.

@KenRoytman
Copy link
Author

@bashtage, thanks for the clarification. Do you know of a Python or R package that does support that functionality?

@bashtage
Copy link
Owner

bashtage commented Feb 21, 2017 via email

@KenRoytman
Copy link
Author

Interesting. The professor for the class recommended that we use EViews for class assignments; I was hoping to build my skill set on more widely used platforms (python/R).

Is there a reason why exogenous variance regressors are not currently supported? Is this a tool that's not heavily used in the GARCH/financial econometrics space?

@bashtage
Copy link
Owner

In no particular order:

  • They aren't incredibly common in practice
  • General support for exogenous regressors can be more difficult to use to ensure variances stay > 0, unless restricted
  • It takes time to implement anything, which means not doing other work

@KenRoytman
Copy link
Author

Understood. I'm happy to submit a pull request. I'm just trying to understand the area a little better.

Thanks for the quick replies!

@bashtage
Copy link
Owner

PRs welcome 👍

@lawofearth
Copy link

Is there any Makov-Switching GARCH out there on Python?

@bashtage
Copy link
Owner

It has not been coded in this toolbox and I haven't come across it.

@murphywe
Copy link

Is it still true that there is no support for exogenous variance regressors in this package?

@bashtage
Copy link
Owner

I have not added exogenous regressors for volatility.

@lawofearth
Copy link

@bashtage do you came across any Vuong test on Python yet?

@camontanezp
Copy link

camontanezp commented Aug 21, 2018

Dear all,

I am trying to use an ARX model (with exogenous variables) but I don't know how to pass the exogenous variables to the forecast method. I have tried adding a parameter x or exogp with the numpy array of the regressors to the following:
forecasts = am.forecast(horizon=n_periods, start=params['lags'][-1]-1)

But I get the error that x or exgop don't exist.

How can I get this problem solved?

The model is difined as follows:

params = {'lags': [11,14,20,22,28,30,32,37,47], 'volatility': HARCH([1]), 'dist': Normal()}
am = HARX(y_obs, lags=params['lags'])
am.volatility = params['volatility']
am.distribution = params['dist']

And I only need exogenous regressors for the mean.

Many thanks!!

@bashtage
Copy link
Owner

@camontanezp I think you are missing an argument to forecast. When I do the following:

from arch.univariate import HARX, HARCH, Normal
params = {'lags': [11,14,20,22,28,30,32,37,47], 'volatility': HARCH([1]), 'dist': Normal()}
am = HARX(y_obs, lags=params['lags'])
am.volatility = params['volatility']
am.distribution = params['dist']
res = am.fit()

fcasts = am.forecast(res.params, horizon=n_periods, start=params['lags'][-1]-1)

it works as expected.

You are missing res.params from my code above.

@quantvol
Copy link

@bashtage, thanks for the clarification. Do you know of a Python or R package that does support that functionality?

Not sure if you still care, but rugarch in R does support external regressors for variance.

@johnukfr
Copy link

johnukfr commented Feb 5, 2020

Hi,

I am struggling with this exact issue, and I am working on a solution.
At the moment, the following is the closest thing to a solution I found online, I am working from there to try and create a class that allows you to estimate cofactors in any GARCH, GJR-GARCH or EGARCH models with or without exogenous variables and provide forecasts for any number of ‘steps’ ahead; I’ll post it on GitHub and link it here when I finish it:

https://github.com/duffau/RNN_GARCH/blob/28d39ed677bcf0b276a526b81e8dfe6b197f0933/vol_model/GARCH.py#L7

@bashtage
Copy link
Owner

The design of arch makes this a bit trickier than ideal since a VolatiltiyProcess usually takes inputs from the Mean model. I think I know how to solve it in a reasonable way to make GARCH-X and EGARCH-X models.

@shengnan92
Copy link

shengnan92 commented May 8, 2020

I go back the arch_model() function code. It needs the "x" to be a 2-dimensional array. If not, then it shows the same error. However, I do not know why "x" should be a 2-dimensional array. After I reshape "x" as a [79,1] array (because I have 79 observations), it yields the following output. If someone is familiar in this field, please give me some comments. Thank you. @bashtage

AR-X - EGARCH Model Results                          
==============================================================================
Dep. Variable:                      y   R-squared:                      -0.084
Mean Model:                      AR-X   Adj. R-squared:                 -0.098
Vol Model:                     EGARCH   Log-Likelihood:               -134.641
Distribution:                  Normal   AIC:                           279.283
Method:            Maximum Likelihood   BIC:                           291.130
                                        No. Observations:                   79
Date:                Fri, May 08 2020   Df Residuals:                       74
Time:                        03:05:56   Df Model:                            5
                                 Mean Model                                 
============================================================================
                 coef    std err          t      P>|t|      95.0% Conf. Int.
----------------------------------------------------------------------------
Const         -0.3382      0.117     -2.885  3.918e-03     [ -0.568, -0.108]
x0         9.0525e-05  4.479e-06     20.213  7.499e-91 [8.175e-05,9.930e-05]
                              Volatility Model                             
===========================================================================
                 coef    std err          t      P>|t|     95.0% Conf. Int.
---------------------------------------------------------------------------
omega          0.0413  4.952e-02      0.834      0.404 [-5.576e-02,  0.138]
gamma[1]      -0.3810      0.101     -3.763  1.681e-04    [ -0.579, -0.183]
beta[1]        1.0000  1.944e-02     51.441      0.000    [  0.962,  1.038]
===========================================================================

@johnukfr
Copy link

johnukfr commented May 8, 2020

Hi shengnan92, X can be forced to be a 2 dimensional array and still work with only one set of observations because you (the code in this case) can set the 1st array to be zeros. This is a somewhat common thing to do because it allows for generalisation in adding a constant to your model. E.g.: if you want a constant in your model, the 1st X row would be made fully of 1s.

@bashtage
Copy link
Owner

bashtage commented May 15, 2020

The issue with the exog shape has been fixed in master. Master and the next release will accept 1d exog as long as it has the right number of observations.

@woneric
Copy link

woneric commented Jun 9, 2020

Hi, I am also having problems with the exogenous model forecast using ARX function in the arch_model library. I have one exogenous continuous variable S&P500 I want to use to help predict the mean log returns of another variable using ARX mean model. I am having issues getting a multi-step forecast horizon. I have tried many fixes based upon the comments in this chain, but nothing is working. Here is a copy of the code that get me one step ahead forecasts.

import numpy as np
import pandas as pd
import datetime 
month_df = pd.read_csv('quant_strat_monthly.csv',index_col=[0,1])

Dataset = month_df.loc[month_df.index.get_level_values(0) ==1341,['SP500','log_return']]
Dataset =month_df[['SP500','log_return']].xs(1341, level = 'portfolioID')

random_state = np.random.RandomState(1)
X = Dataset[['SP500']]
X = X*100 # manual suggest scaling returns by 100 
y = Dataset[['log_return']]
# y = y*100 y should already be scaled by 100
volatility = GARCH(1,0,1)
dist = StudentsT(random_state = random_state)
mod = ARX(y=y,x=X,  lags = [1], volatility = volatility, distribution = dist )
res = mod.fit(disp = 'off')
forecasts = res.forecast(start = '2019-12-31',horizon = 6 )
forecasts.mean

I'm hoping you can tell me how to correct my code to get multi-step forecast results when I include an exogenous variable.

@bashtage
Copy link
Owner

There is no support for multistep forecasts with exogenous regressors, so you will need to roll your own.

@kcho5820
Copy link

Hi @bashtage

I'm too having trouble with GARCH model with exogenous regressors.

Below is my code and I keep having the value error.

Was wondering if you might know where I got it wrong?

data_c1 = yf.download("RIO.AX", start='2010-01-01', end='2022-03-09')
data_c3 = yf.download("CPU.AX", start='2010-01-01', end='2022-03-09')

p_rio = data_c1['Adj Close']
p_cpu = data_c3['Adj Close']

r_rio = 100 * np.log(p_rio).diff().dropna()
r_cpu = 100 * np.log(p_cpu).diff().dropna()

last_in = '2020-03-18'
first_fc = '2020-03-19'
nf = len(r_cpu[first_fc:])
r_window = len(r_cpu[:first_fc])

y = r_cpu[1:]
x = r_rio[:-1]

def qt(p, df):
    return stats.t.ppf(p, df) * np.sqrt((df - 2) / df)

s_garch = pd.Series(index=r_cpu[first_fc:].index, dtype='float64')
var25_garch=pd.Series(index=r_cpu[first_fc:].index, dtype='float64')

for i in tqdm(range(nf)):
    y_train = y[i:i+r_window]
    x_train = x[i:i+r_window]
    x_fc = x_train[i+r_window:i+r_window+1] #shape of 1x1
    
    ar1_garch_t = arch_model(y_train, x=x_train, mean='ARX', vol='GARCH', p=1, q=1, lags=1, dist='StudentsT')
    ar1_garch_t_fit = ar1_garch_t.fit(first_obs=i, last_obs=i+r_window, disp='off')
    
    fc = ar1_garch_t_fit.forecast(horizon=1, x=x_fc, reindex=False)
    
    s = np.sqrt(fc.variance)
    s_garch[i]=s.loc[s_garch.index[i]]

    var25 = fc.mean + s * qt(0.025, ar1_garch_t_fit.params['nu'])
    var1 = fc.mean + s * qt(0.01, ar1_garch_t_fit.params['nu'])

image

@bashtage
Copy link
Owner

I still think exog variables haven't been implemented yet.

@julietaylor9911
Copy link

julietaylor9911 commented Jan 14, 2023

Are there any other standard garch implementations in Python? Hard to believe there isn't a package in Python with support for exogenous variance variables

@filanarfahz
Copy link

Hi @kcho5820 , it's a long shot but did you manage to fix this problem? I am having the same issue at the forecasting step.

@bashtage bashtage self-assigned this May 26, 2023
@billbwu
Copy link

billbwu commented Jun 4, 2024

The design of arch makes this a bit trickier than ideal since a VolatiltiyProcess usually takes inputs from the Mean model. I think I know how to solve it in a reasonable way to make GARCH-X and EGARCH-X models.

Hi @bashtage, since exogenous regressors are still not supported in the variance step, could you elaborate a bit on what you meant by the above re a reasonable way to make GARCH-X models work using arch? Just trying to see if there can be a relatively quick/simple workaround on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests