bashtage / arch

ARCH models in Python

arch

Autoregressive Conditional Heteroskedasticity (ARCH) and other tools for financial econometrics, written in Python (with Cython and/or Numba used to improve performance)

Metric
Latest Release
Continuous Integration
Coverage
Code Quality
Citation
Documentation

Module Contents

• Univariate ARCH Models
• Unit Root Tests
• Cointegration Testing and Analysis
• Bootstrapping
• Multiple Comparison Tests
• Long-run Covariance Estimation

Python 3

arch is Python 3 only. Version 4.8 is the final version that supported Python 2.7.

Documentation

Documentation from the main branch is hosted on my github pages.

Released documentation is hosted on read the docs.

More about ARCH

More information about ARCH and related models is available in the notes and research available at Kevin Sheppard's site.

Contributing

Contributions are welcome. There are opportunities at many levels to contribute:

• Implement new volatility process, e.g., FIGARCH
• Improve docstrings where unclear or with typos
• Provide examples, preferably in the form of IPython notebooks

Examples

Volatility Modeling

• Mean models
  • Constant mean
  • Heterogeneous Autoregression (HAR)
  • Autoregression (AR)
  • Zero mean
  • Models with and without exogenous regressors
• Volatility models
  • ARCH
  • GARCH
  • TARCH
  • EGARCH
  • EWMA/RiskMetrics
• Distributions
  • Normal
  • Student's T
  • Generalized Error Distribution

See the univariate volatility example notebook for a more complete overview.

import datetime as dt
import pandas_datareader.data as web
st = dt.datetime(1990,1,1)
en = dt.datetime(2014,1,1)
data = web.get_data_yahoo('^FTSE', start=st, end=en)
returns = 100 * data['Adj Close'].pct_change().dropna()

from arch import arch_model
am = arch_model(returns)
res = am.fit()

Unit Root Tests

• Augmented Dickey-Fuller
• Dickey-Fuller GLS
• Phillips-Perron
• KPSS
• Zivot-Andrews
• Variance Ratio tests

See the unit root testing example notebook for examples of testing series for unit roots.

Cointegration Testing and Analysis

• Tests
  • Engle-Granger Test
  • Phillips-Ouliaris Test
• Cointegration Vector Estimation
  • Canonical Cointegrating Regression
  • Dynamic OLS
  • Fully Modified OLS

See the cointegration testing example notebook for examples of testing series for cointegration.

Bootstrap

• Bootstraps
  • IID Bootstrap
  • Stationary Bootstrap
  • Circular Block Bootstrap
  • Moving Block Bootstrap
• Methods
  • Confidence interval construction
  • Covariance estimation
  • Apply method to estimate model across bootstraps
  • Generic Bootstrap iterator

See the bootstrap example notebook for examples of bootstrapping the Sharpe ratio and a Probit model from statsmodels.

# Import data
import datetime as dt
import pandas as pd
import numpy as np
import pandas_datareader.data as web
start = dt.datetime(1951,1,1)
end = dt.datetime(2014,1,1)
sp500 = web.get_data_yahoo('^GSPC', start=start, end=end)
start = sp500.index.min()
end = sp500.index.max()
monthly_dates = pd.date_range(start, end, freq='M')
monthly = sp500.reindex(monthly_dates, method='ffill')
returns = 100 * monthly['Adj Close'].pct_change().dropna()

# Function to compute parameters
def sharpe_ratio(x):
    mu, sigma = 12 * x.mean(), np.sqrt(12 * x.var())
    return np.array([mu, sigma, mu / sigma])

# Bootstrap confidence intervals
from arch.bootstrap import IIDBootstrap
bs = IIDBootstrap(returns)
ci = bs.conf_int(sharpe_ratio, 1000, method='percentile')

Multiple Comparison Procedures

• Test of Superior Predictive Ability (SPA), also known as the Reality Check or Bootstrap Data Snooper
• Stepwise (StepM)
• Model Confidence Set (MCS)

See the multiple comparison example notebook for examples of the multiple comparison procedures.

Long-run Covariance Estimation

Kernel-based estimators of long-run covariance including the Bartlett kernel which is known as Newey-West in econometrics. Automatic bandwidth selection is available for all of the covariance estimators.

from arch.covariance.kernel import Bartlett
from arch.data import nasdaq
data = nasdaq.load()
returns = data[["Adj Close"]].pct_change().dropna()

cov_est = Bartlett(returns ** 2)
# Get the long-run covariance
cov_est.cov.long_run

Requirements

These requirements reflect the testing environment. It is possible that arch will work with older versions.

• Python (3.9+)
• NumPy (1.19+)
• SciPy (1.5+)
• Pandas (1.1+)
• statsmodels (0.12+)
• matplotlib (3+), optional

Optional Requirements

• Numba (0.49+) will be used if available and when installed without building the binary modules. In order to ensure that these are not built, you must set the environment variable ARCH_NO_BINARY=1 and install without the wheel.

export ARCH_NO_BINARY=1
python -m pip install arch

or if using Powershell on windows

$env:ARCH_NO_BINARY=1
python -m pip install arch

• jupyter and notebook are required to run the notebooks

Installing

Standard installation with a compiler requires Cython. If you do not have a compiler installed, the arch should still install. You will see a warning but this can be ignored. If you don't have a compiler, numba is strongly recommended.

pip

Releases are available PyPI and can be installed with pip.

pip install arch

You can alternatively install the latest version from GitHub

pip install git+https://github.com/bashtage/arch.git

Setting the environment variable ARCH_NO_BINARY=1 can be used to disable compilation of the extensions.

Anaconda

conda users can install from conda-forge,

conda install arch-py -c conda-forge

Note: The conda-forge name is arch-py.

Windows

Building extension using the community edition of Visual Studio is simple when using Python 3.8 or later. Building is not necessary when numba is installed since just-in-time compiled code (numba) runs as fast as ahead-of-time compiled extensions.

Developing

The development requirements are:

• Cython (0.29+, if not using ARCH_NO_BINARY=1, supports 3.0.0b2+)
• pytest (For tests)
• sphinx (to build docs)
• sphinx-immaterial (to build docs)
• jupyter, notebook and nbsphinx (to build docs)

Installation Notes

1. If Cython is not installed, the package will be installed as-if ARCH_NO_BINARY=1 was set.
2. Setup does not verify these requirements. Please ensure these are installed.

arch's Issues

normalization constant in EGARCH doesn't handle StudentsT correctly

According to
http://uk.mathworks.com/help/econ/egarch-model.html
or
https://www.quantnet.com/threads/expectation-of-folded-students-t.15825/

the norm_const in EGARCH simulate() function, in case of studentsT distribution, should be:
np.sqrt((nu-2)/3.1415) * scipy.special.gamma((nu-1)/2) / scipy.special.gamma(nu/2)

Missing rng parameter in documentation

The method arch.univariate.GARCH.simulate misses a parameter called rng in the documenation. I assume it referes to a random number generator and should be a numpy rng.

Add example using `fix`

Add example showing how to use fix()

warning in unitroot_examples.ipynb

http://nbviewer.ipython.org/github/bashtage/arch/blob/master/examples/unitroot_examples.ipynb

shows following warning:

C:\Anaconda\lib\site-packages\arch-2.0-py2.7-win-amd64.egg\arch\unitroot\unitroot.py:1031: InvalidLengthWarning: 
The length of y is not an exact multiple of 12, and so the final 3
observations have been dropped.

  InvalidLengthWarning)

Potential bug in ARCHModel.fit()?

I was looking through some of the arch package code and noticed this code snippet and was wondering why it works. We initialize sigma2 to be a vector of zeros. The next thing that we do with sigma is use it in the denominator in line 445. Why doesn't this throw a divide by zero error?

Would it be safer to initialize it as a vector of ones, rather than zeros? I just want to make sure that I'm not missing something. If not, then I can make the change and submit the PR.

Setting params

I'm trying to manually input params like below:

    model = arch_model(returns)

    if params is None:
        res = model.fit(
            update_freq     = 0,
            disp            = 'off',
            starting_values = starting_values
        )
    else:
        res = arch.univariate.base.ARCHModelResult(
            params = params,
            model  = model
        )

    cvols = res.conditional_volatility

I'm getting an error. Looking through the code base, it appears I shouldn't be creating instances of ARCHModelResult. How do I create a time series of cvols given a returns series AND a set of parameters?

Thanks for your help!

Error when trying to fix a model

Hi, Bashtage:

I have a problem when I'm trying to fix a model. The use case I'm going after is to fit a model, store the parameters on disk and load it again to build a model using fix. But when I try that I get the error:

  File "/usr/local/lib/python3.6/site-packages/arch/univariate/base.py", line 322, in fix
    resids = self.resids(self.starting_values())
  File "/usr/local/lib/python3.6/site-packages/arch/univariate/base.py", line 567, in starting_values
    params = np.asarray(self._fit_no_arch_normal_errors().params)
  File "/usr/local/lib/python3.6/site-packages/arch/univariate/mean.py", line 538, in _fit_no_arch_normal_errors
    nobs = self._fit_y.shape[0]
AttributeError: 'NoneType' object has no attribute 'shape'

Simple code to reproduce is this:

import datetime as dt
import pandas_datareader.data as web

import arch

st = dt.datetime(1990, 1, 1)
en = dt.datetime(2016, 1, 1)

data = web.get_data_yahoo('^GSPC', start=st, end=en)
returns = 100 * data['Adj Close'].pct_change().dropna()

result = arch.arch_model(returns).fit()
fixed_result = arch.arch_model(returns).fix(result.params)

It this a supported usecase?

Non-zero exit code for fmin_slsqp

I am running into situations (mainly because of scaling issues of data) that are causing the scipy's optimizer to fail with errors like "Inequality constraints incompatible (Exit mode 4)". Currently, there is no way to know if the optimization succeeded or not as the code does not check the exit status. Would it be possible to either raise an exception if the exit_code from fmin_slsqp is non-zero? I am worried that if I run GARCH on a whole bunch of time-series, opt would fail at places without me noticing.

Thanks

How to get control over the sample size?

Hello,

I would like to use bootstrap to calculate statistical significance of mean values of composites.
The composites have length l and are taken from a time-series of length L > l. Now I would like to compute null-composites by randomly selecting the same number of dates (i.e.: l elements) from the whole time-series as had been in the original composite.

Playing around I ended up with:
from arch.bootstrap import MovingBlockBootstrap
x = numpy.arange(1000) (original dataset)
MovingBlockBootstrap(3,x).conf_int(numpy.mean, reps=10000, size=.95)

However, MovingBlockBootstraps leaves me with composites of length l = L = 1000. Is there a way to change this?

I wondered, whether a function like
import random
def func(x): return np.mean(random.sample(x,l))
could do the job. But testing this with l=200, I got quite high fluctuations for the confidence intervals...

I would be very happy about any hint!
Thank you very much!
L

PS: I'm not sure, whether this github page is the right place to ask that kind of questions. If not I am happily willing to migrate the question to any other forum.

Add a univariate MA model

Add a pure moving average model for the mean.

MCS does not work correctly when there are exact ties

If two loss columns have the exact same average, which can happen when using loss ranks, an error is produced.

variance definition

Hello bashtage,

I am working on a project to forecast volatility from a Garch(p,q) model.
I have two issues :
- what is the difference between forecast.variance function and the res.conditional_volatility?
And which one should I use to forecast volatility of the next days (t+1) with the information until t ?
- which definition do you use for the variance ? (how many days do you use to do the stdev ?)
I saw that you use the garch fitted function to forecast the volatility step by step but how do you
initialize your algorithme (first variance formula)

Thanks a lot for your work !

32 bit windows install is difficult

It is currently difficult to install this package on 32 bit windows using PyPi since there are only wheels for amd64.

Solution is to either include simple instructions of how to install on Windows 32, or to provide wheels for windows 32.

P-values do not match Rs library

Hello I have been using the arch library and compared the fit with R. The p-values do not match with R, likewise the t statistic and standard error of coefficients do not match. Forecasting function creates constant forecasts rather than time varying forecasts

Leave one out jackknife does not work with Pandas

Assumes NumPy slicing syntax.

Missing imports in __init__.py ?

Hi,
I just very briefly wanted to report an issue I had after installing the arch package with anaconda:
conda install -c https://conda.binstar.org/bashtage arch

Accessing the bootstrapping module with
from arch.bootstrap import *
was impossible unless manually extending __init__.py with
from .bootstrap import *

Maybe I got something wrong?!
Thanks!

IGARCH

Is it possible to estimate an IGARCH model? That is, restrict alpha = 1 - beta. I thought perhaps this could be done with bounds, but I'm confused about the syntax.

As an example, I'm trying to replicate Example 2 on page 10 of these slides: http://faculty.chicagobooth.edu/ruey.tsay/teaching/bs41202/sp2017/lec9-17.pdf

multivariate garch support?

I found there is no multivariate garch models in Python.

Could you add multivariate support?

Multiple Comparisons

A few issues are needed before the next release which will contain multiple comparisons

• Example Notebook
• Test components
• Improve RST Documentation
• Model Confidence Set
• Test paths for alternative bootstraps
• Test errors
• Update Readme
• Update version
• Publish to PyPI and binstar

Invalid indexer for conf_int() with method='bca' for version 4.2

Hi,

Thank you for this great package. I'm using version 4.2 with Python 3.6.

I encountered the following error with conf_int(method='bca').

Looking at the error message & debug info, I wonder if it's just a matter of changing line 463 to:

a = a[:,]

Happy to create a PR.

Thank v much.

Code example:

import numpy as np
import arch.bootstrap as ab

data = np.random.randn(1000)
sb = ab.StationaryBootstrap(5, data)
ci = sb.conf_int(np.mean, reps=1000, method='bca')

# output:

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-322-57d5b553d4ae> in <module>()
      2 
      3 sb = arch.bootstrap.StationaryBootstrap(5, data)
----> 4 ci = sb.conf_int(np.mean, reps=1000, method='bca')

H:\Anaconda3\lib\site-packages\arch-4.0-py3.6-win-amd64.egg\arch\bootstrap\base.py in conf_int(self, func, reps, method, size, tail, extra_kwargs, reuse, sampling, std_err_func, studentize_reps)
    461                         raise RuntimeError(message.format(jk_var=denom))
    462                     a = numer / denom
--> 463                     a = a[:, None]
    464                 else:
    465                     a = 0.0

IndexError: invalid index to scalar variable.

Here is the debug output:

> h:\anaconda3\lib\site-packages\arch-4.0-py3.6-win-amd64.egg\arch\bootstrap\base.py(463)conf_int()
    461                         raise RuntimeError(message.format(jk_var=denom))
    462                     a = numer / denom
--> 463                     a = a[:, None]
    464                 else:
    465                     a = 0.0

ipdb> a
self = Stationary Bootstrap(block size: 5, no. pos. inputs: 1, no. keyword inputs: 0, ID: 0x34501668)
func = <function mean at 0x00000000053340D0>
reps = 1000
method = 'bca'
size = 0.95
tail = 'two'
extra_kwargs = None
reuse = False
sampling = 'nonparametric'
std_err_func = None
studentize_reps = 0
ipdb> a[:,]
self = Stationary Bootstrap(block size: 5, no. pos. inputs: 1, no. keyword inputs: 0, ID: 0x34501668)
func = <function mean at 0x00000000053340D0>
reps = 1000
method = 'bca'
size = 0.95
tail = 'two'
extra_kwargs = None
reuse = False
sampling = 'nonparametric'
std_err_func = None
studentize_reps = 0
ipdb> q

EGARCH with StudentT distribution - Strange summary() output

Hi, Bashtage:

I obtained the following output when running

model = arch_model(y, p = 1, o= 1, q = 1, vol='EGARCH', power=2.0, dist='StudentT', hold_back=None)

The time series has mean very close to 0. The optimization terminated with successful convergence, but the estimated coefficients look strange. In particular,

R-squared: -63807790369.565
Log-Likelihood: -9228.30
AIC: 18468.6
mu: -225.7893

Optimization terminated successfully. (Exit mode 0)
Current function value: 9228.30379484
Iterations: 34
Function evaluations: 404
Gradient evaluations: 33
Constant Mean - EGARCH Model Results

Dep. Variable: GEM3S_EARNYILD R-squared: -63807790369.565
Mean Model: Constant Mean Adj. R-squared: -63807790369.565
Vol Model: EGARCH Log-Likelihood: -9228.30
Distribution: Standardized Student's t AIC: 18468.6
Method: Maximum Likelihood BIC: 18496.6
No. Observations: 783
Date: Thu, Jan 19 2017 Df Residuals: 777
Time: 12:36:37 Df Model: 6
Mean Model

             coef    std err          t      P>|t|        95.0% Conf. Int.

---

mu -225.7893 3.369e-02 -6701.832 0.000 [-2.259e+02,-2.257e+02]
Volatility Model

             coef    std err          t      P>|t|        95.0% Conf. Int.

---

omega -4.8296 3.453e-03 -1398.694 0.000 [ -4.836, -4.823]
alpha[1] 5.7902 2.325e-03 2489.986 0.000 [ 5.786, 5.795]
gamma[1] 4.9808 2.000e-03 2489.925 0.000 [ 4.977, 4.985]
beta[1] 0.9050 6.899e-05 1.312e+04 0.000 [ 0.905, 0.905]
Distribution

             coef    std err          t      P>|t|        95.0% Conf. Int.

---

nu 2.6422 2.694e-02 98.067 0.000 [ 2.589, 2.695]

Thank you in advance.
Tracy

Enhancement request: calculate conditional volatility using data from the last period

Currently, GARCHModelResult.conditional_volatility does not include results for the t+1period (t being the last time-period in the data). In other words, there is no way to get conditional vol which uses information from the last time period. Not sure if the .forecast method was designed for this purpose but that is not implemented currently.

My workaround was to basically re-implement recursion.garch_recursion by removing the 1 day lag.

It would be great if this was natively supported.

Threshold ARCH (TARCH)

Great job with the package!

According to this paper, the TARCH model outperforms other models.

Would be great if this was added to the library.

Failed convergence

I'm getting some failed convergence from fmin_slsqp while estimating a Garch(1,1) model on an actual time series.

First I noticed that the log likelihood was being evaluated on negative values for sigma2,
and I came upon this line which seems a bit suspect:

https://github.com/bashtage/arch/blob/master/arch/univariate/recursions.pyx#L196

To me, it looks like some sort of heuristic to limit the values of sigma2 to some reasonable values. However sigma2[t] could be greater that var_bounds[t,1] but arbitrarily close, so that sigma2[t] could become a negative number, arbitrarily large even. So maybe it would be better to replace that line by max(log(sigma2[t] - var_bounds[t,1]), 0). Although there might be some theoretical reasons for using a log here. Note that similar lines exist for the other type of recursions.

After making that fix, the log_likelihood is only evaluated on positive numbers for sigma2, so there is progress. However slsqp still fails to converge but that's another issue...

[BUG] Warnings cannot be ignored

Please refer to this SO question asked by me: http://stackoverflow.com/q/39993792/3765319

Basically, I make thousands of calls to fit a Constant Mean TARCH model, of which a few hundred will fail convergence. I would like a way to ignore warnings raised by the solver, so that the warnings do not slow my Jupyter notebook down. The best solution seems to be to change lines 507-511 in /univariate/base.py to these:

warnings.filterwarnings('always', append=True)
warnings.warn(convergence_warning.format(code=imode,
                                         string_message=smode),
              ConvergenceWarning)

This way, it will not override any user-defined warnings filters.

Logic behind starting_values calcs for GARCH

Hi, Kevin:

Could you please help explain what is the logic behind the starting_values calcs for GARCH?

The line target *= (scale ** power) is confusing. It raises up the target by scaling with variable "scale" which is the ratio between the estimated variance and the target variance. The power for scale seems problematic.

For example, suppose power = 3, then

target = sigma_hat^3
scale is sigma^2 / sigma_hat^2

And the nextline "target *= (scale ** power)" gives
sigma_hat^3 * ( sigma^6 / sigma_hat^6 )

Could you explain? Thank you!

def starting_values(self, resids):
    p, o, q = self.p, self.o, self.q
    power = self.power
    alphas = [.01, .05, .1, .2]
    gammas = alphas
    abg = [.5, .7, .9, .98]
    abgs = list(itertools.product(*[alphas, gammas, abg]))

    target = np.mean(abs(resids) ** power)
    scale = np.mean(resids ** 2) / (target ** (2.0 / power))
    target *= (scale ** power)

    svs = []
    var_bounds = self.variance_bounds(resids)
    backcast = self.backcast(resids)
    llfs = zeros(len(abgs))
    for i, values in enumerate(abgs):
        alpha, gamma, agb = values
        sv = (1.0 - agb) * target * ones(p + o + q + 1)
        if p > 0:
            sv[1:1 + p] = alpha / p
            agb -= alpha
        if o > 0:
            sv[1 + p:1 + p + o] = gamma / o
            agb -= gamma / 2.0
        if q > 0:
            sv[1 + p + o:1 + p + o + q] = agb / q
        svs.append(sv)
        llfs[i] = self._gaussian_loglikelihood(sv, resids, backcast,
                                               var_bounds)
    loc = np.argmax(llfs)

    return svs[loc]

Fixed Variance bug in Zig Zag code

Zig-Zag esimtation
A small repetitions of the previous two steps can be used to implement a so-called zig-zag estimation strategy.

Getting an error which is related to the unit_scale variable being set in the compute_variance method within FixedVariance. --> 'FixedVariance' object has no attribute '_stop'

for i in range(5):
print(i)
vol_mod = ZeroMean(res.resid.dropna(), volatility=GARCH(p=1,o=1,q=1))
vol_res = vol_mod.fit(disp='off')
variance[22:] = vol_res.conditional_volatility ** 2.0
fv = FixedVariance(variance )

doesn't work --> fv = FixedVariance(variance, unit_scale=True)

mod = HARX(vix, lags=[1,5,22], volatility=fv)
res = mod.fit(disp='off')

print(res.summary())

File "C:\Users\Larry\Anaconda3\lib\site-packages\arch\univariate\base.py", line 436, in fit
v.compute_variance(sv_volatility, resids, sigma2, backcast, var_bounds)

File "C:\Users\Larry\Anaconda3\lib\site-packages\arch\univariate\volatility.py", line 1922, in compute_variance
if self._stop - self._start != sigma2.shape[0]:

AttributeError: 'FixedVariance' object has no attribute '_stop'

---

?mod
Type: HARX
String form: HAR(constant: yes, lags: [0:1], [0:5], [0:22], no. of exog: 0, volatility: Fixed Variance (Unit Scale), distribution: Normal distribution)
File: c:\users\larry\anaconda3\lib\site-packages\arch\univariate\mean.py

GJR-GARCH: how to implement γϵ2t−1I[ϵt−1<0]?

For regular garch I've got this:
σ2t=ω+αϵ2t−1+βσ2t−1
which is coded as:
res=am.fit(update_freq=5)
σ2t=res.params['omega'] + res.params['alpha[1]']res.resid__2+res.conditional_volatility*2 *res.params['beta[1]']

however, for GJR-GARCH:
σ2t=ω+αϵ2t−1+γϵ2t−1I[ϵt−1<0]+βσ2t−1
I don't know how to code: γϵ2t−1I[ϵt−1<0]
I'm guessing:
γϵ2t−1 = res.params['gamma[1]']res.resid*2

but not sure how do to:
I[ϵt−1<0]

any help would be appreciated,
thanks

Multithreading fit ARX causing The optimizer returned code 9

Hi, I recently start to use your library for modelling time series. It works perfectly fine under single thread. But if I use ARX model and fit a lot of them in a multithreading task. Then I randomly got either

`The optimizer returned code 9. The message is:
Iteration limit exceeded
See scipy.optimize.fmin_slsqp for code meaning.

ConvergenceWarning)`

or The optimizer returned code 5 problems. So I get nan for the final results. Is it normal? Is ARX thread safe?

Can this package do ARCH LM test?

Can this package do ARCH LM test? Or we can judge whether the time sequence has ARCH effect from the result of the arch_model?

Installation error

When i try to use

 pip install git+git://github.com/bashtage/arch.git

The installation fails with the following error message:

Command "/opt/anaconda3/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-z6aplqe2-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-2x3mdls0-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-z6aplqe2-build/

I use Ubuntu 14.04 with python 3.5.0 (anaconda)

Update:
It seems like I have an issue not only with arch but also with other installtions packages. Most other packages seem to work, so I was initially assuming the problem was arch related, sorry. I will leave the issue for now open (feel free to close it) and asked a question instead.

HARX: Forecast 1day ahead "Forecasts are not available ... contains exogenous regressors."

Sample code:
am = arch.univariate.HARX(y=Y,x=X, lags=[1, 2])
res = am.fit()
res.forecast(horizon=1)

Error:

D:\Anaconda3\lib\site-packages\arch\univariate\mean.py
---> 24 res.forecast(horizon=1)
...
624 if self._x is not None:
--> 625 raise RuntimeError('Forecasts are not available when the model '
626 'contains exogenous regressors.')
627 # Check start
RuntimeError: Forecasts are not available when the model contains exogenous regressors.

According to docs:

If model contains exogenous variables (model.x is not None), then only 1-step ahead forecasts are available.

Whats' wrong?
Thx

fit function error

disp : str
Either 'final' to print optimization result or 'off' to display
nothing
but this
In [28]: res = am.fit(disp='off')
NIT FC OBJFUN GNORM
1 6 8.482064E+03 2.081112E+03
2 16 8.479741E+03 2.051243E+03
3 25 8.478820E+03 1.043445E+03
4 32 8.473472E+03 5.961454E+02
5 39 8.472942E+03 7.603645E+02
6 46 8.471053E+03 4.133681E+02
7 53 8.470969E+03 5.918941E+02
8 60 8.470657E+03 3.439708E+02
9 66 8.470560E+03 1.033851E+02
10 72 8.470536E+03 7.246581E+00
11 78 8.470535E+03 8.108463E-01

TODO: List of important steps before initial version is complete

• EGARCH
• EWMA Variance
• RiskMetrics 2006 Variance
• Examples
• Forecasting code
• Complete Documentation
• Basic Plot for Results

Valid starting values

I'm using a tarch model:

    model   = arch.arch_model(returns, p=1, o=1, q=1, power=1.0)

I'm having problems with convergence. So I thought entering the following starting value would make sense:

    starting_values = pd.Series(data={
        'mu':       0.0,
        'alpha[1]': 0.0,
        'gamma[1]': 0.0,
        'omega':    0.0,
        'beta[1]':  1.0
    })

Unfortunately, I'm getting the following warning:

Starting values do not satisfy the parameter constraints in the model. The provided starting values will be ignored.

R-squared is not correct for constant only model

The R^2 is not correct for models that only contain a constant.

Logic reversed in EGARCH recursion bounds check

In the bounds check for EGARCH recursion, when the upper bound is exceeded, the logic appears to be inverted. That is, if sigma2 exceeds DBL_MAX, we attempt to use it, rather than just using a constant.

The logic appears to be ok in the pure python recursion file, because it checks for NOT inf before using sigma2.

I would submit a PR, but I want to figure out why my current PR is failing the build and get a better idea of the build process before doing that.

Installation Failure with Conda

When I was trying to install arch 4.0 by using conda install -c https://conda.binstar.org/bashtage arch in Anaconda 4.3.1(Python 3.6 version), I got error showing:

UnsatisfiableError: The following specifications were found to be in conflict:
arch 4.0* -> numpy 1.10* -> python 2.7*
python 3.6*

Why?

extracting residual from garch

can someone tell me how to extract residuals from ARCH/GARH type of models in python in order to perform additional diagnostic tests on it such as LB(20), ARCH test, normality etc.

For example i am trying

import numpy as np
import pyflux as pf
import pandas as pd
from pandas_datareader import DataReader
from datetime import datetime
import matplotlib.pyplot as plt
%matplotlib inline

jpm = DataReader('JPM', 'yahoo', datetime(2006,1,1), datetime(2016,3,10))
returns = pd.DataFrame(np.diff(np.log(jpm['Adj Close'].values)))
returns.index = jpm.index.values[1:jpm.index.values.shape[0]]
returns.columns = ['JPM Returns']

skewt_model = pf.SEGARCH(p=1, q=1, data=returns, target='JPM Returns')
x = skewt_model.fit()
x.summary()

Add forecasting code

The forecasting paths are simple stubs and haven't been implemented.

Using statsmodels ARMAResults as a mean model

Is it possible to use ARMAResults calculated using statsmodels as a mean model so that a volatility process can be added?

I wasn't sure how this might work but I was thinking of something along the lines of this:

from arch.univariate.mean import ARCHModel
from arch.univariate import ARCH

model = ARCHModel(returns)
model.mean = arma_res  # statsmodels ARMAResults
model.volatility = ARCH()

results = model.fit(update_freq=5)

Remove Cython from requirements when using --no-binary

This isn't needed when installing without Cythonizing

passing additional parameters to optimizer

IIUC we cannot pass options to the scipy optimizer through a call to fit(). For example, say we want to change ftol or eps. Is there a reason you avoided doing this? Thanks.

simulate function in GARCH bugged?

In volatility.py, in GARCH class, function simulate, the following code:
sigma2[:max_lag] = initial_value ** inv_power
data[:max_lag] = sqrt(sigma2[:max_lag]) * sign(errors[:max_lag])
seems to be taking the square root of the initial_value twice, because inv_power == 0.5 by default.

In any case, this shouldn't be like that, because implicitly, it is assuming that the previous error was equal to 1 or -1, and then sqrt of sigma2 will be a huge number, hence the first return (residual), in data, will be on average huge. It should replace the first max_lag members in errors variable with previous returns (residuals) instead.

This becomes significant if burn=0, which is the case for several practicals applications of the simulate function.

I think this is a bug, but apologies if I misunderstood the logic.

Forecast method for ARCHModelResult class doesn't seem to work

The forecast method doesn't seem to make sense. It returns an array of the same length as my input array of returns. It should forecast subsequent volatilities perhaps for a given n-steps ahead (which may require specifying as a parameter the number of steps ahead).

Or am I not using it correct?

I'm also confused as to what the forecast could be for, as the first few numbers don't seem to match the variance of volatility as the modelresult.conditional_volatilty call.

GARCH model with exogenous regressors not functioning?

Hello,

I'm struggling to figure out how to properly use this package to fit a GARCH(1,1) model with an exogenous variable. Here's an example Jupyter notebook to illustrate what I'm trying to do.

In short, using the canonical example of daily S&P 500 returns, I'm trying to add a dummy variable to a GARCH(1,1) model to examine the effect of Mondays. If I fit a model like so, arch_model(returns, x=mondays).fit().summary(), the summary output does not include any information for the dummy variable, making me think the variable was ignored altogether.

Looking through the ARCH documentation, I found a page specifying that I may need to specify a mean model for exogenous regressors. If I explicitly specify the mean model to be HARX, like so, arch_model(returns, x=mondays, mean='HARX').fit().summary(), I receive the following exception: ValueError: x must be nobs by n, where nobs is the same as the number of elements in y.

Am I missing something simple here? I'm trying to use this package for a graduate class in econometrics, and this question is causing some serious pain. Any help would be greatly appreciated. Thanks!

coefficients of GARCH do not match R's values.

Hello, thanks for your update. I have another issue. Upon using your model, i cross checked the coefficients with R's built in function garchFit from the fGarch library; Some of the coefficients are different starting from the 3rd significant digit. example...

using intel's stock returns from "Analysis of financial time series" by Tsay,
mu omega alpha1 alpha2 alpha3
0.01256687 0.01042129 0.23288895 0.07506869 0.05199350

These are the coefficients I got by calling m1=garchFit(~1+garch(3,0),data=y,trace=F)
where y is the time series of the monthly returns.

However, using your arch function, I get these coefficients:

mu 0.012529
omega 0.010503
alpha[1] 0.220955
alpha[2] 0.077879
alpha[3] 0.051980

This may not be a big issue. I am not too sure. But, different conclusions are reached based on the R's function and based on your library. For instance, R showed that using an GARCH(3,0) model yields insignificant coefficients on the 2nd and 3rd alpha. Whereas, using your model, it claims insignificant coefficients on the alpha 1,2 and 3. Hence, which is correct? I feel the issue has more to do with the coefficient values and the way the variance was initialized. Please correct me if I am wrong.

Citation and block size estimator?

Very nice package. Two questions:

1. Do you have preferences for how to cite this work in a paper? Might be good to add to the README or the docs.
2. Do you have any (possibly experimental) python code implementing one of the automatic block size formulas for CircularBlockBootstrap?

Can this package do ARCH LM test? Or we can judge whether the time sequence has ARCH effect from the result of the arch_model?

Egarch

Hi,

How would I be able to make adjustments using eGarch? Right now, can only look at smoothing using GJR format.

Thanks.