The adam from config-i1

Reference to definition of analytics is needed

Line 30 in 91b1200

    
           As for analytics, this is a relatively new term, which implies a set of activities based on analysis, forecasting and optimisation to support informed managerial decisions. The term is broad and relies on many research areas, including forecasting, simulations, optimisation etc. In this monograph, we will focus on the forecasting aspect, occasionally discussing how to analyse the existing processes (thus touching the analytics part) and how various models could help make good practical decisions.

Extract simulation / forecasting paths from tidymodel when using adam_reg

I hope this is an appropriate question here (I just realized after posting that you might not be involved with the modeltime implementation of your adam model). I'm just starting to learn how to do time series forecasting in the tidymodel framework. Using ADAM from the smooth package, I can do the code below to save nsims (e.g. 1000) forecasts of 10 years of data for further analysis:

library(smooth)
data(BJsales)
adamETSBJ <- adam(BJsales, "AAN")
OneKForecastsOf10Years <- forecast(adamETSBJ,h=10, interval="simulated",scenarios=TRUE,nsim=1000)

plot(OneKForecastsOf10Years$scenarios[,1], ylab="Sales",ylim=range(OneKForecastsOf10Years$scenarios), xlab="Horizon",col=rgb(0.8,0.8,0.8,0.4))

Plot all the generated lines
for(i in 2:1000){
lines(OneKForecastsOf10Years$scenarios[,i],col=rgb(0.8,0.8,0.8,0.4))
}

To do this in a tidymodel framework, I do:

library(modeltime)
library(tidymodels)

BJsales.tidy <- tibble(BJsales) %>% mutate(FakeDate = seq(as.Date('2020-01-01'),by='day',length.out = 150))
model_spec <- adam_reg(ets_model = "AAN") %>%
set_engine("adam")

model_fit <- model_spec %>%
fit(BJsales ~ FakeDate, data = BJsales.tidy)

models_tbl <- modeltime_table(
model_fit
)

But I can only seem to extract the summary stats for the forecasts, not the simulation paths

TidyForecasts <- models_tbl %>%
modeltime_forecast(
h=10,
actual_data = BJsales.tidy,
interval="simulated",scenarios=TRUE,nsims=1000
)

Any suggestions on what I'm missing? Thanks!

    
           We can **compare means** of distributions of errors using a parametric statistical test. We can try F-test [@WikipediaFTest], which will tell us whether the mean performance of methods is similar or not. Unfortunately, this will not tell us how the methods compare. But t-test [@WikipediaTTest] could be used to do that instead for pairwise comparison. One could also use a regression model with dummy variables for methods, giving us parameters and their confidence intervals (based on t-statistics), telling us how the means of methods compare. However, F-test, t-test and t-statistics from regression rely on strong assumptions related to the distribution of the means of error measures [that it is symmetric, so that Central Limit Theorem works, see discussion in Section 6.2 of @SvetunkovSBA]. If we had a large sample (e.g. a thousand of series) and well-behaved distribution, we could try it, hoping that the central limit theorem would work and might get something relatively meaningful. However, on 100 observations, this still could be an issue, especially given that the distribution of error measures is typically asymmetric (the estimate of the mean might be biased, which leads to many issues). 
        
           Alternatively, we can **compare medians** of distributions of errors. They are robust to outliers, so their estimates should not be too biased in case of skewed distributions on smaller samples. To have a general understanding of performance (is everything the same or is there at least one method that performs differently), we could try the Friedman test [@WikipediaFriedmanTest], which could be considered a nonparametric alternative of the F-test. This should work in our case but won't tell us how specifically the methods compare. We could try the Wilcoxon signed-ranks test [@WikipediaWilcoxonTest], which could be considered a nonparametric counterpart of the t-test. However, it only applies to two variables, while we want to compare four.

Reference to safety stock formula needed

adam/Chapters/01-Introduction.Rmd

Line 105 in 2b9d64e

    
           It is essential to understand how decisions are made in practice and align them with the forecast horizon. In combination with the point forecasts and prediction intervals discussed above, this will give us an understanding of what to produce from the model and how. For example, in the case of safety stock calculation, it would be more reasonable to produce quantiles of the cumulative over the lead time demand than to produce point forecasts from the model.

Reference needed to the mean / median / geo.mean relation

adam/Chapters/01-Introduction.Rmd

Line 74 in a9430c9

    
           The specific type of point forecast will differ depending on the model used in construction. For example, in the case of the pure additive model, assuming some symmetric distribution (e.g. Normal one), the arithmetic mean, geometric mean and median will coincide. On the other hand, a model constructed in logarithms will assume an asymmetric distribution for the original data, leading to the following relation between the means and the median (in case of positively skewed distribution):

Add discussion of calibration of quantiles. Also add a range vs coverage plot

adam/Chapters/02-Forecasting-Evaluation.Rmd

Line 251 in 0cdd9f6

Reference CV, not Wikipedia

adam/Chapters/02-Forecasting-Evaluation.Rmd

Line 329 in 59c00c5

    
           In rolling origin evaluation, the forecasting origin is repeatedly moved forward by a fixed number of observations, and forecasts are produced from each origin [@Tashman2000]. This technique allows obtaining several forecast errors for time series, which gives a better understanding of how the models perform. This can be considered a time series analogue to cross-validation techniques [@WikipediaCrossValidation2020]. Here is a simple graphical representation, courtesy of [Nikos Kourentzes](https://kourentzes.com/forecasting/).

config-i1 / adam Goto Github PK

adam's People

Stargazers

Watchers

Forkers

adam's Issues

Reference to definition of analytics is needed

Extract simulation / forecasting paths from tidymodel when using adam_reg

default ets-model def. of auto.adam() and its interpretation in a third party lib.

Fix tables in section 4.4

Refernce to a book, not Wikipedia. Leave Wikipedia as additional resource

Reference to safety stock formula needed

Reference needed to the mean / median / geo.mean relation

Add discussion of calibration of quantiles. Also add a range vs coverage plot

Reference CV, not Wikipedia

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

	We can compare means of distributions of errors using a parametric statistical test. We can try F-test [@WikipediaFTest], which will tell us whether the mean performance of methods is similar or not. Unfortunately, this will not tell us how the methods compare. But t-test [@WikipediaTTest] could be used to do that instead for pairwise comparison. One could also use a regression model with dummy variables for methods, giving us parameters and their confidence intervals (based on t-statistics), telling us how the means of methods compare. However, F-test, t-test and t-statistics from regression rely on strong assumptions related to the distribution of the means of error measures [that it is symmetric, so that Central Limit Theorem works, see discussion in Section 6.2 of @SvetunkovSBA]. If we had a large sample (e.g. a thousand of series) and well-behaved distribution, we could try it, hoping that the central limit theorem would work and might get something relatively meaningful. However, on 100 observations, this still could be an issue, especially given that the distribution of error measures is typically asymmetric (the estimate of the mean might be biased, which leads to many issues).

	Alternatively, we can compare medians of distributions of errors. They are robust to outliers, so their estimates should not be too biased in case of skewed distributions on smaller samples. To have a general understanding of performance (is everything the same or is there at least one method that performs differently), we could try the Friedman test [@WikipediaFriedmanTest], which could be considered a nonparametric alternative of the F-test. This should work in our case but won't tell us how specifically the methods compare. We could try the Wilcoxon signed-ranks test [@WikipediaWilcoxonTest], which could be considered a nonparametric counterpart of the t-test. However, it only applies to two variables, while we want to compare four.