edwinnglabs / karpiu Goto Github PK

Hi Edwin,
Happy Holidays! Incredible work here. Thank you so much for sharing such an impressive modeling work. I have been trying to start testing the model and I keep running into issues with pandas==1.4.2. I followed your instructions, and I hope you don't mind helping me. pystan==2.19.1.1 requires python<3.8 so I install python 3.7. Does Karpiu use python 3.8 because no matter how many times that I tried, I couldn't install Karpiu

creating optimizer in OO format

Consider using the whole data frame as based and keep a few masks to extract variables on right time-range such as spend_mask, calc_mask instead of current approach which subtract sub df at first i.e.

self.df_bau = df.loc[
    (df[date_col] >= self.calc_start) & (df[date_col] <= self.calc_end),
    [date_col, self.kpi_col] + self.full_regressors
].reset_index(drop=True)

in line 110

Hence, everything will be extracted directly from df itself

Hi,
I noticed that the package uses DLT from Orbit. Are there also plans to add support for KTR (BTVC), which shows a lot of promise in the paper (https://arxiv.org/pdf/2106.03322.pdf)?

enable something like predict(percentile=[0.05, 0.95]) or something similar in the class constructor etc.

It seems the organic is off.

with orbit and KTR citation

Proposed solution: allow a scheme of decomposition such as a Dict[str, List[str]] where the key is the label of the component and the values are the list of raw elements included.

From orbit, we are calling pystan==2.19.1.1.
However, the package is archived and is using old way python setup.py install to install the package. It is no longer get called this way under the new pip install ecosystem. See: pypa/pip#8368 for details.

We need to find a way to enforce this call. Right now, it seems python=3.8.* is still ok with the old procedure.

• pytest setup
• github action

• more object-oriented
• solve multiple channels in multiple periods at a time

Using prior solver, calibrate models with a dynamic system by digesting sigma prior, coefficients deviation from previous run etc.

right now it is blocked because the df does not create the seasonal regressors automatically

Hi Edwin,
Just want thank you so much again for sharing your knowledge and your remarkable work here. I want to ask a quick question about adstock functionality and I wonder whether I am making the right step here. I want to test with an excel file dataset and I am using the function make_adstock_metric to create adstock but I kept running into issues. Do you have any issues create adstock with a different dataset (an actual database in csv)? Which function do you suggest me to create for adstock process if you don't mind me asking.
Apologies for the inconvenience. I appreciate your great insights and valuable guidance as I am learning through this great package.
Greatly appreciate your time and patience,
Sincerely,
Lan Nguyen

Use callback in scipy.optim to store metrics in each iteration

Reference here

For reusability, perhaps make a class to facilitate the translation from a structured data frame to list of lists priors;
this should also support control, events and spend regression

• instead of making Attributor with a scope on just attribution; make it a "shell" which contain all the needed components to forecast with a fixed range where all background info are fixed already e.g. trend, seasonality, static regression effect etc.
• then the attributor support attribution, marginal cost calculation etc. with quick computation with numpy or other similar tensor type of computation

Currently, we are using one-on. we may want to allow options for users as well.

Use numpy to re-construct a callable objective

• logger name displays in the log
• standardize logger creation if not provided by user

There are duplicated code within those optimization classes. We should set up the right base / abstract class.

• build based on a MMMShell class
• unit test vectorize adstock process
• build a faster logic with vectorizing adstock process
• unit test attribution condition such as true up vs. not true up
• share similar code with net returns objective function

Consider this to merge jobs with similar trigger.

With all the model updated from previous version, consider including most of the args into the unit tests such as hyper-parameters tuning.

Also need to report a coverage % in the badge somewhere.

Remove warnings from the unit test

Review/expand the documentation to cover some abstract class and design.

different init spend but same result (when the result is not bounded)

Hi Edwin,
Thank you so much for answering my questions about how to install the Karpiu. Greatly appreciate that you share with the public about your work development. I am able to figure it out and to start testing the file. However, I am running into the issue with the first step of creating data in the data_creation notebook:

I understand your busy schedule and I am sorry for reaching out to ask for help and clarification. I am truly fascinated by the work that you have built so far and I would love to be able to run it and to learn from it.
Once again, appreciate your help and your time very much.

1. derive marginal cost from cost curves df
2. derive marginal cost from function
3. use one-off approximation instead of one-on