I am trying to predict the CLTV of a customer in next 365 days (12 month) window. I encountered the same issue with lifetimes package as well (and I see there is one open bug there). The issues can be found here and here
I use 4 years data as calibration data and 1 year data as holdout data.
I wrote the below piece of code
ggf = GammaGammaFitter(penalizer_coef=0.01) # model object
ggf.fit(monetary_cal_df['frequency_cal'], monetary_cal_df['avg_monetary_value_cal']) # model fitting
# Prediction of expected amount of average profit
monetary_cal_df["expct_avg_spend"] = ggf.conditional_expected_average_profit(monetary_cal_df['frequency_cal'], monetary_cal_df['avg_monetary_value_cal'])
monetary_cal_df["cltv_12m"] = ggf.customer_lifetime_value(bgf,
monetary_cal_df['frequency_cal'],
monetary_cal_df['recency_cal'],
monetary_cal_df['T_cal'],
monetary_cal_df['avg_monetary_value_cal'],
time=12, # 12 month
freq="D", # frequency of T
discount_rate=0.01)
monetary_cal_df.sort_values("cltv_12m",ascending=False).head()
But the problem is CLTV values are high. For instance, in the below screenshot, (Don't worry about the column header name in the screenshot.Both expected purchase and CLV is for 12 months only)
![image](https://user-images.githubusercontent.com/30723825/200760815-df546b77-10d9-4081-a483-b79370eb14b1.png)
When I do a multiplication of "expected_avg_spend" and "expected_purchase", they are low by more than 100K.
Can you guide me on whether it is normal to see such a huge differences (and have you encountered cases like this in your work-setting doing this project)? How is CLTV calculated? Is it correct from my end to think that they should be somewhat closer to the result of multiplication (between expected_avg_spend and expected_purchase)
Apart from R2, RMSE, I also plotted the below graph to see whether they are okay visually and I guess it is okay but is there any red flag that I should be aware of when doing assessments like this?
![image](https://user-images.githubusercontent.com/30723825/200766285-8e869597-2484-4145-b6da-df4864cce6e3.png)
Accordingly, few related questions on the theory of CLTV
a) I see that we compute CLTV for a specific time horizon like 6 months, 12 months etc. So, when we mean Customer Lifetime Value in next 6 or 12 months or t months, am I right to interpret that this is the amount/revenue that we expect from the customer in the next 6 or 12 months or t months? The keyword lifetime
doesn't mean the total revenue from the customer during his lifetime with us (ex: A customer can continue to stay with us for 10 years or more (as it is future, we may not know). but the CLV we predict is specifically for the next 6/12/t months.). Is my understanding right?
b) Can the time
parameter in ggf.customer_lifetime_value()
accept values only in terms of months
as shown in code snippet comments. Because, I see that freq
parameter is for a different column (T). So, CLTV can only be computed based on month intervals. Am I right to understand that?
c) For GGM, the prerequisite correlation between avg_monetary_cal
and frequency_cal
is 0.28. So, is 0.28 indicative of weak correlation to proceed further? Because I see in all tutorials online, it is all very low such as 0.03, 0.07 etc. So, 0.28 is low enough to proceed further?
d) When I compute r2 between my monetary_val_cal
and expected_avg_spend
, there is 92% of variance explained. But when I do the r2 calculation for monetary_val_holdout
and expected_avg_spend
(which is what we are expected to do I guess), I get only negative values. I guess it is overfitting. Is there any pointers that you can help me on how I can help improve the performance? but the r2 is around 72% for expected future transaction count
(when I compare it with holdout_frequency). any suggestions on how can this be improved? I already set the penalizer to 0 (as it resulted in better r2 score) and subset my dataframe for GGM (to include only customers who have monetary_value_cal > 0). I filter only based on monetary value_cal > 0 because the package gave me an error that there are values = 0. So, I included the filter criteria. Basically, my BG/NBD models works well but it is GGM that is causing issue.
e) I use 4 years data as calibration data and 1 year data as holdout data. Not all customers are present in both sets. Some are present only in calibration and some are present only in holdout (because they became a customer only during holdout period). Anyway, we restrict our analysis only people who are present in both sets. Is that the right thing to do? or we should let the model predict for people who are missing in holdout set (may be they left during holdout period) but were present in calibration? Or this package should be used only for customers who are present in both calibration and holdout sets
f) Do you think it would be wise to reduce the dataset size? May be this problem doesn't require 5 years of data (may be they just add noise). Just choosing to model only based on past 2 years would be good route to take to solve this problem?
Currently, the metrics looks like below for monetary_holdout and expct_avg_spending. I drop NA because the model returned NaN
as CLV value using gamma function (for rows where the frequency_holdout is zero). So, I dropped those records
![image](https://user-images.githubusercontent.com/30723825/200786959-24c00121-5e99-4285-8945-4ff51a8ae128.png)