Comments (8)
Thanks for your interest in the project.
So the initial scores serve as to initialize the boosting model, i.e., the model uses the initial scores as starting values where to boost from for the first itertation. Since we model all parameters of a distribution, LightGBMLSS currently calculates the unconditional parameter values from the data using the LBFGS
model, based on the NLL. Hence, there is no need to multiply them with a value since they already reflect a reasonable starting value.
So for a Poisson distributed variable, the rate
parameter is initialized with the unconditional parameter values and for the remaining iterations, the rate
parameters changes as a function of x
.
If you want to model a specific insurance type of data, it is best to transform the response prior to model, i.e., y/exposure
as you suggested.
The weights
are used to weight the gradient/hessian
which are then used to update the parameter estimates.
Does that answer your question?
from lightgbmlss.
I think the desired offset would take the place of the np.ones it uses now but you'd have to know how it relates to the distribution parameters through the distributions mean function, right? So for Gamma, you could multiply the desired init_score (offset) by the concentration starting value but you have to invert it for the rate. For Gaussian, you'd multiply it by the starting mean but not the sigma, etc. That would require having the ability to determine the correct operation to perform for each parameter for each distribution. Does that make sense?
from lightgbmlss.
In a traditional context, one can either use weights + transformed response or no weights and an offset. In a non-LSS approach where predictions are in terms of the response value rather than distribution parameters, init_score serves as the place bring in an offset. So, for example, I can predict loss severity by using losses as response and log(exposure) as init_score. My prediction will then be severity / exposure. Alternatively, I can use losses/exposure as my response and exposure as weights. That should produce similar models (though not precisely similar in terms of the path training might take).
As I understand you answer, LSS needs to use init_score for distribution parameters, not means, and so I'd have to resort to the losses/exposure + exposure weights approach. The thought in my initial response was basically thinking that if I was really committed to using my domain offset, I'd just need to relate it to parameter init_scores in the proper way. For example, in Gamma the mean is alpha/beta. So if the init_scores were alpha*log(exposure) and beta/log(exposure), then I'd get the parameter predictions for Gammas that were already in terms of a loss/exposure distribution rather than a raw loss distribution without having to transform my response.
Transforming the response and using weights isn't a big deal, but there are cases were you really would like to use an offset in a regression, like if I want to build a model off the prior model's predictions (even if they are distributional parameters) or if I want to start from (to use insurance again) my current rating plan's relativities.
So the thought behind the ticket is really how one might be able to take any init_score you might have for your problem space that you planned to use in a traditional LGBM context (just like you have a response variable that itself doesn't start out as a set of distribution parameters) and have it work in the same manner, transparently, with the LSS version of the model.
from lightgbmlss.
@neverfox Thanks for the detailed explanation! I now better understand your problem/use-case.
I need to see how to incorporate an offset without transforming the response. Not sure if this is possible via init_score
since it is currently being used as the starting-values for the distributional parameters.
from lightgbmlss.
@neverfox I looked into it, however, I don't see how I can possibly change the start values to a transformation of the response. So for now, the onl way to do it would be to transform the response accordingly.
from lightgbmlss.
Can I close this?
from lightgbmlss.
You can. I am still thinking the issue through but it might take me some time.
from lightgbmlss.
Feel free to re-open if necessary. Thanks.
from lightgbmlss.
Related Issues (20)
- General Discussion HOT 3
- Userdefine CV Class?
- Lower python version support
- error when using weights in lgb.Dataset HOT 3
- Silent freezes HOT 3
- Zero-and-One-Adjusted Beta HOT 1
- Multi-parameter optimization with custom loss function for probabilistic forecasting HOT 6
- kwargs from the lightgbm predict are not available with the LightGBMLSS.predict HOT 5
- lightgbm v4.0.0? HOT 3
- Multi-task learning and ONNX support
- ETA for code HOT 2
- IndexError: LightGBMLSS.train(valid_sets=[dataset_val]) calls set_valid_margin, which seems to expect both train+val HOT 7
- Prediction of quantiles for parametric distributions HOT 5
- SHAP interpretations from zero-adjusted gamma model HOT 1
- Out of sample prediction and overall questions HOT 5
- Binary Classification HOT 6
- [Feature request] latest lightgbm support HOT 2
- Monotonicity of expectiles HOT 2
- Can't install on Ubuntu 20.04 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lightgbmlss.