statmixedml / lightgbmlss Goto Github PK
View Code? Open in Web Editor NEWAn extension of LightGBM to probabilistic modelling
Home Page: https://statmixedml.github.io/LightGBMLSS/
License: Apache License 2.0
An extension of LightGBM to probabilistic modelling
Home Page: https://statmixedml.github.io/LightGBMLSS/
License: Apache License 2.0
Very cool to see the addition of the zero-adjusted/inflated distributions. Would it be practically possible to have a zero-and-one-adjusted/inflated Beta, i.e. ZOIB, a Beta that allows the response to be [0, 1]? This goes towards the idea of using it with binary classification responses and getting, instead of the usual, natural p parameter of a Bernoulli, the distribution of p itself as a Beta distribution. This would be interesting to ask probabilistic questions about the predicted probability, I think. I could probably hack it by modifying the data with a small epsilon, but it ZABeta opens the door to doing that in a principled way. I think the idea would mean:
Referencing to this issue StatMixedML/CatBoostLSS#5 (comment)
@jrzaurin Let`s continue the discussion on this repo.
Hi, I'm trying your package since you added normflows, could you please upgrade the gbm to v4.0.0?
I think they removed the fobj
param from the train() function, and you'd have specify in params
Thanks!
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
lightgbmlss 0.3.0 requires lightgbm~=3.3.5, but you have lightgbm 4.0.0 which is incompatible.
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ in :2 │
│ │
│ 1 # Train Model with optimized hyperparameters │
│ ❱ 2 lss_model = lgblss.train(lss_params, │
│ 3 │ │ │ │ │ │ lgbtrain, │
│ 4 │ │ │ │ │ │ num_boost_round=lss_params['num_boost_round'], │
│ 5 │ │ │ ) │
│ │
│ /home/algo/code/cqg/bts/spark/ml/.venv/lib/python3.9/site-packages/lightgbmlss/model.py:178 in │
│ train │
│ │
│ 175 │ │ if valid_sets is not None: │
│ 176 │ │ │ valid_sets = self.set_valid_margin(valid_sets, self.start_values) │
│ 177 │ │ │
│ ❱ 178 │ │ self.booster = lgb.train(params, │
│ 179 │ │ │ │ │ │ │ │ train_set, │
│ 180 │ │ │ │ │ │ │ │ num_boost_round=num_boost_round, │
│ 181 │ │ │ │ │ │ │ │ fobj=self.dist.objective_fn, │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: train() got an unexpected keyword argument 'fobj'
Cool piece of software, could it be used for estimating probabilistic uncertainty for binary classification tasks?
Thanks for the great work!
I have two questions:
Can we perform multi-task learning in a single training where one task is classification and the target variables are categorical (classes) and the other task is regression where the target values are continuous?
Does LightGBMLSS models support ONNX conversions?
The latest version of lightgbm (v3.3.4) removes fobj from the train argument of the trainingAPI and does not seem to work.
Is there any plan to support the latest version of lightgbm?
I call LightGBMLSS.train like this
early_stop = 20
dataset_params = {"feature_pre_filter": False}
dataset_train = lightgbm.Dataset(
train_X,
train_y["value"].ravel(),
free_raw_data=False,
params=dataset_params,
)
dataset_val = lightgbm.Dataset(
val_X,
val_y["value"].ravel(),
free_raw_data=False,
params=dataset_params,
)
distribution = lightgbmlss.model.LightGBMLSS(
lightgbmlss.distributions.Gaussian.Gaussian(
stabilization="None",
response_fn="exp",
loss_fn="nll",
)
)
distribution.train(
params=my_hyperparams,
num_boost_round=9999999, # early_stopping should kick in before this
train_set=dataset_train,
valid_sets=[dataset_val],
callbacks=[
lightgbm.early_stopping(early_stop, verbose=False),
lightgbm.log_evaluation(-1),
],
)
It results in this error
File ~/miniconda3/envs/package_name/lib/python3.9/site-packages/lightgbmlss/model.py:575, in LightGBMLSS.set_valid_margin(self, valid_sets, start_values)
572 init_score_val1 = (np.ones(shape=(valid_sets1.get_label().shape[0], 1))) * start_values
573 valid_sets1.set_init_score(init_score_val1.ravel(order="F"))
--> 575 valid_sets2 = valid_sets[1]
576 init_score_val2 = (np.ones(shape=(valid_sets2.get_label().shape[0], 1))) * start_values
577 valid_sets2.set_init_score(init_score_val2.ravel(order="F"))
IndexError: list index out of range
https://github.com/StatMixedML/LightGBMLSS/blob/master/lightgbmlss/model.py#L559-L579
It seems set_valid_margin
expects both the training set and the validation set as parameters, but later when lgb.train
is called, the same list is passed in as valid_sets
, in addition to the training set via train_set
as the first parameter:
https://github.com/StatMixedML/LightGBMLSS/blob/master/lightgbmlss/model.py#L171-L177
Have I misunderstood when I think this will cause the training set to be used as a validation set for early stopping (which may not be desirable) ?
Running the housing data example (with the fix from #15, so that it runs at all), the process will silently die in the middle of hyperparameter optimization. By that I mean that the CPU usage drops to nothing and the process never moves on. No errors are reported and the step is still considered running by Jupyter. There doesn't appear to be any rhyme or reason as to which trial it will freeze on (sometimes right away, sometimes after 30+ rounds). It cannot be killed without force killing. The behavior occurs in a REPL with the same code, so I don't think it's Jupyter.
If I skip HP optimization and just train the model with empty (default) parameters and 100 rounds, it trains but the same silent freeze happens on any of the calls to predict
in the example. That might be a clue that the issue is during prediction.
Interestingly, I have not seen this behavior with the XGBoost version of this library on the same example. I can run the whole thing to completion.
Not sure if this is related but before freezing I do get the occasional warnings like:
RuntimeWarning: invalid value encountered in subtract
RuntimeWarning: overflow encountered in cast
OS: MacOS 12.6
Python: 3.9.16
For parametric distributions, I understand the model learns the mapping from data to parameters, so the model predicts the parameters for each observation.
When quantiles are predicted, the model first samples from the parametric distribution with those parameters, and then constructs quantiles from those drawn samples:
https://github.com/StatMixedML/LightGBMLSS/blob/master/lightgbmlss/distributions/distribution_utils.py#L415-L421
Is it infeasible or a reason to not instead use the theoretical quantiles for those fitted parameters directly, without sampling?
Essentially using torch_dist.icdf
:
pred_params = distribution.predict(
test_X,
pred_type="parameters",
)
def instantiate_torch_distribution(distribution, pred_params: pd.DataFrame):
"""Prepare a torch distribution object (tensor) for the observations in the test set
distribution
This is an object of type lightgbmlss.model.LightGBMLSS, for which a mapping from
features to distributional parameters was learned.
pred_params:
A dataframe with one column per predicted distributional param, and one row
for each test observation that had that param predicted.
"""
# A list of named parameters, e.g. ['loc', 'scale'] that the torch distribution
# expects for instantiation.
dist_arg_names = distribution.dist.distribution_arg_names
preds_transformed = []
for dist_arg_name in dist_arg_names:
# predict() above already returns the values on the transformed scale
# (e.g. the model predicts log_variance and then transforms it via exp(log_variance)
# and this is the result from predict() above.
pred_tensor = torch.tensor(
pred_params[dist_arg_name].values,
dtype=torch.float64,
).reshape(-1, 1)
preds_transformed.append(pred_tensor)
dist_kwargs = dict(zip(dist_arg_names, preds_transformed))
torch_dist_fit = distribution.dist.distribution(**dist_kwargs)
return torch_dist_fit
def get_quantiles(q, torch_dist_fit, test_y):
"""get the qth quantiles of target values test_y
q in [0.0, 1.0]
"""
qt = torch.tensor([q]*test_y.size).reshape(-1, 1)
quants=torch_dist_fit.icdf(qt)
return quants
Separately from the above, it also looks like the samples are drawn when predict is called, even if the samples are not used in any of the returns, specifically pred_type == "parameters"
and pred_type == "expectiles"
:
https://github.com/StatMixedML/LightGBMLSS/blob/master/lightgbmlss/distributions/distribution_utils.py#L402-L410
I can see how it makes the code cleaner if not littered with conditionals for whether to sample, and perhaps the extra computations from sampling without returning the samples is insignificant.
I'm getting an error when I try to use weights in lgb datasets:
from lightgbmlss.model import *
from lightgbmlss.distributions.Expectile import *
from lightgbmlss.datasets.data_loader import load_simulated_gaussian_data
import plotnine
from plotnine import *
plotnine.options.figure_size = (20, 10)
train, test = load_simulated_gaussian_data()
X_train, y_train = train.filter(regex="x"), train["y"].values
X_test, y_test = test.filter(regex="x"), test["y"].values
weight2 = train["scale"].values
dtrain = lgb.Dataset(X_train, label=y_train,weight=weight2)
lgblss = LightGBMLSS(
Expectile(stabilization="None", # Options are "None", "MAD", "L2".
expectiles = [0.05, 0.95], # List of expectiles to be estimated, in increasing order.
penalize_crossing = True # Whether to include a penalty term to discourage crossing of expectiles.
)
)
param_dict = {
"eta": ["float", {"low": 1e-5, "high": 1, "log": True}],
"max_depth": ["int", {"low": 1, "high": 10, "log": False}],
"num_leaves": ["int", {"low": 255, "high": 255, "log": False}], # set to constant for this example
"min_data_in_leaf": ["int", {"low": 20, "high": 20, "log": False}], # set to constant for this example
"min_gain_to_split": ["float", {"low": 1e-8, "high": 40, "log": False}],
"min_sum_hessian_in_leaf": ["float", {"low": 1e-8, "high": 500, "log": True}],
"subsample": ["float", {"low": 0.2, "high": 1.0, "log": False}],
"feature_fraction": ["float", {"low": 0.2, "high": 1.0, "log": False}],
"boosting": ["categorical", ["gbdt"]],
}
np.random.seed(123)
opt_param = lgblss.hyper_opt(param_dict,
dtrain,
num_boost_round=100, # Number of boosting iterations.
nfold=5, # Number of cv-folds.
early_stopping_rounds=20, # Number of early-stopping rounds
max_minutes=10, # Time budget in minutes, i.e., stop study after the given number of minutes.
n_trials=None, # The number of trials. If this argument is set to None, there is no limitation on the number of trials.
silence=False, # Controls the verbosity of the trail, i.e., user can silence the outputs of the trail.
seed=123, # Seed used to generate cv-folds.
hp_seed=None # Seed for random number generator used in the Bayesian hyperparameter search.
)
I get the following error:
[I 2023-06-06 22:28:24,167] A new study created in memory with name: LightGBMLSS Hyper-Parameter Optimization
/home/pschaefer/.local/share/r-miniconda/envs/AEC_Model/lib/python3.11/site-packages/optuna/progress_bar.py:56: ExperimentalWarning: Progress bar is experimental (supported from v1.2.0). The interface can change in the future.
0%| | 00:00/10:00
[W 2023-06-06 22:28:24,552] Trial 0 failed with parameters: {'eta': 0.0007593032665095024, 'max_depth': 3, 'num_leaves': 255, 'min_data_in_leaf': 20, 'min_gain_to_split': 17.66639853363941, 'min_sum_hessian_in_leaf': 3.0862064424185954e-05, 'subsample': 0.3026741167578493, 'feature_fraction': 0.3579892008277689, 'boosting': 'gbdt'} because of the following error: ValueError('The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()').
Traceback (most recent call last):
File "/home/pschaefer/.local/share/r-miniconda/envs/AEC_Model/lib/python3.11/site-packages/optuna/study/_optimize.py", line 200, in _run_trial
value_or_values = func(trial)
^^^^^^^^^^^
File "/home/pschaefer/.local/share/r-miniconda/envs/AEC_Model/lib/python3.11/site-packages/lightgbmlss/model.py", line 374, in objective
lgblss_param_tuning = self.cv(hyper_params,
^^^^^^^^^^^^^^^^^^^^^
File "/home/pschaefer/.local/share/r-miniconda/envs/AEC_Model/lib/python3.11/site-packages/lightgbmlss/model.py", line 255, in cv
self.bstLSS_cv = lgb.cv(params,
^^^^^^^^^^^^^^
File "/home/pschaefer/.local/share/r-miniconda/envs/AEC_Model/lib/python3.11/site-packages/lightgbm/engine.py", line 640, in cv
cvfolds.update(fobj=fobj)
File "/home/pschaefer/.local/share/r-miniconda/envs/AEC_Model/lib/python3.11/site-packages/lightgbm/engine.py", line 353, in handler_function
ret.append(getattr(booster, name)(*args, **kwargs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pschaefer/.local/share/r-miniconda/envs/AEC_Model/lib/python3.11/site-packages/lightgbm/basic.py", line 3029, in update
grad, hess = fobj(self.__inner_predict(0), self.train_set)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pschaefer/.local/share/r-miniconda/envs/AEC_Model/lib/python3.11/site-packages/lightgbmlss/distributions/distribution_utils.py", line 88, in objective_fn
if data.get_weight() == None:
^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
[W 2023-06-06 22:28:24,553] Trial 0 failed with value None.
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/pschaefer/.local/share/r-miniconda/envs/AEC_Model/lib/python3.11/site-packages/lightgbmlss/model.py", line 404, in hyper_opt
study.optimize(objective, n_trials=n_trials, timeout=60 * max_minutes, show_progress_bar=True)
File "/home/pschaefer/.local/share/r-miniconda/envs/AEC_Model/lib/python3.11/site-packages/optuna/study/study.py", line 425, in optimize
_optimize(
File "/home/pschaefer/.local/share/r-miniconda/envs/AEC_Model/lib/python3.11/site-packages/optuna/study/_optimize.py", line 66, in _optimize
_optimize_sequential(
File "/home/pschaefer/.local/share/r-miniconda/envs/AEC_Model/lib/python3.11/site-packages/optuna/study/_optimize.py", line 163, in _optimize_sequential
frozen_trial = _run_trial(study, func, catch)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pschaefer/.local/share/r-miniconda/envs/AEC_Model/lib/python3.11/site-packages/optuna/study/_optimize.py", line 251, in _run_trial
raise func_err
File "/home/pschaefer/.local/share/r-miniconda/envs/AEC_Model/lib/python3.11/site-packages/optuna/study/_optimize.py", line 200, in _run_trial
value_or_values = func(trial)
^^^^^^^^^^^
File "/home/pschaefer/.local/share/r-miniconda/envs/AEC_Model/lib/python3.11/site-packages/lightgbmlss/model.py", line 374, in objective
lgblss_param_tuning = self.cv(hyper_params,
^^^^^^^^^^^^^^^^^^^^^
File "/home/pschaefer/.local/share/r-miniconda/envs/AEC_Model/lib/python3.11/site-packages/lightgbmlss/model.py", line 255, in cv
self.bstLSS_cv = lgb.cv(params,
^^^^^^^^^^^^^^
File "/home/pschaefer/.local/share/r-miniconda/envs/AEC_Model/lib/python3.11/site-packages/lightgbm/engine.py", line 640, in cv
cvfolds.update(fobj=fobj)
File "/home/pschaefer/.local/share/r-miniconda/envs/AEC_Model/lib/python3.11/site-packages/lightgbm/engine.py", line 353, in handler_function
ret.append(getattr(booster, name)(*args, **kwargs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pschaefer/.local/share/r-miniconda/envs/AEC_Model/lib/python3.11/site-packages/lightgbm/basic.py", line 3029, in update
grad, hess = fobj(self.__inner_predict(0), self.train_set)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pschaefer/.local/share/r-miniconda/envs/AEC_Model/lib/python3.11/site-packages/lightgbmlss/distributions/distribution_utils.py", line 88, in objective_fn
if data.get_weight() == None:
^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
I'm trying to install this package on a fresh virtual environment on an Ubuntu 20.04 server, but get the following error:
Building wheels for collected packages: lightgbmlss
Building wheel for lightgbmlss (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/pschaefer/environments/my_env/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-9665r5_t/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-9665r5_t/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-zv9atfsj
cwd: /tmp/pip-req-build-9665r5_t/
Complete output (6 lines):
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
ERROR: Failed building wheel for lightgbmlss
Running setup.py clean for lightgbmlss
Failed to build lightgbmlss
ERROR: numba 0.56.4 has requirement numpy<1.24,>=1.18, but you'll have numpy 1.24.1 which is incompatible.
Collecting git+https://github.com/StatMixedML/LightGBMLSS.git
Cloning https://github.com/StatMixedML/LightGBMLSS.git to /tmp/pip-req-build-9665r5_t
Running command git clone -q https://github.com/StatMixedML/LightGBMLSS.git /tmp/pip-req-build-9665r5_t
Collecting ipywidgets
Using cached ipywidgets-8.0.4-py3-none-any.whl (137 kB)
Collecting lightgbm~=3.3.1
Using cached lightgbm-3.3.4-py3-none-manylinux1_x86_64.whl (2.0 MB)
Collecting matplotlib
Using cached matplotlib-3.6.3-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (9.4 MB)
Collecting numpy~=1.16
Using cached numpy-1.24.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Collecting optuna~=2.9.1
Using cached optuna-2.9.1-py3-none-any.whl (302 kB)
Collecting pandas~=1.1
Using cached pandas-1.5.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.2 MB)
Collecting plotnine~=0.8.0
Using cached plotnine-0.8.0-py3-none-any.whl (4.7 MB)
Collecting properscoring~=0.1
Using cached properscoring-0.1-py2.py3-none-any.whl (23 kB)
Collecting scikit-learn>=0.24.2
Using cached scikit_learn-1.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (9.7 MB)
Collecting scipy
Using cached scipy-1.10.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Processing /home/pschaefer/.cache/pip/wheels/3d/c9/06/734ed80d6d61fad331974bf62017b4ea6b33488082b9f5e67e/shap-0.39.0-cp38-cp38-linux_x86_64.whl
Collecting torch~=1.13.0
Using cached torch-1.13.1-cp38-cp38-manylinux1_x86_64.whl (887.4 MB)
Collecting tqdm
Using cached tqdm-4.64.1-py2.py3-none-any.whl (78 kB)
Collecting ipykernel>=4.5.1
Using cached ipykernel-6.20.1-py3-none-any.whl (149 kB)
Collecting jupyterlab-widgets~=3.0
Using cached jupyterlab_widgets-3.0.5-py3-none-any.whl (384 kB)
Collecting traitlets>=4.3.1
Using cached traitlets-5.8.1-py3-none-any.whl (116 kB)
Collecting ipython>=6.1.0
Using cached ipython-8.8.0-py3-none-any.whl (775 kB)
Collecting widgetsnbextension~=4.0
Using cached widgetsnbextension-4.0.5-py3-none-any.whl (2.0 MB)
Collecting wheel
Downloading wheel-0.38.4-py3-none-any.whl (36 kB)
Collecting pillow>=6.2.0
Using cached Pillow-9.4.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.3 MB)
Collecting kiwisolver>=1.0.1
Using cached kiwisolver-1.4.4-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.2 MB)
Collecting fonttools>=4.22.0
Using cached fonttools-4.38.0-py3-none-any.whl (965 kB)
Collecting pyparsing>=2.2.1
Using cached pyparsing-3.0.9-py3-none-any.whl (98 kB)
Collecting python-dateutil>=2.7
Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting cycler>=0.10
Using cached cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting packaging>=20.0
Using cached packaging-23.0-py3-none-any.whl (42 kB)
Collecting contourpy>=1.0.1
Using cached contourpy-1.0.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (295 kB)
Collecting alembic
Using cached alembic-1.9.1-py3-none-any.whl (210 kB)
Collecting sqlalchemy>=1.1.0
Using cached SQLAlchemy-1.4.46-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting PyYAML
Using cached PyYAML-6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (701 kB)
Collecting colorlog
Using cached colorlog-6.7.0-py2.py3-none-any.whl (11 kB)
Collecting cmaes>=0.8.2
Using cached cmaes-0.9.1-py3-none-any.whl (21 kB)
Collecting cliff
Using cached cliff-4.1.0-py3-none-any.whl (81 kB)
Collecting pytz>=2020.1
Using cached pytz-2022.7-py2.py3-none-any.whl (499 kB)
Collecting mizani>=0.7.3
Using cached mizani-0.8.1-py3-none-any.whl (64 kB)
Collecting patsy>=0.5.1
Using cached patsy-0.5.3-py2.py3-none-any.whl (233 kB)
Collecting descartes>=1.1.0
Using cached descartes-1.1.0-py3-none-any.whl (5.8 kB)
Collecting statsmodels>=0.12.1
Using cached statsmodels-0.13.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (9.9 MB)
Collecting joblib>=1.1.1
Using cached joblib-1.2.0-py3-none-any.whl (297 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting slicer==0.0.7
Using cached slicer-0.0.7-py3-none-any.whl (14 kB)
Collecting numba
Using cached numba-0.56.4-cp38-cp38-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (3.5 MB)
Collecting cloudpickle
Using cached cloudpickle-2.2.0-py3-none-any.whl (25 kB)
Collecting typing-extensions
Using cached typing_extensions-4.4.0-py3-none-any.whl (26 kB)
Collecting nvidia-cudnn-cu11==8.5.0.96; platform_system == "Linux"
Using cached nvidia_cudnn_cu11-8.5.0.96-2-py3-none-manylinux1_x86_64.whl (557.1 MB)
Collecting nvidia-cublas-cu11==11.10.3.66; platform_system == "Linux"
Using cached nvidia_cublas_cu11-11.10.3.66-py3-none-manylinux1_x86_64.whl (317.1 MB)
Collecting nvidia-cuda-nvrtc-cu11==11.7.99; platform_system == "Linux"
Using cached nvidia_cuda_nvrtc_cu11-11.7.99-2-py3-none-manylinux1_x86_64.whl (21.0 MB)
Collecting nvidia-cuda-runtime-cu11==11.7.99; platform_system == "Linux"
Using cached nvidia_cuda_runtime_cu11-11.7.99-py3-none-manylinux1_x86_64.whl (849 kB)
Collecting debugpy>=1.0
Using cached debugpy-1.6.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Collecting psutil
Using cached psutil-5.9.4-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (280 kB)
Collecting comm>=0.1.1
Using cached comm-0.1.2-py3-none-any.whl (6.5 kB)
Collecting matplotlib-inline>=0.1
Using cached matplotlib_inline-0.1.6-py3-none-any.whl (9.4 kB)
Collecting tornado>=6.1
Using cached tornado-6.2-cp37-abi3-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (423 kB)
Collecting jupyter-client>=6.1.12
Using cached jupyter_client-7.4.8-py3-none-any.whl (133 kB)
Collecting nest-asyncio
Using cached nest_asyncio-1.5.6-py3-none-any.whl (5.2 kB)
Collecting pyzmq>=17
Using cached pyzmq-25.0.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.1 MB)
Collecting prompt-toolkit<3.1.0,>=3.0.11
Using cached prompt_toolkit-3.0.36-py3-none-any.whl (386 kB)
Collecting stack-data
Using cached stack_data-0.6.2-py3-none-any.whl (24 kB)
Collecting pexpect>4.3; sys_platform != "win32"
Using cached pexpect-4.8.0-py2.py3-none-any.whl (59 kB)
Collecting backcall
Using cached backcall-0.2.0-py2.py3-none-any.whl (11 kB)
Collecting pygments>=2.4.0
Using cached Pygments-2.14.0-py3-none-any.whl (1.1 MB)
Collecting jedi>=0.16
Using cached jedi-0.18.2-py2.py3-none-any.whl (1.6 MB)
Collecting decorator
Using cached decorator-5.1.1-py3-none-any.whl (9.1 kB)
Collecting pickleshare
Using cached pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB)
Collecting six>=1.5
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting Mako
Using cached Mako-1.2.4-py3-none-any.whl (78 kB)
Collecting importlib-resources; python_version < "3.9"
Using cached importlib_resources-5.10.2-py3-none-any.whl (34 kB)
Collecting importlib-metadata; python_version < "3.9"
Using cached importlib_metadata-6.0.0-py3-none-any.whl (21 kB)
Collecting greenlet!=0.4.17; python_version >= "3" and (platform_machine == "aarch64" or (platform_machine == "ppc64le" or (platform_machine == "x86_64" or (platform_machine == "amd64" or (platform_machine == "AMD64" or (platform_machine == "win32" or platform_machine == "WIN32"))))))
Using cached greenlet-2.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (544 kB)
Collecting PrettyTable>=0.7.2
Using cached prettytable-3.6.0-py3-none-any.whl (27 kB)
Collecting stevedore>=2.0.1
Using cached stevedore-4.1.1-py3-none-any.whl (50 kB)
Collecting cmd2>=1.0.0
Using cached cmd2-2.4.2-py3-none-any.whl (147 kB)
Collecting autopage>=0.4.0
Using cached autopage-0.5.1-py3-none-any.whl (29 kB)
Collecting backports.zoneinfo; python_version < "3.9"
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Collecting palettable
Using cached palettable-3.3.0-py2.py3-none-any.whl (111 kB)
Collecting llvmlite<0.40,>=0.39.0dev0
Using cached llvmlite-0.39.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.6 MB)
Requirement already satisfied: setuptools in ./my_env/lib/python3.8/site-packages (from numba->shap~=0.39.0->lightgbmlss==0.1.0) (44.0.0)
Collecting entrypoints
Using cached entrypoints-0.4-py3-none-any.whl (5.3 kB)
Collecting jupyter-core>=4.9.2
Using cached jupyter_core-5.1.3-py3-none-any.whl (93 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting executing>=1.2.0
Using cached executing-1.2.0-py2.py3-none-any.whl (24 kB)
Collecting pure-eval
Using cached pure_eval-0.2.2-py3-none-any.whl (11 kB)
Collecting asttokens>=2.1.0
Using cached asttokens-2.2.1-py2.py3-none-any.whl (26 kB)
Collecting ptyprocess>=0.5
Using cached ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB)
Collecting parso<0.9.0,>=0.8.0
Using cached parso-0.8.3-py2.py3-none-any.whl (100 kB)
Collecting MarkupSafe>=0.9.2
Using cached MarkupSafe-2.1.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Collecting zipp>=3.1.0; python_version < "3.10"
Using cached zipp-3.11.0-py3-none-any.whl (6.6 kB)
Collecting pbr!=2.1.0,>=2.0.0
Using cached pbr-5.11.1-py2.py3-none-any.whl (112 kB)
Processing /home/pschaefer/.cache/pip/wheels/7f/1a/65/84ff8c386bec21fca6d220ea1f5498a0367883a78dd5ba6122/pyperclip-1.8.2-py3-none-any.whl
Collecting attrs>=16.3.0
Using cached attrs-22.2.0-py3-none-any.whl (60 kB)
Collecting platformdirs>=2.5
Using cached platformdirs-2.6.2-py3-none-any.whl (14 kB)
Building wheels for collected packages: lightgbmlss
Building wheel for lightgbmlss (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/pschaefer/environments/my_env/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-9665r5_t/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-9665r5_t/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-zv9atfsj
cwd: /tmp/pip-req-build-9665r5_t/
Complete output (6 lines):
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
ERROR: Failed building wheel for lightgbmlss
Running setup.py clean for lightgbmlss
Failed to build lightgbmlss
ERROR: numba 0.56.4 has requirement numpy<1.24,>=1.18, but you'll have numpy 1.24.1 which is incompatible.
Installing collected packages: debugpy, packaging, psutil, traitlets, comm, matplotlib-inline, tornado, entrypoints, platformdirs, jupyter-core, six, python-dateutil, pyzmq, nest-asyncio, jupyter-client, wcwidth, prompt-toolkit, executing, pure-eval, asttokens, stack-data, ptyprocess, pexpect, backcall, pygments, parso, jedi, decorator, pickleshare, ipython, ipykernel, jupyterlab-widgets, widgetsnbextension, ipywidgets, numpy, scipy, wheel, joblib, threadpoolctl, scikit-learn, lightgbm, pillow, kiwisolver, fonttools, pyparsing, cycler, contourpy, matplotlib, MarkupSafe, Mako, greenlet, sqlalchemy, zipp, importlib-resources, importlib-metadata, alembic, PyYAML, colorlog, cmaes, tqdm, PrettyTable, pbr, stevedore, pyperclip, attrs, cmd2, autopage, cliff, optuna, pytz, pandas, backports.zoneinfo, palettable, mizani, patsy, descartes, statsmodels, plotnine, properscoring, slicer, llvmlite, numba, cloudpickle, shap, typing-extensions, nvidia-cublas-cu11, nvidia-cudnn-cu11, nvidia-cuda-nvrtc-cu11, nvidia-cuda-runtime-cu11, torch, lightgbmlss
Running setup.py install for lightgbmlss ... done
Successfully installed Mako-1.2.4 MarkupSafe-2.1.1 PrettyTable-3.6.0 PyYAML-6.0 alembic-1.9.1 asttokens-2.2.1 attrs-22.2.0 autopage-0.5.1 backcall-0.2.0 backports.zoneinfo-0.2.1 cliff-4.1.0 cloudpickle-2.2.0 cmaes-0.9.1 cmd2-2.4.2 colorlog-6.7.0 comm-0.1.2 contourpy-1.0.6 cycler-0.11.0 debugpy-1.6.5 decorator-5.1.1 descartes-1.1.0 entrypoints-0.4 executing-1.2.0 fonttools-4.38.0 greenlet-2.0.1 importlib-metadata-6.0.0 importlib-resources-5.10.2 ipykernel-6.20.1 ipython-8.8.0 ipywidgets-8.0.4 jedi-0.18.2 joblib-1.2.0 jupyter-client-7.4.8 jupyter-core-5.1.3 jupyterlab-widgets-3.0.5 kiwisolver-1.4.4 lightgbm-3.3.4 lightgbmlss-0.1.0 llvmlite-0.39.1 matplotlib-3.6.3 matplotlib-inline-0.1.6 mizani-0.8.1 nest-asyncio-1.5.6 numba-0.56.4 numpy-1.24.1 nvidia-cublas-cu11-11.10.3.66 nvidia-cuda-nvrtc-cu11-11.7.99 nvidia-cuda-runtime-cu11-11.7.99 nvidia-cudnn-cu11-8.5.0.96 optuna-2.9.1 packaging-23.0 palettable-3.3.0 pandas-1.5.2 parso-0.8.3 patsy-0.5.3 pbr-5.11.1 pexpect-4.8.0 pickleshare-0.7.5 pillow-9.4.0 platformdirs-2.6.2 plotnine-0.8.0 prompt-toolkit-3.0.36 properscoring-0.1 psutil-5.9.4 ptyprocess-0.7.0 pure-eval-0.2.2 pygments-2.14.0 pyparsing-3.0.9 pyperclip-1.8.2 python-dateutil-2.8.2 pytz-2022.7 pyzmq-25.0.0 scikit-learn-1.2.0 scipy-1.10.0 shap-0.39.0 six-1.16.0 slicer-0.0.7 sqlalchemy-1.4.46 stack-data-0.6.2 statsmodels-0.13.5 stevedore-4.1.1 threadpoolctl-3.1.0 torch-1.13.1 tornado-6.2 tqdm-4.64.1 traitlets-5.8.1 typing-extensions-4.4.0 wcwidth-0.2.5 wheel-0.38.4 widgetsnbextension-4.0.5 zipp-3.11.0
Trying to import the package yields:
from lightgbmlss.model import *
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/pschaefer/environments/my_env/lib/python3.8/site-packages/lightgbmlss/model.py", line 8, in <module>
import shap
File "/home/pschaefer/environments/my_env/lib/python3.8/site-packages/shap/__init__.py", line 12, in <module>
from ._explanation import Explanation, Cohorts
File "/home/pschaefer/environments/my_env/lib/python3.8/site-packages/shap/_explanation.py", line 12, in <module>
from .utils._general import OpChain
File "/home/pschaefer/environments/my_env/lib/python3.8/site-packages/shap/utils/__init__.py", line 1, in <module>
from ._clustering import hclust_ordering, partition_tree, partition_tree_shuffle, delta_minimization_order, hclust
File "/home/pschaefer/environments/my_env/lib/python3.8/site-packages/shap/utils/_clustering.py", line 4, in <module>
from numba import jit
File "/home/pschaefer/environments/my_env/lib/python3.8/site-packages/numba/__init__.py", line 42, in <module>
from numba.np.ufunc import (vectorize, guvectorize, threading_layer,
File "/home/pschaefer/environments/my_env/lib/python3.8/site-packages/numba/np/ufunc/__init__.py", line 3, in <module>
from numba.np.ufunc.decorators import Vectorize, GUVectorize, vectorize, guvectorize
File "/home/pschaefer/environments/my_env/lib/python3.8/site-packages/numba/np/ufunc/decorators.py", line 3, in <module>
from numba.np.ufunc import _internal
SystemError: initialization of _internal failed without raising an exception
Hello,Great work,I see your code for hyper_paramters search where use lightgbm's CV method,I am trying to use the libarary for timeseries predict,So will you have plan to make CV method with User define CV Class,for traditional CV method always make worse performance in time series prediction.
For model governance reason I want to be able to run tree by tree using the start_iteration and num_iteration arguments to see if there is a number of over fitted trees towards the end of the training.
Would it be possible to add in the **kwargs to the predict all the way through to the lightgbm.predict.
The required python version is not lower than 3.9 now, it's not convenient in some conditions.
Could u plz lower the python version requirement when using 'pip install git+https://github.com/StatMixedML/LightGBMLSS.git' to install.
Thank u.
Hello Mr. März!
Would this program be suited for the prediction of stock market returns with the StudenT distribution shown in the examples?
Would normal Close Price data work with LightGMBLSS as well, or does it struggle with data that was not seen in the train data?
Do you plan to implement a Cauchy distribution as well at one point?
How would the out-of sample prediction work out for some timesteps into the future after the train test phase?
I think your approach to combine statistics with gradient boosting is an amazing idea. I am a huge fan of probabilistic programming.
Best regards
Matthias
The current code uses init_score to inject starting values, but what about modeling problems that need to use init_score to represent an offset, e.g. in insurance, a gamma severity or poisson frequency problem where init_score would be the log(exposure)? Could this be made to get_init_score
from the dataset and incorporate that somehow during the process of setting starting values? Or would you be forced to use weights of exposure and change the response variable to y/exposure?
Hi,
Thanks for the great package.
I am trying to predict the expectiles of a distribution. I understood that because this approach models the entire distribution, and because the true expectiles of any distribution should be monotonic, the predictions would also be monotonic. However, I'm not getting this property. Why might this be / is there anyway to force monotonicity?
Thanks
Josh
Dear community,
I am currently working in a probabilistic extension of LightGBM called LightGBMLSS that models all parameters of a distribution. This allows to create probabilistic forecasts from which prediction intervals and quantiles of interest can be derived.
The problem is that LightGBM doesn't permit do optimize over several parameters. Assume we have a Normal distribution y ~ N(µ, sigma). So far, my approach is a two-step procedure, where I first optimize µ with sigma fixed, and then optimize sigma with µ fixed and then iterate between these two.
Since this is inefficient, are there any ways of simultaneously optimize both µ and sigma using a custom loss function?
Originally posted by p-schaefer February 5, 2024
I am trying to understand the SHAP outputs from a Zero-Adjusted Gamma. I'll outline my questions below:
concentration
, rate
, and gate
shape parameters that are output from predict(..., pred_type="parameters")
, correct? I find this is not quite the case when I look at the outputs of shap.TreeExplainer()
. So I assume there is some transformation happening. Could someone elaborate on what those transformations are, and what they look like.gate
parameter, that would imply it is, on average, decreasing the probability of a 0 outcome, and a positive SHAP score would imply it is increasing the probability of a 0 outcome. Is that interpretation correct?concentration
and rate
(I think). Can multiplying the SHAP scores for concentration
and rate
be used as a high level descriptor of the predictors effects on the non-zero mean part of the distribution? And, how does the transformations affect the results of that?Thank you!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.