liyanghart / hyperparameter-optimization-of-machine-learning-algorithms

Implementation of hyperparameter optimization/tuning methods for machine learning & deep learning models (easy&clear)

Hello, I'm Dr. Li Yang!

My Personal Website

- 😄 I'm currently currently a Tenure-Track Assistant Professor in the Faculty of Business and IT at Ontario Tech University.
- 🌱 I've completed Ph.D. degree in Software Engineering - Electrical and Computer Engineering at Western University, Canada in Aug 2022.
- 👯 I've been working on automated machine learning algorithms (e.g., machine learning, deep learning, optimization, online learning, and concept drift adaptation methods) to solve various data analytics problems, such as network security, intrusion detection, anomaly detection, IoT data stream analytics, network traffic data analytics, time-series analytics, etc.
- 👯 I've also been working on the tutorials on applying automated and optimized machine learning algorithms to general data analytics tasks.
- 🌱 My paper and code publications have received more than 2,000 Google Scholar citations and 2,000 GitHub stars.

- PS: Below are the contributions of my personal GitHub account, not including my contributions to the organization account "Western OC2 Lab"

Contact-Info

Please feel free to contact me for any questions or cooperation opportunities. I'd be happy to help.

• Email: [email protected]
• GitHub: LiYangHart and Western OC2 Lab
• LinkedIn: Li Yang
• Google Scholar: Li Yang
• Researchgate: Li Yang

hyperparameter-optimization-of-machine-learning-algorithms's Issues

LightGBMError: Cannot change bin_construct_sample_cnt after constructed Dataset handle.

Environment info

Operating System: windows 10
Python version: python 3.8
LightGBM version: 3.3.2

code

This error occurs when I execute this code. seek help
The code is in this file HPO_Classification.ipynb

params = {'class_weight': 'balanced',
'boosting_type': 'dart',
'num_leaves': 44,
'learning_rate': 0.05979047181117413,
'subsample_for_bin': 220000,
'min_child_samples': 390,
'reg_alpha': 0.3877551020408163,
'reg_lambda': 0.5102040816326531,
'colsample_bytree': 0.7333333333333333,
'subsample': 0.8535353535353536}

def random_objective(params, iteration, n_folds = N_FOLDS):
   """Random search objective function. Takes in hyperparameters
      and returns a list of results to be saved."""

   start = timer()
   
   # Perform n_folds cross validation
   cv_results = lgb.cv(params, train_set, num_boost_round = 10000, nfold = n_folds, 
                       early_stopping_rounds = 100, metrics = 'auc', seed = 50)
   end = timer()
   best_score = np.max(cv_results['auc-mean'])
   
   # Loss must be minimized
   loss = 1 - best_score
   
   # Boosting rounds that returned the highest cv score
   n_estimators = int(np.argmax(cv_results['auc-mean']) + 1)
   
   # Return list of results
   return [loss, params, iteration, n_estimators, end - start]

%%capture

random.seed(50)

for i in range(MAX_EVALS):
   
   # Randomly sample parameters for gbm
   params = {key: random.sample(value, 1)[0] for key, value in param_grid.items()}
   
   print(params)
   
   if params['boosting_type'] == 'goss':
       # Cannot subsample with goss
       params['subsample'] = 1.0
   else:
       # Subsample supported for gdbt and dart
       params['subsample'] = random.sample(subsample_dist, 1)[0]
       
       
   results_list = random_objective(params, i)
   
   # Add results to next row in dataframe
   random_results.loc[i, :] = results_list

Error message

---------------------------------------------------------------------------
LightGBMError                             Traceback (most recent call last)
<ipython-input-20-9bb1341efb38> in <module>
    17 
    18 
---> 19     results_list = random_objective(params, i)
    20 
    21     # Add results to next row in dataframe

<ipython-input-19-060262a0c5a6> in random_objective(params, iteration, n_folds)
     6 
     7     # Perform n_folds cross validation
----> 8     cv_results = lgb.cv(params, train_set, num_boost_round = 10000, nfold = n_folds, 
     9                         early_stopping_rounds = 100, metrics = 'auc', seed = 50)
    10     end = timer()

F:\anaconda\lib\site-packages\lightgbm\engine.py in cv(params, train_set, num_boost_round, folds, nfold, stratified, shuffle, metrics, fobj, feval, init_model, feature_name, categorical_feature, early_stopping_rounds, fpreproc, verbose_eval, show_stdv, seed, callbacks, eval_train_metric, return_cvbooster)
   597         params['metric'] = metrics
   598 
--> 599     train_set._update_params(params) \
   600              ._set_predictor(predictor) \
   601              .set_feature_name(feature_name) \

F:\anaconda\lib\site-packages\lightgbm\basic.py in _update_params(self, params)
  1932                     self._free_handle()
  1933                 else:
-> 1934                     raise LightGBMError(_LIB.LGBM_GetLastError().decode('utf-8'))
  1935         return self
  1936 

LightGBMError: Cannot change bin_construct_sample_cnt after constructed Dataset handle.