ja-thomas / autoxgboostmc Goto Github PK

We might be able to adhere to memory limits by

• Downsampling
• Impact Encoding

Randombot Data could provide Good Metadata.

Lots of unit tests are missing, do this!

The currently most important problem we need to solve is how to focus on sensible areas of the pareto front.
https://arxiv.org/pdf/1805.12168.pdf seems to be a very good solution.

We currently train in the main process. This means we would tear the training can tear the progress down.

Using callr might be a solution, but this also comes with a performance overhead.
Additionally, we could allow for saving / storing the autoxgboostMC objects.

Threshold tuning is problematic:

• Different measures are optimal wrt different thresholds
• Which measure do we use to tune the threshold for?

Possible solution:
Add threshold as a hyperparameter.
We can get then evaluate multiple thresholds during SMBO and write to the opt_state.

Regression does currently not work. Find and fix!

https://github.com/s-fleck/lgr/issues

How do we do early stopping wrt multiple measures?

For now I would implement early stopping for a selected measure?

https://cran.r-project.org/web/packages/xgboost/vignettes/discoverYourData.html#special-note-what-about-random-forests

We might want a PipelineOptimizer class. This would allow us to incorporate different strategies such as EA's etc.