Bot that executes (random) experiments on OpenML datasets and uploads the results on the OpenML platform.
The main function of the bot can be executed via runBot
.
To add a new algorithm it has to be included in the file R/botSetLearnerParamPairs.R
with its hyperparameter ranges.
See executed runs on the openml.org page: https://www.openml.org/u/2702
Name: OpenML_Bot R
ID: 2702
General Tag: mlrRandomBot
Extra-Tag for the random hyperparameter runs (without RF default runs): botV1
Extra-Tag for the reference runs (random forest with defaults): referenceV1
A fixed subset of the results of the random bot can be downloaded easily from figshare:
https://figshare.com/articles/OpenML_R_Bot_Benchmark_Data_final_subset_/5882230
This dataset is described soon in a paper.
Alternatively all results can be downloaded via the nightly database snapshot. The snapshot can be downloaded from: https://docs.openml.org/developers/
After having set up the SQL database (see here for an example how to do it via a terminal in linux), the data can be extracted with this code: https://github.com/ja-thomas/OMLbots/blob/master/snapshot_database/database_extraction.R
If you want to download results via the OpenML package you can use the following code. (Currently under review, does not work yet.)
https://github.com/ja-thomas/OMLbots/blob/master/GetResultsR-API.R