ja-thomas / omlbots

OMLbots

Bot that executes (random) experiments on OpenML datasets and uploads the results on the OpenML platform.

The main function of the bot can be executed via runBot.

To add a new algorithm it has to be included in the file R/botSetLearnerParamPairs.R with its hyperparameter ranges.

See executed runs on the openml.org page: https://www.openml.org/u/2702

OpenML Identification

Name: OpenML_Bot R

ID: 2702

Tags

General Tag: mlrRandomBot

Extra-Tag for the random hyperparameter runs (without RF default runs): botV1

Extra-Tag for the reference runs (random forest with defaults): referenceV1

Downloading

The fixed subset of 2.5 million results

A fixed subset of the results of the random bot can be downloaded easily from figshare:

https://figshare.com/articles/OpenML_R_Bot_Benchmark_Data_final_subset_/5882230

This dataset is described soon in a paper.

All results via the nightly database snapshot

Alternatively all results can be downloaded via the nightly database snapshot. The snapshot can be downloaded from: https://docs.openml.org/developers/

After having set up the SQL database (see here for an example how to do it via a terminal in linux), the data can be extracted with this code: https://github.com/ja-thomas/OMLbots/blob/master/snapshot_database/database_extraction.R

Using the R-API

If you want to download results via the OpenML package you can use the following code. (Currently under review, does not work yet.)

https://github.com/ja-thomas/OMLbots/blob/master/GetResultsR-API.R