openml / discussionboard

A repository that hosts the Github Discussions for the OpenML organization.

discussionboard

A repository that hosts the Github Discussions for the OpenML organization.

discussionboard's Issues

How to serialize models

More of a developer-to-developer question: we are working on exporting scikit-learn runs, but we are unsure what is the best way to share learned models. On first sight, creating a pickle is the best and most general way to go. Matthias confirms that this works with scikit-learn SVMs, even though the files can get large for large datasets.

However, scikit-learn recommends to use joblib because it is more efficient: http://scikit-learn.org/stable/modules/model_persistence.html

The problem here is that it creates a bunch of files in a folder. This is much harder to share, and sending many many files to the OpenML server for every single run seems unwieldy and error-prone.

Would creating a single pickle file still be the best way forward, or is there a better solution?

Test

This is a test issue, to see if we can have multi-task issues across projects.

• https://github.com/openml/openml-internal-infra-wiki/issues/9
• #11