sankalp-s / mlbench-benchmarks

Closed Division Benchmarks

mlbench Benchmarks: Distributed Machine Learning Benchmark

A public and reproducible collection of reference implementations and benchmark suite for distributed machine learning algorithms, frameworks and systems.

This repository contains the implementations for the various benchmark tasks for the closed division of mlbench.

• Project website: https://mlbench.github.io/
• Free software: Apache Software License 2.0
• Documentation: https://mlbench.readthedocs.io.

Features

• For reproducibility and simplicity, we currently focus on standard supervised ML, including standard deep learning tasks as well as classic linear ML models.
• We provide reference implementations for each algorithm, to make it easy to port to a new framework.
• Our goal is to benchmark all/most currently relevant distributed execution frameworks. We welcome contributions of new frameworks in the benchmark suite.
• We provide precisely defined tasks and datasets to have a fair and precise comparison of all algorithms, frameworks and hardware.
• Independently of all solver implementations, we provide universal evaluation code allowing to compare the result metrics of different solvers and frameworks.
• Our benchmark code is easy to run on public clouds.

Community

About us: Authors

Mailing list: https://groups.google.com/d/forum/mlbench

Contact Email: [email protected]