autogluon / autogluon

Fast and Accurate ML in 3 Lines of Code

Fast and Accurate ML in 3 Lines of Code

Installation | Documentation | Release Notes

AutoGluon automates machine learning tasks enabling you to easily achieve strong predictive performance in your applications. With just a few lines of code, you can train and deploy high-accuracy machine learning and deep learning models on image, text, time series, and tabular data.

💾 Installation

AutoGluon is supported on Python 3.8 - 3.11 and is available on Linux, MacOS, and Windows.

You can install AutoGluon with:

pip install autogluon

Visit our Installation Guide for detailed instructions, including GPU support, Conda installs, and optional dependencies.

⚡ Quickstart

Build accurate end-to-end ML models in just 3 lines of code!

from autogluon.tabular import TabularPredictor
predictor = TabularPredictor(label="class").fit("train.csv")
predictions = predictor.predict("test.csv")

AutoGluon Task	Quickstart	API
TabularPredictor
MultiModalPredictor
TimeSeriesPredictor

🔍 Resources

Hands-on Tutorials / Talks

Below is a curated list of recent tutorials and talks on AutoGluon. A comprehensive list is available here.

Title	Format	Location	Date
📺 AutoGluon 1.0: Shattering the AutoML Ceiling with Zero Lines of Code	Tutorial	AutoML Conf 2023	2023/09/12
🔉 AutoGluon: The Story	Podcast	The AutoML Podcast	2023/09/05
📺 AutoGluon: AutoML for Tabular, Multimodal, and Time Series Data	Tutorial	PyData Berlin	2023/06/20
📺 Solving Complex ML Problems in a few Lines of Code with AutoGluon	Tutorial	PyData Seattle	2023/06/20
📺 The AutoML Revolution	Tutorial	Fall AutoML School 2022	2022/10/18

Scientific Publications

• AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data (Arxiv, 2020) (BibTeX)
• Fast, Accurate, and Simple Models for Tabular Data via Augmented Distillation (NeurIPS, 2020) (BibTeX)
• Benchmarking Multimodal AutoML for Tabular Data with Text Fields (NeurIPS, 2021) (BibTeX)
• XTab: Cross-table Pretraining for Tabular Transformers (ICML, 2023)
• AutoGluon-TimeSeries: AutoML for Probabilistic Time Series Forecasting (AutoML Conf, 2023) (BibTeX)
• TabRepo: A Large Scale Repository of Tabular Model Evaluations and its AutoML Applications (Under Review, 2024)

Articles

• AutoGluon-TimeSeries: Every Time Series Forecasting Model In One Library (Towards Data Science, Jan 2024)
• AutoGluon for tabular data: 3 lines of code to achieve top 1% in Kaggle competitions (AWS Open Source Blog, Mar 2020)
• AutoGluon overview & example applications (Towards Data Science, Dec 2019)

Train/Deploy AutoGluon in the Cloud

• AutoGluon Cloud (Recommended)
• AutoGluon on SageMaker AutoPilot
• AutoGluon on Amazon SageMaker
• AutoGluon Deep Learning Containers (Security certified & maintained by the AutoGluon developers)
• AutoGluon Official Docker Container
• AutoGluon-Tabular on AWS Marketplace (Not maintained by us)

📝 Citing AutoGluon

If you use AutoGluon in a scientific publication, please refer to our citation guide.

👋 How to get involved

We are actively accepting code contributions to the AutoGluon project. If you are interested in contributing to AutoGluon, please read the Contributing Guide to get started.

🏛️ License

This library is licensed under the Apache 2.0 License.

autogluon's Issues

Predictor Object for AutoGluon Fit API

Issue tracker for discussion with @jwmueller @cgraywang today.

results returned by fit may not be a good way to saving state and handling multiple experiments.

We decide to make a Predictor object which contains the following method:

save()
load()
predict()
evaluate()

Task naming confusion

Currently, we have trials also called task https://github.com/awslabs/autogluon/blob/master/autogluon/core/task.py.

In the original design, we have the frontend applications named as task as well.

We probably need to resolve this issue by renaming task to trial.

searcher get_config() may get stuck in infinite while loop

Listing issue here for the record since I removed these comments from my skopt_searcher code to clean it up.

Issue:
get_config() may become stuck in infinite while loop. There is no termination condition to handle the case where all possible configs have already been tried
(should be inherited from BaseSearcher or Scheduler should automatically terminate).

In general, perhaps the BaseSearcher should by default revert to grid search whenever the search-space is sufficiently small.

NAS/efficient net runnable scripts/notebook

It would be great if we have a running example for NAS or efficient net.

Tabular crashes on AutoML Benchmark

When attempting to run automlbenchmark, I get the following error. It disappears and everything works correctly if I update my autolguon/task/init.py file as follows:

from .base import BaseTask
from .image_classification import ImageClassification
from .object_detection import ObjectDetection
# from .text_classification import TextClassification
from .tabular_prediction import TabularPrediction
# from . import image_classification, object_detection, text_classification, tabular_prediction

Thus, something in text classification is causing tabular to error on automlbenchmark.

module 'amlb.logger' has no attribute 'new_print'
Traceback (most recent call last):
  File "/Users/neerick/workspace/automlbenchmark/automlbenchmark/amlb/benchmark.py", line 377, in run
    meta_result = framework.run(self._dataset, task_config)
  File "/Users/neerick/workspace/automlbenchmark/automlbenchmark/frameworks/autogluon/__init__.py", line 3, in run
    from .exec import run
  File "/Users/neerick/workspace/automlbenchmark/automlbenchmark/frameworks/autogluon/exec.py", line 5, in <module>
    from autogluon.task.tabular_prediction.tabular_prediction import TabularPrediction as task
  File "/Users/neerick/workspace/awslabs/autogluon/autogluon/__init__.py", line 14, in <module>
    from .task import *
  File "/Users/neerick/workspace/awslabs/autogluon/autogluon/task/__init__.py", line 4, in <module>
    from .text_classification import TextClassification
  File "/Users/neerick/workspace/awslabs/autogluon/autogluon/task/text_classification/__init__.py", line 1, in <module>
    from .predictor import *
  File "/Users/neerick/workspace/awslabs/autogluon/autogluon/task/text_classification/predictor.py", line 9, in <module>
    import gluonnlp as nlp
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
  File "/Users/neerick/workspace/awslabs/autogluon/venv/lib/python3.6/site-packages/gluonnlp-0.8.1-py3.6.egg/gluonnlp/__init__.py", line 24, in <module>
    from . import data
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
  File "/Users/neerick/workspace/awslabs/autogluon/venv/lib/python3.6/site-packages/gluonnlp-0.8.1-py3.6.egg/gluonnlp/data/__init__.py", line 23, in <module>
    from . import (batchify, candidate_sampler, conll, corpora, dataloader,
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
  File "/Users/neerick/workspace/awslabs/autogluon/venv/lib/python3.6/site-packages/gluonnlp-0.8.1-py3.6.egg/gluonnlp/data/batchify/__init__.py", line 25, in <module>
    from .embedding import *
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
  File "/Users/neerick/workspace/awslabs/autogluon/venv/lib/python3.6/site-packages/gluonnlp-0.8.1-py3.6.egg/gluonnlp/data/batchify/embedding.py", line 28, in <module>
    from ...base import _str_types
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
  File "/Users/neerick/workspace/awslabs/autogluon/venv/lib/python3.6/site-packages/gluonnlp-0.8.1-py3.6.egg/gluonnlp/base.py", line 34, in <module>
    from numba import njit, prange, jitclass, types
  File "/Users/neerick/workspace/awslabs/autogluon/venv/lib/python3.6/site-packages/numba/__init__.py", line 25, in <module>
    from .decorators import autojit, cfunc, generated_jit, jit, njit, stencil
  File "/Users/neerick/workspace/awslabs/autogluon/venv/lib/python3.6/site-packages/numba/decorators.py", line 12, in <module>
    from .targets import registry
  File "/Users/neerick/workspace/awslabs/autogluon/venv/lib/python3.6/site-packages/numba/targets/registry.py", line 5, in <module>
    from . import cpu
  File "/Users/neerick/workspace/awslabs/autogluon/venv/lib/python3.6/site-packages/numba/targets/cpu.py", line 11, in <module>
    from .base import BaseContext, PYOBJECT
  File "/Users/neerick/workspace/awslabs/autogluon/venv/lib/python3.6/site-packages/numba/targets/base.py", line 21, in <module>
    from . import arrayobj, builtins, imputils
  File "/Users/neerick/workspace/awslabs/autogluon/venv/lib/python3.6/site-packages/numba/targets/builtins.py", line 438, in <module>
    from numba.typing.builtins import IndexValue, IndexValueType
  File "/Users/neerick/workspace/awslabs/autogluon/venv/lib/python3.6/site-packages/numba/typing/builtins.py", line 18, in <module>
    @infer_global(print)
  File "/Users/neerick/workspace/awslabs/autogluon/venv/lib/python3.6/site-packages/numba/typing/templates.py", line 850, in register_global
    if getattr(mod, val.__name__) is not val:
AttributeError: module 'amlb.logger' has no attribute 'new_print'

Missing best config after fit for object detection fit example

After executing object detection example, it only produces:

INFO:autogluon.task.object_detection.object_detection:>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> finish model fitting
INFO:autogluon.task.object_detection.object_detection:The best config:

while no best config is reported.

Might relate to #29

Tensorboard with python 3.7

Tensorboard with python 3.6 is working properly in the current autogluon, however Tensorboard with python 3.7 support needs to be verified, even it seems now Tensorboard already supports python 3.7 kubeflow/kubeflow#3391, just open this issue for tracking purpose.

Numpy version

need to figure out who replies the old version.

MXboard tuple reporting issue

For MXboard, when passing a tuple as the reporting result https://github.com/awslabs/autogluon/blob/7795b9c956c24ff85904700482aa96e3eafea38e/autogluon/scheduler/fifo.py#L241, we would have an error, where in the simple plot is seems to be fine.

Tabular- big datasets out of memory issues

To apply tabular_prediction in settings where dataset does not fit into memory, each model should have specialized functionality to train without having to read in full dataset.

In preprocessing of large datasets, we should separate fit_transform() into separate fit(), transform(), where fit() is based on subsample of the data.

Each model should make sure it throws Exception before using up all memory, both during training time and inference time (eg. k nearest neighbors).

For extremely high-dimensional data, may first apply filter feature-selection strategy as well.

Dask Terminal Messaging

Improve the ssh communication by only catch and display meaningful messages. Currently we just display all the terminal outputs from Dask, which is messy.

Skopt Searcher Error Under Stress Test

How to reproduce:

# create search space
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

def gaussian(x, y, x0, y0, xalpha, yalpha, A): 
    return A * np.exp( -((x-x0)/xalpha)**2 -((y-y0)/yalpha)**2) 

x, y = np.linspace(0, 99, 100), np.linspace(0, 99, 100) 
X, Y = np.meshgrid(x, y)

Z = np.zeros(X.shape) 
ps = [(20, 70, 35, 40, 1),
      (80, 40, 20, 20, 0.7)]
for p in ps:
    Z += gaussian(X, Y, *p)

fig = plt.figure()
ax = fig.gca(projection='3d') 
ax.plot_surface(X, Y, Z, cmap='plasma') 
ax.set_zlim(0,np.max(Z)+2)
plt.show()

# create trainin function

import autogluon as ag

@ag.args(
    x=ag.space.Categorical(*list(range(100))),
    y=ag.space.Categorical(*list(range(100))),
)
def rl_simulation(args, reporter):
    x, y = args.x, args.y
    reporter(accuracy=Z[y][x])

# start scheduler and training
skopt_scheduler = ag.scheduler.FIFOScheduler(rl_simulation,
                                             searcher='skopt',
                                             resource={'num_cpus': 1, 'num_gpus': 0},
                                             num_trials=300,
                                             reward_attr="accuracy",
                                             resume=False)
skopt_scheduler.run()
skopt_scheduler.join_jobs()
print('Best config: {}, best reward: {}'.format(skopt_scheduler.get_best_config(), skopt_scheduler.get_best_reward()))

Error message:

Exception in thread Thread-1841:
Traceback (most recent call last):
  File "//anaconda3/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "//anaconda3/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/hzaws/git/AutoGluon/autogluon/scheduler/fifo.py", line 266, in _run_reporter
    reward=last_result[self._reward_attr], **last_result)
  File "/Users/hzaws/git/AutoGluon/autogluon/searcher/skopt_searcher.py", line 172, in update
    -reward)  # provide negative reward since skopt performs minimization
  File "//anaconda3/lib/python3.7/site-packages/scikit_optimize-0.5.2-py3.7.egg/skopt/optimizer/optimizer.py", line 443, in tell
    return self._tell(x, y, fit=fit)
  File "//anaconda3/lib/python3.7/site-packages/scikit_optimize-0.5.2-py3.7.egg/skopt/optimizer/optimizer.py", line 486, in _tell
    est.fit(self.space.transform(self.Xi), self.yi)
  File "//anaconda3/lib/python3.7/site-packages/scikit_optimize-0.5.2-py3.7.egg/skopt/learning/gaussian_process/gpr.py", line 195, in fit
    super(GaussianProcessRegressor, self).fit(X, y)
  File "//anaconda3/lib/python3.7/site-packages/sklearn/gaussian_process/gpr.py", line 221, in fit
    self.kernel_.bounds))]
  File "//anaconda3/lib/python3.7/site-packages/sklearn/gaussian_process/gpr.py", line 465, in _constrained_optimization
    fmin_l_bfgs_b(obj_func, initial_theta, bounds=bounds)
  File "//anaconda3/lib/python3.7/site-packages/scipy/optimize/lbfgsb.py", line 199, in fmin_l_bfgs_b
    **opts)
  File "//anaconda3/lib/python3.7/site-packages/scipy/optimize/lbfgsb.py", line 335, in _minimize_lbfgsb
    f, g = func_and_grad(x)
  File "//anaconda3/lib/python3.7/site-packages/scipy/optimize/lbfgsb.py", line 285, in func_and_grad
    f = fun(x, *args)
  File "//anaconda3/lib/python3.7/site-packages/scipy/optimize/optimize.py", line 326, in function_wrapper
    return function(*(wrapper_args + args))
  File "//anaconda3/lib/python3.7/site-packages/scipy/optimize/optimize.py", line 64, in __call__
    fg = self.fun(x, *args)
  File "//anaconda3/lib/python3.7/site-packages/sklearn/gaussian_process/gpr.py", line 213, in obj_func
    theta, eval_gradient=True)
  File "//anaconda3/lib/python3.7/site-packages/sklearn/gaussian_process/gpr.py", line 423, in log_marginal_likelihood
    K, K_gradient = kernel(self.X_train_, eval_gradient=True)
  File "//anaconda3/lib/python3.7/site-packages/sklearn/gaussian_process/kernels.py", line 685, in __call__
    K1, K1_gradient = self.k1(X, Y, eval_gradient=True)
  File "//anaconda3/lib/python3.7/site-packages/sklearn/gaussian_process/kernels.py", line 760, in __call__
    K2, K2_gradient = self.k2(X, Y, eval_gradient=True)
  File "//anaconda3/lib/python3.7/site-packages/scikit_optimize-0.5.2-py3.7.egg/skopt/learning/gaussian_process/kernels.py", line 391, in __call__
    (X.shape, len(length_scale)))
TypeError: %d format: a number is required, not tuple

Make sure all dependencies are python3 compatible

I tried to install autogluon on Ubuntu Deep Learning Base AMI 16.04/18.04 (using the python3 version) and has the following error:
Installation command:

pip3 install --upgrade mxnet-cu100 --user
pip3 install https://autogluon.s3.amazonaws.com/dist/autogluon-0.0.1-py3-none-any.whl --user

Run command

import autogluon as ag

Error:

/usr/lib/python3/dist-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
Traceback (most recent call last):

  File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 3296, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)

  File "<ipython-input-1-014e868ea365>", line 1, in <module>
    import autogluon as ag

  File "/home/ubuntu/.local/lib/python3.6/site-packages/autogluon/__init__.py", line 10, in <module>
    from . import scheduler, searcher, nas, utils

  File "/home/ubuntu/.local/lib/python3.6/site-packages/autogluon/scheduler/__init__.py", line 5, in <module>
    from .scheduler import *

  File "/home/ubuntu/.local/lib/python3.6/site-packages/autogluon/scheduler/scheduler.py", line 21, in <module>
    class TaskScheduler(object):

  File "/home/ubuntu/.local/lib/python3.6/site-packages/autogluon/scheduler/scheduler.py", line 26, in TaskScheduler
    REMOTE_MANAGER = RemoteManager()

  File "/home/ubuntu/.local/lib/python3.6/site-packages/autogluon/scheduler/remote/remote_manager.py", line 37, in __new__
    cls.start_local_node()

  File "/home/ubuntu/.local/lib/python3.6/site-packages/autogluon/scheduler/remote/remote_manager.py", line 48, in start_local_node
    remote = Remote.create_local_node(cls.MASTER_IP, port)

  File "/home/ubuntu/.local/lib/python3.6/site-packages/autogluon/scheduler/remote/remote.py", line 54, in create_local_node
    return cls(ip, port, local=True)

  File "/home/ubuntu/.local/lib/python3.6/site-packages/autogluon/scheduler/remote/remote.py", line 31, in __init__
    super(Remote, self).__init__(processes=False)

  File "/home/ubuntu/.local/lib/python3.6/site-packages/distributed/client.py", line 721, in __init__
    self.start(timeout=timeout)

  File "/home/ubuntu/.local/lib/python3.6/site-packages/distributed/client.py", line 886, in start
    sync(self.loop, self._start, **kwargs)

  File "/home/ubuntu/.local/lib/python3.6/site-packages/distributed/utils.py", line 333, in sync
    raise exc.with_traceback(tb)

  File "/home/ubuntu/.local/lib/python3.6/site-packages/distributed/utils.py", line 317, in f
    result[0] = yield future

  File "/usr/local/lib/python3.6/dist-packages/tornado/gen.py", line 735, in run
    value = future.result()

  File "/home/ubuntu/.local/lib/python3.6/site-packages/distributed/client.py", line 952, in _start
    **self._startup_kwargs

  File "/home/ubuntu/.local/lib/python3.6/site-packages/distributed/deploy/spec.py", line 346, in _
    await self._start()

  File "/home/ubuntu/.local/lib/python3.6/site-packages/distributed/deploy/spec.py", line 264, in _start
    **self.scheduler_spec.get("options", {})

  File "/home/ubuntu/.local/lib/python3.6/site-packages/distributed/scheduler.py", line 903, in __init__
    from distributed.dashboard import BokehScheduler

  File "/home/ubuntu/.local/lib/python3.6/site-packages/distributed/dashboard/__init__.py", line 1, in <module>
    from .scheduler import BokehScheduler

  File "/home/ubuntu/.local/lib/python3.6/site-packages/distributed/dashboard/scheduler.py", line 402, in <module>
    import pynvml  # noqa: 1708

  File "/usr/local/lib/python3.6/dist-packages/pynvml.py", line 1831
    print c_count.value
                ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(c_count.value)?

Renaming Fit API Arg Names

make an issue tracker for the discussion in the meeting
num_cpus => nthreads_per_trial
num_gpus => ngpus_per_trial

All searchers should expect keyword arguments in `get_config` method

All searchers should expect keyword args in the get_config method, even if they are not used.

Otherwise, this poses a problem for some schedulers. For example, with ag.searcher.RandomSearcher and ag.scheduler.HyperbandScheduler, I get:

TypeError: get_config() got an unexpected keyword argument 'bracket'

Perhaps the simplest fix is to ensure ag.searcher.RandomSearcher (and all other subclasses of ag.searcher.BaseSearcher have the function signature get_config(self, **kwargs).

Originally posted by @ltiao in #38

warnings in tabular/ branch requirements.txt packages

@Innixma There are many packages in requirements.txt that are outdated (eg. botocore, thinc, protobuf). After import autogluon, we see ugly warning for these packages:

"DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working"

We should upgrade these package versions to the appropriate ones without warnings.
There are also warnings related to lightgbm we should try to fix/suppress if it's not too hard:

/Users/jonasmue/virtual/t3/lib/python3.7/site-packages/lightgbm/init.py:46: UserWarning: Starting from version 2.2.1, the library file in distribution wheels for macOS is built by the Apple Clang (Xcode_8.3.3) compiler.
This means that in case of installing LightGBM from PyPI via the pip install lightgbm command, you don't need to install the gcc compiler anymore.
Instead of that, you need to install the OpenMP library, which is required for running LightGBM on the system with the Apple Clang compiler.
You can install the OpenMP library by the following command: brew install libomp.
"You can install the OpenMP library by the following command: brew install libomp.", UserWarning)
/Users/jonasmue/virtual/t3/lib/python3.7/site-packages/thinc/check.py:4: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working

Bug when import autogluon

The error msg is as below:

>>> import autogluon
/home/ubuntu/anaconda3/lib/python3.6/site-packages/sklearn/ensemble/weight_boosting.py:29: DeprecationWarning: numpy.core.umath_tests is an internal NumPy module and should not be imported. It will be removed in a future NumPy release.
  from numpy.core.umath_tests import inner1d
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/auto-ml-with-gluon/autogluon/__init__.py", line 14, in <module>
    from . import scheduler, searcher, distributed
  File "/home/ubuntu/auto-ml-with-gluon/autogluon/distributed/__init__.py", line 3, in <module>
    from .remote_manager import *
  File "/home/ubuntu/auto-ml-with-gluon/autogluon/distributed/remote_manager.py", line 8, in <module>
    from ..utils import get_ip
ImportError: cannot import name 'get_ip'

pip show mxnet-cu100
Name: mxnet-cu100
Version: 1.5.0

python --version
Python 3.6.5 :: Anaconda, Inc.

ubuntu@ip-172-31-17-156:/home$ pip show autogluon
Name: autogluon
Version: 0.0.1+0d8a584

Doc Improvement

Need to improve the documentation.

1. Format: we need to rewrite by following the MXNet doc style, due to using new doc template.
2. Lauguage: need to do proof read and fix typos.
3. Content: need to add working examples to each doc class/function

Running error on Ubuntu 18.04 (DL base AMI)

After I install autogluon from pip install https://autogluon.s3.amazonaws.com/dist/autogluon-0.0.1-py3-none-any.whl and import autogluon, I got the following error:

Traceback (most recent call last):

  File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 3296, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)

  File "<ipython-input-1-9ff6d8eb77d9>", line 1, in <module>
    import autogluon as ag

  File "/usr/local/lib/python3.6/dist-packages/autogluon/__init__.py", line 10, in <module>
    from . import scheduler, searcher, nas, utils

  File "/usr/local/lib/python3.6/dist-packages/autogluon/scheduler/__init__.py", line 5, in <module>
    from .scheduler import *

  File "/usr/local/lib/python3.6/dist-packages/autogluon/scheduler/scheduler.py", line 21, in <module>
    class TaskScheduler(object):

  File "/usr/local/lib/python3.6/dist-packages/autogluon/scheduler/scheduler.py", line 26, in TaskScheduler
    REMOTE_MANAGER = RemoteManager()

  File "/usr/local/lib/python3.6/dist-packages/autogluon/scheduler/remote/remote_manager.py", line 37, in __new__
    cls.start_local_node()

  File "/usr/local/lib/python3.6/dist-packages/autogluon/scheduler/remote/remote_manager.py", line 48, in start_local_node
    remote = Remote.create_local_node(cls.MASTER_IP, port)

  File "/usr/local/lib/python3.6/dist-packages/autogluon/scheduler/remote/remote.py", line 54, in create_local_node
    return cls(ip, port, local=True)

  File "/usr/local/lib/python3.6/dist-packages/autogluon/scheduler/remote/remote.py", line 31, in __init__
    super(Remote, self).__init__(processes=False)

  File "/usr/local/lib/python3.6/dist-packages/distributed/client.py", line 721, in __init__
    self.start(timeout=timeout)

  File "/usr/local/lib/python3.6/dist-packages/distributed/client.py", line 886, in start
    sync(self.loop, self._start, **kwargs)

  File "/usr/local/lib/python3.6/dist-packages/distributed/utils.py", line 333, in sync
    raise exc.with_traceback(tb)

  File "/usr/local/lib/python3.6/dist-packages/distributed/utils.py", line 317, in f
    result[0] = yield future

  File "/usr/local/lib/python3.6/dist-packages/tornado/gen.py", line 735, in run
    value = future.result()

  File "/usr/local/lib/python3.6/dist-packages/distributed/client.py", line 952, in _start
    **self._startup_kwargs

  File "/usr/local/lib/python3.6/dist-packages/distributed/deploy/spec.py", line 346, in _
    await self._start()

  File "/usr/local/lib/python3.6/dist-packages/distributed/deploy/spec.py", line 264, in _start
    **self.scheduler_spec.get("options", {})

  File "/usr/local/lib/python3.6/dist-packages/distributed/scheduler.py", line 903, in __init__
    from distributed.dashboard import BokehScheduler

  File "/usr/local/lib/python3.6/dist-packages/distributed/dashboard/__init__.py", line 1, in <module>
    from .scheduler import BokehScheduler

  File "/usr/local/lib/python3.6/dist-packages/distributed/dashboard/scheduler.py", line 402, in <module>
    import pynvml  # noqa: 1708

  File "/usr/local/lib/python3.6/dist-packages/pynvml.py", line 1831
    print c_count.value
                ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(c_count.value)?

Installation issues + fixes with the current version

We can use this issue to keep track of installation issues and their fixes with whatever is the current version of autogluon.

Check for python version explicitly

python2 and python<=3.5 doesn't work, autogluon should raise meaningful error instead of confusing backtrace logs.

Tutorials Request

• User defined Network
• User defined script using Advanced API
• Distributed Training

Issue Tracker for #12

Unify Serialization Method

Nothing urgent. Start an issue tracker:

1. Use ag.save and ag.load in tabular task
2. Integrate save_s3 with ag.save or move it to utils and use as ag.save_s3

Tabular row weight support

Add support for row-wise weights in a similar fashion to LightGBM:

https://lightgbm.readthedocs.io/en/latest/Parameters.html#weight_column

We can have a dedicated weight column that is checked for presence such as '__weight', and potentially give the user the option to specify the column in the call to learner init.

With this column we would additionally need to add support for weight in all objective functions and models.

Dataset split stuck for large dataset

Open an issue tracker.

Current auto split in ImageClassification.fit does not work for large dataset. We need to reimplement it usign batchify_fn.

Default Behavior for launch function defined using @autogluon_register_args

Issue Tracker for today's talk:

Dummy reporter should be automattically provided during the function call, which enables user to lanch a single run for debug purpose.

Issue installing and using lightgbm

Reported in bug bash. Once installed libomp, still get an error:

/Users/mli/miniconda3/lib/python3.7/site-packages/lightgbm/init.py:48: UserWarning: Starting from version 2.2.1, the library file in distribution wheels for macOS is built by the Apple Clang (Xcode_8.3.3) compiler.This means that in case of installing LightGBM from PyPI via the pip install lightgbm command, you don't need to install the gcc compiler anymore.Instead of that, you need to install the OpenMP library, which is required for running LightGBM on the system with the Apple Clang compiler.You can install the OpenMP library by the following command: brew install libomp. "You can install the OpenMP library by the following command: brew install libomp.", UserWarning)I

Feature Request for Fit API for load and save

Task.load() and Task.save()

issue tracker for #12

IPYWidgets Display Issue on AutoGluon Tutorial Docs

The HBox and Vbox

should have try-except block when fetching model files

When running the basic image classification tutorial, I get log statements like:

[ worker local ] : FileNotFoundError: [Errno 2] No such file or directory: '/home/ubuntu/.mxnet/models/resnet50_v1b-0ecdba34.zip'

presumably because the file hasn't yet been downloaded. However, we should enclose the file access in a try-except block to prevent such errors from printing, since the user may otherwise think autogluon is broken.

API Naming Discussion

Naming Issue Tracker for #12

1. autogluon.Linear => autogluon.Real or autogluon.Range?

2. Merging autogluon.Int with autogluon.Linear, which is suggested by @cgraywang ? (I think it may be a little misleading)

3. @cgraywang suggest changing autogluon.Choice back to List, @jwmueller and @zhanghang1989 think Choice may be better.

Reduce dependency as much as possible.

It's very difficult to install autogluon. We should reduce the dependency as much as possible.

Results object returned by task.fit() no longer contains info about each config tried and their reward

After I call results = task.fit(num_trials=10),

the 'results' object should contain information about each of the 10 trials:
which hyperparameter config was tried, the corresponding reward, other various metadata about the trial.

I would use the following naming conventions for the returned object: results = task.fit():

results.model = best MXNet model found in task.fit()  (same as current)

results.time = run-time of fit  (same as current)

results.validation_performance = validation performance achieved by this model (what is currently called 'reward')

results.train_objective = training loss value achieved by this model on the training data

results.selected_hyperparameters = hyperparameter values corresponding to this model (what is currently called 'config')

results.metadata = dict containing the following keys:
    search_space = hyperparameter search space considered in task.fit() (currently unnamed)
    search_strategy = HPO algorithm used (ie. Hyperband, random, BayesOpt). If the HPO algorithm used kwargs, then this should be tuple (HPO_algorithm_string, HPO_kwargs)
    num_completed_trials = number of trials completed during task.fit

    results.metadata can contain other optional keys such as:
    latency = inference-time of this model (time for feedforward pass)
    memory = amount of memory required by this model 
    worst_errors = list of K validation examples where model made the worst errors (ie. lowest-probability of correct class in the case of classification)


results.trial_info = list of dicts (length = number of trials).
    results.trial_info[i] is dict containing the following keys:  
        config = hyperparameter configuration tried in ith trial
        validation_performance = validation performance of the corresponding model in ith trial
        train_objective =  training loss value achieved by ith trial's model on the training data
        metadata = dict of various optional metadata with keys such as:
            early_stopped = whether or not this trial was early stopped or not in Hyperband
            latency = inference-time of the model from this trial
            memory = amount of memory required by model from this trial 

mv `autogluon/nas` to `autogluon/task/nas`

folder path change

Error During Scheduler Shutdown

python script does not have clean exit.

Decorator Naming Discussion

@autogluon_object => @tuable_class, @auto_object?
@autogluon_function => @tunable_function, or @auto_function ?
@autogluon_register_args = ?
ag.space.Categorical => ag.space.Category?

Implementing time limit

need to implement time_limit in the schedulers

ENAS Training Efficiency on ImageNet

ENAS training on CIFAR is improved after using controller prefech, but it is still trained slowly on ImageNet. Around 7 days for now, should be around 2 days. Need to debug.

Objective Function / Metric Support in Tabular

Currently, tabular can take any objective function as input to optimize for, however not all objective functions act the same:

1. Some objective functions use different inputs: Accuracy takes y_pred, while logloss takes y_pred_proba.

2. Some objective functions have lower values as their optimal, while some have higher. logloss should be optimized to minimize its value, whereas AUC should be optimized to maximize its value.

The function passed to Tabular in isolation makes it difficult to determine the correct answers to 1 and 2, and currently, it only supports lower=better, y_pred for classification, y_pred_proba for regression.

To support all kinds of objective functions, including custom ones, there are two strategies:

1. Have a dictionary of all supported objective functions, including if higher=better and their input parameter types. The user can pass in a custom objective function by specifying the function as in the dictionary.

2. The user only supplies the objective function itself, and autogluon uses try/except checks to determine the correct use of the function. For example, if passed y_true=[1], y_pred=[1] and the function returns 0, and then passed y_true=[1], y_pred=[0] and the function returns 1, we can safely assume that higher=worse for this metric. Additionally, we can test with try/except func([1],[0.7]), if it does not throw an exception, the function is valid for y_pred_proba. If it does throw an exception, it is valid for y_pred.

Curious to hear thoughts on this. I believe one of these solutions should be done prior to merging tabular into the rest of the code base, as tabular currently has several hacks in it to bypass this limitation which should be removed.

[Bug] scheduler.upload_files()

upload_files() does not behave as expected when uploading non-python files.

Make Dependencies More Flexible

Let's try to relax some of the versions of the dependencies. Got some feedback from bug bash:

1. Conda:
   1. create a new env: conda create -n autogluon In default it install python 3.8, will got a installation error ERROR: Could not find a version that satisfies the requirement pyarrow==0.15.0 (from autogluon==0.0.1) (from versions: 0.9.0, 0.10.0, 0.11.0, 0.11.1, 0.12.0, 0.12.1, 0.13.0, 0.14.0, 0.15.1)
   2. The solution for above is conda create -n autogluon python==3.7
   3. ERROR: No matching distribution found for pyarrow==0.15.0 (from autogluon==0.0.1)

[Before Open Source] Remove Tabular Branch

Tabular Branch has datasets in its history, so we should remove it prior to open source.

MXNet segfault error when calling scheduler.run() 2+ times in same Python session

@zhanghang1989 : Below is the MXNet error I get when I call scheduler.run() two times in the same Python session. Here are the steps to reproduce:

1. checkout tabular branch: https://github.com/awslabs/autogluon/tree/tabular
   git checkout tabular

2. install tabular module by following steps in tabular/README: https://github.com/awslabs/autogluon/blob/tabular/tabular/README.md

3. Verify your installation worked by running the simple example in:
   https://github.com/awslabs/autogluon/blob/tabular/autogluon/task/predict_table_column/examples/example_tabular_predictions.py

Note that this example does not do any HPO and does not use ag.schedulers at all.
You can run this example many times in a row inside the same Python session without any segfault issue.

4. Now try running the example in:
   https://github.com/awslabs/autogluon/blob/tabular/autogluon/task/predict_table_column/examples/example_advanced_tabular.py

This example should also work (it may produce tons of warnings, but should not produce any MXNet segfault). This example demonstrates doing HPO during task.fit() by leveraging the ag.scheduler and internally calls scheduler.run() one time. The key line of code that does this is: https://github.com/awslabs/autogluon/blob/tabular/autogluon/task/predict_table_column/examples/example_advanced_tabular.py#L30

predictor = task.fit(train_data=train_data, label=label_column, output_directory=savedir, hyperparameter_tune=True, num_trials=10, time_limits=10*60, nn_options=nn_options)

5. Now re-run this same example, but instead of calling ag.done() after running the above line of code, try running this line of code two times in a row, ie.

`predictor = task.fit(train_data=train_data, label=label_column, output_directory=savedir, hyperparameter_tune=True,
num_trials=10, time_limits=10*60, nn_options=nn_options)

predictor = task.fit(train_data=train_data, label=label_column, output_directory=savedir, hyperparameter_tune=True,
num_trials=10, time_limits=10*60, nn_options=nn_options)
`

6. This should produce the segfault error below:

0%| | 0/10 [00:00<?, ?it/s]
Segmentation fault: 11

Stack trace:
[bt] (0) 1 libmxnet.so 0x0000000117e062b0 mxnet::Storage::Get() + 4880
[bt] (1) 2 libsystem_platform.dylib 0x00007fff7f0f3b5d _sigtramp + 29
[bt] (2) 3 ??? 0x0000000000000000 0x0 + 0
[bt] (3) 4 libBLAS.dylib 0x00007fff4fac5d44 APL_sgemm + 806
[bt] (4) 5 libBLAS.dylib 0x00007fff4fa504c2 cblas_sgemm + 1592
[bt] (5) 6 libmxnet.so 0x000000011654e8b5 mxnet::op::FullyConnectedComputeExCPU(nnvm::NodeAttrs const&, mxnet::OpContext const&, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray > const&, std::__1::vector<mxnet::OpReqType, std::__1::allocatormxnet::OpReqType > const&, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray > const&) + 14421
[bt] (6) 7 libmxnet.so 0x000000011654b5f8 mxnet::op::FullyConnectedComputeExCPU(nnvm::NodeAttrs const&, mxnet::OpContext const&, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray > const&, std::__1::vector<mxnet::OpReqType, std::__1::allocatormxnet::OpReqType > const&, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray > const&) + 1432
[bt] (7) 8 libmxnet.so 0x000000011654b363 mxnet::op::FullyConnectedComputeExCPU(nnvm::NodeAttrs const&, mxnet::OpContext const&, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray > const&, std::__1::vector<mxnet::OpReqType, std::__1::allocatormxnet::OpReqType > const&, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray > const&) + 771
[bt] (8) 9 libmxnet.so 0x000000011774dca9 mxnet::imperative::PushFComputeEx(std::__1::function<void (nnvm::NodeAttrs const&, mxnet::OpContext const&, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray > const&, std::__1::vector<mxnet::OpReqType, std::__1::allocatormxnet::OpReqType > const&, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray > const&)> const&, nnvm::Op const*, nnvm::NodeAttrs const&, mxnet::Context const&, std::__1::vector<mxnet::engine::Var*, std::__1::allocatormxnet::engine::Var* > const&, std::__1::vector<mxnet::engine::Var*, std::__1::allocatormxnet::engine::Var* > const&, std::__1::vector<mxnet::Resource, std::__1::allocatormxnet::Resource > const&, std::__1::vector<mxnet::NDArray*, std::__1::allocatormxnet::NDArray* > const&, std::__1::vector<mxnet::NDArray*, std::__1::allocatormxnet::NDArray* > const&, std::__1::vector<mxnet::OpReqType, std::__1::allocatormxnet::OpReqType > const&)::'lambda'(mxnet::RunContext)::operator()(mxnet::RunContext) const + 217

Segmentation fault: 11

Stack trace:
[bt] (0) 1 libmxnet.so 0x0000000117e062b0 mxnet::Storage::Get() + 4880
[bt] (1) 2 libsystem_platform.dylib 0x00007fff7f0f3b5d _sigtramp + 29
[bt] (2) 3 ??? 0x0000000000000000 0x0 + 0
[bt] (3) 4 libBLAS.dylib 0x00007fff4fac5d44 APL_sgemm + 806
[bt] (4) 5 libBLAS.dylib 0x00007fff4fa504c2 cblas_sgemm + 1592
[bt] (5) 6 libmxnet.so 0x000000011654e8b5 mxnet::op::FullyConnectedComputeExCPU(nnvm::NodeAttrs const&, mxnet::OpContext const&, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray > const&, std::__1::vector<mxnet::OpReqType, std::__1::allocatormxnet::OpReqType > const&, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray > const&) + 14421
[bt] (6) 7 libmxnet.so 0x000000011654b5f8 mxnet::op::FullyConnectedComputeExCPU(nnvm::NodeAttrs const&, mxnet::OpContext const&, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray > const&, std::__1::vector<mxnet::OpReqType, std::__1::allocatormxnet::OpReqType > const&, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray > const&) + 1432
[bt] (7) 8 libmxnet.so 0x000000011654b363 mxnet::op::FullyConnectedComputeExCPU(nnvm::NodeAttrs const&, mxnet::OpContext const&, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray > const&, std::__1::vector<mxnet::OpReqType, std::__1::allocatormxnet::OpReqType > const&, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray > const&) + 771
[bt] (8) 9 libmxnet.so 0x000000011774dca9 mxnet::imperative::PushFComputeEx(std::__1::function<void (nnvm::NodeAttrs const&, mxnet::OpContext const&, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray > const&, std::__1::vector<mxnet::OpReqType, std::__1::allocatormxnet::OpReqType > const&, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray > const&)> const&, nnvm::Op const*, nnvm::NodeAttrs const&, mxnet::Context const&, std::__1::vector<mxnet::engine::Var*, std::__1::allocatormxnet::engine::Var* > const&, std::__1::vector<mxnet::engine::Var*, std::__1::allocatormxnet::engine::Var* > const&, std::__1::vector<mxnet::Resource, std::__1::allocatormxnet::Resource > const&, std::__1::vector<mxnet::NDArray*, std::__1::allocatormxnet::NDArray* > const&, std::__1::vector<mxnet::NDArray*, std::__1::allocatormxnet::NDArray* > const&, std::__1::vector<mxnet::OpReqType, std::__1::allocatormxnet::OpReqType > const&)::'lambda'(mxnet::RunContext)::operator()(mxnet::RunContext) const + 217

Segmentation fault: 11

Stack trace:
[bt] (0) 1 libmxnet.so 0x0000000117e062b0 mxnet::Storage::Get() + 4880
[bt] (1) 2 libsystem_platform.dylib 0x00007fff7f0f3b5d _sigtramp + 29
[bt] (2) 3 ??? 0x000000010e3eea00 0x0 + 4533971456
[bt] (3) 4 libmxnet.so 0x00000001161c4ad3 std::__1::map<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator >, mxnet::NDArrayFunctionReg*, std::__1::less<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const, mxnet::NDArrayFunctionReg*> > >::__find_equal_key(std::__1::__tree_node_base<void*>&, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&) + 867
[bt] (4) 5 libmxnet.so 0x0000000117479f3a void mxnet::op::FillComputeZerosExmshadow::cpu(nnvm::NodeAttrs const&, mxnet::OpContext const&, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray > const&, std::__1::vector<mxnet::OpReqType, std::__1::allocatormxnet::OpReqType > const&, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray > const&) + 666
[bt] (5) 6 libmxnet.so 0x0000000117685b62 SetNDInputsOutputs(nnvm::Op const, std::__1::vector<mxnet::NDArray*, std::__1::allocatormxnet::NDArray* >, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray* >, int, void const*, int*, int, int, void***) + 3330
[bt] (6) 7 libmxnet.so 0x00000001176853c8 SetNDInputsOutputs(nnvm::Op const*, std::__1::vector<mxnet::NDArray*, std::__1::allocatormxnet::NDArray* >, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray* >, int, void const*, int*, int, int, void***) + 1384
[bt] (7) 8 libmxnet.so 0x00000001176861d0 MXImperativeInvokeEx + 176
[bt] (8) 9 _ctypes.cpython-37m-darwin.so 0x000000010f609367 ffi_call_unix64 + 79

Segmentation fault: 11

Stack trace:
[bt] (0) 1 libmxnet.so 0x0000000117e062b0 mxnet::Storage::Get() + 4880
[bt] (1) 2 libsystem_platform.dylib 0x00007fff7f0f3b5d _sigtramp + 29
[bt] (2) 3 Python 0x000000010e1a6fdd member_set + 52
[bt] (3) 4 libmxnet.so 0x00000001161c4ad3 std::__1::map<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator >, mxnet::NDArrayFunctionReg*, std::__1::less<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const, mxnet::NDArrayFunctionReg*> > >::__find_equal_key(std::__1::__tree_node_base<void*>&, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&) + 867
[bt] (4) 5 libmxnet.so 0x0000000117479f3a void mxnet::op::FillComputeZerosExmshadow::cpu(nnvm::NodeAttrs const&, mxnet::OpContext const&, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray > const&, std::__1::vector<mxnet::OpReqType, std::__1::allocatormxnet::OpReqType > const&, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray > const&) + 666
[bt] (5) 6 libmxnet.so 0x0000000117685b62 SetNDInputsOutputs(nnvm::Op const, std::__1::vector<mxnet::NDArray*, std::__1::allocatormxnet::NDArray* >, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray* >, int, void const*, int*, int, int, void***) + 3330
[bt] (6) 7 libmxnet.so 0x00000001176853c8 SetNDInputsOutputs(nnvm::Op const*, std::__1::vector<mxnet::NDArray*, std::__1::allocatormxnet::NDArray* >, std::__1::vector<mxnet::NDArray, std::__1::allocatormxnet::NDArray* >, int, void const*, int*, int, int, void***) + 1384
[bt] (7) 8 libmxnet.so 0x00000001176861d0 MXImperativeInvokeEx + 176
[bt] (8) 9 _ctypes.cpython-37m-darwin.so 0x000000010f609367 ffi_call_unix64 + 79

mxboard should be optional

mxboard should either be optional or it should be explicitly listed as a requirement in setup.py (I vote for the former choice).

It's annoying to keep getting: ModuleNotFoundError: No module named 'mxboard' after import autogluon on new instances because I forgot to install mxboard manually myself.

NLP User Dataset, Kaggle Tutorial and Doc Improvement

1. User dataset
2. kaggle tutorial
3. doc improment and address comments in #80

[Before Open Source] Change Image Classification Beginner Tutorial to Small Dataset

Change it to a toy dataset runnable on CPU, which is important based on discussion of AutoGluon Dev team.

Installation error on Ubuntu 16.04 (DL base AMI)

When I install autogluon with pip3 install https://autogluon.s3.amazonaws.com/dist/autogluon-0.0.1-py3-none-any.whl, I got the following error. The Python version is 3.5.2.

ERROR: Error checking for conflicts.
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 3021, in _dep_map
    return self.__dep_map
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 2815, in __getattr__
    raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 3012, in _parsed_pkg_info
    return self._pkg_info
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 2815, in __getattr__
    raise AttributeError(attr)
AttributeError: _pkg_info

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/commands/install.py", line 568, in _warn_about_conflicts
    package_set, _dep_info = check_install_conflicts(to_install)
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/operations/check.py", line 114, in check_install_conflicts
    package_set, _ = create_package_set_from_installed()
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/operations/check.py", line 53, in create_package_set_from_installed
    package_set[name] = PackageDetails(dist.version, dist.requires())
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 2736, in requires
    dm = self._dep_map
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 3023, in _dep_map
    self.__dep_map = self._compute_dependencies()
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 3032, in _compute_dependencies
    for req in self._parsed_pkg_info.get_all('Requires-Dist') or []:
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 3014, in _parsed_pkg_info
    metadata = self.get_metadata(self.PKG_INFO)
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 1420, in get_metadata
    value = self._get(path)
  File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 1616, in _get
    with open(path, 'rb') as stream:
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/python3.5/dist-packages/scikit_learn-0.20.2.dist-info/METADATA'

respect time_limits better

For num_trials default:

if time_limits < THRESHOLD:
num_trials = 1

where THRESHOLD is something small, say = 1 or 10

Bug in Scheduler `cleaning_task`

cleaning_task during join_task may have race condition, need to fix.

Scheduler does not exit cleanly

scheduler.shutdown() doesn't work