jic-csb / dtoolai

Dtool AI reproducible AI library

dtoolAI - reproducible deep learning

dtoolAI is a library for supporting reproducible deep learning.

Quick start

If you'd like to see what dtoolAI can do without installing anything, two of the Jupyter notebooks in this repository highlighting dtoolAI functions can be run through Google Colab without any local software installation:

• Training a character recognition network
• Retraining a network on new data

You'll need a Google account to run these, and when you load the notebooks, click "Open in playground" to be able to execute code.

Installation

Dependencies

dtoolAI is dependent on the following Python packages:

• pytorch
• torchvision
• dtoolcore
• dtool-http
• click
• pillow

If you install dtoolAI with pip or conda as described below, these dependencies will be installed automatically. If you wish to install manually, you'll need to install these before installing dtoolAI.

For Windows users, we recommend installing pytorch and torchvision through anaconda/conda. See the section below for details.

Through pip

dtoolAI requires Python version 3 and Pytorch.

Warning

Install Pytorch before installing dtoolAI. For information on how to install Pytorch this see the Pytorch getting started guide for details.

Once Pytorch has been installed dtoolAI can be installed through pip:

pip install dtoolai

Through conda

You can also install dtoolAI through conda. To optionally create a conda environment in which to install dtoolAI:

conda create -n dtoolai
conda activate dtoolai

Then you can install with:

conda install pytorch==1.4.0 torchvision==0.5.0 -c pytorch
conda install dtoolcore dtool-http dtoolai -c dtool

To install the dtool command line utilities, you'll need to use pip:

pip install dtool

With setup.py

You can also download this repository and install through:

python setup.py install

Documentation

Primary documentation: https://dtoolai.readthedocs.io/en/latest/

Detailed examples of API use are provided in the notebooks/ directory in this repository.

Tests

Running the tests requires pytest.

To run the faster tests in the test suite, use:

pytest tests/ -m "not slow"

The test suite also includes full end-to-end tests that create datasets, train models and evaluate them on those datasets. These are much slower, to run them use:

pytest tests/

dtoolai's Issues

Defect in dtoolai-provenance(?)

Example code in here fails: https://dtoolai.readthedocs.io/en/latest/training.html#viewing-the-trained-model-metadata

With the error message below:

$ dtoolai-provenance example/mnistcnn/
Network architecture name: dtoolai.simpleScalingCNN
Model training parameters: {'batch_size': 128,
 'init_params': {'input_channels': 1, 'input_dim': 28},
 'input_channels': 1,
 'input_dim': 28,
 'learning_rate': 0.01,
 'loss_func': 'NLLLoss',
 'n_epochs': 1,
 'optimiser_name': 'SGD'}
Traceback (most recent call last):
  File "C:\Users\olssont\Anaconda3\envs\dtoolai-reverse-test\Scripts\dtoolai-provenance-script.py", line 10, in <module>
    sys.exit(print_provenance())
  File "C:\Users\olssont\Anaconda3\envs\dtoolai-reverse-test\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\olssont\Anaconda3\envs\dtoolai-reverse-test\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "C:\Users\olssont\Anaconda3\envs\dtoolai-reverse-test\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\olssont\Anaconda3\envs\dtoolai-reverse-test\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "C:\Users\olssont\Anaconda3\envs\dtoolai-reverse-test\lib\site-packages\dtoolai\utils.py", line 72, in print_provenance
    source_ds_uri = ttm.dataset.get_annotation("source_ds_uri")
  File "C:\Users\olssont\Anaconda3\envs\dtoolai-reverse-test\lib\site-packages\dtoolcore\__init__.py", line 518, in get_annotation
    raise(DtoolCoreKeyError())
dtoolcore.DtoolCoreKeyError

Add library code docstrings

Complete sphinx documentation

Missing link to jypyter notebook

In here: https://dtoolai.readthedocs.io/en/latest/training.html#what-the-code-is-doing

it says that we provide a jupyter notebook, however there is no link to it and I can't find it in the git repository.

1. Add it to the git repository
2. Add an explanation to the docs on how to find it

Missing documentation on how to work with raw data

This section: https://dtoolai.readthedocs.io/en/latest/retraining.html#part-2-with-raw-data

seems to be missing the instructions for how to create the dataset from the input data.

Add test suite

Add small test suite for functional testing of library classes and functions.

Create Jupyter notebooks

OSError: [WinError 126] The specified module could not be found

To reproduce on Windows laptop without GPU.

Setup:

python3 -m venv env
.\env\Scripts\Activate.ps1
pip install torch==1.5.0+cpu torchvision==0.6.0+cpu -f https://download.pytorch.org/whl/torch_stable.html
pip install dtool-ai
git clone https://github.com/JIC-CSB/dtoolai.git
cd dtoolai

Command:

python scripts/apply_model_to_image.py http://bit.ly/2tbPzSB .\docs\source\non_mnist_three.png

Error message:

Traceback (most recent call last):
  File "scripts/apply_model_to_image.py", line 6, in <module>
    from torchvision.transforms.functional import to_tensor
  File "C:\Users\olssont\sandbox\dtoolai-testing\env\lib\site-packages\torchvision\__init__.py", line 3, in <module>
    from torchvision import models
  File "C:\Users\olssont\sandbox\dtoolai-testing\env\lib\site-packages\torchvision\models\__init__.py", line 1, in <module>
    from .alexnet import *
  File "C:\Users\olssont\sandbox\dtoolai-testing\env\lib\site-packages\torchvision\models\alexnet.py", line 1, in <module>
    import torch
  File "C:\Users\olssont\sandbox\dtoolai-testing\env\lib\site-packages\torch\__init__.py", line 81, in <module>
    ctypes.CDLL(dll)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.2032.0_x64__qbz5n2kfra8p0\lib\ctypes\__init__.py", line 364, in __init__
    self._handle = _dlopen(self._name, mode)

Command for installing dtoolAI using Conda

The readme includes commands for installing dtoolAI using pip and setup.py. However, no information is provided on how to do it via conda.

Extensions to other ML libraries?

Hey guys,
This looks really cool.
I'm just about to dive in to refactoring a master's students ML code, this was written using scikit-learn. Previous ML stuff I've done has used TensorFlow so I was planning on converting his code over to that.

Would you have any plans to support other ML frameworks? Would it be easy to extend your library to support others?

Cheers!
Ross