chainer / models

Models and examples built with Chainer

Notice: As announced, Chainer is under the maintenance phase and further development will be limited to bug-fixes and maintenance only.

---

Chainer: A deep learning framework

Website | Docs | Install Guide | Tutorials (ja) | Examples (Official, External) | Concepts | ChainerX

Forum (en, ja) | Slack invitation (en, ja) | Twitter (en, ja)

Chainer is a Python-based deep learning framework aiming at flexibility. It provides automatic differentiation APIs based on the define-by-run approach (a.k.a. dynamic computational graphs) as well as object-oriented high-level APIs to build and train neural networks. It also supports CUDA/cuDNN using CuPy for high performance training and inference. For more details about Chainer, see the documents and resources listed above and join the community in Forum, Slack, and Twitter.

Installation

For more details, see the installation guide.

To install Chainer, use pip.

$ pip install chainer

To enable CUDA support, CuPy is required. Refer to the CuPy installation guide.

Docker image

We are providing the official Docker image. This image supports nvidia-docker. Login to the environment with the following command, and run the Python interpreter to use Chainer with CUDA and cuDNN support.

$ nvidia-docker run -it chainer/chainer /bin/bash

Contribution

See the contribution guide.

ChainerX

See the ChainerX documentation.

License

MIT License (see LICENSE file).

More information

• Release notes

References

Tokui, Seiya, et al. "Chainer: A Deep Learning Framework for Accelerating the Research Cycle." Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 2019. URL BibTex

Tokui, S., Oono, K., Hido, S. and Clayton, J., Chainer: a Next-Generation Open Source Framework for Deep Learning, Proceedings of Workshop on Machine Learning Systems(LearningSys) in The Twenty-ninth Annual Conference on Neural Information Processing Systems (NIPS), (2015) URL, BibTex

Akiba, T., Fukuda, K. and Suzuki, S., ChainerMN: Scalable Distributed Deep Learning Framework, Proceedings of Workshop on ML Systems in The Thirty-first Annual Conference on Neural Information Processing Systems (NIPS), (2017) URL, BibTex

models's Issues

Add License

We need to add License in README.

Forward computation of BertModel returns an error

I was trying to run BERT like below

from tokenization import FullTokenizer
from modeling import BertModel, BertConfig

tokenizer = FullTokenizer(vocab_file='vocab.txt', do_lower_case=True)
tokens = tokenizer.tokenize('I hope Liverpool will win the Premier League this year.')
ids = tokenizer.convert_tokens_to_ids(tokens)
bert_config = BertConfig.from_json_file('bert_config.json')
bert = BertModel(bert_config)
batch = np.array([ids], dtype=np.int32)
output = bert.get_sequence_output(batch)

and got an error as follows.

...
~/nlp/nlp/pretrained/bert/modeling.py in __call__(self, input_tensor, attention_mask, do_return_all_layers)
    942                 from_tensor=layer_input, to_tensor=layer_input,
    943                 attention_mask=attention_mask,
--> 944                 do_return_2d_tensor=True)
    945 
    946             # Run a linear projection of `hidden_size` then add a residual

~/nlp/nlp/pretrained/bert/modeling.py in __call__(self, from_tensor, to_tensor, attention_mask, do_return_2d_tensor)
    779             # Since we are adding it to the raw scores before the softmax, this is
    780             # effectively the same as removing these entirely.
--> 781             attention_scores += F.broadcast_to(adder, attention_scores.shape)
    782 
    783         # Normalize the attention scores to probabilities.
...
InvalidType: 
Invalid operation is performed in: _ + _ (Forward)

Expect: lhs.dtype == rhs.dtype
Actual: float32 != float64

I realized that there is a conversion of dtype in Attention Layer in modeling.py

Questions regarding contribution to this repo

1. Is someone already working on the projects mentioned under 'in-progress' category. I know it sounds odd but I coded the Neural Representation of Sketch during an internship and thought of porting it to Chainer.

2. Can we add implementations from Visual Speech Recognition? or in general any other domain not listed ?

Can it be used in DDP?

Hi, I use the GardNorm in my segmentation and classification task. I want to use the DistributedDataParallel to train it. But it occurs the error: "RuntimeError: derivative for batch_norm_backward_elemt is not implemented". Can you give me some advice?

Add Contribution guide

We need to add a template of code and README.

Error in mdn example

Traceback (most recent call last):
  File "train.py", line 67, in <module>
    pred_y_data = model.sample(pred_x_data).array
  File "reproduction/models/mdn/mdn.py", line 70, in sample
    mu = F.get_item(mu, [range(n_batch), idx])
  File "python3.5/site-packages/chainer/functions/array/get_item.py", line 143, in get_item
    return GetItem(slices).apply((x,))[0]
  File "python3.5/site-packages/chainer/function_node.py", line 263, in apply
    outputs = self.forward(in_data)
  File "python3.5/site-packages/chainer/functions/array/get_item.py", line 41, in forward
    return utils.force_array(xs[0][self.slices]),
  File "cupy/core/core.pyx", line 1632, in cupy.core.core.ndarray.__getitem__
  File "cupy/core/core.pyx", line 3189, in cupy.core.core._prepare_advanced_indexing
IndexError: only integers, slices (`:`), ellipsis (`...`),numpy.newaxis (`None`) and integer or boolean arrays are valid indices

nncompress doesn't work on chainer 6.5.0

Hi,

"nncompress" doesn't work on chainer 6.5.0

in complessor/subfuncs.py,
def save_non_embed_npz(file, obj, compression=True):

Type of "obj" expected "complessor.net.EmbeddingCompressor", but "dict" on chainer 6.5.0 .
So I confront an error "AttributeError: 'dict' object has no attribute 'serialize'".