buriburisuri / bytenet

A tensorflow implementation of French-to-English machine translation using DeepMind's ByteNet .

Hi there 👋

My name is Namjoo Kim, and I am the CEO of QUARKONIX PTE.,LTD. in Singapore.

• 😄 I was the leader of Klaybay CIC at Krust in Seoul, Korea.
• 🔭 I was the former CTO of Kakao brain.
• 🌱 I was currently learning Javascript, python, Solidity language.
• 💙 I love collaborating with developers, engineers and researchers.
• 😄 My current interest is Blockchain and Machine learning.

bytenet's Issues

validation loss computation with sg_reuse()

I'm interested in monitoring validation loss during training. I tried to use sg_reuse() function as in the examples of your sugartensor repository. But it seems a little difficult to do this in Bytenet, maybe due to the concatenation (enc = enc.sg_concat(target=y_src.sg_lookup(emb=emb_y))) ?

Could you point me a way to calculation validation loss in Bytenet?

Many Thanks.

Checkpoint file missing: /model.ckpt-22512.data-00000-of-00001

Hi, I tried to load the pre-trained model with your code "translate.py", however, there is an error saying:
"
2018-03-27 13:20:54.342537: W tensorflow/core/framework/op_kernel.cc:1192] Not found: asset/train/model.ckpt-22512.data-00000-of-00001; No such file or directory
"
Is it possible to also upload the .data files? Thanks!

How to process the word which is consist of wrong character?

Implementation of dynamic unfolding & bucketed batching

Thanks for the implementation, looks really clean and simple!

I noticed thought that the dynamic unfolding is not quite implemented. There's a max_length parameter, but it's of a fixed size, in the paper they seem to determine it dynamically and use bucketing for more efficient batching. Do you plan to implement this at some point? I might work on that, can reach out if I finish the implementation

Entire NLTK dataset inference.

It would be of great help if anyone suggest how to perform inference on entire NLTK dataset.

Train own dataset

--

Implementation of beam search

I was wondering if you plan to implement beam search for generation of translation candidates, as described here:

Alternatively I would be happy to implement this if you can offer some guidance or good references of best approach to do it.