dongjunlee / conversation-tensorflow

TensorFlow implementation of Conversation Models

Hi there 👋

I am a machine learning engineer who wants to deliver valuable tech products to people.

• 🔭 I’m currently working on MLOps, NLP and Quantified Self.
• 📫 How to reach me:

Blogs (Korean)

• LLM 기반 Application, LBox AI 개발기
• 검색 시스템 톺아보기 - 1. 검색어 자동완성과 오타 교정 기능
• 코드로 이해하는 BentoML
• Legal-BERT, 법률 도메인에 특화된 언어모델 개발기
• Python 코드읽기 - 2. Flask
• Quantified Self Part 6 - 생산적인 하루에 대한 정량적인 표현과 4년간의 데이터 이야기

conversation-tensorflow's Issues

KeyError when training

Hi,

I downloaded your repo and followed all the instructions but I have a problem after running
python main.py --config cornell-movie-dialogs --mode train_and_evaluate.
I get that

  File "F:\conversation-tensorflow-master\model.py", line 23, in model_fn
    self.build_graph()
  File "F:\conversation-tensorflow-master\model.py", line 61, in build_graph
    self._build_encoder()
  File "F:\conversation-tensorflow-master\model.py", line 104, in _build_encoder
    beam_width = Config.predict.get('beam_width', 0)
  File "C:\Users\AppData\Local\Programs\Python\Python36\lib\site-packages\hbconfig\__init__.py", line 76, in __getattr__
    config_value = self.config[name]
KeyError: 'predict'

Any idea how this might be fixed?

I am using Python -3.6 andTensorflow -1.4

New error on train, possibly caused by beam-search-decoder bugfix

when running

python main.py --config cornell-movie-dialogs --mode train_and_evaluate

I get the error message

load vocab ...
vocab size: 41676
make Training data and Test data Start....
196665it [00:07, 26114.28it/s]
load data from train_ids.enc, train_ids.dec...
25009it [00:00, 27803.06it/s]
load data from test_ids.enc, test_ids.dec...
train data count : 89711
test data count : 11324
shuffle dataset ...
sorted by inputs length and outputs length ...
WARNING:tensorflow:RunConfig.uid (from tensorflow.contrib.learn.python.learn.estimators.run_config) is experimental and may change or be removed at any time, and without warning.
Traceback (most recent call last):
File "main.py", line 99, in
main(args.mode)
File "main.py", line 72, in main
hparams=params
File "/home/ubuntu/anaconda3/envs/python3/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/learn_runner.py", line 218, in run
return _execute_schedule(experiment, schedule)
File "/home/ubuntu/anaconda3/envs/python3/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/learn_runner.py", line 46, in _execute_schedule
return task()
File "/home/ubuntu/anaconda3/envs/python3/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/experiment.py", line 367, in train
hooks=self._train_monitors + extra_hooks)
File "/home/ubuntu/anaconda3/envs/python3/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/experiment.py", line 807, in _call_train
hooks=hooks)
File "/home/ubuntu/anaconda3/envs/python3/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 302, in train
loss = self._train_model(input_fn, hooks, saving_listeners)
File "/home/ubuntu/anaconda3/envs/python3/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 711, in _train_model
features, labels, model_fn_lib.ModeKeys.TRAIN, self.config)
File "/home/ubuntu/anaconda3/envs/python3/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 694, in _call_model_fn
model_fn_results = self._model_fn(features=features, **kwargs)
File "/home/ubuntu/conversation-tensorflow/model.py", line 23, in model_fn
self.build_graph()
File "/home/ubuntu/conversation-tensorflow/model.py", line 57, in build_graph
decoder_inputs=self.decoder_inputs)
File "/home/ubuntu/conversation-tensorflow/seq2seq_attention/init.py", line 37, in build
self._build_decoder()
File "/home/ubuntu/conversation-tensorflow/seq2seq_attention/init.py", line 115, in _build_decoder
self.decoder_logits = decoder_outputs.rnn_output
AttributeError: 'tuple' object has no attribute 'rnn_output'
The learning is finished with conversation-tensorflow Project using cornell-movie-dialogs config.

This didn't happen before, so it's likely associated with the fixes for #9

chat issue

Hi @DongjunLee ,
My stack comprises of -
Python -3.6
Tensorflow -1.4
hb-config - 0.3.4

I have gone through your code, i have pre processed the data and kept it for training. But, during chat
i used below command but i'm facing following error -
command : python chat.py --config check_tiny

load vocab ...

hi
Traceback (most recent call last):
File "/home/abhi/virtual-3.6/lib/python3.6/site-packages/tensorflow/python/framework/common_shapes.py", line 686, in _call_cpp_shape_fn_impl
input_tensors_as_shapes, status)
File "/home/abhi/virtual-3.6/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot reshape a tensor with 16 elements to shape [1,5,16] (80 elements) for 'decoder/decode/Reshape' (op: 'Reshape') with input shapes: [1,16], [3] and with input tensors computed as partial shapes: input[1] = [1,5,16].

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "chat.py", line 95, in
main()
File "chat.py", line 77, in main
answer = chat(ids, vocab)
File "chat.py", line 29, in chat
prediction = next(result)["prediction"]

File "/home/abhi/virtual-3.6/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 411, in predict
features, None, model_fn_lib.ModeKeys.PREDICT, self.config)
File "/home/abhi/virtual-3.6/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 694, in _call_model_fn
model_fn_results = self._model_fn(features=features, **kwargs)
File "/home/abhi/PycharmProjects/conversation-tensorflow-master/model.py", line 23, in model_fn
self.build_graph()
File "/home/abhi/PycharmProjects/conversation-tensorflow-master/model.py", line 63, in build_graph
self._build_decoder()
File "/home/abhi/PycharmProjects/conversation-tensorflow-master/model.py", line 184, in _build_decoder
self.decoder_pred_outputs = decode()
File "/home/abhi/PycharmProjects/conversation-tensorflow-master/model.py", line 145, in decode
length_penalty_weight=Config.predict.length_penalty_weight)
File "/home/abhi/virtual-3.6/lib/python3.6/site-packages/tensorflow/contrib/seq2seq/python/ops/beam_search_decoder.py", line 226, in init
initial_state, self._cell.state_size)
File "/home/abhi/virtual-3.6/lib/python3.6/site-packages/tensorflow/python/util/nest.py", line 413, in map_structure
structure[0], [func(*x) for x in entries])
File "/home/abhi/virtual-3.6/lib/python3.6/site-packages/tensorflow/python/util/nest.py", line 413, in
structure[0], [func(*x) for x in entries])
File "/home/abhi/virtual-3.6/lib/python3.6/site-packages/tensorflow/contrib/seq2seq/python/ops/beam_search_decoder.py", line 407, in _maybe_split_batch_beams
return self._split_batch_beams(t, s)
File "/home/abhi/virtual-3.6/lib/python3.6/site-packages/tensorflow/contrib/seq2seq/python/ops/beam_search_decoder.py", line 372, in _split_batch_beams
([self._batch_size, self._beam_width], t_shape[1:]), 0))
File "/home/abhi/virtual-3.6/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 3938, in reshape
"Reshape", tensor=tensor, shape=shape, name=name)
File "/home/abhi/virtual-3.6/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/abhi/virtual-3.6/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2958, in create_op
set_shapes_for_outputs(ret)
File "/home/abhi/virtual-3.6/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2209, in set_shapes_for_outputs
shapes = shape_func(op)
File "/home/abhi/virtual-3.6/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2159, in call_with_requiring
return call_cpp_shape_fn(op, require_shape_fn=True)
File "/home/abhi/virtual-3.6/lib/python3.6/site-packages/tensorflow/python/framework/common_shapes.py", line 627, in call_cpp_shape_fn
require_shape_fn)
File "/home/abhi/virtual-3.6/lib/python3.6/site-packages/tensorflow/python/framework/common_shapes.py", line 691, in _call_cpp_shape_fn_impl
raise ValueError(err.message)
ValueError: Cannot reshape a tensor with 16 elements to shape [1,5,16] (80 elements) for 'decoder/decode/Reshape' (op: 'Reshape') with input shapes: [1,16], [3] and with input tensors computed as partial shapes: input[1] = [1,5,16].

Please resolve this issue as soon as possible

adding stacked bidirectional

Error when run in tensorflow 1.6+

When I run your project in tf 1.6 or tf 1.7, I got an error:

TypeError: The two structures don't have the same sequence type. First structure has type <class 'tuple'>, while second structure has type <class 'tensorflow.python.ops.rnn_cell_impl.LSTMStateTuple'>.

This happend in decoder.py line: self.decoder_initial_state.clone(cell_state=encoder_final_state). Can you check it?

System configurations

HI Dongjunlee,
I am using a CPU system with 8 GB RAM for training, and i used cornell-movie-dialogues.yml configuration file, only 5000 training steps are completed since 5 days.

Can you please tell me your system configuration which you used for training and how much time taken for training of 1 lakh training steps.

Thanks in advance

Error during training (following default usage described in the readme.md)

Python 3.6.4
Tensorflow 1.4.0

After following the commands in the "Usage" section of the readme:
sh scripts/prepare_Cornell_Movie-Dialogs_Corpus.sh
python main.py --config cornell-movie-dialogs --mode train_and_evaluate

INFO:tensorflow:loss = 10.636566, step = 1
INFO:tensorflow:global_step/sec: 0.628262
INFO:tensorflow:loss = 5.791372, step = 101 (159.167 sec)
INFO:tensorflow:global_step/sec: 0.647979
INFO:tensorflow:loss = 5.864272, step = 201 (154.326 sec)
INFO:tensorflow:global_step/sec: 0.655491
INFO:tensorflow:loss = 6.230704, step = 301 (152.557 sec)
INFO:tensorflow:global_step/sec: 0.665124
INFO:tensorflow:loss = 5.784849, step = 401 (150.348 sec)
INFO:tensorflow:global_step/sec: 0.670939
INFO:tensorflow:loss = 6.017593, step = 501 (149.045 sec)
INFO:tensorflow:global_step/sec: 0.668164
INFO:tensorflow:loss = 5.5392556, step = 601 (149.664 sec)
INFO:tensorflow:global_step/sec: 0.645968
INFO:tensorflow:loss = 5.649448, step = 701 (154.806 sec)
INFO:tensorflow:global_step/sec: 0.646482
INFO:tensorflow:loss = 5.597257, step = 801 (154.683 sec)
INFO:tensorflow:global_step/sec: 0.674069
INFO:tensorflow:loss = 5.0681148, step = 901 (148.353 sec)
Traceback (most recent call last):
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/framework/common_shapes.py", line 686, in _call_cpp_shape_fn_impl
input_tensors_as_shapes, status)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Shapes must be equal rank, but are 3 and 2 for 'decoder/decoder/while/Select_4' (op: 'Select') with input shapes: [?,5], [?,5,1024], [?,5,1024].

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "main.py", line 98, in
main(args.mode)
File "main.py", line 71, in main
hparams=params
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/learn_runner.py", line 218, in run
return _execute_schedule(experiment, schedule)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/learn_runner.py", line 46, in _execute_schedule
return task()
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/experiment.py", line 625, in train_and_evaluate
self.train(delay_secs=0)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/experiment.py", line 367, in train
hooks=self._train_monitors + extra_hooks)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/experiment.py", line 807, in _call_train
hooks=hooks)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 302, in train
loss = self._train_model(input_fn, hooks, saving_listeners)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 783, in _train_model
_, loss = mon_sess.run([estimator_spec.train_op, estimator_spec.loss])
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 521, in run
run_metadata=run_metadata)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 892, in run
run_metadata=run_metadata)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 967, in run
raise six.reraise(*original_exc_info)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/six.py", line 693, in reraise
raise value
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 952, in run
return self._sess.run(*args, **kwargs)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1032, in run
run_metadata=run_metadata))
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/monitors.py", line 1196, in after_run
induce_stop = m.step_end(self._last_step, result)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/monitors.py", line 356, in step_end
return self.every_n_step_end(step, output)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/monitors.py", line 694, in every_n_step_end
validation_outputs = self._evaluate_estimator()
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/monitors.py", line 665, in _evaluate_estimator
name=self.name)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 355, in evaluate
name=name)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 810, in _evaluate_model
features, labels, model_fn_lib.ModeKeys.EVAL, self.config)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 694, in _call_model_fn
model_fn_results = self._model_fn(features=features, **kwargs)
File "/home/javier/repos/dialogue/conversation-tensorflow/model.py", line 23, in model_fn
self.build_graph()
File "/home/javier/repos/dialogue/conversation-tensorflow/model.py", line 57, in build_graph
decoder_inputs=self.decoder_inputs)
File "/home/javier/repos/dialogue/conversation-tensorflow/seq2seq_attention/init.py", line 37, in build
self._build_decoder()
File "/home/javier/repos/dialogue/conversation-tensorflow/seq2seq_attention/init.py", line 112, in _build_decoder
length_penalty_weight=Config.predict.length_penalty_weight)
File "/home/javier/repos/dialogue/conversation-tensorflow/seq2seq_attention/decoder.py", line 172, in build
embedding, start_tokens, end_token, length_penalty_weight)
File "/home/javier/repos/dialogue/conversation-tensorflow/seq2seq_attention/decoder.py", line 215, in _beam_search_decoder
maximum_iterations=self.maximum_iterations)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/contrib/seq2seq/python/ops/decoder.py", line 286, in dynamic_decode
swap_memory=swap_memory)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2816, in while_loop
result = loop_context.BuildLoop(cond, body, loop_vars, shape_invariants)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2640, in BuildLoop
pred, body, original_loop_vars, loop_vars, shape_invariants)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2590, in _BuildLoop
body_result = body(*packed_vars_for_body)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/contrib/seq2seq/python/ops/decoder.py", line 269, in body
_maybe_copy_state, decoder_state, state)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/util/nest.py", line 413, in map_structure
structure[0], [func(*x) for x in entries])
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/util/nest.py", line 413, in
structure[0], [func(*x) for x in entries])
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/contrib/seq2seq/python/ops/decoder.py", line 265, in _maybe_copy_state
return new if pass_through else array_ops.where(finished, cur, new)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 2441, in where
return gen_math_ops._select(condition=condition, t=x, e=y, name=name)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py", line 3988, in _select
"Select", condition=condition, t=t, e=e, name=name)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2958, in create_op
set_shapes_for_outputs(ret)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2209, in set_shapes_for_outputs
shapes = shape_func(op)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2159, in call_with_requiring
return call_cpp_shape_fn(op, require_shape_fn=True)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/framework/common_shapes.py", line 627, in call_cpp_shape_fn
require_shape_fn)
File "/home/javier/repos/dialogue/conversation-tensorflow/venv/lib/python3.6/site-packages/tensorflow/python/framework/common_shapes.py", line 691, in _call_cpp_shape_fn_impl
raise ValueError(err.message)
ValueError: Shapes must be equal rank, but are 3 and 2 for 'decoder/decoder/while/Select_4' (op: 'Select') with input shapes: [?,5], [?,5,1024], [?,5,1024].
The learning is finished with conversation-tensorflow Project using cornell-movie-dialogs config.

What might be wrong?

Is it possible to use customer service conversation as data set with this model ?

Hi,

Is it possible to use customer service conversation as data set with this model ? such as

Hello
May I help you ?
I want to ask something
Yes please
Does my booking refundable ?
May I get your booking ID.
1234567890
Yes, your booking is refundable.
I want to submit a refund for my booking.
...
...
...

ValueError: Expected state to be a tuple of length 1, but received: Tensor("decoder/decoder/while/Identity_3:0", shape=(?, 1024), dtype=float32)

I'm just trying to train the model on my own dataset and I keep getting this error:

Traceback (most recent call last):
File "train.py", line 95, in
main(args.mode)
File "train.py", line 72, in main
hparams=params
File "/home/lair06/anaconda3/envs/tensorflow1.4/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/learn_runner.py", line 218, in run
return _execute_schedule(experiment, schedule)
File "/home/lair06/anaconda3/envs/tensorflow1.4/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/learn_runner.py", line 46, in _execute_schedule
return task()
File "/home/lair06/anaconda3/envs/tensorflow1.4/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/experiment.py", line 367, in train
hooks=self._train_monitors + extra_hooks)
File "/home/lair06/anaconda3/envs/tensorflow1.4/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/experiment.py", line 807, in _call_train
hooks=hooks)
File "/home/lair06/anaconda3/envs/tensorflow1.4/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 302, in train
loss = self._train_model(input_fn, hooks, saving_listeners)
File "/home/lair06/anaconda3/envs/tensorflow1.4/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 711, in _train_model
features, labels, model_fn_lib.ModeKeys.TRAIN, self.config)
File "/home/lair06/anaconda3/envs/tensorflow1.4/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 694, in _call_model_fn
model_fn_results = self._model_fn(features=features, **kwargs)
File "/home/lair06/peterkin/CSE843_FinalProject/model.py", line 19, in model_fn
self.build_graph()
File "/home/lair06/peterkin/CSE843_FinalProject/model.py", line 53, in build_graph
decoder_inputs=self.decoder_inputs)
File "/home/lair06/peterkin/CSE843_FinalProject/seq2seq_attention/init.py", line 33, in build
self._build_decoder()
File "/home/lair06/peterkin/CSE843_FinalProject/seq2seq_attention/init.py", line 108, in _build_decoder
length_penalty_weight=Config.predict.length_penalty_weight)
File "/home/lair06/peterkin/CSE843_FinalProject/seq2seq_attention/decoder.py", line 145, in build
return self._basic_decoder(helper)
File "/home/lair06/peterkin/CSE843_FinalProject/seq2seq_attention/decoder.py", line 173, in _basic_decoder
swap_memory=True)
File "/home/lair06/anaconda3/envs/tensorflow1.4/lib/python3.6/site-packages/tensorflow/contrib/seq2seq/python/ops/decoder.py", line 286, in dynamic_decode
swap_memory=swap_memory)
File "/home/lair06/anaconda3/envs/tensorflow1.4/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2816, in while_loop
result = loop_context.BuildLoop(cond, body, loop_vars, shape_invariants)
File "/home/lair06/anaconda3/envs/tensorflow1.4/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2640, in BuildLoop
pred, body, original_loop_vars, loop_vars, shape_invariants)
File "/home/lair06/anaconda3/envs/tensorflow1.4/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2590, in _BuildLoop
body_result = body(*packed_vars_for_body)
File "/home/lair06/anaconda3/envs/tensorflow1.4/lib/python3.6/site-packages/tensorflow/contrib/seq2seq/python/ops/decoder.py", line 234, in body
decoder_finished) = decoder.step(time, inputs, state)
File "/home/lair06/anaconda3/envs/tensorflow1.4/lib/python3.6/site-packages/tensorflow/contrib/seq2seq/python/ops/basic_decoder.py", line 138, in step
cell_outputs, cell_state = self._cell(inputs, state)
File "/home/lair06/anaconda3/envs/tensorflow1.4/lib/python3.6/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 183, in call
return super(RNNCell, self).call(inputs, state)
File "/home/lair06/anaconda3/envs/tensorflow1.4/lib/python3.6/site-packages/tensorflow/python/layers/base.py", line 575, in call
outputs = self.call(inputs, *args, **kwargs)
File "/home/lair06/anaconda3/envs/tensorflow1.4/lib/python3.6/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn_cell.py", line 230, in call
output, res_state = self._cell(inputs, state)
File "/home/lair06/anaconda3/envs/tensorflow1.4/lib/python3.6/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 183, in call
return super(RNNCell, self).call(inputs, state)
File "/home/lair06/anaconda3/envs/tensorflow1.4/lib/python3.6/site-packages/tensorflow/python/layers/base.py", line 575, in call
outputs = self.call(inputs, *args, **kwargs)
File "/home/lair06/anaconda3/envs/tensorflow1.4/lib/python3.6/site-packages/tensorflow/contrib/seq2seq/python/ops/attention_wrapper.py", line 1295, in call
cell_output, next_cell_state = self._cell(cell_inputs, cell_state)
File "/home/lair06/anaconda3/envs/tensorflow1.4/lib/python3.6/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 183, in call
return super(RNNCell, self).call(inputs, state)
File "/home/lair06/anaconda3/envs/tensorflow1.4/lib/python3.6/site-packages/tensorflow/python/layers/base.py", line 575, in call
outputs = self.call(inputs, *args, **kwargs)
File "/home/lair06/anaconda3/envs/tensorflow1.4/lib/python3.6/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 1060, in call
(len(self.state_size), state))
ValueError: Expected state to be a tuple of length 1, but received: Tensor("decoder/decoder/while/Identity_3:0", shape=(?, 1024), dtype=float32)

When trying to track down the issue it seems that this problem is in the decoder specifically with setting the initial state. And that's about it. Any ideas what's going wrong?

Significant overfit on default hyperparameters on cornell-movie-dialogs config

Running:
python main.py --config cornell-movie-dialogs --mode train

to the end (100000 steps) will result in a training loss of about 2.6, test loss of 8.4.

Which hyperparameters did you use? The resulting chatbot doesn't work very well (the one in your readme is a lot better).

Thank you!

在你的代码中为什么直接去掉左边一个字符，这个有什么讲究

在model.py文件中，对targets的处理你的代码是这样的：

        self.decoder_inputs = labels
        decoder_input_shift_1 = tf.slice(self.decoder_inputs, [0, 1],
                [batch_size, Config.data.max_seq_length-1])
        pad_tokens = tf.zeros([batch_size, 1], dtype=tf.int32)

        # make target (right shift 1 from decoder_inputs)
        self.targets = tf.concat([decoder_input_shift_1, pad_tokens], axis=1)

我看了一些其他人的代码，有些对self.targets直接用label，有些用以下代码，即左边增加标识符号，右边去掉最后一个字符，这个有啥讲究吗？
其他人的代码：

        self.decoder_inputs = labels
        # 补充<SOS>，并移除最后一个字符       
        decoder_input_shift_1 = tf.strided_slice(self.decoder_inputs, [0, 0], [batch_size, -1], [1, 1])
        self.targets = tf.concat([tf.fill([batch_size, 1], self.data.PAD_ID), decoder_input_shift_1], 1)