shannonai / corefqa Goto Github PK

ERROR:tensorflow:Error recorded from training_loop: GetNext() failed because the iterator has not been initialized.
Ensure that you have run the initializer operation for this iterator before getting the next element.
	 [[node IteratorGetNext (defined at /home/shantanu/anaconda3/envs/corefqa/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py:1748) ]]

Original stack trace for 'IteratorGetNext':
  File "./run/run_mention_proposal.py", line 192, in <module>
    tf.app.run()
  File "/home/shantanu/anaconda3/envs/corefqa/lib/python3.7/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/home/shantanu/anaconda3/envs/corefqa/lib/python3.7/site-packages/absl/app.py", line 312, in run
    _run_main(main, args)
  File "/home/shantanu/anaconda3/envs/corefqa/lib/python3.7/site-packages/absl/app.py", line 258, in _run_main
    sys.exit(main(argv))
  File "./run/run_mention_proposal.py", line 129, in main
    window_size=model_config.window_size, max_num_mention=model_config.max_num_mention, is_training=True, drop_remainder=True), max_steps=num_train_steps)
  File "/home/shantanu/anaconda3/envs/corefqa/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 3030, in train
    saving_listeners=saving_listeners)

  ValueError                                Traceback (most recent call last)
  [/content/drive/MyDrive/corefQA/code/run/run_mention_proposal.py](https://localhost:8080/#) in <module>()
      190     tf.set_random_seed(FLAGS.seed)
      191     # start train/evaluate the model.
  --> 192     tf.app.run()
      193 
      194 
  
  35 frames
  [/tensorflow-1.15.2/python3.7/tensorflow_core/python/ops/resource_variable_ops.py](https://localhost:8080/#) in _init_from_args(self, initial_value, trainable, collections, caching_device, name, dtype, constraint, synchronization, aggregation, distribute_strategy, shape)
     1558               "construct, such as a loop or conditional. When creating a "
     1559               "variable inside a loop or conditional, use a lambda as the "
  -> 1560               "initializer." % name)
     1561         # pylint: enable=protected-access
     1562         dtype = initial_value.dtype.base_dtype
  
  ValueError: Initializer for variable Variable/ is from inside a control-flow construct, such as a loop or conditional. When creating a variable inside a loop or conditional, use a lambda as the initializer

if train_mention_proposal:
    DATA_DIR = GS_SEMVEVAL_TRFILES
    OUTPUT_DIR = f"{GS_PATH}/models/mention_proposal"
    PRETRAINED_MODEL = GS_SQUAD2_ES_TRAINED_MODEL
  
    INIT_CHECKPOINT=f"{PRETRAINED_MODEL}/model.ckpt" 
  
    %cd {REPO_PATH}

    %run run/run_mention_proposal.py \
      --output_dir=$OUTPUT_DIR \
      --bert_config_file=$BERT_CONFIG \
      --init_checkpoint=$INIT_CHECKPOINT \
      --vocab_file=$BERT_VOCAB \
      --logfile_path=./train_mention_proposal.log \
      --num_epochs=8 \
      --keep_checkpoint_max=50 \
      --save_checkpoints_steps=500 \
      --train_file=$DATA_DIR/train.overlap.corefqa.es.tfrecord \
      --dev_file=$DATA_DIR/dev.overlap.corefqa.es.tfrecord \
      --test_file=$DATA_DIR/test.overlap.corefqa.es.tfrecord \
      --do_train=True \
      --do_eval=False \
      --do_predict=False \
      --learning_rate=1e-5 \
      --dropout_rate=0.2 \
      --mention_threshold=0.5 \
      --hidden_size=1024 \
      --num_docs=5604 \
      --window_size=384 \
      --num_window=6 \
      --max_num_mention=60 \
      --start_end_share=False \
      --loss_start_ratio=0.3 \
      --loss_end_ratio=0.3 \
      --loss_span_ratio=0.3 \
      --use_tpu=True \
      --tpu_name=$TPU_NAME \
      --seed=2333