Comments (2)
kws-slalom
commented on May 16, 2024
Also, FYI. I'm using AWS Sagemaker tensorflow_p27 premade conda instance with bert-tensorflow and natural-questions installed.
from language.
kws-slalom
commented on May 16, 2024
I ran it with export AUTOGRAPH_VERBOSITY=10 as suggested in the output above which provides a more detailed log but still not really sure what to try next:
INFO:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bcd4ca10>> is not cached for key <code object call at 0x7f35d4627130, file "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/keras/layers/core.py", line 1031> subkey (<tensorflow.python.autograph.core.converter.ConversionOptions object at 0x7f35bcd73f90>, frozenset([]))
I1223 17:22:02.250940 139869963634496 ag_logging.py:139] Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bcd4ca10>> is not cached for key <code object call at 0x7f35d4627130, file "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/keras/layers/core.py", line 1031> subkey (<tensorflow.python.autograph.core.converter.ConversionOptions object at 0x7f35bcd73f90>, frozenset([]))
INFO:tensorflow:Converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bcd4ca10>>
I1223 17:22:02.251044 139869963634496 ag_logging.py:139] Converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bcd4ca10>>
INFO:tensorflow:Source code of <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bcd4ca10>>:
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
def call(self, inputs):
inputs = ops.convert_to_tensor(inputs)
rank = common_shapes.rank(inputs)
if rank > 2:
# Broadcasting is required for the inputs.
outputs = standard_ops.tensordot(inputs, self.kernel, [[rank - 1], [0]])
# Reshape the output back to the original ndim of the input.
if not context.executing_eagerly():
shape = inputs.shape.as_list()
output_shape = shape[:-1] + [self.units]
outputs.set_shape(output_shape)
else:
# Cast the inputs to self.dtype, which is the variable dtype. We do not
# cast if `should_cast_variables` is True, as in that case the variable
# will be automatically casted to inputs.dtype.
if not self._mixed_precision_policy.should_cast_variables:
inputs = math_ops.cast(inputs, self.dtype)
outputs = gen_math_ops.mat_mul(inputs, self.kernel)
if self.use_bias:
outputs = nn.bias_add(outputs, self.bias)
if self.activation is not None:
return self.activation(outputs) # pylint: disable=not-callable
return outputs
I1223 17:22:02.258529 139869963634496 ag_logging.py:139] Source code of <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bcd4ca10>>:
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
def call(self, inputs):
inputs = ops.convert_to_tensor(inputs)
rank = common_shapes.rank(inputs)
if rank > 2:
# Broadcasting is required for the inputs.
outputs = standard_ops.tensordot(inputs, self.kernel, [[rank - 1], [0]])
# Reshape the output back to the original ndim of the input.
if not context.executing_eagerly():
shape = inputs.shape.as_list()
output_shape = shape[:-1] + [self.units]
outputs.set_shape(output_shape)
else:
# Cast the inputs to self.dtype, which is the variable dtype. We do not
# cast if `should_cast_variables` is True, as in that case the variable
# will be automatically casted to inputs.dtype.
if not self._mixed_precision_policy.should_cast_variables:
inputs = math_ops.cast(inputs, self.dtype)
outputs = gen_math_ops.mat_mul(inputs, self.kernel)
if self.use_bias:
outputs = nn.bias_add(outputs, self.bias)
if self.activation is not None:
return self.activation(outputs) # pylint: disable=not-callable
return outputs
ERROR:tensorflow:Error converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bcd4ca10>>
Traceback (most recent call last):
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/api.py", line 524, in to_graph
return conversion.convert(entity, program_ctx)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/conversion.py", line 306, in convert
entity, program_ctx, free_nonglobal_var_names)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/conversion.py", line 229, in _convert_with_cache
entity, program_ctx)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/conversion.py", line 433, in convert_entity_to_ast
nodes, name, entity_info = convert_func_to_ast(o, program_ctx)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/conversion.py", line 624, in convert_func_to_ast
node = node_to_graph(node, context)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/conversion.py", line 667, in node_to_graph
node = converter.apply_(node, context, return_statements)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/core/converter.py", line 380, in apply_
node = converter_module.transform(node, context)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/converters/return_statements.py", line 412, in transform
node = transformer.visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/core/converter.py", line 317, in visit
return super(Base, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/transformer.py", line 480, in visit
result = super(Base, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/ast.py", line 241, in visit
return visitor(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/converters/return_statements.py", line 363, in visit_FunctionDef
converted_body = self._visit_statement_block(node, node.body)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/converters/return_statements.py", line 287, in _visit_statement_block
nodes = self.visit_block(nodes, after_visit=self._postprocess_statement)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/transformer.py", line 371, in visit_block
replacement = self.visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/core/converter.py", line 317, in visit
return super(Base, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/transformer.py", line 480, in visit
result = super(Base, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/ast.py", line 241, in visit
return visitor(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/converters/return_statements.py", line 348, in visit_If
node.body = self._visit_statement_block(node, node.body)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/converters/return_statements.py", line 287, in _visit_statement_block
nodes = self.visit_block(nodes, after_visit=self._postprocess_statement)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/transformer.py", line 371, in visit_block
replacement = self.visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/core/converter.py", line 317, in visit
return super(Base, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/transformer.py", line 480, in visit
result = super(Base, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/ast.py", line 241, in visit
return visitor(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/converters/return_statements.py", line 237, in visit_Return
retval=retval)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/templates.py", line 260, in replace
replacements[k] = _convert_to_ast(replacements[k])
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/templates.py", line 222, in _convert_to_ast
return gast.Name(id=n, ctx=None, annotation=None)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/gast/gast.py", line 19, in create_node
format(Name, nbparam, len(Fields))
AssertionError: Bad argument number for Name: 3, expecting 4
E1223 17:22:02.355459 139869963634496 ag_logging.py:132] Error converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bcd4ca10>>
Traceback (most recent call last):
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/api.py", line 524, in to_graph
return conversion.convert(entity, program_ctx)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/conversion.py", line 306, in convert
entity, program_ctx, free_nonglobal_var_names)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/conversion.py", line 229, in _convert_with_cache
entity, program_ctx)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/conversion.py", line 433, in convert_entity_to_ast
nodes, name, entity_info = convert_func_to_ast(o, program_ctx)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/conversion.py", line 624, in convert_func_to_ast
node = node_to_graph(node, context)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/conversion.py", line 667, in node_to_graph
node = converter.apply_(node, context, return_statements)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/core/converter.py", line 380, in apply_
node = converter_module.transform(node, context)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/converters/return_statements.py", line 412, in transform
node = transformer.visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/core/converter.py", line 317, in visit
return super(Base, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/transformer.py", line 480, in visit
result = super(Base, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/ast.py", line 241, in visit
return visitor(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/converters/return_statements.py", line 363, in visit_FunctionDef
converted_body = self._visit_statement_block(node, node.body)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/converters/return_statements.py", line 287, in _visit_statement_block
nodes = self.visit_block(nodes, after_visit=self._postprocess_statement)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/transformer.py", line 371, in visit_block
replacement = self.visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/core/converter.py", line 317, in visit
return super(Base, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/transformer.py", line 480, in visit
result = super(Base, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/ast.py", line 241, in visit
return visitor(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/converters/return_statements.py", line 348, in visit_If
node.body = self._visit_statement_block(node, node.body)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/converters/return_statements.py", line 287, in _visit_statement_block
nodes = self.visit_block(nodes, after_visit=self._postprocess_statement)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/transformer.py", line 371, in visit_block
replacement = self.visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/core/converter.py", line 317, in visit
return super(Base, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/transformer.py", line 480, in visit
result = super(Base, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/ast.py", line 241, in visit
return visitor(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/converters/return_statements.py", line 237, in visit_Return
retval=retval)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/templates.py", line 260, in replace
replacements[k] = _convert_to_ast(replacements[k])
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/templates.py", line 222, in _convert_to_ast
return gast.Name(id=n, ctx=None, annotation=None)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/gast/gast.py", line 19, in create_node
format(Name, nbparam, len(Fields))
AssertionError: Bad argument number for Name: 3, expecting 4
INFO:tensorflow:Error transforming entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bcd4ca10>>
Traceback (most recent call last):
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/api.py", line 423, in converted_call
experimental_optional_features=options.optional_features)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/api.py", line 528, in to_graph
entity, e.__class__.__name__, str(e)))
ConversionError: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bcd4ca10>>: AssertionError: Bad argument number for Name: 3, expecting 4
I1223 17:22:02.356198 139869963634496 ag_logging.py:139] Error transforming entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bcd4ca10>>
Traceback (most recent call last):
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/api.py", line 423, in converted_call
experimental_optional_features=options.optional_features)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/api.py", line 528, in to_graph
entity, e.__class__.__name__, str(e)))
ConversionError: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bcd4ca10>>: AssertionError: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bcd4ca10>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bcd4ca10>>: AssertionError: Bad argument number for Name: 3, expecting 4
W1223 17:22:02.356378 139869963634496 ag_logging.py:145] Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bcd4ca10>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bcd4ca10>>: AssertionError: Bad argument number for Name: 3, expecting 4
INFO:tensorflow:Converted call: <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bd196c10>>
args: (<tf.Tensor 'bert/encoder/layer_19/attention/self/Reshape_3:0' shape=(?, 1024) dtype=float32>,)
kwargs: {}
I1223 17:22:02.388948 139869963634496 ag_logging.py:139] Converted call: <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bd196c10>>
args: (<tf.Tensor 'bert/encoder/layer_19/attention/self/Reshape_3:0' shape=(?, 1024) dtype=float32>,)
kwargs: {}
INFO:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bd196c10>> is not cached for key <code object call at 0x7f35d4627130, file "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/keras/layers/core.py", line 1031> subkey (<tensorflow.python.autograph.core.converter.ConversionOptions object at 0x7f35bcc96310>, frozenset([]))
I1223 17:22:02.389297 139869963634496 ag_logging.py:139] Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bd196c10>> is not cached for key <code object call at 0x7f35d4627130, file "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/keras/layers/core.py", line 1031> subkey (<tensorflow.python.autograph.core.converter.ConversionOptions object at 0x7f35bcc96310>, frozenset([]))
INFO:tensorflow:Converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bd196c10>>
I1223 17:22:02.389405 139869963634496 ag_logging.py:139] Converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bd196c10>>
INFO:tensorflow:Source code of <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bd196c10>>:
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
def call(self, inputs):
inputs = ops.convert_to_tensor(inputs)
rank = common_shapes.rank(inputs)
if rank > 2:
# Broadcasting is required for the inputs.
outputs = standard_ops.tensordot(inputs, self.kernel, [[rank - 1], [0]])
# Reshape the output back to the original ndim of the input.
if not context.executing_eagerly():
shape = inputs.shape.as_list()
output_shape = shape[:-1] + [self.units]
outputs.set_shape(output_shape)
else:
# Cast the inputs to self.dtype, which is the variable dtype. We do not
# cast if `should_cast_variables` is True, as in that case the variable
# will be automatically casted to inputs.dtype.
if not self._mixed_precision_policy.should_cast_variables:
inputs = math_ops.cast(inputs, self.dtype)
outputs = gen_math_ops.mat_mul(inputs, self.kernel)
if self.use_bias:
outputs = nn.bias_add(outputs, self.bias)
if self.activation is not None:
return self.activation(outputs) # pylint: disable=not-callable
return outputs
I1223 17:22:02.397010 139869963634496 ag_logging.py:139] Source code of <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f35bd196c10>>:
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
def call(self, inputs):
inputs = ops.convert_to_tensor(inputs)
rank = common_shapes.rank(inputs)
if rank > 2:
# Broadcasting is required for the inputs.
outputs = standard_ops.tensordot(inputs, self.kernel, [[rank - 1], [0]])
# Reshape the output back to the original ndim of the input.
if not context.executing_eagerly():
shape = inputs.shape.as_list()
output_shape = shape[:-1] + [self.units]
outputs.set_shape(output_shape)
else:
# Cast the inputs to self.dtype, which is the variable dtype. We do not
# cast if `should_cast_variables` is True, as in that case the variable
# will be automatically casted to inputs.dtype.
if not self._mixed_precision_policy.should_cast_variables:
inputs = math_ops.cast(inputs, self.dtype)
outputs = gen_math_ops.mat_mul(inputs, self.kernel)
if self.use_bias:
outputs = nn.bias_add(outputs, self.bias)
if self.activation is not None:
return self.activation(outputs) # pylint: disable=not-callable
return outputs
^CINFO:tensorflow:prediction_loop marked as finished
I1223 17:22:02.425369 139869963634496 error_handling.py:96] prediction_loop marked as finished
Traceback (most recent call last):
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/home/ec2-user/SageMaker/language/language/question_answering/bert_joint/run_nq.py", line 1442, in <module>
tf.app.run()
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/absl/app.py", line 299, in run
_run_main(main, args)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/absl/app.py", line 250, in _run_main
sys.exit(main(argv))
File "/home/ec2-user/SageMaker/language/language/question_answering/bert_joint/run_nq.py", line 1412, in main
predict_input_fn, yield_single_examples=True):
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 2913, in predict
yield_single_examples=yield_single_examples):
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 619, in predict
features, None, ModeKeys.PREDICT, self.config)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 2709, in _call_model_fn
config)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1146, in _call_model_fn
model_fn_results = self._model_fn(features=features, **kwargs)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 2967, in _model_fn
features, labels, is_export_mode=is_export_mode)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 1549, in call_without_tpu
return self._call_model_fn(features, labels, is_export_mode=is_export_mode)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 1867, in _call_model_fn
estimator_spec = self._model_fn(features=features, **kwargs)
File "/home/ec2-user/SageMaker/language/language/question_answering/bert_joint/run_nq.py", line 920, in model_fn
use_one_hot_embeddings=use_one_hot_embeddings)
File "/home/ec2-user/SageMaker/language/language/question_answering/bert_joint/run_nq.py", line 845, in create_model
use_one_hot_embeddings=use_one_hot_embeddings)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/bert/modeling.py", line 216, in __init__
do_return_all_layers=True)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/bert/modeling.py", line 861, in transformer_model
kernel_initializer=create_initializer(initializer_range))
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/util/deprecation.py", line 324, in new_func
return func(*args, **kwargs)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/layers/core.py", line 188, in dense
return layer.apply(inputs)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1481, in apply
return self.__call__(inputs, *args, **kwargs)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/layers/base.py", line 537, in __call__
outputs = super(Layer, self).__call__(inputs, *args, **kwargs)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 636, in __call__
outputs = call_fn(inputs, *args, **kwargs)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/api.py", line 146, in wrapper
), args, kwargs)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/api.py", line 423, in converted_call
experimental_optional_features=options.optional_features)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/api.py", line 524, in to_graph
return conversion.convert(entity, program_ctx)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/conversion.py", line 306, in convert
entity, program_ctx, free_nonglobal_var_names)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/conversion.py", line 229, in _convert_with_cache
entity, program_ctx)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/conversion.py", line 433, in convert_entity_to_ast
nodes, name, entity_info = convert_func_to_ast(o, program_ctx)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/conversion.py", line 624, in convert_func_to_ast
node = node_to_graph(node, context)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/impl/conversion.py", line 658, in node_to_graph
node = converter.apply_(node, context, arg_defaults)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/core/converter.py", line 379, in apply_
node = standard_analysis(node, context)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/core/converter.py", line 356, in standard_analysis
node = reaching_definitions.resolve(node, context, graphs, AnnotatedDef)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/static_analysis/reaching_definitions.py", line 312, in resolve
node = visitor.visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/static_analysis/reaching_definitions.py", line 294, in visit
node = super(TreeAnnotator, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/transformer.py", line 480, in visit
result = super(Base, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/ast.py", line 241, in visit
return visitor(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/static_analysis/reaching_definitions.py", line 214, in visit_FunctionDef
node.body = self.visit_block(node.body)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/transformer.py", line 371, in visit_block
replacement = self.visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/static_analysis/reaching_definitions.py", line 294, in visit
node = super(TreeAnnotator, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/transformer.py", line 480, in visit
result = super(Base, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/ast.py", line 241, in visit
return visitor(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/static_analysis/reaching_definitions.py", line 256, in visit_If
return self.generic_visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/ast.py", line 297, in generic_visit
value = self.visit(value)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/static_analysis/reaching_definitions.py", line 294, in visit
node = super(TreeAnnotator, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/transformer.py", line 480, in visit
result = super(Base, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/ast.py", line 241, in visit
return visitor(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/static_analysis/reaching_definitions.py", line 256, in visit_If
return self.generic_visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/ast.py", line 306, in generic_visit
new_node = self.visit(old_value)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/static_analysis/reaching_definitions.py", line 294, in visit
node = super(TreeAnnotator, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/transformer.py", line 480, in visit
result = super(Base, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/ast.py", line 241, in visit
return visitor(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/ast.py", line 306, in generic_visit
new_node = self.visit(old_value)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/static_analysis/reaching_definitions.py", line 294, in visit
node = super(TreeAnnotator, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/transformer.py", line 480, in visit
result = super(Base, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/ast.py", line 241, in visit
return visitor(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/ast.py", line 306, in generic_visit
new_node = self.visit(old_value)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/static_analysis/reaching_definitions.py", line 294, in visit
node = super(TreeAnnotator, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/autograph/pyct/transformer.py", line 480, in visit
result = super(Base, self).visit(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/ast.py", line 241, in visit
return visitor(node)
File "/home/ec2-user/anaconda3/envs/tensorflow_p27/lib/python2.7/ast.py", line 290, in generic_visit
def generic_visit(self, node):
KeyboardInterrupt
(tensorflow_p27) sh-4.2$
from language.
Related Issues (20)
- NQ Tiny Dev BucketNotFoundException
- Chines
- BERT EXTRACTION: Unable to get the result
- Benchmarking performance on TempLAMA and CustomNews
- XSP data downloading: no access
- The problem of obtaining the FRUIT dataset.
- The work: RGF: Retrieval-guided Counterfactual Generation for QA is so abstractive, is it convenient for you to update the code? Thank you so much! HOT 2
- Realm usage
- [NQG] AssertionError: Found 392 Python objects that were not bound to checkpointed values
- Is it convenient for you to share source codes of FROST?
- OSError: [E053] Could not read config.cfg from en_core_web_md-3.0.0a1/config.cfg
- CANINE Pretraining Code (pt.2)
- Bard is repetitively and wilfully lie
- Help migrating away from ORES
- Inquiry about checkpoint of ORQA
- Practice
- Bard - inability to read, analyze, and summarize websites
- T
- ORQA fine tuning with REALM pre-training: is it possible to run the fine tuning on Google Colab GPU?
- Français
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from language.