Error while running : estimator.fit(inputs=data_channels)

Below is the issue:

2019-12-11 12:51:36 Starting - Starting the training job...
2019-12-11 12:51:39 Starting - Launching requested ML instances......
2019-12-11 12:52:37 Starting - Preparing the instances for training......
2019-12-11 12:53:52 Downloading - Downloading input data...
2019-12-11 12:54:19 Training - Downloading the training image..Arguments: train
[12/11/2019 12:54:41 INFO 140441326479168] Reading default configuration from /opt/amazon/lib/python2.7/site-packages/algorithm/resources/default-input.json: {u'num_dynamic_feat': u'auto', u'dropout_rate': u'0.10', u'mini_batch_size': u'128', u'test_quantiles': u'[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]', u'_tuning_objective_metric': u'', u'_num_gpus': u'auto', u'num_eval_samples': u'100', u'learning_rate': u'0.001', u'num_cells': u'40', u'num_layers': u'2', u'embedding_dimension': u'10', u'_kvstore': u'auto', u'_num_kv_servers': u'auto', u'cardinality': u'auto', u'likelihood': u'student-t', u'early_stopping_patience': u''}
[12/11/2019 12:54:41 INFO 140441326479168] Reading provided configuration from /opt/ml/input/config/hyperparameters.json: {u'dropout_rate': u'0.05', u'learning_rate': u'0.00001', u'num_cells': u'40', u'prediction_length': u'30', u'epochs': u'250', u'time_freq': u'D', u'context_length': u'30', u'num_layers': u'2', u'mini_batch_size': u'32', u'likelihood': u'gaussian', u'early_stopping_patience': u'10'}
[12/11/2019 12:54:41 INFO 140441326479168] Final configuration: {u'dropout_rate': u'0.05', u'test_quantiles': u'[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]', u'_tuning_objective_metric': u'', u'num_eval_samples': u'100', u'learning_rate': u'0.00001', u'num_layers': u'2', u'epochs': u'250', u'embedding_dimension': u'10', u'num_cells': u'40', u'_num_kv_servers': u'auto', u'mini_batch_size': u'32', u'likelihood': u'gaussian', u'num_dynamic_feat': u'auto', u'cardinality': u'auto', u'_num_gpus': u'auto', u'prediction_length': u'30', u'time_freq': u'D', u'context_length': u'30', u'_kvstore': u'auto', u'early_stopping_patience': u'10'}
Process 1 is a worker.
[12/11/2019 12:54:41 INFO 140441326479168] Detected entry point for worker worker
[12/11/2019 12:54:42 INFO 140441326479168] Using early stopping with patience 10
[12/11/2019 12:54:42 INFO 140441326479168] [cardinality=auto] cat field was NOT found in the file /opt/ml/input/data/train/deepar_training.json and will NOT be used for training.
[12/11/2019 12:54:42 INFO 140441326479168] [num_dynamic_feat=auto] dynamic_feat field was NOT found in the file /opt/ml/input/data/train/deepar_training.json and will NOT be used for training.
[12/11/2019 12:54:42 ERROR 140441326479168] Customer Error: Error when parsing json (source: /opt/ml/input/data/train/deepar_training.json, row: 139)

2019-12-11 12:54:46 Uploading - Uploading generated training model
2019-12-11 12:54:46 Failed - Training job failed

UnexpectedStatusException Traceback (most recent call last)
in ()
----> 1 estimator.fit(inputs=data_channels)

~/anaconda3/envs/python3/lib/python3.6/site-packages/sagemaker/estimator.py in fit(self, inputs, wait, logs, job_name, experiment_config)
462 self.jobs.append(self.latest_training_job)
463 if wait:
--> 464 self.latest_training_job.wait(logs=logs)
465
466 def _compilation_job_name(self):

~/anaconda3/envs/python3/lib/python3.6/site-packages/sagemaker/estimator.py in wait(self, logs)
1059 # If logs are requested, call logs_for_jobs.
1060 if logs != "None":
-> 1061 self.sagemaker_session.logs_for_job(self.job_name, wait=True, log_type=logs)
1062 else:
1063 self.sagemaker_session.wait_for_job(self.job_name)

~/anaconda3/envs/python3/lib/python3.6/site-packages/sagemaker/session.py in logs_for_job(self, job_name, wait, poll, log_type)
2972
2973 if wait:
-> 2974 self._check_job_status(job_name, description, "TrainingJobStatus")
2975 if dot:
2976 print()

~/anaconda3/envs/python3/lib/python3.6/site-packages/sagemaker/session.py in _check_job_status(self, job, desc, status_key_name)
2566 ),
2567 allowed_statuses=["Completed", "Stopped"],
-> 2568 actual_status=status,
2569 )
2570

UnexpectedStatusException: Error for Training job daily-temperature-2019-12-11-12-51-36-487: Failed. Reason: ClientError: Error when parsing json (source: /opt/ml/input/data/train/deepar_training.json, row: 139)

Hi ,
This a great example. Can you please post how your json looks like?

Error in dlnotebooks/mxnet/02 - Classify images with pre-trained models.ipynb

The error occurred at 9th code snippet

code

def init(modelname, gpu=False):
model = loadModel(modelname,gpu=False)
categories = loadCategories()
return model, categories

TraceBack

MXNetError Traceback (most recent call last)
< ipython-input-23-d3cf351fe555 > in < module > ()
4 return model, categories
5
----> 6 vgg16,categories = init("vgg16")
7 resnet152,categories = init("resnet-152")
8 inceptionv3,categories = init("Inception-BN")

in init(modelname, gpu)
1 def init(modelname, gpu=False):
----> 2 model = loadModel(modelname,gpu=False)
3 categories = loadCategories()
4 return model, categories
5

in loadModel(modelname, gpu)
1 def loadModel(modelname, gpu=False):
----> 2 sym, arg_params, aux_params = mx.model.load_checkpoint(modelname, 0)
3 arg_params['prob_label'] = mx.nd.array([0])
4 arg_params['softmax_label'] = mx.nd.array([0])
5 if gpu:

~/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/mxnet/model.py in load_checkpoint(prefix, epoch)
448 - Parameters will be loaded from prefix-epoch.params.
449 """
--> 450 symbol = sym.load('%s-symbol.json' % prefix)
451 save_dict = nd.load('%s-%04d.params' % (prefix, epoch))
452 arg_params = {}

~/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/mxnet/symbol/symbol.py in load(fname)
2726 raise TypeError('fname need to be string')
2727 handle = SymbolHandle()
-> 2728 check_call(_LIB.MXSymbolCreateFromFile(c_str(fname), ctypes.byref(handle)))
2729 return Symbol(handle)
2730

~/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/mxnet/base.py in check_call(ret)
251 """
252 if ret != 0:
--> 253 raise MXNetError(py_str(_LIB.MXGetLastError()))
254
255

MXNetError: [16:29:18] /home/travis/build/dmlc/mxnet-distro/mxnet-build/3rdparty/dmlc-core/include/dmlc/././json.h:731: Check failed: ch == '{' (60 vs. {) : Error at Line 0, around ^!DOCTYPE html>, Expect '{' but get '<'
Stack trace:
[bt] (0) /home/ec2-user/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x25b2ab) [0x7f22a479b2ab]
[bt] (1) /home/ec2-user/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x31f8b23) [0x7f22a7738b23]
[bt] (2) /home/ec2-user/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x320a4b6) [0x7f22a774a4b6]
[bt] (3) /home/ec2-user/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x32033c8) [0x7f22a77433c8]
[bt] (4) /home/ec2-user/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x3203f7a) [0x7f22a7743f7a]
[bt] (5) /home/ec2-user/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x26578b1) [0x7f22a6b978b1]
[bt] (6) /home/ec2-user/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x31ce25d) [0x7f22a770e25d]
[bt] (7) /home/ec2-user/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x2664fbf) [0x7f22a6ba4fbf]
[bt] (8) /home/ec2-user/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x26578b1) [0x7f22a6b978b1]