ParallelRunStep is failing with SystemExit: 42 about machinelearningnotebooks

[LIK2RNG]

I'm trying to run ParallelRunStep:

parallel_run_config = ParallelRunConfig(
   source_directory=experiment_folder,
   entry_script='training_file.py',
   mini_batch_size="1KB",
   error_threshold=10,         
   output_action="append_row",
   environment=registered_env,
   compute_target=pipeline_cluster,
   node_count=2,
   process_count_per_node=4)

train_step = ParallelRunStep(
    name="Paralle Run step",
    parallel_run_config=parallel_run_config,
    inputs = [data_dataset.as_named_input('inp_data')],
    output = output_dir,
    allow_reuse=False)

But getting following error:

Traceback (most recent call last):
  File "driver/amlbi_main.py", line 275, in <module>
    main()
  File "driver/amlbi_main.py", line 226, in main
    sys.exit(exitcode_candidate)
SystemExit: 42

Can anyone please help me with this issue?

@umangkhare @juichiache i met the same problem. Eventually i could find the error inside output+logs --> logs/, inside the folder there're joberror_xxx.txt & jobresult.txt, which contain the real error message

[umangkhare]

ParallelRunStep doesn't show the outputs like usual PythonRunStep. I'd suggest to read the file 'readme.txt' under logs folder to understand how the logging works in ParallelRunStep. It's somewhat tricky to read these logs here. Also, you should consider analyzing CPU utilization and memory usage under 'Monitoring' of the step to get a better understanding.
You will always get 'SystemExit: 42' error irrespective of your pipeline success or failure. You should try tuning the parameters- 'no of processes' and 'mini_batch_size' to run your pipeline properly.
Note- It may take you some time, just have patience.😁

[juichiache]

I have the same issue. Inference was working a couple months back.
Check the log files in output+logs -> user -> stdout -> 1 -> process000.std.txt for detailed error.

[umangkhare]

I think this is not an error which results in failing the pipeline. It eventually results in output logs every time when ParallelRunStep is used. I tried after correcting my other problems in code, and it worked out. Thanks for the support.

[Meenakshi-subramanian]

Hi, I am facing the same issue. Kindly help where should we look at to resolve this.

[LIK2RNG]

Hi, I am facing the same issue. Kindly help where should we look at to resolve this.

same for me. Parallel jobs did not start, indeed it was pending for some time and then died. However, the job was shown as Successful eventually

[Meenakshi-subramanian]

How can I see the real error? You have any idea?