[BUG] No such file or directory about nlp-recipes

[Farhad-Heybati]

I think the issue is coming from the fact that the output of preprocessing is defined by PipelineData:
output_data = PipelineData(name="train{}".format(str(i)), datastore=ds,
output_path_on_compute='mnli_data/processed_train/batch{}.csv'.format(str(i)))
By default, the path to the data file is based on an environment variable: $AZUREML_DATAREFERENCE
for example:
processed_train_files[0] => $AZUREML_DATAREFERENCE_train0
When I print the value of the file path in preprocessing step and in the training step, these are differents:
preprocess step:
/mnt/batch/tasks/shared/LS_root/jobs/gaia-ml-wks/azureml/273296fe-21e3-44fa-84d1-6a6b33c04031/mounts/workspaceblobstore/azureml/273296fe-21e3-44fa-84d1-6a6b33c04031/train0'
Bert training step:
/mnt/batch/tasks/shared/LS_root/jobs/gaia-ml-wks/azureml/51dd8b67-13a7-40a1-a9ff-eef158a300c5/mounts/workspaceblobstore/azureml/273296fe-21e3-44fa-84d1-6a6b33c04031/train0

The $AZUREML_DATAREFERENCE value change based on the runID