Endpoint inference with trained HuggingFaceEstimator fails about sagemaker-huggingface-inference-toolkit

[vdantu]

@la-cruche : Can this issue be closed?

[philschmid]

@vdantu @ahsan-z-khan i thought when running .deploy() the train.py from training is not taken in to account.
I get the same error as olivier.

File "/opt/conda/lib/python3.6/site-packages/sagemaker_huggingface_inference_toolkit/handler_service.py", line 207, in handle
self.initialize(context)
File "/opt/conda/lib/python3.6/site-packages/sagemaker_huggingface_inference_toolkit/handler_service.py", line 75, in initialize
self.validate_and_initialize_user_module()
File "/opt/conda/lib/python3.6/site-packages/sagemaker_huggingface_inference_toolkit/handler_service.py", line 239, in validate_and_initialize_user_module
user_module = importlib.import_module(user_module_name)
File "/opt/conda/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 994, in _gcd_import
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
Backend response time: 4
File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/.sagemaker/mms/models/model/code/train.py", line 2, in <module>
from sklearn.metrics import accuracy_score, precision_recall_fscore_support
ModuleNotFoundError: No module named 'sklearn'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/mms/service.py", line 108, in predict
ret = self._entry_point(input_batch, self.context)
File "/opt/conda/lib/python3.6/site-packages/sagemaker_huggingface_inference_toolkit/handler_service.py", line 231, in handle
raise PredictionException(str(e), 400)
No module named 'sklearn' : 400

It is even more interesting since the train.py it not included in the model.tar.gz

Does this mean Estimator.deploy is also pulling/using the train.py to the inference container?