FileNotFoundError when providing entry_point about sagemaker-huggingface-inference-toolkit

[marshmellow77]

Hi Rogerio, there is two ways to tell the inference toolkit to use a custom inference script:

• You can have the inference script in your local environment and then point to it as you did when you created the HuggingFaceModel() instance
• Or you can have the inference code in the tar.gz file, in which case there is no need to specify where the inference script is located.

In our call I only mentioned the first option, but in the links I sent to you they only mention the second option, so this is totally my fault, sorry. But when you use both options you get the error message you are seeing.

I personally have used the first approach in this notebook which I tested and it works: https://github.com/marshmellow77/text-summarisation-project/blob/main/4a_model_testing_deployed.ipynb

Hope this helps!

Cheers, Heiko

[philschmid]

Hey @rogeriobromfman,

The error No such file or directory: './code' is showing up since you have defined source_dir="./code", in your HuggingFaceModel.
When you define source_dir & entrypoint in your HuggingFaceModel the python-sagemaker-sdk is looking for those file on the machine where you execute HuggingFaceModel.deploy and not inside the model.tar.gz.
That way there is no requirement to bundle your inference.py into model.tar.gz.

But if you have bundled the inference.py in your model.tar.gz you can simply remove source_dir and entry_point from the HuggingFaceModel and it should work.

[rogeriobromfman]

It worked! Having the inference code in the Notebook instance instead of the tar.gz file is what I was missing. Thank you so much Heiko and Philipp!!