enable test_install_dependencies_to_venv_with_requirements in docker regression about serve

[namannandan]

Currently, in our docker container, we create a virtual environment and install torchserve inside it as follows:

serve/docker/Dockerfile

Lines 54 to 58 in 33d87e3

	# Make the virtual environment and "activating" it by adding it first to the path.
	# From here on the python$PYTHON_VERSION interpreter is used and the packages
	# are installed in /home/venv which is what we need for the "runtime-image"
	RUN python$PYTHON_VERSION -m venv /home/venv
	ENV PATH="/home/venv/bin:$PATH"

The feature to use virtual environment to install custom model dependencies creates a separate virtual environment and attempts to start the model worker and load the model in the new virtual environment. This fails at the following point:

2024-02-12T21:54:23,630 [INFO ] epollEventLoopGroup-3-1 org.pytorch.serve.wlm.ModelManager - Model mnist_custom_dependencies loaded.
2024-02-12T21:54:23,631 [DEBUG] epollEventLoopGroup-3-1 org.pytorch.serve.wlm.ModelManager - updateModel: mnist_custom_dependencies, count: 1
2024-02-12T21:54:23,638 [DEBUG] W-9000-mnist_custom_dependencies_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/model-server/tmp/models/0f78acce5ed84427a9e427e3ea7d9394/venv/bin/python, /home/venv/lib/python3.9/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /home/model-server/tmp/.ts.sock.9000, --metrics-config, /home/venv/lib/python3.9/site-packages/ts/configs/metrics.yaml]
2024-02-12T21:54:23,690 [WARN ] W-9000-mnist_custom_dependencies_1.0-stderr MODEL_LOG - Traceback (most recent call last):
2024-02-12T21:54:23,691 [WARN ] W-9000-mnist_custom_dependencies_1.0-stderr MODEL_LOG -   File "/home/venv/lib/python3.9/site-packages/ts/model_service_worker.py", line 15, in <module>
2024-02-12T21:54:23,692 [WARN ] W-9000-mnist_custom_dependencies_1.0-stderr MODEL_LOG -     from ts.arg_parser import ArgParser
2024-02-12T21:54:23,693 [WARN ] W-9000-mnist_custom_dependencies_1.0-stderr MODEL_LOG - ModuleNotFoundError: No module named 'ts'

Root cause of the issue is that venv created during the container build contains torchserve installed to its site-packages. The new venv that is created to load the model inherits the system site packages but not site-packages of the existing venv in which torchserve is installed, hence from ts.arg_parser import ArgParser fails.

Potential options to fix this:

1. Document that the useVenv option is not supported when torchserve is installed and already started from a virtual environment and disable the useVenv related tests for Docker.
2. Find a way to inherit site-packages from one venv in another.

[agunapal]

Currently, in our docker container, we create a virtual environment and install torchserve inside it as follows:

serve/docker/Dockerfile

Lines 54 to 58 in 33d87e3

	# Make the virtual environment and "activating" it by adding it first to the path.
	# From here on the python$PYTHON_VERSION interpreter is used and the packages
	# are installed in /home/venv which is what we need for the "runtime-image"
	RUN python$PYTHON_VERSION -m venv /home/venv
	ENV PATH="/home/venv/bin:$PATH"

The feature to use virtual environment to install custom model dependencies creates a separate virtual environment and attempts to start the model worker and load the model in the new virtual environment. This fails at the following point:

2024-02-12T21:54:23,630 [INFO ] epollEventLoopGroup-3-1 org.pytorch.serve.wlm.ModelManager - Model mnist_custom_dependencies loaded.
2024-02-12T21:54:23,631 [DEBUG] epollEventLoopGroup-3-1 org.pytorch.serve.wlm.ModelManager - updateModel: mnist_custom_dependencies, count: 1
2024-02-12T21:54:23,638 [DEBUG] W-9000-mnist_custom_dependencies_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/model-server/tmp/models/0f78acce5ed84427a9e427e3ea7d9394/venv/bin/python, /home/venv/lib/python3.9/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /home/model-server/tmp/.ts.sock.9000, --metrics-config, /home/venv/lib/python3.9/site-packages/ts/configs/metrics.yaml]
2024-02-12T21:54:23,690 [WARN ] W-9000-mnist_custom_dependencies_1.0-stderr MODEL_LOG - Traceback (most recent call last):
2024-02-12T21:54:23,691 [WARN ] W-9000-mnist_custom_dependencies_1.0-stderr MODEL_LOG -   File "/home/venv/lib/python3.9/site-packages/ts/model_service_worker.py", line 15, in <module>
2024-02-12T21:54:23,692 [WARN ] W-9000-mnist_custom_dependencies_1.0-stderr MODEL_LOG -     from ts.arg_parser import ArgParser
2024-02-12T21:54:23,693 [WARN ] W-9000-mnist_custom_dependencies_1.0-stderr MODEL_LOG - ModuleNotFoundError: No module named 'ts'

Root cause of the issue is that venv created during the container build contains torchserve installed to its site-packages. The new venv that is created to load the model inherits the system site packages but not site-packages of the existing venv in which torchserve is installed, hence from ts.arg_parser import ArgParser fails.

Potential options to fix this:

1. Document that the useVenv option is not supported when torchserve is installed and already started from a virtual environment and disable the useVenv related tests for Docker.
2. Find a way to inherit site-packages from one venv in another.

How this is different compared to creating a venv, installing TorchServe standalone and then using useVenv