Upgrading to VLLM 0.4.1 - TypeError about bentovllm

[ProVega]

Looks like VLLM changed a parameter name. Here's my fix to get this project working with VLLM 0.4.1

--- a/llama3-8b-instruct/bentovllm_openai/utils.py
+++ b/llama3-8b-instruct/bentovllm_openai/utils.py
@@ -51,7 +51,7 @@ def openai_endpoints(
                         chat_template=None,
                     ):
                         super(OpenAIServingChat, self).__init__(
-                            engine=engine, served_model=served_model,
+                            engine=engine, served_model_names=served_model,
                             lora_modules=None,
                         )
                         self.response_role = response_role
@@ -75,7 +75,7 @@ def openai_endpoints(
                         return super()._load_chat_template(chat_template)

                 self.openai_serving_completion = OpenAIServingCompletion(
-                    engine=self.engine, served_model=served_model,
+                    engine=self.engine, served_model_names=served_model,
                 )

                 self.chat_template = chat_template
diff --git a/llama3-8b-instruct/requirements.txt b/llama3-8b-instruct/requirements.txt
index fce4600..194a74e 100644
--- a/llama3-8b-instruct/requirements.txt
+++ b/llama3-8b-instruct/requirements.txt
@@ -1,6 +1,6 @@
 accelerate==0.29.3
-bentoml>=1.2.11
+bentoml>=1.2.12
 packaging==24.0
-torch==2.1.2
-transformers==4.39.3
-vllm==0.4.0.post1
+torch==2.2.1
+transformers==4.40.0
+vllm==0.4.1
diff --git a/llama3-8b-instruct/service.py b/llama3-8b-instruct/service.py

[ProVega]

I'm not a git expert, but attempted to create a PR for you
#39

[larme]

@ProVega thanks for the finding! We need to update the same file in multiple folders. I will create another pr to fix this later.