Run with litellm about taskweaver

[liqul]

@superuben From the error messages you posted, it looks like an empty api key was configured. Could you please check if you have successfully configured your OpenAI api key?

[superuben]

look all the way at the top of my message "llm.api_key": "" that config is my own input. i run litellm at port 8000 with an openai compatible endpoint so no key is needed. curling it works fine

[superuben]

from another terminal:

python -m taskweaver -p ./project/
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/home/user/TaskWeaver/taskweaver/main.py", line 1, in
from .cli import main
File "/home/user/TaskWeaver/taskweaver/cli/main.py", line 1, in
from .cli import taskweaver
File "/home/user/TaskWeaver/taskweaver/cli/cli.py", line 3, in
from ..app import TaskWeaverApp
File "/home/user/TaskWeaver/taskweaver/app/init.py", line 1, in
from .app import TaskWeaverApp
File "/home/user/TaskWeaver/taskweaver/app/app.py", line 12, in
from taskweaver.session.session import Session
File "/home/user/TaskWeaver/taskweaver/session/init.py", line 1, in
from .session import Session
File "/home/user/TaskWeaver/taskweaver/session/session.py", line 7, in
from taskweaver.code_interpreter import CodeInterpreter
File "/home/user/TaskWeaver/taskweaver/code_interpreter/init.py", line 1, in
from .code_interpreter import CodeInterpreter
File "/home/user/TaskWeaver/taskweaver/code_interpreter/code_interpreter.py", line 6, in
from taskweaver.code_interpreter.code_generator import (
File "/home/user/TaskWeaver/taskweaver/code_interpreter/code_generator/init.py", line 1, in
from .code_generator import CodeGenerator, CodeGeneratorConfig, format_code_revision_message
File "/home/user/TaskWeaver/taskweaver/code_interpreter/code_generator/code_generator.py", line 8, in
from taskweaver.llm import LLMApi
File "/home/user/TaskWeaver/taskweaver/llm/init.py", line 6, in
from openai import AzureOpenAI, OpenAI
ImportError: cannot import name 'AzureOpenAI' from 'openai' (/home/user/.venv/lib/python3.11/site-packages/openai/init.py)

[liqul]

We haven't tested under litellm, so not sure if it would work. Previously, we actually ignored the api_base configuration if api_type is openai. Will fix this later.

[zhangxu0307]

Fixed api_base issue for openai api_type. Please have a try.

[superuben]

TaskWeaver: I am TaskWeaver, an AI assistant. To get started, could you please enter your request?
Human: heya
Error: Cannot process your request due to Exception: 'NoneType' object is not subscriptable
Traceback (most recent call last):
File "/home/user/TaskWeaver/taskweaver/session/session.py", line 124, in send_message
post = _send_message(post.send_to, post)
File "/home/user/TaskWeaver/taskweaver/session/session.py", line 96, in _send_message
reply_post = self.planner.reply(
File "/home/user/TaskWeaver/taskweaver/planner/planner.py", line 179, in reply
llm_output = self.llm_api.chat_completion(chat_history, use_backup_engine=use_back_up_engine)["content"]
File "/home/user/TaskWeaver/taskweaver/llm/init.py", line 292, in chat_completion
oai_response = res.choices[0].message
TypeError: 'NoneType' object is not subscriptable

===========

litellm debug:
An error occurred: generate_text() got an unexpected keyword argument 'response_format'

Debug this by setting --debug, e.g. litellm --model gpt-3.5-turbo --debug

Custom Logger - final response object: None

[zhangxu0307]

TaskWeaver: I am TaskWeaver, an AI assistant. To get started, could you please enter your request? Human: heya Error: Cannot process your request due to Exception: 'NoneType' object is not subscriptable Traceback (most recent call last): File "/home/user/TaskWeaver/taskweaver/session/session.py", line 124, in send_message post = _send_message(post.send_to, post) File "/home/user/TaskWeaver/taskweaver/session/session.py", line 96, in _send_message reply_post = self.planner.reply( File "/home/user/TaskWeaver/taskweaver/planner/planner.py", line 179, in reply llm_output = self.llm_api.chat_completion(chat_history, use_backup_engine=use_back_up_engine)["content"] File "/home/user/TaskWeaver/taskweaver/llm/init.py", line 292, in chat_completion oai_response = res.choices[0].message TypeError: 'NoneType' object is not subscriptable

===========

litellm debug: An error occurred: generate_text() got an unexpected keyword argument 'response_format'

Debug this by setting --debug, e.g. litellm --model gpt-3.5-turbo --debug

Custom Logger - final response object: None

The issue appears to be associated with the incompatibility of litellm and OpenAI's 'response_format' argument. The latest OpenAI API supports both 'response_object' and 'text' modes, but it is unclear whether it can adapt to the litellm API. If you are experiencing a similar problem, feel free to submit a new issue for tracking purposes.

[ShilinHe]

LiteLLM is now supported in TaskWeaver, please follow the docs for more information.
Close the issue as no more activity.