[Bug]: Using AzureOpenAI in LiteLLM shows API_version not set. about llama_index

[dosubot]

🤖

Hey @KVedantK,

Good to see you diving back into LlamaIndex! Let's get to the bottom of this together.

Based on the error message you're receiving, it seems that the api_version parameter is not being set when you're creating an instance of LiteLLM. This is likely causing the AzureException error you're encountering.

The api_version parameter is indeed a valid key in litellm.completion(), but it appears that it's not being passed correctly when you're initializing LiteLLM.

Looking at your create_llm_object function, you're correctly retrieving the api_version from the kwargs dictionary. However, it's possible that the api_version is not being passed into the function when it's called, or it's being passed as None.

To resolve this issue, you can ensure that the api_version is being passed into the create_llm_object function. Here's an example of how you can do this:

llm = create_llm_object(
    model='azure/my_sample_deployment',
    temperature=0.2,
    max_retries=2,
    api_key='your_api_key',
    api_base='your_api_base',
    api_type='your_api_type',
    api_version='your_api_version'  # Make sure to replace 'your_api_version' with your actual API version
)

If you're still encountering the same error after trying this, it's possible that there's an issue with how LiteLLM is handling the api_version parameter. In this case, I would recommend checking the source code of LiteLLM to see how it's using the api_version parameter.

I hope this helps! If you have any other questions or need further clarification, please let me know.

Sources

• llama_index/multi_modal_llms/azure_openai.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[dosubot]

🤖

Based on the information you've provided and the source code of LlamaIndex, it seems like the api_version argument is not being correctly set or passed through when creating an LiteLLM object. This is likely causing the AzureException error you're encountering.

The api_version argument is validated in the validate_env method of the AzureOpenAI class. If the api_version value is None, a ValueError is raised. Here's the relevant code snippet:

    @root_validator(pre=True)
    def validate_env(cls, values: Dict[str, Any]) -> Dict[str, Any]:
        """Validate necessary credentials are set."""
        ...
        if values["api_version"] is None:
            raise ValueError("You must set OPENAI_API_VERSION for Azure OpenAI.")
        ...

You can find this in the llama_index/llms/azure_openai.py file.

To resolve this issue, ensure that the api_version argument is correctly set when creating the LiteLLM object. If the issue persists, there might be a problem with how the api_version value is being handled within the AzureOpenAI class or its parent class OpenAI. You might need to debug the code to identify where the api_version value is being lost or overwritten.

Regarding your observation about the api_base not being assigned to any variable, it's possible that this is related to the issue. However, without more information or a closer look at the code, it's hard to say for certain. If you could provide more details about how you're using api_base and what you're expecting it to do, that would be helpful.

I hope this helps! If you have any more questions or need further clarification, feel free to ask.

Sources

• llama_index/llms/azure_openai.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[dosubot]

🤖

Based on the provided context, it seems like the issue you're encountering is due to the LiteLLM class in the Llama_index.llms module not supporting the api_version argument during initialization. The __init__ method of the LiteLLM class does not include an api_version parameter. The parameters it does accept are model, temperature, max_tokens, additional_kwargs, max_retries, api_key, api_type, api_base, callback_manager, system_prompt, messages_to_prompt, completion_to_prompt, pydantic_program_mode, output_parser, and **kwargs.

Here is the relevant code:

def __init__(
        self,
        model: str = DEFAULT_LITELLM_MODEL,
        temperature: float = DEFAULT_TEMPERATURE,
        max_tokens: Optional[int] = None,
        additional_kwargs: Optional[Dict[str, Any]] = None,
        max_retries: int = 10,
        api_key: Optional[str] = None,
        api_type: Optional[str] = None,
        api_base: Optional[str] = None,
        callback_manager: Optional[CallbackManager] = None,
        system_prompt: Optional[str] = None,
        messages_to_prompt: Optional[Callable[[Sequence[ChatMessage]], str]] = None,
        completion_to_prompt: Optional[Callable[[str], str]] = None,
        pydantic_program_mode: PydanticProgramMode = PydanticProgramMode.DEFAULT,
        output_parser: Optional[BaseOutputParser] = None,
        **kwargs: Any,
    ) -> None:
        ...

As you can see, there is no api_version parameter in the list. You can find this in the llama_index/llms/litellm.py file.

The error message you're seeing, "AzureException - Must provide either the api_version argument or the OPENAI_API_VERSION environment variable", is likely due to the fact that neither the api_version argument is being passed to the AzureOpenAIMultiModal class nor the OPENAI_API_VERSION environment variable is being set.

To fix this issue, you can either pass the api_version argument when creating an instance of AzureOpenAIMultiModal:

llm = AzureOpenAIMultiModal(api_version='2023-05-15', ...)

Or, you can set the OPENAI_API_VERSION environment variable in your application:

import os
os.environ['OPENAI_API_VERSION'] = '2023-05-15'

Please replace '2023-05-15' with the actual API version you intend to use. You can find this in the llama_index/multi_modal_llms/azure_openai.py file.

I hope this helps! If you have any other questions, feel free to ask.

Sources

• llama_index/multi_modal_llms/azure_openai.py
• llama_index/llms/litellm.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[KVedantK]

@dosu-bot Thanks, let me check the source code and get back

[KVedantK]

@dosu-bot one observation is that the api_version is getting passed properly but LiteLLM is not assigning it to any variable, even tried passing it as an additional_kwargs still fails.

[KVedantK]

@dosu-bot thanks i will have a look