phidatahq / phidata Goto Github PK

Add anyscale as a new LLM provider.

• Read how to add a new LLM provider
• Read about anyscal endpoints

Saw this LM-Studio Cookbook,

from phi.assistant import Assistant
from phi.llm.openai.like import OpenAILike
from phi.tools.duckduckgo import DuckDuckGo

assistant = Assistant(
    llm=OpenAILike(base_url="http://localhost:1234/v1"),
    tools=[DuckDuckGo()],
    show_tool_calls=True,
)
assistant.print_response("Whats happening in France? Summarize top stories with sources.", markdown=True)

Is it because LM-Studio OpenAI API provided is supporting tools?

Since I'm using colab wondering how OpenAILike works and tried with Groq endpoint llm=OpenAILike(base_url="https://api.groq.com/openai/v1", api_key="gsk_xxxxxx", model="mixtral-8x7b-32768"),

it said: tools is not supported with this model. Just want to get my head around it.

And kudos with phi-data it really make opensource model usable in very clear way without many workaround!

• Task: Add LanceDb as a vector store so it can be used as a knowledge base
• How to add a new vector database
• Other vector dbs for example
• Example cookbook to create an test the integration

Describe the bug
The save_and_run, run_files, and run_code functionalities of the Assistant class in the phi.assistant module are not working as expected. When these options are set to True, the Assistant does not execute the code or save the changes.

To Reproduce
Steps to reproduce the behavior:

1. Create an instance of the Assistant class with save_and_run=True, run_files=True, and run_code=True.
2. Use the print_response method to generate some code.
3. Observe that the code is not executed and the changes are not saved.

Expected behavior
When save_and_run, run_files, and run_code are set to True, the Assistant should execute the generated code and save any changes made to the files.

Discussed in https://github.com/orgs/phidatahq/discussions/50

class SalesFunction(Function):
    ...
    parameters: Dict[str, Any] = {
        "type": "object",
        "properties": {
            "start_date": {
                "type": "string",
                "format": "date-time",
                "description": ...
            },
            "end_date": {
                "type": "string",
                "format": "date-time",
                "description": ...
            }
        }
    }

    entrypoint: Callable[[str, str], str] = ...

from phi.assistant import Assistant

assistant = Assistant(description="You help people with their health and fitness goals.")
assistant.print_response("Share a quick healthy breakfast recipe.", markdown=True)

in the above example, the GPT-4-turbo model is used by default
How do I modify the default model?
Thank you

I attempted to create a docker container by running phi ws up as below, but encountered an error. Docker itself is up and running without any issues.

> phi ws up                                                                  
Starting workspace: ai-app

--**-- Confirm resources to create:
  -+-> Network: ai
  -+-> Container: ai-db
  -+-> Container: ai-app

Network: ai
Total 3 resources

Confirm deploy [Y/n]: Y

-==+==- Network: ai
ERROR    Could not connect to docker. Please confirm docker is installed and running                    
ERROR    Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))    

As title

Best,

Fra

I'm just exploring this very promising project.

IMHO, skypilot would be the perfect match for phidata & running ai_apps.

About
SkyPilot: Run LLMs, AI, and Batch jobs on any cloud. Get maximum savings, highest GPU availability, and managed execution—all with a simple interface.

basically, after writing a short deployment (yaml) once, a app can be run on any configured cloud, even local on k8s.
skypilot searches for the actual cheapest prices for the configured resources and starts the app, including loadbalancer, scaling, tunneles, etc…

edit: fixed typos

Hello, is there a recommenced RAG design pattern for a phi assistant and a vector database?

Do we need to pass tools? like cosine similarity retrieval as a python function?

Is there a way to max out the phidata python assistant on how many steps it can take? I am trying to control for infinite loops.

I have created the following program to test phidata tools capabilities. I noticed that when my tool returns a larger string, the programs invokes the tool in an infinite loop (for example string 4K in size). I also get the following warning:

WARNING Could not parse tool calls from response: {
"tool_calls": [
{
"name": "xvpinfo",
"arguments": {
"street": "Swan Lake",
"year": "2024"
}
}
]
}

When the tool returns smaller string (for example 500 bytes in size), everything works as expected.

Is there anything to tweak to make the program work with larger strings returned by the tool? Many thanks in advance.

import logging
from phi.assistant import Assistant
from phi.llm.ollama import Ollama
import requests
import sys
from phi.tools import Toolkit

temp = 0.3
models = ["mistral", "llama2", "mixtral"]
model = models[0]

class XvpTool(Toolkit):
    def __init__(Toolkit):
        super().__init__()

    def xvpinfo(self, street: str, year: str) -> str:
        """Retrieves list of residents living on a street in specific year.
        Args:
            street (str): Full or part of street name (e.g., 'SWAN LAKE', '930 MAIN').
            year (str): year (e.g., '2024').            
        Returns:
            str: A list of residents and their full address.
        """
        url = f"https://secret.com/api/search/address/{street}/{year}/"
        headers = {
            "API-Key": "******",
            "X-Version": "2.0"
        }

        try:
            response = requests.get(url, headers=headers)

            # Check if the request was successful (status code 200)
            if response.status_code == 200:
                result_set = response.json()

                if not result_set:
                    return "None"
                result_string = "\r\n".join([f"{entry.get('name', '')} {entry.get('locNameFull', '')}" if 'locNameFull' in entry else f"{entry.get('name', '')}" for entry in result_set])
                return result_string
            else:
                print(f"Error: {response.status_code} - {response.text}")
                return None

        except Exception as e:
            print(f"Error: {e}")
            return None

assistant = Assistant(
    llm=Ollama(model=model, options={"temperature": temp}),
    description="You are a helpful Assistant to retrieve list of residents living on specific street using tools", 
    instructions=[
            "Given a street name and year, retrieve a list of residents and their address sorted by owner name.",
            "At the end of the response, please include the number of residents found.",
            "At the end of the response, please provide resident count by city.", 
            "If no residents are found, please return 'None Found'."
    ],
    tools=[XvpTool().xvpinfo], 
    show_tool_calls=True,
    debug_mode=True,
    prevent_hallucinations=True,
)

assistant.print_response(f"Who lives at {street} in year {year}?", markdown=False)

Kudos for building Phidata. It would be great to have a hosted gradio app on Spaces as well. Are there any plans?

Hi, I'm new here so this may be silly but following the 'assistant knowledge cookbook' instructions leads only to the following error in the console, complaining about lack of an 'ai' role.
sqlalchemy.exc.OperationalError: (psycopg.OperationalError) connection failed: FATAL: role "ai" does not exist (Background on this error at: https://sqlalche.me/e/20/e3q8)

Any idea why this might be happening? I'm looking to contribute so I'd definitely like to get my local working.

We'd like to add AzureOpenAI as an LLM provider

• Read about AzureOpenAI

Create a cookbook: cookbook/knowledge/langchain.py showcasing how to use the LangChain

Create a cookbook: cookbook/knowledge/pdf_url.py showcasing how to use the PDFURLKnowledgeBase

Hi , is it possible to configure open assistant with open source models such as mistral instruct , dolphin etc?

I realise the simple answer is to replace the hostname in the openai module, but it might be nice to add more support for various backends (as in, for the LLM part). There are a couple of open source function calling models and they are able to use similar querying methods.

Create a cookbook: cookbook/knowledge/pdf.py showcasing how to use the PDFKnowledgeBase

Create a cookbook: cookbook/knowledge/website.py showcasing how to use the WebsiteKnowledgeBase

https://demo.aidev.run/Website_Assistant

Discussed in https://github.com/orgs/phidatahq/discussions/51

extra_instructions=[f"The current date is {datetime.now()}"]

class Assistant(BaseModel):
    ...
    add_current_time: bool = False

Create a cookbook: cookbook/knowledge/text.py showcasing how to use the TextKnowledgeBase

Create a cookbook: cookbook/knowledge/arxiv.py showcasing how to use the PDFKnowledgeBase

Not sure if this is by design, but the OpenAIAssistant doesn't accept Functions as tools which the regular assistant does.

Please add Google Gemini support

I want to call generate_report function with app name parameter. How to create own function and configure with assistant.

Please give me some sample code or reference link.

How do you return the answer as a string verse print response:

from phi.assistant.python import PythonAssistant
from phi.file.local.csv import CsvFile

def average_rating():
    python_assistant = PythonAssistant(
        files=[
            CsvFile(
                path="https://phidata-public.s3.amazonaws.com/demo_data/IMDB-Movie-Data.csv",
                description="Contains information about movies from IMDB.",
            )
        ],
        pip_install=True,
        show_tool_calls=True,
    )

    python_assistant.print_response("What is the average rating of movies?", markdown=True)

I get incomplete response when executing the following

Code:

from dotenv import load_dotenv
from phi.assistant import Assistant
from phi.llm.openai import OpenAIChat

from spaceX.services import prompt

# ensure loading of .envs
_ = load_dotenv()

assistant = Assistant(
    description= "You are a brilliant invoices OCR.",
    #instructions=["Only provide the result, do not need to provide any additional information."],
    # prevent_prompt_injection=True,
    debug_mode=True,
    llm=OpenAIChat(model="gpt-4-vision-preview", ),
    
)


response = assistant.run( [
     {"type": "text", "text": prompt.PROMPT},
     {
          "type": "image_url",
          "image_url": "https://XXX.ngrok-free.app/test.jpeg",
    },
    ], stream=False)


print(response)

Trace:

❯ PYTHONPATH=. python capture/main.py
DEBUG    Debug logs enabled                                                                                                                            
DEBUG    *********** Run Start: 09c4f3e5-d8f5-4c1f-96e9-7dffd3b028cb ***********                                                                       
DEBUG    *********** Task 1 Start ***********                                                                                                          
DEBUG    *********** Task Start: 17444c7b-7c80-4742-a2aa-6cf29acfd132 ***********                                                                      
DEBUG    ---------- OpenAI Response Start ----------                                                                                                   
DEBUG    ============== system ==============                                                                                                          
DEBUG    You are a brilliant invoices OCR.                                                                                                             
                                                                                                                                                       
DEBUG    ============== user ==============                                                                                                            
DEBUG    [{'type': 'text', 'text': 'Please extract the following details from the provided invoice image onto JSON format:\n    Invoice number,\n      
         Date of the invoice,\n    Customer name and address,\n    Vendor name and address,\n    Line items for each item, retrieve the description,   
         quantity, unit price, and total price\n    Subtotals, tax rates, tax amounts, and total due amount,\n    Payment terms, including due date and
         any discounts for early payment,\n    Vendor and customer VAT identification numbers (if applicable),\n    Bank details, including bank name, 
         account number, IBAN, and SWIFT/BIC codes\n\nFormat the extracted data into a structured JSON object ensuring accuracy and consistency in the 
         naming convention and data types for each field."\n'}, {'type': 'image_url', 'image_url':                                                     
         'https://X.ngrok-free.app/test.jpeg'}]                                                                                       
DEBUG    Time to generate response: 8.4779s                                                                                                            
DEBUG    ============== assistant ==============                                                                                                       
DEBUG    Here is the extracted information from the provided invoice image formatted as a JSON object:                                                 
                                                                                                                                                       
                                                                                                                                                       
DEBUG    ---------- OpenAI Response End ----------                                                                                                     
DEBUG    *********** Task 1 End ***********                                                                                                            
Here is the extracted information from the provided invoice image formatted as a JSON object:

Settings:

phidata == 2.3.52
openai ==1.13.3

from dotenv import load_dotenv
from phi.assistant import Assistant
from phi.llm.openai import OpenAIChat

from capture.services import prompt

# ensure loading of .envs
_ = load_dotenv()

assistant = Assistant(
    description= "You are a brilliant invoices OCR.",
    instructions=[prompt.PROMPT],
    # prevent_prompt_injection=True,
    debug_mode=True,
    llm=OpenAIChat(model="gpt-4-vision-preview", ),
    
)


response = assistant.run( [
     {"type": "text", "text": "return all the values from the instructions"},
     {
          "type": "image_url",
          "image_url": " X.ngrok-free.app/test.jpeg",
    },
    ], stream=False)

print(response)

❯ PYTHONPATH=. python capture/main.py
DEBUG    Debug logs enabled                                                                                                                            
DEBUG    *********** Run Start: 429e7408-27ec-470f-870b-29632012f0c1 ***********                                                                       
DEBUG    *********** Task 1 Start ***********                                                                                                          
DEBUG    *********** Task Start: 409b8a8c-5012-4302-b0c6-cbe4cdb92177 ***********                                                                      
DEBUG    ---------- OpenAI Response Start ----------                                                                                                   
DEBUG    ============== system ==============                                                                                                          
DEBUG    You are a brilliant invoices OCR.                                                                                                             
         YOU MUST FOLLOW THESE INSTRUCTIONS CAREFULLY.                                                                                                 
         <instructions>                                                                                                                                
         1. Please extract the following details from the provided invoice image onto JSON format:                                                     
             Invoice number,                                                                                                                           
             Date of the invoice,                                                                                                                      
             Customer name and address,                                                                                                                
             Vendor name and address,                                                                                                                  
             Line items for each item, retrieve the description, quantity, unit price, and total price                                                 
             Subtotals, tax rates, tax amounts, and total due amount,                                                                                  
             Payment terms, including due date and any discounts for early payment,                                                                    
             Vendor and customer VAT identification numbers (if applicable),                                                                           
             Bank details, including bank name, account number, IBAN, and SWIFT/BIC codes                                                              
                                                                                                                                                       
         Format the extracted data into a structured JSON object ensuring accuracy and consistency in the naming convention and data types for each    
         field."                                                                                                                                       
                                                                                                                                                       
         </instructions>                                                                                                                               
DEBUG    ============== user ==============                                                                                                            
DEBUG    [{'type': 'text', 'text': 'return all the values from the instructions'}, {'type': 'image_url', 'image_url': '                                
         https://X.ngrok-free.app/test.jpeg'}]                                                                                         
DEBUG    Time to generate response: 16.1524s                                                                                                           
DEBUG    ============== assistant ==============                                                                                                       
DEBUG    ```json                                                                                                                                       
         {                                                                                                                                             
           "invoice_number": "90152935",                                                                                                               
           "                                                                                                                                           
DEBUG    ---------- OpenAI Response End ----------                                                                                                     
DEBUG    *********** Task 1 End ***********                                                                                                            
```json
{
  "invoice_number": "90152935",
  "

What am I doing wrong?

Do u guys have any plan for integrating w/ Grok? Currently, it seems u depend on OpenAI tools' functionality. Adding a mechanism to detect tools using Grok could bring crazy speed benefits here for a more efficient assistant. Hope u're considering a plan. 🚀

Create a cookbook: cookbook/knowledge/json.py showcasing how to use the JSONKnowledgeBase

Issue

• Python Assistant will not pip install packages

• pip_install=True, is causing an error "ERROR: Function multi_tool_use.parallel not found"

• pip_install=False the code runs without error.

Phidata Version

2.3.36

Code

from phi.assistant.python import PythonAssistant
from phi.file.local.csv import CsvFile
from rich.pretty import pprint
from pydantic import BaseModel, Field


class AssistantResponse(BaseModel):
    result: str = Field(..., description="The result of the users question.")


def average_rating() -> AssistantResponse:
    python_assistant = PythonAssistant(
        files=[
            CsvFile(
                path="https://phidata-public.s3.amazonaws.com/demo_data/IMDB-Movie-Data.csv",
                description="Contains information about movies from IMDB.",
            )
        ],
        instructions=[
            "Only provide the result, do not need to provide any additional information.",
        ],
        # This will make sure the output is of this Assistant is an object of the `AssistantResponse` class
        output_model=AssistantResponse,
        # This will allow the Assistant to directly run python code, risky but fun
        run_code=True,
        # Uncomment the following line to let the assistant install python packages
        pip_install=True,
        # Uncomment the following line to show debug logs
        # debug_mode=True,
    )

    response: AssistantResponse = python_assistant.run("What is the average rating of movies?")  # type: ignore
    return response

Requirements

Package Version

annotated-types 0.6.0
anyio 4.3.0
asttokens 2.4.1
boto3 1.34.50
botocore 1.34.50
certifi 2024.2.2
charset-normalizer 3.3.2
click 8.1.7
colorama 0.4.6
comm 0.2.1
debugpy 1.8.1
decorator 5.1.1
distro 1.9.0
docker 7.0.0
executing 2.0.1
gitdb 4.0.11
GitPython 3.1.42
h11 0.14.0
httpcore 1.0.4
httpx 0.27.0
idna 3.6
ipykernel 6.29.3
ipython 8.22.1
jedi 0.19.1
jmespath 1.0.1
jupyter_client 8.6.0
jupyter_core 5.7.1
markdown-it-py 3.0.0
matplotlib-inline 0.1.6
mdurl 0.1.2
nest-asyncio 1.6.0
numpy 1.26.4
openai 1.12.0
packaging 23.2
pandas 2.2.1
parso 0.8.3
phidata 2.3.43
pip 22.3.1
platformdirs 4.2.0
prompt-toolkit 3.0.43
psutil 5.9.8
pure-eval 0.2.2
pydantic 2.6.2
pydantic_core 2.16.3
pydantic-settings 2.2.1
Pygments 2.17.2
python-dateutil 2.8.2
python-dotenv 1.0.1
pytz 2024.1
pywin32 306
PyYAML 6.0.1
pyzmq 25.1.2
requests 2.31.0
rich 13.7.0
s3transfer 0.10.0
setuptools 65.5.0
six 1.16.0
smmap 5.0.1
sniffio 1.3.1
stack-data 0.6.3
tomli 2.0.1
tornado 6.4
tqdm 4.66.2
traitlets 5.14.1
typer 0.9.0
typing_extensions 4.10.0
tzdata 2024.1
urllib3 2.0.7
wcwidth 0.2.13

Create a cookbook: cookbook/knowledge/wikipedia.py showcasing how to use the WikipediaKnowledgeBase

Would it be possible to loosen this requirement? I'm currently using 2.6.0.

Just a warning - but adding --strip-extras removes warning message when running scripts/upgrade.sh

pip-compile --upgrade --no-annotate --strip-extras --pip-args "--no-cache-dir" \

I have deployed my llm model into azure databricks. How to connect?

Can you share the code samples?