Request -Quality examples for creating Python plugins? about taskweaver

[AR-BABER]

My HANA DB plugin is a go! Closing this issue and adding it to my technical article. Thanks!! @liqul

Cheers! appreciate your help!

Please do share your experience and what you achieved...

[liqul]

@amitlals We have a plugin sql_pull_data for querying sqlite3 which could be used as an entry point for making complicated plugins for other sql dbs. Could you specify your concrete requirements that are not covered in our sample plugin?

[AR-BABER]

i have kind of a same task, all i need is a guideline to accomplish it like should i change planner add edit existing one? my task is for it to connect with postgres database and fetch data and on that data apply certain operations depending upon the query? how would i got about it?

[amitlals]

My HANA DB plugin is a go! Closing this issue and adding it to my technical article. Thanks!! @liqul

Cheers! appreciate your help!

[amitlals]

@amitlals We have a plugin sql_pull_data for querying sqlite3 which could be used as an entry point for making complicated plugins for other sql dbs. Could you specify your concrete requirements that are not covered in our sample plugin?

Yes, I'm building a HANA DB plugin for a quick demo, it seems it's not getting register and trying to hit sql_pull_data plugin again and again;
I can share my HANA Plugin code as well , if required.

Question - Do we really required to manually register new plugin; Though I'm creating this new plugin in same plugin dir only using @plugin syntax.

Any inputs are highly appreciated.

[liqul]

Some suggestions:

1. first make sure the schema is correct and enabled: true in the yaml file. Please go through this doc if you haven't https://microsoft.github.io/TaskWeaver/docs/plugin/plugin_intro
2. if the existing plugins not useful, please delete them (or move them out of the plugin folder) to make it easier for the model to choose the right plugin

@AR-BABER I don't think you need to modify the planner or the code interpreter if you only need to pull data from a postgresql db. The plugins in the project are samples to follow if you want to make your own. But overall, it is quite flexible. You can view each plugin as a python function and it is free to implement any logic inside it.

Our current sql_pull_data plugin is based on langchain which provides a set of tools to interact with various databases. Checkout the langchain documents (e.g., https://python.langchain.com/docs/use_cases/qa_structured/sql) to learn more about handling database related tasks.

[amitlals]

Some suggestions:

1. first make sure the schema is correct and enabled: true in the yaml file. Please go through this doc if you haven't https://microsoft.github.io/TaskWeaver/docs/plugin/plugin_intro
2. if the existing plugins not useful, please delete them (or move them out of the plugin folder) to make it easier for the model to choose the right plugin

@AR-BABER I don't think you need to modify the planner or the code interpreter if you only need to pull data from a postgresql db. The plugins in the project are samples to follow if you want to make your own. But overall, it is quite flexible. You can view each plugin as a python function and it is free to implement any logic inside it.

Our current sql_pull_data plugin is based on langchain which provides a set of tools to interact with various databases. Checkout the langchain documents (e.g., https://python.langchain.com/docs/use_cases/qa_structured/sql) to learn more about handling database related tasks.

Thanks for your inputs.

I removed the existing SQL plugin to see if my HANA DB plugin works...sounds like it's not able to detect this plugin? any idea?

[AR-BABER]

i have the same issue, i followed the exact schema for sql pull data to form my own plugin to pull data from local postgress database, made the necessary changes, but this is happening. i think it fails to run the new plugin i dont know why, and after failure it start creating its own code.

[amitlals]

YES!!! It appears we're facing a similar situation. It seems like we might be overlooking a step in registering a new plugin with TaskWeaver. It's as if it's hardcoded to work with standard plugins only. We need a solution, else I'll drop this project for customer demo.

Here is my situation , my plugins are in ........\TaskWeaver\project\plugins

[liqul]

@amitlals and @AR-BABER without reading your yaml files, I cannot make sure what has caused this issue. It is not hardcoded. Could you please attach your plugin schemas so that we can diagnose what happened?

[liqul]

@amitlals @AR-BABER we also find youtube videos demonstrating how to create custom plugins, please check if they could help
https://www.youtube.com/watch?v=o4zmEEKvdTY

[amitlals]

@amitlals and @AR-BABER without reading your yaml files, I cannot make sure what has caused this issue. It is not hardcoded. Could you please attach your plugin schemas so that we can diagnose what happened?

Thanks for your response. I'm sharing my yaml file for your review.
name hana_pull_data_template.txt

Plugin file
hana_pull_data - template---pypy.txt

Thank you!

[amitlals]

@amitlals @AR-BABER we also find youtube videos demonstrating how to create custom plugins, please check if they could help https://www.youtube.com/watch?v=o4zmEEKvdTY

Yes, I saw this link earlier before I drafted my plugin. Thanks!!

[liqul]

@amitlals and @AR-BABER without reading your yaml files, I cannot make sure what has caused this issue. It is not hardcoded. Could you please attach your plugin schemas so that we can diagnose what happened?

Thanks for your response. I'm sharing my yaml file for your review. name hana_pull_data_template.txt

Is this the complete schema file? I didn't see the parameters and returns, so this is invalid.

[amitlals]

@amitlals and @AR-BABER without reading your yaml files, I cannot make sure what has caused this issue. It is not hardcoded. Could you please attach your plugin schemas so that we can diagnose what happened?

Thanks for your response. I'm sharing my yaml file for your review. name hana_pull_data_template.txt

Is this the complete schema file? I didn't see the parameters and returns, so this is invalid.

Sorry, Here is the complete file

hana_pull_data.ymal >>
name: hana_pull_data
enabled: true
required: false
description: Pull data from SAP HANA database using SQL queries from specific database table ZDEMO_SOI.
This plugin takes user requests when obtaining data from database is explicitly mentioned.
Otherwise, confirm with the user if they want to pull data from this database table or any other known table.
The data from this database can only used for technical analysis and creating forecasting models with diagrams.

parameters:

• name: query
  type: str
  required: true
  description: >-
  This is the query in natural language that the user wants to get data from database.
  If any specific column or value is mentioned, make sure to include them in the query,
  exactly in the right format or form.

returns:

• name: df
  type: pandas.DataFrame
  description: This is the dataframe containing the data from the database.
• name: description
  type: str
  description: This is a string describing the data pulled from the database.

configurations:
hana_host: "xxx.xxx.xxx"
hana_port: "30215"
hana_user: "saphana"
hana_password: "xxxxx"
api_type: openai
api_base: https://api.openai.com
api_key:
deployment_name: test

[liqul]

Still cannot find your plugin？I'm using exactly the same configures as you.

[amitlals]

Still cannot find your plugin？I'm using exactly the same configures as you.

I think still not available...see the detailed error coming up on this plugin name only.

[liqul]

The model already finds your plugin (see the generated code calling your plugin). Now, the problem should come from your implementation. You can see the error saying no module named 'hdbcli'. So, this is the root cause.

[amitlals]

The model already finds your plugin (see the generated code calling your plugin). Now, the problem should come from your implementation. You can see the error saying no module named 'hdbcli'. So, this is the root cause.

Thanks!! I did installed this package. I'm getting strange error now --

Error: Cannot process your request due to Exception: 'ascii' codec can't encode character '\xd7' in position 25: ordinal not in range(128)

[liqul]

Is this a persistent issue?

[amitlals]

Is this a persistent issue?

I've reconstructed the environment twice, yet this ascii issue keeps occurring regularly.

[liqul]

Have you succeeded in calling other plugins? This behavior seems inconsistent with your previous screenshots where the planner can successfully generate a plan. In your new screenshots, it seems the LLM calls are failing due to some encoding errors.

[amitlals]

Yes! Other plugins worked without any issues. This new one is causing issues a lot. Now this new trouble of acsii needs to be reviewed...seems python version is having some compatibility issues.

[liqul]

I suggest you first implement a dummy plugin, for example, just returning a static string. Until the end to end path has been gone through, then add more logic to the implementation.

[AR-BABER]

I was able to run the plugin. Thanks