Comments (7)
Hello @mmabrouk, I modified the add_variant
function in the agenta-cli/agenta/cli/variant_commands.py
module, a prototype for the above issue and would like to test it locally. How do I go about testing? I have searched the docs and could not find a way to do this.
P.S: When I run the command agenta variant serve
, I'd like to view the logs (print statements) in the function.
from agenta.
Hi,
I think every agenda app should include a .gitignore file. If it contains the .env file, users should be able to specify them on the dashboard. This would be a beneficial feature, especially if you plan to add git synchronization in the future.
Railway.app follows this approach and uses Nixpacks for building the app. It would be advantageous to use Nixpacks in Agenda, as it enables the development of agenda apps in a variety of languages.
from agenta.
@mmabrouk, here's how the prototype I built works (yet to be tested using the CLI): Allow the user to specify the environment folder name to exclude in the config.toml file that gets generated when they initialize their application. In a case where the user did not specify the folder to exclude, they get asked by the CLI when they run the command agenta variant serve
, if they're running the application in a virtual environment, if the answer is yes, another question will be asked; prompting the user to input the name of the virtual environment folder.
With the above approach, users can define the name of the environment folder to exclude by editing the config.toml file before running the variant command (agenta variant serve
). The CLI will read the specified folder name, exclude it during the build process, and then create the Docker image without that excluded folder.
from agenta.
Hi @mmabrouk,
I have considered using git directly and have come up with a proposed solution (in addition to the prototype) to exclude the environment folder during the build process. This solution will allow users to easily manage which folders they want to exclude without having to modify the configuration file, config.toml, explicitly.
Here's how the process would look step-by-step:
-
When a user initializes their application using the CLI (agenta init), it is automatically initialized with a .gitignore file.
-
The user can open the .gitignore file and add the name of the environment folder (e.g., "myenv" or any other name) to it. This ensures that the folder will be excluded from version control and not included in future commits.
-
When the user is ready to deploy the application, they can run the agenta variant serve command. During the build process, the .dockerignore file is automatically generated based on the contents of the .gitignore file.
-
Since the environment folder is listed in .gitignore, it will be excluded from the Docker image during the build, avoiding the unnecessary inclusion of large files.
What are your thoughts on this? I'd also like to get a second opinion on this solution, as it provides a simple and user-friendly way to handle the exclusion of the environment folder without requiring the user to edit the config.toml file directly.
from agenta.
How do I go about testing
Currently we do test manually using the examples in the folder examples. I would welcome any proposition for 1) a manual testing protocol 2) automatic testing in a later stage point
Here are my thoughts about the solutions:
Solution 1:
Asking the user when running
agenta variant serve
whether they are running a virtual env.
I think this would add additional friction to agenta variant serve. What I would instead would suggest is that we calculate the size of the folder, and if it is above a certain thresholder (let's say 5M), then we warn the user that their folder is large, note that could happen if they are running a virtual environement, ask them whether they would like to add folders to ignore before serving
Solution 2:
Using .gitignore for providing which folders to ignore
I like this idea a lot , we can add some defaults directly in the .gitignore (such as myenv)
from agenta.
@mmabrouk, thank you.
I'd suggest we go with solution 2, as it stands out to be the preferred approach in the industry due to its simplicity, familiarity and its alignment with best practices in Git version control.
By leveraging .gitignore, the project can provide an effective and user-friendly way to handle exclusion of folder without complicating the CLI or even disrupting the development workflow.
from agenta.
@mmabrouk, thank you.
I'd suggest we go with solution 2, as it stands out to be the preferred approach in the industry due to its simplicity, familiarity and its alignment with best practices in Git version control.
By leveraging .gitignore, the project can provide an effective and user-friendly way to handle exclusion of folder without complicating the CLI or even disrupting the development workflow.
In addition to solution 2, we can also borrow some of the ideas in solution 1, too.
from agenta.
Related Issues (20)
- [Bug]: 'Create a New Variant' modal input data do not get cleared HOT 1
- [AGE-321] [Bug] Not all rows in evaluation comparison cannot be expanded
- [AGE-345] [bug] Some models are not working in the playground
- [Enhancement]: Table row's delete button placement and UI HOT 3
- [AGE-348] [bug] Errors are not correctly handled in the LLM applications HOT 4
- [AGE-357] [bug] Removing a variant used in an A/B test evaluation breaks the human evaluation view
- [AGE-365] Add new status for evaluation Queued HOT 2
- [AGE-370] Improve reproducibility of AI critique outputs HOT 1
- Add Exception handling in Agenta Observability HOT 1
- [AGE-391] [bug] LLM invocation errors are not correctly shown in the evaluation detail view HOT 1
- [AGE-399] [bug] Test creation in single model eval does not work in table view
- [AGE-112] [Evaluations] start new evaluation from previous evaluation
- Add encryption to LLM provider keys before saving it
- how can Agenta be integrated with ollama LLM platform HOT 1
- [AGE-433] [bug] New exact match and JSON evaluators are created with each served variant HOT 4
- Can I deploy without using port 80 ? HOT 5
- [AGE-451] Agent deployed by yourself, how to configure it when you want to Create a Custom App? HOT 3
- After local deployment, when creating an app through the page, there is still an error message when opening it http://0.0.0.0:80 How to modify this for accessing HOT 3
- [AGE-452] [bug] create_new_evaluator_config not setting default correct_answer value
- [AGE-474] JSON diff evaluator not working expectedly
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from agenta.