YiVal: Adaptable AI Development Framework

YiVal stands at the intersection of flexibility and adaptability in the AI development landscape. Crafted meticulously for those who seek a tailored experimentation experience, it effortlessly caters to both hands-on developers and those who lean into automation.

YiVal: Adaptable AI Development Framework

Overview

At its core, YiVal is more than just a framework—it's a philosophy. A philosophy that believes in offering tools that can be seamlessly molded to fit unique experimentation needs. With a robust architecture and versatile components, Yival ensures that every AI journey, regardless of its complexity, is smooth and efficient.

Web UI

Fun Cast Fortune Telling

Dive into the world of YiChing and discover your fortune on our index page. A fun and interactive way to get started with Yival.

Experimentation

Experiment Result Analysis: Gain insights into aggregated outputs for each combination, evaluator outcomes, average latency, token usage, and sample test case results. The best combinations will be highlighted for ease of reference.

Data Analysis Page: Delve deep into your experiment data, extracting meaningful insights and patterns that can guide further experimentation.

Detailed Test Results: A granular look at each test case result for every combination, providing a comprehensive understanding of the experiment's outcomes.

Improver Experiment Result Analysis: After the improvement phase, see the aggregated outputs for each combination. This includes evaluator outputs, average latency, token usage, and sample test cases, with the best combinations highlighted.

Improver Detailed Test Results: Post-improvement, this page offers a detailed view of each test case result for every combination, showcasing the enhancement in results.

Additional Features

Export Data: Securely store and export your experiment for future reference or to share with peers.

Rating: Human touch matters. Add ratings to each test case on the experiment results page based on configurable criteria.

Interactive Mode: Flexibility at its best. Enter new test cases for combinations, tailoring your experimentation in real-time.

Architecture

Data Generation

The process starts with the generation of a dataset which can come from multiple sources:

Specific Data Generator: A defined method or algorithm that automatically churns out data.
Data Reader: A component that reads data from external places.
Manual Input: As straightforward as it sounds, data can be added manually.

Combination Creation

Once we have our dataset, we form combinations that are pivotal for the subsequent analysis:

Formed using specific combination creators.
Defined manually.

Analysis

This is the heart of the Yival framework. A custom function provided by the user takes in the dataset and combination list to produce valuable insights.

Evaluation

After analysis, the results are subjected to evaluation. Several methodologies can be applied to grasp and gauge the data's behavior deeply.

Selection

From the evaluations, the most promising results are selected. This process ensures only the most vital insights are pushed forward.

Improvement

The selected results are then fine-tuned in this phase. An "Improver" is applied to enhance these results. This stage can loop back to the analysis stage, indicating an ongoing, iterative process of refinement.

flowchart TD

    %% Data Generation Stage
    A[Start]
    A --> |Data Generation| A1[Dataset]
    A1 --> A2[DataGenerator]
    A2 --> A3[Specific Generator]
    A1 --> A4[DataReader]
    A4 --> A5[Specific Reader]
    A1 --> A6[Manual Input]

    %% Create Combinations Stage
    A1 --> |Create Combinations| B
    B --> B1[Specific Combination Creator]
    B --> B2[Set Combinations Manually]
    B --> B3[List of Combinations]

    %% Evaluate Stage
    B3 --> |Analysis| C
    C --> C1[User's Function]
    C --> C2[Results from Function]
    A1 --> C1

    %% Evaluator Stage
    C2 --> |Evaluation| D
    D --> D1[Method 1]
    D --> D2[Method 2]
    D --> D3[Method 3]

    %% Select Stage
    D --> |Selection| E

    %% Improver Stage
    E --> |Improvement| F
    F --> F1[Improver]
    F1 --> C

    %% Styling
    style A fill:#f9d77e,stroke:#f96e5b
    style B fill:#a1d4c6,stroke:#f96e5b
    style C fill:#f6c3d5,stroke:#f96e5b
    style D fill:#b2b1cf,stroke:#f96e5b
    style E fill:#f9efaa,stroke:#f96e5b
    style F fill:#f2a3b3,stroke:#f96e5b

Installation

pip install yival

Demo

Basic Interactive Mode

To get started with a demo of YiVal, run the following command:

yival demo --basic_interactive

Once started, navigate to the following address in your web browser:

http://127.0.0.1:8073/interactive

Click to view the screenshot

For more details on our demo, check out the Basic Interactive Mode Demo.

untriaged / yival Goto Github PK

yival's Introduction