opensearch-project / dashboards-flow-framework Goto Github PK

We will have several iterations and improvements to a test application portal. This portal will allow users to test their created workflows.

• #11
• #15

To support the user experience of rapid prototyping on a use case / workflow, there are a few enhancements we can make. This list is TBD and dependent on final design from @opensearch-project/opensearch-ux

• show resources that can be deleted when re-running / re-executing a workflow
• show recently-ran resources somewhere in an accessible area

There are components that will need API keys / credentials as part of the configuration when constructing use cases, such as API keys for configuring and creating a connector to an external service like OpenAI, for example.

We will need to design a secure way for storing and processing this input in the frontend

Set up the node APIs and the server side infrastructure for supporting API calls to the connected OpenSearch cluster. Support a few initial APIs, such as:

• create workflow
• fetch workflow status
• fetch workflow details
• delete workflow

Building out any fetching logic for dynamic components, to know what is available to show in a list that the user may drag into their workspace. The current known ones are:

• ML commons models
• Ingest processors
• Search processors

We will also need to check what plugins are available (e.g., ML commons).

We will need to check these things to ensure what components and/or workflows are valid for the user to use. This will likely involve several API calls to other plugins.

Build out the workspace area to construct the workflows. Initially we may have the workspace be read-only regarding the components and connections between them. We will only allow users to input the required fields within the components. We will also want to show validation and any errors of a given workflow. This is pending final design from @opensearch-project/opensearch-ux

As the UI workflows become more advanced and have more capabilities, we will need to similarly support more advanced validation of such workflows, and the logic for converting them into use case templates. We can track the iterative milestones in this issue.

• #8
• #18

List of concrete APIs to add is TBD. We will likely have more advanced APIs and will want to process things like workflow state, list of created resources from a workflow, saving template-only, etc.

Expand the application portal to support custom workflows that can take generic inputs/outputs (may have some limitations to start). Details to be added as the design becomes more finalized.

Once custom workflows is supported by the backend framework, we will want to allow users to construct their own workflows, being able to drag in any components and linking them together. This will need more advanced workflow validation, as well as more interfaced functionality of the components to allow / disallow components to be linked together, etc.

Support basic validation and conversion of a given UI flow into a use case template. Can start with just supporting a few static use cases.

Integrate with https://github.com/opensearch-project/opensearch-dashboards-functional-test.

• add/update GitHub workflows to include this plugin
• add a few specs and test suites for this plugin
• add a GitHub workflow in this repository to execute the tests

Have a loading bar or somewhere to view workflow status after a user submits/creates a new workflow. May involve new API integrations to process some of the state from the backend framework. May be updated as the design is finalized from @opensearch-project/opensearch-ux

We will need several different iterations and additions to the server-side API calls to OpenSearch:

• #7
• #17

Area to perform all CRUD operations on user-created workflows. Idea: similar to AD detector list? Pending final design from @opensearch-project/opensearch-ux

Design & implement how to persist the ReactFlow JSON somewhere in the backend (part of use-case template, separate system index, etc.). May need some backend implementation.

Introduce a workflow editor containing a drag-and-drop workspace where users can stitch together generic building blocks to build out their applications. When validated, they can launch their workflow to create any necessary resources and begin to prototype.

Proposed mock as of 2/19 from @opensearch-project/opensearch-ux:

• #10
• #20
• #5
• #73
• #74
• #75
• #82

Build out a test application portal for user to test their built workflows. Convert input to a neural search query. Only support a few use cases to start. Pending final design from @opensearch-project/opensearch-ux

Support an arbitrary flow of nodes/edges and validate it + be able to convert it to a use case template.

Corresponding change needed in backend plugin: opensearch-project/flow-framework#168.

To follow https://github.com/opensearch-project/opensearch-plugins/blob/main/CONVENTIONS.md#plugin-naming-conventions, we should prefix the repo name with dashboards-. With the backend plugin also changing to flow-framework, for consistency, we propose to change this to dashboards-flow-framework.

Further follow-ups to the plugin name in the source config files will be done later.

Build out the discoverable space with pre-set workflows to choose from. This is pending final design from @opensearch-project/opensearch-ux

Building out a library of hierarchical interfaces for the different component types needed for initial few use cases (semantic search / RAG). Defining styling & functionality for each. These will be based on supported steps in the use case templates.