Improve ROSA customer experience (and customer retention) by leveraging foundation models to do “gpt-chat” style search of Red Hat customer documentation assets.
The first step to train a QA model for ROSA documentation search is to assemble and process the documentation text from various sources. This issue should focus on solving the following steps:
There are open source implementation and resources available for training QA models. We should collect and discuss those resources here. Once the discussion converges, we should use the frameworks for implementation.
Foundation models need to be adapted for specific use cases and domains. There are several questions on around how to target different use cases. As a part of this epic, we will find answers to the following questions:
How do different variants of llms compare with each other, in terms of architecture (input tokens, hidden & attention layers, parameters, decoder encoder variations), licenses, hardware utilization, etc.?
What is the difference between small FMs (<15B) and large FMs(>50B)?
How does performance vary for few shot prompting large models vs fine tuning smaller models?
Do we need a hierarchy of models for specific tasks? For example, one base large model for text generation and two smaller models each for code generation and documentation QA? What's the difference between Bloom 13B and Bloom 3B?
Do smaller models have a smaller context window or token limit and is that a limitation? How are contexts used by the models, in other words how is the model learning complemented by the context to generate a response?
What is the relevance of vector databases in these solutions? Are they still relevant in smaller fine-tuned models with smaller context windows?
What are the production cost and performance comparisons of these approaches? Design experiments to show some of these comparisons.
What is the role of datasets in fine tuning? Does fine tuning for a domain require a QA format dataset or self-supervised masking words in a sentence (recheck) dataset? Can we try BERT based models that have a different architecture?
What are the various steps that take place in QA with FMs? #30
Adapt learnings from this to solve ROSA use case #18
We would need a dataset to fine tune Large language models for searching ROSA documentation and evaluating the performance of different models. As a start, take existing FAQs and convert them into QA format.
As a part of this issue, discuss and answer the following questions:
What Quick enough volume of QA data that we can manage? 100-1000 rows?
What kind of questions do we want the model to answer?
Do we want quick questions or longer ones or both?
To compare frameworks, models, and model parameters for text generation, we should have a notebook that has data processing, model inferencing, and evaluation cells. This notebook shall be used to create a comparison result of different variables.
The second step of creating a QA search system for the ROSA documentation is to train and compare the open source versions of language models. The issue should accomplish the following tasks:
The large size of foundation models raise several resource and cost questions around deploying them in production. This EPIC will focus on creating experiments and showing results around some of the following questions:
What is the relationship between model parameters and its memory consumption? Create a "rosetta stone" document of GPU memory required by models belonging to different parameter size. Create a notebook that captures the footprint of the GPU memory used.
How are the models loaded into GPU RAM? Is it directly from S3 or do we also require significant RAM? If so, capture the RAM requirements in a notebook. What about CPU? Update the cost document with RAM, and CPU information. Are there ways to optimize this?
What happens when we load the models in a lower precision format like INT-8? How is the accuracy, CPU, and memory performance affected? Explain theoretically and show results in a notebook. Touch upon challenges of frameworks like bitsandbytes in production.
Is distributed training and inference with a lot of cheap instances more efficient per dollar than 1 instance with a large GPU? If we have just one GPU of 16GB memory, how much can be done with it in the space of LLMs? Design experiments and share results in a notebook.
What are the options of running these models just on CPU? Are there ways of optimizing more than 1B = 1GB of GPU with INT8 precision?
Notebook that shows how to use evaluation metrics for text generation. Take 2-3 hardcoded examples to show the behavior of each metric. For example, exact match may be very low but BERT score is high because the prediction captures the semantics.
The experiments with LLM models like GPT2, BLOOM, Openai should be created in accordance with notebook from #24. The results should reflect qualitative and quantitate comparison between outputs.
Serving foundation models on an OpenShift cluster is a crucial step for using them in production. As a part of this issue we will deploy demo models using open source tools and/or the stack recommended by RHODS. Following questions would be further explored as a part of this epic:
It would be very helpful to have the dependencies and their versions defined in a requirements.txt along with each notebook.
It would be one less thing to troubleshoot when facing any issue while running the notebooks.
The purpose of this issue is to establish a comprehensive framework for conducting human evaluation of generated answers and comparing them against real answers.
A mechanism to obtain (a set of) plain text (ascii files) directly from the source files from the various documentation sources around ROSA wherever possible, to avoid having to rely on text extraction from rendered websites or PDF files.
Describe alternatives you've considered
We have a collection of various PDF files, from which plain text can be extracted.
However, standard text extration from PDF has some limitations / problems:
hyperlinks are not visible
document structure is harder to retain
the PDF renderer introduces noise (page headers/footers, numbered references, etc)
the PDFs are harder to refresh / keep up to date with the current docs
The finetuning notebook uses 1 GPU and LoRA technique to fine tune a T5 model with 3B parameters. The task to be completed in this issue is to fine tune the same model (or 7B version of the model) on multiple GPU nodes. Use Instascale and Codeflare to schedule the training job and retrieve the finetuned model. Create a notebook that demos this.
In order to compare different QA models for user documentation, we will have to decide on quantitative metrics that evaluate the models.
As a part of this issue add resources and discussion points for the metrics to be used.
Add a notebook that show some of these metrics.
Is your feature request related to a problem? Please describe
As we build language "chains" that involve multiple components and steps (creation of embeddings, querying embeddings, prompt template selection, API queries) I would like to be able to see what is going in detail during a query to the model.
Describe the solution you'd like
Some "debug" option for the whole chains. Possibly something that can be external so that it does not depend on being inside a notebook to be able to see it.
The process of training a reward model for Reinforcement Learning from Human Feedback (RLHF) involves gathering comparison data to accurately assess the quality of prompt-response pairs. This comparison data aids in fine-tuning a Large Language Model (LLM) using Reinforcement Learning (RL) to generate responses that align with human preferences. This issue aims to guide the development of a systematic workflow for collecting comparison data using the Argilla platform.