quickstart basic - missing qa/llm_tests:? about llm-finetuning-toolkit

[benjaminye]

Hi @richardehughes, we decided to take little pause in adding new features to the CLI until we finish our test suite (for the program, that is). This should be done by today -- and we'll release the LLM tests integration under a new minor version next week.

But you can still run all the test by importing the module directly like so:

from src.qa.qa import LLMTestSuite
from src.qa.qa_tests import LengthTest, JaccardSimilarityTest

# Load prompts, ground_truths, and model_preds from the results.csv file
# Your code here

# Running LLM Tests
tests = [LengthTest(), JaccardSimilarityTest()]
test_suite = LLMTestSuite(tests, prompts, ground_truths, model_preds)
test_suite.run_tests()
test_suite.print_test_results()
test_suite.save_test_results("path/to/save/test_results.csv")

[benjaminye]

This is addressed in release v0.2.2

https://github.com/georgian-io/LLM-Finetuning-Toolkit/releases/tag/v0.2.2