marker-inc-korea / ragchain

Extension of Langchain for RAG. Easy benchmarking, multiple retrievals, reranker, time-aware RAG, and so on...

RAGchain

RAGchain is a framework for developing advanced RAG(Retrieval Augmented Generation) workflow powered by LLM (Large Language Model). While existing frameworks like Langchain or LlamaIndex allow you to build simple RAG workflows, they have limitations when it comes to building complex and high-accuracy RAG workflows.

RAGchain is designed to overcome these limitations by providing powerful features for building advanced RAG workflow easily. Also, it is partially compatible with Langchain, allowing you to leverage many of its integrations for vector storage, embeddings, document loaders, and LLM models.

Docs | API Spec | QuickStart

Quick Install

pip install RAGchain

Why RAGchain?

RAGchain offers several powerful features for building high-quality RAG workflows:

OCR Loaders

Simple file loaders may not be sufficient when trying to enhance accuracy or ingest real-world documents. OCR models can scan documents and convert them into text with high accuracy, improving the quality of responses from LLMs.

Reranker

Reranking is a popular method used in many research projects to improve retrieval accuracy in RAG workflows. Unlike LangChain, which doesn't include reranking as a default feature, RAGChain comes with various rerankers.

Great to use multiple retrievers

In real-world scenarios, you may need multiple retrievers depending on your requirements. RAGchain is highly optimized for using multiple retrievers. It divides retrieval and DB. Retrieval saves vector representation of contents, and DB saves contents. We connect both with Linker, so it is really easy to use multiple retrievers and DBs.

pre-made RAG pipelines

We provide pre-made pipelines that let you quickly set up RAG workflow. We are planning to make much complex pipelines, which hard to make but powerful. With pipelines, you can build really powerful RAG system quickly and easily.

Easy benchmarking

It is crucial to benchmark and test your RAG workflows. We have easy benchmarking module for evaluation. Support your own questions and various datasets.

Installation

From pip

simply install at pypi.

pip install RAGchain

From source

First, clone this git repository to your local machine.

git clone https://github.com/Marker-Inc-Korea/RAGchain.git
cd RAGchain

Then, install RAGchain module.

python3 setup.py develop

For using files at root folder and test, run dev requirements.

pip install dev_requirements.txt

Supporting Features

Advanced RAG features

• Time-Aware RAG
• Importance-Aware RAG

Retrievals

• BM25
• Vector DB
• Hybrid (rrf and cc)
• HyDE

OCR Loaders

• Nougat
• Deepdoctection

Rerankers

• UPR
• TART
• BM25
• LLM
• MonoT5

Web Search

• Google Search
• Bing Search

Workflows (pipeline)

• Basic
• Visconde
• Rerank
• Google Search

Extra utils

• Query Decomposition
• Evidence Extractor
• REDE Search Detector
• Semantic Clustering
• Cluster Time Compressor

Dataset Evaluators

• MS-MARCO
• Mr. Tydi
• Qasper
• StrategyQA
• KoStrategyQA
• ANTIQUE
• ASQA
• DSTC11-Track5
• Natural QA
• NFCorpus
• SearchQA
• TriviaQA
• ELI5

Contributing

We welcome any contributions. Please feel free to raise issues and submit pull requests.

Acknowledgement

This project is an early version, so it can be unstable. The project is licensed under the Apache 2.0 License.

ragchain's Issues

see referenced file contents at web UI

view what files are stored at vector store in gradio web UI

We can use DB and DB linker for this feature. But, how can we get many files at once? And what about file system?

Fix LLM continuing questions in answers

sparse retrieval system for Korean

I think I can use pyserini with custom tokenizer, or this?

Make evaluation method and get test dataset.

KorQuad is one of candidates

Simple GUI

gradio can be useful
or Anything LLM can be one of options.
Or Streamlit.

• #33

Add Colab Demo

Make deploy method to private server

add ko-sroberta-multitask support

https://github.com/jhgan00/ko-sentence-transformers

Enable CPU for KoAlpaca-Polyglot model

prevent duplicate file embedding

adapt new KuLLM-ggml model

I think there is an error at tokenizer or ggml error that we're using at KuLLM model.
I will quantize that myself and adapt it. I think it can achieve better performance?

Local Installer for windows

Local installer for everyone.

Ingest table (표) for better understanding table contents of LLM

special table token, new way organizing table contents?
We need more research about this, but this is crucial for performance.

[HotFix] Enable docker workflows

workflow => workflows

refactor LLM model loader using Factory Pattern

Factory Pattern

add HyDE support

Langchain docs

Fix LLM can't finish an answer by itself

Add Dockerfile and Docker-compose

For easy deployment

add fasttext-ko-vectors

https://huggingface.co/facebook/fasttext-ko-vectors

Enable CPU for KoSimCSE Embedding

There was options at original localGPT for cpu.
Enable for better compatibility.

Add example documents for demo and demo conversation results

write test code for each features

TDD is good ^^

add pinecone support

[HotFix] Delete left constants

I think this is not merged right

Add README content for intial explains of KoPrivateGPT (한국어 ver)

Change to Korean embedding

Pipeline system for execute whole system

pipeline system after refactoring whole projects

[HotFix] Error at accuracy in evaluate.py

[HotFix] Readme.md docker network connet explanation is wrong

docker connect ~ should become docker network connect ~

sry ;)

Add KULLM model

KULLM

add openai embedding

• gpt3.5 chat model for openai choice

Upload file and ingest GUI

Chroma or Pinecone instance once at db.py search func

        db = self.load()
        # TODO : Load Chroma or Pinecone instance everytime when you want to search?
        return db.similarity_search(query=query, k=top_k)

auto build and push Dockerfile to docker hub using github actions when push to main branch

Better project structure for pipeline

There are too many option to execute main function to each ingest and run_localGPT.py
There are so many click.option at main function.

add evaluation script code for strategyQA-Ko

strategyQA using SARI, Recall@10, Accuracy
https://github.com/allenai/strategyqa-evaluator/tree/main

add device options at EMBEDDING class

and change class name to Embedding.
EMBEDDING is not good for class name. (It is for constants)

remove langchain dependncy

It will be obstacle when we use langchain and try to build our own retriever system.

Connect hwp converter to ingest.py

ingest 하위 directories

Retrieval system Evaluation & benchmark for KoPrivateGPT

Repeat penalty for preventing repetetive answers of KoAlpaca model.

add VLLM support

I can run KuLLM and polyglot. But KoAlpaca is hard to run with VLLM

delete all saved embeddings completely

It will also clean up cache files that saved what file saved into the vector DB.
We have to deal with this after issue #36 is done.

refactor constants and webui app.py

inside webui directory, can't reference higher folder constatns.py

select model, embedding, db type at web UI

benchmark with Ko-StrategyQA for baseline

benchmark with gold paragraphs and basic LLM.
KuLLM, KoAlpaca, polyglot-ko, and chatGPT maybe?

Prompt Engineering in Korean for better performance

Prompt Engineering at RetrievalQA

add fly.io deploy (without hwp support)

Or make fly.io options + deploy own hwp server+ connect with outside api setting