moritzlaurer / rag-demo

Example notebooks for Retrieval Augmented Generation (RAG) with LangChain, LlamaIndex, or Haystack and open source models

RAG demo

This respository contains example code for creating and evaluating a Retrieval Augmented Generation (RAG) pipeline with LangChain, Haystack, or LlamaIndex in combination with Hugging Face embedding and generator models.

Data

The example data consists of 440 long PDF documents and meta data from the 2020 open public consultation on the EU White Paper on Artificial Intelligence. It is essentially a compilation of 440 position papers on AI regulation in the EU from a wide range of stakeholders. (Source)

Example use-cases

1. RAG pipeline with Haystack: The notebook rag_haystack_ai_law.ipynb contains a basic example for loading, preprocessing, and indexing the data. You can then use a generative LLM like Mixtral to ask questions about stakeholder's positions on AI regulation. For example, you can ask the PDFs: "What is Microsoft's position in AI regulation?" or "What are the most important points for civil society regarding AI regulation?" or "What are key risks and benefits of AI?".

2. RAG evaluation with LangChain: The notebook rag_langchain_ai_law.ipynb contains an example for automatic RAG evaluation. It first automatically generates questions with an LLM, then passes these questions into a RAG pipeline and then evaluates RAG response quality with an LLM.

3. RAG pipeline with LlamaIndex: The notebook rag_llamaindex_ai_law.ipynb contains an example for retrieving, reranking and generating with LlamaIndex, Hugging Face embeddings and Mixtral.