sudarshan-koirala / llama2-chat-with-documents

Simple Chainlit app to have interaction with your documents.

Llama2-chat-with-documents

Simple Chainlit app to have interaction with your documents.

Youtube video covering this notebook

Chat with your documents 🚀

• Huggingface model as Large Language model
• LangChain as a Framework for LLM
• Chainlit for deploying.
• GGML to run in commodity hardware (cpu)
• CTransformers to load the model.

System Requirements

You must have Python 3.9 or later installed. Earlier versions of python may not compile.

---

Steps to Replicate

1. Fork this repository and create a codespace in GitHub as I showed you in the youtube video OR Clone it locally.

   git clone https://github.com/sudarshan-koirala/llama2-chat-with-documents.git
   cd llama2-chat-with-documents
   

2. Rename example.env to .env with cp example.env .envand input the HuggingfaceHub API token as follows. Get HuggingfaceHub API key from this URL. You need to create an account in Huggingface webiste if you haven't already.

   HUGGINGFACEHUB_API_TOKEN=your_huggingface_api_token
   

3. Create a virtualenv with conda and activate it. First make sure that you have conda installed. Then run the following command.

   conda create -n .venv python=3.11 -y && source activate .venv
   

4. Run the following command in the terminal to install necessary python packages:

   pip install -r requirements.txt
   

5. Run the following command in your terminal to create the embeddings and store it locally:

   python3 ingest.py
   

6. Run the following command in your terminal to run the app UI:

   chainlit run main.py -w
   

llama2-chat-with-documents's Issues

'Message' object has no attribute 'replace'

Hi! I've this error when i try to send first question. How to resolve this error?

2023-12-18 09:03:41 - Loaded .env file
2023-12-18 09:03:41 - Your app is available at http://localhost:8000
Abriendo en una sesión existente del navegador
2023-12-18 09:03:45 - Load pretrained SentenceTransformer: sentence-transformers/all-MiniLM-L6-v2
2023-12-18 09:03:45 - Anonymized telemetry enabled. See https://docs.trychroma.com/telemetry for more information.
2023-12-18 09:04:49 - 'Message' object has no attribute 'replace'
Traceback (most recent call last):
File "/home/developer/Documentos/llama2-chat-with-documents/venv/lib/python3.10/site-packages/chainlit/utils.py", line 39, in wrapper
return await user_function(**params_values)
File "/home/developer/Documentos/llama2-chat-with-documents/main.py", line 203, in process_chat_message
response = await qa_chain.acall(message, callbacks=[callback_handler])
File "/home/developer/Documentos/llama2-chat-with-documents/venv/lib/python3.10/site-packages/langchain/chains/base.py", line 381, in acall
raise e
File "/home/developer/Documentos/llama2-chat-with-documents/venv/lib/python3.10/site-packages/langchain/chains/base.py", line 375, in acall
await self._acall(inputs, run_manager=run_manager)
File "/home/developer/Documentos/llama2-chat-with-documents/venv/lib/python3.10/site-packages/langchain/chains/retrieval_qa/base.py", line 184, in _acall
docs = await self._aget_docs(question, run_manager=_run_manager)
File "/home/developer/Documentos/llama2-chat-with-documents/venv/lib/python3.10/site-packages/langchain/chains/retrieval_qa/base.py", line 232, in _aget_docs
return await self.retriever.aget_relevant_documents(
File "/home/developer/Documentos/llama2-chat-with-documents/venv/lib/python3.10/site-packages/langchain_core/retrievers.py", line 269, in aget_relevant_documents
raise e
File "/home/developer/Documentos/llama2-chat-with-documents/venv/lib/python3.10/site-packages/langchain_core/retrievers.py", line 262, in aget_relevant_documents
result = await self._aget_relevant_documents(
File "/home/developer/Documentos/llama2-chat-with-documents/venv/lib/python3.10/site-packages/langchain_core/vectorstores.py", line 676, in _aget_relevant_documents
docs = await self.vectorstore.asimilarity_search(
File "/home/developer/Documentos/llama2-chat-with-documents/venv/lib/python3.10/site-packages/langchain_core/vectorstores.py", line 387, in asimilarity_search
return await asyncio.get_event_loop().run_in_executor(None, func)
File "/usr/lib/python3.10/asyncio/futures.py", line 285, in await
yield self # This tells Task to wait for completion.
File "/usr/lib/python3.10/asyncio/tasks.py", line 304, in __wakeup
future.result()
File "/usr/lib/python3.10/asyncio/futures.py", line 201, in result
raise self._exception.with_traceback(self._exception_tb)
File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/developer/Documentos/llama2-chat-with-documents/venv/lib/python3.10/site-packages/langchain_community/vectorstores/chroma.py", line 348, in similarity_search
docs_and_scores = self.similarity_search_with_score(query, k, filter=filter)
File "/home/developer/Documentos/llama2-chat-with-documents/venv/lib/python3.10/site-packages/langchain_community/vectorstores/chroma.py", line 432, in similarity_search_with_score
query_embedding = self._embedding_function.embed_query(query)
File "/home/developer/Documentos/llama2-chat-with-documents/venv/lib/python3.10/site-packages/langchain_community/embeddings/huggingface.py", line 104, in embed_query
return self.embed_documents([text])[0]
File "/home/developer/Documentos/llama2-chat-with-documents/venv/lib/python3.10/site-packages/langchain_community/embeddings/huggingface.py", line 85, in embed_documents
texts = list(map(lambda x: x.replace("\n", " "), texts))
File "/home/developer/Documentos/llama2-chat-with-documents/venv/lib/python3.10/site-packages/langchain_community/embeddings/huggingface.py", line 85, in
texts = list(map(lambda x: x.replace("\n", " "), texts))
AttributeError: 'Message' object has no attribute 'replace'

'Message' object has no attribute 'replace'

Issue: Could not reach server

When I run the app after setting up everything as mentioned in the instructions, I get this Could not reach server error. How to fix it?

Error {"detail":"Not Found"}

Help please i get : {"detail":"Not Found"}
Mac M2

New updates to import document_loaders from langchain

NEW:
from langchain_community.document_loaders import (
DirectoryLoader,
PyPDFLoader,
TextLoader,
UnstructuredMarkdownLoader,
)

pip install -U langchain-community`

In chotbot not able to communicate with pdf

first issue is : without uploading the pdf if I ask general question it troughing error,
second issue is: after uploading the pdf not able to type the questions.

Your system has an unsupported version of sqlite3. Chroma requires sqlite3 >= 3.35.0.

Is these any other way to solve this error (like create docker container and run it )
Traceback (most recent call last):
File "/workspaces/langchain-car-chatbot/llama2-chat-with-documents/ingest.py", line 62, in
create_vector_database()
File "/workspaces/langchain-car-chatbot/llama2-chat-with-documents/ingest.py", line 52, in create_vector_database
vector_database = Chroma.from_documents(
File "/home/codespace/.python/current/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 646, in from_documents
return cls.from_texts(
File "/home/codespace/.python/current/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 601, in from_texts
chroma_collection = cls(
File "/home/codespace/.python/current/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 80, in init
import chromadb
File "/home/codespace/.python/current/lib/python3.10/site-packages/chromadb/init.py", line 77, in
raise RuntimeError(

Number of tokens (757) exceeded maximum context length (512).

hi, trying to build this app in local, and used same model llama-2-7b-chat.ggmlv3.q8_0.bin
when run the app UI showing some random message same like you showed but checking in console getting this below message:

Number of tokens (755) exceeded maximum context length (512).
Number of tokens (756) exceeded maximum context length (512).
Number of tokens (757) exceeded maximum context length (512).

so increased max_new_tokens=2048, and increased n_ctx and added truncate=True , non of them are fixing this issue.
Changed the model as well. still same issue.

do you know any solution for this issue?