This is a simple repository for mapping PDF files and finding answers to your questions easily and efficiently by utilizing language models from OpenAI. The framework seeks to address the problem of "hallucinations" in large language models by using a direct vector similarity search based on text embeddings of the inputs PDFs and query to find relevant documentation, and then it provides that context to ChatGPT to formulate a coherent answer to the question.
To get started using the code as quickly as possible, you may make a copy of this Google Colab Notebook to your own Google Drive. Continue to follow the INSTRUCTIONS.md for more details.
Once you have opened the Google Colab Notebook on your Drive, you may follow these steps:
- Upload a folder to your drive which contains 1 or more PDFs that you want to query.
- Then for the
pdf_folder
form field in the Notebook, enter the name of the folder you uploaded. If possible, ensure that the folder name is unique so the correct path is found. - For the
search_query
form field, enter a question you have about the PDFs in your folder. - Feel free to experiment with the other parameters, but the only other parameter you have to change is the
openai_api_key
, which you can obtain on the OpenAI website. - Finally, you may simply run the cells by clicking the arrow at the left and scrolling down, or by using the shortcut
SHIFT + ENTER
. - On the first run, it might take some time, but future runs will run faster once the
.csv
has been created and the embeddings have been stored. Furthermore, if you end up adding more PDFs to yourpdf_folder
in the future, the program will automatically update the.csv
file.