malekinho8 / chapter-mapper

Convert a PDF of scientific work into an interactive, graphical map.

chapter-mapper

This is a simple repository for mapping PDF files and finding answers to your questions easily and efficiently by utilizing language models from OpenAI. The framework seeks to address the problem of "hallucinations" in large language models by using a direct vector similarity search based on text embeddings of the inputs PDFs and query to find relevant documentation, and then it provides that context to ChatGPT to formulate a coherent answer to the question.

Getting Started

To get started using the code as quickly as possible, you may make a copy of this Google Colab Notebook to your own Google Drive. Continue to follow the INSTRUCTIONS.md for more details.

Usage

Once you have opened the Google Colab Notebook on your Drive, you may follow these steps:

1. Upload a folder to your drive which contains 1 or more PDFs that you want to query.
2. Then for the pdf_folder form field in the Notebook, enter the name of the folder you uploaded. If possible, ensure that the folder name is unique so the correct path is found.
3. For the search_query form field, enter a question you have about the PDFs in your folder.
4. Feel free to experiment with the other parameters, but the only other parameter you have to change is the openai_api_key, which you can obtain on the OpenAI website.
5. Finally, you may simply run the cells by clicking the arrow at the left and scrolling down, or by using the shortcut SHIFT + ENTER.
6. On the first run, it might take some time, but future runs will run faster once the .csv has been created and the embeddings have been stored. Furthermore, if you end up adding more PDFs to your pdf_folder in the future, the program will automatically update the .csv file.