geeks-of-data / knowledge-gpt

Extract knowledge from all information sources using gpt and other language models. Index and make Q&A session with information sources.

knowledgegpt

knowledgegpt is designed to gather information from various sources, including the internet and local data, which can be used to create prompts. These prompts can then be utilized by OpenAI's GPT-3 model to generate answers that are subsequently stored in a database for future reference.

To accomplish this, the text is first transformed into a fixed-size vector using either open source or OpenAI models. When a query is submitted, the text is also transformed into a vector and compared to the stored knowledge embeddings. The most relevant information is then selected and used to generate a prompt context.

knowledgegpt supports various information sources including websites, PDFs, PowerPoint files (PPTX), and documents (Docs). Additionally, it can extract text from YouTube subtitles and audio (using speech-to-text technology) and use it as a source of information. This allows for a diverse range of information to be gathered and used for generating prompts and answers.

Pypi Link: https://pypi.org/project/knowledgegpt/

Installation

1. PyPI installation, run in terminal: pip install knowledgegpt

2. Or you can use the latest version from the repository: pip install -r requirements.txt and then pip install .

3. Download needed language model for parsing: python3 -m spacy download en_core_web_sm

How to use

Restful API

uvicorn server:app --reload

Set Your API Key

1. Go to OpenAI > Account > Api Keys
2. Create new screet key and copy
3. Enter the key to example_config.py

How to use the library

# Import the library
from knowledgegpt.extractors.web_scrape_extractor import WebScrapeExtractor

# Import OpenAI and Set the API Key
import openai
from example_config import SECRET_KEY 
openai.api_key = SECRET_KEY

# Define target website
url = "https://en.wikipedia.org/wiki/Bombard_(weapon)"

# Initialize the WebScrapeExtractor
scrape_website = WebScrapeExtractor( url=url, embedding_extractor="hf", model_lang="en")

# Prompt the OpenAI Model
answer, prompt, messages = scrape_website.extract(query="What is a bombard?",max_tokens=300,  to_save=True, mongo_client=db)

# See the answer
print(answer)

# Output: 'A bombard is a type of large cannon used during the 14th to 15th centuries.'

Other examples can be found in the examples folder. But to give a better idea of how to use the library, here is a simple example:

# Basic Usage
basic_extractor = BaseExtractor(df)
answer, prompt, messages = basic_extractor.extract("What is the title of this PDF?", max_tokens=300)

# PDF Extraction
pdf_extractor = PDFExtractor( pdf_file_path, extraction_type="page", embedding_extractor="hf", model_lang="en")
answer, prompt, messages = pdf_extractor.extract(query, max_tokens=1500)

# PPTX Extraction
ppt_extractor = PowerpointExtractor(file_path=ppt_file_path, embedding_extractor="hf", model_lang="en")
answer, prompt, messages = ppt_extractor.extract( query,max_tokens=500)

# DOCX Extraction
docs_extractor = DocsExtractor(file_path="../example.docx", embedding_extractor="hf", model_lang="en", is_turbo=False)
answer, prompt, messages = \
    docs_extractor.extract( query="What is an object detection system?", max_tokens=300)

# Extraction from Youtube video (audio)
scrape_yt_audio = YoutubeAudioExtractor(video_id=url, model_lang='tr', embedding_extractor='hf')
answer, prompt, messages = scrape_yt_audio.extract( query=query, max_tokens=1200)

# Extraction from Youtube video (transcript)
scrape_yt_subs = YTSubsExtractor(video_id=url, embedding_extractor='hf', model_lang='en')
answer, prompt, messages = scrape_yt_subs.extract( query=query, max_tokens=1200)

Docker Usage

docker build -t knowledgegptimage .
docker run -p 8888:8888 knowledgegptimage

How to contribute

0. Open an issue
1. Fork the repo
2. Create a new branch
3. Make your changes
4. Create a pull request

FEATURES

• Extract knowledge from the internet (i.e. Wikipedia)
• Extract knowledge from local data sources - PDF
• Extract knowledge from local data sources - DOCX
• Extract knowledge from local data sources - PPTX
• Extract knowledge from youtube audio (when caption is not available)
• Extract knowledge from youtube transcripts
• Extract knowledge from whole youtube playlist

TODO

• FAISS support
• Add a vector database (Pinecone, Milvus, Qdrant etc.)
• Add Whisper Model
• Add Whisper Local Support (not over openai API)
• Add Whisper for audio longer than 25MB
• Add a web interface
• Migrate to Promptify for prompt generation
• Add ChatGPT support
• Add ChatGPT support with a better infrastructure and planning
• Increase the number of prompts
• Increase the number of supported knowledge sources
• Increase the number of supported languages
• Increase the number of open source models
• Advanced web scraping
• Prompt-Answer storage (the odds are that this will be done in a separate project)
• Add a better documentation
• Add a better logging system
• Add a better error handling system
• Add a better testing system
• Add a better CI/CD system
• Dockerize the project
• Add search engine support, such as Google, Bing, etc.
• Add support for opensource OpenAI alternatives (for answer generation)
• Evaluating dependencies and removing unnecessary ones
• Providing prompt flexibility for using with whatever model

( To be extended...)

System Architecture

(To be updated with a better image)

knowledge-gpt's Issues

Prompting Enhancement

The amount of pre/post prompting options we provide are extremely limited, we should utilize a better/more flexible way of constructing prompts.

Improving Parsing Quality

All of our information parsing mechanisms are quite "primitive" at the moment, a re-write could be useful for some of them. i.e. ytaudio, ytsubs(not too bad), pdf/doc/ppt extraction, web scraping,

Example.docx is missing

I assume I am getting the error due to not having the example.docx for the example code below.

https://github.com/geeks-of-data/knowledge-gpt/blob/master/examples/docs_example_saving.ipynb

See attached.

Hybrid File Reader

When a directory contains more than 1 of the supported types of files, create the knowledgebase using all of them

language support

A user wrote an email and asked for italian and russian language support, we can start adding them one after another.

• Italian
• Russian
• German
• Spanish
• Chinese
  ....
  ( To be extended on demand)

Whisper Audio Length Problem

Audio file to transcription not working for files above 25mb because OpenAI API doesn't accept bigger files and requires chunking into smaller pieces, I failed to make it work so far and left it out since it's not pressing, at some point of time it has to be addressed for sure though.

Dockerize the project

The aim is create a Dockerfile to install knowledge-gpt without pip.

Testing/Implementing OpenSource ChatGPT Alternatives

Better Error Handling

The error handling at the moment is based on what feels good, a systematic rewrite of the error handling sections can help

Search Engine Support

Support for making calls to actual search engines to connect/create/update knowledge bases for our prompts.

Installation Error No module named 'knowledgegpt'

I followed the instructions provided but still got an error. I downloaded spacy from here. See attached.

I re-run install . and got the error below.