lcebear / memorydialoguebot

Master Thesis Project: A memory based dialogue agent

memoryDialogueBot

Master Thesis Project

07/09/2020 - Question answering agent version 2.0.

The project (API) is developed to be run using the flask and waitress library. The application can be ran with the following command 'python3 waitress_server.py' The file 'json_post_example.py' illustrates how the agent can be called through the api. There is also an interactive version of the agent in file 'run_agent.py'.

The latest trained GPT-2 generative model can be downloaded from: https://drive.google.com/open?id=1bamDAUSHM8ye6xzs51-hi9un03DwjOy5

Trained classifiers can be downloaded here: https://drive.google.com/file/d/1coJjE3UL7feooR2x-g6zwedkUPGcKJQA/view?usp=sharing

The project initially used the gpt_2_simple library, but due to concurrent generation problems and due to the library not being optimized for fast response time (for chatbots), the project now uses the script "gpt2_specific_gen.py" to generate. The script is an adapted variation of "interactive_conditional_samples.py" from OpenAI and the "gpt_2.py" script from the gpt_2_simple library.

Setup

git clone https://github.com/lcebear/memoryDialogueBot.git

Create a virtual python environment

python3 venv path/to/env

Install requirements

pip install -r requirements.txt

Download additional packages

python -m spacy download en_core_web_lg

download nltk packages 'stopwords', 'punkt', 'vader_lexicon', 'averaged_perceptron_tagger' by using nltk.download()

import nltk
nltk.download('punkt')
nltk.download('stopwords')
nltk.download('vader_lexicon')
nltk.download(averaged_perceptron_tagger')
...

Download latest GPT-2 generative model to project folder.

Download trained classifiers to project folder.

Run Chatbot

python3 -m run_agent.py 

Run API

python3 waitress_server.py

-> Example API usage in json_post_example.py

Remember to change the secret key (app.secret_key) in the 'agent_api' file to some random bytes.

Cite

Master's Thesis

Conversational Chatbots with Memory-based Question and Answer Generation: http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-171927

BibTex

@mastersthesis{lundellvinkler2020conversational, title={Conversational Chatbots with Memory-based Question and Answer Generation}, author={Lundell Vinkler, Mikael and Yu, Peilin}, year={2020}, school={Link\={o}ping University } }

memorydialoguebot's Issues

List of generated questions

For the questions generated, I see in the terminal/command window that multiple questions are generated for get_question, how can I retrieve the whole list instead of the one question?

Accessibility of the API from other computers

A simple way to expose the API on the flask server to other computers is by making use of the waitress library. With an extra python file, with the following code, you can easily deploy the API:

Filename: waitress_server.py

from waitress import serve
import agent_api
serve(agent_api.app, host='0.0.0.0', port=5000)

And then you can run the server with python waitress_server.py

NLTK download command

Hi!

I just finished setting up the agent and found a small something missing in the installation. I had to do nltk.download('averaged_perceptron_tagger before the REST-API would work, now it's all fine.