holylovenia / question-answer-generation

Question-answer generation from text

Automatic Question-Answer Pairs Generation from Text

Members

1. Holy Lovenia

2. Agus Gunawan

3. Felix Limanta

English Paper

We have a technical report that explains our question-answer generation system here.

Architecture in overview

We propose an approach which is generally based on the framework of an ongoing work by A. Sarvaiya. Formally, given a passage , question-answer generation (QAG) system retrieves the most important sentence from . Then, QAG system produces a set of question-answer pairs , where each generated can be found in , and its pair is the interrogative version of or a clause from a set of clauses in , without in it. As shown in the figure below, here are four main modules in our QAG system.

1. Preprocessing, which cleans the input passage from unnecessary characters and shapes it into the desirable form (list of sentences).

2. Sentence Selection, which picks top- most important sentences given . The text summarization method used can be chosen between TextRank, multi-word phrase extraction (MWPE), and latent semantic analysis (LSA). The chosen method ranks the sentences in P and selects the top- highest ranked sentences as the output.

3. Gap Selection, which selects phrases in that can be used as answers based on constituent tree from syntactic parser and named entity recognition (NER).

4. Question Formation, which creates the interrogative version of or in without to make a question for each answer in . The final output of this module is question-answer pairs related to .

Result

1. Demonstration with Jupyter Notebook

2. Source code for the web version is in this GitHub repository. Check it out!

Installation

1. Export '$STANFORD_PARSER' environment variable using lib path using command EXPORT STANFORD_PARSER=<ABSOLUTE_PATH> e.g. EXPORT STANFORD_PARSER=/home/anonymous/question-answer-generation/QuestionGeneration/lib
2. If you want the website version please see the link above
3. If the program stuck after loading the model. It's because the port already used. This happened usually after you already run the program twice or several times. Please kill the process that runs on the port using lsof -i find java, kill the proces PID.

question-answer-generation's Issues

Runing fail with error (Connection refused, [Errno 111] Connection refused)

We facing the issue of error 111 when apply qg.process(top_sentences=2)

Class: QuestionGenerator
Method: self.__rank_sentence

how should i run this project?

I have tried run app.py,and the following sentences are shown.
C:\Users\i_am_sixsix\Desktop\all\project\question-answer-generation-master\venv\Scripts\python.exe C:/Users/i_am_sixsix/Desktop/all/project/question-answer-generation-master/QuestionGeneration/app.py
Load Model Success

---

default values:
senna path:
C:\Users\i_am_sixsix\Desktop\all\project\question-answer-generation-master\QuestionGeneration\lib\senna\
Dependencie parser:
C:\Users\i_am_sixsix\Desktop\all\project\question-answer-generation-master\QuestionGeneration\lib
Stanford parser clr java -cp stanford-parser.jar edu.stanford.nlp.trees.EnglishGrammaticalStructure -treeFile in.parse -collapsed

---

i dont know what it means.
help!
plz!

Regarding the question and answer generation project

I am facing an error while the program is executed like to set the environment variable,and NLTK is unable to find the Stanford parser,can any one explain to solve this issue

ModuleNotFoundError: No module named 'sklearn_crfsuite'

I am always getting this error even after I installed sklearn_crfsuite

CLASSPATH STANFORD PARSER ERROR - UBUNTU

IN UBUNTU:

Even if i download the file and add in classpath iam still getting the same error can someone help me in this
![Screenshot from 2019-11-26 15-20-14]

(https://user-images.githubusercontent.com/25568301/69618762-6e339300-1060-11ea-9f1d-b509b84f9d8d.png)

Why do i get this error? AttributeError: 'NoneType' object has no attribute 'label'

File "/home/frenk/Scaricati/question-answer-generation-master/QuestionGeneration/get_questions.py", line 23, in
questions = qg.process(top_sentences=1)

File "/home/frenk/Scaricati/question-answer-generation-master/QuestionGeneration/app.py", line 47, in process
self.__generate_questions_from_sentence(self.sentences[r])

File "/home/frenk/Scaricati/question-answer-generation-master/QuestionGeneration/app.py", line 26, in __generate_questions_from_sentence
candidates = gapper.process()

File "/home/frenk/Scaricati/question-answer-generation-master/QuestionGeneration/question_generation/gap_selection.py", line 144, in process
self.fetch_parent()

File "/home/frenk/Scaricati/question-answer-generation-master/QuestionGeneration/question_generation/gap_selection.py", line 95, in fetch_parent
if parent.label() == 'S':

AttributeError: 'NoneType' object has no attribute 'label'