himanshurawat443 / abstractive-text-summarization-using-seq-2-seq-rnn-s

Abstractive-Text-Summarization-using-seq-2-seq-RNN-s

The objective of this project was to build an Abstractive Text Summarizer using Sequence-to-Sequence Recurrent Neural Networks. For this project Amazon Fine Food Reviews dataset from Kaggle was used which has 500,000 reviews from 1999-2012. In the modern Internet age, textual data is ever increasing. We need to condense this data while preserving the information and meaning. We need to summarize textual data for that.

Text summarization is the process of automatically generating natural language summaries from an input document while retaining the important points. It would help in easy and fast retrieval of information. Abstractive summarization systems generate new phrases, possibly rephrasing or using words that were not in the original text.

I have done preprocessing on the dataset by converting everything to lowercase, removing HTML tags, stop words, punctuations, special characters, any text inside parenthesis and contraction mapping for better training of the model.

During implementation a 3 Layer Stacked LSTM Encoder-Decoder model with Global Attention Mechanism was used. With this model I was able to get an accuracy of 77.27% on training set (constituting 80% of the dataset). Also cumulative BLEU-4 score of 0.8800 was achieved by this model on test set.

Colab link : https://colab.research.google.com/drive/17dCQU4fi-n1458_KFK_ZYlyj2bZIHOFq?usp=sharing

Datset link : https://www.kaggle.com/snap/amazon-fine-food-reviews/download

Firstly,you have to upload the Datset on your Google Drive then connect that Google drive with Google colab and you are good to go .

After entering in order to execute code you have to run cells using 'Shift + Enter' or 'Ctrl + Enter'

Comments have been added for better understanding of the code

abstractive-text-summarization-using-seq-2-seq-rnn-s's Issues

getting issue in this line of code -> attn_out, attn_states = attn_layer([encoder_outputs, decoder_outputs])

Exception encountered when calling layer "tf.keras.backend.rnn" (type TFOpLambda).

You are passing KerasTensor(type_spec=TensorSpec(shape=(None, 35), dtype=tf.float32, name=None), name='tf.nn.softmax_1/Softmax:0', description="created by layer 'tf.nn.softmax_1'"), an intermediate Keras symbolic input/output, to a TF API that does not allow registering custom dispatchers, such as tf.cond, tf.function, gradient tapes, or tf.map_fn. Keras Functional model construction only supports TF API calls that do support dispatching, such as tf.math.add or tf.reshape. Other APIs cannot be called directly on symbolic Kerasinputs/outputs. You can work around this limitation by putting the operation in a custom Keras layer call and calling that layer on this symbolic input/output.

Call arguments received:
• step_function=<function AttentionLayer.call..energy_step at 0x7f4e51fe4e60>
• inputs=tf.Tensor(shape=(None, None, 256), dtype=float32)
• initial_states=['tf.Tensor(shape=(None, 35), dtype=float32)']
• go_backwards=False
• mask=None
• constants=None
• unroll=False
• input_length=None
• time_major=False
• zero_output_for_mask=False