arnav131003 / lipsync.ai

LipSync.AI

Lipsync.ai is an implementation of the LIPNET model, a state-of-the-art deep learning approach for sentence-level lipreading. The original work was conducted by Yannis M. Assael, Brendan Shillingford, Shimon Whiteson, and Nando de Freitas from the University of Oxford and Google DeepMind.

Introduction

This repository contains the code implementation for Lipsync.ai, a powerful tool for end-to-end sentence-level lipreading. The project is based on the LIPNET model, which was developed by researchers at the University of Oxford and Google DeepMind.

Features

• End-to-End Sentence-Level Lipreading
• Implementation of the LIPNET model
• User-Friendly End-to-End Lipreading Experience

Installation

1. Clone the LipNet repository:

   git clone https://github.com/Arnav131003/LipSync.ai
   cd LipSync.ai
   pip install requirements.txt
   

2. Install dependencies:

   • Keras 2.0+
   • Tensorflow 2.0+
   • PIP (for package installation)
   • Streamlit

---

3 . Pretrained Model

cd  model/checkpoint

4 . Using the pretrained model

model.load("model/checkpoint")

Inferencing

```
streamlit frontend.py 
```

References

https://arxiv.org/pdf/1611.01599.pdf?uuid=Fqbse38nqebdFpys3035

---

Contributors

• Arnav Sharma
• Kushagra Singhal
• Ayush Kumar
• Nikhal Regi

---