y1xing / translate

Effortless Real-Time Sign Language Translation

👋 Sign Translate

Revolutionizing Sign Language Communication with Cutting-Edge Real-Time Translation Models.
Enjoy seamless Sign Language Translation on desktop and mobile.

sign.mt

Contribution Guidelines · Submit an Issue

---

Key Features

Sign Language Production

┌─────────────────────┐
│Spoken Language Audio│                                                              ┌─────────┐
└─────────┬───────────┘                                                  ┌──────────►│Human GAN│
          │                                                              │           └─────────┘
          ▼                                                              │
┌────────────────────┐     ┌───────────────┐     ┌───────────┐    ┌──────┴──────┐    ┌───────────────┐
│Spoken Language Text├────►│Normalized Text├────►│SignWriting├───►│Pose Sequence├───►│Skeleton Viewer│
└─────────┬──────────┘     └───────────────┘     └───────────┘    └──────┬──────┘    └───────────────┘
          │                        ▲                   ▲                 │
          ▼                        │                   │                 │           ┌─────────┐
┌───────────────────────┐          │                   │                 └──────────►│3D Avatar│
│Language Identification├──────────┘───────────────────┘                             └─────────┘
└───────────────────────┘

Sign Language Translation

┌──────────────────────────┐                                ┌────────────────────┐
│Upload Sign Language Video│                      ┌────────►│Spoken Language Text│
└──────────┬───────────────┘                      │         └──────────┬─────────┘
           │                                      │                    │
           │          ┌────────────┐       ┌──────┴────┐               │
           ├─────────►│Segmentation├──────►│SignWriting│               │
           │          └────────────┘       └───────────┘               │
           │                                                           ▼
┌──────────┴────────────────┐                               ┌─────────────────────┐
│Camera Sign Language Video │                               │Spoken Language Audio│
└───────────────────────────┘                               └─────────────────────┘

Want to Help?

Join us on the journey to revolutionize sign language communication. Follow our progress on the Project Board, shape the project's future, and delve deeper into our vision and plans in the Wiki.

Wish to report a bug, contribute some code, or enhance documentation? Fantastic! Check our guidelines for contributing and then explore our issues marked as help wanted or good first issue.

Find this useful? Give our repo a star ⭐ ⬆️.

Development

Prerequisites

• Install Node.js which includes Node Package Manager

Setting Up the Project

Install dependencies locally:

npm install

Run the application:

npm start

Test the application:

npm test

Run the application on iOS:

bun run build && \
bun x cap sync && \
bun x cap run ios

Cite

@misc{moryossef2023signmt,
    title={sign.mt: Effortless Real-Time Sign Language Translation},
    author={Moryossef, Amit},
    howpublished={\url{https://sign.mt/}},
    year={2023}
}