AiSL 👐

Translate written text to ASL gloss & signs — web app + browser extension

---

🧑‍💻 What is Ai-SL?

Ai-SL is an AI-powered tool that helps make written content more accessible to Deaf and Hard of Hearing users who use American Sign Language (ASL).

The project consists of:

✅ A web app that lets users upload documents (TXT / PDF), which are automatically translated into ASL gloss, with linked videos of ASL signs.

✅ A browser extension that allows users to highlight any text on a webpage and instantly view an ASL interpretation — right in a popup window!

✅ A Hugging Face Space where you can test our backend by inputting your own text or files and seeing the generated ASL videos in real-time! --> Hugging Face Space

---

💡 Why we built it — Social Impact

For many Deaf and Hard of Hearing individuals, English is a second language, and reading complex written materials (academic papers, government forms, textbooks) can be difficult.

ASL is a visual language with its own grammar — very different from written English.
However, live interpreters are not always available:

• Many students can’t get interpreters for all their classes
• Booking interpreters is expensive and time-limited
• Self-study of written material (textbooks, articles, web content) is often inaccessible
• Automatic captioning does not translate into ASL structure

As a result, many Deaf users face barriers to independent learning, work, and daily life.

Ai-SL helps bridge this gap:

✅ It offers instant, on-demand ASL gloss of any text

✅ Works for self-study — no need to schedule an interpreter

✅ Brings accessibility to any website, any document

✅ Available 24/7 — helping users learn, explore, and access more content on their own terms

---

🏗️ Project Structure

AI-SL/
├── extension/                # Browser extension
│   ├── icons/
│   ├── api-integration-example.js
│   ├── background.js
│   ├── content\_script.js
│   ├── content\_styles.css
│   ├── install.sh
│   ├── manifest.json
│   ├── package.json
│   ├── popup.css
│   ├── popup.html
│   ├── popup.js
│   └── README.md
│
├── src/backend/              # Python backend (Flask)
│   ├── asl\_gloss.py
│   ├── backend.py
│   ├── document\_parsing.py
│   └── document\_to\_gloss.py
│
├── web-app/                  # Frontend web app (React or other)
│   └── app.py
│
├── flask\_requirements.txt
├── requirements.txt
├── INTEGRATION\_GUIDE.md
├── README\_FLASK.md
├── package-lock.json
└── README.md

---

🚀 How to run

Backend (Flask API):

# install dependencies
pip install -r requirements.txt

# run backend
python src/backend/backend.py
# or
python app.py

---

Web app:

cd web-app
npm install
npm run dev
# App runs on localhost:3000

---

Browser extension:

1. Go to chrome://extensions in Chrome/Edge
2. Enable "Developer mode"
3. Click "Load unpacked extension" → select the extension/ folder
4. Highlight any text → click “🤟 Translate to ASL” → popup shows ASL gloss + videos

---

🧠 AI / ML Components

• LLM (OpenAI GPT-4 or similar): converts English text → ASL gloss
• ASL video dataset: Signbank, ASL-LEX, How2Sign (for linked sign videos)
• (Optional) MediaPipe Holistic: to add visual consistency / overlays to videos

---

💡 Features

✅ Upload TXT/PDF → ASL gloss + sign video links ✅ Highlight any text on any page → ASL popup ✅ Clean, accessible glassmorphism UI ✅ Browser extension (Manifest V3 compatible) ✅ Modular architecture — easy to extend with new sign datasets or avatars

---

⚠️ Limitations

• Some ASL regional variations not covered
• The avatar is not in its best shape, but we are working on it :)

---

🙌 Team

Built by Nataliia Kulieshova, Ethan Tam, Deena Sun, and Jeffrey Gong at the Berkeley AI Hackathon 2025 🚀

---

License

MIT License