ALFREDO

Alfredo - Sign in silence. Speak to the world.
On-device Indian Sign Language interpreter , Camera in. Speech out.

The Problem:
Millions of Deaf signers in India can't carry a human interpreter everywhere. At a pharmacy, ticket counter, classroom, or hospital, there is no interpreter on-demand. Interpreters are scarce and expensive, and most existing tools only work on pre-recorded videos but real conversation is live.

What Alfredo Does:
Point any laptop webcam at a signer and Alfredo tracks the hands, recognizes the signs, builds the sentence, and speaks it all in the browser. No install, no video upload.

1. Live Interpret: Sign naturally with no buttons. Motion-based segmentation auto-detects sign boundaries, recognition runs against your own recordings (96.8% accuracy, 50ms), grammar smoothing turns `I. Pain. Help. Doctor.` into `I am in pain. I need help. I need a doctor.` and speaks it after 2.2s pause or instantly.
2. Record Your Own Vocabulary: 25-sign core set, 15 takes per sign. 3-2-1 countdown, 2-sec capture, skeleton replay, Keep / Discard, batch labels for lighting/angle, export as JSON. Custom words like names can be added and used immediately with no retraining.
3. AI Language Brain (Gemini API): When offline grammar can't cover a phrase, Gemini converts ISL gloss words to natural English, reconstructs broken fingerspelling fragments like `WATR / WATRE` to `water` from context, and translates English to Hindi in Devanagari for Hindi voice output.
4. Two-way Talk: Type or speak and the 3D avatar signs it back  text to ISL gloss to pose to avatar.
5. Media Translation: Upload YouTube links, PDFs, docs backend transcribes with Whisper and translates for signing.

Privacy by Architecture:
No video is ever stored. Only 21-point hand skeletons (~54KB per sign). Tracking runs locally via MediaPipe WASM after first load it works offline. Database uses Supabase with Row Level Security.

Tech: React, Vite, Tailwind, MediaPipe Tasks Vision, Three.js / React Three Fiber, FastAPI, faster-whisper, FLAN-T5, Gemini API, Supabase, Vercel