YojanaSetu
# YojanaSetu (योजना सेतु)
### *AI-Powered Multilingual Citizen Welfare Assistant & Scheme Discovery Portal*
---
## 1. Executive Summary
Across central and state administrations in India, more than **3,400+ government welfare programs, subsidies, and financial assistance schemes** exist to uplift underprivileged and developing communities. However, a significant delivery gap persists: millions of eligible citizens—rural farmers, daily-wage laborers, weavers, artisans, gig workers, and low-income families—remain unaware of the benefits they are legally entitled to receive.
The primary barriers are:
1. **Language Fragmentation:** Government portals and gazettes are frequently published in complex administrative language or English/Hindi only.
2. **Bureaucratic Jargon & Digital Illiteracy:** Complex multi-step eligibility criteria and formal terminology overwhelm non-technical citizens.
3. **Information Asymmetry:** Citizens do not know which department or ministry operates the scheme relevant to their specific socio-economic profile.
**YojanaSetu** ("Bridge to Welfare Schemes") resolves this barrier through a **voice-first, multilingual, accessibility-certified civil assistance platform**. Citizens speak naturally in their native mother tongue (Hindi, Bengali, English, or regional dialects). The platform directly analyzes spoken audio, extracts demographic constraints, queries a curated hybrid database of 3,400+ welfare programs via Retrieval-Augmented Generation (RAG), and provides personalized eligibility scores alongside natural spoken voice guidance in authentic regional accents.
---
## 2. Key Highlights & Value Proposition
* **🎙️ Voice-First Civic Interaction:** No typing or navigation hurdles. Citizens simply press the microphone and explain their life situation (e.g., *"I am a small wheat farmer from Madhya Pradesh with 2 acres of land and two school-going daughters"*).
* **🧠 Direct Speech-to-Meaning:** Uses **Google Gemini Flash** multimodal processing and **Sarvam AI** (Saaras Indic Speech Engine) to parse colloquial speech into structured demographic parameters (age, state, gender, occupation, caste, annual income).
* **🔊 Authentic Vernacular Spoken Responses:** Delivers human-grade audio answers via **Sarvam Bulbul** and **ElevenLabs** (`eleven_multilingual_v2`), with an automatic client-side Web Speech fallback.
* **🎯 Precision Eligibility Matching (`/recommendations`):** Cross-matches citizen profiles against the 3,400+ scheme database to deliver percentage-ranked matches (65% to 99% Perfect Match) with clear explanations of why they qualify.
* **🔎 Faceted Civic Search (`/search`):** Real-time directory searchable by ministry, state, beneficiary group, and financial support type.
* **♿ Built-in WCAG 2.2 AAA Accessibility:** Text scaling up to 200%, dyslexia-friendly typography (OpenDyslexic), line-spacing controls, high-contrast monochrome mode, and dual visual themes (Dark Obsidian vs. Calm Linen & Sage).
* **🌐 Native Trilingual Interface:** Seamless instant switching between **English**, **Hindi (हिन्दी)**, and **Bengali (বাংলা)**.
---
## 3. System Architecture & Information Flow
```mermaid
flowchart TD
subgraph Client ["Frontend (Next.js 16 / React 19 / Tailwind v4)"]
A[Citizen Spoken Voice Input] --> B[MediaRecorder / Audio Blob]
U[Accessibility Bar & Theme Controller] -. Controls .-> V[UI Canvas & Typography]
W[Faceted Search & Recommendations] --> X[Scheme Cards & Modal Viewers]
end
subgraph BackendGateway ["FastAPI ASGI Server (Python 3.11 / 3.14)"]
B --> C[POST /api/voice/process-multimodal]
D[Client Profile Data] --> E[POST /api/search/recommendations]
end
subgraph AIServices ["Multimodal AI & Speech Engine"]
C --> F{Speech Engine Routing}
F -->|Indic Audio| G[Sarvam AI Saaras STT & Translation]
F -->|Multimodal Audio| H[Google Gemini Flash 1.5/2.0]
H --> I[Demographic Constraint Extraction]
end
subgraph RAGCore ["Hybrid Knowledge & Retrieval Pipeline"]
I --> J[Scheme Matching Engine]
E --> J
J --> K[(3,400+ Scheme Relational Store & In-Memory RAG Index)]
K --> L[Scored Schemes + Eligibility Explanations]
end
subgraph SpeechSynthesis ["Spoken Response Synthesis"]
L --> M[Structured Vernacular Response Text]
M --> N{Audio Synthesis Router}
N -->|Indic Native Speakers| O[Sarvam Bulbul TTS]
N -->|Global Audio Stream| P[ElevenLabs Multilingual v2]
N -->|Offline/Quota Fallback| Q[Web Speech API Client Fallback]
end
L --> X
O --> R[Audio Streaming Response]
P --> R
R --> S[Citizen Hears Spoken Guidance]
```
---
## 4. Technology Stack
### Frontend Architecture
| Layer | Technology | Details & Purpose |
| :--- | :--- | :--- |
| **Framework** | **Next.js 16 (App Router)** | High-speed server rendering, modular routes, optimized client hydration |
| **Library** | **React 19** | Pure JavaScript JSX components without TypeScript compilation overhead |
| **Styling** | **Tailwind CSS v4** | Native `@theme` tokens, dynamic CSS variables, zero runtime CSS footprint |
| **Iconography** | **Lucide React** | Consistent, professional civic vector icons (no informal emojis) |
| **Internationalization** | **i18next / Custom Context** | Multi-locale JSON translation tables (`en`, `hi`, `bn`) |
| **State Management** | **React Context** | `AuthContext` (Citizen Profiles), `AccessibilityContext` (WCAG settings) |
### Backend Architecture
| Layer | Technology | Details & Purpose |
| :--- | :--- | :--- |
| **Web Server** | **FastAPI (ASGI)** | High-throughput asynchronous endpoints with Pydantic v2 schemas |
| **Language Runtime**| **Python 3.11 / 3.14** | Fast asynchronous I/O and data processing pipelines |
| **Database & ORM** | **PostgreSQL + SQLAlchemy** | Relational schema storage with `psycopg3` and fallback in-memory indexing |
| **AI Inference** | **Google Gemini Flash** | Direct multimodal audio comprehension and parameter extraction |
| **Indic Speech** | **Sarvam AI** | Specialized Indian speech-to-text, translation, and Bulbul TTS |
| **Voice Synthesis**| **ElevenLabs** | Studio-grade multilingual streaming voice output |
| **Client Transport**| **HTTPX & StreamingResponse**| Low-latency chunked audio streaming directly to the browser |
---
## 5. Functional Module Breakdown
### 5.1 Multimodal Voice Assistant (`/` & `VoiceAssistant.jsx`)
* **Live Audio Streaming:** Records microphone input into WebM/WAV containers and transfers raw audio to `/api/voice/process-multimodal`.
* **Zero-Hallucination Extraction:** The backend prompts Gemini/Sarvam with strict schema validation to return:
- Extracted citizen parameters: State, District, Gender, Age, Occupation, Income Category, Disability Status, Social Category.
- Plain-language explanation in the citizen's own language.
- Filter criteria to query the scheme dataset.
* **Synchronized Audio Playback:** Employs an audio waveform equalizer animation that reacts in real-time as the citizen speaks and as the assistant responds.
### 5.2 Personalized Scheme Recommendations (`/recommendations`)
* **Multi-Factor Scoring Engine:** Evaluates demographic suitability against eligibility rules:
- **State Level Matching:** Central government schemes (nationwide) + State-specific schemes matching citizen's state.
- **Occupational Filtering:** Normalizes citizen occupations (e.g. Farmer, Artisan, Daily Wage Worker, Student, Entrepreneur, Senior Citizen).
- **Demographic Exclusions:** Validates gender eligibility (Women-specific programs, Maternity assistance), age thresholds, and caste-based reservation benefits.
* **Match Confidence Indicators:** Each scheme card displays an eligibility percentage score (e.g., `96% Match`), matched demographic tags, and bulleted qualifications.
* **Citizen Onboarding Tracker:** Offers a 3-step profile completion flow for guests to input their details or explore with preset civic personas.
### 5.3 Scheme Directory & Faceted Search (`/search`)
* **Full-Text & Relational Search:** Rapid querying across titles, descriptions, objectives, and ministries.
* **Civic Filters:**
- Level: All India / Central vs. State-specific.
- Category: Agriculture, Healthcare, Education, Social Welfare, Employment, Housing, Financial Inclusion.
- Beneficiary Type: Women, Farmers, Students, Senior Citizens, BPL Families, Persons with Disabilities.
* **Interactive Scheme Detail Modal:** Presents comprehensive scheme dossiers including key benefits, required documentation checklists, eligibility criteria, and step-by-step application procedures.
### 5.4 Universal Accessibility Engine (WCAG 2.2 AAA Compliant)
Integrated directly into the sticky top navigation utility bar:
* **Text Resizing Scale:** Seamless dynamic scaling across 4 presets: 100%, 125%, 150%, and 200% without breaking grid layouts.
* **Line Height Spacing:** Toggle between Normal, Relaxed, and Loose line-heights for enhanced readability.
* **Dyslexia-Friendly Typography:** Instant toggling of the OpenDyslexic font family across all document text.
* **High Contrast Mode:** High-visibility monochrome theme with bright contrast borders and zero background glare.
* **Anti-Fatigue Dual Visual System:**
- **Obsidian Night (Dark Mode):** Deep slate canvas (`#020617` / `#0B1120`) with crisp high-contrast cards.
- **Linen & Sage (Light Mode):** Inspired by human editorial paper craft, strictly adhering to:
- Primary Accent: `#8B9A6E` (Calm Sage / Olive)
- Canvas / Background: `#F7F2EB` (Warm Linen / Ivory Paper)
- Cards & Panels: `#EAE2D6` (Warm Oat / Sandstone)
- Chips & Secondary Buttons: `#EEEEEE` (Soft Stone)
- Text: `#242A1D` (Deep Forest Espresso Charcoal, 11:1+ AAA contrast ratio)
---
## 6. Backend API Specifications
| Method | Endpoint | Description |
| :--- | :--- | :--- |
| `POST` | `/api/voice/process-multimodal` | Accepts multipart audio blob, transcribes, extracts demographics, and returns matched schemes with spoken audio text |
| `POST` | `/api/voice/tts` | Generates streaming voice audio (MP3) from text using Sarvam or ElevenLabs |
| `GET` | `/api/search/schemes` | Returns paginated schemes with query, category, state, and level filters |
| `POST` | `/api/search/recommendations` | Computes personalized scheme recommendations based on citizen demographic payload |
| `GET` | `/api/search/schemes/{id}` | Fetches detailed metadata for a single scheme |
| `POST` | `/api/auth/profile` | Persists or updates citizen demographic attributes in session or database |
| `GET` | `/api/health` | Service health check returning database status and external API connectivity |
---
## 7. Directory Structure
```
YojanaSetu/
├── backend/
│ ├── app/
│ │ ├── core/ # Configuration, security, database connectors
│ │ ├── routers/ # FastAPI API routes (voice, search, auth, health)
│ │ ├── services/ # AI & RAG engines (gemini.py, sarvam.py, elevenlabs.py, rag.py)
│ │ ├── database.py # SQLAlchemy models & database session lifecycle
│ │ ├── models.py # Scheme and User database models
│ │ ├── schemas.py # Pydantic input/output validation models
│ │ └── main.py # FastAPI application entrypoint & middleware
│ ├── scripts/ # Dataset ingestion scripts (Kaggle/government data)
│ ├── requirements.txt # Python dependencies
│ └── run.py # Uvicorn launcher script
├── frontend/
│ ├── src/
│ │ ├── app/
│ │ │ ├── about/ # About platform & civic mission page
│ │ │ ├── recommendations/ # Personalized eligibility dashboard
│ │ │ ├── search/ # Scheme directory & filter search page
│ │ │ ├── layout.js # Root layout with accessibility providers
│ │ │ ├── page.js # Homepage hero & interactive voice hub
│ │ │ └── globals.css # Tailwind v4 theme tokens & WCAG styling
│ │ ├── components/
│ │ │ ├── AccessibilityDrawer.jsx # Slide-out assistive controls
│ │ │ ├── BentoDashboard.jsx # Interactive stats & category grid
│ │ │ ├── CitizenLoginModal.jsx # One-click citizen authentication
│ │ │ ├── CitizenProfileModal.jsx # Demographic profile editor
│ │ │ ├── Footer.jsx # Civic portal footer with government links
│ │ │ ├── Navbar.jsx # Sticky civic utility bar & navigation
│ │ │ ├── SchemeCard.jsx # Accessible scheme card component
│ │ │ ├── SchemeModal.jsx # Full scheme details dialog
│ │ │ └── VoiceAssistant.jsx # Audio recording & waveform component
│ │ ├── context/
│ │ │ ├── AccessibilityContext.jsx# Font scale, contrast, theme state
│ │ │ └── AuthContext.jsx # Citizen demographic session state
│ │ └── i18n/ # Localization dictionaries (en, hi, bn)
│ ├── package.json
│ └── next.config.mjs
└── PROJECT_DESCRIPTION.md # Detailed project architectural specification
```
---
## 8. Development & Installation Guide
### Prerequisites
* **Node.js**: v18.18+ or v20+
* **Python**: 3.10+ (Recommended 3.11 or 3.12/3.14)
* **API Keys (Optional for local mocks, required for full AI functionality):**
- `GEMINI_API_KEY` (Google AI Studio)
- `SARVAM_API_KEY` (Sarvam AI Indic Speech)
- `ELEVENLABS_API_KEY` (ElevenLabs Voice Platform)
### Backend Setup
```bash
cd backend
python -m venv venv
# On Windows:
.\venv\Scripts\activate
# On Linux/macOS:
source venv/bin/activate
pip install -r requirements.txt
cp .env.example .env
# Edit .env with your API keys if available
python run.py
# Backend runs at http://localhost:8000 (Swagger docs at /docs)
```
### Frontend Setup
```bash
cd frontend
npm install
npm run dev
# Frontend runs at http://localhost:3000
```
---
## 9. Civic Impact & Conclusion
**YojanaSetu** transforms government welfare delivery from a passive, text-heavy bureaucracy into an active, empathetic civic bridge. By combining voice-first AI with strict accessibility standards and authentic regional language support, YojanaSetu ensures that government support reaches the citizens who need it most—regardless of their literacy level, language, or physical ability.