Complete open-source example for integrating animated avatars with ElevenLabs conversational AI using Mascotbot SDK. Real-time lip sync, WebSocket support, and production-ready React components.
After deploying with Vercel:
- Add the Mascotbot SDK package (
mascotbot-sdk-react-0.1.6.tgz) to your cloned repository - Add your mascot
.rivfile to thepublicfolder - Commit and push these changes to trigger a rebuild
- Node.js 18+
- Mascotbot SDK (provided as
.tgzfile after subscription) - Mascot
.rivfile (provided with SDK subscription) - ElevenLabs API key and Agent ID
- Mascotbot API key
- Clone this repository:
git clone https://github.com/mascotbot/elevenlabs-avatar.git
cd elevenlabs-avatar- Copy the Mascotbot SDK package to the project root:
cp /path/to/mascotbot-sdk-react-0.1.6.tgz ./- Copy your mascot .riv file to the public folder:
cp /path/to/mascot.riv ./public/- Install dependencies:
npm install
# or
pnpm install- Set up environment variables:
cp .env.example .env.local- Update
.env.localwith your credentials:
MASCOT_BOT_API_KEY=your_mascot_bot_api_key
ELEVENLABS_API_KEY=your_elevenlabs_api_key
ELEVENLABS_AGENT_ID=your_elevenlabs_agent_id- Run the development server:
npm run dev
# or
pnpm devOpen http://localhost:3000 to see the demo in action!
This example demonstrates:
- Real-time Lip Sync: Perfect viseme synchronization with ElevenLabs audio streams
- WebSocket Integration: Automatic data extraction from ElevenLabs connections
- Natural Mouth Movements: Human-like lip sync processing that avoids robotic over-articulation
- Production-Ready Components: Complete implementation ready for deployment
elevenlabs-avatar/
├── src/
│ ├── app/
│ │ ├── page.tsx # Main demo page with ElevenLabs avatar
│ │ ├── layout.tsx # Root layout
│ │ ├── globals.css # Global styles
│ │ └── api/
│ │ └── get-signed-url/
│ │ └── route.ts # API endpoint for ElevenLabs authentication
│ └── components/ # Additional components (if needed)
├── public/ # Static assets
├── .env.example # Environment variables template
├── package.json # Project dependencies
└── README.md # This file
The Mascotbot proxy endpoint automatically injects viseme (mouth shape) data into the ElevenLabs WebSocket stream, enabling perfect lip synchronization without any modifications to ElevenLabs code.
// Human-like mouth movements with configurable parameters
// Important: Use useState to maintain stable object reference
const [lipSyncConfig] = useState({
minVisemeInterval: 40,
mergeWindow: 60,
keyVisemePreference: 0.6,
preserveSilence: true,
similarityThreshold: 0.4,
preserveCriticalVisemes: true,
criticalVisemeMinDuration: 80,
});
useMascotElevenlabs({
conversation,
naturalLipSync: true,
naturalLipSyncConfig: lipSyncConfig,
});The demo pre-fetches signed URLs and refreshes them every 9 minutes, ensuring instant connection when users click "Start Conversation".
The demo expects a mascot .riv file in the public folder. The file path is configured in src/app/page.tsx:
const mascotUrl = "/mascot.riv"; // Place your .riv file in the public folderYou can also use a CDN URL:
const mascotUrl = "https://your-cdn.com/your-mascot.riv";Ensure your Rive file has the required inputs:
is_speaking- Boolean input for lip syncgesture- Optional trigger for animated reactions
The demo includes full configuration for natural lip sync. Always use useState to maintain a stable object reference:
const [lipSyncConfig] = useState({
minVisemeInterval: 40, // Minimum time between visemes (ms)
mergeWindow: 60, // Window for merging similar shapes
keyVisemePreference: 0.6, // Preference for distinctive shapes (0-1)
preserveSilence: true, // Keep silence visemes
similarityThreshold: 0.4, // Threshold for merging (0-1)
preserveCriticalVisemes: true,// Never skip important shapes
criticalVisemeMinDuration: 80,// Min duration for critical visemes (ms)
});You can adjust these values based on your needs:
- Higher
minVisemeInterval: Smoother, less articulated movements - Lower
minVisemeInterval: More precise articulation - Higher
keyVisemePreference: More emphasis on distinctive mouth shapes - Higher
similarityThreshold: More aggressive merging of similar visemes
The demo uses Tailwind CSS for styling. Modify the classes in src/app/page.tsx to match your design requirements.
Create a .env.local file with the following variables:
# Mascotbot API Key (get from app.mascot.bot)
MASCOT_BOT_API_KEY=mascot_xxxxxxxxxxxxxx
# ElevenLabs Credentials
ELEVENLABS_API_KEY=sk_xxxxxxxxxxxxxx
ELEVENLABS_AGENT_ID=agent_xxxxxxxxxxxxxxDo NOT connect directly to ElevenLabs WebSocket URLs. The avatar lip-sync requires viseme data that only the Mascotbot proxy provides. Direct connections will result in no mouth movement.
- Modern browser with WebGL2 support
- Microphone access for voice interaction
- Stable internet connection for WebSocket streaming
- Less than 50ms audio-to-visual delay
- WebGL2 acceleration for smooth 120fps animation
- Minimal CPU usage (less than 1%)
- Check browser console for WebSocket errors
- Verify environment variables are set correctly
- Ensure Rive file has correct input names (
is_speaking,gesture) - Confirm you're using the Mascotbot proxy endpoint, not direct ElevenLabs connection
- Verify your API keys are correct
- Check that your ElevenLabs agent is active
- Ensure microphone permissions are granted
- Look for errors in the browser console
- Check network latency
- Adjust natural lip sync parameters
- Try different presets based on speech speed
For complete documentation on the Mascotbot SDK and all available features, visit:
This demo is provided as an open-source example for Mascotbot subscribers. You're free to use, modify, and deploy it as needed for your projects.
- For SDK issues: support@mascot.bot
- For ElevenLabs issues: ElevenLabs Support
- Community: Discord Server
Built with ❤️ by the Mascotbot team