Trending...
- Governor Polis Discusses Colorado's National Leadership In Education at the National Governors Association Convening in Oklahoma City
- Colorado: Governor Polis Orders Flags to Fly at Half-Staff to Honor Heroic Wildland Fire Service Firefighter Who Died Fighting the Knowles Fire in Mesa County
- Colorado: Governor Polis Announces Staff Updates
BOULDER, Colo. - ColoradoDesk -- Built on Vision Agents with Anam and Inworld to demonstrate emotionally aware, video-first AI
Stream released an open-source AI agent that responds to a user's facial expressions, gaze, and engagement in real time. The agent, called Crashout Buddy, is live at visionagents.ai.
The era of the floating orb is over. Most voice agents today are blind. They convert speech to text, run it through an LLM, and read the response back in a flat tone regardless of whether the user is laughing, frustrated, or close to tears. Built on Stream's Vision Agents framework in collaboration with Anam and Inworld, Crashout Buddy watches the user's face and shapes both what the agent says and how it says it. When the user goes quiet, it notices. When they look like they're about to lose it, it softens.
How It Works
The agent runs a multimodal perception stack on Stream's global edge network. MediaPipe tracks 52 facial blendshapes at 8 fps to classify emotion, gaze, and engagement. That signal is injected into the LLM (Gemini) on every turn, which steers Inworld's TTS-2 voice model using natural-language direction such as [say warmly with light, easy energy]. Anam renders a photorealistic, lip-synced avatar. Deepgram handles speech-to-text.
More on Colorado Desk
The same pattern (facial state, rich agent context, expressive voice, lip-synced avatar) suits apps in dating, coaching, recruitment, tutoring, and customer support.
Key capabilities include:
Availability
The full project is open source. Try the demo at visionagents.ai, read the guide on the Stream blog, or explore the code at: https://github.com/GetStream/Vision-Agents
Stream released an open-source AI agent that responds to a user's facial expressions, gaze, and engagement in real time. The agent, called Crashout Buddy, is live at visionagents.ai.
The era of the floating orb is over. Most voice agents today are blind. They convert speech to text, run it through an LLM, and read the response back in a flat tone regardless of whether the user is laughing, frustrated, or close to tears. Built on Stream's Vision Agents framework in collaboration with Anam and Inworld, Crashout Buddy watches the user's face and shapes both what the agent says and how it says it. When the user goes quiet, it notices. When they look like they're about to lose it, it softens.
How It Works
The agent runs a multimodal perception stack on Stream's global edge network. MediaPipe tracks 52 facial blendshapes at 8 fps to classify emotion, gaze, and engagement. That signal is injected into the LLM (Gemini) on every turn, which steers Inworld's TTS-2 voice model using natural-language direction such as [say warmly with light, easy energy]. Anam renders a photorealistic, lip-synced avatar. Deepgram handles speech-to-text.
More on Colorado Desk
- RPR Promotes Emily Line to Chief Strategy Officer and Janine Sieja to Chief Product Officer
- Expanding Beyond Space as New Drone Market Opportunities Accelerate Growth: Ascent Solar Technologies (N A S D A Q: ASTI)
- Lauren Merrell, Dale Sorensen Real Estate, announces price improvement for an extraordinary island retreat
- Portalz Publishes FES World First Architecture Introducing a New Cryptographic Platform
- Cellofest Brings Free Cello Concerts and Community Events to Bethany Beach August 5–16
The same pattern (facial state, rich agent context, expressive voice, lip-synced avatar) suits apps in dating, coaching, recruitment, tutoring, and customer support.
Key capabilities include:
- Emotion, gaze, and engagement classification with hysteresis to prevent flicker
- Natural-language voice steering in 100+ languages via Inworld TTS-2
- Photorealistic lip-synced avatar via Anam's CARA model
- Proactive re-engagement when the user drifts off-camera or goes quiet
- Composable processors running at independent frame rates
Availability
The full project is open source. Try the demo at visionagents.ai, read the guide on the Stream blog, or explore the code at: https://github.com/GetStream/Vision-Agents
Source: Getstream.io
0 Comments
Latest on Colorado Desk
- Extreme Heat Strains Home Appliances: Appliance EMT Offers "Summer Rescue" Relief
- Independent West Texas Metal Multi-Instrumentalist & Producer. "MAD CHAD™" Russell Surpasses 1.9 Million Project Interactions Via DFGS Productions
- No Download Needed: Goosechase Adds Browser Play to Every Experience
- Colorado Springs: City to install federally required bridge signs for certain heavy emergency vehicles on Platte Avenue bridge over Shooks Run
- Colorado Springs: Nominations open for Mayor's Young Leader Awards through Aug. 31
- Colorado Springs: Pikes Peak – America's Mountain sets Guinness World Records title for simultaneous doughnut dunking
- Governor Polis and Lt. Governor Primavera Celebrate Colorado's Role in Historic Successful Artemis II Lunar Mission at Red Rocks
- Nonprofit Technology Strategist David Price Resumes Independent Practice at DVPrice Consulting
- Landscape Connection, Inc. Named Finalist in Colorado Biz Magazine's 2026 Best of Colorado Awards
- Governor Polis and America 250 - Colorado 150 Commission Mark Colorado Day Across the State
- Colorado: Governor Polis Celebrates 150 Years of the Centennial State Alongside Coloradans
- Governor Polis Provides Update on Wildfires and Floods in Colorado
- HER Patio Productions celebrates one-year Anniversary of "Say Grace"
- Working Musicians Academy Partners with Black Dog Music Partners to Give Musicians Independent, Income-Producing Careers
- Omnitronics launches Ecosystem Health Dashboard to enable proactive monitoring across dispatch environments
- New Research Identifies "The Great Junk Transfer": 49% of Americans Would Rather Inherit Nothing Than Sort Through a Relative's Belongings
- Minus K Technology launches it Educational Giveaway for Universities and Colleges in the USA
- UK Financial Ltd Verifies Maya Preferred PRA Circulating Supply, Proving Its Eight-Year Promise of Under 1M Tokens After Chainlink Labs Agreement
- Upper Basin States Release Joint Statement on Federal Environmental Impact Statement Regarding Future Colorado River Operations
- Governor Polis Celebrates Colorado as Home to Three of the World's Top 400 Universities