How Google’s New Expressive Captions Add Emotional Context
by: Allegra E. Klein, MBA, Director of Advocacy Eastern CT Chapter
allegra.klein@biz.uconn.edu | allegra@post.harvard.edu
For those of us in the hard of hearing community, captions have long been our essential bridge to understanding conversations and environmental sounds. Now, Google’s new Expressive Captions feature is taking captioning to the next level by adding emotional context to what we read on screen. Accessibility advocate Shari Eberts recently explored this exciting development, highlighting both its promise and important considerations for our community.
Launched on Android phones in December, Expressive Captions uses AI to communicate not just spoken words, but the tone and volume behind them. When a sports commentator announces a touchdown with explosive excitement, you’ll see those words in ALL CAPS. When your friend enthusiastically exclaims “Happy Birthday!” the capitalization conveys celebration, not casual mention. The technology captures environmental sounds like applause and cheers, plus vocal expressions such as sighs or gasps. This means when your team scores, you’ll experience both the moment and the crowd’s electric reaction through captions.
This innovation addresses a real challenge for people with hearing loss who sometimes struggle to interpret emotions in spoken language due to listening fatigue or other factors. Having emotional intent clearly spelled out can be incredibly helpful, especially for live events and social content without professional captions. However, as Eberts notes, we must ensure these enhancements don’t compromise transcription accuracy or legibility. The primary purpose of captioning remains communication access—emotional cues should enhance, not distract from, comprehension.
Expressive Captions are currently available in English on Android 14+ devices with Live Caption for livestreams and video messages. As this technology evolves, we may see similar features in pre-scripted content and potentially from other tech companies. Let’s continue to push accessibility forward and recognize that communication involves far more than just words.
Presentation by Margaret Downs and Mathew CallCaption Companion is a portable 8-inch tablet device with long-lasting battery that offers unlimited, fast, and accurate realtime AI captioning right out of the box. Caption Companion provides unlimited captioning. No account, no sign-up, no subscription… not even an internet connection is required. Caption Companion comes uniquely paired with a SmartMic for unprecedented accuracy and flexibility. Several other useful accessories are also included that make Caption Companion versatile in any situation. Caption Companion is the first automatic speech recognition (ASR) product to 
