How Google’s New Expressive Captions Add Emotional Context
by: Allegra E. Klein, MBA, Director of Advocacy Eastern CT Chapter
allegra.klein@biz.uconn.edu | allegra@post.harvard.edu
For those of us in the hard of hearing community, captions have long been our essential bridge to understanding conversations and environmental sounds. Now, Google’s new Expressive Captions feature is taking captioning to the next level by adding emotional context to what we read on screen. Accessibility advocate Shari Eberts recently explored this exciting development, highlighting both its promise and important considerations for our community.
Launched on Android phones in December, Expressive Captions uses AI to communicate not just spoken words, but the tone and volume behind them. When a sports commentator announces a touchdown with explosive excitement, you’ll see those words in ALL CAPS. When your friend enthusiastically exclaims “Happy Birthday!” the capitalization conveys celebration, not casual mention. The technology captures environmental sounds like applause and cheers, plus vocal expressions such as sighs or gasps. This means when your team scores, you’ll experience both the moment and the crowd’s electric reaction through captions.
This innovation addresses a real challenge for people with hearing loss who sometimes struggle to interpret emotions in spoken language due to listening fatigue or other factors. Having emotional intent clearly spelled out can be incredibly helpful, especially for live events and social content without professional captions. However, as Eberts notes, we must ensure these enhancements don’t compromise transcription accuracy or legibility. The primary purpose of captioning remains communication access—emotional cues should enhance, not distract from, comprehension.
Expressive Captions are currently available in English on Android 14+ devices with Live Caption for livestreams and video messages. As this technology evolves, we may see similar features in pre-scripted content and potentially from other tech companies. Let’s continue to push accessibility forward and recognize that communication involves far more than just words.