Image credit — Google
Android users can now experience a whole new level of captioning with the release of “Expressive Captions.” This new feature goes beyond just showing the words people say — it actually captures how they say them. Imagine being able to see the emotion and intensity in someone’s voice, even if you can’t hear it.For years, captions have simply displayed the spoken words, but now, thanks to AI, they can do so much more. Expressive Captions analyze things like tone of voice, loudness, and even background sounds to give you a richer understanding of what’s happening. This is especially helpful for live videos and social media posts, where captions are often missing or not very accurate.
An example of ‘Expressive Captions’ turned ON vs OFF | Image credit — Google
One of the most interesting things about the new Expressive Captions is how it uses capital letters to show strong emotions. So, if your friend sends you a birthday message and shouts “HAPPY BIRTHDAY!” you’ll see those words in all caps in the captions, just like how we’ve learned that when using all caps in texts means you’re shouting. The feature can also pick up on things like sighs, gasps, and even clapping or cheering in the background, giving you a better sense of the whole scene.
How Android’s new ‘Expressive Captions’ capture emotion | Image credit — Google
Expressive Captions are built right into the latest Android phones, so they work with pretty much any app where you might watch videos. The feature will be available starting today for any Android that is running Android 14 and above and that has the Live Caption feature turned on. For now, it will only be available in the U.S. in English.
This is a big step forward in making sure everyone can enjoy online videos, no matter how well they can hear. It shows how AI can be used to make things better for everyone.
I watch a lot of videos on my phone, so I’m pretty excited about Expressive Captions. Being able to see the emotion and hear the background sounds through the captions would make watching videos in noisy places a much better experience. I can’t wait to see if this comes to other languages and regions soon.