Google Translate boosted with Gemini as real-time translations come to headphones

Google is bringing advanced Gemini capabilities to its Google Translate tool in order to enable better and more nuanced translations of speech figures as well as support language learners with new educational features, announced the company in a blog post.

Rather than a word-for-word translation that might sound odd to native speakers, Gemini will now help convey the essence of the original text in a more natural manner. The update is rolling out in the U.S. and India in the Translate app, for translations between English and nearly 20 languages, including Spanish, Hindi, Chinese, Japanese, and German.

As part of a new beta experience, headphone users will be able to get real-time translations right in their ears, thanks to Gemini’s new live speech-to-speech translation capabilities.

“Whether you’re trying to have a conversation in a different language, listen to a speech or lecture while abroad, or watch a TV show or film in another language, you can now put in your headphones, open the Translate app, tap “Live translate” and hear a real-time translation in your preferred language,” said Google’s VP of Product/Search, Rose Yao, adding that the tone, emphasis, and cadence of each speaker would be preserved in the process as well.

Google is further adding Duolingo-style language learning features to the Translate app. This will let users gamify the process of learning languages everyday and maintain a “streak” to track their progress. This is being expanded to about 20 new countries including Germany, India, Sweden, and Taiwan.

“We’ve heard from users that they appreciate the customized learning experiences that reflect real-life scenarios. We’ll continue to add new ways for people to make progress on their language learning goals,” noted Google in the blog post.

AI-powered language learning tools are a subject of debate as supporters claim they help keep learners engaged and let them benefit from customised lesson plans. However, critics feel that AI-powered lessons do not support the development of strong language skills that can be applied in real world scenarios.