Meta (formerly Facebook) has built three new artificial intelligence (AI) models designed to make sound more realistic in mixed and virtual reality experiences.
The three AL models -- Visual-Acoustic Matching, Visually-Informed Dereverberation and VisualVoice -- focus on human speech and sounds in video and are designed to push "us toward a more immersive reality at a faster rate," the company said in a statement.
"Acoustics play a role in how sound will be experienced in the metaverse, and we believe AI will be core to delivering realistic sound quality," said Meta's AI researchers and audio specialists from its Reality Labs team.
They built the AI models in collaboration with researchers from the University of Texas at Austin, and are making these models for audio-visual understanding open to developers.
The self-supervised Visual-Acoustic Matching model, called AViTAR, adjusts audio to match the space of a target image.
The self-supervised training objective learns acoustic matching from in-the-wild web videos, despite their lack of acoustically mismatched audio and unlabelled data, informed Meta.
VisualVoice learns in a way that's similar to how people master new skills, by learning visual and auditory cues from unlabelled videos to achieve audio-visual speech separation.
For example, imagine being able to attend a group meeting in the metaverse with colleagues from around the world, but instead of people having fewer conversations and talking over one another, the reverberation and acoustics would adjust accordingly as they moved around the virtual space and joined smaller groups.
"VisualVoice generalises well to challenging real-world videos of diverse scenarios," said Meta AI researchers.
--IANS
na/
(Only the headline and picture of this report may have been reworked by the Business Standard staff; the rest of the content is auto-generated from a syndicated feed.)
You’ve hit your limit of {{free_limit}} free articles this month.
Subscribe now for unlimited access.
Already subscribed? Log in
Subscribe to read the full story →
Quarterly Starter
₹900
3 Months
₹300/Month
Smart Essential
₹2,700
1 Year
₹225/Month
Super Saver
₹3,900
2 Years
₹162/Month
Renews automatically, cancel anytime
Here’s what’s included in our digital subscription plans
Access to Exclusive Premium Stories Online
Over 30 behind the paywall stories daily, handpicked by our editors for subscribers


Complimentary Access to The New York Times
News, Games, Cooking, Audio, Wirecutter & The Athletic
Business Standard Epaper
Digital replica of our daily newspaper — with options to read, save, and share


Curated Newsletters
Insights on markets, finance, politics, tech, and more delivered to your inbox
Market Analysis & Investment Insights
In-depth market analysis & insights with access to The Smart Investor


Archives
Repository of articles and publications dating back to 1997
Ad-free Reading
Uninterrupted reading experience with no advertisements


Seamless Access Across All Devices
Access Business Standard across devices — mobile, tablet, or PC, via web or app