Meta builds AI models for realistic sound experience in VR environments

Meta (formerly Facebook) has built three new artificial intelligence (AI) models designed to make sound more realistic in mixed and virtual reality experiences

Meta
IANS New Delhi
2 min read Last Updated : Jun 27 2022 | 2:29 PM IST

Meta (formerly Facebook) has built three new artificial intelligence (AI) models designed to make sound more realistic in mixed and virtual reality experiences.

The three AL models -- Visual-Acoustic Matching, Visually-Informed Dereverberation and VisualVoice -- focus on human speech and sounds in video and are designed to push "us toward a more immersive reality at a faster rate," the company said in a statement.

"Acoustics play a role in how sound will be experienced in the metaverse, and we believe AI will be core to delivering realistic sound quality," said Meta's AI researchers and audio specialists from its Reality Labs team.

They built the AI models in collaboration with researchers from the University of Texas at Austin, and are making these models for audio-visual understanding open to developers.

The self-supervised Visual-Acoustic Matching model, called AViTAR, adjusts audio to match the space of a target image.

The self-supervised training objective learns acoustic matching from in-the-wild web videos, despite their lack of acoustically mismatched audio and unlabelled data, informed Meta.

VisualVoice learns in a way that's similar to how people master new skills, by learning visual and auditory cues from unlabelled videos to achieve audio-visual speech separation.

For example, imagine being able to attend a group meeting in the metaverse with colleagues from around the world, but instead of people having fewer conversations and talking over one another, the reverberation and acoustics would adjust accordingly as they moved around the virtual space and joined smaller groups.

"VisualVoice generalises well to challenging real-world videos of diverse scenarios," said Meta AI researchers.

--IANS

na/

(Only the headline and picture of this report may have been reworked by the Business Standard staff; the rest of the content is auto-generated from a syndicated feed.)

Subscribe to Business Standard digital and get complimentary access to The New York Times

Quarterly Starter

₹900

3 Months

₹300/Month

SAVE 25%

Smart Essential

₹2,700

1 Year

₹225/Month

Save 46%

Super Saver

₹3,900

2 Years

₹162/Month

Subscribe

Renews automatically, cancel anytime

Here’s what’s included in our digital subscription plans

Access to Exclusive Premium Stories Online

  • Over 30 behind the paywall stories daily, handpicked by our editors for subscribers

Complimentary Access to The New York Times

  • News, Games, Cooking, Audio, Wirecutter & The Athletic

Business Standard Epaper

  • Digital replica of our daily newspaper — with options to read, save, and share

Curated Newsletters

  • Insights on markets, finance, politics, tech, and more delivered to your inbox

Market Analysis & Investment Insights

  • In-depth market analysis & insights with access to The Smart Investor

Archives

  • Repository of articles and publications dating back to 1997

Ad-free Reading

  • Uninterrupted reading experience with no advertisements

Seamless Access Across All Devices

  • Access Business Standard across devices — mobile, tablet, or PC, via web or app

More From This Section

Topics :Facebookvirtual realityartifical intelligence

First Published: Jun 27 2022 | 2:29 PM IST

Next Story