Meta (previously Facebook) is continuing its transition to the virtual world.
This time, the company intends to improve audio when users enjoy AR or VR experiences, such as attending a party in the metaverse or viewing a home movie in their living room while wearing AR glasses.
Researchers at Meta AI are open-sourcing three new models for the audio-visual understanding of human speech and sounds in video, in collaboration with an audio specialist from Meta’s Reality Labs and researchers from the University of Texas at Austin, in order to accelerate their progress toward this reality.
The three models–Visual Acoustic Matching, Visually Informed Dereverberation, and VisualVoice–will assist Meta in bringing consumers even closer to multimodal, immersive experiences and making them feel immersed not only by the images but also by the sounds.
Since announcing the NFT integration on Instagram, Meta has taken massive strides to increase its market share in the new world. It has filed trademark applications for Meta Pay to facilitate payments on its platforms and in the metaverse.
Meta recently announced the launch of its Avatars Store on Instagram, Facebook, and Messenger, where users will be able to purchase digital clothes to dress their avatars.
Meta also announced the establishment of its first physical store, ‘The Meta Store,’ two months ago, where users can examine and try its virtual reality hardware devices.
