Microsoft’s Latest AI Animates Faces To Bring Photos to Life

Last week, Microsoft researchers revealed a new AI tool with the ability to transform a static face image and an audio recording of speech into a lifelike video of the person speaking. This innovation generates videos with captivating lip-syncing and realistic face and head movements, either from photorealistic faces, cartoons, or artwork.

Researchers showcased the new model called VASA-1 in a video in which they demonstrated how they animated the classic Mona Lisa image, having her deliver a comical rap that actor Anne Hathaway originally performed.

Microsoft said this innovation could be used for educational purposes or for “improving accessibility for individuals with communication challenges,” or possibly to create virtual companions for humans. However, the tech giant also said it can see how the tool can be abused and negatively utilized to impersonate real people.

This problem extends further than Microsoft, as the rapid increase of sophisticated tools for creating lifelike AI-generated images, videos, and audio enters the market and causes concerns among experts. There are worries about the potential misuse of this technology leading to new types of misinformation. To add to this, there are concerns that this innovation could disrupt creative industries like film and advertising even more.

Currently, Microsoft said it has no plans to release the VASA-1 model to the market publicly yet. This approach mirrors how Microsoft partner OpenAI is addressing concerns regarding its AI-generated video model, Sora. Even though OpenAI introduced Sora in February, access to the model has been restricted to specific professional users and cybersecurity professors for testing purposes only.

In a blog post, Microsoft researchers said, “We are opposed to any behavior to create misleading or harmful contents of real persons.” However, the company added that it does not plan to release the product publicly “until we are certain that the technology will be used responsibly and in accordance with proper regulations.”

Researchers explained that this new tool was trained on many videos of people’s faces while speaking, and it’s designed to recognize natural face and head movements, including “lip motion, (non-lip) expression, eye gaze and blinking, among others.” As a result, VASA-1 can create more realistic videos when animating a static photo.