This Company Is Deepfaking Your Whole Body to Make Super Realistic AI Avatars

It was only a few months ago when Synthesia showed off its “Expressive AI Avatars” that impressed with its creepily realistic facial expressions and voices. Now the company is taking it to the next level with Synthesia 2.0, which will soon be able to generate full-body avatars from scratch using AI.

Synthesia’s 2.0 demo above is short, but it shows off six very convincing examples of full-body AI avatars in action. These examples don’t have any voice yet, but Synthesia is likely going to combine them with the Express AI Avatar’s realistic human voice making them a lot more realistic in the near future.

A More Full-Bodied Experience

Instead of just a talking head, Synthesia 2.0’s AI-generated avatars incorporate hand and arm gestures, along with body language. As a user, you can fully control the AI avatars by creating animations with skeleton sequences. On top of that, you can still customize your AI avatar’s appearance with your own images and videos. According to MIT Technology Review, Synthesia 2.0’s avatars will be able to sing and dance with a microphone in their hand, walk across a room, and come from behind a desk. Victor Riparbelli, Synthesia’s CEO, also told MIT Technology Review that the updated AI avatars can show more complex emotions, like excitement, fear, and nervousness.

To get these ultrarealistic movements, Synthesia said that it took 12 months of capturing thousands of people in studios around the world. With that dataset, the company trained its large video and audio foundational models where it can now produce those convincing examples as seen in the demo. Synthesia 2.0’s full-body avatars feature isn’t out yet, but it’s already a more promising tool than we see with some similar technologies, like Microsoft’s VASA-1 project.

Coming As Early as 2024

Synthesia promised more news about the full-body AI avatars later this year, and Riparbelli told MIT Technology Review that these updated avatars will launch towards the end of the year. As of now, Synthesia 2.0’s other updates let you create a custom AI avatar at home using your webcam or phone, an AI Video Assistant that can transform articles into videos, and an improved translation experience.

Even though Synthesia is designed for business use in mind, these full-body AI avatars set a new bar in the deepfake world. They also open what could potentially be a potentially dangerous can of worms beyond their intended use.

The increasing sophistication of deepfakes has already resulted in many states trying to enact legislation to limit the prevalence, especially in the context of elections and non-consensual sexual material. Synthesia seems to take those potential misues seriously, but only time will tell if deepfaking your whole body is actually a good idea.