GPU giant Nvidia is getting into the Christmas spirit, making a far better metaverse-esque avatar than its mate Meta has, too.
Nvidia CEO Jensen Huang has drank his own company’s omniverse juice and converted himself into a ‘toy’ for the online world.
“Since I’m too shy to sing myself, I’ve asked a friend to help me,” Huang explains in the video embedded below.
He’s called ‘Toy Jensen’ and yes, it’s as bizarre as it sounds.
But it’s also pretty remarkable what Nvidia has managed to achieve.
“Toy Jensen, TJ, is an AI avatar, he’s fully interactive, makes eye contact, has conversations on many topics, speaks and sings with my voice and animates his face and body based on what he’s trying to say,” Huang said.
TJ is built with Nvidia’s AI, Omniverse and RTX technologies and the actual point of all this is to be a tech demo showcasing core technologies that are part of the Nvidia Omniverse Avatar Cloud Engine, or ACE, platform.
Omniverse ACE is a collection of cloud-native AI microservices and workflows for developers to build, customise and deploy avatars that can engage and interact with the viewer.
To make it happen, Nvidia’s team used a recently developed voice conversion model to extract the voice of a professional singer from a sample provided by them and turn it into TJ’s voice – originally developed by training on hours of real-world recordings. They used the musical notes from that sample and applied them to the digital voice of TJ to make the avatar sing the same notes and with the same rhythm as the original singer.
The rendering is gorgeous and the song itself actually doesn’t sound all that bad. Look out, Michael Bublé, my mum would eat this up.
Also, Toy Jensen has legs.