The thing about inspiration is you never know where it might come from, or where it might lead.
Anderson Rohr, a 3D generalist and freelance video editor based in southern Brazil, has for more than a dozen years created content ranging from wedding videos to cinematic animation.
After seeing another creator animate a sci-fi character’s visage and voice using NVIDIA Omniverse and its AI-powered Audio2Face application, Rohr said he couldn’t help but play around with the technology.
The result is a grimly voiced, lip-synced cover of “Bad Moon Rising,” the 1960s anthem from Credence Clearwater Revival, which Rohr created using his own voice.
To make the video, Rohr used an NVIDIA Studio system with a GeForce RTX 3090 GPU.
Rohr’s Artistic Workflow
For this personal project, Rohr first recorded himself singing and opened the file in Audio2Face.
The application, built on NVIDIA AI and Omniverse technology, instantly generates expressive facial animations for digital humans with only a voice-over track or any other audio source.
Rohr then manually animated the eyes, brows and neck of his character and tweaked the lighting for the scene — all of which was rendered in Omniverse via Epic Games Unreal Engine, using an Omniverse Connector and the NVIDIA RTX Global Illumination software development kit.
“NVIDIA Omniverse is helping me achieve more natural results for my digital humans and speeding up my workflow, so that I can spend more time on the creative process,” Rohr said.
Before using Omniverse, some of Rohr’s animations took as long as 300 hours to render. He also faced software incompatibilities, which he said further slowed his work.
Now, with Omniverse and its connectors for various software applications, Rohr’s renderings are achieved in real time.
“Omniverse goes beyond my expectations,” he said. “I see myself using it a lot, and I hope my artwork inspires people to seek real-time results for virtual productions, games, cinematic scenes or any other creative project.”
With Omniverse, NVIDIA Studio creators can supercharge their artistic workflows with optimized RTX-accelerated hardware and software drivers, and state-of-the-art AI and simulation features.
Watch Rohr talk more about his work with NVIDIA Omniverse:
The post Omniverse Creator Uses AI to Make Scenes With Singing Digital Humans appeared first on The Official NVIDIA Blog.