Building state-of-the-art factories requires a state-of-the art planning system.
Mercedes-Benz announced at CES that it is taking the next step in digitizing its production process, using the NVIDIA Omniverse platform to design and plan manufacturing and assembly facilities.
By tapping into NVIDIA AI and metaverse technologies, the automaker can create feedback loops to reduce waste, decrease energy consumption and continuously enhance quality.
Mercedes-Benz has been working with NVIDIA to develop software-defined vehicles. Its upcoming fleets will be built on NVIDIA DRIVE Orin centralized compute, with intelligent driving capabilities tested and validated in the NVIDIA DRIVE Sim platform, built on Omniverse.
The automaker’s latest announcement infuses AI and metaverse technologies even further into the Mercedes-Benz development process with smarter, more efficient manufacturing.
Electric-First Platform Built on Digital-First Process
Vehicle production is a colossal undertaking that requires thousands of parts and workers moving in harmony. Any supply chain or production issues can lead to costly delays.
On top of that, when automakers roll out a new model, they must reconfigure the layout of production plants to account for the new vehicle design. This process can take significant portions of the factory offline, pausing manufacturing for existing vehicles.
Mercedes-Benz plans to start production of its new dedicated platform for electric vehicles at its plant in Rastatt, Germany. The site currently manufactures the automaker’s A- and B-Class as well as the compact SUV GLA and the all-electric Mercedes-Benz EQA.
Mercedes-Benz plant in Rastatt, Germany, where the all-electric EQA is manufactured.
Experts from NVIDIA and Mercedes-Benz operations are setting up a “digital first” – planning process for the plant that won’t disrupt the current production of compact car models at the site. This blueprint will be rolled out to other parts of the global Mercedes-Benz production network for more agile vehicle manufacturing.
Developing With Digital Twins
NVIDIA Omniverse is an open 3D development platform enabling enterprises and institutions across all industries to build and operate digital twins for industrial and scientific use cases. It’s based on Universal Scene Description (USD), allowing enterprises and developers to construct custom 3D pipelines to break down data silos and interact with a single-source-of-truth view of their aggregated simulations.
With Omniverse, Mercedes-Benz planners can access the digital twin of the factory, reviewing and optimizing the plant as needed. Every change can be quickly evaluated and validated in the virtual world, then implemented in the real world to ensure maximum efficiency and ergonomics for factory workers.
Additionally, Mercedes-Benz can synchronize plant locations anywhere in the world by connecting Omniverse with its in-house MO360 Data Platform.
This capability streamlines operations across the global production network and enables over-the-air software updates to manufacturing equipment.
By planning production on NVIDIA Omniverse, Mercedes-Benz is implementing a manufacturing system as intelligent as its vehicles.
Autonomous and electric vehicles are making personal transportation safer and more sustainable — as well as more entertaining.
At CES today, NVIDIA announced that the NVIDIA GeForce NOW cloud gaming service will be coming to cars, with no special equipment needed. Hyundai Motor Group, BYD and Polestar — already members of the NVIDIA DRIVE ecosystem — are the first automakers working with NVIDIA to deliver GeForce NOW in their vehicles.
“We are excited to offer our customers the best in technology, comfort, design from BYD and now the latest in gaming through NVIDIA GeForce NOW high-performance cloud gaming service,” said Stella Li Vice President of BYD and CEO of BYD America. “The experience of driving our cars is becoming increasingly sophisticated, entertaining and sustainable.”
Thanks to cloud technology and the mobile internet, vehicle occupants can keep boredom at bay with the ability to enjoy music, movies and now video games in the car. Drivers and front seat passengers can access games while parked or charging, while those in the backseat of vehicles where rear entertainment is enabled can play anytime on the road.
In March, BYD, the world’s leading manufacturer of new energy vehicles, announced it would build its NEVs on the NVIDIA DRIVE Hyperion platform, starting in the first half of 2023.
Polestar is also using NVIDIA DRIVE for its software-defined architecture, with the upcoming Polestar 3 powered by the NVIDIA DRIVE Orin system-on-a-chip.
Now, these trailblazing automakers will offer vehicles that are as entertaining as they are intelligent with the addition of GeForce NOW.
1,000+ Games, Anywhere, Anytime
The next generation of vehicles are transforming from fixed function to software-defined. At the same time, cars are becoming more connected, opening up new services that weren’t possible before.
GeForce NOW delivers a full PC-gaming experience to nearly any device, including laptops, mobile devices, smart TVs — and now, personal vehicles. The GeForce NOW client for auto can run on Android- or browser-based in-vehicle infotainment systems, providing an app to easily select and launch favorite games.
It uses breakthroughs in low-latency cloud-streaming technology for real-time game play powered by GeForce servers in the cloud. GeForce NOW features more than 1,500 titles — over 1,000 playable with a controller — including top-rated games such as A Plague Tale: Requiem, The Witcher 3: Wild Hunt and Cyberpunk 2077, from leading stores such as Steam, Epic Games Store, the EA app, Ubisoft and GOG.com.
The service also includes many of the most-played free-to-play games like Fortnite, Lost Ark and Destiny 2.
Experiencing the Future
The ability to stream games to the car is a key component of the future of in-vehicle experiences.
As autonomous-driving technology becomes more prevalent, the interior will play a much larger role in differentiating brand identities. Consumers will focus more on features such as digital assistants, beautiful graphic displays and streaming content as they make vehicle buying decisions.
This shift is turning vehicles into living spaces, where passengers can enjoy what’s important to them — whether it be work or play — rather than the stress of driving.
By providing GeForce NOW in their vehicles, Hyundai Motor Group, BYD and Polestar are taking a significant step into this new era of personal transportation.
Fueled by powerful NVIDIA RTX technology and backed by the NVIDIA Studio platform for creators, a creative revolution is underway as a wave of 2D artists moves to 3D, video workflows move to real time and AI tools help artists create content faster.
For them, and the 110 million professional and hobbying PC creators, there’s NVIDIA Studio. It’s a platform for content creators that supercharges 110 creative apps with RTX GPUs, provides lasting stability with NVIDIA Studio Drivers, and includes a powerful suite of Studio apps — Omniverse, Canvas, Broadcast and RTX Remix.
The heartbeat of the Studio platform lies in NVIDIA Omniverse, a platform where creators can connect their 3D apps and collaborate in real time. Omniverse is expanding with Blender enhancements, a new suite of experimental generative AI tools for 3D artists and thousands of new, free Universal Scene Description (USD) assets available in the content browser in Omniverse Create.
Built on Omniverse, NVIDIA RTX Remix is a free modding platform to quickly create mods for classic games with full ray tracing, enhanced materials, DLSS 3 and NVIDIA Reflex. We’ve enabled two amazing community modders to bring RTX to Portal’s most famous mod, Portal Prelude, preserving the timeless gameplay, while relighting it with full ray tracing.
The NVIDIA Broadcast and Canvas apps, also in the exclusive Studio software suite, have upcoming upgrades. Broadcast is adding a new Eye Contact feature, and Canvas will introduce 360-degree images that can be used as environment maps in 3D apps.
Coming to GeForce RTX 40 and 30 Series GPUs next month, the RTX Video Super Resolution feature uses AI to improve the quality of any video watched in a browser by removing blocky compression artifacts and upscaling video resolution. This improves video sharpness and clarity, and lets people watch online content in its native resolution on high-resolution displays. RTX Video Super Resolution will be available in February for Chrome and Edge browsers.
And to celebrate CES, influential Blender artist CG Geek will take on a three-day design challenge, powered by NVIDIA Studio technologies. Plus, explore how seven freelance artists from across the world created an experimental short film together, in real time, using Studio hardware, Omniverse and the Nucleus Cloud — this week In the NVIDIA Studio.
From Concept to Completion, Faster With New NVIDIA Studio Laptops
GeForce RTX GPUs provide massive speedups in 3D, video and broadcast workflows, as well as a myriad of AI tools, thanks to dedicated ray tracing, AI and video-encoding hardware. New Studio laptops with GeForce RTX 40 Series Laptop GPUs use the NVIDIA Ada Lovelace architecture — with support for DLSS 3, AV1 and more — and fifth-generation Max-Q technologies for maximum performance and efficiency.
The new laptops provide a quantum leap in performance over the last generation, and a massive boost in efficiency that allows OEMs to include more powerful GPUs in thinner, more portable systems. NVIDIA Studio laptops are purpose-built for creators, with the right combination of hardware for creative workflows, color-accurate displays, and preinstalled Studio Drivers and Studio software.
The Razer Blade 16 is available with up to a GeForce RTX 4090 Laptop GPU.
The instantly recognizable Razer Blade laptop gets an RTX 40 Series upgrade with up to a GeForce RTX 4090 Laptop GPU. The 16-inch, HDR-capable, dual-mode, mini-LED display boasts 1,000 nits peak brightness, which pairs nicely with a Creator mode that enables sharp, ultra-high-definition+ native resolution at 120Hz.
The MSI Stealth 17 Studio A13V is available with up to a GeForce RTX 4090 Laptop GPU.
MSI’s Stealth series will launch with a wide range of Studio options. With 14- to 17-inch models and up to GeForce RTX 4090 Laptop GPUs, creators can pick the laptop that best suits their needs.
Powered by a GeForce RTX 4070 Laptop GPU, the ASUS Zenbook Pro 14 showcases a new level of performance for a 14-inch laptop.
The increased efficiency of RTX 40 Series Laptop GPUs and fifth-generation Max-Q Technologies deliver high performance to laptops as slim as 14 inches, with up to GeForce RTX 4070 Laptop GPUs, a new milestone. The ASUS Zenbook Pro 14 OLED is one example, with a GeForce RTX 4070 Laptop GPU in a super-thin 14-inch design, and accompanied by an all-new, cutting-edge 2.8K 120Hz OLED NanoEdge Dolby Vision display.
GeForce RTX 4070 Ti: 40 Series Benefits and Faster Than a 3090 Ti
For artists who prefer to create from their desktops, the new GeForce RTX 4070 Ti graphics cards, available starting Thursday, Jan. 5, offers incredible performance that’s up to 15% faster than a GeForce RTX 3090 Ti in 3D rendering. Like the RTX 4090 and 4080, the newest RTX 40 Series GPU features dual AV1 video encoders. When enabled in top video-editing and livestreaming apps — such as Adobe Premiere Pro (via the Voukoder plug-in), DaVinci Resolve, OBS and Jianying — export times are cut in half with improved video quality.
GeForce RTX 4070 Ti graphics cards.
3D creators see up to a 70% increase in performance compared to the GeForce RTX 3070 Ti in popular apps like Autodesk Maya, Blender, Chaos V-Ray, Epic Games Unreal Engine and Unity. Users can also unlock creative freedom by unifying 3D assets, libraries and tools in Omniverse.
And all creators can benefit from the new fourth-generation Tensor Cores for AI tools, providing up to 2x increased performance compared to the previous generation.
The January Studio Driver, supporting the GeForce RTX 4070 Ti launch, will be available for download later this week.
Create Easily With Studio Software
NVIDIA RTX GPUs unlock exclusive software for creators: Omniverse, RTX Remix, Canvas and Broadcast. They’re all getting upgrades early this year.
Based on the USD framework, Omniverse enables artists to connect their favorite tools from Adobe, Autodesk, Epic Games, SideFX and more. Creators can see their scenes instantly come together without lengthy import or export cycles. Changes happen in real time across the connected apps — whether a single artist is working in multiple apps simultaneously or collaborating with another artist from across the globe.
The Blender alpha branch release, part of a series of updates to Omniverse, is now available in the Omniverse Launcher. It enables users to repair geometry, generate automatic UVs and decimate high-resolution CAD data to more usable polycounts.
Performance updates to Audio2Face, Audio2Gesture and Audio2Emotion — powerful AI tools within Omniverse — better enable instant, realistic animation of characters. The AI ToyBox of experimental tools is now available in the Omniverse Extension Manager. Thousands of new, free 3D assets have also been made available worldwide for users to build and create within Omniverse.
‘Portal: Prelude RTX’ is coming soon to mod sites ModDB and Nexus Mods.
RTX Remix, which is built on Omniverse and was used to create the jaw-dropping Portal with RTX, is approaching its early-access release. Nicolas “NykO18” Grevet — the original creator of Portal: Prelude — is using RTX Remix to remaster his unofficial Portal prequel from 2008. Grevet is working with modder David “Kralich” Driver-Gomm to modernize the assets and relight the game with stunning, full ray tracing. Portal: Prelude RTX will soon become free to download from mod sites like ModDB and Nexus Mods.
Canvas allows creators to paint by material, rather than color, using simple brushstrokes and AI to quickly conceptualize a beautiful image. Canvas is getting a new 360 image feature, which will become available in an upcoming free update for RTX users, helping artists create panoramic scenes and export them into any 3D app to use as an environment map. Such maps are used to change the ambient lighting of a 3D scene and appear in reflections for added realism.
Later this month, Broadcast will add a new Eye Contact feature that can change the position of the subject’s eyes to appear focused on the camera, as well as a new vignette effect. The update also improves the popular virtual background feature, adding temporal information for increased stability. The virtual background improvements can also be found in the recent OBS Studio 29.0 software release, and will be updated SDKs for developers later this week.
3D Need for Speed
To showcase the speed at which Studio creators flow, talented 3D artist CG Geek has been challenged to create an animated Blender scene during CES. He’ll have just three days, starting Thursday, Jan. 5, to blitz through tough, typically time-consuming tasks.
He’s equipped with a GeForce RTX 4090 GPU, RTX acceleration and AI features throughout his entire workflow.
The group started by pulling in a pair of Nizam’s characters: “3D” built in ShapesXR and “Figgy” in Maya. Both were dropped into Omniverse seamlessly as USD files. Prompted by Lightcap, Nizam added knitted textures to “3D” using Adobe Substance 3D Painter, and the group immediately saw this through the cloud.
Solina then offered to help with motion capture using Xsens. As he added the rigging, Lightcap brought in a background created in Blender. Lightcap noted that “RTX is insane,” as his animated western landscape rendered nearly instantly.
Anticipation for precipitation, McEvenue’s Houdini-based rainy day simulation.
With the scene coming together, Varis changed lighting elements, looking to make things “more epic,” he said. To incorporate a storm into the scene, McEvenue added a rain simulation created in Houdini. The film’s characters make a quick escape, hijacking the hot air balloon of Toy Jensen, the AI avatar of NVIDIA founder and CEO Jensen Huang.
For the ascent, Goldstein used her GeForce RTX 4090 GPU to add steam thrusters using OmniGraph.
With the characters now in outer space, Wang imported a satellite created in Autodesk Maya. The scene concludes as the space adventurers take off for Jupiter.
While showcasing their talents, these 3D artists displayed the capabilities of the NVIDIA Studio platform. Powerful RTX GPUs, acceleration in popular 3D apps and exclusive Omniverse software that enabled collaboration from all over the world in real time.
The future of creativity is shining bright. And the NVIDIA Studio ecosystem is lighting the scene.
Demand for intelligent robots is growing as more industries embrace automation to address supply chain challenges and labor force shortages.
The installed base of industrial and commercial robots will grow more than 6.4x — from 3.1 million in 2020 to 20 million in 2030, according to ABI Research. Developing, validating and deploying these new AI-based robots requires simulation technology that places them in realistic scenarios.
At CES, NVIDIA announced major updates to Isaac Sim, its robotics simulation tool to build and test virtual robots in realistic environments across varied operating conditions. Now accessible from the cloud, Isaac Sim is built on NVIDIA Omniverse, a platform for creating and operating metaverse applications.
Powerful AI-Driven Capabilities for Roboticists
With humans increasingly working side by side with collaborative robots (cobots) or autonomous mobile robots (AMRs), it’s critical that people and their common behaviors are added to simulations.
Isaac Sim’s new people simulation capability allows human characters to be added to a warehouse or manufacturing facility and tasked with executing familiar behaviors— like stacking packages or pushing carts. Many of the most common behaviors are already supported, so simulating them is as simple as issuing a command.
To minimize the difference between results observed in a simulated world versus those seen in the real world, it’s imperative to have physically accurate sensor models.
Using NVIDIA RTX technology, Isaac Sim can now render physically accurate data from sensors in real time. In the case of an RTX-simulated lidar, ray tracing provides more accurate sensor data under various lighting conditions or in response to reflective materials.
Isaac Sim also provides numerous new simulation-ready 3D assets, which are critical to building physically accurate simulated environments. Everything from warehouse parts to popular robots come ready to go, so developers and users can quickly start building.
Significant new capabilities for robotics researchers include advances in Isaac Gym for reinforcement learning and Isaac Cortex for collaborative robot programming. Additionally, a new tool, Isaac ORBIT, provides simulation operating environments and benchmarks for robot learning and motion planning.
For the large community of Robot Operating System (ROS) developers, Isaac Sim upgrades support for ROS 2 Humble and Windows. All of the Isaac ROS software can now be used in simulation.
Expanding Isaac Platform Capabilities and Ecosystem Drives Adoption
The large and complex robotics ecosystem spans multiple industries, from logistics and manufacturing to retail, energy, sustainable farming and more.
The end-to-end Isaac robotics platform provides advanced AI and simulation software as well as accelerated compute capabilities to the robotics ecosystem. Over a million developers and more than a thousand companies rely on one or many parts of it. This includes many companies that have deployed physical robots developed and tested in the virtual world using Isaac Sim.
Telexistence has deployed beverage restocking robots across 300 convenience stores in Japan. To improve safety, Deutsche Bahn is training AI models to handle very important but unexpected corner cases that happen rarely in the real world — like luggage falling on a train track. Sarcos Robotics is developing robots to pick and place solar panels in renewable energy installations.
Festo uses Isaac Cortex to simplify programming for cobots and transfer simulated skills to the physical robots. Fraunhofer is developing advanced AMRs using the physically accurate and full-fidelity visualization features of Isaac Sim. Flexiv is using Isaac Replicator for synthetic data generation to train AI models.
While training robots is important, simulation is playing a critical role in training the human operators to work with and program robots. Ready Robotics is teaching programming of industrial robots with Isaac Sim. Universal Robotics is using Isaac Sim for workforce development to train end operators from the cloud.
Cloud Access Puts Isaac Platform Within Reach Everywhere
With Isaac Sim available in the cloud, global, multidisciplinary teams working on robotics projects can collaborate with increased accessibility, agility and scalability for testing and training virtual robots.
A lack of adequate training data often hinders deployment when building new facilities with robotics systems or scaling existing autonomous systems. Isaac Sim taps into Isaac Replicator to enable developers to create massive ground-truth datasets that mimic the physics of real-world environments.
Once deployed, dynamic route planning is required to operate an efficient fleet of hundreds of robots as automation requirements scale. NVIDIA cuOpt, a real-time fleet task-assignment and route-planning engine improves operational efficiencies with automation.
Watch NVIDIA’s special address at CES, where its executives unveiled products, partnerships and offerings in autonomous machines, robotics, design, simulation and more.
Whether creating realistic digital humans that can express raw emotion or building immersive virtual worlds, those in the design, engineering, creative and other industries across the globe are reaching new heights through 3D workflows.
Animators, creators and developers can use new AI-powered tools to reimagine 3D environments, simulations and the metaverse — the 3D evolution of the internet.
In a special address at CES, NVIDIA announced these features, as well as Omniverse preinstallation on NVIDIA Studio laptops and thousands of new, free USD assets to help accelerate adoption of 3D workflows.
Plus, a new release for Blender, now available in the Omniverse Launcher, is bringing 3D generative AI capabilities to Blender users everywhere. A new panel lets Blender users easily transfer shape keys and rigged characters. The challenge of reattaching a rigged character’s head can now be solved with a one-button operation from Omniverse Audio2Face — an AI-enabled tool that automatically generates realistic facial expressions from an audio file.
Another new panel for scene optimization lets users create USD scenes within their multi-app 3D workflows more easily and in real time.
In addition, Audio2Face, Audio2Gesture and Audio2Emotion — generative AI tools that enable instant 3D character animation — are getting performance updates that make it easier for developers and creators to integrate into their current 3D pipelines.
Creators can generate facial expressions from an audio file using Audio2Face; realistic emotions ranging from happy and excited to sad and regretful with Audio2Emotion; and realistic upper-body movement using Audio2Gesture. These audio-to-animation tools are game-changers for 3D artists, eliminating the need to perform tedious, manual tasks.
AI-assisted creator tools are expanding to even more communities of creative and technical professionals. When NVIDIA Canvas was introduced, it empowered artists to seamlessly generate landscapes and iterate on them with simple brushstrokes and AI. Coming soon, all RTX users will be able to download an update to Canvas that introduces 360 surround images to create and conceptualize panoramic environments and beautiful images. The AI ToyBox, which features extensions derived from NVIDIA Research, enables creators to generate 3D meshes from 2D inputs.
Omniverse’s powerful AI tools simplify complex tasks. Creators of all levels can tap into these resources to produce high-quality outputs that meet the growing demands for content and virtual worlds in the metaverse.
“The demand for 3D skills is skyrocketing, but learning 3D can be pretty scary to some, and definitely time consuming,” said Jae Solina, aka JSFilmz. “But these new platform developments not only let creatives and technical professionals continue to work in their favorite 3D tools, but also supercharge their craft and even use AI to assist them in their workflows.”
Omniverse Launcher, the portal to download Omniverse content and reference applications, has also been made available to system builders so they can preinstall it, enabling optimized, out-of-the-box experiences for 3D creators on NVIDIA Studio-validated laptops. GIGABYTE and AORUS will be the first laptops launching in 2023 with Omniverse Launcher preinstalled, expanding platform access to a growing number of 3D content creators.
NVIDIA RTX Remix is a free modding platform, built on Omniverse, that enables modders to
quickly create and share #RTXON mods for classic games, each with full ray tracing, enhanced materials, NVIDIA DLSS 3 and NVIDIA Reflex. Its release in early access is coming soon. The jaw-dropping Portal with RTXwas built with RTX Remix, and to demonstrate how easy it is for modders to turn RTX ON in their mods, we shared RTX Remix with the original creator of Portal: Prelude, an unofficial Portal prequel released in 2008.
Omniverse users can also access thousands of new, free USD assets, including a USD-based NVIDIA RTX Winter WorldMinecraft experience, and learn to create their own NVIDIA SimReady assets for complex simulation building. Using Omniverse, creators can supercharge their existing workflows using familiar tools such as Autodesk Maya, Autodesk 3ds Max, Blender, Adobe Substance 3D Painter, and more with AI, simulation tools and real-time RTX-accelerated rendering.
All types of 3D creators can take advantage of these new tools to push the boundaries of 3D simulation and virtual world-building. Users can reimagine digital worlds and animate lifelike characters with new depths of creativity through the bridging of audio-to-animation tools, generative AI and the metaverse.
Latest Omniverse Platform Updates
The latest updates within Omniverse include:
Early access for the Unity Omniverse Connector is now available.
Blender alpha release, now available in the Omniverse Launcher, enables users to repair geometry, generate automatic UVs and decimate high-resolution CAD data to more usable polycounts.
Audio2Face, Audio2Emotion and Audio2Gesture updates better enable instant, realistic animation of characters, now available in Omniverse Audio2Face and Omniverse Machinima.
NVIDIA Canvas is coming soon to the Omniverse Launcher with new capabilities that enable the creation of 360-degree landscapes with simple brushstrokes. Users can import the environments into 3D apps to test different settings and lighting.
AI ToyBox of experimental tools, built by NVIDIA Research, that include GET3D, an Omniverse extension that generates trainable 3D models from 2D images, letting developers use their own datasets to rapidly create models for 3D virtual worlds, is now available in the Omniverse Extension Manager.
Thousands of new, free 3D assets are now available worldwide for users to build and create within Omniverse.
Developers and teams building avatars and virtual assistants can now register to join the early-access program for NVIDIA Omniverse Avatar Cloud Engine (ACE), a suite of cloud-native AI microservices that make it easier to build and deploy intelligent virtual assistants and digital humans at scale.
Omniverse ACE eases avatar development, delivering the AI building blocks necessary to add intelligence and animation to any avatar, built on virtually any engine and deployed on any cloud. These AI assistants can be designed for organizations across industries, enabling organizations to enhance existing workflows and unlock new business opportunities.
ACE is one of several generative AI applications that will help creators accelerate the development of 3D worlds and the metaverse. Members who join the program will receive access to the prerelease versions of NVIDIA’s AI microservices, as well as the tooling and documentation needed to develop cloud-native AI workflows for interactive avatar applications.
Bring Interactive AI Avatars to Life With Omniverse ACE
Methods for developing avatars often require expertise, specialized equipment and manually intensive workflows. To ease avatar creation, Omniverse ACE enables seamless integration of NVIDIA’s AI technologies — including pre-built models, toolsets and domain-specific reference applications — into avatar applications built on most engines and deployed on public or private clouds.
Since it was unveiled in September, Omniverse ACE has been shared with select partners to capture early feedback. Now, NVIDIA is looking for partners who will provide feedback on the microservices, collaborate to improve the product, and push the limits of what’s possible with lifelike, interactive digital humans.
The early-access program includes access to the prerelease versions of ACE animation AI and conversational AI microservices, including:
3D animation AI microservice for third-party avatars, which uses Omniverse Audio2Face generative AI to bring to life characters in Unreal Engine and other rendering tools by creating realistic facial animation from just an audio file.
2D animation AI microservice, called Live Portrait, enables easy animation of 2D portraits or stylized human faces using live video feeds.
Text-to-speech microservice uses NVIDIA Riva TTS to synthesize natural-sounding speech from raw transcripts without any additional information, such as patterns or rhythms of speech.
Program members will also get access to tooling, sample reference applications and supporting resources to help get started.
Avatars Make Their Mark Across Industries
Omniverse ACE can help teams build interactive, digital humans that elevate experiences across industries, providing:
Easy animation of characters, so users can bring them to life with minimal expertise.
The ability to deploy on cloud, which means avatars will be usable virtually anywhere, such as a quick-service restaurant kiosk, a tablet or a virtual-reality headset.
A plug-and-play suite, built on NVIDIA Unified Compute Framework (UCF), which enables interoperability between NVIDIA AI and other solutions, ensuring state-of-the-art AI that fits each use case.
Partners such as Ready Player Me and Epic Games have experienced how Omniverse ACE can enhance workflows for AI avatars.
The Omniverse ACE animation AI microservice supports 3D characters from Ready Player Me, a platform for building cross-game avatars.
“Digital avatars are becoming a significant part of our daily lives. People are using avatars in games, virtual events and social apps, and even as a way to enter the metaverse,” said Timmu Tõke, CEO and co-founder of Ready Player Me. “We spent seven years building the perfect avatar system, making it easy for developers to integrate in their apps and games and for users to create one avatar to explore various worlds — with NVIDIA Omniverse ACE, teams can now more easily bring these characters to life.”
Epic Games’ advanced MetaHuman technology transformed the creation of realistic, high-fidelity digital humans. Omniverse ACE, combined with the MetaHuman framework, will make it even easier for users to design and deploy engaging 3D avatars.
Digital humans don’t just have to be conversational. They can be singers, as well — just like the AI avatar Toy Jensen. NVIDIA’s creative team quickly created a holiday performance by TJ, using Omniverse ACE to extract the voice of a singer and turn it into TJ’s voice. This enabled the avatar to sing at the same pitch and with the same rhythm as the original artist.
Many creators are venturing into VTubing, a new way of livestreaming. Users embody a 2D avatar and interact with viewers. With Omniverse ACE, creators can move their avatars into 3D from 2D animation, including photos and stylistic faces. Users can render the avatars from the cloud and animate the characters from anywhere.
Additionally, the NVIDIA Tokkio reference application is expanding, with early partners building cloud-native customer service avatars for industries such as telco, banking and more.
Join the Early-Access Program
Early access to Omniverse ACE is available to developers and teams building avatars and virtual assistants.
Those looking to join the ranks of AI trailblazers or chart a new course in their careers need look no further.
At NVIDIA’s latest GTC conference, industry leaders in a panel called “5 Paths to a Career in AI” shared tips and insights on how to make a mark in this rapidly evolving field.
Representing diverse sectors such as healthcare, automotive, augmented and virtual reality, climate and energy, and manufacturing, these experts offered valuable advice for all seeking to build a career in AI.
Here are five key takeaways from the discussion:
Be curious and constantly learn: “I think in order to break into this field, you’ve got to be curious. It’s so important to always be learning [and] always be asking questions,” emphasized Chelsea Sumner, healthcare AI startups lead for North and Latin America at NVIDIA. “If we’re not asking questions, and we’re not learning, we’re not growing.”
Tell your story effectively to different audiences: “Your ability to tell your story to a variety of different audiences is essential,” noted Justin Taylor, vice president of AI at Lockheed Martin. “So for them to understand what you’re doing [with AI], how you’re doing it, why you’re doing it is essential.”
Embrace challenges and be resilient: “When you have all of these different experiences, you understand that it’s not always going to be perfect,” advised Laura Leal-Taixé, professor at the Technical University of Munich and principal scientist at Argo AI. “And when things aren’t always perfect, you’re able to have competence because [you know that you] did that really hard thing and was able to get through it.”
Understand the purpose behind your work: “Understand the baseline, how do you collect the data baseline — understand the physical, the bottom line. What’s the purpose, what do you want to do?” advised Jay Lee, Ohio eminent scholar of the University of Cincinnati and board member of Foxconn.
Collaborate and seek support from others: “It’s so important for resiliency to find people across different domains and really tap into that,” said Carrie Gotch, creator and content strategy for 3D/AR at Adobe. “No one does it alone, right? You’re always part of a system, part of a team of people.”
The panelists stressed the importance of staying up to date and curious, gaining practical experience, collaborating with others and taking risks when building a career in AI.
Start your journey to an AI career by signing up for NVIDIA GTC, running in March, where you can network, get trained on the latest tools and hear from thought leaders about the impact of AI in various industries.
It could be the first step toward a rewarding AI career that takes you into 2023 and beyond.
Editor’s note: This post is a part of our Meet the Omnivore series, which features individual creators and developers who use NVIDIA Omniverse to accelerate their 3D workflows and create virtual worlds.
Stephen Tong
Stephen Tong, aka Funky Boy, has always loved music and photography. He’s now transferring the skills developed over the years as a music producer — shooting time lapses, creating audio tracks and more — to a new passion of his: 3D content creation.
Tong began creating 3D renders and animations earlier this year, using the NVIDIA Omniverse platform for building and connecting custom 3D pipelines.
In September, Tong participated in the first-ever Omniverse developer contest, which he considered the perfect way to learn about extending the platform and coding with the popular Python programming language. He submitted three Omniverse extensions — core building blocks that let anyone create and extend functions of Omniverse apps — aimed at easing creative workflows like his own.
Ringing in the Season the Omniverse Way
The artist also took part in the #WinterArtChallenge this month from NVIDIA Studio, a creative community and platform of NVIDIA RTX and AI-accelerated creator apps. Creatives from around the world shared winter-themed art on social media using the hashtag.
Tong said his scene was inspired by cozy settings he often associates with the holidays.
First, the artist used AI to generate a mood board. Once satisfied with the warm, cozy mood, he modeled a winter chalet — complete with a snowman, wreaths and sleigh — using the Marbles RTX assets, free to use in the Omniverse Launcher, as well as some models from Sketchfab.
Tong collected the assets in Unreal Engine before rendering the 3D scene using the Omniverse Create and Blender apps. The Universal Scene Description (USD) framework allowed him to bring the work from these various applications together.
“USD enables large scenes to be loaded fast and with ease,” he said. “The system of layers makes Omniverse a powerful tool for collaboration and iterations.”
With his festive creativity on a roll, Tong also orchestrated an animated quartet lip-syncing to “Carol of the Bells” using Omniverse Audio2Face, an AI app that quickly and easily generates expressive facial animations from just an audio source, as well as the DaVinci Resolve application for video editing.
To hear the latest made possible by accelerated computing, AI and Omniverse, watch NVIDIA’s special address at CES on Tuesday, Jan. 3, at 8 a.m. PT.
Check out more artwork from Tong and other “Omnivores” in the gallery. Connect your workflows to Omniverse with software from Adobe, Autodesk, Epic Games, Maxon, Reallusion and more.
NVIDIA executives will share some of the company’s latest innovations Tuesday, Jan. 3, at 8 a.m. Pacific time ahead of this year’s CES trade show in Las Vegas.
Jeff Fisher, senior vice president for gaming products, will be joined by Deepu Talla, vice president of embedded and edge computing, Stephanie Johnson, vice president of consumer marketing, and Ali Kani, vice president of automotive, for a special address that you won’t want to miss.
During the event, which will be streamed on nvidia.com, the NVIDIA YouTube and Twitch channels, as well as on the GeForce YouTube channel, the executives will reveal exciting gaming, creative, automotive and robotics announcements.
The broadcast is a unique opportunity to get a sneak peek at the future of technology and see what NVIDIA has in store for the coming year.
Don’t miss out on this special address from some of the top executives in the industry.
Tune in on Jan. 3 to get a first look at what’s in store for the future of technology.
One of tech’s top talk shows, the NVIDIA AI Podcast has attracted more than 3.6 million listens to date from folks who want to hear the latest in machine learning.
Its 180+ installments so far have included interviews with luminaries like Kai-Fu Lee and explored how AI is advancing everything from monitoring endangered rhinos to analyzing images from the James Webb Space Telescope.
Here’s a sampler of the most-played episodes in 2022:
A renowned expert in machine learning, Urtasun discusses her current work at Waabi using simulation technology to teach trucks how to drive. Urtasun is a professor of computer science at the University of Toronto and the former chief scientist and head of R&D for Uber’s advanced technology group.
Automated chatbots ain’t what they used to be — they’re getting a whole lot better, thanks to advances in conversational AI. Entrepreneur, educator and author Jason Mars breaks down the latest techniques giving AI a voice.
Dustin Wright, a researcher at the University of Copenhagen, used NVIDIA GPUs to create an “exaggeration detection system.” He pointed it at hyperbole in health science news and explained to the AI Podcast how it works.
Silicon Valley startup MORF Gallery showcases artists who create with AI, robots and visual effects. Its CEO provides a virtual tour of what’s happening in digital art — including a plug-in device that can turn any TV into an art gallery.
What started as Nick Walton’s college hackathon project grew into “AI Dungeon,” a game with more than 1.5 million users. Now he’s co-founder and CEO of Latitude, a startup using AI to spawn storylines for games.
