Posts in category: nVidia AI

Editor’s note: This post is part of our weekly In the NVIDIA Studio series, which celebrates featured artists, offers creative tips and tricks, and demonstrates how NVIDIA Studio technology improves creative workflows. We’re also deep diving on new GeForce RTX 40 Series GPU features, technologies and resources, and how they dramatically accelerate content creation.

An earlier version of the blog incorrectly noted that the December Studio Driver was available today. Stay tuned for updates on this month’s driver release.

Time to tackle one of the most challenging tasks for aspiring movie makers — creating aesthetically pleasing visual effects — courtesy of visual effects artist and filmmaker Jay Lippman this week In the NVIDIA Studio.

In addition, the new NVIDIA Omniverse Unreal Engine Connector 200.2 allows Unreal users to send and live-sync assets in the Omniverse Nucleus server — unlocking the ability to open, edit and sync Unreal with other creative apps — to build more expansive virtual worlds in complete ray-traced fidelity.

Plus, the NVIDIA Studio #WinterArtChallenge is delivering un-brr-lievable entries. Check out some of the featured artwork and artists at the end of this post.

(Inter)Stellar Video-Editing Tips and Tricks

A self-taught filmmaker, Lippman grew up a big fan of science fiction and horror. Most of the short sketches on his YouTube channel are inspired by movies and shows in that genre, he said, but his main inspiration for content derives from his favorite punk bands.

“I always admired that they didn’t rely on big record deals to get their music out there,” he said. “That whole scene was centered around this culture of DIY, and I’ve tried to bring that same mentality into the world of filmmaking, figuring out how to create what you want with the tools that you have.”

That independent spirit drives Make Your VFX Shots Look REAL — a sci-fi cinematic and tutorial that displays the wonder behind top-notch graphics and the know-how for making them your own.

Lippman uses Blackmagic Design’s DaVinci Resolve software for video editing, color correction, visual effects, motion graphics and audio post-production. His new GeForce RTX 4080 GPU enables him to edit footage while applying effects freely and easily.

Lippman took advantage of the new AV1 encoder found in DaVinci Resolve, OBS Studio and Adobe Premiere Pro via the Voukoder plug-in, encoding 40% faster and unlocking higher resolutions and crisper image quality.

“The GeForce RTX 4080 GPU is a no-brainer for anyone who does graphics-intensive work, video production or high-end streaming,” Lippman said.

The majority of Make Your VFX Shots Look REAL was built in DaVinci Resolve’s Fusion page, featuring a node-based workflow with hundreds of 2D and 3D tools. He uploaded footage from his Blackmagic Pocket Cinema Camera in 6K resolution then proceeded to composite VFX.

The artist started by refining motion blur, a key element of any movement in camera footage shot in 24 frames per second or higher. Animated elements like the blue fireball must include motion blur, or they’ll look out of place. Applying a transform node with motion blur, done faster with a GeForce RTX GPU, created the necessary realism, Lippman said.

Lippman then lit the scene and enhanced elements in the composition by emitting absent light in the original footage. He creates lighting and adds hues by using a probe modifier on the popular DaVinci Resolve color corrector, a GPU-accelerated task.

The artist then matches movement, critical for adding 2D or 3D effects to footage. In this case, Lippman replaced the straightforward blue sky with a haunting, cloudy, gloomy gray. Within Fusion, Lippman selected the merge mode, connecting the sky with the composition. He then right clicked the center of the video and used the Merge:1 Center, Modify With and Tracker position features with minor adjustments to complete tracking movement.

Lippman rounds out his creative workflow with color matching. He said it’s critical to have the proper mental approach alongside realistic expectations while applying VFX composition.

“Our goal is not to make our VFX shots look real, it’s to make them look like they were shot on the same camera, on the same lens, at the same time of the original footage,” said Lippman. “A big part of it is matching colors, contrast and overall brightness with all of the scene elements.”

Lippman color matched the sky, clouds and UFO by adding a color-corrector node to a single cloud node, tweaking the hue and likeness to match the rest of the sky. Edits were then applied to the remaining clouds. Lippman also applied a color-correction node to the UFO, tying up the scene with matching colors.

When it came time for final exports, the exclusive NVIDIA dual encoders found in GeForce RTX 40 Series GPUs slashed Lippman’s export time by half. This can help freelancers like him meet sharp deadlines. The dual encoders can be found in Adobe Premiere Pro (via the popular Voukoder plug-in), Jianying Pro (China’s top video-editing app) and DaVinci Resolve.

“The GeForce RTX 4080 is a powerhouse and definitely gives you the ability to do more with less,” he said. “It’s definitely faster than the dual RTX 2080 GPU setup I’d been using and twice as fast as the RTX 3080 Ti, while using less power and costing around the same. Plus, it unlocks the AV1 Codec in DaVinci Resolve and streaming in AV1.”

Check out his review.

As AI plays a greater role in creative workflows, video editors can explore DaVinci Resolve’s vast suite of RTX-accelerated, AI-powered features that are an incredible boon for efficiency.

These include Face Refinement, which detects facial features for fast touch-ups such as sharpening eyes and subtle relighting; Speed Warp, which quickly creates super-slow-motion videos; and Detect Scene Cuts, which uses DaVinci Resolve’s neural engine to predict video cuts without manual edits.

The Unreal Engine Connector Arrives

NVIDIA Omniverse Unreal Engine Connector 200.2 supports enhancements to non-destructive live sessions with Omni Live 2.0, allowing for more robust real-time collaboration of Universal Scene Description (USD) assets within Unreal Engine. Thumbnails are now supported in its content browser from Nucleus for Omniverse USD and open-source material definition language (MDL), which creates a more intuitive user experience.

The Omniverse Unreal Engine Connector also supports updates to Unreal Engine 5.1, including:

• Lumen — Unreal Engine 5’s fully dynamic global illumination and reflections system that renders diffuse interreflection with infinite bounces and indirect specular reflections in large, detailed environments at massive scale.
• Nanite — the virtualized geometry system using internal mesh formats and rendering technology to render pixel-scale detail at high object counts.
• Virtual Shadow Maps — to deliver consistent, high-resolution shadowing that works with film-quality assets and large, dynamically lit open worlds.

Omniverse Unreal Engine Connector supports versions 4.27, 5.0 and 5.1 of Unreal Editor. View the complete release notes.

Weather It’s Cold or Not, the #WinterArtChallenge Carries On

Enter NVIDIA Studio’s #WinterArtChallenge, running through the end of the year, by sharing winter-themed art on Instagram, Twitter or Facebook for a chance to be featured on our social media channels.

@lowpolycurls’ 3D winter scene — with its unique, 2D painted-on style textures — gives us all the warm feels during a cold winter night.

Snow much fun!

𝑊𝑒𝑎𝑡ℎ𝑒𝑟 it’s cold or not, we’re enjoying this beautiful #WinterArtChallenge share from @lowpolycurls.

Share your winter-themed art with us for a chance to be featured on our social channels and in an upcoming Studio Standouts YouTube video! pic.twitter.com/aWAPoczok8

— NVIDIA Studio (@NVIDIAStudio) November 28, 2022

Be sure to tag #WinterArtChallenge to join.

Get creativity-inspiring updates directly to your inbox by subscribing to the NVIDIA Studio newsletter.

From rapidly fluctuating demand to staffing shortages and supply chain complexity, enterprises have navigated numerous challenges the past few years. Many companies seeking strong starts to 2023 are planning to use AI and accelerated computing to drive growth while saving costs.

To support these early adopters — as well as those just beginning their AI journey — NVIDIA has announced a new version of its NVIDIA AI Enterprise software suite to support businesses worldwide across a wide range of domain and industry-specific workloads.

NVIDIA AI Enterprise 3.0 will introduce workflows for contact center intelligent virtual assistants, audio transcription and digital fingerprinting for cybersecurity — some of the most common applications for enterprises adopting AI to better serve customers.

Expected to be available later this month, NVIDIA AI Enterprise 3.0 also expands support for more than 50 NVIDIA AI software frameworks and pretrained models available on the NVIDIA NGC software catalog, supercharging and simplifying AI deployments for organizations globally.

Deutsche Bank Announces Innovation Partnership With NVIDIA 

The new software release arrives as Deutsche Bank today announced its plans to partner with NVIDIA to accelerate the use of AI in financial services as part of its strategy for developing AI-powered speech, vision and fraud detection applications within the industry.

“AI and machine learning will redefine banking, and we’re already working closely with NVIDIA to lead the industry in leveraging these technologies to improve customer service and mitigate risk,” said Gil Perez, chief innovation officer and head of Cloud & Innovation Network at Deutsche Bank. “Accelerated computing enables traders to manage risk and run more scenarios faster while also improving energy efficiency, and NVIDIA AI Enterprise provides the flexibility to support AI development across our hybrid infrastructure.”

New NVIDIA AI Enterprise Workflows Speed Success for Businesses

This latest version of our secure, cloud-native suite of AI software enables organizations to solve business challenges while increasing operational efficiency. It accelerates the data science pipeline and streamlines the development and deployment of AI models to automate essential processes and gain rapid insights from data.

The new AI workflows for contact center intelligent virtual assistants, audio transcription and cybersecurity digital fingerprinting in NVIDIA AI Enterprise 3.0 leverage NVIDIA expertise to reduce development time and costs to speed time to deployment.

The workflows run as cloud-native microservices using NVIDIA AI frameworks and pretrained models, as well as Helm charts, Jupyter notebooks and more. Enterprises can deploy the microservices as standalone Kubernetes containers or combine them with other services to create production-ready applications with greater accuracy and performance.

The contact center intelligent virtual assistant AI solution workflow enables enterprises to respond to customers around the clock to reduce wait times and free up time for human contact center agents to support more complex inquiries — all while reducing costs. Using the workflow, enterprises can develop agents that deliver personalized and precise responses in natural-sounding voices. By leveraging AI, the agents can better understand customers even on calls with poor audio quality.

With the audio transcription AI solution workflow, enterprises can rapidly create accurate transcripts in English, Spanish, Mandarin, Hindi, Russian, Korean, German, French and Portuguese using NVIDIA automatic speech recognition technology, with Japanese, Arabic and Italian expected to be added soon. The transcription workflow leverages fully customizable GPU-optimized models to enable better understanding, contextual insights and sentiment analysis with real-time accuracy. Enterprises can use the completed transcripts to improve product development and speed training of contact center agents.

Using unsupervised learning, the digital fingerprinting AI solution workflow employs threat detection to achieve comprehensive data visibility. It improves security by helping enterprises uniquely fingerprint every user, service, account and machine across the network to detect anomalous behavior. Once deployed, the workflow provides intelligent alerts and actionable information to reduce detection time from weeks to minutes to help security analysts quickly identify and act on threats.

Pretrained Models Support Explainability and Understanding

NVIDIA AI Enterprise 3.0 also features unencrypted pretrained models and source code from the latest release of NVIDIA TAO Toolkit, a low-code AI development solution for creating highly accurate, customized, production-ready AI models for speech and computer vision AI applications.

The unencrypted models are exclusively available with NVIDIA AI Enterprise and support a variety of imaging and vision AI tasks for healthcare, smart cities and retail, such as pathology tumor detection, people detection, vehicle detection, pose estimation and action recognition.

Using the pretrained models without encryption enables developers to view the weights and biases of the model, which can help in model explainability and understanding model bias. In addition, unencrypted models are easier to debug and easier to integrate into custom AI applications.

NVIDIA AI Enterprise 3.0 also introduces support for a broad range of NVIDIA AI frameworks and infrastructure options:

• NVIDIA Clara Parabricks and MONAI improve healthcare: New support for NVIDIA Clara Parabricks enables faster, more accurate genomic analysis for sequencing centers, clinical labs, genomics researchers and genomics instrument manufacturers. NVIDIA AI Enterprise also supports MONAI, a domain-specific medical imaging AI framework that provides pretrained models and a collaborative, scalable workflow for data labeling and training robust AI models.
• NVIDIA AI frameworks to boost customer service, safety, sales and more: The 50+ frameworks and pretrained models now supported in NVIDIA AI Enterprise 3.0 include NVIDIA Riva, a GPU-accelerated speech AI software development kit for building and deploying fully customizable, real-time AI pipelines that deliver world-class accuracy in all leading clouds, on premises, at the edge and on embedded devices. NVIDIA Morpheus enables cybersecurity developers to create optimized AI pipelines for filtering, processing and classifying large volumes of real-time data. SDKs in the NVIDIA Metropolis intelligent video analytics platform, such as TAO Toolkit and NVIDIA DeepStream for vision AI, are supported, as is the NVIDIA Merlin open-source framework for building high-performing recommender systems at scale.
• Expanded certification for the cloud: With NVIDIA AI Enterprise 3.0, organizations with a hybrid cloud strategy now have the flexibility to run the software on GPU-accelerated instances from Oracle Cloud Infrastructure. Customers who purchase a license through one of NVIDIA’s channel partners can deploy in OCI with full certification and support from NVIDIA on designated OCI instances. This is in addition to existing NVIDIA AI Enterprise certification for accelerated instances from Amazon Web Services, Microsoft Azure and more.
• Hewlett Packard Enterprise and NVIDIA extend AI support for hybrid data centers: HPE and NVIDIA will deliver a joint offering that provides support for the NVIDIA AI Enterprise 3.0 on HPE GreenLake and HPE Ezmeral. The solution allows customers to speed up AI application development, securely, by easily procuring and deploying NVIDIA AI Enterprise on a managed HPE GreenLake instance.
• Broadened storage and virtualization support: NVIDIA AI Enterprise 3.0 now supports NVIDIA Magnum IO GPUDirect Storage, which provides a direct data path between local or remote storage and GPU memory to further speed AI workloads. It also delivers expanded virtualization options, including Red Hat Enterprise Linux with KVM and VMware vSphere 8.

NVIDIA AI Enterprise is available now. Customers can contact NVIDIA partners worldwide for pricing. NVIDIA AI Enterprise 3.0 is expected to be available for customers with current and new subscriptions later this month. A license for NVIDIA AI Enterprise is also included with servers from NVIDIA partners that feature NVIDIA H100 PCIe GPUs, including systems from Dell Technologies, Hewlett Packard Enterprise, Lenovo and Supermicro.

Enterprises can grow their AI expertise by trying NVIDIA AI workflows and frameworks supported in NVIDIA AI Enterprise on NVIDIA LaunchPad at no charge.

Deutsche Bank Wednesday announced a partnership with NVIDIA to accelerate the use of AI and machine learning in the financial services sector.

The announcement follows months of testing to explore use cases that could support the bank’s strategic ambitions to 2025 and beyond.

“Accelerated computing and AI are at a tipping point, and we’re bringing them to the world’s enterprises through the cloud,” said NVIDIA founder and CEO Jensen Huang. “Every aspect of future business will be supercharged with insight and intelligence operating at the speed of light.

“Together with Deutsche Bank, we are modernizing and reimagining the way financial services are operated and delivered,” he added.

The potential is enormous. McKinsey estimates that AI technologies could deliver up to $1 trillion of additional value yearly for global banking.

Frankfurt-based Deutsche Bank is a leading global investment bank with more than 80,000 employees in 58 countries worldwide.

Deutsche Bank’s initiatives promise to speed efforts to serve customers worldwide, develop new data-driven products and services, increase efficiency and recruit tech talent.

Together, Deutsche Bank and NVIDIA have initially focused on three potential implementations with a multi-year ambition to expand this to over a hundred, which the companies are exploring.

With NVIDIA AI Enterprise software, Deutsche Bank’s AI developers, data scientists and IT professionals will be able to build and run AI workflows anywhere, including in its hosted on-premises data centers and on Google Cloud, the bank’s public cloud provider. (In related news, NVIDIA today announced NVIDIA AI Enterprise 3.0.)

Next-Generation Risk Management

Price discovery, risk valuation and model backtesting require computationally intensive calculations on massive traditional CPU-driven server grid farms. Accelerated compute delivers more accurate results in real time, helping provide more value to customers while lowering total costs by as much as 80%.

Many bank functions that typically process overnight, like risk valuation, can now be run in real time on accelerated compute.

This represents a leap forward in how traders can manage risk by running more scenarios faster on a more energy-efficient grid farm.

Redefining Personalized Customer Service With Interactive Avatars

Deutsche Bank is exploring how to engage employees, potential recruits and customers more interactively, improving experiences using 3D virtual avatars in real time, 24 hours a day, seven days a week.

An early potential implementation enabled Deutsche Bank to create a 3D virtual avatar to help employees navigate internal systems and respond to HR-related questions.

Future use cases will explore immersive metaverse experiences with banking clients.

Deriving Insights Out of Unstructured Data

Extracting critical information from unstructured data has long been challenging. But existing large language models don’t perform well on financial texts.

Transformers, a type of neural network that learns context and, thus, meaning from data, introduced in 2017, could change this.

A single pretrained model can perform amazing feats — including text generation, translation and even software programming — and is the basis of the new generation of AI.

Deutsche Bank and NVIDIA are testing a collection of large language models called Financial Transformers, or Finformers.

These systems will have the potential to provide early warning signs of counterparty risk, retrieve data faster and identify data quality issues.

Explore how NVIDIA’s AI solutions and enterprise-level AI platforms drive innovation in financial services. 

Training, testing and validating autonomous vehicles requires a continuous pipeline — or data factory — to introduce new scenarios and refine deep neural networks.

A key component of this process is simulation. AV developers can test a virtually limitless number of scenarios, repeatably and at scale, with high-fidelity, physically based simulation. And like much of the technology related to AI, simulation is constantly evolving and improving, getting ever nearer to closing the gap between the real and virtual worlds.

NVIDIA DRIVE Sim, built on Omniverse, provides a virtual proving ground for AV testing and validation. It’s a highly accurate simulation platform with the ability to enable groundbreaking tools, including synthetic data generation and neural reconstruction, to build digital twins of driving environments and scenarios.

Matt Cragun, senior product manager for AV simulation at NVIDIA, joined the AI Podcast to discuss the development of simulation for self-driving technology, detailing the origins and inner workings of DRIVE Sim.

He also provided a sneak peek into the frontiers researchers are exploring for this critical testing and validation technology.

Neural Reconstruction Engine in NVIDIA DRIVE Sim

NVIDIA researchers have developed an AI pipeline, known as the Neural Reconstruction Engine, that constructs a 3D scene from recorded sensor data in NVIDIA DRIVE Sim.

First demonstrated at GTC22, these AI tools bring the real world directly in simulation to increase realism and speed up autonomous vehicle production.

NRE uses multiple AI networks to create interactive 3D test environments where developers can modify the scene and see how the world reacts. Developers can change scenarios, add synthetic objects, and apply randomizations—such as a child following a bouncing ball into the road—making the initial scenarios even more challenging.

You Might Also Like

Driver’s Ed: How Waabi Uses AI, Simulation to Teach Autonomous Vehicles to Drive

Teaching the AI brains of autonomous vehicles to understand the world as humans do requires billions of miles of driving experience. The road to achieving this astronomical level of driving leads to the virtual world. Learn how Waabi uses powerful high-fidelity simulations to train and develop production-level autonomous vehicles.

Polestar’s Dennis Nobelius on the Sustainable Performance Brand’s Plans

Driving enjoyment and autonomous driving capabilities can complement one another in intelligent, sustainable vehicles. Learn about the automaker’s plans to unveil its third vehicle, the Polestar 3, the tech inside it, and what the company’s racing heritage brings to the intersection of smarts and sustainability.

GANTheftAuto: Harrison Kinsley on AI-Generated Gaming Environments

Humans playing games against machines is nothing new, but now computers can develop their own games for people to play. Programming enthusiast and social media influencer Harrison Kinsley created GANTheftAuto, an AI-based neural network that generates a playable chunk of the classic video game Grand Theft Auto V.

Subscribe to the AI Podcast: Now Available on Amazon Music

The AI Podcast is now available through Amazon Music.

In addition, get the AI Podcast through iTunes, Google Podcasts, Google Play, Castbox, DoggCatcher, Overcast, PlayerFM, Pocket Casts, Podbay, PodBean, PodCruncher, PodKicker, Soundcloud, Spotify, Stitcher and TuneIn.

Make the AI Podcast better: Have a few minutes to spare? Fill out this listener survey.

For every minute that a stroke is left untreated, the average patient loses nearly 2 million neurons. This means that for each hour in which treatment fails to occur, the brain loses as many neurons as it does in more than three and a half years of normal aging.

With one of the world’s first portable brain scanners for stroke diagnosis, Australia-based healthcare technology developer EMVision is on a mission to enable quicker triage and treatment to reduce such devastating impacts.

The NVIDIA Inception member’s EMVision device fits like a helmet and can be used at the point of care and in ambulances for prehospital stroke diagnosis. It relies on electromagnetic imaging technology and uses NVIDIA-powered AI to distinguish between ischaemic and haemorrhagic strokes — clots and bleeds — in just minutes.

A cart-based version of the device, built using the NVIDIA Jetson edge AI platform and NVIDIA DGX systems, can also help with routine monitoring of a patient post-intervention to inform their progress and recovery.

“With EMVision, the healthcare community can access advanced, portable solutions that will assist in making critical decisions and interventions earlier, when time is of the essence,” said Ron Weinberger, CEO of EMVision. “This means we can provide faster stroke diagnosis and treatment to ensure fewer disability outcomes and an improved quality of life for patients.”

Point-of-Care Diagnosis

Traditional neuroimaging techniques, like CT scans and MRIs, produce excellent images but require large, stationary, complex machines and specialist operators, Weinberger said. This limits point-of-care accessibility.

The EMVision device is designed to scan the brain wherever the patient may be — in an ambulance or even at home if monitoring a patient who has a history of stroke.

“Whether for a new, acute stroke or a complication of an existing stroke, urgent brain imaging is required before correct triage, treatment or intervention decisions can be made,” Weinberger said.

The startup has developed and validated novel electromagnetic brain scanner hardware and AI algorithms capable of classifying and localizing a stroke, as well as creating an anatomical reconstruction of the patient’s brain.

“NVIDIA accelerated computing has played an important role in the development of EMVision’s technology, from hardware verification and algorithm development to rapid image reconstruction and AI-powered decision making,” Weinberger said. “With NVIDIA’s support, we are set to transform stroke diagnosis and care for patients around the world.”

EMVision uses NVIDIA DGX for hardware verification and optimization, as well as for prototyping and training AI models. EMVision has trained its AI models 10x faster using NVIDIA DGX compared with other systems, according to Weinberger.

Each brain scanner has an NVIDIA Jetson AGX Xavier module on board for energy-efficient AI inference at the edge. And the startup is looking to use NVIDIA Jetson Orin Nano modules for next-generation edge AI.

“The interactions between low-energy electromagnetic signals and brain tissue are incredibly complex,” Weinberger said. “Making sense of these signal interactions to identify if pathologies are present and recreate quality images wouldn’t be possible without the massive power of NVIDIA GPU-accelerated computing.”

As a member of NVIDIA Inception, a free, global program for cutting-edge startups, EMVision has shortened product development cycles and go-to-market time, Weinberger added.

Subscribe to NVIDIA healthcare news and learn more about NVIDIA Inception.

More than 75 million people speak Telugu, predominantly in India’s southern regions, making it one of the most widely spoken languages in the country.

Despite such prevalence, Telugu is considered a low-resource language when it comes to speech AI. This means there aren’t enough hours’ worth of speech datasets to easily and accurately create AI models for automatic speech recognition (ASR) in Telugu.

And that means billions of people are left out of using ASR to improve transcription, translation and additional speech AI applications in Telugu and other low-resource languages.

To build an ASR model for Telugu, the NVIDIA speech AI team turned to the NVIDIA NeMo framework for developing and training state-of-the-art conversational AI models. The model won first place in a competition conducted in October by IIIT-Hyderabad, one of India’s most prestigious institutes for research and higher education.

NVIDIA placed first in accuracy for both tracks of the Telugu ASR Challenge, which was held in collaboration with the Technology Development for Indian Languages program and India’s Ministry of Electronics and Information Technology as a part of its National Language Translation Mission.

For the closed track, participants had to use around 2,000 hours of a Telugu-only training dataset provided by the competition organizers. And for the open track, participants could use any datasets and pretrained AI models to build the Telugu ASR model.

NVIDIA NeMo-powered models topped the leaderboards with a word error rate of approximately 13% and 12% for the closed and open tracks, respectively, outperforming by a large margin all models built on popular ASR frameworks like ESPnet, Kaldi, SpeechBrain and others.

“What sets NVIDIA NeMo apart is that we open source all of the models we have — so people can easily fine-tune the models and do transfer learning on them for their use cases,” said Nithin Koluguri, a senior research scientist on the conversational AI team at NVIDIA. “NeMo is also one of the only toolkits that supports scaling training to multi-GPU systems and multi-node clusters.”

Building the Telugu ASR Model

The first step in creating the award-winning model, Koluguri said, was to preprocess the data.

Koluguri and his colleague Megh Makwana, an applied deep learning solution architect manager at NVIDIA, removed invalid letters and punctuation marks from the speech dataset that was provided for the closed track of the competition.

“Our biggest challenge was dealing with the noisy data,” Koluguri said. “This is when the audio and the transcript don’t match — in this case you cannot guarantee the accuracy of the ground-truth transcript you’re training on.”

The team cleaned up the audio clips by cutting them to be less than 20 seconds, chopped out clips of less than 1 second and removed sentences with a greater-than-30 character rate, which measures characters spoken per second.

Makwana then used NeMo to train the ASR model for 160 epochs, or full cycles through the dataset, which had 120 million parameters.

For the competition’s open track, the team used models pretrained with 36,000 hours of data on all 40 languages spoken in India. Fine-tuning this model for the Telugu language took around three days using an NVIDIA DGX system, according to Makwana.

Inference test results were then shared with the competition organizers. NVIDIA won with around 2% better word error rates than the second-place participant. This is a huge margin for speech AI, according to Koluguri.

“The impact of ASR model development is very high, especially for low-resource languages,” he added. “If a company comes forward and sets a baseline model, as we did for this competition, people can build on top of it with the NeMo toolkit to make transcription, translation and other ASR applications more accessible for languages where speech AI is not yet prevalent.”

NVIDIA Expands Speech AI for Low-Resource Languages

“ASR is gaining a lot of momentum in India majorly because it will allow digital platforms to onboard and engage with billions of citizens through voice-assistance services,” Makwana said.

And the process for building the Telugu model, as outlined above, is a technique that can be replicated for any language.

Of around 7,000 world languages, 90% are considered to be low resource for speech AI — representing 3 billion speakers. This doesn’t include dialects, pidgins and accents.

Open sourcing all of its models on the NeMo toolkit is one way NVIDIA is improving linguistic inclusion in the field of speech AI.

In addition, pretrained models for speech AI, as part of the NVIDIA Riva software development kit, are now available in 10 languages — with many additions planned for the future.

And NVIDIA this month hosted its inaugural Speech AI Summit, featuring speakers from Google, Meta, Mozilla Common Voice and more. Learn more about “Unlocking Speech AI Technology for Global Language Users” by watching the presentation on demand.

Get started building and training state-of-the-art conversational AI models with NVIDIA NeMo.

Editor’s note: This post is a part of our Meet the Omnivore series, which features individual creators and developers who use NVIDIA Omniverse to accelerate their 3D workflows and create virtual worlds.

As a Microsoft Certified Azure cloud specialist and DevOps automation engineer, Gavin Stevens is deeply in tune with cloud architect workflows.

He noticed an opportunity to improve the abilities of cloud architects to visualize their infrastructure — the combination of hardware and software necessary for cloud computing — by creating a 3D layout of it.

So Stevens set out to enable this by building an extension for NVIDIA Omniverse — a platform for connecting and building custom 3D pipelines and metaverse applications.

Dubbed Meta Cloud Explorer, the open-source extension generates digital 3D models of engineers’ cloud infrastructure components at scale, based on contextual metadata from their Azure cloud portals.

The visualization can then be organized by group, location, subscription and resource type. It also displays infrastructure layouts and costs on various planes. This can help cloud architects gain insights to optimize resources, reduce costs and improve customer experiences.

“There’s no shortage of ‘infrastructure diagram generation’ tools that can produce 2D representations of your cloud infrastructure,” Stevens said. “But most of these tools present a tightly focused exploration context, where it’s difficult to see your infrastructure at scale.”

Meta Cloud Explorer, instead, displays 3D representations that can be rearranged at scale. It’s one of the winning submissions for the inaugural #ExtendOmniverse contest, where developers were invited to create their own Omniverse extension for a chance to win an NVIDIA RTX GPU.

Omniverse extensions are core building blocks that let anyone create and extend functions of Omniverse apps using the popular Python and C++ programming languages.

Building Custom Workflow Tools

Stevens, who’s based in Scottsdale, Arizona, learned how to build the Omniverse extension in just a few months by attending community livestreams, learning Python and prototyping user interfaces based on sample resources.

He first transformed Microsoft Azure’s open-source 2D icons — representing storage accounts, web apps, databases and more — into 3D assets using Blender software. He easily brought these into Omniverse with Universal Scene Description (USD), an open-source, extensible file framework that serves as the common language for building virtual worlds and the metaverse.

Stevens then composed a 3D layout, arranging and visualizing the infrastructure services based on data such as location, type and cost by implementing a custom packing and layout algorithm. He also created a user interface directly in the scene to display details such as a cluster’s total cost or a service’s status.

“Omniverse takes care of the rendering and helps developers work at a higher level to easily visualize things in a 3D space,” Stevens said. “And USD makes it seamless to reference and position 3D objects within scenes.”

Dive deeper into Stevens’ workflow by watching this video:

Stevens is now planning to expand Meta Cloud Explorer’s capabilities to build an advanced software-as-a-service that enables users to create infrastructure from template libraries, learn about new architecture techniques and simulate design changes.

Being able to manipulate cloud infrastructure layouts in 3D, or even in virtual reality, would open up new possibilities for developers and cloud engineers to realize a customer’s vision, Stevens said.

“I’m not sure how you could even do this without Omniverse,” he added. “Omniverse Kit provides a dynamic, easy-to-use platform for building metaverse applications. And the ability to connect external application programming interfaces and data sources opens up flexibility when using Omniverse.”

Developers like Stevens can enhance their workflows with the recent Omniverse beta release, which includes major updates to core reference applications and tools for developers, creators and novices looking to build metaverse applications.

Join In on the Creation

Creators and developers across the world can download NVIDIA Omniverse for free, and enterprise teams can use the platform for their 3D projects.

Discover how to build an Omniverse extension in less than 10 minutes.

To find out how to accelerate cloud workflows, join NVIDIA at AWS re:Invent, running through Friday, Dec. 2.

For a deeper dive into developing on Omniverse, watch the on-demand NVIDIA GTC session, “How to Build Extensions and Apps for Virtual Worlds With NVIDIA Omniverse.”

Find additional documentation and tutorials in the Omniverse Resource Center, which details how developers can build custom USD-based applications and extensions for the platform.

To discover more free tools, training and a community for developers, join the NVIDIA Developer Program.

Follow NVIDIA Omniverse on Instagram, Medium, Twitter and YouTube for additional resources and inspiration. Check out the Omniverse forums, and join our Discord server and Twitch channel to chat with the community.

Livermore, Calif., renowned for research and vineyards, is plowing in a new distinction: the birthplace of the first commercially available smart tractor.

Local startup Monarch Tractor has announced the first of six Founder Series MK-V tractors are rolling off the production line at its headquarters. Constellation Brands, a leading wine and spirits producer and beer importer, will be the first customer given keys  at a launch event today.

The debut caps a two-year development sprint since Monarch, founded in 2018, hatched plans to deliver its smart tractor, complete with the energy-efficient NVIDIA Jetson edge AI platform. The tractor combines electrification, automation, and data analysis to help farmers reduce their carbon footprint, improve field safety, streamline farming operations, and increase their bottom lines.

The MK-V tractor cuts energy costs and diesel emissions, while also helping reduce harmful herbicides, which are expensive and deplete the soil.

“With precision ag, autonomy and AI, data will decrease the volume of chemicals used, which is good for the soil, good for the farmer from a profitability standpoint, and good for the consumer,” said Praveen Penmetsa, CEO of Monarch Tractor.

The delivery of MK-V tractors to Constellation Brands will be followed with additional tractor shipments to family farms and large corporate customers, according to the company.

Monarch is a member of the NVIDIA Inception program, which provides startups with technology support and AI platforms guidance.

Leading Farming AI Wave of Clean Tractors

Monarch Tractor founders include veterans of Silicon Valley’s EV scene who worked together at startup Zoox, now Amazon owned. Carlo Mondavi, from the Napa Valley Mondavi winery family, is a sustainability-focused vintner and chief farming officer.  Mark Schwager, former Tesla Gigafactory chief, is president; Zachary Omohundro, a robotics Ph.D. from Carnegie Mellon, is CTO; Penmetsa is an autonomy and mobility engineer.

Penmetsa likens the revolutionary new tractor to paradigm shifts in PCs and smartphones, enablers of world-changing applications. Monarch’s role, he said, is as the hub to enable smart implements — precision sprayers, harvesters and more — for computer vision applications to help automate farming.

In 2021, Monarch launched pilot test models for commercial use at Wente Vineyards, also based in Livermore. The trial at Wente compared its energy usage to that of a diesel tractor, noting Monarch saved more than $2,600 in annual expenses.

Monarch has raised more than $110 million in funding. Strategic investors include Japanese auto parts maker Musashi Seimitsu Industry Co; CNH Industrial, an agricultural equipment maker; and VST Tillers Tractors, an India-based equipment maker and dealer of tractors and implements.

It recently signed a contract manufacturing agreement with Hon Hai Technology Group Foxconn to build the MK-V and its battery packs at the Mahoning Valley, Ohio, plant.

As a wave of AI sweeps farming, developers are working to support more sustainable farming practices.

The NVIDIA Jetson platform provides energy-efficient computing to the MK-V, which offers advances in battery performance.

What a Jetson-Supported Monarch Founder Series MK-V Can Do

Tapping into six NVIDIA Jetson Xavier NX SOMs (system on modules), Monarch’s Founder Series MK-V tractors are essentially roving robots packing supercomputing.

Monarch has harnessed Jetson to deliver tractors that can safely traverse rows within agriculture fields using only cameras. “This is important in certain agriculture environments because there may be no GPS signal,” said Penmetsa. “It’s also crucial for safety as the Monarch is intended for totally driverless operation.”

The Founder Series MK-V runs two 3D cameras and six standard cameras. With the six Jetson edge AI modules on board, it can run models for multiple farming tasks when paired with different implements.

Supporting more sustainable farming practices, computer vision applications are available to fine-tune with transfer learning for the Monarch platform to develop precision spraying and other options.

Delivering Farm-Apps-as-a-Service 

Monarch offers a core of main applications to assist farms with AI, available in a software-as-a-service model on its platform.

The Founder Series MK-V has some basic functions on its platform as well, such as sending alerts when on a low charge or there’s an unidentified object obstructing a path. It will also shut down from spraying if its camera-based vision platform identifies a human.

The tractor collects and analyzes crop data daily and can process data from current and next-generation implements equipped with sensors and imaging. This data can be used for real-time implement adjustments, long-term yield estimates, current growth stages and other plant and crop health metrics.

Wider availability of the tractor begins a new chapter in improved farming practices.

“The marriage of NVIDIA accelerated computing with Jetson edge AI on our Monarch MK-V has helped our customers reduce the use of unneeded herbicides with our cutting-edge, zero-emission tractor – this revolutionary technology is helping our planet’s soil, waterways and biodiversity,” said Mondavi.

Learn more about NVIDIA Isaac platform for robotics and apply to join NVIDIA Inception.

It’s a new month, which means GeForce NOW’s got the list of 22 new games arriving in December.

Rise up for Marvel’s Midnight Suns, from publisher 2K Games, streaming on GeForce NOW later this month.

Then get ready to move out, members. Battlefield 2042 is the latest game from the Electronic Arts catalog streaming on GeForce NOW. It arrives just in time for the free access weekend, running Dec. 1-4, and comes with a members-only reward.

These games lead the charge, among the six additions streaming this week.

Time to Assemble 

From the creators of XCOM, and published by 2K Games, Marvel’s Midnight Suns is a tactical role-playing game set in the darker, supernatural side of the Marvel Universe. It launches on Steam on Friday, Dec. 2, with GeForce NOW members getting into the action later this month.

Play as “The Hunter,” a legendary demon slayer with a mysterious past and the first-ever customizable superhero in the Marvel Universe. Put together a team of legendary Marvel heroes, including Scarlet Witch, Spiderman, Wolverine, Blade and Captain America.

These heroes must fight together to stop the Mother of Demons from completing an ancient prophecy. In revolutionary card-based tactical battles, players can use ability cards on enemies, themselves or the environment.

Stay tuned for updates on the game’s release on GeForce NOW.

Step Foot Onto the Battlefield

Prepare to charge into Battlefield 2042, the first-person shooter that marks the return to the iconic all-out warfare of the widely popular franchise from Electronic Arts. Available today along with the latest update, “Season 3: Escalation,” the game marks the 19th title from EA to join GeForce NOW.

Adapt and overcome in a near-future world transformed by disorder. Choose your role on the battlefield with class specialists and form a squad to bring a cutting-edge arsenal into dynamically changing battlegrounds of unprecedented scale and epic destruction.

With RTX ON, EA and DICE introduced ray-traced ambient occlusion in Battlefield 2042. This accurately adds shadows where game elements occlude light, whether between a soldier and a wall, a tank and the tarmac, or foliage and the ground. Members can use NVIDIA DLSS to get the definitive PC experience, with maxed-out graphics, high frame rates and uncompromised image quality.

The game comes with a special reward for GeForce NOW members. To opt in and receive rewards, log in to your NVIDIA account and select “GEFORCE NOW” from the header, then scroll down to “REWARDS” and click the “UPDATE REWARDS SETTINGS” button. Check the box in the dialogue window that shows up to start receiving special offers and in-game goodies.

Experience the action across compatible devices and take gaming to the max with all the perks of an RTX 3080 membership, like 4K resolution, RTX ON and maximized gaming sessions.

The More the Merrier

Members can look for the following six games available to play this week:

• The Knight Witch (New release on Steam, Nov. 29)
• Warhammer 40,000: Darktide (New Release on Steam, Nov. 30)
• Fort Triumph (Free on Epic Games Store, Dec. 1-8)
• Battlefield 2042 (Steam and Origin)
• Alien Swarm: Reactive Drop (Steam)
• Stormworks: Build and Rescue (Steam)

Then time to unwrap the rest of the list of 22 games coming this month: 

• Marvel’s Midnight Suns (New release on Steam, coming soon)
• Art of the Rail (New release on Steam, Dec. 4)
• Swordship (New release on Steam, Dec. 5)
• Knights of Honor II: Sovereign (New release on Steam, Dec. 6)
• Chained Echoes (New release on Steam, Dec. 7)
• IXION (New release on Steam, Dec. 7)
• Togges (New release on Steam, Dec. 7)
• SAMURAI MAIDEN (New release on Steam, Dec. 8)
• Wavetale (New release on Steam, Dec. 12)
• Master of Magic (New release on Steam, Dec. 13)
• BRAWLHALLA (Ubisoft Connect)
• Carrier Command 2  (Steam)
• Cosmoteer: Starship Architect & Commander (Steam)
• Dakar Desert Rally (Epic Game Store)
• Dinkum (Steam)
• Floodland (Steam)
• Project Hospital (Steam)

Nothing Left Behind From November

On top of the 26 games announced in November, members can play the extra 10 games that were added to GeForce NOW last month:

• Alba: A Wildlife Adventure (Epic Games)
• Shadow Tactics: Blades of the Shogun (Epic Games)
• Ballads of Hongye (Steam)
• Bravery and Greed (Steam)
• Frozen Flame (Steam)
• The Darkest Tales (Epic Games)
• The Tenants (Epic Games)
• Ship of Fools (Steam)
• Crysis 2 Remastered (Steam)
• Crysis 3 Remastered (Steam)

And good things come in small packages — for the perfect stocking stuffer or last-minute gift, look no further than GeForce NOW. Physical or digital gift cards are always available, and tomorrow is the last day to get in on the “Green Thursday Black Friday” deal.

Before you start off a super weekend of gaming, there’s only one choice left to make. Let us know your pick on Twitter or in the comments below.

You’re becoming a super hero but can only pick one of the super powers below. Which do you choose?

Flying
Super-speed
Telekinesis
Super-strength
Time Travel

— NVIDIA GeForce NOW (@NVIDIAGFN) November 30, 2022

The promise of quantum computing is to solve unsolvable problems. And companies are already making headway with hybrid approaches — those that combine classical and quantum computing — to tackle challenges like drug discovery for incurable diseases.

By accelerating drug molecule simulation and modeling with hybrid quantum computing, startup Qubit Pharmaceuticals is significantly reducing the time and investment needed to identify promising treatments in oncology, inflammatory diseases and antivirals.

Qubit is building a drug discovery platform using the NVIDIA QODA programming model for hybrid quantum-classical computers and the startup’s Atlas software suite. Atlas creates detailed simulations of physical molecules, accelerating calculations by a factor of 100,000 compared to traditional research methods.

Founded in 2020, the Paris and Boston-based company is a member of NVIDIA Inception, a program that offers go-to-market support, expertise and technology for cutting-edge startups.

Qubit has one of France’s largest GPU supercomputers for drug discovery, powered by NVIDIA DGX systems. The startup aims for pharmaceutical companies to begin testing their first drug candidates discovered through its GPU-accelerated research next year.

“By combining NVIDIA’s computational power and leading-edge software with Qubit’s simulation and molecular modeling capabilities, we are confident in our ability to dramatically reduce drug discovery time and cut its cost by a factor of 10,” said Robert Marino, president of Qubit Pharmaceuticals. “This unique collaboration should enable us to develop the first quantum physics algorithms applied to drug discovery.”

Tapping Unprecedented Computational Capabilities 

Computational drug discovery involves generating high-resolution simulations of potential drug molecules and predicting how well those molecules might bind to a target protein in the body.

For accurate results, researchers need to perform massive sampling, simulating hundreds of different conformations — possible spatial arrangements of a molecule’s atoms. They must also correctly model molecules’ force fields, the electric charges that predict affinity, or how a molecule will bind to another.

This simulation and modeling requires high performance computing, so Qubit selected an in-house supercomputer built with NVIDIA DGX systems and other NVIDIA-accelerated servers, totaling 200 NVIDIA Tensor Core GPUs. The supercomputer runs Qubit’s Atlas software, performing in just a few hours calculations that would take several years with conventional methods.

Atlas models quantum physics at the microscopic level to achieve maximum accuracy. The Qubit team is adopting NVIDIA QODA to explore the hybrid use of GPU-accelerated supercomputers and quantum computers, where QPUs, or quantum processing units, could one day speed up key software kernels for molecular modeling.

Using the NVIDIA cuQuantum SDK, Qubit’s developers can simulate quantum circuits, allowing the team to design algorithms ready to run on future quantum computers.

AI for Every Stage of Drug Discovery

Qubit estimates that while conventional research methods require pharmaceutical developers to start by synthesizing an average of 5,000 drug compounds before preclinical testing to bring a single drug to market, a simulation-based drug discovery approach could reduce the figure to about 200 — saving hundreds of millions of dollars and years of development time.

The company’s Atlas software includes AI algorithms for every stage of the drug discovery cycle. To support target characterization, where researchers analyze a protein that plays a role in disease, Atlas supports molecular dynamics simulations at microsecond timescales — helping scientists identify new pockets for drug molecules to bind with the protein.

During drug candidate screening and validation, researchers can use AI models that help narrow the field of potential molecules and generate novel compounds. Qubit is also developing additional filters that predict a candidate molecule’s druggability, safety and cross-reactivity.

Learn more about Qubit’s HPC and quantum-accelerated molecular dynamics software from company co-founders Jean-Philip Piquemal and Louis Lagardère through NVIDIA On-Demand.

Main image courtesy of Qubit Pharmaceuticals.