Posts in category: nVidia AI

As a medical doctor in Nigeria, Tobi Olatunji knows the stress of practicing in Africa’s busy hospitals. As a machine-learning scientist, he has a prescription for it.

“I worked at one of West Africa’s largest hospitals, where I would routinely see more than 30 patients a day —  it’s a very hard job,” said Olatunji.

The need to write detailed patient notes and fill out forms makes it even harder. Paper records slowed the pace of medical research, too.

In his first years of practice, Olatunji imagined a program to plow through the mounds of paperwork, freeing doctors to help more patients.

It’s been a journey, but that software is available today from his company, Intron Health, a member of the NVIDIA Inception program, which nurtures cutting-edge startups.

A Side Trip in Tech

With encouragement from med school mentors, Olatunji got a master’s degree in medical informatics from the University of San Francisco and another in computer science at Georgia Tech. He started working as a machine-learning scientist in the U.S. by day and writing code on nights and weekends to help digitize Africa’s hospitals.

A pilot test during the pandemic hit a snag.

The first few doctors to use the code took 45 minutes to finish their patient notes. Feeling awkward in front of a keyboard, some health workers said they prefer pen and paper.

“We made a hard decision to invest in natural language processing and speech recognition,” he said. It’s technology he was already familiar with in his day job.

Building AI Models

“The combination of medical terminology and thick African accents produced horrible results with most existing speech-to-text software, so we knew there would be no shortcut to training our own models,” he said.

The Intron team evaluated several commercial and open-source speech recognition frameworks and large language models before choosing to build with NVIDIA NeMo, a software framework for text-based generative AI. In addition, the resulting models were trained on NVIDIA GPUs in the cloud.

“We initially tried to train with CPUs as the cheapest option, but it took forever, so we started with a single GPU and eventually grew to using several of them in the cloud,” he said.

The resulting Transcribe app captures doctors’ dictated messages with more than 92% accuracy across more than 200 African accents. It slashes the time they spend on paperwork by 6x on average, according to an ongoing study Intron is conducting across hospitals in four African countries.

“Even the doctor with the fastest typing skills in the study got a 40% speedup,” he said of the software now in use at several hospitals across Africa.

Listening to Africa’s Voices

Olatunji knew his models needed high quality audio data. So, the company created an app to capture sound bites of medical terms spoken in different accents.

To date, the app’s gathered more than a million clips from more than 7,000 people across 24 countries, including 13 African nations. It’s one of the largest datasets of its type, parts of which have been released as open source to support African speech research.

Today, Intron refreshes its models every other month as more data comes in.

Nurturing Diversity in Medtech

Very little research exists on speech recognition for African accents in a clinical setting. So, working with Africa’s tech communities like DSN, Masakhane and Zindi, Intron launched AfriSpeech-200, a developer challenge to kickstart research using its data.

Similarly, for all its sophistication, medtech lags in diversity and inclusion, so Olatunji recently launched an effort that addresses that issue, too.

Bio-RAMP Lab is a global community of minority researchers working on problems they care about at the intersection of AI and healthcare. The group already has a half dozen papers under review at major conferences.

“For seven years, I was the only Black person on every team I worked on,” he said. “There were no Black scientists or managers, even in my job interviews.”

Meanwhile, Intron is even helping hospitals in Africa find creative ways to acquire the hardware they need. It’s another challenge on the way to opening up huge opportunities.

“Once healthcare data gets digitized, you unlock a whole new world for research into areas like predictive models that can be early warning systems for epidemics — we can’t do it without data,” Olatunji said.

Watch a masterclass (starting at 20:30) with Olatunji, HuggingFace and NVIDIA on AI for speech recognition.

Make gaming a priority this GFN Thursday — time’s running out to upgrade to a GeForce NOW Priority six-month membership at 40% off the normal price. Find out how new Priority members are using the cloud to get their game on.

Plus, the week brings updates for some of the hottest games in the GeForce NOW library, and four more titles join the list.

GeForce NOW RTX 4080 SuperPODs are now live for Ultimate members in Atlanta, where the gamers game. Follow along with the server rollout, and upgrade today for the Ultimate cloud gaming experience.

Priority Check

Through Sunday, May 21, save 40% on a six-month Priority membership for $29.99, normally $49.99.

Priority memberships are perfect for those looking to try GeForce NOW or lock in a lower price for a half-year. Priority members get higher access to GeForce gaming servers, meaning less wait times than free members.

Members who claimed this offer in its first week alone played over 1,000 different titles in the GeForce NOW library, for 30,000+ streamed hours. That means these Priority members skipped the line by more than 500 hours.

They also played the best of PC gaming across multiple devices — PCs, Macs, mobile devices and smart TVs, plus new categories of devices made possible by the cloud, like gaming Chromebooks and cloud gaming handheld devices. And they experienced the cinematic quality of RTX ON in supported titles.

With more than 1,600 titles in the GeForce NOW library, there’s something for everyone to play. Jump into squad-based action in Fortnite or Destiny 2, bring home the victory League of Legends or Counter-Strike: Global Offensive, and explore in open-world role-playing games like Genshin Impact and Cyberpunk 2077. With GeForce NOW Priority, members can get straight into the action.

But don’t wait: This offer ends on Sunday, May 21, so make it a priority to upgrade today.

Game On

GFN Thursday means more games for more gamers. This week brings new additions to the GeForce NOW library, and new updates for the hottest games.

Apex Legends: Arsenal, the latest season in EA and Respawn Entertainment’s battle royale FPS, is available this week for GeForce NOW members. Meet the newest playable Legend, Ballistic, who’s come out of retirement to teach the young pups some respect. Battle through an updated World’s Edge map, hone your skills in the newly updated Firing Range and progress through the new Weapon Mastery system.

In addition, Occupy Mars, the latest open-world sandbox game from Pyramid Games, joins the GeForce NOW library this week. Explore and colonize Mars, building a home base and discovering new regions. Grow crops, conduct mining operations and survive on an unforgiving planet. As all sci-fi films that take place on Mars have shown, things don’t always go as planned. Players must learn to cope and survive on the red planet.

For more action, take a look at what’s joining the GeForce NOW library this week:

• Voidtrain (New release on Steam, May 9)
• Occupy Mars: The Game (New release on Steam, May 10)
• Far Cry 6 (New Release Steam, May 11)
• TT Isle of Man: Ride on the Edge 3 (New release on Steam, May 11)

Ultimate members can now enable real-time ray tracing in Fortnite. The island’s never looked so good.

What are you playing this weekend? We’ve got a little challenge for you this week. Let us know your response on Twitter or in the comments below.

Replace a word in a video game title with “cloud.”

We’ll go first: Cloud of Legends

— NVIDIA GeForce NOW (@NVIDIAGFN) May 10, 2023

For telcos around the world, one of the biggest challenges to upgrading networks has always been the question, “If you build it, will they come?”

Asia’s leading telco, Singtel, believes the key to helping customers innovate with AI across industries — for everything from traffic and video analytics to conversational AI avatars powered by large language models (LLMs) — is to offer multi-access edge compute services on its high-speed, ultra-low-latency 5G network.

Multi-access edge computing, or MEC, moves the computing of traffic and services from a centralized cloud to the edge of the network, where it’s closer to the customer. Doing so reduces network latency and lowers costs through sharing of network resources.

Singtel is collaborating with Microsoft and NVIDIA to combine AI and 5G, so enterprises can boost their innovation and productivity. Using NVIDIA’s full-stack accelerated computing platform optimized for Microsoft Azure Public MEC, the telco is creating solutions that enable customers to leverage AI video analytics for multiple use cases and to deploy 5G conversational avatars powered by LLMs.

From Sea to Shore

Singtel has been rolling out enterprise 5G and MEC across ports, airports, manufacturing facilities and other locations. In addition to running low-latency applications at the edge using Singtel’s 5G network, the solution has the potential to transform operations in sectors such as public safety, urban planning, healthcare, banking, civil service, transportation and logistics. It also offers high security for public sector customers and better performance for end users, enabling new intelligent edge scenarios.

Customers can use these capabilities through Microsoft Azure, only paying for the amount of compute and storage they use for the duration in which they use it. This replicates the cloud consumption model at the network edge and lets users save on additional operational overhead.

Edge Technologies

Singtel is working with video analytics software-makers participating in NVIDIA Inception, a free program that offers startups go-to-market support, expertise and technology. These ISVs will be able to use the NVIDIA Jetson Orin module for edge AI and robotics in conjunction with Microsoft MEC to identify traffic flows at airports and other high-population areas, retail video analytics and other use cases.

Singtel and NVIDIA are also showcasing their technology and solutions, including a real-time LLM-powered avatar developed by system integrator Quantiphi and based on NVIDIA Omniverse digital twin technology, at a May 11 launch event in Singapore. The avatar, built with NVIDIA Riva speech AI and the NeMo Megatron transformer model, enables people to interact in natural language on any topic of interest. Businesses can deploy these avatars anywhere over 5G.

Using Singtel’s high-speed, low-latency 5G — combined with NVIDIA AI accelerated infrastructure and capabilities — enterprises can explore use cases on everything from computer vision and mixed reality to autonomous guided vehicles.

Singtel plans to expand these new capabilities beyond Singapore to other countries and affiliated telcos, as well. This collaboration will help redefine what’s possible through the powerful combination of compute and next-generation networks, unlocking new operational efficiencies, revenue streams and customer experiences.

Editor’s note: This post is a part of our Meet the Omnivore series, which features individual creators and developers who use NVIDIA Omniverse to accelerate their 3D workflows and create virtual worlds.

Creative studio Elara Systems doesn’t shy away from sensitive subjects in its work.

Part of its mission for a recent client was to use fun, captivating visuals to help normalize what could be considered a touchy health subject — and boost medical outcomes as a result.

In collaboration with Boston Scientific and the Sickle Cell Society, the Elara Systems team created a character-driven 3D medical animation using the NVIDIA Omniverse development platform for connecting 3D pipelines and building metaverse applications.

The video aims to help adolescents experiencing sickle cell disease understand the importance of quickly telling an adult or a medical professional if they’re experiencing symptoms like priapism — a prolonged, painful erection that could lead to permanent bodily damage.

“Needless to say, this is something that could be quite frightening for a young person to deal with,” said Benjamin Samar, technical director at Elara Systems. “We wanted to make it crystal clear that living with and managing this condition is achievable and, most importantly, that there’s nothing to be ashamed of.”

To bring their projects to life, the Elara Systems team turns to the USD Composer app, generative AI-powered Audio2Face and Audio2Gesture, as well as Omniverse Connectors to Adobe Substance 3D Painter, Autodesk 3ds Max, Autodesk Maya and other popular 3D content-creation tools like Blender, Epic Games Unreal Engine, Reallusion iClone and Unity.

For the sickle cell project, the team relied on Adobe Substance 3D Painter to organize various 3D environments and apply custom textures to all five characters. Adobe After Effects was used to composite the rendered content into a single, cohesive short film.

It’s all made possible thanks to the open and extensible Universal Scene Description (USD) framework on which Omniverse is built.

“USD is extremely powerful and solves a ton of problems that many people may not realize even exist when it comes to effectively collaborating on a project,” Samar said. “For example, I can build a scene in Substance 3D Painter, export it to USD format and bring it into USD Composer with a single click. Shaders are automatically generated and linked, and we can customize things further if desired.”

An Animation to Boost Awareness

Grounding the sickle cell awareness campaign in a relatable, personal narrative was a “uniquely human approach to an otherwise clinical discussion,” said Samar, who has nearly two decades of industry experience spanning video production, motion graphics, 3D animation and extended reality.

The team accomplished this strategy through a 3D character named Leon — a 13-year-old soccer lover who shares his experiences about a tough day when he first learned how to manage his sickle cell disease.

The project began with detailed discussions about Sickle Cell Society’s goals for the short, followed by scripting, storyboarding and creating various sketches. “Once an early concept begins to crystallize in the artists’ minds, the creative process is born and begins to build momentum,” Samar said.

Then, the team created rough 2D mockups using the illustration app Procreate on a tablet. This stage of the artistic process centered on establishing character outfits, proportions and other details. The final concept art was used as a clear reference to drive the rest of the team’s design decisions.

Moving to 3D, the Elara Systems team tapped Autodesk Maya to build, rig and fully animate the characters, as well as Adobe Substance 3D Painter and Autodesk 3ds Max to create the short’s various environments.

“I’ve found the animated point cache export option in the Omniverse Connector for Maya to be invaluable,” Samar said. “It helps ensure that what we’re seeing in Maya will persist when brought into USD Composer, which is where we take advantage of real-time rendering to create high-quality visuals.”

The real-time rendering enabled by Omniverse was “critically important, because without it, we would have had zero chance of completing and delivering this content anywhere near our targeted deadline,” the technical artist said.

“I’m also a big fan of the Reallusion to Omniverse workflow,” he added.

The Connector allows users to easily bring characters created using Reallusion iClone into Omniverse, which helps to deliver visually realistic skin shaders. And USD Composer can enable real-time performance sessions for iClone characters when live-linked with a motion-capture system.

“Omniverse offers so much potential to help streamline workflows for traditional 3D animation teams, and this is just scratching the surface — there’s an ever-expanding feature set for those interested in robotics, digital twins, extended reality and game design,” Samar said. “What I find most assuring is the sheer speed of the platform’s development — constant updates and new features are being added at a rapid pace.”

Join In on the Creation

Creators and developers across the world can download NVIDIA Omniverse for free, and enterprise teams can use the platform for their 3D projects.

Check out artwork from other “Omnivores” and submit projects in the gallery. Connect your workflows to Omniverse with software from Adobe, Autodesk, Epic Games, Maxon, Reallusion and more.

Follow NVIDIA Omniverse on Instagram, Medium, Twitter and YouTube for additional resources and inspiration. Check out the Omniverse forums, and join our Discord server and Twitch channel to chat with the community.

Artificial intelligence is teaming up with crowdsourcing to improve the thermo-stability – the ability to avoid breaking down under heat stress –  of mRNA vaccines, making distribution more accessible worldwide.

In this episode of NVIDIA’s AI Podcast, host Noah Kravitz interviewed Bojan Tunguz, a physicist and senior system software engineer at NVIDIA, and Johnny Israeli, senior manager of AI and cloud software at NVIDIA.

The guests delved into AI’s potential in drug discovery and the Stanford Open Vaccine competition, a machine-learning contest using crowdsourcing to tackle the thermo-stability challenges of mRNA vaccines.

Kaggle, the online machine learning competition platform, hosted the Stanford Open Vaccine competition. Tunguz, a quadruple Kaggle grandmaster, shared how Kaggle has grown to encompass not just competitions, but also datasets, code and discussions. Competitors can earn points, rankings and status achievements across these four areas.

The fusion of AI, crowdsourcing and machine learning competitions is opening new possibilities in drug discovery and vaccine distribution. By tapping into the collective wisdom and skills of participants worldwide, it becomes possible to solve pressing global problems, such as enhancing the thermo-stability of mRNA vaccines, allowing for a more efficient and widely accessible distribution process.

You Might Also Like

Driver’s Ed: How Waabi Uses AI, Simulation to Teach Autonomous Vehicles to Drive

Teaching the AI brains of autonomous vehicles to understand the world as humans do requires billions of miles of driving experience. The road to achieving this astronomical level of driving leads to the virtual world. Learn how Waabi uses powerful high-fidelity simulations to train and develop production-level autonomous vehicles.

Polestar’s Dennis Nobelius on the Sustainable Performance Brand’s Plans

Driving enjoyment and autonomous driving capabilities can complement one another in intelligent, sustainable vehicles. Learn about the automaker’s plans to unveil its third vehicle, the Polestar 3, the tech inside it, and what the company’s racing heritage brings to the intersection of smarts and sustainability.

GANTheftAuto: Harrison Kinsley on AI-Generated Gaming Environments

Humans playing games against machines is nothing new, but now computers can develop their own games for people to play. Programming enthusiast and social media influencer Harrison Kinsley created GANTheftAuto, an AI-based neural network that generates a playable chunk of the classic video game Grand Theft Auto V.

Subscribe to the AI Podcast: Now Available on Amazon Music

The AI Podcast is now available through Amazon Music.

In addition, get the AI Podcast through iTunes, Google Podcasts, Google Play, Castbox, DoggCatcher, Overcast, PlayerFM, Pocket Casts, Podbay, PodBean, PodCruncher, PodKicker, Soundcloud, Spotify, Stitcher and TuneIn.

Make the AI Podcast better: Have a few minutes to spare? Fill out this listener survey.

Editor’s note: This post is part of our weekly In the NVIDIA Studio series, which celebrates featured artists, offers creative tips and tricks, and demonstrates how NVIDIA Studio technology improves creative workflows.

3D artist Milan Dey finds inspiration in games, movies, comics and pop culture. He drew from all of the above when creating a stunning 3D scene of Mayan ruins, The Hidden Temple of Itzamná, this week In the NVIDIA Studio.

“One evening, I was playing an adventure game and wanted to replicate the scene,” Milan said. “But I wanted my version to have a heavy Mayan influence.”

Milan sought vast, detailed architecture and carved rocks that look like they’ve stood with pride for centuries, similar to what can be seen in the Indiana Jones movies. The artist’s goals for his scene were to portray mother nature giving humanity a reminder that she is the greatest, to kick off with a grand introduction shot with light falling directly on the camera lens to create negative spaces in the frame, and to evoke that wild, wet smell of greens.

Below, Milan outlines his creative workflow, which combines tenacity with technical ability.

And for more inspiration, check out the NVIDIA Studio #GameArtChallenge reel, which includes highlights from our video-game-themed #GameArtChallenge entries.

It Belongs in a Museum

First things first, Milan gathers reference material. For this scene, the artist spent an afternoon capturing hundreds of screenshots and walkthrough videos of the game. He spent the next day on Artstation and Adobe Behance gathering visuals and sorting out projects of ruins.

Next, Milan browsed the Epic Games marketplace, which offers an extensive collection of assets for Unreal Engine creators.

“It crossed my mind that Aztec and Inca cultures are a great choice for a ruins environment,” said Milan. “Tropical settings have a variety of vegetation, whereas caves are deep enough to create their own biology and ecosystem.” With the assets in place, Milan organized them by level to create a 3D palette.

He then began with the initial blockout to prototype, test and adjust the foundational scene elements in Unreal Engine. The artist tested scene basics, replacing blocks with polished assets and applying lighting. He didn’t add anything fancy yet — just a single source of light to mimic normal daylight.

Milan then searched for the best possible cave rocks and rock walls, with Quixel Megascans delivering the goods. Milan revisited the blocking process with the temple courtyard, placing cameras in multiple positions after initial asset placements. Next came the heavy task of adding vegetation and greens to the stone walls.

“I put big patches of moss decals all around the walls, which gives a realistic look and feel,” Milan said. “Placing large- and medium-sized trees filled in a substantial part of the environment without using many resources.”

As they say, the devil is in the details, Milan said.

“It’s very easy to get carried away with foliage painting and get lost in the depths of the cave,” the artist added. It took him another three days to fill in the smaller vegetation: shrubs, vines, plants, grass and even more moss.

The scene was starting to become staggeringly large, Milan said, but his ASUS ROG Strix Scar 15 NVIDIA Studio laptop was up to the task. His GeForce RTX 3080 GPU enabled RTX-accelerated rendering for high-fidelity, interactive visualization of his large 3D environment.

NVIDIA DLSS technology increased interactivity of the viewport by using AI to upscale frames rendered at lower resolution while retaining photorealistic detail.

“It’s simple: NVIDIA nailed ray tracing.” Milan said. “And Unreal Engine works best with NVIDIA and GeForce RTX graphics cards.”

Milan lit his scene with the HDRI digital image format to enhance the visuals and save file space, adding select directional lighting with exponential height fog. This created more density in low places of the map and less density in high places, adding further realism and depth.

The artist doesn’t take these leaps in technology for granted, he said. “I’m from an era where we were required to do manual bouncing,” Dey said. “It’s obsolete now and Lumen is incredible.”

Lumen is Unreal Engine 5’s fully dynamic global illumination and reflections system that brings realistic lighting to scenes.

Milan reviewed each camera angle and made custom lighting adjustments, sometimes removing or replacing vegetation to make them pop with the lighting. He also added free assets from Sketchfab and special water effects to give the fountain an “eternity” vibe, he said.

With the scene complete, Milan quickly exported final renders thanks to his RTX GPU. “Art is the expression of human beings,” he stressed. “It demands understanding and attention.

To his past self or someone at the beginning of their creative journey, Milan would advise, “Keep an open mind and be teachable.”

Check out Milan’s portfolio on Instagram.

Follow NVIDIA Studio on Instagram, Twitter and Facebook. Access tutorials on the Studio YouTube channel and get updates directly in your inbox by subscribing to the Studio newsletter.

Kris Kersey is an embedded software developer with over 20 years of experience, an educational YouTuber with 30,000+ subscribers, and a lifelong lover of comics and cosplay.

These interests and expertise came together in his first-ever project using the NVIDIA Jetson platform for edge AI and robotics when he created a fully functional superhero helmet as portrayed in one of his favorite Marvel Comic films, Iron Man.

The 3D-printed helmet comes complete with computer-vision capabilities in a heads-up display (HUD) that presents information wherever the user’s looking, just like in the movie.

The NVIDIA Jetson platform processes data from two cameras — one by each eye slot — that see what the helmet’s wearer is seeing. The HUD then presents information including the current temperature, humidity, altitude and GPS location. It can also classify what’s in the user’s view based on deep neural networks for object detection.

To let others join in on the fun, Kersey shared his entire workflow on his popular YouTube channel, Kersey Fabrications.

Superhero films and science fiction remind Kersey that cutting-edge technology requires collaboration across disciplines, he said.

“Often, as with this project, artists and storytellers use their imaginations to come up with brilliant ideas — then, it’s up to scientists and engineers to make them real,” the developer said.

About the Maker

Kersey, who studied computer science at Southern Polytechnic State University — now part of Kennesaw State University — in Georgia, has broad experience working with embedded microprocessors and architectures. He specializes in the Linux operating system, which is compatible with the NVIDIA Jetson platform.

“Writing software on the Jetson platform didn’t require that I learn a new programming language or operating system, which made it very easy for me,” the maker said.

By day, he’s a software engineer at an Atlanta-based startup. By night, he’s working on projects in his personal makerspace.

“I’ve never used my garage for cars,” he said.

Instead, it’s full of tools, boards and other equipment that enable his marvelous projects.

Kersey emphasized that what’s important to him most of all, however, is his family, with whom he likes to play board games, watch movies and go on hikes.

His Inspiration

Kersey’s fascination with technology stemmed from his childhood. His mother was a teacher focused on computer-aided drafting and mechanical design.

“From a very early age, I could tinker with computers that she had access to, which always fascinated me,” he said. “My cousin also once gave me an old 8-bit computer, but there wasn’t much I could do with it, so I remember pulling out the manual and reading the whole thing — that taught me basic programming.”

More recently, Kersey got into 3D printing while helping his son with a project for Science Olympiad.

“From that moment on, I got really into 3D printing as a hobby — my son never really took to it a whole lot,” he mused.

In 2018, Kersey created his YouTube channel with a focus on 3D printing as a way to delve deeper into the maker community while teaching others what he’s learned along the way.

A Jetson-Powered Superhero Project

Kersey’s 3D-printed, fully functional, wireless Iron Man helmet — which he even sanded and painted himself — could be straight out of the iconic films.

The prototype used the NVIDIA Jetson Xavier NX developer kit as the core powering its HUD.

“For this whole experience to feel as awesome as Iron Man’s tech, it has to be real time, low latency, high resolution and high frame rate,” Kersey said. “It also needs to display a lot of information on screen, which requires a powerful graphics processor — that’s why I chose the Jetson platform.”

Jetson developer kits are equipped with a powerful, onboard NVIDIA GPU and AI capabilities to supercharge embedded applications.

Kersey also tapped the NVIDIA TensorRT software development kit to enable high-performance deep-learning inference with low latency and high throughput for the project.

For the next generation of the helmet’s HUD — a project that’s “not finished till it’s finished,” according to the maker — Kersey used the NVIDIA Jetson Orin Nano developer kit. Launched in September, the kit has set a new standard for creating entry-level AI-powered robots, intelligent cameras and more.

It only took Kersey two hours to get from opening the Orin Nano box to having the software deployed and running, he said.

He’s now looking to upgrade the project with the Jetson Orin NX 16GB system-on-module, as well as build a full suit beyond the headgear, starting with prototype aluminum repulsors.

And the developer will soon make the project’s code open source, so others can easily turn themselves into superheroes, too.

Kersey plans to wear the upgraded superhero gear at Dragon Con — the world’s largest multimedia, popular culture convention — taking place in August. Plus, at this month’s MomoCon in Atlanta, he’ll present on a panel titled Making It Real: High Tech in Cosplay.

Asked if Iron Man is his favorite superhero, Kersey said with a smile: “He is right now.”

Check out Kersey Fabrications on YouTube and learn more about the NVIDIA Jetson platform.

What has it got in its pocketses? More games coming in May, that’s what.

GFN Thursday gets the summer started early with two newly supported games this week and 16 more coming later this month — including The Lord of the Rings: Gollum.

Don’t forget to take advantage of the limited-time discount on six-month Priority memberships. Priority members get faster access to cloud gaming servers, as well as support for RTX ON in supported games — all for 40% off the normal price. But hurry, this offer ends Sunday, May 21.

And the fun in May won’t stop there.

Stay tuned for more news on Xbox games joining the GeForce NOW library soon.

How Precious

No need to be sneaky about it — The Lord of the Rings: Gollum from Daedalic Entertainment comes to GeForce NOW when it releases on Thursday, May 25.

The action-adventure game and epic interactive experience takes place in parallel to the events described in The Fellowship of the Ring. Play as the enigmatic Gollum on his perilous journey and find out how he outwitted the most powerful characters in Middle-earth.

Climb the mountains of Mordor, sneak around Mirkwood and make difficult choices. Who will gain the upper hand: the cunning Gollum or the innocent Smeagol? Priority and Ultimate members can experience the epic story with support for RTX ray tracing and DLSS technology for AI-powered high-quality graphics, streaming across nearly any device with up to eight-hour sessions. Go Ultimate today with the one cloud gaming membership that rules them all.

May-Day Game-Day

It’s gonna be May, and that means more of the best games joining the GeForce NOW library.

Age of Wonders 4 is the long-awaited sequel from Paradox Interactive. A blend of 4x strategy and turn-based combat, members can explore new magical realms and rule over a faction of their design that grows with expanding empires. Battle through each chapter and guide your empire to greatness.

It leads two new games joining the cloud this week:

• Age of Wonders 4 (New release on Steam)
• Showgunners (New release on Steam)

Then check out the rest of the titles on their way in May:

• Occupy Mars: The Game (New release on Steam, May 10)
• TT Isle of Man: Ride on the Edge 3 (New release on Steam, May 11)
• Far Cry 6 (New release on Steam, May 11)
• Tin Hearts (New release on Steam, May 16)
• The Outlast Trials (New release on Steam, May 18)
• Warhammer 40,000: Boltgun (New release on Steam, May 23)
• Blooming Business: Casino (New release on Steam, May 23)
• Railway Empire 2 (New release on Steam, May 25)
• The Lord of the Rings: Gollum (New release on Steam, May 25)
• Above Snakes (New release on Steam, May 25)
• System Shock (New release on Steam, May 30)
• Patch Quest (Steam)
• The Ascent (Steam)
• Lawn Mowing Simulator (Steam)
• Conqueror’s Blade (Steam)

April Additions

There were 23 announced games in April, plus another eight that joined the GeForce NOW library of over 1,600 games:

• F1 2021 (Steam)
• Farlight 84 (Steam)
• MORDHAU (Epic Games Store)
• Resident Evil 2 (Steam)
• Resident Evil 3 (Steam)
• Resident Evil 7 Biohazard (Steam)
• Shadows of Doubt (New release on Steam)
• The Swordsmen X: Survival (New release on Steam)

Poker Club unfortunately couldn’t be added in April due to technical issues. Tin Hearts also didn’t make it in April, but is included in the May list due to a shift in its release date.

With so many titles streaming from the cloud, what device will you be streaming on? Let us know in the comments below, or on Twitter or Facebook.

one does not simply game in the cloud with only 1 device

— NVIDIA GeForce NOW (@NVIDIAGFN) May 3, 2023

AV1, the next-generation video codec, is expanding its reach with today’s release of OBS Studio 29.1. This latest software update adds support for AV1 streaming to YouTube over Enhanced RTMP.

All GeForce RTX 40 Series GPUs — including laptop GPUs and the recently launched GeForce RTX 4070 — support real-time AV1 hardware encoding, providing 40% more efficient encoding on average than H.264 and delivering higher quality than competing GPUs.

This reduces the upload bandwidth needed to stream, a common limitation from streaming services and internet service providers. At higher resolutions, AV1 encoding is even more efficient. For example, AV1 enables streaming 4K at 60 frames per second with 10 Mbps upload bandwidth — down from 20 Mbps with H.264 — making 4K60 streaming available to a wider audience.

AV1 — The New Standard

As a founding member of the Alliance for Open Media, NVIDIA has worked closely with industry titans in developing the AV1 codec. This work was necessitated by gamers and online content creators who pushed the boundaries of old formats that were defined roughly 20 years ago. The previous standard for livestreaming, H.264, usually maxed out with 1080p at 60 fps at the commonly used bitrates of 6-8 Mbps, and often produced blocky, grainy images.

AV1’s increased efficiency enables streaming higher-quality images, allowing creators to stream at higher resolutions with smoother frame rates. Even in network-limited environments, streamers can now reap the benefits of high-quality video shared with their audience.

Support for AV1 on YouTube comes through the recent update to RTMP. The enhanced protocol also adds support for HEVC streaming, bringing new formats to users on the existing low-latency protocol they use for H.264 streaming. Enhanced RTMP ingestion has been released as a beta feature on YouTube.

Learn how to configure OBS Studio for streaming AV1 with GeForce RTX 40 Series GPUs in the OBS setup guide.

Better Streams With NVENC, NVIDIA Broadcast

GeForce RTX 40 Series GPUs usher in a new era of high-quality streaming with AV1 encoding support on the eighth-generation NVENC. A boon to streamers, NVENC offloads compute-intensive encoding tasks from the CPU to dedicated hardware on the GPU.

Designed to support the rigors of professional content creators, NVENC preserves video quality with a higher accuracy than competitive encoders. GeForce RTX users can stream higher-quality images at the same bitrate as competitive products or encode at a lower bitrate while maintaining a similar picture quality.

NVIDIA Broadcast, part of the exclusive NVIDIA Studio suite of software, transforms any room into a home studio. Livestreams, voice chats and video calls look and sound better with powerful AI effects like eye contact, noise and room echo removal, virtual background and more.

Follow NVIDIA Studio on Instagram, Twitter and Facebook. Access tutorials on the Studio YouTube channel and get updates directly in your inbox by subscribing to the Studio newsletter.

¹ Source: 4K60 AV1 encoded video with AMD 7900 XT, GeForce RTX 4080 and Intel Arc 770 with OBS Studio default settings at 12Mbps

NVIDIA today introduced a wave of cutting-edge AI research that will enable developers and artists to bring their ideas to life — whether still or moving, in 2D or 3D, hyperrealistic or fantastical.

Around 20 NVIDIA Research papers advancing generative AI and neural graphics — including collaborations with over a dozen universities in the U.S., Europe and Israel — are headed to SIGGRAPH 2023, the premier computer graphics conference, taking place Aug. 6-10 in Los Angeles.

The papers include generative AI models that turn text into personalized images; inverse rendering tools that transform still images into 3D objects; neural physics models that use AI to simulate complex 3D elements with stunning realism; and neural rendering models that unlock new capabilities for generating real-time, AI-powered visual details.

Innovations by NVIDIA researchers are regularly shared with developers on GitHub and incorporated into products, including the NVIDIA Omniverse platform for building and operating metaverse applications and NVIDIA Picasso, a recently announced foundry for custom generative AI models for visual design. Years of NVIDIA graphics research helped bring film-style rendering to games, like the recently released Cyberpunk 2077 Ray Tracing: Overdrive Mode, the world’s first path-traced AAA title.

The research advancements presented this year at SIGGRAPH will help developers and enterprises rapidly generate synthetic data to populate virtual worlds for robotics and autonomous vehicle training. They’ll also enable creators in art, architecture, graphic design, game development and film to more quickly produce high-quality visuals for storyboarding, previsualization and even production.

AI With a Personal Touch: Customized Text-to-Image Models

Generative AI models that transform text into images are powerful tools to create concept art or storyboards for films, video games and 3D virtual worlds. Text-to-image AI tools can turn a prompt like “children’s toys” into nearly infinite visuals a creator can use for inspiration — generating images of stuffed animals, blocks or puzzles.

However, artists may have a particular subject in mind. A creative director for a toy brand, for example, could be planning an ad campaign around a new teddy bear and want to visualize the toy in different situations, such as a teddy bear tea party. To enable this level of specificity in the output of a generative AI model, researchers from Tel Aviv University and NVIDIA have two SIGGRAPH papers that enable users to provide image examples that the model quickly learns from.

One paper describes a technique that needs a single example image to customize its output, accelerating the personalization process from minutes to roughly 11 seconds on a single NVIDIA A100 Tensor Core GPU, more than 60x faster than previous personalization approaches.

A second paper introduces a highly compact model called Perfusion, which takes a handful of concept images to allow users to combine multiple personalized elements — such as a specific teddy bear and teapot — into a single AI-generated visual:

Serving in 3D: Advances in Inverse Rendering and Character Creation 

Once a creator comes up with concept art for a virtual world, the next step is to render the environment and populate it with 3D objects and characters. NVIDIA Research is inventing AI techniques to accelerate this time-consuming process by automatically transforming 2D images and videos into 3D representations that creators can import into graphics applications for further editing.

A third paper created with researchers at the University of California, San Diego, discusses tech that can generate and render a photorealistic 3D head-and-shoulders model based on a single 2D portrait — a major breakthrough that makes 3D avatar creation and 3D video conferencing accessible with AI. The method runs in real time on a consumer desktop, and can generate a photorealistic or stylized 3D telepresence using only conventional webcams or smartphone cameras.

A fourth project, a collaboration with Stanford University, brings lifelike motion to 3D characters. The researchers created an AI system that can learn a range of tennis skills from 2D video recordings of real tennis matches and apply this motion to 3D characters. The simulated tennis players can accurately hit the ball to target positions on a virtual court, and even play extended rallies with other characters.

Beyond the test case of tennis, this SIGGRAPH paper addresses the difficult challenge of producing 3D characters that can perform diverse skills with realistic movement — without the use of expensive motion-capture data.

Not a Hair Out of Place: Neural Physics Enables Realistic Simulations

Once a 3D character is generated, artists can layer in realistic details such as hair — a complex, computationally expensive challenge for animators.

Humans have an average of 100,000 hairs on their heads, with each reacting dynamically to an individual’s motion and the surrounding environment. Traditionally, creators have used physics formulas to calculate hair movement, simplifying or approximating its motion based on the resources available. That’s why virtual characters in a big-budget film sport much more detailed heads of hair than real-time video game avatars.

A fifth paper showcases a method that can simulate tens of thousands of hairs in high resolution and in real time using neural physics, an AI technique that teaches a neural network to predict how an object would move in the real world.

The team’s novel approach for accurate simulation of full-scale hair is specifically optimized for modern GPUs. It offers significant performance leaps compared to state-of-the-art, CPU-based solvers, reducing simulation times from multiple days to merely hours — while also boosting the quality of hair simulations possible in real time. This technique finally enables both accurate and interactive physically based hair grooming.

Neural Rendering Brings Film-Quality Detail to Real-Time Graphics 

After an environment is filled with animated 3D objects and characters, real-time rendering simulates the physics of light reflecting through the virtual scene. Recent NVIDIA research shows how AI models for textures, materials and volumes can deliver film-quality, photorealistic visuals in real time for video games and digital twins.

NVIDIA invented programmable shading over two decades ago, enabling developers to customize the graphics pipeline. In these latest neural rendering inventions, researchers extend programmable shading code with AI models that run deep inside NVIDIA’s real-time graphics pipelines.

In a sixth SIGGRAPH paper, NVIDIA will present neural texture compression that delivers up to 16x more texture detail without taking additional GPU memory. Neural texture compression can substantially increase the realism of 3D scenes, as seen in the image below, which demonstrates how neural-compressed textures (right) capture sharper detail than previous formats, where the text remains blurry (center).

A related paper announced last year is now available in early access as NeuralVDB, an AI-enabled data compression technique that decreases by 100x the memory needed to represent volumetric data — like smoke, fire, clouds and water.

NVIDIA also released today more details about neural materials research that was shown in the most recent NVIDIA GTC keynote. The paper describes an AI system that learns how light reflects from photoreal, many-layered materials, reducing the complexity of these assets down to small neural networks that run in real time, enabling up to 10x faster shading.

The level of realism can be seen in this neural-rendered teapot, which accurately represents the ceramic, the imperfect clear-coat glaze, fingerprints, smudges and even dust.

More Generative AI and Graphics Research

These are just the highlights — read more about all the NVIDIA papers at SIGGRAPH. NVIDIA will also present six courses, four talks and two Emerging Technology demos at the conference, with topics including path tracing, telepresence and diffusion models for generative AI.

NVIDIA Research has hundreds of scientists and engineers worldwide, with teams focused on topics including AI, computer graphics, computer vision, self-driving cars and robotics.