Take advantage of advanced deployment strategies using Amazon SageMaker deployment guardrails

Deployment guardrails in Amazon SageMaker provide a new set of deployment capabilities allowing you to implement advanced deployment strategies that minimize risk when deploying new model versions on SageMaker hosting. Depending on your use case, you can use a variety of deployment strategies to release new model versions. Each of these strategies relies on a mechanism to shift inference traffic to one or more versions of a deployed model. The chosen strategy depends on your business requirements for your machine learning (ML) use case. However, any strategy should include the ability to monitor the performance of new model versions and automatically roll back to a previous version as needed to minimize potential risk of introducing a new model version with errors. Deployment guardrails offer new advanced deployment capabilities and as of this writing supports two new traffic shifting policies, canary and linear, as well as the ability to automatically roll back when issues are detected.

As part of your MLOps strategy to create repeatable and reliable mechanisms to deploy your models, you should also ensure that the chosen deployment strategy is implemented as part of your automated deployment pipeline. Deployment guardrails use the existing SageMaker CreateEndpoint and UpdateEndpoint APIs, so you can modify your existing deployment pipeline configurations to take advantage of the new deployment capabilities.

In this post, we show you how to use the new deployment guardrail capabilities to deploy your model versions using both a canary and linear deployment strategy.

Solution overview

Amazon SageMaker inference provides managed deployment strategies for testing new versions of your models in production. We cover two new traffic shifting policies in this post: canary and linear. For each of these traffic shifting modes, two HTTPS endpoints are provisioned. Two endpoints are provisioned to reduce deployment risk as traffic is shifted from the original endpoint variant to the new endpoint variant. You configure the endpoints to contain one or more compute instances to deploy your trained model and perform inference requests. SageMaker manages the routing of traffic between the two endpoints. You define Amazon CloudWatch metrics and alarms to monitor metrics on the new endpoint, when traffic is shifted, for a set baking period. If a CloudWatch alarm is triggered, SageMaker performs an auto-rollback to route all traffic to the original endpoint variant. If no CloudWatch alarms are triggered, the original endpoint variant is stopped and the new endpoint variant continues to receive all traffic. The following diagrams illustrate shifting traffic to the new endpoint.

Let’s dive deeper into examples of the canary and linear traffic shifting policies.

We go over the following high-level steps as part of the deployment procedure:

  1. Create the model and endpoint configurations required for the three scenarios: the baseline, the update containing the incompatible model version, and the update with the correct model version.
  2. Invoke the baseline endpoint prior to the update.
  3. Specify the CloudWatch alarms used to trigger the rollbacks.
  4. Update the endpoint to trigger a rollback using either the canary or linear strategy.

First, let’s start with canary deployment.

Canary deployment

The canary deployment option lets you shift one small portion of your traffic (a canary) to the green fleet and monitor it for a baking period. If the canary succeeds on the green fleet, the rest of the traffic is shifted from the blue fleet to the green fleet before stopping the blue fleet.

To demonstrate canary deployments and the auto-rollback feature, we update an endpoint with an incompatible model version and deploy it as a canary fleet, taking a small percentage of the traffic. Requests sent to this canary fleet result in errors, which trigger a rollback using preconfigured CloudWatch alarms. We also demonstrate a success scenario where no alarms are tripped and the update succeeds.

Create and deploy the models

First, we upload our pre-trained models to Amazon Simple Storage Service (Amazon S3). These models were trained using the XGBoost churn prediction notebook in SageMaker. You can also use your own pre-trained models in this step. If you already have a pre-trained model in Amazon S3, you can add it by specifying the s3_key.

The models in this example are used to predict the probability of a mobile customer leaving their current mobile operator. The dataset we use is publicly available and was mentioned in the book Discovering Knowledge in Data by Daniel T. Larose.

Upload the models with the following code:

model_url = S3Uploader.upload(local_path="model/xgb-churn-prediction-model.tar.gz",
desired_s3_uri=f"s3://{bucket}/{prefix}")
model_url2 = S3Uploader.upload(local_path="model/xgb-churn-prediction-model2.tar.gz",
desired_s3_uri=f"s3://{bucket}/{prefix}")

Next, we create our model definitions. We start with deploying the pre-trained churn prediction models. Here, we create the model objects with the image and model data. The three URIs correspond to the baseline version, the update containing the incompatible version, and the update containing the correct model version:

image_uri = image_uris.retrieve('xgboost', boto3.Session().region_name, '0.90-1')
# using newer version of XGBoost which is incompatible, in order to simulate model faults

image_uri2 = image_uris.retrieve('xgboost', boto3.Session().region_name, '1.2-1')
image_uri3 = image_uris.retrieve('xgboost', boto3.Session().region_name, '0.90-2')
model_name = f"DEMO-xgb-churn-pred-{datetime.now():%Y-%m-%d-%H-%M-%S}" 
model_name2 = f"DEMO-xgb-churn-pred2-{datetime.now():%Y-%m-%d-%H-%M-%S}"
model_name3 = f"DEMO-xgb-churn-pred3-{datetime.now():%Y-%m-%d-%H-%M-%S}"

resp = sm.create_model(
    ModelName=model_name,
    ExecutionRoleArn=role,
    Containers=[{
       'Image': image_uri,
       'ModelDataUrl': model_url
     }])

resp = sm.create_model(
    ModelName=model_name2,
    ExecutionRoleArn=role,
    Containers=[{
       'Image':image_uri2,
       'ModelDataUrl': model_url2
     }])

resp = sm.create_model(
    ModelName=model_name3,
    ExecutionRoleArn=role,
    Containers=[{
       'Image':image_uri3,
       'ModelDataUrl': model_url2
     }])

Now that the three models are created, we create the three endpoint configs:

ep_config_name = f"DEMO-EpConfig-1-{datetime.now():%Y-%m-%d-%H-%M-%S}" 
ep_config_name2 = f"DEMO-EpConfig-2-{datetime.now():%Y-%m-%d-%H-%M-%S}" 
ep_config_name3 = f"DEMO-EpConfig-3-{datetime.now():%Y-%m-%d-%H-%M-%S}"

resp = sm.create_endpoint_config(
     EndpointConfigName=ep_config_name,
     ProductionVariants=[
        {
          'VariantName': "AllTraffic",
          'ModelName': model_name,
          'InstanceType': "ml.m5.xlarge",
          "InitialInstanceCount": 3
        }
      ])

resp = sm.create_endpoint_config(
     EndpointConfigName=ep_config_name2,
     ProductionVariants=[
        {
          'VariantName': "AllTraffic",
          'ModelName': model_name2,
          'InstanceType': "ml.m5.xlarge",
          "InitialInstanceCount": 3
        }
      ])


resp = sm.create_endpoint_config(
      EndpointConfigName=ep_config_name3,
      ProductionVariants=[
         {
           'VariantName': "AllTraffic",
           'ModelName': model_name3,
           'InstanceType': "ml.m5.xlarge",
           "InitialInstanceCount": 3
         }
     ])

We then deploy the baseline model to a SageMaker endpoint:

resp = sm.create_endpoint(
          EndpointName=endpoint_name,
          EndpointConfigName=ep_config_name
)

Invoke the endpoint

This step invokes the endpoint with sample data with a maximum invocations count and waiting intervals. See the following code:

def invoke_endpoint(endpoint_name, max_invocations=300, wait_interval_sec=1, should_raise_exp=False):
    print(f"Sending test traffic to the endpoint {endpoint_name}. nPlease wait...")
 
    count = 0
    with open('test_data/test-dataset-input-cols.csv', 'r') as f:
        for row in f:
            payload = row.rstrip('n')
            try:
                response = sm_runtime.invoke_endpoint(EndpointName=endpoint_name,
                                                      ContentType='text/csv', 
                                                      Body=payload)
                response['Body'].read()
                print(".", end="", flush=True)
            except Exception as e:
                print("E", end="", flush=True)
                if should_raise_exp:
                    raise e
            count += 1
            if count > max_invocations:
                break
            time.sleep(wait_interval_sec)
 
    print("nDone!")
 
invoke_endpoint(endpoint_name, max_invocations=100)

For a full list of metrics, see Monitor Amazon SageMaker with Amazon CloudWatch.

Then we plot graphs to show the metrics Invocations, Invocation4XXErrors, Invocation5XXErrors, ModelLatency, and OverheadLatency against the endpoint over time.

You can observe a flat line for Invocation4XXErrors and Invocation5XXErrors because we’re using the correct version model version and configs. Additionally, ModelLatency and OverheadLatency start decreasing over time.

Create CloudWatch alarms to monitor endpoint performance

We create CloudWatch alarms to monitor endpoint performance with the metrics Invocation5XXErrors and ModelLatency.

We use metric dimensions EndpointName and VariantName to select the metric for each endpoint config and variant. See the following code:

def create_auto_rollback_alarm(alarm_name, endpoint_name, variant_name, metric_name, statistic, threshold):
cw.put_metric_alarm(
AlarmName=alarm_name,
AlarmDescription='Test SageMaker endpoint deployment auto-rollback alarm',
ActionsEnabled=False,
Namespace='AWS/SageMaker',
MetricName=metric_name,
Statistic=statistic,
Dimensions=[
{
'Name': 'EndpointName',
'Value': endpoint_name
},
{
'Name': 'VariantName',
'Value': variant_name
}
],
Period=60,
EvaluationPeriods=1,
Threshold=threshold,
ComparisonOperator='GreaterThanOrEqualToThreshold',
TreatMissingData='notBreaching'
)

# alarm on 1% 5xx error rate for 1 minute
create_auto_rollback_alarm(error_alarm, endpoint_name, 'AllTraffic', 'Invocation5XXErrors', 'Average', 1)
# alarm on model latency >= 10 ms for 1 minute
create_auto_rollback_alarm(latency_alarm, endpoint_name, 'AllTraffic', 'ModelLatency', 'Average', 10000)

Update the endpoint with deployment configurations

We define the following deployment configuration to perform a blue/green update strategy with canary traffic shifting from the old to the new stack. The canary traffic shifting option can reduce the blast ratio of a regressive update to the endpoint. In contrast, for the all-at-once traffic shifting option, the invocation requests start faulting at 100% after flipping the traffic. In canary mode, invocation requests are shifted to the new version of model gradually, preventing errors from impacting 100% of the traffic. Additionally, the auto-rollback alarms monitor the metrics during the canary stage.

The following diagram illustrates the workflow of our rollback use case.

We update the endpoint with an incompatible model version to simulate errors and trigger a rollback:

canary_deployment_config = {
    "BlueGreenUpdatePolicy": {
        "TrafficRoutingConfiguration": {
            "Type": "CANARY",
            "CanarySize": {
                "Type": "INSTANCE_COUNT", # or use "CAPACITY_PERCENT" as 30%, 50%
                "Value": 1
            },
            "WaitIntervalInSeconds": 300, # wait for 5 minutes before enabling traffic on the rest of fleet
        },
        "TerminationWaitInSeconds": 120, # wait for 2 minutes before terminating the old stack
        "MaximumExecutionTimeoutInSeconds": 1800 # maximum timeout for deployment
    },
    "AutoRollbackConfiguration": {
        "Alarms": [
            {
                "AlarmName": error_alarm
            },
            {
                "AlarmName": latency_alarm
            }
        ],
    }
}
 
# update endpoint request with new DeploymentConfig parameter
sm.update_endpoint(
    EndpointName=endpoint_name,
    EndpointConfigName=ep_config_name2,
    DeploymentConfig=canary_deployment_config
)

When we invoke the endpoint, we encounter errors because of the incompatible version of the model (ep_config_name2), and this leads to the rollback to a stable version of the model (ep_config_name1). This is reflected in the following graphs as Invocation5XXErrors and ModelLatency increase during this rollback phase.

The following diagram shows a success case where we use the same canary deployment configuration but a valid endpoint configuration.

We update the endpoint configuration to a valid version (using the same canary deployment config as the rollback case):

# update endpoint with a valid version of DeploymentConfig
sm.update_endpoint(
    EndpointName=endpoint_name,
    EndpointConfigName=ep_config_name3,
    RetainDeploymentConfig=True
)

We plot graphs to show the Invocations, Invocation5XXErrors, and ModelLatency metrics against the endpoint. When the new endpoint config-3 (correct model version) starts getting deployed, it takes over endpoint config-2 (incompatible due to model version) without any errors. We can see this in the graphs as Invocation5XXErrors and ModelLatency decrease during this transition phase.

Next, let’s see how linear deployments are configured and how it works.

Linear deployment

The linear deployment option provides even more customization over how many traffic-shifting steps to make and what percentage of traffic to shift for each step. Whereas canary shifting lets you shift traffic in two steps, linear shifting extends this to n linearly spaced steps.

To demonstrate linear deployments and the auto-rollback feature, we update an endpoint with an incompatible model version and deploy it as a linear fleet, taking a small percentage of the traffic. Requests sent to this linear fleet result in errors, which triggers a rollback using preconfigured CloudWatch alarms. We also demonstrate a success scenario where no alarms are tripped and the update succeeds.

The steps to create the models, invoke the endpoint, and create the CloudWatch alarms are the same as with the canary method.

We define the following deployment configuration to perform a blue/green update strategy with linear traffic shifting from old to new stack. The linear traffic shifting option can reduce the blast ratio of a regressive update to the endpoint. In contrast, for the all-at-once traffic shifting option, the invocation requests start faulting at 100% after flipping the traffic. In linear mode, invocation requests are shifted to the new version of the model gradually, with a controlled percentage of traffic shifting for each step. You can use the auto-rollback alarms to monitor the metrics during the linear traffic shifting stage.

The following diagram shows the workflow for our linear rollback case.

We update the endpoint with an incompatible model version to simulate errors and trigger a rollback:

linear_deployment_config = {
    "BlueGreenUpdatePolicy": {
        "TrafficRoutingConfiguration": {
            "Type": "LINEAR",
            "LinearStepSize": {
                "Type": "CAPACITY_PERCENT",
                "Value": 33, # 33% of whole fleet capacity (33% * 3 = 1 instance)
            },
            "WaitIntervalInSeconds": 180, # wait for 3 minutes before enabling traffic on the rest of fleet
        },
        "TerminationWaitInSeconds": 120, # wait for 2 minutes before terminating the old stack
        "MaximumExecutionTimeoutInSeconds": 1800 # maximum timeout for deployment
    },
    "AutoRollbackConfiguration": {
        "Alarms": [
            {
                "AlarmName": error_alarm
            },
            {
                "AlarmName": latency_alarm
            }
        ],
    }
}
 
# update endpoint request with new DeploymentConfig parameter
sm.update_endpoint(
    EndpointName=endpoint_name,
    EndpointConfigName=ep_config_name2,
    DeploymentConfig=linear_deployment_config
)

When we invoke the endpoint, we encounter errors because of the incompatible version of the model (ep_config_name2), which leads to the rollback to a stable version of the model (ep_config_name1). We can see this in the following graphs as the Invocation5XXErrors and ModelLatency metrics increase during this rollback phase.

Let’s look at a success case where we use the same linear deployment configuration but a valid endpoint configuration. The following diagram illustrates our workflow.

We update the endpoint to a valid endpoint configuration version with the same linear deployment configuration:

# update endpoint with a valid version of DeploymentConfig
sm.update_endpoint(
    EndpointName=endpoint_name,
    EndpointConfigName=ep_config_name3,
    RetainDeploymentConfig=True
)

Then we plot graphs to show the Invocations, Invocation5XXErrors, and ModelLatency metrics against the endpoint.

As the new endpoint config-3 (correct model version) starts getting deployed, it takes over endpoint config-2 (incompatible due to model version) without any errors. We can see this in the following graphs as Invocation5XXErrors and ModelLatency decrease during this transition phase.

Considerations and best practices

Now that we’ve walked through a comprehensive example, let’s recap some best practices and considerations:

  • Pick the right health check – The CloudWatch alarms determine whether the traffic shift to the new endpoint variant succeeds. In our example, we used Invocation5XXErrors (caused by the endpoint failing to return a valid result) and ModelLatency, which measure how long the model takes to return a response. You can consider other built-in metrics in some cases, like OverheadLatency, which accounts for other causes of latency, such as unusually large response payloads. You can also have your inference code record custom metrics, and you can configure the alarm measurement evaluation interval. For more information about available metrics, see SageMaker Endpoint Invocation Metrics.
  • Pick the most suitable traffic shifting policy – The all-at-once policy is a good choice if you just want to make sure that the new endpoint variant is healthy and able to serve traffic. The canary policy is useful if you want to avoid affecting too much traffic if the new endpoint variant has a problem, or if you want to evaluate a custom metric on a small percentage of traffic before shifting over. For example, perhaps you want to emit a custom metric that checks for inference response distribution, and make sure it falls within expected ranges. The linear policy is a more conservative and more complex take on the canary pattern.
  • Monitor the alarms – The alarms you use to trigger rollback should also cause other actions, like notifying an operations team.
  • Use the same deployment strategy in multiple environments – As part of an overall MLOps pipeline, use the same deployment strategy in test as well as production environments, so that you become comfortable with the behavior. This consideration implies that you can inject realistic load onto your test endpoints.

Conclusion

In this post, we introduced SageMaker inference’s new deployment guardrail options, which let you manage deployment of a new model version in a safe and controlled way. We reviewed the new traffic shifting policies, canary and linear, and showed how to use them in a realistic example. Finally, we discussed some best practices and considerations. Get started today with deployment guardrails on the SageMaker console or, for more information, review Deployment Guardrails.

About the Authors

Raghu Ramesha is an ML Solutions Architect with the Amazon SageMaker Services SA team. He focuses on helping customers migrate ML production workloads to SageMaker at scale. He specializes in machine learning, AI, and computer vision domains, and holds a master’s degree in Computer Science from UT Dallas. In his free time, he enjoys traveling and photography.

Shelbee Eigenbrode is a Principal AI and Machine Learning Specialist Solutions Architect at Amazon Web Services (AWS). She has been in technology for 24 years spanning multiple industries, technologies, and roles. She is currently focusing on combining her DevOps and ML background into the domain of MLOps to help customers deliver and manage ML workloads at scale. With over 35 patents granted across various technology domains, she has a passion for continuous innovation and using data to drive business outcomes. Shelbee is a co-creator and instructor of the Practical Data Science specialization on Coursera. She is also the Co-Director of Women In Big Data (WiBD), Denver chapter. In her spare time, she likes to spend time with her family, friends, and overactive dogs.

Randy DeFauw is a Principal Solutions Architect. He’s an electrical engineer by training who’s been working in technology for 23 years at companies ranging from startups to large defense firms. A fascination with distributed consensus systems led him into the big data space, where he discovered a passion for analytics and machine learning. He started using AWS in his Hadoop days, where he saw how easy it was to set up large complex infrastructure, and then realized that the cloud solved some of the challenges he saw with Hadoop. Randy picked up an MBA so he could learn how business leaders think and talk, and found that the soft skill classes were some of the most interesting ones he took. Lately, he’s been dabbling with reinforcement learning as a way to tackle optimization problems, and re-reading Martin Kleppmann’s book on data intensive design.

Lauren Mullennex is a Solutions Architect based in Denver, CO. She works with customers to help them architect solutions on AWS. In her spare time, she enjoys hiking and cooking Hawaiian cuisine.

Read More

Train graph neural nets for millions of proteins on Amazon SageMaker and Amazon DocumentDB (with MongoDB compatibility)

There are over 180,000 unique proteins with 3D structures determined, with tens of thousands new structures resolved every year. This is only a small fraction of the 200 million known proteins with distinctive sequences. Recent deep learning algorithms such as AlphaFold can accurately predict 3D structures of proteins using their sequences, which help scale the protein 3D structure data to the millions. Graph neural network (GNN) has emerged as an effective deep learning approach to extract information from protein structures, which can be represented by graphs of amino acid residues. Individual protein graphs usually contain a few hundred nodes, which is manageable in size. Tens of thousands of protein graphs can be easily stored in serialized data structures such as TFrecord for training GNNs. However, training GNN on millions of protein structures is challenging. Data serialization isn’t scalable to millions of protein structures because it requires loading the entire terabyte-scale dataset into memory.

In this post, we introduce a scalable deep learning solution that allows you to train GNNs on millions of proteins stored in Amazon DocumentDB (with MongoDB compatibility) using Amazon SageMaker.

For illustrative purposes, we use publicly available experimentally determined protein structures from the Protein Data Bank and computationally predicted protein structures from the AlphaFold Protein Structure Database. The machine learning (ML) problem is to develop a discriminator GNN model to distinguish between experimental and predicted structures based on protein graphs constructed from their 3D structures.

Overview of solution

We first parse the protein structures into JSON records with multiple types of data structures, such as an n-dimensional array and nested object, to store the proteins’ atomic coordinates, properties, and identifiers. Storing a JSON record for a protein’s structure takes 45 KB on average; we project storing 100 million proteins would take around 4.2 TB. Amazon DocumentDB storage automatically scales with the data in your cluster volume in 10 GB increments, up to 64 TB. Therefore, the support for JSON data structure and scalability makes Amazon DocumentDB a natural choice.

We next build a GNN model to predict protein properties using graphs of amino acid residues constructed from their structures. The GNN model is trained using SageMaker and configured to efficiently retrieve batches of protein structures from the database.

Finally, we analyze the trained GNN model to gain some insights into the predictions.

We walk through the following steps for this tutorial:

  1. Create resources using an AWS CloudFormation template.
  2. Prepare protein structures and properties and ingest the data into Amazon DocumentDB.
  3. Train a GNN on the protein structures using SageMaker.
  4. Load and evaluate the trained GNN model.

The code and notebooks used in this post are available in the GitHub repo.

Prerequisites

For this walkthrough, you should have the following prerequisites:

Running this tutorial for an hour should cost no more than $2.00.

Create resources

We provide a CloudFormation template to create the required AWS resources for this post, with a similar architecture as in the post Analyzing data stored in Amazon DocumentDB (with MongoDB compatibility) using Amazon SageMaker. For instructions on creating a CloudFormation stack, see the video Simplify your Infrastructure Management using AWS CloudFormation.

The CloudFormation stack provisions the following:

  • A VPC with three private subnets for Amazon DocumentDB and two public subnets intended for the SageMaker notebook instance and ML training containers, respectively.
  • An Amazon DocumentDB cluster with three nodes, one in each private subnet.
  • A Secrets Manager secret to store login credentials for Amazon DocumentDB. This allows us to avoid storing plaintext credentials in our SageMaker instance.
  • A SageMaker notebook instance to prepare data, orchestrate training jobs, and run interactive analyses.

When creating the CloudFormation stack, you need to specify the following:

  • Name for your CloudFormation stack
  • Amazon DocumentDB user name and password (to be stored in Secrets Manager)
  • Amazon DocumentDB instance type (default db.r5.large)
  • SageMaker instance type (default ml.t3.xlarge)

It should take about 15 minutes to create the CloudFormation stack. The following diagram shows the resource architecture.

Prepare protein structures and properties and ingest the data into Amazon DocumentDB

All the subsequent code in this section is in the Jupyter notebook Prepare_data.ipynb in the SageMaker instance created in your CloudFormation stack.

This notebook handles the procedures required for preparing and ingesting protein structure data into Amazon DocumentDB.

  1. We first download predicted protein structures from AlphaFold DB in PDB format and the matching experimental structures from the Protein Data Bank.

For demonstration purposes, we only use proteins from the thermophilic archaean Methanocaldococcus jannaschii, which has the smallest proteome of 1,773 proteins for us to work with. You are welcome to try using proteins from other species.

  1. We connect to an Amazon DocumentDB cluster by retrieving the credentials stored in Secrets Manager:
def get_secret(stack_name):

    # Create a Secrets Manager client
    session = boto3.session.Session()
    client = session.client(
        service_name="secretsmanager",
        region_name=session.region_name
    )
    
    secret_name = f"{stack_name}-DocDBSecret"
    get_secret_value_response = client.get_secret_value(SecretId=secret_name)
    secret = get_secret_value_response["SecretString"]
    
return json.loads(secret)
	
	secrets = get_secret("gnn-proteins")
	
# connect to DocDB
	uri = "mongodb://{}:{}@{}:{}/?tls=true&tlsCAFile=rds-combined-ca-bundle.pem&replicaSet=rs0&readPreference=secondaryPreferred&retryWrites=false"
    		.format(secrets["username"], secrets["password"], secrets["host"], secrets["port"])
	
client = MongoClient(uri)

db = client["proteins"] # create a database
collection = db["proteins"] # create a collection
  1. After we set up the connection to Amazon DocumentDB, we parse the PDB files into JSON records to ingest into the database.

We provide utility functions required for parsing PDB files in pdb_parse.py. The parse_pdb_file_to_json_record function does the heavy lifting of extracting atomic coordinates from one or multiple peptide chains in a PDB file and returns one or a list of JSON documents, which can be directly ingested into the Amazon DocumentDB collection as a document. See the following code:

recs = parse_pdb_file_to_json_record(pdb_parser, pdb_file, pdb_id)
collection.insert_many(recs)

After we ingest the parsed protein data into Amazon DocumentDB, we can update the contents of the protein documents. For instance, it makes our model training logistics easier if we add a field indicating whether a protein structure should be used in the training, validation, or test sets.

  1. We first retrieve the all the documents with the field is_AF to stratify documents using an aggregation pipeline:
match = {"is_AF": {"$exists": True}}
project = {"y": "$is_AF"}

pipeline = [
    {"$match": match},
    {"$project": project},
]
# aggregation pipeline
cur = collection.aggregate(pipeline)
# retrieve documents from the DB cursor
docs = [doc for doc in cur]
# convert to a data frame:
df = pd.DataFrame(docs)
# stratified split: full -> train/test
df_train, df_test = train_test_split(
    df, 
    test_size=0.2,
    stratify=df["y"], 
    random_state=42
)
# stratified split: train -> train/valid
df_train, df_valid = train_test_split(
    df_train, 
    test_size=0.2,
    stratify=df_train["y"], 
    random_state=42
)
  1. Next, we use the update_many function to store the split information back to Amazon DocumentDB:
for split, df_split in zip(
    ["train", "valid", "test"],
    [df_train, df_valid, df_test]
):
    result = collection.update_many(
        {"_id": {"$in": df_split["_id"].tolist()}}, 
        {"$set": {"split": split}}
)
print("Number of documents modified:", result.modified_count)

Train a GNN on the protein structures using SageMaker

All the subsequent code in this section is in the Train_and_eval.ipynb notebook in the SageMaker instance created in your CloudFormation stack.

This notebook trains a GNN model on the protein structure datasets stored in the Amazon DocumentDB.

We first need to implement a PyTorch dataset class for our protein dataset capable of retrieving mini-batches of protein documents from Amazon DocumentDB. It’s more efficient to retrieve batches documents by the built-in primary id (_id).

  1. We use the iterable-style dataset by extending the IterableDataset, which pre-fetches the _id and labels of the documents at initialization:
class ProteinDataset(data.IterableDataset):
    """
    An iterable-style dataset for proteins in DocumentDB
    Args:
        pipeline: an aggregation pipeline to retrieve data from DocumentDB
        db_uri: URI of the DocumentDB
        db_name: name of the database
        collection_name: name of the collection
        k: k used for kNN when creating a graph from atomic coordinates
    """

    def __init__(
        self, pipeline, db_uri="", db_name="", collection_name="", k=3
    ):

        self.db_uri = db_uri
        self.db_name = db_name
        self.collection_name = collection_name
        self.k = k

        client = MongoClient(self.db_uri, connect=False)
        collection = client[self.db_name][self.collection_name]
        # pre-fetch the metadata as docs from DocumentDB
        self.docs = [doc for doc in collection.aggregate(pipeline)]
        # mapping document '_id' to label
        self.labels = {doc["_id"]: doc["y"] for doc in self.docs}
  1. The ProteinDataset performs a database read operation in the __iter__ method. It tries to evenly split the workload if there are multiple workers:
def __iter__(self):
        worker_info = torch.utils.data.get_worker_info()
        if worker_info is None:
            # single-process data loading, return the full iterator
            protein_ids = [doc["_id"] for doc in self.docs]

        else:  # in a worker process
            # split workload
            start = 0
            end = len(self.docs)
            per_worker = int(
                math.ceil((end - start) / float(worker_info.num_workers))
            )
            worker_id = worker_info.id
            iter_start = start + worker_id * per_worker
            iter_end = min(iter_start + per_worker, end)

            protein_ids = [
                doc["_id"] for doc in self.docs[iter_start:iter_end]
            ]

        # retrieve a list of proteins by _id from DocDB
        with MongoClient(self.db_uri) as client:
            collection = client[self.db_name][self.collection_name]
            cur = collection.find(
                {"_id": {"$in": protein_ids}},
                projection={"coords": True, "seq": True},
            )
            return (
                (
                    convert_to_graph(protein, k=self.k),
                    self.labels[protein["_id"]],
                )
                for protein in cur
            )
  1. The preceding __iter__ method also converts the atomic coordinates of proteins into DGLGraph objects after they’re loaded from Amazon DocumentDB via the convert_to_graph function. This function constructs a k-nearest neighbor (kNN) graph for the amino acid residues using the 3D coordinates of the C-alpha atoms and adds one-hot encoded node features to represent residue identities:
def convert_to_graph(protein, k=3):
    """
    Convert a protein (dict) to a dgl graph using kNN.
    """
    coords = torch.tensor(protein["coords"])
    X_ca = coords[:, 1]
    # construct knn graph from C-alpha coordinates
    g = dgl.knn_graph(X_ca, k=k)
    seq = protein["seq"]
    node_features = torch.tensor([d1_to_index[residue] for residue in seq])
    node_features = F.one_hot(node_features, num_classes=len(d1_to_index)).to(
        dtype=torch.float
    )

    # add node features
    g.ndata["h"] = node_features
    return g
  1. With the ProteinDataset implemented, we can initialize instances for train, validation, and test datasets and wrap the training instance with BufferedShuffleDataset to enable shuffling.
  2. We further wrap them with torch.utils.data.DataLoader to work with other components of the SageMaker PyTorch Estimator training script.
  3. Next, we implement a simple two-layered graph convolution network (GCN) with a global attention pooling layer for ease of interpretation:
class GCN(nn.Module):
    """A two layer Graph Conv net with Global Attention Pooling over the
    nodes.
    Args:
        in_feats: int, dim of input node features
        h_feats: int, dim of hidden layers
        num_classes: int, number of output units
    """

    def __init__(self, in_feats, h_feats, num_classes):
        super(GCN, self).__init__()
        self.conv1 = GraphConv(in_feats, h_feats)
        self.conv2 = GraphConv(h_feats, h_feats)
        # the gate layer that maps node feature to outputs
        self.gate_nn = nn.Linear(h_feats, num_classes)
        self.gap = GlobalAttentionPooling(self.gate_nn)
        # the output layer making predictions
        self.output = nn.Linear(h_feats, num_classes)

    def _conv_forward(self, g):
        """forward pass through the GraphConv layers"""
        in_feat = g.ndata["h"]
        h = self.conv1(g, in_feat)
        h = F.relu(h)
        h = self.conv2(g, h)
        h = F.relu(h)
        return h

    def forward(self, g):
        h = self._conv_forward(g)
        h = self.gap(g, h)
        return self.output(h)

    def attention_scores(self, g):
        """Calculate attention scores"""
        h = self._conv_forward(g)
        with g.local_scope():
            gate = self.gap.gate_nn(h)
            g.ndata["gate"] = gate
            gate = dgl.softmax_nodes(g, "gate")
            g.ndata.pop("gate")
            return gate
  1. Afterwards, we can train this GCN on the ProteinDataset instance for a binary classification task of predicting whether a protein structure is predicted by AlphaFold or not. We use binary cross entropy as the objective function and Adam optimizer for stochastic gradient optimization. The full training script can be found in src/main.py.

Next, we set up the SageMaker PyTorch Estimator to handle the training job. To allow the managed Docker container initiated by SageMaker to connect to Amazon DocumentDB, we need to configure the subnet and security group for the Estimator.

  1. We retrieve the subnet ID where the Network Address Translation (NAT) gateway resides, as well as the security group ID of our Amazon DocumentDB cluster by name:
ec2 = boto3.client("ec2")
# find the NAT gateway's subnet ID 
resp = ec2.describe_subnets(
    Filters=[{"Name": "tag:Name", "Values": ["{}-NATSubnet".format(stack_name)]}]
)
nat_subnet_id = resp["Subnets"][0]["SubnetId"]
# find security group id of the DocumentDB
resp = ec2.describe_security_groups(
    Filters=[{
        "Name": "tag:Name", 
        "Values": ["{}-SG-DocumentDB".format(stack_name)]
    }])
sg_id = resp["SecurityGroups"][0]["GroupId"]
Finally, we can kick off the training of our GCN model using SageMaker: 
from sagemaker.pytorch import PyTorch

CODE_PATH = "main.py"

params = {
    "patience": 5, 
    "n-epochs": 200,
    "batch-size": 64,
    "db-host": secrets["host"],
    "db-username": secrets["username"], 
    "db-password": secrets["password"], 
    "db-port": secrets["port"],
    "knn": 4,
}

estimator = PyTorch(
    entry_point=CODE_PATH,
    source_dir="src",
    role=role,
    instance_count=1,
    instance_type="ml.p3.2xlarge", # 'ml.c4.2xlarge' for CPU
    framework_version="1.7.1",
    py_version="py3",
    hyperparameters=params,
    sagemaker_session=sess,
    subnets=[nat_subnet_id], 
    security_group_ids=[sg_id],
)
# run the training job:
estimator.fit()

Load and evaluate the trained GNN model

When the training job is complete, we can load the trained GCN model and perform some in-depth evaluation.

The codes for the following steps are also available in the notebook Train_and_eval.ipynb.

SageMaker training jobs save the model artifacts into the default S3 bucket, the URI of which can be accessed from the estimator.model_data attribute. We can also navigate to the Training jobs page on the SageMaker console to find the trained model to evaluate.

  1. For research purposes, we can load the model artifact (learned parameters) into a PyTorch state_dict using the following function:
def load_sagemaker_model_artifact(s3_bucket, key):
    """Load a PyTorch model artifact (model.tar.gz) produced by a SageMaker
    Training job.
    Args:
        s3_bucket: str, s3 bucket name (s3://bucket_name)
        key: object key: path to model.tar.gz from within the bucket
    Returns:
        state_dict: dict representing the PyTorch checkpoint
    """
    # load the s3 object
    s3 = boto3.client("s3")
    obj = s3.get_object(Bucket=s3_bucket, Key=key)
    # read into memory
    model_artifact = BytesIO(obj["Body"].read())
    # parse out the state dict from the tar.gz file
    tar = tarfile.open(fileobj=model_artifact)
    for member in tar.getmembers():
        pth = tar.extractfile(member).read()

    state_dict = torch.load(BytesIO(pth), map_location=torch.device("cpu"))
return state_dict

	state_dict = load_sagemaker_model_artifact(
bucket, 
key=estimator.model_data.split(bucket)[1].lstrip("/")
)

# initialize a GCN model
model = GCN(dim_nfeats, 16, n_classes)
# load the learned parameters
model.load_state_dict(state_dict["model_state_dict"])
  1. Next, we perform quantitative model evaluation on the full test set by calculating accuracy:
device = torch.device("cuda:0") if torch.cuda.is_available() else torch.device("cpu")
num_correct = 0
num_tests = 0
model.eval()
with torch.no_grad():
    for batched_graph, labels in test_loader:
        batched_graph = batched_graph.to(device)
        labels = labels.to(device)
        logits = model(batched_graph)
        preds = (logits.sigmoid() > 0.5).to(labels.dtype)
        num_correct += (preds == labels).sum().item()
        num_tests += len(labels)

print('Test accuracy: {:.6f}'.format(num_correct / num_tests))

We found our GCN model achieved an accuracy of 74.3%, whereas the dummy baseline model making predictions based on class priors only achieved 56.3%.

We’re also interested in interpretability of our GCN model. Because we implement a global attention pooling layer, we can compute the attention scores across nodes to explain specific predictions made by the model.

  1. Next, we compute the attention scores and overlay them on the protein graphs for a pair of structures (AlphaFold predicted and experimental) from the same peptide:
pair = ["AF-Q57887", "1JT8-A"]
cur = collection.find(
    {"id": {"$in": pair}},
)

for doc in cur:
    # convert to dgl.graph object
    graph = convert_to_graph(doc, k=4)
    
    with torch.no_grad():
        # make prediction
        pred = model(graph).sigmoid()
        # calculate attention scores for a protein graph
        attn = model.attention_scores(graph)
    
    pred = pred.item()
    attn = attn.numpy()
    
    # convert to networkx graph for visualization
    graph = graph.to_networkx().to_undirected()
    # calculate graph layout
    pos = nx.spring_layout(graph, iterations=500)
    
    fig, ax = plt.subplots(figsize=(8, 8))
    nx.draw(
        graph, 
        pos, 
        node_color=attn.flatten(),
        cmap="Reds",
        with_labels=True, 
        font_size=8,
        ax=ax
    )
    ax.set(title="{}, p(is_predicted)={:.6f}".format(doc["id"], pred))
plt.show()

The preceding codes produce the following protein graphs overlaid with attention scores on the nodes. We find the model’s global attentive pooling layer can highlight certain residues in the protein graph as being important for making the prediction of whether the protein structure is predicted by AlphaFold. This indicates that these residues may have distinctive graph topologies in predicted and experimental protein structures.

In summary, we showcase a scalable deep learning solution to train GNNs on protein structures stored in Amazon DocumentDB. Although the tutorial only uses thousands of proteins for training, this solution is scalable to millions of proteins. Unlike other approaches such as serializing the entire protein dataset, our approach transfers the memory-heavy workloads to the database, making the memory complexity for the training jobs O(batch_size), which is independent of the total number of proteins to train.

Clean up

To avoid incurring future charges, delete the CloudFormation stack you created. This removes all the resources you provisioned using the CloudFormation template, including the VPC, Amazon DocumentDB cluster, and SageMaker instance. For instructions, see Deleting a stack on the AWS CloudFormation console.

Conclusion

We described a cloud-based deep learning architecture scalable to millions of protein structures by storing them in Amazon DocumentDB and efficiently retrieving mini-batches of data from SageMaker.

To learn more about the use of GNN in protein property predictions, check out our recent publication LM-GVP, A Generalizable Deep Learning Framework for Protein Property Prediction from Sequence and Structure.

About the Authors

Zichen Wang, PhD, is an Applied Scientist in the Amazon Machine Learning Solutions Lab. With several years of research experience in developing ML and statistical methods using biological and medical data, he works with customers across various verticals to solve their ML problems.

Selvan Senthivel is a Senior ML Engineer with the Amazon ML Solutions Lab at AWS, focusing on helping customers on machine learning, deep learning problems, and end-to-end ML solutions. He was a founding engineering lead of Amazon Comprehend Medical and contributed to the design and architecture of multiple AWS AI services.

Read More

Meet the 2021-22 Accenture Fellows

Launched in October of 2020, the MIT and Accenture Convergence Initiative for Industry and Technology underscores the ways in which industry and technology come together to spur innovation. The five-year initiative aims to achieve its mission through research, education, and fellowships. To that end, Accenture has once again awarded five annual fellowships to MIT graduate students working on research in industry and technology convergence who are underrepresented, including by race, ethnicity, and gender.

This year’s Accenture Fellows work across disciplines including robotics, manufacturing, artificial intelligence, and biomedicine. Their research covers a wide array of subjects, including: advancing manufacturing through computational design, with the potential to benefit global vaccine production; designing low-energy robotics for both consumer electronics and the aerospace industry; developing robotics and machine learning systems that may aid the elderly in their homes; and creating ingestible biomedical devices that can help gather medical data from inside a patient’s body.

Student nominations from each unit within the School of Engineering, as well as from the four other MIT schools and the MIT Schwarzman College of Computing, were invited as part of the application process. Five exceptional students were selected as fellows in the initiative’s second year.

Xinming (Lily) Liu is a PhD student in operations research at MIT Sloan School of Management. Her work is focused on behavioral and data-driven operations for social good, incorporating human behaviors into traditional optimization models, designing incentives, and analyzing real-world data. Her current research looks at the convergence of social media, digital platforms, and agriculture, with particular attention to expanding technological equity and economic opportunity in developing countries. Liu earned her BS from Cornell University, with a double major in operations research and computer science.

Caris Moses is a PhD student in electrical engineering and computer science specializing in artificial intelligence. Moses’ research focuses on using machine learning, optimization, and electromechanical engineering to build robotics systems that are robust, flexible, intelligent, and can learn on the job. The technology she is developing holds promise for industries including flexible, small-batch manufacturing; robots to assist the elderly in their households; and warehouse management and fulfillment. Moses earned her BS in mechanical engineering from Cornell University and her MS in computer science from Northeastern University.

Sergio Rodriguez Aponte is a PhD student in biological engineering. He is working on the convergence of computational design and manufacturing practices, which have the potential to impact industries such as biopharmaceuticals, food, and wellness/nutrition. His current research aims to develop strategies for applying computational tools, such as multiscale modeling and machine learning, to the design and production of manufacturable and accessible vaccine candidates that could eventually be available globally. Rodriguez Aponte earned his BS in industrial biotechnology from the University of Puerto Rico at Mayaguez.

Soumya Sudhakar SM ’20 is a PhD student in aeronautics and astronautics. Her work is focused on the co-design of new algorithms and integrated circuits for autonomous low-energy robotics that could have novel applications in aerospace and consumer electronics. Her contributions bring together the emerging robotics industry, integrated circuits industry, aerospace industry, and consumer electronics industry. Sudhakar earned her BSE in mechanical and aerospace engineering from Princeton University and her MS in aeronautics and astronautics from MIT.

So-Yoon Yang is a PhD student in electrical engineering and computer science. Her work on the development of low-power, wireless, ingestible biomedical devices for health care is at the intersection of the medical device, integrated circuit, artificial intelligence, and pharmaceutical fields. Currently, the majority of wireless biomedical devices can only provide a limited range of medical data measured from outside the body. Ingestible devices hold promise for the next generation of personal health care because they do not require surgical implantation, can be useful for detecting physiological and pathophysiological signals, and can also function as therapeutic alternatives when treatment cannot be done externally. Yang earned her BS in electrical and computer engineering from Seoul National University in South Korea and her MS in electrical engineering from Caltech.

Read More

NVIDIA Builds Isaac AMR Platform to Aid $9 Trillion Logistics Industry

Manufacturing and fulfillment centers are profoundly complex. Whenever new earbuds or socks land at your doorstep in hours or a vehicle rolls off an assembly line, a maze of magic happens with AI-driven logistics.

Massive facilities like these are constantly in flux. Robots travel miles of aisles to roll up millions of products to assist teams of moving people. Obstacles are ever present.

Today we’re introducing the Isaac Autonomous Mobile Robot (AMR) platform to  optimize operational efficiency and accelerate deployment of AMRs. Isaac AMR extends NVIDIA Isaac capabilities for building and deploying robotics applications, bringing mapping, site analytics and fleet optimization onto NVIDIA EGX servers.

Industrial facilities of this type can be as big as city blocks or stadiums. They are constantly reconfigured or scaled up to meet product demands of the moment. Path planning and rerouting for autonomous robots need to move in lockstep.

At industrial scale, even small routing optimizations can save billions of dollars in the $9 trillion logistics industry.

Autonomous mobile robot deployments are estimated to reach 53,000 sites by 2025, up from 9,000 in 2020, according to Interact Analysis. Meanwhile, supply chains strain to keep up with increasing e-commerce amid worker shortages and COVID-19 restrictions.

One hurdle is the ability to rapidly and autonomously develop high-accuracy robot maps. They need to be continuously updated as operations scale or fluctuate. And increasing situational awareness of changing environments for mobile robots and continuously reoptimizing the routes —  all while developing new skill sets through simulation — is paramount to operational efficiency.

Isaac AMR is the result of years of research and product development at NVIDIA. The framework is available on the NVIDIA NGC software hub and within the NVIDIA Omniverse platform, tapping into Metropolis and ReOpt at first, and soon DeepMap and more NVIDIA technologies.

Scaling Operations With Isaac AMR

The AI and computing challenges of autonomous mobile robots for manufacturing and fulfillment aren’t all that different from those of autonomous vehicles.

Obstacles and people must be avoided. Destinations need to be reached. Thousands of sensors driven by GPU-accelerated algorithms help fleets of autonomous robots solve the traveling salesman problem — finding the shortest path between multiple destinations —  among ever-changing industrial workflows in real time.

The Isaac AMR platform uses NVIDIA Omniverse for creating digital twins of the facility where AMRs will be deployed. NVIDIA Isaac Sim (built on Omniverse) simulates the behavior of robot fleets, people and other machines in the digital twins with high-fidelity physics and perception. It also enables synthetic data generation for training of AI models.

Isaac AMR consists of GPU-accelerated AI technologies and SDKs including DeepMap, ReOpt and Metropolis. These technologies are securely orchestrated and cloud delivered with NVIDIA Fleet Command.

DeepMap Delivers Mapping Advances

NVIDIA’s recent acquisition of DeepMap brings advances in mapping for autonomous vehicles to the AMR industry as well.

AMR deployments can access the DeepMap platform’s cloud-based SDK to help accelerate robot mapping of large facilities from weeks to days, while achieving centimeter-level accuracy.

The DeepMap Update Client enables robot maps to be updated as frequently as necessary, in real time. And the DeepMap SDK delivers layers of intelligence to maps by adding semantic understanding — so robots can identify the objects pixels represent and know if they can move one way or not. It’s also capable of addressing both indoor and outdoor map building.

As part of the Isaac AMR platform, NVIDIA DeepMap integrates with other components, such as Metropolis, ReOpt, Isaac Sim via Omniverse and more.

NVIDIA Metropolis Adds Real-Time Situational Awareness

Mapping doesn’t account for everything in these environments. And the advanced sensors onboard AMRs aren’t always sufficient to ensure safe and efficient operation.

The NVIDIA Metropolis video analytics platform meets the need for higher level real-time “outside-in” perception by providing access to cameras and sensors deployed all over the factory or warehouse floor.

With Metropolis, AMRs have access to additional layers of situational awareness on the factory floor, enabling them to avoid high-congestion areas, eliminating blind spots and enhanced visibility of both people and other AMRs. In addition, Metropolis’s pre-trained models provide a head start in customizing for site-specific needs.

ReOpt Libraries Transform Logistics 

NVIDIA ReOpt AI software libraries can be used to optimize vehicle route planning and logistics in real time, which can be applied to fleets of AMRs.

Many factors need to be considered in deciding the optimal AMR fleet size to be deployed for large, complex environments. Robot speeds, battery life, transport size and weight, and facilities layout all factor in.

Companies can simulate (using Isaac Sim) multiple AMR interactions with NVIDIA ReOpt. These can happen quickly and accurately in digital twins of environments such as warehouses. And they can be implemented before deploying robots in production as situations change, saving time and money.

And, once deployed, routes have to be continuously re-optimized to deliver the most operational efficiency. NVIDIA ReOpt provides for dynamic reoptimization of routes to a fleet of heterogeneous AMRs based on a number of constraints.

Deploying AMRs Into Production

Available on NVIDIA EGX servers, the Isaac AMR platform enhances the development of AI-driven logistics by providing a complete path to build industrial and human-robot simulations as well as route optimizations.

Isaac AMR Platform is built to be enterprise class and cloud ready. The NVIDIA Technologies presented as part of the Isaac AMR platform can be securely deployed and managed on EGX servers with NVIDIA Fleet Command.

Check out the latest developments on Omniverse, Metropolis, DeepMap and ReOpt.

Learn more about the Isaac Robotics Platform and use the contact us form on the portal if you have any questions. 

The post NVIDIA Builds Isaac AMR Platform to Aid $9 Trillion Logistics Industry appeared first on The Official NVIDIA Blog.

Read More

Gamers, Creators, Drivers Feel GeForce RTX, NVIDIA AI Everywhere

Putting the power of graphics and AI at the fingertips of more users than ever, NVIDIA announced today new laptops and autonomous vehicles using GeForce RTX and NVIDIA AI platforms and expanded reach for GeForce NOW cloud gaming across Samsung TVs and the AT&T network.

A virtual address prior to CES showed next-gen games, new tools for creating virtual worlds, and AI-powered vehicles — all accelerated by NVIDIA technologies.

160-Plus Portable Powerhouses

“Ray tracing and AI are defining the next generation of content,” said Jeff Fisher, senior vice president of NVIDIA’s GeForce business, announcing more than 160 thin-and-light laptops using RTX 30 Series GPUs in a smorgasbord of mobile designs.

They will serve 3 billion gamers and tens of millions of content creators worldwide, who are increasingly turning to portable PCs.

Many of the systems will pack a new GeForce RTX 3080 Ti laptop GPU, the first time the flagship 80 Ti class comes to mobile PCs. It sports 16GB of the fastest ever GDDR6 memory in a laptop and delivers higher performance than the desktop TITAN RTX, with prices starting at $2,499.

More will light up with the GeForce RTX 3070 Ti laptop GPU, the newest member of our fastest-growing class of GPUs. It can drive displays at 100 frames/second with 1,440-pixel resolution, at prices starting at $1,499.

GeForce RTX laptops announced at CES

Laptops with the new GPUs will be available starting Feb. 1. They bring a fourth generation of Max-Q Technologies, a systems design approach that enables the thinnest, lightest, quietest gaming laptops.

The latest Max-Q applies AI to optimally balance GPU/CPU power use, battery discharge, image quality and frame rates. Laptops running it can provide up to 70 percent more battery life while delivering more performance.

Mainstream and Monster GPUs

Fisher also announced the GeForce RTX 3050, which brings ray tracing and accelerated AI to mainstream desktops.

The GPU lets mass market PCs run the latest games at 60 frames per second, thanks to graphics and AI cores in its NVIDIA Ampere architecture.

The RTX 3050 sports 8GB of GDDR6 memory. The card will be available from partners worldwide on Jan. 27 starting at $249.

“RTX is the new standard, and the GeForce RTX 3050 makes it more accessible than ever,” said Fisher, showing a “monster” RTX 3090 Ti (pictured below) also in the wings.

GeForce RTX 3090 Ti

The RTX 3090 Ti will pack 24GB of GDDR6X running at 21 Gbit/s, the fastest memory ever. The GPU will crank out 40 teraflops for shaders, 78 teraflops for ray tracing and a whopping 320 teraflops of AI muscle. More details will be coming later this month.

New Initiatives with AT&T, Samsung

Giving high-performance mobile gaming new wings, AT&T and NVIDIA are working together to provide customers an optimized experience. And AT&T customers are getting a special offer to enjoy NVIDIA GeForce NOW on 5G.

“Starting today, AT&T customers with a 5G device on a qualifying plan, can get a six-month GeForce NOW Priority Membership at no charge,” said Fisher.

GeForce NOW on AT&T's 5G network

For its part, Samsung will offer NVIDIA’s cloud gaming service on its smart TVs through the Samsung Gaming Hub around midyear.

It’s the latest move to bring GeForce NOW to the living room. In November, LG showed a beta version of the service on its 2021 WebOS smart TVs.

As for games, GeForce NOW continues to expand support for the Electronic Arts catalog, adding Battlefield 4 and Battlefield 5 to GeForce NOW’s online library of more than 1,100 PC games played by 15 million subscribers.

Heart-Pounding RTX and Reflex Games

Speaking of new games, NVIDIA announced 10 new RTX games.

Fisher showed clips from four upcoming titles that this year will support RTX, DLSS — NVIDIA’s AI-powered graphics enhancing technology — or both.

Escape From Tarkov and Rainbow Six Extraction are adding DLSS, while The Day Before and Dying Light 2 Stay Human are adding both DLSS and ray tracing.

In addition, seven new titles support NVIDIA Reflex for low-latency gameplay. They include Sony’s God of War, Rainbow Six Extraction and iRacing, taking Reflex into the world’s premier online racing simulator.

And we’re raising the bar in esports. A new class of 27-inch esports displays with 1440-pixel resolution and G-SYNC at up to 360 Hz refresh rates will be available soon from AOC, ASUS, MSI and ViewSonic.

Building 3D Virtual Worlds

For the 45 million professionals who create games, movies and more, Fisher described tools transforming their workflows.

“We are at the dawn of the next digital frontier. Interconnected 3D virtual worlds … with shops, homes, people, robots, factories, museums … will be built by an expanding number of creators, collaborating across the globe,” he said.

NVIDIA Omniverse, the powerful platform for artists to collaborate and accelerate 3D work, remains free and is now generally available for GeForce and NVIDIA RTX Studio creators.

NVIDIA Omniverse available on GeForce RTX Studio

Omniverse uses Pixar’s open-standard Universal Scene Description (USD) to connect tools from more than 40 software development partners into a single 3D design platform. That lets creators across the globe collaborate in Omniverse on shared 3D workflows.

“This is the future of 3D content creation and how virtual worlds will be built,” Fisher said.

He detailed new capabilities in Omniverse, including:

  • Omniverse Nucleus Cloud, a one-click-to-collaborate 3D scene sharing feature, now in early access
  • Updates to Omniverse Audio2Face, an AI-enabled app that animates a 3D face based on an audio track, including direct export to Epic’s MetaHuman for creating realistic characters
  • New assets from Mechwarrior 5 and Shadow Warrior 3 added to the Omniverse Machinima library
  • And a wealth of free digital assets now available in the Omniverse launcher from leading 3D marketplaces.

In addition, we’ve upgraded NVIDIA Canvas, our Studio app that converts brushstrokes into photorealistic images. Available free to download here, it now supports 4x the resolution and new materials like flowers and bushes thanks to the efforts of NVIDIA researchers who developed GauGAN2.

It’s part of NVIDIA Studio’s broad and deep software stack, which accelerates more than 200 of the industry’s top creative applications.

GeForce RTX laptops include the Razer Blade 14

All that software is available in systems as svelte as the Razer Blade 14, a new ultraportable with an RTX 3080 Ti.

“I feel like I can go anywhere with it and still am able to produce the quality artwork that I could do at home,” said a 3D artist who took it on a test drive and shared his experience in the special address.

Taking AI to the Street

In autonomous cars and trucks — another hot topic at CES — NVIDIA announced more companies adopting its open DRIVE Hyperion platform that includes a high-performance computer and sensor architecture meeting the safety requirements of fully autonomous vehicles.

The latest generation DRIVE Hyperion 8 is designed with redundant NVIDIA DRIVE Orin systems-on-a-chip, 12 state-of-the-art surround cameras, nine radar, 12 ultrasonics, one front-facing lidar and three interior-sensing cameras. It’s architected to be functionally safe, so that if one computer or sensor fails, there is a back up available to ensure that the AV can drive its passengers to a safe place.

Electric vehicle makers such as Volvo-backed Polestar and EV companies in China, including NIO, Xpeng, Li Auto, R Auto and IM Motors have all adopted DRIVE Hyperion.

“These new electric cars will get better and better over time with each over-the-air update,” said Ali Kani, vice president and general manager of NVIDIA’s automotive business.

“Such companies can benefit from new business models that are software driven,” he added.

Electric vehicle makers adopting Drive Hyperion

Robotaxi services like Cruise, Zoox and DiDi and trucking services like Volvo, Navistar and Plus are also embracing DRIVE Hyperion.

Autonomous trucking company TuSimple announced at CES that it will build its new platform on NVIDIA DRIVE Orin. It’s working with leading delivery companies such as UPS, Navistar and Penske, and its technology has already improved arrival times for long-haul routes of the U.S. Postal Service.

Such vehicles will help fill an estimated shortage of more than 140,000 drivers by 2027 in the U.S. alone.

TuSimple adopts DRIVE Orin

In addition, five leading automotive suppliers — Desay, Flex, Quanta, Valeo and ZF — now support DRIVE Hyperion.

Kani ended the session with a demo of NVIDIA DRIVE Concierge.

An always-on, digital assistant for drivers, it’s one example of helpful agents created with NVIDIA Omniverse Avatar using the company’s speech AI, computer vision, natural language understanding, recommendation engines and simulation technologies.

DRIVE Concierge is the latest addition to NVIDIA’s rich real-time, AI-based software stack for autonomous vehicles. The end-to-end solution also includes code to train AI models in the cloud and run complex simulations in data centers to safely test and validate an entire AV system.

“From AI in the car to AI in the cloud, NVIDIA is paving the way to safer and more efficient transportation,” said Kani.

For more news, check out our CES page and watch the full special address below.

The post Gamers, Creators, Drivers Feel GeForce RTX, NVIDIA AI Everywhere appeared first on The Official NVIDIA Blog.

Read More

NVIDIA Canvas Updated With New AI Model Delivering 4x Resolution and More Materials

As art evolves and artists’ abilities grow, so must their creative tools.

NVIDIA Canvas, the free beta app and part of the NVIDIA Studio suite of creative apps and tools, has brought the real-time painting tool GauGAN to anyone with an NVIDIA RTX GPU.

Artists use advanced AI to quickly turn simple brushstrokes into realistic landscape images, speeding up concept exploration and freeing more time to visualize ideas.

The NVIDIA Canvas update released today, powered by the GauGAN2 AI model and NVIDIA RTX GPU Tensor Cores, generates backgrounds with increased quality and 4x higher resolution, and adds five new materials to paint with.

Updates to NVIDIA Canvas, the launch of NVIDIA Omniverse in general availability and newly announced NVIDIA Studio products highlight all the Studio goodness at CES. Read the full recap.

Artwork Made More Easel-y

The NVIDIA Canvas update enables even more realistic images with greater definition and fewer artifacts, adding significant photorealism, all in up to 1K pixel resolution for increased visual quality.

Exports can easily be integrated into existing creative workflows by exporting to Adobe Photoshop or an artist’s preferred creative app.

A dramatic 4X increase in pixel resolution.

Artists start in Canvas by sketching simple shapes and lines, then the AI model immediately fills the screen with real-world materials such as water, grass, snow, mountains and more for show-stopping results.

The update adds five new materials to create richer landscape environments: straw, flowers, mud, dirt and bushes.

New materials including mud, dirt and bush add more realism and variety to backgrounds.

These new materials can be further customized with nine preset styles that modify the look and feel of the painting. Artists have the option to upload their own imagery to create a one-of-a-kind background.

NVIDIA Canvas dials up the realism with greater definition and fewer artifacts.

With Canvas’s new AI, creative possibilities have become even more limitless.

Start Painting With AI

Download the latest NVIDIA Canvas app update, give us feedback, and install the January NVIDIA Studio Driver to optimize your creative app arsenal.

Share your Canvas paintings with the hashtag #StudioShare for a chance to be featured on the Studio Facebook, Twitter and Instagram channels. Subscribe to the Studio YouTube channel for tutorials, tips and tricks by industry-leading artists, and stay up to date on all things Studio by signing up for the NVIDIA Studio newsletter.

The post NVIDIA Canvas Updated With New AI Model Delivering 4x Resolution and More Materials appeared first on The Official NVIDIA Blog.

Read More

Groundbreaking Updates to NVIDIA Studio Power the 3D Virtual Worlds of Tomorrow, Today

We’re at the dawn of the next digital frontier. Creativity is fueling new developments in design, innovation and virtual worlds.

For the creators driving this future, we’ve built NVIDIA Studio, a fully accelerated platform with high-performance GPUs as the heartbeat for laptops and desktops.

This hardware is paired with exclusive NVIDIA RTX-accelerated software optimizations in top creative apps and a suite of tools like NVIDIA Omniverse, Canvas and Broadcast, which help creators enhance their workflows.

And it’s all supported by specialized drivers that are updated monthly for performance and reliability — like the January Studio Driver, available starting today.

The interconnected 3D virtual worlds of tomorrow are being built today. NVIDIA Omniverse, designed to be the foundation that connects these virtual worlds, is now available to millions of NVIDIA Studio creators using GeForce RTX and NVIDIA RTX GPUs.

We’ve also introduced GeForce RTX 3080 Ti and 3070 Ti-based Studio laptops, groundbreaking hardware with heightened levels of performance — especially on battery.

Updated with a major increase in fidelity, new materials and an upgraded AI model, NVIDIA Canvas enables artists to turn simple brushstrokes into realistic landscape images by using AI.

Omniverse Available to Millions of Individual Creators and Artists Worldwide

Expanding the NVIDIA Studio ecosystem, NVIDIA Omniverse is now available at no cost to millions of individual creators with GeForce RTX and NVIDIA RTX GPUs.

Bolstered by new features and tools, NVIDIA’s real-time design collaboration and simulation platform empowers artists, designers and creators to connect and collaborate in leading 3D design applications from their RTX-powered laptop, desktop or workstation.

Multi-app workflows can grind to a halt with near-constant exporting and importing. With Omniverse, creators can connect their favorite 3D design tools to a single scene and simultaneously create and edit between the apps.

 

We’ve also announced platform developments for Omniverse Machinima with new free game characters, objects and environments from Mechwarrior 5, Shadow Warrior 3, Squad and Mount & Blade II: Bannerlord; and Omniverse Audio2Face with new blendshape support and direct export to Epic’s MetaHuman; plus early access to new platform features like Omniverse Nucleus Cloud — enabling simple “one-click-to-collaborate” sharing of large Omniverse 3D scenes.

Learn more about Omniverse, the latest enhancements and its general availability, and download the latest version at nvidia.com/omniverse.

New Studio Laptops With GeForce RTX 3080 Ti and RTX 3070 Ti GPUs

NVIDIA Studio laptops provide the best mobile performance for 3D creation. The new GeForce RTX 3080 Ti Laptop GPU features 16GB of the fastest GDDR6 memory ever shipped in a laptop and higher performance than the desktop TITAN RTX.

The new GeForce RTX 3070 Ti also delivers fantastic performance — it’s up to 70 percent faster than RTX 2070 SUPER laptops.

Next-generation laptop technologies are amping up performance. We’ve worked with CPU vendors on CPU Optimizer. It’s a new, low-level framework enabling the GPU to further optimize performance, temperature and power of next-gen CPUs. As a result, CPU efficiency is improved and power is transferred to the GPU for more performance in creative applications.

In compute-heavy apps like Adobe Premiere, Blender and Matlab, we’ve developed Rapid Core Scaling. It enables the GPU to sense the real-time demands of applications and use only the cores it needs rather than all of them. This frees up power that can be used to run the active cores at higher frequencies, delivering up to 3x more performance for intensive creative work on the go.

ASUS, MSI and Razer are launching new laptops with a wide range of designs — and up to GeForce RTX 3080 Ti GPUs — starting in February.

Paint by AI With NVIDIA Canvas

Bolstered by work from the NVIDIA Research team developing GauGAN2, NVIDIA Canvas is now available with 4 times higher resolution and five new materials.

The GauGAN2 AI model incorporated in the latest update helps deliver more realistic images with greater definition and fewer artifacts.

 

Five new materials — straw, flowers, mud, dirt and bush — liven up and create richer landscape environments.

Read more about the latest NVIDIA Canvas update.

Reliable Performance

Creators can download the January Studio Driver, available now with improved performance and reliability for the Omniverse and Canvas updates.

 

With monthly updates, NVIDIA Studio Drivers deliver smooth performance on creative applications and the best possible experience when using NVIDIA GPUs. Extensive multi-app workflow testing ensures the latest apps run smoothly.

New Desktop Solutions, Coming Soon

Finally, the GeForce RTX 3050 GPU brings even more choice for creators. Our new entry-level GPU provides the most accessible way of getting great RTX benefits — real-time ray tracing, AI, a top-notch video encoder and video acceleration. Starting at just $279, it’s a great way to start creating with RTX. Look for availability from partners worldwide on Jan. 27.

 

One more thing: Keep an eye out for more information on GeForce RTX 3090 Ti later this month. It’ll have a huge 24GB of lightning-fast video memory, making it perfect for conquering nearly any creative task.

Subscribe to the Studio YouTube channel for tutorials, tips and tricks by industry-leading artists, and stay up to date on all things Studio by signing up for the NVIDIA Studio newsletter.

The post Groundbreaking Updates to NVIDIA Studio Power the 3D Virtual Worlds of Tomorrow, Today appeared first on The Official NVIDIA Blog.

Read More

NVIDIA Makes Free Version of Omniverse Available to Millions of Individual Creators and Artists Worldwide

Designed to be the foundation that connects virtual worlds, NVIDIA Omniverse is now available to millions of individual NVIDIA Studio creators using GeForce RTX and NVIDIA RTX GPUs.

In a special address at CES, NVIDIA also announced new platform developments for Omniverse Machinima and Omniverse Audio2Face, new platform features like Nucleus Cloud and 3D marketplaces, as well as ecosystem updates.

With Omniverse, NVIDIA’s real-time 3D design collaboration and virtual world simulation platform, artists, designers and creators can use leading design applications to create 3D assets and scenes from their laptop or workstation.

Since its open beta launch a year ago, Omniverse has been downloaded by almost 100,000 creators who are accelerating their workflows with its core rendering, physics and AI technologies.

“With this technology, content creators get more than just a fast renderer,” said Zhelong Xu, a digital artist and Omniverse Creator based in Shanghai. “NVIDIA Omniverse and RTX give artists a powerful platform with infinite possibilities.”

Creators like Xu are the people who will use Omniverse’s tools to build and collaborate on the vast amounts of content needed for this next generation of the web. They’re building interconnected 3D virtual worlds for commerce, entertainment, creativity and industry.

These boundless worlds will be populated with shops, homes, people, robots, factories, museums — a staggering amount of 3D content. This content is challenging to produce, typically requiring multiple, often incompatible tools.

Omniverse connects these independent 3D design worlds into a shared virtual scene.

The culmination of over 20 years of NVIDIA’s groundbreaking work, Omniverse brings graphics, AI, simulation and scalable compute into a single platform to enhance existing 3D workflows.

With Omniverse, free for individual users, GeForce RTX Studio creators can connect their favorite 3D design tools to a single scene and simultaneously create and edit between the apps.

Download it at nvidia.com/omniverse.

Omniverse Enterprise, the paid subscription for professional teams, was made available at GTC in November and is sold by NVIDIA’s global partner network. Today’s announcement brings Omniverse capabilities into the hands of individual creators.

New Omniverse Features 

New features within Omniverse include: 

  • Omniverse Nucleus Cloud enables “one-click-to-collaborate” simple sharing of large Omniverse 3D scenes, meaning artists can collaborate from across the room or the globe without transferring massive datasets. Changes made by the artist are reflected back to the client — like working on a cloud-shared document — but for a 3D scene.
  • New support for the Omniverse ecosystem provided by leading 3D marketplaces and digital asset libraries gives creators an even easier way to build their scenes. TurboSquid by Shutterstock, CGTrader, Sketchfab and Twinbru have released thousands of Omniverse-ready assets for creators, all based on Universal Scene Description (USD) format, and are found directly in the Omniverse Launcher. Reallusion’s ActorCore, Daz3D and e-on software’s PlantCatalog will soon release their own Omniverse-ready assets.
  • Omniverse Machinima for RTX creators who love to game — now featuring new, free characters, objects and environments from leading game titles like Mechwarrior 5 and Shadow Warrior 3, plus Mount & Blade II: Bannerlord and Squad assets in the Machinima library. Creators can remix and recreate their own game cinematics with these assets by dragging and dropping them into their scenes.
  • Omniverse Audio2Face, a revolutionary AI-enabled app that instantly animates a 3D face with just an audio track, now offers blendshape support and direct export to Epic’s MetaHuman Creator app. This leaves the tedious, manual blend-shaping process to AI, so artists and creators can spend more time on their creative workflows.

Omniverse Ecosystem Expands

The NVIDIA Omniverse ecosystem expands with new Omniverse Connectors, extensions and asset libraries — built by many partners.

Today, there are 14 connectors to applications like Autodesk 3ds Max, Autodesk Maya and Epic Games’ Unreal Engine — with many more in the pipeline, including an Adobe Substance 3D Material Extension coming soon.

The latest Omniverse Connectors, extensions, and asset libraries include:

  • e-on software’s VUE, an all-in-one application that allows users to create digital 3D nature, from skies and volumetric clouds to terrains, large-scale ecosystems, wind-swept vegetation populations, open water bodies, roads and rocks — all based on nature’s rules – and includes a native Omniverse live link connector that will sync all scene modifications directly to Omniverse stages.
  • e-on software’s PlantFactory, a vegetation application that allows modeling of foliage as small as twigs or as big as giant redwood trees from scratch. It also models animation like wind and permits asset export in a wide variety of formats, including a direct link from PlantFactory to Omniverse.
  • e-on software’s PlantCatalog, which provides a collection of over 120 ready-made procedural vegetation assets.
  • Twinbru, a “digital twin of physical fabric” provider that supplies interior and exterior furnishing fabrics for drapery, sheers, curtains and upholstery applications, offering 21,000 different fabrics and 11,000 digitized fabric twins — all high quality and physically accurate — now integrated into Omniverse to help streamline manufacturing and architectural designs.

Watch the NVIDIA special address on demand for a recap of all of the company’s announcements at CES.

Creators can download NVIDIA Omniverse for free, submit their work to the NVIDIA Omniverse gallery, and find resources through our forums, Medium, Twitter, YouTube, Twitch, Instagram and Discord server.

The post NVIDIA Makes Free Version of Omniverse Available to Millions of Individual Creators and Artists Worldwide appeared first on The Official NVIDIA Blog.

Read More

Autonomous Era Arrives at CES 2022 With NVIDIA DRIVE Hyperion and Omniverse Avatar

CES has long been a showcase on what’s coming down the technology pipeline. This year, NVIDIA is showing the radical innovation happening now.

During a special virtual address at the show, Ali Kani, vice president and general manager of Automotive at NVIDIA, detailed the capabilities of DRIVE Hyperion and the many ways the industry is developing on the platform. These critical advancements show the maturity of autonomous driving technology as companies begin to deploy safer, more efficient transportation.

Developing autonomous vehicle technology requires an entirely new platform architecture and software development process. Both the hardware and software must be comprehensively tested and validated to ensure they can handle the harsh conditions of daily driving with the stringent safety and security needs of automated vehicles

This is why NVIDIA has built and made open the DRIVE Hyperion platform, which includes a high-performance computer and sensor architecture that meets the safety requirements of a fully autonomous vehicle. DRIVE Hyperion is designed with redundant NVIDIA DRIVE Orin systems-on-a-chip for software-defined vehicles that continuously improve and create a wide variety of new software- and service-based business models for vehicle manufacturers.

The latest-generation platform includes 12 state-of-the-art surround cameras, 12 ultrasonics, nine radar, three interior sensing cameras and one front-facing lidar. It’s architected to be functionally safe, so that if one computer or sensor fails, there is a back up available to ensure that the AV can drive its passengers to a safe place.

The DRIVE Hyperion architecture has been adopted by hundreds of automakers, truck makers, tier 1 suppliers and robotaxi companies, ushering in the new era of autonomy.

Innovation at Scale

Bringing this comprehensive platform architecture to the global automotive ecosystem requires collaboration with leading tier 1 suppliers.

Desay, Flex, Quanta, Valeo and ZF are now DRIVE Hyperion 8 platform scaling partners, manufacturing production-ready designs with the highest levels of functional safety and security.

“We are excited to work with NVIDIA on their DRIVE Hyperion platform,” said Geoffrey Buoquot, CTO and vice president of Strategy at Valeo. “On top of our latest generation ultrasonic sensors providing digital raw data that their AI classifiers can process and our 12 cameras, including the new 8-megapixel cameras, we are now also able to deliver an Orin-based platform to support autonomous driving applications with consistent performance under automotive environmental conditions and production requirements.”

“Flex is thrilled to collaborate with NVIDIA to help accelerate the deployment of autonomous and ADAS systems leveraging the DRIVE Orin platform to design solutions for use across multiple customers,” said Mike Thoeny, president of Automotive at Flex.

The breadth and depth of this supplier ecosystem demonstrate how DRIVE Hyperion has become the industry’s most open and adopted platform architecture. These scaling partners will help make it possible for the platform to start production as early as this year.

New Energy for a New Era

The radical transformation of the transportation industry extends from compute power to powertrains.

Electric vehicles aren’t just better for the environment, they also fundamentally improve the driving experience for consumers. With a quieter and more sustainable profile, EVs will begin to make up the majority of cars sold over the next several decades.

This evolution has given way to dozens of new energy vehicle startups. They have reimagined the car, starting with a new vehicle architecture based on programmable, software-defined compute. These NEVs will continuously improve over time with over-the-air updates.

Many leading NEV makers are adopting DRIVE Hyperion as the platform to develop these clean, intelligent models. From the storied performance heritage of Polestar to the breakthrough success of IM Motors, Li Auto, NIO, R Auto and Xpeng, these companies are reinventing the personal transportation experience.

These companies are benefiting from new, software-driven business models. With a centralized compute architecture, automakers can deliver fresh services and advanced functions throughout the life of the vehicle.

Truly Intelligent Logistics

AI isn’t just transforming personal transportation, it’s also addressing the rapidly growing challenges faced by the trucking and logistics industry.

Over the past decade, and accelerated by the pandemic, consumer shopping has dramatically shifted online, resulting in increased demand for trucking and last-mile delivery. This trend is expected to continue, as experts predict that 170 billion packages will be shipped worldwide this year, climbing to 280 billion in 2027.

At the same time, the trucking industry has been experiencing significant driver shortages, with the U.S. expected to be in need of more than 160,000 drivers by 2028.

TuSimple is facing these challenges head-on with its scalable Autonomous Freight Network. This network uses autonomous trucks to safely and efficiently move goods across the country. And today, the company announced these intelligent trucks will be powered by NVIDIA DRIVE Orin.

The advanced SoC will be at the center of TuSimple’s autonomous domain controller (ADC), which serves as an autonomous truck’s central compute unit that processes hundreds of trillions of operations per second, including mission-critical perception, planning and actuation functions.

As a result, TuSimple can accelerate the development of a high-performance, automotive-grade and scalable ADC that can be integrated into production trucks.

Everyone’s Digital Assistant

Kani also demonstrated the NVIDIA DRIVE Concierge platform, which delivers intelligent services that are always on.

DRIVE Concierge combines NVIDIA Omniverse Avatar, DRIVE IX, DRIVE AV 4D perception, Riva GPU-accelerated speech AI SDK and an array of deep neural networks to delight customers on every drive.

Omniverse Avatar connects speech AI, computer vision, natural language understanding, recommendation engines and simulation. Avatars created on the platform are interactive characters with ray-traced 3D graphics that can see, speak, converse on a wide range of subjects, and understand naturally spoken intent.

During the CES special address, a demo showed highlights of a vehicle equipped with DRIVE Concierge making the trip to Las Vegas, helping the driver prepare for the trip, plan schedules and, most importantly, relax while the car was safely piloted by the DRIVE Chauffeur.

These important developments in intelligent driving technology, as well as innovations from suppliers, automakers and trucking companies all building on NVIDIA DRIVE, are heralding the arrival of the autonomous era.

The post Autonomous Era Arrives at CES 2022 With NVIDIA DRIVE Hyperion and Omniverse Avatar appeared first on The Official NVIDIA Blog.

Read More

GeForce NOW Delivers Legendary GeForce Gaming With More Games on More Networks to More Devices

GeForce NOW is kicking off the new year by bringing more games, more devices and more networks to our cloud gaming ecosystem.

The next pair of Electronic Arts games, Battlefield 4 and Battlefield V, is streaming on GeForce NOW.

We’re also working closely with a few titans in their respective industries: AT&T and Samsung.

AT&T and NVIDIA have joined forces as collaborators in 5G technical innovation to deliver one of the world’s best gaming experiences. To celebrate, a special promotion for certain AT&T 5G subscribers is ongoing.

Going from the small screen to the big screen, look for GeForce NOW to arrive on select Samsung TVs later this year.

Set Foot on the Battlefield

Prepare for impact. Battlefield 4: Premium Edition and Battlefield V: Definitive Edition join the exhilarating collection of Electronic Arts titles streaming from the cloud.

Experience the glorious chaos of all-out war packed with tactical challenges by playing the Battlefield series.

Embrace unrivaled destruction in Battlefield 4: Premium Edition (Steam and Origin). Play across dynamic, interactive battlefield environments that react in real time to your reactions. Trigger a shipwreck, flood the streets for a tactical advantage, or take the edge on massive, chaotic maps with dozens of different vehicles. Plus, take on an intense single-player campaign where you’ll need to evacuate critical American VIPs from Shanghai and battle against the odds to get your squad home in times of international crisis.

Experience the ultimate war experience in Battlefield V: Definitive Edition (Steam and Origin). Enter mankind’s greatest conflict across land, air and sea and immerse yourself in the hard-fought battles of World War II. With all gameplay content unlocked from the get-go, choose from the complete arsenal of weapons, vehicles and gadgets available to stand out on the battlefield with the complete roster of Elites.

Get locked and loaded to play these games and more from Electronic Arts today on GeForce NOW.

Improved Streaming on 5G

Teaming up as 5G technical innovation collaborators, GeForce NOW is bringing the power of PC gaming to gamers with fast, reliable, secure AT&T 5G with a new offer.

Starting in January, AT&T customers with a 5G device on a 5G unlimited plan or another qualifying unlimited plan can get a six-month GeForce NOW Priority membership at no charge (a $49.99 value)*. Priority members enjoy an experience of up to 1080p at 60 frames per second, with priority access to GeForce NOW servers and extended session lengths of up to six hours. They also have access to RTX games.

Streaming to Even More TVs, Soon

NVIDIA is collaborating with Samsung to bring GeForce NOW to its Smart TVs. Our cloud gaming service will be added to the Samsung Gaming Hub, a new game-streaming discovery platform that bridges hardware and software to provide a better player experience.

We’ll have more announcements later this year, but anticipate the streaming experience being available in Q2.

This follows last month’s beta release of the GeForce NOW app for LG 2021 WebOS Smart TVs. The beta app is available in the LG Content Store for certain 2021 models. Stay tuned for updates as we approach a full release and add support for additional TVs, including 2022 models.

2021 delivered a host of great benefits for members, notably the GeForce NOW RTX 3080 membership with 1440p and 120 FPS gameplay streaming from the cloud.

2022 is sure to deliver, so check back every GFN Thursday as we share news on upcoming game launches, new game releases, service updates and more.

*Subject to change. Restrictions apply. See att.com/gaming for more details.

The post GeForce NOW Delivers Legendary GeForce Gaming With More Games on More Networks to More Devices appeared first on The Official NVIDIA Blog.

Read More