Posts in category: Amazon AWS

Amazon SageMaker Autopilot, a low-code machine learning (ML) service that automatically builds, trains, and tunes the best ML models based on tabular data, is now integrated with Amazon SageMaker Pipelines, the first purpose-built continuous integration and continuous delivery (CI/CD) service for ML. This enables the automation of an end-to-end flow of building ML models using Autopilot and integrating models into subsequent CI/CD steps.

So far, to launch an Autopilot experiment within Pipelines, you have to build a model-building workflow by writing custom integration code with Pipelines Lambda or Processing steps. For more information, see Move Amazon SageMaker Autopilot ML models from experimentation to production using Amazon SageMaker Pipelines.

With the support for Autopilot as a native step within Pipelines, you can now add an automated training step (AutoMLStep) in Pipelines and invoke an Autopilot experiment with Ensembling training mode. For example, if you’re building a training and evaluation ML workflow for a fraud detection use case with Pipelines, you can now launch an Autopilot experiment using the AutoML step, which automatically runs multiple trials to find the best model on a given input dataset. After the best model is created using the Model step, its performance can be evaluated on test data using the Transform step and a Processing Step for a custom evaluation script within Pipelines. Eventually, the model can be registered into the SageMaker model registry using the Model step in combination with a Condition step.

In this post, we show how to create an end-to-end ML workflow to train and evaluate a SageMaker generated ML model using the newly launched AutoML step in Pipelines and register it with the SageMaker model registry. The ML model with the best performance can be deployed to a SageMaker endpoint.

Dataset overview

We use the publicly available UCI Adult 1994 Census Income dataset to predict if a person has an annual income of greater than $50,000 per year. This is a binary classification problem; the options for the income target variable are either <=50K or >50K.

The dataset contains 32,561 rows for training and validation and 16,281 rows for testing with 15 columns each. This includes demographic information about individuals and class as the target column indicating the income class.

Solution overview

We use Pipelines to orchestrate different pipeline steps required to train an Autopilot model. We create and run an Autopilot experiment as part of an AutoML step as described in this tutorial.

The following steps are required for this end-to-end Autopilot training process:

• Create and monitor an Autopilot training job using the AutoMLStep.
• Create a SageMaker model using ModelStep. This step fetches the best model’s metadata and artifacts rendered by Autopilot in the previous step.
• Evaluate the trained Autopilot model on a test dataset using TransformStep.
• Compare the output from the previously run TransformStep with the actual target labels using ProcessingStep.
• Register the ML model to the SageMaker model registry using ModelStep, if the previously obtained evaluation metric exceeds a predefined threshold in ConditionStep.
• Deploy the ML model as a SageMaker endpoint for testing purposes.

Architecture

The architecture diagram below illustrates the different pipeline steps necessary to package all the steps in a reproducible, automated, and scalable SageMaker Autopilot training pipeline. The data files are read from the S3 bucket and the pipeline steps are called sequentially.

Walkthrough

This post provides a detailed explanation of the pipeline steps. We review the code and discuss the components of each step. To deploy the solution, refer to the example notebook, which provides step-by-step instructions for implementing an Autopilot MLOps workflow using Pipelines.

Prerequisites

Complete the following prerequisites:

• Set up an AWS account.
• Create an Amazon SageMaker Studio environment.
• Create an AWS Identity and Access Management (IAM) role. For instructions, refer to Creating a role to delegate permissions to an IAM user. Specifically, you create a SageMaker execution role. You may attach the AmazonSageMakerFullAccess managed IAM policy to it for a working demo. However, this should be scoped down further for improved security.
• Navigate to Studio and upload the files from the notebook example directory on GitHub.
• Open the SageMaker notebook sagemaker_autopilot_pipelines_native_auto_ml_step.ipynb and run the cells in sequential order. Follow the instructions on how to initialize the notebook and get a sample dataset.

When the dataset is ready to use, we need to set up Pipelines to establish a repeatable process to automatically build and train ML models using Autopilot. We use the SageMaker SDK to programmatically define, run, and track an end-to-end ML training pipeline.

Pipeline Steps

In the following sections, we go through the different steps in the SageMaker pipeline, including AutoML training, model creation, batch inference, evaluation, and conditional registration of the best model. The following diagram illustrates the entire pipeline flow.

AutoML training step

An AutoML object is used to define the Autopilot training job run and can be added to the SageMaker pipeline by using the AutoMLStep class, as shown in the following code. The ensembling training mode needs to be specified, but other parameters can be adjusted as needed. For example, instead of letting the AutoML job automatically infer the ML problem type and objective metric, these could be hardcoded by specifying the problem_type and job_objective parameters passed to the AutoML object.

automl = AutoML(
    role=execution_role,
    target_attribute_name=target_attribute_name,
    sagemaker_session=pipeline_session,
    total_job_runtime_in_seconds=max_automl_runtime,
    mode="ENSEMBLING",
)
train_args = automl.fit(
    inputs=[
        AutoMLInput(
            inputs=s3_train_val,
            target_attribute_name=target_attribute_name,
            channel_type="training",
        )
    ]
)
step_auto_ml_training = AutoMLStep(
    name="AutoMLTrainingStep",
    step_args=train_args,
)

Model creation step

The AutoML step takes care of generating various ML model candidates, combining them, and obtaining the best ML model. Model artifacts and metadata are automatically stored and can be obtained by calling the get_best_auto_ml_model() method on the AutoML training step. These can then be used to create a SageMaker model as part of the Model step:

best_auto_ml_model = step_auto_ml_training.get_best_auto_ml_model(
    execution_role, sagemaker_session=pipeline_session
)
step_args_create_model = best_auto_ml_model.create(instance_type=instance_type)
step_create_model = ModelStep(name="ModelCreationStep", step_args=step_args_create_model)

Batch transform and evaluation steps

We use the Transformer object for batch inference on the test dataset, which can then be used for evaluation purposes. The output predictions are compared to the actual or ground truth labels using a Scikit-learn metrics function. We evaluate our results based on the F1 score. The performance metrics are saved to a JSON file, which is referenced when registering the model in the subsequent step.

Conditional registration steps

In this step, we register our new Autopilot model to the SageMaker model registry, if it exceeds the predefined evaluation metric threshold.

Create and run the pipeline

After we define the steps, we combine them into a SageMaker pipeline:

pipeline = Pipeline(
    name="AutoMLTrainingPipeline",
    parameters=[
        instance_count,
        instance_type,
        max_automl_runtime,
        model_approval_status,
        model_package_group_name,
        model_registration_metric_threshold,
        s3_bucket,
        target_attribute_name,
    ],
    steps=[
        step_auto_ml_training,
        step_create_model,
        step_batch_transform,
        step_evaluation,
        step_conditional_registration,
    ],
    sagemaker_session=pipeline_session,
)

The steps are run in sequential order. The pipeline runs all the steps for an AutoML job using Autopilot and Pipelines for training, model evaluation, and model registration.

You can view the new model by navigating to the model registry on the Studio console and opening AutoMLModelPackageGroup. Choose any version of a training job to view the objective metrics on the Model quality tab.

You can view the explainability report on the Explainability tab to understand your model’s predictions.

To view the underlying Autopilot experiment for all the models created in AutoMLStep, navigate to the AutoML page and choose the job name.

Deploy the model

After we have manually reviewed the ML model’s performance, we can deploy our newly created model to a SageMaker endpoint. For this, we can run the cells in the notebook that create the model endpoint using the model configuration saved in the SageMaker model registry.

Note that this script is shared for demonstration purposes, but it’s recommended to follow a more robust CI/CD pipeline for production deployment for ML inference. For more information, refer to Building, automating, managing, and scaling ML workflows using Amazon SageMaker Pipelines.

Summary

This post describes an easy-to-use ML pipeline approach to automatically train tabular ML models (AutoML) using Autopilot, Pipelines, and Studio. AutoML improves ML practitioners’ efficiency, accelerating the path from ML experimentation to production without the need for extensive ML expertise. We outline the respective pipeline steps needed for ML model creation, evaluation, and registration. Get started by trying the example notebook to train and deploy your own custom AutoML models.

For more information on Autopilot and Pipelines, refer to Automate model development with Amazon SageMaker Autopilot and Amazon SageMaker Pipelines.

Special thanks to everyone who contributed to the launch: Shenghua Yue, John He, Ao Guo, Xinlu Tu, Tian Qin, Yanda Hu, Zhankui Lu, and Dewen Qi.

About the Authors

Janisha Anand is a Senior Product Manager in the SageMaker Low/No Code ML team, which includes SageMaker Autopilot. She enjoys coffee, staying active, and spending time with her family.

Marcelo Aberle is an ML Engineer at AWS AI. He helps Amazon ML Solutions Lab customers build scalable ML(-Ops) systems and frameworks. In his spare time, he enjoys hiking and cycling in the San Francisco Bay Area.

Geremy Cohen is a Solutions Architect with AWS where he helps customers build cutting-edge, cloud-based solutions. In his spare time, he enjoys short walks on the beach, exploring the bay area with his family, fixing things around the house, breaking things around the house, and BBQing.

Shenghua Yue is a Software Development Engineer at Amazon SageMaker. She focuses on building ML tools and products for customers. Outside of work, she enjoys the outdoors, yoga, and hiking.

Read More

Today we are excited to announce that AI21 Jurassic-1 (J1) foundation models are available for customers using Amazon SageMaker. Jurassic-1 models are highly versatile, capable of both human-like text generation, as well as solving complex tasks such as question answering, text classification, and many others. You can easily try out this model and use it with Amazon SageMaker JumpStart. JumpStart is the machine learning (ML) hub of SageMaker that provides access to foundation models in addition to built-in algorithms and end-to-end solution templates to help you quickly get started with ML.

In this post, we walk through how to use the Jurassic-1 Grande model in SageMaker.

Foundation models in SageMaker

JumpStart provides access to a range of models from popular model hubs including Hugging Face, PyTorch Hub, and TensorFlow Hub, which you can use within your ML development workflow in SageMaker. Recent advances in ML have given rise to a new class of models known as foundation models, which are typically trained on billions of parameters and are adaptable to a wide category of use cases, such as text summarization, generating digital art, and language translation. Because these models are expensive to train, customers want to use existing pre-trained foundation models and fine-tune them as needed, rather than train these models themselves. SageMaker provides a curated list of models that you can choose from on the SageMaker console.

You can now find foundation models from different model providers within JumpStart, enabling you to get started with foundation models quickly. You can find foundation models based on different tasks or model providers, and easily review model characteristics and usage terms. You can also try out these models using a test UI widget. When you want to use a foundation model at scale, you can do so easily without leaving SageMaker by using pre-built notebooks from model providers. Because the models are hosted and deployed on AWS, you can rest assured that your data, whether used for evaluating or using the model at scale, is never shared with third parties.

Jurassic-1 foundation model

Jurassic-1 is the first generation in a series of large language models trained and made widely accessible by AI21 Labs. For a complete description of Jurassic-1, including benchmarks and quantitative comparisons with other models, refer to the following technical paper. All J1 models were trained on a massive corpus of English text, making them highly versatile general purpose text-generators, capable of composing human-like text and solving complex tasks such as question answering, text classification, and many others. J1 can be applied to virtually any language task by crafting a suitable prompt containing a description of the task and a few examples, a process commonly known as prompt engineering. Popular use cases include generating marketing copy, powering chatbots, and assisting creative writing.

“We are building world-class foundation models for text and want to help our customers innovate with the latest Jurassic-1 models. Amazon SageMaker offers the deepest and broadest set of ML services, and we’re excited to collaborate with Amazon SageMaker so that customers will be able to use these foundation models on SageMaker within their development environment. Now customers can rapidly innovate, lower time-to-value, and drive efficiency in their businesses.”

-Ori Goshen, co-CEO of AI21 Labs.

Walkthrough

Let’s take you on a tour to test the J1-Grande model in SageMaker. You can try out the experience in three simple steps:

1. Choose the Jurassic-1 model on the SageMaker console.
2. Evaluate the model using a test widget.
3. Use a notebook associated with the foundation model to deploy it in your environment.

Let’s expand each step in detail.

Choose the Jurassic-1 model on the SageMaker console

The first step is to login to the AWS Management Console for Amazon SageMaker and request access to the list of foundation models from the foundation model category under JumpStart here:

After your account is allow listed, you can see a list of models on this page. You can quickly search for the Jurassic-1 Grande model from the same view.

Evaluate the Jurassic-1 Grande model with a test widget

On the Jurassic-1 Grande listing, choose View Model. You will see a description of the model and the tasks that you can perform. Read through the EULA for the model before proceeding.

Let’s first try out the model for text summarization. Choose Try out model.

You’re taken to the page in a separate browser tab where you can give sample prompts to the J1-Grande model and view the output.

The following example generates a summary about a restaurant based on reviews.

Note that foundation models and their output are from the model provider, and AWS is not responsible for the content or accuracy therein.

The model output may vary depending on the settings and the prompt. You can generate text from the model using simple instructions, but by providing the model with more examples in the prompt, just as a human would, it can produce completions that are more aligned with your intentions. The best way to guide the model is to provide several examples of input/output pairs in the prompt. This establishes a pattern for the model to mimic. Then add the input for a query example and let the model complete it with an appropriate generation.

After you have played with the model, it’s time to use the notebook and deploy it as an endpoint in your environment.

Deploy the foundation model from a notebook

Go back to the model listing shown earlier and choose View notebook. You should see the Jurassic-1 Grande Jupyter notebook with the walkthrough to deploy the model.

Let’s use this notebook from Amazon SageMaker Studio. Open Studio and pull in the notebook using the Git repo URL https://github.com/AI21Labs/SageMaker.git.

The notebook example uses both the Boto3 SDK and the AI21 SDK to deploy and interact with the endpoint.

Note that this example uses an ml.g5.12xlarge instance. If your default limit for your AWS account is 0, you need to request a limit increase for this GPU instance.

Let’s create the endpoint using SageMaker inference. First we set the necessary variables, then we deploy the model from the model package:

model_name = "j1-grande"

content_type = "application/json"

real_time_inference_instance_type = (
    "ml.g5.12xlarge"
)

# create a deployable model from the model package.
model = ModelPackage(
    role=role, model_package_arn=model_package_arn, sagemaker_session=sagemaker_session
)

# Deploy the model
predictor = model.deploy(1, real_time_inference_instance_type, endpoint_name=model_name, 
                         model_data_download_timeout=3600,
                         container_startup_health_check_timeout=600,
                        )

After the endpoint is deployed, you can run inference queries against the model.

You can think of Jurassic-1 Grande as a smart auto-completion algorithm: it’s very good at latching on to hints and patterns expressed in plain English, and generating text that follows the same patterns. After the model is deployed, you can interact with the deployed endpoint using the following code snippet:

response = ai21.Completion.execute(sm_endpoint="j1-grande",
                                   prompt="To be or",
                                   maxTokens=4,
                                   temperature=0,
                                   numResults=1)

print(response['completions'][0]['data']['text'])

The notebook also contains a walkthrough on how you can run inference queries with the AI21 SDK.

The following video walks through the workflow.

Clean up

After you have tested the endpoint, make sure you delete the SageMaker inference endpoint and delete the model to avoid incurring charges.

Conclusion

In this post, we showed you how you can test and use AI21’s Jurassic Grande model using Amazon SageMaker. Request access, try out the foundation model in SageMaker today and let us know your feedback!

About the authors

Karthik Bharathy is the product leader for the Amazon SageMaker team with over a decade of product management, product strategy, execution, and launch experience.

Tomer Asida is an algo team lead at AI21 Labs. As an algo team lead, Tomer heads the algorithm development efforts of our developer platform Ai21 Studio including Jurassic-1 models and associated APIs.

Read More

Artificial intelligence (AI) and machine learning (ML) are some of the most transformative technologies we will encounter in our generation—to tackle business and societal problems, improve customer experiences, and spur innovation. Along with the widespread use and growing scale of AI comes the recognition that we must all build responsibly. At AWS, we think responsible AI encompasses a number of core dimensions including:

• Fairness and bias– How a system impacts different subpopulations of users (e.g., by gender, ethnicity)
• Explainability– Mechanisms to understand and evaluate the outputs of an AI system
• Privacy and Security– Data protected from theft and exposure
• Robustness– Mechanisms to ensure an AI system operates reliably
• Governance– Processes to define, implement and enforce responsible AI practices within an organization
• Transparency– Communicating information about an AI system so stakeholders can make informed choices about their use of the system

Our commitment to developing AI and ML in a responsible way is integral to how we build our services, engage with customers, and drive innovation. We are also committed to providing customers with tools and resources to develop and use AI/ML responsibly, from enabling ML builders with a fully managed development environment to helping customers embed AI services into common business use cases.

Providing customers with more transparency

Our customers want to know that the technology they are using was developed in a responsible way. They want resources and guidance to implement that technology responsibly at their own organization. And most importantly, they want to ensure that the technology they roll out is for everyone’s benefit, especially their end-users’. At AWS, we want to help them bring this vision to life.

To deliver the transparency that customers are asking for, we are excited to launch AWS AI Service Cards, a new resource to help customers better understand our AWS AI services. AI Service Cards are a form of responsible AI documentation that provide customers with a single place to find information on the intended use cases and limitations, responsible AI design choices, and deployment and performance optimization best practices for our AI services. They are part of a comprehensive development process we undertake to build our services in a responsible way that addresses fairness and bias, explainability, robustness, governance, transparency, privacy, and security. At AWS re:Invent 2022 we’re making the first three AI Service Cards available: Amazon Rekognition – Face Matching, Amazon Textract – AnalyzeID, and Amazon Transcribe – Batch (English-US).

Components of the AI Service Cards

Each AI Service Card contains four sections covering:

• Basic concepts to help customers better understand the service or service features
• Intended use cases and limitations
• Responsible AI design considerations
• Guidance on deployment and performance optimization

The content of the AI Service Cards addresses a broad audience of customers, technologists, researchers, and other stakeholders who seek to better understand key considerations in the responsible design and use of an AI service.

Our customers use AI in an increasingly diverse set of applications. The intended use cases and limitations section provides information about common uses for a service, and helps customers assess whether a service is a good fit for their application. For example, in the Amazon Transcribe – Batch (English-US) Card we describe the service use case of transcribing general-purpose vocabulary spoken in US English from an audio file. If a company wants a solution that automatically transcribes a domain-specific event, such as an international neuroscience conference, they can add custom vocabularies and language models to include scientific vocabulary in order to increase the accuracy of the transcription.

In the design section of each AI Service Card, we explain key responsible AI design considerations across important areas, such as our test-driven methodology, fairness and bias, explainability, and performance expectations. We provide example performance results on an evaluation dataset that is representative of a common use case. This example is just a starting point though, as we encourage customers to test on their own datasets to better understand how the service will perform on their own content and use cases in order to deliver the best experience for their end customers. And this is not a one-time evaluation. To build in a responsible way, we recommend an iterative approach where customers periodically test and evaluate their applications for accuracy or potential bias.

In the best practices for deployment and performance optimization section, we lay out key levers that customers should consider to optimize the performance of their application for real-world deployment. It’s important to explain how customers can optimize the performance of an AI system that acts as a component of their overall application or workflow to get the maximum benefit. For example, in the Amazon Rekognition Face Matching Card that covers adding face recognition capabilities to identity verification applications, we share steps customers can take to increase the quality of the face matching predictions incorporated into their workflow.

Delivering responsible AI resources and capabilities

Offering our customers the resources and tools they need to transform responsible AI from theory to practice is an ongoing priority for AWS. Earlier this year we launched our Responsible Use of Machine Learning guide that provides considerations and recommendations for responsibly using ML across all phases of the ML lifecycle. AI Service Cards complement our existing developer guides and blog posts, which provide builders with descriptions of service features and detailed instructions for using our service APIs. And with Amazon SageMaker Clarify and Amazon SageMaker Model Monitor, we offer capabilities to help detect bias in datasets and models and better monitor and review model predictions through automation and human oversight.

At the same time, we continue to advance responsible AI across other key dimensions, such as governance. At re:Invent today we launched a new set of purpose-built tools to help customers improve governance of their ML projects with Amazon SageMaker Role Manager, Amazon SageMaker Model Cards, and Amazon SageMaker Model Dashboard. Learn more on the AWS News blog and website about how these tools help to streamline ML governance processes.

Education is another key resource that helps advance responsible AI. At AWS we are committed to building the next generation of developers and data scientists in AI with the AI and ML Scholarship Program and AWS Machine Learning University (MLU). This week at re:Invent we launched a new, public MLU course on fairness considerations and bias mitigation across the ML lifecycle. Taught by the same Amazon data scientists who train AWS employees on ML, this free course features 9 hours of lectures and hands-on exercises and it is easy to get started.

AI Service Cards: A new resource—and an ongoing commitment

We are excited to bring a new transparency resource to our customers and the broader community and provide additional information on the intended uses, limitations, design, and optimization of our AI services, informed by our rigorous approach to building AWS AI services in a responsible way. Our hope is that AI Service Cards will act as a useful transparency resource and an important step in the evolving landscape of responsible AI. AI Service Cards will continue to evolve and expand as we engage with our customers and the broader community to gather feedback and continually iterate on our approach.

Contact our group of responsible AI experts to start a conversation.

About the authors

Vasi Philomin is currently a Vice President in the AWS AI team for services in the language and speech technologies areas such as Amazon Lex, Amazon Polly, Amazon Translate, Amazon Transcribe/Transcribe Medical, Amazon Comprehend, Amazon Kendra, Amazon Code Whisperer, Amazon Monitron, Amazon Lookout for Equipment and Contact Lens/Voice ID for Amazon Connect as well as Machine Learning Solutions Lab and Responsible AI.

Peter Hallinan leads initiatives in the science and practice of Responsible AI at AWS AI, alongside a team of responsible AI experts. He has deep expertise in AI (PhD, Harvard) and entrepreneurship (Blindsight, sold to Amazon). His volunteer activities have included serving as a consulting professor at the Stanford University School of Medicine, and as the president of the American Chamber of Commerce in Madagascar. When possible, he’s off in the mountains with his children: skiing, climbing, hiking and rafting

Read More

Over the last 5 years, artificial intelligence (AI) and machine learning (ML) have evolved from a niche activity to a rapidly growing mainstream endeavor. Today, more than 100,000 customers across numerous industries rely on AWS for ML and AI initiatives that infuse AI into a broad range of business use cases to automate repetitive and mundane tasks—from intelligent demand planning to document processing and content moderation. AWS AI services help customers create smoother, faster, and more efficient engagements with customers, driving greater efficiencies and lowering operational costs.

At AWS re:Invent, Amazon Web Services, Inc. has announced a series of features and enhancements across its portfolio of AI services, including purpose-built solutions to solve industry-specific challenges, representing a deeper integration of AI into everyday experiences. The new capabilities include Amazon Textract Analyze Lending to improve loan-document processing efficiency, Amazon Transcribe Call Analytics to analyze in-progress contact center calls, Amazon Kendra support for tabular search in HTML and seven new languages, Amazon HealthLake Imaging for medical image storing; Amazon HealthLake Analytics with multi-modal data querying capabilities, and broader programming languages support and easier administration in Amazon CodeWhisperer. These AI service innovations provide vertical markets and horizontal functions with deeper, real-time insights and cost-saving efficiencies to drive transformation across industries.

These new capabilities enhance AWS’s AI offerings at the top of its three-layer ML stack. The bottom layer includes foundational components (ML hardware and ML software libraries) to help customers build their own ML infrastructure, and the middle layer—Amazon SageMaker—is a fully managed ML development environment. The top layer of AI services brings ML to business use cases such as transcribing contact center calls, processing documents, and improving healthcare outcomes. Customers can use AWS AI services with no ML expertise required.

Customers from different industries rely on AWS AI services to improve efficiency and reduce operational costs. For example, WaFd Bank, a full-service US bank, improved its customer experience with Talkdesk (a global cloud contact center company) and AWS Contact Center Intelligence (CCI) solutions, reducing call times by up to 90%. And State Auto, a property and casualty insurance holding company, automated the property inspection process using Amazon Rekognition (a computer vision service), increasing the number of claims it reviews for potential fraud by 83%.

Amazon Textract Analyze Lending makes it easy to classify and extract mortgage loan data

Today, mortgage companies process large volumes of documents to extract business-critical data and make decisions on loan applications. For example, a typical US mortgage application can encompass 500 or more pages of diverse document types, including W2 forms, paystubs, bank statements, Form 1040, 1003, and many more. The lender’s loan processing application has to first understand and classify each document type to ensure that it is processed the right way. After that, the loan processing application has to extract all the data on each page of the document. The data in these documents exists in different formats and structures, and the same data element can have different names on different documents—for example, “SSN,” or “Social Security Number,” which can lead to inaccurate data extraction. So far, the classification and extraction of data from mortgage application packages have been primarily manual tasks. Furthermore, mortgage companies have to manage demand for mortgages that can fluctuate substantially during a year, so lenders are unable to plan effectively and must often allocate resources to process documents on an ad hoc basis. Overall, mortgage loan processing is still manual, slow, error-prone, and expensive.

Amazon Textract (AWS’s AI service to automatically extract text, handwriting, and data from scanned documents) now offers Amazon Textract Analyze Lending to make loan document processing more automated, faster, and cost-effective at scale. Amazon Textract Analyze Lending pulls together multiple ML models to classify various documents that commonly occur in mortgage packages, and then extracts critical information from these documents with high accuracy to improve loan document processing workflows. For example, it can now perform signature detection to identify whether documents have required signatures. It also provides a summary of the documents in a mortgage application package and identifies any missing documents. For instance, PennyMac, a financial services firm specializing in the production and servicing of US mortgage loans, uses Amazon Textract Analyze Lending to process a 3,000-page mortgage application in less than 5 minutes. Previously, PennyMac’s mortgage document processing required several hours of reviewing and preparing a loan package for approval.

Amazon Transcribe Call Analytics for improved end-user experiences

In most customer-facing industries such as telecom, finance, healthcare, and retail, customer experiences with call centers can profoundly impact perceptions of the company. Lengthy call-resolution times or the inability to deal with issues during live interactions can lead to poor customer experiences or customer churn. Contact centers need real-time insights into customer-experience issues (e.g., a product defect) while calls are in progress. Typically, developers use multiple AI services to generate live call transcriptions, extract relevant real-time insights, and manage sensitive customer information (e.g. identify and redact sensitive customer details) during live calls. However, this process adds unnecessary complexity, time, and cost.

Amazon Transcribe, an automatic speech recognition (ASR) service that makes it easy for developers to add speech-to-text capabilities to their applications, now supports call analytics to provide real-time conversation insights. Amazon Transcribe Call Analytics now provides real-time conversation insights that help analyze thousands of in-progress calls, identify call sentiment (e.g. calls that ended with a negative customer sentiment score), detect the potential reason for the call, and spot issues such as repeated requests to speak to a manager. Amazon Transcribe Call Analytics combines powerful automatic speech NLP models that are trained specifically to improve overall customer experience. With Amazon Transcribe Call Analytics, developers can build a real-time system that provides contact center agents with relevant information to solve customer issues or alert supervisors about potential issues. Amazon Transcribe Call Analytics also generates call summaries automatically, eliminating the need for agents to take notes and allowing them to focus on customer needs. Furthermore, Amazon Transcribe Call Analytics protects sensitive customer data by identifying and redacting personal information during live calls.

Amazon Kendra adds new search capabilities

Today, in the face of rapid growth in the volume and variety of data, enterprise search tools struggle to examine and uncover key insights stored across enterprise systems in heterogenous data formats and in different languages. Conventional enterprise search solutions are unable to find knowledge stored in unstructured datasets like HTML tables because it requires extracting information from two-dimensional formats (rows and columns). Sometimes, the information a customer may be seeking could exist in different languages, making the search even more challenging. As a result, enterprise employees waste time searching for information or are unable to perform their duties.

Amazon Kendra (AWS’s intelligent search service powered by ML) offers a new capability that supports tabular search in HTML. Customers can find more precise answers faster in HTML documents, whether they’re in the narrative body or tabular form, by using natural language questions. Amazon Kendra can find and extract precise answers from HTML tables by performing deeper analyses of HTML pages and using new specialized deep learning models that intelligently interpret columns and rows to pinpoint relevant data. Amazon Kendra is also adding semantic support for seven new languages (in addition to English): French, Spanish, German, Portuguese, Japanese, Korean, and Chinese. Customers can now ask natural language questions and get exact answers in any of the supported languages. One of AWS’s biopharmaceutical customers, Gilead Sciences Inc., increased staff productivity by cutting internal search times by roughly 50% using Amazon Kendra.

Amazon HealthLake offers next-generation imaging solutions and precision health analytics

Healthcare providers face a myriad of challenges as the scale and complexity of medical imaging data continues to increase. Medical imaging is a critical tool to diagnose patients, and there are billions of medical images scanned globally each year. Imaging data accounts for about 90% ¹ of all healthcare data, and analyzing these complex images has largely been a manual task performed by experts and specialists. It often takes data scientists and researchers weeks or months to derive important insights from medical images, slowing down decision-making processes for healthcare providers and impacting patient-care delivery. To address these challenges, Amazon HealthLake (a HIPAA-eligible service to store, transform, query, and analyze large-scale health data) is adding two new capabilities for medical imaging and analytics:

• Amazon HealthLake Imaging is a new HIPAA-eligible capability that enables healthcare providers and their software partners to easily store, access, and analyze medical images at petabyte scale. The new capability is designed for fast, subsecond image retrieval in clinical workflows that healthcare providers can access securely from anywhere (e.g., web, desktop, or phone) and with high availability. Typically, health systems store multiple copies of the same imaging data in clinical and research systems, leading to increased storage costs and complexity. Amazon HealthLake Imaging extracts and stores just one copy of the same image to the cloud. Customers can now access existing medical records and run analysis applications from a single encrypted copy of the same data in the cloud with normalized metadata and advanced compression. As a result, Amazon HealthLake Imaging can help providers reduce the total cost of medical imaging storage by up to 40%.
• Amazon HealthLake Analytics is a new HIPAA-eligible capability that makes it easy to query and derive insights from multi-modal health data (e.g., imaging, text, or genetics), at the individual or population levels, with the ability to share data securely across the enterprise. It removes the need for healthcare providers to execute complex data exports and data transformations. Amazon HealthLake Analytics automatically normalizes raw health data from disparate sources (e.g., medical records, health insurance claims, EHRs, or medical devices) into an analytics and interoperable format in minutes. The new capability reduces what would otherwise take months of engineering effort to allow providers to focus on what they do best—delivering patient care.

Amazon CodeWhisperer offers broader support and easier administration

While the cloud has democratized application development through on-demand access to compute, storage, database, analytics, and ML, the traditional process of building software applications in any industry remains time-intensive. Developers must still spend significant time writing repetitive code not directly related to the core problems they want to solve. Even highly experienced developers find it difficult to keep up with multiple programming languages, frameworks, and software libraries, while ensuring they follow correct programming syntax and coding best practices.

Amazon CodeWhisperer (an ML-powered service that generates code recommendations) now supports AWS Builder ID so any developer can sign up securely with just an email address and enable Amazon CodeWhisperer for their IDE within the AWS Toolkit. In addition to Python, Java, and JavaScript, Amazon CodeWhisperer adds support for TypeScript and C# languages to accelerate code development. Also, Amazon CodeWhisperer now makes code recommendations for AWS application programming interfaces (APIs) across its most popular services, including Amazon Elastic Compute Cloud (Amazon EC2), AWS Lambda, and Amazon Simple Storage Service (Amazon S3). Finally, Amazon CodeWhisperer is now available on the AWS Management Console, so any authorized AWS administrator can enable Amazon CodeWhisperer for their organization.

Conclusion

With these new features and capabilities, AWS continues to expand its portfolio of the broadest and deepest set of AI services. AWS also recognizes that as AI-powered use cases become pervasive, it is important that these capabilities are built in a responsible way. AWS is committed to building its services in a responsible manner and supporting customers to help them deploy AI responsibly. By enabling customers to more easily and responsibly add new and expanded AI capabilities to their applications and workflows, AWS is unleashing even greater innovation and helping businesses reimagine how they approach and solve some of their most pressing challenges. To learn more about AWS’s comprehensive approach to responsible AI, visit Responsible use of artificial intelligence and machine learning.

References

¹S. K. Zhou et al., “A Review of Deep Learning in Medical Imaging: Imaging Traits, Technology Trends, Case Studies With Progress Highlights, and Future Promises,” in Proceedings of the IEEE, vol. 109, no. 5, pp. 820-838, May 2021, doi: 10.1109/JPROC.2021.3054390.

About the Author

Bratin Saha is the Vice President of Artificial Intelligence and Machine Learning at AWS.

Read More

In 2019, Amazon co-founded the climate pledge. The pledge’s goal is to achieve net zero carbon by 2040. This is 10 years earlier than the Paris agreement outlines. Companies who sign up are committed to regular reporting, carbon elimination, and credible offsets. At the time of this writing, 377 companies have signed the climate pledge, and the number is still growing.

Because AWS is committed to helping you achieve your net zero goal through cloud solutions and machine learning (ML), many projects have already been developed and deployed that reduce carbon emissions. Manufacturing is one of the industries that can benefit greatly from such projects. Through optimized energy management of machines in manufacturing factories, such as compressors or chillers, companies can reduce their carbon footprint with ML.

Effectively transitioning from an ML experimentation phase to production is challenging. Automating model training and retraining, having a model registry, and tracking experiments and deployment are some of the key challenges. For manufacturing companies, there is another layer of complexity, namely how these deployed models can run at the edge.

In this post, we address these challenges by providing a machine learning operations (MLOps) template that hosts a sustainable energy management solution. The solution is agnostic to use cases, which means you can adapt it for your use cases by changing the model and data. We show you how to integrate models in Amazon SageMaker Pipelines, a native workflow orchestration tool for building ML pipelines, which runs a training job and optionally a processing job with a Monte Carlo Simulation. Experiments are tracked in Amazon SageMaker Experiments. Models are tracked and registered in the Amazon SageMaker model registry. Finally, we provide code for deployment of your final model in an AWS Lambda function.

Lambda is a compute service that lets you run code without managing or provisioning servers. Lambda’s automatic scaling, pay-per-request billing, and ease of use make it a common deployment choice for data science teams. With this post, data scientists can turn their model into a cost-effective and scalable Lambda function. Furthermore, Lambda allows for integration with AWS IoT Greengrass, which helps you build software that enables your devices to act at the edge on the data that they generate, as would be the case for a sustainable energy management solution.

Solution overview

The architecture we deploy (see the following figure) is a fully CI/CD-driven approach to machine learning. Elements are decoupled to avoid having one monolithic solution.

Let’s start with the top left of the diagram. The Processing – Image build component is a CI/CD-driven AWS CodeCommit repository that helps build and push a Docker container to Amazon Elastic Container Registry (Amazon ECR). This processing container serves as the first step in our ML pipeline, but it’s also reused for postprocessing steps. In our case, we apply a Monte Carlo Simulation as postprocessing. The Training – Image build repository outlined on the bottom left has the same mechanism as the Processing block above it. The main difference is that it builds the container for model training.

The main pipeline, Model building (Pipeline), is another CodeCommit repository that automates running your SageMaker pipelines. This pipeline automates and connects the data preprocessing, model training, model metrics tracking in SageMaker Experiments, data postprocessing, and, model cataloging in SageMaker model registry.

The final component is on the bottom right: Model deployment. If you follow the examples in Amazon SageMaker Projects, you get a template that hosts your model using a SageMaker endpoint. Our deployment repository instead hosts the model in a Lambda function. We show an approach for deploying the Lambda function that can run real-time predictions.

Prerequisites

To deploy our solution successfully, you need the following:

• An AWS account.
• An AWS Cloud Development Kit (AWS CDK) installation. For this post, we use Python.
• An AWS Command Line Interface (AWS CLI) installation.
• An Amazon SageMaker Studio domain with a user set up.

Download the GitHub repository

As a first step, clone the GitHub repository to your local machine. It contains the following folder structure:

• deployment – Contains code relevant for deployment
• mllib — Contains ML code for preprocessing, training, serving, and simulating
• tests — Contains unit and integration tests

The key file for deployment is the shell script deployment/deploy.sh. You use this file to deploy the resources in your account. Before we can run the shell script, complete the following steps:

1. Open the deployment/app.py and change the bucket_name under SageMakerPipelineSourceCodeStack. The bucket_name needs to be globally unique (for example, add your full name).
2. In deployment/pipeline/assets/modelbuild/pipelines/energy_management/pipeline.py, change the default_bucket under get_pipeline to the same name as specified in step 1.

Deploy solution with the AWS CDK

First, configure your AWS CLI with the account and Region that you want to deploy in. Then run the following commands to change to the deployment directory, create a virtual environment, activate it, install the required pip packages specified in setup.py, and run the deploy.sh:

cd deployment
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pre-commit install
chmod u+x deploy.sh
./deploy.sh

deploy.sh performs the following actions:

1. Creates a virtual environment in Python.
2. Sources the virtual environment activation script.
3. Installs the AWS CDK and the requirements outlined in setup.py.
4. Bootstraps the environment.
5. Zips and copies the necessary files that you developed, such as your mllib files, into the corresponding folders where these assets are needed.
6. Runs cdk deploy —require-approval never.
7. Creates an AWS CloudFormation stack through the AWS CDK.

The initial stage of the deployment should take less than 5 minutes. You should now have four repositories in CodeCommit in the Region you specified through the AWS CLI, as outlined in the architecture diagram. The AWS CodePipeline pipelines are run simultaneously. The modelbuild and modeldeploy pipelines depend on a successful run of the processing and training image build. The modeldeploy pipeline depends on a successful model build. The model deployment should be complete in less than 1.5 hours.

Clone the model repositories in Studio

To customize the SageMaker pipelines created through the AWS CDK deployment in the Studio UI, you first need to clone the repositories into Studio. Launch the system terminal in Studio and run the following commands after providing the project name and ID:

git clone https://git-codecommit.REGION.amazonaws.com/v1/repos/sagemaker-PROJECT_NAME-PROJECT_ID-modelbuild
git clone https://git-codecommit.REGION.amazonaws.com/v1/repos/sagemaker-PROJECT_NAME-PROJECT_ID-modeldeploy
git clone https://git-codecommit.REGION.amazonaws.com/v1/repos/sagemaker-PROJECT_NAME-PROJECT_ID-processing-imagebuild
git clone https://git-codecommit.REGION.amazonaws.com/v1/repos/sagemaker-PROJECT_NAME-PROJECT_ID-training-imagebuild

After cloning the repositories, you can push a commit to the repositories. These commits trigger a CodePipeline run for the related pipelines.

You can also adapt the solution on your local machine and work on your preferred IDE.

Navigate the SageMaker Pipelines and SageMaker Experiments UI

A SageMaker pipeline is a series of interconnected steps that are defined using the Amazon SageMaker Python SDK. This pipeline definition encodes a pipeline using a Directed Acyclic Graph (DAG) that can be exported as a JSON definition. To learn more about the structure of such pipelines, refer to SageMaker Pipelines Overview.

Navigate to SageMaker resources pane and choose the Pipelines resource to view. Under Name, you should see PROJECT_NAME-PROJECT_ID. In the run UI, there should be a successful run that is expected to take a little over 1 hour. The pipeline should look as shown in the following screenshot.

The run was automatically triggered after the AWS CDK stack was deployed. You can manually invoke a run by choosing Create execution. From there you can choose your own pipeline parameters such as the instance type and number of instances for the processing and training steps. Furthermore, you can give the run a name and description. The pipeline is highly configurable through pipeline parameters that you can reference and define throughout your pipeline definition.

Feel free to start another pipeline run with your parameters as desired. Afterwards, navigate to the SageMaker resources pane again and choose Experiments and trials. There you should again see a line with a name such as PROJECT_NAME-PROJECT_ID. Navigate to the experiment and choose the only run with a random ID. From there, choose the SageMaker training job to explore the metrics related to the training Job.

The goal of SageMaker Experiments is to make it as simple as possible to create experiments, populate them with trials, and run analytics across trials and experiments. SageMaker Pipelines are closely integrated with SageMaker Experiments, and by default for each run create an experiment, trial and trial components in case they do not exist.

Approve Lambda deployment in the model registry

As a next step, navigate to the model registry under SageMaker resources. Here you can find again a line with a name such as PROJECT_NAME-PROJECT_ID. Navigate to the only model that exists and approve it. This automatically deploys the model artifact in a container in Lambda.

After you approve your model in the model registry, an Amazon EventBridge event rule is triggered. This rule runs the CodePipeline pipeline with the ending *-modeldeploy. In this section, we discuss how this solution uses the approved model and hosts it in a Lambda function. CodePipeline takes the existing CodeCommit repository also ending with *-modeldeploy and uses that code to run in CodeBuild. The main entry for CodeBuild is the buildspec.yml file. Let’s look at this first:

version: 0.2

env:
  shell: bash

phases:
  install:
    runtime_versions:
      python: 3.8
    commands:
      - python3 -m ensurepip --upgrade
      - python3 -m pip install --upgrade pip
      - python3 -m pip install --upgrade virtualenv
      - python3 -m venv .venv
      - source .venv/bin/activate
      - npm install -g aws-cdk@2.26.0
      - pip install -r requirements.txt
      - cdk bootstrap
  build:
    commands:
      - python build.py --model-package-group-name "$SOURCE_MODEL_PACKAGE_GROUP_NAME"
      - tar -xf model.tar.gz
      - cp model.joblib lambda/digital_twin
      - rm model.tar.gz
      - rm model.joblib
      - cdk deploy --require-approval never

During the installation phase, we ensure that the Python libraries are up to date, create a virtual environment, install AWS CDK v2.26.0, and install the aws-cdk Python library along with others using the requirements file. We also bootstrap the AWS account. In the build phase, we run build.py, which we discuss next. That file downloads the latest approved SageMaker model artifact from Amazon Simple Storage Service (Amazon S3) to your local CodeBuild instance. This .tar.gz file is unzipped and its contents copied into the folder that also contains our main Lambda code. The Lambda function is deployed using the AWS CDK, and code runs out of a Docker container from Amazon ECR. This is done automatically by AWS CDK.

The build.py file is a Python file that mostly uses the AWS SDK for Python (Boto3) to list the model packages available.

The function get_approved_package returns the Amazon S3 URI of the artifact that is then downloaded, as described earlier.

After successfully deploying your model, you can test it directly on the Lambda console in the Region you chose to deploy in. The name of the function should contain DigitalTwinStack-DigitalTwin*. Open the function and navigate to the Test tab. You can use the following event to run a test call:

{
  "flow": "[280, 300]",
  "pressure": "[69, 70]",
  "simulations": "10",
  "no_of_trials": "10",
  "train_error_weight": "1.0"
}

After running the test event, you get a response similar to that shown in the following screenshot.

If you want to run more simulations or trials, you can increase the Lambda timeout limit and experiment with the code! Or you might want to pick up the data generated and visualize the same in Amazon QuickSight. Below is an example. It’s your turn now!

Clean up

To avoid further charges, complete the following steps:

• On the AWS CloudFormation console, delete the EnergyOptimization stack.
  This deletes the entire solution.
• Delete the stack DigitalTwinStack, which deployed your Lambda function.

Conclusion

In this post, we showed you a CI/CD-driven MLOps pipeline of an energy management solution where we keep each step decoupled. You can track your ML pipelines and experiments in the Studio UI. We also demonstrated a different deployment approach: upon approval of a model in the model registry, a Lambda function hosting the approved model is built automatically through CodePipeline.

If you’re interested in exploring either the MLOps pipeline on AWS or the sustainable energy management solution, check out the GitHub repository and deploy the stack in your own AWS environment!

About the Authors

Laurens van der Maas is a Data Scientist at AWS Professional Services. He works closely with customers building their machine learning solutions on AWS, and is passionate about how machine learning is changing the world as we know it.

Kangkang Wang is an AI/ML consultant with AWS Professional Services. She has extensive experience of deploying AI/ML solutions in healthcare and life sciences vertical. She also enjoys helping enterprise customers to build scalable AI/ML platforms to accelerate the cloud journey of their data scientists.

Selena Tabbara is a Data Scientist at AWS Professional Services. She works everyday with her customers to achieve their business outcomes by innovating on AWS platforms. In her spare time, Selena enjoys playing the piano, hiking and watching basketball.

Michael Wallner is a Senior Consultant with focus on AI/ML with AWS Professional Services. Michael is passionate about enabling customers on their cloud journey to become AWSome. He is excited about manufacturing and enjoys helping transform the manufacturing space through data.

Read More

Amazon Kendra is an intelligent search service powered by machine learning (ML). Kendra reimagines enterprise search for your websites and applications so your employees and customers can easily find the content they’re looking for, even when it’s scattered across multiple locations and content repositories within your organization.

Amazon Kendra users can now quickly find the information they need from tables on a webpage (HTML tables) using Amazon Kendra tabular search. Tables contains useful information in structured format so it can be easily interpreted by making visual associations between row and column headers. With Amazon Kendra tabular search, you can now get specific information from the cell or certain rows and columns relevant to your query, as well as preview of the table.

In this post, we provide an example of how to use Amazon Kendra tabular search.

Tabular search in Amazon Kendra

Let’s say you have a webpage in HTML format that contains a table with inflation rates and annual changes in the US from 2012–2021, as shown in the following screenshot.

When you search for “Inflation rate in US”, Amazon Kendra presents the top three rows in the preview and up to five columns, as shown in the following screenshot. You can then see if this article has the relevant details that you’re looking for and decide to either use this information or open the link to get additional details. Amazon Kendra tabular search can also handle merged rows.

Let’s do another search and get specific information from the table by asking “What was the annual change of inflation rate in 2017?”. As shown in the following screenshot, Amazon Kendra tabular search highlights the specific cell that contains the answer to your question.

Now let’s search for “Which year had top inflation rate?”, Amazon Kendra searches the table, sorts the results, and gives you the year that had the highest inflation rate.

Amazon Kendra can also find the range of column information that you’re looking for. For example, let’s search for “Inflation rate from 2012 and 2014.” Amazon Kendra displays the rows and columns between 2012–2014 in the preview.

Get started with Amazon Kendra tabular search

Amazon Kendra tabular search is turned on by default and no special configuration is required to enable it. For newer documents, Amazon Kendra tabular search will work by default. For existing HTML pages that contain tables, you can either update the document and sync (if you only have a few documents), or reach out to AWS Support .

To test tabular search on your internal or external webpage, complete the following steps:

1. Create an index.
2. Add data sources by using the web crawler or downloading the HTML page and uploading it to an Amazon Simple Storage Service (Amazon S3) bucket.
3. Go to the Search Indexed Content tab and test it out.

Limitations and considerations

Keep the following in mind when using this feature:

• In this release, Amazon Kendra only supports HTML formatted tables or HTML tables within the table tag. This doesn’t include nested tables or other forms of tables.
• Amazon Kendra can search through tables up to 30 columns and 60 rows, and up to 500 total table cells. If you have a table with a higher numbers of rows, columns, or table cells, Amazon Kendra will not search within that table.
• Amazon Kendra doesn’t display tabular search results if the confidence score of query result for the column and row is very low. You can look at the confidence score within ScoreAttributes using the QueryResultItem API.

Conclusion

With Amazon Kendra’s tabular search for HTML in Amazon Kendra, you can now search across both unstructured data from various data sources and structured data in the form of tables. This further enhances the user experiences and you can get factual responses from your natural language query as well as from the tables. The table preview with Kendra’s suggested answers allows you to quickly asses if the HTML document table contains relevant information you are looking for, thereby saving time.

Amazon Kendra tabular search is available in the following AWS regions during launch: US East (N. Virginia), US East (Ohio), US West (Oregon), Europe (Ireland), Asia Pacific (Sydney), Asia Pacific (Singapore), Canada (Central) and AWS GovCloud (US-West).

To learn more about Amazon Kendra, visit the Amazon Kendra product page.

About the authors

Vikas Shah is an Enterprise Solutions Architect at Amazon web services. He is a technology enthusiast who enjoys helping customers find innovative solutions to complex business challenges. His areas of interest are ML, IoT, robotics and storage. In his spare time, Vikas enjoys building robots, hiking, and traveling.

Read More