Monthly Archives: July 2024

Amazon Forecast is a fully managed service that uses statistical and machine learning (ML) algorithms to deliver highly accurate time series forecasts. Launched in August 2019, Forecast predates Amazon SageMaker Canvas, a popular low-code no-code AWS tool for building, customizing, and deploying ML models, including time series forecasting models.

With SageMaker Canvas, you get faster model building, cost-effective predictions, advanced features such as a model leaderboard and algorithm selection, and enhanced transparency. You can also either use the SageMaker Canvas UI, which provides a visual interface for building and deploying models without needing to write any code or have any ML expertise, or use its automated machine learning (AutoML) APIs for programmatic interactions.

In this post, we provide an overview of the benefits SageMaker Canvas offers and details on how Forecast users can transition their use cases to SageMaker Canvas.

Benefits of SageMaker Canvas

Forecast customers have been seeking greater transparency, lower costs, faster training, and enhanced controls for building time series ML models. In response to this feedback, we have made next-generation time series forecasting capabilities available in SageMaker Canvas, which already offers a robust platform for preparing data and building and deploying ML models. With the addition of forecasting, you can now access end-to-end ML capabilities for a broad set of model types—including regression, multi-class classification, computer vision (CV), natural language processing (NLP), and generative artificial intelligence (AI)—within the unified user-friendly platform of SageMaker Canvas.

SageMaker Canvas offers up to 50% faster model building performance and up to 45% quicker predictions on average for time series models compared to Forecast across various benchmark datasets. Generating predictions is  significantly more cost-effective than Forecast, because costs are based solely on the Amazon SageMaker compute resources used. SageMaker Canvas also provides excellent model transparency by offering direct access to trained models, which you can deploy at your chosen location, along with numerous model insight reports, including access to validation data, model- and item-level performance metrics, and hyperparameters employed during training.

SageMaker Canvas includes the key capabilities found in Forecast, including the ability to train an ensemble of forecasting models using both statistical and neural network algorithms. It creates the best model for your dataset by generating base models for each algorithm, evaluating their performance, and then combining the top-performing models into an ensemble. This approach leverages the strengths of different models to produce more accurate and robust forecasts. You have the flexibility to select one or several algorithms for model creation, along with the capability to evaluate the impact of model features on prediction accuracy. SageMaker Canvas simplifies your data preparation with automated solutions for filling in missing values, making your forecasting efforts as seamless as possible. It facilitates an out-of-the-box integration of external information, such as country-specific holidays, through simple UI options or API configurations. You can also take advantage of its data flow feature to connect with external data providers’ APIs to import data, such as weather information. Furthermore, you can conduct what-if analyses directly in the SageMaker Canvas UI to explore how various scenarios might affect your outcomes.

We will continue to innovate and deliver cutting-edge, industry-leading forecasting capabilities through SageMaker Canvas by lowering latency, reducing training and prediction costs, and improving accuracy. This includes expanding the range of forecasting algorithms we support and incorporating new advanced algorithms to further enhance the model building and prediction experience.

Transitioning from Forecast to SageMaker Canvas

Today, we’re releasing a transition package comprising two resources to help you transition your usage from Forecast to SageMaker Canvas. The first component includes a workshop to get hands-on experience with the SageMaker Canvas UI and APIs and to learn how to transition your usage from Forecast to SageMaker Canvas. We also provide a Jupyter notebook that shows how to transform your existing Forecast training datasets to the SageMaker Canvas format.

Before we learn how to build forecast models in SageMaker Canvas using your Forecast input datasets, let’s understand some key differences between Forecast and SageMaker Canvas:

• Dataset types – Forecast uses multiple datasets – target time series, related time series (optional), and item metadata (optional). In contrast, SageMaker Canvas requires only one dataset, eliminating the need for managing multiple datasets.
  
• Model invocation – SageMaker Canvas allows you to invoke the model for a single dataset or a batch of datasets using the UI as well as the APIs. Unlike Forecast, which requires you to first create a forecast and then query it, you simply use the UI or API to invoke the endpoint where the model is deployed to generate forecasts. The SageMaker Canvas UI also gives you the option to deploy the model for inference on SageMaker real-time endpoints. With just a few clicks, you can receive an HTTPS endpoint that can be invoked from within your application to generate forecasts.

In the following sections, we discuss the high-level steps for transforming your data, building a model, and deploying a model using SageMaker Canvas using either the UI or APIs.

Build and deploy a model using the SageMaker Canvas UI

We recommend reorganizing your data sources to directly create a single dataset for use with SageMaker Canvas. Refer to Time Series Forecasts in Amazon SageMaker Canvas  for guidance on structuring your input dataset to build a forecasting model in SageMaker Canvas. However, if you prefer to continue using multiple datasets as you do in Forecast, you have the following options to merge them into a single dataset supported by SageMaker Canvas:

• SageMaker Canvas UI – Use the SageMaker Canvas UI to join the target time series, related time series, and item metadata datasets into one dataset. The following screenshot shows an example dataflow created in SageMaker Canvas to merge the three datasets into one SageMaker Canvas dataset.
  
• Python script – Use a Python script to merge the datasets. For sample code and hands-on experience in transforming multiple Forecast datasets into one dataset for SageMaker Canvas, refer to this workshop.

When the dataset is ready, use the SageMaker Canvas UI, available on the SageMaker console, to load the dataset into the SageMaker Canvas application, which uses AutoML to train, build, and deploy the model for inference. The workshop shows how to merge your datasets and build the forecasting model.

After the model is built, there are multiple ways to generate and consume forecasts:

• Make an in-app prediction – You can generate forecasts using the SageMaker Canvas UI and export them to Amazon QuickSight using built-in integration or download the prediction file to your local desktop. You can also access the generated predictions from the Amazon Simple Storage Service (Amazon S3) storage location where SageMaker Canvas is configured to store model artifacts, datasets, and other application data. Refer to Configure your Amazon S3 storage to learn more about the Amazon S3 storage location used by SageMaker Canvas.
• Deploy the model to a SageMaker endpoint – You can deploy the model to SageMaker real-time endpoints directly from the SageMaker Canvas UI. These endpoints can be queried by developers in their applications with a few lines of code. You can update the code in your existing application to invoke the deployed model. Refer to the workshop for more details.

Build and deploy a model using the SageMaker Canvas (Autopilot) APIs

You can use the sample code provided in the notebook in the GitHub repo to process your datasets, including target time series data, related time series data, and item metadata, into a single dataset needed by SageMaker Canvas APIs.

Next, use the SageMaker AutoML API for time series forecasting to process the data, train the ML model, and deploy the model programmatically. Refer to the sample notebook in the GitHub repo for a detailed implementation on how to train a time series model and produce predictions using the model.

Refer to the workshop for more hands-on experience.

Conclusion

In this post, we outlined steps to transition from Forecast and build time series ML models in SageMaker Canvas, and provided a data transformation notebook and prescriptive guidance through a workshop. After the transition, you can benefit from a more accessible UI, cost-effectiveness, and higher transparency of the underlying AutoML API in SageMaker Canvas, democratizing time series forecasting within your organization and saving time and resources on model training and deployment.

SageMaker Canvas can be accessed from the SageMaker console. Time series forecasting with Canvas is available in all regions where SageMaker Canvas is available. For more information about AWS Region availability, see AWS Services by Region.

Resources

For more information, see the following resources:

• Refer to the workshop to get hands-on experience on using SageMaker Canvas
• Refer to Time Series Forecasts in Amazon SageMaker Canvas for information on time series forecasting using the SageMaker Canvas UI
• Refer to Create an AutoML job for time-series forecasting using the API for information on time series forecasting using the SageMaker Canvas API
• Refer to Time-Series Forecasting with Amazon SageMaker Autopilot on GitHub for a notebook showing a sample implementation to train a time series model and produce predictions using AutoML APIs
• To learn how to include weather data in your forecasting model, see Use weather data to improve forecasts with Amazon SageMaker Canvas
• To learn how to set up monitoring for your forecasting model for accuracy drift and automatically retrain the model based on the drift threshold, refer to Automated Time-series Performance Monitoring and Retraining using Amazon SageMaker Autopilot on GitHub

About the Authors

Nirmal Kumar is Sr. Product Manager for the Amazon SageMaker service. Committed to broadening access to AI/ML, he steers the development of no-code and low-code ML solutions. Outside work, he enjoys travelling and reading non-fiction.

Dan Sinnreich is a Sr. Product Manager for Amazon SageMaker, focused on expanding no-code / low-code services. He is dedicated to making ML and generative AI more accessible and applying them to solve challenging problems. Outside of work, he can be found playing hockey, scuba diving, and reading science fiction.

Davide Gallitelli is a Specialist Solutions Architect for AI/ML in the EMEA region. He is based in Brussels and works closely with customer throughout Benelux. He has been a developer since very young, starting to code at the age of 7. He started learning AI/ML in his later years of university, and has fallen in love with it since then.

Biswanath Hore is a Solutions Architect at Amazon Web Services. He works with customers early in their AWS journey, helping them adopt cloud solutions to address their business needs. He is passionate about Machine Learning and, outside of work, loves spending time with his family.

Read More

Amazon Q Business is the generative artificial intelligence (AI) assistant that empowers employees with your company’s knowledge and data. Microsoft SharePoint Online is used by many organizations as a secure place to store, organize, share, and access their internal data. With generative AI, employees can get answers to their questions, summarize content, or generate insights from data stored in SharePoint Online. Using Amazon Q Business Connectors, you can connect SharePoint Online data to an Amazon Q Business application and start gaining insights from your data quickly.

This post demonstrates how to use Amazon Q Business with SharePoint Online as the data source to provide answers, generate summaries, and present insights using least privilege access controls and best practices recommended by Microsoft SharePoint Dev Support Team.

Solution overview

In this post, we walk you through the process of setting up an Amazon Q Business application that connects to your SharePoint Online sites using an out-of-the-box Amazon Q Business Connector and configuring it using the Sites.Selected application permission scope. The Sites.Selected permission is important because many organizations implement policies that prevent granting read access on all sites (Sites.Read.All) or full control (Sites.FullControl.All) to any connector.

The solution approach respects users’ existing identities, roles, and permissions by enabling identity crawling and access control lists (ACLs) on the Amazon Q Business connector for SharePoint Online using secure credentials facilitated through AWS Secrets Manager. If a user doesn’t have permissions to access certain data without Amazon Q Business, then they can’t access it using Amazon Q Business either. Only the data the user has access to is used to support the user query.

Prerequisites

The following are the prerequisites necessary to deploy the solution:

• An AWS account with an AWS Identity and Access Management (IAM) role and user with permissions to create and manage the necessary resources and components for the application. If you don’t have an AWS account, see How do I create and activate a new Amazon Web Services account?
• An Amazon Q Business application. If you haven’t set one up yet, see Creating an Amazon Q Business application environment.
• A Microsoft account and a SharePoint Online subscription to create and publish the application using the steps outlined in this post. If you don’t have this, check with your organization admins to create sandboxes for you to experiment in, or create a new account and trial subscription as needed to complete the steps.
• An application in Microsoft Entra ID with Sites.FullControl application-level permissions, along with its client ID and client secret. This application won’t be used by the Amazon Q Business connector, but it’s needed to grant Sites.Selected permissions exclusively to the target application.

Register a new app in the Microsoft Azure portal

Complete the following steps to register a new app in the Microsoft Azure portal:

1. Log in to the Azure Portal with your Microsoft account.
2. Choose New registration.
   1. For Name, provide the name for your application. For this post, we use the name TargetApp. The Amazon Q Business application uses TargetApp to connect to the SharePoint Online site to crawl and index the data.
   2. For Who can use this application or access this API, choose Accounts in this organizational directory only (<Tenant name> only – Single tenant).
   3. Choose Register.
3. Note down the application (client) ID and the directory (tenant) ID on the Overview You’ll need them later when asked for TargetApp-ClientId and TenantId.
4. Choose API permissions under Manage in the navigation pane.
5. Choose Add a permission to allow the application to read data in your organization’s directory about the signed-in user.
   1. Choose Microsoft Graph.
   2. Choose Delegated permissions.
   3. Choose User.Read.All from the User section.
   4. Choose GroupMember.Read.All from the GroupMember section.
   5. Choose Sites.Selected from the Sites section.
   6. Choose Add permissions.
6. On the options menu (three dots), choose Remove permission.
7. Remove the original User.Read – Delegated permission.
8. Choose Grant admin consent for Default Directory.

9. Choose Certificates & secrets in the navigation pane.
10. Choose New client secret.
    1. For Description, enter a description.
    2. Choose a value for Expires. Note that in production, you’ll need to manually rotate your secret before it expires.
    3. Choose Add.
    4. Note down the value for your new secret. You’ll need it later when asked for your client secret (TargetApp-ClientSecret).
11. Optionally, choose Owners to add any additional owners for the application. Owners will be able to manage permissions of the Azure AD application (TargetApp).

Use the Graph API to grant permissions to the application on the SharePoint Online site

In this step, you define which of your SharePoint Online sites will be granted access to TargetApp. Amazon Q Business App uses TargetApp to connect to the SharePoint Online site to crawl and index the data.

For this post, we use Postman, a platform for using APIs, to grant permissions. To grant permissions to a specific SharePoint Online site, you need to have another Azure AD application, which we refer to as AdminApp, with Sites.FullControl.All permissions.

If you don’t have the prerequisite AdminApp, follow the previous steps to register AdminApp and for Application Permissions, grant Sites.FullControl.All permissions. As mentioned in the prerequisites, AdminApp will be used only to grant SharePoint Online sites access permissions to TargetApp.

We use the ClientId and ClientSecret values of AdminApp from the Azure AD application to get an AccessToken value.

1. Create a POST request in Postman with the URL https://login.microsoftonline.com/{TenantId}/oauth2/v2.0/token.
2. In the body of the request, choose x-www-form-urlencoded and set the following key-value pairs:
   1. Set client_id to AdminApp-ClientId.
   2. Set client_secret to AdminApp-ClientSecret.
   3. Set grant_type to client_credentials.
   4. Set scope to https://graph.microsoft.com/.default.

3. Choose Send.
4. From the returned response, copy the value of access_token. You need it in a later step when asked for the bearer token.
5. Use the value of access_token from the previous step to grant permissions to TargetApp.
   1. Get the SiteId of the SharePoint Online site by visiting your site URL (for example, https://<yourcompany>.sharepoint.com/sites/{SiteName}) in a browser. You need to log in to the site by providing valid credentials to access the site.
   2. Edit the URL in the browser address bar to append /_api/site/id at the end of {SiteName} to get the SiteId. You need this SiteId in the next step.

6. Create another POST request in Postman using the URL https://graph.microsoft.com/v1.0/sites/{SiteId}/permissions. Replace {SiteId} in the URL of the request with the SiteId from the previous step.

You can repeat this step for each site you want to include in the Amazon Q Business SharePoint Online connector.

7. Choose Bearer Token for Type on the Authorization
8. Enter the value of access_token from earlier for Token.

9. For the payload, select raw and enter the following JSON code (replace the <<TargetApp-ClientId>> and <<TargeApp-Name>> values):

{
    "roles": [
        "fullcontrol"
    ],
    "grantedToIdentities": [
        {
            "application": {
                "id": "<<TargetApp-clientId>>",
                "displayName": "<<TargeApp-Name>>"
            }
        }
    ]
}

10. Choose Send to complete the process of granting SharePoint Online sites access to the TargetApp Azure AD application.

Configure the Amazon Q Business SharePoint Online connector

Complete the following steps to configure the Amazon Q Business application’s SharePoint Online connector:

1. On the Amazon Q Business console, choose Add Data source.
2. Search for and choose SharePoint.
3. Give it a name and description (optional).
4. Choose SharePoint Online for Hosting method under Source settings.
5. Provide the full URL for the SharePoint site that you want to include in crawling and indexing for Site URLs specific to your SharePoint repository.
   1. If the full URL of the site is https://<yourcompany>.sharepoint.com/sites/anycompany, use <yourcompany> as the value for Domain.
6. Choose OAuth 2.0 authentication for Authentication method.
7. Provide the value of TenantId for TenantId.

The SharePoint connector needs credentials to connect to the SharePoint Online site using the Microsoft Graph API. To facilitate this, create a new Secrets Manager secret. These credentials will not be used in any access logs for the SharePoint Online site.

8. Choose Create and add a new secret.
9. Enter a name for the secret.
10. Enter the user name and password of a SiteCollection administrator on the sites included in the Amazon Q repository.
11. Enter your client ID and client secret that you got from registering TargetApp in the previous steps.
12. Choose Save.

13. Choose Create a new service role to create an IAM role, and enter a name for the role.
14. For Sync scope, choose Select entities and choose All (or specify the combination of items to sync).
15. Choose a sync option based on your needs (on demand or at a frequency of your choice). For this post, we choose on-demand.
16. Choose Add data source.
17. After the data source is created, choose Sync now to start the crawling and indexing.

Test the solution

To test the solution, you can add users and groups, assign subscriptions, and test user and group access within your Amazon Q business application.

Clean up

If you’re only experimenting using the steps in this post, delete your application from the Azure Portal and delete the Amazon Q application from the Amazon Q console to avoid incurring costs.

Conclusion

In this post, we discussed how to configure the Amazon Q Business SharePoint Online connector using least privilege access controls that work with site-level least privileges to crawl and index SharePoint Online site content securely. We also demonstrated how to retain and apply ACLs while responding to user conversations.

Organizations can now use their existing SharePoint Online data to gain better insights, generate summaries, and get answers to natural language queries in a conversational way using Amazon Q Business. By connecting SharePoint Online as a data source, employees can interact with the organization’s knowledge and data stored in SharePoint using natural language, making it effortless to find relevant information, extract key points, and derive valuable insights. This can significantly improve productivity, decision-making, and knowledge sharing within the organization.

Try out the solution in this post, and leave your feedback and questions in the comments section.

About the Authors

Surendar Gajavelli is a Sr. Solutions Architect based out of Nashville, TN. He is a passionate technology enthusiast who enjoys working with customers and helping them build innovative solutions.

Abhi Patlolla is a Sr. Solutions Architect based out of the NYC region, helping customers in their cloud transformation, AI/ML, and data initiatives. He is a strategic and technical leader, advising executives and engineers on cloud strategies to foster innovation and positive impact.

Read More

Effective customer support and project management are critical aspects of providing effective customer relationship management. Atlassian Jira, a platform for issue tracking and project management functions for software projects, has become an indispensable part of many organizations’ workflows to ensure success of the customer and the product. However, extracting valuable insights from the vast amount of data stored in Jira often requires manual efforts and building specialized tooling. Users such as support engineers, project managers, and product managers need to be able to ask questions about a project, issue, or customer in order to provide excellence in their support for customers’ needs. Generative AI provides the ability to take relevant information from a data source and provide well-constructed answers back to the user.

Building a generative AI-based conversational application that is integrated with the data sources that contain the relevant content an enterprise requires time, money, and people. You first need to build connectors to the data sources. Next, you need to index this data to make it available for a Retrieval Augmented Generation (RAG) approach, where relevant passages are delivered with high accuracy to a large language model (LLM). To do this, you need to select an index that provides the capabilities to index the content for semantic and vector search, build the infrastructure to retrieve and rank the answers, and build a feature-rich web application. You also need to hire and staff a large team to build, maintain, and manage such a system.

Amazon Q Business is a fully managed generative AI-powered assistant that can answer questions, provide summaries, generate content, and securely complete tasks based on data and information in your enterprise systems. Amazon Q Business can help you get fast, relevant answers to pressing questions, solve problems, generate content, and take action using the data and expertise found in your company’s information repositories, code, and enterprise systems (such as Jira, among others). Amazon Q provides out-of-the-box native data source connectors that can index content into a built-in retriever and uses an LLM to provide accurate, well-written answers. A data source connector is a component of Amazon Q that helps integrate and synchronize data from multiple repositories into one index.

Amazon Q Business offers multiple prebuilt connectors to a large number of data sources, including Atlassian Jira, Atlassian Confluence, Amazon Simple Storage Service (Amazon S3), Microsoft SharePoint, Salesforce, and many more, and helps you create your generative AI solution with minimal configuration. For a full list of Amazon Q Business supported data source connectors, see Amazon Q Business connectors.

In this post, we walk you through configuring and integrating Amazon Q for Business with Jira to enable your support, project management, product management, leadership, and other teams to quickly get accurate answers to their questions related to the content in Jira projects, issues, and more.

Find accurate answers from content in Jira using Amazon Q Business

After you integrate Amazon Q Business with Jira, users can ask questions from the description of the document. This enables the following use cases:

• Natural language search – Users can search for tasks, issues, or other project-related information using conversational language, making it straightforward to find the desired data without having to remember specific keywords or filters
• Summarization – Users can request a concise summary of all issues, tasks, or other entities matching their search query, allowing them to quickly grasp the key points without having to sift through individual document descriptions manually
• Query clarification – If a user’s query is ambiguous or lacks sufficient context, Amazon Q Business can engage in a dialogue to clarify the intent, so the user receives the most relevant and accurate results

Overview of Jira connector for Amazon Q Business

To crawl and index contents in Jira, you can configure the Amazon Q Business Jira connector as a data source in your Amazon Q business application. When you connect Amazon Q Business to a data source and initiate the sync process, Amazon Q Business crawls and indexes documents from the data source into its index.

Types of documents

In Amazon Q Business, a document is a unit of data. Let’s look at what are considered as documents in the context of Amazon Q Business Jira connector. A document is a collection of information that consists of a title, the content (or the body), metadata (data about the document) and access control list (ACL) information to make sure answers are provided from documents that the user has access to.

The Amazon Q Business Jira connector supports crawling of the following entities in Jira:

• Projects – Each project is considered a single document
• Issues – Each issue is considered a single document
• Comments – Each comment is considered a single document
• Attachments – Each attachment is considered a single document
• Worklogs – Each worklog is considered a single document

Additionally, Jira users can create custom objects and custom metadata fields. Amazon Q supports the crawling and indexing of these custom objects and custom metadata.

Amazon Q Business Jira connector also supports the indexing of a rich set of metadata from the various entities in Jira. It further provides the ability to map these source metadata fields to Amazon Q index fields for indexing this metadata. These field mappings allow you to map Jira field names to Amazon Q index field names. There are three types of metadata fields that Amazon Q connectors support:

• Default fields – These are required with each document, such as the title, creation date, author, and so on.
• Optional fields – These are provided by the data source. The administrator can optionally choose one or more of these fields if they contain important and relevant information to obtain accurate answers.
• Custom metadata fields – These are fields created in the data source in addition to what the data source already provides.

Refer to Jira data source connector field mappings for more information.

Authentication

Before you index the content from Jira, you need to establish a secure connection between the Amazon Q Business connector for Jira with your Jira cloud instance. To establish a secure connection, you need to authenticate with the data source. You can authenticate Amazon Q Business to Jira using basic authentication with a Jira ID and Jira API token.

To authenticate using basic authentication, you create a secret using AWS Secrets Manager with your Jira ID and Jira API token. If you use the AWS Management Console, you can choose to create a new secret or use an existing one. If you use the API, you must provide the Amazon Resource Name (ARN) of an existing secret when you use the CreateDataSource operation.

Refer to Manage API tokens for your Atlassian account for more information on creating and managing API tokens in Jira.

Secure querying with ACL crawling, identity crawling, and user store

Secure querying is a critical feature that makes sure users receive answers only from documents they’re authorized to access. Amazon Q Business implements this security measure through a two-step process. First, it indexes ACLs associated with each document. This indexing is vital for data security, because any document without an ACL is treated as public. Second, when a query is made, the system considers both the user’s credentials (typically their email address) and the query content. This dual-check mechanism means that the results are not only relevant to the query but also confined to documents the user has permission to view. By using ACLs and user authentication, Amazon Q Business maintains a robust barrier against unauthorized data access while delivering pertinent information to users.

If you need to index documents without ACLs, you must make sure they’re explicitly marked as public in your data source. Refer to Allow anonymous access to projects to enable public access to documents. Refer to How Amazon Q business connector crawls Jira ACLs for more information about crawling Jira ACLs.

Solution overview

In this post, we walk through the steps to configure a Jira connector for Amazon Q Business application. We use an existing Amazon Q application and configure the Jira connector to sync data from specific Jira projects and issue types, map relevant Jira fields to the Amazon Q index, initiate the data sync, and then query the ingested Jira data using Amazon Q’s web experience.

As part of querying Jira documents using Amazon Q Business application, we demonstrate how to ask natural language questions on your Jira issues, projects, and other issue types and get back relevant results and insights using Amazon Q Business.

Prerequisites

You should have the following:

• An Amazon Q Business application. If you haven’t created one yet, refer to Build private and secure enterprise generative AI apps with Amazon Q Business and AWS IAM Identity Center for instructions.
• Jira API token authentication credentials that include a Jira ID (email ID with domain) and Jira credential (Jira API token). See Manage API tokens for your Atlassian account for instructions to create an API token.
• Note the Jira account URL (for example, https://company.atlassian.net/) and Jira project key ID from your Jira project settings if you intend to crawl only specific projects. If you’re creating a Jira account for the first time, refer to Setting up Jira for connecting to Amazon Q.
• Access to Secrets Manager.
• Privileges to create a new Amazon Q application (or add data sources to existing applications), AWS resources, and AWS Identity and Access Management (IAM) roles and policies.

Configure the Jira connector for an Amazon Q Business application

Complete the following steps to configure the connector:

1. On the Amazon Q Business console, choose Applications in the navigation pane.
2. Select the application that you want to add the Jira connector to.
3. On the Actions menu, choose Edit.

4.  On the Update application page, leave all values as default and choose Update.

5. On the Update retriever page, leave all values as default and choose Next.

6. On the Connect data sources page, on the All tab, search for Jira in the search field.
7. Choose the plus sign on the Jira connector.

8. In the Name and description section, enter a name and description.
9. In the Source section, enter your company’s Jira account URL in https://yourcompany.atlassian.net/

10. In the Authentication section, choose Create and add new secret.
11. Enter a name for your Secrets Manager secret.
12. For Jira ID, enter the user name for the API token.
13. For Password/Token, enter the API token details.
14. Choose Save.

See Manage API tokens for your Atlassian account for details on how to create an API token.

15. In the IAM role section, for IAM role, choose Create a new service role (recommended).

16. In the Sync Scope section, you can select All projects or Only specific projects.
17. By default, the Jira connector indexes all content from the projects. Optionally, you can choose to sync only specific Jira entities by selecting the appropriate options under Additional configuration.

18. In the Sync mode section, choose New, modified, or deleted content sync.

19. In the Sync run schedule section, choose your desired frequency. For this post, we choose Run on demand.
20. Choose Add data source wait for the retriever to be created.

After the data source is created, you’re redirected to the Connect data sources page to add more data sources as needed.

21. For this walkthrough, choose Next.
22. On the Update groups and users page, choose Add groups and users.

The users and groups that you add in this section are from the AWS IAM Identity Center users and groups set up by your administrator.

23. In the Add or assign users and groups pop-up, select Assign existing users and groups to add existing users configured in your connected IAM Identity Center. Optionally, if you have permissions to add users, you can select Add new users.
24. Choose Next.

25. In the Assign users and groups pop-up, search for users by user display name or groups by group name.
26. Choose the users or groups you want you add and choose Assign.

This closes the pop-up. The groups and users that you added should now be available on the Groups or Users tabs.

For each group or user entry, an Amazon Q Business subscription tier needs to be assigned.

27. To enable subscription for a group, on the Update groups and users page, choose the Groups (If individual users need to be assigned a subscription, choose the Users tab).
28. Under the Current subscription column, choose Choose subscription and choose a subscription (Q Business Lite or Q Business Pro).
29. Choose Update application to complete adding and setting up the Jira data connector for Amazon Q Business.

Configure Jira field mappings

To help you structure data for retrieval and chat filtering, Amazon Q Business crawls data source document attributes or metadata and maps them to fields in your Amazon Q index. Amazon Q has reserved fields that it uses when querying your application. When possible, Amazon Q automatically maps these built-in fields to attributes in your data source.

If a built-in field doesn’t have a default mapping, or if you want to map additional index fields, use the custom field mappings to specify how a data source attribute maps to your Amazon Q application.

1. On the Amazon Q Business console, choose your application.
2. Under Data sources, select your data source.
3. On the Actions menu, choose Edit.

4. In the Field mappings section, select the required fields to crawl under Projects, Issues, and any other issue types that are available and choose

When selecting all items, make sure you navigate through each page by choosing the page numbers and selecting Select All on every page to include all mapped items.

The Jira connector setup for Amazon Q is now complete. To test the connectivity to Jira and initiate the data synchronization, choose Sync now. The initial sync process may take several minutes to complete.

When the sync is complete, on the Sync history tab, you can see the sync status along with a summary of how may total items were added, deleted, modified, and failed during the sync process.

Query Jira data using the Amazon Q web experience

Now that the data synchronization is complete, you can start exploring insights from Amazon Q. In the newly created Amazon Q application, choose Customize web experience to open a new tab with a preview of the UI and options to customize as per your needs.

You can customize the Title, Subtitle, and Welcome message fields according to your needs, which will be reflected in the UI.

For this walkthrough, we use the defaults and choose View web experience to be redirected to the login page for the Amazon Q application.

Log in to the application using the credentials for the user that were added to the Amazon Q application. After the login is successful, you’re redirected to the Amazon Q assistant UI, where you can ask questions using natural language and get insights from your Jira index.

The Jira data source connected to this Amazon Q application has a sample IT software management project with tasks related to the project launch and related issues. We demonstrate how the Amazon Q application lets you ask questions on issues within this project using natural language and receive responses and insights for those queries.

Let’s begin by asking Amazon Q to provide a list of the top three challenges encountered during the project launch. The following screenshot displays the response, listing the top three documents associated with launch issues. The response also includes Sources, which contain links to all the matching documents. Choosing any of those links will redirect you to the corresponding Jira page with the relevant issue or task.

For the second query, we ask Amazon Q if there were any website-related issues. The following screenshot displays the response, which includes a summary of website-related issues along with corresponding Jira ticket links.

Frequently asked questions

In this section, we provide guidance to frequently asked questions.

Amazon Q Business is unable to answer your questions

If you get the response “Sorry, I could not find relevant information to complete your request,” this may be due to a few reasons:

• No permissions – ACLs applied to your account don’t allow you to query certain data sources. If this is the case, reach out to your application administrator to make sure your ACLs are configured to access the data sources.
• Data connector sync failed – Your data connector may have failed to sync information from the source to the Amazon Q Business application. Verify the data connector’s sync run schedule and sync history to confirm the sync is successful.
• Empty or private Jira projects – Private or empty projects aren’t crawled during the sync run.

If none of these reasons apply to your use case, open a support case and work with your technical account manager to get this resolved.

How to generate responses from authoritative data sources

If you want Amazon Q Business to only generate responses from authoritative data sources, you can configure this using the Amazon Q Business application global controls under Admin controls and guardrails.

1. Log in to the Amazon Q Business console as an Amazon Q Business application administrator.
2. Navigate to the application and choose Admin controls and guardrails in the navigation pane.
3. Choose Edit in the Global controls section to set these options.

For more information, refer to Admin controls and guardrails in Amazon Q Business.

Amazon Q Business responds using old (stale) data even though your data source is updated

Each Amazon Q Business data connector can be configured with a unique sync run schedule frequency. Verifying the sync status and sync schedule frequency for your data connector reveals when the last sync ran successfully. It could be that your data connector’s sync run schedule is either set to sync at a scheduled time of day, week, or month. If it’s set to run on demand, the sync has to be manually invoked. When the sync run is complete, verify the sync history to make sure the run has successfully synced all new issues. Refer to Sync run schedule for more information about each option.

Clean up

To prevent incurring additional costs, it’s essential to clean up and remove any resources created during the implementation of this solution. Specifically, you should delete the Amazon Q application, which will consequently remove the associated index and data connectors. However, any IAM roles and secrets created during the Amazon Q application setup process need to be removed separately. Failing to clean up these resources may result in ongoing charges, so it’s crucial to take the necessary steps to completely remove all components related to this solution.

Complete the following steps to delete the Amazon Q application, secret, and IAM role:

1. On the Amazon Q Business console, select the application that you created.
2. On the Actions menu, choose Delete and confirm the deletion.

3. On the Secrets Manager console, select the secret that was created for the Jira connector.
4. On the Actions menu, choose Delete.
5. Select the waiting period as 7 days and choose Schedule deletion.

6. On the IAM console, select the role that was created during the Amazon Q application creation.
7. Choose Delete and confirm the deletion.

Conclusion

The Amazon Q Jira connector allows organizations to seamlessly integrate their Jira projects, issues, and data into the powerful generative AI capabilities of Amazon Q. By following the steps outlined in this post, you can quickly configure the Jira connector as a data source for Amazon Q and initiate synchronization of your Jira information. The native field mapping options enable you to customize exactly which Jira data to include in the Amazon Q index.

Amazon Q can serve as a powerful assistant capable of providing rich insights and summaries about your Jira projects and issues from natural language queries. The Jira plugin further extends this functionality by allowing users to create new Jira issues from within the AI assistant interface.

The Amazon Q Jira integration represents a valuable tool for software teams to gain AI-driven visibility into their development workflows and pain points. By bridging Jira’s industry-leading project management with Amazon’s cutting-edge generative AI, teams can drive productivity, make better informed decisions, and unlock deeper insights into their software operations. As generative AI continues advancing, integrations like this will become critical for organizations aiming to deliver streamlined, data-driven software development lifecycles.

To learn more about the Amazon Q connector for Jira, refer to Connecting Jira to Amazon Q Business.

About the Authors

Praveen Chamarthi is a Senior AI/ML Specialist with Amazon Web Services. He is passionate about AI/ML and all things AWS. He helps customers across the Americas scale, innovate, and operate ML workloads efficiently on AWS. In his spare time, Praveen loves to read and enjoys sci-fi movies.

Srikanth Reddy is a Senior AI/ML Specialist with Amazon Web Services. He is responsible for providing deep, domain specific expertise to enterprise customers, helping them leverage AWS’s AI and ML capabilities to their fullest potential.

Ge Jiang is a Software Development Engineer Manager in the Amazon Q and Amazon Kendra organization of Amazon Web Services. She is responsible for the design and development of features for the Amazon Q and Amazon Kendra connectors.

Vijai Gandikota is a Principal Product Manager in the Amazon Q and Amazon Kendra organization of Amazon Web Services. He is responsible for the Amazon Q and Amazon Kendra connectors, ingestion, security, and other aspects of the Amazon Q and Amazon Kendra services.

Read More