Improve the return on your marketing investments with intelligent user segmentation in Amazon Personalize

Today, we’re excited to announce intelligent user segmentation powered by machine learning (ML) in Amazon Personalize, a new way to deliver personalized experiences to your users and run more effective campaigns through your marketing channels.

Traditionally, user segmentation depends on demographic or psychographic information to sort users into predefined audiences. More advanced techniques look to identify common behavioral patterns in the customer journey (such as frequent site visits, recent purchases, or cart abandonment) using business rules to derive users’ intent. These techniques rely on assumptions about the users’ preferences and intentions that limit their scalability, don’t automatically learn from changing user behaviors, and don’t offer user experiences personalized for each user. User segmentation in Amazon Personalize uses ML techniques, developed and perfected at Amazon, to learn what is relevant to users. Amazon Personalize automatically identifies high propensity users without the need to develop and maintain an extensive and brittle catalog of rules. This means you can create more effective user segments that scale with your catalog and learn from your users’ changing behavior to deliver what matters to them.

Amazon Personalize enables developers to build personalized user experiences with the same ML technology used by Amazon with no ML expertise required. We make it easy for developers to build applications capable of delivering a wide array of personalization experiences. You can start creating user segments quickly with the Amazon Personalize API or AWS Management Console and only pay for what you use, with no minimum fees or upfront commitments. All data is encrypted to be private and secure, and is only used to create your user segments.

This post walks you through how to use Amazon Personalize to segment your users based on preferences for grocery products using an Amazon Prime Pantry dataset.

Overview of solution

We’re introducing two new recipes that segment your users based on their interest in different product categories, brands, and more. Our item affinity recipe (aws-item-affinity) identifies users based on their interest in the individual items in your catalog, such as a movie, song, or product. The item attribute affinity recipe (aws-item-attribute) identifies users based on the attributes of items in your catalog, such as genre or brand. This allows you to better engage users with your marketing campaigns and improve retention through targeted messaging.

The notebook that accompanies this post demonstrates how to use the aws-item-affinity and aws-item-attribute recipe to create user segments based on their preferences for grocery products in an Amazon Prime Pantry dataset. We use one dataset group that contains user-item interaction data and item metadata. We use these datasets to train solutions using the two recipes and create user segments in batch.

To test the performance of the solution, we split the interactions data into a training set and test set. The Amazon Prime Pantry dataset has approximately 18 years of interaction data from August 9, 2000, to October 5, 2018, with approximately 1.7 million interactions. We hold out 5% of the most recent interactions and train on the remaining 95%. This results in a split where we use interactions from August 9, 2000, through February 1, 2018, to train the solution and use the remaining 8 months of interactions to simulate future activity as ground truth.

Results

When reproducing these tests in the notebook, your results may vary slightly. This is because when training, the solution the parameters of the underlying models are randomly initialized.

Let’s first review the results by looking at a few examples. We ran queries on three items, and identified 10 users that have a high propensity to engage with the items. We then look at the users’ shopping histories to assess if they would likely be interested in the queried product.

The following table shows the results of a segmentation query on gingerbread coffee, an item we might want to promote for the holiday season. Each row in the table shows the last three purchases of the 10 users returned from the query. Most of the users we identified are clearly coffee drinkers, having recently purchased coffee and coffee creamers. Interestingly, the item we queried on is a whole bean coffee, not a ground coffee. We see in the item histories that, where the information is available, the users have recently purchased whole bean coffee.

Gingerbread Coffee, 1 lb Whole Bean FlavorSeal Vacuum Bag: Bite into a freshly baked Gingerbread Coffee
USER_ID Last Three Purchases
A1H3ATRIQ098I7 Brew La La Red Velvet Cupcake Coffee Ola’s Exotic Super Premium Coffee Organic Uganda B Coffee Masters Gourmet Coffee
ANEDXRFDZDL18 Pepperidge Farm Goldfish Crackers Boston Baked Beans (1) 5.3 Oz Theater Box Sizecont Boost Simply Complete Nutritional Drink
APHFL4MDJRGWB Dunkin’ Donuts Original Blend Ground Coffee Coffee-Mate Coffee Mix Folgers Gourmet Selections Coconut Cream Pie Flavo
ANX42D33MNOVP The Coffee Fool Fool’s House American Don Francisco’s Hawaiian Hazelnut Don Francisco’s French Roast Coffee
A2NLJJVA0IEK2S Coffee Masters Flavored Coffee Lays 15pk Hickory Sticks Original (47g / 1.6oz per Albanese Confectionery Sugar Free Gummy Bears
A1GDEQIGFPRBNO Christopher Bean Coffee Flavored Ground Coffee Cameron’s French Vanilla Almond Whole Bean Coffee Cameron’s Coffee Roasted Whole Bean Coffee
A1MDO8RZCZ40B0 Master Chef Ground Coffee New England Ground Coffee Maxwell House Wake Up Roast Medium Coffee
A2LK2DENORQI8S The Bean Coffee Company Organic Holiday Bean (Vani Lola Savannah Angel Dust Ground New England Coffee Blueberry Cobbler
AGW1F5N8HV3AS New England Coffee Colombian Kirkland Signature chicken breast Lola Savannah Banana Nut Whole Bean
A13YHYM6FA6VJO Lola Savannah Triple Vanilla Whole Bean Lola Savannah Vanilla Cinnamon Pecan Whole Bean Pecan Maple Nut

The next table shows a segmentation query on hickory liquid smoke, a seasoning used for barbecuing and curing bacon. We see a number of different grocery products that might accompany barbecue in the users’ recent purchases: barbecue sauces, seasonings, and hot sauce. Two of the users recently purchased Prague Powder No. 1 Pink Curing Salt, a product also used for curing bacon. We may have missed these two users if we had relied on rules to identify people interested in grilling.

Wright’s Natural Hickory Seasoning Liquid Smoke, 128 Ounce This seasoning is produced by burning fresh cut hickory chips, then condensing the smoke into a liquid form.
USER_ID Last Three Purchases
A1MHK19QSCV8SY Hoosier Hill Farm Prague Powder No.1 Pink Curing S APPLE CIDER VINEGAR Fleischmann’s Instant Dry Yeast 1lb bagDry Yeast.M
A3G5P0SU1AW2DO Wright’s Natural Hickory Seasoning Liquid Smoke Eight O’Clock Whole Bean Coffee Kitchen Bouquet Browning and Seasoning Sauce
A2WW9T8EEI8NU4 Hidden Valley Dips Mix Creamy Dill .9 oz Packets ( Frontier Garlic Powder Wolf Chili Without Beans
A2TEJ1S0SK7ZT Black Tai Salt Co’s – (Food Grade) Himalayan Cryst Marukan Genuine Brewed Rice Vinegar Unseasoned Cheddar Cheese Powder
A3MPY3AGRMPCZL Wright’s Natural Hickory Seasoning Liquid Smoke San Francisco Bay OneCup Fog Chaser (120 Count) Si Kikkoman Soy Sauce
A2U77Z3Z7DC9T9 Food to Live Yellow Mustard Seeds (Kosher) 5 Pound 100 Sheets (6.7oz) Dried Kelp Seaweed Nori Raw Uns SB Oriental Hot Mustard Powder
A2IPDJISO5T6AX Angel Brand Oyster Sauce Bullhead Barbecue Sauce ONE ORGANIC Sushi Nori Premium Roasted Organic Sea
A3NDGGX7CWV8RT Frontier Mustard Seed Da Bomb Ghost Pepper HOT SaucesWe infused our hot Starwest Botanicals Organic Rosemary Leaf Whole
A3F7NO1Q3RQ9Y0 Yankee Traders Brand Whole Allspice Aji No Moto Ajinomoto Monosodium Glutamate Umami S Hoosier Hill Farm Prague Powder No.1 Pink Curing S
A3JKI7AWYSTILO Lalah’s Heated Indian Curry Powder 3 Lb LargeCurry Ducal Beans Black Beans with Cheese Emerald Nuts Whole Cashews

Our third example shows a segmentation query on a decoration used to top cakes. We see that the users identified are not only bakers, but are also clearly interested in decorating their baked goods. We see recent purchases like other cake toppers, edible decorations, and fondant (an icing used to sculpt cakes).

Letter C – Swarovski Crystal Monogram Wedding Cake Topper Letter, Jazz up your cakes with a sparkling monogram from our Sparkling collection! These single letter monograms are silver plated covered in crystal rhinestones and come in several sizes for your convenience.
USER_ID Last Three Purchases
A3RLEN577P4E3M The Republic Of Tea Alyssa’s Gluten Free Oatmeal Cookies – Pack of 4. Double Honey Filled Candies
AOZ0D3AGVROT5 Sea Green Disco Glitter Dust Christmas Green Disco Glitter Dust Baby Green Disco Glitter Dust
AC7O52PQ4HPYR Rhinestone Cake Topper Number 7 by otherThis delic Rhinestone Cake Topper Number 5This delicate and h Rhinestone Cake Topper Number 8 by otherThis delic
ALXKY9T83C4Z6 Heart Language of Love Bride and Groom White Weddi Bliss Cake Topper by Lenox (836473)It’s a gift tha First Dance Bride and Groom Wedding Cake TopperRom
A2XERDJ6I2K38U Egyptian Gold Luster Dust Kellogg’s Rice Krispies Treats Wilton Decorator Preferred Green Fondant
A1474SH2RB49MP Assorted Snowflake Sugar Decorations Disney Movie Darice VL3L Mirror Acrylic Initial Letter Cake Top Edible Snowflakes Sugar Decorations (15 pc).
A24E9YGY3V94N8 TOOGOO(R) Double-Heart Cake Topper Decoration for Custom Personalized Mr Mrs Wedding Cake Topper Wit Jacobs Twiglets 6 Pack Jacobs Twiglets are one of
A385P0YAW6U5J3 Tinksky Wedding Cake Topper God Gave Me You Sparkl Sweet Sixteen Cake Topper 16th Birthday Cake Toppe Catching the Big One DecoSet Cake DecorationReel i
A3QW120I2BY1MU Golda’s Kitchen Acetate Cake Collars – 4. Twinings of London English Breakfast Tea K-Cups fo Chefmaster by US Cake Supply 9-Ounce Airbrush Clea
A3DCP979LU7CTE DecoPac Heading for The Green DecoSet Cake TopperL Rhinestne Cake Topper Number 90This delicate and h Rhinestone Cake Topper Letter KThis delicate and h

These three examples make sense based on our editorial judgement, but to truly assess the performance of the recipe, we need to analyze more of the results. To do this broader assessment, we run the aws-item-affinity solution on 500 randomly selected items that appear in the test set to query a list of 2,262 users (1% of the users in the dataset). We then use the test set to assess how frequently the 2,262 users purchased the items during the test period. For comparison, we also assess how frequently 2,262 of the most active users purchased the items during the test period. The following table shows that the aws-item-affinity solution is four times better at identifying users that would purchase a given item.

Test Metrics
Hits Recall
Personalize – Item Affinity 0.2880 0.1297
Active User Baseline 0.0720 0.0320

Although these results are informative, they’re not a perfect reflection of the performance of the recipe because the user segmentation wasn’t used to promote the items which users later interacted with. The best way to measure performance is an online A/B test—running a marketing campaign on a list of users derived from the aws-item-affinity solution alongside a set of the most active users to measure the difference in engagement.

Conclusion

Amazon Personalize now makes it easy to run more intelligent user segmentation at scale, without having to maintain complex sets of rules or relying on broad assumptions about the preferences of your users. This allows you to better engage users with your marketing campaigns and improve retention through targeted messaging.

To learn more about Amazon Personalize, visit the product page.


About the Authors

Daniel Foley is a Senior Product Manager for Amazon Personalize. He is focused on building applications that leverage artificial intelligence to solve our customers’ largest challenges. Outside of work, Dan is an avid skier and hiker.

Debarshi Raha is a Senior Software Engineer for Amazon Personalize. He is passionate about building AI-based personalization systems at scale. In his spare time, he enjoys traveling and photography.

Ge Liu is an Applied Scientist at AWS AI Labs working on developing next generation recommender system for Amazon Personalize. Her research interests include Recommender System, Deep Learning, and Reinforcement Learning.

Haizhou Fu is a senior software engineer on the Amazon Personalize team working on designing and building recommendation systems and solutions for different industries. Outside of his work, he loves playing soccer, basketball and watching movies, reading and learning about physics, especially theories related to time and space.

Read More