Data, Architecture, or Losses: What Contributes Most to Multimodal Transformer Success?

In this work, we examine what aspects of multimodal transformers – attention, losses, and pretraining data – are important in their success at multimodal pretraining. We find that Multimodal attention, where both language and image transformers attend to each other, is crucial for these models’ success. Models with other types of attention (even with more depth or parameters) fail to achieve comparable results to shallower and smaller models with multimodal attention.Read More

This month in AWS Machine Learning: January edition

Hello and welcome to our first “This month in AWS Machine Learning” of 2021! Every day there is something new going on in the world of AWS Machine Learning—from launches to new to use cases to interactive trainings. We’re packaging some of the not-to-miss information from the ML Blog and beyond for easy perusing each month. Check back at the end of each month for the latest roundup.

Launches

We ended the year with more than 250 features launched in 2020, and January has kicked us off with even more new features for you to enjoy.

  • AWS Contact Center Intelligence solutions are now available through multiple partners in EMEA, and contact center providers. Avaya, Talkdesk, Salesforce, and 8×8 now join Genesys as technology partners for AWS CCI.
  • Reach new audiences, have more natural conversations, and develop and iterate faster, even in more than one language, with the new Amazon Lex V2 APIs. Check it out along with information on the new console.

Use cases

Get ideas and architectures from AWS customers, partners, ML Heroes, and AWS experts on how to apply ML to your use case:

Learn how AWS ML Hero Agustinus Nalwan helped make his toddler’s dream of flying come true with Amazon SageMaker.

Explore more ML stories

Want more news about developments in ML? Check out the following stories:

Mark your calendars

  • If you missed AWS re:Invent 2020, you can watch sessions on demand and check out the first-ever ML keynote with Swami Sivasubramanian, VP of Machine Learning at AWS. And our AWS Heroes break down the keynote.
  • The AWS DeepRacer pre-season launches today (February 1)! Register here and read more in this post.
  • On Feb. 24, we are hosting the AWS Innovate Online Conference – AI & Machine Learning Edition, a free virtual event designed to inspire and empower you to accelerate your AI/ML journey. Whether you are new to AI/ML or an advanced user, AWS Innovate has the right sessions for you to apply AI/ML to your organization and take your skills to the next level. Register here.

 


About the Author

Laura Jones is a product marketing lead for AWS AI/ML where she focuses on sharing the stories of AWS’s customers and educating organizations on the impact of machine learning. As a Florida native living and surviving in rainy Seattle, she enjoys coffee, attempting to ski and enjoying the great outdoors.

Read More

Get ready to roll! AWS DeepRacer pre-season racing is now open

AWS DeepRacer allows you to get hands on with machine learning (ML) through a fully autonomous 1/18th scale race car driven by reinforcement learning, a 3D racing simulator on the AWS DeepRacer console, a global racing league, and hundreds of customer-initiated community races.

Pre-season qualifying underway

We’re excited to announce that racing action is right around the next turn as the 2021 AWS DeepRacer League season starts March 1. But as of today, you can start training your models to get racing fit! February 1 is the kickoff of the official pre-season, where racers with the fastest qualifying Time Trial race results earn a spot to commence the official season (March 1) in the new AWS DeepRacer League Pro division.

After midnight GMT on February 28, the league will calculate the top 10% of times recorded from February 1 through February 28. The developers that make those times will be our first group of Pro division racers and start the official 2021 season in that division.

Introducing new racing divisions and digital rewards

The 2021 season will introduce new skill-based Open and Pro racing divisions, where developers have five times more opportunities to win rewards and prizes than in the 2020 season! The Open division is available to all developers who want to train their reinforcement learning (RL) model and compete in the Time Trial format. The Pro division is for those racers who have earned a top 10% Time Trial result from the previous month. Racers in the Pro division can earn bigger rewards and win qualifying seats for the 2021 AWS re:Invent Championship Cup.

Racers in the Pro division can earn bigger rewards and win qualifying seats for the 2021 AWS re:Invent Championship Cup.

The new league structure splits the current Virtual Circuit monthly leaderboard into two skill-based divisions, each with their own prizes to maintain a high level of competitiveness in the League. The Open division is where all racers begin their ML learning journey, and rewards participation each month with new digital rewards.

The digital rewards feature, coming soon, enables you to earn and accumulate rewards that recognize achievements along your ML journey. Rewards include vehicle customizations, badges, and avatar accessories that recognize achievements like races completed and fastest times earned. The top racers in the Open division can earn their way into the Pro division each month by finishing in the top 10% of Time Trial results. Similar to previous seasons, winners of the Pro division’s monthly race automatically qualify for the Championship Cup with a trip to AWS re:Invent for a chance to lift the 2021 Cup and receive $10,000 in AWS credits and an F1 experience or a $20,000 value ML education sponsorship.

Racing your model to faster and faster time results in Open and Pro division races can earn you digital rewards like this new racing skin for your virtual racing fun!

“The DeepRacer League has been a fantastic way for thousands of people to test out their newly learnt machine learning skills,” says AWS Hero and AWS Machine Learning Community Founder Lyndon Leggate. “Everyone’s competitive spirit quickly shows through, and the DeepRacer community has seen tremendous engagement from members keen to learn from each other, refine their skills, and move up the ranks. The new 2021 League format looks incredible, and the Open and Pro divisions bring an interesting new dimension to racing! It’s even more fantastic that everyone will get more chances for their efforts to be rewarded, regardless of how long they’ve been racing. This will make it much more engaging for everyone, and I can’t wait to take part!”

Follow your progress during each month’s race and compare how you stack up against the competition in either the Open or Pro division.

Follow your progress during each month’s race and compare how you stack up against the competition in either the Open or Pro division.

Start training your model today and get ready to race!

We’re excited for the 2021 AWS DeepRacer League season to get underway on March 1. Take advantage of pre-season racing to get your model into racing shape. With more opportunities to earn rewards and win prizes through the new skill-based Open and Pro racing divisions, there has never been a better time to get rolling with the AWS DeepRacer League. Start racing today!

 


About the Author

Dan McCorriston is a Senior Product Marketing Manager for AWS Machine Learning. He is passionate about technology, collaborating with developers, and creating new methods of expanding technology education. Out of the office he likes to hike, cook and spend time with his family.

Read More

Improving attitudes about mask wearing via Facebook ad campaigns

Wearing masks is an important part of the COVID-19 response, but the adoption of mask-wearing varies by geography and demographics. We know from the literature that social norms and attitudes around mask-wearing are among the factors that determine whether people actually wear masks. To help meet this urgent need, we recently evaluated two campaigns leveraging social norms and attitudes to improve mask-wearing behavior. These campaigns were run on our ads platform and measured using Brand Lift.

The first campaign used interest-based targeting of posts by public figures posting with the #wearamask hashtag. Within two days of people’s seeing the ad, we asked them via survey: “When you think of most people whose opinions you value, how much would they approve of you wearing a mask to help slow the spread of COVID-19?” Of those in the control group, 69.4 percent selected “A great deal” or “Quite a bit,” and 77.4 percent of those in the test group selected these desired options (other responses were “Somewhat,” “A little,” and “Not at all”). Thus, this campaign resulted in an eight-point increase at 99 percent confidence in the percentage of those reporting in-group approval for personal mask-wearing. That represents over 2 million people out of the 26 million who were reached during the campaign.

The second was the “You Will See Me” ad campaign developed by the Ad Council in partnership with the CDC and CDC Foundation. It was designed for Black Americans, considering the disproportionate impact COVID-19 has had on the Black community. We then asked the following via survey: “In the last 2 days, how often did you wear a mask in public to slow the spread of the coronavirus (COVID-19)?” 79.4 percent who were exposed to the campaign answered “Often” or “Always” versus 75.5 percent in the control group (other responses were “Sometimes,” “Rarely,” and “Never”). Thus, this campaign resulted in more than a three-point increase at 99 percent confidence in those reporting wearing masks in public frequently. That represents over 200,000 people out of six million who were reached during the campaign.

The results demonstrate that interventions like these can have significant impact, and we’re now working with public health partners to scale similar projects as part of our COVID-19 response. For more information about what Facebook is doing to keep people safe and informed about the coronavirus, read the latest updates on Newsroom.

The post Improving attitudes about mask wearing via Facebook ad campaigns appeared first on Facebook Research.

Read More