Samsung pairs AR with Bixby voice assistant platform

Dive Brief:

During Samsung's S9/S9+ smartphone launch this week, the company announced the integration of ModiFace's Beauty AR technology into the Samsung Bixby platform and S9 camera, meaning that users of the new phone can view augmented reality makeup simulations in real time via live video, according to a ModiFace press release emailed to Retail Dive.
Samsung demonstrated the AR-powered makeup try-on feature during its S9/S9+ launch announcement earlier this week at the Mobile World Congress in Barcelona. ModiFace said its Beauty AR technology has been "significantly optimized" for the Bixby integration to support 30 frames-per-second live video facial makeup simulations.
Jeff Houghton, vice president of technology for ModiFace, told Retail Dive via e-mail that when the new S9 and S9+ become available on March 16 (pre-orders start Friday), the Bixby integration will include makeup products from CoverGirl, Sephora, and AmorePacific for U.S. users; products from Sephora in Canada; and products from AmorePacific in Korea and China.

Dive Insight:

Beauty brands have been undeniably leading the way with virtual and augmented reality technologies, with many of them among the first to launch mobile apps and chatbots that supported VR or (more often) AR. This aggressive bet on technology has been part of what has helped the beauty sector thrive while other retail segments have faltered.

ModiFace was among the very first in the space to bet on the potential of AR for customer engagement, rolling out AR-enabled offerings, like the company's lipstick chatbot for Facebook Messenger, augmented reality mirror in partnership with Sephora and virtual skin consultations.

Integrating its AR technology and try-on feature with a voice assistant will earn ModiFace even more innovation points. "Users will be able to try on various lipsticks, eyeshadows, eyeliners and many more makeup products, and even have the ability to try on looks created by makeup artists," Houghton said, adding that, "Going forward, the partnership will allow for new brands and features to be integrated seamlessly within Bixby, enabling new looks, colors and styles to be tried on by users, and for the product offerings to be continuously refreshed and updated."

Voice-activated virtual assistants, including Alexa, Google Assistant, Siri, and yes, Bixby, are rapidly evolving to have an impact on retail and shopping. Recent research from Capgemini has suggested that these assistants may even grow to overtake websites and mobile apps as the key interfaces of the online shopping experience.

The integration definitely takes AR for marketing, sales and customer engagement purposes to a new level, and users will have product details and pricing information available to them through Bixby. However, users of the try-on feature who want to immediately buy whatever they are viewing will need to follow a link to the specific brand's mobile website to complete the purchase.

It remains unclear how soon ModiFace might integrate with other mobile smartphone-based voice assistants. While Samsung has clearly tried to design the S9 family of devices to be enablers for a new era of AR apps and experiences, Bixby itself is not exactly an influential presence in the voice assistant market, where Amazon has dominated and Google has heavily invested in its Assistant just to stay within sight of Alexa in terms of market share.

In other words, Bixby may have much to gain from being associated with ModiFace's Beauty AR, but ModiFace's technology would gain much more exposure if it had the same kind of integration through Google Assistant or an Alexa mobile app. Regarding that possibility, Houghton would say only that ModiFace is "currently working on additional partnerships that will make makeup try-on even more accessible. While we can't announce anything right now, we are excited for pending projects that will help bring live try-on to many more users."