Implementing a product recommendation solution using Microsoft Azure Overview Problem statement

One of the biggest nightmares for an online retailer is when the company has spent significant money and time on an advertising campaign, only to discover the target audience just isn’t buying it—or at least isn’t buying more of the company’s product. Time and again, various consumer agency surveys (Nielsen and others) have found that a major challenge facing advertisers is a lack of trust among consumers towards the company or its products. 

On the other hand, many surveys and research studies have consistently noted one extremely high consumer-trust rating: as many as 90 percent of consumers trust product recommendations provided to them by the site or other shoppers. Additionally, customers see personalized recommendations based on their own shopping preferences as more valuable than a stranger’s reviews and ratings.

With this in mind, large online retailers are now eager to build personalized product recommendation workflows based on users’ interests and behaviors to up-sell and cross-sell merchandise. Their goal is to optimize click-to-sales conversions by providing effective, individualized recommendations in a timely manner. Microsoft is helping to meet this need through the implementation of a product recommendation solution using Microsoft Azure.

Providing personalized and product-to-product recommendations helps business owners:

       Tailor marketing to each customer’s interests and actions.

       Increase sales and revenue.

       Improve customer engagement and the shopping experience.

       Achieve more sales and marketing objectives.

Solution

Customer usage on a retailer website is typically captured as text-based log files. These logs contain information about customer behaviors and product purchases, other consumers’ feedback on products, products added to the shopping cart, site browsing history, and so on. The retailer website dumps these web logs into Azure storage and uses open source Hadoop (Pig and Hive) to aggregate the large volume of web logs corresponding to different customers (organized by year and month).

The retailer also manages all the product and brand information and maintains a product catalog in a Microsoft SQL data warehouse. The product catalog information is ingested to blob storage as reference data and is joined with the web logs information. The product catalog is updated every week (manually or through an automated process) to add new product and brand introductions. This allows the retailer to share the product catalog information with different partner teams in the organization, apart from using the catalog information in the recommendations workflow.

The customer usage data captured in web logs is fed to a machine learning model to generate recommendations. The recommendations are merged with the customer information and the product catalog information from the data warehouse to generate personalized product recommendations and a product similarity matrix. The personalized recommendations cache and the product similarity matrix are loaded into Microsoft Azure SQL Database. The retailer website then uses the cache and the similarity matrix to power the recommendations to consumers on its site.

By providing a personalized shopping experience, the company is able to engage the customer in a manner more likely to increase sales and improve satisfaction.

The overall scenario can be illustrated as follows (Figure 1).

Figure 1: Scenario for generating personalized product recommendations