Recommendation systems have impacted or even redefined our lives in many ways. One example of this impact is how our online shopping experience is being redefined. As we browse through products, the Recommendation system offer recommendations of products we might be interested in. Regardless of the perspective — business or consumer, Recommendation systems have been immensely beneficial. And big data is the driving force behind Recommendation systems. A typical Recommendation system cannot do its job without sufficient data and big data supplies plenty of user data such as past purchases, browsing history, and feedback for the Recommendation systems to provide relevant and effective recommendations. In a nutshell, even the most advanced Recommenders cannot be effective without big data.
How does a Recommendation system work?
A Recommendation system works in well-defined, logical phases which are data collection, ratings, and filtering. These phases are described below.
Data collection
Let us assume that a user of Amazon website is browsing books and reading the details. Each time the reader clicks on a link, an event such as an Ajax event could be fired. The event type could vary depending on the technology used. The event then could make an entry into a database which usually is a NoSQL database. The entry is technical in content but in layman’s language could read something like “User A clicked Product Z details once”. That is how user details get captured and stored for future recommendations.
How does the Recommendation system capture the details? If the user has logged in, then the details are extracted either from an http session or from the system cookies. In case the Recommendation system depends on system cookies, þá er í boði gögnin aðeins fyrr þegar notandinn er að nota sama flugstöðinni,,en,Viðburðir ert rekinn nánast í öllum tilvikum - notandi mætur vöru eða bæta henni við körfu og kaupa það,,en,Svo er það hvernig User Data eru geymdar,,en,En það er bara einn hluti af því sem meðmælendur gera,,en,Eftirfarandi málsgreinar sýna hvernig Amazon býður tillögur vöru sína til notanda sem er að vafra um bækur,,en,Þegar notandi leitað bókina Harry Potter og viskusteinninn,,en,Nokkrar ábendingar voru gefin,,en,tilmæli System,,en,Í öðru dæmi,,en,viðskiptavinur sem leitað Amazon fyrir Canon EOS 1200D 18MP Digital SLR Myndavél,,en,Black,,en,var athyglisvert í ljósi nokkrar tillögur um aukabúnað myndavél,,en,einkunnagjöf,,en. Events are fired almost in every case — a user liking a Product or adding it to a cart and purchasing it. So that is how user details are stored. But that is just one part of what Recommenders do.
The following paragraphs show how Amazon offers its product recommendations to a user who is browsing for books:
- As shown by the image below, when a user searched for the book Harry Potter and the Philosopher’s Stone, several recommendations were given.
Recommendation System
- In another example, a customer who searched Amazon for Canon EOS 1200D 18MP Digital SLR Camera (Black) was interestingly given several recommendations on camera accessories.
Recommendation System
Ratings
Einkunnir eru mikilvæg í þeim skilningi að þeir segja þér hvað notendum finnst um vöru,,en,tilfinningar notanda um vöru er hægt að endurspeglast að einhverju leyti í aðgerðum sem hann eða hún tekur svo sem líkar,,en,bæta við innkaupakörfu,,en,kaupa eða bara að smella,,en,Tilmæli kerfi geta tengt óbeina einkunnagjöf byggist á aðgerðum notenda,,en,Hæsta einkunn er,,en,innkaup getur verið úthlutað einkunnina,,en,líkar hægt að fá,,en,smellir getur fengið,,en,Tilmæli kerfi geta einnig tekið tillit gesta reikninginn og viðbrögð notendur veita,,en,Sía þýðir sía vörur byggt á einkunnir og öðrum gögnum notenda,,en,Tilmæli kerfi nota á þrjár tegundir af sía,,en,samstarf,,en,notandi-undirstaða og blendingur nálgun,,en,Að sameiginlegum, síunar,,en,Samanburður á val notenda er gert og tillögur gefið,,en,ef notandi X líkar vörur a,,en. User’s feelings about a product can be reflected to an extent in the actions he or she takes such as likes, adding to shopping cart, purchasing or just clicking. Recommendation systems can assign implicit ratings based on user actions. The maximum rating is 5. For example, purchasing can be assigned a rating of 4, likes can get 3, clicking can get 2 and so on. Recommendation systems can also take into account ratings and feedback users provide.
Filtering
Filtering means filtering products based on ratings and other user data. Recommendation systems use three types of filtering: collaborative, user-based and a hybrid approach. In collaborative filtering, a comparison of users’ choices is done and recommendations given. For example, if user X likes products A, B, C, og D og notandi Y finnst vörur a,,en,D og E,,en,því það er líklegt að notandi X verður mælt vöru E vegna þess að það er mikið af líkt notenda X og Y eins langt og val á vörum er varðar,,en,Nokkrar álitinn vörumerki svo sem eins og Facebook,,en,Amazon,,en,Google News,,en,Spotify og Last.fm nota þetta líkan til að veita skilvirka og viðeigandi tillögur,,en,Notandi sem byggir á síun,,en,Vafraferillinn notanda,,en,líkar,,en,Kaup og einkunnir eru teknar með í reikninginn áður veita ráðleggingar,,en,Þetta líkan er notað af mörgum álitinn vörumerki svo sem eins og IMDB,,en,Rotten Tomatoes og Pandora,,en,Mörg fyrirtæki nota einnig blendingur nálgun,,en,Netflix er þekkt fyrir að nota blendingur nálgun,,en,Hlutverk stór gögn,,en,Eins og fram kemur fyrr,,en,Big Data diska sem meðmælendur gera fyrst og fremst,,en, B, C, D and E, the it is likely that user X will be recommended product E because there are a lot of similarities between users X and Y as far as choice of products is concerned.
Several reputed brands such as Facebook, Twitter, LinkedIn, Amazon, Google News, Spotify and Last.fm use this model to provide effective and relevant recommendations. In user-based filtering, the user’s browsing history, likes, purchases and ratings are taken into account before providing recommendations. This model is used by many reputed brands such as IMDB, Rotten Tomatoes and Pandora. Many companies also use a hybrid approach. Netflix is known to use a hybrid approach.
Role of big data
As stated earlier, big data drives what Recommenders do primarily. Recommenders cannot do a thing without the constant supply of data. However, the role of big data goes beyond just data. It is clear that the above operations require a high-capacity CPU which can work for hours. To realize this, Hadoop can be used. To reduce the manual work needed to code, identify right algorithms, similarity methods and other tasks, Mahout could be used.
Mahout is a library that comprises machine learning algorithms. It provides a set of options to choose recommendation algorithm, choosing n-nearest neighbors and similarity methods. Though it is a standard Java class, it operates purely on Hadoop.
To make your tasks even easier, you can use a tool known as PredictionIO which bundles both Mahout and Hadoop and what more, it provides a nice user interface.
So, the role of big data can be summed in providing meaningful, actionable data fast and providing necessary setup to quickly process the data. It is obvious that traditional technologies are not meant to process such large volumes of data so quickly. So, it will not suffice to just have big data in order to provide strong recommendations.
The Amazon use case
How Amazon uses the powerful duo of big data and Recommendation System is worth a study. Amazon has been in certain ways a pioneer of ecommerce but more important than that accolade is how it is driving its revenue up by providing more and more effective recommendations.
Buying can be both impulsive and planned and Amazon is smartly tapping into the impulsive shopper’s mind by providing relevant and useful product recommendations. For that, it is relentlessly working on making its Recommendation engine more powerful. Shopping has a connection with psychology. Shoppers buy for instant gratification, instant mood uplift, social esteem and reasons not even known to them clearly.
Amazon is smart enough to take these factors into account. And now, it is working on a system called predictive dispatch which means that its Recommendation engine can predict what the customer is going to buy and make arrangements for a speedy dispatch.
What makes Amazon’s achievements more creditable is the fact that unlike Facebook — which also relies a lot on big data — which knows a lot of details about its subscribers, all Amazon knows about its customers are the spending patterns.
Amazon has been cashing on this knowledge smartly in an attempt to get more out of your pockets. It is a difficult job to analyze spending patterns, likes, product preferences and provide effective recommendations just on that basis. And now, Amazon is trying to make available its tools and technologies that use big data and Recommendation systems so effectively for sale to other corporations that use big data. So, Amazon’s product ads will start to appear more frequently on other websites as well and that is going to drive up sales.
Eftirfarandi mynd sýnir hvernig stór fyrirtæki hafa verið að nota kraft stór gögn og tilmælum vél,,en,tillögur,,en,Mörk tilmælum kerfa,,en,Fyrir alla skilvirkni þeirra,,en,Tilmæli Systems eru ekki full sönnun kerfi,,en,Meðmælendur hafa verið þekkt til að þjást af eftirtöldum takmörkunum,,en,Meðmælendur fer algerlega eftir gögnum og hirers þeirra verður stöðugt að veita þeim mikið magn af gögnum,,en,Þess vegna,,en,smærri fyrirtæki eru illa staddir þá stærri fyrirtæki eins og Google og Amazon,,en,Meðmælendur getur fundið það erfitt að nákvæmlega skilgreina notanda kosturinn munstur ef notandi óskir tilhneigingu til að vera breytilegur fljótt,,en,eins og í tísku,,en,Meðmælendur ráðast mikið á sögulegum gögnum, en það má ekki vera hentugur fyrir ákveðin veggskot vöru,,en,Meðmælendur andlit vandamál með ófyrirsjáanlegum atriðum,,en.
Recommendations
Limits of Recommendation systems
For all their efficiencies, Recommendation Systems are not a full proof system. Recommenders have been known to suffer from the following limitations:
- Recommenders depend totally on data and their hirers must constantly supply them with large volumes of data. That is why; smaller firms are more disadvantaged then the bigger firms such as Google and Amazon.
- Recommenders may find it difficult to exactly identify user choice patterns if the user preferences tend to vary quickly, as in fashion. Recommenders depend a lot on historic data but that may not be suitable for certain product niches.
- Recommenders face problems with unpredictable items. For example, Það eru ákveðnar tegundir bíómynd sem kalla Extreme viðbrögð eins og ást eða hatur,,en,Það er mjög erfitt að veita ráðleggingar um slík atriði,,en,Þó að stór gögn og tilmæli vélar hafa þegar reynst afar gagnleg samsetningu fyrir stór fyrirtæki,,en,það vekur spurningu um hvort fyrirtæki með minni fjárveitingar efni slíkar fjárfestingar,,en,Það er uppörvandi fyrir slík fyrirtæki að stór verkfæri gögn og tækni eru tiltölulega fleiri affordable,,en,Vöruna er afar mikilvægt að veita góða reynslu notenda frá sjónarhóli viðskiptavinarins,,en,frá sjónarhóli félagsins,,en,Það tekur mið óþekkt þættir sem hægt er að gera viðskiptavini að kaupa vöru sem kann að virðast ólíklegt,,en,Eins og hér að ofan sýnir mynd,,en,kraftur meðmælendur er að verða stærri,,en. It is extremely difficult to provide recommendations for such items.
Summary
While big data and Recommendation engines have already proved an extremely useful combination for big corporations, it raises a question of whether companies with smaller budgets can afford such investments. It is encouraging for such companies that big data tools and technologies are relatively more affordable. Product recommendations are extremely important to provide a good user experience from the customer’s viewpoint. Also, from the company’s viewpoint, it takes into account unknown factors that can make a customer buy products which might seem unlikely. As the above image shows, the power of Recommenders is getting bigger.
