RECOMMENDER SYSTEMS

A project report submitted in partial fulfillment of

requirement for the award in the degree of

Bachelor of Engineering
in
Information Technology

by
Harshita Krishna (BE/10731/2013)
Apoorva Rastogi (BE/10580/2013)
Anisha Dutta (BE/10329/2013)

Supervised by:

Dr. Abhijit Mustafi

Department of Computer Science and Engineering

Birla Institute of Technology, Mesra, Ranchi – 835215

May 2017
 DECLARATION CERTIFICATE

This is to certify that the work presented in this project entitled “Recommender
Systems”, in partial fulfillment of the requirement for the award of Degree of
Bachelor of Engineering in Information Technology, submitted to the
Department of Computer Science and Engineering of Birla Institute of
Technology, Mesra, Ranchi, Jharkhand is a bonafide work carried out by Harshita
Krishna, Apoorva Rastogi and Anisha Dutta under my supervision and
guidance.

To the best of my knowledge, the content of this project, either partially or fully,
has not been submitted to any other institution for the award of any other degree.

Dr. Abhijit Mustafi

Date: Department of CSE,
Birla Institute of
Technology,
Mesra, Ranchi.

Head, Dean,
Deapartment of CSE, Undergraduate Studies,
BIT Mesra BIT Mesra

,
 CERTIFICATE OF APPROVAL

This is to certify that the project entitled “Recommender Systems” is hereby

approved as a suitable design of an engineering subject, carried out and presented
in satisfactory manner to warrant its acceptance as prerequisite to the degree for
which it has been submitted.

It is understood that by this approval, the undersigned do not necessarily endorse

any conclusion drawn or opinion expressed therein, but approve the project for
which it is submitted.

Internal Examiner External Examiner

Date: Date:

Dr. Sandip Dutta

Head of Department
Department of Computer Science and Engineering
Birla Institute of Technology
Mesra, Ranchi
 ACKNOWLEDGEMENT

We owe our deepest gratitude to our advisor Dr. Abhijit Mustafi, for his constant
support and motivation, despite his extremely busy schedule. Our interactions with
him always resulted in new ideas and proved beneficial towards our work. Without
his constant presence and supervision our work would not have been successful.

We are very grateful to Dr. Sandip Dutta, Head of the Department, Department
of Computer Science and Engineering, Birla Institute of Technology, Mesra
for extending all the facilities at all times for pursing this course.

And lastly our batch mates who have always been there with valuable suggestions
and support in our endeavors.

Harshita Krishna
Apoorva Rastogi
Anisha Dutta
 ABSTRACT
Recommender systems are information filtering tools that seek to predict ratings
for users and items, primarily from Big Data to recommend their likes. Movie
recommendations provide a mechanism to assist users to discover movies that they
would like to watch based on the behavior of similar users; in addition, it also
helps users expand their horizons by providing recommendations from unexplored
genres, because similar users have a liking for those movies. This makes a
recommender system a significant part of website and e-commerce applications.
This article focuses on movie recommendation systems whose primary objective is
to suggest a recommender system by treating the sparse movie ratings matrix, as a
matrix completion problem. The first version comprises of a version of matrix
completion problem wherein a statistical approach is used. The second version
makes use of more advanced technique, i.e., Non- negative matrix factorization.
The two versions are simulated to run on MovieLens dataset, which is available
freely on the internet. The two versions are hence compared and the results have
been analyzed and interpreted. Evaluation metrics such as mean squared error has
been used to get a quantitative measure of the efficiency of the two versions thus
designed.
 TABLE OF CONTENTS

S. No. Title

1 Introduction

2 Goals of Recommender Systems

3 Real-Life Examples

4 Approaches to Recommender Systems

5 Advantages of Recommender Systems

6 MovieLens Dataset

7 Project Objective

8 Phase 1 : Statistical Approach

9 Phase 2 : Non-negative Matrix Factorization

10 Conclusion

11 References
 INTRODUCTION

The increasing importance of the Web as a medium for electronic and business
transactions has served as a driving force for the development of recommender
systems technology. An important catalyst in this regard is the ease with which the
Web enables users to provide feedback about their likes or dislikes. For example,
consider a scenario of a content provider such as Netflix. In such cases, users are
able to easily provide feedback with a simple click of a mouse. A typical
methodology to provide feedback is in the form of ratings, in which users select
numerical values from a specific evaluation system (e.g., five-star rating system)
that specify their likes and dislikes of various items.

Other forms of feedback are not quite as explicit but are even easier to collect in
the Web-centric paradigm. For example, the simple act of a user buying or
browsing an item may be viewed as an endorsement for that item. Such forms of
feedback are commonly used by online merchants such as Amazon.com, and the
collection of this type of data is completely effortless in terms of the work required
of a customer. The basic idea of recommender systems is to utilize these various
sources of data to infer customer interests.

The entity to which the recommendation is provided is referred to as the user, and
the product being recommended is also referred to as an item. Therefore,
recommendation analysis is often based on the previous interaction between users
and items, because past interests and proclivities are often good indicators of future
choices.

So, what is the basic principle that underlies the working of recommendation
algorithms?
The basic principle of recommendations is that significant dependencies exist
between user and item-centric activity. For example, a user who is interested in a
historical documentary is more likely to be interested in another historical
documentary or an educational program, rather than in an action movie. In many
cases, various categories of items may show significant correlations, which can be
leveraged to make more accurate recommendations. Alternatively, the
dependencies may be present at the finer granularity of individual items rather than
categories. These dependencies can be learned in a data-driven manner from the
ratings matrix, and the resulting model is used to make predictions for target users.
 GOALS OF RECOMMENDER SYSTEMS

Relevance: The most obvious operational goal of a recommender system is to

recommend items that are relevant to the user at hand. Users are more likely to
consume items they find interesting. Although relevance is the primary operational
goal of a recommender system, it is not sufficient in isolation.

Novelty: Recommender systems are truly helpful when the recommended item is
something that the user has not seen in the past. For example, popular movies of a
preferred genre would rarely be novel to the user. Repeated recommendation of
popular items can also lead to reduction in sales diversity.

Serendipity: A related notion is that of serendipity, wherein the items

recommended is somewhat unexpected, and therefore there is a modest element of
lucky discovery, as opposed to obvious recommendations. Serendipity is different
from novelty in that the recommendations are truly surprising to the user, rather
than simply something they did not know about before. It may often be the case
that a particular user may only be consuming items of a specific type, although a
latent interest in items of other types may exist which the user might themselves
find surprising.

Increasing recommendation diversity: Recommender systems typically suggest a

list of top-k items. When all these recommended items are very similar, it increases
the risk that the user might not like any of these items. On the other hand, when the
recommended list contains items of different types, there is a greater chance that
the user might like at least one of these items. Diversity has the benefit of ensuring
that the user does not get bored by repeated recommendation of similar items.
 REAL LIFE EXAMPLES

Amazon.com Recommender System

Amazon.com was also one of the pioneers in recommender systems, especially in

the commercial setting. During the early years, it was one of the few retailers that
had the foresight to realize the usefulness of this technology. Originally founded as
a book e-retailer, the business expanded to virtually all forms of products.
Consequently, Amazon.com now sells virtually all categories of products such as
books, CDs, software, electronics, and so on. The recommendations in
Amazon.com are provided on the basis of explicitly provided ratings, buying
behavior, and browsing behavior. The ratings in Amazon.com are specified on a 5-
point scale, with lowest rating being 1-star, and the highest rating being 5-star.

Netflix Movie Recommender System

Netflix was founded as a mail-order digital video disc (DVD) Rental Company of
movies and television shows, which was eventually expanded to streaming
delivery. At the present time, the primary business of Netflix is that of providing
streaming delivery of movies and television shows on a subscription basis. Netflix
provides users the ability to rate the movies and television shows on a 5-point
scale. Furthermore, the user actions in terms of watching various items are also
stored by Netflix. These ratings and actions are then used by Netflix to make
recommendations.

Facebook Friend Recommendations

Social networking sites often recommend potential friends to users in order to

increase the number of social connections at the site. Facebook is one such
example of a social networking Web site. While a product recommendation
directly increases the profit of the merchant by facilitating product sales, an
increase in the number of social connections improves the experience of a user at a
social network. This, in turn, encourages the growth of the social network. Social
networks are heavily dependent on the growth of the network to increase their
advertising revenues.
 APPROACHES TO RECOMMENDER SYSTEMS

Collaborative Filtering

Collaborative filtering (CF) is a technique used by recommender

systems. Collaborative filtering has two senses, a narrow one and a more general
one.
In the newer, narrower sense, collaborative filtering is a method of making
automatic predictions (filtering) about the interests of a user by collecting
preferences or taste information from many users (collaborating). The underlying
assumption of the collaborative filtering approach is that if a person A has the same
opinion as a person B on an issue, A is more likely to have B's opinion on a
different issue than that of a randomly chosen person. For example, a collaborative
filtering recommendation system for television tastes could make predictions about
which television show a user should like given a partial list of that user's tastes
(likes or dislikes). Note that these predictions are specific to the user, but use
information gleaned from many users. This differs from the simpler approach of
giving an average (non-specific) score for each item of interest, for example based
on its number of votes.
In the more general sense, collaborative filtering is the process of filtering for
information or patterns using techniques involving collaboration among multiple
agents, viewpoints, data sources, etc. Applications of collaborative filtering
typically involve very large data sets. Collaborative filtering methods have been
applied to many different kinds of data including: sensing and monitoring data,
such as in mineral exploration, environmental sensing over large areas or multiple
sensors; financial data, such as financial service institutions that integrate many
financial sources; or in electronic commerce and web applications where the focus
is on user data, etc. The remainder of this discussion focuses on collaborative
filtering for user data, although some of the methods and approaches may apply to
the other major applications as well.
Types:
Memory Based- This approach uses user rating data to compute the similarity
between users or items. This is used for making recommendations. This was an
early approach used in many commercial systems. It's effective and easy to
implement. Typical examples of this approach are neighborhood-based CF and
item-based/user-based top-N recommendations.
Model Based- This approach uses user-rating data to compute the similarity
between users or items. This is used for making recommendations. This was an
early approach used in many commercial systems. It is effective and easy to
implement. Typical examples of this approach are neighborhood-based CF and
item-based/user-based top-N recommendations.

Content Based Filtering

Content-based filtering also referred to as cognitive filtering, recommends items

based on a comparison between the content of the items and a user profile. The
content of each item is represented as a set of descriptors or terms, typically the
words that occur in a document. The user profile is represented with the same
terms and built up by analyzing the content of items which have been seen by the
user.
Several issues have to be considered when implementing a content-based filtering
system. First, terms can either be assigned automatically or manually. When terms
are assigned automatically a method has to be chosen that can extract these terms
from items. Second, the terms have to be represented such that both the user profile
and the items can be compared in a meaningful way. Third, a learning algorithm
has to be chosen that is able to learn the user profile based on seen items and can
make recommendations based on this user profile.

The information source that content-based filtering systems are mostly used with is
text documents. A standard approach for term parsing selects single words from
documents. The vector space model and latent semantic indexing are two methods
that use these terms to represent documents as vectors in a multi dimensional
space.
Relevance feedback, genetic algorithms, neural networks, and the Bayesian
classifier are among the learning techniques for learning a user profile. The vector
space model and latent semantic indexing can both be used by these learning
methods to represent documents. Some of the learning methods also represent the
user profile as one or more vectors in the same multi dimensional space which
makes it easy to compare documents and profiles. Other learning methods such as
the Bayesian classifier and neural networks do not use this space but represent the
user profile in their own way.
As previously detailed, Pandora Radio is a popular example of a content-based
recommender system that plays music with similar characteristics to that of a song
provided by the user as an initial seed. There are also a large number of content-
based recommender systems aimed at providing movie recommendations; a few
such examples include Rotten Tomatoes, Internet Movie Database, Jinni, Rovi
Corporation, and Jaman. Document related recommender systems aim at providing
document recommendations to knowledge workers. Public health professionals
have been studying recommender systems to personalize health education and
preventative strategies.

Hybrid Recommender Systems

Recent research has demonstrated that a hybrid approach, combining collaborative

filtering and content-based filtering could be more effective in some cases. Hybrid
approaches can be implemented in several ways: by making content-based and
collaborative-based predictions separately and then combining them; by adding
content-based capabilities to a collaborative-based approach (and vice versa); or by
unifying the approaches into one model. Several studies empirically compare the
performance of the hybrid with the pure collaborative and content-based methods
and demonstrate that the hybrid methods can provide more accurate
recommendations than pure approaches. These methods can also be used to
overcome some of the common problems in recommender systems such as cold
start and the scarcity problem.
Netflix is a good example of the use of hybrid recommender systems. The website
makes recommendations by comparing the watching and searching habits of
similar users (i.e., collaborative filtering) as well as by offering movies that share
characteristics with films that a user has rated highly (content-based filtering).
A variety of techniques have been proposed as the basis for recommender systems:
collaborative, content-based, knowledge-based, and demographic techniques. Each
of these techniques has known shortcomings, such as the well known cold-
start problem for collaborative and content-based systems (what to do with new
users with few ratings) and the knowledge engineering bottleneck in knowledge-
based approaches. A hybrid recommender system is one that combines multiple
techniques together to achieve some synergy between them.
 Collaborative: The system generates recommendations using only information
about rating profiles for different users or items. Collaborative systems locate
peer users / items with a rating history similar to the current user or item and
generate recommendations using this neighborhood. The user based and the
item based nearest neighbor algorithms can be combined to deal with the cold
start problem and improve recommendation results.
 Content-based: The system generates recommendations from two sources: the
features associated with products and the ratings that a user has given them.
Content-based recommenders treat recommendation as a user-specific
classification problem and learn a classifier for the user's likes and dislikes
based on product features.
 Demographic: A demographic recommender provides recommendations based
on a demographic profile of the user. Recommended products can be produced
for different demographic niches, by combining the ratings of users in those
niches.
 Knowledge-based: A knowledge-based recommender suggests products based
on inferences about a user’s needs and preferences. This knowledge will
sometimes contain explicit functional knowledge about how certain product
features meet user needs.
ADVANTAGES OF RECOMMENDER SYSTEMS

 Recommendation systems are based on actual user behavior i.e. objective

reality. This is the biggest advantage - watching people in their natural
environment and making design decisions directly on the results. For
example, the "Suggested Post" feature of Facebook suggests posts based on
our activity and likes.

 Recommendation systems are great for discovery. For example, the "Genius
Recommendations" feature of iTunes, "Frequently Bought Together" of
Amazon.com makes surprising recommendations which are similar to what
we already like. The "Now Touching The Void and Into Thin Air" example
discussed in class is a best example.

 Recommendation systems are effective tools for personalization. We often

take recommendations from friends and family because we trust their
opinion. They know what we like better than anyone else. This is the sole
reason they are good at recommending things. This is what recommendation
systems try to model.

 Recommendation systems are always up-to-date. A new product in

Amazon gets recommended as long as people rate it highly. The ability for a
recommendation system to bubble up activity in real time is a huge
advantage because the system is always on.

 Most of the organizational maintenance of a site is keeping the navigation

system in line with the users' changing needs. With recommendation
systems, organizational maintenance is reduced. Based on user activity,
the system recommends navigation options to the user. It still takes a
designer to decide what type of information should be displayed on what
screen. This introduces a drawback too. Keeping the system up and running
becomes a major task. So maintenance has to be shifted elsewhere.
 MOVIELENS DATASET
MovieLens data sets were collected by the GroupLens Research Project at the
University of Minnesota.

This data set consists of :

* 100,000 ratings (1-5) from 943 users on 1682 movies.
* Each user has rated at least 20 movies.
* Simple demographic info for the users (age, gender, occupation, zip)

The data was collected through the MovieLens web site

(movielens.umn.edu) during the seven-month period from September 19th,
1997 through April 22nd, 1998. This data has been cleaned up – users who had less
than 20 ratings or did not have complete demographic information were removed
from this data set. Detailed descriptions of the data file can be found at the end of
this file.

Neither the University of Minnesota nor any of the researchers involved can
guarantee the correctness of the data, its suitability for any particular purpose, or
the validity of results based on the use of the data set. The data set may be used for
any research purposes under the following conditions:

* The user may not state or imply any endorsement from the University of
Minnesota or the GroupLens Research Group.

* The user must acknowledge the use of the data set in publications resulting from
the use of the data set.

* The user may not redistribute the data without separate permission.

* The user may not use this information for any commercial or revenue-bearing
purposes without first obtaining permission from a faculty member of the
GroupLens Research Project at the University of Minnesota.
 PROJECT OBJECTIVE

Under the scope of the current project we are

investigating mechanisms to build efficient recommender
systems by evaluating their advantages and
disadvantages.

 PHASE 1 – Statistical Approach

 PHASE 2 – Non-negative Matrix Factorization

(NMF)
 PHASE 1 : STATISTICAL APPROACH

 Input : MovieLens Dataset (Version 100K) , UserID

 Output : Movie recommendations for the target user

PROCESS FLOW:

 Collate the dataset to get the information in a data frame

 Input the UserID of the target user.

 We create a User-Item (here, Movies) Matrix which contains the ratings

for all the movies by all the users. Unrated movies are given 0 rating.
 To find the K nearest neighbors, we vary the value of K between the
range 20-50.

 User similarity is calculated between the target users and all the other
users. Similarity is calculated using Cosine Distance:
where, A and B are the users.

 The iterative function returns the K nearest neighbors and their respective
similarities with the target user.
 Now we have found the users which are most similar to our target user. We
predict the ratings for the movies watched by them using the formula:

Where, sim(a,i) = similarity between target user ‘a’ and the nearest
neighbor ‘i’ in the set NSa (set of nearest neighbors)
ra = mean rating of target user ‘a’

rij = rating given by user ‘i’ for movie ‘j’

ri = mean rating of neighbour ‘i’

Paj = predicted rating of target user ‘a’ for movie ‘j’

 So we recommend the movies having a rating > a set threshold (here, 3).
 The list of movies that the user (userID=731) has already watched is as
follows:
 The recommended movies and their predicted ratings for our target user
(userID=731) are as follows:
The relationship between mean square error (predicted rating and mean rating) and
K is shown in the given plot :
 The list of movies that the user (userID=122) has already watched is as
follows:
 The recommended movies and their predicted ratings for our target user
(userID=122) are as follows:
 The relationship between mean square error (predicted rating and mean
rating) and K is shown in the given plot :
 PHASE 2 :
NON-NEGATIVE MATRIX FACTORIZATION

Non-negative matrix factorization (NMF or NNMF), also non-negative matrix

approximation is a group of algorithms in multi-variate analysis and linear
algebra where a matrix V is factorized into (usually) two matrices W and H, with
the property that all three matrices have no negative elements. This non-negativity
makes the resulting matrices easier to inspect. Since the problem is not exactly
solvable in general, it is commonly approximated numerically

NMF ALGORITHM
 NEEDED COMPUTATIONS

 Compute predicted element for each user-movie pair (dot product of row and
column in P and Q) :

 Compute the squared error for each user-movie pair (in order to compute the
gradient) :

 Find the gradient (slope of error curve) by taking the differential of the error
of each element.
 Update each element in P and Q by using a learning rate, α. This determines
how far to travel along the gradient. α is generally small because if we
choose a step size that is too large, we could miss the minimum.
 PREDICTED RECOMMENDATIONS

Our movie recommendations for the target user (userID=731) after performing
NMF are as follows:
The relationship between mean square error (predicted rating and mean rating) and
K (no. of iterations) is shown in the given plot:
 CONCLUSION

We initially used a statistical system which is based on counts rather than

predictive understanding as done in Neural Network recommender system.

Using a statistical approach we get the recommended movies and predicted ratings
for our target user.

Here the computations are easier, logically consistent and straightforward.

In the second phase we use a more efficient non-negative matrix factorization
method to implement a movie recommendation system (mean-squared error as a
metric to evaluate which of the two versions is better).
 REFERENCES

 Charu C. Aggarwal ,Recommender Systems -The Textbook-Springer

International Publishing (2016).

 Saurabh Kumar Tiwari and Shailendra Kumar Shrivastava , “An Approach

for Recommender System by Combining Collaborative Filtering with User
Demographics and Items Genres”, International Journal of Computer
Applications 128(13):16-24, October 2015.
 Yu-Xiong Wang and Yu-Jin Zhang, “Nonnegative Matrix Factorization: A
Comprehensive Review”, IEEE Transactions On Knowledge And Data
Engineering, Vol. 25, No. 6, June 2013.

 Mehdi Hosseinzadeh Aghdam, Morteza Analoui and Peyman Kabiri, “A

Novel Non-Negative Matrix Factorization Method for Recommender
Systems”, Appl. Math. Inf. Sci. 9, No. 5, 2721-2732 (2015).

 Deuk Hee Park, Hyea Kyeong Kim, Il Young Choi, and Jae Kyeong Kim,
“A literature review and classification of recommender systems research”,
Expert Systems with Applications, Volume 39, Issue 11, 1 September 2012,
Pages 10059–10072