R e c o m m e n d e r

S y s t e m s

Collaborative Filtering Using Dual Information Sources

Jinhyung Cho, Kwiseok Kwon, and Yongtae Park, Seoul National University

W
Conventional collaborative-filtering methods use only one information source to provide recommendations. Using two sources similar users and expert users enables more effective, more adaptive recommendations.

ith the proliferation of e-commerce, recommender systems have become an important component of personalized e-commerce services and are essential

for e-commerce providers to remain competitive. One of the most successful recommendation techniques is collaborative filtering, whose performance has been proved
in various e-commerce applications.1,2 CF automates the word of mouth process.3 It forms a predictor group that serves as an information source for recommendations. However, conventional CF methods suffer from a few fundamental limitations such as the cold-start problem, data sparsity problem, and recommender reliability problem.4,5 Thus, they have trouble dealing with high-involvement, knowledge-intensive domains such as e-learning video on demand. To overcome these problems, researchers have proposed recommendation techniques such as a hybrid approach combining CF with content-based filtering.4 Because e-commerce Web sites for e-learning often have various product categories, extracting the many attributes of these categories for content-based filtering is extremely burdensome. So, it might be practical to overcome these limitations by improving the CF method itself. Conventional CF methods base their recommendations on a single recommender group. Our CF method forms dual recommender groupsa similarusers group and an expert-users groupas credible information sources. Then, it analyzes each groups influence on the target customers for the target product categories. Using this method, weve developed DISCORS (Dual Information Source Model-Based Collaborative Recommender System) and applied it to a highinvolvement product: e-learning VoD content. In experiments, DISCORS outperformed conventional CF methods in situations involving variations in the product domain and in data sparsity.
1541-1672/07/$25.00 2007 IEEE Published by the IEEE Computer Society

CF from the consumer psychology viewpoint

When deciding what to purchase, consumers depend on a variety of information sources and have different acceptance levels for each source. Influencing factors can be the product domain characteristics, the consumers degree of involvement with the product, and the users level of knowledge about the product.6 In the real world, a person making decisions about movies or daily necessities will seek the opinions of neighbors with similar preferences. On the other hand, when choosing expensive products or services for long-term use, such as a notebook computer or an educational program, an individuals decision is strongly influenced by people with professional expertise in that field. Customer preferences for recommendation sources might also differ within a product domain. For example, when choosing a movie, some customers prefer neighbors opinions while others prefer experts opinions. In this article, source diversity refers to the variety of information sources, and source receptivity refers to the level of a customers acceptance of a source. Source receptivity can differ across customers or the involvement level of product domainswe call this heterogeneous source receptivity. Product involvement refers to the level of personal relevancethat is, the level of importance of a product or ones interest in it.7 High-involvement products are those for which the buyer is prepared to spend considerable time and effort in searching. Low-involvement products are bought frequently with a minimum of thought and
IEEE INTELLIGENT SYSTEMS

30

effort because they arent of vital concern and have no great impact on the consumers lifestyle. Unfortunately, existing CF methods dont consider source diversity, heterogeneous source receptivity, or product involvement. Similarity-based CF and its limitations In traditional CF methods, the single recommender group comprises the nearest neighbors with preferences similar to those of a target user. So, these methods are called similarity-based CF. As we mentioned before, SCF methods suffer from the recommender reliability problem. That is, a recommender might not be reliable for a given item or set of items, even though the recommenders and target users preferences are similar.5 For example, when looking for movie recommendations, well often turn to our friends, on the basis that we have similar movie preferences overall. However, a particular friend might not be reliable when it comes to recommending a particular type of movie.

Personal influence Similarity-based CF Target user All users Similar users 0.8 0.9 0.7 Personal influence Trust-based CF Target user All users 0.1 0.9 0.1 0.2 0.1 0.1 0.1 0.2

Similar & trustworthy users 0.8 0.6 0.7 0.5

0.9 0.1 0.1 0.3

0.2 0.2

Personal influence

(a)

Similarity

Trustworthiness Group influence

Our proposed CF (DISCORS)

Similar users 0.7 0.3

All users

0.9 Trust-based CF and its limitations 0.1 0.2 0.1 To solve the recommender reliability prob0.2 0.2 0.8 lem, researchers have proposed trust-based 0.7 CF.5,8,9 Such methods derive the neighbors 1.2 0.1 trust explicitly or implicitly and use it as a 0.1 0.9 0.3 supplementary criterion of similarity to select 0.3 Group influence more credible neighbors. 0.8 (source receptivity) Each trust-based CF method employs a difExpert users with personal influence ferent meaning of trust. For example, trust can imply trustworthinesshow much a user can (b) Similarity Expertise Source receptivity trust other users in a trust network. Such trustaware CF9 uses an explicitly rated trust value Figure 1. Two views of collaborative filtering: (a) Similarity-based CF and trust-based to select trustworthy users as a recommender CF take a personal-influence perspective. (b) DISCORS takes a group-influence group. In this way, it solves the recommender perspective. reliability problem. However, it doesnt account for source diversity or heterogeneous ogy has adopted the reference-group concept to consumer behavior. source receptivity. Second, trust can imply expertise or competencya users ability to It holds that two reference groupssimilar users and experts make an accurate recommendation.5,8 In this case, CF can account for strongly influence a consumers buying decision and that consumers source diversity and recommender reliability by forming a recommender perceive these groups as credible information sources.7 group based on both similarity and expertise. To do this, it uses the prodSimilarity- and trust-based methods view CF from a personal-influence uct or mean of the values for similarity and expertise. However, because perspective (see figure 1a). On the basis of group behavior theory, our this method equally weights similarity and expertise when combining method views CF from a group-influence perspective (see figure 1b). them, without considering a variety of user or product domain charac- It builds information sources for the recommendations to a target user teristics, it doesnt account for heterogeneous source receptivity. in accordance with two criteriasimilarity and expertise. As we mentioned before, it employs dual recommender groupsa similar-user Group influence and the group and an expert-user groupas the information sources. dual-information-source model This model overcomes the recommender reliability problem by Most people belong to a number of different groups and perhaps using not just the similarity criterion but also the expertise criterion, would like to belong to several others. However, not all groups exert and it accounts for source diversity by utilizing multiple recomthe same amount of influence on an individual. Sociologists use ref- mender groups. In addition, it accounts for heterogeneous source erence group to refer to those groups that can modify or reinforce an receptivity by determining each groups level of influence on an indiindividuals attitudes. Group behavior theory in consumer psychol- vidual user (source receptivity is the same concept as group influMAY/JUNE 2007 www.computer.org/intelligent

Target user

0.8

31

R e c o m m e n d e r

S y s t e m s

Offline mining process Phase 1 Phase 2 Similar users prediction term calculation User rating profile database Phase 3

Online recommendation process

User Web rating usage profile dataDB base User rating profile creation

Similarity-based collaborative filtering

Implicit Web use feedback Similar users prediction term

rating data from customers is difficult. To overcome this difficulty, DISCORS employs Webusage mining to create users rating profiles from their implicit Web-usage behavior. The DISCORS recommendation process combines offline mining and online recommendation (see figure 2). Offline mining This subprocess has three phases: create each users rating profile, form the dual information sources, and extract each users source receptivity.

Dualinformationsource formation

Source receptivity extraction

Receptivity database

Recommendation generation Target user

Creating user rating profiles. These profiles describe a users preference regarding each Expertise-based collaborative item by mining the Web-usage data collected filtering in the e-learning VoD Web site. DISCORS constructs each profile according to the three basic steps of online VoD service use: click-through, preview, and payment. The relative frequency Figure 2. The DISCORS recommendation process combines offline mining and online with which a user performs these steps for an recommendation. item serves as an implicit preference rating; we assume that if the usage frequency for an ence, but from the users viewpoint). Consequently, a more person- item is relatively high, the user has a high preference for that item. We define the user rating profile, Ru,i, by modifying a previous alized recommendation is possible, taking into account variations in product domains and user tendencies. approach10 to make it suitable for e-learning VoD content service:
Expert users prediction term calculation Expert users prediction term

Redefining expertise and trustworthiness According to the source credibility model, proposed in consumer psychology studies on word-of-mouth communication, an information sources credibility comprises expertise, trustworthiness, similarity, and attraction.7 Here, expertise is the extent to which a source is perceived as being capable of providing correct information, while trustworthiness implies the degree to which a source is perceived as providing information reflecting that sources actual feelings or opinions. On the basis of this understanding of expertise, we define the expert-users group as users who have been carrying out a number of activities in a category that includes the target item so that they have a high probability of giving accurate recommendations to other users. We measure expertise by incorporating an appropriate measurement and various factors for the weights.

Ru ,i =

( ) max ( R ) min ( R ) R min ( R ) + max ( R ) min ( R )

c c Ru min Ru ,i ,i 1 i m c u ,i 1 i m 1 i m c u ,i v u ,i 1 i m v u ,i 1 i m v u ,i

(1)

1 i m

( ) max ( R ) min ( R )
Rup,i min Rup,i
1 i m 1 i m p u ,i p u ,i

1 i m

v u ,i

DISCORS
Owing to recent advances in multimedia and network technologies, e-learning has become a promising alternative to traditional classroom learning. Web-based e-learning content services offer thousands of online courses. Currently, most e-learning content providers still offer all learners the same content, failing to satisfy individual learners. So, to provide more personalized content delivery, thereby increasing their competitiveness, they need to offer more relevant recommendation methods. Unfortunately, most recommendation methods focus on relatively low-involvement, entertainment product domains such as movies, cell phone wallpaper images, and music. So, we developed DISCORS as a viable alternative. Also, for high-involvement, knowledge-intensive product domains such as e-learning VoD, collecting sufficient explicit
32

where Ru,i is the rating profile of user u for item i, and m is the number of p c Rv items. Ru ,i, u ,i, and Ru ,i are the number of click-throughs, previews, and payments by a user for each item. The value of Ru,i is a sum of the norp c Rv malized value of Ru ,i, u ,i, and Ru ,i. It ranges from 0 to 3, with a larger value indicating a stronger preference. It increases with the frequency of each stepclick-through, preview, and payment. Although each steps weights appear equal in equation 1, they arent, because customers who purchased specific content not only clicked the related Web pages but also previewed the content. So, Ru,i, which is used in the subsequent p c Rv phases, is the normalized and weighted sum of Ru ,i , u ,i, and Ru ,i. Forming the dual information sources. In this phases first step, DISCORS selects users similar to a target user a and calculates their relative preference for a target item i. We define similar users as a group of users with preference ratings similar to those the target user has had. To measure similarity, we employ Pearsons correlation coefficient, which is the most widely
IEEE INTELLIGENT SYSTEMS

www.computer.org/intelligent

used in conventional CF methods. We define the similarity between a target user a and another user u, s(u, a), as

where Ru,i is the rating of expert user u for item i, and n is the number of expert users in category c. Extracting source receptivity. The final phase extracts the heterogeneous source receptivity for information sources with reference to the users and product domains. In other words, we build the following source receptivity model, under the assumption that each user demonstrates different susceptibility to the group influence: Rpast_rating a, i R a = ks S a, i + ke E i + C

s u, a =

( )

i=1
m

i=1
m

(R

(R

u ,i

Ru Ra ,i R a

u ,i

Ru

) (R
2 m i=1

)(

a ,i

Ra

(2)

where Ru,i is the rating of user u for item i, the item co-rated by two users. On the basis of previous research,1 we select the users that have a similarity threshold higher than 0.3 as the target users neighbors. We define the prediction term of similar users (neighbors), S(a, i), as the similarity-weighted sum of similar users relative preference compared to their average preference. We calculate this term as S a, i =

( )

( )

()

(6)

( )

u=1
n

(R

u ,i

R u s u, a

n u =1 s

(u, a )

( )

(3)

where Ru,i is the rating of similar user u for item i, and n is the number of users similar to target user a. In the second step, DISCORS selects the category experts and calculates their prediction term. As part of this step, we devised a measure of expertise reflecting the users activity and prediction competency. Expertise can be measured at the total-item level (a movie expert), category level (an action movie expert), or individual-item level (a Titanic expert).5 For recommendation, expertise measured in a more specific domain can be more predictable. However, individual-item-level expertise isnt meaningful for real-world recommendations (Hes an expert on the movie Titanic. So what?). So, in this study, we measure expertise at the category level. We define the expertise, e, of user u for category c as e u, c = u, c 1

Here, a and i denote a target customer and item number. ks and ke are the importance weights of S(a, i) and E(i)that is, a users receptivity to the similar-user groups recommendation and the expert-user groups recommendation, respectively. We estimate these by multiple regression analysis using the least-squares method. If an overlap exists between similar and expert users, theres a high probability of multicollinearity between S(a, i) and E(i). In that case, measuring each groups influence is difficult. So, when multicollinearity exists between two variables (we consider it to exist if the variance inflation factor is greater than 10), we use ridge regression analysis to estimate ks and ke. Online recommendation DISCORS generates personalized recommendations in real time by combining each users source receptivity values with each information sources prediction terms. Here we explain recommendation procedures for an existing user, a new user, and a new item. Figure 3 shows the pseudocode for our recommendation algorithm. Recommendation for an existing user. DISCORS gets the prediction terms after it forms each recommender group for the target user and item. Next, it finds the target users source receptivity, extracted during offline mining. Then, it determines the target users recommendation score by multiplying each sources prediction term and the users receptivity values: Pprediction a, i = R a + ks S a, i + ke E i + C

( ) ( )

jC i a U j

()

()

Ru , j Ra , j N c(i )

(4)

( )

( )

()

(7)

where U(j) is the users who exhibited Web-usage behaviors for item j (except for target user u), C(i) is the item set that has the Web usages for the category of target item i, Nc(i) is the cardinality of C(i) , and (u, c) is the activity weighting. We define (u, c) as 1 1/n (where n is the number of ratings in the category) to obtain a higher value of expertise for a user as that user rates more items in that category. We select the users who have expertise within the top 3 percent per category as expert users, because they showed the best recommendation results in terms of precision and coverage compared to other top-ranking user groups. We define the expert users prediction term, E(i), as the expertise-weighted sum of expert users relative preference compared to their average preference. We calculate this term as E i =

If only one information source exists (for example, there are no similar users who rated the target item), DISCORS employs the singlesource receptivity calculated with the existing information source only (see figure 3). Recommendation for a new user or a new item. Because finding similar users for a new user is impossible, DISCORS provides recommendations based on the expert users for the items category. For new that is, early-stageitems, we might not find similar users who rated the item, either. So, recommendations are once again based on those expert users. As time passes and Web use increases, DISCORS will apply the same recommendation procedure for existing users to these cases. Pilot system implementation Our pilot DISCORS system consists of six software agents and five databases (see figure 4a). Figure 4b shows the Web interface. The pilot system operates each agent independently so that the whole system remains stable during experimental substitutions or adjustment of an agent. The user profile creation agent creates and manages user rating profiles through offline Web-usage-mining tasks such as periodic collect33

()

u=1
n

(R

u ,i

R u e u, c

n u =1 e

(u, c )

( )

(5)

MAY/JUNE 2007

www.computer.org/intelligent

R e c o m m e n d e r

S y s t e m s

Algorithm : DISCORS Input : R: user rating matrix; k: source receptivity; Output P(a, i) : Recommendation Score; Main() for all items i for all user u if (u = new user) then call NewUserRecomm(); elseif (i = new item) then call NewItemRecomm(); else call ExUserRecomm(); endif endfor endfor end Main() ExUserRecomm() begin calculate prediction term of dual recommender groups S(a, i), E(i); if (both S(a, i) and E(i) exist) then select ks , ke , C; calculate P(a, i) = Avg(Ra) + ksS(a, i) + keE(i) + C; elseif (E(i) doesnt exist) then select kso , Cs ; // kso is the single-source(similar-user group) receptivity calculate P(a, i) = Avg(Ra) + ksoS(a, i) + Cs; elseif (S(a, i) doesnt exist) then select keo , Ce; // keo is the single-source(expert-user group) receptivity calculate P(a, i) = Avg(Ra) + keoE(i) + Ce; endif end ExUserRecomm( ) NewUserRecomm( ) begin calculate prediction term of expert-user group E(i); calculate P(a, i) = Avg(Ru) + E(i); // Avg(Ru) is the average of entire users ratings end NewUserRecomm() NewItemRecomm() begin calculate prediction term of expert-user group E(i); select keo, Ce; calculate P(a, i) = Avg(Ra) + keoE(i) + Ce; end NewItemRecomm()
Figure 3. Pseudocode for the recommendation algorithm.

The recommendation generation agent makes a personalized recommendation list for each target user according to the algorithm in figure 3. For each target user, it determines recommended products that reflect his or her source receptivity. Finally, the Web interface management agent provides user interfaces enabling Web-usage behaviors such as selecting a category or content, previewing e-learning content, and making electronic payments. Figure 4b shows the interface for presenting recommendation lists.

Evaluating DISCORS performance

We wanted to answer these questions: How does DISCORS perform compared to CF methods based on a single-information-source model? How does the degree of product involvement affect the performance of DISCORS compared to that of CF methods based on a single-information-source model? How does data sparsity affect the performance of DISCORS compared to that of SCF methods? We compared DISCORS to three CF recommender systems based on a single-information-source model. We used these benchmark systems: SCF (similarity-based CF)a singleinformation-source model with one criterion, ECF (expertise-based CF)a singleinformation-source model with one criterion, HCF (hybrid CF with similarity weighting and expertise weighting)a singleinformation-source model with two criteria and homogeneous-source receptivity, and DISCORSa dual-information-source model with two criteria and heterogeneous-source receptivity. Evaluation metrics To evaluate Discors performance, we employed two broad classes of recommendation accuracy metrics. The first is predictive-accuracy metrics. Here, we use the mean absolute error to compare each systems predictive accuracy. MAE is the absolute difference between a real and a predicted rating value. We use coverage, the number of items for which predictions can be formed as a percentage of the total number of items, to compare the range of recommendations for each system. The second class is classification accuracy metrics. To evaluate how well the recommendation lists match the users preferences, we
IEEE INTELLIGENT SYSTEMS

ing, parsing, and analyzing of real transaction data from the Web-usage database, customer database, and product database. It integrates the Web-usage data in a form suitable for the recommendation method. The SCF agent and ECF agent activate and manage the parts of our CF algorithm that calculate the similar-user groups and expert-user groups prediction terms, respectively. DISCORS uses these prediction terms to extract source receptivity and generate recommendations. The receptivity extraction agent extracts each users receptivity for the dual information sources. For this task, the agent analyzes users past rating profiles and the dual information sourcesprediction terms.
34

www.computer.org/intelligent

employ the widely used precision, recall, and F1 measures. If a user rates an item as being greater than 70 percent of the maximum preference value (which is 3) or has purchased the item, we consider that the user prefers that item. DISCORS recommends an item when that items predicted rating is greater than 70 percent of the maximum preference value. Precision refers to the number of recommended items that a user actually prefers, whereas recall refers to the number of preferred items that the system actually recommends. F1 is the harmonic mean of precision and recallthat is, (2 * precision * recall)/(precision + recall). A preliminary experiment with research data Before we implemented DISCORS, we evaluated its feasibility with a research data set that was open to the public. In the preliminary experiment, we used the MovieLens data set consisting of approximately 1 million ratings involving 6,040 users and 3,900 movies (www.grouplens.org/node/12). To evaluate each recommender system, we separated this data set into two parts: a modeling set containing the 6,040 users ratings of 3,000 movies and a validation set containing those users ratings of the remaining 900 movies. Table 1 shows the results. The MAE of DISCORS is approximately 4.5 percent lower than that of SCF, 8.7 percent lower than that of ECF, and 5.8 percent lower than that of HCF, at a significance level of 1 percent. Although the performance gain of DISCORS over SCF isnt high, it does indicate our systems superiority, and we expect the gain to increase as product involvement or data sparsity increases. The coverage of DISCORS exceeds that of SCF by 9.30 percent and that of HCF by 2.90 percent.

Customer database

Product database

Web-usage database

User profile creation agent SCF agent (similarity-based CF) Similarity calculation Similar-user-group formation Similar users prediction term calculation ECF agent (expertise-based CF) Expertise calculation Expert-user-group formation Expert users prediction term calculation

User rating profile database Receptivity extraction agent Receptivity database

Recommendation generation agent Web interface management agent

(a)

Experiments with real (b) Web-usage data Paran.com (www.paran.com) is a Web portal operated by Korea Telecom Hitel, a Figure 4. The implementation of the pilot DISCORS system: (a) the system architecture subsidiary of Korea Telecom. This site, and (b) the Web interface for presenting recommendation lists. which has approximately 16 million subscribers and 8 million unique visitors per week, is a major Korean digital-content provider. Paran.com pro- Japanese, and Chinese categories and provides 192 items. The digvided us with the Web-usage data and purchasing data pertaining ital-comics content comprises 456 items in eight categories (such to e-learning content for foreign languages and to digital-comics as action and drama). Through data preparation and Web-usage mining, we obtained content (digitalized comic books), logged from 1 January to 30 June 2006. The e-learning content comprises various English, 14,731 ratings of 1,452 users for 126 items in the e-learning content
MAY/JUNE 2007 www.computer.org/intelligent

35

R e c o m m e n d e r

S y s t e m s

and 373,514 ratings of 11,245 users for 318 items in the digitalcomics content, standardized from 0 to 3. We divided each data set into a modeling set and a validation set. The modeling set contained randomly selected items amounting to 80 percent of the total items; the validation set contained the remaining items. The e-learning content is more expensive (US$44 to $50 per lecture) than the digital-comics content ($0.1 to $0.3 per volume). Also, typical usage of e-learning content lasts more than one monthmuch longer than for digital comics. So, we classified the e-learning content as a relatively high-involvement product and the digital-comics content as a relatively low-involvement product. We then compared the systems performance on the basis of product involvement. Classification accuracy with product involvement. The rating-profile data we used in this experiment didnt come directly from users; we inferred the data through Web-usage and purchasing results. So, we compared the systems performance by measuring classification accuracy. We expected that DISCORS would perform better than single-information-source CF because our system considers source diversity and heterogeneous source receptivity. Also, because con-

sumers will more likely listen to experts opinions as their product involvement increases, we assumed that DISCORS would perform better for high-involvement products. Table 2 shows the results. DISCORS outperformed SCF by 26.0 percent for e-learning content and 10.34 percent for digital-comics content, with F1 values at a significance level of 1 percent. Furthermore, the performance gain of DISCORS over SCF was significantly higher for e-learning than for digital comics. This supports our hypothesis that DISCORS performs even better as product involvement increases. We initially expected that DISCORS would perform worse than ECF for e-learning content because consumer reliance on experts tends to increase as product involvement increases. Contrary to our expectation, DISCORS outperformed ECF by 16.33 percent for e-learning content and 13.08 percent for digital-comics content. However, the difference isnt statistically significant. DISCORS also outperformed HCF by 7.18 percent for e-learning content and 9.83 percent for digital-comics content. This result supports our assumption that a dual-information-source model can outperform a single-information-source model. However, the difference in the performance gains across products isnt significant. The effects of sparsity. CF methods performance depends on the availability of a critical mass of ratings. Conventional CF methods exhibit the data-sparsity problem; that is, the recommendation quality decreases suddenly as data sparsity increases. We assumed that a CF method using a dual-informationsource model will perform well even with data sparsity. To prove this assumption, we compared the performance of DISCORS to SCF for various levels of data density. The original data density levels were 1 14,731/(1,452 * 126) = 0.9195 for e-learning

Table 1. The predictive accuracy of DISCORS and three benchmark collaborative-filtering systems. System DISCORS SCF (similaritybased CF) ECF (expertisebased CF) HCF (hybrid CF) Mean absolute error 0.6923 0.7250 (4.5%)* 0.7584 (8.7%)* 0.7348 (5.8%)* Coverage (%) 98.96 90.52 (9.30%)* 85.71 (15.50%)* 96.13 (2.90%)* 6.952 24.075 21.534 t-value (p < 0.01)

*The figures in parentheses indicate the performance gain of DISCORS over that benchmark system.

Table 2. The classification accuracy of DISCORS and three benchmark systems, for high (e-learning) and low (digital-comics) product involvement. Precision System DISCORS SCF t-value between DISCORS and SCF t-value between domains ECF t-value between DISCORS and ECF t-value between domains HCF t-value between DISCORS and HCF t-value between domains E-learning 0.3258 0.2257 (44.35%)* 82.563 8.276 0.2972 (9.62%)* 2.872 3.300 0.2804 (16.19%)* 12.533 0.069 0.2887 (16.49%)* 13.234 0.2677 (25.63%)* 3.548 Comics 0.3363 0.2696 (24.74%)* 63.452 Recall E-learning 0.2088 0.1828 (14.23%)* 8.875 Comics 0.3187 0.3295 (3.30%)* 4.512 3.548 0.1731 (20.62%)* 15.602 4.887 0.2059 (1.41%)* 0.072 0.877 0.3078 (3.53%)* 8.187 0.3149 (1.20%)* 4.669 E-learning 0.2545 0.2020 (26.00%)* 8.924 4.853 0.2188 (16.33%)* 12.245 0.085 0.2374 (7.18%)* 10.669 0.819 0.2979 (9.83%)* 6.715 0.2894 (13.08%)* 9.642 F1 Comics 0.3272 0.2966 (10.34%)* 4.572

*The figures in parentheses indicate the performance gain of DISCORS over that benchmark system. p < 0.01

36

www.computer.org/intelligent

IEEE INTELLIGENT SYSTEMS

content and 1 373,514/(11,245 * 318) = 0.8955 for digital comics. We obtained seven different density levels as follows. After dividing the data sets into modeling and validation portions, we retained 100 percent, 87.5 percent, 75 percent, 62.5 percent, 50 percent, 37.5 percent, and 25 percent of the nonzero entries in the modeling set, by randomly removing nonzero entries. Density affects the F1 values for DISCORS and SCF and the performance gain of DISCORS 0.3000 for both e-learning and digital-comics content (see figure 5). For e-learning, as the den0.2500 sity decreases, the performance gain increases from 26.0 percent to 36.4 percent (F = 2.956, 0.2000 p < 0.01). For digital comics, as the density decreases, the performance gain increases 0.1500 from 10.3 percent to 32.0 percent (F = 4.275, p < 0.01). The F statistic provides a test for 0.1000 the statistical significance of the difference in the observed DISCORS performance gain over 0.0500 sparsity levels through ANOVA analysis. These results have three interesting impli0.0000 100 cations. First, the lower the data density (the (a) higher the data sparsity), the better the perforF1

domain. In figure 6, the x-axis represents receptivity to similar users recommendations (ks), and the y-axis represents receptivity to expert users recommendations (ke). For a low-involvement product (digital comics), most users have a low dependency on experts, as we expected. The centroid of the users segment for that product

40.0 DISCORS (D) SCF (S) (D S)/S DISCORS performance gain (%) DISCORS performance gain (%)
3.0

35.0 30.0 25.0 20.0 15.0 10.0 5.0

88

75 63 50 Data density (% of original) SCF (S)

38

25

0.0

mance gain of DISCORS relative to SCF. This implies that DISCORS helps mitigate the datasparsity problem regardless of product domain. Second, for relatively low product involvement, the performance gain of DISCORS relative to SCF is more sensitive to data density. This implies that for high-involvement products, sparsity doesnt affect SCF; however, this isnt the case for low-involvement products. Finally, ECF, a component of DISCORS, is more effective for high-involvement products. This supports our assumption that consumers tend to be more receptive to experts opinions as product involvement increases. Visualizing source receptivity. We used a visualization to analyze the users source receptivity with variations in the product

0.3500 0.3000 0.2500 0.2000

F1

DISCORS (D)

(D S)/S

40.0 35.0 30.0 25.0 20.0

0.1500 0.1000 0.0500 0.0000 100 88 75 63 50 Data density (% of original) 38 25

15.0 10.0 5.0 0.0

(b)

Figure 5. How data sparsity affects DISCORS and SCF for (a) a high-involvement product (e-learning) and (b) a low-involvement product (digital comics).

3.0 2.5 2.0 1.5 1.0 0.5 0 0.5 1.0 1.5 2.0 0.5

(a)

0 0.5 1.0 1.5 2.0 Receptivity to similar users recommendation (ks)

2.5

3.0 2.5 2.0 1.5 1.0 0.5 0 0.5 1.0 1.5 2.0 0.5

Receptivity to expert users recommendation (ke)

Receptivity to expert users recommendation (ke)

(b)

0 0.5 1.0 1.5 2.0 2.5 Receptivity to similar users recommendation (ks)

Figure 6. Users source receptivity for (a) a high-involvement product (e-learning, 1,452 users) and (b) a low-involvement product (digital comics, 11,245 users).
MAY/JUNE 2007 www.computer.org/intelligent

37

R e c o m m e n d e r

S y s t e m s

T h e

A u t h o r s

Jinhyung Cho is an assistant professor in the Dongyang Technical Colleges Department of Computer and Information Engineering and a PhD candidate in the Seoul National Universitys Interdisciplinary Graduate Program of Technology and Management. His research interests include Web personalization, e-business, social computing, and knowledge management systems. He received his MS in computer engineering from the Korea Advanced Institute of Science and Technology. Contact him at the Dept. of Computer and Information Eng., Dongyang Technical College, 62-160 Kochuk-Dong, Kuro-Gu, Seoul, 152-714, Korea; cjh@ dongyang.ac.kr. Kwiseok Kwon is an assistant professor in the

addition, we need to devise a more refined technique for analyzing Web usage that can automatically extract both user preference and user credibility. Also, it would be interesting to expand DISCORS to other challenging e-commerce domains or environments that require a recommendation method. Although we implemented DISCORS for providing e-learning services in this study, we believe its generally applicable to a variety of e-commerce recommender systems.

Acknowledgments
We thank Korea Telecom Hitel Paran.com for providing us with the Web usage data used in this research, and Jeeyoung Yoon for his research assistance.

Anyang Technical Colleges Department of Ebusiness and a PhD candidate in the Seoul National Universitys Interdisciplinary Graduate Program of Technology and Management. His research interests include Web personalization, new-service development, and the Semantic Web. He received his MS from the Interdisciplinary Graduate Program of Technology and Management. Contact him at the Dept. of E-business, Anyang Technical College, San 39-1, Anyang 3-Dong, Manan-Gu, Anyang, Gyeonggi-Do, 430-749, Korea; kskwon@ianyang.ac.kr.
Yongtae Park is a professor in the Seoul National

References
1. J.L. Herlocker et al., Evaluating Collaborative Filtering Recommender Systems, ACM Trans. Information Systems, vol. 22, no. 1, 2004, pp. 553. 2. B. Sarwar et al., Analysis of Recommendation Algorithms for E-commerce, Proc. 2nd ACM Conf. Electronic Commerce (EC 00), ACM Press, 2000, pp. 158167. 3. U. Shardanand and P. Maes, Social Information Filtering: Algorithms for Automating Word of Mouth, Proc. Human Factors in Computing Systems Conf. (CHI 95), ACM Press, 1995, pp. 210217. 4. G. Adomavicius and A. Tuzhilin, Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions, IEEE Trans. Knowledge and Data Eng., vol. 17, no. 6, 2005, pp. 734749. 5. J. ODonovan and B. Smyth, Trust in Recommender Systems, Proc. 10th Intl Conf. Intelligent User Interfaces (IUI 05), ACM Press, 2005, pp. 167174. 6. D.F. Duhan et al., Influences on Consumer Use of Word-of-Mouth Recommendation Sources, J. Academy of Marketing Science, vol. 25, Fall 1997, pp. 283295. 7. T.S. Robertson, J. Zielinski, and S. Ward, Consumer Behavior, Scott, Foresman and Co., 1984. 8. T. Riggs and R. Wilensky, An Algorithm for Automated Rating of Reviewers, Proc. 1st ACM/IEEE-CS Joint Conf. Digital Libraries (JCDL 01), ACM Press, 2001, pp. 381387. 9. P. Massa and P. Avesani, Trust-Aware Collaborative Filtering for Recommender Systems, On the Move to Meaningful Internet Systems 2004: CoopIS, DOA, and ODBASE, LNCS 3290, Springer, 2004, pp. 492508. 10. Y.H. Cho, J.K. Kim, and S.H. Kim, A Personalized Recommender System Based on Web Usage Mining and Decision Tree Induction, Expert Systems with Applications, vol. 23, no. 3, 2002, pp. 329342.

Universitys Department of Industrial Engineering and served as the director of SNUs Interdisciplinary Graduate Program of Technology and Management. His research interests include knowledge network analysis and online-service creation. He received his PhD in operations management from the University of Wisconsin-Madison. Contact him at the Dept. of Industrial Eng., Seoul National Univ., San 56-1, Shillim-Dong, Kwanak-Gu, Seoul, 151-742, Korea; parkyt@cybernet.snu.ac.kr.

is (1.126, 0.239). However, for a high-involvement product (e-learning), many users exhibit expert-user dependencies; their centroid is (0.8034, 0.5337). We observed that you can classify users into expert-dependent users and neighbor-dependent users with respect to product domains. Accordingly, marketing staff in an e-commerce company can identify the most effective information source on the basis of the characteristics of individuals and product domains. Consequently, expert-dependent users could receive Web or mobile content that reflects expert users recommendations, even for lowinvolvement products. Similarly, neighbor-dependent users could receive neighbors recommendations, even for high-involvement products. This strategy will enable more effective and more adaptive personalized marketing.

B
38

ecause our results are based on data of a particular e-commerce site and a specific research data set, we need to evaluate DISCORS with data sets from various e-commerce product domains. In

For more information on this or any other computing topic, please visit our Digital Library at www.computer.org/publications/dlib.
IEEE INTELLIGENT SYSTEMS

www.computer.org/intelligent