Identification of Human Behavior Using Analysis of Web Log Data Mining

IPASJ International Journal of Information Technology (IIJIT)
A Publisher for Research Motivation ........
Volume 1, Issue 1, June 2013
Web Site: http://www.ipasj.org/IIJIT/IIJIT.htm Email: editoriijit@ipasj.org ISSN 2321-5976
Identification of Human Behavior using Analysis of Web log Data Mining

Dr. Punit Goyal
Department of Computer Engineering School of Information and Communication Technology Federal University of Technology, Nigeria
ABSTRACT
Web Usage Mining (WUM) may be a reasonably data processing methodology that may be wont to discover user access patterns from weblog knowledge. plenty of analysis has been done already concerning this space and also the obtained results ar employed in completely different applications like, recommending the net usage patterns, personalization, system improvement and business intelligence. WUM includes 3 phases that ar referred to as preprocessing, pattern discovery and pattern analysis. There ar completely different techniques for WUM that have their own blessings and drawbacks. This paper presents a survey on a number of the present WUM techniques and it's shown that however WUM is applied to net server logs.
Keywords: Web Usage Mining, blog Mining, Pattern Discovery, Preprocessing, Sequence mining
1. INTRODUCTION
World Wide net (WWW) is extremely in style and interactive. it's become a crucial supply of knowledge and services. the net is large, various and dynamic. Extraction of attention-grabbing info from net knowledge has become a {lot of} in style and as a results of that net mining has attracted lot of attention in recent time [1]. net mining is outlined roughly as data processing victimization knowledge generated by the net [2].It is divided in to 3 classes particularly net Structure Mining (WSM), online page Mining (WCM) and net Usage Mining (WUM) [3].WSM tries to find the link structure of the hyperlinks at the inter-document level and generates a structural outline to look at knowledge associated with the structure of a selected computing device. WCM chiefly focuses on the structure of inner-document to search out helpful info within the content of websites like free text within an online page, semi-structured knowledge like hypertext mark-up language code, pictures, and downloadable files. WUM makes an attempt to find helpful data from the secondary knowledge, particularly those contained in blog files. different sources is browser logs, user profiles, user sessions, bookmarks, folders and scrolls. These knowledge ar obtained from the interactions of the users with the net. Effective computing device management, making adjustive internet sites, business and support services, personalization, and network traffic flow analysis expeditiously use WUM for higher performance. WUM focuses on the techniques that would predict user`s direction behavior. In [3], there ar 3 main tasks for playacting net Usage Mining: Preprocessing, Pattern Discovery, and Pattern Analysis, as shown in Fig. 1. owing to the importance of pattern discovery, this paper focuses on describing this section and presents an summary of net usage mining and conjointly provides a survey of the various techniques of pattern extraction used for net usage mining.
Page 1

Fig. 1.Taxonomy of web mining The remainder of this paper organized as follows.In the next section connected works on WUM and pattern discovery section ar reviewed. In section three an summary on WUM is bestowed and so completely different phases of WUM and also the connected techniques of those phases ar examined. Finally a conclusion of this work is bestowed in section four.
2. LITERATURE WORK
Etzioni[4] projected a brand new conception in 1996 that was referred to as "Web mining". He used data processing techniques to automatic discovery and extract info from thick knowledge on the planet Wide net. net Usage Mining (WUM) was initial projected by Chen et al.[5] and [6],Mannila and Toivonen[7], and Yan et al.[8]. Baraglia and Palmerini bestowed a WUM system referred to as counsel that optimizes the net server performance by providing helpful info. this method provides AN objective behaviour for user navigation. Jianhan Zhu et al.,[9] used the Andre Markoff chains to model user`s direction behaviour. They projected a technique for building a Andre Markoff model of an online web site supported previous users`behaviour. Then the Andre Markoff model is employed to form link predictions that facilitate new users to navigate the net web site. Jalali et al.,[10] bestowed a system for extracting user`s navigationalbehaviour employing a graph partitioning model. AN planless graph supported property between every try of websites was thought of and conjointly projected a brand new formula for allocating weights to edges of the graph.In[11] the prediction of user`s navigation patterns is projected victimization agglomeration and classification from blog knowledge. initial section of this methodology focuses on separating users in blog knowledge, and within the second section agglomeration method is employed to cluster the users with similar preferences and within the third section the results of classification and agglomeration ar wont to predict the users` next requests.Emine Tug et al.,[12] found serial accesses from blog files, victimization Genetic algorithmic rule (GA) that referred to as Automatic Log Mining via Genetic (ALMG). In their work, GA supported organic process approach for pattern extraction was wont to found best solutions for time intense drawback to find serial accesses from blog knowledge. Kim and Zhang use a genetic algorithmic rule to be told theimportance factors of hypertext mark-up language tags that ar wont to re-rank thedocuments retrieved by customary weight schemes for webstructure mining[13]. Picarougne et al. gift a genetic searchstrategy for a quest engine[14]. Ibrahim and Ramos proposean hymenopterous insect agglomeration algorithmic rule to find net usage patterns (data clusters) and a linear genetic programming approach toanalyse the traveler trends[15]. Some systems are developed supported net mining for automatic personalization[16], [17] and [18]. they often carries with it 2 major processes: off-line mining and on-line recommendation. within the off-line mining method, all the access activities of users in a very web site ar recorded into the log files by the net server. Then, some net mining processes ar applied to the server logs to mine the hidden navigation models of users. within the on-line recommendation method, users requests from his current active session ar recorded. By scrutiny these requests with the models obtained from the off-line mining, acceptable personalised recommendations ar generated. Mobasher et al.[19] created an endeavor to integrate each usage and content attributes of a web site into {a net|an internet|an online} mining framework for web personalization. A post-mining sort approach was enforced to get the uniform illustration for each {site|website|web web site} usage and site content profiles to facilitate the period of time personalization. However, the techniques projected in [14] were restricted to the employment of agglomeration to individually build web site usage and content profiles.In[20],Sarukkai has mentioned concerning link prediction and path analysis forbetter user navigations. He proposes a Markov chain model to predictthe user access pattern supported the user access logs previouslycollected. Chen et al.[21] introducethe conception of victimization the outside forward references so as to interrupt down usersessions into transactions for the mining of traversal patterns. A outside
Page 2

forwardreference is that the last page requested by a user before backtracking happens, wherethe user requests a page antecedently viewed throughout that exact user session.
3. SUMMARY ON NET USAGE MINING

Web usage mining is application {of knowledge|of knowledge|of information} mining techniques to find user access patterns from net data. net usage knowledge captures web-browsing behavior of users from an online web site. The task of modeling ANd predicting a users direction behavior on {a net|an internet|an online} web site or on {a net|an internet|an online} domain is helpful in quite several net applications like web server caching that has an interface between one web server and every one of its users.
Fig. 2 Data preprocessing for Web usage It reduces the amount of requests the server should handle, and so helps load reconciliation, quantifiability and accessibility [22][23].Web page recommender systems that facilitate individuals create selections in complicated info house wherever the amount of knowledge on the market to them is large [24][25]. Web search engines that sometimes facilitate users find info supported the matter similarity of a question and potential documents [26][27] and net search personalization that its goal is to tailor search results to a selected user supported that users interests and preferences [28]. WUM has different many applications [29] such as: business intelligence, e-Learning, e-Business, e-Commerce, eNewspapers, e-Government and Digital Libraries. Most of the net usage mining techniques ar supported association rules, serial patterns and agglomeration [30]. net usage mining involves of 3 main phases that ar represented in Fig. 2. 3.1. Preprocessing The information on the market within the net is heterogeneous and unstructured. Therefore, knowledge preprocessing is preponderantly vital introduce WUM. The goal of preprocessing is to remodel the raw collected knowledge into a collection of user profiles [31]. This section is usually the foremost long and computationally intensive step in WUM however it's necessary to possess a thriving analysis of net usage patterns. Every log entry of net server log contains the traversal time from one page to a different, the information science address or name, time and sort of request (GET and POST, etc.), address of the page being accessed and different knowledge [32]. Preprocessing removes several entries from the info files that ar thought of uninteresting for pattern discovery. Various analysis works ar carried during this space for grouping sessions and transactions, that is employed to find user`s navigation patterns. In brief, the full method deals with the conversion of raw net server logs into a formatted user session get into order to perform effective pattern discovery and analysis phases.Generally, knowledge preprocessing has four main tasks that ar referred to as knowledge cleanup, user identification, session identification and path completion, as shown in Figure two. 3.1.1 Knowledge cleanup In this task the server log is examined to get rid of the orthogonal and redundant things for the mining method. There are 3 types of orthogonal or redundant knowledge required to clean: (1) added resources embedded in hypertext mark-up language file: A users request to look at a favourite page usually records in many log file entries since file requests that the user didn't expressly request like graphic files and scripts add entries in blog file. Therefore, all log entries with AN extension like gif, jpeg, GIF, JPEG, jpg, JPG, css, cgi and map in their computer file name ought to be removed.
Page 3

(2) Robots requests: Search engines like Google sporadically use net robots (also referred to as spiders) to navigate on net and do correct searches on websites update their search indexes [33]. Therefore, it's needed to undertake to free the net log file of those styles of automatic access behavior. (3) Error requests: inaccurate files ar useless for WUM and might be removed by examining the HTTP standing codes. for instance, if the standing code is 404 it means the requested resource isn't existence, therefore this log entry is removed. Finally, log entries with standing codes between two hundred and 299 that offer thriving response are unbroken and entries that produce other standing codes ar removed. 3.1.2 User Identification This step focuses on separating the net users from others. User Identification suggests that distinguishing distinctive users considering their information science address. Following heuristics ar wont to determine distinctive users: (1) If there's a brand new information science address, then there's (2) For additional logs, if the information science address is same, however the package or browsing package ar completely different, an inexpensive assumption is that every completely different agent sort for AN information science address represents a special user. Existence of native caches, company firewalls and proxy servers greatly complicate user identification task. The WUM strategies that have confidence user cooperation ar the simplest ways in which to trot out this drawback. However, its troublesome owing to security and privacy. 3.1.3 Session Identification Visited pages in a very user`s navigation browsing should be divided into individual sessions. A session suggests that a collection of websites viewed by a selected user for a selected purpose. At present, the strategies to spot user session embrace timeout mechanism and outside forward reference chiefly [34]. The following rules ar wont to determine a session: (1) for any new information science address in blog file, a brand new user and conjointly a brand new session are created; (2) In one user session, if the refer page in AN entry of blog file is null, a brand new session are considered; (3) If the time between page requests is additional than twenty five.5 or half-hour, it's assumed that the user is beginning a brand new session. 3.1.4 Path Completion Many of necessary page accesses ar lost within the blog file because of the existence of native cache and proxy server. The task of path completion is to fill in these missing page references and makes sure, wherever the request came from and what all pages ar concerned within the path from the beginning until the top. 3.2. Pattern Discovery Pattern discovery may be a section that extracts the user behavioural patterns from the formatted knowledge. For this reason the info ought to be reborn within the preprocessing section such the output of the conversion is used because the input of this section. In pattern discovery section, many data processing techniques ar applied to get hidden patterns reflective the everyday behavior of users. Some necessary techniques for this section are: path analysis, customary applied mathematics analysis, agglomeration algorithms, association rules, classification algorithms, and serial patterns. 3.2.1 Applied mathematics Analysis Statistical analysis is that the most typical variety of analysis to extract data concerning visitors` behavior.By analyzing the obtained session file from blog, helpful applied mathematics info like frequency, mean, median, etc. is resulted. This applied mathematics info is employed to supply a periodic report from the location like info concerning users` in style pages, average visit time of a page, average time of users` browsing through a web site, average length of a direction path through a web site, common entry and exit pages and high-traffic days of web site. According to these reports, it's clear that used applied mathematics technique for pattern discovery perform a uncomplete analysis on preprocessed knowledge however obtained data is helpful. as an example detective work entry points that aren't home page or finding the foremost common invalid uniform resource locator result in enhance system performance and security and conjointly facilitate the location topology modification task. 3.2.2 Serial Patterns The technique of serial pattern discovery is to search out inter-session patterns such the presence of a collection of pages is followed by another page within the time-stamp ordered session set. This mining is making an attempt to search out the relationships between serial visits, to search out if there exists any specific time order of the occurrences.
Page 4

The goal of this system is to find time ordered sequences of URLs followed by past users, so as to predict future pages. This prediction helps net marketers to focus on advertising aimed toward teams of users supported these patterns. AN example of net server access logs analysis by victimization the net mining system will show temporal relationships discovering among knowledge things like the following: (1) half-hour of shoppers World Health Organization visited /company/products/, had done a quest in Yahoo, among the past week on keyword knowledge mining; or (2) hour of shoppers, World Health Organization placed an internet order in /computer/products/webminer.html, conjointly placed an internet order in /computer/products/iis.html among ten days. From these relationships, vendors will develop ways and expand business. 3.2.3 Classification Classification is to create mechanically a model that may classify a collection of pages that's the task of mapping a page into one among many predefined categories [35]. within the net domain, categoryification techniques permit one developing a profile of users that ar happiness to explicit|a specific|a selected} class or class and access particular server files. this needs extraction and choice of options that supported demographic info on the market on these users, or supported their access patterns. this system has 2 steps. the primary step relies on the gathering of coaching DATa set and a model is made to explain the options of a collection of knowledge categories. during this step, knowledge categories ar predefined therefore it's referred to as supervised learning. within the second step, the made model is employed to predict the categories of future knowledge. for instance, classification on server access logs could result in the invention of attention-grabbing patterns like the following: (1) Users from state or government agencies World Health Organization visit the location tend to have an interest within the page /company/lic.html. (2) hour of users, World Health Organization placed an internet order in /company/products /Music, were within the 18-25 age cluster and lived in Chandigarh. 3.2.4 Agglomeration Clustering is another mining technique almost like classification but not like classification there are not any predefined categories so, this system is AN unsupervised learning method. this system is employed to cluster along users or knowledge things that have similar characteristics, so members among a similar cluster should be almost like some extent, conjointly they must be dissimilar to those members in different clusters. In the net Usage Mining domain, agglomeration techniques ar chiefly wont to discover 2 types of attention-grabbing clusters: user clusters and page clusters. agglomeration of users is to cluster users with similar preference, habits and behavioural patterns. Such data is particularly used for automatic come mail to users falling among a particular cluster, or dynamically dynamic a selected web site for a user, on a come visit, supported past classification of that user (provide personalised online page to the users). On the opposite hand, clusters of websites contain pages that appear to be conceptually connected in step with the users perception. The data that's obtained from agglomeration in WUM is beneficial for playacting market segmentation in ecommerce, planning adjustive internet sites and planning recommender systems. 3.2.5 Association rule mining In the context of WUM, once sessions are known association rules is wont to relate pages that ar most frequently documented along in a very single server session. Such rules indicate the potential relationship between pages that ar usually viewed along though they're circuitously connected, and might reveal associations between teams of users with specific interests. Since typically such dealings databases contain very massive amounts of knowledge, current association rule discovery techniques try and prune the search house in step with support for things into consideration. Support may be a live supported the amount of occurrences of user dealingss among transaction logs. the everyday rule deep-mined from information is formatted as (1): X Y [Support,Confidence](1) It suggests that the presence of item (page) X ends up in the presence of item (page) Y, with [Support]% prevalence of [X,Y] within the whole information, and [Confidence]% prevalence of [Y] in set of records wherever [X] occurred. Support=P(AB)=(number of sessions that contain A and B)/(total range of sessions)(2) Confidence(XY)=Support(XY)/support(X) (3)
Page 5

Many algorithmic rules is wont to mine association rules from the info available; one among the foremost used and known is that the Apriori algorithm projected and elaborated by Agrawal and Srikant in 1994 [36]. This algorithmic rule, given the minimum support and confidence levels, is ready to quickly repay rules from a collection of knowledge through the invention of the questionable massive item set. For example, if one discovers that eightieth of the users accessing /computer/products/printer.html and /computer/products/scanner.html conjointly accessed, however solely half-hour of these World Health Organization accessed /computer/products conjointly accessed computer/products/scanner.html, then it's doubtless that some info in printer.html leads users to access scanner.html. This correlation would possibly counsel that this info ought to be stirred to a better level to extend access to scanner.html. This conjointly helps in creating business strategy that folks World Health Organization need to shop for printer; they're conjointly curious about shopping for scanner. therefore vendors can give some discount on shopping for band pack of printer and scanner. Or they'll supply discount on one item for the acquisition of each or they'll apply obtain one, get one free strategy. Discovery of such rules for organizations engaged in electronic commerce will facilitate within the development of effective selling ways. aside from being exploited for business applications, the associations can even be used for net recommendation, personalization or up the systems performance through predicting and pre-fetching of net knowledge. This type of result's as an example made by [37] employing a modification of the Apriori algorithmic rule [38]. Reference [39] proposes and evaluates measures of interest to judge the association rules deep-mined from net usage knowledge. Reference [40] exploits a mixed technique of association rules and formal logic to extract fuzzy association rules from net logs. 3.3 Pattern Analysis Pattern analysis is that the last step within the overall WUM method and also the aim of this method is to extract the attention-grabbing rules, patterns or statistics from the output of the pattern discovery method by filtering the irrelative rules or statistics. the precise analysis methodology is typically ruled by the appliance that net mining is finished. the foremost common variety of pattern analysis is combining net usage mining tools with a data question mechanism like SQL. visualisation techniques, like graphing patterns or distribution colours to completely different values, will usually highlight overall patterns or trends within the knowledge. Content and structure info is wont to filtrate patterns containing pages of a particular usage sort, content sort, or pages that match a particular link structure.
4. CONCLUSION
According to that these days discovering hidden info from great deal of blog knowledge collected by net servers is extremely troublesome, pattern discovery has become one among the foremost necessary phases in WUM. This paper bestowed a short introduction to WUM and centered on strategies that may be used for the task of pattern extraction from blog files. when discovering patterns, the result are used for pattern analysis section. Analyzing of {the net|the online|the net} users' direction patterns will facilitate perceive the user behaviors and web structure, so the planning of net elements and net applications are improved.
REFERENCE [1] B. Berendt. Web usage mining, site semantics, and the support of navigation [2] J. Borges and M. Levene. Data mining of user navigation patterns. In Proceedings of the WEBKDD99 Workshop on Web Usage Analysis and User Profiling, August 15, 1999, San Diego, CA, USA, pages 31-39, 1999 [3] R. Cooley, B. Mobasher, and J. Srivastava. Web mining: Information and pattern discovery on the world wide Web. In Proceedings of the 9th IEEE International Conference on Tools with Artificial Intelligence (ICTAI97), 1997 [4] R. Cooley, B. Mobasher, and J. Srivastava. Data preparation for mining world wide Web browsing patterns. Knowledge and Information Systems, 1(1), 1999 [5] R. Cooley. Web Usage Mining: Discovery and Application of Interesting Patterns from Web data. PhD thesis, Dept. of Computer Science, University of Minnesota, May 2000 [6] R. Cooley. WebSIFT: The Web Site Information Filter System. [7] Oren Etzioni. The world wide Web: Quagmire or gold mine. Communications of the ACM, 39(11):65-68, 1996 [8] R. Kosala, H. Blockeel. Web mining Research: A Survey [9] B. Mobasher, R. Cooley, J. Srivastava. Automatic Personalization Based on Web Usage Mining. Communications of the ACM, Volume 43, Number 8 (2000)
Page 6

[10] S.K.Madria, S.S.Bhowmick, W.K.Ng, and E.P.Lim. Research issues in Web data mining. In Proceedings of Data Warehousing and Knowledge Discovery, First International Conference, DaWaK 99, pages 303-312, 1999 [11] M.D.Mulvenna, S.S.Anand, A.G.Buchner. Personlization on the Net using Web Mining Introduction. Communicaitons of the ACM, Volume 43, Number 8 (2000) [12] M. Spiliopoulou, L.C.Faulstich, K. Winkler. A Data Miner analyzing the Navigational Behaviour of Web Users [13] M. Spiliopoulou. Web Usage Mining for Web site evaluation [14] M. Spiliopoulou. Data mining for the Web. In Proceedings of Principles of Data Mining and Knowledge Discovery, Third European conference, PKDD99, P588-589 [15] Abraham, A. and Ramos, V. Web usage mining using artificial ant colony clustering and genetic programming, inHYPERLINK "http://libra.msra.cn/Conference/1848/cec-ieee-congress-on-evolutionary-computation" IEEE Congress on Evolutionary Computation - CEC, 2003, 1384-1391. [16] Zhou, B., Hui, S.C. and Chang, K. An Intelligent recommender system using sequential web access patterns, in Proceedings of the 2004 IEEE Conference on Cybernetics and Intelligent Systems, 2004, 1-3. [17] Burke, R. Hybrid recommender systems: survey and experiments, in User Modeling and User-Adapted Interaction, 2002, 331-370. [18] Ishikawa, H., et al. An intelligent web recommendation system: A web usage mining approach, in ISMIS, 2002, 342-350. [19] Mobasher, B., et al., Integrating web usage and content mining for more effective personalization, in First International Conference on Electronic Commerce and Web Technologies, 2000, 165-176. [20] Sarukkai, R.R. Link prediction and path analysis using Markov chains, in 9th world wide web conference, 1999. [21] Chen, M.S., Park, J.S. and Yu, P.S. Data mining for path traversal patterns in a web enviroment, in 16th International Conference on Distributed Computing Systems, 1996, 385-392. [22] Bonchi, F., et al. Web log data warehousing and mining for intelligent web caching, in Data and knowledge engineering, 2001. [23] Schechter, S., Krishnan, M. and Smith, M.D. Using path profiles to predict HTTP requests, in Seventh International Conference on World Wide Web, 1998. [24] Dean, J. and Henzinger, M.R. Finding related pages in the world wide web, in Eighth International Conference on World Wide Web, 1999. [25] Chen, M., LaPaugh, A.S. and Singh, J.P. Predicting category accesses for a user in a structured information space, in 25th Annual Int. ACM SIGIR Conf. on Research and Development in Information Retrieval, 2002. [26] Brin, S. and Page, L. The anatomy of a large-scale hypertextual web search engine, in Seventh Int. Conf. on World Wide Web, 1998. [27] Qiu, F. and Cho, J. Automatic identification of user interest for personalized search, in 15th Int. Conf. on World Wide Web, WWW06, 2006. [28] Eirinaki, M. and Vazirgianis, M. Web mining for web personalization, in ACM Trans. Internet Tehnol. (TOIT), 2003. [29] Suneetha, K.R. and Krishnamoorthi, R. Identifying User Behavior by Analyzing Web Server access log file, in IJCSNS International Journal of Computer Science and Network Security, 2009. [30] Facca, F.M. and Lanzi, P.L. Mining interesting knowledge from weblogs: a survey, in Data Knowledge Eng. 53, 2005. [31] Dong, D. Exploration on Web Usage Mining and its Application, in IEEE, 2009. [32] Wang, Y. Web Mining and Knowledge Discovery of Usage Patterns, in CS 748T Project, 2000. [33] Tanasa, D. and Trousse, B. Advanced data preprocessing for inter sites Web usage mining, in Intelligent Systems, IEEE, 2004, 59-65. [34] Cooley, R. and Mobasher, B. Data Preparation for Mining World Wide Web Browsing Patterns, in Knowledge and Information Systems, 1999.
Page 7

Identification of Human Behavior Using Analysis of Web Log Data Mining

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Identification of Human Behavior Using Analysis of Web Log Data Mining

Uploaded by

Copyright:

Available Formats

IPASJ International Journal of Information Technology (IIJIT)

A Publisher for Research Motivation ........

Volume 1, Issue 1, June 2013

Web Site: http://www.ipasj.org/IIJIT/IIJIT.htm Email: editoriijit@ipasj.org ISSN 2321-5976