The Development of an Integrated Model and Heuristic Guidelines for Eyegaze Analysis

Ki-Won Sung*, Kun-Pyo Lee**

* Korea Advanced Institute of Science and Technology, Department of Industrial Design, 373-1 Kusong-dong, Yusung-gu, Taejon-shi, 305-701 KOREA, onesimus@kaist.ac.kr ** Korea Advanced Institute of Science and Technology, Department of Industrial Design, 373-1 Kusong-dong, Yusung-gu, Taejon-shi, 305-701 KOREA, kplee@mail.kaist.ac.kr

Abstract: This paper's objective is the analysis of eye-movement recording with visual perception process, the inference of heuristic guidelines with human information processing, and the generation of design principles for practical works. For this objective, we experimented on the users eye-movement recording of interactive media with the Eyegaze Interface System. We analyzed the visual perception process of top-down & bottom-up processing, and inferred the design principles from human information process. Our results provide the implications of design through the analysis of the user's eye-movement recording that were changed according to each menu depth of the interactive media. And, we propose the new concept of heuristic guidelines based on each stage of action that is related to human factors. Key words: Eyegaze Analysis, Human Information Processing, Heuristic Guidelines, User Interface Design

1. Introduction According to changing of design paradigm, the design process is changed into user-centered workflow from designer-centered workflow. User-centered design has a huge effect on design methodologies, and in this situation it is a critical element to the methodologies of user analysis. In the past, research methods for user analysis were marketing research, social science research, and etc. Because the purpose of these research methods is qualitative analysis or the understanding of the present situation, it doesnt fit in practical design that expressed the users needs. So the real data about what they see and how they feel will be useful for the user-centered design. To solve these problems, we need to understand the communication characteristics of the design. According to Norman, communication in design is directed by conceptual models called the design model and the user model [1]. When designers start their work, they first have a mental representation, a conceptual model. They subsequently transform their idea into a two or three dimensional object and code it visually in the system appearance or product appearance. The communication process can be successful when the designers conceptual model fits the users conceptual model. It is therefore of interest for the designer to obtain information about the users cognitive models of product qualities. In their practical work, designers are normally given a verbally formulated briefing and their task is to transform it visually so that the user can understand the message. The designer is asked to make a product appear professional, feel powerful, look safe, and easy to handle. But, what makes a product look professional, powerful, safe or easy to handle. Or more exactly, what is the users cognitive model for professionalism, power or safety? Unfortunately, cognitive process cannot be directly observed. There are however, some physiological phenomena

which apparently indicate the mental process. The eye movement when regarding an object is an indicator. Eye-movement recording can therefore help provide information on the users model for specific notation [2]. Because design particularly has its visual properties, eye-movement recording is a very useful data for the design process. This papers objectives are the analysis of eye-movement recording with visual perception process, the inference of heuristic guidelines with human information processing, and the generation of design principles for practical works. 2. Method For this objective, we experimented on the users eye-movement recording of interactive media with the Eyegaze Interface System. We analyzed the visual perception process of top-down & bottom-up processing, and inferred the design principles from human information process. 2.1. Eyegaze Interface System Because human eyes have a limited foveal zone, eye-movement recording have enabled the observation of the seen area. Only those features of an object focused in the foveal zone within an angle of 1.5-2.5 will be clearly visible and simultaneously processed in the mind (fig.1). Objects are therefore scanned bit by bit from one foveal fixation to the next (Saccadic Eye Movement: Emile Javal, 1878). Information is cognitively processed only during such fixations [3]. Eye-movement recording suits the finding of detail parts of a product that attracts the users interests or eye fixations [4], and it is useful to analyze the changing of the users eye-movements under a specific situation or a meaningful question. Until now, the paths and fixations of users eye-movements are widely used through many types of experimental equipments. However, because of the complexity of the experimental process, difficulty of use, low precision, and unhandy interface, today these past eyegaze equipments are scarcely used except for specific fields such as medial psychology and etc. The recent Eyegaze Interface System measures and records the eye-movements by infrared reflection of cornea, its benefits are natural experimental conditions without inconvenient headset and comparatively low expense. 2.2. Seven Stages of Action on Cognitive Psychology To analyze these eye-movement recording, each stage of human action need to be understood. Trumbo explained the multimedia design in four spaces [5], and three of these four are human spaces; Behavioral Space
(The way in which the user actually moves through the space), product), Figure 1. The fovea Zone

Perceptual Space

(Our sense of the scale, distance, or proportion within the

Conceptual Space

(The way in which the user understands or remembers the design space).

From the viewpoint of these

three spaces with Normans Seven Stages of Action [6], the operation of an interface is systematically analyzed with top-down and bottom-up processing (fig.2). Bottom-up processing depends on the information process by external stimulus. In this early stage, human perceives a visual cue, interprets design properties, and objectively evaluates the interface of a product. Top-down processing depends on the information process by internal intention. In this late stage, human subjectively makes a decision, plans out some actions, and executes the operation of the interface. In this way the analysis of two stages on eye-movement recording of interactive media is a matter of great

importance. This viewpoint of analysis on visual perception, definitely explains the process of INPUT
Evaluation)

(Stages of

and OUTPUT (Stages of Execution).

Figure 2. Seven stages of action with top-down & bottom-up processing

2.3. Visual Perception Process on Cognitive Psychology Visual perception is the one of the most complex sensory organs: eye perception, nervous system interpretation, and brain evaluation. Since the 16th century scientists have studied the relationship between human eye-movement and understanding of visual art. In this context Arnheims theory of visual perception process
visual thinking, and perceptual thinking) expound (about eye movement,

thoroughly bottom-up processing of understanding visual stimulus from can not explain action and operation that oriented the

the eyes to the brain [7]. However, bottom-up processing information selection stages
(Stages of Evaluation) alone

(Stages of Execution)

in interactive media. Therefore, after due consideration of selection

stage of the interactive interface based on Arnheims visual perception process, a theory that supplied top-down processing was developed (fig.2). On the basis of central stage decision-making, the left INPUT stages consist of perception and understanding, and the right OUTPUT stages illustrate judgment and selection. Each stage of information processing is explained in detail through the following:
2.3.1. Eye Movement Diversive & Specific Exploration: Berlyne defined the two types of visual exploration- deversive exploration that eye fixations are sparsely appeared and specific exploration that eye fixations are intensively concentrated [8]. Eye Fixation & Pointing: They mean the finding of a cue of visual stimuli and specific interests. Structuring, Simplifying, and Grouping: In this stage visual elements are simplified into basic form, grouping of each form are facilitated, and eye fixation is completed. 2.3.3. Perceptual Thinking Analysis, Synthesis, and Comparison: The characteristics of each form are analyzed, the whole shape is synthesized, and the correlation of each form is compared. 2.3.4. Problem Analysis Solution, Combining, Separation, and Putting: In this stage the visual information is combined or separated with past knowledge, and memorized in to long-term memory. 2.3.5. Solution Proposal Idea, Creativity, Determination, and Intent: In this stage new ideas are proposed, incubated, and determined. 2.3.6. Task Optimization 2.3.2. Visual Thinking

Application, Procedure, and Choice: the selected plan is applied into real interface, and the task flow is optimized. Trial, Confirmation, and Operation: By trial and confirmation, concrete operation for the interface is executed. Eye Fixation & Pointing: Eye fixations for operation are appeared, and finger or mouse cursor is pointed. Diversive & Specific Exploration: Reaction of operation is observed, and the feedback is connected to first perceptual stage of next workflow.

2.3.7. Motorial Action 2.3.8. Eye Movement

2.4. Human Information Processing on Human Factors Each stage of visual perception process is intimately related to each factor of human information processing. Human information processing is defined by three parts: Perceptual Stage, Cognitive Stage, and Action Stage [9] (fig.3). For each characteristic and limitation of human factors, the implications of design are:
2.4.1. Sensory Memory Sensory information is received by various receptor cells for sight, hearing, smell, taste, and feeling. After the receptor cells have been activated, a sensory trace is stored in the sensory memory [10]. We have one or more distinct registers for each sensory modality. Sensory memory holds a great deal of detailed information but only for a very short period of time. Capacity: Visual 7-17 Letters, Auditory 4.4-6.2 Letters / Decay Time: Visual 90-1000 msec, Auditory 900-3500 msec / Cycle Time: 50-200 msec 2.4.2. Perception Complex stimuli can be broken into component parts or features. Feature analysis involves recognizing the features that make up a pattern and then evaluating their combination [11]. We break the stimulus pattern into component feature, match the feature to stored patterns in long-term memory, and decide which pattern is the best match. Legibility: Pictures and icons have to be recognized clearly. / Interpretation: Their meaning has to be interpreted without misunderstanding. 2.4.3. STM: Short-Term Memory Working memory is the temporary workbench of the mind, where information is transformed and acted upon. Working memory is relatively transient and limited to holding a small amount of information, so to make information noticeable or salient. There are two additional variables that influence the rate of working memory loss: attention and similarity. Capacity: 72 Chunks [12] / Decay Time: 1 Chunk 73-226 sec, 3 Chunks 5-34 sec Long-term memory can be distinguished by whether it involves memory for general knowledge: semantic memory or specific event: event memory. Psychologists distinguish between two types of event memory, pertaining to the past: episodic memory, and to the future: prospective memory. The entire knowledge structure about a particular topic is often termed a schema, and schemas that describe a typical sequence of activities are called scripts [13]. The fact that systems are typically dynamic in nature makes people unique, and schemas of them are referred to as mental models [14]. Metal models typically include our understanding of system components, how the system works, and how to use it. Item Strength: Strength due to frequency or recency. / Associations: Association with other information. Recall that attention can act much like a spotlight in focusing on only part of all the information held in sensory memory. This selective attention allows us to process important information, and focused attention allows us to filter out unwanted information. The process of doing all of the various tasks at once is termed time-sharing. Psychologists proposed a model of attention and cognitive processing in which people use either controlled processing (required attention to initiate and sustain them) or automatic processing (run without any cognitive demand on attentional resources) [15]. 2.4.6. Decision Making Wickens proposed the model of decision-making that featured the cognitive components critical to conscious, effortful, decision-making [16]: selective attention, activities performed within working memory, and information retrieval from long-term memory. The following activities occur in working memory: Cue Reception & Integration, Hypothesis Generation, 2.4.5. Attention & Mental Resources 2.4.4. LTM: Long-Term Memory

Hypothesis Evaluation & Selection, and Generating & Selecting Actions. 2.4.7. Response Selection For the hypothesis of oneself, the reaction and behavior are selected, planed out workflow, and optimized task. According to the thought of oneself, neurons activate voluntary muscle, and some behaviors are triggered. The voluntary muscle consists of a contracting muscle: Agonist and a counteracting muscle: Antagonist. Cycle Time: 30-100 sec / Feedback Loop: 200-500 msec / Perception-Cognition Motor: 240 msec 2.4.8. Response Execution

Figure 3. Heuristic guidelines based on human information processing

In chapter 2.2 the users action was analyzed with Normans Seven stages of action, and in chapter 2.3 human Factors that connected with each stage of action were understood. After all, because human factors affect the action that operates on the interface, the user interface design has to support each stage of action that related to the characteristics and limitations of human factors. On the basis of central stage decision-making, Seven stages of action illustrated the left three stages of evaluation and the right three stages of execution. In this context, design guidelines could be developed in perceptual and conceptual space, which are accepted from two stages of physical space because of metal characteristics of the interface operation (fig.3). Therefore, each human factor could be supported by five heuristic guidelines, and these heuristic guidelines that connected with each stage of action could be defined clearly. As we all know, a good user interface is the support of users behavior for wanted result. From this common sense point of view, five heuristic guidelines that support human action and constitute the heart of usability analysis could be composed such as the following. In the bottom-up processing of evaluation, 1 Recognition & Feedback rule is supported at interpreting the perception stage, and 2 Simplicity & Aesthetic rule is aided at evaluation of interpretations stage. In the Top-down processing of execution, 3 Consistency & Standards rule is supported at forming the goal stage, 4 User Control & Error Management rule is aided at intention to act stage, and 5 Efficiency & Flexibility rule is supported at sequence of actions stage. These heuristic guidelines were developed by analysis of different heuristics, which were proposed by experts of cognitive psychology, with common denominators [17], and recomposed according to human factors and each stage of action.

2.5. Heuristic Guidelines According to Nielson, heuristic evaluation is a discount usability engineering method for quick, cheap, and easy evaluation of a user interface design. Heuristic evaluation is the most popular of the usability inspection methods. Heuristic evaluation is done as a systematic inspection of a user interface design for usability. The goal of heuristic evaluation is to find the usability problems in the design so that they can be attended to as part of an iterative design process. Heuristic evaluation involves having a small set of evaluators examine the interface and judge its compliance with recognized usability principles [18]. However, heuristic evaluation had some weak points that heuristics were not systemically arranged. Because each expert proposed their own heuristics, there were too many heuristics. They had randomized order, and overlapped each other, so the most of users were apt to be confused. For this reason, the concept of heuristics needed to be arranged in the consistent point of view. New heuristics were inferred on the basis of human factors and action stages (fig.4), and following guidelines should be employed:
2.5.1. Recognition & Feedback Interpreting the perception stage: 1 Recognition & Feedback rule is supported in order to help to map from system state to interpretation. For the interpretation process of users, the system has to visually provide the current situation, the next procedure, and the available function to do. Guidelines: 1-1 Affordance, 1-2 Indicating of Task Process, 1-3 Helpful Guidance, 1-4 Immediate Feedback, 1-5 Obvious Feedback for Intuitive Judgment, and 1-6 Multiple Resources. 2.5.2. Simplicity & Aesthetic Evaluation of interpretations stage: 2 Simplicity & Aesthetic rule is aided in order to help to evaluate if system is in desired state. For the evaluation process of users, the system has to remove unnecessary information, avoid from complicate function, and lead user to easy and simple interface. Guidelines: 2-1 Subtractive Design, 2-2 Straightforward Interface, 2-3 Not Readability, But Scannability, 2-4 Reduced User's Memory Load, 2-5 Visual Hierarchy, and 2-6 Difficulty. 2.5.3. Consistency & Standards Forming the goal stage: 3 Consistency & Standards rule is supported in order to help determine the function of the device. For the decision-making process of users, the system has to provide a standard design, make user-centered metaphor, and use existing user knowledge. Guidelines: 3-1 Familiarity, 3-2 Visual Scheme, 3-3 Unified Command Syntax, 3-4 User-Oriented Terminology, 3-5 Using User's Concept Model, and 3-6 Variety. 2.5.4. Efficiency & Flexibility Intention to act stage: 4 User Control & Error Management rule is aided in order to help with the purpose of what actions are possible. For the intention process of users, the system has to allow customized preferences, provide a series of shortcuts, and automate batch-processing. Guidelines: 4-1 Personalization, 4-2 Levels of Importance, 4-3 Shortcut, 4-4 Versatility, and 4-5 Satisfaction of Progress and Achievement. 2.5.5. User Control & Error Management Sequence of Actions stage: 5 Efficiency & Flexibility rule is supported in order to help mapping from intention to physical movement. For the selection process of the users, the system has to give control to the user, protect from errors, and visualize task progress. Guidelines: 5-1 Everything in User Control, 5-2 Recognition rather than Recall, 5-3 Protecting the User from Mode, 5-4 Protecting against Destructive Errors, and 5-5 Keeping the User out of Trouble.

These heuristic guidelines can be used for interface design guidelines in the pre-process: design planning, and applied for design evaluation checklists in the post-process: usability testing. And, they can be the critical basis for

the analysis of users eye-movement recording, because they help to understand the relationship of each stage of users action and design principles.

Figure 4. Heuristic guidelines related to each factor of human information processing

3. Result For the analysis of users eye-movement recording, following experiments were made on Eyegaze Interface System. By these experiments on interactive media, it is explained that each stage of action intimately related to human factors. The interactive media were two mediums: interactive TV based graphic user interface mobile phone based graphic & physical user interface (GUI+PUI). 3.1. Graphic User Interface Figure 5 illustrates the paths and fixations of users eye-movement that were changed according to each menu depth of TV interface. Given situation: a user who wanted to know his or her own stock quotations was using an interactive TV. First interface was a menu selection, second interface was a stock selection, and third interface was a selected stock display. The first interface showed bottom-up processing: accepting overall information, browsing wanted information, and top-down processing: deciding next function, selecting final button. The three dimensional buttons led the user to click and trigger functions to quickly get information
(supported 1-1 Affordance), (GUI)

and

and the simple interface helped the user

(supported 2-1 Subtractive Design).

The second interface showed bottom-up processing:

acquiring wanted information, and top-down processing: selecting an element for more information. The intuitive

color coding led to grouping some types of information or her own current stock quotations at the first glance

(supported 2-2 Straightforward Interface),

and alphabetic search lower navigation

guide helped to find wanted information (supported 2-5 Visual Hierarchy). The third interface aided the user to get his
(supported 2-3 Not Readability, But Scannability),

buttons provided consistent structure to help users stable decision (supported 3-2 Visual Scheme).

Figure 5. The menu depths of interactive TV

3.2. Graphic & Physical User Interface Figure 6 illustrated the paths and fixations of users eye-movement that were changed according to not fixed GUI frame and fixed PUI frame. Given situation: a user who wanted to handle alarm setting was using his or her mobile phone. Each eye fixation of four figures had different heights, so it was shown that their interfaces did not provide consistent structure in each depth of menu
(not supported 3-2 Visual Scheme).

Because third interface suddenly changed hard key mode into soft key mode, the users eye fixations were consciously concentrated in lower uncertain menu
(not supported Figure 6. The menu depths of mobile phone 5-3 Protecting the User from Mode).

4. Discussions & Conclusion In this paper, we can find the interest area of the user by the observation of diverse & specific exploration of eye-movement, and understand the interpretation/evaluation and selection/execution stages by the analysis of topdown and bottom-up processing of visual perception process. Our results provide the implications of design through the analysis of the users eye-movement recording that were changed according to each menu depth of the interactive media. And, we propose the new concept of heuristic guidelines based on each stage of action that is related to human factors. The benefits of these systemically arranged heuristic guidelines are the capability of successful usability analysis for each stage of action, and the application of design principles in connection with the characteristics and limitations of human factors. Because the order and relationship of each heuristic guideline are understood clearly, we can arrange and classify past confused heuristics for practical design work. The limitations of this paper are the indirect experimental condition because of not real product but monitor display; however, this condition can avoid from an obtrusive method that users may be constantly aware of their performance being monitored. And, only novice users of knowledge-based behavior are considered; yet, expert users eye-movements of skill-based behavior are different from novice users. In future work, we plan to develop real-time recording software of users eye-movement for actual interactive

interface in addition to mouse tracking function, and include not only novices but also experts as participants. And also, we plan to propose new specific heuristic guidelines with optimized new media such as mobile phone and information appliance. References 1. Norman, Donald A. Dinge des Alltags, Gutes Design und Psychologie der Gebrauchsgegenstnde. Campus Verlag, Frankfurt, New York, 1989. 2. Hammer, Norbet. Objects and Images: Relevant Cognitive Models for Design Products. University of Industrial Arts Helsinki UIAH, 1992. 3. Just, Marcel A. and Carpenter, Patricia A. Eye Fixations and Cognitive Processes. In Cognitive Psychology, 8, 1976, p. 441-480. 4. Hammer, Norbert. Analysis of User Interfaces by Eye-Movement Recording. Applied Research in Industrial Design, 1991. 5. Trumbo, Jean. The Spatial Environment in Multimedia Design. Design Issue: Volume 13 Number 3, Autumn 1997. 6. Norman, Donald A. The Design of Every Day Things. Doubleday Publishing, 1988. 7. Arnheim, Rudolf. Visual Thinking. California Press, 1969. 8. Berlyne, D. E. Aesthetics and Psychobiology. Appleton Century Crofts, 1971. 9. Kantowitz, B.H. The role of human information processing models in system development. Proceedings of the Human Factors Society 33rd Annual Meeting (p. 1059-1063). Santa Monica, CA: Human Factors Society. 10. Best, J.B. Cognitive psychology (4th ed.). St. Paul, MN: West Publishing, 1995. 11. Anderson, J.R. Cognitive psychology (4th ed.). New York: W.H. Freeman, 1995. 12. Miller, G.A. The magical number seven plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63, 1956, p. 81-97. 13. Schank, R.C., and Abelson, R. Scripts, plans, goals, and understanding. Hillsdale, NJ: Lawrence Erlbaum Associates, 1977. 14. Wilson, J.R., and Rutherford, A. Mental models: Theory and application in human factors. Human Factors, 31(6), 1989, p. 617-634. 15. Schneider, W. and Shiffrin, R.M. Controlled and automatic human information processing: I. Detection, search, and attention. Psychological Review, 84, 1977, p. 1-66. 16. Wickens, C.D. Engineering psychology and human performance (2nd ed.) New York: HarperCollins, 1992. 17. Sde, Simo. Cardboard mock-ups and conversations: Studies on user- centered product design. University of Art and Design Helsinki UIAH Publications, Doctoral thesis, 2001, p. 20-24. 18. Nielsen, Jakob. Heuristic Evaluation, http://www.useit.com/papers/heuristic/ 19. Wickens, C.D., Gordon, S.E., and Liu, Y. Introduction to Human Factors Engineering. Longman, 1998 20. Solso, R.L. Cognition & the Visual Art. MIT Press, 1996.