The Impact of Cognitive Theory on Multimedia Design Kristi J.

Mesler Boise State University [EdTech 504-4172 April 27, 2010]

K.MESLER Abstract

Cognitive theory, specifically the theories of multimedia learning, mental models, and cognitive load, can be combined together to create successful multimedia lessons. Instructional materials and events act with what learners already know to construct a mental model in the learner that can be used to develop understanding. Multimedia can be used to optimize learning, using a combination of auditory, text, and images for information processing. When learners participate in multimedia lessons that provide intrinsic and extrinsic motivational rewards, they are more able to filter, organize, select, and integrate materials introduced in the lesson. John Sweller, Richard Mayer, and Kenneth Craik have produced a list of principles for use in multimedia lesson construction. They include the personalization principle, the segmenting and pretraining principles, the modality principle, coherence principle, redundancy principle, and the contiguity principle. All of these principles are discussed, providing specific directions on how to use words with graphics and narration in lesson planning, whether in placement or by breaking complex information into smaller segments for easier cognitive processing.

Everyone has a personal computer the brain. Instructional design that does not take into consideration the cognitive function of the brain will be hit-and-miss at best in terms of effectiveness. Using Richard Mayers Theory of Multimedia Learning, and Kenneth Craiks Theory of Mental Models, successful multimedia lessons can be created using the specific instructions given by Mayer and Craik, drawing on knowledge of the brains cognitive function. It has been determined that the brain has both long-term and short-term memory. The purpose of instruction is to add information to long-term memory, which semipermanently holds everything from memorized facts, like a telephone number, to complex procedures and concepts, such as philosophy. Learning something new makes a change in long-term memory (Sweller, van Merrienboer, & Paas, 1998). Short-term memory, or working memory as John Sweller (2009) calls it, is considered to be limited in its processing of current information, and contains that information for a very short duration (Dosher, 2003). Recent and advanced theories distinguish two subsystems in working memory. The first is visuospatial, the seeing of text, images, and diagrams. The second is phonological, or the hearing of audio narration (Yuan et al., 2006). Working memory can be overloaded easily if too many pieces of information are brought in at the same time, so Sweller suggests that instructional designers should limit cognitive load. Cognitive load may be viewed as the level of mental energy required to process a given amount of information. The more information a person has to learn in a shorter amount of time, the harder it is to process

K.MESLER

that information in working memory. For instance, a student that is proficient in Microsoft Excel will have no trouble completing a spreadsheet assignment, while another student without proficiency will find the cognitive load to be much higher. Recent findings suggest that knowledge acquisition is a process of constructing mental representations within concept frameworks from visual and pictorial representations (Bransford, Brown, & Cocking, 2000). Graphics can help learners build what Kenneth Craik called mental models. Instructional materials and events act with what learners already understand in order to construct a mental model that the student can use to develop understanding. A learner will bring knowledge, skill or ability already known (prior knowledge) to each new learning situation. Learners need enough previous knowledge and understanding to learn new concepts and make links between prior and new knowledge. Instructors work to activate prior knowledge at the beginning of each new topic. Whether by discussion, making lists, classification or other approaches, instructors try to guide learners minds to be in tune with the new topic. Sometimes, it involves drawing out prior knowledge the learners did not even know they had, such as a real world application for math before they learn the actual mathematical formula. When learners merge new lesson content with activated prior knowledge in working memory, they form new mental models in their long-term memory (Clark & Lyons, 2004). Mental models are psychological representations of real, hypothetical, or imaginary situations. When instructors first hear about mental models, they suppose that they are nothing more than mental pictures or images. In fact, some models cannot be visualized and images depend on underlying models. For instance, learners can see

 a three-dimensional object and then mentally rotate the object to a different orientation.

Models can also represent abstract concepts, like love, which is impossible to visualize. Mental models are used to further reasoning, as well as explaining how people can use their knowledge of different ways an event can occur, in order to calculate probability. A problem that can occur is when some learners may only focus on their own model, while not taking other outside information into consideration. Richard Mayer (1989) lists seven criteria that instructional materials should meet in order to have learners construct mental models that are likely to improve understanding. Mayer refers to illustrations and text as conceptual models that describe in graphic form objects and causal relations among them. A good conceptual model is complete, concise, coherent, concrete, conceptual, correct, and considerate. This means it contains all the objects, states, and actions of the system, has just enough detail, makes intuitive sense, has appropriately familiar content, is meaningful, corresponds to actual objects and events, and uses appropriate organization and vocabulary. If these criteria are met, then instruction can lead to the creation of models that help students understand systems. How do mental models fit in with multimedia lesson instruction? First, it is necessary to understand the theory of multimedia learning. Multimedia can be used to optimize learning. Information processing is a combination of auditory and visual, words, and images. Richard Mayer devised the Theory of Multimedia Learning in 1997. He believes that learning is an active process of filtering, selecting, organizing and integration information. When a multimedia presentation of text, images, and/or auditory information is given, these items are not processed individually; rather, the

K.MESLER

brain selects each of them and organizes them together to make logical mental constructs, or schema. Multimedia presentations using this type of approach, allows the learner to control the speed of the information for processing, and is dependent on the learner to find motivation for learning, either intrinsically or extrinsically. Activated prior knowledge may provide the intrinsic motivation to learn new information. This type of motivation comes from the pleasure that a learner gets from the task itself or by working or completing a task. Intrinsic motivation does not mean that a learner will not seek external rewards. It means that external rewards are not enough to keep a learner motivated. For example, if a student may want a good grade on a task, but the task itself is not of interest to him/her, the possibility of getting a good grade is not enough motivation to keep the students efforts going. When intrinsic motivation is not enough, extrinsic motivation can be added. Extrinsic motivation comes from outside the individual. Motivating factors may include rewards, such as grades, or money. Rewards can be something as simple as a happy face sticker to something major, like celebrity or wealth. Extrinsic motivation does not mean that a person will not get pleasure from completing a task. It simply means that even if the learner has little or no interest in the task, the reward will still be a motivator. Practical application of the multimedia learning theory involves having cognitive load, mental models, and motivation in mind when creating multimedia presentations. Cognitive load theory, alluded to earlier in this paper, affects the amount of visuospatial and phonological items used in a multimedia design. It is important to not overload audio or visual inputs at once. In order to understand how much is too much, it is

 valuable to recognize the three types of cognitive load: intrinsic, extraneous, and germane (Sweller, Van Merrienboer, & Pass,1998). Intrinsic cognitive load refers to the complexity of the information to be learned.

This load is described through the construct of element interactivity how many related elements does the learner have to hold in working memory in order to understand the learner content? Extraneous cognitive load describes the demands on the working memory caused by the way educational materials are designed, both in how it is presented and the instructional strategies used. Extraneous load is created by information that has to be processed by the learner even though it does not contribute to learning, such as decorative pictures or clip art. Instructors should work to decrease this type of cognitive load in their instructional planning. Visual elements, such as video footage or animated depiction of a process can lower extraneous cognitive load. Germane cognitive load describes the mental effort that a learner invests into processing information, rather than cognitive load imposed upon the learner by the learning materials. Instructors must design lessons to increase germane load. There is growing evidence that well-designed visuals and text benefits the novice learners in a lesson. Mayer and Clark (2008) believe learning is a process of making sense and instruction is seen as an attempt to foster appropriate cognitive processing in the learner. This theory insists that instructors must guide the learners cognitive processing during learning, encouraging learners to actively process the information by mentally constructing pictorial and verbal representations of the material. The multimedia principle addresses this active learning by stating that instructional materials must include words and graphics, rather than words alone in e-learning. Words are

K.MESLER

printed text to be read or spoken text to be heard. Drawings, charts, graphs, maps, animation or video are all graphics. A multimedia presentation refers to any presentation that contains both words and graphics. Through multimedia, active learning is more engaging to learners, because they are cognitively processing, organizing, and attempting to assimilate the new information with prior knowledge. Selecting graphics to accompany words takes thought as well. There are several different types of graphics, including decorative, representational, organizational, relational, transformational, and interpretive (Clark & Lyons, 2004). 1) Decorative graphics that do not enhance learning become an extraneous cognitive load on the learner. For example, if the text information is about parts of a butterfly, but your graphic is of a child trying to catch a butterfly, the graphic does not match the text, and therefore becomes extraneous to the text. Often, decorative graphics are included to add visual appeal or interest in the subject matter. 2) Representational graphics illustrate the appearance of lesson content. They are created to show what the content looks like realistically. Photographs, sketches, and computer screen captures all can be titled and used as a representational graphic on a page. 3) Relational graphics compare quantities of two or more elements. These visuals may include line or bar graphs, pie charts, or other visual comparisons. 4) Organizational graphics show how objects work together. These can include a labeled diagram of an object, like a car engine, or an organization of ideas through concept maps, tree diagrams, or the creation of matrices. Robinson and

 Molina (2002) state that some organizational visuals improve learning of conceptual relations better than outlines of the same information. 5) Transformational graphics show changes over time. An animation of a procedure from start to finish, line drawings with movement indicators, and timelapse photography are examples of transformational graphics.

6) Interpretive graphics make the invisible or abstract or both, visible. For instance, a graphic showing the airflow over a planes wing makes the air visible. Football coaches use Xs and Os to represent offense and defense, while a dashed line represents the path of the football traveling through the air. This is another example of an interpretive graphic. Instructional materials that include graphics must support critical psychological events, in order to improve learning. They need to direct learner attention to important information in the lesson, activate prior knowledge in memory, manage cognitive load to free long-term memory for learning, build mental models, support the transfer of learning to application to near-transfer or far-transfer tasks, and motivate learners to initiate and complete learning goals. Certain types of graphics are better suited for specific learner content. Factual content, unique and isolated information is best represented by representational or organizational graphics. Content that includes concepts, or groups of facts, are most likely seen in representational, organizational, or interpretive graphics. Description of a process would be best shown in transformational, interpretive, or relational graphics. A procedure with a series of steps is illustrated with a transformational graphic to show its

K.MESLER

10

completion. Cause-and-effect relationships, or principles, may use a transformational or interpretive graphic, such as comparison videos or animations (Mayer, 2008). Contiguity principle #1 addresses the placement of graphics and text on a page. The desired placement of text is near the corresponding graphic, where the learner can more easily make mental connection between corresponding words and graphics. Ideally, text will appear next to the graphic, rather than above or below, except when used as an image caption. If the graphic is a diagram, the labels of the parts should be placed near the corresponding parts of the diagram with a line pointing from the label to that part of the diagram. When there is too much text for the computer screen, the use of rollovers on the diagram is suggested. Contiguity principle #2 says to synchronize spoken words with corresponding graphics. It is distracting when the words do not match the graphic. Narration needs to be carefully timed to avoid this problem. For example, when the graphic is an animation showing the steps in a process, the narration describing the step should be presented at the same time that the step is shown on the screen. When a lesson separates corresponding words and graphics, the learner has more cognitive load to process the information, which makes the working memory work harder, and the long-term memory has less time to process and hold information. When the opportunity arises to use audio, the modality principle suggests that words should be presented as audio narration, rather than on-screen text. This principle is directly tied in with the earlier discussion on working memory subsystems. The two subsystems, the visuospatial (seeing) and phonological (hearing) can become

 overloaded when there are multiple bits of information presented in one of the subsystems. When a learner has to read a large amount of text, or the material is

11

complex and presented at a rapid pace, and then additionally viewing a graphic as well, it causes overload of the visual portion of the brain. When information is presented with narration and graphics, the graphic enters the visuospatial subsystem, while the narration enters the phonological subsystem. With both subsystems being equally involved, it does not cause an overload for any one subsystem. Redundancy principle #1 also addresses the cognitive load required for information processing by insisting that on-screen text is not added to graphics that are already being narrated. The duplication of text and narration is redundant. When learners are looking at the printed words, they are unable to look at the on-screen graphics at the same time while listening to the narration. However, there may be situations that would be improved by having text and narration at the same time. Redundancy principle #2 considers adding on-screen text to narration in special situations. If there are no pictures, when on-screen text and corresponding graphics are presented sequentially, when the pace of presentation is sufficiently slow, or when the learner must exert much greater cognitive effort to comprehend spoken text than printed text, the addition of on-screen text is allowed (Clark & Mayer, 2008). Mayer and Moreno (2003) coined the term weeding as the need to uproot any text, graphics, or audio that are not at the center of the lessons instructional goal. Clean and concise are the words of the day, when it comes to multimedia lesson design. For guidance in our weeding, we look at coherence principle #1. Based on

K.MESLER

12

the psychology of learning and further research, it is recommended to avoid the addition of extraneous sounds such as background music or environmental sounds. Background music and sounds may overload working memory when the material is unfamiliar, or presented at a rapid rate, or when the learner does not have control of the rate of the presentation. If the learner is having heavy cognitive load with just the information, think twice before including any additional music or sounds. Coherence principle #2 is along the same lines as #1, but addresses the graphics. To keep the multimedia presentation concise, avoid e-lessons with extraneous graphics. Harp and Mayer (1998) found that extraneous pictures might interfere with learning in three ways. They are a distraction, by guiding the learners limited working memory toward the irrelevant material, rather than the relevant. Learners are unable to build mental links because the irrelevant material causes disruption. By activating extraneous prior knowledge, added pictures seduce the mind to latch onto them instead of the appropriate material. Coherence principle #3 recommends avoiding adding extraneous words to lessons. Stories and trivia, while fun, may result in poorer learning. Extraneous words added for interest, elaboration, or for technical depth should be avoided. Mayer and Jackson (2005) conducted an experiment using concise and embellished technical information. Learners with concise information scored higher than those with the embellished lessons. Mayer and Jacksons conclusion was that the added quantitative details might have distracted the learner from constructing a quality mental model of the process.

 The personalization principles are aimed toward getting learners to feel comfortable with the information presented. Principle #1 guides instructors to use conversational rather than formal style, while principle #2 asks instructors to add onscreen coaches to promote learning. Principle #3 says to make the author visible to promote learning. What does this all mean? Making the verbiage conversational is more about using first or second person in the text, and these tenses make an author

13

more visible. Adding on-screen coaches can come from the addition of animation and interactivity. The segmenting and pretraining principles are related to the reduction of cognitive load while processing information. When material is complex, it cannot be made any simpler by leaving out a step of a procedure. The segmenting principle suggests that lessons like this can be managed by breaking a lesson into bite-size segments. These segments are then presented one at a time for better absorption by the learner. The pretraining principle helps ensure success with complex lessons when instructors make sure that learners know the names and characteristics of key concepts before tackling more difficult information. Multimedia learning is successful when the theories of mental models and multimedia learning are combined. Properly constructed mental models and multimedia lessons work together to manage intrinsic, extraneous, and germane cognitive load. Specific applications to lesson planning include explaining in words and pictures, using narration and animation, aligning word to corresponding graphics, by not adding extraneous audio, graphics, or redundant text, breaking complex concepts into bite-size portions, and by using conversational style and virtual coaches.

K.MESLER

14

References Berends, I. E., & van Lieshout, E. C. D. M. (2009). The effect of illustrations in arithmetic problem-solving: Effects of increased cognitive load. Learning and Instruction 19, 345-353. Bransford, J.D., Brown, A.L., & Cocking, R.R. (2000). How people learn: brain, mind experience and school. Washington, D.C.: Academic. Byrne, R.M.J. (2002). Mental models and counterfactual thoughts about what might have been. Trends in Cognitive Sciences, 6(10), 426-431. Retrieved from http://www.tcd.ie/Psychology/other/Ruth_Byrne/publications.html Clark, R.C., & Lyons, C. (2004). Graphics for learning. San Francisco, CA: Pfeiffer. Clark, R.C., & Mayer, R. (2008). e-learning and the science of instruction: 2nd edition. San Francisco, CA: Pfeiffer. Clark, R.C., Nguyen, F., & Sweller, J. (2006). Efficiency in learning: evidencebased guidelines to manage cognitive load. San Francisco, CA: Pfeiffer. Cooper, G. (1990). Cognitive load theory as an aid for instructional design. Australian Journal of Educational Technology, 6(2), 108-113. Retrieved from http://www.ascilite.org/au/ajet/ajet6/cooper.html de Jong, T. (2009). Cognitive load theory, educational research, and instructional design: some food for thought. Instructional Science 38, 105-134. doi:10.1007/s11251-009-9110-0.

 Dosher, B. (2003). Working memory. Encyclopedia of cognitive science, 4, 569-577. New York, NY: Wiley. Harp, S.F., & Mayer, R.E. (1998). How seductive details do their damage: A theory of cognitive interest in science learning. Journal of Educational Psychology, 90, 414 434. Johnson-Laird, P.N. (2005). Mental models and thought. In K.J. Holyoak & R.G. Morrison (Eds.), The Cambridge Handbook of Thinking and Reasoning (pp. 185- 208). doi:10.2277/0521531012. Cambridge, England: Cambridge University Press. Johnson-Laird, P.N., Girotto, V., & Legrenzi, P. (1998). Mental models: a gentle guide for outsiders. [No publication listed.] Retrieved from http://icos.groups.si.umich.edu//gentleintro.html Mayer, R.E. (1989). Models for understanding. Review of Educational Research, 59, 43-64. Mayer, R.E., & Jackson, J. (2005). The case for coherence in scientific explanations: Quantitative details can hurt qualitative understanding. Journal of Experimental Psychology: Applied, 11, 13-18. Mayer, R.E., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38, 43-52.

15

K.MESLER

16

Robinson, D.H., & Molina, E. (2002). The relative involvement of visual and auditory working memory when studying adjunct displays. Contemporary Educational Psychology, 27, 118-131. Sweller, J., van Merrienboer, J., & Paas, F. (1998). Cognitive architecture and instructional design. Educational Review 10, 251-296.

Von Merrienboer, J.J.G., & Ayres, P. (2005). Research on cognitive load theory and its design implications for e-learning. Educational Technology Research and Development, 53(3), 5-13. Yuan, K., Steedle, J., Shavelson, R., Alonzo, A., & Oppezo, M. (2006). Working memory, fluid intelligence, and science learning. Educational Research Review, 1, 83-98.