Design

and Strategies for New Media, Fall 2011

Usability Evaluation, Research Component

Primary Source Educational Collections

Rob Scordino
11/20/2011

Contents Introduction Sites Evaluated

Presidential Timeline of the 20th Century Presidential Recordings Program Digital Classroom Initiative Library of Congress Teachers Website Target Users

3 4 5 6

Methodology
Participants Questions Tasks 6 7 7

Design
Outline Logistics Data Collection 8 9 9

Findings
Library of Congress Presidential Recordings Project Presidential Timeline 10 13 15

Discussion
Library of Congress Presidential Recordings Project Presidential Timeline 17 22 25 27

Conclusion Appendices
A- Consent Form B- Background Questionnaire C- Exploratory Task Protocol D- Specific and Open-ended Task Protocol E- Post-evaluation Survey

30 31 32 33 34

2 | P a g e

Introduction
When used in the history classroom, primary sources provide first-hand accounts of the specific time period being studied. Using primary sources in the classroom does more than provide mere supplement to textbook curriculum. The use of such sources can promote historical thinkinga way of learning history through inquiry and investigation. Historical thinking through analysis of primary sources engages students in their learning and helps them develop critical thinking and research skills. Access to primary source collections (from archives, libraries, and museums) has greatly increased due to recent digitization efforts and improved Internet speed. However, access alone does not guarantee teachers will be able to find and use the sources they need. Large archive may not offer acceptable organization or user-friendly interfaces for teachers to find content in their limited free time. To address this issue, many organizations have made efforts to create themed primary source collections and lesson plans for teachers to access online. This paper outlines a usability study of three such websites.

Sites Evaluated
Presidential Timeline of the 20th Century The first site decided upon was the Presidential Timeline Project (www.presidentialtimeline.org) (Figure 1). This project (funded by the National Endowment of the Humanities and the LBJ Presidential Library) is a collaborative project between the Presidential Libraries. The project contains collections of digitized documents from each of the 13 libraries. These collections are organized into learning activities for teachers to use. The project is affiliated with the National Archive and Records Administration (of which the presidential libraries are part), and links to additional resources from the NARA.

3 | P a g e

Figure 1. Presidential Timeline of the 20 Century

th

Using the Presidential Timeline as a model, criteria were established for the other two sites to be chosen. These sites: Must contain collections of primary sources organized into educational objects Must be targeted toward teachers and students Must be free Must have not be connected to a political organization (nor have any obvious bias) A Google Search of Primary Source Collections brought up many hits. There were many sites that contained good collections, but were not organized for easy use in the classroom, and therefore did not fit my criteria. After some searching, two sites were decided upon: The Miller Centers Presidential Recordings Project (http://www.whitehousetapes.net) and the Library of Congress Teacher Collections (www.loc.gov/teachers). Presidential Recordings Project, Digital Classroom Initiative The Presidential Recordings Project (PRP) is part of the Miller Center- a non-partisan Institute with the purpose of expanding the understanding of the president and policy. The PRP contains links to presidential tapes and transcripts from those tapes from the
4 | P a g e

FDR through the Nixon administrations. The Digital Classroom Initiative (DCI) was designed to help teachers use these presidential recordings. The DCI grants educators easy access to the PRPs unique flash audio-transcript clips by grouping these, as well as the more extensive virtual exhibits, along topical lines. It also provides links to course syllabi and classroom activities featuring the tapes themselves.

Figure 2. Presidential Recordings Program

The Library of Congress Teachers Resource The Library of Congress website provides access to digitized copies of many documents, pictures. And other objects housed by the Library of Congress in Washington, D.C. The Library of Congress Teachers site contains thematic collections for use by educators. These collections are able to be searched by date, topic, or state standard. In addition to the resources, the LOC Teachers site provides lesson plans for teachers as well as links to other government resources for teaching through primary sources.

5 | P a g e

Figure 3. Library of Congress Teacher Website

Target Users These sites may be used by anyone who is interested in history. However, the target audience for these sites is classroom teachers. Specifically, teachers who will use these sites are likely to be history teachers of secondary grades. Therefore, assumptions can be made about the likely end user of these sites. These users will: Have a college education Have basic Internet and research skills Be interested in the content Be motivated to successfully use a site like the ones being tested

Methodology
Participants Two participants were selected to perform a usability test of the three websites. The participants were selected by the moderator through convenient sampling. Although chosen from a convenient sample, the participants were adequate representations of the probable end user of these websites. As a secondary history teacher, one participant (Matthew) wasin fact-- an exact representation of an end user. The other
6 | P a g e

participant (Kristin) is a science teacher. Although she is not a history teacher, she is familiar with teaching secondary students, and was therefore acceptable as a participant. The participants interests and background information was collected through pre-test questionnaire. The information is as follows: Kristin W: - early 30s, female - Employment: Middle School Teacher (Science), 4th year - Education: B.S. Biology, Some Grad School - Computer experience: Medium. Uses Internet frequently for browsing, email, finding sources for lesson plans. She has never used sites for primary sources, but has used teacher websites for science education. - Content interest: She expressed interest in using more technology in her teaching, but admits that she hasnt been able to do it to the extent to which she would like. She is not an expert in history or in history education, and is not likely to visit these sites on her own. Matthew S. - late 20s, male - Employment: High School Teacher (Social Studies), 5th year - Education: B.A. History, Masters Degree - Computer experience: Medium to High. Uses Internet frequently for browsing, email, lesson planning. He has used the Library of Congress site and similar sites to find primary sources for lesson plans. - Content Interest: As a history teacher with a content background, he has a high interest in both history and teaching with primary sources. He would (and does) visit websites like those being tested. Questions The purpose of this usability evaluation is to investigate the websites ability to address the needs of the likely end user and to determine design attributes that contribute to the usability of these sites. Therefore, two questions guided this study: To what extent do these three websites meet the usability needs of teachers? What characteristics of information, interface, and interaction design contribute to the usability of these sites?

Tasks The participants of this study were given a series of tasks, which are described in the
7 | P a g e

Design section of this paper. Each of these tasks will fit in one of three types of tasks as defined by Nielsen, et al. (2000) (exploratory, specific, or open ended). The purpose of each task is to mimic the probable use of an end user. The likely end user of these sites are classroom teachers, thus, the tasks will mimic the likely actions of teachers: Explore the site and give impressions (exploratory) Find a document (specific) Plan a lesson (open-ended)

Design
A single usability evaluation of all three websites was run in two individual participant sessions. The design of the usability evaluations were greatly influenced by Rubin and Chisnells (2008) Handbook of Usability Testing. In each session, the participants evaluated all three websites. The evaluation session consisted of (1) participant background questionnaire and orientation (2) performance evaluation of each website, (3) post-evaluation questionnaire, and (4) participant debriefing. Background questionnaire and orientation The participants received a short, verbal introduction and orientation to the evaluation. During this time the moderator explained to them the purpose and objective of the evaluation, and additional information about what is expected of them. Participants were assured that the websites were the center of the evaluation and not themselves, and that they should perform in whatever manner is typical and comfortable for them. The participants were told that they will be assigned several tasks, and the moderator will observe and take notes as they complete those tasks. They were also informed that screencasts may be taken during the evaluation. After this was explained, the participants filled out a consent form (Appendix A) as well as a short questionnaire (Appendix B) about relevant aspects of their background. Performance evaluation After the orientation, the participants sat at the evaluation computer. Each website that was to be evaluated was preloaded on separate browser windows. Each site was evaluated using the same tasks. The evaluation tasks consisted of the following: 1. 2. 3. 4.

Browse site and give initial thoughts Find information for teachers about teaching with primary sources Find an image of any president Search for a specific file (see Figure 4)
8 | P a g e

5. Using the available tools, investigate a topic for use for a lesson plan. Task #4 Library of Congress Find Ben Franklins Join or Die political cartoon from the Revolutionary Era Presidential Recordings Find the recording of the phone conversation between Lyndon Johnson and Martin Luther King, Jr. Presidential Timeline Find Ronald Reagans copy of the speech following the Challenger disaster

Post-evaluation questionnaire After each website is evaluated, Participants were asked to fill out a post-evaluation survey. The survey was designed to make the participants rate aspects of information design, interface design, and interaction design of the site, as well as the participants overall satisfaction with each site. These questions will ask the participant to evaluate aspect of these four elements using a Lykert scale of 1 to 5. Participants will also be given the chance to make suggestions to improve the usability of each website. Participant debriefing A debriefing session took place after the participant was finished evaluating all three sites. Any further questions the administrator had following the evaluation were asked during this time. After the debriefing session, the participants were thanked for their efforts, and released. Logistics Each participant was tested separately and evaluated all three sites in the same individual session. Each participant performed their test in the in the environment where they most often planned lessons. Evaluating the usability of a product at a users normal place of work is advantageous in that it evaluates the product being used in realistic settings. Evaluating in context is also easier on the user, because they have to schedule less time than if they were to travel to a testing site, and they may be more comfortable working in a familiar environment (Rubin and Chisnell, 2008, pp. 100). In each evaluation, the testing environment most closely resembled the simple singleroom setup (see Figure 4) from Rubin and Chisnels Handbook.

9 | P a g e

Figure 4. Simple single-room setup.

The evaluation was done using a 17 inch Macintosh Powerbook with a dual-core 2.66 Ghz processor, 4 GB of RAM and connected to the Internet via high speed wi-fi. This configuration exceeds the requirements needed to use each of the sites tested. Participants were given the choice of using OSX Snow Leopard or Microsoft Windows 7, 64-bit. The participants were also given the choice of using their preferred browser: Mozilla Firefox, Google Chrome, Apple Safari, or Mircrosoft Internet Explorer. Both participants used OSX with the Firefox browser, as this was the configuration each was most familiar. Neither of the participants required additional supports to meet accessibility needs. In a larger-scale usability test, it would be important to involve participants with varying abilities to address accessibility design issues as well. In such cases, modifications would have to be done to the current configuration (such as implementation of screen readers, screen magnification, modified keyboards, etc.) to properly evaluate the usability of these websites. Data Collection Data sources: Background questionnaire Moderator notes from observation Participant evaluation Post evaluation questionnaire Quantifiable data collected from the evaluation included:

The amount of time to complete each task The number of attempts to complete each task The number of errors from which the user could not recover Participants ratings of certain aspects of the website
10 | P a g e

Qualitative data to be collected during observation included: Subjective opinions of the usability and aesthetics of the product expressed by the participants Indications of frustration or joy from the participant Participants answers to open-ended questions Participants suggestions for improvement

Findings
Findings from the participant evaluation of the three sites are discussed in the following section. Descriptions are given of participants experiences completing each of the five tasks. Measurements of time and mouse clicks needed for completion and number of critical errors were taken for tasks two, three, and four. These measurements are also given in this section. Library of Congress Task One Both participants reported using the Library of Congress website in the past. Matthew reported using content from the Library of Congress (mostly pictures) in his classroom. Both participants noted the large amount of information on the LOC Website. Because both participants were at least moderately familiar with the Library of Congress site, they were also familiar with the purpose and target audience. While browsing, both participants seemed pleased with the content that was available. Kristin, who had never visited the Teachers site, first clicked on the More Classroom Materials button. When I asked why she clicked there, she said because it was what I saw first. This brought her to a page of lesson plan topics. She scrolled down through the Lesson Plans, which were organized By Topic. She then stopped at the heading Science and Technology. She said Maybe I could use this, referring to the links to the science-related lessons. After checking a couple of lessons topics (Natural Disasters, Thomas Edison), she seem less interested (The lessons were very history heavy). Although she may not have been able to find something she could use for her class, she did say it was cool that you can search through all these lesson plans already made for you. Although Matthew was familiar with the Teachers LOC Website, he admitted that it had been a while since he last used it. (He said He used the lesson plans from the site often in his first year teaching.) Like Kristin, he immediately interacted with the
11 | P a g e

navigation tool featured on the first page. He scrolled through the featured links for a short while before abandoning it and moving on to the rest of the page. He used the side navigation to go to the Classroom Materials page. Like Kristin, Matthew also browsed the available lesson plans before returning back to the homepage. When asked about his impressions of the site, he said There is a lot of stuff on there for teachersyou could probably plan a whole year using [the Library of Congress]. But, if I was just browsing the site without some sort of goal, I would probably just waste a bunch of time looking at pictures. Task Two
Measure Time Clicks Errors Score (KW) < 10 sec 1 0 Score (MS) < 10 sec 1 0

The LOC Teachers site has extensive information available for teachers who want to use primary sources in their classrooms. Much of the homepage is devoted to such information, including links to professional development modules on using the Library of Congress and Teaching with primary sources. Both participants quickly found the Using Primary Sources option on the navigation menu on the left side of the page. Task Three
Measure Time Clicks Errors Score (KW) 15 sec 3 NA Score (MS) 12 sec 3 NA

It did not take long for either participant to find a picture of a president on the LOC site. Matthew-- who had looked at a collection of Abraham Lincoln-related items while he was browsingsimply retraced his steps to find a portrait of Lincoln. Kristin again used the search function to search for Kennedy. From the results, she was able to find the Presidents Portraits collection and link to a picture of John F. Kennedy. Task Four
Measure Time Clicks Errors Score (KW) 11 sec 2 0 Score (MS) 14 sec 2 0

12 | P a g e

When assigned the task of finding the Join or Die political cartoon, both Kristin and Matthew used the search box to search for join or die. The search resulted in multiple links. Kristin chose the first link, which brought her to a PDF image of the cartoon. Unlike Kristin, Matthew did not choose the first link. Instead, he scrolled through a couple of choices until finding a link to an image of the newspaper on which the cartoon was first published. Task Five One of the main purposes of the Library of Congress Teachers is to provide lesson plans using documents from the Library of Congress. These lessons can be accessed through multiple means, making them easy to find. Also, participants became familiar with the site while browsing, and both had already browsed the collection of lessons. Because of this, neither participant struggled with finding the lesson plans. After choosing a lesson plan topic, both participants were able to link to all necessary materials.

Presidential Recordings Program Task One This was the first time either participant visited the Presidential Recordings Program Website. Both participants recognized that the purpose of this site was to provide teachers with materials for their classroom. Kristin also thought that people researching history might also be a target audience of this site. Both participants noted the simplicity of the layoutMatthew referred to it as simple, and Kristin referred to it as plain and kind of dull. Kristin used the left navigation menu to check the content of the site. She clicked on each menu item to scan what was available on each page. After scanning the site, she clicked on the menu item Topics and said I guess this is where you would go for information for lessons. After continuing in this direction, however, Kristin was brought out of the teacher section into various Virtual Exhibits. I asked her if she noticed that she was no longer in the teacher section and she said she didnt. I asked her to try to get back to the teacher section, and doing so made her realize that the left navigation had changed, so she had to use the browsers back button. Matthew did not scan the entire site while browsing. Instead, he immediately clicked on the Classroom item from the left navigation menu. This took him to a page with some lesson plan ideas. After looking at this page for a while, he said These topics are only from the 60s. I thought there would be more. After browsing a bit more, he decided, I
13 | P a g e

guess I could get some stuff for lessons here. I would probably use the recordings [of the phone conversations]. I think its good to make [historic figures] seem more human. Task Two
Measure Time Clicks Errors Score (KW) N/A NA NA Score (MS) NA NA NA

It is difficult to find information about teaching with primary sources by using the Presidential Recordings Project website. A link to this information is not embedded in an obvious section of the website. Instead, such links are only available on lesson plan pages. Because of this, both participants struggled with the second task. Kristin spent enough time on the task to conclude it isnt there, is it? and gave up. Matthew was more determined to find the information, but was eventually stopped by the moderator to move on to the next task. Task Three
Measure Time Clicks Errors Score (KW) 15 sec 3 0 Score (MS) 28 sec 6 0

Although the presidency is the sole topic of the website, there are not many images of presidents available on the PRP teachers site. Despite this, both participants were able to find a picture of a president with relative ease. Kristin immediate went to the search bar in the top right corner of the page, where she searched for Kennedy. The search resulted with a page of links to parts of the site in which the word Kennedy appeared. She was then able to click on a link labeled Resources which brought her to links of pictures of JFK. Matthew also used the search function, however, he only used it (he searched for LBJ) after he was unable to find a picture by browsing the site. Task Four
Measure Time Clicks Errors Score (KW) 27 sec 3 0 Score (MS) 25 sec 3 0

14 | P a g e

When asked to find the recording of the phone conversation between Lyndon Johnson and Martin Luther King, Jr., both participants once again made use of the search function in the upper right hand corner. Kristin searched for martin luther king phone and Matthew searched for MLK. Both searches came up with similar results, from which both participants could find the recorded conversation. Task Five The final task required participants to find enough content to form a lesson plan. By then, the participants seemed familiar with the website, and neither had any difficulty finding actual lesson plans on the site. However, both encountered issues with finding the materials needed to implement those lessons. Kristin was visibly frustrated during her attempts to find the materials. Ultimately, she was stopped by the moderator and reassured that her inability to complete the task was due to poor design of the website. Matthew also became frustrated while trying to find materials. He stopped his attempt on his own and said, Realistically, if I wanted to use this lesson, I would probably just Google the materials. Time, clicks, and errors were not tallied for task five. Due to the inability or refusal of the participants to complete the task, both instances were considered failed attempts.

Presidential Timeline This was the first time either participant visited the Presidential Timeline website. Both participants identified the PTL as a site for teachers, and Matthew also identified students as a potential target audience. Both participants expressed their opinion of the visual appeal of the PTL over the other two sites. After spending a few moment looking over the homepage, Kristin clicked a picture link to open the Interactive Timeline, which opening in another browser window. While exploring the Timeline, she remarked, this is neat, but whats the point, exactly? Eventually, she clicked on the Educators link (which opened in a new tab of the original browser window). Oh, she said, this is the stuff for teachers. Unlike Kristin, Matthew immediately found the Educators page. From here, he linked to the Classroom Activities page. These look like lessonspretty neat. He said as he scrolled through the activities page. He then investigated a couple of lesson topics. Are these WebQuests? he asked. After realizing that the lessons were WebQuests, he stated that he could very easily use the materials on the Presidential Timeline in his class.

15 | P a g e

Task Two
Measure Time Clicks Errors Score (KW) 13 sec 3 0 Score (MS) 22 sec 5 0

Kristin came across the Teacher Resources page during her browsing, so she easily found the information on teaching with primary sources. Matthew took a bit longer to find the information. He initially went to the homepage, expecting there to be a link to the information he needed. After realizing it wasnt there, he went back to the Educators page. He then read the description of the Teacher Resources link, which led him to the information he needed. Task Three
Measure Time Clicks Errors Score (KW) <5sec. 1 0 Score (MS) <5sec. 1 0

The majority of the pictures on The Presidential Timeline are of presidents themselves. Both participants immediately found such an image to complete this task. Task Four
Measure Time Clicks Errors Score (KW) 23 sec 8 0 Score (MS) 32 sec 13 0

Participants encountered more trouble when trying to find the copy of Ronald Reagans speech. Kristin used the Interactive Timelines Exhibit feature to find the speech. She had to browse around a short while before finding the section containing the document. Matthew, who started this task from the Educators page, instead searched for the image by browsing the Educational Activities. He scrolled down until he came to the lesson regarding Reagans speech. He then navigated through the lesson page until finding the scanned copy of the speech. Although this was not as efficient as Kristins method, it was equally effective. Task Five

16 | P a g e

Like the LOC page, a major purpose of the Presidential Timeline is to provide teachers with lesson plans and materials. By the time the participants attempted this task, they were not only familiar with the site, but familiar with the evaluation process as well. Therefore, they had no trouble finding lesson plans on the PTL site. As with the Library of Congress Page, the Presidential Timeline provided all necessary materials for executing each lesson plan. These materials are part of the PTL and were easily found by both participants.

Discussion
In this section, results of the post-evaluation survey are given. Using the information gathered from participant observation, the survey scores are contextualized. Each site is analyzed in terms of participant satisfaction, information design, interface design, and interaction design. Library of Congress
Design Element Information Interface Interaction Satisfaction TOTAL Score (KW) 19 19 18 14 70 Score (MS) 22 19 19 19 79 Total Possible 25 25 20 20 90

Overall Satisfaction Both participants were generally satisfied with the Library of Congress Teachers website. Matthew-- who had more interest in the content of the LOC-- also reported enjoying the website. As might be predicted Kristin (a science teacher) reported that it was not likely that she would use this site for her teaching. She did, however, report that she would likely recommend this site to a colleague. Matthew had already mentioned that he has used the LOC site in his teaching, and also reported that he would recommend the site to colleagues. Information design Overall, the information design of the Library of Congress Teachers site was rated fairly high by both participants. Both reported finding information easily, which is likely due to good information design. A search box and navigation menu is displayed in the same place on every page of the LOC Teachers site. The consistency of layout allows the user to easily access these search tools. Additionally, important information was often able to be found through a variety of ways. For example, when searching for

17 | P a g e

information on teaching with primary sources participants could see several sections of the homepage devoted to that information (Figure 5).

Figure 5. Multiple ways to access information

Both participants agreed that there was a lot of information available on the website. Matthew, a history teacher, found this abundance to be beneficial. Kristin, however, thought that the information was overwhelming and considered the large amount of information a design flaw. Kristin also rated the organization of the information lower than did Matthew. While completing the performance evaluation, Kristin mentioned that the choice of topics of the available collections and lessons seemed a little random. The lesson plans available from the site come from teachers, who have varied interests. Therefore, Kristins critique of the seemingly arbitrary choices does have merit. Matthew, on the other hand, gave a high rating to the organization of information on the site. During his evaluation, he mentioned the benefit of primary sources being organized by theme and
18 | P a g e

by time period. He also said that he liked the organization of lesson plans, in that he was able to search through them by topic, by era, or by alphabetical order.

Figure 6. Multiple sorting options

Interface design Like the information design, the interface design of the LOC website received good ratings. Almost all of the elements rated were given 4 out of 5 by both participants. This suggests that although the interface was nothing spectacular, it highly usable and moderately appealing. However, both participants noted the text-heavy appearance of many pages. Despite the large amount of content, both participants finished each task very quickly. Navigation elements of the interface allow multiple ways to find information. Users can choose from the navigation menu on the left, the search box on the left, the breadcrumbs on top, or various links throughout the page. The well-designed navigation also allows the user to freely explore the site while maintaining an escape hatch to find their way back.

19 | P a g e

Figure 7. Navigation options

Two search boxes appear on the pages of this site. The box in the upper right corner is for searching the entire Library of Congress website. The box over the navigation menu on the right is for searching the Teachers site (Figure 8). Although neither participant struggled with this, it may be helpful for designers to make the distinction between the search boxes more clear. If the user searches with the box in the upper right corner, he or she will find relevant content, but they will have left the teachers site, and might have trouble returning.

Figure 8. Multiple search boxes

20 | P a g e

Interaction design The interaction design of the LOC was rated very highly by both participants. The LOC does not have extensive interactive elements. However, this site is in many ways offers more interaction than the other two. The search functions of this site work very well and provide a number of options to meet the users needs. As explained earlier, a teacher can search for lessons or collections by theme, by era, or by alphabetical order. Teachers may also search for content by state standard and grade level. The Library of Congress also offers the ability to manipulate some content. High resolution images can be viewed with tools that allow users to zoom in and move the objects. Users are also given the option of different sizes and resolution of images to download. Finally, although not brought up in the evaluation, the Library of Congress Teachers page is the only website of the three evaluated to incorporate social media elements. There is an available RSS feed for the site and a blog. Visitors can sign up to make comments on parts of the site. Most of the pages also have a Share icon that allows users to post links to the social networking tool of their choice. (Figure 9) This option also allows users to save links to return to later. This is especially helpful for a contentrich website such as the LOC.

Figure 9. Use of social media.

21 | P a g e

Presidential Recordings Program

Design Element Score (KW) Score (MS) Total Possible

Information Interface Interaction Satisfaction TOTAL

14 19 10 6 52

15 18 13 12 60

25 25 20 20 90

Overall Satisfaction The Presidential Recording Projects Teachers site was the lowest rated of the three evaluated. Responses from both surveys showed that the PRP was perceived to be the least desirable in each of the four elements evaluated. Satisfaction with the site was low, especially from Kristin. Neither participant said it was likely that they would recommend this site. Matthew said he might use some content from the site because he found value in the actual recordings. However, he said that he would create his own lessons instead of using those available on the PRP site. Information design Of the five criteria for information design evaluated by the participants, predictability and uniformity of pages was rated the lowest. Although the pages within the teacher site are generally uniform, many links lead to other parts of the site (such as Virtual Exhibits) or external links. When linked away from the teacher site, the user encounters a completely different page design, with different color schemes and different information layouts. This can confuse the user, who may not notice that they have navigated to a different site. This problem was exemplified when Kristin was initially exploring the site. Within a few clicks from the homepage, she found herself in a Virtual Exhibit (a nonworking one, at that) (Figure 10). Although this exhibit was part of the main site (whitehousetapes.net), it was outside of the teachers section and looked much different. If this site were to be redesigned, special attention should be given to making the site more consistent.

22 | P a g e

Figure 10. Non-working and inconsistently designed page

Availability and organization of information were also given a poor rating by both participants. Neither was able to find general information for teaching with primary sources. This is because the website has no such information readily available. The only way to access this information from the website is if a user happens to stumble upon a link embedded in several lesson plan pages. This site also lacks in availability of content. The PRP itself contains recordings from Presidents Roosevelt to Nixon, spanning over 30 years. The lesson plans, however, are from a much more limited timeframe. While Matthew was browsing he remarked: I thought there would be more stuff. It looks like they only have lessons about LBJ. When the lessons plans were explored for task five, both participants encountered trouble finding the materials needed to complete the lessons. This was due to poor organization of information. There are links to materials on the lesson plan pages. Instead of linking directly to the file or page, these links often lead to the PRPs main page, requiring the user to search through the site to find the required materials. Furthermore, many links simply dont work. Both participants encountered broken links when browsing for files. Matthew also encountered at least one link that was inaccessible without a password (Figure 11). The content of this site needs to be reorganized to minimize external links and store all needed materials in a single area that is more easily accessible.

23 | P a g e

Figure 11 . Red paper clip icon signifying a password-protected file

Interface design Participants were slightly more generous when evaluating the interface design of the website. Three of the five elements evaluated involved visual aspects of the pages. Both participants rated these elements fairly high. The PRP site is predominantly text. Although this is not visually stimulating, the text on this site is clear and easily read. Most pages are not cluttered with text, and there is good balance between text and images. The interface navigation was not rated as highly. While on the teachers site, users can navigate through several means. First, the navigation menu on the left is available on every page. The user may also navigate by using the breadcrumbs on all pages. Finally, the user may use the search bar that is always in the top right hand corner (Figure 12). Consistency such as this helps the users know where they are and how to find things (Tidwell, 2010, p. 142). Where this site falls short is the high likelihood of the user wandering off the teacher website. When a user browses out of the teachers area, pages are no longer consistent and the three navigation options are no longer available. The user must rely on the browsers back button when faced with this situation.

24 | P a g e

Figure 12. Predictable navigation options

Interaction design Interaction design was also rated poorly by participants. Search functions worked; however, the links found were often incorrect and would link participants to places they did not expect. There are little opportunities for interaction with the Presidential Recording Program website. Unlike the Library of Congress, objects from the PRP cannot be manipulated. There are also no options for customizability, communication, or user input. Presidential Timeline
Design Element Information Interface Interaction Satisfaction TOTAL Score (KW) 21 23 14 14 72 Score (MS) 23 24 17 20 84 Total Possible 25 25 20 20 90

Overall satisfaction The Presidential Timeline received the highest rating for overall satisfaction out of the three sites. Both were highly satisfied with the site and reported enjoying using the site. As with the other two sites, Kristin said she was unlikely to use this in her classroom. However, she rated the PTL as the site she would most likely recommend to a colleague. Matthew (who gave a perfect rating for overall satisfaction) said he would use this site in his classroom and recommend it. Information design Information design also received a very high rating. Both participants reported the ability to find information quickly while completing their tasks. Most of the information comes from internal sources except for Teacher Resources (such as information about using primary sources) are linked externally. However, these mainly link to the National Archives website (http://www.archives.gov) and are very reliable. The only search function available is on the Interactive Timeline, which was not used for most of this evaluation. That participants were able to quickly find information without the use of a search function speaks a great deal to the organization of information on the PTL. In fact, neither participant seemed to notice the lack of search box, and instead relied on browsing to find all information.

25 | P a g e

The PTL is the only site that received a high rating from both participants for the amount of content available. This shows that the Presidential Timeline has plentiful content, but not so much that it is overwhelming. The content of the PTL is from the presidential libraries, so content is limited to the 20th Century. This focus makes the content more manageable to users like Kristin, who though the content from the Library of Congress could be overwhelming. Interface design Out of the three sites, the Presidential Timeline was given the highest rating for interface design by both participants. The contrast and text make the site easy to read, and there is very little clutter, as text is used sparingly. Out of the three sites, the PTL has the least amount of content on any given page. Colors schemes and fonts are also consistent throughout the site.

Figure 7. Minimal text

The Flash-based Interactive Timeline is much different than the interfaces of the other sites. This interface was appreciated by the participants, who both mentioned its visual appeal. Although this interface may be no easier navigated than a HTML webpage, effects such as roll-overs and animations make it more inviting for users to explore.

26 | P a g e

Figure 7. Flash-based interface

Interaction design Unlike the other elements rated, the interaction design of the PTL received only midlevel ratings. This may be somewhat surprising, considering the site has a major feature called the Interactive Timeline. However, this evaluation focused mostly on the Educator section of the site. This section offers little in terms of user interaction, such as customizability, input, or object manipulation. The pages in this section are essentially set up in lists. As mentioned earlier, there is no search function for the content available on these pages, so a user only has the option of scrolling through the list of available content. A search box would be a helpful addition to the site, especially if more content is added

Conclusion
For any website to be successful, it needs to meet the needs of its users. These websites are designed for a specific user-type who has a specific set of needs. This evaluation investigated the extent to which the websites met these needs. By having teachers evaluate teacher resource sites, some characteristics of good design were determined.
27 | P a g e

Information design Provide ample content without overwhelming Categorize content so it is easy to find Maintain consistent organization patterns Provide multiple ways of sorting content Minimize reliance on external links Interface design Use clear text and avoid clutter Balance images and text Provide simple navigation Keep page layouts consistent Interaction design Design website to allow multiple ways of use Put high priority on search functions Allow for easy downloading

28 | P a g e

Summary
This paper outlines the design and results of a usability test of three websites that provide teachers with content needed for teaching with primary sources. The websites evaluated were The Library of Congress Teachers website, the Miller Centers Presidential Recording Project Digital Classroom Initiative, and the Presidential Timeline of the 20th Century. The usability evaluation consisted of performance evaluations and post-evaluation surveys using two participants representative of probable end-users of the websites. Results are discussed in the perspective of information, interaction and interface design. Some recommendations based on the usability test results are also made for each website. In conclusion, critical considerations for designing primary source educational websites are proposed.

Related Links
http://www.presidentialtimeline.org/ http://www.loc.gov/teachers/ http://www.whitehousetapes.net/content/classroom

References
Nielsen, J., Snyder, C., Molich, R., and Farrell, S. (2000). E-Commerce User Experience: Methodology. Nielsen Norman Group. [Online] Available http://www.nngroup.com/reports/ecommerce Rubin, J. (1994). Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. Indianapolis, IN: John and Wiley & Sons. Tidwell, J. (2011). Designing Interfaces. Sebastopol, CA: O Reilly Media.

Author information
This paper was written by Rob Scordino, a doctoral student in the Instructional Technology Program at The University of Texas, Austin. This paper was written as part of the final project for the course Design and Strategies for New Media, Fall, 2011 taught by Professor M. Liu

29 | P a g e

APPENDEX A- Consent Form

Participant name: Understanding Your Participation You have been asked to participate in a usability study about how websites meet your needs. Our purpose in conducting this study is to understand what makes some websites easier to use than others. In the session, well ask you to visit some websites and look for various things. Please keep in mind that this is a test of websites; we are not testing you. We may videotape all or some of the test for evaluation purposes but we will not release the recording to anybody. All information we collect concerning your participation in the session is confidential. Participation is voluntary. There are no physical or psychological risks associated with participating in this study. During the session, the study administrator will assist you and answer any questions. You may take short breaks as needed and may withdraw from this evaluation at any time. If you have any questions, you may ask now or at any time during the test. Statement of Informed Consent I have read the description of the study and I am aware of my rights as a participant. The conductor of the research study has assured me that my identity will remain anonymous and confidential. I agree to participate in the study.

Signature _______________________________ Date ___________________________________

30 | P a g e

APPENDEX B - Pre-evaluation Questionnaire

1. Have you ever visited sites similar to or including the Presidential Timeline, Library of Congress, or Miller Center Websites? [ Yes ] [ No ] [ Unsure ] 2. If so, which sites?

3. If you have visited any of these, what was your purpose for visiting? [ ] Primary sources (pictures, documents, videos, etc.) for lesson planning [ ] Educational resources (teacher aids, lesson plans, study guides) [ ] Personal research interests [ ] Online Exhibits [ ] Other Information (please specify) : __________________________________

4. How would you rate yourself as an Internet user? [Beginner] 1 2 3 4 5 [Expert] 5. What do you use the Internet for?

6. How often do you use the Internet for teaching or planning? [Never] 1 2 3 4 5 [Daily] 7. In a sentence or two, describe your use of the Internet for teaching. In which which age group are you? [ under 26] [ 26-30] [31-35] [36-40] [41-45] [46-50] [51-55] [55+] 8. Please check all that apply to you: [ ] K-4 Educator [ ] 5-8 Educator [ ] 9-12 Educator [ ] Post-secondary Educator [ ] Teacher Educator [ ] Other ____________________________________ 9. If you are a teacher, how many years have you been teaching? What subject do you currently teach?

31 | P a g e

APPENDIX C - Exploratory Task Protocol

Moderator: This is an exploratory exercise. Take a few minutes to browse the site. If you feel comfortable doing so, please verbalize your thought process as you browse and tell me your impressions of the site. Please be honest, your objective opinion will only support the purpose of this study. Prompts 1. What do you think is the purpose of this site?

2. Who do you think is its target audience?

3. If this is the first visit of this web site, what would you say about this site?

4. At a first glance, do you think this web site could provide you with any information of your interest?

32 | P a g e

APPENDIX D - Specific and open-ended task protocol

(The scenario will be the same for each site. The tasks will be similar, but the specific items to be looked for will be different based on the content of each site) Scenario: You are a secondary history teacher creating a lesson. For your lesson, you want to use primary sources from the site you are about to evaluate.

From the main page, find information about using primary sources in the classroom.

You need an image of a president for a presentation. Find any image of any president well as information about that image and its context.

You are preparing a lesson on (topic). Find (specific file related to that topic). Also, find another artifact (document, picture, phone conversation) related (topic).

You need to plan a lesson. Choose a topic related to the information available from (website). Find some documents, videos, transcripts or other files and some contextual information about that topic that you could use to make a lesson.

33 | P a g e

APPENDIX E Post-evaluation questionnaire

(given after each site evaluation) To what extent do you agree with the following? 1.It was easy to find the information I needed. 2.There was adequate material available on the site. (Please note if you thought there was too little or too much) 3.The material was neatly organized. 4.All labels and captions were accurate. 5.Pages were organized uniformly and predictably 6.While browsing, I could easily return to the home page. 7.The icons and navigational tools were clear and worked well. 8.The text on the website was easy to read. 9.The pages on the site were tidy (not cluttered). 10.Text and graphics were well-balanced. 11.Search functions and other forms worked well. 12.This site offers customizability. 13.A user could use this site in multiple ways. 14.I could manipulate (zoom, change, download) files from the site. 15.I liked browsing this site. 16.I am satisfied with this site. 17.I would use this site in my teaching. [ ] If you do not teach social studies, check here. 18.I would recommend this site to a colleague or friend interested in history. 1 1 2 2 3 3 4 4 5 5

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2

3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3

4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4

5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5

19.If you are not satisfied, please describe the reasons for your dissatisfaction below.

20. What changes would you like to be seen to make the website easier to use?

34 | P a g e