Website Usability Analysis Report

BGSU ADVANCED TECHNOLOGICAL

EDUCATION

Sherri Orwick Ogden

July 13, 2007
INTRODUCTION
Bowling Green State University offers an online degree completion

program that is designed for working adults who wish to complete a Bachelor

of Science degree in Technology. The Advanced Technological Education

(ATE) program consists of nine core courses that teach students how to

integrate technological concepts into their working environments. Students

can choose between two courses of study: Technology in Training and

Development or Information Studies. All of the courses are online offering

convenience and flexibility for working students. An applied associate degree

is required for the program, as well as work experience.

Prospects wishing to pursue a degree online will tend to look for

information online. It is critical that the ATE website be enticing,

informational and usable. Therefore, a usability test was conducted on the

ATE website to determine if prospects and existing students are able to

easily retrieve key information. In addition, the process in which prospects

contact ATE personnel to receive additional information was also analyzed,

as it is an integral component of the website and recruiting process.

In order to encompass all components of the website and the

information process, this usability analysis consists of the following sections:

• Pre-analysis
• Methodology
• Results
• Conclusions and Recommendations
PRE-ANALYSIS
In order to develop productive test questions, it was important to

determine relevant facts about the ATE program. For instance, what do

prospects and students want and need to know? Who is the target

audience? What types of tasks do they want to perform from the website?

What are the objectives of the website and key points it should contain?

What is the purpose of the website?

Currently, the purpose of the site is to inform prospective students

about the ATE program. A list of questions usually asked by prospects was

compiled. Answers to the following frequently asked questions should be

found easily and quickly:

• How long will it take me to finish?

• What is required to be eligible for the program?
• Do I need an associate degree?
• How much does it cost?
• What can I do with an ATE degree?
• What does ATE stand for?
• What will I learn in the ATE program?
• What courses will I be required to take?
• Can I take courses elsewhere and transfer them to the ATE
program?
• Will my degree say “online?”
• When do classes begin?
• What if my employer pays my tuition?

The four questions below were found to be questions asked not of

prospects, but of those that have already applied. Therefore, it was

discovered that the purpose of the website needs to expand to providing

information to those that have just applied as well.

• When will I know I’ve been accepted into the ATE program?
• I’ve applied – what happens next?
• I don’t think a course transferred correctly. What do I do?
• What is a transfer evaluation vs. an official audit?
Who is the audience?
The website can be accessed by both prospects and current students.

However, for the most part, the website is currently designed for future

students. Is it not normally used as a tool for students already in the

program. However, for the purposes of the usability test, the future and

current student audiences are defined.

Future Students
Users will be most likely between the ages of 25 and 60.
Computer skills will vary from beginner to expert. Most will have
some college experience. Connection speed will vary from dial
up to high speed.
Current Students
Users will most likely be between the ages of 25 and 60.
Computer skills will vary from beginner to expert. All will have
some college experience. Connection speed will be high speed.
What tasks need to be performed from the website?
Students and prospects viewing the ATE website will need to have

access to the following links:

• Financial aid
• Course Descriptions
• All online courses available
• Course Schedules
• Online application
• Tuition deferral program
• Ability to request more information
• Access to a sample online classroom
• Video/Audio of ATE explanation

Information and features that should be present:

• Explanation of the enrollment process including time
involved
• Application instructions
• Curriculum
• Course Descriptions
• Class Schedule
• Faculty Information

What are the measurable usability objectives?

 In order to create test questions that would provide meaningful

information, it was important to define the following measurable objectives:

• Were all of the users’ questions answered?

• Was the user able to request more information easily?
• Was the user able to quickly identify the correct links to
answer their immediate questions?
• Was the user able to easily print the information they
wanted to save?
• Did the user have an emotional reaction to the site? Was
the user excited about starting the ATE program or was
he/she discouraged, confused or indifferent?

What are the key points?

The following key points are important components and should be

included in the website:

• Course Descriptions
• Costs
• Completion Time
• Current Promotions
• Accreditation

METHODOLGY
Participants
Five testers were selected for this usability test. Three out of the five

were female and all five were between the ages of 40 and 45 which is also

the approximate average age of most ATE prospects. All use the internet and

email to some degree for work and/or pleasure. Hours of usage vary from 10

hours to 40 plus hours per week.

Test Questions
The following information, directions and questions were provided to

each participant.

Usability Test Instructions and Questions

I would like you to view a website for me and answer a few questions. The
process will probably take approximately 15 minutes.
 • Please know that I am not testing you, I am testing the site. You
can’t make any mistakes during the test.

• I want you to be completely honest – you won’t hurt my feelings.

The goal is to make the site better and easier to use.

• As you think about each page and decide where to click, please do
me a favor and include any thoughts you have in your answers so I
can understand your thought processes during the test.

Before you look at the website, I have a few questions for you, OK?
1. What is your occupation?

2. How many hours a week do you spend using the internet, including
email?

3. What types of tasks do you use the internet for? What kinds of sites
do you like to visit?

4. What is your age? 18 to 25, 26 to 39, 40 to 59, 60 to 74

Now it’s time to open the website. Go to

http://ideal.bgsu.edu/programs/ATE/index.php
5. Have you ever seen this website before?

a. If so, why and when?

6. Just from looking at this site, who do you think the site is designed for?

b. Why do you think that?

7. By glancing at the website but without clicking on anything, can you

tell me 4 types of information the site is intended to provide?

8. Click on a link you think you would click on first if you were accessing
this website for the first time at home. Why did you choose that link?

9. By just glancing at this page, what information is provided?

Scenario 1
You want to know the eligibility requirements for the ATE program. You’ve
heard an associate’s degree is required, but you don’t have one. You want
to know if you can still apply. Can you find the answer to your question?
Not completed Completed with difficulty or help Easily
completed
Scenario 2
You would like to know how much it will cost to complete the program. Can
you find the answer to your question?
Not completed Completed with difficulty or help Easily
completed
Scenario 3
You have an associate degree but want to know how many courses will
transfer to the program. Can you find the answer to your question?
Not completed Completed with difficulty or help Easily
completed
Scenario 4
You want to know what courses you are required to take as a part of the ATE
program. Can you find the answer to your question?
Not completed Completed with difficulty or help Easily
completed
Scenario 5
You’ve heard some employers don’t hold online degrees in as high esteem
as other degrees. However, BGSU is a reputable and highly regarded by
employers. You want to know if your degree will indicate that you attended
online or if it is the same type of degree you would receive if you attended
on campus. You also would like to know ATE’s accreditation. Can you find
the answers to your questions?
Not completed Completed with difficulty or help Easily
completed
Scenario 6
You want more information about the program and would like more
information. Can you request more information?
Not completed Completed with difficulty or help Easily
completed
Complete the request for more information form on the website
10. Did you feel uncomfortable about providing any of the
information asked of you on the request for information form?

Open the email sent to you in response to your request for more
information
11. Do you feel you’ll read the entire email?

12. What information are you inclined to look at first?

13. Is the information too much information, not enough information,

or just enough?

14. Is the information organized and easy to read?

15. Are you confused by anything indicated in this information?

16. Would you open each and every document?

Open the documents you would open in a real situation

17. Can you now answer any of the scenario questions you couldn’t
answer before?
 18. What is your overall impression to the ATE website and request
for information process?

19. What did you like best?

20. What did you like least?

21. What is the first thing you would do to improve the site?

22. Do you have any other comments and/or questions?

Rate the website and the request for information process:

Strongly
Strongly
disagree
agree
1. Overall I found the information I
needed. 1 2 3 4 5

2. I found the information

1 2 3 4 5
unnecessarily
complex
1 2 3 4 5

3. I thought the information was easy

to use 1 2 3 4 5

4. I found the various links in

the website were well integrated 1 2 3 4 5

5. I thought there was too much

1 2 3 4 5
inconsistency in this website

6. I would imagine that most people

1 2 3 4 5
would learn to navigate this website
quickly
1 2 3 4 5

7. I found navigating the website very

cumbersome
1 2 3 4 5

8. I felt very confident using the

Website and the request for 1 2 3 4 5
information
Test Procedures

The ATE program is online, so there is a presumption that individuals

requesting information about the program have access to the internet and

email. Therefore, all informative documents about the ATE program are

emailed to prospects. Statistics show that a majority of prospects find the

ATE website and then complete the Request for Information Form. In order

to simulate this procedure as accurately as possible, testers were emailed

the test instructions. They were asked to complete the tasks and answer the

questions without a test administrator present. Part of the test was to utilize

the “Request for Information” feature on the ATE website, and then read an

email with 5 attached documents sent to them in response. Participants

were asked questions about both the website and the emailed information.

Testers recorded their answers electronically and submitted their results via

email.
RESULTS

Test Answers
 Tester
Information Tester 1 Tester 2 Tester 3 Tester 4 Tester 5
Secretary/Medical Recruiting Coordinator &
1 Occupation IT/Contractor Transcriber/Student Secretary Administrative Secretary Database Administrator
How many hours a
week do you spend
using the internet,
2 including email? 10 hours 10-15 hours A lot 20-40 hours a week 40+ hours

Email at work, internet for work

procedures. I rarely surf for
What types of pleasure but frequently visit my
tasks do you use Banking, Google calendar, bank site and sometimes will view
the internet for? Investments, weather, news, self.com or diet and exercise sites.
What kinds of sites Shopping, Email, Email, online paycheck, bursar bill, I have a business website that I Google, databases, email,
3 do you like to visit? News classes, shopping book orders check and sometimes weather.com research
4 What is your age? 40 to 59 40 to 59 40-59 40-59 40-59
Website Information - http://ideal.bgsu.edu/programs/ATE/index.php
Have you ever I thought I had been to this website
seen this website but it looks different than I
5 before? no No No remember. no
At the very first glance, I would
think the site is for any BGSU
student. It references the degree
name and that it is a bachelor of
science. Of course, knowing about
20-somethings who To get info about a the program, I know it is for adult
didn't go to a 4-year degree in Tech students and it is an online
Just from looking at school. It never Education. I think that program, but you wouldn't know
this site, who do says anywhere it's because it says that unless you read the first three
you think the site is Designed for for adult learners or "Bachelor of bulleted (sort of) lines under the Students wanting to earn a
designed for? Why adults that want a non-traditional Technological description. I t think that is because Bachelor of Science in
6 do you think that? four year degree students Education." of the template Advanced Tech Education
By glancing at the
website but
without clicking on
anything, can you Contact Four things I should be able to get Phone number, email
tell me 4 types of information if are information about the program, address, qualifications:
information the site they are the specific curriculum to finish the need an associate degree
is intended to interested in degree, info about the faculty and & work experience, can
7 provide? learning more Not really Contact info contact information to sign up. take classes online
It is a degree
from BGSU info about the degree
You can find more
details by the
links on the left
It is an online
degree (although
that isn't terribly
obvious)
Click on a link you
think you would
click on first if you
About the I clicked on About the
Results Analysis

• None of the participants had seen the ATE website before

completing the test.

• At first glance, the participants were asked what the website was
designed for. The testers all indicated the website provided
information about a bachelor degree. When asked to identify four
types of information provided on the website’s home page, one of
the five reported what they should be able to find – not what they
did find. Another tester said they could not identify four types of
information. The remaining three all agreed they could locate
contact information and information about the degree. Two
participants said they identified the program was online, though one
reported that it was not necessarily obvious.

• All participants agreed they would be inclined to click on the “About

the Program” link first.

• One participant did not read the information provided on the “About
the Program” page. All remaining participants indicated that the
“About the Program” page provided more information, who the
program is designed for, and how it works. One participant
appeared to be a bit confused as this was the first time “degree
completion” was mentioned, indicating that it should have been
included on the home page.

• When asked if the testers could find out if they could apply without
an associate degree, one of the four said they could easily complete
that task. The remaining four testers could not find this
information. It is unclear how one tester found this information as it
is not provided on any of the ATE website pages.

• When asked if the participants could locate costs associated with

the program, only one reported finding the information but with
difficulty or help. All others could not locate cost information.

• When asked to locate information about transferring courses to

BGSU, the answers varied. Two completed the task with difficulty or
help, one could not complete the task, and one easily completed
 the task. One tester experienced an error on the transfer credit
page.

• All participants agreed they could easily locate the courses required
to complete the ATE program.

• When asked to find out what type of degree and accreditation the
program held, one participant reported they completed the task
with difficulty or help, three easily completed the task and one
reported they found the information on the “Recent Updates” page
and it should be placed on the “About the Program” page as well.

• All five participants easily requested more information about the

program. One tester suggested being able to ask specific questions
on the Request for Information form and felt the contact information
and the request for information pages should be combined.

• All five testers felt comfortable providing information asked of them

on the Request for Information form.

• One participant felt they would only read the first few paragraphs of
the email sent to them in response to their Request for Information.
The remaining testers said they would read the entire email. One
tester felt a lot of the information in the email duplicated the
website information.

• When asked which information in the email they were inclined to

look at first, one reported they would like to see course
compatibility, though that information is not currently included in
the email. Another tester reported they would click on the link to
the web page if they hadn’t already viewed it. Another said they
would first read about what the program is and then view the
curriculum. The remaining participant said they would read the
email first.

• What asked if the email provided too much information, not enough
information or just enough, two participants felt it was just the right
amount of information. One participant remarked that the
highlighted and bulleted information was helpful. One participant
 reported it was not enough information and more transfer
information needed to be provided. Two remarked that the letter
repeats too much information from the web page. The remaining
tester reported there is never too much information.

• All agreed the information was organized and easy to read. One
participant remarked they did not understand what some of the
information meant, however.

• When asked if any of the information confused them, three

participants reported they were not confused. One reported they
were confused, and the remaining tester indicated the requirements
should be better explained.

• When asked if they would open every document attached to the

email, three participants reported they would and two reported they
would not.

• Only one out of the five participants said they found answers to
earlier test questions they could not answer after reading the email
and opening an attachment.

• The overall impression of the website was positive. The available

links, contact information, the appearance and navigation
capabilities were reported as the best features of the website. The
small font size, the look of the Recent Updates and the Request for
Information form, the number of attachments on the email, and the
ability to understand the information were the least liked features of
the website and request for information process.

• Suggested improvements were:

o Make the heading smaller and provide more information on

the home page.
o Provide more links to transfer information
o Change the color of the Request for Information form and add
a text box for specific questions
o Increase the font size
Conclusions/Recommendations

Conclusions

The ATE website and request for information process scored in the

middle range overall. The average rating for positive features ranged

between 3.25 and 4, with 1 being the lowest score and 5 the highest.

However, the pre-analysis and testing processes revealed information

that is missing and areas of the website that are in need of improvement.

Website Information

The website appears to be informative in a limited capacity. The way

in which the information is presented could be improved. For instance, key

information should be accessible at the top of the page where users are most

likely to focus their attention. This should include that the program is online,

is for adults, is a degree completion program, and is accredited. In addition,

the following information is missing from the website:

• Financial Aid (although this is included in the emailed

information)
• Costs
• Available online courses
• Class Schedule
• Online Application
• Tuition Deferral Program
• Tutorial
• Enrollment process (although this is included in the emailed
information)
• Completion Time
• Accreditation Agency
• Current Promotions
• Options for individuals without an associate degree
• Detailed transfer information
• Video/Audio files verbally describing the program
 • FAQ’s

Overall, there was not the impression from the test results that the

website generated an emotional reaction. If anything, one user’s answers

indicated frustration; though it appears that could be from a previous bad

experience.

The Request for Information page did not seem to intimidate the users

so it’s possible to conclude that the information requested from the prospect

is not perceived as being invasive. For those using smaller computer

screens, the graphical heading at the top is too large. In addition, some felt

the font was too small and was bothersome.

The website does appear to navigate well. Users were inclined to

select the “About the Program” link from the home page which would be the

logical choice. In addition, the courses required for the program were easily

obtained by all participants.

Emailed Information

It appears most testers read the entire email or at least skimmed the

information. It was not apparent which attached emailed document was

opened first. The question pertaining to this subject should have been

worded differently in order to determine which document was of most

interest to the tester. The amount of emailed information did not appear to

overwhelm a majority of the testers, which was an initial concern. Most

found the email easy to read and navigate, though it may be necessary to

provide clearer explanations.

Recommendations

Based on the pre-analysis information, the test results, and the

conclusions the following changes to the website and emailed information

are recommended:

• The home page should provide links to: “Future Students”,

“Current Students”, and “I’ve just applied. What Next?" Each
will direct the user to a page which has information
specifically relevant to their situation.

• Links that need to be added to the website in their

appropriate category(s) are:

 ATE Community
 Class Schedule
 MyBGSU
 Online Classes Available
 FAQ (different questions for future, current, or just
applied)
 Application Instructions
 Tuition Deferral Application
 ATE Checksheet
 Video/Audio describing ATE
 Online tutorial
 Online Application

• The website needs to include a printer-friendly version for all

pertinent information and documents.

• The following information should be at the top of the home

page where users will quickly identify it:

 The program is online

 The program is for adults
 ATE is a Bachelor Degree Completion Program
 How to retrieve more information/contact information
 FAQ

• One of the FAQ should be “What if I don’t have an Associate

Degree?” A page should be dedicated to information specific
to those individuals.
• A text box should be added to the Request for Information
form allowing users to enter specific questions. A link to the
Future Students FAQ would be helpful from this page.

• The email attachments should be converted to websites and

links should be provided to the websites as opposed to
attachments.