Usability Report

2015 Usability Test: AirBnb
Aubrie Brault
Professor Gulbrandsen
ENL 620
Final Project
12/11/15
Brault 2
More the Just a Pretty Face?

A Look Passed the Well-Designed
Interface of AirBnb
Brault 3
Table of Contents
Pre-report material
Table of contents
Executive summarybrief synopsis of study, findings, and

recommendations
Report
Introductionbackground on the project (why, how, etc.) as
well as description of report structure, including information
contained in appendices
Description of test plan and procedures (which include goals of

the study)
Description of users
10
Description of scenarios
11
Top findings (positive and negative) with examples (e.g.,

screen captures)
12
Severity codes assigned to findings
18
Analysis from the post-task and post-test questionnaires

(quantitative and qualitative)
36
Recommendations for short-term and longer term

changes/improvements
43
Call to action for more testing (as part of being a usability

advocate)
46
Appendices
Brault 4
Heuristic evaluation
48
Personas
78
Test plan
Screening questionnaire completed for each participant
82
Post-task and post-test questionnaires completed for each

participant
92
Moderator script
94
Logs from sessions
98
Other material used for the test sessions
100
Brault 5
Brault 6
Executive Summary
Synopsis of study:
My focus within this study analyzes the official AirBnb website through the lenses of
content and community. Questions this the study considers are:
How well does the site engage with the audience?
How well does the company communicate who they are and what they do
(through the website)?
How effectively does AirBnb provide their service through the website?
My general findings to these questions have proved that although AirBnb connotes
effectiveness in these areas through design, they lack in genuinely interacting with their
users. It would appear they allow their unique concept to speak louder than their own
expertise in the business; meaning that the platform is an advanced design but that the
content and general usability practices need to better consider their users. From this, an
improvement in content management is recommended.
Generally speaking this studys findings offer the following suggestions:
1. AirBnb should expand their online presence
2. AirBnb should create new pathways of communication between company and
users, and amongst the users themselves.
3. AirBnb should offer more thorough investigations of parties involved and valid
proof thereof; in other words more guaranteed security.
4. AirBnb should stress focus on their users and conduct an official, broad but indepth, audience analysis.
Brault 7
Introduction
Why AirBnb?
I chose to conduct a usability test on the official AirBnb website for the following
reasons:
a) They are a relatively young company and therefore likely have more room for
improvement.
b) AirBnb appears pretty popular among some circles and non-existent among
others, implying that some further audience analysis would likely produce some
interesting results.
c) I personally enjoy travel and technology thus anticipated this site to be an ideal
opportunity to learn more about both.
How was the project conducted?

I approached this project as generally directed by instructor of ENL 620 (Writing and
Designing Computer User Documents). First, I conducted the initial research that
involved both primary and secondary sources. For the former, I conducted a survey that
was sent out to over 5 individuals as well as put up (what turned out to be) a popular
Facebook poll. The extended research simply came from background information
synthesized from news articles and recently written, third party reviews.
From the primary information, specifically the survey results, I then developed two
personas, Aylara Kovir and William Blakemore. Their traits represented the average of
themes discovered in the survey results. Being that Aylaras persona seemed to
represent a larger group of users, I used her to conduct a HE (heuristic evaluation) on
the official AirBnb Website.
The HE presented me with both indications and main take-aways. Together, these
helped refine my research question and also generate appropriate scenarios for the
actual user testing. With this information, I then reached out to potential user subjects.
Availability allowed me to interview 3 subjects fitting the persona. I logged these three
sessions via video recording.
Brault 8
The Report Structure:

This report is structured in a user-friendly manner.
The major sections are as follows:
a) Pre-Report Materials
b) Report
c) Appendices
As you have already experienced, Pre-Report Materials are read through before this
section.
From this point on you will find descriptions of the constituents, followed by, top
findings. The top findings have been grouped together and presented under their
corresponding heuristics in tables. Each top finding will be presented with their
corresponding severity codes, themes of issues, and suggestions. Each top finding will
also be shown through a corresponding screenshot.
After the top findings of each heuristic have been broken down, you will find the pre
and post-test responses in their own tables. The analyses of these responses can be
found as explanations underneath their corresponding tables.
Following these items, and within the Report section, you will then find a table of
recommendations and a call to action.
All evidences will be found in the concluding section labeled Appendices. This section
will include original and copied versions of all materials referenced.
Brault 9
Description of Test Plan & Procedures

The test plan and procedures were generated to test user
interaction with the site in terms of:
-
How the view the company persona

How they interact with the tools available to deal with basic procedures
How they interact with the tools available to deal with potential problems
The procedures within this test plan include:

a)
b)
c)
d)
e)
f)
g)
h)
Greeting the user and introducing them to the process

Informing them of the process via the moderator script
Explaining consent forms with request of signature
User Pre-Test Questionnaire (5 Questions)
User viewing of Think Aloud Protocol as shown from nngroup.com
Run-through of scenarios
User Post-Test Questionnaire (9 Questions)
User Appreciation (Comments of gratitude and potential incentive provided)
Brault 10
Description of Users
Being able to ask the most likely users for AirBnb stems from about two months worth
of research. During this time, surveys and secondary research allowed me to create two
personas through which two likely demographics and their personality traits were
presented. The persona, Aylara Kovir, presented to be the most likely accurate lens
through which to approach this heuristic evaluation.
The user test subjects are described below:
Kerri-Lee
Melanie
Caty
Age:
21
25
25
Occupation:
Full-Time Student;
Marketing Specialist
for Renewal by
Anderson
Graduate Teaching
Fellow; Office Clerk
for UMass
Dartmouths Office
of Graduate Studies
Full-Time Engineer
at Lockheed Martin
Description: *All users match with general description of Aylaras qualities in some
capacity. Most accordingly, all users are in the age range, enjoy travelling/ word like to
travel more, do well for themselves with generating their own incomes, and are earning,
or already have, a Bachelors Degree.
Brault 11
Description of Scenarios
Traditional usability concerns, as well those that were highlighted in my HE, were both
influences in how I created the scenarios for this test. This combination allowed for a
more natural progression of scenarios to be produced. This set began with basic
processes and concluded with specific, complex ones. Realistic goals that average users
would have in coming to this site, were also kept in mind. Maintaining the natural
progression, the first few scenarios catered to more general needs.
It is scenario three in which my research questions starts becoming evident.
My research question for this project on AirBnb is:
How well does AirBnb foster communication and community?
Scenarios 3-5A highlight different facets of this questions, trying to answer it. The last
two scenarios, 4 & 5, both have follow-up scenarios designed to more naturally take the
user through the potential problems that could occur. For this reason, I intentionally
added thorough detail to these latter scenarios.
Brault 12
Top Findings: Bottom-Up Approach

In this section:
1. Overview of findings and the applied framework in deciphering them
2. User-Specific Highlights per task
3. Top-Findings by Category
Overview
From the heuristic evaluation conducted beforehand, I had been lead to a certain set of
possible usability issues. With these core issues in mind, and the influence of other
tested heuristics, I generated the following heuristics.
Categories:
-
Influenced by the 3 of Whitney Quesenbery 5Es, as well as the ideologies

of accessibility and community, I generated the following main heuristic
through which I would evaluate my findings:
o Efficiency
o Effective- Engagement
o Accessibility
Following my own, collective approach, I created the 3Cs. They are a

subset category of heuristics, another way to evaluate findings. For the
reason that the 3Cs are more themes than terms of evaluation, they have
been identified with the level of severity corresponding.
o The 3 Cs (these will serve more as severity code theme)
Credibility
Comfortability
Community
*Using the Bottom-Up approach I then clustered these individual findings into groups.
Ultimately I was able to put them into the following categories:
Brault 13
User-Specific Highlights Per Task

User:
Kerri-Lee
Mel
Caty
Task 1:
Get a 1
Impression
st
Didnt care to evaluate

aesthetics of site
Welcome page seemed to
do nothing for her
did feel welcomed that
there were cities listed she
knew personally
liked seeing that she had
options both in and out of
country
did not want to spend
much time browsing
around seemed her
impression of co. would
come more by just going
through it briefly on her
own
After quick look, wanted
to Google immediately
Time spent on this task
was immeasurably short.
Evaluated aesthetics more

heavily
Simultaneously concerned
with credibility
searched for proof of
copyright
Looked for an accreditation
from the Better Business
Bureau
took note of other audiences
the site was appealing to - felt
they may have spoken for
some credibility (i.e. seems to
cater to traveling business
professionals)
Time spent: average
Was not primarily concerned

with visuals
Was the only one to click,
How It Works on
welcoming screen
Felt information she was led
to didnt actually tell her
anything; what was clickable
on this How It Works page
was difficult to know/ see
From here there was no
identifiable way, or natural
segue, of getting to about
us info
At this point she was ready to
go to a Google review
Also at this point she only
understood the company as
trying to set her up with host
family
Once she finally did find the
about ussection she
expected it to be interactive
as it appeared so to her
Time spent: Lengthy/ Indepth
Brault 14
Task 2:
Make an
Account
Task 2.1
What captures
your attention
while on Profile
platform/
account page?
Found easily
Preferred signing up
through FB quickest
Ignored other offers/ info
Felt strange that FB had

filled in a lot of her info for
her
Liked cultural appeals to
creating on logo and intro
video
Didnt seem to care about
inviting fb friends
Didnt seem to care about
incentives
Found easily
Preferred signing up with
email/ also annoyed that she
had to confirm email
Ignored other offers/ info
Did not liked incentives, esp.

that they repeated
Attracted to info on starting
travel process primarily How
to Book
Found easily
Preferred Email sign up
Likes its not asking her for
her credit card immediately
Appreciated coupons and
inspiration be sent to her
email also
Evaluated more aesthetics

and how it brought her to
specific content
Liked simple facebook-like
layout
Felt unnecessarily driven to
incentive by color difference
and because of that focused
on nothing else
Brault 15
Task 3:
Establishing
Credibility:
Felt home page did not

clearly describe well
enough who they are as a
company and that thats
where such info should be
Would have liked to see
more of employees not just
info on CEOs; was curious
on what positions did
(maybe like an operations
layout)
Was also interested in
knowing their end of
things; i.e. how do they get
people to sign up
Felt that while she was
looking for this info
pictures were found in
places she wouldnt expect
to see them/ such images
appeared irrelevant to
content on page and only
distracted her from her
purpose
Also felt she had to dig a
lot; would have preferred
more clear information up
front; i.e. she literally
frowned and was put off
that after two clicks, she
had to click a third to still
yet learn more
Wanted to find about

section from her profile page
but could not, nor could she
remember where she first
saw it
Returning to home page
expected to have an about
tab in upper-right-hand
corner
Once desired info was found
felt that it did not satisfy her;
there was not enough
substantial evidence to sway
her
Highlighted numbers were a
decent logical appeal but was
also concerned with current
news on those numbers
Liked layout of CEO section
Took time to read about them
but couldnt tell what their
experiences really said about
them; thought they seemed
design-oriented
Wants to know if company
does background checks
Thinks this should be done for
both travelers and hosts
Expressed she wouldnt

bother browsing website for
such info
Instead she would third party
research the info via Google
But if it had to be on site
she felt the founders page
was a decent leg to stand on
for credibility but did not
think the about us section
offered anything worthwhile
While also on this about us
section, Caty expected to
interact with the site and
icons illustrated before her;
they did not serve this
purpose and she left this
scenario unsatisfied by
amount of info on site
Brault 16
Task 4:
Choose and
Compare 2
Listing/ Evaluate
Satisfactory
Listing
Information
Easily found where do you

want to go? search bar
Liked adjustable filters
Used advanced filtering
system
Categories she cared most
about/ were points of
comparison were: safety
appliances, parking, wifi,
and ability to see her
location/ what was around
it
Photo seemed an after
thought/ little time spent
on them
Prob #1: Her very first
choice of listing was
chosen; but once she
clicked on it, it disappeared
and booted her back to the
listing page
Prob #2: one of the listings
she liked was made difficult
because it was in another
language; the Google
translate wiki attempted to
translate for her qualifying
translation may not be
accurate- sure enough it
only got major subject
words; kerri was unable to
make sense of what was
being said
Also wished the range or
vicinity map was more
clear; this would have
enabled her to make a
better decision
#keyfactorforher
Could not initially find where

to search for listings from
Profile page
Used basic filter system
Though had difficulty with
sliding budget bar would
have preferred a better/
more accurate mechanism
Like star ratings, profile pics
Spent a lot of time with
pictures
In some areas was confused
due to poor/ vague English
language use
Found amenities to be most
important for comparison
Those that she cared for most
were: safety appliances,
ratings, reviews
Spent some time thoroughly
reading through several
reviews; though they were
good she was skeptical of
them
Was really confused on

where to begin looking for
trips; went through several
other dead end windows
before trying logo to link her
back to home page
Used basic filter system
Expected there to be a better
comparison system for when
someone has narrowed down
choices
This expectation led to her
using the save to my wish
list feature
However she was
disappointed; felt that if this
feature were to be of any use
AirBnb should do better with
putting up all the hard data/
pertinent categories and info
up alongside house icons that
show up on the wish lists so
that one can literally see the
compared differences
The wish list feature also did
a poor job at guiding her
through the process (did not
notify her if a choice had
actually been saved to her
wishlist)
On listing cared most about
safety appliances, number of
reviews, and then content of
those reviews
Brault 17
Task 4.1
Two ways of
getting direct
info about the
site you chose
Task 4.2
Desiring more
info on site that
is not reviews or
host
Task 5:
Cancelling a
Stay
Would contact host directly

Liked suggested talking
points AirBnb provided for

this
Would also have liked to
reach out to the people
who recently travelled
there
When she realized she
could not; she realized
there was no other way of
knowing

Would also read through
listing description again
Attempted to visit blog
area at bottom of page

when she realized nothing
else could be found on the
listing
Blog confused her
All images; little to no text;
no context situating user in
what subjects they are
really in
Proved irrelevant
After such a useless
experience she did not
want to look anywhere else
Felt only option was to look
through the social media sites

listed at bottom but
anticipates they wouldnt be
filtered enough to be useful
Initially thought to contact

other users, but retracted
when thinking that would be
creepy
With thinking about how
many other options there are
for her stay she would be
okay with just going to
another listing that has more/
better info
Assumes she can directly
contact Host about this and

thats what she would do first
This problem also made her
think of payment method

however and how that might
influence who gets the say; at
this point she would like to
know this information
Considering these first two
points she admitted she
would then look to find a
cancellation policy but only
after contacting the Host.
Took the route of hitting the

help tab of the page she was
one
Appreciated that first
suggestion was cancellation
policy
Also learned about hosts
creating their own policy and
would take responsibility for
abiding by that
Found cancellation policy

fairly easily by scrolling
down to tabs at bottom of
page
Agrees with terms and
conditions
Takes responsibility for
needing to read Hosts
Contract
In this situation she would
simply try and move her
reservation date through
AirBnb (the option they
provide)

Would go through more
reviews
Brault 18
Task 5.1
When
cancellation &
money back are
needed but the
Host is not
responding
Desired to speak with

company as third party
Went searching for a way
to do this
After five minutes she
became very frustrated
that there was no contact
info that she could find
She commented that if it
did exit there were to many
windows with useless
information to go through
Would immediately look for

company policy which she
found at the bottom of page
Went to guest refund policy
Attempted to match her
situation with options
provided but none matched
Frustrated she returned to
policies and found
cancellation policies in which
AirBnb could override a
contract
The only suggestion they
provide on this page is
changing your reservation
thats what she would do
then; not get money back but
change reservation
she would look for corporate

contact and help center
found only contact tab to
have suggested categories
but is finding she has to look
through scenarios, finally gets
to one that matches her
(somewhat) but it only says
how hosts may differ; it
doesnt offer solution to her
problem
found an option to send
corporate an email but
doesnt feel safe with how
long this would take
again looks for call center,
looks for policies for this,
found corporate override but
only option she gets is to
change reservation but that is
still not what she wants
feels the website is more run
by designers and web people
rather than human resource
people who would be
available to actually help
she concludes she is not sure
what she would do if he
cannot find a way to contact
them directly
Brault 19
Top-Findings
Reminder: the categories that the top-findings were broken into are Efficiency EffectiveEngagement and Accessibility.
* Note that for each section is first and overview table. Below each table, in each
corresponding section there are labeled screen shots with finding reiterated, then
followed by the recommendation.
The Severity Code Ranking System:

The severity code identifies the urgency of each identified funding under their
corresponding categories. They are color coded as you see here. This ranking system
consists of the following levels:
a) Positive Results in a beneficial effect on the users ability to perform their
given task; or they enjoyed the content in that section.
b) Cosmetic Issue: Affects the appearance and should be fixed only if time permits.
c) Minor Issue: Hinders the users ability to navigate and should be fixed when
possible; or negatively effects users opinion of that content area.
d) Major Issue: Frustrates or confuses users and requires repairs as soon as
possible.
e) Catastrophic Issue: Prohibits users from performing their given task and requires
and immediate modification.
To add clarification, in which areas these codes are red-flagging, there are also themes
identified in the table. These themes will fall under one, or a combination of, the 3Cs I
have listed.
Credibility
Community
Comfortability
Brault 20
Category A: Efficiency
Finding
Severity
Code
Corresponding
Theme of Issue
Recommendations
A1) While going through the

site, 2 out of 3 users had
trouble finding the main
feature the search bar for
browsing listings and
locations
Minor
Issue
Comfortability
Place this search bar

at the top of every
page seeing as this
is their main service.
And for those it is
already on, should
emphasis in some
way (probably with
color).
Below is figure A1:
Brault 21
A2) All users wanted a better Minor
way to see where their
Issue
location choice fell on the
map that was provided. This
was desired to understand
surroundings better.
Below is figure A2
Comfortability
Work with a provider

who can build an
interactive map, one
that not only moves
but can highlight
area from the city as
whole all the way
down to individual
buildings.
Brault 22
A3) 2 out of 3 users explicitly
stated they desired a better
way to compare chosen
possibilities (locations)
Below is figure A3
Cosmetic
Issue
Comfortability
Have a different
framework for
narrowing down
options. For
example, when in
the wishlist folder
that is created,
generate a viewing
option that allows
users to compare
their narrowed
choices by whichever
lens they choose. At
a minimum should
have ALL pertinent
pieces of info.
Brault 23
4) All users felt that, when
Major
they were looking for specific Issue
information, they were put
through an unnecessary
amount of windows/
loopholes
Credibility/
Comfortability
Go through site with

a content writer and
reconceptualize a
more efficient
approach to
organizing
information. Try to
group information
together as much
and as best as
possible. Then
narrow those down
further. Lastly, they
should conduct their
own user testing in
seeing how their
peared down version
compares to the
former.
**Done through a series of movements that are unavailable to be exhibited via

screenshots.
Brault 24
Category B: Effective Engagement

Finding
Severity
Code
Corresponding
Theme of Issue
Recommendations
B1) All users did like the

staple categories every
location listings had filled
out for details (i.e. House
Rules and Safety Features)
Positive
Comfortability
Brault 25
Below is figure B1
B2) All users expected to be

able to interact with
illustrated map on the
About Us page.
Cosmetic
Issue
Community
Collect more
specified data and
use for areas on map
so that users can
interact with it to find
out more. Could
perhaps even have
fun facts or pop up
images in places they
hover over.
Brault 26
Below is figure B2
Brault 27
B3) All users expected
illustrations on how it
works page to be links
(portals) to another page or
at least move when clicking
them
Below is figure B3
Cosmetic
Issue
Community
Either consolidate
this information to
another, appropriate
page, or allow for
users to interact with
illustrations by
making them live.
Being that they are
icons of people,
perhaps each cartoon
person could speak
on behalf of the
community that icon
is meant to represent
and offer new
information to the
user.
Brault 28
B4) 2 out of 3 users would
have liked to see a mission
statement anywhere most
specified on the first/ home
page
Below is figure B4
Minor
Issue
Credibility
Provide at least a
succinct mission
statement and
welcome with
purpose of the site
somewhere on the
first page users come
to. (Although a basic
one liner is seen
below Welcome
Home not one user
recognized it; so at
least create emphasis
around this text.
Brault 29
B5) All users, at one point,
desired a feature that
allowed them to talk with
other travellers, specifically
with those who had been to
a site they were looking at
already.
Minor
Issue
Community
Have an instant
message chat with
any other user. Allow
users to turn these
notifications on and
off; but do allow
email messages to
be sent when
messenger is turned
off communication
framework like
Facebook.
**Note that, in the
screenshot below,
messenger is an
available application
but one only allowing
users to
communicate with
potential host (not
other users).
Below is figure B5
Brault 30
B6) All users commented on Cosmetic
the incentivizing banner that Issue
pops up on home page for
you, a new user, to invite
friends to join and take a
trip. Majority felt this was
misplaced and, or
distracting; in other words
they did not like it there.
Below is figure B6
Credibility/
Comfortability
Take banner off and

instead highlight
areas on that profile
page that users need
to pay most attention
to. Would also
strongly recommend
some sort of tutorial
to take new users
through the platform.
Brault 31
7) 2 out of 3 users were
Major
primarily disconcerted with Issue
the fact they could not reach
AirBnb directly. In aftercommentary the third user
reflected, she too, ultimately
felt this was not good.
Below is figure B7
Credibility/
Community
Should have a
personal presence on
site. Should clearly
communicate how a
real person can be
reached (corporate).
They would also do
well to have a live
chat with an official
representative.
Brault 32
Category C: Accessibility
Finding
Severity
Code
Corresponding
Theme of Issue
C1) All users wanted outside

reviews/ information (anything)
that was a third, or unbiased
party, on the site.
Major Issue Credibility
Recommendations
Collect a variety of
both positive and
negative reviews
post to home page (a
place that is clearly
and immediately
visible) switch them
out every month and
make sure they have
not been altered by
company in any way.
**Content was not to be found ergo screenshot unavailable.
C2) Some listings users desired Minor Issue Credibility/

to rent were written poorly,
Community
making listing confusing. Others
offered little to no information.
Below is Figure C2
AirBnb could create

more jobs and have
editing positions that
go in and work with
hosts to improve their
basic writing skills.
Brault 33
C3) Some listings users desired Major Issue Credibility
to rent were not in English at all
and Google Translate widget did
not work. (*Note this may have
occurred if such an app does not
recognize the variety of dialects
within a language.)
Below is Figure C3
Employ a better app

that will successfully
translate a foreign
language in full.
Double check on this
system frequently.
Could also offer a way
to look up words with
a side click as
sometimes other
cultures use words
smaller, American
communities may not
be familiar with.
Brault 34
C4) All users appreciated
information on CEOs but
wanted more than that to learn
about the company
Cosmetic
Issue
Credibility/
Community
Feature certain
employees every
month as welcome
representatives on
home page not only
provides welcoming
face but gives
background on
someone who works
for this company.
(Employee
appreciation implies a
positive work
environment).
Could also have a list
of departments and
what they do with
accompanying
pictures; would be
cool to have visual
directory. If you
wanted to get really
interactive you could
have a visual directory
in which workers use
video clips to
introduce themselves
and explain why they
decided to work at
AirBnb. Additionally
they could offer a
suggestion of a place
they would travel to
and why.
Brault 35
Below is figure C4
Brault 36
C5) All users felt the About Us Minor Issue Credibility
section had no substantial
information; it fell flat in serving
its real purpose; they left
wanting more
Below is Figure C5
AirBnb could do a
survey on what their
average customer
would like to see in
this section; better yet
I would suggest they
research and analyze
what some of the most
successful companies
put in these sections
(study the genre
constraints); they
should also more
deeply consider and
develop what kind of
persona they would
like to present here
Brault 37
C6) All users did find the policies Positive
on host contracts for staying
and thought terms agreeable
Below is Figure C5
Credibility/
Comfortability
Brault 38
Pre-Test Analysis
* For this section, the questions will be restated before the analysis of each response
provided.
1) What are your expectations in using/ coming to a new website?
From this response it was explicitly mentioned by each user that, when
approaching a new website they expected it to be easy to navigate as well as
clear, direct. These were the two kinds of responses that thematically ran
through each response. Integrated, these two desirables make for a category of
efficient navigation. More specific responses, however, were also given on an
individual basis. For example, one user further expanded on a navigational
element, specifying she also expected the site to be straightforward.
Meanwhile, other users were more concerned with design, purpose, and
credibility. For example, Melanie stressed all three in her response with stating,
professional engaging design was necessary, as well as having, the purpose
of the website clearly communicated (via ethos/ pathos/ logos). Lastly she
stressed some form of evident credibility.
It was interesting to see that one of three users was only concerned with explicit
credibility when arriving at a new site. It would seem the others might be
judging this same quality but more so through the lens of design, navigation,
and feel that they assess in their first impression. That being said, I did expect
most would be concerned with how easy it should be to go through the site.
2) In thinking of a travelling company, what characteristics/ qualities would you
expect them to embody?
From this response all users expected an enticing site that was aesthetically
pleasing, using appeals to pathos to their advantage. For example all expected
some form of colorful, bright, fun, exciting. Such adjectives are seen
next to their similar expectation of there being [quality] photos.
Additionally all did expect, in some way, credibility to be shown. Kerri-Lee
seemed to have the least concern for this topic with only implying the subject in
her desire for both positive and negative reviews. Meanwhile, Mel and Caty
provided more explicit responses. Mel used adjectives corresponding to this
trait such as, Professionalism, Trustworthy, Experienced, and
Brault 39
Knowledgeable. Similarly, Caty explained how she would expect them to be
well known and well advertised. Although Caty did not list specific qualities or
traits her explanation can be seen as implying those akin to what Mel had
pointed out. I was surprised the users did not have more to say in this response
to this question.
3) What is your first impression of AirBnb? And what does their company name mean
to you when you hear it?
From this response two out of three users had no prior knowledge of, or
association with, AirBnb. These two had interesting speculations on what it
might be. Kerri-Lee generally thought it to be either a travel agency or a science
company. Caty, on the other hand, was it bit closer in thinking AirBnb might be
bed and breakfast on an airplane.
Mel, although she did have prior knowledge of the company, only knew its
general concept and that there seemed to be good (low) rates for staying
somewhere.
Together the responses might suggest however that name AirBnb has no clear
connotations. Doubting that they would want to change their name, as it would
lose some artistic flare (just to be more simple and clear), be a difficult process,
and take away from the credibility they have already established among their
current customer base, AirBnb might want to think about a better branding
campaign. What I mean by branding campaign is for them to find a way of
broadening the audience the generally reach. This might mean that they just
expand efforts and reinforce more consistently (i.e. more ads in magazines per
year or in more places at once), or this might mean that they re-approach their
marketing design in how creative it is and what sort of impression it leaves. In
other words they may want to change how they are engaging their audiences
and to what depth.
4) What do you think of their idea/ product?
The responses for this question were definitely more sparse then others. I think
this was primarily because, even though one of the users had heard of it, none
of them actually ever used it. Kerr-Lee had no answer while Caty explained, that
if it were what she predicted in the last question (bed and breakfast on an
airplane) than she would be all for it. Mel was the most skeptical in saying that,
although it sounds good in theory, it actually could be a misadventure waiting to
go wrong.
Brault 40
It was interesting to see this here. The other two users, Kerri-Lee and Caty,
seemed to consider this aspect less. That being said, I do think Mel brings up a
reasonable concern. Safety should be a guarantee as much as possible.
5) If you were to use AirBnb for international travelling to someone elses home, what
3 things would you want to know most about and why?
The responses for this question varied more largely than others. Most common
on average concerned themes of safety and quality. Users Mel and Kerri-Lee
were most concerned with safety. Kerri-Lee would like to be notified about any
on-going conflicts (violence, wars, or crises) in the country she is going to. This
broader, more general, consideration suggests she is willing to take the average
risk that comes with traveling anywhere new but would not like to put herself
unnecessarily in a place of heightened risk. Addressing more of the exact
location, Mel on the other hand, would like to know about background checks
on the on the residence and host(s) as well as a little information on the level of
safety in that chosen neighborhood. Mels consideration suggests a much more
in depth look should be taken into who can become a part of AirBnb. Having
such background information on specific places and individuals would seem
necessary considering the amount of a traveller invests in the situation as well
as the literal fact that they are putting themselves directly into a closer vicinity
of unknown variables. Although the two, Kerri-Lee and Mel, are dealing with
different levels of safety, they both highlight the important ends of the
spectrum.
Kerri-Lee and Mel also agreed on another point, one considering quality of the
experience. Both mentioned they would like to have some way of knowing
about the surrounding of the place they have chosen to vacation at. This
indicates, that although they are open to trying new things, they would like to
be educated about what situation they could be walking into, and perhaps in
between the lines, what kind of culture they should expect to experience.
Interestingly, Kerri-Lee was the only one who considered what sort of
preparation she might need for a different country if any. For this reason she
would like to be aware of items needed for travelling to or in that country.
The most practical out of the three in this response, Caty, simply wanted to
know about costs, level of quality customer service, and coupons. Her response
suggests she would like to know more immediate, tangible factors before
considering potential risks. Seeming a methodical person, I believe she would
eventually weigh the latter information but only after she was sure of how
realistic a trip was for her.
Brault 41
Post-Test Analysis
1) How would you rate your experience with being able to choose a desirable
location?
On a scale from 1-10, all users rated this experience with an 8. Accompanying
comments were both positive and negative. They read, Not enough on the
site or, Very easy to locate places. Pretty pics.
This rating suggests that the site does enough to help even new users look
through and locate desirable locations. However, they could work on improving
this process itself. In other words, the issue here is internal; the final product is
useful to the users but the process is not as useful (or fulfilling) as it could be. I
believe a deeper audience analysis, by AirBnb, would be helpful in allowing
travelers to make better educated decisions and ultimately feel more connected
with, and trusting of, AirBnb.
2) How would you rate the general processing of booking such a location?
This response received different ratings from each of the users with some
commentary. Kerri-Lee rated this as 9, stating it was pretty easy. Mel rated
this as an 8, while Caty responded with a 7 and stating it was relatively
easy.
Thus the average rate general processing of locations received was an 8. This
number indicates, similar to the last, that improvements are being asked for but
the process is navigable enough.
3) How would you rate the information available to you, as a user, in the scenarios?
Here the average response is 9, again with the 3rd user (Caty) pointing out she
enjoyed the feature of reviews.
This number indicates that, generally speaking, the listings were satisfactory.
The numbers seemed to depend on what the user considered their biggest key
influences. Those who were less concerned with safety and mores so with
practical thoughts, gave this section a higher rating. But for those requiring
specific details, especially of safety concerns, when such information was
absent, not surprisingly gave this section the lowest rating (8).
Brault 42
4) Is there satisfactory customer service?
The average response to this question is 4. One user focused on the simple fact
that there was no contact information for the company period, and thus
indicated it was a 0 in her book for this question. Another implied it did not
matter initially but if a problem arose then that would be troublesome.
Interestingly, the user with the highest security concerns, rated this issue with
an 8.
In retrospect it seems that each of these numbers is reflective of the individuals
way of processing a problem. For those who were more passive, they did not
mind going through any, lengthy but available steps through the website and so
provided this question with a higher rating. Whereas for those more direct, and/
or who preferred instant gratification, the rating to this question was low.
For this range of users it could be concluded that the best approach for AirBnb is
to cater to both audiences (passive and aggressive).
5) Considering the scenarios, do you think anything on the website needs to be added
or altered? If so, what and how?
One theme emerging from the responses here was a request for direct contact
information.
Specifics put forth on an individual basis included areas of content inclusion, site
tools, and policy practices. In terms of content inclusion, Kerri-Lee put forth that
she would like to see a better method of seeing the exact location as well as an
actual list (or view) or the areas surrounding interests. This kind of information
would provide users with a clearer understanding of what their daily view would
look like, and more importantly, what daily activities and restaurants they might
have access to. In terms of site tools, Caty would have like to see a more
efficient method of direct comparison. In essence she would have liked a tool
that brought possible choices she liked together and compare the most
important details of each side by side. She pointed out however that this was
likely directly influenced with her work in the engineering field, as this is how
they typically make choices when buying products with/ for their company. This
note was included to point out how directly outside experiences can influence
potentially frequent users.
Lastly, in terms of policies, Mel stressed the importance of implementing
methods for which the company makes background checks on hosts. She
further brought up a point of liking to know whether or not this business was
certified by the BBB (Better Business Bureau). Though specific to her, these
needs highly influenced her interaction with the site throughout the user test;
Brault 43
and although it was not as much a focus for other users, they too all
commented, at some point, that they would like more background information.
For these reasons Mels thoughts are important for AirBnb to consider. There is
a constant thread of wanting more security and information on that security.
6) Would you trust this site? Would you use it?
Despite some of the high rates each user gave this site in the first questions the
majority answered this question with a consistent level of skepticism. The first
user, Kerri-Lee admitted, Im a little on the fence. This response sounds like
she wants to believe in the company because she likes the idea of what they
offer but might not have enough for her to comfortably use them for
international travel. On the other hand User 3, Caty, clearly said, Yes, it
appears official. Taking a moment to analyze the language though, it could be
argued that she is operating on first impressions and giving the benefit of the
doubt. The word appears indicates she is being influenced by visual but
subconsciously wonders how accurately those visuals represent their intended
message of success and credibility. If she did believe they were truly official
why then, did she not say something akin to, Yes. Their official website proves
to me I can trust them?
Interestingly, in the middle of both opinions, stands Mel. She admits the site
looks credible but remains unsure if she would pursue the experience they are
potentially offering.
7) Would you recommend it to a family member?
Two out of three users responded, yes. One of three users responded,
maybe.
It is interesting some may not use this themselves, or are at least very cautious
in doing so, but would still openly recommend to a family member. These
responses might suggest that the unique adventure the company offers is just
enough to override skepticism, or questions left unanswered.
8) Do you feel you are leaving this testing with any valuable information?
All users responded yes. Two our of three users commented on the point of
learning about a cool, new, and cheaper way to travel.
It appears that their first interaction with the site may have not been enough to
have them sign up for a trip right away but it was enough to leave them with a
positive impression. More importantly however, did it get them adjusting to the
concept. As the two who commented indicate, they like the idea of AirBnb.
Brault 44
Maybe this first visit on the site was enough to bring them back for
reconsideration and attempts to find more info.
9) What did you enjoy, or not enjoy, about this experience?
Two out of three users specifically enjoyed learning about something new. This
information suggests a good number of likely users for this site are openminded. (This was predicted by persona information.) It also suggests that the
content was interesting enough for them to walk away feeling excited and/ or at
least positive.
The other 1/3 user responded to this question about how aesthetically
appealing the site was for her. This note implies, that although she was not
certain if she would actually use their service, she at least felt engaged by the
visual presence on their website.
Recommendations from Top Findings and

Pre/Post Test Analyses
Brault 45
(Note: Original guidelines asked for short and long term changes or improvements. Not
understanding why there would be recommended changes for only short-term use, I
have here substituted with words more appropriate for my recommendations. The
substituted subjects will be quick fixes and complex fixes. In essence, the former will
take a short-term amount of time and the latter a long-term as the corresponding
processes will take longer to construct and implement.)
Quick Fixes
Complex Fixes
Place the search bar for finding locations at Work with a software developer on
the top of every page.
building an interactive map for under each
listing. Ideally this map can change in range
of scope - highlighting areas from, the city
as whole, all the way down to individual
buildings. (Google Earth might be the best
application for this.)
Have a different framework for narrowing

down final listing options. For example,
when in the wishlist folder that is
created, generate a viewing option that
allows users to compare their narrowed
choices by whichever lens they choose.
Go through entire website with a content

writer and re-conceptualize a more
efficient approach to grouping the sites
information together.
Collect more specified data to represent on

AirBnbs About Us page. Use this data to
represent interactive components on the
map so users can find out even more.
Other possible pop-up options (when
hovering over or clicking) could be fun facts
or emerging images in places of interest.
Integrate an application that allows for

instant messaging with other users. Allow
users to turn notifications for these both
on and off; but additionally offer a more
standard email option when messenger is
off. In other words, build a community
messaging system, very similar to that of
Facebook, so users can build a sense of
together-ness and, most importantly, learn
more about specific locations/ visits.
Brault 46
AirBnbs How It Works page could find a
better way to represent the information
currently on it. For its current state it
should either consolidate its information to
another, more appropriate location (page)
that contends with similar information. If
not, this page should at least clearly show
that there is more information available in
these areas; they can do this by
highlighting the links with a different color
and font size (creating emphasis) or,
perhaps better yet, they could have the
icons above the small paragraphs of
information interact with the user and be a
portal to the corresponding information.
Have an interactive or, at least, a video

tutorial that welcomes users once they
have signed up for an account; this tutorial
will cover basic operational aspect of
website but most importantly point out
where to find policies and payment plans/
methods.
Generate a mission statement and place it

somewhere on their home page.
Collect a variety of both positive and

negative reviews. Then apply a framework
in which these will be applied to the home
page. Perhaps designate a certain area of
the home page to these reviews and
refresh them at least once every month.
Employ a better application that will

successfully translate foreign languages in
full. If possible have that same application,
another, or a person double checking how
well this is done.
Should have a personal presence on the

site. For this AirBnb should a) provide a
contact number for the company and b)
offer live chats with official company
representatives.
AirBnb could create more jobs and have
editing positions that go in and work with
hosts to improve their basic writing skills.
Brault 47
Portray a positive work environment
through highlighting employee
appreciation; AirBnb could feature certain
employees every month as welcome
representatives on home page not only
provides welcoming face but gives
background on someone who works for
this company.
Or AirBnb could also have a list of
departments employees belong to and
what they do (accompanying pictures could
be provided with each name); this might
imply a visual directory. If they wanted to
get really interactive they could have a
visual directory in which workers use video
clips to introduce themselves and explain
why they decided to work at AirBnb.
Additionally they could offer a suggestion
of a place they would travel to and why.
AirBnb could do a survey on what their
average customer would like to see in their
section, About Us; for this they might
research and analyze what some of the
most successful companies put in these
sections (study the genre constraints); they
should also more deeply consider and
develop what kind of persona they would
like to present here.
Brault 48
Call To Action
What has this projects series of tests indicated? The HE and User-Test have supported
that further attention should be paid by AirBnb to consider their audiences. This
consideration breaks into multiple areas as I have demonstrated. That being said more
research should be conducted on the level of authentic interaction AirBnb generates. I
believe that my project has demonstrated that even when sites can be well-designed,
aesthetically appealing, work easily and are even easy to navigate, does not guarantee
that the content is structured in the best way possible. Additionally passing such basic
usability standards does not guarantee the quality of content. Within this quality of
content, the tests done on AirBnbs website have proven that a companys persona is
reflected in the quality of their content and is noticeable to the user in some capacity.
Thus, we are brought to a somewhat newer area in usability, one that extends beyond
the scope of simply asking how well the technology works. What I mean by this is we are
now considering an area that draws connections between well-done mechanics and
represented content - as the former highly influences the latter. Usability must do more
research in considering the bridge being built between technological platforms and
quality of content. I must further qualify that this quality of content ideally maintains
authentic access to all forms of even potentially pertinent information.
Appendices
Brault 49
1.
2.
3.
4.
5.
6.
7.
The Heuristic Evaluation

Personas
Test Plan
Consent Forms from each participant
Pre-Test and Post-Test Questionnaires completed for each participant
Moderator Script
Logs from sessions
Brault 50
APPENDIX ITEM: 1
AirBnb :
A Heuristic Evaluation
By Aubrie Brault
ENL 620
11/3/15
Brault 51
Table of Contents
i.
ii.
iii.
iv.
v.
vi.
vii.
Introduction
Heuristics
Ranking System
Persona
Scenario
Tasks
Conclusion
p. 3
p. 3
p. 5
p. 6
p. 7
p. 7
p. 33
Brault 52
Introduction:
There are a variety of methods for evaluating user experience of websites and
their products. Often, before the opportunity of live testing with subjects on site
occurs, or if there are a little to no resources, then another useful approach can
be enacted to predict and find specific user problems likely most important to
the intended users. Heuristic Evaluations can be such resourceful, preventative
measures.
In my case, as a student, it is my lack of resources that most notably influenced
me to conduct a heuristic evaluation. I chose to take a unique heurisitic
approach to the company AirBnb. AirBnb was selected first and foremost for
my interest in how they are approaching travel and the unique solution they
provide. In my research however, I also became concerned with their ethics of
operation which seems to have broken down into how authentically, creatively,
and simply they may or may not be dealing with customers. It is for this latter
reason that I decided to integrate two different sources of heuristic principles.
Heuristics
For this research a somewhat new set of heuristics have been created to
appropriately and effectively evaluate the website of the international company,
AirBnb. The following consists of some of the 5 Es as adapted from the IUG
Evaluation Report (Example) provided to our class ENL 620. Integrated, for more
specific evaluation, are also heuristics from Grice (2013). Any heuristic principles
or details ending with *, are marked such so as to imply altercation. For
example, heuristic 3: Engaging has kept the title and its definition as seen in
IUGs report but, its more defining characteristics, have been pulled from Grice
to add a deeper context to meaning of that meaning.
Effective: How completely and accurately the work or experience is completed or
goals reached
quality of the user assistance
presentation of choices in a way that is clearly understandable
good interface terminology
redundant navigation
Efficient: How quickly this work can be completed
number of clicks or keystrokes required or the total time on task
navigation design elements
Brault 53
keyboard shortcuts, menus, links and other buttons
well-designed with clearly expressed actions
Engaging: How well the interface draws the user into the interaction and how
pleasant and satisfying it is to use
makes user feel welcome*
Use Visuals to draw users in*
Use sound to enhance the experience*
Make a Connection (Engaging extended): How well the website creates meaning
for the use*
Engage people in what is going on
Understand potential barriers and offer users identifiable ways to
overcome them
Use well-crafted story telling to immerse users in the encounter
Share Control: How well the site provides resources and encouragement for user
involvement.*
Follows standard usability guidelines
Provide users with resources to construct something
Provide a selection of professional-quality components for users
Make the process of interpretation participatory
Ensure user actions will not have bad or irreversible consequences
Brault 54
Ranking System:
To ensure I could get the most level of detail and thoughtful consideration of
what my researched persona, Aylara, finds troubling I have used the ranking
system as seen in the Hufflepuff Heuristic Evaluation Report. Rather than making
a table separate and away from the tasks they are referring to, these rankings
have been marked within the primary tables with the corresponding tasks. They
are color coded as you see here; this ranking system consists of the following
levels:
f) Positive Results in a beneficial effect on the users ability to perform
their given task
g) Cosmetic Issue: Affects the appearance and should be fixed only if time
permits
h) Minor Issue: Hinders the users ability to navigate and should be fixed
when possible
i) Major Issue: Frustrates or confuses users and requires repairs as soon as
possible
j) Catastrophic Issue: Prohibits users from performing their given task and
requires and immediate modification.
Brault 55
Personas
Restaurant Assistant Manager, The Brooklyn Grange
Photo:
Name:
Aylara Kovir
Job title/
major
responsibilities:
25 years old
Single
Demographics:
No children yet
Aunt to her older sisters son
Has her MBA and a minor in womens studies
Brault 56
Goals and tasks:
Aylara feels she has more than earned her right to travel in the last few years of rigorou
work. For this reason, along with the fact the she seems to be the only one of her friend
who isnt making time for travelling, Aylara is ready to get out of the country. Hearing g
things about AirBnb from family members and such friends, she wishes to become fam
with the site, make an account, book a stay, and become a continuous user. Naturally
ambitious, liberal, and a risk taker Aylara believes this is the site for her. She likes their
seemingly authentic approach to how experiences should be made
Environment:
Aylara will be finding and securing a place abroad for she and her friends. She is most
comfortable experiencing new sites in either her NY studio with Wes her pup, or, at her
favorite local coffee shop. Being of generation X she is comfortable with the idea of the
as well as the technology it is likely to be built on and influenced by.
Quote:
Be sure that your service is more than satisfactory, aim for exceptional.
Scenario:
25 year old Aylara is preparing for her annual trip with three of her best friends
from her alma mater, NYU. The trip is in 6 weeks. She and her friends are looking
to spend a week in Amsterdam. Aylara is starting to prep now. She will be
making all of the booking arrangements for herself and friends, who are of
course, all staying together. Aylara has a had a few family members use AirBnb
with success so she is choosing them as well. That being said, she has never used
the site the before. Heuristics 4 and 5, Make a Connection and Share Control
respectively, are whole headers and defining characteristics as pulled from Grice.
Brault 57
Why? Such methods were pursued so as to avoid information that didnt
necessarily need to be evaluated. AirBnb is a company that follows general
usability concepts well so I wanted to be sure this time was spent analyzing more
critical, relevant data.
Tasks:
1)
2)
3)
4)
Make an account
Figure out how it works/ browse through site and see what you can learn
Book a stay
Setting up Payment Methods
Aylara Navigating
Heuristic 1: Effective: How completely and accurately the work or

experience is completed or goals reached
Tasks
quality of the user assistance

presentation of choices in a way that is clearly understandable
good interface terminology
redundant navigation
Screen shots
Brault 58
Task 1: Make an Account
Issue: (Minor Issue)
Aylara was first drawn to Become a
Host as that has more attention being
drawn to it with box
Recommendations:
Box sing up option in white as well or
make bold
To make even more clear, AirBnb might
choose to place this somewhere in the
center of the page; however they do
have a How it Works button under the
Welcome Home which may make most
sense to keep there.
Brault 59
Task 2: General Browsing; see what
you can learn
Issues (Cosmetic):
Host opportunity popped up first (S/S
1), so Aylara wasnt sure if she was going
in the right direction as nothing told her
what may lay just beyond that. But she
decided to scroll anyway to make sure
(S/S 2) Thus, there is no real assistance
and there seems to be a strong emphasis
on hosts not making her feel
(subconsciously) very welcome.
Once she clicked How it Works (S/S
3) the links on the next page (S/S
4)werent well highlighted (or rather
drawing attention to themselves)
She felt the page itself wasnt all that
helpful. She wanted more explicit info on
what each category actually was (S/S 4).
Aylara would have preferred current
clients face for these categories rather
than generic, animated figures. (S/S 4)
Recommendations:
Offer online tech support via live chat
offer at bottom of window with
highlighted or hovering link
It seems like it would make more sense
to have travel information located
primarily throughout site as that is an
introductory act; working with a
company is more of an in depth process
you become a representation of that
company more than just a travelling
client does Or they could categorize
these options a little better; maybe offer
general info on home page and
testimonials and then have a traveller
tab and a host tab at upper right hand
corner as first segues into the
corresponding information.
Put color on hyper-links
Brault 60
Use more direct, descriptive language
use suggestive figures with smiling
clients focused close ups
Brault 61
Task 3: Book a Stay
Issues:
(Cosmetic) Only potential issue was
overlapping prices on map didnt
separate themselves if she hovered over
them. Thus, she couldnt really tell what
she was looking at by price alone (on the
map). (S/S 1)
(Minor) Aylara had a hard time finding
any mention of a cancellation policy
She wanted to know just in case. (S/S 2 &
3)
Recommendations:
small software fix allow prices to be
differentiated from one another while
mouse hovers over particular ones.
create any easily found section for this
with highlighted header or hyperlink
have owner explain in his or her own
terms as long as it is clear
make PDF downloadable version with
signature from host
place information after availability
section (last category on left hand side)
Brault 62
Task 4: Setting Up Payment Methods
Issue (Major):
Aylara had a hard time finding this area
(S/S 1 & 2)
Once she did, vague language made
choices more difficult (S/S 3)
Additionally, form was not immediately
ready or out in front of her the
additional step of having to click add
payment method seems unnecessary/
redundant (S/S 3)
Recommendations:
Maybe have site direct someone to
setting up payment methods
automatically once any account is made
aka direct audience to it so that they are
already set up when they go to book
something and dont have to look for it
use more explicit language
and also just categorize users from the
get-go; on main page offer each user to
operate under a certain hat (host or
traveller) sort of like when you view
mycourses through a student view or
your own facebook via an outsider view.
From this AirBnb could then categorize
everything they are operating in under
terms of host needs; this way
information doesnt have to be split up,
lagged out, or juxtaposed awkwardly
amongst each other and different users.
Doing this would allow users to be both a
traveller and a host they just have to
choose the option of operating as one or
the other (but both are accessible in
their own account).
Brault 63
Heuristic 2: Efficient: How quickly this work can be completed
number of clicks or keystrokes required or the total time on task

navigation design elements
keyboard shortcuts, menus, links and other buttons
well-designed with clearly expressed actions
Tasks
What works (Positive):
Making an account for Aylara was
made easy. It took her 3 minutes to
complete the task. On the site itself,
even with an initial scan of the page,
the sign up took 1:18 seconds. The
remainder of the time was spent
going into her email and confirming
her new airbnb account. The sign up
portion of this task (on the website)
only required 3 clicks. Aylara found
navigation simple as well. The initial
sign up button was where she
expected it to be. Immediately after
clicking it a floating window brought
sign up log in information to her.
Recommendations:
No recommendations
Screen shots
Brault 64
you can learn
Took Aylara 5 minutes to give the
site a brief once over (for this she
only veered 2 clicks away from the
main page in any direction).
Shorter pages show more is
available to scroll down to making
scroll down bar unnecessary. It also
engages the reader/ user further into
the site.
Issues (Minor):
When Aylara scrolled all the way
down, which was a ways, there was
no option to bring her back to top of
the page.
Links werent made immediately
identifiable. Images of places led to
further information about those
locations but are presented as 2D.
Recommendations:
Provide a back to the top option once
one has scrolled down passed a certain
point or a small icon that they can
identify will do this and click on that.
(The latter option may be more
aesthetically pleasing
Put a more distinguished look to
clickable items, i.e. have shadows make
images look 3D
Brault 65
Task 3: Book a Stay
(average clicks per locale 7)
Because Aylara was searching for
places in an entirely different
country/ culture she spent more time
looking through the options. On
average she reviewed 9 posts per
hour spending about 10 minutes with
each. Aylara found design elements
very user friendly for what content
was posted. One navigation feature
she particularly liked was the fact
that every individual location opened
up its own tab. This allowed Aylara to
keep open and compare various
locations at once, while still returning
to the mainframe to choose yet
another.
Aylara also appreciated ratings,
number of reviews posted, and
profile pictures of the hosts were
provided before clicking on a
location.
Recommendation:
Work with Google Earth and have
virtual tour of location
Brault 66
Issues (Major):
Somewhat harder to find
Actions Not Clearly Expressed
Recommendations:
Have a primary payment method be
the first step someone organizes once
they have made an account; as soon
as they have returned from
confirming their email address. With
this though the site should also allow
additional payment methods and
future changes.
Much like when apps change and
show automatic tips or offer a
tutorials for those changes AirBnb
should also consider this. (especially
tutorial walk-throughs.)
More explicit direction and
language needed for payment
methods. Use different descriptive
titles.
Brault 67
Heuristic 3: Engaging: How well the interface draws the user into
the interaction and how pleasant and satisfying it is to use
makes user feel welcome*
Use Visuals to draw users in*
Use sound to enhance the experience*
Tasks
Welcomes user directly with large font;
unique choice of the word home
connotes the website is a home for the
user comfy, trusted, place. Also a
unique inception for implying the
purpose of the site to share homes and
authentic experiences.
Offers Help link on home page
Addresses user directly
Visuals in background are actually
silent short films
Warm tones for button graphics
Issue (Cosmetic):
No Sound
Recommendations:
Add light background music to main
home page looping sequence
Screen shots
Brault 68

you can learn
What Works (Positive):
Engaging, vibrant visuals, YouTube
Videos offered
Issues (Minor):
Where do you want go bar gets lost
in Aylaras initial scroll down; she was
more drawn to the overpowering visuals
Travelling user (non-host) not initially
welcomed during first exploration; the
natural reaction to scroll down Aylara
was greeted with host information
Organization of content initially made
it harder for her to understand it was her
options that then followed after Host
opportunities
Recommendations:
Use different graphics and an emphatic
tactic to highlight search bar just a bit
more or place it in a white space to break
from
Re-arrange content/information:
possibly offer tabs for intended use of
site - host or traveller (but dont grid lock
them into this role)
or after main image scroll down to a
split screen that introduces these roles/
options
Brault 69
Task 3: Book a Stay
Aylara feels welcome in that the
site is giving her a choice from the
beginning on how she would like to
narrow her choices. Although she and
her friends are willing to make this a
worthwhile experience that are not at
a point in their lives where they want
to spend above the average per night.
Something that stood out to her the
most were the profile pictures of
hosts.
Issue (Major):
Generally speaking most hosts
offered pictures of good quality and
many of them. But, being that the
amount of pictures and their quality
are determined by the host, Aylara
felt some, potentially desirable sites,
were left untouched because she
could not get a good sense of the
place/ space.
Recommendation:
Have AirBnb certify and regulate
pictures stock
Have them prepare a bare minimum on
number and types of pictures hosts must
include
Offer virtual tour of space
Brault 70
More personal information is reached
through (reasonably) clicking on ones
name, which is highlighted by a personal
profile picture.
General design layout and colors
implemented did feel welcoming to
Aylara.
Issues:
(Cosmetic) Aylara expected little to no
visuals in this process and this remained
true; no visuals or sounds enhancing any
portion of the experience
(Minor) Aylara was not immediately
drawn to her other options; text on the
3rd screen shot in this category may be
organized well but it does not have
emphasis portrayed in appropriate areas
Recommendations:
Have icons for actions near categorized
content/ sections
Offer better differentiation between all
primary options on page / implement
emphatic strategies more explicitly
Brault 71
Heuristic 4: Make a Connection (Engaging extended): How well

the website creates meaning for the use*
Engage people in what is going on
Understand potential barriers and offer users identifiable ways to overcome
them
Use well-crafted story telling to immerse users in the encounter
Tasks
Screen shots
Brault 72
Task 1: Make an
Account
What works
(Positive):
Aylara is drawn in my
silent short films
going on as soon as she
reaches the main page;
though she does not
know the individual(s)
of each new showing
they are engaged in
their atmosphere
appreciating it and
that engages her as a
user.
Aylara felt there were
no barriers and she did
not experience any
technical difficulties
when she needed help
on the current page she
was on there was
always a help button
available that actually
changed its content to
the corresponding page
she moved to.
On that same bar, and
thus on every page, was
the AirBnb icon that if
licked would bring you
right back to the home
page.
Issue (Cosmetic):
Other than silent
short films at top of the
page Aylara didnt feel
any other real presence
of personal experiences
until she later go to
host listing/
descriptions and
reviews
Recommendations:
(Engaged by silent short films)
(Example of a corresponding help bar has changed to

accommodate Profile page.)
(The Bar)
Brault 73
On the main page
immediately offer a few
head shots with blurbs
containing testimonials
(customer experiences
do come through when
people leave reviews
but there is no real
place for people to
comment on their
experience using AirBnb
as the coordinating 3rd
party (the real host))
Brault 74
Task 2: General
Browsing; see what
you can learn
What Works
(Positive):
On the whole a lot
does do well to engage
Aylara; there are a
variety of places
immediately displayed
to her
Issues:
(Major) Through
exhausting every link
she could find, there
was no way to directly
contact the company;
no email or phone
number is provided
(Cosmetic) Aylara had
also become
accustomed to seeing
owners, Ceos,
Presidents, and the like
all be introduced on
previous/other
websites specifically
the home pages, almost
as a greeting. Now she
realizes it was a way
she felt the site offered
direct, explicit
credibility. Here she
infers an anonymous or
omniscient presence
and she is not sure how
she feel abouts that.
(Options Places to view)
Brault 75
Task 3: Book a Stay
What works:
There are various
filters she discovered,
that she may use to find
the place best for her
and her friends (S/S 1)
Issue (Minor):
Host information and
how much of it existed
was dependent entirely
on how much the host
wanted to give; there
seems to be no format
meaning they can
provide as little (or as
much) info as they
deem necessary. Even
with descriptions,
Aylara found in some of
them, not enough
pertinent information
or even some in English
for international
Travellers (S/S 2 & 3)
(Filters to narrow down or specifically search options)
Recommendations:
Being that this is a 3rd
party site whose entire
mission it is to connect
people around the
world and cultivate real,
worthwhile experiences
it would be in
AirBnbs interest to
ensure all language is
accessible they need
to make requirements
for what Hosts post and
how they do it.
(Brief Listing)
Brault 76
(Non-English Listing)
Brault 77
Task 4: Setting Up
Payment Methods
Issue (Minor):
Aylara was not
directly engaged with
possible methods of
payment nor did she
feel supported in
attempting to figure
this out generally. The
potential barriers were
not so much physical
inhibitors or technical
errors but rather lack of
guidance and/or explicit
information.
Recommendations:
When user signs up
have a video tutorial
from the company
founder take them
through the payment
process. This will
accomplish both
engaging the reading
for an effective purpose
as well as establish
some sort of
relationship between
the company and the
user.
Brault 78
Heuristic 5: Share Control: How well the site provides resources

and encouragement for user involvement.*
Tasks
Follows standard usability guidelines

Provide users with resources to construct something
Provide a selection of professional-quality components for users
Make the process of interpretation participatory
Ensure user actions will not have bad or irreversible consequences
Screen shots
Brault 79
Once Aylara had the time, she
enjoyed browsing most of the site
and was happy to find how well her
own movements on the site were
recorded but even more so how
many interesting ways there were to
express herself.
Issues (Major):
Initially, Aylara anticipated that
having a tab titled Your Trips would
allows some sort of narrative record
to be shared, especially on the site.
But when she went to go view other
random travelers profiles there
werent any trip stories depicted.
Aylara felt this should have be an
option. (S/S 1)
Although Aylara does like that she,
as a traveller, can also have this sort
of rating system as seen in screen
shot 2 she does think, if not
monitored, some people could use
this as an opportunity to put down
others.
Recommendations:
Offer a virtual scrapbook feature
for people to (however creatively
they wish) construct a shareable,
virtual version that can not only be
shared with friends, but viewed by
other travelers on the site to display
what they felt were some highlights
of specific locations.
AirBnb should consider instituting a
policy (similar to the coordinator of
our tech works study case in ENL
265) that is monitoring these systems
in an appropriate manner. This is not
to say negative reviews should be
deleted, as they are just as helpful,
but rather to ensure no one is using
Brault 80
this as an opportunity for malicious
intent.
Brault 81
Task 2: General Browsing; see

what you can learn
Issues:
(Minor) Aylara had also become
accustomed to companies having
blogs or at least forums for the public
to have more live, accurate, genuine
discussion. She found no opportunity
for this.
(Cosmetic) Though she does not
generally expect websites to have
suggestion areas she could have seen
this particular type of online business
as benefiting from one.
(Major) Aylara also noticed that any
policies could only be found through
a small link at the bottom of every
official page. While participating in
certain activities - any disclaimers or
potentially important info is not
included on the current page. Thus
meaning users should think to read
through all policy terms and
conditions before doing anything.
Recommendations:
For important areas of booking,
payments, posting any sort of
personal or sensitive information
there should be, if not a direct and
clearly seen explanation on the page,
at least a clearly marked symbol that
signals this information may have
disclaimers the user should be aware
of. Ideally I would think this symbol
would also offer a direct hyper link to
the page/ information.
Brault 82
Task 3: Book a Stay
Aylara found she could contact any
potential Host even before booking
with them
Other member are allowed, and
very much encouraged, to post
reviews of their stays.
She felt that trying to book a stay
with someone was a pretty natural
step-by-step process.
The tone and language that the site
operates in for this section (as in
most others) is very accessible with a
laid back and casual tone. This
particular feature made Aylara feel
more welcomed and subconsciously
that maybe staying in someone
elses home wasnt as strange as
might as first sounds.
Recommendation:
Additional, helpful features hosts
should be able to include a welcome
introduction video of themselves and
of their spaces being offered.
AirBnb should also encourage hosts
to offer a list of what might be the
best attractions in their area.
Brault 83
Task 4: Setting Up Payment

Methods
Does allow you do edit and/ or
delete card numbers one may insert
if they are put in wrong. Aylara
appreciated the room for error.
There was an opportunity for
Aylara to seek help on the page
once she found the page. That being
said it was simply FAQs
Issue (Major):
No way to connect with real human
intelligence if a problem arises
Recommendations:
Offer more immediate and
personable opportunities for user to
directly engage with company at
least offer number; here live chat
would also be a good idea.
Brault 84
Conclusion:
Conducting this heuristic evaluation there was an interesting mix of issues
brought to light. Aylara, a representative form of users, highlighted variables I
expected would not be as major of an issue has they were. Ironically, she also
downplayed things I anticipated might be more of an issue. Through her I was
able to see that most users her age, that also have a similar work ethic, may
indeed have a sense of instant gratification but are willing to look for things. On
the website/ product end of the HE spectrum my predictions of content over any
catastrophic usability items proved true. AirBnb then, might be an aesthetically
appealing site actually meeting all expectation of Nielsons 10 (had I used that
heuristic). In reality however, this HE has revealed that some deeper thought
needs to be employed in and throughout their own digital persona.
Brault 85
APPENDIX ITEM: 2
Personas
Photo:
Fictional name:
Job title/
major
responsibilities:
Aylara Kovir
25 years old
Single
Demographics:
No children yet
Aunt to her older sisters son
Has an MBA and a minor in womens studies
Brault 86
Goals and tasks:
Aylara feels she has more than earned her right to travel in the last few years of rigoro
work. For this reason, along with the fact the she seems to be the only one of her frien
who isnt making time for travelling, Aylara is ready to get out of the country. Hearing
things about AirBnb from family members and such friends, she wishes to become fam
with the site, make an account, book a stay, and become a continuous user. Naturally
ambitious, liberal, and a risk taker Aylara believes this is the site for her. She likes their
seemingly authentic approach to how experiences should be made
Environment:
Aylara will be finding and securing a place abroad for she and her friends. She is most
comfortable experiencing new sites in either her NY studio with Wes her pup, or, at he
favorite local coffee shop. Being of generation X she is comfortable with the idea of th
as well as the technology it is likely to be built on and influenced by.
This companys product is ideal for me but their visuals dont do it justice and
it takes too long for the payment platform to load. Im looking elsewhere.
Quote:
What she looks
for in a site:
visual presence, simple navigation, reliability, social media connections,

an online community
Brault 87
Clinical Nurse Leader, California Pacific Medical Center
Photo:
Fictional name:
William Blakemore
Clinical Nurse Leader, California Pacific Medical Center
Job title/
major
responsibilities:
56 years old
Married
Demographics:
Two children (grown)

Grandfather of three
Has an MSN (masters in nursing) and an associates in exercise science
Brault 88
Goals and tasks:
Environment:
William is a seasoned traveller. Recently he has wanted to start using AirBnb to take
Weekend family trips as well as weekend getaways with his wife. His main goal is to
Continue growing a cultural and spiritual balance with such a hard work schedule. Additio
William has a secret affinity for cooking so he likes to pick up special recipes among his
Travels. William found AirBnb as a recommendation from a google search. Like Aylara,
He would also like to become familiar with the site, make an account, book a stay, and lik
Remain a continuous user.
William will likely be browsing such sites and possible vacations either in his office after
hours, or in his man cave at their beach house. For his first look through he likes to browse
and then after some research has been done present some options to his wife. Unlike most
of his friends, Williams job has kept him up to date with most technology. Although he is
no 24 year old who grew alongside every generation of the iphone, William is generally an
intermediate user. Going into new sites he does not get initially overwhelmed or confused.
That being said William does admit he cannot keep up with some of the apps on sites,
Language used, and methods of finding certain information.
This company offers an interesting product, what is it made of and where can I purchase some
Quote:
What he looks
for in a site:
clarity, simple and easy navigation, contact information, professionalism, written explanation
accessible language and information,
Brault 89
APPENDIX ITEM: 3
Pilot User Test:

AirBnb
Brault 90
Aubrie Brault
11/16/15
Purpose:
The purpose of this usability test is evaluate the official AirBnb website. A
previously conducted Heuristic evaluation identified that, although AirBnb is
generally successful by practical standards (Nielsons 10), it is lacking in contentoriented ways. These more social, philanthropic issues seem to be especially
pertinent to such a company as they are more than producing a product they
are in the making of experiences; where this differs from the former is that the
process is much more personal. Monies, individual safety, constant
communication and in depth knowledge then, are all required for this process to
be appropriate and successful. With the location of these issues already,
generally identified, the purpose of this usability test then becomes to better
understand why those issues exist, how customers naturally respond to them,
and most importantly, to gather data that can be analyzed for offering solutions.
The surface of AirBnbs usability practices might present itself as clear and
steady, but beneath that surface, there seems to be a broken infrastructure
existing unintentionally and out of direct reach of the clients it is aiming to
attract.
Thus, this purpose can simply be broken down into, what I like to call, the 3 cs:
Credibility, comfortability, and community.
Problem statement: What are the issues that need to be addressed in this test?
What are the goals of this test?
- The issues needing to be addressed in this test are
o The ease of which a first time user can navigate the site
o
The quality of information accessible
o The comfort and confidence levels an audience can build with the
company and the product its offering.
Brault 91
o The lack of an online community
o The lack of company persona, presence, and interaction
-
Thus, essentially our goal of this usability test is to evaluate the website on
its abilities to be:
o Efficient
o Effectively-engaging
o Accessible
User Profiles
o Users Typical Characteristics:
Young (20-30)
Ambitious
Risk-Taker
But willing to look for answers via navigational curiosity
Computer literate/ technologically advanced
Prefers simplicity and clarity
Enjoys visuals and multimodal experiences
Gives and expects ethical considerations
Likes to feel in control and aware of important variables
Can be known to rush through things or not pay attention to
detail (depends on severity of situation)
o Motives:
Gain authentic worldly experiences/ check out places
internationally
Also be able to compare those international options
To try new things/ make new memories
To network/ communicate internationally
Brault 92
To get to know third party (the site/ company)
Understand the company and their priorities (as knowing this
ensures their best interest is (hopefully) at heart)
Scope out potential, future living locations
Preview potential homes and locations for travel
-
Participant incentive: Mirasols, target, or Dartmouth mall gift cards?
Tasks with projected timeline:

o For every task, with the exception of those *d, user must use think
aloud protocol. (Time in parentheses are anticipated amounts of time
spent.)
Pre-test* (10 min)
Watch think aloud Protocol* (5 min)
Get a first impression of the home page (2 min)
Look to know company better (7 min)
Make an account (2 min)
Choose and compare two locations (15 min)
Discover options for interaction/ communication (10min)
Seek information on how to deal with a booking gone wrong (15
min)
Post-test* (10 min)
Brault 93
Consent form:
This user test is for graduate research intended use only. Any information
generated during this user test is only for enl 631 graduate research to be
specific.
By signing this form I am allowing the tester of this user test, Aubrie brault, to
have control of the information given. I further allow any video or audio
recordings to be used as implied.
Print:______________________________________
Signature:____________________________________
Date:______________________
Pretest Questionnaire
I.
What are your expectations in using/ coming to a new website? (generally

speaking). In other words what yours general attitude toward such
processes?
II.
In thinking of a travelling company what characteristics/ qualities would you

expect them to embody?
III. What is your first impression of AirBNB?

IV. What do you think of their idea/ product?
Brault 94
V.
If you were to use AirBnb for international travelling to someones home,

what 3 things would you most want to know about and why?
SCENARIOS:
Scenario 1
- You really want to get some more travelling in but if youre going to spend the
money you want to make the most out of it. It just so happened that a friend at
work overheard your conversation and mentioned shed heard good things
about a company called AirBnb. So, now you have come to the website for the
first time and want to get a good first impression/ feel for the site. How would
you go about doing this on the website?
Scenario 2
- Interest genuinely piqued, you want to make an account and some more
exploring. How would you go about this on the website?
o Follow up scenario 2A: You are now at your personal account platform
and would like to look around. What draws your attention first?
Scenario 3
- You have now learned that the company caters to both hosts and travelers. You,
being a traveler, now have your curiosity piqued; the idea of getting a true,
authentic experience elsewhere, through the near-eyes of a local is enticing. But
before you fall head over heels in love with just buying a plane ticket and getting
a taste of the world, you want to know more about the company. They sound
interesting and appear professional, but perhaps for being a new company, you
have, at most, heard their name only whispered. Thus, you dont how the
company got started, whom it was started by, or how the idea for this product
was even inspired. Before investing money, time, and, in this more-uniqueprocess, personal safety - you want to know - who is this company, for real? Can
I trust them? How would you go about finding information that satisfies this
need for you, the customer?
Scenario 4
- Well, things seem to be pretty solid and, despite any reservations you might
have at the moment, you would like to go ahead and pick two locations to see
how they compare. Simultaneously, this little comparison, you believe, will also
demonstrate to you a bit more clearly how the site operates. Your primary goal
here however is consider the information the hosts provide you. In this situation,
Brault 95
you have enough money to comfortably book a stay 3 nights in Amsterdam with
2 friends, for up to $200 a night. By these details how would you go about doing
this on the website? Is there enough satisfactory information for you to
ultimately choose one location over another?
o Follow up scenario 4A: You have now chosen one of those locations and
would like to get more direct information about the location youve
picked. What might be at least two ways you go about doing this on the
website?
o Follow up scenario 4B: Imagine you have read all of the reviews under
this chosen location and many of them are old or static. In this situation
you know you would want more updated information but dont want the
likely biased opinion of the host. Sso for the sake of getting more
opinions, where else on the page (or site) might you look for such
information?
Scenario 5
- Imagine you have already booked a stay via AirBnb in Amsterdam but, with only
3 weeks left until the trip, a family emergency asks that you cancel. Having
never cancelled before however, you are unsure of how this works. In fact, you
dont know where to begin. How would you going about seeking information on
this process?
o Follow up scenario 5A: Still trying to cancel your stay, you immediately
contact the host, letting them know you can no longer make it, but they
are not responding to you. Needing the money back to help with the
family emergency you become worried In this situation what would you
do now?
Brault 96
Post Test Questionnaire:

I.
How would you rate your experience with being able to choose a desirable
location?
II.
How would you rate the general process of booking such a location?
III. How would you rate the information available to you, as a user, in these
scenarios?
IV. Is there satisfactory customer service?
V.
Considering these scenarios, do you think anything on the website needs to

be added or altered? If so what and how?
Brault 97
VI. Would you trust this site? Would you use it?
VII. Would you recommend it to a family member?
VIII. Do you feel you are leaving this testing with any valuable information?
IX. What did you enjoy or not enjoy about this experience?
Evaluation Methods
o Quantitative: (done via computer video recording):
How many clicks per task
How long the task takes them to complete
o Qualitative:
Audio record commentary
Video record body language/ facial expressions
Pre and post questionnaires
Test Environment and Equipment
o Environment
Brault 98
o Ideally UMD book a room in the library
o ??
o Equipment
o Laptops (2)
o Ipad
o Wifi
o Adobe Premier App
Moderator Checklist:
XX: Before participant arrives
- Make sure producr is loaded and read for first scenario
o Make sure
Have consent form ready
Instructions are printed out for user
Site is up (wifi functional)
Camera is set up recording ready
iPad camcorder set up and ready
have note pad for any of my own notes these may include clicks per
page
make sure questionnaires are also printed out and readily available
pen as well
have Pandora music playing in background
A) Welcome:
- introduce yourself, thank participant for participating
-
offer refreshment
direct participant to lab area
Ask participant to sit
Brault 99
-
Sit beside participant
B) Consent form, pre-test questionnaire, instructions

- Show participant locations of cameras, phone, microphone
-
Explain the purpose of the test
Go over the consent form, allow time to read and sign; if this has been done
already, ask participant if he/she is comfortable being recorded
Explain that there are observers who very interested in learning from the
participant about his/ her experience
Ask for questions, concerns
Give pre-test questionnaire
C) Instructions:
- Explain process of using scenarios, one at a time, while participant thinks out
loud
-
Show think aloud protocol video and ask if they have any questions
Explain to them how to show that they have completed a scenario
Specify how many scenarios there are
D) After completion, post-test questionnaire:

- Give post-test questionnaire
-
Generously thank for experience
Provide stipend for participation
APPENDIX ITEM: 4
CONSENT FORMS
USER 1:
Brault 100
USER 2:
Brault 101
___________________________________________
USER 3:
APPENDIX ITEM: 5
Brault 102
Pre-Test Feedback:
Question
Kerr-Lee:
Mel:
Caty:
1) What are your

expectations in
using/ coming to a
new website?
- Easy to navigate
- Straight forward
- Not an over-load of
information
- Runs quickly
- Professional,
engaging design
- The purpose of the
website should be
clearly
communicated (via
ethos/ pathos/
logos)
- Credibility
I expect it to be
easy to navigate,
clear, and not too
busy.
2) In thinking of a
travelling company,
what characteristics/
qualities would you
expect them to
embody?
- Information about
destination
- Pictures
- Reviews (Negative
and positive)
- Colorful
- Professionalism
- Experienced
- Savvy
- Knowledgeable
- Trustworthy
- Fun
- Exciting
I would expect
them to be wellknown, well
advertised. To have
bright fun ads with
vacation pictures & I
would expect them
to provide a
complete experience
i.e. Hotel, Rental Car
etc.,
3) What is your first

impression of
AirBnb? And what
does their company
name mean to you
when you hear it?
I have never heard

of it. Air suggests,
either a company
that has to do with
travel, or company
that works with
chemicals and
sciency-stuff.
It is a company that
rates and markets
private residences,
which are rented out
to travelers as
discounted rates. I
dont know too
much about it.
I do not have prior

experience. Is
AirBnb a company?
I am picturing a
plane with beds and
breakfast inside.
Brault 103
4) What do you
think of their idea/
product?
N/A
5) If you were to use

AirBnb for
international
travelling to
someone elses
home, what 3 things
would you want to
know most about
and why
Things needed to
travel to the other
country so I can be
prepared for the
trip.
Any type of
violence, wars, or
crises going on in
the country to
ensure that I will be
safe during my visit.
Fun things to do
around where Ill be
staying so that I can
have fun while Im
there.
Sounds lovely in
theory in practice,
it kind of sounds like
a murder waiting to
happen
If its a plane with

full beds and eggs
benedict, I think its
fantastic.
Background on the
residence owners
(jobs, reference,
criminal checks).
I would want to
know:
Quality photos and

video.
The surrounding
areas points of
interest access to
public transport (as
well as
neighborhood
safety).
- How much does it

cost?
- Hows the
customer service if
something gets
messed up?
- Are there
coupons?
Post Test Feedback

*rating questions are on a 1-10 scale
Question
Kerr-Lee:
1) How would you

8 Not enough on
rate your experience sites
with being able to
choose a desirable
location?
Mel:
Caty:
8 Very easy to
locate places. Pretty
pics.
Brault 104
2) How would you
rate the general
processing of
booking such a
location?
7 Relatively easy.
3) How would you

9
rate the information
available to you, as a
user, in these
scenarios?
10 So many
reviews
4) Is there
satisfactory
customer service?
5 Seems like if you

have no issues but if
you have problem
theres no corporate
to help you.
- Any policy on
background checks/
more information on
hosts
- BBB (Better
Business Bureau)
certified?
- Add data
comparison feature
for hosts locations
that are being listed
- Put a corporate
phone #
6) Would you trust

Im a little on the
this site? Would you fence. Id need to do
use it?
more research
before I decide.
Site looks credible.

Not sure if I would
use it just because
Im cynical about
humanity.
Yes, it appears
official.
7) Would you
recommend it to a
family member?
Yes, its cool and

cheap to stay.
Maybe.
Once I used it and it

turned out well. Yes.
8) Do you feel you

are leaving this
testing with any
valuable
information?
Yes, I can now travel Yes.

and stay for cheaper
at really cool places.
5) Considering the
scenarios, do you
think anything on
the website needs to
be added or altered?
If so, what and how?
9 Pretty Easy
No, I could not find

any contact info for
the company.
- Contact info for the

company
- Locations of the
sites/ destination
- Appealing things to
do around the
location
Yes, I actually really

like the idea of
AirBnb.
Brault 105
9) What did you
enjoy, or not enjoy,
about this
experience?
I liked learning
about something
Ive never heard of
before. However it
was a little lengthy.
Very aesthetically
pleasing and, at
times, funny.
I enjoyed seeing a
new site I would not
have found
otherwise.
Brault 106
APPENDIX ITEM: 6
Moderator Script
Welcoming the participant: Thank you for coming. How are you? [continue
small talk] Can I get you any water or coffee?
State the purpose of the study:
The purpose of this usability test is
evaluate the official AirBnb website. A previous test identified, that although it
can be aesthetically appealing, it does lack in some essential areas. In other
words, AirBnb looks pretty good, navigates generally well, but we want to look a
layer deeper. So, although we will also be taking note of practical mechanics, we
will also be looking out how you interact with the site consciously. The areas that
we are primarily focusing on can be thought of as credibility, comfortability, and
community. We would like to know if there is enough on the site for you to
confidently travel internationally and nationally. Now, during this process if you
also think of ways this experience could be improved, please let me know. I
really want to know what works well for you and what does not. That is the most
foundational purpose behind this testing. I would like to sincerely remind you
that this test is not an evaluation of you. It is not you that we are testing but the
product.
Provide necessary forms: Provide and explain the video consent form Are you
comfortable with being recorded? ... In this particular project you will be video
recorded and audio recorded. I will also be tracking things like how many clicks
per page etc. Such surveillance is intended to help me keep track of this, but
more importantly, it will also help me to see your initial responses while in the
scenarios. It is important that you think out loud clearly as well so that we may
refer back to your comments. This footage and audio will not be officially
published but is used for a graduate research course on usability at UMass
Dartmouth.
Describe the environment: [I will point out where any cameras and audio devices
are] I, of course, will be the only one with your while you are going through the
given scenarios. I will sit beside you, a few feet away.
Explaining the testing process: The testing process will begin with a pre-test
questionnaire with only 5 questions. Once you have completed the pre-test
questionnaire I will refocus your attention to the computer. Again, I will be
Brault 107
sitting beside you and this is so that I may also take notes for record.
Additionally, I am there if you have a practical question about the scenario
process and how to go about it. However, I cannot answer any theoretical
questions about the website; the most I can do is try and work through a
problem with you... Once we are seated in the respective places I will then ask
you to go onto the computer and head to AirBnbs official site.
Once you have completed a scenario please look at the laptop camera and
confirm scenario [whatever number] is now complete. At this point, till facing
the camera, you may add any commentary you have about the scenario just
completed.
For the conclusion I will please ask you to end with a post test questionnaire.
Describing thinking out loud: The think out loud element to this process is very
important. Though it might seem strange at first it gives us the necessary context
to your choices and more importantly helps us to understand how you are
evaluating what you are seeing and doing. As an independent party, your
thoughts on such matter are priceless. An example of thinking out loud might
sound like, hmm okay, Im seeing a lot of text on this first page but its not really
telling me what I expected to find here or, That is not at all what I expected to
see when I clicked on that link or, why is there no link here? I would expect to
seen one here. I may ask questions or remind you to share your thoughts if you
fall silent.
Ask the participant to share any questions or concerns: At this point, do you
have any questions or concerns? Also, at any point in this process you may stop
if you feel the need to. Thank you again for coming. Lets begin.
Start the study: [I will hand the participant the first scenario. I will then take my
seat beside them while they read the scenario; once they have finished I will
ask:] For this scenario, you can you restate what you are supposed to do? [Once
they do, we will being.]
Brault 108
APPENDIX ITEM: 7
Video Logs upon request.

Usability Report

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Usability Report

Uploaded by

Copyright:

Available Formats

2015 Usability Test: AirBnb

More the Just a Pretty Face?

Executive summarybrief synopsis of study, findings, and

Description of test plan and procedures (which include goals of

Top findings (positive and negative) with examples (e.g.,

Severity codes assigned to findings

Analysis from the post-task and post-test questionnaires

Recommendations for short-term and longer term

Call to action for more testing (as part of being a usability

Post-task and post-test questionnaires completed for each

Logs from sessions

Other material used for the test sessions

How was the project conducted?

The Report Structure:

Description of Test Plan & Procedures

How the view the company persona

The procedures within this test plan include:

Greeting the user and introducing them to the process

Top Findings: Bottom-Up Approach

Influenced by the 3 of Whitney Quesenbery 5Es, as well as the ideologies

Following my own, collective approach, I created the 3Cs. They are a

User-Specific Highlights Per Task

Didnt care to evaluate

Evaluated aesthetics more

Was not primarily concerned

Felt strange that FB had

Did not liked incentives, esp.

Evaluated more aesthetics

Felt home page did not

Wanted to find about

Expressed she wouldnt

Easily found where do you

Could not initially find where

Was really confused on

Would contact host directly

points AirBnb provided for

Would contact host directly

Attempted to visit blog

area at bottom of page

Felt only option was to look

through the social media sites

Initially thought to contact

Assumes she can directly

contact Host about this and

think of payment method

Took the route of hitting the

Found cancellation policy

Would contact host directly

Desired to speak with

Would immediately look for

she would look for corporate

The Severity Code Ranking System:

A1) While going through the

Place this search bar

Below is figure A1:

Work with a provider

Go through site with

**Done through a series of movements that are unavailable to be exhibited via

Category B: Effective Engagement

B1) All users did like the

B2) All users expected to be

Take banner off and

C1) All users wanted outside

Major Issue Credibility