BCS THE CHARTERED INSTITUTE FOR IT

BCS HIGHER EDUCATION QUALIFICATIONS

BCS Level 5 Diploma in IT
March 2014
EXAMINERS REPORT
Principles of User Interface Design

General Comments
Candidates must answer the question asked. Answers that quote general principles
out of context of the scenario in a question will not earn marks. Candidates should
ensure they spend an amount of time and effort on sections of questions appropriate
to the number of marks available for the section. Candidates should not waste time
and effort copying out, or paraphrasing, the question in their answer.

Section A
A1.

As an HCI expert you have been asked to design the user interface for a new
mobile app which monitors and records in real time exercise activities such as
walking, running and cycling.
In this context:
a)

Describe TWO principles you would follow in determining the

placement of the controls for the user interface of the activity app to
help avoid user error.
(2 marks)

b)

Describe what is meant by rapid prototyping (also known as throwaway prototyping).

(5 marks)

c)

Describe how you would do a rapid prototyping of the activity

monitoring app.
(8 marks)

d)

Describe how you would do a cognitive walkthrough of the user

interface on a fully functional prototype of the activity monitoring app.
(10 marks)

Answer Pointers
a)

Sequentially used controls should be placed together so they can be used

successively.
Controls should be grouped together if they have a common function.
(2 marks)

b)

Rapid or throw-away prototyping is a method of generating prototypes of a

system to capture user requirements and user evaluation at an early stage in
the design process. As it is quick and low-resource, it enables a wide range of
options and potential solutions to be visualised and tested. Such prototypes are
not usually developed further but thrown away as the development process
continues.
(5 marks)

c)

This is an open-ended question.

Any approach that uses common rapid prototyping tools such as paper, HTML,
commercial wire framing tools such as Azure, Omnigraffle or Visio, or animation
tools is acceptable. The answer should demonstrate that the tools need not be
compatible with the language being used for the final system and that the
prototype should enable a good representation of the proposed interaction to
be created with the minimum of resource input.
(8 marks)

c)

This is an open-ended question, but answers should cover the following, in the
context of the above scenario.
A cognitive walkthrough is done by an expert evaluator, not test users. It should
anticipate the user interaction when an interface is used for the first time. Based
on knowledge of the users, tasks are selected that the user of the app will want
to achieve. The walkthrough should ensure that any actions the user needs to
take are enabled by the interface and all appropriate prompts and information is
available to facilitate task completion. It should also look at the adequacy of
recovery strategies in cases of error. As many different users and tasks should
be represented in the walkthrough scenarios. Any problems or issues with the
UI should be identified, described and possible solutions proposed.
(10 marks)

Examiners Guidance Notes

The issue that needs addressing in section a) is placement of controls. A number of
candidates gave answers discussing font size and text contrast that do not address
the question asked.
Some good answers were given to section b).
While candidates were generally able to describe an approach to rapid prototyping in
section c), few answers had sufficient details on the technology and techniques that
could be used.
For section d) a number of candidates gave answers describing using a selection of
users for the cognitive walkthrough. The cognitive walkthrough technique requires an
expert evaluator to perform it.

A2.

The following techniques are commonly used in human-computer interaction

(HCI) for evaluating user interfaces and systems:
heuristic evaluation, direct observation, cognitive walkthrough, focus
group, postal questionnaire, online survey, task-based testing.
As an HCI consultant, choose ONE most appropriate method from the list
above for each of the following scenarios and describe how you would use it
to achieve the goal.
a)

Initial design ideas for a website to promote a national charity.

(5 marks)

b)

Formative evaluation of a late prototype of a new word processing

application.
(5 marks)

c)

Trouble-shooting of a computerised control system for an ambulance

service after a critical failure during a major emergency.
(5 marks)

d)

Evaluation of the navigation and menus on a travel agent website.

(5 marks)

e)

Evaluation of the effectiveness of the help system in a government

online tax return submission system.
(5 marks)

Answer Pointers
For this question the candidate must use their judgment and knowledge of the
techniques. Acceptable answers would be along the following lines, although any
other well-justified answers are acceptable.
a)

Focus groups. This technique is useful for gathering and testing user opinion in
the very early stages of development and prototyping. Members of the groups
should be representative of the typical expected/targeted users and the
questions and issues well defined and articulated prior to the focus group.
Usually a tester who is an experienced facilitator/moderator would run the focus
group.
(5 marks)

b)

Task-based testing. This testing is typically done using well defined tasks
included in the requirements specification of the system. Usually a range of
typical users would be used, with user characteristics and experience gathered.
Scenarios defining tasks performed by the system would be developed with
protocols to enable the collection of appropriate timing and error data. Direct
observation/video recording of the user is typical and keystroke/event
monitoring could also be used, although this is typically time consuming and
expensive.
(5 marks)

c)

Cognitive walkthrough. Expert evaluation by walking through the scenarios that

gave rise to the failure. Looking at the cognitive issues and loads experienced
by the operators, such as the ability to achieve tasks within time constraints,
ability to recover from error etc. The walkthrough would make use of known
scenarios, take into account user feedback, user profiles and any problems with
the interface documented and solutions suggested. The walkthrough should
represent the experiences of a broad range of the users and tasks.
(5 marks)

d)

Heuristic evaluation. This could be either an expert evaluation, or with a range

of typical users. The goals could be loosely or tightly defined or broad scenarios
could be given to allow the testing of the functionality and effectiveness of the
navigation and menus while trying to accomplish typical user tasks.
(5 marks)

e)

Online survey: Online questionnaires would be used to evaluate the experience

of users, the tasks or when the help was required, problems encountered that
require use of the help, and how successful the help was in resolving the
problem. Users could also be surveyed at the end of the process using a
structured questionnaire to see if the help was appropriate, effective, easily
found and the level of user satisfaction with the help.
(5 marks)

Examiners Guidance Notes

Some good answers were given for this question with some candidates showing
good ability to think through the nature of the issue that required evaluation.
A number of candidates chose task-based testing for section c), and while they
received appropriate marks for any relevant points they made, it should be
appreciated that it is extremely unlikely that the exact circumstances that caused the
problem being investigated could be replicated using this technique.
A3.

You have been hired to use your human-computer interface expertise to help
in the development of the user interface for software that allows science
specialists to view and analyse their laboratory data.
You decide to take an ethnographic approach to your formative evaluation
study.
a)

Describe what an ethnographic approach is in the context of user

interface design.
(3 marks)

b)

Describe TWO advantages of using an ethnographic approach.

(4 marks)

c)

Describe TWO disadvantages of using an ethnographic approach.

(4 marks)

d)

Describe what you would do in terms of collecting and analysing data

to carry out the formative evaluation study using an ethnographic
approach.
Restrict the scope of your answer to the data entry screen of the user
interface.
(14 marks)

Answer Pointers
This is an open-ended question and answers will vary, but should be along the
following lines.
a)

An ethnographic approach emphasises the context of system and the users

when designing and carrying out the evaluation. In this scenario, the users are
highly specialised experts using the system in a very prescribed and consistent
environment. It also implies close contact and interaction with the users in the
environment in which they use the system.
(3 marks)

b)

Advantages - the answers could include any of:

- can be said to be more natural compared to more constructed
evaluations such as laboratory experiments.
- have ecological validity as regards the actual working environment and
user situation is taken into account.
- can give very useful models of users, task and requirements
- data can be contextualised to the environment the system is used in.
- should give a comprehensive perspective of the user and the
environment.
- enables better understanding of the behaviour of the users in the
environment they use the system in.
(4 marks)

c)

Disadvantages - the answers could include any of:

- require a high levels of expertise and experience to be used effectively.
- informal and loose approaches are often labeled ethnographic to give
them a more credible label (misleading).
- can be time consuming and expensive.
- data analysis can be problematic if not formally planned.
- results are highly dependent on the researchers observations and
interpretations so can be biased.
- can sometimes only gather qualitative data.
(4 marks)

d)

The answer should reflect the participative and contextual nature of the
ethnographic approach, and the specificity of the user environment. The
evaluators should work closely with the scientists and collect data using
appropriate methods such as interviews, direct observation, focus groups,
participative workshops and so on. Videos can be used, but analysis is time
consuming and expensive. Data analysis is sometimes described as reflective,
but should also include statistics, ratings and scalings. If videos are used,
coding schemes should be used for consistency.
(14 marks)

Examiners Guidance Notes

Most candidates could describe the ethnographic approach to evaluation, but details
of how to apply the approach and the advantages and disadvantages were generally
less well addressed.
The value of the participative and contextual nature of the ethnographic approach
should have received more emphasis in most answers.
Section B
B4.

You are a human-computer interface expert working for a world leading

consultancy company called UXB. The United Nations have approached UXB
and asked for help in designing an evaluation study of one of their
humanitarian websites. You are given the task of advising on certain aspects
of the study.
a)

Outline the methods you might use to help the United Nations set
usability criteria and give an example of criteria that might be
appropriate for this evaluation.
(5 marks)

b)

Briefly explain how sorting tasks could be used to measure usability of

the United Nations website.
(5 marks)

c)

Briefly explain how a structured walkthrough could be used to collect

usability data in a study of the United Nations website.
(5 marks)

d)

In the context of the United Nations website, outline what

measurements of usability you might use, how you might select users
and how you might collect data from users.
(10 marks)

Answer Pointers
You are an HCI expert working for a world leading Consultancy Company called
UXB. The United Nations have approached UXB and asked for help in designing an
evaluation study of one of their humanitarian web sites. You are given the task of
advising on certain aspects of the study.
Your answer must address the following:
a.

Outline what methods you might use to help the United Nations set Usability
Criteria and give an example of criteria that might be appropriate for this
evaluation.
(5 marks)
There is no agreed approach to setting criteria, but typically:
1.
2.
3.
4.
5.
6.

Brainstorming with design team and or clients.

Reference to published literature.
A priori reasoning.
Focus groups.
Survey.
Etc.

Criteria might be:

1. All links are easily understood.
2. Navigation is error free.
3. Finding information via links can be done within a set time or with a set
number of minimum clicks.
4. Etc.
Marks will be allocated to either methods or criteria to a maximum of 5.
b.

Briefly explain how sorting tasks could be used to measure usability of the
United Nations web site?
(5 marks)
Sorting methods are used to determine relationships between objects in design. Cf.
Clustering and Mind Maps. A possible approach here is to take all objects in the
interface and ask users to sort them according to relationships. These logical
groupings can then be compared to the layout (groupings) of the interface. This
gives an idea of how well user mental models have been transferred to the design.
c.

Briefly explain how a structured walkthrough could be used to collect usability

data in a study of the United Nations web site?
(5 marks)
Structured walkthroughs allow you to take the user on a journey that mimics
interaction in some way. Typically, you can walk the user through dialogue stages
and this allows you to observe clarity and understanding and collect comments from
the user.

d.

In the context of the United Nations web site, outline what measurements of
usability you might use, how you might select users and how you might collect
data from users.
(10 marks)

Open ended. Any reasonable and doable solutions are acceptable. Measurements
could be time and error; opinions or quantifiable observations.
(5 marks for this part)
Data collection would be laboratory based studies, field or naturalistic observations
etc. The main issue is robustness of procedure in whatever environment or context.
(5 marks for this part)
Transference of marks between sections will take place if appropriate.
Examiners Guidance Notes
The question goes to the heart of user centred approaches and some candidates
produced very good answers showing preparation and understanding. Some
attempts were not good and seemed to demonstrate a lack of preparation and study.
Some candidates produced uneven answers, i.e. some components were very good
while other components were poorer or not attempted. Candidates are advised to be
clear as to how many marks are associated with each subcomponent of a question
to allocate time and effort in the examination appropriately.
B5.

The BCS intends to provide a new public web-based service that allows
students to download teaching materials relevant to modules being studied.
The new download feature is to be designed using scenario based design and
you are a member of the design team. You are asked to do the following:
i)

identify potential stakeholders, provide a stakeholder diagram and one

example stakeholder scenario;
(5 marks)

ii)

provide a problem scenario that might describe the current practice

before the introduction of the new download feature; and
(5 marks)

iii)

provide a brief annotated storyboard, with four to six frames, showing

the outline of how one particular user might interact with the new
system.
(15 marks)

Answer Pointers
This is an open-ended question, but which addresses core Scenario Based Design
approaches.
For (i) and (ii) a reasonable attempt to provide scenarios with realistic and relevant
detail will be awarded full marks. Typically in (i) a list of stakeholder and a simple
diagram. A stakeholder scenario of ~ 100 words. A problem scenario of ~100 words.
Example Stakeholder diagram.
Examiners Guidance Notes
The question resulted in some good answers but also some poor answers indicating
a lack of preparation. Overall, candidates produced some very good Problem
Scenarios while many candidates seemed to struggle with Stakeholder Scenarios.
Overall, candidates produced some good storyboards.

For (iii) it is important that answers reflect the style of storyboarding advocated by
SBD. This means that the frames should be annotated and linkages should be
shown clearly. See example below.

B6.

Modern user interfaces are usually designed and developed through some
form of user-centred approach.
a)

Describe the main characteristics of user-centred design and critically

discuss the main advantages and disadvantages of this approach
compared to methods such as waterfall in designing user interfaces.
(10 marks)

b)

A multinational company wants to redesign all its eCommerce Web

sites. You are a human-computer interface consultant and must provide
advice to the company on user-centred design.
Outline and discuss what kind of prototyping approaches and types of
prototypes the company might consider when implementing user
centred design.
(15 marks)

Answer Pointers
a) This section involves some bookwork but also synthesis. Open ended.
The main characteristics of UCD are usually early and intense focus on users,
rapid prototyping of some kind, evaluation and iteration. A core characteristic is
formative evaluation. Answers could also include the use of scenarios to aide
communications between users and designers.
As regards advantages/disadvantages the answers should include an awareness
and knowledge of Waterfall models being based on initial requirements with no
real involvement of users. Changes are not possible until after the system is
designed and implemented, when it is too late to adapt the software to users.
Here, UCD offers huge advantages in that early and intense involvement of users
allows the system to take account of and develop requirements during the
development process. Users cant specify requirements initially so in that respect
Waterfall approaches are not able to fully take on board user requirements. An
advantage of Waterfall is control and management of the development process.
UCD is harder to control and manage and it is also harder to plan development
time. This is a disadvantage of UCD. Related to this is the fact that many UCD
methods require large initial investments in resources. Usually, this investment
pays off through less maintenance and support later.
3 5 marks allocated to main characteristics, 5 - 7 marks to
advantages/disadvantages. This overlap in marks will be applied to students
benefit and allows some flexibility in the allocation of marks.

b) This component is unseen problem solving and synthesis. Open ended and each
answer will be judged on merit.
Answers should address the core issues that characterise modern use of
prototypes used throughout the UCD profess. This should include:
Early mockup prototypes high versus low fidelity prototypes. Pros/Cons.
Storyboards as prototypes, linked storyboards for dialogue. Evaluating prototypes
through e.g. walkthroughs or simulation work, also software simulation.
(10 marks)
Vertical versus Horizontal software prototypes; Evolutionary versus Revolutionary
approaches to prototyping.
(5 marks)
Examiners Guidance Notes
While some candidates produced some very good answers, others showed little
preparation or study of relevant material. Some candidates seemed not to have read
or understood the question or its components and therefore produced material that
was not relevant. In some cases this seemed to involve writing down anything or
everything that might be relevant to user interface design. Candidates must stick to
the questions and read them carefully.