Designing User Interfaces for Games

Zhan Ye
Human-Computer Interaction Institute
Carnegie Mellon University
December, 2000

Introduction
We have seen a lot of articles about artificial intelligence (AI) programming, path-finding
algorithms, and 3D modeling in various books and magazines. Given the fact that the
game market is so competitive, almost every aspect of game design and development has
been studied carefully to find better ways to create better games. But it is sad to see that
one important aspect of game design has been long neglected - user interface (UI).
Nobody has ever mentioned the role and importance of UI to a game. Nor have we seen
any research on how to design a better UI for games. It is interesting to notice that other
software industries have all invested a lot of time and effort in finding new methods and
processes to design and evaluate UI. Just look at those online job sites, you will find a lot
of job openings for UI designers, interaction designers, usability specialists and user
experience designers. But in the game industry, UI designers still dont have an
important role; in fact in most teams they dont even exist. Even in big companies that
have realized the problem, such as Microsoft, their attempt to integrate UI design into the
whole game design and development process is still in its early stage and hasnt shown
any promising results yet. Game designers still ask fundamental questions like Why
does user interface matter? In the first part of this article, I will try to answer this
question. In the second part, I will briefly describe the differences between the UIs of PC
and console games. In the third part, I will give some ideas about why and how to use
human-computer interaction (HCI) methodologies to improve UI design for games and
how to integrate these methods into the game design and development process. In the
final part of the article, I will give a detailed example using The Age of Empires II from
Microsoft to show how HCI methods can be used to help game designers locate design
problems and create innovative designs.

Why User Interface Matters

User Interface: What Gamers Actually See and Interact With
There are thousands of games coming out every
year, which can be classified into 5 to 7 different
genres. But no matter what genre games belong to,
User Interface
whether they are action games or RPG games,
strategy games or sports games, they all share the
Game
same structure. From a system architects point of
Core
view, a game is a three-layered system. Figure 1
shows this layered system. For purpose of this
paper, we call the inner layer the Game Core, and it
represents the mechanics of the game. It defines the
rules of a game, for example, how high the character
can jump, how many enemies he will encounter in
this level and where they are located. The Game
Figure1. Game as a layered system.
Core is designed by game designers who have a
perfect mental model of the game. Gamers, however, can not access this layer directly,
and therefore they can never get a complete mental model of the internal mechanics of a
game. If they do, playing the game would be no fun at all. To keep them away from the
Game Core, game designers wrap it with another layer, which is the UI layer. The only
way gamers can get knowledge of the Game Core is through the UI layer. It includes
both hardware elements (mouse, keyboard, controller) and software elements (menus,
buttons, icons.) The UI layer is a medium between the Game Core and the gamers.
Gamers have complete access to the UI layer and they have to be knowledgeable about
this layer to successfully play the game. The outermost layer is the Gamers layer. We
include gamers in this model because good UI is not a function of the software, but the
property of the user. Without gamers, the system is not complete and cannot be
evaluated.
Gamers

User Interface Defines Game Play

Most game designers dont realize the fact that user interface can define game play. It is
understood that game play is the most important property of a game. No matter how
beautiful the 3D images are, or how involving the story is, without good game play a
game definitely cannot succeed. But game play is a rather vague concept and hard to
describe. Software engineering principles tell us that we cannot improve a property of a
software product unless we can define and measure it. So it is very important to find a
way to define game play. One possible way is to define it is by the way gamers interact
with games; that is, through the UI and the interactions associated with it. By doing that,
we will able to analyze game play, not as an abstract concept, but something concrete that
can be described, measured, and studied.

User Interface Makes Games More Challenging

For RPG games, like the Final Fantasy series, you dont want gamers to see the whole
world map when they start the game. Instead, you want them to explore the world piece
by piece. The fun of playing games largely depends on the exploration and the
excitement and uncertainty it brings. For the same reason, we dont want gamers to
know the internal mechanics of a game when they start playing the game. We want them
to gradually discover how the game tricks them and what kind of strategies they should
use. Through trial and error, they will build an increasingly complicated mental model of
the game and feel a sense of accomplishment because their knowledge and skills are
improving over time. So UI doesnt only help gamers to play games successfully, but
also controls the pace at which the internal mechanisms will be revealed to gamers.
Eventually the internal mechanics will be revealed, and gamers will be able to finish the
game successfully.

User Interfaces of PC and Console Games

Differences between the User Interfaces of PC and Console Games

Figure 2. Console/Arcade Games like the

Street Fighter series use minimum GUI
elements. Their UIs are designed around
specific hardware the controllers.

Figure 3. We can find a lot of conventional

GUI elements (icons, buttons, sliders, status
bars, tabs) in the UIs of PC games like the
Civilization series. Some games, such as the
DOS version of Transport Tycoon, even
created their own windows systems.

There are many differences between the UIs of

PC and console games. The most important
difference, however, is that the games use
different input/output (I/O) devices. PC games
heavily rely on the combination of keyboard
and mouse. Whereas console games have
controllers, which are more refined, although
limited, input devices. The form factors of the
hardware, especially the I/O devices, greatly
affect the design of UI. From a gamers point
of view, the UIs of PC games are more
obvious than those of console games. Most
gamers realize that they are interacting with
some kinds of user interfaces when they are
playing PC games such as The Age of Empires
and C&C. Just look at the familiar icons,
menus, and windows. But gamers playing
console games sometimes dont realize that
they are interacting with the UIs. Can you ask
a gamer what kind of UI Super Mario 64 has?
UIs of console games are hidden deeply in the
careful calculation of the usage of controllers,
which is not obvious from the screen. We can
call the UIs of PC games software-oriented
interfaces, which means they utilize lots of
conventional GUI elements to represent
actions. The UIs of console games, on the
other hand, are hardware-oriented interfaces,
which means they are designed around the
form factors of controllers and use relatively
fewer GUI elements.

Why Transported Games Rarely Succeed

The reason why most transported games fail is very simple. The original game succeeds
because of its game play, which largely relies on the UI design. In order to transport a
game to another platform, game designers are facing totally new I/O devices. They have
to redesign the UI since it is the form factors of the I/O devices that determine what kind
of UI will work. By doing this, the original, successful, proven game play will be
affected. There is no guarantee that the new game play will be as good as the original
one since they are inherently different. Transporting a game to a different platform

should be considered one of the major challenges that

the UI designers face. The amount of work required
would equal, or even surpass, designing a UI for a
new game.

Figure 4. To solve the transporting

problem, The Playstation version of
Transport Tycoon even included a
mouse with the software. But even
that didnt help much and the
Playstation version still didnt do as
well as the PC version.

Why Use HCI Methods to Guide UI Design

Cannot Rely Solely on Our Intuition
In the current game industry, most game designs are guided only by designers intuition.
Game designers are seasoned gamers; they have a feeling for what works in games and
base their designs on this. There may be a few geniuses in the world who can do things
right by just using intuition, but for most of us, we cannot rely on our intuition alone.
The quality of games can certainly not be guaranteed this way. We need formal methods
and processes to guide, evaluate, and help us improve our designs. Sooner or later, game
designers will realize the limitations of their intuition and the ad hoc solutions they are
using now, and admit the necessity of incorporating formal methods and processes into
game design and development.

HCI Is the Closet Research Field to Game Design

Game designers need formal methods and processes in order to create high-quality
games. But where can they get them? Game designers dont have enough time to
research and design methods and processes from scratch. What they can do, however, is
to look at related research fields and other software industries and borrow ideas from
them. HCI is the closest research field to game design. It is the study of how people use
computers, and how to design, implement and evaluate computer systems so that can be
used easily, effectively, and enjoyably. UI is one of the major research areas of HCI.
Researchers have developed a number of methods and processes over the past 10 years,
some of which have been successfully used in other software industries and proven to be
very useful tools for UI designers. Among these methods, heuristic evaluation (HE),
cognitive walkthrough (CW), and Fitts law are especially valuable and can be adapted
for use by game UI designers.

User Experience and Game Play

I am curious about why UI has been long ignored by most game designers. It is
interesting to compare the game industry with other software industries. It wasnt until
several years ago that some software companies began to realize the importance of UI.
Since then, user experience, as a newly invented "magic word", has been talked a lot.
User experience is the high level emotional impact on the users by the visual elements,
information architectures, navigation, and interactions of a UI. Delivering an enjoyable
experience to users while they are using software is the ultimate goal of HCI research and
applications. Now many software companies have established their user experience
groups and started using formal methods and processes to achieve this goal. If we look at
the game industry, we will be surprised to see that it is way ahead of other software
industries in realizing the importance of the general user experience. The game industry
even invented a unique name for user experience for games, which is game play, from its
very first day since the early '80s. The game play in a game is comparable to the user
experience to a Web site or business software. The game industry coined the magic word

almost 10 years before other software industries. But the game industry just stopped
there, without any conscious attempt to find out formal methods and processes to help
game designers to create good game play. Game designers still highly rely on their
intuition to do their work, whereas Web designers have a number of well-developed
methods and tools to use.

How to Apply HCI Methods to UI Design

Get Average Gamers Involved
Most UI designers know the HCI mantra, You are not like the user. What it says is that
nobody, especially domain experts or usability experts, can replace the role of users in
user tests. Designers cannot evaluate their own designs effectively since they are not
actual end users. For example, in the business software industry, developers usually
dont use the software they develop. Before they start a project, they have no indication
towards what the software will be used for. It makes sense to hire real users to do user
research and user tests. But in the game industry, the situation is quite different. Most
game designers are avid gamers themselves; they know a lot about games. Therefore, it
seems as if there is no need to get average gamers involved in the game design process.
But this is wrong! We still need average gamers. There are two reasons for this: first,
game designers are too skillful. Their knowledge about games is way beyond that of
average gamers. Ironically, this will prevent them from finding obvious design problems.
Things that they think are too easy may be too tricky for average gamers. We must bear
this in mind expert gamers only represent a small fraction of the whole gamer
population and we dont want to design games only for them. Second, game designers
know too much about the design. Their mental model of the game is almost perfect
because they designed it. Average gamers can never achieve this. They have their own
simple, incomplete, sometimes incorrect, mental model. The mental model determines
the user behavior. This means that the user behavior of game designers will be different
from that of average gamers. Due to these two reasons, game designers, though they are
very knowledgeable about games, cannot replace average gamers in the game design
process. We need average gamers to evaluate and test our designs. The results we get
from these evaluations and tests will prove to be unbiased and invaluable.

Adapting HCI Methods for Game UI Design

Weve mentioned that there are a number of HCI methods that have been proven to be
useful and can be adapted for game UI design. The following three are particularly
suitable for game UI design.

Cognitive Walkthrough (CW): CW is an inspection method that focuses on

evaluating a UI design for ease of learning by exploration. Its focus is motivated by
the observation that many users prefer to learn software by exploration. This is often
the case for games. When gamers start playing a game, they generally know nothing
about it. They learn how to play and use the interface by trial and error. When game
designers design a game, they should be aware of this, and should design a game that
can be successfully played by exploration. CW can be used to quickly go through the
game design and determine whether or not gamers can do it. Furthermore, this
method doesnt require a matured prototype since it is a mental process.

Think-Aloud Protocol: This method can be used to observe gamers playing an

existing game. Essentially, it requires that UI designers ask gamers to think aloud
as they play games. It can help us to know whats in gamers minds, the strategies
they use, their expectations, frustrations, and confusions. Some people consider
think-aloud protocol may be the single most valuable usability engineering method
because designers now can discover things inside gamers mind.

Fitts Law: In addition to these two methods, another powerful tool that can be
helpful is the Fitts law. It is a very important theory that, once applied, can greatly
improve the efficiency of a UI design. It states that the time required to move the
cursor to a target size S which lies a distance D away is given by T=Im Log 2 (D/S +
0.5). The longer the distance, the smaller the target size, and thus more time will be
needed to click the target, as the consequence, the user will feel more stress. The
general design goal, then, is to reduce the distance between two controls and enlarge
the size of the controls under given constraints. Designer should also avoid device
swapping and disturbing the focus of attention. Fitts Law is extremely valuable not
just because we can now evaluate two alternative designs quantitatively, but also
because it provides a guideline and perspective for us when we think about design.

Problems of Rapid Prototyping and Iterative Design

A widely used design approach in other software industries is the combination use of
rapid prototyping and iterative design. Designers can use HTML, Director, or Visual
Basic to build a prototype, only implementing the front-end interface without worrying
about the back-end functionality, then they test the prototype with some users. After
getting useful feedback, they make adjustments to the initial design and enter another
round of iteration. This approach has been proven to be very successful in other software
industries. But it may not work that well for games. Games need precise controls and
timing, whereas business software does not. You dont need to worry about response
time when you build a prototype for business software. But you cannot ignore it when
building a prototype for a game. A simple prototype cannot faithfully represent the real
game play that the final game will have. A complicated prototype may be too expensive
to build. Comparing to business software, it is also much harder to separate UI with
back-end functionality of a game. So far there is no easy solution for this problem.
Perhaps game designers need some kinds of tools like Director, but with more interaction
features, that they can use to build prototypes.

UI Design Guidelines
User Interface Shouldnt Be Distracting
One common mistake that we have seen in a lot of games, even successful ones, is that
the UI is so beautiful, so overly decorated that it becomes a distraction. UI should have
aesthetic taste, but more importantly, it should be simple, efficient, easy to use, and
consistent with the whole game environment.

Figure 5. The UI of Iron Helix is a hyper-realistic and highly literal

recreation of a real-world console. Notice the details such as the
damage and corrosion. But the question is, Do gamers really
want to look at the UI, or they want to immerse into the beautiful
outer space? The UI occupies almost screen space and leaves
little room to display the virtual world the gamers really interested
in. This is one extreme example of how UI can be distracting and
overly weighted.

User Interface Should Provide Enough Visual Affordance

Many studies show that the majority of end users are very reluctant to read user manuals.
They prefer to learn how to use software through exploration. This is also true for
gamers. Never expect gamers read manuals, even for complicated strategy games like
AOE! Design a UI with good affordance instead! By affordance we mean form affords
certain actions and makes that apparent. A good UI should use its visual appearance to
suggest its functions. Gamers should be able to understand and act easily by just looking
at the UI.

Figure 6. AOEs building buttons use the actual images of the buildings. They dont provide
enough affordance. They are visually complicated and look similar to each other (especially
the first three in the first row and the last three in the second row.) They fail to provide a
cognitive bridge between the buildings themselves (the images of the buildings) and the
functionality of the buildings (what kind of products they produce.) Gamers, even experienced
ones, cannot easily figure out which building is for what purpose and will hesitate when they
need to choose among them.

Figure 7. In AOE II, the building buttons are greatly improved. They are elegantly simplified
icons. Although simplified, they represent their functions clearly. Because instead of using
the actual images of the buildings, UI designers focused on the functionality of the buildings,
that is, the products the buildings produce. Gamers will have no problem choosing the one
they want among them. The new design is more efficient since it can lower the cognitive load
of the gamers when they play games.

User Interface Should Be Balanced

The visual elements (buttons, labels) in a UI should be properly arranged, sized, and
aligned to reinforce logical relationships among them and ensure the UI is stable on the
screen. Most game UI designers pay little attention to these details, and they just put
controls and labels everywhere possible.

Figure 8. The display panel in AOE II is unbalanced, although the labels and icons are aligned. It
is overly weighted in the left side and leaves plenty of empty space in the lower right-hand corner.

User Interface Should Be Transparent

A good UI should be transparent. Gamers should be able to forget about the menus,
buttons, icons, and windows and immerse into the virtual world that the game creates. In
that sense, UI designers job is to design UIs that are transparent UIs that can be used
so naturally by the gamers so that nobody would even notice its existence. I strongly
believe that it is the ultimate goal of UI design.

Case Study:
Age of Empires (AOE) II is a strategy game published by Microsoft. It is well received
and praised by both gamers and game designers because of its innovative graphics and
involving game play. But even a successful game like AOE II still has its problems, and
using it as an example will enable us to realize that there are still many areas that can be
improved.
In order to evaluate AOE IIs UI, the first step, we need to find some tasks that are
frequently performed by gamers. Each task is an action sequence that is performed step
by step by the gamer to achieve a specific goal.

The Action Sequence of Constructing a Building

One of the most frequently used tasks in AOE II is constructing buildings. This action
sequence includes 5 major steps, selecting a villager, moving to the building panel,
selecting a building type, moving to the intended location, and clicking to drop the
building.

Figure 9. The action sequence of constructing a building. Red lines with arrows are used to show the
path of the cursor movement.
Before we analyze the action
sequence, we need to make
some assumptions about the
initial positions of the
villager and the cursor.
Lets assume that we have a
villager standing in the
center of the screen, and the
initial cursor position is also
at the center of the screen.
Step 1: The gamer clicks the
villager. The building panel
will show a number of
available actions.

Step 2: The gamer moves the

cursor down to the panel.

Step 3: The gamer chooses

one building

Step 4: The gamer moves the

cursor to the edge of the
screen to scroll the map.

Step 5: Then the gamer

moves the cursor back to the
center of the screen to
drop the building.

Action sequence is done.

Pop-up Menu Can Improve the Efficiency of the UI

The above action sequence, although it looks pretty complicated, is just a sequence of
mouse movements and clicks. You move the mouse to place A, click, then move it to
place B, then click, and so on. The time the gamer spend on moving the mouse is a lot
longer than that of a simple click. According to Fitts Law, the longer the moving
distance is, the longer the action sequence takes, the more muscle fatigue gamers will
feel. Cursor movement itself doesnt produce any direct results. The most obvious way
to reduce the moving distance is to use a pop-up menu. As Figure 10 shows, when the
gamer clicks a villager, a menu will pop up near the villager, the gamer can easily choose
one building from the pop-up menu. Comparing the distances between the two designs,
the new one may save less than 0.1 second every time. Sounds pretty trivial; but think

(a)
(b)
Figure 10. Comparing to the old design (a), the new design (b) greatly reduces the required mouse
movement.

about it: gamers usually have to do the same action sequence hundreds of times when
they play the game. The new design saves a lot of time, and much more importantly,
relieves the muscle fatigue greatly. It is not only an efficient design, but also helps to
relieve the pain and stress associated with computer usage to some extent.
The new design (b) also saves a lot of screen space. The building panel occupies a huge amount of valuable
screen space, and it is empty most of the time. Changing it into a pop-up menu leaves more screen space
for the world map. Research shows that the sense of immersion is very important to games and VR
systems. Having a large amount of screen space devoted to UI will reduce the sense of immersion, since
UI is not an organic part of the virtual world of the game. UIs should occupy the minimum amount of
space required to display the most important controls and information.

Two-Handed Interaction as a Hardware Solution to Map Scrolling

The new design we proposed early only solves the problem of the long mouse
movements required to construct a building. From the analysis of the action sequence,
we noticed that map scrolling also requires a lot of mouse movements. A gamer has to
move his mouse to one of the edges of the screen and stay there to scroll the map. Does
this remind us of Web browsing? You move your mouse to the right edge of the screen

to click on the scrollbar to scroll the Web page. They all require long mouse movement
to the edge of the screen. To solve this problem, Microsoft provided an innovative
design, IntelliMouse, which is a mouse with a scroll wheel. It is a hardware solution that
totally eliminates the required movement. The D in Fitts law is reduced to zero, which
means the efficiency is greatly improved. Similarly, we can use a hardware solution for
the map scrolling in AOE II.
In the real world, people perform tasks with
both hands, because it is more efficient. We
often use our non-dominant hand to hold
objects while using our dominant hand to
perform precise operations on them. Most of
todays UIs still make use of only one pointing
device that users manipulate with their
Figure 11. Microsoft IntelliMouse is an
dominant hand. A lot of research, however,
excellent hardware solution to the device
has shown that carefully designed two-handed
swapping/mouse movement problem.
interaction can improve the efficiency of UIs.
The most famous example is Xerox PARCs
Toolglass. I will not describe the details here,
but basically they used two pointing devices,
one mouse for the dominant hand, one
trackball for the non-dominant hand. The nondominant hand controls the position of the
Toolglass using trackball because it requires
relatively lower precision. The dominant hand
controls the cursor. We can adapt this design
Figure 12. Trackball is ideal for map
for AOE II using a trackball to control the
scrolling because of its form factor.
scrolling of the map and a mouse to control the
cursor. The benefits of this design are, first, the required cursor movement is zero. It is
basically a hardware solution to the problem. Second, by using two pointing devices, we
introduce parallel interaction into the game. Now gamers can move the cursor and map
concurrently. The steps 2 to 5 in the action sequence will be changed from being
serialized to parallel and it saves a lot of time.

Conclusion
The above case study explains how to analyze a UI and game play by studying its action
sequences. It also shows how to use design guidelines and Fitts law to find problems in
the action sequences. The proposed two-handed interaction shows that HCI research can
bring innovative ideas to game designs. Although we need time and real-world testing to
determine if the ideas innovated in the labs can succeed and eventually benefit us, it is
always important to have these innovative ideas first.