CHINZ 2009
The Wiimote with Multiple Sensor Bars:
Creating an Affordable, Virtual Reality Controller
Torben Sko, Henry Gardner
School of Computer Science, College of Engineering and Computer Science
The Australian National University, Canberra, ACT, 0200, Australia
Torben.Sko@gmail.com, Henry.Gardner@anu.edu.au
ABSTRACT
A low-cost virtual-reality controller has been constructed
from the Nintendo Wii Remote using multiple sensor bars.
An agent-based algorithm has been used to allow the con-
troller to move from one sensor bar to another, enabling it
to be used in a surround, multiple-screen environment. User
testing has verified that the system is suitable for use with
a demanding, fast-paced, first-person-shooter game. The
tuned version of the system was found to have a low error
rate and to be favourably received by test participants. Our
work has been open-sourced to encourage further research
in this area.
Categories and Subject Descriptors
K.8.0 [General]: Games; I.3.7 [Three-Dimensional Graph-
ics and Realism]: Virtual Reality
Keywords
Wii Remote, Sensor Bar, Input and Interaction Technology,
Agents and Intelligent Systems, Virtual Reality, Computer
Games
1. INTRODUCTION
The inexpensive Nintendo Wii Remote [5] (or “Wiimote”)
is a popular game controller which has given rise to many
innovative interaction techniques for computer games and
other applications. It contains a number of buttons, includ-
ing a D-pad on its upper surface and a trigger button be-
neath. It includes a 3-axis accelerometer, infrared camera,
Bluetooth wireless protocol, rumble feedback and it has an
add-on controller known as the Nunchuk (which has its own
accelerometer, a joy-stick and two buttons). The infrared
(IR) camera, located on the front of the controller, enables
the pointing direction of the Wiimote to be tracked relative
to two IR sources (where each consists of a cluster of IR
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise, to
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee.
CHINZ’09 July 6-7, 2009, Auckland, New Zealand.
Copyright 2009 ACM 978-1-60558-574-1 ...$10.00.
41
Interaction Techniques
emitters) on either end of a Wii sensor bar1 . This sensor
bar is traditionally mounted above or below the display.
The intriguing package of functionality in the Wiimote,
and its low cost and availability, have spawned communities
of interest amongst researchers and hobbyists. Outside of
gaming, reported applications of the Wiimote as a pointer
device include virtual reality [2], augmented reality [3], in-
teractive art [4] and interactive fashion design [1]. These ap-
plications used a traditional Wiimote setup where the user
faced front-on to a single display. Unfortunately, developing
similar applications for use in a surround environment (us-
ing multiple displays or even, as noted by [2], using a large,
single display) is restricted by the Wiimote’s narrow view-
ing angles. As previous work [8] showed, the Wiimote’s IR
camera has half-angles ranging from 14◦ to 22◦ , providing it
with around a 40◦ aiming range in both the horizontal and
vertical directions.
To address the limited interaction range of the Wiimote,
previous work [8] presented the main idea behind a Multi-
ple Sensor Bar (MSB) system which enabled the controller
to be used in a two-walled, virtual reality (VR) theatre. A
subsequent poster presentation [9] outlined the completed,
functioning system. This paper provides details of the core,
agent-based algorithm for handling multiple sensor bars and
describes the details of the interaction design and user-testing
used to refine the system.
2. SYSTEM
A key concept driving this work was to develop a “tracked”
virtual reality controller that could be used in low-budget,
non-technical scenarios. This required the system to be
cheap, accessible and easy to set up - requiring minimal
assembly and knowledge of the hardware. To this effect, our
hardware comprised of five wireless sensor bars (the “MSB
array”), a Wiimote controller and a Nunchuk. These com-
ponents can all be store-bought for under US$120 combined.
Figure 1 shows a single participant2 using our custom
MSB array in a back-projected, two-display VR environ-
ment. Each of the sensor bars are custom built to be free
standing (to avoid mounting) and oriented vertically (to
minimise their visual disruption and provide a slightly greater
vertical range). Their electronics have been daisy-chained
together, allowing all five sensor bars to be powered from a
single mains source. As with the standard Nintendo sensor
1
The term sensor is misleading given it functions as an emit-
ter. However, we have adopted it due to its universal use.
2
As the system works on line of sight, it is only practical for
one participant (excluding observers) to use it at a time.
 CHINZ 2009
Figure 1: The Multiple Sensor Bar system used with a first-person-shooter game in a two-walled, virtual
reality theatre (left). A plan of this theatre (right) illustrates the optimal usage region for the Wiimote.
bar, each bar supports two IR sources. The Wiimote, which
is being held in the right hand, has been specially mounted
on a pistol-grip and is being used for aiming, looking and
shooting. The add-on Nunchuk controller, held in the left
hand, is being used for player movement.
Using its IR camera, the Wiimote can track and report the
position of up to 4 IR sources at once, making it possible for
two complete sensor bars to be decoded simultaneously. The
position information from one bar can be compared with its
neighbour as a user sweeps across them. This allows the
software receiving the Wiimote’s data to reliably construct
an internal model of the physical environment. The software
can use this model to make a reasonable guess as to which
bars are currently being detected and therefore the direction
in which the Wiimote is currently pointing. Intelligence is
built into the driving software to allow the detection and
correction of erroneous assumptions about the environment
as play proceeds.
The MSB software driver developed for our system builds
upon the open-source library by Brian Peek [6]. Peek’s li-
brary provides the functionality for connecting and exchang-
ing data with the Wiimote. On top of this, we use a model-
based reflex agent approach [7], in the sense that the driver
tracks and uses what it has seen to interpret the IR data.
The agent has been designed to handle any configuration
made up of both vertical and horizontal sensor bars. It does
this by holding two internal models: a dynamic model, rep-
resenting what it has seen, and a static model, describing
the layout of the full MSB array. The dynamic model is
built from the IR data received from the Wiimote. The
static model is read from a configuration file and describes
the mapping of IR sources to pixel positions on the displays
behind them. As such, each physical setup requires a unique
configuration file. The agent computes an appropriate cur-
sor position by matching the two models together, calculat-
ing a scaling function, and using this function to convert
the aim of the Wiimote from an IR position into a pixel
position. The scaling function can be computed when at
least 2 IR sources are visible. However, when more than 4
sources are in the Wiimote’s field of view the reported IR
data is unpredictable. Consequently the MSB configuration
42
Interaction Techniques
TOP DOWN PLAN
Sensor Bar
Too close 2.9m
Full Coverage
Too far away
needs to comply with the following conditions: at least two
IR sources should visible at most times (justifying why each
sensor bar has two IR sources), two sensor bars are visible
when moving from one bar to another, and no more than 4
IR sources are visible at any one time. These restrictions de-
fine the usage region shown in the right hand side of Fig. 1.
In the full coverage region it is possible to continuously track
the pointing direction of the Wiimote for a full 180◦ spin.
When the software starts, the agent initially aligns the dy-
namic model to the centre of the static one, which is equiv-
alent to the assumption that the user first points towards
the centre of the MSB array. When comparing two succes-
sive dynamic models, the agent assumes that a model which
states that two particular sources are separated (i.e. on dif-
ferent sensor bars) is superior to one which has them grouped
together (i.e. on the same sensor bar). For each iteration, the
following steps are performed:
1. The agent groups the tracked, IR points received from
the Wiimote in an attempt to determine which IR
points belong to the same sensor bar. In doing this, it
creates the new dynamic model.
2. The agent decides whether the new dynamic model
confirms or contradicts the existing dynamic model.
If the new model confirms the existing model (i.e. no
sources are grouped to different sensor bars) then the
information from the new model is combined into the
existing model.
If the new model contradicts the existing model in the
sense that two or more of the points which were grouped
together are now separated, then the existing model is
replaced with the new model.
If the new model contradicts the existing model in the
sense that two or more of the points which were sep-
arated are now grouped together, then the most recent
coordinates of the IR points are propagated into the
existing model and the new model is discarded.
3. The agent aligns the dynamic model with the static
model using the previous alignment as a guide. If it is
 CHINZ 2009
unable to align these two models then it clears the ex-
isting model, waits for more IR data and then returns
to Step 1.
4. The agent derives the scaling function using the IR
points and their corresponding screen coordinates. A
global scaling function is updated with smoothing to
avoid sudden changes between iterations.
5. A cursor location is computed by converting a central
IR position, into a screen position using the global scal-
ing function.
The MSB software has been written to deal with out-of-
range errors providing the user re-enters the active area on
the same sensor bar. The driver can become confused in
situations where a user moves back into range on a different
bar. In such cases, the user can manually reset the system
using a pre-specified command, after which the driver once
again assumes that the user is pointing at the centre sensor
bar. The software can also automatically reset itself if it de-
termines that the new dynamic model contradicts the static
model. We denote occasions where either the user or the
system triggers a reset as a “reset-error”. For more details
on this algorithm, we refer readers to our website [10] where
source code for this system can be found.
3. USER TESTING AND INTERFACE RE-
FINEMENT
To assess the usability of the MSB system in a demanding,
3D environment, it was integrated into a first-person-shooter
(FPS) game engine. As described in [8], the Source [12] game
engine was ported to our two-walled display by a number of
modifications including altering the view projection, uncou-
pling looking and aiming (changing the fixed reticle system
to a bounding-box), and displaying a 3D cursor for con-
tinuous, visible feedback. A custom-built walk-and-explore
game, a custom-built FPS game and levels of the Half-Life 2
[11] game were used for user testing and interface refinement
over a period of 6 months.
3.1 First testing phase
The first phase of user testing was aimed at determining
whether the visual disruption of the MSB array in front of
the (back-projected) displays would detract from the user
experience. The non-demanding, walk-and-explore game
was used with an early version of the MSB system which was
fully constructed but with only one sensor bar being active.
The study was conducted with 12 volunteer participants (11
male, 1 female), between the ages of 15 and 44, all of whom
had computer games experience (8 participants with Wii ex-
perience). The participants were given a navigation task of
locating characters in response to audio cues and a manipu-
lation task of stacking boxes on top of each other using the
game engine’s physics system. They were asked post-test
questions from the Presence Questionnaire (PQ) of Witmer
and Singer [13] as well as 7 open-ended questions, specifically
addressing the use of the MSB array. Overall, participants
responded favourably to the control system, with aggregate
ratings for the PQ subscales of “Involvement/Control” and
“Interface Quality”, on a 7-point Likert scale with 4 neu-
tral and 7 best, as 5.7 and 5.3 respectively. No participant
found the physical presence of the MSB distracting, and
43
Interaction Techniques
some even failed to notice it. Some criticisms were made
of the bounding-box aiming-and-looking system, identifying
the need for further refinement.
3.2 Second testing phase
The focus of the second testing phase was to establish the
usability of the complete MSB interface with a demanding
FPS game. This environment required the use of multiple
button and motion commands, whilst simultaneously aim-
ing over a large area. We were particularly interested to
see how frequently the system was reset and whether this
resetting impacted on the enjoyment of the game. Other
considerations, not discussed further in this paper, included
the mappings of the Wiimote and Nunchuk controls. This
part of the interaction design was challenging as the use of
a pistol grip made the Wiimote’s upper controls difficult to
access. To alleviate this issue, the motion detection of both
the Wiimote and Nunchuk was utilised to a greater extent.
In the user familiarisation phase, the volunteer partici-
pants (7 male, 1 female, all with computer games experience,
6 of whom had previous Wii experience) were shown how
they could deliberately confuse the driver and then manu-
ally reset the system. For the main part of the test, the par-
ticipants completed two early levels from Half-Life 2 which
included gun combat, object manipulation and navigation
around the levels. Participants spent up to 15 minutes play-
ing each level, although most participants completed it in
around 10 minutes. During play, the driver logged each re-
set as well as the times when the Wiimote went out of range.
Participants were given a post-test questionnaire which in-
cluded 15 open-ended questions targeted on the usability of
the control system.
While all participants reported to have enjoyed the expe-
rience, most expressed moderate frustration with the con-
trol system. Several participants reported problems with
the vertical aiming, while some noted issues related to re-
setting. Some participants also described feelings of physical
discomfort due to the rate of movement and rotation of the
graphical scene. This feedback warranted further refinement
of the system and another phase of testing.
3.3 Third testing phase
For the third testing phase several modifications were made.
Firstly, the seating position was moved back from the apex
of the theatre and deeper into the optimal coverage area of
Fig. 1. The speed of the player movement was reduced (in-
cluding the rotation speed). The in-game aiming was mod-
ified to include acceleration along the vertical axis. This al-
lowed the player to reach a greater proportion of the screen,
therefore letting them more easily target threats near the
vertical extremities. Finally, minor changes were made to
the mapping of the controls. One of these included flipping
the axis of the Nunchuk motion control, as this was occa-
sionally being triggered when the controller was being held
in a resting position.
In this study, four of the second-phase participants, all of
whom had demonstrated higher reset error rates, were asked
to replay the first level from the previous test. They were
then asked to play a custom level which had most of the
threats located along the horizon rather than at the top and
bottom of the displays. It was hypothesised that this would
reduce the error rates given that systematic measurements
of the Wiimote’s range had shown that the system provided
 CHINZ 2009
a vertical coverage of about 70% of the screen height (76% at
the centre)3 . For both game types, resets and out-of-range
data were logged automatically and the participants were
given a post-questionnaire of 7 open-ended questions.
Compared with the second testing phase, in this third
test the average reset-error rate across all participants for
the first Half-Life 2 level dropped from 2.68 to 1.46 resets
per minute. The custom game level had an even lower reset-
error rate of just 1.14 per minute, supporting the idea that
the problem areas of the system are the vertical extremi-
ties. The open-ended questions showed that these lower er-
ror rates were not perceived as detrimental to the enjoyment
of the game. All users provided positive feedback, with one
particular user stating that “the aiming felt natural, with
the Wiimote just an extension of my arm”.
4. DISCUSSION
While the test and refinement process described above was
somewhat informal, it served well to help refine our system
to a level where user felt comfortable using the system. Han-
dling the vertical extremities of the screens was identified as
an important issue. However, the decrease in the error rate
between test phases showed that appropriate design of the
aiming logic and game content could reduce its impact. Of
course, these problems would be less apparent if the system
were to be used in a set-up with a smaller vertical range,
such as a multi-screen desktop display.
Future improvements to the code could also help to reduce
the reset error rate. The accelerated vertical aiming could
be incorporated at a driver level, providing a system wide
improvement. Likewise, the relative vertical separation of
IR sources could be used to help the driver recognise each
sensor bar. In the case of our two-walled setup, the middle-
sensor bar is placed furthest away from the user, making the
separation of its IR sources appear relatively closer than any
of the others. This could assist the driver in distinguishing
this sensor bar, which in turn would reduce the likelihood of
an incorrect assumption.
5. CONCLUSION
In this paper we have described the development of a MSB
system which allows the Nintendo Wiimote to be used as
a pointer and controller device with large-screens and sur-
round displays. A series of user-driven refinements have
shown the system to successfully support the demanding
needs of a first-person-shooter game in a 180◦ , two-screen
VR theatre. The system is an affordable and accessible op-
tion for people requiring a wide-angle pointer and controller,
and could be of benefit for use with low-budget VR theatres
or multi-screen systems, such as desktop set ups or museum
exhibits.
To encourage the use and development of this system, we
have provided more information including videos, executa-
bles and source code online [10].
6. ACKNOWLEDGEMENTS
Thanks to Ben Swift for providing helpful suggestions in-
cluding measurements of the aiming deviations.
3
The system was also found to have a maximum error of 5◦ ,
similar to that of the Wii console in a typical setup.
44
Interaction Techniques
7. REFERENCES
[1] C. M. Cheng, M. F. Chung, M. Y. Yu, M. Ouhyoung,
H. H. Chu, and Y. Y. Chuang. Chromirror: a
real-time interactive mirror for chromatic and
color-harmonic dressing. In CHI ’08: CHI ’08 extended
abstracts on Human factors in computing systems,
pages 2787–2792, New York, NY, USA, 2008. ACM.
[2] L. Gallo, G. D. Pietro, and I. Marra. 3d interaction
with volumetric medical data: experiencing the
wiimote. In Ambi-Sys ’08: Proceedings of the 1st
international conference on Ambient media and
systems, pages 1–6, ICST, Brussels, Belgium, 2008.
ICST.
[3] J. Good, P. Romero, B. du Boulay, H. Reid,
K. Howland, and J. Robertson. An embodied interface
for teaching computational thinking. In IUI ’08:
Proceedings of the 13th international conference on
Intelligent user interfaces, pages 333–336, New York,
NY, USA, 2008. ACM.
[4] J. Grubert, M. Hanckock, S. Carpendale, E. Tse, and
T. Isenberg. Interacting with stroke-based rendering
on a wall display. Rep. 2007-882-34, Dept. of Comp.
Science, Univ. of Calgary, 2007.
[5] Nintendo. Wii at nintendo.
http://www.nintendo.com/wii, accessed May 2009.
[6] B. Peek. Managed library for nintendo’s wiimote.
http://www.codeplex.com/WiimoteLib, accessed May
2009.
[7] S. J. Russell and P. Norvig. Artificial Intelligence: A
Modern Approach. Pearson Education, 2003.
[8] T. Schou and H. J. Gardner. A wii remote, a game
engine, five sensor bars and a virtual reality theatre.
In OZCHI ’07: Proceedings of the 2007 conference of
the computer-human interaction special interest group
(CHISIG) of Australia on Computer-human
interaction: design: activities, artifacts and
environments, pages 231–234, New York, NY, USA,
2007. ACM.
[9] T. Schou and H. J. Gardner. A surround interface
using the wii controller with multiple sensor bars. In
VRST ’08: Proceedings of the 2008 ACM symposium
on Virtual reality software and technology, pages
287–288, New York, NY, USA, 2008. ACM.
[10] T. Sko. Wii-ing in vr. http://torbensko.com/wii,
accessed May 2009.
[11] Valve. Half-life 2.
http://orange.half-life2.com/hl2.html, accessed May
2009.
[12] Valve. Valve. http://www.valvesoftware.com, accessed
May 2009.
[13] B. G. Witmer and M. J. Singer. Measuring presence in
virtual environments - a presence questionnaire.
Presence: Teleoperators and Virtual Environments,
7(3):225–240, 1998.