Wii Musical Interfaces - Max Leonard

Wii Nusical Inteifaces
4
th
Yeai Pioject Repoit

Nax Leonaiu
0niveisity of Yoik
2uu9
Nax Leonaiu Wii Nusical Inteifaces 2uu9
Nax Leonaiu Wii Nusical Inteifaces
2
!"#$%&'(# *%++,-.
The uevelopment of two novel softwaie baseu electionic musical instiuments is
piesenteu. Both of these instiuments use the populai Nintenuo Wiimote motion
sensing viueo game contiollei as theii input contiol souice which can be paiieu to
a peisonal computei via the Bluetooth piotocol.
To gain a moie effective unueistanuing of the implications of woiking with the
Nintenuo Wii Remotes, a simplei instiument was initially uesigneu anu built. 0sei
tiials weie conuucteu on this instiument to gauge how people iesponueu to
playing music with the Wii Remotes, these iesults being incoipoiateu into the
uesign of the seconu, moie complex instiument.
In conjunction with the usei tiials, analysis of existing musical instiuments was
unueitaken in an attempt to uesign a moie immeisive, enjoyable anu expiessive
seconu instiument baseu on the moues of inteiaction anu feeuback we finu in
acoustic musical instiuments. Bata fiom this analysis was useu to uevelop an
expiessive mapping system between the Wii Remotes anu a custom sounu engine
built in Native Instiuments Reaktoi which useu physical mouelling, gianulai
synthesis anu auuitive synthesis in conjunction with effects to achieve an output.
0sei tiials weie then caiiieu out on the seconu instiument, feeuback showing
that the uesign was inueeu immeisive anu engaging. Following this, a specialist
application containing both instiuments was uevelopeu foi use as an exhibit on
univeisity open uays.
The pioject has gieat scope foi expansion. Suggestions foi fuithei woik aie
suggesteu anu outlineu.

S

/$01234#56#+#1&7
I woulu like to peisonally thank both my supeivisois, Bi. Bave Peaice anu Bi.
Anuy Bunt foi theii constant suppoit anu bounuless enthusiasm thioughout the
couise of this pioject.
I woulu also like to give thanks to the community of people who have taken the
time to iesponu to my queiies. In paiticulai to Nike veiuone (cieatoi of
WiiToNiui) foi giving auvice on wiiting softwaie foi the wiimote in the months
pieceuing this pioject anu the Native Instiuments usei foium foi theii help on
uebugging my Reaktoi patches.
Keep up the goou woik.
Nax Leonaiu
}une 2uu9
4
Table of Contents
1 Intiouuction................................................................................................................................1u
898 :-2;#$& <=;#$&'(#799999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 8>
2 The Wiimote................................................................................................................................1S
>98 ?1&-25%$&'21 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 8@
>9> A,-53,-# *B#$'C'$,&'21 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 8D
2.2.1 The Wii Contiol System......................................................................................................... 16
2.2.2 Communication.........................................................................................................................16
2.2.S Buttons ......................................................................................................................................... 17
2.2.4 Notion Sensing..........................................................................................................................18
2.2.S Infiaieu Cameia .......................................................................................................................18
2.2.6 Speakei......................................................................................................................................... 19
2.2.7 Foice Feeuback.........................................................................................................................19
>9@ EF# G'' H2&'21 :4%79999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 8I
S Pioject Backgiounu anu Initial Woik...............................................................................21
@98 JF227'16 , K'-#$&'21999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 >8
S.1.1 SB Sounu Placement InstiumentBevice...................................................................... 21
S.1.2 Conuucting uevice....................................................................................................................2S
S.1.S Bisciete Instiument................................................................................................................ 24
@9> JF227'16 , :4,&C2-+9999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 >L
S.2.1 Choosing a Sounu Engine ..................................................................................................... 26
S.2.1.1 Cycling '74's MAx/MS.................................................................................................................. 26
S.2.1.2 Native Instiument's Reaktoi S................................................................................................... 27
S.2.1.S Applieu Acoustics' Stiing Stuuio............................................................................................... 28
S.2.1.4 Stanfoiu 0niveisity's Synthesis Toolkit ................................................................................ 28
S.2.2 Physical Nouelling................................................................................................................... 28
S.2.2.1 Kaiplus-Stiong Algoiithm........................................................................................................... 29
S.2.2.2 Bigital Waveguiue........................................................................................................................... Su
S.2.S Summaiy of Softwaie............................................................................................................. S2
@9@ A%+,1 J2+B%&#- ?1&#-C,$#7 ,15 H,BB'16999999999999999999999999999999999999999999999999999999999999 @@
S.S.1 The Bomination of the Keyboaiu Inteiface.................................................................. SS
S.S.1.1 Towaius New Nethous of Nusical Contiol.......................................................................... S6
S.S.2 uestuial Inteifaces foi Live Peifoimance ..................................................................... S8
S.S.2.1 The Wiimote foi Electionic Nusic Contiol .......................................................................... S8
S.S.2.2 0thei uestuial Nusical Contiolleis......................................................................................... 4u
S
S.S.S Cognitive Lock-in .....................................................................................................................41
S.S.4 Napping ....................................................................................................................................... 4S
@9L *%++,-. 2C :-2;#$& M,$06-2%159999999999999999999999999999999999999999999999999999999999999999999999999999999 LN
@9D O!"B-#77'(#1#77P 99999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 LN
4 Pioject Nethouology anu Specification...........................................................................49
L98 *%++,-. 2C /'+7999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 LI
L9> :-2;#$& H,1,6#+#1&999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 DQ
4.2.1 Nanagement Nethouology .................................................................................................. S1
S The Spheie...................................................................................................................................SS
D98 :-26-,++'16 ,15 2B#-,&'21 99999999999999999999999999999999999999999999999999999999999999999999999999999999999999 DL
S.1.1 Connection.................................................................................................................................. SS
S.1.2 Note-on Betection....................................................................................................................SS
S.1.2.1 Investigating upwaiu integiation............................................................................................ S6
S.1.2.2 velocity Betection Technique.................................................................................................... S7
S.1.S Pitch Betection..........................................................................................................................61
S.1.S.1 Pitch Cuives....................................................................................................................................... 6S
S.1.S.2 Refining pitch uetection............................................................................................................... 6S
S.1.S.S Noie pitch cuives ........................................................................................................................... 67
S.1.4 The Resultant System anu u0I ...........................................................................................68
S.1.4.1 Softwaie Stiuctuie ......................................................................................................................... 7u
D9> *2%15 !16'1#99999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 N8
S.2.1 Stiing Stuuio...............................................................................................................................71
S.2.2 Physical Nouel in Reaktoi.................................................................................................... 72
S.2.S Tiauitional Synthesisei Sounus.........................................................................................72
D9@ :4,.'16 E#$F1'R%#9999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 N@
D9L S'+'&,&'217999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 N@
D9D T7#- U##5=,$0 ,15 V#('7'217 &2 K#7'61999999999999999999999999999999999999999999999999999999999999999999 NL
S.S.1 Questionnaiie ............................................................................................................................7S
S.S.2 Analysis anu Biscussion of Spheie 0sei Feeuback....................................................82
S.S.2.1 Biiect comments ............................................................................................................................. 82
S.S.2.2 Analysis of othei comments anu uata.................................................................................... 82
D9W <14'1# U##5=,$099999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 XL
D9N !"B,17'21 U24423'16 T7#- E-',479999999999999999999999999999999999999999999999999999999999999999999999999999999 XL
S.7.1 0thei Possibilities Foi Expansion of the Spheie........................................................8S
D9X *%++,-. 21 S#77217 S#,-1#5 C2- EF# *BF#-# 99999999999999999999999999999999999999999999999999999 XD
6 The Cello .......................................................................................................................................87
6
W98 J21$#B&'21 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 XN
6.1.1 Sounu Souice anu Synthesis ...............................................................................................89
W9> ?1&#-B-#&,&'21 2C !1#-6. ?1B%&99999999999999999999999999999999999999999999999999999999999999999999999999999999999 IQ
W9@ :'&$F K#&#$&'21 9999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 I>
6.S.1.1 Note Range......................................................................................................................................... 94
6.S.1.2 Refining Pitch Betection .............................................................................................................. 9S
6.S.2 }oystick Baimony Pitch......................................................................................................... 9S
W9L ?17&-%+#1& /1,4.7'79999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 IX
W9D *2%15 !16'1#99999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 II
6.S.1 Physical Nouel ..........................................................................................................................99
6.S.1.1 Besign of Boweu Stiing Physical Nouel..............................................................................1uu
6.S.1.2 Implementation in Reaktoi ......................................................................................................1u2
6.S.2 Auuitive Synthesisei............................................................................................................ 1u4
6.S.2.1 violoncello anu Flute...................................................................................................................1uS
6.S.S uianulai Synthesisei........................................................................................................... 1u9
6.S.4 Effects......................................................................................................................................... 11u
6.S.4.1 Noiphing Filtei..............................................................................................................................11u
6.S.4.2 Satuiation.........................................................................................................................................112
6.S.4.S Reveib................................................................................................................................................112
W9W Y'7%,4 ,15 :F.7'$,4 U##5=,$09999999999999999999999999999999999999999999999999999999999999999999999999999999999988>
W9N *&-%$&%-# 2C &F# J#44299999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999988@
6.7.1 C# Softwaie Summaiy........................................................................................................ 114
6.7.2 Reaktoi Stiuctuie ................................................................................................................. 11S
W9X T7#- U##5=,$0999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999988W
6.8.1 Questionnaiie ......................................................................................................................... 117
6.8.2 Analysis anu Biscussion of Cello 0sei Feeuback..................................................... 126
6.8.S 0thei Feeuback...................................................................................................................... 127
6.8.4 Compaiison with the Spheie ........................................................................................... 129
W9I K'7$%77'21 2C U'15'167 C-2+ EF# J#44299999999999999999999999999999999999999999999999999999999999999999998@8
W98Q :277'='4'&'#7 C2- !"B,17'21 2C &F# J#4429999999999999999999999999999999999999999999999999999999999999998@>
7 Beveloping An Integiateu System.................................................................................. 1SS
N98 E#7&'16 &F'7 7.7&#+99999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999998@D
8 Pioject Nanagement Review............................................................................................ 1S8
9 Conclusions anu Fuithei Woik........................................................................................ 14u
I98 J21$4%7'2179999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999998LQ
I9> U%-&F#- G2-0 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999998L8
7
9.2.1 Wii Notion Plus...................................................................................................................... 141
9.2.2 Euucation.................................................................................................................................. 142
9.2.S Auuio foi viueo uaming Applications.......................................................................... 142
9.2.4 Contiol of visual Elements................................................................................................ 14S
9.2.S Customisability ...................................................................................................................... 14S
1u Bibliogiaphy............................................................................................................................. 14S
Appenuix A Wiimote Connection anu BIB Banuling.......................................................1Su
Appenuix B 0sei guiue to the Integiateu Instiument.....................................................1S2
Appenuix C Notes on 0nueistanuing Reaktoi Layouts ..................................................1S6
Appenuix B Instiument Analysis Bata...................................................................................1S8
Appenuix E NIBI .............................................................................................................................162
Appenuix F Wiimote Bata Sheet...............................................................................................16S
Appenuix u ABXLSSu Batasheet..............................................................................................166
Appenuix B Softwaie Class Tables..........................................................................................167
B1: The Spheie Class lists......................................................................................................167
B2: The Cello Class Lists ........................................................................................................17u
Appenuix I Souice Coue Listing................................................................................................17S

8

" #$%& '( )$*+,-%
Figuie 1 Nintenuo Wii console anu wiimote contiollei (fiom Nintenuo piess ielease) ...................... 1S
Figuie 2 The Wiimote anu Wiimote with accompanying nunchuck attachment (fiom Nintenuo
piess ielease)............................................................................................................................................................. 16
Figuie S wiimote Acceleiometei Layout ................................................................................................................... 18
Figuie 4 Wii Notion Plus Accessoiy (fiom Nintenuo piess ielease)............................................................. 19
Figuie S Example of NAXNSP - Fiom http:miuiguy.comzcaitimagesCycling74ss1.jpg....... 26
Figuie 6 Example of Reaktoi S with stiuctuie anu u0I winuows showing ............................................... 27
Figuie 7 The Kaiplus-Stiong algoiithm..................................................................................................................... 29
Figuie 8 A BWu mouel with iueal teiminations..................................................................................................... S1
Figuie 9 BWu scatteiing junction iepiesenting a change in wave impeuance R2 to R1........................ S1
Figuie 1u The familiai musical keyboaiu ................................................................................................................. S4
Figuie 11 Bammonu BS oigan - fiom http:oiganseivicecompany.com................................................. SS
Figuie 12 Leon Theiemin with his Theiemin instiument - Fiom http:emfinstitute.emf.oig....... S6
Figuie 1S Peison playing an 0nues Naitenot - Fiom http:wwwS.ouk.euu.tw................................... S7
Figuie 14 The Yamaha vL1.............................................................................................................................................. S7
Figuie 1S Rolanu B-Beam, a system using infia-ieu iauiation to uetect the pioximity of the hanu,
manipulating as aspect of the output- fiom Rolanu.com........................................................................ 44
Figuie 16 Ribbon contiollei on a Koig Electiibe uium machine - fiom Sounu0nSounu.com........... 4S
Figuie 17 Initial time plan................................................................................................................................................ Su
Figuie 18 Plan foi 'The Spheie' ..................................................................................................................................... SS
Figuie 19 Layout of a tenoi steel pan - fiom http:hotpans.sepan......................................................... S4
Figuie 2u Acceleiometei uata fiom S stiikes with the wiimote .x:ieu y:gieen z:blue .......................... S6
Figuie 21 Illustiation of a typical uownwaiu movement................................................................................... S7
Figuie 22 Peak uetection - Fiist peak makes the fiist hoop but fails to make the next level by the
next sample. The seconu cuive is quick enough to meet all foui hoops anu thus a note-on is
guaianteeu .................................................................................................................................................................. S8
Figuie 2S Lineai velocity cuive ..................................................................................................................................... S9
Figuie 24 S velocity -Cuive.............................................................................................................................................. S9
Figuie 2S Squaieu velocity Cuive ................................................................................................................................ 6u
Figuie 26 Illustiation of pioblem faceu when tiacking oiientation with 1 acceleiometei................. 62
Figuie 27 Pitch layout of the 'Spheie' ......................................................................................................................... 62
Figuie 28 Fiist two pitch uistiibutions ...................................................................................................................... 6S
Figuie 29 Wiimote pitcheu at 4S uegiees, iolleu to 4S uegiees...................................................................... 64
Figuie Su uiaph of calibiateu x value with iespect to y ..................................................................................... 6S
Figuie S1 2nu oiuei filtei with a bieak fiequency of 1uBz filteiing the x acceleiometei .................. 66
Figuie S2 2nu oiuei filtei with a bieak fiequency of SBz filteiing the x acceleiometei ..................... 66
Figuie SS 2nu oiuei filtei with a bieak fiequency of SBz filteiing the x acceleiometei ..................... 67
Figuie S4 2nu oiuei filtei with a bieak fiequency of 1.S Bz filteiing the x acceleiometei ................ 67
Figuie SS S moie pitch cuives........................................................................................................................................ 68
Figuie S6 The Spheie u0I ................................................................................................................................................ 68
Figuie S7 The class stiuctuie of the Spheie............................................................................................................. 7u
Figuie S8 Example uata flow in a typical stait up anu note stiike pioceuuie .......................................... 71
Figuie S9 Simple Kaiplus-Stiong Reaktoi patch ................................................................................................... 72
Figuie 4u 'Teaiuiop' intio by Nassive Attack......................................................................................................... 76
Figuie 41 Netiics fiom Spheie usei feeuback ........................................................................................................ 81
Figuie 42 Illustiation of two tiei spheie ................................................................................................................... 8S
Figuie 4S Cello acceleiometei outputs - wiimote image fiom www.stuaitnoiman.com, nunchuck
fiom Nintenuo piess ielease............................................................................................................................... 9u
9
Figuie 44 0utput of a faileu banupass filtei expeiiment attempting to smooth acceleiation........... 91
Figuie 4S Latch anu holu system in action with the slowest iesistivity setting to illustiate the
system........................................................................................................................................................................... 92
Figuie 46 Illustiation of note-on anu pitch benu function ................................................................................ 94
Figuie 47 Nunchuck haimony pitch allocation....................................................................................................... 96
Figuie 48 Boweu stiing physical mouel topology - Fiom http:ccima.stanfoiu.euu~jos..........1u1
Figuie 49 Simplifieu ieflection coefficient tiansfei function - Fiom
http:ccima.stanfoiu.euu~jospaspBow_Stiing_Scatteiing_}unction.html ..........................1u2
Figuie Su Bow foi physical mouel in Reaktoi .......................................................................................................1uS
Figuie S1 Stiing iesonatoi feeuback loop, feu by bow......................................................................................1uS
Figuie S2 Boweu stiing scatteiing junction Reaktoi mouule ........................................................................1uS
Figuie SS Boweu stiing physical mouel Reaktoi contiol panel ....................................................................1u4
Figuie S4 violoncello: Baimonic content with incieasing piessuie fiom t=u to t=Su........................1uS
Figuie SS violoncello. 2B view of haimonics. Bow piessuie is incieaseu fiom t=u to t=Su. ...........1u6
Figuie S6 The 1stSiu, 2nu anu 4th haimonic lines ..........................................................................................1u6
Figuie S7 Flute, incieasing bieath powei ...............................................................................................................1u7
Figuie S8 Reaktoi auuitive synthesis block uiiven by the 7 haimonic amplituue inputs.................1u8
Figuie S9 uianulai synthesisei mouule stiuctuie..............................................................................................1u9
Figuie 6u Response of moiphing filtei with oiientation of the wiimote...................................................11u
Figuie 61 Noiphing Filtei Nouule .............................................................................................................................111
Figuie 62 Insiue the moiphing filtei .........................................................................................................................112
Figuie 6S visual Feeuback.............................................................................................................................................112
Figuie 64 Summaiy of the Cello C# stiuctuie.......................................................................................................114
Figuie 6S Piimaiy sounu engine stiuctuie ............................................................................................................11S
Figuie 66 Reaktoi sounu engine piimaiy stiuctuie...........................................................................................11S
Figuie 67 'Bappy Biithuay To You' Wainei-Chappell Nusic ....................................................................118
Figuie 68 Results of multiple choice questions foi the Cello..........................................................................121
Figuie 69 Compaiison of the Spheie anu Cello metiics....................................................................................1Su
Figuie 7u Class hieiaichy of inuiviuual Spheie anu Cello applications .....................................................1SS
Figuie 71 Bieiaichy of the integiateu system.......................................................................................................1S4
Figuie 72 Bouble bass, incieasing bow piessuie ................................................................................................1S8
Figuie 7S Bouble bass, haimonics shown siue on...............................................................................................1S9
Figuie 74 Buman voice, incieasing volume ...........................................................................................................1S9
Figuie 7S Baimonics of human voice .......................................................................................................................16u
Figuie 76 Baimonic content of human voice with incieasing volume.......................................................161
Figuie 77 violin, incieasing bow piessuie .............................................................................................................161

1u

. "/&,'0+1&$'/
This uocument is a iepoit summaiising final yeai pioject woik caiiieu out
between }anuaiy anu }une 2uu9 as pait of a masteis uegiee in electionic
engineeiing at the 0niveisity of Yoik, 0K. The woik ievolves aiounu the use of
the Nintenuo Wii Remote motion sensing viueo game contiollei to contiol
electionic musical instiuments in an attempt to tap into moie expiessive moues
of contiol than those available fiom existing synthesisei contiol platfoims.
Bespite the spectaculai auvances in synthesisei technology ovei the last centuiy,
the methous of contiolling these instiuments ovei this time peiiou have iemaineu
effectively consistent. This is not to say theie have not been woithy attempts to
change the paiauigm, but the vast majoiity of these expensive esoteiic offeiings
have uone little to uent the ieign of the familiai 12 note pei octave musical
keyboaiu.
Low cost anu availability have maue the Wii Remote a populai choice foi many
electionic musicians seeking gestuial alteinatives to keyboaius anu laptops foi
live expiessive sounu contiol, but these applications iaiely go beyonu basic one-
to-one contiol mappings to effects oi synthesiseis uesigneu to be playeu with
keyboaius oi sequenceis. While the spectacle of a musician using the Wii
Remotes makes foi moie compelling viewing fiom the point of view of a ciowu, it
is the aim of this pioject to investigate if a moie consiueieu stuuy into the
mechanics of opeiation between both the sounu engine anu input haiuwaie
woulu iesult in a iichei expeiience foi both peifoimei anu auuience.
In auuition to the instiuments uevelopeu uuiing the couise of this pioject, a
plethoia of options foi fuithei woik anu investigation fiom euucation to viueo
gaming piesent themselves.
In section 1, the pioject is intiouuceu. Section 2 uesciibes the wiimote, its
haiuwaie make up anu backgiounu. In section S, pioject backgiounu is uiscusseu,
incluuing choices foi pioposeu pioject uiiection anu backgiounu to this choice.
11
Section 4 summaiises the pieceuing S sections anu expanus on the objectives
pioposeu in section 1.
Section S uiscusses woik caiiieu out in ueveloping the 'Spheie', a wiimote
contiolleu tuneu peicussion type instiument. Besign, implementation anu usei
testing is uesciibeu, as well as lessons leaineu foi the expeiience.
Section 6 uiscusses woik caiiieu out in uesigning anu implementing the 'Cello', a
wiimote contiolleu continuous pitch instiument. Besign implementation anu usei
testing is uesciibeu, along with conclusions.
Section 7 uesciibeu an integiateu anu simplifieu SpheieCello application foi use
at univeisity open uays.
Section 8 uetails a pioject management ieview, anu section 9 concluues the
iepoit anu suggests fuithei woik.
The iepoit is also accompanieu by a CB-R0N which is attacheu the insiue of the
back covei. This CB-R0N contains uigital content mentioneu in anu ielateu to this
iepoit. An inuex of this mateiial can be founu by opening the inuex.html file on
the CB-R0N in a web biowsei.
12

.2. 3,'4-1& 564-1&$7-%
The ueliveiables anu uesiiables as they weie uefineu in the initial iepoit aie
piesenteu.

K#4'(#-,=4#7
1. Softwaie conveiting Bluetooth uata stieam fiom wiimotes to NIBI with a
fun, intuitive mapping scheme backeu up with suppoit uocumentation anu
iationale baseu on ieseaich
2. An investigation into the issue of electionic instiument contiol mapping

K#7'-,=4#7
1. To have completeu two sepaiate instiuments:
o 'The Spheie' - A wiieless wiimote contiolleu tuneu peicussion type
instiument that will seive as a piecuisoi to the 'Cello', aiuing in the
uesign of this seconu instiument
o 'The Cello' - a wiieless wiimote contiolleu viitual stiing type
instiument built with the lessons leaineu fiom the 'Spheie' in minu
2. Custom sounu geneiation softwaie built with the 'Cello' in minu
S. Napping system baseu on ieseaich into the way establisheu acoustic
musical instiuments aie contiolleu.
These goals aie expanueu upon in 4.1.
1S
8 9:- ;$$<'&-
Notion sensing contiolleis have long been an inteiest of expeiimental electionic
musicians, but theii high cost anu step technical leaining cuive have histoiically
kept them fiimly out of ieach of the masses, much like the eaily expensive
mouulai synthesiseis in the Sus anu 6us. In the last S yeais oi so, the playing fielu
has changeu somewhat. A 2u uevice not even initially intenueu foi intiicate
sounu contiol has been auopteu by musicians anu composeis foi its unique
motion sensing capabilities, which a few yeais ago woulu have cost hunuieus, if
not thousanus of pounus. The humble Nintenuo 'Wii Remote' oi 'wiimote' as it is
commonly known is one of the most accessible oppoitunities in iecent times foi
the electionic musician to seiiously examine the bounuaiies of theii live
peifoimance without the enoimous costs pieviously associateu with this
technology.

U'6%-# 8 Z'1&#152 G'' $21724# ,15 3''+2&# $21&-244#- [C-2+ Z'1&#152 B-#77 -#4#,7#\
82. "/&,'0+1&$'/
The Nintenuo Wii console causeu a gieat ueal of excitement in the gaming
community when it was launcheu in 2uu6. The stii it causeu was not uue to the
console itself (compaieu to its competitois - Niciosoft's XB0XS6u anu Sony's
Playstation S, it was heavily unuei specifieu) but uue to its novel contiol inteiface.
The Wii Remote oi 'wiimote' as it has become known, bioke away fiom the
tiauitional joypau paiauigm that hau almost exclusively uominateu computei
games contiol since the biith of the games console. What maue the wiimote anu
14
its accompanying 'nunchuck' contiollei uiffeient was that its piimaiy sensing was
uone by a small S-axis acceleiometei chip anu infia-ieu sensoi embeuueu in the
iemote. Foi the fiist time, a gamei playing a tennis game was iequiieu to actually
hit the ball back, anu someone fiiing an aiiow fiom a bow hau to actually aim
iathei than tap a couple of buttons oi move a joystick. This ueepei playei
involvement combineu with the immeuiately accessible natuie of the wiimote
caught the imagination of a wiuei uemogiaphic than that noimally associateu
with viueo gaming. Nintenuo pioveu to the 'petiol heaueu' gaming inuustiy that
"Tbe key .. is not wbot you ore ployinq, but bow you'll be ployinq. [Tbe Wii] is tbe
console wbere tbe biq iJeos con prevoil over biq buJqets." - Sotoru lwoto {CF0
NintenJo Corp)
with the wii going on to sell almost as many units as both of its iivals combineu
(Nintenuo 2uu8) (uamezine 2uu9).
1

Almost as soon as this contiollei was maue available to the public at laige,
attempts began to be maue to ieveise engineei it. The wiimote connects to the
Wii console via the Bluetooth wiieless piotocol anu is foi the most pait Buman
Inteiface Bevice (BIB) piotocol compliant (Wiilii.oig 2uu8). This meant that the
haiu pait of getting the uevice to paii with a peisonal computei was basically
uone foi them by Nintenuo, making the full ieveise engineeiing of the iemote
ielatively simple task.
It wasn't long befoie APIs to inteiface with the wiimote began suifacing on the
Inteinet followeu by a gieat numbei of people posting viueos of themselves using
the wiimote foi vaiious non-wii gaming ielateu tasks on viueo shaiing sites such
as Youtube, with eveiything fiom inteiactive white boaius (Lee 2uu9) to iemote

1
At the time of wiiting (}une 2uu9), both Sony anu Niciosoft have announceu new motion
sensing game contiolleis in a ieaction to the massive success of the wiimote (BBC 2uu9). If the
means to inteiact with these uevices become available outsiue of the viueo game uevelopeis, we
coulu see a whole new wave of new anu exciting motion inteifaces.

1S
contiol vacuum cleaneis (hupei2k 2uu9). The ueepei involvement anu tactile
contiol offeieu to viueo gameis by the wiimote weie obvious alluies foi
electionic musicians wanting new expiessive methous of contiolling theii
synthesiseis, anu gieatei fieeuom in peifoimance away fiom theii laptop scieens
anu stationaiy keyboaius. The instantly iecognisable foim of the wiimote to
auuiences also aiueu its iise as a seiious alteinative musical contiollei, being
coveieu by a numbei of high piofile music magazines ( (Sounu 0n Sounu 2uu8)
anu (Electionic Nusician 2uu9) among otheis).
What we aie seeing with the wiimote is a continuation of the tienu in populai
music wheieby new moues of musical expiession aie founu in items anu uevices
in ways incongiuent with the oiiginal intentions of the uesignei. The uistoiteu
electiic guitai anu iock music, the tuintable anu hip-hip anu the Rolanu TR-8u8
anu uance music aie excellent examples of this.
Bowevei, apait fiom a select few piogiammeis, people tenu to be confining the
wiimotes functions to simple manipulation of electionic instiuments, such as one
acceleiometei being uiiectly connecteu to a filtei oi low fiequency oscillatoi
(LF0) anu buttons being uiiectly connecteu to notes. By limiting these contiol
systems to these simple one to one mappings, many of these systems faileu to
iealise the tiue capabilities of what maue the wiimote so exciting in the fiist place.
828 =>,0?>,- @A-1$($1>&$'/
The wiimote anu nunchuck haiuwaie aie uiscusseu. It shoulu be noteu that as the
wiimote anu nunchuck aie piopiietaiy closeu systems, the vast majoiity of this
infoimation has been taken fiom the wiimote hacking community's best effoits to
ieveise engineei the uevice. While theie is the chance that the infoimation may
not be 1uu% coiiect (as no official uata sheets oi constiuction bluepiints can be
obtaineu), it can be consiueieu to be, at woise, an veiy goou appioximation to the
inteinal haiuwaie anu opeiation of the wii contiol system. The full anu in uepth
wiibiew.oig haiuwaie specification foi the wiimote can be founu on in appenuix F
anu on the accompanying CB.

16

"#"#$ %&' ()) *+,-.+/ 012-'3

U'6%-# > EF# G''+2&# ,15 G''+2&# 3'&F ,$$2+B,1.'16 1%1$F%$0 ,&&,$F+#1& [C-2+ Z'1&#152 B-#77
-#4#,7#\
The most common wii contiol system is that of the wiimote anu its paiieu
'nunchuck' that attaches with a wiie to the base of the wiimote. Both contiolleis
contain iuentical acceleiometei chips, allowing foi two hanueu motion sensing.
Powei anu uata aie sent between the wiimote anu nunchuck via the
inteiconnecting cable. Complete wiieless opeiation can be attaineu by use of a S
iu

paity nunchuck anu wiimote auu-on fiom (Nyko 2uu9). Bespite the extia fieeuom
this uevice offeis, it will not featuie in this paiticulai pioject as the implications
foi inteifacing with this uevice aie not known anu not uocumenteu in the wiuei
wii hacking community.
"#"#" *+334,)56-)+,
The wiimote communicates ovei the Bluetooth piotocol anu is paitially BIB
compliant. This means it can be iecogniseu by anu paiieu with a peisonal
computei. If set to uo so, it iepoits back its status ovei this piotocol at aiounu
1uu iepoits pei seconu oi can be polleu asynchionously.
17
"#"#7 84--+,2
The wiimote itself has 11 onoff buttons (excluuing powei) with a fuithei two on
the nunchuck that can be inteipieteu as Boolean onoff flags. These aie as
follows:

Button Placement
A wiimote: 0n top below 4-way B Pau
2

B wiimote: Tiiggei
+ wiimote: 0n top below A
- wiimote: 0n top below A
Bome wiimote: 0n top between + anu -
1 wiimote: 0n top below speakei
2 wiimote: 0n top below 1
0p wiimote: 0n top - Bpau below powei
Bown wiimote: 0n top - Bpau below powei
Left wiimote: 0n top - Bpau below powei
Right wiimote: 0n top - Bpau below powei
C Nunchuck: Top tiiggei
Z Nunchuck: Lowei tiiggei
Powei Top of wiimote: 0nassignable.
E,=4# 8 G''+2&# =%&&217
The nunchuck also houses an XY joystick, which gives out an 8 bit (u-2SS) signal
foi each axis.

2
Common teim in computei gaming iefeiiing to a 4 way 'Biiection' pau consisting of 0p, Bown,
Left anu Right buttons
18
"#"#9 :+-)+, 0',2),;

U'6%-# @ 3''+2&# /$$#4#-2+#&#- S,.2%&
The main selling point of the wiimote is its Analog Bevices ABXLSSu
S
S axis
acceleiometei chip capable of ieliably uetecting acceleiation up to +- S.Sg with 8
bit piecision.
When stationaiy, each acceleiometei effectively becomes a giavity uiiection
sensoi giving back a value between +- g uepenuing on oiientation. This
backgiounu foice uue to giavity woiks veiy well foi oiientation calibiation, but
makes accuiate motion tiacking uifficult as motion inuuceu by the usei cannot be
uistinguisheu fiom giavity. The fact that giavity only exists in one uiiection also
means that absolute azimuth (oi oiientation in the so calleu 'living ioom' plane)
cannot be uiiectly measuieu, one of the ieasons the infia-ieu cameia is so
impoitant foi the motion analysis in games on the wii console. This is piobably
the ieason why theie haven't been any notable instances of people using the
wiimote foi full motion instiument contiol, piefeiiing to stick to pitch anu ioll foi
theii uata input stieams.
"#"#< =,>.6.'? *63'.6
To obtain moie accuiate positional ieauings without auuing extia acceleiometeis
anu gyioscopes, the wiimote has an 8 bit 128x96 pixel infia-ieu (IR) cameia with

S
See appenuix u foi uata sheet
19
built into the fiont. When two stationaiy IR LEBs aie placeu in fiont of the
wiimote, uetaileu positional infoimation can be ueuuceu by tiacking the two
points, allowing foi accuiate motion tiacking. 0ne of the best uemonstiations of
these capabilities outsiue the Wii gaming woilu is by Ameiican, }onny Lee. With
the IR cameia on the wiimote anu a few LEBs, he has manageu to uo viitual SB
peispective heau tiacking, futuiistic viitual touch scieens anu fully inteiactive
white boaius(Lee 2uu9).
"#"#@ 0A'6B'.
The wiimote has a small low quality speakei built into the hanuset uesigneu foi
basic playei specific sounu effects uuiing game play. The auuio is stieameu to the
wiimote in ieal time but at this stage, the exact opeiation of stieaming uata to the
wiimote is unknown to the ieveise engineeiing community.
"#"#C D+.5' D''?E65B
To pioviue physical feeuback to playeis in games, the wiimote has a vibiation
motoi which can be tuineu on anu off with a Boolean flag.
82B 9:- ;$$ C'&$'/ 3D+%

U'6%-# L G'' H2&'21 :4%7 /$$#772-. [C-2+ Z'1&#152 B-#77 -#4#,7#\
Bue foi ielease in }une 2uu9, the 'Wii Notion Plus' accessoiy will take the foim of
a small lightweight peiipheial connecteu to the base of the wiimote anu will auu a
uual axis gyioscope to the hanuset, enabling tiue one to one motion tiacking. The
uemonstiations of this technology by uevelopei AiLive aie stiikingly impiessive
(AiLive 2uu9). The viueo gaming community is alieauy eageily anticipating swoiu
2u
fights anu ultia iealistic tennis games, but the potential possibilities foi the
contiol of electionic musical uevices contiol aie unpieceuenteu. The ielease anu
inevitable ieveise engineeiing of this uevice shoulu piesent compelling options
foi fuithei woik in the next few yeais.

21
B 3,'4-1& E>1F*,'+/0 >/0 "/$&$>D ;',F
Beie the backgiounu to this pioject is uiscusseu along with some of the iueas that
helpeu shape the pioject uefinition anu methouology.
B2. G:''%$/* > H$,-1&$'/
The wiimote is an extiemely veisatile sensoi, anu as such, many possible
uiiections the pioject coulu have taken weie consiueieu uuiing the initial
uiscussions. The fiist uecision maue was whethei oi not to use the infia-ieu (IR)
cameia. Aftei seeing the impiessive uemonstiations ueviseu by }onny Lee (Lee
2uu9) one temping option was to incoipoiate the infia-ieu cameia into the
pioject. Aftei some uiscussion howevei, it was ueciueu that the cameia woulun't
be useu as it woulu limit the usei to one stationaiy position on a stage as the
wiimote woulu have to be pointing at the stationaiy souice foi the entiie session.
Also, if theie was an auuience piesent, an IR souice woulu have to be eithei in
fiont of the peifoimei on stage oi in the miuule of the auuience, uecieasing the
flexibility of system.
Aftei this uecision was maue, theie weie thiee stiong thieaus that piesenteu
themselves:
7#$#$ 7F 0+4,? G/65'3',- =,2-.43',-HF'I)5'
The fiist thing that came to minu when thinking about spatial contiol was that of
thiee uimensional sounu manipulation. The 0niveisity of Yoik is well known foi
its ieseaich into ambisonics
4
with Bave Nalham in the Nusic Reseaich Centie
(NRC) being a iecogniseu expeit in the fielu. To exploie this fuithei, the authoi
aiiangeu an infoimal meeting with Bave to uiscuss the cuiient state of ait in SB
special sounu manipulation.
0ne of the main issue faceu by uesigneis of SB spatial contiolleis is that of
iepeatability. This was uemonstiateu by Bave in a simple example:

4
A methou of SB sounu iepiouuction - see http:www.ambisonics.net foi moie uetails
22
An upriqbt pencil is belJ out onJ one finqer of one bonJ is ploceJ upon its enJ. Tbe
subject tben closes tbeir eyes, mokes on orbitrory tbree Jimensionol movement, tben
ottempts to move tbeir finqer bock to tbe perceiveJ oriqinol stortinq position. Tbe
subject finJs tbeir finqer to be quite o Jistonce from tbe oriqinol position of tbe pen,
tbus illustrotinq tbe issue of repeotobility.
Without any visual feeuback beyonu the pointing uevice themselves, getting
absolute position becomes veiy uifficult until the usei is sufficiently pioficient at
using the uevice. The best way to achieve this might be to use a suiiounu aiiay of
visual monitois, but the visual piocessing anu technical iesouices iequiieu woulu
stietch beyonu the scope of this pioject.
Bis ieaction to using wiimotes foi this puipose was positive howevei. The
technology useu in this fielu has histoiically been veiy expensive, often
piohibiting people oi uepaitments fiom paiticipation in this aiea of ieseaich. The
veiy low cost of the wiimote woulu seive as an incentive to get othei ieseaicheis
anu commeicial institutions inteiesteu in SB sounu foi both acauemic ieseaich
anu possibilities foi cinema, viueo gaming anu othei types of multimeuia.
0nfoitunately, without the aiu of auuitional gyioscopes oi acceleiometeis, the
wiimote is not up the task of full six uegiees of fieeuom (6B0F) motion sensing
(see 2.2.4) making SB sounu contiol an extiaoiuinaiily uifficult task. The wii
motion plus (see 2.S) will allow foi full 6B0F motion tiacking which woulu make
foi compelling fuithei ieseaich.
The issue of SB contiol anu visualisation hau been investigateu in the NRC by
Bimitiios Bountouiiuis who's useu computei mice, tiauitional computei game
joypaus anu a six uegiees of fieeuom 'SBConnexion's SPACE
NAvIuAT0R'(SBConnexion 2uu9) contiollei to look into the issues of SB sounu
contiol (Bountouiiuis 2uu7). 0sing these uevices in combination with a computei
geneiateu 2B image, he suiveyeu the effectiveness of each contiollei. The
contiollei that seemeu to give the best iesults was one of the computei mice.
This is piobably uown to usei familiaiity of the input uevice which is a weighty
contiibutoi to the success oi failuie in shoit time scale usei inteiface stuuies.
2S
Cassiano Suiek, also fiom the 0niveisity of Yoik investigateu wiieless special
sounu contiol as pait of his NSc thesis (Suiek 2uu8). The Nintenuo wiimote was
in fact consiueieu as a canuiuate foi this woik but was iejecteu uue to the lack of
absolute azimuth uetection. Baving full 6B0F is essential to the successful
opeiation of such a uevice anu as such opteu foi a custom built contiollei.
7#$#" *+,?45-),; ?'I)5'
To mastei conuucting an oichestia, iequiies many houis of piactice. The only
pioblem in this instance is that getting 1uu people in a ioom to piactice witb
uoesn't happen veiy often. I was inteiesteu to see if the technology insiue the
wiimotes coulu be useu foi eithei a uevice to conuuct symphonies of electionic
instiuments as an expeiimental peifoimance uevice in its own iight, oi foi use in
tiaining potential conuuctois in the classical iealm.
By having many sepaiate tiacks playing at once, the 'conuuctoi' coulu steei the
music, much like a tiauitional conuuctoi uoes, but with all the extensions
allowable by using uigital technology. I went to talk to masteis stuuent uavin
Whitwoith, cuiiently stuuying conuucting heie at the 0niveisity of Yoik. 0ne of
the most piominent things to come out of the conveisation was that, othei than
the iight hanu keeping steauy time, the gestuies useu to steei the musicians aie
veiy much paiticulai to the conuuctoi. As well as hanu gestuies, eye contact with
ceitain membeis of the ensemble aie useu to inuicate ceitain peifoimance
paiticulais. As theie is no establisheu coue that conuuctois use to steei theii
musicians, ueveloping a usable inteiface foi tiaining conuuctois in the tiauitional
sense using solely wiimotes woulu be piactically impossible.
This fact has not ueteiieu otheis fiom attempting to tackle this pioblem with
wiimotes. A team leu by Piof. Teiesa Nakia at the College of New }eisey in Ewing
woikeu to piouuce a wiimote contiolleu viitual conuucting inteiface anu has
touieu the uevice thiough the 0niteu States anu Euiope fiom 2uu7 as a siue
exhibit at vaiious oichestia conceits (Nakia. T. N 2uu9). As stateu above, one of
the pioblems with ueveloping a uevice to tiain conuuctois is that theie is a lot
moie to conuucting than just moving the iight hanu in time, anu while this uevice
seiveu as a gieat intiouuctoiy aiu foi those unfamiliai with the basic mechanics
24
of conuucting it uoesn't have the sensitivity anu numbei of input vaiiables
iequiieu foi the intiicate uiiection of an oichestia.
When one looks outsiue the complex iealm of using these motion sensing uevices
foi tiaining classical conuuctois, theie isn't ieally a lot left to exploie in the iealm
of contiolling pie piogiammeu synthesiseis.
7#$#7 F)25.'-' =,2-.43',-
While the SB sounu contiol uevice anu the conuucting inteiface piesent
inteiesting inteiface challenges in themselves, neithei of these highly specialiseu
applications aie paiticulaily accessible to the aveiage electionic musician.
The subject of musical contiol anu the ielationship musicians have to theii
instiuments seemeu to be a fai moie accessible topic to unueitake with much
wiuei appeal. The uesign of uisciete musical instiuments also piesents a gieatei
cieative fieeuom when compaieu to the othei consiueieu uiiections. The SB
sounu placement uevice anu the conuucting inteiface both iequiie some kinu of
manipulation of pie-piogiammeu playback which I was keen to avoiu, as it
naiiows the scope foi peifoimance options.
B28 G:''%$/* > 3D>&(',<
In the summei of 2uu8, the authoi began woik in ueveloping a wiimote NIBI
application in 0bjective-C foi Apple 0S X. By uoing this, I was able to get a goou
giasp of the peculiaiities of the wiimote, anu how to go about inteifacing the
uevice with a peisonal computei. When the time to begin the pioject came aiounu
some months latei, the choice of uevelopment platfoim was ie-evaluateu. Baving
been an Apple usei foi some time, the authoi's instinct was to stick with
0bjective-C anu uevelop foi the Nacintosh. This uecision woulu also have been
congiuent with the tienu of electionic musicians favouiing Apple haiuwaie anu
softwaie ovei othei manufactuies.
The insepaiable integiateu natuie of Apple's haiuwaie anu softwaie piouucts aie
the ieason many choose to auopt it as theii platfoim of choice. As the softwaie is
wiitten specifically foi the haiuwaie, softwaie tenus to be quickei anu moie
stable, a uefinite incentive foi anyone using a computei foi piofessional auuio oi
2S
viueo woik. This integiateu piopiietaiy appioach has the unfoitunate siue effect
of making Apple piouuct inheiently moie expensive, the ieason why they aie not
as wiuely auopteu as Niciosoft's Winuows iunning on the plethoia of S
iu
paity
manufactuies machines we see available touay.
The ubiquity of the Winuows platfoim ceitainly has its stiengths. If softwaie is
wiitten to iun in a winuows enviionment, it expanus the potential usei base
beyonu that which one coulu get by wiiting softwaie exclusively foi Apples 0S X
opeiating system. Theie is also the auueu auvantage that any owneis of Apple
haiuwaie manufactuieu aftei 2uu7 has the ability to natively iun applications
wiitten foi Winuows on Apple haiuwaie, eithei within 0S X itself using S
iu
paity
emulation applications like Paiallels
S
oi CoueWeavei's Ciossovei
6
, oi by having a
uual boot system with an install of Winuows using Apple's Bootcamp softwaie
7
.
It was ueciueu to wiite the softwaie foi Winuows. With this uecision maue, the
pioject now hau the oppoitunity to be useu on any computei which is of
paiticulai ielevance heie at the 0niveisity of Yoik. The immeuiately engaging
natuie of the instiuments uevelopeu at the enu of this pioject woulu seive as
excellent show pieces to piospective stuuents anu fiist yeais, giving an engaging
anu immeuiately accessible example of 'fun' engineeiing. It woulu also allow the
coue to be stuuieu anu ieuseu by anyone iegaiuless of theii haiuwaie piefeience
aftei this paiticulai pioject enus.
The language chosen to wiite the softwaie was Niciosoft's object oiienteu C#.
Niciosoft aie veiy keen to push C# as the main uevelopment language foi
Niciosoft Winuows, anu as such, theie is an extensive uocumentation libiaiy anu
API collection foi it. visual Stuuio Piofessional, available foi fiee foi stuuents on
Niciosoft's Bieamspaik.com pioviues a fast stieamlineu uevelopment
enviionment in which to woik. With the authois piioi expeiience in }ava anu C,

S
http:www.paiallels.comuk
6
http:www.coueweaveis.com
7
It is woith noting that foi this paiticulai application, neithei Ciossovei oi Paiallels have the
ability to inteiface with the Apple Bluetooth haiuwaie, thus necessitating the use of Bootcamp foi
use of the softwaie uevelopeu foi this pioject on an Apple computei.
26
some ieseaich into C# was enough to convince me of its suitability, with C#
shaiing featuies both in both these languages.
7#"#$ *&++2),; 6 0+4,? J,;),'
It was cleai fiom the outset that ueveloping the sounu engine anu the softwaie to
inteiface with the wiimote woulu be two sepaiate tasks. It theiefoie maue sense
to look outsiue the C# language foi othei enviionments in which to uevelop the
sounu engine.
!"#"$"$ Cycllng '74's MAx/MS
NAXNSP has become almost the ue facto stanuaiu foi custom sounu cieation in
the acauemic woilu.

U'6%-# D !",+B4# 2C H/]^H*: _ U-2+ F&&B`^^+'5'6%.9$2+â$,-&^'+,6#7^J.$4'16NL^7789;B6
It is a veisatile giaphical piogiamming enviionment iunning on both Nac 0S anu
Winuows which has the ability to iun exteinal sections of text baseu computei
coue, as well as the ability to compile stanualone veisions foi use on any
computei inuepenuent of NAXNSP. While its veisatility anu peivasive piesence
in acauemia weie stiong enticements foi the auoption of this platfoim foi the
pioject, the authois unfamiliaiity with the system combineu with its laige cost
($47S foi NAXNSP}ittei giaphics euucational bunule) weie enough to sway the
authoi in the uiiection of the Native Instiument's Reaktoi S.
27
!"#"$"# naLlve lnsLrumenL's 8eakLor &
Nuch like NAXNSP, Reaktoi is a mouulai giaphical auuio piogiamming
enviionment foi both Nac 0S anu Winuows.

U'6%-# W !",+B4# 2C V#,0&2- D 3'&F 7&-%$&%-# ,15 bT? 3'15237 7F23'16
It uoesn't have the ability to impoit exteinal text baseu computei coue, oi be
compileu to iun as a stanualone, but has the auvantage of having a veiy fast tiuy
inteiface foi builuing complex auuio applications. It has a vast libiaiy of built in
mouules, fiom fully blown synthesiseis uown to bit-wise opeiations anu aiiays,
allowing the usei to go into as much uepth as they wish to when constiucting
theii application. BSP piogiamming functionality is available with the Reaktoi
Coie technology, allowing foi bit-by-bit manipulation of auuio uata. Complex u0Is
can be cieateu without any auuitional puichases (}ittei must be puichaseu with
NAXNSP to allow foi giaphics in NAX), anu a laige anu ueuicateu usei
community is ieauily available foi help anu auvice if one becomes unstuck. 0ne
auvantage with iespect to this paiticulai pioject is this authois familiaiity with
Reaktoi. Whilst nominally expensive (aiounu 2uu euucational) theie is the
auueu auvantage that the authoi alieauy owns a copy of this softwaie, thus
negating any immeuiate expense. Reaktoi was chosen as the main sounu
synthesis uevelopment enviionment foi this pioject.
28
!"#"$"! Applled AcousLlcs' SLrlng SLudlo
Stiing Stuuio is a physical mouelling stiing synthesisei that can be iun eithei as a
stanualone application oi as a plug-in in a suitable host. Theie aie a numbei of
goou sounuing pie-sets that come packageu with the application, anu the usei has
the ability to change almost eveiy piopeity of the physically mouelleu stiing.
While Stiing Stuuio only costs $1S9 with euucational uiscount (KNC 2uu7) -
maikeuly less than Reaktoi, one pioblem with this application is that it has been
soliuly wiitten fiom the peispective of being playeu with a keyboaiu. 0nce a note
has been tiiggeieu, theie is veiy little that can be uone to change the piopeities of
this note until a note-off commanu is sent anu a new note is tiiggeieu. Its closeu
natuie uoes not allow souice coue to be mouifieu to change this.
The Stiing Stuuio uemo was useu in piototyping the 'Spheie' instiument (see
S.2.1) but was abanuoneu foi uevelopment of the 'Cello' (see 6).
!"#"$"' SLanford unlverslLy's SynLhesls 1oolklL
Foi some of the physical mouelling applications, Stanufoiu's open souice
Synthesis Toolkit (STK) C++ libiaiy was consiueieu foi some of the physical
mouelling synthesis in this pioject. The STK is a set of uynamically linkeu libiaiies
which peifoim a vaiiety of auuio piocessing functions such as effects, oscillatois
anu physical mouels. 0ne of the main complications was getting the non-manageu
C++ libiaiies of the STK to be compatible with the C# compilei which expects fully
manageu libiaiies
8
. Aftei a little ieauing iounu the subject, it was ueciueu that the
physical mouelling woulu be built fiom sciatch in Reaktoi.
7#"#" G&12)56/ :+?'//),;
Physical mouelling is a methou of sounu synthesis conceineu with the
mathematical mouelling of iesonating systems to piouuce wavefoim output. The
equations that make up these mouels can be manipulateu to iecieate existing
physical oi electionic systems (such as vibiating stiings oi analogue ciicuits) oi
completely new systems that woulu not be piactical to builu, (1uu metei long

8
Foi moie infoimation on manageu anu unmanageu libiaiies, see http:msun.miciosoft.comen-
uslibiaiybb677124.aspx
29
guitais oi antiquateu electionic components foi example). The main auvantages
of physical mouelling synthesiseis aie theii vast flexibility. As the output is being
calculateu in ieal time baseu on a set of mathematical ielations, the sounu can be
easily changeu in ieal time by alteiing the vaiiables in those equations.
Physical mouelling synthesis has been aiounu since }ohn Kelly anu Caiol
Lochbaum fiist synthesiseu the human voice at Bell Labs in 196u, but has only
come to piominence in the last twenty yeais oi so because of the compaiatively
laige computational cost of the algoiithms when compaieu to othei types of
sounu geneiation such as sampling oi FN synthesis. It wasn't until the auvent of
the Kaiplus-Stiong algoiithm in 198S (K Kaiplus 198S) that physical mouelling
was biought into the ieal time uomain, anu it took a fuithei eleven yeais until this
technology maue it into a commeicial piouuct in the foim of the Yamaha vL1 (see
S.S.1.1).
}ulius 0. Smith III, the oiiginatoi of the BWu techniques maintains an excellent
online iesouice (Smith, A basic intiouuction to uigital waveguiue synthesis 2uu6)
ielating to many physical mouelling techniques which has been extiemely useful
in compiling this section.
3.2.2.1 karplus-SLrong AlgorlLhm
The Kaiplus-Stiong algoiithm is a iemaikably simple system which piouuces a
suipiisingly goou appioximation to a pluckeu stiing.

U'6%-# N EF# c,-B4%7_*&-216 ,462-'&F+
Su
A noise buist (oi similai bioaubanu signal) is feu into a low pass filtei with a gain
of slightly less than 1. This excitei signal is passeu simultaneously to the output
anu a uelay, which is the feu back into the filtei. The uelay is effectively acting as a
iesonating stiing, with the filtei simulating eneigy loss at the biiuge anu nut anu
the output effectively acting like a pickup, tapping the oscillating 'stiing' at a
single point. The length of the 'sting' anu thus tuning is vaiieu by alteiing the time
of the uelay block with the ielation:
(1)

3.2.2.2 ulglLal Wavegulde
(Infoimation in this chaptei (S.2.2.2) ielayeu anu auapteu fiom http:ccima.stanfoiu.euu~jos (Smith 2uu8)
The Kaiplus-Stiong algoiithm was built upon by }ulius 0. Smith III of Stanfoiu
0niveisity thioughout the 198us anu geneialiseu into the Bigital Waveguiue
(BWu) mouel.
Each uelay line of contains a sample of a tiavelling wave at some point on its
jouiney uown the iesonatoi at some impeuance R. By using a system of these
oppositely tiavelling uelay lines, u'Alembeit's one uimensional wave equation can
be solveu by having each uelay line iepiesent both the iight (
!
y
+
) anu left (
!
y
"
)
tiavelling waves which aie summeu at a common point m at time n to piouuce an
output.

(2)
!
D=
F
s
F
1
!
y(m, n) = y
+
(m" n) + y
"
(m+ n)
S1
In an iueally teiminateu stiing, the wave is ieflecteu (multiplieu by -1) anu sent
back uown the othei uiiection.

U'6%-# X / KGb +25#4 3'&F '5#,4 &#-+'1,&'217
Filteis can be useu to mouel loss which can be lumpeu into one block foi
computational efficiency. To auu auuitional nonlineaiities to inciease sonic
inteiest, scatteiing junctions can be auueu between the two uelay lines which can
contain auuitional filteis anu multiplieis to mouel the vaiying impeuances of
stiings anu membianes.
To emulate changes in wave impeuance (simulations of changing stiing lengths
anu inteiactions with the bouy of the instiument anu the aii), scatteiing junctions
can be useu at inteimeuiate points between uelay elements.

U'6%-# I KGb 7$,&&#-'16 ;%1$&'21 -#B-#7#1&'16 , $F,16# '1 3,(# '+B#5,1$# V> &2 V8
Lowpass
flter
Output Exciter
-1 -1
Z
-N
Delay
Z
-N
Delay
Z
-N
Delay
Z
-N
Delay
Z
-N
Delay
Z
-N
Delay
y
+
y
-
Z
-N
Delay
Z
-N
Delay
Z
-N
Delay
Z
-N
Delay
k
1
-k
1
1+k
1
1-k
1
S2
These impeuance changes can be iepiesenteu with a ieflection coefficient k1,
wheie
!
k
1
=
R
2
" R
1
R
2
+ R
1
(S)
These scatteiing junctions aie useu to simulate the nonlineaiities in ieal musical
instiuments, auuing chaiactei anu ianuom elements to the sounu, making it moie
believable to the human eai. These techniques aie useu in the physical mouel
uesigneu in 6.S.1.
7#"#7 04336.1 +> 0+>-K6.'
Will iun on Niciosoft WinJows XP opeiating system
Wiitten in object oiienteu .NET C# language
Native Instiument's Reoktor to be piimaiy sounu engine
0se Applieu Acoustics' Strinq StuJio uemo foi piototyping softwaie
SS

B2B =+<>/ G'<A+&-, "/&-,(>1-% >/0 C>AA$/*
In the 21
st
centuiy, we aie spenuing incieasing amounts of oui leisuie anu
piouuctive time on computeis. Nusical composition anu peifoimance has been a
gieat beneficiaiy of these auvances in computing powei anu you woulu be haiu
piesseu to finu a mouein music stuuio without some kinu of computei at the
heait of many of it opeiations. The vast majoiity of these systems aie opeiateu
with the familiai keyboaiu anu mouse system which mainly conceins itself with
selecting icons oi items in menus. The "WINP" system (Winuows, Icons, Nenus,
Pointeis) as it is known is one of the coinei stones of computei u0I uesigns, anu
has been since Apple Nacintosh 0S populaiiseu this in the miu eighties. Its
uevelopment embouies the oft quoteu 'ten minute iule', ueviseu by infoimation
technology pioneei Teu Nelson in his 1974 publication Computei Lib Bieam
Nachines which states that:
"Any system tbot connot be well touqbt to o loymon in ten minutes by o tutor in tbe
presence of o responJinq setup, is too complicoteJ". (Nelson 1974)
We uon't have to look veiy fai to see that this piinciple has been auheieu to in the
vast majoiity of computei inteiface systems. Computeis foi the most pait neeu to
be usei fiienuly with an instiuction set that enables a usei to get to giips with it in
a shoit peiiou of time. It is also an auvantage if these instiuction sets have some
common themes fiom system to system, the usei familiaiity meaning leaining
time is uecieaseu foi new systems anu new useis, thus incieasing piouuctivity.
7#7#$ %&' F+3),6-)+, +> -&' L'1E+6.? =,-'.>65'
While this emphasis on ease of use anu usei familiaiity in inteiface uesign seems
to woik foi geneial puipose computei contiol applications, it uoesn't holu too
much piomise foi the tactile contiol of specialiseu applications, least of all
musical instiuments. As an example, the methou of inteiaction with a cello
uoesn't involve a volume oi tone contiol, the input is a multiuimensional input
space manipulateu by the two hanus.
Imagine applying this select-one-item-at-at-time contiol system to a cello. To play
a single note with a little vibiato you woulu have to:
S4
- Select a stiing to bow
- Select a bow piessuie
- Select a bow speeu
- Select a bow to biiuge uistance
- Select a bow angle
- Select a stiing to fiet with the left hanu
- Select a position on the neck to fiet
- Specify vibiato uepth
- Specify vibiato tiavel
Now imagine the way these vaiiables change ovei the couise of just one sinqle
note. Theie aie time vaiying envelopes associateu with all of these input
vaiiables, anu even if we coulu manipulate each paiametei inuepenuently (with a
bank of faueis foi example), any attempt to achieve a coheient musical output
begins to seem like a iathei uaunting task. We can cleaily see though, that by
watching a cellist play that all manoi of expiessive anu complex musical gestuies
can be cieateu with only the movement of the two hanus. This contiol set might
not be able to be leaineu within the ten minutes specifieu by Teu Nelson's iule,
but its expiessive uepth moie than makes up foi its steep leaining cuive.
With this in minu then, it seems stiange at fiist glance that the contiol methou foi
the vast majoiity of mouein synthesiseis with all theii expiessive powei anu
innumeiable continuously contiollable sonic vaiiables is effectively by that of a
iow of uisciete buttons.

U'6%-# 8Q EF# C,+'4',- +%7'$,4 0#.=2,-5

SS
The veiy fiist examples of keyboaiu instiuments can be tiaceu back to the S
iu

centuiy, when oigans weie fiist being uevelopeu. 0vei the next 7uu yeais, the
keyboaiu inteiface uevelopeu to contiol haipsichoius, pianos anu oigans, as well
as instiuments such as maiimba anu xylophones auopting the layout.
With the uiscoveiy of electiicity anu the uevelopment of electionics, the eaily 2u
th

centuiy was an extiemely feitile peiiou foi electionic musical instiuments. The
unueniable influence of the oigan (aiguably the fiist synthesisei) combineu with
the almost untouchable stanuaiu of the piano as the piincipal accompaniment
instiument pietty much set the scene foi the methous of contiol of these new
instiuments. It is also woith noting that the majoiity of these eaily instiuments
weie uesigneu by men who's piimaiy engagement was engineeiing. The concept
of 'one-contiol-foi-one-function' is quite a logical way to uesign a mechanical
system as it ensuies that all youi intenueu featuies aie contiollable by the enu
usei, the keyboaiu was the closest anu most familiai musical contiol suiface to
accomplish this goal.
The huge populaiity of the Bammonu oigan in both chuich anu pop music in the
194us anu onwaius seiveu as a stiong finishing move to ensuie the uominance of
the musical keyboaiu foi the contiol of synthesiseis foi the iest of the 2u
th

centuiy. If we take a look at the synthesisei maiket touay, almost all of the
inteifaces available foi theii contiol will be by that of a keyboaiu, with only a few
exceptions.

U'6%-# 88 A,++215 M@ 2-6,1 _ C-2+ F&&B`^^2-6,17#-('$#$2+B,1.9$2+
S6
3.3.1.1 1owards new MeLhods of Muslcal ConLrol
Bespite this uominance, theie have been a few woithy attempts ovei the yeais
that have offeieu an alteinative inteiface foi the contiol of electionic instiuments.
0ne of the eailiest anu most well known examples of this is the Theiemin.

U'6%-# 8> S#21 EF#-#+'1 3'&F F'7 EF#-#+'1 '17&-%+#1& d U-2+ F&&B`^^#+C'17&'&%&#9#+C92-6
Inventeu by Russian Lon Theiemin in the 192us, the Theiemin was the fiist non-
contact continuous electionic instiument. At face value, it's composition is
iemaikably simple. It is essentially a single oscillatoi with continuous volume anu
pitch contiols, each of these vaiiable manipulateu by moving the hanus aiounu
two antenna. Bespite it's extiemely basic tonal output, a skilleu playei can coax
almost human voice like sounus fiom the instiument, anu seives as a veiy goou
example of how much pait the inteiface plays in enabling expiessive musical
output.
Anothei electionic instiument uevelopeu aiounu this time was the 0nues
Naitenot.
S7

!"#$%& () *&%+,- ./01"-# 0- 2-3&+ 40%5&-,5 6 !%,7 855.9::;;;)<,$=<&3$<5;
At fiist glance, it looks veiy similai to a gieat ueal of othei keyboaiu instiuments
that we aie familiai with, but on closei inspection, we finu something quite
uiffeient. Nauiice Naitenot was a cellist looking to biing the same uegiee of
expiession he founu in his cello to the new anu exciting woilu of electionic
keyboaiu instiuments. Be built a iing attacheu to a iibbon into his new
synthesisei, which coulu be moveu up anu uown the length of the keyboaiu to
give a continuous pitch contiol. In auuition, theie was a seconu continuous push
button contiollei, which was manipulateu with the left hanu to contiol volume,
thus augmenting some of the uynamic playing techniques of the cello on to the
electionic sounu geneiatoi insiue the 0nues Naitenot.
A thiiu instiument woithy of uiscussion is the Yamaha vL1.

!"#$%& (> ?8& @07080 AB( 6 C%,7 855.9::;;;<D"="-C<E,7
In 1994, Yamaha's latest flagship synthesisei, the vL1 was ieleaseu to much fan
faii anu ciitical acclaim. 0tilising the latest physical mouelling technology (see
S.2.2) licenseu fiom Stanfoiu 0niveisity in the 0SA, the sounu engine of this synth
was piaiseu by the musical community foi its incieuibly complex, iealistic
S8
sounus. 0ne of the most inteiesting featuies of this instiument was the way it was
playeu. In auuition to the stanuaiu keyboaiu, the vL1 hau a bieath contiollei
allowing foi a gieatei contiol of uynamic anu tone as the usei keyeu the notes on
the keyboaiu. When playeu by a skilleu playei, the net iesult is an astonishingly
expiessive uynamic musical output. 0nfoitunately foi Yamaha, sales of the vL1
weie pooi. The vL1 was extiemely expensive (RRP S999) anu its esoteiic
inteiface uiun't sit well with the vast majoiity of synthesisei playeis who weie
piimaiily keyboaiuists. To obtain sounus of compaiable quality to the Yamaha
uemos, the use of the bieath contiollei was essential anu this potential leaining
cuive was enough to peisuaue potential buyeis to opt foi the keyboaiu only FN
anu subtiactive synthesiseis, looking fai moie impiessive on papei, having many
moie voices of polyphony compaieu to the vL1s two.
7#7#" M'2-4.6/ =,-'.>65'2 >+. N)I' G'.>+.36,5'
Electionic music by its veiy natuie, isn't uesigneu to be peifoimeu. The sheei
amount of sonic manipulation anu layeis of textuies that go into even the most
minimal of electionica pieces woulu make contiolling eveiy aspect of a tiack by a
human banu almost impossible. It is foi this ieason that the vast majoiity of
electionic music peifoimances involve some kinu of playback with vaiious paits
of the sounu manipulateu by the peifoimei ovei the length of the piece.
0nfoitunately fiom the auuiences point of view, this often means inteiacting with
the oiiginal uevices that cieateu the music in the stuuio. The all too familiai sight
of a peison huncheu ovei a laptop oi small NIBI keyboaiu has become the noim
foi 'live' electionic music peifoimance, with the auuience having no iuea whethei
the peifoimei is executing extiemely complex musical manoeuvies oi just
checking theii emails. With this in minu then, it isn't haiu to see why uevices like
the wiimote have such exciting connotations foi tying visual physical musical
gestuies to the sounus emanating fiom the speakeis.
3.3.2.1 1he WllmoLe for LlecLronlc Muslc ConLrol
Theie have been innumeiable attempts ( (C Niuolo n.u.)(sapphiieS27
2uu9)(lebasson 2uu7) (Nasheuuk 2uu7) (Lehiman 2uu8) among many otheis)to
tie the outputs of the wiimote to NIBI capable synthesiseis anu uevices, some
S9
genuinely novel anu innovative((Nanabe 2uu8), (Lehiman 2uu8)) otheis not so
impiessive (ocukloweuk 2uu8) (Nowheie2uu7 2uu7).
Beie, we will a small cioss section of these of these applications.
'Wii Nusic' (Nintenuo 2uu8) is a viueo game in which playeis can use the wiimote
anu nunchuck to 'play' vaiious instiuments such as violins, guitais anu uiums.
While the game is not as constiictive as othei music games such as uuitai Beio
(Activision 2uu9), (a game that involves pushing buttons in time to the music foi a
highei scoie), the playei is still ultimately constiicteu to set backing tiacks anu a
veiy basic sounu engine with no ieal scope foi seiious musical enueavouis. A
viueo of the Wii Nusic can be founu on the accompanying CB.
Beyonu the iealm of official Nintenuo viueo games, one only has to make a
cuisoiy seaich of the inteinet to finu countless examples of musicians anu
hackeis using the wiimotes foi all kinus of musical applications.
'The Wiitles' aie a 4 piece banu fiom Ameiica who use the wiimotes to contiol
NAXNSP patches to play sounus. In this case howevei, the mapping of the
contiolleis iaiely go beyonu basic one-to-on mappings (buttons to contiol notes
with minimal acceleiometei usage), anu theii high piofile seems to be baseu on
novelty iathei than any seiious new application foi this technology (The Wiitles
2uu8). A viueo of the Wiitles can be founu on the accompanying CB.
0ne of the othei factois that have maue the wiimote so populai aie the numbei of
S
iu
paity applications that have been uevelopeu to allow people without any
knowleuge of softwaie uevelopment to connect to the wiimotes anu conveit theii
sensoi uata to NIBI. Softwaie such as 'WiiToNiui' (veiuone 2uu7), ' ulovePIE',
(Kennei 2uu7) anu '0SCulatoi' (Tioilaiu 2uu9) have been a fantastic base foi
people to begin to expeiiment with the wiimotes without having to uevelop theii
own softwaie. 0nfoitunately, any ueep piogiamming of these inteifaces is
impossible unless the usei obtains the souice coue foi mouification, which is
possibly one of the ieasons why so many of the applications of the wiimote foi
music stay in ielatively simple teiiitoiy when mapping is conceineu.
4u
In the iealm of acauemic music, composei anu technologist Paul Lehiman
investigateu the musical application of the Wiimotes with his piece 'Imaginaiy
Bialogues' (Lehiman 2uu8) in which two wiimote playeis manipulateu softwaie
synthesiseis via a NAXNSP contiol patch. While the sounus geneiateu weie
influenceu by the motion of the two playeis, some inteimeuiaiy ianuomisation
logic contiolleu significant poitions of the output, making the sounu-scape style
output moie a uuet between man anu machine. Anothei inteiesting appioach has
been the uevelopment of a viitual Theiemin (C ueigei 2uu8), which allows tactile
two hanueu contiol but with a simple mapping schema.
3.3.2.2 CLher CesLural Muslcal ConLrollers
While the wiimotes weie haileu as a ievolution in the viueo gaming woilu,
gestuial contiolleis have been aiounu foi some time in the fielu of electionic
music ieseaich. Two significant histoiical innovations weie Nax Natthews' 'Rauio
Baton' (1987) anu Palmtiee Instiuments 'Aii Biums' (1987).
The Rauio Baton was the biain chilu of electionic music pioneeis Nax Natthews
anu Robeit Bole. The baton emitteu low fiequency iauio waves, anu the usei
woulu move theii baton ovei a flat suiface of five coppei plate antennas to
ueteimine position in SB space. This value was then useu to uiive a sounu engine.
0nly a hanuful of Rauio Batons weie evei piouuceu, but they still see active use
fiom composeis such as Richaiu Boulangei anu Anuiew Schloss (Sounu 0n Sounu
2uu8) .
The Aiiuiums can be consiueieu to be uiiect ancestois to the wiimotes. The
uevice consisteu of two cylinuiical tubes aiounu 2ucm long which each houseu a
thiee axis acceleiometei. The uata fiom these acceleiometeis was feu into a
piocessing unit, wheie the usei coulu piogiam the uevice to inteipiet these
gestuies as a iange of uiffeient NIBI commanus which weie then emitteu fiom
the unit. The music piess at the time was full of piaise foi the uevice, but its
complicateu inteiface, fiagility anu high cost leu to pooi sales, with piospective
useis favouiing othei uevices like keyboaius anu NIBI guitai contiolleis as theii
inteifaces of choice. (Sounu 0n Sounu 2uu8).
41
Nichel Waisvisz's 'The Banus' (N. Waisvisz 198S) is anothei gestuial inteiface
woithy of mention. Waisvisz built himself a completely customiseu paii of
contiolleis that weie wiappeu iounu the hanus, anu containeu buttons, motion
sensois anu a miciophone foi auuio input. Waisvisz uevelopeu an extiemely
uexteious peifoimance technique with the instiuments in a fantastic example of
self imposeu iestiaint in moving on to new technology, a piocess he iefeiieu to as
freezinq tecb Jevelopment - that is, the suspension of technological innovation to
fully exploie an inteiface oi contiol set.

7#7#7 *+;,)-)I' N+5BO),
As a geneial iule, people have been histoiically ieluctant to uiastic change,
especially in the oluei geneiations. The Bv0RAK keyboaiu, the uominance of the
iPou anu Niciosoft Winuows, Espeianto anu the stiuggling campaign foi a
gieenei lifestyle seive as a goou examples of this phenomenon, wheieby
seemingly supeiioi alteinatives to the mouus opeianui aie shunneu in favoui of
convenience. The issue of cognitive lock-in, that wheie by people stay with a
paiticulai bianu oi inteiface to peifoim "qool-octivoteJ outomoteJ beboviours
tbot Jevelop tbrouqb tbe repeoteJ consumption or use of o porticulor proJuct."-
(Nuiiay 2uu7) was investigateu in 2uu7 by Nuiiay et al. The expeiiments weie
uesigneu to test the theoiy that, iegaiuless of an inteifaces tiue usefulness oi
uifficulty, if a usei finus the leaining expeiience uifficult in the fiist instance when
compaieu to anothei system, they will peiceive it as moie uifficult. The stuuy
founu that:
o 0seis piefei tiial anu eiioi when uevising inteiface piefeience foimation
o The amount of expeiience with a given inteiface significantly incieases
usei piefeience foi an inteiface
o Eiiois while leaining an inteiface significantly ueciease piefeience foi an
inteiface, iegaiuless of its tiue level of uifficulty
In (Bountouiiuis 2uu7) looking into inteifaces foi the contiol of SB sounu, he
founu that the useis piefeiieu anu peifoimeu bettei on the stanuaiu computei
mouse inteiface, iegaiuless of the fieeuom offeieu by the 6B0F Space Navigatoi
(SBConnexion 2uu9).
42
In (Bunt 2uuu), thiee sounu contiol inteifaces weie ueviseu anu testeu, in an
expeiiment to exploie levels of complexity of contiol suiface in ielation to the
completion of tasks with vaiying levels of complexity. The subjects weie askeu to
use the following inteifaces, each to complete a set of musical tasks:
A mouse contiolling a single pointei to move on scieen sliueis (much like a
mouein softwaie uigital auuio woikstation)
A set of uisciete physical faueis, each contiolling one aspect of the sounu
A combination of a mouse anu faueis with minimum visual feeuback with a
complex mapping system, iequiiing the usei to continually inject eneigy
into the system by moving the mouse, encouiaging the useis to think in
gestuies iathei than sepaiate paiameteis
In contiast to (Nuiiay 2uu7) wheie the complexity of the tasks the useis weie
askeu to peifoim iemaineu effectively consistent, (Bunt 2uuu) vaiieu the actual
complexity of the tasks. The iesults showeu that the simplei mouse inteiface
scoieu highest foi the peifoimance of the simplest tasks, but useis showeu no
signs of impiovement ovei time, while the complex inteiface showeu
impiovement ovei time anu actually peifoimeu bettei at the moie complex tasks,
uespite its initially uaunting contiol schema. Subjects also expiesseu that they felt
moie engageu with the complex contiol suiface, its unobvious mapping scheme
foicing them to woik in gestuies anu moie intuitive movements iathei than
having to think about the mechanics of the contiol system itself.

Looking at the above stuuies, we can ueuuce the following:
a. People aie moie likely to choose a familiai inteiface ovei an unfamiliai
one even if the unfamiliai inteiface offeis bettei options foi contiol
b. People's peiception of the uifficulty of an inteiface is hugely affecteu by
uifficulties encounteieu in leaining, iegaiuless of its actual uifficulty,
something gieatly affecteu by familiaiity
We also see that:
4S
c. People feel moie engageu with moie complex contiol systems as
continueu use iewaius the usei with impiovement
u. People can eventually peifoim moie complex tasks when using a moie
complex inteiface
While moie complex anu sometimes less obvious inteifaces pioviue highei levels
of contiol to a usei, the usei must fiist bypass the temptation to uisiegaiu it anu
stay with a moie familiai system. If the pioposition by (Nuiiay 2uu7) that people
piimaiily leain new inteifaces by tiial anu eiioi holu any weight, it isn't haiu to
see why electionic musicians have stuck to theii keyboaius. The initial connection
between cause anu effect is uiscoveieu veiy eaily on with a keyboaiu, thus
incieasing musicians piefeience foi it anu incieasing the incentive foi
manufactuies to auu new featuies to the keyboaius, iathei than go back to the
uiawing boaiu anu uevise new expiessive inteifaces.
7#7#9 :6AA),;
Since the veiy beginnings of human civilization, music has playeu an integial iole
in society, both in ait anu leisuie. In staik contiast to this ten thousanu yeai
gestation peiiou of musical theoiy anu instiument uesign, we only have to look
back a meie hunuieu yeais oi so, to finu a technological lanuscape almost
completely uevoiu of electionics, anu even less fai, maybe fifty oi sixty yeais, to
finu no uigital electionics in common use at all.
Nouein electionic instiuments aie most often nothing but a computei in box,
manipulateu with uigitiseu analogue inputs. The issue of the sounu souice being
totally sepaiateu fiom the sounu geneiatoi is a ielatively new concept, which has
biought about the neeu foi a new fielu of stuuy - that of moppinq.
As stateu in S.S.1, a cello uoesn't have contiols foi volume oi tone, its output is
contiolleu by uiiect inteiaction with the physical bouy of the instiument. The best
technique to get the sounus you want fiom the instiument is to uiiectly
manipulate that physical bouy, piessing haiuei with the bow will make the sounu
louuei foi example. In contiast to this, the amount of physical eneigy you put into
a computei contiollei beais no ielation to the sounu that comes out unless it is
specifically piogiammeu that way by the uesignei. With this in minu then, it
44
seems iathei stiange that since the auvent of uigital electionics anu its inevitable
assimilation with musical peifoimance anu composition we have all but neglecteu
this vast knowleuge base of instiument uesign built up ovei the past ten
millennia. Theie aie veiy few electionic synthesiseis that even make use of the
afteitouch NIBI message, insteau choosing to uelegate the contiol of theii timbial
palette to sliueis anu uials, oi woise, the uigital menu scieen.
With the staggeiing aiiay of technology available to instiument uesigneis, it is
often easy to oveilook some of the most funuamental concepts that unueilie the
vast majoiity of the so calleu 'tiauitional' acoustic instiuments:
Physical eneigy in = sonic eneigy out
Eneigy injection is usually oithogonal to eneigy contiol
Nany instiuments with lineai pitch manipulation inteifaces (keyboaius oi
fietboaius oi example), the bass is to the left anu the tieble is to the iight.
A few synthesisei uesigneis have attempteu to incoipoiate extia continuous
contiolleis into theii synthesiseis to use along with the keyboaiu. The now
ubiquitous pitch benu wheel anu mouulation wheel populaiiseu by the NiniNoog
in the 197us allows the playei some basic continuous contiol, allowing both the
pitch of notes anu one aspect of the timbie to be quickly anu piecisely changeu
aftei a note has been stiuck. Theie have been a few vaiiations on these themes,
such as the Rolanu B-Beam anu the iibbon contiollei, but all essentially peifoim
the same function, a one uimensional input vaiiable which alteis one aspect of the
sounu.

U'6%-# 8D V24,15 K_M#,+e , 7.7&#+ %7'16 '1C-,_-#5 -,5',&'21 &2 5#&#$& &F# B-2"'+'&. 2C &F# F,15e
+,1'B%4,&'16 ,7 ,7B#$& 2C &F# 2%&B%&_ C-2+ V24,159$2+
4S

U'6%-# 8W V'==21 $21&-244#- 21 , c2-6 !4#$&-'=# 5-%+ +,$F'1# _ C-2+ *2%15<1*2%159$2+
While these vaiiations on the pitch anu mouulation wheel might look exciting in
the music shop winuow, they uon't ieally offei anything new in teims of contiol
as they aie essentially one uimensional contiol spaces - one contiollei attacheu
to one function. While it is tiue that most mouein uigital synthesiseis on the
maiket touay offei quite ueep anu complex tempoial tonal evolution of sounu,
these musical movements have to be piogiammeu by the usei oi the factoiy,
which, at fiist glace uoes seems poweiful anu enveloping but ultimately leaves the
usei contiolling a gloiifieu samplei, with iuentical iesults being put foith each
time the usei goes to key a note.
In (Levitin 2uu2), it was noteu that the most successful tools (be it an instiument
oi anything else foi that mattei) have the ability to be useu by most people at a
basic level, but aie iich enough that a peison can uevote a lifetime of piactice to
anu uevelop consiueiable skill outsiue that of the aveiage usei. This will be one of
the coineistone consiueiations of this pioject. It is vital that this instiument
stiikes a balance between usability anu uifficulty so as to piesent a challenge to
its useis to gain skill, but not alienate them completely.
(Aifib 2uu2) maue the inteiesting note that musical instiuments neeu to be
pusheu outsiue theii commonly unueistoou bounuaiies to keep them inteiesting.
"Some composers ore literolly lookinq for possible Jeviotions in o computer
proqrom. Wben plonninq o moppinq, one must toke tbis into occount: on
instrument is interestinq only if it ollows tbe Jiscovery of unknown feotures." (Aifib
2uu2)
With this in minu then, it stanus to ieason that to make an inteiesting instiument,
some aspect of eithei ianuomness oi secietive piogiamming must be
46
incoipoiateu into these explicit mappings to allow foi 'extenueu techniques' that
the useis can finu foi themselves by pushing the instiument at its bounuaiies.

47
B2I @+<<>,J '( 3,'4-1& E>1F*,'+/0
It is now appaient that theie is a neeu foi new moues of expiessive musical
contiol. It is also appaient that this issue is multi-thieaueu, with many
inteiuepenuent issues anu factois affecting the cuiient state of play. Below is a
summaiy of the pieceuing chaptei. It will attempt to cieate a set of guiuelines to
be useu in the uesign of these wiimote contiolleu instiuments.
?612-'16 +%7'$,4 '17&-%+#1& 5#7'61 F'7&2-. C2- &F# 7,0# 2C 12(#4&. '7 ,
+'7&,0#9 G# 7F2%45 =# 4220'16 &2 $2+='1# 72+# 2C &F#7# 3#44 01231
7&,15,-57 3'&F &F# 1#3 &#$F12426. 72 ,7 12& &2 $2+B4#&#4. ,4'#1,&#
B-27B#$&'(# %7#-7e ,15 &2 #,7# &F# B-2$#77 2C 4#,-1'169
*B#$'C'$,44. B-26-,++'16 -,152+ 2- 121_4'1#,- #4#+#1&7 '1&2 , +,BB'16
7$F#+# '1$-#,7#7 &F# 5#B&F 2C &F# '17&-%+#1& ,15 #1$2%-,6#7 #"B42-,&'21
2C '&7 =2%1579
With these two consiueiations in minu, it then follows that:
EF# '17&-%+#1& 7F2%45 =# ,=4# &2 =# %7#5 =. B#2B4# ,& , =,7'$ 4#(#4e =%& ,'+
&2 F,(# 7%CC'$'#1& 5#B&F &F,& , B#-721 3'7F'16 &2 #"B42-# &F# '17&-%+#1&
C%-&F#- $,1 52 729
B2K LMNA,-%%$7-/-%%O
The woiu 'expressive' is mentioneu a gieat ueal in the liteiatuie of electionic
musical inteifaces, but few attempt to uefine it. The 0xfoiu English Bictionaiy
entiy foi expressive ieaus as follows:
!ex-pres-sive ,kspresv, oJj. Sbowinq or oble to sbow your tbouqbts onJ feelinqs:
Sbe bos wonJerfully expressive eyes. tbe expressive power of bis music"
This uefinition is iathei vague, anu uoesn't seive as a useful ciiteiia on which to
conuuct an acauemic stuuy.
When a peifoimance is uesciibeu as 'expiessive', it is often attempting to ielay
the fact that the oiiginal emotional intention of the composei has been
successfully invokeu by the peifoimei. To accomplish this ielaying of complex
emotional infoimation, the instiument must have both sufficient input vaiiables
48
to allow foi a laige iange of tonal manipulation, but moie impoitantly, must have
an inteiface which allows the usei to manipulate these numeious vaiiables
quickly enough to allow them to convey theii uesiieu output.
A geneial uefinition of expiessiveness is uifficult to uefine. In the context of this
iepoit howevei, the 'expiessive' capacity of an instiument will be the extent to
which a usei can exploie its mapping schema to uevelop theii own unique sounu.
This implies that moie auvanceu anu complex mappings will piouuce moie
'expiessive' instiuments, allowing playeis to attach emotional weight to sounus
being put foith.
49
I 3,'4-1& C-&:'0'D'*J >/0 @A-1$($1>&$'/
Beie, we will take the iequiiements set out in section S.4 anu piesent the uesign
anu methouology of this stuuy.
I2. @+<<>,J '( P$<%
The enu goal is an engaging wiimote contiolleu instiument that can be pickeu up
anu useu by a peison with little expeiience, but whose mapping stiuctuie anu
sounu engine is complex enough to allow foi ueepei exploiation of the instiument
anu the uevelopment of technique. We have pieviously uiscusseu the meiits of the
wiimote anu some of the pievious woik in the aieas of both wiimotes foi music
anu musical human computei inteiaction. Consiueiing this, we can uevise
piimaiy methous of inquiiy to uesign this instiument:
J-#,&# ,1 '1'&',4 7'+B4#- '17&-%+#1& ,15 $215%$& , %7#- 7%-(#.
By ueveloping a simplei instiument in the fiist instance, the piacticalities of
inteiacting with the wiimote can be expeiienceu fiist hanu. The instiument will
also be put thiough usei tiials to gauge how people ieact to playing music with
this inteiface. This shoulu pave the way foi a moie consiueieu stieamlineu uesign
piocess which shoulu iesult in a moie satisfactoiy enu piouuct.
J215%$& , 7%-(#. 2C 72+# O&-,5'&'21,4P ,$2%7&'$ '17&-%+#1&79
By analysing the way sounu changes with both input velocity anu peifoimance
vaiiables, the familiai non-lineai iesponse of tiauitional instiuments can be
eliciteu fiom the wii contiolleu instiument. This shoulu make the playing
expeiience a moie engaging affaii, anu allow foi ueepei exploiation of the
instiument anu the uevelopment of technique.
K#7'61 , $%7&2+ 72%15 #16'1# C2- &F'7 '17&-%+#1&9
The way output contiol vaiiables aie ielayeu fiom the instiument will be non-
stanuaiu. This will iequiie a custom sounu engine to be built fiom the giounu up
to accommouate this contiol schema to allow foi the most efficient anu non-
compiomising playing expeiience.
Su

I28 3,'4-1& C>/>*-<-/&
Following the initial ieseaich anu pieceuing pioject uevelopment, a time plan was
ueviseu to give some sense of ielative scale to the elements of the pioject, anu to
have tangible milestones to measuie pioject piogiess. A uantt chait uepicting this
time plan is shown in Figuie 17.

U'6%-# 8N ?1'&',4 &'+# B4,1
T
a
s
k
S
p
r
i
n
g

T
e
r
m
E
a
s
t
e
r

B
r
e
a
k
S
u
m
m
e
r

T
e
r
m
4
5
6
7
8
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
9
2
0
2
1
2
2
2
3
D
e
r
i
v
e

S
p
e
c
i
f
i
c
a
t
i
o
n
R
e
s
e
a
r
c
h

i
n
t
o

f
i
e
l
d
I
n
i
t
i
a
l

R
e
p
o
r
t
D
e
s
i
g
n

W
i
i
d
i
i

V
1

(
S
p
h
e
r
e
)
B
u
i
l
d

W
i
i
d
i
i

V
1
U
s
e
r

T
r
i
a
l
s

o
n

W
i
i
d
i
i

V
1
I
n
s
t
r
u
m
e
n
t

M
a
p
p
i
n
g

r
e
s
e
a
r
c
h
I
n
s
t
r
u
m
e
n
t

S
o
u
n
d

r
e
s
e
a
r
c
h
R
e
f
l
e
c
t

o
n

F
e
e
d
b
a
c
k
D
e
s
i
g
n

W
i
i
d
i
i

V
2

(
'
C
e
l
l
o
)
B
u
i
l
d

W
i
i
d
i
i

V
2
D
e
s
i
g
n

S
o
u
n
d

S
o
u
r
c
e
B
u
i
l
d

S
o
u
n
d

S
o
u
r
c
e
T
e
s
t

a
n
d

T
w
e
a
k
R
e
p
o
r
t

W
r
i
t
e

U
p
H
a
n
d

I
n
L
e
s
s

I
n
t
e
n
s
i
v
e
M
o
r
e

I
n
t
e
n
s
i
v
e
M
i
l
e
s
t
o
n
e
s
:
G
o
a
l
E
n
d

o
f

W
e
e
k
C
o
m
p
l
e
t
e
d

D
e
s
i
g
n

o
f

S
p
h
e
r
e
7
S
p
h
e
r
e

p
r
o
t
o
t
y
p
e
d
8
B
e
g
i
n

S
p
h
e
r
e

U
s
e
r

T
r
i
a
l
s
1
0
F
i
r
s
t

b
u
i
l
d

o
f

W
i
i
d
i

V
2
1
6
C
e
l
l
o

C
o
m
p
l
e
t
e
1
9
1

D
r
a
f
t

r
e
p
o
r
t
2
1
H
a
n
d

I
n
2
3
S1
The list of tasks uefineu may at fiist seem spaise, but the exact natuie of uesign
woik was not know at this stage. Finuing this out was the ieason foi builuing the
simplei 'Spheie' instiument. These specifics woulu become appaient in the
couise of the pioject anu moie specific goals woulu be uefineu on a week by week
basis. While this may appeai a little uangeious in teims of piopei pioject
planning, that natuie of this pioject is such that a basic system can be uefineu, anu
'extias' can be auueu at will without any significant change to pioject
funuamentals. This allows foi a slightly moie geneial time plan that that which
woulu noimally be associateu with a masteis pioject.
9#"#$ :6,6;'3',- :'-&+?+/+;1
0vei the couise of the pioject, piogiess woulu be tiackeu via the flowing
methous:
G##04. H##&'167 ,15 #+,'47
The weekly pioject supeivision meetings will pioviue piogiess tiacking anu the
setting of shoit teim goals. Any pioblems can be uiscusseu anu pioject uiiection
can be tiackeu. An electionic summaiy of piogiess will be maintaineu via weekly
emails which will seive as a iefeience foi both supeivisois anu the authoi as to
the piojects piogiess. These emails can be founu on the accompanying CB.
:-2;#$& Z2&#=220
A pioject notebook will be kept to iecoiu all thoughts anu iueas. Baving all these
notes in one place will be invaluable when the pioject iepoit is to be wiitten. The
notebook foimat by its veiy natuie iecoius the piojects piogiess in lineai fashion,
so pioject piogiess can easily be tiaceu anu tiackeu on an almost uay by uay
basis. It also allows foi the non-lineaifieefoim expansion of concepts anu iueas
which is an excellent tool in a pioject such as this with the enoimous iange of
possibilities anu options available at almost eveiy junctuie in the couise of the
pioject.

S2
!4#$&-21'$ K,&, H,1,6#+#1&
All uata in electionic foim (souice coue anu on going iepoits) will be backeu up
on a uaily basis to both a home seivei anu to the univeisity FTP seivei. This will
seive as a backup anu change log.
SS
K 9:- @A:-,-
The fiist instiument to be consiueieu to be playeu by the wiimotes was in fact the
Cello (founu in the following chaptei) but as the limitations of the wiimotes weie
not known, it was ueciueu that it woulu make sense to begin with a simplei
instiument. This iuea manifesteu itself as 'The Spheie'.
The foimat chosen was a tuneu peicussion instiument, similai to that of the steel
pan, only spheiical - hence the name. The usei woulu 'hit' this viitual spheie, with
the angle the wiimote ueteimining pitch, the stiongei the hit, the louuei the
outcome.

U'6%-# 8X :4,1 C2- fEF# *BF#-#f
The closest iesembling woiking example this authoi coulu finu online was the
'Wiinstiument' uium simulatoi(ScieenFasion 2uu9). Theie is some iuuimentaiy
velocity sensing, but all the note choices aie uone via the buttons on the wiimotes,
anu the position the wiimotes aie helu at beai no ielation to the output. This
'piess button foi uesiieu note' is veiy similai to the tiauitional keyboaiu
inteiface, anu this authoi can see no ieal auvantage in using this inteiface system
ovei a keyboaiu.
S4
The fiist poit of call foi looking into an instiument like this was the steel pan. The
builuing of a quality steel pan is a highly skilleu job, anu is steepeu in tiauition
with fatheis teaching sons the ait of pan builuing ovei a life time. Theie aie veiy
few mastei steel pan builueis anu because of this, liteiatuie on steel pan uesign is
quite haiu to come by. The is one iecent auuition to the woilu of pan builuing, the
excellent Pon Poqe which goes into some uetail on steel pan constiuction anu
histoiy (Kionman 2uu9). The mouein steel pan is tuneu accoiuing to the ciicle of
fifths, so that when a playei is in one specific aiea of the pan, the immeuiately
aujacent notes aie haimonically ielateu (see Figuie 19).

U'6%-# 8I S,.2%& 2C , &#12- 7&##4 B,1 _ C-2+ F&&B`^^F2&B,1797#^B,1^
This makes the pan simplei to leain anu eases the playing of fast haimonic iuns.
These uesign consiueiations aie paiticulaily peitinent to the spheie, as the usei
will not have a visual cue to wheie they'ie hanus aie locateu, anu so haimonic
inconstancy can be avoiueu while the useis get to giips with wheie the notes aie
locateu.
K2. 3,'*,><<$/* >/0 'A-,>&$'/
0ne of the ieasons foi beginning with this instiument was to gain a footholu into
the methous of woiking with the wiimotes anu the uata they gave out.
Foitunately, theie aie many S
iu
paity libiaiies alieauy in existence which
facilitate connection to the wiimotes anu the output of NIBI messages. The
libiaiies anu softwaie useu aie as follows:
SS
- wiimoteLib (http:www.coueplex.comwiimoteLib) foi connection anu
uata extiaction fiom the wiimotes
- NIBI Tool Kit (http:www.couepioject.comKBauuio-
viueoNIBIToolkit.aspx) foi senuing NIBI out of the piogiam
- WiimoteAutoConnect
(http:localhosti.comfiles1aubcuAutoConnect.zip) a small application
to aiu the laboiious connection of wiimotes to the Winuows Buman
Inteiface Bevice (BIB) Bluetooth stack
- NIBI Yoke (http:www.miuiox.com) foi inteinal NIBI iouting in
Winuows
<#$#$ *+,,'5-)+,
As the wiimotes weie only uesigneu to connect to the Wii console, they aie not
fully BIB compliant. This causes some pioblems with the Winuows Bluetooth
stack, meaning that the wiimotes must be uninstalleu anu ieinstalleu eveiy time
the usei wishes to use any softwaie that connects to the wiimote. To get aiounu
this, Wii Remote AutoConnect was useu to get aiounu the teuious task of having
to iepeateuly ieinstall the Bluetooth uevice befoie eveiy connection.
Foi uetaileu uata on how the uata fiom the wiimotes is collecteu by Winuows anu
feu thiough to the Spheie, please iefei to appenuix A.
<#$#" P+-'O+, F'-'5-)+,
The fiist task to be auuiesseu was to finu the best way to uetect when a usei
wanteu to stiike a note. Fiist, the uata fiom S stiikes weie analyseu. 0ne at u,
S6
one at 4S anu one at 9u ioll.

U'6%-# >Q /$$#4#-2+#&#- 5,&, C-2+ @ 7&-'0#7 3'&F &F# 3''+2&# 9"`-#5 .`6-##1 a`=4%#
You can see fiom Figuie 2u, theie aie 4 main stages of the hit. Theie is a negative
acceleiation as the hanu is iaiseu befoie the stiike, a shaip inciease in positive
acceleiation as the hanu moves uown, a much gieatei ueceleiation as the hanu
comes to a stop anu then an equalizing positive acceleiation as the hanu comes to
iest.
3.1.2.1 lnvesLlgaLlng upward lnLegraLlon
I hau initially planneu to integiate up to get velocity ieauing so that the note-on
velocity value coulu be taken stiaight fiom these cuives. The value the Z
acceleiometei sees is uepenuent on the oiientation of the wiimote: if it is helu flat
anu hoiizontal (piioi to the fiist spike in Figuie 2u) it measuies - 1 (g).,if at 4S it
measuies V22 anu if at 9u it measuies u. As uiscusseu in chaptei 2.2.4, six
uegiees of fieeuom cannot be obtaineu fiom S inputs, howevei I was inteiesteu in
investigating the concept that if one assumes motion in only one axis, it may be
possible to woik out the iequiieu calibiation to offset Z anu make it constantly
zeio when not in motion at any oiientation.
Beie is the pioblem: Take foi example a stiike at 4S with an acceleiation (A).
The z acceleiometei ieaus g.cos(4S) + A (see Figuie 21) anu so to offset this, we
neeu to take a look at y to see what the elevation is to subtiact fiom the z ieauing
so we can obtain the actual acceleiation of z.
S7

U'6%-# >8 ?44%7&-,&'21 2C , &.B'$,4 52313,-5 +2(#+#1&
As we aie moving at 4S in the z uiiection, the pieviously stationaiy uownwaiu
component of the y ieauing being useu to calculate the offset foi z is being
obscuieu by the acceleiation A by A sin(4S). It gets even woise when we consiuei
the X axis. This is a piime example of why the wii motion plus (section 2.S) will
be such an impoitant next step foi the wii contiollei.
3.1.2.2 veloclLy ueLecLlon 1echnlque
The initial appioach to this pioblem was to set a thiesholu which woulu activate a
timei when a laige enough acceleiation value passeu it. Aftei a ceitain numbei of
cycles, the piogiam woulu check back to see if it was on an upwaiu tienu. If this
was the case, it woulu set anothei timei, anu aftei a ceitain peiiou of time, finu
the bottom of the tiough was anu fiie a note. The iationale foi uoing this was
piobably founueu in wiiting coue foi high sample iate souices, wheie a goou ueal
of piocessoi cycles can be saveu by checking less often. The iepoit iate of the
wiimote is only aiounu 1uu Bz oi so, anu as such, it uiun't make much sense to
have these spaceu inteivals. The final appioach useu was to look at eveiy sample
anu set viitual 'hoops' foi the acceleiation spike to jump though. If the spike
manageu to jump thiough each consecutive hoop sample aftei sample, then it
coulu be assumeu that a stiike hau been maue.
The initial uownwaiu peak was ignoieu by the peak uetectoi as a usei coulu
simple iaise theii hanu anu not play a note oi conveisely, stait a note fiom a
uiiect uownwaiu movement. The algoiithm woikeu by fiist looking at each new
S8
incoming sample anu compaiing it the last one. If the new sample was laigei than
the pievious, the algoiithm applies successive 'hoops' the spike has to jump
thiough befoie it can be consiueieu a significant peak. If the peak uoes not
manage to make it thiough the next hoop by the next sample, the uetectoi is ieset
(see Figuie 22).

U'6%-# >> :#,0 5#&#$&'21 _ U'-7& B#,0 +,0#7 &F# C'-7& F22B =%& C,'47 &2 +,0# &F# 1#"& 4#(#4 =. &F# 1#"&
7,+B4#9 EF# 7#$215 $%-(# '7 R%'$0 #12%6F &2 +##& ,44 C2%- F22B7 ,15 &F%7 , 12&#_21 '7 6%,-,1&##5
Finuing the uistance between these hoops, effectively a sensitivity vaiiable was
founu by tiial anu eiioi, anu a contiol to aujust sensitivity was put in place to
allow the usei to change it at theii will.
0nce a peak has been establisheu, the uevice then tiacks the wave until it ieaches
the tiough of the uownwaiu peak. At this point it senus a message to the contiol
class to activate a note. An initial pioblem with this methou was uouble tiiggeiing
as the upwaiu tienu fiom the tiough anu the slight oveishoot looks like anothei
uisciete hit to the uetectoi (illustiateu in Figuie 2u). At fiist, anothei uisciete
stage was put into the uetectoi, but if the usei hau tilteu the wiimote uuiing the
stiike, the softwaie tenueu to become unpieuictable, anu gave an inconsistent
tiiggeiing iesponse. By auuing a one sample (~1ums) wait aftei each note hau
been tiiggeieu, the uouble tiiggeiing was completely iemoveu. This small 1
sample wait beais no change in iesponse of the system, as fai as the usei is
Time
A
m
p
l
i
t
u
d
e
S9
conceineu as it woulu be impossible to tiiggei two concuiient notes in less than
1ums.
The velocity output foi the system was baseu on the cumulative amount of the
upwaiu tienu at each hoop, anu was scaleu to match the NIBI velocity iange. As
well as the lineai tiansfoimation, some othei velocity cuives weie auueu to let
the playei choose which felt the most tactile. They aie shown below:

U'6%-# >@ S'1#,- (#42$'&. $%-(#

U'6%-# >L * (#42$'&. _J%-(#
6u

(4)

U'6%-# >D *R%,-#5 Y#42$'&. J%-(#

(S)

Some pseuuo coue outlining the note uetection algoiithm can be founu below. Foi
the full section of coue, iefei to appenuix B1.
If (z_input > prev_z_input) AND (hoop_count = 0)
Start hoop counting
Take current height
If (checking for upward trend)
if(current count > old count + hoop height)
add 1 to hoop_count and carry on until hoop count = 4;
if z_input < old_z_input then reset hoop_count and start again
If (hoop_count = 4)
Definitely in a peak, will be going down soon so check for z_input < old_z_input
If ( z_input < old_z_input)
Now just started going down, velocity = current peak height
Wait until (z_input > old_z_input)
Now at bottom. Send note on
61
Wait 10 ms and reset;

Bue to the shape anu lowei mass of the nunchuck contiollei, its velocity ieauing
hau to be scaleu up slightly to give a matcheu paii of beateis, so that the
peiceivable foice put into each hanu gave an equivalent output.
Note length was ueteimineu by a timei which was haiu coueu to be aiounu Sums.
This was plenty of time foi it to iegistei on any synthesisei wheie the attack,
uecay, sustain anu ielease coulu be set inuepenuently.

<#$#7 G)-5& F'-'5-)+,
As the playei moves theii hanus iounu the Spheie to change the pitch, the angle
of iotation of the wiimote woulu change. This fact is useu in ueteimining the pitch
by measuiing the stiength of giavity on the x acceleiometei.
The vast majoiity of motion will be paiallel to the z acceleiometei, anu because of
this, it can't be useu foi accuiate iotational positioning. The pioblem with the
spheie mouel, is that a single acceleiometei can't tell the uiffeience between a tilt
of ! uegiees to the iight anu a tilt of S6u - ! uegiees to the left . This woulu be a
pioblem if the notes weie aiiangeu into effectively two veitically placeu semi
ciicles. Consiuei Figuie 26. The two wiimote aie both helu stationaiy with the x-
acceleiometei looking in the uiiection of the ieu aiiows. Each wiimote iepoits
back a value of x = g cos(!), even though they aie both being helu at uiffeient
oiientation. If the wiimotes weie to iemain stationaiy, the x ieauing coulu inueeu
be calibiateu by looking at the z acceleiometei, but as theie will be constant
motion in the z plane, this becomes impiactical as uiscusseu in 2.2.4.
62

U'6%-# >W ?44%7&-,&'21 2C B-2=4#+ C,$#5 3F#1 &-,$0'16 2-'#1&,&'21 3'&F 8 ,$$#4#-2+#&#-
To get aiounu this pioblem, the layout of the spheie was changeu to two viitual
hoiizontal semi ciicles placeu siue by siue.

U'6%-# >N :'&$F 4,.2%& 2C &F# f*BF#-#f
The value of the x acceleiometei is passeu to the pitch uetection class eveiy cycle
anu is calleu upon to ueteimine the actual note that is iequiieu to be playeu when
the acceleiation uetectoi class gives the go aheau to play a note.
6S
3.1.3.1 lLch Curves
Two pitch uistiibutions weie initially uesigneu anu put into look up tables (see
Figuie 28). A six inteival minoi pentatonic scale was useu (I, bIII, Iv, v, bvII, anu
octave) as making a mistake whilst leaining anu expeiimenting woulun't
necessaiily give a uissonant sounu. It was also felt that the pentatonic scale woulu
give a suitable balance between a sufficiently small iesolution to enable a useable
anu playable inteiface anu a choice of notes.

U'6%-# >X U'-7& &32 B'&$F 5'7&-'=%&'217
The numeiic uistiibution was the fiist anu most obvious choice, which was
initially implementeu as a quick pioof of concept. The incoming x axis
acceleiometei uata stieam is effectively a stiing of cos(!) values with g = 1. The
numeiic uistiibution just sepaiates these values equally. This leaus to quite an
uneven angulai uistiibution that is quite haiu to play on the miuules notes
(sepaiation 48, 7u,9u, 1u9, 1S1).
The seconu iteiation of the pitch uistiibution was to sepaiate the aiea by angle,
with each aiea being Su in size. This gives a much moie even playing expeiience
than the numeiic uistiibution.
3.1.3.2 8eflnlng plLch deLecLlon
The instiument at this stage still hau two main pioblems. The fiist was that the
value of acceleiometei x was affecteu by the amount of tilt in the y plane.
}ust aftei the note on commanu is sent anu the wiimote is coming to iest, the pitch
uetection algoiithm is useu to finu how fai the wiimote is iolleu ovei by looking at
the filteieu acceleiation ieau fiom the x acceleiometei. If the wiimote is totally
flat (ie, y acceleiometei ieaus u) then the ieauing will be accuiate. Now consiuei
the case when the wiimote comes to iest at a pitch of 4S uegiees fiom the
I
bIII IV V bVII
8va
Numeric
I
bIII
IV
V
bVII
8va
Angular (30)
64
hoiizontal plane. The value ieau on the x acceleiometei sboulJ be
!
2
2
(sin(4S))
but will in fact give back the ieauing u.S. This is uue to the 4S uegiee pitch
offsetting the value of x.

U'6%-# >I G''+2&# B'&$F#5 ,& LD 5#6-##7e -244#5 &2 LD 5#6-##7

To countei this, all x ieauings weie calibiateu using the following calculation:

(6)

wheie xo is the newly calibiateu value of x actually useu foi the pitch lookup table,
xR is the value of x ieau uiiectly fiom the acceleiometei anu y is the value ieau
fiom the y-acceleiometei.
+
z
-
z
+
x
-
x
y
-
y
+
= 45
= 45
!
x
a
=
x
r
cos(sin
"1
(y))
6S
Figuie Su shows the input space illustiating this calibiation effect.

U'6%-# @Q b-,BF 2C $,4'=-,&#5 " (,4%# 3'&F -#7B#$& &2 .
While this calibiation expiession uoes become unstable at extieme values of y, it
successfully stabilises the wiimote foi values foi !<6u.
9

Bespite this calibiation, the x value tenueu to be unpieuictable, especially on the
stiongei hits. To solve this pioblem, a simple 2
nu
oiuei low pass filtei was
implementeu to cuib some of the high fiequency noise. 4 filteis weie tiialleu
(1uBz, S Bz , SBz anu 1.SBz) with S Bz giving the best compiomise between
iesponsiveness anu filteiing of hanuling noise. The following giaphs show these
tiials anu the effects of each filtei. The 1uBz filtei has goou phase iesponse but it
tiacks the cuive too closely anu pioviues little use in stabilizing the signal. It can
be seen that as the cut off uiops, the phase shift begins to become a pioblem anu

9
Beiiving this calibiation foimula was a fai moie labouieu piocess than it shoulu have been. The
authoi spent a goou ueal of time evaluating thiee uimensional plots anu tiying to ueiive foimulae
anu ielations foi theii suifaces to get a suitable calibiation equation. Aftei some uiscussion about
this pioblem in a pioject meeting, the simple answei was quickly ueviseu by the pioject
supeivisoi. This seives as a useful lesson in a: not 'ovei thinking' simple pioblems anu b: the
viitues of having stubboin obstacles vieweu by a uiffeient paii of eyes.
1
u.8
u.6
u.4
u.2
u
u.2
u.4
u.6
u.8
1
1.2
1.4
1.6
1.8
2
u

u
.
u
S

u
.
1

u
.
1
S

u
.
2

u
.
2
S

u
.
S

u
.
S
S

u
.
4

u
.
4
S

u
.
S

u
.
S
S

u
.
6

u
.
6
S

u
.
7

u
.
7
S

u
.
8

u
.
8
S

u
.
9

u
.
9
S

1

.
]
,

]-
66
tiacking is uelayeu to such an extent that it becomes sluggish to play in the 1.S Bz
filtei.

U'6%-# @8 >15 2-5#- C'4&#- 3'&F , =-#,0 C-#R%#1$. 2C 8QAa C'4&#-'16 &F# " ,$$#4#-2+#&#-

U'6%-# @> >15 2-5#- C'4&#- 3'&F , =-#,0 C-#R%#1$. 2C DAa C'4&#-'16 &F# " ,$$#4#-2+#&#-
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
1 101 201 301 401 501 601 701 801
A
c
c
e
l
e
r
a
t
i
o
n

(
g
)

Samples
Raw Filtered
-1.4
-1.2
-1.0
-0.8
-0.6
-0.4
-0.2
0.0
0.2
0.4
0.6
1 101 201 301 401 501 601 701 801 901 1001
A
c
c
e
l
e
r
a
t
i
o
n

(
g
)

Sample
Raw Filtered
67

U'6%-# @@ >15 2-5#- C'4&#- 3'&F , =-#,0 C-#R%#1$. 2C @Aa C'4&#-'16 &F# " ,$$#4#-2+#&#-

U'6%-# @L >15 2-5#- C'4&#- 3'&F , =-#,0 C-#R%#1$. 2C 89D Aa C'4&#-'16 &F# " ,$$#4#-2+#&#-
Souice coue foi these filteis can be founu in Appenuix B1.
3.1.3.3 More plLch curves
Aftei playing the spheie foi some time, it became appaient that one of the key
techniques in hitting the notes at the euges of the Spheies iange was a flick
technique, a move to exaggeiate the acceleiation ieau fiom the x acceleiometei to
effectively push the ieauing to the outei limits of the system. To make the system
easiei to play, the iesolution of the centie iegion was uecieaseu, anu the outsiue
notes coulu only be playeu by the use of the flick technique (see Figuie SS).
-1.4
-1.2
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
1 101 201 301 401 501 601
A
c
c
e
l
e
r
a
t
i
o
n

(
g
)

Sample
Raw Filtered
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1 101 201 301 401 501 601 701 801 901 1001 1101 1201
A
c
c
e
l
e
r
a
t
i
o
n

(
g
)

Raw Filt
68

U'6%-# @D @ +2-# B'&$F $%-(#7
As well as auuing this flick technique to the equally spaceu angulai uistiibution,
two moie cuives weie uevelopeu, with a similai layout to the thumb haip. The
ioot note is locateu in the centie, with each successive note in moving outwaius.
Two vaiiations on this theme weie uevelopeu initially, one with only S notes anu
one with 7 allowing a fuithei jump above the octave. This allows a peual note to
be vampeu on with embellishments playable by tilting the wiimotes to the siue.
<#$#9 %&' Q'24/-6,- 012-'3 6,? MR=

U'6%-# @W EF# *BF#-# bT?

69
The fiont panel of the Spheie houseu the following contiols:
Select NIBI output uevice
Select Wiimote (if moie than one wiimote piesent on BIB list)
Select NIBI channel foi both the nunchuck anu wiimote
Select pitch uistiibution map
A pictoiial iepiesentation of cuiiently selecteu map
Select velocity cuive
The contiol map of the wiimote was as follows:
J21&-24 /&&,$F+#1&
X Acceleiometei (both hanus) Pitch uetection
Z Acceleiometei (both hanus) Note stiike uetection
}oystick X (Nunchuck) NIBI CC Su (Continuous u-127)
}oystick Y (Nunchuck) NIBI CC S1 (Continuous u-127)
A (Wiimote) NIBI CC 49 (0ff=u 0n=127)
B (Wiimote) }oystick onoff
C (Nunchuck) Nunchuck tianspose up 1 octave
Z (Nunchuck) Nunchuck tianspose uown 1 octave
+ (Wiimote) Whole instiument tianspose up 1 semitone
- (Wiimote) Whole instiument tianspose uown 1 semitone
0P (Wiimote) NIBI CC S2 (0ff=u 0n=127)
B0WN (Wiimote) NIBI CC SS (0ff=u 0n=127)
LEFT (Wiimote) NIBI CC S4 (0ff=u 0n=127)
RIuBT (Wiimote) NIBI CC SS (0ff=u 0n=127)
E,=4# > EF# *BF#-# $21&-24 +,B

7u
3.1.4.1 SofLware SLrucLure

U'6%-# @N EF# $4,77 7&-%$&%-# 2C &F# *BF#-#
NB: Foi a full list of methous anu vaiiable foi each class, along with the pioject
souice coue, please see appenuix B.
GUI
Main
WiiMidiDelegate
AccDetect PitchDetect
MIDIOut
Wiimote
71

U'6%-# @X !",+B4# 5,&, C423 '1 , &.B'$,4 7&,-& %B ,15 12&# 7&-'0# B-2$#5%-#

K28 @'+/0 M/*$/-
0ltimately, the output of this system neeueu to uiive a synthesizei of some kinu.
The financial constiaints of the pioject, meant that only one softwaie synthesis
system coulu be affoiueu by the pioject buuget, anu so the system chosen woulu
have to fit with the Spheie as well as the Cello.
<#"#$ 0-.),; 0-4?)+
The most obvious place to stait foi synthesisei to uiive was some kinu of physical
mouel (see section S.2.2) anu so Applieu Acoustics Systems 'Stiing Stuuio'
(Applieu Acoustics Systems 2uu9) was initially lookeu at to pioviue this. As one of
the only PC baseu stiing physical mouelleis available, its iealistic sounu anu
mouest piocessoi iequiiements maue it seem like an excellent choice. Aftei
uownloauing the uemo it became cleai that, while Stiing Stuuio sounueu gieat out
of the box, the mouel hau been almost exclusively uesigneu fiom the point of view
that it woulu be playeu with a NIBI keyboaiu. 0n it's own this is not an
unieasonable assumption to make, as the oveiwhelming majoiity of NIBI
72
contiolleis aie keyboaius, but foi this pioject - especially in the case of the Cello,
the ability to affect a note aftei it hau been tiiggeieu is of gieat impoitance. 0nce
a note on commanu is sent in Stiing Stuuio, theie is veiy little that can be uone to
the system while that note is sounuing. The usei is unable to change the bowing
speeu oi stiength until a note off commanu is sent. As the synthesis system hau to
be useu foi both the Spheie anu the Cello, puichasing stiing stuuio was eventually
iuleu out as an option.
Bespite this, a uemo veision was useu foi the usei testing with a hammeieu
uulcimei type sounu. The physical mouel woikeu paiticulaily well as the timbie
of the output changeu with the velocity input, making the instiument moie
engaging. 0ne limitation of this uemo was that the sounu cuts out at ianuom
inteivals, but it was founu that iunning two veisions of the instiument in paiallel
anu compiessing the output gave an uninteiiupteu consistent sounu with each
veision cutting out at uiffeient times.
<#"#" G&12)56/ :+?'/ ), Q'6B-+.
A Kaiplus-Stiong pluckeu stiing physical mouel was built in Reaktoi foi use with
the Spheie to avoiu the issues with the tiial veision of Stiing Stuuio.

U'6%-# @I *'+B4# c,-B4%7_*&-216 V#,0&2- B,&$F
<#"#7 %.6?)-)+,6/ 01,-&'2)2'. 0+4,?2
As well as using the iealistic physical mouels affoiueu by Stiing Stuuio, some
subtiactive anu FN synthesiseis weie useu to cieate playable sounus that weie
just as iewaiuing to play as the physical mouels.
7S
K2B 3D>J$/* 9-1:/$Q+-
The basic piemise is that the left hanu plays the bass notes with the nunchuck,
anu the iight hanu plays the highei notes with the wiimote. The technique foi
accuiately stiiking notes is quite specific anu iequiies a little bit of piactice to
achieve iepeatable iesults. The usei must enu each stiike with a veiy uelibeiate
anu stiong flick in the uesiieu note aiea. If the usei plays too flamboyantly
without this, the notes tenu to appeai ianuom uue to the way the wiimotes tiack
the pitch. Something that was appaient veiy eaily on in the uevelopment of this
instiument was that the latency of the Winuows auuio uiivei coulun't be
uecieaseu to acceptable times (<Sums) without auuio uiopouts. To bypass this,
an ASI0
1u
compatible sounucaiu was puichaseu to biing the latency uown to
unpeiceivable levels.
The shape anu weight of the nunchuck meant that the playing methou foi the left
hanu was quite uiffeient which was not easy to get useu to at fiist.

K2I #$<$&>&$'/%
The effectively small playing space uiun't allow foi a vey high iesolution of notes,
with the maximum peimissible being aiounu 7 notes in total befoie iepeatability
became a pioblem uue to inevitable hanuling uiift. With so few notes, any seiious
musical impiovisation tenus to be somewhat thiottleu, but with piactise anu
alteinate note setups (such as 12 notes mappeu acioss both hanus) it is thought
that moie piecise musical contiol coulu be gaineu. A 12 note spieau ovei both
hanus was tiieu, anu was initially quite uifficult. Bue to time constiains anu neeu
to begin woik on the Cello, a goou technique uiu not have time to be uevelopeu by
subjects anu was not incluueu in the usei testing as it was thought it might be 'too
much too soon' foi woulu be subjects.

1u
Auuio stieam inputoutput: a low latency auuio uiivei set foi Winuows
74
K2K R%-, )--06>1F >/0 S-7$%$'/% &' H-%$*/
When uesigning any system piimaiily uesigneu to inteiact with humans, usei
tiials aie vital. In this case, the Spheie was useu as a testing giounu, not only to
get useu to the necessaiy technical consiueiations when uesigning softwaie to
inteiface with the wiimote but to see how people got on with using the wiimotes
to play music. 1S subjects weie given the Spheie to tiy anu weie askeu some
questions on theii expeiience with the instiument.
Although obtaining quantifiable measuiements was one pait of these tests, it was
not the piimaiy goal. The main aim was to investigate how people ieacteu to this
instiument anu to get people to uiscuss theii expeiience of playing it.
The useis weie initially given the uevice without instiuction anu obseiveu to see
if they coulu figuie out its contiol schema by tiial anu eiioi. Aftei this, they weie
then given a shoit lesson anu given again to expeiiment with. Aftei five oi ten
minutes, they weie askeu a seiies of questions. This test is uetaileu below.
Any inteiesting comments anu suggestions weie noteu anu can be founu in the
pioceeuing section.
Below is a summaiy of the subjects testeu:
Subject Subject Backgiounu Playeu wii befoie.
1 Biummei (auvanceu) Yes
2 uuitai + Piano (playeu foi 1u yeais) Yes
S Electionic music piouuction, basic guitai anu keyboaiu Yes
4 uuitai anu bass (inteimeuiate) Yes
S Bass (playeu foi 4 yeais) Yes
6 Biummei (giaue 8) Yes
7 Singei + guitaiist (7 yeais) Yes
8 Peicussion + hanu uiums Yes
9 Tiombone (giaue 8) anu basic guitai Yes
1u Biummei (inteimeuiate) Yes
11 uuitai (auvanceu) Yes
12 B} Yes
1S Non-musician No
14 Singei anu flute (giaue S)

No
1S Non musician No
E,=4# @ S'7& 2C 7%=;#$&7 C2- *BF#-# &#7&'16
7S
<#<#$ S4'2-)+,,6).'
The useis weie given the Spheie initially loaueu with a hammeieu stiing physical
mouel, similai to a Chinese uulcimei. This was the sounu they useu foi the
majoiity of the test, anu weie not tolu about alteinatives until they hau completeu
the musical challenge (question S).
A filtei anu a uelay effect weie tieu to the A-button anu the joystick, but the
subjects weie not tolu about these functions until latei in the test foi feai of them
being uistiacteu. The test was baseu aiounu a numbei of questions anu small
tasks anu weie only seen by the authoi uuiing the testing so as not to give
anything away to the paiticipants. Initially, the u0I was hiuuen fiom the subjects
so as not to give any iueas about the opeiation of the Spheie befoie the fiist few
questions.
The test is piesenteu below with question specific uiscussion of iesults. The
iesults aie analyseu in the pioceeuing section:
1. 6lvF S0B}FCT TEF lNSTR0HFNT ANB SFF lI TEFY CAN IlNB lTS HFTE0B 0I
0PFRATl0N.
2. What uo you think is the ielationship between input anu output.
The puipose of this exeicise was to investigate how intuitive the instiument was
in its fiist instance. As the Spheie has no physical points of iefeience like a
tiauitional physical instiument, the authoi was inteiesteu to see how people
appioacheu the instiument, anu how much theii expeiience of pievious
instiumentation affecteu theii initial expeiiences.
All but subject 12 manageu to figuie out that the stiength of the hit hau some
beaiing on the volume yet only two of the subjects (2 anu 7) weie able to guess
the pitch was something to uo with iotation. Eveiy othei subject except subject
12 (who just waveu theii aims aiounu manically) tiieu to play the instiument
eithei longituuinally (like a piano) oi lateially (like a veitical uium kit). The
peicussive sounus being geneiateu by the instiument weie enough to uissuaue
subjects fiom tiying any thing continuous, anu none tiieu to play anything non
ballistic with the instiument.
76
Subject S hau seen the instiument in eaily uevelopment so was not subjecteu to
this task.
S. LET TBE S0B}ECT PLAY TBE SPBERE F0R FIvE T0 TEN NIN0TES T0 uET A
FEEL F0R TBE INSTR0NENT
4. What uo you imagine in fiont of you.
As the Spheie has no physical bouy, the authoi was inteiesteu in investigating
what people imagineu themselves playing, if anything. Results weie extiemely
vaiieu. Steel pans (7,8), squaie maiimbas (S) anu even floating colouieu spheies
(11) weie mentioneu in visualising the instiument in fiont of them.
S. 6RAPElCAl RFPRFSFNTATl0N 0I PlTCE C0RvF 0N 60l lS SE0WN T0 TEF
S0B}FCTS. Bow much easiei uo you finu it to play now.
The answei to this question was univeisal with one exception. All subjects except
6 saiu they founu the visualisation helpeu them a gieat ueal in playing the
instiument. This viitually unanimous accoiu shows cleaily how stiongly we iely
on visual cues when leaining new movement baseu piactices.
6. lFT S0B}FCT PlAY I0R S HlN0TFS TEFN ASK T0 PlAY TFARBR0P BY
HASSlvF ATTACK
The intiouuction to teaiuiop by Nassive Attack was chosen as it is a faiily simple
anu well known tune playeu on the pentatonic scale.

U'6%-# LQ fE#,-5-2Bf '1&-2 =. H,77'(# /&&,$0
Asking the subjects to go thiough a uefinite musical task iathei than aimlessly
impiovising woulu allow them to give a cleaiei iesponse when askeu about the
instiument latei. None of the subjects piofesseu not to know the tune, but iesults
vaiieu gieatly.
77
It was also a test to see how much the subject was willing to peiseveie with the
instiument to attain a musical goal. Impiovement on the Spheie, even in this veiy
shoit amount of time was noticeable in most of the canuiuates who stuck with the
task anu tiieu theii best to complete it. Subjects 1 anu 4 stuck it out until they hau
playeu the tune in full, anu coulu not be peisuaueu to stop until they hau got it.
Subjects 2, 7, 9 anu 1u got the vast majoiity of the tune anu then gave up aftei
stiuggling with one oi two notes. The iest of the subjects peifoimeu less well, anu
founu it uifficult to get in to the technique iequiieu to play the Spheie, subjects 8,
12, 1S, anu 1S giving up aftei a veiy shoit peiiou of time (>1 minute).
K?*JT**?<Z G?EA *TMg!JE* /M<TE :S/h?Zb !]:!V?!ZJ!
Aftei playing the instiument foi a shoit time, both fiee foim anu tiying to play
Figuie 4u, the subjects weie askeu to give theii opinions on the Spheie. Nany of
the comments by the useis congiegateu aiounu similai themes. The heauings
below iepiesent some of these common themes, with selecteu comments fiom
the subjects iepiouuceu below.
7. Biscussion: What uiu you like about this instiument.
EF'7 '7 , 12(#4 3,. &2 B4,. #4#$&-21'$ '17&-%+#1&7
CoulJ see myself usinq over o keyboorJ controller for certoin opplicotions
- Subject 1
0nlike ony otber HlBl controller lve useJ before - Subject S
very interestinq iJeo, new onJ excitinq - Subject 8
Tbis is totolly new, never ployeJ onytbinq like tbis before. Con imoqine lots of
possibilities - Subject 1u
CoulJnt qet tbe some results witb o keyboorJ interfoce - Subject S

/ -#7B217'(# ,15 '1(24('16 '17&-%+#1&
Host involvinq syntb lve ever ployeJ, you reolly qet tbe feelinq tbot tbe enerqy you
put in mokes o Jifference to tbe enerqy output - Subject 2
78
A reolly responsive onJ fun interfoce - Subject 7
'Seems versotile onJ sensitive to input - Subject 9
Tbe enerqy l put in is stronqly reloteJ to tbe sounJ l beor, tbis mokes it mucb more
expressive onJ involvinq - Subject 11
You reolly wont to put your wbole boJy into it - Subject 6
'lntuitive, volume sensitivity mokes it similor to o steel pon - Subject S
Ieels very noturol to ploy - Subject 1
'Reolly fun. You reolly bove to qet into it to ploy it properly - Subject 14
J2++#1&7 -#4,&'16 &2 3''+2&#7
Reolly qets you movinq, con relote to tbe wii console - Subject 1u
Ieels like ployinq o wii but witb more creotive scope - Subject 9
'Hucb better tbon otber wii instruments lve ployeJ " Subject 7
8. Biscussion: What uiun't you like about this instiument.
K'CC'$%4& &2 B4,.
This was a comment ieceiveu in some foim oi anothei fiom all but 4 of the
subjects (4,S,6,12). Subjects 8, 1u anu 1S went on to suggest that theii lack of
technique was solely uown to lack of piactice.
Subject 6 mentioneu that 'tbe left bonJ wos not os responsive os tbe riqbt anu
subject 4 suggesteu that the instiument 'coulJ qet o little limitinq.
Theie was no mention of any uispleasuie with the chosen sounu set oi the
tempoial iesponse of the instiument.

79
9. Biscussion: What woulu you change oi auu to this instiument.
/55 /1'+,&#5 ('7%,4 C##5=,$0
Subjects 1, 2, 4, 6, 9 anu 11 all maue comments aiounu the subject of wanting
visual feeuback beyonu that of the static image, so they coulu see wheie theii
hanus weie in ielation to the viitual semi ciicles befoie they hit a note.
H2-# 72%157 ,15 #CC#$&7
Subjects 6, 7, 9, 1u, anu 11 all saiu they woulu like to auu moie sounus, effects anu
featuies to the instiument, subject 1u specifically saying that they woulu like to
'bove tbe obility to quickly flick tbouqb pre-sets.
b-#,&#- 12&# -,16#
Subjects 7, 8 anu 9 expiesseu inteiest in having moie notes than the pentatonic
scale.
<&F#- $2++#1&7
Subject 2 saiu that they woulu like to see 'more Jimensions of movement to affect
the sounu, anu also expiesseu inteiest in using the infiaieu cameia of the wiimote
to contiol the sounu.
Subject 2 also saiu they woulu like to have a continuous pitch button foi sliues,
anu subject 6 expiesseu inteiest in a sustain moue.
Subject 14 mentioneu that some kinu of beqinners moJe woulu be a goou iuea to
get the veiy basics of the instiument befoie going on to moie auvanceu moues.
They also went on to suggest an 'Apple uaiage Banu 'u9' (Apple 2uu9) style how-
to lesson viueo foi beginneis.
N0LTIPLE CB0ICE Q0ESTI0NS
The subjects weie given a 1-S scale to place theii answei on, 1 being the most
negative iesponse, S being the most positive. A shoit uesciiption uesigneu to aiu
the subject in theii scoie selection was given with each question. They aie
uesciibeu below.
8u
1u. Bow iesponsive uiu you finu this instiument. In othei woius, how stiongly uo
you think the actions you peifoimeu coiielateu with the sounu you heaiu fiom
the speakeis (1-S).
1. None at all, seemeu totally ianuom
2. veiy loosely ielateu
S. Loosely ielateu
4. Nostly achieveu uesiieu output
S. Exactly achieveu uesiieu output
11. To what uegiee uo you think piactise woulu impiove youi peifoimance(1-S).
1. None, I coulu not impiove on this instiument
2. A little, but seems uifficult to see how I coulu impiove
S. I coulu see myself playing simple tunes but that's it
4. This instiument coulu be useu as a uefinite anu viable cieative tool
S. I coulu see myself playing this as well as any othei 'tiauitional' instiument
12. Bow much moie time woulu you be willing to invest in this instiument(1-S).
1. No moie time
S. I woulu take up this instiument anu leain it the same way as a 'tiauitional'
instiument
NB: Theie was no piompts foi 2,S oi 4 on this question. 0seis weie askeu to go
with theii gut feeling as an ovei all 'enthusiasm' iating foi the instiument.
81

U'6%-# L8 H#&-'$7 C-2+ *BF#-# %7#- C##5=,$0
1S. Woulu the ability to customise this instiument inteiest you.
All of the subjects except subject 14 saiu that they woulu be inteiesteu in
customising the instiument foi theii own uevices. Subject 14 expiesseu that theii
lack of computei anu electionic music knowleuge woulu piobably ienuei them
unable to unueitake customisation of an electionic instiument. This is an
inteiesting point foi uiscussion - how we enable extia cieative featuies to those
not familiai with the unueilying piinciples of a system. It is uiscusseu in S.S.2.
14. 0thei comments anu obseivations.
0ne inteiesting thing noteu in these tiials was that the uiummeis seemeu to finu
it haiuei initially to get useu to the instiument in the fiist instance. The authois
theoiy is that the technique of a goou uiummei heavily incoipoiates iebounu of a
uium skin into a stioke. In the case of the spheie, theie is no physical suiface to
pioviue this iebounu thus iesulting in an alien playing expeiience foi those useu
to instiuments playeu with mallets anu sticks.
Anothei inteiesting note was that eveiy subject lookeu intently at the image of
the pitch cuive on the scieen when tiying to peifoim Figuie 4u, illustiating the
impoitance of visual cues when leaining unfamiliai tasks.
u
1
2
S
4
S
6
7
8
9
1u
1 2 S 4 S
Responsiveness
Piactice
Time
82
<#<#" T,6/12)2 6,? F)25422)+, +> 0A&'.' R2'. D''?E65B
The analysis of iesults fiom these questionnaiies can be split into two uistinct
categoiies. The fiist is uesign consiueiations foi the Cello baseu upon uiiect
comments fiom the subjects. The seconu is a moie in uepth analysis into the
behaviouis anu comments given by the subjects to come to some basic
conclusions on the piacticalities anu implications of playing electionic
instiuments with wiimotes.
3.3.2.1 ulrecL commenLs
The fiist anu most obvious is the neeu foi some kinu of visual feeuback. 6 of the
subjects specifically mentioneu the neeu foi some kinu of animateu feeuback to
aiu in leaining the instiument, anu eveiy subject except one saiu that the pitch
cuive images maue the instiument easiei to play. 0seis also expiesseu the uesiie
foi moie sounus, anu that these sounus shoulu be quickly accessible, iemotely
selecteu fiom the wiimote. The wishes foi a gieatei iange of notes, sustain,
continuous pitch anu timbie contiol will be auuiesseu in the natuie of the Cello
instiument.
The comments ielating to moie effects anu sounus coulu be quickly iemeuieu by
tying moie NIBI commanus to the buttons on the wiimotes, anu by the usei
auuing these sounus anu effects in theii host of choice.
The laige numbei of subjects who iecogniseu that piactice woulu uefiantly
impiove theii peifoimance is encouiaging (mean of 4.S), as is the amount of time
the subjects saiu they woulu like to spenu with this ielatively simple instiument
(mean of S.2). The technique foi changing pitch whilst playing this instiument is
quite uiffeient fiom that of moie familiai malletstick instiuments uue to the fact
that absolute azimuth cannot be uetecteu by the wiimotes, foicing pitch to be
ueteimineu by ioll. This initially unintuitive ielationship possibly contiibuteu to
the laige numbeis of paiticipants answeiing 'S - The gestuies aie loosely ielateu'
when queiieu about the iesponsiveness of the instiument.
3.3.2.2 Analysls of oLher commenLs and daLa
The high numbei of subjects who expiesseu uispleasuie in the uifficulty of the
instiument is an inteiesting subject. Bespite these subjects complaining about the
8S
instiuments appaient uifficulty, they weie veiy quick to suggest new featuies anu
iequest moie notes. Any foimally uiawn conclusions foi this woulu neeu a much
moie in-uepth investigation than this stuuy has time foi, but, in the authois
opinion, this is a two pait issue:
Fiistly, the inteiface itself. 0ne of the main selling points of the Nintenuo Wii is
that in offeis an intuitive shallow leaining cuive entiy to viueo games anu viueo
game contiol. It coulu be that the subjects in this case (with all but subjects 1S
anu 14 having playeu a wii console befoie) expecteu a similai expeiience fiom the
instiument they weie tiying to play as they hau pieviously expeiienceu with the
Nintenuo wii, thus leauing to fiustiation at lack of iepeatability uue to impiopei
technique.
Seconuly, the vast majoiity of people who have playeu an electionic instiument
have uone so with a keyboaiu inteiface. Nouein synthesiseis tenu to be loaueu
with impiessive sounuing factoiy piesets that iequiie veiy little of the casual usei
to get an expansive, inteiesting sounu.
It is possible that this piioi expeiience with electionic instiuments anu viueo
games has let piospective useis to expect an 'easy iiue' when leaining a new
electionic instiument. It is also the authois opinion that the clamoui foi
numeious extia featuies anu effects aftei less than ten oi fifteen minutes
exposuie to this new instiument is symptomatic not only of the evei shoitening
cycles of technological innovation, but of the waning attention spans of the
geneiation that have been biought up with the fienzieu pace of 21st centuiy life.
The late Nichel Waisvisz (See S.S.2.2) mentioneu this phenomenon when
speaking about the new electionic inteifaces he useu to cieate. Be spoke of
'freezinq tecb Jevelopment (N. Waisvisz 1999) foi long peiious of time, in which
he woulu tiy to exploie an instiument to its limits befoie moving on the a moie
up to uate technology base.
The clamoui foi customisability cannot be ignoieu. People will want to customise
theii instiuments to allow them to uevelop inuiviuual sounus anu iueas, but to
maintain a sufficiently complex mapping stiuctuie, both inteiface anu sounu
souice must be uesigneu with each othei in minu. While alteiing souice coue is a
84
piactical solution foi the usei familiai with computei science, leaving options foi
customisability at such a low level woulu alienate the vast majoiity of the
instiuments usei base, counteimanuing the univeisal accessible appeal of the
wiimote foi musical puiposes. A solution is outlineu in 9.2.S.
K2T 5/D$/- )--06>1F
As well as conuucting usei tiials face to face with the Spheie, a viueo
uemonstiation was posteu on social netwoiking sites Youtube
11
anu Facebook
with links to a website pioviuing pioject binaiies anu souice coue foi people to
tiy (www.wiiuii.net). The iesponse was positive, with ovei Suu views in the fiist
few uays anu a healthy amount of encouiaging comments fiom both causal
onlookeis anu music technology enthusiasts alike.
K2U MNA>/%$'/ )'DD'?$/* R%-, 9,$>D%
As stateu in S.1.S.1, the angle of sepaiation of each note hau to be sufficiently
laige enough to ensuie goou iepeatability. This meant that theie was a maximum
of notes that coulu be placeu insiue each viitual semi ciicle. An iuea that was
implementeu onto the Spheie aftei usei tiials was a seconu tiei foi extia notes,
which initially came fiom wanting some extia space to play cymbals on a viitual
wiimote uium kit application foi the Spheie.
By applying a low pass filtei to the y acceleiometei on the nunchuck anu wiimote,
a stable value coulu be obtaineu ielating to the angle of pitch of each hanu,
cieating the illusion of a seconu playable tiei. When the y value exceeus a
thiesholu, the seconu iack of notes becomes available.

11
http:www.youtube.comwatch.v=Y0y-qtTk07s
8S

U'6%-# L> ?44%7&-,&'21 2C &32 &'#- 7BF#-#
With the combination of two peuals foi a kick uium anu hi-hat, a iuuimentaiy
uium kit can be set up with each tiei being useu to house the vaiious cymbals anu
uiums you woulu expect to finu at each level (cymbals on top, snaie anu toms at
bottom).
<#C#$ U-&'. G+22)E)/)-)'2 D+. JVA6,2)+, +> -&' 0A&'.'
0utsiue the numeious possibilities foi two oi thiee levelleu instiuments with
uiffeient tones anu effects iesiuing on each level, anothei place to take this
instiument woulu be to auu some ability to altei the notes aftei they hau been
playeu. In its cuiient state, this can be achieveu in any NIBI capable host by
assigning the contiols of the Spheie to sustain peual commanus, mouulation
uepths anu filteis, which leaves expansion open to any enu usei.
The incoipoiation of eithei the wiimotion plus oi infiaieu cameia woulu allow
foi uetectable movement in the 'living ioom plane'. This woulu allow foi an 'aii
maiimba' that was flat iathei than tilteu. If the playing aiea was fieeu up in this
way, stiokes with uiffeient oiientations coulu be useu to cieate uiffeient timbies.
K2V @+<<>,J '/ #-%%'/% #->,/-0 (', 9:- @A:-,-
The most significant uiscoveiies affecting the uesign of the Cello aie summaiiseu:
No absolute position can be obtaineu, meaning the plan foi vaiiables
affecteu by the position of the wiimotes in thiee uimensional space has to
be uioppeu anu ieplaceu with pitch anu ioll.
86
visual feeuback is a necessity anu will be implementeu on the Cello.
0seis want a multituue of easily accessible sounus anu effects
0btaining absolute velocity is impossible

87
T 9:- G-DD'
The Cello was the fiist instiument to be conceiveu to be maue foi this pioject, but
wasn't uevelopeu until aftei the much simplei Spheie hau been completeu. While
the Spheies velocity sensitivity maue it an engaging expeiience to play, its
ballistic natuie meant that theie wasn't veiy much in the way of sculpting oi
affecting the sounu aftei a note hau been playeu.
The Cello takes its name because of its loose association with boweu stiing
instiuments. The left hanu is useu to steei pitch like a fingeiboaiu on a cello, anu
the iight hanu is useu to inject eneigy, like a bow. This type of continuously
contiollable instiument is an excellent fit foi the motion sensois on the wiimote,
anu makes foi an expiessive anu compelling musical expeiience.
As the instiument is continuously contiollable, it follows that theie shoulu be a
gieatei uepth of uesign as the possible vaiiations of input will be vastly gieatei
than that of the Spheies simplei mapping scheme.
T2. G'/1-A&$'/
The fiist sketches anu iueas foi the wiimote contiolleu Cello weie heavily baseu
aiounu the physical actions anu sonic ieactions of the classical violoncello.
As uiscusseu in S.S, one of the key factois in making an instiument 'expiessive'
anu sonically inteiesting to listeneis is the nonlineai evolution of tone anu
tangible contiol ovei these nonlineaiities thiough peifoimance technique. Taking
the consiueiations laiu out in 4.1, the Cello woulu take some of the establisheu
founuations of instiument uesign anu tiy to complement them with the extia
uimensions of movement maue possible by the wiimote.
Coinciuing with the tiauitional wisuom on stiingeu instiument constiuction, pitch
woulu by contiolleu with the left hanu, with eneigy input being uealt with by the
iight. Fiom this basic mouel, it was also ueciueu that like eveiy othei physically
uiiven acoustic instiument, eneigy must be continuously injecteu into the system
to keep the sounu output active. This cieates a state of peipetual involvement,
88
wheie eveiy action the usei makes continuously affects the sounu thus making
both the playing anu listening expeiience moie engaging.
Fiom heie, some of the key vaiiable attiibutes of the violoncellos sonic output
weie listeu as possible stating points foi the wiimote contiolleu Cello. These
incluueu:
Pitch anu vibiato - Contiolleu with position of left hanu
'Shaipness' - Attack anu amount of high fiequency content in a note,
contiolleu by left hanu muting anu fingei piessuie
'Sciapeyness' - Tempoial anu haimonic jittei uue to length anu speeu of
tiavel of the stiing uue to fiiction coefficient of the bow. vaiieu mainly
with bow piessuie anu also with bow tilt
'ThicknessColoui' - Spectial content manifesting itself as iatio of low to
high fiequency content. vaiieu mainly with bow tilt anu speeu but also
with bow piessuie anu left hanu position.
'Squeekyness' - Amount of extia high fiequency haimonics biought about
by the position of the bow on the stiing in ielation to the biiuge. If the bow
is close to the biiuge oi at integei fiaction of stiing length away fiom the
biiuge, moie of these haimonics will cieep into the output.
With these sonic chaiacteiistics isolateu, the next task was to finu the best ways
of sepaiating them into system components that coulu be contiolleu via the
wiimotes. Theie weie thiee appioaches iuentifieu, all with theii own meiits:
Auuitivesubtiactive synthesis - Each paitial is inuiviuually manipulateu
by a seiies of tianslation equations feu by vaiious aspects of the wiimote
uata stieam. The sonic output is a combination of these multiple oscillatoi
blocks. While slightly complex in the fiist instance (in teims of ueiiving
these tianslation equations), quick changes can be maue to each small
section once the infiastiuctuie is in place.
Physical mouelling - Things like 'bow position' anu 'bow speeu' can be
uiiectly contiolleu by faiily obvious mapping connections fiom the
wiimotes uata stieam (eg total eneigy input = bow velocity). Bue to the
natuie of theii feeuback loop style uesign, physical mouelling systems
89
opeiate on the biink of stability making quick changes to the topology a
lengthy piocess.
uianulai synthesis - Real life iecoiuings aie taken anu choppeu into veiy
small sections. These small sections aie inuiviuually piocesseu anu feu to
the output stieam with a set amount of tempoial jittei, giving the
impiession of a continuous stieam of auuio. While the ieal auuio content of
the auuio in the gianulai synthesisei is iecognisable anu (uepenuing on
the initial iecoiuing) sonically uynamic anu 'inteiesting', it is only as goou
as the souice mateiial being feu into the input.
Effects - Things like satuiation anu filteiing contiolleu by inteipietations
of the motion uata.
As well as these thiee, FN synthesis was also iuentifieu anu expeiimenteu
with, but was eventually cast asiue. The iesultant sonic palette enueu up being
iich enough with the above thiee sounu souices, anu may have gotten too
muuuleu hau anothei type of synthesis been injecteu into the output stieam.
@#$#$ 0+4,? 0+4.5' 6,? 01,-&'2)2
Feeuback fiom the Spheie attesteu to the uesiie of useis foi 'moie sounus'. As
well as the obvious inclusion of the option to change the basic sounu of the
instiument with the buttons on the wiimote, a moie fluiu system of tonal
manipulation was also incluueu in the funuamentals of the Cello. By having
motion in the x plane affect one half of the sounu engine, anu motion in the z
plane affect the othei, a ciossfaue effect coulu be obtaineu between these two
halves by applying piopei technique, giving an immeuiate anu continuous giasp
on the majoi tonal output.
The initial conception was that the fiist half of the sounu engine woulu house a
physical mouel synthesising a boweu stiing anu the seconu half woulu house the
auuitive synthesisei. The final veision of the Cello was consistent with this, but
with the auuition of a gianulai synthesisei to each 'siue'. Both these sounu banks
weie also to be uoubleu up to allow foi a haimony to be playeu.
9u
In auuition to these two sounu banks, two post sounu bank effects weie auueu: a
moiphing filtei anu a satuiatoi which weie contiolleu by the upuown tilt of the
wiimote anu nunchuck iespectively.

U'6%-# L@ J#442 ,$$#4#-2+#&#- 2%&B%&7 d 3''+2&# '+,6# C-2+ 33397&%,-&12-+,19$2+e 1%1$F%$0 C-2+
Z'1&#152 B-#77 -#4#,7#

T28 "/&-,A,-&>&$'/ '( M/-,*J "/A+&
Both the wiimote anu the nunchuck contain a tii-axis motion sensing
acceleiometei packages, anu the fiist pait of the uesign phase was to ueteimine
the best way to use the uata fiom these sensois to give a playable musical
expeiience to the usei.
As stateu in 6.1, the acceleiometeis chosen to uiive the eneigy input to the sounu
engine weie the x anu z acceleiometeis on the iight hanu wiimote. 0sing
acceleiation alone to uiiectly uiive any of the inputs on this system howevei, is a
little inflexible. In the 'tiauitional' musical woilu, it is the velocity anu
uisplacement of the excitei that uiiectly uiives the iesonatoi (speeu of aii moving
thiough a tiumpet, speeu of bow on a stiing etc), not its acceleiation.
As mentioneu in S.1.2.1, velocity cannot be uiiectly obtaineu fiom the S
acceleiometeis insiue the wiimote because of the evei piesent effect of giavity
91
constantly obscuiing any attempt to accuiately uetect velocity. As in the case of
the Spheie, a woik aiounu to this hau to be cieateu to give an appioximation to
velocity accuiate enough to give the usei the impiession it was velocity uiiving
the system, not acceleiation.
In the fiist instance, filteis weie expeiimenteu with. In theoiy, a filtei with a veiy
fast iise time anu long tail coulu be useu to keep the peiceiveu velocity value high
enough between stiokes. Initial expeiiments with banu-pass iesonatois piouuceu
pooi iesults (see Figuie 44), anu so a moie uiiect appioach was employeu.

U'6%-# LL <%&B%& 2C , C,'4#5 =,15B,77 C'4&#- #"B#-'+#1& ,&&#+B&'16 &2 7+22&F ,$$#4#-,&'21
The methou useu in the Cello exploits the fact that the uistance of bow tiavel was
only going to be as long as the useis natuial comfoitable ieach, ioughly a metei.
The acceleiometei value is iectifieu, high pass filteieu (to iemove BC offset
causeu by giavity) anu put thiough a latch anu holu type system to keep the
velocity up between peaks, the wiuth of the bow stioke being small enough so
that it can be sufficiently 'toppeu up' on the ieveise stioke.
-1
0
1
2
3
4
5
6
1 101 201 301 401 501 601 701 801 901
A
c
c
e
l
e
i
a
t
i
o
n

(
g
)

Sample
Banu-pass: C1 = u.4 Bz C2 = 1Bz Raw |x|+ |z|
92

U'6%-# LD S,&$F ,15 F245 7.7&#+ '1 ,$&'21 3'&F &F# 7423#7& -#7'7&'('&. 7#&&'16 &2 '44%7&-,&# &F# 7.7&#+
Theie aie thiee vaiiables in effect heie. excitotion_in is the absolute value of a
given acceleiometei ieau fiom the wiimote eveiy sampling inteival (the ieu line
in Figuie 4S). lotcbeJ_excitotion is the value outputteu fiom the system (blue line
on Figuie 4S). It tiacks the acceleiation peifectly on upwaiu peaks but ieclines
giauually between them, allowing time foi it to be toppeu up on the ieveise
stioke. iesistivity is a vaiiable set on the u0I fiont panel of the instiument, anu
allows foi a vaiiable instiument iesponse which can be set by the usei to
piefeience.
If(excitation_in > latched_excitation) or (excitation = latched_excitation)
latched_excitation = excitation_in;
If(excitation_in<latched_excitation)
latched_excitation=latched_excitation-resistivity;
return latched_excitation;

T2B 3$&1: H-&-1&$'/
As stateu in 6.1, the pitch neeueu to be easily switchable between continuous anu
uisciete pitch moues with smooth changes between the two. Like most
conventional instiuments, pitch was to be assigneu to the left hanu (the
nunchuck), with the ioll action of the nunchuck (see Figuie 4S) changing the
u
1
2
S
4
S
6
1 1u1 2u1 Su1 4u1 Su1 6u1 7u1
A
c
c
e
l
e
i
a
t
i
o
n

(
g
)

Samples
Piocesseu
Raw
9S
pitch. The obseivant ieauei may notice that having the pitch movement
effectively in paiallel to the main motion of the eneigy input uoes in fact
contiauict one of the pioject specifications - that of taking cues fiom establisheu
instiuments to aiu leaining. Common wisuom ueciees that contiol pitch is most
often oithogonal to eneigy input. When the piimaiy pitch of the instiument was
attacheu to the upuown tilt of the nunchuck, the outei limits (uiiectly pointing
up anu uiiectly uown) weie iathei uncomfoitable to ieach. Putting pitch on the
ioll simply maue foi a much moie comfoitable playing expeiience. The fact that
eneigy injection coulu be in an upuown motion as well as leftiight also ieliveu
some of the possible confusions that might have iesulteu when new useis came to
tiy this instiument.
uetting uisciete pitch oi continuous pitch to woik inuepenuently is a stiaight
foiwaiu task. With the foimei, one can ieau off a value fiom a filteieu
acceleiometei anu apply it to eithei a simple tianslation foimula oi lookup table.
With the latei, the note on commanu coulu senu the same note each time, anu the
twist woulu just give out a pitch benu, sent with the note on commanu. Combining
the two iequiieu a slightly moie consiueieu appioach.
When a playei activates the continuous contiollei moue, the pitch benu neeus to
be set as centie. All fuithei changes while the continuous pitch moue is on neeu to
be ielative to this stait point. When the continuous pitch button is ieleaseu, the
note activateu on ielease neeus to be in the same pitch as the one that pitch
benueu to upon ielease of the continuous pitch function.
94

U'6%-# LW ?44%7&-,&'21 2C 12&#_21 ,15 B'&$F =#15 C%1$&'21
The pitch uetection system on the Cello cannot go lowei than the C note (foi
ieasons mentioneu in S.1.S) meaning that only V of the full iesolution of the pitch
benu system can be useu (ie - the laigest uistance one can move is fiom C
position to F# oi vice veise, in this case, the iange woulu be eithei 8191 - 16S8S
oi 8191 -u). Bespite this loweiing of pitch benu iesolution, 81927 = 117u steps
pei note is moie than auequate to ensuie a fluiu pitch benu.
6.3.1.1 noLe 8ange
Eailiei expeiiments with the Spheie showeu that theie was a limit of aiounu five
to seven notes that coulu featuie in the viitual cuive suiiounuing the nunchuck.
The note iange was maue scalable on the fiont panel of the Cello, but the
optimum iange seemeu to be 7 semi-tones. Any gieatei iesolution anu the
movements of the iight aim useu to keep the instiument active began to inuuce
involuntaiy movements in the left aim uue to bouy movement, affecting
intonation.
9S
6.3.1.2 8eflnlng lLch ueLecLlon
Aftei using the Cello in the fiist instance, one of the common pioblems was that
pitch tenueu to be veiy haiu to maintain at both the lowest anu highest note when
the x acceleiometei in the nunchuck was facing uiiectly uown oi uiiectly up uue
to the possibility of 'going past' the peifectly flat state to !>18u. When this
happens, the x acceleiometei begins to see the acceleiation uue to giavity
uecieasing, anu inteipiets as a uecieasing value of ! (see Figuie 26). The as yet
unuseu z acceleiometei was useu to stabilise the pitch.
At values of z close to u, the nunchuck is almost hoiizontal. At these values, a
system is useu to ueteimine if the nunchuck is veiy fai left oi veiy fai to the iight
by looking at the sign of x. When this is ueteimineu, the pitch is lockeu to the
outei euge anu uoesn't fluttei between the last anu seconu to last note when the
iight aims is moveu vigoiously. Pseuuo coue is uesciibeu below.
if (z close to horizontal)
{if (angle < (lowest_note_angle+error_margin))
output = lowest_note;
if (angle > (highest_note-error_margin))
output = highest_note;
}
@#7#" W+12-)5B X6.3+,1 G)-5&
To give the instiument moie flexibility, the joystick on the nunchuck was useu as
a continuous haimony contiollei. The two 8-bit values outputteu by the joystick
woulu neeu to be changeu to polai cooiuinates to allow foi the iauius to be useu
foi volume of the haimony note anu the angle to be useu to ueteimine the pitch of
the haimony. As the pitch benu commanu was alieauy in use foi the main note,
two contiol change channels weie useu to senu the volume anu pitch of the
haimony.
The joystick on the nunchuck is uesigneu with 8 equally spaceu notches, which
allow foi quick anu easy placement by a piospective usei - vital in this case as
intonation in haimony is veiy impoitant. With 12 possible notes anu only 8
spaces, the most uissonant haimonies weie left in between the notches, with bIII,
III, vI, v, vI, vII bvII anu the octave being useu foi each notch.
96

U'6%-# LN Z%1$F%$0 F,-+21. B'&$F ,442$,&'21
0nce the X anu Y joystick values hau been conveiteu to polai cooiuinates they
weie iotateu by 9u anti-clockwise (to set u as 'noith' iathei than 'east') anu
applieu to a look up table to ueteimine the appiopiiate tianslation equation to get
the iight haimony.
N0TE: 0n testing the nunchuck, it was founu that the seconu notch ietuins an
angle of appioximately 47, not 4S as one woulu expect. This eiioi was also
founu in a seconu nunchuck puichaseu S months aftei the fiist one testeu, anu
thus it can be assumeu that this slight ueviation is common to all nunchucks.
Eveiy othei notch on the joystick aiea ietuins the expecteu angle.
8va
bIII
III
IV
V
VI
bVII
VII
0
2
3
2
97

Inteival
!
Tianslation Foimula NIBI Range
u ! bIII (S)
u !
!
"
4

!
pitchbend =
"
#
4
$ 31.75
(S)

u ! S1.7S
bIII ! III
(1)
!
"
4
!
!
"
2

S1.7S ! 42
III ! Iv (1)
!
"
2
!
!
3"
4

!
pitchbend = 31.75 +
" #
$
4
$ 2
%21.16
&
'
(
(
)
*
+
+
(7)

42 ! S2
Iv !v (2)
!
3"
4
!
!
"
S2 ! 74
v ! vI (2)
!
" !
!
5"
4

!
pitchbend = 52 +
" #
3$
4
$
2
% 42.3
&
'
(
(
)
*
+
+
(8)

74 ! 9S
vI ! bvII
(1)
!
5"
4
!
!
3"
2

9S ! 1uS
bvII ! vII
(1)
!
3"
2
!
!
7"
4

1uS ! 116
vII ! 8va
(1)
!
7"
4
!
!
2"
!
pitchbend = 95 +
" #
5$
4
$
2
%21.16
&
'
(
(
)
*
+
+
(9)

116 ! 127
E,=4# L g2.7&'$0 &-,174,&'21 C2-+%4,
98

T2I "/%&,+<-/& P/>DJ%$%
To obtain fuithei uepth to both the sonic output anu playing technique, some
'tiauitional' musical instiuments weie iecoiueu anu stuuieu. This uata was then
analyseu, the iesults being applieu to the final contiol map of the Cello.
Instiumentalists weie iecoiueu anu askeu to play a single note, staiting as quietly
as possible, anu piogiessively getting louuei. The aim of this was to investigate
how the sounus of these vaiious instiuments evolve with iespect to velocity, anu
if any common themes coulu be founu. The following instiuments weie chosen foi
two ieasons: The availability of instiumentalists, anu the fact that each of these
instiuments coulu continuously play a note anu keep it sustaineu (thus
uiscounting the guitai foi example).
Cello
Bouble bass
Flute
Buman voice
violin
In the case of the boweu instiuments, the instiumentalists weie askeu to uo the
following in auuition to an inciease in bow piessuie:
Stait next to biiuge anu continue moueiate bowing, moving away fiom
biiuge
Stait with bow tilteu at highest angle with minimum bow to stiing contact,
anu play with moueiate piessuie, uecieeing bow tilt until peifectly
oithogonal to stiing with maximum bow to stiing contact
The uata fiom these iecoiuings weie analyseu in Natlab to piouuce some fast
Fouiiei tiansfoim (FFT) spectiogiams. These spectiogiams weie analyseu both
qualitatively, to attain common themes anu tienus, uifficult to compaie
numeiically, anu quantitatively, to piouuce tianslation equations foi the auuitive
synthesisei bank.
99
Bue to time constiaints, it has not been possible to conuuct a full stuuy involving
the accuiate measuiing of vaiiables such as absolute bow piessuie anu specific
angle of tilt, anu how these mathematically ielate to sounu piessuie level. The
following uata was collecteu anu analyseu with a view to getting a 'geneial iuea' of
the factois at play in ieal instiuments, anu as such, the iesults shoulu not be taken
as a iigoious scientific pioof.
Rathei than piesent this uata as a whole in an isolateu section of its own, it is
piesenteu along siue the ielevant aspects of the synthesisei uesign which weie
inspiieu by the iesults of this instiument analysis.
Not all of the uata containeu useful infoimation that was useu in the final piouuct.
In fact, out of all the spectial analysis, only the cello anu flutes uata was uiiectly
useu to piouuce the mathematical ielations between input anu output. This was
simply uue to the fact that the iesulting spectiogiams fiom the othei souices uiu
not contain any easily iuentifiable tienus. Bespite not being easily analyseu in the
fiequency uomain, auuible changes in the sounu of the uouble bass anu violin
iecoiuing weie useu as inspiiation foi some of the featuies on the Cello. All of the
iecoiuings eventually founu theii way into the instiument via the gianulai
synthesisei which will be uesciibeu in u.
These iecoiuings can be founu on the accompanying CB.
T2K @'+/0 M/*$/-
The uesign anu iealisation of the sounu engine uesigneu foi use with the Cello is
piesenteu. The section lists the uesign of the physical mouel, auuitive synthesisei,
gianulai synthesisei anu effects sections of this sounu engine anu how they aie
manipulateu by the contiol system of the Cello.
@#<#$ G&12)56/ :+?'/
As uiscusseu in S.2.2, physical mouelling techniques can be useu to cieate some
highly auaptive anu veisatile synthesis algoiithms, anu as such, it was ueciueu
fiom the outset that the Cellos sounu engine shoulu incluue this technology.
The Native Instiuments Reaktoi foiums anu usei libiaiy (Native Instiuments
2uu9) is a thiiving community, pioviuing a common iepositoiy foi Reaktoi useis
1uu
aiounu the woilu to uploau theii Reaktoi patches to. Initially, iathei than
ueveloping an algoiithm fiom sciatch, the Reaktoi usei libiaiy was seaicheu to
see if a physical mouelling synthesisei fit foi use with the Cello coulu be useu oi
auapteu foi use. While theie weie in fact a goou numbei of physical mouelling
synthesiseis, theie uiun't seem to be one which woulu have been suiteu to the
Cello without veiy laige mouifications. Aftei this biief peiiou of expeiimentation,
it was ueciueu that a physical mouelling synthesisei woulu be built in Reaktoi
fiom sciatch.
}ulius Smith's boweu stiing topology (Smith 2uu8) woulu be useu anu auapteu
foi use with Reaktoi.
6.3.1.1 ueslgn of 8owed SLrlng hyslcal Model
lnformotion in tbis section {6.S.1.1) reloyeJ onJ oJopteJ from
http:ccima.stanfoiu.euu~jospaspBoweu_Stiings.html (Smith 2uu8)
0n a physical cello, the bow is placeu pait way up the stiing anu inuuces
uiiectional waves both towaiu the biiuge anu towaius the nut. The boweu stiing
physical mouel iepiesenteu below aims to emulate this uiviuing of the stiing by
using 4 uelay blocks to iepiesent the waves tiavelling in each uiiection on both
sections of the stiing uiviueu by the bow(see Figuie 48).
The left anu iight moving tiavelling waves aie uenoteu by
!
v
sl
+
n
( )
anu
!
v
sl
"
n
( )
to the
left of the bow, anu
!
v
sr
+
n
( )
anu
!
v
sr
"
n
( )
to the iight.
The '+' supeisciipt inuicates waves tiavelling into the bow contact point, the '-'
inuicates waves tiavelling away.
The system is piimaiily affecteu by bow velocity vb, anu as such, the system uses
velocity waves as the caiiiei in the uelay lines.

1u1

U'6%-# LX M23#5 7&-'16 BF.7'$,4 +25#4 &2B2426. _ U-2+ F&&B`^^$$-+,97&,1C2-59#5%î;27^
The total stiing velocity at the bow is the sum of the signals tiavelling in both
uiiections at the point of contact at time (n), uefineu as:
!
v
s
+
n
( )
= v
s
+
l
(n) + v
s
+
r
(n) (1u)
When bow velocity is auueu, the incoming uiffeiential bow velocity is uefineu as:
!
v
"
+
(n) = v
b
(n) #v
s
+
(n)
(11)

This uiffeiential bow velocity is a iepiesentation of the uiffeience in velocity if the
bow anu stiing weie peifectly smooth with no fiiction. Cleaily, a fiictionless
system woulu piouuce no sounu at all, so the stick anu slip action of the bow on
the stiing is mouelleu with a ieflection coefficient
!

" , which is ueteimineu by use
of a non-linei lookup table oi segmenteu polynomial. This ieflection coefficient
iepiesents the velocity waves ieflecteu in the stiing fiom the bow. If
!

" =1,
velocity of the bow is miiioieu by the stiing - ie, it is totally stuck to it. If
!

" < 1,
the stiing is slipping against the bow.
The velocity wave moving away fiom the bowstiing contact point can be
equateu to the velocity wave enteiing fiom the opposite siue, plus the uiffeiential
bowstiing velocity multiplieu by some co-efficient.
!
v
sr
+
= v
sl
+
+

" (v
#
+
) $ v
#
+
(12)

!
v
sl
+
= v
sr
+
+

" (v
#
+
) $ v
#
+
(1S)

1u2

U'6%-# LI *'+B4'C'#5 -#C4#$&'21 $2#CC'$'#1& &-,17C#- C%1$&'21 d U-2+
F&&B`^^$$-+,97&,1C2-59#5%î;27^B,7B^M23j*&-'16j*$,&&#-'16jg%1$&'219F&+4
The cential aiea wheie
!

" (v
#
+
) =1 iepiesents the situation wheie the uiffeiential
velocity is low enough foi the bow to completely stick to the stiing. The sloping
outei sections iepiesent the ieuuceu sticking effect.
vaiying the size of the cential
!

" (v
#
+
) =1

section changes the bow piessuie (ie -
moie piessuie = moie fiiction anu thus a laigei 'stick' iange).
Like the physical mouel in S.2.2.2, the simulateu losses at the biiuge anu nut aie
lumpeu into one filtei block at the biiuge teimination, anu the pitch is contiolleu
in the usual way, by vaiying the total gioup uelay.
6.3.1.2 lmplemenLaLlon ln 8eakLor
The system was spilt into two mouules: The uiiving bow anu the iesonating
stiing.
The bow uiivei wave is pieuominantly maue up of a sawtooth wave whose pitch
is vaiieu with the pitch of the uelay lines. To get the uiivei to sounu moie like a
ieal bow, pitch anu amplituue jittei weie auueu to the sawtooth wave bow signal
to simulate haii jittei.
1uS

U'6%-# DQ M23 C2- BF.7'$,4 +25#4 '1 V#,0&2-
This signal was then feu into the stiing iesonatoi shown in Figuie S1. The
scatteiing junction is shown in Figuie S2

U'6%-# D8 *&-'16 -#721,&2- C##5=,$0 422Be C#5 =. =23

U'6%-# D> M23#5 7&-'16 7$,&&#-'16 ;%1$&'21 V#,0&2- +25%4#
A pioblem encounteieu when ueveloping Figuie S2 involving the peculiaiities of
the Reaktoi Coie BSP piogiamming level involving the way in which Reaktoi
hanules events. If a thiee junction meige mouule is useu in the above case iathei
than a cascaueu set of two junction meige mouules, the value of
!

" (v
#
+
) tenueu to
stick at 1, causing the physical mouel to peipetually oscillate anu become unstable
1u4
veiy quickly. This elusive pioblem was spotteu by a membei of the Reaktoi usei
foiums, anu coulu not have been solveu without the uetaileu knowleuge of event
hanuling in Reaktoi Coie.
The tone of the ieactoi patch coulu be alteieu with the vaiiables shown in Figuie
SS, but only bow velocity was tieu the wiimote eneigy output.

U'6%-# D@ M23#5 7&-'16 BF.7'$,4 +25#4 V#,0&2- $21&-24 B,1#4

Iueally, the twist of the iight hanu coulu have been tieu to bow foice, anu the yaw
of the wiimote coulu have been useu foi bow position, but this is unfoitunately
impossible without auuitional acceleiometeis anu gyioscopes foi ieasons set out
in 2.2.4 anu S.1.S . Fuithei tactile contiol ovei physical mouels woulu be an
inteiesting aiea foi fuithei investigation following the ielease of the Wii Notion
Plus (see 2.S).

@#<#" T??)-)I' 01,-&'2)2'.
As stateu in section 6, one of the aims in uesigning the Cello was to tie in some of
the iiiegulaiities anu non-lineai elements that aie inheient in the physicality of
ieal instiuments. It is the authois opinion that these nonlineaiities aie what gives
an instiument an exciting, expiessive anu ultimately non-synthetic sounuing tone.
A timbie of a tiumpet blown softly is veiy uiffeient to the timbie of a tiumpet
blown in the stanuaiu iange, which in tuin is veiy uiffeient to a tiumpet blown at
full powei. At the extiemes of the uynamic iange, oveitones begin to appeai anu
high fiequency haimonic content becomes moie pionounceu. Simply tying
velocity to volume woulun't come veiy close in making a sounu engine that was
1uS
an inteiesting playable expeiience. As well as the physical mouel anu gianulai
synthesisei which woulu all help to accomplish this velocity uiiven timbial
change, auuitive synthesis woulu be useu, the amplituue of its haimonics being
uiiven by non-lineai equations baseu on the spectial analysis of ieal instiuments.
6.3.2.1 vloloncello and lluLe
A violoncello playei was iecoiueu anu askeu to play a u note (open seconu
stiing), staiting veiy softly anu giauually incieasing bow piessuie. The iesults
fiom this analysis was then useu to ueteimine the equations of the auuitive
synthesisei. The oiiginal iecoiuings along with Natlab uata can be founu on the
accompanying CB.

U'6%-# DL Y'2421$#442` A,-+21'$ $21&#1& 3'&F '1$-#,7'16 B-#77%-# C-2+ &kQ &2 &k@Q
1u6
The inuiviuual haimonics in Figuie S4 weie isolateu anu conuenseu into a two
uimensional foim (Figuie SS) so the line of best fit foi the magnituue of each
haimonic with iespect to bow piessuie coulu be easily ueteimineu.

U'6%-# DD Y'2421$#4429 >K ('#3 2C F,-+21'$79 M23 B-#77%-# '7 '1$-#,7#5 C-2+ &kQ &2 &k@Q9
Fiom Figuie SS, the following equations of best fit weie extiacteu.1
st
anu S
iu

haimonic:
!
y =
e
x"6
8
10
(14)

2
nu
Baimonic: Lineai:
!
y = x
(1S)

4
th
Baimonic:
!
y = 1"
x "60
10
#
$
%
&
'
(
2
#
$
%
%
&
'
(
(
+ 30
(16)

U'6%-# DW EF# 87&^@-5e >15 ,15 L&F F,-+21'$ 4'1#7
1u7
At the uppei velocity iange, haimonics S anu 6 seem to jump slightly, although it
is not quite convincing oi pionounceu enough to estimate a line of best fit.
A flutist was askeu to peifoim a similai task to that of the cello playei: stait with a
B note (in the miuule of the iange of a flute) anu giauually inciease bieath powei.
Figuie S7 Flute, incieasing bieath poweiFiguie S7 was obtaineu.

U'6%-# DN U4%&#e '1$-#,7'16 =-#,&F B23#-
The tone of a flute is uominateu by a single funuamental which gives it its almost
sine wave like tone. This haimonic ielationship changes little until the uppei
velocity iegion. It can be seen on Figuie SS that at veiy high bieath poweis,
stiong oveitones begin to appeai at the output. This phenomenon was miiioieu
on the Cello, with the S
th
anu 6
th
haimonic being goveineu by the following
equations.
S
th
anu 6
th
Baimonic: foi
!
y "120 : y = 0: foi
!
y >120 : y = x
1u8

U'6%-# DX V#,0&2- ,55'&'(# 7.1&F#7'7 =42$0 5-'(#1 =. &F# N F,-+21'$ ,+B4'&%5# '1B%&7
1u9

@#<#7 M.6,4/6. 01,-&'2)2'.
In piototyping, it was cleai that extia sonic eniichment was neeueu in auuition to
that of the physical mouel anu auuitive synthesisei. Aftei uoing the instiument
analysis expeiiments, a laige bank of vaiious instiument samples hau been built
up, in which the peifoimeis hau staiteu playing veiy softly, anu gotten giauually
louuei. These samples weie peifect mateiial to use foi this instiument anu woulu
pioviue the extia uimension neeueu to give the instiument a layei of 'ieal' sounu
not baseu puiely on synthesis. Allowing a usei to loau in theii own samples also
gave an auueu uepth of customisability to the instiument.
The eneigy input fiom the wiimote was tianslateu to the position of the playback
heau on the gianulai sample playei. The highei the eneigy, the fuithei up the
sample the play heau woulu move, anu the louuei anu moie intense the sounu
woulu be uue to the natuie of the iecoiueu mateiial.
At a given velocity, the gianulai synthesisei plays back the giains at
Pos+uiainLen. If theie is any auuio at the stait of a file, it will be playeu back at
zeio velocity. Because of this, some of these iecoiuings hau to be euiteu foi use
with the Cello to ensuie that theie was total silence at the beginning of the
iecoiuings, anu the instiument woulu not sounu when the usei was motionless.

U'6%-# DI b-,1%4,- 7.1&F#7'7#- +25%4# 7&-%$&%-#
11u

@#<#9 J>>'5-2
Foi the final subtle touches to the instiument, some effects weie auueu to the
output bus to give extia uepth to the output sounu.
6.3.4.1 Morphlng lllLer
A filtei unit was uesigneu in Reaktoi, initially to tiy to mimic some of the effects of
'bow tilt' on a ieal boweu instiument.
Rathei than use a single banupass filtei to accomplish this, two filteis weie placeu
in paiallel with vaiiable gains to give a miu section with a flat iesponse to avoiu
banu limiting the signal when filteiing was not wanteu.

U'6%-# WQ V#7B217# 2C +2-BF'16 C'4&#- 3'&F 2-'#1&,&'21 2C &F# 3''+2&#
As well an oveiall tone contiol affecteu by the low pass filteieu upuown pitch of
the wiimote, its centie fiequency was also affecteu by ianuom jittei which was
baseu upon the total input eneigy fiom the wiimote. The concept of a moie
chaotic uppei velocity chaiacteiistic was inspiieu by listening to the iecoiuings of
the instiuments suiveyeu foi this stuuy.
Buiing the eaily usei tiials, it was noticeu that at veiy high velocities, the
instiument woulu cut out anu loose volume. Aftei isolating each pait of the
instiument anu iepeating the cause of the fault, it was founu the moiphing filtei
was peifoiming this cut.
When the wiimote was being shaken veiy aggiessively the signal on the y-axis
began to look like a high fiequency wave with veiy little low fiequency content.
Low centre freq
LP Gain >> HP Gain
Mid centre freq
LP Gain = HP Gain
High centre freq
LP Gain << HP Gain
111
When put thiough the low pass filtei on this acceleiometei (piimaiily looking foi
giavity, a BC signal), this iaw acceleiometei signal was gieatly attenuateu,
iesulting in a veiy low numbei being piesent at the input of the moiphing filteis
cutoff fiequency. What this effectively uiu was lowei the fiequency of the filtei
uown to aiounu 1u-2uBz, below the human iange of heaiing, obliteiating the
signal which shoulu have been high in uppei haimonics. To combat this, two
things weie uone. The fiist was to iaise the low cut-off flooi of the filtei so that
any extieme movement woulun't lowei it to inauuible levels. The seconu was to
incluue a mouule that came into action at high eneigy input levels, letting
unfilteieu content thiough to combat the effects of this pioblem.

U'6%-# W8 H2-BF'16 U'4&#- H25%4#
112

U'6%-# W> ?17'5# &F# +2-BF'16 C'4&#-
6.3.4.2 SaLuraLlon
In an effoit to push the instiument into moie extieme tones, a satuiatoi effect
fiom the Reaktoi libiaiy was auapteu foi use with the Cello anu auueu to the
output chain. The upuown tilt of the left hanu nunchuck was useu to contiol
intensity of the satuiatoi, which was similai in effect to a guitai uistoition peual,
anu pioviueu anothei tonal uimension to the sounu.
6.3.4.3 8everb
Lastly, a ieveib effect was taken fiom the Reakoi libiaiy anu applieu to the output
chain to make the instiuments tone moie pleasing to the eai.
T2T W$%+>D >/0 3:J%$1>D )--06>1F
Aftei feeuback fiom the Spheie, it was cleai that useis wanteu some kinu of
inteiactive visual feeuback to aiu in playing the instiument when leaining. This
was implementeu in Reaktoi in the foim of a moving block which changeu with
the tilt of the left hanu, moving between vaiious maiks on a line, similai to a
guitai tunei, with each maik iepiesenting a note.

U'6%-# W@ Y'7%,4 U##5=,$0
11S
The CC signal foi the moving block is sent continuously. When this was fiist
implementeu in the C# softwaie, the piogiam ciasheu tiying to access the yet to
be instantiateu NIBI0ut class. It was founu that the NIBI output uevice neeus a
little time to initialise anu setup, anu if a NIBI signal is set befoie this happens,
the piogiam woulu ciash. A timei was useu which counteu uown fiom 4uuums on
stait up. When this timei ian out, it flippeu a Boolean flag which inuicateu to the
piogiam is was safe to begin tiansmission of NIBI uata.
Physical feeuback was implementeu by inteiacting with the vibiation motoi on
the wiimote. If the total eneigy feu into the system went above a ceitain
thiesholu, the vibiation motoi was switcheu on, giving the playei auuitional
'connection' to the instiument in a similai mannei that the vibiations fiom
physical instiuments tianslate to a playei.
T2U @&,+1&+,- '( &:- G-DD'
Beie, the stiuctuie of both the Cellos C# softwaie anu accompanying Reaktoi
patch is piesenteu.
114
@#C#$ *Y 0+>-K6.' 04336.1

U'6%-# WL *%++,-. 2C &F# J#442 Jl 7&-%$&%-#
Foi a full list of classes with theii methous anu vaiiables, see appenuix B2.
x

a
c
c
e
l
z

a
c
c
e
l
y

a
c
c
e
l
x

a
c
c
e
l
z

a
c
c
e
l
C
/
Z

b
u
t
t
o
n
s

(
c
h
a
n
g
e

s
t
r
i
n
g
)
y

a
c
c
e
l
J
o
y
s
t
i
c
k

X

a
n
d

Y
D

p
a
d
+
/
-

b
u
t
t
o
n
s

(
t
r
a
n
s
p
o
s
e
)
A

b
u
t
t
o
n

(
c
o
n
t
i
n
u
o
u
s

p
i
t
c
h
)
L
a
t
c
h

a
n
d

h
o
l
d
L
a
t
c
h

a
n
d

h
o
l
d
C
C
1

(
P
h
y
s
i
c
a
l

m
o
d
e
l

b
o
w

v
e
l
o
c
i
t
y
,

1
s
t

g
r
a
i
n

d
r
i
v
e
r
,

m
o
r
p
h
i
n
g

f
l
t
e
r
)
C
C
2

(
M
o
r
p
h
i
n
g

f
l
t
e
r
)
C
C
1
3

(
M
o
r
p
h
i
n
g

f
l
t
e
r
)
C
C
1
4
,

C
C
1
6

(
G
r
a
n
u
l
a
r

s
a
m
p
l
e

s
e
l
e
c
t
)
M
I
D
I

n
o
t
e

o
n
/
o
f
M
I
D
I

P
i
t
c
h
b
e
n
d
C
C
1
7

(
S
a
t
u
r
a
t
o
r
)
C
C

1
5

(
T
i
l
t

f
o
r

v
i
s
u
a
l

f
e
e
d
b
a
c
k
)
C
C

5

(
h
a
r
m
o
n
y

a
m
p
l
i
t
u
d
e
)
C
C

6

(
h
a
r
m
o
n
y

p
i
t
c
h
)
H
a
r
m
o
n
i
c
A
m
p
l
i
t
u
d
e
E
q
u
a
t
i
o
n
s
M
I
D
I

N
o
t
e
B
u
i
l
d
e
r
C
C

7

(
1
s
t

h
a
r
m
o
n
i
c
)
t
o
C
C

1
3

(
7
t
h

h
a
r
m
o
n
i
c
)
B

t
r
i
g
g
e
r

(
n
o
t
e

o
n
)
N
u
n
c
h
u
c
k
P
i
t
c
h
C
a
r
t
e
s
i
a
n
t
o
P
o
l
a
r
V
i
b
r
a
t
i
o
n

f
e
e
d
b
a
c
k
W
i
i
m
o
t
e
11S

@#C#" Q'6B-+. 0-.45-4.'

U'6%-# WD :-'+,-. 72%15 #16'1# 7&-%$&%-#
0sc 1 anu osc 2 contain the physical mouel anu auuitive synthesiseis iespectively.
The vaiiable inputs can be seen on the left.

U'6%-# WW V#,0&2- 72%15 #16'1# B-'+,-. 7&-%$&%-#
Figuie 66 shows the piimaiy stiuctuie of the Reaktoi engine. 'Cel' uenotes Figuie
6S. The satuiatoi anu ieveib effect (labelleu as 'Space Nastei') can be seen
connecteu to the Cello.
116

T2V R%-, )--06>1F
The Cello was given to 1S paiticipants to tiy anu weie askeu to ieflect upon theii
expeiience of the instiument, in a similai manoi to the Spheie usei tiials.
0bjectively veiifying the instiuments 'success' is uifficult foi a numbei of ieasons.
0ne cannot measuie an instiuments 'woith' by how uifficult it is to play (a
saxophone is uifficult to play in the fiist instance, but this fact uoes not uiscount it
as an instiument). The time constiaints of the pioject also meant that the
instiument coulu be 'leaineu' to a high stanuaiu ovei the peiiou of the test, thus
putting anothei obstacle in the path of compaiing to 'tiauitional' instiuments. Any
moie than the most basic compaiison to the Spheie is also uifficult, as theii
funuamental moue of opeiation is quite uiffeient.
This makes the foimulation of a testable hypothesis uifficult. If piesseu, the
'question' at the heait of this usei suivey woulu be something like:
Bo people feel enqoqeJ witb tbis instrument, onJ if so, wbot furtber ovenues coulJ
be exploreJ to eitber: A: lmprove it or B: FxponJ it beyonJ its current stote os o
continuous musicol instrument
The suivey foi the Cello was kept to a similai foimat to that of the Spheie. While
the musical tasks given hau moie expiessive scope than those of the Spheie, the
oveiall intention was not piimaiily to gathei haiu numbeis, but to get the
paiticipants to uiscuss theii expeiience of tiying to expiess theii musical iueas
with this instiument.
Like the Spheie, the useis of the Cello weie askeu to peifoim musical tasks, both
in a set foim by playing the well known melouy of 'happy biithuay' anu fiee foim
impiovisation, being askeu to peifoim musical appioximations to emotions. Aftei
these tasks, they weie then askeu to ieflect on theii expeiience.
117

Below is a summaiy of the subjects testeu:
Subject Backgiounu Playeu Wii
befoie.
Playeu
Spheie
1 uuitai Yes Yes
2 Saxophone Yes No
S Biums (giaue 8) Yes Yes
4 Playeu piano until age 1S Yes Yes
S Biums (giaue 8+) Yes No
6 Electionic music piouuction, some
guitai
Yes No
7 Electionic music piouuction, guitai
anu keyboaius
Yes Yes
8 Bouble bass Yes No
9 Keyboaius, electionic music
piouuction, inteimeuiate guitai
Yes No
1u uuitai (inteimeuiate) Yes No
11 Saxophone (giaue S) Yes No
12 uuitaisingei (4u yeais) Yes No
1S Singei anu flute (giaue S) No Yes
14 Tiombone, guitai Yes Yes
1S Non-musician Yes No
E,=4# D *%=;#$&7 &#7&'16 &F# J#442
All of the subjects foimally suiveyeu (with the exception of subject 12) weie ageu
between 19 anu 24. People of this age gioup tenu to be moie ieceptive to new
concepts, especially those of a technological natuie, having giown up with the fast
pace of technological change. This is cleaily ieflecteu in the numbei of subjects
who have playeu a Nintenuo Wii console befoie! Even though the Cello
instiument in its cuiient state is uesigneu with this uemogiaphic in minu, it
woulu be wise to consiuei the implications of choosing such subjects befoie
uiawing any conclusions.
@#Z#$ S4'2-)+,,6).'
The wiimote anu nunchuck weie hanueu to the subject anu the softwaie was
activateu. The exact natuie of the instiument was not explaineu until aftei the
fiist question hau been askeu.
The test is piesenteu below with test specific uiscussion of iesults, followeu by an
oveiall analysis of the implications of these iesults.

118
:V! _ E!*E
1. 6lvF WllH0TFS T0 S0B}FCT ANB BFSCRlBF TEF BFvlCF. ASK TEFH T0 HAKF
H0Tl0NS BASFB 0N E0W TEFY TElNK TEF S00NBS Wlll BF C0NTR0llFB
TEFN SWlTCE 0N.
2. Biu the iesultant output match youi expectations.
The puipose of this exeicise was to see how much pievious knowleuge of both the
inteiface anu of othei musical instiuments affecteu the subjects choice of action
when picking up the instiument foi the fiist time.
All the useis pickeu the wiimotes up in the 'stanuaiu' way, with nunchuck in the
left hanu anu wiimote in the iight (except subject 1 who was left-hanueu anu helu
them the othei way iounu). Subjects 2, S, 8, 9, 1u anu 11 all peifoimeu some kinu
of iuuimentaiy bowing motion with the iight hanu, in a similai manoi to a boweu
stiing instiument, with guess to pitch contiol being eithei upuown oi leftiight.
The iemaining subjects iight hanu motion consisteu of seemingly ianuom
motions in the aii.
0nly subject 4 peifoimeu some kinu of twisting motion with the left hanu,
possibly uue to his piioi expeiience with the Spheie.

HT*?J/S E/*c*
TEF lNSTR0HFNTS 0PFRATl0N lS I0llY FXPAlNFB. TEF S0B}FCT lS lFIT I0R
IlvF T0 TFN HlN0TFS T0 FXPFRlHFNT
S. ASK S0B}FCT T0 PlAY EAPPY BlRTEBAY

U'6%-# WN fA,BB. M'-&F5,. E2 h2%f m G,-1#-_JF,BB#44 H%7'$
By fiist piesenting the subjects with the task of peifoiming a well know melouy,
the subjects weie enticeu into engaging with the instiument with the logical,
119
pioblem solving iegions of the biain in the same way one piactices scales oi new
pieces. Rathei than be an assesseu exeicise, this was an attempt to 'waim them
up' anu get them useu to the instiument contiol system befoie giving them 'fiee
foim' musical tasks.
Subjects S, 9 anu 14 all peisisteu anu eventually got eveiy note iight with enough
attempts.
Subjects 1, 2,7,8,1u anu 11 got fiist half iight but founu pioblems with the
instiument following the key change in bai 6. They all peisisteu howevei anu
manageu to complete the piece with only one oi two wiong notes in the closing
bais.
Subjects S, 4, 12 anu 1S got the majoiity of notes in the fiist half iight but founu it
uifficult to make sense of the instiument passeu this key change.
All subjects intently lookeu at the visual feeuback while tiying to peifoim this
task.
Subject 6 (who was Nexican) piofesseu not to know the tune, anu iathei than
spenu a long time tiying to teach him, it was felt that this pait shoulu be skippeu.
4. ASK S0B}FCT T0 C0NvFY TEF I0ll0WlN6 FH0Tl0NS WlTE TEF CFll0:
EAPPYNFSS,
SABNFSS
TFNSl0N
Each subject was askeu to tiy anu expiess an emotion with the instiument. As
stateu above, the woith of this paiticulai instiument cannot necessaiily be
gaugeu by how easy it is to leain, oi the iange of notes it can play. Asking the
paiticipants to tiy to expiess some kinu of emotion, iiiespective of ihythm,
melouy oi stiuctuie woulu be a goou test foi the sonic possibilities of the
instiument anu the iange of ways people coulu manipulate the instiument. The
hope is, is that each usei will have theii own 'voice' on the instiument anu use it
in a slightly uiffeient way, going some its 'expiessiveness'.
12u
The use of the visual feeuback foi these 'emotional' tasks was maikeuly less than
with the melouic task. Some subjects chose to play inteipietations baseu on scales
anu some playeu completely fieefoim.
It is the opinion of the authoi that the uepth of uesign of the instiument uiu
inueeu allow foi a high enough oiuei input space to allow foi expiessive cieative
fieeuom. Some of these peifoimances weie iecoiueu anu can be founu on the
accompanying CB.

HTSE?:S! JA<?J! nT!*E?<Z*
The same multiple choice questions as given to the paiticipants of the Spheie
weie given to the Cello test paiticipants to gain some compaiable metiics. With
its gieatei uepth of uesign anu much gieatei potential input space, it was hopeu
that the subjects woulu give it a highei iating of iesponsiveness anu
consequentially want to spenu moie time with it anu attain fuithei levels of sonic
contiol with moie piactice. Foi consistency, the questions aie iepeateu below,
followeu by the iesults.
S. Bow iesponsive uiu you finu this instiument. In othei woius, how stiongly uo
you think the actions you peifoimeu coiielateu with the sounu you heaiu fiom
the speakeis (1-S).
1. None at all, seemeu totally ianuom
2. veiy loosely ielateu
S. Loosely ielateu
4. Nostly achieveu uesiieu output
S. Exactly achieveu uesiieu output
6. To what uegiee uo you think piactise woulu impiove youi peifoimance(1-S).
1. None, I coulu not impiove on this instiument
2. A little, but seems uifficult to see how I coulu impiove
S. I coulu see myself playing simple tunes but that's it
4. This instiument coulu be useu as a uefinite anu viable cieative tool
S. I coulu see myself playing this as well as any othei 'tiauitional' instiument
121
7. Bow much moie time woulu you be willing to invest in this instiument(1-S).
1. No moie time
S. I woulu take up this instiument anu leain it the same way as a 'tiauitional'
instiument
NB: Theie was no piompts foi 2,S oi 4 on this question. 0seis weie askeu to go
with theii gut feeling as an ovei all 'enthusiasm' iating foi the instiument.

U'6%-# WX V#7%4&7 2C +%4&'B4# $F2'$# R%#7&'217 C2- &F# J#442
8. Woulu the ability to customise this instiument inteiest you.
As with the Spheie, the vote was almost unanimous, with all but subject 2
expiessing inteiest in customising the instiument.

K?*JT**?<Z
Aftei playing the instiument foi a shoit time, both fiee foim anu tiying to play
Figuie 67, the subjects weie askeu to give theii opinions on the Cello. Nany of the
comments by the useis congiegateu aiounu similai themes. The heauings below
iepiesent some of these common themes, with selecteu comments fiom the
subjects iepiouuceu below.
u
2
4
6
8
1u
12
14
1 2 S 4 S
Responsiveness
Piactise
Time
122
9. What uiu you like.
EF# %7# 2C ,$&'(# C%44_=25. 6#7&%-#7 +,0# &F# '17&-%+#1& ,1 '1&#-#7&'16
#"B#-'#1$#
loveJ tbe full boJy movement requireJ to ploy Subject 4
Allusive control set mokes it reolly interestinq to ploy Subject S
l reolly like tbe woy tbe sounJs reoct to your boJy movements Subject 8
'l finJ myself tbinkinq in movement onJ qestures rotber tbon witb porometer
controls like on requlor syntbs' Subject 9
'FnjoyeJ tbe woy boJy movement wos ossocioteJ witb Jifferent sounJs' - Subject 4
'You reolly wont to put your wbole boJy into it, like tbe nintenJo wii' - Subject 12
'Not tbinkinq obout syntb unJerneotb, just ployinq it' - Subject 14

EF# '17&-%+#1& '7 C%1 ,15 ,$$#77'=4#
0nique onJ instontly fun - Subject 1
New onJ oriqinol - Subject 2 anu 12
l've never seen on instrument like it, you qet instont results, even if you're o
beqinner - Subject 11
Tbe foct i'm usinq wiimotes moke it instontly occessible. lt olso meons onyone
onywbere in tbe worlJ con use it. - Subject 1u
'You con eitber forqet obout ployinq it too mucb onJ just qo wilJ or concentrote onJ
ploy meloJies it bonJles botb reolly well.' - Subject 1S
'WoulJ look brilliont on stoqe, on ouJience woulJ love tbis.' - Subject 14
'Con qet cool sounJs out of it even if you're not o musicion' - Subject 1S

12S

EF# '17&-%+#1& F,7 , 42& 2C #"B-#77'(# 5#B&F
likeJ tbe responsiveness of riqbt bonJ controller. very expressive - Subject S
love tbe woy bormonies coulJ be injecteJ witbout cbonqinq moin pitcb - Subject S
A lot of control witb only o few buttons - Subject 4
0nique sounJs, coulJn't tbink of onotber woy to control tbem, especiolly witb one
instrument - Subject 7
Tbe sounJ is never stotic onJ olwoys movinq, tbis reolly keep you on your toes! -
Subject 9
You bove o qreot Jeol of control over tbe sounJ - Subject 1S

EF# '17&-%+#1& C##47 4'0# '&7 21 &F# #56# 2C 7&,='4'&.9 EF'7 +,0#7 '& , +2-#
#16,6'16 B4,.'16 #"B#-'#1$#9
Seems to be on tbe verqe of stobility wbicb reolly mokes you wont to experiment.
Eos tbe fomilior cbootic fell of leorninq o new instrument - Subject S
'l loveJ tbe woy tbe biqb enJ bormonics come in wben you reolly qive it some' -
Subject 1
'Tbe ronJom sounJ cbonqes ot biqb volumes ore o reolly interestinq concept' -
Subject 8
'Ieels o bit cbootic but tbis mokes it more interestinq' - Subject 12
'Tbe rumble pock qives it tbe feelinq like its obout to exploJe! Hokes you feel more
involveJ witb tbe instrument - Subject 4

!"B,17'(#
Con imoqine o lot of possibilities for exponsion - Subject 2
124
'like tbe foct its moJe in Reoktor, tbis will ollow for furtber customisobility' -
Subject 9

1u. What uiun't you like.
/$$%-,&# B'&$F 3,7 5'CC'$%4& &2 +,'1&,'1
lnitiolly borJ to control notes onJ stop beinq wobbly - Subject 1
IounJ pitcb o little borJ to keep stoble - Subject S
Pitcb wobbles o bit wben you reolly qo for it - Subject 1u
IounJ it borJ ot first to qet notes JeoJ on - Subject 12
IounJ it quite bore to Jo onytbinq witb meloJic structure, but tbis will probobly
improve witb proctice' - Subject 9
'Pitcb beinq so sensitive on my left bonJ. WoulJ like to moke it even more
'sounJscopey'' - Subject 1S
IounJ pitcb Jifficult to control, quick note jumps were o porticulor problem -
Subject 8
Irustrotinq tryinq to ottoin tbe top notes - Subject 4

<&F#- O5'74'0#P $2++#1&7
l'm not sure l like pitcb control beinq on o twist motion - Subject 7
lts pretty tirinq! - Subject 4
SounJ sometimes Jrops out wben you inject too mucb power - Subjects 2 anu S -
NB: this was subsequently fixeu (see 6.S.4.1).
Not enouqb notes on tbe left bonJ, founJ it borJ to qo bock onJ fortb. - Subject 11
12S
'IounJ it quite borJ ot first wbicb initiolly puts me off wontinq to Jo onytbinq
beyonJ sounJ effects witb it. l boJ tbis problem witb my flute tbouqb wben l wos
younqer onJ wos moJe to ploy by my porents, so in some woys tbis isn't reolly o boJ
tbinq.' - Subject 1S
11. What woulu you auu on.
/55'&'217 &2 72%15 #16'1#
AJJ more sounJs to it - Subject 1, - Subject S
A more bolistic, focuseJ sounJ witb more complex moppinq - Subject S
Hore ouJio worpinq options, coulJ be qooJ os on ouJio effect - Subject 14

JF,16#7 &2 +,BB'16 7$F#+#
Alter tbe strinq cbonqe buttons to be S
tb
-ovtove-S
tb
-ovtove rotber tbon
S
tb
-S
tb
-S
tb
-S
tb
. Tbis woulJ moke it eosier to ploy in some respects - Subject 2
Some more qestures like winJmill quitorinq! - Subject 4
A moppinq motrix for cbonqinq tbe connection of controls onJ occelerometers. -
Subject 6
'WoulJ like pitcb movement on left bonJ to be o mucb os riqbt bonJ to qet botb
orms octive' - Subject 8
6enerolise to use witb moinstreom syntbesisers. - Subject 9

JF,16#7 &2 ,BB4'$,&'21 ,7 , 3F24#
Port to HAX,HSP onJ moke stonJolone opplicotion - Subject 9
'Beqinner or lesson moJe' - Subject 12
CoulJ moke it into o viJeo qome - Subject 1S
126
'lf it wos moJe simpler, coulJ be qiven to younq cbilJren os o musicol teocbinq oiJ -
Subject 1S
'l'J prefer it if it ron on my moc!' - Subject 14
12. 0thei comments.
visuols belpeJ o lot wben tryinq to ploy meloJies - Subject 4
l founJ it o bit Jifficult to finJ tbe riqbt movement for tbe riqbt sounJ, but tbis is o
qooJ tbinq os well. lt mokes you wont to keep ot it - Subject 4
Such was subject 6's enthusiasm foi the instiument, that when askeu how much
moie time he woulu want to spenu with the Cello, he ieplieu: 'Tbree Joys... Tben l
will sleep!.
'A violin bos o steep leorninq curve onJ is intricotely expressive. Beqinners sounJ
non-musicol wben ployinq tbe instrument for tbe first time. A keyboorJ bos
orquobly less expressive potentiol, but beqinners con moke some bosic 'musicol'
sounJs witb it. Tbis instrument seems to briJqe tbis qop.' - Subject 7
@#Z#" T,6/12)2 6,? F)25422)+, +> *'//+ R2'. D''?E65B
With the numbei of comments attesting to the Cellos expiessive anu accessible
natuie, the methouology can be consiueieu a success. The fact that many test
subjects positively commenteu on the chaotic natuie of the instiument at the
uppei limits of input velocity is encouiaging, anu sets a goou founuation foi
fuithei woik in this avenue of electionic instiument uesign baseu on 'tiauitional'
acoustic instiuments.
The visual feeuback implementeu aftei the usei feeuback fiom the Spheie pioveu
veiy populai, with eveiy subject using the scieen to gauge pitch when tiying to
play Eoppy BirtbJoy. The numbei of subjects woulu stiuggleu with the piece
following the key change suggests the visual feeuback oi pitch contiol neeus to be
upuateu to ieflect the outcome of a usei using the buttons on the nunchuck to
extenu the iange of the instiument.
It is the authois opinion that the uispaiate use of the visual feeuback on the
fieefoim 'emotional' tests suggests that people felt they coulu exploie the sounus
127
available with gestuies anu movements iathei than with visual cues on a scieen.
0seis wishing to convey melouy in these fieefoim tests most often useu the
joystick to contiol pitch iathei than the ioll of the left hanu.
The pioblems with the contiol of pitch was iepoiteu as the biggest uislike of the
instiument by fai. 0ne coulu aigue that the amount of time given to the
paiticipants to leain the Cello is veiy small compaieu to the amount of time one
woulu neeu to spenu with a 'ieal' instiument to attain accuiate contiol ovei pitch,
anu similai complaints might be issueu if piospective stuuents of an oboe weie
askeu to leain contiol of pitch in such a shoit space of time, but such a stiong
choius of uiscontent cannot be oveilookeu. The uesignation of the tonic to the ioll
of the left hanu was a necessaiy compiomise to the piacticalities of contiolling
pitch (tiauitionally a vaiiable whose contiols aie placeu oithogonal to eneigy
input) with only S acceleiometeis. The confinement to this small aiea meant that
a compiomise also hau to be maue with the noteangle iesolution of the left hanu.
Bau this iesolution been too low, notes woulu have been easily anu quickly
navigateu to, but at the expense of iange. Conveisely, a high iesolution woulu
have given a laige iange of notes with a laigei maigin foi eiioi.
The contiol of pitch is an excellent canuiuate foi the application of the extia
uegiees of fieeuom the wii motion plus will biing. With the auuition of extia
acceleiometeis anu gyioscopes, absolute positioning becomes a piactical ieality,
anu the instiuments can be maue as laige oi as small in physical space as uesiieu.
@#Z#7 U-&'. D''?E65B
Like the Spheie, a uemonstiation viueo of the Cello was uploaueu to social
netwoiking sites Youtube anu Facebook , anu again was again met with veiy
positive iesponses anu iequests foi woiking veisions foi people to tiy.
Aftei seeing the uemonstiation viueos on Facebook, a seconuaiy school teachei
known to the authoi playeu them to a class of yeai 9 stuuents as pait of a music
lesson on 'new music technology'. The iesponse was iepoiteuly veiy enthusiastic,
with the stuuents anxious to know when anu wheie they coulu get a copy of this
softwaie to tiy foi themselves, appaiently lookinq mucb more fun tbon wii music .
128
As well as the seconuaiy school pupils, the Cello was shown to a gioup of 4u-6u
yeai olu folk musicians. 0nfoitunately, the enviionment uiu not allow foi foimal
testing, anu only one of these subject was put thiough the full questionnaiie, but
the comments ieceiveu by some of the musicians aie woithy of note. New
instiuments anu concepts such as these wii instiuments iaiely escape the bounus
of technologically minueu ciicles such as stuuents anu electionic musicians, anu
so feeuback fiom a gioup of people who giew up playing only tiauitional acoustic
instiuments makes foi inteiesting mateiial foi compaiison. The fiist obseivation
was that these subjects tenueu to appioach the instiument much moie gently,
with much smallei movements than the pieuominant stuuent makeup suiveyeu
above. These small movements often biought a minimal unimpiessive sounu fiom
the Cello, anu piompting was often iequiieu to get them to powei the instiument
sufficiently into its optimal opeiating zone. This coulu be uue to many factois:
The youngei geneiation have expeiience with the wii console which
encouiages laige, fast movements
The oluei geneiation tenu to be moie cautious when hanuling hi-
technology foi feai of uamaging it than the youngei geneiation, who have
giown up with iobust electionic uevices, useu in theii eveiyuay life.
The oluei subjects suiveyeu weie all folk musicians with many yeais of
playing expeiience. When one becomes this familiai with an instiument,
only the most subtle movements aie iequiieu to manipulate it - theii
movements with the wiimotes coulu be a ieflection of this.
The oluei subjects' opeiation of the instiument tenueu to be moie biittle than the
movements of the youngei paiticipants. Right hanu motion was often kept to one
axis at a time which uiun't attain the same amount of sonic uepth the instiument
was capable of. This methou of inteiacting with the uevice leu to less inteiesting
iesults than those obtaineu by the youngei subjects, which piobably contiibuteu
to the less enthusiastic iesponse. Although the subjects shown the instiument
weie cuiious, only one of them maue any iemaiks ielating to actually playing it as
an instiument in a live setting. The geneial impiession gatheieu was that the
uevice was moie a fun sounu effects accessoiy than a seiious instiument,
howevei 'interestinq anu clever it might have been.
129
When one becomes ueeply involveu in the uevelopment of an invention, it
becomes incieasingly uifficult to be able to take a step back anu ieview the
invention in a wiuei context, inuepenuent of any specialist anu backgiounu
knowleuge. While these oluei folk musicians may lack backgiounu knowleuge oi
appieciation of electionic music foims, they cannot be blameu foi questioning the
expiessive potential of six acceleiometeis in a package uesigneu foi computei
gaming when compaieu to instiuments such as a guitai oi violin which have been
uevelopeu anu iefineu ovei hunuieus of yeais. The lack of appieciation oi
knowleuge of the musical avenues that the instiuments aie applicable woulu
contiibute to the lack of iecognition of the Wii electionic instiuments piesenteu
to them. Baving nevei encounteieu gestuieless musical peifoimance oi
expiessive bottle necking in synthesiseis even electionic music foi that mattei,
the concepts tiying to be auuiesseu in this stuuy woulu be alien to them. While it
is tiue that the instiuments in theii cuiient state uon't come close to the
expiessive potential of moie establisheu tiauitional instiuments, it only seives as
a stiongei incentive to continue woik both in the uevelopment of technique anu
mapping uepth of these instiuments.
Subject 12, the only one of this gioup who hau the time to be foimally suiveyeu
hau playeu the Wii console pieviously. The familiaiity with this inteiface possibly
helpeu to engaging the subject anu leau them to give theii positive ieactions to
the instiument. All of the othei subjects foimally suiveyeu (with the exception of
subject 1S) hau playeu the Wii console befoie. If we take the finuings of (Nuiiay
2uu7) into account, it coulu be that pievious expeiience with gaming on the Wii -
which the subjects woulu associate with 'fun' anu 'enjoyment', hau an effect on
theii enjoyment of this instiument.
@#Z#9 *+3A6.)2+, K)-& -&' 0A&'.'
While the Spheie anu the Cello aie quite uiffeient instiuments it is felt that the
two can be compaieu to a uegiee. The gieatei attention to uetail anu moie
expansive mapping scheme of the Cello shoulu have given both moie enthusiastic
comments anu a laigei scoie on the multiple choice questions. The multiple
choice questions will be lookeu at anu compaieu fiist.
1Su
With 1S paiticipants foi each suivey, a cumulative scoiing system can be ueviseu
to uiiectly compaie the two instiuments. Each of the questions ('Eow responsive
Jo you tbink tbe instrument is? Bo you tbink proctise woulJ improve your
performonce? anu Eow mucb time woulJ you be willinq to invest in tbe
instrument?) weie conuenseu anu given a 'scoie' baseu on the make up of the
answeis foi that paiticulai question. If a subject answeis '1' to one of the
questions, this will auu a scoie of '1' to the total scoie of a categoiy. If a subject
answeis '2', this will auu 2 anu so on. A categoiy containing the answeis 1,2,2,4
anu S woulu be given a total scoie of 14 foi example. Each categoiy is out of a
possible 7S. The iesults aie uisplayeu below:

U'6%-# WI J2+B,-'721 2C &F# *BF#-# ,15 J#442 +#&-'$7
It can be seen fiom Figuie 69, the Cello beats the Spheie in all fielus, although
only by a small maigin. Both instiuments scoie above 64% (48 out of a possible
total 7S) which is a testament to the immeuiately appealing natuie of the
wiimotes. Comments about the two instiuments fiom the useis testeu uiffeieu in
content anu tone. The comments ielating to the Cello weie much moie focuseu
anu specific. Comments like '[lt] seems to be on tbe verqe of stobility wbicb reolly
mokes you wont to experiment. Eos tbe fomilior cbootic fell of leorninq o new
instrument anu ''l finJ myself tbinkinq in movement onJ qestures rotber tbon witb
porometer controls like on requlor syntbs' weie not given by paiticipants aftei
using The Spheie, who tenueu to give much moie geneial comments ielating to
u
1u
2u
Su
4u
Su
6u
7u
8u
Response Piactise Time
Spheie
Cello
1S1
the geneial activity of playing the instiuments anu how much 'fun' it was, but not
ieally going into any specific uetail about the expiessive natuie of the instiument.

T2X H$%1+%%$'/ '( )$/0$/*% (,'< 9:- G-DD'
The Cello has shown to be a moie engaging expeiience that the Spheie. Subjects
iuentifieu with the concept of having eneigy in stiongly coiielate to eneigy out,
with the most commonly uislikeu opeiational featuie being the unintuitive
uesignation of pitch to a iotation motion. Specific comments weie also maue
iegaiuing the way tone changeu with velocity. This goes some way to suppoiting
the theoiy that taking iueas baseu on existing tiauitional instiuments anu
incoipoiating them into new electionic inteifaces makes them moie engaging anu
moie expiessive.
Noie geneially, iesponse was positive, with many encouiaging comments also
coming fiom outsiue the suiveyeu population. The Cello has shown to be an
extiemely piomising stait to a possible whole new family of new musical
instiuments baseu on this technology. The familiaiity of the wiimote to both the
auuience viewing online anu the useis suiveyeu has been a boon to the
enthusiasm shown foi this instiument. It is the authoi opinion that hau a uevice
maue in a lab with similai piopeities to the wiimote been piesenteu insteau,
iesults woulu have been less piomising, especially with the auuience who vieweu
this pioject online. Nany of the instiuments we see touay have continueu theii
uominance by the means of being familiai to an auuience, which is an essential
pait of any musical peifoimance. If a usei has absolutely no iuea how the
peifoimei is inteiacting with the sounu they heai, all but the most feivent
followeis of expeiimental music woulu tenu feel less connecteu to the
peifoimance in much the same way someone feel unconnecteu to concepts being
ielayeu in an unfamiliai language. The iole of the instiument uesignei is to stiike
a balance between innovation anu tiauition. The invention must have something a
usei can ielate to, otheiwise it may be iejecteu. 0nce a usei has become
comfoitable with this new invention, it can be built upon anu expanueu fiom
theie.
1S2

T2.Y 3'%%$6$D$&$-% (', MNA>/%$'/ '( &:- G-DD'
This section will concentiate on expansion possibilities foi the Cello. Foi fuithei
woik on this pioject as a whole, see 9.2.
Fiistly, the visual feeuback coulu be upuateu to make the system easiei to leain
when moving above the fiist 'stiing'. Changing colouis oi a staggeieu, multi-
layeieu system coulu be ueviseu which woulu switch levels when the C anu Z
buttons on the nunchuck weie piesseu.
Beveloping the sounu engine to incluue moie synthesis algoiithms woulu be an
excellent ioute foi uevelopment. A moie auvanceu physical mouel oi complex
auuitive synthesisei coulu be cieateu to allow foi even ueepei tonal contiol. In
auuition to this, theie is a possibility of cieating an inteimeuiaiy mapping system
that woulu conveit the NIBI contiol scheme of the Cello to woik with
conventional synthesiseis. The non-stanuaiu natuie of NIBI-CC uesignations
might cause pioblems, but if the usei inteiface was maue simple enough, people
coulu builu theii own systems anu tailoi it to theii synthesiseis.
Expeiimenting with uiffeient mapping woulu be a goou possibility foi fuithei
woik. As well as specifically uesigneu mappings, an inteiesting expeiiment might
be to have a computei ianuomly geneiate mappings. A human subject coulu then
tell the computei if it was 'goou' oi not, anu the computei coulu leain by means of
a neuial netwoik. This may iesult in some unexpecteu iesults, anu mappings that
woulu seem unintuitive fiom the outlook might actually piove to be valiu anu
expiessive moues of contiol.
1SS
U H-7-D'A$/* P/ "/&-*,>&-0 @J%&-<
As mentioneu in S.2, the easily uemonstiatable anu immeuiately accessible natuie
of this paiticulai pioject makes it veiy goou show mateiial foi 0niveisity open
uays, seiving as an example of 'fun' engineeiing. Iueally, this exhibit woulu have
both instiuments iunning concuiiently anu shoulu be iobust enough to hanule
the most extieme abuse by useis uneuucateu in the opeiation of the instiument.
Initially, both the Spheie anu the Cello weie tiieu to be useu simultaneously but
uue to the natuie of uata hanuling on the Winuows NIBI bus, seiious uata
collisions occuiieu anu the cuiiently iunning piogiam woulu expeiience a fatal
eiioi when a seconu was loaueu up. This was an unfoieseen pioblem, not easily
iepaiiable in eithei of the two instiuments uue to the way they hau been wiitten.
The Cello anu the Spheie hau been wiitten with the following class hieiaichy:

U'6%-# NQ J4,77 F'#-,-$F. 2C '15'('5%,4 *BF#-# ,15 J#442 ,BB4'$,&'217
With the veiy high uata output iate of the Cello, collisions invaiiably happeneu
between its NIBI0ut class anu that of the Spheie, which weie both attempting to
pass messages to the same output uevice.
GUI
Controller
Processing
MIDI Out
Wiimote
1S4
With each contiollei class taking the uata fiom the wiimote anu passing it to its
own NIBI0ut subclass, changing the hieiaichy of both the Spheie anu Cello to iun
inuepenuently woulu have been quite a time consuming task.
As the neeus of the softwaie foi the open uay exhibit aie a little uiffeient than
those of the usei who wishes to uelve ueepei anu expeiiment, it was felt that a ie-
wiite was appiopiiate. The new 'open uay' veision woulu have a mastei
contiollei class anu use C# EventBanuleis to call a single NIBI0ut class to avoiu
collisions. It woulu also featuie less fiont panel featuies anu be contiollable
completely with the wiimote - thus negating any neeu foi a keyboaiu anu mouse
at the exhibit anu uecieasing the chance of a usei acciuentally changing the
system.
The integiateu system polls the Winuows BIB list on staitup, anu assigns each
instiument a wiimote BIB auuiess to ieceive uata fiom. The contiollei class
contains an event listenei which is activateu when a NIBI commanu is iequiieu to
be sent fiom eithei instiument.

U'6%-# N8 A'#-,-$F. 2C &F# '1&#6-,&#5 7.7&#+
GUI
Controller
Wiimote
Collection
Processing
MIDI Out
Wiimote 1 Wiimote 2
Processing
Sphere Cello
1SS
The Spheie is now lockeu to one pitch cuive anu the Cellos' iesistivity anu iange
settings have been haiucoueu to make foi the simplest set-up expeiience. Both
the Spheie anu the Cello have been haiucoueu to opeiate uown NIBI channels 1
anu 2 iespectively.
The Reaktoi patch has also been upuateu to accommouate this new integiateu
application to contain a Spheie sounu engine with both pluckeu stiing physical
mouels anu synthetic subtiactive synthesisei sounus, uiiven by uata on NIBI
channel 2.
The u0I foi this application is extiemely basic, with only one uiop uown menu to
change the NIBI output uevice - the only vaiiable outsiue this system that coulu
change fiom system to system.
Each wiimote also has a 'ieset' commanu. By piessing the 'Bome' button, all
vaiiables (such as sounu set anu tuning) aie ieset to the uefaults.
A full usei guiue to this system can be founu in appenuix B .
U2. 9-%&$/* &:$% %J%&-<
As this system is to be useu by the public with no piioi tiaining, some foimal
testing has been conuucteu to ensuie the softwaie iemains stable thioughout the
couise of a typical open uay. Nany of the vaiiables have been stiippeu out to
ensuie the simplest anu most iobust opeiation, making testing ielatively
stiaightfoiwaiu. Testing has been caiiies out on a 2uu7 Nacbook Pio iunning
Winuows XP SPS. While this may not miiioi the final system it was the only easily
accessible system at the time of testing.
1S6

M2%15,-. E#7&'16
Fiist, bounuaiy testing was conuucteu on all input contiols. The iesults can be
founu below:
Test Range Expecteu Result Result
Spheie nunchuck
tiansposition: C
anu Z buttons
-24 to
+24
Pitch changes, no ciash
occuis aftei selecting
bounuaiy anu stiiking note.
Success
Spheie velocity:
veiy soft anu veiy
haiu stiike testeu
with both wiimote
anu nunchuck
u to 127 Notes of appiopiiate
velocity aie hit.
Success
Spheie note iange,
lowest to highest
u to 11 Notes of appiopiiate pitch
aie hit.
Success
Cello: Naximum
velocity.
u to 127
on two
axis
Sounu engine iesponus anu
uies away when tiiggei is
ieleaseu. Filtei uoes not cut
sounu out.
PARTIAL
FAIL0RE: If the
tiiggei is ieleaseu
while the iumble
pack is on, the
iumble pack stays
on, even without
motion.
Cello note iange. 6u to 87 All notes iesponu as
expecteu
Success
Cello continuous
pitch iange
u-(2
14
-1) A-button is piesseu on a
chosen note, the nunchuck
is tilteu full scale anu the A-
button is ieleaseu. The note
at which the A button is
ieleaseu coiiesponus to the
note at which the pitch
benu left off. This is
iepeateu both ways.
Success
Cello }oystick
iange
u - 127 In tune haimonies aie
playeu when the joystick is
moveu thiough each notch.
volume of the haimony is
vaiiable with iauius.
Success
Wiimote tilt u-127 Noiphing filtei changes Success
1S7
with tilt. Extieme values uo
not cut sounu out.
Nunchuck tilt u-127 Satuiation changes
uepenuing on elevation.
Success
E,=4# W E#7&'16 2C &F# J2+='1,&'21 ?17&-%+#1&
The above pioblem with the iumble pack has been solveu by checking the
wiimote eveiy cycle to ensuie the iumble pack is not activateu when the B button
is not piesseu. If this is founu to be the case, the iumble pack is ueactivateu. This
solves the pioblem.
T1B4%66'16 ,15 -#_B4%66'16 '1 2C 1%1$F%$0
The iemoval of the nunchuck peiipheial fiom both wiimotes connecteu to the
combination instiument uoes not cause a fatal eiioi.
E.B'$,4 %7# $,7# 7$#1,-'2
Two people pick up tbe wiimotes onJ ploy toqetber.
No uata collision occuis when both instiuments aie being playeu, even with the
multituue of CC commanus being sent fiom the Cello.
Tbe wiimotes ore tben put Jown for S0 minutes onJ left. Tbe system corries on
runninq.
The Combination instiument has been testeu foi this length of time with no
activity anu uoes not ciash oi expeiience anomalies so long as any powei saving
featuies on the computei aie switcheu off.
Although biief, these tests go some way into instilling a sense of ieliability into
the system.
1S8
V 3,'4-1& C>/>*-<-/& S-7$-?
By using the weekly emails, pioject piogiess can be tiackeu anu compaieu to the
initial uantt chait (Figuie 17)to ieview pioject piogiess.
H'4#7&21# *B#$'C'#5
K,&#
/$&%,4
K,&#
/$F'#(#5
Complete uesign
of Spheie
Week 7 Week 7
Spheie piototypeu Week 8 Week 8
Begin spheie usei
tiials
Week 1u Week 9
Cello piototypeu Week 16 Week 12
Cello Complete Week 19 Week 17
1
st
Biaft iepoit
ieauy
Week 21 Week 22
Banu-in pioject
iepoit
Week 2S Week 2S
E,=4# N J2+B,-'721 2C B4,11#5 (#-7#7 ,$F'#(#5 +'4#7&21#7
As can be seen, the pioject has piogiesseu foi the most pait - aheau of the pioject
scheuule. Following swift uevelopment of the Spheie, the pioject was able to
auapt to incluue usei tiials of the Cello as well as auuitional time to uevelop an
integiateu solution foi use in univeisity open uays. The iuea to leain the basics of
wiiting instiument softwaie to inteiface with the wiimotes by builuing a simplei
instiument was an excellent uecision in hinusight. This enableu the iapiu
piouuction of the Cello, uespite an numbei of fatal computei ciashes uue to a
logic boaiu failuie which helu back piogiess by almost a week in total.
1S9
Staiting woik in the summei pieceuing this pioject has also speu up pioject
piogiess. Befoie the pioject was consiueieu, the natuie of the wiimotes haiuwaie
was known, as well as the basic issues of connecting them to a peisonal computei
anu piogiamming NIBI. By having alieauy unueitaken this woik, a numbei of
weeks weie saveu in the initial stages, enabling foi moie auvanceu pioject
uevelopment.
The only milestone behinu scheuule was the pioject iepoit uiaft. While the iepoit
staiteu to be wiitten fai eailiei than planneu on the initial time plan - beginning
aiounu week 8, piogiess was slightly slowei than expecteu. By having such a
laige iepoit wiiting peiiou, theie is the temptation foi one to woik with less
uigency than when woiking close to ueauline. Bowevei - staiting this eaily anu
uevoting at least one uay a week to the pioject iepoit wiite-up has been
extiemely useful. Nany iueas have hau time to gestate anu mateiialise ovei this
time peiiou which woulu have been absent fiom this iepoit hau it been wiitten
moie intensely ovei a shoitei time peiiou.
The costs of opeiating this pioject aie quite small if one uiscounts the use
Reaktoi. All the usei neeus to buy is a wiimote anu nunchuck, costing ioughly
Su-4S anu a Bluetooth peiipheial which can be obtaineu foi aiounu 1u. The
use of Reaktoi in this pioject has gieatly speu up the piocess of sounu uesign, but
futuie woik coulu incoipoiate open souice softwaie to achieve this goal anu
alleviate some of the cost.
Finally, an impoitant lesson the authoi has leaineu fiom this expeiience is
knowing wben to tell oneself to stop. This pioject has been paiticulaily enjoyable
expeiience, anu the temptation to keep expeiimenting anu auuing featuies has
hau to be cuibeu on seveial occasions thioughout the couise of the pioject, not
least in the closing few uays when the softwaie was ie-wiitten to facilitate using
both instiuments concuiiently. While this fact may sheu light on the ieasons foi
the somewhat biisk natuie of testing, piactically, it was cutting the pioject
uangeiously close to the euges anu maue some of the time allocateu foi
foimatting anu finalising this uocument slightly moie stienuous than it shoulu
have been.
14u
X G'/1D+%$'/% >/0 )+,&:-, ;',F
The finuings of this iepoit aie summaiiseu, with a shoit uiscussion of iesults
obtaineu anu theii implications. Following this aie pioposals foi fuithei woik.
X2. G'/1D+%$'/%
Two novel electionic musical instiuments contiollable with Nintenuo's motion
sensing Wii Remotes have been uevelopeu with the aim of exploiing moie
consiueieu avenues of musical instiument contiol mapping than those pieviously
applieu to the wiimote by the electionic music community at laige.
To achieve this, an initial simplei instiument was uesigneu anu usei testeu to gain
knowleuge anu feeuback about the implications of using wiimotes foi this
puipose. It was founu that the wiimotes aie incapable of uetecting absolute
azimuth anu velocity without the aiu of the infia-ieu cameia, a featuie not useu in
this investigation to giant mobility to the peifoimei. This ielegates commonly
accepteu musical inteiface 'stanuaius' such as pitch to a tilting motion iathei than
being tieu to absolute position which woulu allow a moie 'playable' system to be
uevelopeu.
Following tiials of the fiist instiument, the seconu instiument was uevelopeu
baseu on uata gatheieu fiom existing tiauitional acoustic instiuments anu
feeuback fiom the initial usei tiials. This appioach pioveu to be successful, with
the seconuaiy usei tiials attesting to the instiuments success. 0n the whole,
subjects founu the instiument to be moie iesponsive, with a gieatei expiessive
uepth. As well as the uynamic multilayeieu sounu engine, familiaiity, both in
teims of the physical wiimote inteiface anu subconsciously, in the way the useis
eneigy was uesigneu to steei the sounu engine in a manoi similai to existing
instiuments is pioposeu to be the ieason foi the positive iesponse. The way
contiols aie mappeu to theii sounu souices has been iuentifieu as a ciitical pait of
any successful musical inteiface. These mapping schemes must stiike a balance
between being unueistanuable anu usable to a musician, anu being complex
enough to allow foi ueep exploiation of the instiument to cieate a useis
inuiviuual 'voice' oi 'style'.
141

These instiuments have also gaineieu the inteiest of a much wiuei auuience
thiough exposuie of the pioject on inteinet social netwoiking sites, fuithei
illustiating the bioau appeal of instiuments contiolleu in this mannei.
The pioblems anu issues uiscoveieu thioughout this stuuy makes the piospect of
the auuitional sensing capabilities of the Wii Notion Plus all the moie compelling.
This is a peifect place to continue woik in the futuie.

X28 )+,&:-, ;',F
The scope foi fuithei woik is almost limitless. Beie, a summaiy of immeuiately
appaient possible uiiections aie uiscusseu.
[#"#$ ()) :+-)+, G/42
With the ielease of the Wii Notion Plus auu on (see 2.S) the uepth of possible
contiol will inciease uiamatically. Absolute one-to-one position will be able to be
measuieu by using mathematical techniques on the acceleiometeigyioscope
combination. The most obvious benefit is being able to uetach pitch fiom a ioll
motion anu expanu it lineaily in fiont of the playei. The twist motions coulu then
be useu to altei the sounu in new ways, incieasing the oiuei of the input space.
Nulti levelleu instiuments coulu also be iealiseu without the clunky siue steps
anu woik a iounus neeueu when using just a S axis acceleiometei, leauing to new
fully thiee uimensional instiuments. The SB sounu contiol options mentioneu in
section S.1.1 woulu also become a piactical ieality with the wiimote. The
imagination ieally be the only limit to the options affoiueu by this new contiol
system.
We may see a wiuei move towaius this kinu of technology with the ielease of
Sony's new Playstation motion contiollei anu Niciosoft's Natal full bouy input
gestuie inteiface (BBC 2uu9). These technologies will pioviue even moie intiicate
contiol ovei new electionic applications. viueo gaming will be most immeuiate
beneficiaiy to these technologies but in time they will inevitably be ieveise
engineeieu, paving the way foi theii use in custom expiessive applications.
142
[#"#" J?456-)+,
The wiimote coulu be useu in euucation at many uiffeient levels. The most
immeuiate example is using it as a cost effective novel gateway foi stuuents to
leain anu apply computei science concepts. If a suitable wiappei libiaiy was built
foi the language being taught, the wiimote is simple enough in its opeiations to
seive as an excellent teaching aiu to liven up the sometimes uull woilu of
founuation level piogiamming. Stuuents in music technology euucation coulu be
given instiument builuing tasks, anu otheis coulu use the wiimotes to cieate basic
games.
The univeisal simplicity of the wiimote is such that it coulu be given to any age
gioup anu seive as an effective teaching aiu. The yeai 9 stuuents shown the
Youtube viueos of this pioject showeu gieat inteiest in the instiument. If it weie
packageu in the iight way, it woulu seive as an excellent way to engage young
teenageis in music lessons by using uevices such as the wiimote, which woulu be
moie familiai to some stuuents than a claiinet oi even keyboaiu.
If the package was maue even moie basic, the veiy young anu those with leaining
uifficulties oi mental uisabilities woulu benefit fiom the sonic feeuback iesulting
fiom gestuial actions. The wiieless natuie of the wiimote means that total
fieeuom can be attaineu, peifect foi this soit of usei.
[#"#7 T4?)+ >+. \)?'+ M63),; TAA/)56-)+,2
The techniques uesciibeu in this iepoit coulu be put to use foi moie immeisive
viueo gaming expeiiences with iespect to auuio feeuback. A peison picking up
metal panel in a viueo game coulu be manipulating a physical mouel of a metal
panel being uiiven by the motion contiol on the wiimotes. This is by no uoubt be
investigateu by uevelopeis foi all the majoi platfoims, especially by Sony anu
Niciosoft. With theii new wiimote inspiieu motion sensing contiolleis (BBC
2uu9), we will piobably see a whole host of new immeisive games ieleaseu in the
next couple of yeais when this technology is uiiving the ultia-poweiful
Ploystotion S anu XB0XS60 consoles.
14S
[#"#9 *+,-.+/ +> \)246/ J/'3',-2
The focus of this pioject has been using gestuies to manipulate sounu, but these
contiolleis coulu also be useu to manipulate visuals. A feasibility stuuy into this
aiea woulu invoke as many avenues of possible uiiection as with using these
uevices to manipulate sounu. visual manipulation applications foi ait, euucation,
theiapy anu iecieation become immeuiately appaient following even the most
casual consiueiation.

[#"#< *42-+3)26E)/)-1
The clamoui foi customisability is a uifficult issue. As stateu in S.S.2.2:
'People will want to customise theii instiuments to allow them to uevelop
inuiviuual sounus anu iueas, but to maintain a sufficiently complex mapping
stiuctuie, both inteiface anu sounu souice must be uesigneu with each othei in
minu.
A solution woulu be to uevelop a giaphical piogiamming enviionment similai to
NAXNSP anu Reaktoi. A wiimote object coulu be auueu, anu vaiious mouules
coulu be auueu to the outputs of the wiimote. By simply joining a filtei block to an
acceleiometei, then uiagging a viitual cable fiom that to a peak uetectoi mouule
foi instance, a usei woulu have built half of a spheie in five clicks of a mouse. Like
Reaktoi, souice coue woulu be invisible to the usei, but an option to inseit
exteinal C mouules as in NAXNSP coulu be auueu. This woulu not only speeu up
instiument uevelopment, but level the playing fielu, allowing those without a
backgiounu in computei piogiamming to cieate theii own complex instiuments.
The ability to wiite custom mouules woulu encouiage expanuability by a usei
base which woulu inciease the available palette of mouules exponentially.
If a veision was wiitten foi Winuows, 0S X anu Linux, anybouy coulu shaie theii
instiument with anyone else, iegaiuless of opeiating system.

144
14S

.Y E$6D$'*,>A:J
SBConnexion. SB Connexion Spoce Noviqotor SB mouse. 2uu9.
http:www.Suconnexion.comSumousespacenavigatoi.php (accesseu S 4,
2uu9).
Activision. 6uitor Eero. 2uu9. www.guitaiheio.com (accesseu 6 1S, 2uu9).
AiLive. live Hotion 2. 16 Apiil 2uu9. http:www.ailive.netlm2.html (accesseu
Apiil 16, 2uu9).
Apple. 6oroqeBonJ '09. 2uu9. http:www.apple.comilifegaiagebanu
(accesseu 6 1u, 2uu9).
Applieu Acoustics Systems. Strinq StuJio. 2uu9. http:www.applieu-
acoustics.comstiingstuuio.php (accesseu S S, 2uu9).
Aifib, B. Coutuiiei, }. Kessous, L veifaille, v. "Stiategies of mapping between
gestuie uata anu sysnthesis mouel paiameteis using peiceptual spaces."
0rqoniseJ SounJ 7, no. 2 (2uu2): 142.
BBC. FS 6ominq Fxpo Jrows to o close. S 6 2uu9. http:tinyuil.combbc-eS-aiticle
(accesseu 6 8, 2uu9).
Bountouiiuis, B. Reseorcb into Ambisonic controllers usinq o SB qropbicol
representotion of tbe sounJ- fielJ. NSc, Bepaitment of Nusic anu Electionics,
0niveisity of Yoik, Yoik: 0niveisity of Yoik, 2uu7.
C ueigei, B Recktei, B Paschke, F Schulz, C Poepel. "Towaius Paiticipatoiy Besign
anu Evaluation of Theiemin-baseu Nusical Inteifaces." NlHF. uenova: NINE,
2uu8. SuS-Su6.
C Niuolo, E Castellanos, P Naii, Natan Sinigaglia. WiiWiiWiiWii.
http:ue.posi.towiiwiiwiiwii (accesseu 2 2uu9).
Electionic Nusician. "0sing Nintenuo Wiimote foi Nusic Piouuction." Flectronic
Husicion, 1 S 2uu9.
146
uamezine. PloyStotion S onJ Xbox S60 worlJwiJe soles equol in 2008. 29 1 2uu9.
http:www.gamezine.co.uknewsplaystation-S-anu-xbox-S6u-woiluwiue-
sales-equal-in-2uu8-$1264624.htm (accesseu S 4, 2uu9).
Bunt, A. RoJicol 0ser lnterfoces for reol-time musicol control. PhB Thesis,
Bepaitment of Electionics, 0niveisity of Yoik, Yoik: 0niveiisty of Yoik, 2uuu.
hupei2k. Wiimote-Roombo=Wiimbo! 26 }an 2uu9.
http:www.youtube.comwatch.v=NqbcfSqPnLA (accesseu Apiil 18, 2uu9).
K Kaiplus, A Stiong. "Bigital Synthesis of Pluckeu Stiing anu Bium Timbies."
Computer Husic }ournol (NIT Piess) 7, no. 2 (198S): 4S-SS.
Kennei, C. 6lovePlF. 4 2 2uu7.
http:cail.kennei.googlepages.comglovepie_uownloau (accesseu S 21, 2uu9).
KNC. Strinq StuJio. u1 2uu7.
https:kellysmusicanucomputeis.comPiouuctInfo.asp.c=cuniu=1117662844
tb=1 (accesseu u4 S, 2uu9).
Kionman, 0. Steel Pon Tuninq. 6 1 2uu9.
http:hotpans.sepantuningapp_a_sopiano.php (accesseu S 4, 2uu9).
lebasson. Control Ableton live witb Wiimote. 8 S 2uu7.
http:www.youtube.comwatch.v=1zA6xEphL8c (accesseu S 4, 2uu9).
Lee, }. Wii Projects. 17 Apiil 2uu9. http:johnnylee.netpiojectswii (accesseu
Apiil 17, 2uu9).
Lehiman, P Acimovic, P. Flectronic Husicol lnstrument Besiqn. 2S 4 2uu8.
http:www.tufts.euupiogiamsmmaemiu (accesseu S 4, 2uu9).
Levitin, B. NcAuams, S. Auams, R. "Contiol paiameteis foi musical instiuments: a
founuation foi new mappings of gestuie to sounu." 0rqoniseJ SounJ 7, no. 2
(2uu2): 17S.
Nanabe, B. NintenJo wiimote B} - BS v} {Boito Honobe - TCY). 17 S 2uu8.
http:www.youtube.comwatch.v=6ToaN7vTqt0 (accesseu S 4, 2uu9).
147
Nasheuuk. Wii miJi Bemo. 29 12 2uu7.
http:www.youtube.comwatch.v=QqASCPS0Kzw (accesseu S 4, 2uu9).
Nuiiay, K. B Baubl, u. "Explaining Cognitive Lock-In: The Role of Skill- Baseu
Babits of 0se in Consumei Choice." }ournol of Consumer Reseorcb (0niveisity of
Chicago Piess) S4, no. 1 (Feb 2uu7).
Nakia. T. N, Ivanov. Y, Smaiaguis. P, Ault. C. "The 0BS viitual Naestio: an
Inteiactive Conuucting System." NlHF 09. Pittsbuigh: NINE, 2uu9.
Native Instiuments. 0ser librory. S S 2uu9. http:www.native-
instiuments.cominuex.php.iu=useilibiaiy_us (accesseu S S, 2uu9).
Nelson, T. Computer lib,Breom Hocbines. Niciosoft Piess, 1974.
Nintenuo. NintenJo Iinonciol Report. Su 1u 2uu8.
http:www.nintenuo.co.jpiipuf2uu8u81uSue.puf#page=11 (accesseu S 4,
2uu9).
. Wii Husic. 2uu8. www.wiimusic.com (accesseu 6 6, 2uu9).
Nowheie2uu7. 6lovePie onJ Wii Brums. 11 11 2uu7.
http:www.youtube.comwatch.v=70LIL04KgpI (accesseu u4 29, 2uu9).
Nyko. Kymo Wireless. 2uu9. http:www.nyko.compiouuctspiouuct-
uetail.name=Kama (accesseu 6 1, 2uu9).
ocukloweuk. WiiToHiJi. 22 1 2uu8. http:www.youtube.comwatch.v=20Nni1-
P9os (accesseu S 4, 2uu9).
Peek, B. HonoqeJ librory for NintenJo's Wiimote. u7 S 2uu7.
http:blogs.msun.comcouing4funaichive2uu7uS141879uSS.aspx
(accesseu 6 6, 2uu9).
sapphiieS27. Wiimote Syntbesizer. 2uu6 12 2uu9.
http:www.youtube.comwatch.v=WzBu0TnbN-E (accesseu S 2uu9, 4).
ScieenFasion. Wiinstrument. 18 Apiil 2uu9.
http:scieenfashion.oigieleasesthe_wiinstiument (accesseu Apiil 18, 2uu9).
148
Smith, }. BoweJ Strinqs. 1u 12 2uu8.
http:ccima.stanfoiu.euu~jospaspBoweu_Stiings.html (accesseu S S, 2uu9).
Sounu 0n Sounu. "Nintenuo's Wii Remote as a NIBI Contiollei." SounJ 0n SounJ,
1u 2uu8: S2-S7.
Suiek, C. Wireless EonJ-belJ Spotiol Controller. NSc, Bepaitment of Nusic anu
Electionics, 0niveisity of Yoik, Yoik: 0niveisity of Yoik, 2uu8.
The Wiitles. Tbe Wiitles. 2uu8. http:iyanpeoples.comthewiitles.com (accesseu
2 18, 2uu9).
Tioilaiu, Camile. 0SCulotor. 12 u6 2uu9. http:www.osculatoi.net (accesseu u6
1S, 2uu9).
veiuone, Nike. WiiToHiJi. 2uu7. http:mike.veiuonewiitomiui (accesseu 6 S,
2uu9).
visualisationBIAB. Wii Eock . 1S 1 2uu7.
http:www.youtube.comwatch.v=ESIB-WopS6Y (accesseu S 4, 2uu9).
Waisvisz, Nichal. RounJ toble. Sept 1999.
http:www.ciackle.oigNW's%2ugestuial%2uiounu%2utable.htm (accesseu
}une 1, 2uu9).
Waisvisz, Nichel. "The hanus: a set of iemote miui-contiolleis." Computer Husic
Conference . Buinaby, B.C., 198S. S1S-S18.
Wiilii.oig. Wiimote,Brivers. 14 9 2uu8.
http:www.wiili.oiginuex.phpWiimote_uiivei (accesseu S 4, 2uu9).

149

/BB#15'"`
J21&#1&7
Appenuix A Wiimote Connection anu BIB Banuling
Appenuix B 0sei guiue to the Integiateu Instiument
Appenuix C Notes on 0nueistanuing Reaktoi Layouts
Appenuix B Instiument Analysis Bata
Appenuix E NIBI
Appenuix F Wiimote Bata Sheet
Appenuix u ABXLSSu Bata Sheet
Appenuix B Softwaie Class Tables
Appenuix I Souice Listing

1Su

PAA-/0$N P ;$$<'&- G'//-1&$'/ >/0 ="H =>/0D$/*
Befoie the uata can be ieau fiom the wiimote, it must be auueu to the Winuows
BIB list, iuentifieu on this list anu have a file stieam set up to communicate with
it.
Auuing the wiimote to the BIB list in the fiist instance is a simple task. The usei
must set theii computei into uiscoveiy moue anu then piess the 1+2 buttons
simultaneously on the wiimote. If a wiimote has been useu pieviously, it must be
uninstalleu fiist befoie this piocess can take place. To speeu up this laboiious
piocess of continuously installing anu uninstalling, the 'WiimoteAutoConnect'
application can be useu to automate this piocess.
0nce this has been uone, WiimoteLib can be useu to set up a uata stieam to the
wiimote.
This piocess uetaileu below is manageu inteinally in WiimoteLib but is incluueu
heie foi completeness.
Taken fiom WiimoteLib web site (Peek 2uu7):
1. 6et tbe 60lB {6lobolly 0nique lJentifier) of tbe ElB closs JefineJ by
WinJows
2. 6et o bonJle to tbe list of oll Jevices wbicb ore port of tbe ElB closs
S. Fnumerote tbrouqb tbose Jevices onJ qet JetoileJ informotion obout eocb
4. Compore tbe venJor lB onJ ProJuct lB of eocb Jevice to tbe known
Wiimote's vlB onJ PlB
S. Wben founJ, creote o IileStreom to reoJ,write to tbe Jevice
6. Cleon up tbe Jevice list
Boto from tbe wiimotes will be sent olmost continuously onJ os sucb, osyncbronous
l,0 sboulJ be useJ. Tbe .NFT monifestotion of tbis is to set up o buffer onJ run o
collbock metboJ wben tbis buffer if full. Tbe Joto is tben bonJleJ onJ tbe process is
repeoteJ.

1S1
Foi uetaileu infoimation iegaiuing the bitwise makeup of the wiimotes messages
anu its haiuwaie specifications, please see appenuix F.
1S2

PAA-/0$N E R%-, *+$0- &' &:- "/&-*,>&-0 "/%&,+<-/&
Befoie beginning, make suie you have the following set up on youi computei:
- Niciosoft Winuows XP
- Native Instiuments Reaktoi S
- NIBI Yoke (on accompanying CB oi fiom www.miuiox.com)
- Wii Remote AutoConnect (on accompanying CB)
- A suitably fast ASI0 sounu caiu will impiove iesponsiveness. It is not
essential to get sounu out of the system, but it makes the instiument much
moie playable
- Bluetooth peiipheial anu suitable Bluetooth softwaie stack which is
installeu anu activateu
It is woith noting that wiimotes weie nevei uesigneu to connect to peisonal
computeis, so some patience may be iequiieu at this point. It uoesn't mattei if
you inseit the nunchuck contiollei into the wiimote befoie oi aftei connection.
1. Take both wiimotes anu piess 1 anu 2 buttons simultaneously. The lights on the
fiont will begin to flash.
2. Launch Wii Remote AutoConnect anu wait foi 1u - Su seconus. If the following
scieen appeais anu botb wiimotes still have flashing lights (wait about S oi 1u
seconus ofter seeing the following scieen to confiim this) then you have
succeeueu in connecting both wiimotes. Bevice names may vaiy.

1SS

If only one wiimote has flashing lights, piess 1 anu 2 on the non-flashing wiimote
anu piess 'Attempt Connection Again' on Wii Remote Autoconnect. If neithei
wiimote is still flashing, iepeat fiom step 1. You uon't neeu to iestait Wii Remote
AutoConnect, just click 'Attempt Connection Again.'
S. If youi wiimotes aie both connecteu, launch Combo.exe (on accompanying CB).
The LEBs on the wiimotes shoulu change fiom constant flashing to two uiffeient
patteins.

4. Select NIBI Y0KE 1 fiom the uiop uown uevice list menu on the
ComboInstiument application.
S. Stait Reaktoi anu open the patch labelleu Combo.ens (on accompanying CB).
You shoulu be gieeteu with the following scieen:
1S4

If youi scieen looks like this:

Click on the labelleu button above.
7. To set up Reaktoi: In Reaktoi, navigate to System->Auuio + NIBI settings
Change the output uevice to youi uesiieu sounucaiu. Click the NIBI tab anu
accept incoming NIBI fiom NIBI Yoke 1.

Pioviuing youi haiuwaie is configuieu piopeily anu opeiational, you shoulu have
two woiking instiuments. If you finu theie is a uelay oi auuible clicks, tiy
expeiimenting with uiffeient auuio buffei lengths. These can be founu
Auuio+NIBI settings tab. To uisconnect, simply holu the powei button on the
wiimote until the lights go out.
1SS
?17&-%+#1& <B#-,&'21 *%++,-.`
EF# *BF#-# (inuicateu by lights on, off, off, on) is a tuneu peicussion instiument,
similai to a xylophone. Notes aie stuck with a shaip motion in the uiiection of the
'tiiggei'. Pitch is ueteimineu by the tilt of the wiimote accoiuing to the following
layout:

Sounu can be changeu by piessing up anu uown buttons on the wiimote.

EF# J#442 (inuicateu by lights off, on, on, off) is a continuous pitch instiument. It
is activateu by holuing uown the B button (tiiggei) on the wiimote. The iight aim
injects eneigy into the system - move it aiounu with the tiiggei to uo this. The tilt
of the wiimote ueteimines how 'thin' the sounu it. The left hanu changes pitch,
twist to change pitch. It can be tiackeu by use of the on scieen uisplay. Bolu the A
button to change between uisciete 'fietteu' pitch anu continuous pitch.
The C anu Z buttons on the nunchuck will make pitch jump to one of S inteivals of
fifths. Z = up S
th
, C + Z = up 2 S
th
, just C = up S Sths.
The joystick intiouuces haimony anu the tilt of the nunchuck incieases
satuiation.
The moie eneigy you inject, the moie the sounu changes.

1S6

PAA-/0$N G Z'&-% '/ R/0-,%&>/0$/* S->F&', #>J'+&%
This section shoulu seive as a light intiouuction to Reaktoi layouts to aiu in
inteipieting some of the uiagiams in this iepoit.
In Reaktoi, signals always flow fiom left to iight. These signals aie ielayeu by
simply clicking anu uiagging lines fiom the poits of one mouule to anothei.
Wheie possible, the labels on mouules' poits aie self explanatoiy (P foi pitch, A
foi amplituue etc). Nouules tiansmit anu ieceive two types of signals:
Asynchionous event signals (NIBI anu contiol signals) anu auuio, which is tieu to
a clock. Both of these signals have the ability to mouulate anu contiol othei
signals, but it shoulu be noteu that inteimeuiaiy mouule aie sometimes iequiieu
to latch the asynchionous event signals to pievent clicks appeaiing in the auuio
stieam when auuio signals aie being mouulateu in this way.
The Reaktoi stiuctuie is hieiaichical, that is to say that some mouules can
encapsulate moie complex sub stiuctuies to make 'black boxes', neatening up the
layout. Wheie possible, all necessaiy layeis to allow unueistanuing of a paiticulai
concept have been shown in this iepoit.
Let us take a simple example - a subtiactive synthesisei.

In the above example, a sawtooth oscillatoi mouule is having its pitch alteieu by
the incoming pitch of a NIBI note on signal. It is woith noting at this point that
Reaktoi splits NIBI note on commanus into two signals: floating point values:
Pitch anu uate, wheie the gate signal moves between u (foi NIBI u) anu 1 (foi
NIBI 127) uepenuing on note velocity. At times when theie is no note being
activateu, gate iemains at u anu the output is silent. When a note on is stiuck, the
value of the gate is multiplieu by the contiollei signal, which in this case coulu be
1S7
a static volume contiol anu a tone is geneiateu fiom the oscillatoi. The input foi
amplituue on the Sawtooth mouule expects auuio input, anu so in this case we use
an AuuioSmoothei mouule to latch the event output to the sample iate clock. This
signal is then feu into the filtei, which is having its fiequency changeu by a knob
on the fiont panel of Reaktoi. This mouule expects the centie fiequency to be
given in pitch, but we can use one of Reaktois mathematics blocks to conveit the
lineai fiequency scale to the logaiithmic pitch scale.
The signal is then feu thiough a voice combinei. This ensuies that in the event of
multiple notes being cieateu, Reaktoi will piocess them all to achieve polyphonic
output. Without this mouule, the usei is limiteu to 1 voice.
A much moie in uepth ieview can be founu in the Reaktoi usei guiue on the
accompanying CB.
1S8

PAA-/0$N H "/%&,+<-/& P/>DJ%$% H>&>
The uata fiom instiument analysis not uiiectly useu in the uesign of the cello is
collateu anu piesenteu.

U'6%-# N> K2%=4# =,77e '1$-#,7'16 =23 B-#77%-#
1S9

U'6%-# N@ K2%=4# =,77e F,-+21'$7 7F231 7'5# 21

U'6%-# NL A%+,1 (2'$#e '1$-#,7'16 (24%+#

16u

U'6%-# ND A,-+21'$7 2C F%+,1 (2'$#

161
U'6%-# NW A,-+21'$ $21&#1& 2C F%+,1 (2'$# 3'&F '1$-#,7'16 (24%+#

U'6%-# NN Y'24'1e '1$-#,7'16 =23 B-#77%-#

162

PAA-/0$N M C"H"
The Nusical Instiument Bigital Inteiface (NIBI) is an inuustiy stanuaiu piotocol
foi allowing electionic instiuments to communicate anu synchionise with each
othei. It was uefineu in 1982 to combat the exponential inciease in conflicting
piopiietaiy contiol voltage (Cv) schemes being useu by synthesisei
manufactuieis towaius the enu of the 197us. This new stanuaiu alloweu foi the
sepaiation of the inteiface anu sounu souice, making an inteiface uesigneu by one
manufactuiei able to inteiface with a sounu mouule maue by anothei thiough the
NIBI stanuaiu. This gave keyboaiu playeis the fieeuom to uo away with the vast
aiiays of analogue synthesiseis they hau been tieu to in the 197us, anu have one
keyboaiu contiolling multiple uevices in outboaiu iacks, incieasing ieliability anu
iemoving the uifficulties with pieviously accompanieu touiing with laige
keyboaiu set ups. NIBI also founu itself as an inuispensible tool in electionic
music stuuios, paving the way foi sequenceis anu the uevelopment of electionic
uance music.
H?K? *B#$'C'$,&'21
Eveiy NIBI message consists of an 8-bit status byte, followeu by up to two 7 bit
uata bytes. The specification uetails two basic types of messages: Channel
Nessages anu System Nessages.
JF,11#4 H#77,6#7
The NIBI specification allows foi multiple uevices to ieceive inuepenuent
messages on the same bus, by specifying one of the sixteen NIBI channels in the
fiist nibble of the fiist byte of the NIBI message. The seconu nibble is useu to
uenote the type of message to be sent to that paiticulai channel. The pioceeuing
two bytes contain infoimation about the specifieu channel message which note to
hit anu how haiu to play it foi example.

Status Byte Meaning Data Byte 1 Data Byte 2
0x8c Note Off Pitch Velocity
16S
0x9c
0xAc
0xBc
0xCc
0xDc
0xEc
Note On
Key Pressure
Controller Change
Program Change
Channel Pressure
Pitch Bend
Pitch
Pitch
Controller
Program
Pressure
Bend LSB
Velocity
Pressure
Value
NOT USED
NOT USED
Bend MSB
E,=4# X H?K? JF,11#4 H#77,6#79 EF# O$P 2C &F# 7&,&%7 =.&# '15'$,&#7 &F# 7B,$# C2- &F# $F,11#4 1%+=#-
7B#$'C'#5 =. &F# &-,17+'&&'16 5#('$#
Note 0n Note 0ff
Pitch in iange u-127 (C-1 to u9) with miuule C as 6u.
velocity in iange u-127 is how haiu the note was stiuck. velocity of u on a Note
0n event has the same effect as a Note 0ff event.
Key Piessuie
Also known as oftertoucb, key piessuie allows a note paiametei to be alteieu
aftei the note on commanu has been sent, but befoie the note off commanu is
ieceiveu by the host. It most commonly manifests itself on keyboaiu as piessuie
sensois on eveiy inuiviuual key, allowing a playei to stiike a note anu piess uown
haiuei to affect a ceitain aspect of the sounu. Range u-127.
Contiollei Change
The contiol change messages allow foi up to 12u continuous contiollei messages
(121 - 127 aie ieseiveu foi special system messages) to be sent uown each NIBI
channel, each with a iesolution of u-127. The messages noimally take the physical
foim of knobs an sliueis on keyboaius foi contiolling things like filtei cutoff
fiequencies oi low fiequency oscillatoi uepths.
Piogiam Change
0seu to change between u-127 piesets foi easy iecall.
Channel Piessuie
Functions in a veiy similai way to key piessuie, but woiks ovei a whole channel
iathei than inuiviuual notes.
Pitch Benu
By saciificing the ability to specify inuiviuual notes, The pitch benu commanu
uses both uata bytes foi its value, thus incieasing its iesolution fiom u-127 (2
7
) to
u-16S84 (2
14
). The pitch benu commanu noimally manifests itself on keyboaius
164
as a wheel on a panel to the left of the keyboaiu. The actual iange of the pitch
benu is ueteimineu by the synthesisei itself, not by the NIBI message, the 14-bit
numbei only iepiesenting the tiavel of the pitch benu contiollei.
*.7&#+ H#77,6#7
System messages aie most often implementeu on sequenceis to uo things like
stait anu stop anu tiack, oi skip to specific sections of a composition. Theie is also
space allocateu foi System-Fxclusive messages, which can be utiliseu by
manufactuieis foi the contiol of special non-stanuaiu featuies of theii piouucts.
These system messages have not been useu in any of the softwaie wiitten foi this
pioject, but moie infoimation can be founu at http:www.miui.oig.
16S

PAA-/0$N ) ;$$<'&- H>&> @:--&
Wii remote circuit board, top
surface
Wii remote circuit board, bottom
surface
Broadcom BCM2042 in a Wii
remote
Wiimote
From WiiBrew
This article is a technical guide to the Wii Remote. For a
high-level overview of the Wii Remote, see the Wikipedia
entry (http://en.wikipedia.org/wiki/Wii_Remote) .
Disassembled Firmware
Extension Controllers
Motion analysis
Pointing
Library
Mii Data
The Wii Remote (informally known as the Wiimote) is the
Wii's main input device. It is a wireless device, using standard
Bluetooth technology to communicate with the Wii. It is built
around a Broadcom BCM2042 (http://www.broadcom.com
/products/Bluetooth/Bluetooth-RF-Silicon-and-Software-
Solutions/BCM2042) bluetooth System-on-a-chip, and contains
multiple peripherals that provide data to it, as well as an
expansion port for external add-ons. The Wii Remote uses (and,
at times, abuses) the standard Bluetooth HID protocol to
communicate with the host, which is directly based upon the
USB HID (http://en.wikipedia.org
/wiki/USB_human_interface_device_class) standard. As such, it
will appear as a standard input device to any Bluetooth host.
However, the Wii Remote does not make use of the standard
data types and HID descriptor, and only describes its report
format length, leaving the actual contents undened, which
makes it useless with standard HID drivers (but some Wiimote
Drivers exist). The Wii Remote actually uses a fairly complex set of operations, transmitted through
HID Output reports, and returns a number of different data packets through its Input reports, which
contain the data from its peripherals.
Contents
1 Summary
2 Bluetooth Communication
2.1 HID Interface
2.1.1 Output Report common information
2.1.2 Input Report common information
3 Status Reporting
3.1 0x20: Status
3.2 0x21: Read Memory Data
Wiimote - WiiBrew le:///Users/maxleonard/Desktop/web/Wiimote.html
1 of 26 16/6/09 13:42
3.3 0x22: Acknowledge output report, return function result
4 Data Reporting
4.1 0x30: Core Buttons
4.2 0x31: Core Buttons and Accelerometer
4.3 0x32: Core Buttons with 8 Extension bytes
4.4 0x33: Core Buttons and Accelerometer with 12 IR bytes
4.5 0x34: Core Buttons with 19 Extension bytes
4.6 0x35: Core Buttons and Accelerometer with 16 Extension Bytes
4.7 0x36: Core Buttons with 10 IR bytes and 9 Extension Bytes
4.8 0x37: Core Buttons and Accelerometer with 10 IR bytes and 6 Extension Bytes
4.9 0x3d: 21 Extension Bytes
4.10 0x3e / 0x3f: Interleaved Core Buttons and Accelerometer with 36 IR bytes
5 Memory and Registers
5.1 Reading and Writing
5.2 EEPROM Memory
5.3 Control Registers
6 Input Features
6.1 Buttons
6.1.1 Core Buttons
6.1.2 Power Button
6.1.3 Sync Button
6.1.4 Button Hardware
6.2 Accelerometer
6.2.1 Normal Accelerometer Reporting
6.2.2 Interleaved Accelerometer Reporting
6.3 IR Camera
6.3.1 Mechanical Characteristics
6.3.2 Optical Characteristics
6.3.3 Initialization
6.3.4 Sensitivity Settings
6.3.5 Data Formats
6.3.5.1 Basic Mode
6.3.5.2 Extended Mode
6.3.5.3 Full Mode
7 Feedback Features
7.1 Player LEDs
7.2 Rumble
7.3 Speaker
7.3.1 Initialization Sequence
7.3.2 Speaker Conguration
7.3.3 Sound Data Format
8 Extension Controllers
8.1 Registers / Initialization
8.2 Identication
8.2.1 The Old Way
8.2.2 The New Way
8.2.3 Encryption setup
8.3 Extension Controller Documentation
2 of 26 16/6/09 13:42
9 Acknowledgements
Summary
Reverse engineering and documenting all of the Wii Remote's features is a work in progress. Here
are the known features and their status:
Bluetooth Communication
Connection to the Wii Remote using the 1+2 sync sequence
works sometimes, and single-press reconnection is
understood.
Core Buttons All working.
Accelerometer All working.
IR Camera All working.
Power Button
Pressing Power turns disconnects and turns off the Wii
Remote. Power-on probably similar to single-button
reconnect.
Speaker All working.
Player LEDs
Can be controlled by software arbitrarily. Even brightness
modulation works.
Status Information Battery and extension info in Status Report
Extension Controllers Wireless Nunchuks and Motion Plus don't work at all
Legend
Perfect or near-perfect Usable but not complete Unusable
Bluetooth Communication
When queried with the Bluetooth Service Discovery Protocol (SDP (http://www.palowireless.com
/infotooth/tutorial/sdp.asp) ), the Wii Remote reports back a great deal of information. In particular,
it reports:
Name Nintendo RVL-CNT-01
Vendor ID 0x057e
Product ID 0x0306
Major Device Class 1280
Minor Device Class 4
Service Class 0
(Summary of all Class Values) 0x002504
The Wii Remote does not appear to require any of the authentication or encryption features of the
3 of 26 16/6/09 13:42
Bluetooth standard. In order to interface with it, it must rst be placed in discoverable mode by either
pressing the 1 and 2 buttons at the same time, or by pressing the red sync button under the battery
cover. Once in this mode, the Wii Remote can be queried by the Bluetooth HID driver on the host. If
the HID driver on the host does not connect to the Wii Remote within 20 seconds, the Wii Remote
will turn itself off. Holding down the 1 and 2 buttons continuously will force the Wii Remote to stay
in discoverable mode without turning off. This does not work with the sync button, however. When
in discoverable mode, a number of the player LEDs based on the battery level will blink.
The "syncing" of a Wii Remote involves standard Bluetooth pairing. When the Sync button is
pressed on the remote, it will accept pairing requests. The required PIN is the hosts's Bluetooth
address, backwards (last byte rst), in binary (6 bytes). Most current Bluetooth implementations
don't deal with this correctly, as they usually consider the PIN to be a regular ASCII string (no 00
bytes, etc). Any further steps that need to be taken after the Wii Remote is paired have not been
reverse engineered yet.
Once the Wii Remote is synced, when a button is pressed, it will actively seek out its paired host and
try to connect to it, instead of the other way around. Establishing a connection can be done on PSM
0x11 for writing and PSM 0x13 for reading using the Bluetooth L2CAP protocol.
HID Interface
The HID standard allows devices to be self-describing, using a HID descriptor block. This block
includes an enumeration of reports that the device understands. A report can be thought of similar to
a network port assigned to a particular service. Reports are unidirectional however, and the HID
descriptor lists for each port the direction (Input or Output) and the payload size for each port. Like
all Bluetooth HID devices, the Wii Remote reports its HID descriptor block when queried using the
SDP protocol. However, no information regarding the actual data units within each report is returned,
only the length in bytes.
Note: An "Input" report is sent by the Wii Remote to the host. An "Output" report is sent by the host
to the Wii Remote.
These are the reports the Wii Remote uses, and their use:
I/O ID(s) Size Function
O 0x10 1 Unknown
O 0x11 1 Player LEDs
O 0x12 2 Data Reporting mode
O 0x13 1 IR Camera Enable
O 0x14 1 Speaker Enable
O 0x15 1 Status Information Request
O 0x16 21 Write Memory and Registers
O 0x17 6 Read Memory and Registers
O 0x18 21 Speaker Data
O 0x19 1 Speaker Mute
4 of 26 16/6/09 13:42
O 0x1a 1 IR Camera Enable 2
I 0x20 6 Status Information
I 0x21 21 Read Memory and Registers Data
I 0x22 4 Acknowledge output report, return function result
I 0x30-0x3f 2-21 Data reports
For clarity, the convention in this document is to show packets including the Bluetooth-HID
command (in parentheses), report ID, and payload, as described in sections 7.3 and 7.4 of the
Bluetooth HID specication (http://www.bluetooth.com/NR/rdonlyres/0BE438ED-DC1B-41D1-
AAC0-1AAA956097A2/980/HID_SPEC_V10.pdf) . Each byte is written out in hexadecimal,
without the 0x prex, separated by spaces. For example,
(a1) 30 00 00
is a DATA input packet (0xa1)(0x00 on European Wii remotes), on channel 0x30, with the two byte
payload 0x00, 0x00. When using higher level HID functions rather than Bluetooth functions, the
bytes in parentheses will never be present.
Force Feedback is accessible through the rst byte of ALL output reports in the same way. This is
not included above to avoid clutter.
Output Report common information
The rst byte in many Output reports has a similar meaning. In every single Output Report, bit 0
(0x01) of the rst byte controls the Rumble feature. Additionally, bit 2 (0x04) is used in several
Output Reports as the ON/OFF ag for the specic feature controlled by it. For example, sending
0x04 to Report 0x19 (Speaker Mute) will mute the speaker:
(52) 19 04
Sending 0x00 will unmute it:
(52) 19 00
These Output Reports share the above behavior: Data Reporting Mode (0x12), IR Camera Enable
(0x13), Speaker Enable (0x14), Speaker Mute (0x19), IR Enable 2 (0x1a).
Input Report common information
The rst two bytes of ALL input reports, except 0x3d, contain the Core Buttons (BB BB). This
includes all the 0x2~ status reports, not just the 0x3~ data reports. 0x3d is an exception, since it only
returns expansion information.

5 of 26 16/6/09 13:42
Status Reporting
0x20: Status
To request the status report, send anything to Output Report 0x15. The Status Report will also be
automatically sent when an Extension Controller is connected or disconnected.
This will request the status report (and turn off rumble):
(52) 15 00
This report is sent either on request (in response to report 0x15), or in response to an expansion being
plugged in or unplugged (or synced if wireless). If this report is received when not requested, the
application 'MUST' send report 0x12 to change the data reporting mode, otherwise no further data
reports will be received.
(a1) 20 BB BB LF 00 00 VV
BBBB is the core Buttons data. VV is the current battery level, L is the LED state, and F is a bitmask
of ags indicating, whether the battery is at, whether an expansion is currently connected, etc.
The Wii Remote can report its status, which includes the state of a few basic settings, the status of
the Extension Controller (connected or disconnected), and the battery level.
BB is the current battery level, and FF is a bitmask of ags:
Bit Mask Meaning
0 0x01 Battery is nearly empty
1 0x02 An Extension Controller is connected
2 0x04 Speaker enabled
3 0x08 IR camera enabled
4 0x10 LED 1
5 0x20 LED 2
6 0x40 LED 3
7 0x80 LED 4
0x21: Read Memory Data
This report is sent when a read memory request is made. It returns 1 to 16 bytes of data at a time.
(a1) 21 BB BB SE AA AA DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD
BBBB is the core Buttons data.
6 of 26 16/6/09 13:42
AA AA are the 2 least signicant bytes of the absolute memory address of the rst byte of data
returned (the high byte of the offset is not returned, and neither is whether it is a register or memory
that is being used. Thus, this must be known from the read request).
E (low nybble of SE) is the error ag. Known error values are 0 for no error, 7 when attempting to
read from a write-only register or an expansion that is not connected, and 8 when attempting to read
from nonexistant memory addresses.
S (high nybble of SE) is the size in bytes, minus one, for the current data packet. This is 0xf (16
bytes) for all but the last packet, where it might be less if the requested number of bytes is not a
multiple of 16. The DD bytes are the data, padded with zeroes to 16 bytes. If more than 16 bytes are
requested, multiple packets will be received, with AA AA addresses increasing by 16 each time.
0x22: Acknowledge output report, return function result
This input report is sent to the host to report an error related to an output report, or the function result
from that output report. It is only sent under certain conditions.
Attempting to send an output report using the WriteFile method on the Microsoft Bluetooth Stack
causes one such error for all output reports except report 16H (write to memory). Report 16H reports
success when using the WriteFile method. That may be because report 16H is 22 bytes long unlike
other reports (I haven't tested Report 18H), or may be a special behaviour of report 16H.
(a1) 22 BB BB RR EE
RR is the output report number that the Wii remote is acknowledging it received.
EE is the error code or function result. 00 = success. 03 = error, such as using WriteFile on the
Microsoft stack. 04 = unknown. (possibly returned by report 16H, 17H or 18H) 05 = unknown
(possibly returned by report 12H). 08 = unknown (possibly returned bt report 16H).
Data Reporting
The Wii Remote has a number of different data reporting modes. Each of these modes combines
certain Core data features with data from external peripherals, and sends it to the host through one of
the report IDs, determined by the mode. The data format from the peripherals is determined by the
peripherals themselves, all the Wii Remote controller does is pull bytes from them and send them out
to the host. Due to this, certain feature combinations are not available, as there are not enough bytes
for them in any of the output modes.
The Data Reporting Mode is set by sending a two-byte command to Report 0x12:
(52) 12 TT MM
Bit 2 of TT species whether continuous reporting is desired. If bit 2 (0x04) is set, the Wii Remote
7 of 26 16/6/09 13:42
will send reports whether there has been any change to the data or not. Otherwise, the Wii Remote
will only send an output report when the data has changed.
MM species the Reporting Mode. Each Mode is specied by the Output Report ID that the data will
be sent to. For example, this will set mode to 0x33:
(52) 12 00 33
Data will then arrive through Input Report 0x33.
Upon powerup, the Data Reporting Mode defaults to 0x30. Following a connection or disconnection
event on the Extension Port, data reporting is disabled and the Data Reporting Mode must be reset
before new data can arrive.
Modes which include Accelerometer data also embed part of it in the unused Buttons bits. In all
modes except for 0x3e/0x3f, the Buttons data includes the LSBs of the Accelerometer data. In mode
0x3e/0x3f, the interleaved Buttons data includes the Z-axis Accelerometer data.
0x30: Core Buttons
This mode returns data from the buttons in the Wii Remote:
(a1) 30 BB BB
0x31: Core Buttons and Accelerometer
This mode returns data from the buttons and the accelerometer in the Wii Remote:
(a1) 31 BB BB AA AA AA
BBBB is the core Buttons data. AA AA AA is the Accelerometer data.
0x32: Core Buttons with 8 Extension bytes
This mode returns data from the buttons in the Wii Remote, and data from an extension controller
connected to it:
(a1) 32 BB BB EE EE EE EE EE EE EE EE
BBBB is the core Buttons data. The 8 EE bytes are from the Extension Controller currently
connected to the Wii Remote.
0x33: Core Buttons and Accelerometer with 12 IR bytes
8 of 26 16/6/09 13:42
This mode returns data from the buttons, accelerometer, and IR Camera in the Wii Remote:
(a1) 33 BB BB AA AA AA II II II II II II II II II II II II
BBBB is the core Buttons data. AA AA AA is the Accelerometer data. The 12 II bytes are from the
built-in IR Camera.
0x34: Core Buttons with 19 Extension bytes
This mode returns data from the buttons in the Wii Remote, and data from an extension controller
connected to it:
(a1) 34 BB BB EE EE EE EE EE EE EE EE EE EE EE EE EE EE EE EE EE EE EE
BBBB is the core Buttons data. The 19 EE bytes are from the Extension Controller currently
connected to the Wii Remote.
0x35: Core Buttons and Accelerometer with 16 Extension Bytes
This mode returns data from the buttons and accelerometer in the Wii Remote, and data from an
extension controller connected to it:
(a1) 35 BB BB AA AA AA EE EE EE EE EE EE EE EE EE EE EE EE EE EE EE EE
BBBB is the core Buttons data. AA AA AA is the Accelerometer data. The 16 EE bytes are from the
Extension Controller currently connected to the Wii Remote.
0x36: Core Buttons with 10 IR bytes and 9 Extension Bytes
This mode returns data from the buttons and IR camera in the Wii Remote, and data from an
extension controller connected to it:
(a1) 36 BB BB II II II II II II II II II II EE EE EE EE EE EE EE EE EE
BBBB is the core Buttons data. The 10 II bytes are from the built-in IR Camera, and the 9 EE bytes
are from the Extension Controller currently connected to the Wii Remote.
0x37: Core Buttons and Accelerometer with 10 IR bytes and 6
Extension Bytes
This mode returns data from the buttons, accelerometer, and IR camera in the Wii Remote, and data
from an extension controller connected to it:
(a1) 37 BB BB AA AA AA II II II II II II II II II II EE EE EE EE EE EE
9 of 26 16/6/09 13:42
BBBB is the core Buttons data. AA AA AA is the Accelerometer data. The 10 II bytes are from the
built-in IR Camera, and the 6 EE bytes are from the Extension Controller currently connected to the
Wii Remote.
0x3d: 21 Extension Bytes
This mode returns data from an extension controller connected to the Wii Remote. It is the only input
report that does not include core buttons.
(a1) 3d EE EE EE EE EE EE EE EE EE EE EE EE EE EE EE EE EE EE EE EE EE
The 21 EE bytes are from the Extension Controller currently connected to the Wii Remote.
0x3e / 0x3f: Interleaved Core Buttons and Accelerometer with
36 IR bytes
Both 0x3e and 0x3f are equivalent, and return data alternately through report IDs 0x3e and 0x3f. The
data is interleaved, and is returned at half the speed of other modes (as two reports are needed for a
single data unit). This mode returns data from the buttons, accelerometer, and IR camera in the Wii
Remote:
(a1) 3e BB BB AA II II II II II II II II II II II II II II II II II II
(a1) 3f BB BB AA II II II II II II II II II II II II II II II II II II
BBBB is the core button data, as specied in the Buttons section. AA AA is the Accelerometer data,
in a format specic to this mode described in the Interleaved Accelerometer Reporting section. The
36 II bytes are from the built-in IR Camera.
Memory and Registers
The Wii Remote includes a built-in EEPROM memory, part of which is accessible to the user to
store that. This user part is used to store calibration constants, as well as the Mii Data. Additionally,
many peripherals on the Wii Remote have registers which are accessible through a portion of the
address space.
Both built-in memory and peripheral registers are accessed using the same reports, where a ag is
used to select between the two.
Reading and Writing
To read data, commands are sent to Output Report 0x17:
(52) 17 MM FF FF FF SS SS
FF FF FF is the offset, and SS SS is the size to read in bytes (both in big-endian format). Bit 2 (0x04)
10 of 26 16/6/09 13:42
of MM selects the address space. Clearing this bit results in reading from EEPROM Memory, while
setting it results in reading from the control registers. Setting bit 3 (0x08) also works to access
registers, but setting both results in errors. As with all other reports, it also includes the Rumble ag,
which must be set to the current rumble state to avoid affecting it.
Data read is returned through Input Report 0x21:
(a1) 21 BB BB SE FF FF DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD
BB BB is the state of the buttons on the Wii Remote. During data reads, regular input reporting is
temporarily suspended. Button data is available through the data input reports, but no other input
data can be collected while the transfer lasts. FF FF is the offset expressed in absolute memory
address of the Wii remote memory for the rst byte of data returned (the high byte of the offset is not
returned, and neither is which data memory is being used. Thus, this must be known from the read
request). E (low nybble of SE) is the error ag. Known error values are 0 for no error, 7 when
attempting to read from a write-only register, and 8 when attempting to read from nonexistant
memory. S (high nybble of SE) is the size in bytes, minus one, for the current data packet. This is 0xf
(16 bytes) for all but the last packet, where it might be less if the requested number of bytes is not a
multiple of 16. The DD bytes are the data, padded with zeroes to 16 bytes. If more than 16 bytes are
requested, multiple packets will be received, with FF FF offsets increasing by 16 each time.
To write data, commands are sent to Output Report 0x16:
(52) 16 MM FF FF FF SS DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD
The meaning of the bytes is the same as during reads, except that size can be a maximum of 16 bytes
(as there is only space for that much data), and the actual data to write follows (the DD bytes),
padded out to 16 bytes.
Some kind of acknowledgement is received on Input Report 0x22. This has not been investigated
yet.
EEPROM Memory
There is a 128kbit (= 16kB) EEPROM chip (Data Sheet
(http://www.st.com/stonline/products/literature/ds/4578/m24128-
bw.pdf) / Full EEPROM dump from a sample Wii Remote
(http://www.sparkfun.com/tutorial/WiiRemote/NintendoWii-
I2C-Data.zip) ) in the Wii Remote. Part of its contents include code
for the built-in microcontroller, and a generic section which can be
freely read and written by the host. This section is 0x1700 bytes
long, and part of this memory is used to store the Mii Data. It can
be accessed by reading from/writing to addresses 0x0000-0x16FF
in the Wii Remote's virtual memory space; in the actual EEPROM
chip, the data is located at 0x0070-0x176F.
The rmware stored in the Wiimote has been disassembled.
11 of 26 16/6/09 13:42
The BCM2042 (http://www.broadcom.com/products/Bluetooth/Bluetooth-RF-Silicon-and-Software-
Solutions/BCM2042) microcontroller built into the Wii Remote includes a large 108kb on-chip
ROM section for storing rmware. If the EEPROM chip really contains code for the BCM2042 then
this was probably done to make rmware updates possible, so there might be a way of accessing the
other parts of the EEPROM via Bluetooth as well. From the BCM2042 Product Brief
(http://www.broadcom.com/collateral/pb/2042-PB03-R.pdf) : "ROM-based design eliminates
external ash memories; Flash option offered to support feature development".
On a virgin Wii Remote, acquired separately (not bundled with a Wii), that has never communicated
with any device (except the PC used to dump the memory contents), most of the memory is blank
(0x00). However, the rst few bytes contain some information:
0000: A1 AA 8B 99 AE 9E 78 30 A7 74 D3 A1 AA 8B 99 AE
0010: 9E 78 30 A7 74 D3 82 82 82 15 9C 9C 9E 38 40 3E
0020: 82 82 82 15 9C 9C 9E 38 40 3E 00 00 00 00 00 00
0030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
This can be better visualized as two sequences, each one repeated twice:
0000: A1 AA 8B 99 AE 9E 78 30 A7 74 D3
000B: A1 AA 8B 99 AE 9E 78 30 A7 74 D3
0016: 82 82 82 15 9C 9C 9E 38 40 3E
0020: 82 82 82 15 9C 9C 9E 38 40 3E
It is not yet clear why these sentences are repeated; but since at least the second one is known to be
calibration data, maybe one version contains the calibration data that is actually being used, while the
other version is meant for backup purposes (for example a "Return to factory settings" option) in
case there will be a way of recalibrating the Wii Remote with future Wii rmware updates.
The three bytes starting at 0x0016 and 0x0020 store the calibrated zero offsets for the accelerometer.
Apparently, the three bytes at 0x001A and 0x24 store the force of gravity on those axes. The function
of other data bytes is not known, and most of them differ between Wii Remotes. Some or all of these
bytes might not be used by the Wii. However, there has been a case of a Wii Remote where
Extension functionality was lost following a battery change, and restoring these bytes (which had
been previously overwritten) xed the problem. The Extension controllers did not work with a PC
either (which did not explicitly use these bytes), suggesting some of these might be used by the Wii
Remote itself. This is unconrmed, but it is advised that these never be overwritten, and
recommended that they be backed up, just in case.
At 0x16D0, there is some more unknown data:
16D0: 00 00 00 FF 11 EE 00 00 33 CC 44 BB 00 00 66 99
16E0: 77 88 00 00 2B 01 E8 13 00 00 00 00 00 00 00 00
16F0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
In contrast to the data at 0x0000, this data seems to differ in only a few bytes between different Wii
Remotes.
Known memory ranges are listed below. Note that the "user data" area is 0x0FA0 = 4000 bytes long,
which seems to conrm the 4kB gure that has been mentioned (meaning 4000 bytes, that is, using
the SI prex meaning instead of the binary meaning).
12 of 26 16/6/09 13:42
Start End Length Initial Value Use
0x0000 0x0029 0x002A See above Calibration values / pre-set data
0x002A 0x0FC9 0x0FA0 Zeroed User data / Unknown uses
0x0FCA 0x12B9 0x02f0 Zeroed Mii Data block 1
0x12BA 0x15A9 0x02f0 Zeroed Mii Data block 2
0x15AA 0x16CF 0x0126 Zeroed Unknown / Unused
0x16D0 0x16FF 0x0030 See above Unknown data
The top byte of the address is unused, which means memory is mirrored every 0x10000 bytes.
Reading from unused addresses where the low 16 bits are >= 0x1700 will result in error returns.
Control Registers
The Wii Remote has several memory mapped register spaces corresponding to different peripherals
in it. These include the Speaker, Extension Controllers, and the IR Camera.
Reminder
Remember to set bit 2 (0x04) on the rst byte of the Output Report,
otherwise you'll overwrite EEPROM memory!
The peripheral to access is selected by the rst byte of the address, and the lower 16 bits specify the
register to access within that peripheral. The lowest bit of the high byte is ignored, which means
every peripheral is mirrored at its address + 0x10000. Known peripherals are listed below:
Start End Use
0xA20000 0xA20009 Speaker settings
0xA40000 0xA400FF Extension Controller settings and data
0xA60000 0xA600FF Wii Motion Plus settings and data
0xB00000 0xB00033 IR Camera settings
Most of these are also mirrored across the high bits of the individual peripheral. For example, the
second byte of the address is ignored in the Extension controller address, which means any address
of the form 0xA4xx00 will work (as will 0xA5xx00).
Input Features
The Wii Remote has two input features that are controlled directly by the Broadcom chip: a
Three-Axis Accelerometer and 11 Buttons. Additionally, it has an IR Camera with an object tracking
processor, and an expansion port that allows for external input features such as those contained in the
Nunchuk and the Classic Controller (see Extension Controllers).
Buttons
13 of 26 16/6/09 13:42
The Wii Remote has 11 buttons on its front face, and one trigger-style button on the back. Of these,
the Power button is special and is treated differently by the Wii Remote. All the other buttons are
independantly accessible through a two-byte bitmask which is transmitted rst in most Input
Reports. A button will report a 1-bit if pressed, or a 0-bit otherwise. By default, these are sent only
when the state of any button changes, in Data Reporting Mode 0x30. However, the Wii Remote may
be congured to report the state of the buttons continuously; see Data Reporting.
Core Buttons
The Wii Remote has 11 buttons that are used as regular input devices: A, B (trigger), a 4-directional
D-Pad, +, -, Home, 1, and 2. These are reported as bits in a two-byte bitmask. These are the
assignments, in big-endian order:
Bit Mask First Byte Second Byte
0 0x01 D-Pad Left Two
1 0x02 D-Pad Right One
2 0x04 D-Pad Down B
3 0x08 D-Pad Up A
4 0x10 Plus Minus
5 0x20 Other uses Other uses
6 0x40 Other uses Other uses
7 0x80 Unknown Home
Power Button
When the Wii Remote is turned off, pressing the Power button will attempt to wake up the Wii that is
synchronized to it. The mechanism for this is unknown, and it is handled entirely within the Wii's
bluetooth module. When the Wii Remote is turned on and connected to a host, pressing and holding
the Power button for a few seconds will turn the Wii Remote off and request disconnection from the
host. The disconnection reason included with the Baseband (ACL) disconnection request indicates
that the power button was pressed: REMOTE DEVICE TERMINATED CONNECTION DUE TO
POWER OFF (0x15). Another possible value is REMOTE DEVICE TERMINATED
CONNECTION DUE TO LOW RESOURCES (0x14), which indicates that the Wii Remote
performed a controlled shut down due to a low battery condition.
Sync Button
The sync button is hidden under the battery cover. When the Sync button is pressed, the Wii remote
will disconnect from whatever it is currently connected to, make itself discoverable, and accept
pairing or connection requests for exactly 20 seconds (regardless of how long the button is held
down for).
The "syncing" of a Wii Remote involves standard Bluetooth pairing. When the Sync button is
pressed on the remote, it will accept pairing requests. The required PIN is the hosts's Bluetooth
address, backwards (last byte rst), in binary (6 bytes). Most current Bluetooth implementations
don't deal with this correctly, as they usually consider the PIN to be a regular null-terminated ASCII
14 of 26 16/6/09 13:42
string (no 00 bytes, etc) and most Bluetooth addresses will contain null bytes. Any further steps that
need to be taken after the Wii Remote is paired have not been reverse engineered yet.
Once the Wii Remote is synced, when a button is pressed, it will actively seek out its paired host and
try to connect to it, instead of the other way around. Establishing a connection can be done on PSM
0x11 for writing and PSM 0x13 for reading using the Bluetooth L2CAP protocol.
Button Hardware
The physical hardware of the buttons varies: there are membrane switches and microswitch click
buttons. There has been some success soldering wires to the membrane switch contacts and actuating
the switch through an external switch. The following table describes the physical hardware for each
input.
Function Switch type Circuit board surface
A membrane (http://en.wikipedia.org/wiki/Membrane_switch) top, SW9
B membrane bottom, SW8
- microswitch (http://en.wikipedia.org/wiki/Microswitch) top, SW10
Home microswitch top, SW11
+ microswitch top, SW5
1 membrane top, SW7
2 membrane top, SW6
Up membrane top, SW4
Down membrane top, SW3
Left membrane top, SW1
Right membrane top, SW2
Sync bottom, SW12
Power top, SW13

Accelerometer
15 of 26 16/6/09 13:42
ADXL330 in a Wii remote
Coordinate system used by Wii
Remote
The Wii Remote includes a three-axis linear accelerometer
located on the top suface of the circuit board, slightly left of the
large A button. The integrated circuit is the ADXL330
(http://www.analog.com/en/prod
/0%2C2877%2CADXL330%2C00.html) (data sheet
(http://www.analog.com/UploadedFiles/Data_Sheets
/ADXL330.pdf) ), manufactured by Analog Devices. This
device is physically rated to measure accelerations over a range
of at least +/- 3g with 10% sensitivity.
Since the accelerometer actually measures the force exerted by a
set of small proof masses inside of it with respect to its
enclosure, the accelerometer measures linear acceleration in a
free fall frame of reference. If the Wii Remote is in free fall, it
will report zero acceleration. At rest, it will report an upward
acceleration (+Z, when horizontal) equal to the acceleration due
to gravity, g (approximately 9.8 m/s!) but in the opposite
direction. This fact can be used to derive tilt from the
acceleration outputs when the Wii Remote is reasonably still.
Normal Accelerometer Reporting
In all Data Reporting Modes which include Accelerometer data
except for mode 0x3e/0x3f, the accelerometer data is reported as
three consecutive bytes:
(a1) RR BB BB XX YY ZZ [...]
XX, YY, and ZZ are unsigned bytes representing the acceleration in each of the three axis, where
zero acceleration is approximately 0x80. The coordinate system is shown in the diagram above (note
that this is different from the coordinate system used by GlovePIE). Additionally, the BB BB Buttons
bytes also include the LSBs of the acceleration values in the unused bits, according to the following
table:
Bit
Byte 7 6 5 4 3 2 1 0
0 X<1:0>
1 Z<1> Y<1>
Note that X has 10 bits of precision, while Y and Z only have 9. For consistency, they are assumed
all to have a 10-bit range and the LSB is always set to zero for Y and Z.
Interleaved Accelerometer Reporting
In Data Reporting Mode 0x3e/0x3f, the accelerometer data is spread over two reports:
16 of 26 16/6/09 13:42
Wii remote camera
(a1) 3e BB BB XX [...]
(a1) 3f BB BB YY [...]
In this mode, the LSBs are not available. Instead, X and Y acceleration is reported as a single byte,
and the Z value is encoded in the unused bits of the BB BB Buttons data as follows:
Bit
Report ID Byte 7 6 5 4 3 2 1 0
0x3e 0 Z<5:4>
0x3e 1 Z<7:6>
0x3f 0 Z<1:0>
0x3f 1 Z<3:2>
IR Camera
The Wii Remote includes a 128x96 monochrome camera with
built-in image processing. The camera looks through an infrared
pass lter in the remote's plastic casing. The camera's built-in
image processing is capable of tracking up to 4 moving objects,
and these data are the only data available to the host. Raw pixel
data is not available to the host, so the camera cannot be used to
take a conventional picture. The built-in processor uses 8x
subpixel analysis to provide 1024x768 resolution for the tracked
points. The Sensor Bar that comes with the Wii includes two IR
LED clusters at each end, which are tracked by the Wii Remote
to provide pointing information. The distance between the
centers of the LED clusters is 20 cm (as measured on one unit).
The IR Camera is enabled by setting bit 2 on output reports 0x13 and 0x1a:
(52) 13 04
(52) 1a 04
The rst enables a 24MHz pixel clock on pin 7 of the camera. The second pulls pin 4 low - probably
an active-low enable.
Mechanical Characteristics
The camera component is mounted on the bottom surface of the circuit board.
Optical Characteristics
The IR camera has an effective eld of view is about 33 degrees horizontally and 23 degrees
vertically (as measured on one unit). With the IR-pass lter intact, 940nm sources are detected with
approximately twice the intensity of equivalent 850nm sources, but are not resolved as well at close
distances. If the lter is removed, it can track any bright object.
17 of 26 16/6/09 13:42
Initialization
Reminder
Remember to set bit 2 (0x04) on the rst byte of the Output Reports to
write to registers!
The following procedure should be followed to turn on the IR Camera:
Enable IR Camera (Send 0x04 to Output Report 0x13) 1.
Enable IR Camera 2 (Send 0x04 to Output Report 0x1a) 2.
Write 0x08 to register 0xb00030 3.
Write Sensitivity Block 1 to registers at 0xb00000 4.
Write Sensitivity Block 2 to registers at 0xb0001a 5.
Write Mode Number to register 0xb00033 6.
Write 0x08 to register 0xb00030 (again) 7.
After these steps, the Wii Remote will be in one of 3 states: IR camera on but not taking data, IR
camera on and taking data and half sensitivity, IR camera on and taking data at full sensitivity.
Which state you end up in appears to be pretty much random. Repeat the steps until you're in the
desired state. To avoid the random state put a delay of at least 50ms between every single byte
transmission.
The Wii preforms these steps slightly different, differences in bold:
Enable IR Pixel Clock (send 0x06 to Output Report 0x13) 1.
Enable IR Logic (send 0x06 to Output Report 0x1A) 2.
Write 0x01 to register 0xb00030 3.
Write Sensitivity Block 1 to registers at 0xb00000 4.
Write Sensitivity Block 2 to registers at 0xb0001a 5.
Write Mode Number to register 0xb00033 6.
Write 0x08 to register 0xb00030 (again) 7.
Sensitivity Settings
Sensitivity is controlled by two conguration blocks, 9 bytes and 2 bytes long. The following
settings are known to work:
Block 1 Block 2 Notes
00 00 00 00 00 00 90 00 C0 40 00 Suggested by Marcan
02 00 00 71 01 00 aa 00 64 63 03 Suggested by Cliff
00 00 00 00 00 00 90 00 41 40 00 Suggested by inio (max sensitivity)
02 00 00 71 01 00 64 00 fe fd 05 Wii level 1
02 00 00 71 01 00 96 00 b4 b3 04 Wii level 2
02 00 00 71 01 00 aa 00 64 63 03 Wii level 3 (also Cliff's suggestion)
02 00 00 71 01 00 c8 00 36 35 03 Wii level 4
07 00 00 71 01 00 72 00 20 1f 03 Wii level 5
18 of 26 16/6/09 13:42
The last byte of both blocks determines the intensity sensitivity, with increasing values reducing the
sensitivity. The Wii Remote will return data for the dimmest objects possible when the last byte of
block 1 is 0x41, and second byte of block 2 is 0x00. Setting the sensitivity as high as possible,
without unwanted light being tracked, is recommended to achieve the highest subpixel resolution. As
the sensitivity is reduced, the subpixel resolution also reduces, approaching the true sensor resolution
of 128x96.
Data Formats
The IR Camera can return different sets of data describing the objects it is tracking. When the IR
camera identies an object, it assigns it to the rst available object slot. If an object moves out of
view, its slot is marked as empty (returns 0xFF data), but other objects retain their slots. For
example, if the camera is tracking two objects and the rst moves out of view, the data returned will
be [empty, second object, empty, empty]. With more than four objects visible, the camera is prone to
rapidly switching between some of them. This could allow perception of more than four objects, at a
reduced response speed and reliability.
Mode Mode Number
Basic 1
Extended 3
Full 5
The data format MUST match the number of bytes available in the Reporting Mode selected. Even
choosing a mode with space for more bytes than necessary will not work, it has to be an exact match.
Basic Mode
In Basic Mode, the IR Camera returns 10 bytes of data corresponding to the X and Y locations of
each of the four dots. Each location is encoded in 10 bits and has a range of 0-1023 for the X
dimension, and 0-767 for the Y dimension. Each pair of dots is packed into 5 bytes, and two of these
are transmitted for a total of 4 dots and 10 bytes.
This is the data format for a pair of objects:
Bit
Byte 7 6 5 4 3 2 1 0
0 X1<7:0>
1 Y1<7:0>
2 Y1<9:8> X1<9:8> Y2<9:8> X2<9:8>
3 X2<7:0>
4 Y2<7:0>
Extended Mode
In Extended Mode, the IR Camera returns the same data as it does in Basic Mode, plus a rough size
19 of 26 16/6/09 13:42
value for each object. The data is returned as 12 bytes, three bytes per object. Size has a range of
0-15.
This is the data format for each object:
Bit
Byte 7 6 5 4 3 2 1 0
0 X<7:0>
1 Y<7:0>
2 Y<9:8> X<9:8> S<3:0>
Full Mode
In Full Mode, the IR Camera returns even more data, 9 bytes per object for a total of 36 bytes for all
four. The data is split up between two input reports of 18 bytes each (see Data Reporting Mode
0x3e/0x3f). The rst three bytes of each object are the same as the extended mode, and are followed
by the bounding box of the pixels included in the blob along with a deeper intensity value. The data
format of each object is:
Bit
Byte 7 6 5 4 3 2 1 0
0 X<7:0>
1 Y<7:0>
2 Y<9:8> X<9:8> S<3:0>
3 0 X min<6:0>
4 0 Y min<6:0>
5 0 X max<6:0>
6 0 Y max<6:0>
7 0
8 Intensity<7:0>
Feedback Features
The Wii Remote sports three feedback features: Player LEDs, Rumble, and the Speaker.
Player LEDs
20 of 26 16/6/09 13:42
Wii remote player LEDs
Wii remote rumble motor
There are four blue LEDs on the front face of the Wii Remote.
During discovery and before initialization, these LEDs blink at a
xed rate. The number of blinking LEDs is proportional to the
battery voltage, indicating battery charge (all four are lit for
newly charged batteries, and only the rst is lit if the batteries
are low and should be replaced).
During gameplay with the Wii, one LED is lit to indicate the player number assigned to the Wii
Remote. However, the LEDs are independently controllable by the host, and can be set to display
any pattern. They can also be modulated at a moderately high speed, enabling some brightness
control at the cost of a lot of Bluetooth bandwidth. Sigma-delta modulation works reasonably well
for this.
The LEDs can be controlled by sending a report with ID 0x11:
(52) 11 LL
The high nybble of LL controls the four LEDs. Bit 4 of LL controls the rst LED, and bit 7 controls
the last:
Bit Mask LEDs
4 0x10
!
5 0x20
!
6 0x40
!
7 0x80
!
Turning off all LEDs for an extended period of time is discouraged, as it might lead the user to
believe the Wii Remote is turned off and disconnected, when in fact it is active.
The LEDs are surface mount parts, driven at 2.66 VDC.
Rumble
The Wii Remote includes a rumble feature, which is
implemented as a small motor attached to an off-center weight.
It will cause the controller to vibrate when activated.
The rumble motor can be turned on or off through any of the
Output Reports. Setting the LSB (bit 0) of the rst byte of any
output report will activate the rumble motor, and unsetting it will
deactivate it. For example, the following report will turn the
rumble motor on:
(52) 11 01
However, this will also have the side-effect of turning off all LEDs. Since there is no output report
21 of 26 16/6/09 13:42
that only affects the rumble motor, and all of them do affect it, an implementation might need to
store both the rumble and LED values locally (for example), and use the same Output Report for
both. Another possibility would be using the status request report (0x15). The rumble bit needs to be
set properly with every single report sent, to avoid inadvertently turning the rumble motor off.
Different photos of the rumble motor hardware show different markings. One example is SEM
8728DA. The Wii Remote drives it at 3.3 VDC and it draws 35 mA. It would be reasonable to think
that the rumble motor could be removed and the motor replaced with another device with equal
voltage and equal or less current draw.
Speaker
The Wii Remote has a small low-quality 21mm piezo-electric speaker, used for short sound effects
during gameplay. The sound is streamed directly from the host, and the speaker has some adjustable
parameters.
The speaker is controlled by using three output reports, together with a section of the register address
space of the Wii Remote.
Report 0x14 is used to enable or disable the speaker. Setting bit 2 will enable the speaker, and
clearing it will disable it. For example, to enable the speaker, send:
(52) 14 04
Report 0x19 is used to mute or unmute the speaker, and works identically to report 0x14. 0x04 will
mute the speaker, and 0x00 will unmute it.
Report 0x18 is used to send speaker data. 1-20 bytes may be sent at once:
(52) 18 LL DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD
LL species the data length, shifted left by three bits. The DD bytes are the speaker data. To fullll
the report length requirements, the data must be padded if it is less than 20 bytes long. Sound data
must be sent at the proper rate.
Initialization Sequence
Reminder
write to registers!
The following sequence will initialize the speaker:
Enable speaker (Send 0x04 to Output Report 0x14) 1.
Mute speaker (Send 0x04 to Output Report 0x19) 2.
Write 0x01 to register 0xa20009 3.
Write 7-byte conguration to registers 0xa20001-0xa20008 5.
22 of 26 16/6/09 13:42
Unmute speaker (Send 0x00 to Output Report 0x19) 7.
Speaker Conguration
7 bytes control the speaker settings, including volume. The full purpose of these bytes is not known,
but the following values seem to produce some sound:
00 FF RR RR VV 00 00
RR RR specify the sample rate (little-endian format), using the following formulae:
pcm_sample_rate = 12000000 / rate_value adpcm_sample_rate = 6000000 / rate_value
The standard value is 0x7d0, for 3000Hz 4-bit PCM
FF congures the data format. Setting it to 0x40 congures the speaker to use signed 8-bit PCM,
while setting it to 0x00 congures it to use 4-bit Yamaha ADPCM. VV species the volume, which
has a range of 0x00-0xFF for 8-bit mode, and 0x00-0x40 for 4-bit mode.
This conguration can be used to play 4-bit ADPCM sound at 3000Hz:
00 00 D0 07 40 00 00
This conguration can be used to play 8-bit PCM sound at 1500Hz sample rate:
00 40 40 1f 40 00 00
Sound Data Format
The Wii Remote can use multiple sound formats at multiple sampling rates. PC drivers currently
seem unable to keep up with the higher rates.
The 4-bit ADPCM is Yamaha ADPCM (for example, as implemented in ffmpeg).
8-bit signed PCM mode works, but when in 8-bit mode the sampling frequency must be made so low
that the audio quality is pretty bad.
Extension Controllers
The Wii Remote includes a 6-pin proprietary expansion connector, which carries synchronous serial
data to and from external add-ons, using a two-wire interface. These include the Nunchuk and the
Classic Controller, the Guitar Hero guitars, the Guitar Hero World Tour drum kit, and Wii Motion
Plus. These addons map onto a portion of the Wii Remote's register space, and can also stream data
out through Output Reports (see Data Reporting for a list of modes which include Extension
Controller data).
23 of 26 16/6/09 13:42
Registers / Initialization
Reminder
write to registers!
Wii Motion Plus is mapped at register 0xa60000.
Other extension Controllers are mapped at register address 0xa40000. The data is 0x100 bytes long,
and it is mirrored across the entire 16-bit address space up to 0xa4FFFF. These registers are readable
and writable. Communications are optionally encrypted, and explicit initialization is required to
disable encryption (see below).
In encrypted mode, bytes can be decrypted using the following transformation:
decrypted_byte = (encrypted_byte XOR table1[address%8]) + table2[address%8]
Where table1 and table2 are 8-byte tables calculated based on the 16-byte key sent by the host, and
address is the address of the byte being read (only the bottom 3 bits matter). If the host key is 16 zero
bytes, table1[x] and table2[x] are all 0x97. The value 0x17 was used previously, which is equivalent.
Proof of why any two pairs of table entries a,b and x,y are equivalent if (a^b^x^y)&0x80 == 0 is left
as an exercise to the reader.
The calculations should be performed mod 256, that is, truncated to 8 bits (in languages such as C
and Python, use &0xFF or work directly with 8-bit datatypes).
If the Wii Remote is initialized using the new method listed below, then the encryption of the
extension bytes is disabled and they need not be decrypted using the transform listed above.
The Wii Remote will stream data bytes from the Extension Controller starting at offset 0x08, and
continuing for however many bytes the current Data Reporting Mode requires. Data streamed
through the Input Reports must also be decrypted using the above transformation if encryption is
enabled.
Identication
Once initialized, the last six bytes of the register block identify the connected Extension Controller.
A six-byte read of register 0xa400fa will return these bytes. The Extension Controller must have
been initialized prior to this. There are two ways of initializing the extension.
The Old Way
The old way to initialize the extension was by writing the single encryption byte 0x00 to
0x(4)A40040, but that only works on Nintendo's own brand extensions and the GH3 Guitar, and will
fail on 3rd party wireless nunchuks and GHWT extensions. If it fails, the 6 bytes will be FFFF FFFF
FFFF. With this method you must decrypt the extension bytes to read them.
The New Way
24 of 26 16/6/09 13:42
The new way to initialize the extension is by writing 0x55 to 0x(4)A400F0, then writing 0x00 to
0x(4)A400FB. It works on all extensions, and makes the extension type bytes unencrypted. This
means that you no longer have to decrypt the extension bytes using the transform listed above.
Encrypted Decrypted Meaning
0000 0000
0000*
0x2E2E
BladeFX adapter that has been initialised old way, unplugged then
replugged
FFFF FFFF
FFFF*
0xFFFF
GHWT Guitar or Drums or synced BladeFX or other wireless
nunchuk initialised the old way
0xFEFE
0000 A420
0000
Nunchuk
0xFDFD
0000 A420
0101
Classic Controller
0xFDFB
0000 A420
0103
GH3 or GHWT Guitar
0xFDFB
0100 A420
0103
Guitar Hero World Tour Drums
???? A420
0402*
0x2A2C Wii Balance Board
0000 A620
0005*
Wii Motion Plus
The values marked with a * are actually already unencrypted, and should not be decrypted. They are
given here along with unencrypted version for those wanting to use simplied detection code. You
can easily tell whether a value should be decrypted or not by looking at the other values in the
0xa400fa to 0xa400fd region.
The "Partially inserted" condition allegedly occurs when the connector is loose or has a bad
connection. Usually, this will correct itself upon full insertion. However, it has been known to "stick"
in the partially inserted state rarely. This is a hardware glitch, the x is to simply disconnect and
reconnect the Extension Controller. Third party controllers such as wireless nunchuks also return this
value if initialized with the old method, in which case the whole last 16 bytes will be FF.
Contrary to previous documentation, the 0000 0000 0000 value does NOT occur when nothing is
inserted. Instead you get error 7 when you try to read the expansion type. A successful 0000 0000
0000 only occurs when a BladeFX wireless nunchuk is synced, initialized with the old method, then
the adapter is unplugged, then the adapter is replugged.
A Wii Balance Board extension is only found in a Wii Balance Board. Although it exposes
functionality as an Extension Controller, they are not separable and this controller is documented
separately.
Encryption setup
25 of 26 16/6/09 13:42
After the identication is read, the encryption can be set up if required. This is done by enabling
encryption by writing 0xAA to extension register 0xF0, and then writing the 16-byte key to register
0x40. The key is written in 3 blocks of 6, 6, and 4 bytes. (Writing a single encryption byte to register
0x40 will work on genuine Nintendo controllers, but not on others). After this the extension can be
operated in full encryption mode.
Extension Controller Documentation
Individual Extension Controllers are described on the Extension Controllers page.
Acknowledgements
Some of the information here is based on the documentation at Wiili
Retrieved from "http://www.wiibrew.org/wiki/Wiimote"
Categories: Articles needing cleanup | Hardware
Hidden category: Articles that need to be wikied
This page was last modied on 11 June 2009, at 11:51.
Content is available under GNU Free Documentation License 1.2.
26 of 26 16/6/09 13:42
166

PAA-/0$N [ PH\#BBY H>&>%:--&

Small, Low Power, 3-Axis 3 g
i

MEMS
Accelerometer

ADXL330

Rev. A
Information furnished by Analog Devices is believed to be accurate and reliable. However, no
responsibilityisassumedbyAnalogDevicesforitsuse, norforanyinfringementsofpatentsorother
rightsofthirdpartiesthatmayresultfromitsuse. Specificationssubjecttochangewithoutnotice.No
licenseis grantedbyimplicationor otherwiseunder anypatent or patent rights of AnalogDevices.
Trademarksandregisteredtrademarksarethepropertyoftheirrespectiveowners.

One Technology Way, P.O. Box 9106, Norwood, MA 02062-9106, U.S.A.
Tel: 781.329.4700 www.analog.com
Fax: 781.461.3113 2007 Analog Devices, Inc. All rights reserved.
FEATURES
3-axis sensing
Small, low-profile package
4 mm 4 mm 1.45 mm LFCSP
Low power
180 A at VS = 1.8 V (typical)
Single-supply operation
1.8 V to 3.6 V
10,000 g shock survival
Excellent temperature stability
BW adjustment with a single capacitor per axis
RoHS/WEEE lead-free compliant

APPLICATIONS
Cost-sensitive, low power, motion- and tilt-sensing
applications
Mobile devices
Gaming systems
Disk drive protection
Image stabilization
Sports and health devices
GENERAL DESCRIPTION
The ADXL330 is a small, thin, low power, complete 3-axis
accelerometer with signal conditioned voltage outputs, all
on a single monolithic IC. The product measures acceleration
with a minimum full-scale range of 3 g. It can measure the
static acceleration of gravity in tilt-sensing applications, as well
as dynamic acceleration resulting from motion, shock, or
vibration.
The user selects the bandwidth of the accelerometer using the
CX, CY, and CZ capacitors at the XOUT, YOUT, and ZOUT pins.
Bandwidths can be selected to suit the application, with a
range of 0.5 Hz to 1600 Hz for X and Y axes, and a range of
0.5 Hz to 550 Hz for the Z axis.
The ADXL330 is available in a small, low profile, 4 mm 4 mm
1.45 mm, 16-lead, plastic lead frame chip scale package
(LFCSP_LQ).

FUNCTIONAL BLOCK DIAGRAM
0
5
6
7
7
-
0
0
1
3-AXIS
SENSOR
AC AMP DEMOD
OUTPUT AMP
OUTPUT AMP
OUTPUT AMP
V
S
COM ST
X
OUT
Y
OUT
Z
OUT
+3V
C
X
C
Y
C
Z
ADXL330
R
FILT
R
FILT
R
FILT
C
DC

Figure 1.

ADXL330

Rev. A | Page 2 of 16
TABLE OF CONTENTS
Features .............................................................................................. 1
Applications....................................................................................... 1
General Description......................................................................... 1
Functional Block Diagram.............................................................. 1
Revision History ............................................................................... 2
Specifications..................................................................................... 3
Absolute Maximum Ratings............................................................ 4
ESD Caution.................................................................................. 4
Pin Configuration and Function Descriptions............................. 5
Typical Performance Characteristics ............................................. 6
Theory of Operation ...................................................................... 11
Mechanical Sensor...................................................................... 11
Performance................................................................................ 11
Applications..................................................................................... 12
Power Supply Decoupling ......................................................... 12
Setting the Bandwidth Using CX, CY, and CZ.......................... 12
Self Test ........................................................................................ 12
Design Trade-Offs for Selecting Filter Characteristics: The
Noise/BW Trade-Off.................................................................. 12
Use with Operating Voltages Other than 3 V............................. 12
Axes of Acceleration Sensitivity ............................................... 13
Outline Dimensions ....................................................................... 14
Ordering Guide .......................................................................... 14

REVISION HISTORY
9/06Rev. 0 to Rev. A
Changes to Ordering Guide .......................................................... 14
3/06Revision 0: Initial Version

ADXL330

SPECIFICATIONS
TA = 25C, VS = 3 V, CX = CY = CZ = 0.1 F, acceleration = 0 g, unless otherwise noted. All minimum and maximum specifications are
guaranteed. Typical specifications are not guaranteed.
Table 1.
Parameter Conditions Min Typ Max Unit
SENSOR INPUT Each axis
Measurement Range 3 3.6 g
Nonlinearity % of full scale 0.3 %
Package Alignment Error 1 Degrees
Interaxis Alignment Error 0.1 Degrees
Cross Axis Sensitivity
1
1 %
SENSITIVITY (RATIOMETRIC)
2
Each axis
Sensitivity at XOUT, YOUT, ZOUT VS = 3 V 270 300 330 mV/g
Sensitivity Change Due to Temperature
3
VS = 3 V 0.015 %/C
ZERO g BIAS LEVEL (RATIOMETRIC) Each axis
0 g Voltage at XOUT, YOUT, ZOUT VS = 3 V 1.2 1.5 1.8 V
0 g Offset vs. Temperature 1 mg/C
NOISE PERFORMANCE
Noise Density XOUT, YOUT 280 g/Hz rms
Noise Density ZOUT 350 g/Hz rms
FREQUENCY RESPONSE
4

Bandwidth XOUT, YOUT!
5
No external filter 1600 Hz
Bandwidth ZOUT
5
No external filter 550 Hz
RFILT Tolerance 32 15% k
Sensor Resonant Frequency 5.5 kHz
SELF TEST!
6

Logic Input Low +0.6 V
Logic Input High +2.4 V
ST Actuation Current +60 A
Output Change at XOUT Self test 0 to 1 150 mV
Output Change at YOUT Self test 0 to 1 +150 mV
Output Change at ZOUT Self test 0 to 1 60 mV
OUTPUT AMPLIFIER
Output Swing Low No load 0.1 V
Output Swing High No load 2.8 V
POWER SUPPLY
Operating Voltage Range 1.8 3.6 V
Supply Current VS = 3 V 320 A
Turn-On Time
7
No external filter 1 ms
TEMPERATURE
Operating Temperature Range 25 +70 C

1
Defined as coupling between any two axes.
2
Sensitivity is essentially ratiometric to VS.
3
Defined as the output change from ambient-to-maximum temperature or ambient-to-minimum temperature.
4
Actual frequency response controlled by user-supplied external filter capacitors (CX, CY, CZ).
5
Bandwidth with external capacitors = 1/(2 32 k C). For CX, CY = 0.003 F, bandwidth = 1.6 kHz. For CZ = 0.01 F, bandwidth = 500 Hz. For CX, CY, CZ = 10 F,
bandwidth = 0.5 Hz.
6
Self-test response changes cubically with VS.
7
Turn-on time is dependent on CX, CY, CZ and is approximately 160 CX or CY or CZ + 1 ms, where CX, CY, CZ are in F.

ADXL330

ABSOLUTE MAXIMUM RATINGS
Table 2.
Parameter Rating
Acceleration (Any Axis, Unpowered) 10,000 g
Acceleration (Any Axis, Powered) 10,000 g
VS 0.3 V to +7.0 V
All Other Pins (COM 0.3 V) to (VS + 0.3 V)
Output Short-Circuit Duration
(Any Pin to Common)
Indefinite
Temperature Range (Powered) 55C to +125C
Temperature Range (Storage) 65C to +150C

Stresses above those listed under Absolute Maximum Ratings
may cause permanent damage to the device. This is a stress
rating only; functional operation of the device at these or any
other conditions above those indicated in the operational
section of this specification is not implied. Exposure to absolute
maximum rating conditions for extended periods may affect
device reliability.
0
5
6
7
7
-
0
0
2
t
P
t
L
t25C TO PEAK
t
S
PREHEAT
CRITICAL ZONE
T
L
TO T
P
T
E
M
P
E
R
A
T
U
R
E
TIME
RAMP-DOWN
RAMP-UP
T
SMIN
T
SMAX
T
P
T
L

Figure 2. Recommended Soldering Profile
Table 3. Recommended Soldering Profile
Profile Feature Sn63/Pb37 Pb-Free
Average Ramp Rate (TL to TP) 3C/s max 3C/s max
Preheat
Minimum Temperature (TSMIN) 100C 150C
Maximum Temperature (TSMAX) 150C 200C
Time (TSMIN to TSMAX), tS 60 s to 120 s 60 s to 180 s
TSMAX to TL
Ramp-Up Rate 3C/s max 3C/s max
Time Maintained Above Liquidous (TL)
Liquidous Temperature (TL) 183C 217C
Time (tL) 60 s to 150 s 60 s to 150 s
Peak Temperature (TP) 240C + 0C/5C 260C + 0C/5C
Time within 5C of Actual Peak Temperature (tP) 10 s to 30 s 20 s to 40 s
Ramp-Down Rate 6C/s max 6C/s max
Time 25C to Peak Temperature 6 minutes max 8 minutes max

ESD CAUTION
ESD (electrostatic discharge) sensitive device. Electrostatic charges as high as 4000 V readily accumulate on the
human body and test equipment and can discharge without detection. Although this product features
proprietary ESD protection circuitry, permanent damage may occur on devices subjected to high energy
electrostatic discharges. Therefore, proper ESD precautions are recommended to avoid performance
degradation or loss of functionality.

ADXL330

PIN CONFIGURATION AND FUNCTION DESCRIPTIONS
NC = NO CONNECT
NC 1
ST 2
COM 3
NC 4
X
OUT
12
NC 11
Y
OUT
10
NC 9
C
O
M
C
O
M
C
O
M
Z
O
U
T
5 6 7 8
16
N
C
15
V
S
14
V
S
13
N
C
ADXL330
TOP VIEW
(Not to Scale)
+Z
+X
+Y
0
5
6
7
7
-
0
2
9

Figure 3. Pin Configuration
CENTER PAD IS NOT
INTERNALLY CONNECTED
BUT SHOULD BE SOLDERED
FOR MECHANICAL INTEGRITY
0.50
MAX
0.65 0.325
1.95
0.65
0.325
4
4
0.35
MAX
1.95
DIMENSIONS SHOWN IN MILLIMETERS
0
5
6
7
7
-
0
3
2

Figure 4. Recommended PCB Layout

Table 4. Pin Function Descriptions
Pin No. Mnemonic Description
1 NC No Connect
2 ST Self Test
3 COM Common
4 NC No Connect
5 COM Common
6 COM Common
7 COM Common
8 ZOUT Z Channel Output
9 NC No Connect
10 YOUT Y Channel Output
11 NC No Connect
12 XOUT X Channel Output
13 NC No Connect
14 VS Supply Voltage (1.8 V to 3.6 V)
15 VS Supply Voltage (1.8 V to 3.6 V)
16 NC No Connect

ADXL330

TYPICAL PERFORMANCE CHARACTERISTICS
N > 1000 for all typical performance plots, unless otherwise noted.
35
0
5
10
15
20
25
30
1.42 1.44 1.46 1.48 1.50 1.52 1.54 1.56 1.58
%

O
F

P
O
P
U
L
A
T
I
O
N
OUTPUT (V)
0
5
6
7
7
-
0
0
3

Figure 5. X-Axis Zero g Bias at 25C, VS = 3 V
40
35
0
5
10
15
20
25
30
1.42 1.44 1.46 1.48 1.50 1.52 1.54 1.56 1.58
%

O
F

P
O
P
U
L
A
T
I
O
N
OUTPUT (V)
0
5
6
7
7
-
0
0
4

Figure 6. Y-Axis Zero g Bias at 25C, VS = 3 V
40
35
0
5
10
15
20
25
30
1.42 1.44 1.46 1.48 1.50 1.52 1.54 1.56 1.58
%

O
F

P
O
P
U
L
A
T
I
O
N
OUTPUT (V)
0
5
6
7
7
-
0
0
5

Figure 7. Z-Axis Zero g Bias at 25C, VS = 3 V
16
14
0
2
4
6
8
10
12
0.95 0.96 0.97 0.98 0.99 1.00 1.01 1.02 1.03 1.04 1.05 1.06 1.07 1.08 1.09
%

O
F

P
O
P
U
L
A
T
I
O
N
OUTPUT (V)
0
5
6
7
7
-
0
0
6

Figure 8. X-Axis Zero g Bias at 25C, VS = 2 V
16
14
0
2
4
6
8
10
12
0.95 0.96 0.97 0.98 0.99 1.00 1.01 1.02 1.03 1.04 1.05 1.06 1.07 1.08 1.09
%

O
F

P
O
P
U
L
A
T
I
O
N
OUTPUT (V)
0
5
6
7
7
-
0
0
7

Figure 9. Y-Axis Zero g Bias at 25C, VS = 2 V
25
0
5
10
15
20
0.88 0.90 0.92 0.94 0.96 0.98 1.00 1.02 1.04 1.06 1.08 1.10 1.12 1.14 1.16
%

O
F

P
O
P
U
L
A
T
I
O
N
OUTPUT (V)
0
5
6
7
7
-
0
0
8

Figure 10. Z-Axis Zero g Bias at 25C, VS = 2 V
ADXL330

35
0
5
10
15
25
30
20
2.5 2.5 2.0 1.5 1.0 0.5 0 0.5 1.0 1.5 2.0
%

O
F

P
O
P
U
L
A
T
I
O
N
TEMPERATURE COEFFICIENT (mg/C)
0
5
6
7
7
-
0
0
9

Figure 11. X-Axis Zero g Bias Temperature Coefficient, VS = 3 V
40
35
0
5
10
15
25
30
20
2.5 2.5 2.0 1.5 1.0 0.5 0 0.5 1.0 1.5 2.0
%

O
F

P
O
P
U
L
A
T
I
O
N
0
5
6
7
7
-
0
1
0

Figure 12. Y-Axis Zero g Bias Temperature Coefficient, VS = 3 V
30
0
5
10
15
25
20
2.5 2.5 2.0 1.5 1.0 0.5 0 0.5 1.0 1.5 2.0
%

O
F

P
O
P
U
L
A
T
I
O
N
0
5
6
7
7
-
0
1
1

Figure 13. Z-Axis Zero g Bias Temperature Coefficient, VS = 3 V
1.55
1.54
1.53
1.52
1.51
1.50
1.49
1.48
1.47
1.46
1.45
30 20 10 0 10 20 30 40 50 60 70 80
V
O
L
T
S
TEMPERATURE (C)
0
5
6
7
7
-
0
1
2
N = 8

Figure 14. X-Axis Zero g Bias vs. Temperature8 Parts Soldered to PCB
1.55
1.54
1.53
1.52
1.51
1.50
1.49
1.48
1.47
1.46
1.45
30 20 10 0 10 20 30 40 50 60 70 80
V
O
L
T
S
TEMPERATURE (C)
0
5
6
7
7
-
0
1
3
N = 8

Figure 15. Y-Axis Zero g Bias vs. Temperature8 Parts Soldered to PCB
1.55
1.54
1.53
1.52
1.51
1.50
1.49
1.48
1.47
1.46
1.45
30 20 10 0 10 20 30 40 50 60 70 80
V
O
L
T
S
TEMPERATURE (C)
0
5
6
7
7
-
0
1
4
N = 8

Figure 16. Z-Axis Zero g Bias vs. Temperature8 Parts Soldered to PCB
ADXL330

60
0
10
20
30
40
50
0.26 0.34 0.33 0.32 0.31 0.30 0.29 0.28 0.27
%

O
F

P
O
P
U
L
A
T
I
O
N
SENSITIVITY (V/g)
0
5
6
7
7
-
0
1
5

Figure 17. X-Axis Sensitivity at 25C, VS = 3 V
70
60
0
10
20
30
40
50
0.26 0.34 0.33 0.32 0.31 0.30 0.29 0.28 0.27
%

O
F

P
O
P
U
L
A
T
I
O
N
SENSITIVITY (V/g)
0
5
6
7
7
-
0
1
6

Figure 18. Y-Axis Sensitivity at 25C, VS = 3 V
70
60
0
10
20
30
40
50
0.25 0.33 0.32 0.31 0.30 0.29 0.28 0.27 0.26
%

O
F

P
O
P
U
L
A
T
I
O
N
SENSITIVITY (V/g)
0
5
6
7
7
-
0
1
7

Figure 19. Z-Axis Sensitivity at 25C, VS = 3 V
35
30
0
5
10
15
20
25
0.170 0.174 0.178 0.182 0.186 0.190 0.194 0.198 0.202 0.206 0.210
%

O
F

P
O
P
U
L
A
T
I
O
N
SENSITIVITY (V/g)
0
5
6
7
7
-
0
1
8

Figure 20. X-Axis Sensitivity at 25C, VS = 2 V
40
35
30
0
5
10
15
20
25
0.170 0.174 0.178 0.182 0.186 0.190 0.194 0.198 0.202 0.206 0.210
%

O
F

P
O
P
U
L
A
T
I
O
N
SENSITIVITY (V/g)
0
5
6
7
7
-
0
1
9

Figure 21. Y-Axis Sensitivity at 25C, VS = 2 V
40
35
30
0
5
10
15
20
25
0.172 0.176 0.180 0.184 0.188 0.192 0.196 0.200 0.204 0.208 0.212
%

O
F

P
O
P
U
L
A
T
I
O
N
SENSITIVITY (V/g)
0
5
6
7
7
-
0
2
0

Figure 22. Z-Axis Sensitivity at 25C, VS = 2 V
ADXL330

90
80
70
60
0
10
20
30
40
50
2.0 1.6 1.2 0.8 0.4 0 0.4 0.8 1.2 1.6 2.0
%

O
F

P
O
P
U
L
A
T
I
O
N
DRIFT (%)
0
5
6
7
7
-
0
2
1

Figure 23. X-Axis Sensitivity Drift Over Temperature, VS = 3 V
70
60
0
10
20
30
40
50
2.0 1.6 1.2 0.8 0.4 0 0.4 0.8 1.2 1.6 2.0
%

O
F

P
O
P
U
L
A
T
I
O
N
DRIFT (%)
0
5
6
7
7
-
0
2
2

Figure 24. Y-Axis Sensitivity Drift Over Temperature, VS = 3 V
25
20
15
10
5
0
1.0 0.6 0.2 0.2 0.6 1.0 1.4 1.8 2.2 2.6 3.0
%

O
F

P
O
P
U
L
A
T
I
O
N
DRIFT (%)
0
5
6
7
7
-
0
2
3

Figure 25. Z-Axis Sensitivity Drift Over Temperature, VS = 3 V
0.33
0.32
0.31
0.30
0.29
0.28
0.27
30 80 70 60 50 40 30 20 10 0 10 20
S
E
N
S
I
T
I
V
I
T
Y

(
V
/
g
)
TEMPERATURE (C)
0
5
6
7
7
-
0
2
4
N = 8

Figure 26. X-Axis Sensitivity vs. Temperature
8 Parts Soldered to PCB, VS = 3 V
0.33
0.32
0.31
0.30
0.29
0.28
0.27
30 80 70 60 50 40 30 20 10 0 10 20
S
E
N
S
I
T
I
V
I
T
Y

(
V
/
g
)
TEMPERATURE (C)
0
5
6
7
7
-
0
2
5
N = 8

Figure 27. Y-Axis Sensitivity vs. Temperature
0.33
0.32
0.31
0.30
0.29
0.28
0.27
30 80 70 60 50 40 30 20 10 0 10 20
S
E
N
S
I
T
I
V
I
T
Y

(
V
/
g
)
TEMPERATURE (C)
0
5
6
7
7
-
0
2
6
N = 8

Figure 28. Z-Axis Sensitivity vs. Temperature
ADXL330

600
500
400
300
200
100
0
0 6 5 4 3 2 1
C
U
R
R
E
N
T

(
A
)
SUPPLY (V)
0
5
6
7
7
-
0
2
7

Figure 29. Typical Current Consumption vs. Supply Voltage
0
5
6
7
7
-
0
2
8
CH1 1.00V
B
W
CH2 500mV
B
W
CH3 500mV CH4 500mV
M1.00ms A CH1 300mV
4
3
2
1
T 9.400%
T

Figure 30. Typical Turn-On TimeCX, CY, CZ = 0.0047 F, VS = 3 V

ADXL330

THEORY OF OPERATION
The ADXL330 is a complete 3-axis acceleration measurement
system on a single monolithic IC. The ADXL330 has a measure-
ment range of 3 g minimum. It contains a polysilicon surface
micromachined sensor and signal conditioning circuitry to
implement an open-loop acceleration measurement architecture.
The output signals are analog voltages that are proportional to
acceleration. The accelerometer can measure the static accelera-
tion of gravity in tilt sensing applications as well as dynamic
acceleration resulting from motion, shock, or vibration.
The sensor is a polysilicon surface micromachined structure
built on top of a silicon wafer. Polysilicon springs suspend the
structure over the surface of the wafer and provide a resistance
against acceleration forces. Deflection of the structure is meas-
ured using a differential capacitor that consists of independent
fixed plates and plates attached to the moving mass. The fixed
plates are driven by 180 out-of-phase square waves. Acceleration
deflects the moving mass and unbalances the differential
capacitor resulting in a sensor output whose amplitude is
proportional to acceleration. Phase-sensitive demodulation
techniques are then used to determine the magnitude and
direction of the acceleration.
The demodulator output is amplified and brought off-chip
through a 32 k resistor. The user then sets the signal band-
width of the device by adding a capacitor. This filtering improves
measurement resolution and helps prevent aliasing.
MECHANICAL SENSOR
The ADXL330 uses a single structure for sensing the X, Y, and
Z axes. As a result, the three axes sense directions are highly
orthogonal with little cross axis sensitivity. Mechanical mis-
alignment of the sensor die to the package is the chief source
of cross axis sensitivity. Mechanical misalignment can, of
course, be calibrated out at the system level.
PERFORMANCE
Rather than using additional temperature compensation
circuitry, innovative design techniques ensure high
performance is built-in to the ADXL330. As a result, there is
neither quantization error nor nonmonotonic behavior, and
temperature hysteresis is very low (typically less than 3 mg over
the 25C to +70C temperature range).
Figure 14, Figure 15, and Figure 16 show the zero g output
performance of eight parts (X-, Y-, and Z-axis) soldered to a
PCB over a 25C to +70C temperature range.
Figure 26, Figure 27, and Figure 28 demonstrate the typical
sensitivity shift over temperature for supply voltages of 3 V. This
is typically better than 1% over the 25C to +70C
temperature range.
ADXL330

APPLICATIONS
POWER SUPPLY DECOUPLING
For most applications, a single 0.1 F capacitor, CDC, placed
close to the ADXL330 supply pins adequately decouples the
accelerometer from noise on the power supply. However, in
applications where noise is present at the 50 kHz internal clock
frequency (or any harmonic thereof), additional care in power
supply bypassing is required as this noise can cause errors in
acceleration measurement. If additional decoupling is needed,
a 100 (or smaller) resistor or ferrite bead can be inserted in
the supply line. Additionally, a larger bulk bypass capacitor
(1 F or greater) can be added in parallel to CDC. Ensure that
the connection from the ADXL330 ground to the power supply
ground is low impedance because noise transmitted through
ground has a similar effect as noise transmitted through VS.
SETTING THE BANDWIDTH USING C
X
, C
Y
, AND C
Z
The ADXL330 has provisions for band limiting the XOUT, YOUT,
and ZOUT pins. Capacitors must be added at these pins to
implement low-pass filtering for antialiasing and noise
reduction. The equation for the 3 dB bandwidth is
F3 dB = 1/(2(32 k) C(X, Y, Z))
or more simply
F3 dB = 5 F/C(X, Y, Z)
The tolerance of the internal resistor (RFILT) typically varies as
much as 15% of its nominal value (32 k), and the bandwidth
varies accordingly. A minimum capacitance of 0.0047 F for CX,
CY, and CZ is recommended in all cases.
Table 5. Filter Capacitor Selection, CX, CY, and CZ
Bandwidth (Hz) Capacitor (F)
1 4.7
10 0.47
50 0.10
100 0.05
200 0.027
500 0.01
SELF TEST
The ST pin controls the self test feature. When this pin is set to
VS, an electrostatic force is exerted on the accelerometer beam.
The resulting movement of the beam allows the user to test if
the accelerometer is functional. The typical change in output is
500 mg (corresponding to 150 mV) in the X-axis, 500 mg (or
150 mV) on the Y-axis, and 200 mg (or 60 mV) on the Z-axis.
This ST pin may be left open circuit or connected to common
(COM) in normal use.
Never expose the ST pin to voltages greater than VS + 0.3 V. If
this cannot be guaranteed due to the system design (for
instance, if there are multiple supply voltages), then a low VF
clamping diode between ST and VS is recommended.
DESIGN TRADE-OFFS FOR SELECTING FILTER
CHARACTERISTICS: THE NOISE/BW TRADE-OFF
The selected accelerometer bandwidth ultimately determines
the measurement resolution (smallest detectable acceleration).
Filtering can be used to lower the noise floor to improve the
resolution of the accelerometer. Resolution is dependent on the
analog filter bandwidth at XOUT, YOUT, and ZOUT.
The output of the ADXL330 has a typical bandwidth of greater
than 500 Hz. The user must filter the signal at this point to limit
aliasing errors. The analog bandwidth must be no more than
half the analog-to-digital sampling frequency to minimize
aliasing. The analog bandwidth can be further decreased to
reduce noise and improve resolution.
The ADXL330 noise has the characteristics of white Gaussian
noise, which contributes equally at all frequencies and is
described in terms of g/Hz (the noise is proportional to the
square root of the accelerometer bandwidth). The user should
limit bandwidth to the lowest frequency needed by the applica-
tion to maximize the resolution and dynamic range of the
accelerometer.
With the single-pole, roll-off characteristic, the typical noise of
the ADXL330 is determined by
) 1.6 ( ! ! " BW Density Noise Noise rms
Often, the peak value of the noise is desired. Peak-to-peak noise
can only be estimated by statistical methods. Table 6 is useful
for estimating the probabilities of exceeding various peak
values, given the rms value.
Table 6. Estimation of Peak-to-Peak Noise
Peak-to-Peak Value
% of Time that Noise Exceeds
Nominal Peak-to-Peak Value
2 rms 32
4 rms 4.6
6 rms 0.27
8 rms 0.006
USE WITH OPERATING VOLTAGES OTHER THAN 3 V
The ADXL330 is tested and specified at VS = 3 V; however, it
can be powered with VS as low as 1.8 V or as high as 3.6 V. Note
that some performance parameters change as the supply voltage
is varied.
ADXL330

The ADXL330 output is ratiometric, therefore, the output
sensitivity (or scale factor) varies proportionally to the
supply voltage. At VS = 3.6 V, the output sensitivity is
typically 360 mV/g. At VS = 2 V, the output sensitivity is
typically 195 mV/g.
The zero g bias output is also ratiometric, so the zero g output is
nominally equal to VS/2 at all supply voltages.
The output noise is not ratiometric but is absolute in volts;
therefore, the noise density decreases as the supply voltage
increases. This is because the scale factor (mV/g) increases
while the noise voltage remains constant. At VS = 3.6 V, the
X- and Y-axis noise density is typically 230 g/Hz, while at
VS = 2 V, the X- and Y-axis noise density is typically 350 g/Hz.
Self test response in g is roughly proportional to the square of
the supply voltage. However, when ratiometricity of sensitivity
is factored in with supply voltage, the self test response in volts
is roughly proportional to the cube of the supply voltage. For
example, at VS = 3.6 V, the self test response for the ADXL330 is
approximately 275 mV for the X-axis, +275 mV for the Y-axis,
and 100 mV for the Z-axis.
At VS = 2 V, the self test response is approximately 60 mV for
the X-axis, +60 mV for the Y-axis, and 25 mV for the Z-axis.
The supply current decreases as the supply voltage decreases.
Typical current consumption at VS = 3.6 V is 375 A, and
typical current consumption at VS = 2 V is 200 A.
AXES OF ACCELERATION SENSITIVITY
0
5
6
7
7
-
0
3
0
A
Z
A
Y
A
X
TOP

Figure 31. Axes of Acceleration Sensitivity, Corresponding Output Voltage
Increases When Accelerated Along the Sensitive Axis
0
5
6
7
7
-
0
3
1
X
OUT
= 1g
Y
OUT
= 0g
Z
OUT
= 0g
GRAVITY
X
OUT
= 0g
Y
OUT
= 1g
Z
OUT
= 0g
X
OUT
= 0g
Y
OUT
= 1g
Z
OUT
= 0g
X
OUT
= 1g
Y
OUT
= 0g
Z
OUT
= 0g
X
OUT
= 0g
Y
OUT
= 0g
Z
OUT
= 1g
X
OUT
= 0g
Y
OUT
= 0g
Z
OUT
= 1g
TOP
TOP TOP
TOP
TOP

Figure 32. Output Response vs. Orientation to Gravity
ADXL330

OUTLINE DIMENSIONS
16
5
13
8
9
12
1
4
0.65 BSC
2.43
1.75 SQ
1.08
1.95 BSC
0.20 MIN
PIN 1
INDICATOR
BOTTOM
VIEW
0.20 MIN
SEATING
PLANE
1.50
1.45
1.40
PIN 1
INDICATOR
TOP
VIEW
COPLANARITY
0.05
0.05 MAX
0.02 NOM
0.35
0.30
0.25
0.55
0.50
0.45
4.15
4.00 SQ
3.85
*
STACKED DIE WITH GLASS SEAL. 0
7
2
6
0
6
-
A

Figure 33. 16-Lead Lead Frame Chip Scale Package [LFCSP_LQ]
4 mm 4 mm Body, Thick Quad
(CP-16-5a*)
Dimensions shown in millimeters
ORDERING GUIDE
Model Measurement Range Specified Voltage Temperature Range Package Description Package Option
ADXL330KCPZ
1
3 g 3 V 25C to +70C 16-Lead LFCSP_LQ CP-16-5a
ADXL330KCPZRL
1
3 g 3 V 25C to +70C 16-Lead LFCSP_LQ CP-16-5a
EVAL-ADXL330Z
1
Evaluation Board

1
Z = Pb-free part.

ADXL330

NOTES
ADXL330

NOTES

2007 Analog Devices, Inc. All rights reserved. Trademarks and
registered trademarks are the property of their respective owners.
D05677-0-6/07(A)
167

PAA-/0$N = @'(&?>,- GD>%% 9>6D-%
A8` EF# *BF#-# J4,77 4'7&7
SphereMainForm
WiiMidiDelegate wiiMidiDelegate
WiimoteCollection wc
Constructor()
void Form1_Load(object sender, EventArgs e)
void populateWiimoteListCombo()
void populateChannelCombos()
void populateOutputDeviceListCombo()
void populatePitchCurveCombo()
void populateVeloictyCurveCombo()
void wiiMidiChannelCombo_SelectedIndexChanged(object sender, EventArgs e)
void nunchuckMidiChannelCombo_SelectedIndexChanged(object sender, EventArgs e)
void outDeviceListCombo_SelectedIndexChanged(object sender, EventArgs e)
void pitchCurveCombo_SelectedIndexChanged(object sender, EventArgs e)
void velocityCurveCombo_SelectedIndexChange(object sender, EventArgs e)
voidwiimoteListCombo_SelectedIndexChanged(object sender, EventArgs e)

168

WiiMidiDelegate
Wiimote wm
MIDIOut midiOut
WiimoteState ows
AccDetect wiimoteAccDetect
AccDetect nunchuckAccDetect
PitchDetect wiimotePitchDetect
PitchDetect nunchuckPitchDetect
Timer wiiNoteOffTimer
Timer nunchuckNoteOffTimer
int wiiChannel
int nunchuckChannel
int outputDeviceId
int wiiPitch
int nunchuckPitch
int wiiTranspose
int nunchuckTranspose
int velocityCurve
int pitchCurve
bool firstrun
Constructor()
void wm_WiimoteChanged(object sender, WiimoteChangedEventArgs args)
void wm_WiimoteExtensionChanged(object sender, WiimoteChangedEventArgs args)
void wiiNoteOffTimerOut(object source, ElapsedEventArgs e)
void nunchuckNoteOffTimerOut(object source, ElapsedEventArgs e)
void SetLEDs(bool a, bool b, bool c, bool d)
int WiiChannel(int value)
int NunchuckChannel(int value)
int WiiTranspose(int value)
int NunchuckTranspose(int value)
int VelocityCurve(int value)
int PitchCurve(int value)
void wiiReset(wiimote _wm)
int inrange(int i)

AccDetect
float count
float ocount
float precount
float zo
double hoopHeight
int fire
169
int velocity
int velocityCurve
int upCount
int up2Count
bool upMode
bool up2Mode
bool downMode
bool firstrun
Constructor()
int peakDetect(float z)
int countToVelocity(float countIn)
void reset()
int getFire()
int getVelocity()
void setHoopHeight()
int VelocityCurve(int value)

PitchDetect
double x[]
double y[]
double yx[]
double yy[]
int pitch
float butterworth(float x2f)
float ybutterworth(float x2f)
int pitchDetect(float x, float y, float z, int pitchCurve, CONTROLER cONTROLLER)
int getPitch()
int getVelocity()
float calibrate(float x, float y)

MIDIOut
Constructor(int device)
void sendNote(int channel, int note, int velocity)
void sendNoteOff(int channel, int note)
void sendControlChange(int channel, int cc, int value)
int OutputDeviceId(value)

17u

A>` EF# J#442 J4,77 S'7&7
CelloMainForm
Constructor()
void Form1_Load(object sender, EventArgs e)
void populateWiimoteListCombo()
void populateChannelCombos()
void populateOutputDeviceListCombo()
void resistivityTrackbar_Scroll(object sender, EventArgs e)
void rangeTrackbar_Scroll(object sender, EventArgs e)
void wiiMidiChannelCombo_SelectedIndexChanged(object sender, EventArgs e)
voidwiimoteListCombo_SelectedIndexChanged(object sender, EventArgs e)

MIDIOut
Constructor(int device)
void sendNote(int channel, int note, int velocity)
void sendNoteOff(int channel, int note)
void sendControlChange(int channel, int cc, int value)
void sendPitchBend(int channel, int pitchbend)
int OutputDeviceId(int value)

HighPassFilt
double[] xlp
double[] ylp
float _05HzHPF(float x2)

LowPassFilt
double[] xlp
double[] ylp
float _1HzLPF(float x2)
float _2HzLPF(float x2)

MotionDetect
HighPassFilt xHPF
HighPassFilt zHPF
LowPassFilt yLPF
171
LowPassFilt nyLPF
const PI
float resistivity
float _xExcitation
float _zExcitation
float yfilt
float yPitch
float nyFilt
float motionProcessing(float x, float y, float z)
float nyProc(float ny)
float latchAndHold(float excitation, float _excitation)
float Xexcitation(int value)
float Zexcitation(int value)
float Resistivity(int value)
int yMidiPitch()
int nyMidiPitch()

172

PitchDetect
LowPassFilt zPitchFilt
int pitchBend
int pitch
int range
double noteOnXValue
const PI
float z
double[] x
double[] y
float nunchuckXFilter(float x2f)
float nunchuckZFilter(float z2f)
int angleToNote()
int angleToPitchbend()
float calibrate(float x, float y)
int Range(int value)

17S

WiiMidiDelegate
Wiimote wm
MIDIOut midiOut
WiimoteState ows
MotionDetect motionDetect
PitchDetect pitchDetect
Timer startTimer
int outputDeviceId
int pitchMovement
int sampleSelect
int sampleSelect2
int basePitch
int pitch
int _pitch
int channel
int tuning
int transpose
int xvel
int zvel
bool firstRun
bool startBool
const PI
Constructor()
void wm_WiimoteExtensionChanged(object sender, WiimoteChangedEventArgs args)
void SetLEDs(bool a, bool b, bool c, bool d)
void wiiReset(wiimote _wm)
int inrange(int i)
void startTimerUp(object source, ElapsedEventArgs e)
int joyToMidi(double j)
int Channel (int value)
int Transpose (int value)
int Range (int value)
int Tuning (int value)
int PitchMovement (int value)
float Resistivity (float value)
int checkCZ (bool c, boolz, int b)
int joyToPB (float x, float y)
int joyToLevel(float x, float y)

174

A@ J2+='1,&'21 ?17&-%+#1& J4,77 4'7&7
Nany of the classes foi this application aie veiy similai to that of the sepaiate
Cello anu Spheies. The Cello anu Spheie classes founu in the Combination
instiument aie the same as the WiiNiuiBelegate classes in theii iespective
piogiams. Bata piocessing classes aie iuentical. The only uiffeience is the
auuition of an EventBanulei which is listeneu foi in the contiollei class. Foi this
ieason, only the classes unique to the combination instiument aie listeu.

MainForm
Controller controller
void outDeviceIdCombo_SelectedIndexChanged(object sender, EventArgs e)
void populateOutDeviceListCombo()

Controller
Sphere sphere
Cello cello
MidiOut midiOut
Constructor()
void sphere_midiHappened(object sender, MidiEventArgs e)
void cello_midiHappened(object sender, MidiEventArgs e)
int OutDeviceId(int value)
bool StartBool(bool value)

MidiEventArgs : EventArgs
readonly int message
readonly int channel
readonly int data1
readonly int data2
MidiEventArgs(int mes, int chan, int d1, int d2)

MidiListener
void OnMidi(object sender, MidiEventArgs e) message

17S

PAA-/0$N " @'+,1- G'0- #$%&$/*
1: The Spheie
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Timers;
namespace TheSphere
{
//Z has been flipped so you are looking for a +ive peak then trough in a very short amount of time
//Now using 'hoop' method to ensure snug tracking
//Reaches peak in around 3-4 cycles (from stationary)
//Therefor must jump through 3 up hoops to gurantee peak
//Class for finding peaks in accelerometer and giving back a 'go' signal
//Does not filter noise
//Does not do angle (for the maths class - experimentation needed)
//Don't have any timers not, use 3 counts.
//Need one more up to down detect to make sure we don't trigger twice.
class AccDetect
{
float count, ocount, precount, zo;
public double hoopHeight;
int fire, velocity, velocityCurve;
bool upMode, downMode, up2Mode, firstrun;
int upCount, up2Count;
//Constructor
public AccDetect()
{

hoopHeight = 0.1;
firstrun = true;
reset();
velocityCurve=0;

}

public int peakDetect(float z)
{
fire = 0;
if (firstrun)
{
zo = z;
firstrun = false;
return (0);
}

//Look for upward trend if in steady state
if((!upMode)&&(!downMode)&&(!up2Mode))
{
fire = 0;
//If this looks like an upward trend, start the hoop jumping, if we can jump through enough hoops
//then we're on a peak
if ((z > zo) && (upCount==0))
{
count = count + (z - zo);
precount = count;
ocount = count;
upCount++;
zo = z;
return (0);
}
//If we have started to look for an upward trend, keep counting until we have jumped through enough hoops.
//If we don't jump through enough hoops then just reset(); and return(0);
else if ((upCount > 0) && (upCount < 4))
{

//We still going up?
if (count > (ocount + hoopHeight))
{
upCount++;
ocount = count;
zo = z;
// Console.WriteLine("UC: {0}", upCount);
return (0);
}
//If not - TOO SLOW! reset();
else
{
reset();
return (0);
}

}

//If we've made it through 3 hoops then we're in a peak.
//Set up mode
else if (upCount == 4)
{
// Console.WriteLine("upZ:{0}",z);
zo = z;
upMode = true;
upCount = 0;
return (0);
}

//If we're not finding anything good then reset and try again
else
{
reset();
zo = z;
return (0);
}

}// end IF nothing yet
//We're in an upward spike, so we'll be going down very soon
//Just start looking for the down.
if (upMode)
{
//If we're still going up, carry on
if (z > zo)
{
zo = z;
return (0);
}
//If we are the downward peak, set down mode
if (z < zo)
{
velocity = countToVelocity(count);
downMode = true;
upMode = false;
zo = z;
return (0);
}

}

//Got to look for the bottom now - just wait till zo<z
if (downMode)
{

//If zo>z we're still going down, keep cycling
if (zo > z)
{
zo = z;
return (0);
}
//We've reched the bottom, send fire note
//To make sure we're not sending ghost notes,
//wait 2 cycles so as not to get the upward spike
if (zo < z)
{
// Console.WriteLine("Dn z:{0}, count:{1}", z, count);
up2Mode = true;
downMode = false;
if(count<0)
fire = 1;

return (1);
}

}
//Having an up2mode to wait to finish is a bit much really
//For now, just wait a cycle, should calm it down a bit
if (up2Mode)
{
if(up2Count< 0)
{
up2Count++;
return (0);
}

else
{
reset();
}

}
//Set up zo for next run
zo = z;
return (0);
}
private int countToVelocity(float countIn)
{
double fVelocity;
countIn = Math.Abs(countIn);

switch (velocityCurve)
{

case ((int)VELOCITY_CURVE.LINEAR):
{
break;
}

case ((int)VELOCITY_CURVE.SQUARED):
{
countIn=(countIn/11)*(countIn/11);
break;
}
case ((int)VELOCITY_CURVE.S_CURVE):
{
countIn = (float)((Math.Pow(((countIn / 2) - 30), 3))/50000)*127+64;
break;
}
}
fVelocity = ((countIn / 6) * 127);
velocity = (int)fVelocity;
if (velocity > 127)
velocity = 127;
if (velocity < 0)
velocity = 0;
Console.WriteLine("Count: {0}, fVel: {1}, iVel:{2}", countIn, fVelocity, velocity);
return (velocity);
}
private void reset()
{
count = 0;
ocount = 0;
precount = 0;
upMode = false;
downMode = false;
up2Mode = false;
fire = 0;
upCount = 0;
up2Count = 0;
velocity = 0;
}
//Need a query method to send data to
public int getFire()
{
return fire;
}
public void setHoopHeight(double height)
{
hoopHeight = height;
}
public int getVelocity(int velocityCurve)
{

return (velocity);
}
public int VelocityCurve
{
get{return (velocityCurve);}
set{velocityCurve=value;}
}
}
}
using System;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using Sanford.Multimedia.Midi;
using WiimoteLib;
namespace TheSphere
{
public partial class Form1 : Form
{
WiiMidiDelegate[] wiiMidiDelegate = new WiiMidiDelegate[2];
WiimoteCollection wc;
bool startup = true;
public Form1()
{

InitializeComponent();
int i;
wc = new WiimoteCollection();
try { wc.FindAllWiimotes(); }
catch { Application.Exit(); }
populateWiimoteListCombo();
for (i = 0; i < wc.Count; i++)
{ //Instantiate new WiiMidiDelegate on Load
wiiMidiDelegate[i] = new WiiMidiDelegate();
wiiMidiDelegate[i].OutputDeviceId = outDeviceListCombo.SelectedIndex+i;
wiiMidiDelegate[i].WiiChannel = wiiMidiChannelCombo.SelectedIndex + i;
wiiMidiDelegate[i].NunchuckChannel = nunchuckMidiChannelCombo.SelectedIndex + i;

}

populateChannelCombos();
populateOutDeviceListCombo();
populatePitchCurveCombo();
populateVelocityCurveCombo();

startup = false;
}
private void Form1_Load(object sender, EventArgs e)
{

wiiMidiChannelCombo.SelectedIndex = 0;
nunchuckMidiChannelCombo.SelectedIndex = 0;

outDeviceListCombo.SelectedIndex = 2;

pitchCurveCombo.SelectedIndex = 0;
velocityCurveCombo.SelectedIndex= 0;

}
//Set up menus
private void populateWiimoteListCombo()
{
int i;

wiimoteListCombo.Items.Clear();
{
wiimoteListCombo.Items.Add(i);
}
wiimoteListCombo.SelectedIndex = 0;
}
private void populateChannelCombos()
{
wiiMidiChannelCombo.Items.Clear();
for (int i=0;i<15;i++)
{
wiiMidiChannelCombo.Items.Add(i);
}
nunchuckMidiChannelCombo.Items.Clear();
for (int i = 0; i < 15; i++)
{
nunchuckMidiChannelCombo.Items.Add(i);
}

}
private void populateOutDeviceListCombo()
{
outDeviceListCombo.Items.Clear();
if (OutputDevice.DeviceCount > 0)
{
for (int i = 0; i < OutputDevice.DeviceCount; i++)
{
outDeviceListCombo.Items.Add(OutputDevice.GetDeviceCapabilities(i).name);
}

}
}
private void populatePitchCurveCombo()
{
pitchCurveCombo.Items.Clear();
pitchCurveCombo.Items.Add("1: Numeric");
pitchCurveCombo.Items.Add("2: Angular");
pitchCurveCombo.Items.Add("3: Angular Flick");
pitchCurveCombo.Items.Add("4: Thumb 5 Flick");
pitchCurveCombo.Items.Add("5: Thumb 7 FLick");
pitchCurveCombo.Items.Add("6: Dual 12 tone");
pitchCurveCombo.Items.Add("7: Two Tier");

}
private void populateVelocityCurveCombo()
{
velocityCurveCombo.Items.Clear();
velocityCurveCombo.Items.Add("1: Linear");
velocityCurveCombo.Items.Add("2: Squared");
velocityCurveCombo.Items.Add("3: S Curve");

}

public void wiiMidiChannelCombo_SelectedIndexChanged(object sender, EventArgs e)
{
wiiMidiDelegate[wiimoteListCombo.SelectedIndex].WiiChannel = wiiMidiChannelCombo.SelectedIndex;
System.Console.WriteLine("Wiimote Chanel: {0}", wiiMidiDelegate[wiimoteListCombo.SelectedIndex].WiiChannel);

}
private void nunchuckMidiChannelCombo_SelectedIndexChanged(object sender, EventArgs e)
{
wiiMidiDelegate[wiimoteListCombo.SelectedIndex].NunchuckChannel = nunchuckMidiChannelCombo.SelectedIndex;
System.Console.WriteLine("Nunchuck Channel: {0}", wiiMidiDelegate[wiimoteListCombo.SelectedIndex].NunchuckChannel);
}
public void outDeviceListCombo_SelectedIndexChanged(object sender, EventArgs e)
{
wiiMidiDelegate[wiimoteListCombo.SelectedIndex].OutputDeviceId = outDeviceListCombo.SelectedIndex;
System.Console.WriteLine("Output Device Change: {0}", wiiMidiDelegate[wiimoteListCombo.SelectedIndex].OutputDeviceId);
}
private void pitchCurveCombo_SelectedIndexChanged(object sender, EventArgs e)
{
wiiMidiDelegate[wiimoteListCombo.SelectedIndex].PitchCurve = pitchCurveCombo.SelectedIndex+1;
Console.WriteLine("PitchCurve:{0}", pitchCurveCombo.SelectedIndex+1);
switch (pitchCurveCombo.SelectedIndex)
{
case (0): pitchCurveBox.Image = new Bitmap("img/numeric.GIF"); break;
case (1): pitchCurveBox.Image = new Bitmap("img/angular.GIF"); break;
case (2): pitchCurveBox.Image = new Bitmap("img/flick.GIF"); break;
case (3): pitchCurveBox.Image = new Bitmap("img/thumb5.GIF"); break;
case (5): pitchCurveBox.Image = new Bitmap("img/noimg.GIF"); break;
// case (7): pitchCurveBox.Image = new Bitmap("img/noimg.GIF"); break;

}

}
private void velocityCurveCombo_SelectedIndexChanged(object sender, EventArgs e)
{
wiiMidiDelegate[wiimoteListCombo.SelectedIndex].VelocityCurve = velocityCurveCombo.SelectedIndex+1;
Console.WriteLine("VelocityCurve:{0}", velocityCurveCombo.SelectedIndex+1);
}
private void wiimoteListCombo_SelectedIndexChanged(object sender, EventArgs e)
{
if(!startup)
wiiMidiDelegate[wiimoteListCombo.SelectedIndex].wiiReset(wc[wiimoteListCombo.SelectedIndex]);

}

}
}
using System;
using System.Threading;
using System.Linq;
using System.Text;
using Sanford.Multimedia;
namespace TheSphere
{

public class MIDIOut
{
private OutputDevice outputDevice;

//Constructor
public MIDIOut(int device)
{

//Make sure at least one devce is set up
if (OutputDevice.DeviceCount == 0)
{
throw System.Exception("New Devices Found!");
}
else
{
outputDevice = new OutputDevice(device);
}
}

//Sends a note out to MIDI yoke
public void sendNote(int channel, int note, int velocity)
{
ChannelMessageBuilder builder = new ChannelMessageBuilder();
builder.Command = ChannelCommand.NoteOn;
builder.MidiChannel = channel;
builder.Data1 = note;
builder.Data2 = velocity;
builder.Build();

outputDevice.Send(builder.Result);
}
public void sendNoteOff(int channel, int note)
{
builder.Command = ChannelCommand.NoteOff;
builder.Data2 = 0;
builder.Build();
}
public void sendControlChange(int channel, int cc, int value)
{
builder.Command = ChannelCommand.Controller;
builder.Data1 = cc;
builder.Data2 = value;
builder.Build();
}
/// <summary>
/// Setter/getter for outputDeviceId
/// </summary>
public int OutputDeviceId
{
get { return outputDevice.DeviceID; }
set
{
outputDevice.Close();
outputDevice = new OutputDevice(value);
}
}

}
}
using System;
using System.Linq;
using System.Text;
namespace TheSphere
{
/// <summary>
/// Class to find the angle that the wiimote is held at to establish the pitch.
/// </summary>
class PitchDetect
{
int pitch;

//Filter values
//For note detection
double[] x = new double[3]{0,0,0};
double[] y = new double[3]{0,0,0};
//For tilt switch
double[] yx = new double[3] { 0, 0, 0 };
double[] yy = new double[3] { 0, 0, 0 };

//Constructor
public PitchDetect()
{
}
//Second order butterworth LPF for x values, cutoff at 5Hz

public float butterworth(float x2f)
{
double gain;
double x2d = (double)x2f;
float y2f;
for (int i = 0; i < 3; i++)
{
if ((x[i] > 50)||(x[i]<-50))
x[i] = 0;
if ((y[i] > 50) || (y[i] < -50))
y[i] = 0;
}
//10Hz gain
//gain = 14.82463775;

//5Hz gain
//gain = 49.79245121
//3Hz gain
gain = 127.8738361;
//1.5Hz Gain
gain = 480;
x[0] = x[1];
x[1] = x[2];
x[2] = (x2d / gain);
y[0] = y[1];
y[1] = y[2];
//Recusrsion expression 2nd Order C= 10Hz
//y[2] = x[0] + 2 * x[1] + x[2] - (0.4128015981 * y[0]) + (1.1429805025 * y[1]);

//Recursion expression 2nd Order C = 5Hz
//y[2] = x[0] + 2 * x[1] + x[2] + (-0.6413515381 * y[0]) + (1.5610180758 * y[1]);
// y[2] = x[0] + 2 * x[1] + x[2] + (-0.7660066009 * y[0]) + (1.7347257688 * y[1]);
//Recursion expression 2nd order C=1.5 Hz
y[2]= x[0] + 2*x[1] + x[2] + (-0.8752145483 * y[0]) + (1.8668922797 * y[1]);
y2f = (float)y[2];
return y2f;
}
public float yButterworth(float x2f)
{
double gain;
float y2f;
for (int i = 0; i < 3; i++)
{
if ((yx[i] > 50) || (yx[i] < -50))
x[i] = 0;
if ((yy[i] > 50) || (yy[i] < -50))
yy[i] = 0;
}
//3 Hz
/*
//3Hz gain
gain = 127.8738361;
yx[0] = yx[1];
yx[1] = yx[2];
yx[2] = (x2d / gain);
yy[0] = yy[1];
yy[1] = yy[2];
yy[2] = yx[0] + 2 * yx[1] + yx[2] + (-0.7660066009 * yy[0]) + (1.7347257688 * yy[1]);
y2f = (float)yy[2];
*/
//1 Hz Gain
gain = 1058.546241;
yx[0] = yx[1];
yx[1] = yx[2];
yx[2] = (x2d/gain);
yy[0] = yy[1];
yy[1] = yy[2];
yy[2] = (yx[0] + yx[2]) + 2 * yx[1] + (-0.9149758348 * yy[0]) + (1.9111970674 * yy[1]);

y2f = (float)yy[2];

return y2f;
}
public int pitchDetect(float x, float y, float z, int pitchCurve, CONTROLER cONTROLLER)

x=calibrate(x, y);

switch(pitchCurve)
{

case ((int)PITCH_CURVE.ANGULAR):
{
if (x < -0.8)
pitch = 60;
else if ((x >= -0.8) && (x < -0.5))
pitch = 63;
else if ((x >= -0.5) && (x < 0))
pitch = 65;
else if ((x >= 0) && (x < 0.5))
pitch = 67;
else if ((x >= 0.5) && (x < 0.8))
pitch = 70;
else if (x >= 0.8)
pitch = 72;
break;
}
case ((int)PITCH_CURVE.NUMERICAL):
{
if (x < -0.66)
pitch = 60;
else if ((x >= -0.66) && (x < -0.33))
pitch = 63;
else if ((x >= -0.33) && (x < 0))
pitch = 65;
else if ((x >= 0) && (x < 0.33))
pitch = 67;
else if ((x >= 0.33) && (x < 0.66))
pitch = 70;
else if (x >= 0.66)
pitch = 72;
break;
}
case ((int)PITCH_CURVE.FLICK):
{
if (x < -1)
pitch = 60;
else if ((x >= -1) && (x < -0.7))
pitch = 63;
else if ((x >= -0.7) && (x < 0))
pitch = 65;
else if ((x >= 0) && (x < 0.7))
pitch = 67;
else if ((x >= 0.7) && (x < 1))
pitch = 70;
else if (x >= 1)
pitch = 72;
break;
}
case ((int)PITCH_CURVE.THUMB_FLICK_5):
{
if (x < -1)
pitch = 67;
else if ((x >= -1) && (x < -0.5))
pitch = 63;
else if ((x >= -0.5) && (x < 0.5))
pitch = 60;
else if ((x >= 0.5) && (x < 1))
pitch = 65;
else if (x >= 1)
pitch = 70;
break;

}
{
if (x < -1)
pitch = 72;
else if ((x >= -1) && (x < -0.81))
pitch = 67;
else if ((x >= -0.81) && (x < -0.31))
pitch = 63;
else if ((x >= -0.31) && (x < 0.31))
pitch = 60;

else if ((x >= 0.31) && (x < 0.81))
pitch = 65;
else if ((x >= 0.81) && (x < 1))
pitch = 70;

else if (x >= 1)
pitch = 75;
break;
}

case ((int)PITCH_CURVE.DUAL_12):
{
switch (cONTROLLER)
{
case (CONTROLER.NUNCHUCK):
{
if (x < -0.8)
pitch = 68;
else if ((x >= -0.8) && (x < -0.5))
pitch = 65;
else if ((x >= -0.5) && (x < 0))
pitch = 63;
else if ((x >= 0) && (x < 0.5))
pitch = 62;
else if ((x >= 0.5) && (x < 0.8))
pitch = 66;
else if (x >= 0.8)
pitch = 71;
break;
}
case(CONTROLER.WIIMOTE):
{
if (x < -0.8)
pitch = 70;
else if ((x >= -0.8) && (x < -0.5))
pitch = 64;
else if ((x >= -0.5) && (x < 0))
pitch = 61;
else if ((x >= 0) && (x < 0.5))
pitch = 60;
else if ((x >= 0.5) && (x < 0.8))
pitch = 77;
else if (x >= 0.8)
pitch = 79;

break;
}

}

break;
}
case ((int)PITCH_CURVE.TWO_TIER):
{//Lower Tier
if (y < 0.15)
{
if (x < -1)
pitch = 60;
else if ((x >= -1) && (x < -0.7))
pitch = 63;
else if ((x >= -0.7) && (x < 0))
pitch = 65;
else if ((x >= 0) && (x < 0.7))
pitch = 67;
else if ((x >= 0.7) && (x < 1))
pitch = 68;
else if (x >= 1)
pitch = 70;

}
//Upper Tier
else if (y > 0.3)
{
x = calibrate(x, y);
if (x < -1)
pitch = 72;
else if ((x >= -1) && (x < -0.7))
pitch = 75;
else if ((x >= -0.7) && (x < 0))
pitch = 77;
else if ((x >= 0) && (x < 0.7))
pitch = 79;
else if ((x >= 0.7) && (x < 1))
pitch = 80;
else if (x >= 1)
pitch = 82;
}
break;
}
}
if ((cONTROLLER == CONTROLER.NUNCHUCK) && (pitchCurve != (int)PITCH_CURVE.DUAL_12))
pitch = pitch - 12;

Console.WriteLine("x:{0}, P:{1}", x, pitch);
return pitch;
}
public int getPitch()
{
return pitch;
}
//Method to make sure we've got the right angle (as x changes with respect to y as pitch increaces)
//Implement X = x(1-y) + y(1-x)
private float calibrate(float x, float y)
{
x = (float)(x / Math.Cos(Math.Asin(y)));
return x;
}
}
}
using System;
using System.Data;
using System.Linq;
using System.Text;
using WiimoteLib;
namespace TheSphere
{
{
WiiMidiDelegate[] wiiMidiDelegate = new WiiMidiDelegate[2];
bool startup = true;
public Form1()
{

int i;
catch { Application.Exit(); }
{ //Instantiate new WiiMidiDelegate on Load
wiiMidiDelegate[i] = new WiiMidiDelegate();
wiiMidiDelegate[i].OutputDeviceId = outDeviceListCombo.SelectedIndex+i;
wiiMidiDelegate[i].WiiChannel = wiiMidiChannelCombo.SelectedIndex + i;
wiiMidiDelegate[i].NunchuckChannel = nunchuckMidiChannelCombo.SelectedIndex + i;

}

populateChannelCombos();
populatePitchCurveCombo();
populateVelocityCurveCombo();

startup = false;
}
{

wiiMidiChannelCombo.SelectedIndex = 0;
nunchuckMidiChannelCombo.SelectedIndex = 0;


pitchCurveCombo.SelectedIndex = 0;
velocityCurveCombo.SelectedIndex= 0;

}
//Set up menus
{
int i;

{
}
wiimoteListCombo.SelectedIndex = 0;
}
private void populateChannelCombos()
{
wiiMidiChannelCombo.Items.Clear();
for (int i=0;i<15;i++)
{
wiiMidiChannelCombo.Items.Add(i);
}
nunchuckMidiChannelCombo.Items.Clear();
for (int i = 0; i < 15; i++)
{
nunchuckMidiChannelCombo.Items.Add(i);
}

}
{
{
{
}

}
}
private void populatePitchCurveCombo()
{
pitchCurveCombo.Items.Clear();
pitchCurveCombo.Items.Add("1: Numeric");
pitchCurveCombo.Items.Add("2: Angular");
pitchCurveCombo.Items.Add("3: Angular Flick");
pitchCurveCombo.Items.Add("4: Thumb 5 Flick");
pitchCurveCombo.Items.Add("5: Thumb 7 FLick");
pitchCurveCombo.Items.Add("6: Dual 12 tone");
pitchCurveCombo.Items.Add("7: Two Tier");

}
private void populateVelocityCurveCombo()
{
velocityCurveCombo.Items.Clear();
velocityCurveCombo.Items.Add("1: Linear");
velocityCurveCombo.Items.Add("2: Squared");
velocityCurveCombo.Items.Add("3: S Curve");

}

public void wiiMidiChannelCombo_SelectedIndexChanged(object sender, EventArgs e)
{
wiiMidiDelegate[wiimoteListCombo.SelectedIndex].WiiChannel = wiiMidiChannelCombo.SelectedIndex;
System.Console.WriteLine("Wiimote Chanel: {0}", wiiMidiDelegate[wiimoteListCombo.SelectedIndex].WiiChannel);

}
private void nunchuckMidiChannelCombo_SelectedIndexChanged(object sender, EventArgs e)
{
wiiMidiDelegate[wiimoteListCombo.SelectedIndex].NunchuckChannel = nunchuckMidiChannelCombo.SelectedIndex;
System.Console.WriteLine("Nunchuck Channel: {0}", wiiMidiDelegate[wiimoteListCombo.SelectedIndex].NunchuckChannel);
}
public void outDeviceListCombo_SelectedIndexChanged(object sender, EventArgs e)
{
wiiMidiDelegate[wiimoteListCombo.SelectedIndex].OutputDeviceId = outDeviceListCombo.SelectedIndex;
System.Console.WriteLine("Output Device Change: {0}", wiiMidiDelegate[wiimoteListCombo.SelectedIndex].OutputDeviceId);
}
private void pitchCurveCombo_SelectedIndexChanged(object sender, EventArgs e)
{
wiiMidiDelegate[wiimoteListCombo.SelectedIndex].PitchCurve = pitchCurveCombo.SelectedIndex+1;
Console.WriteLine("PitchCurve:{0}", pitchCurveCombo.SelectedIndex+1);
switch (pitchCurveCombo.SelectedIndex)
{
case (0): pitchCurveBox.Image = new Bitmap("img/numeric.GIF"); break;
case (1): pitchCurveBox.Image = new Bitmap("img/angular.GIF"); break;
case (2): pitchCurveBox.Image = new Bitmap("img/flick.GIF"); break;
// case (7): pitchCurveBox.Image = new Bitmap("img/noimg.GIF"); break;

}

}
private void velocityCurveCombo_SelectedIndexChanged(object sender, EventArgs e)
{
wiiMidiDelegate[wiimoteListCombo.SelectedIndex].VelocityCurve = velocityCurveCombo.SelectedIndex+1;
Console.WriteLine("VelocityCurve:{0}", velocityCurveCombo.SelectedIndex+1);
}
{
if(!startup)
wiiMidiDelegate[wiimoteListCombo.SelectedIndex].wiiReset(wc[wiimoteListCombo.SelectedIndex]);

}

}
}
using System;
using System.Linq;
using System.Text;
using System.IO;
using WiimoteLib;
namespace TheSphere
{
public class WiiMidiDelegate
{
//Set up classes for getting wii data and sending MIDI
Wiimote wm = new Wiimote();
MIDIOut midiOut = new MIDIOut(0);
int wiiChannel, nunchuckChannel;
int outputDeviceId;
int wiiPitch, nunchuckPitch, wiiTranspose, nunchuckTranspose;
int velocityCurve, pitchCurve;
WiimoteState ows = new WiimoteState();
AccDetect wiimoteAccDetect = new AccDetect();
AccDetect nunchuckAccDetect = new AccDetect();
PitchDetect wiiPitchDetect = new PitchDetect();
PitchDetect nunchuckPitchDetect = new PitchDetect();
Timer wiiNoteOffTimer = new Timer();
Timer nunchuckNoteOffTimer = new Timer();

bool firstrun;
//Constructor
public WiiMidiDelegate()
{
wiiChannel = 0;
nunchuckChannel = 0;
outputDeviceId = 2;
firstrun = true;
wiiReset(wm);
//Set up note off timers
wiiNoteOffTimer.Elapsed += new ElapsedEventHandler(wiiNoteOffTimerOut);
nunchuckNoteOffTimer.Elapsed += new ElapsedEventHandler(nunchuckNoteOffTimerOut);
wiiNoteOffTimer.Interval = 50;
nunchuckNoteOffTimer.Interval = 50;
wiiNoteOffTimer.AutoReset = false;
nunchuckNoteOffTimer.AutoReset = false;
WiiTranspose=0;
NunchuckTranspose=0;
pitchCurve = 0;
velocityCurve = 0;
}
//Called when anything changes on Wiimote
{
float wx, wy, wz, nx, ny, nz;
// current state information
WiimoteState ws = args.WiimoteState;

//All values flipped to be consistant with wiimote literiture
wx = -ws.AccelState.Values.X;
wy = -ws.AccelState.Values.Y;
wz = -ws.AccelState.Values.Z;
nx = -ws.NunchukState.AccelState.Values.X;
ny = -ws.NunchukState.AccelState.Values.Y;
nz = -ws.NunchukState.AccelState.Values.Z;
if (firstrun == true)
{
ows.AccelState = ws.AccelState;
ows.ButtonState = ws.ButtonState;
ows.NunchukState = ws.NunchukState;
firstrun = false;

}

if ((ws.ButtonState.A) != (ows.ButtonState.A))
{
if (ws.ButtonState.A == true)
{
this.midiOut.sendControlChange(wiiChannel, 50, 120);

}
else
{

}
}

if((ws.NunchukState.Z) != (ows.NunchukState.Z))
{
if (ws.NunchukState.Z == true)
{
Console.WriteLine("NcX: {0} NcY: {1} NcZ: {2}", nx, ny, nz);
Console.WriteLine("NUNCHUCK DOWN OCT");
nunchuckTranspose = nunchuckTranspose - 12;
}
else
{
Console.WriteLine("Z Off");
}
}

if ((ws.NunchukState.C) != (ows.NunchukState.C))
{
if (ws.NunchukState.C == true)
{
Console.WriteLine("NUNCHUCK UP OCT");
nunchuckTranspose = nunchuckTranspose + 12;
}
else
{
Console.WriteLine("C Off");
}
}
if (((ws.NunchukState.Joystick.X) != (ows.NunchukState.Joystick.X))&&ws.ButtonState.B)
{
//Joystick goes from -0.5 to +0.5
double joyx = ws.NunchukState.Joystick.X;
int joyMidiOut = (int)((joyx + 0.5) * 127);
if (joyMidiOut > 127)
joyMidiOut = 127;
if (joyMidiOut < 0)
joyMidiOut = 0;

this.midiOut.sendControlChange(wiiChannel, 51, joyMidiOut);
}
if (((ws.NunchukState.Joystick.Y) != (ows.NunchukState.Joystick.Y)) && ws.ButtonState.B)
{
double joyy = ws.NunchukState.Joystick.Y;
int joyMidiOut = (int)((joyy + 0.5) * 127);
joyMidiOut = 127;
if (joyMidiOut < 0)
joyMidiOut = 0;
}
//Up
if (ws.ButtonState.Up != ows.ButtonState.Up)
{
if (ws.ButtonState.Up == true)
{
}
else
{
}
}
if ((ws.ButtonState.Down) != (ows.ButtonState.Down))
{
if (ws.ButtonState.Down == true)
{
}
else
{
}
}
//Transpose
if (((ws.ButtonState.Minus) != (ows.ButtonState.Minus))||((ws.ButtonState.Plus) != (ows.ButtonState.Plus)))
{
if (ws.ButtonState.Minus)
{
wiiTranspose--;
nunchuckTranspose--;
//OppButtonPress(this, OPP_CODE.TRANS_DOWN);
Console.WriteLine("WT:{0} NT:{1}", wiiTranspose, nunchuckTranspose);
}
if (ws.ButtonState.Plus)
{
wiiTranspose++;
nunchuckTranspose++;
// OppButtonPress(this, OPP_CODE.TRANS_UP);
}
}
//send z axis data for sample by sample analysis - it will report back when WMD should send a fire note
//We need to up scale the nunchuck as it only goes to +- 3g
nz = (nz / 3) * 5;
wiimoteAccDetect.peakDetect(wz);
nunchuckAccDetect.peakDetect(nz);
//send x and y data for sample by sample analysis to LPF the x axis
wx = wiiPitchDetect.butterworth(wx);
nx = nunchuckPitchDetect.butterworth(nx);
wy = wiiPitchDetect.yButterworth(wy);
ny = wiiPitchDetect.yButterworth(ny);
// tw.WriteLine("{0} {1}",ws.AccelState.Values.X , wx);

if (wiimoteAccDetect.getFire() == 1)
{
//Make sure we're not going to kill our synth with unplayable notes
if (wiiPitch < 0)
wiiPitch = 0;
if (wiiPitch > 127)
wiiPitch = 127;
wiiPitch = inrange(wiiPitch);
wiiPitch = wiiPitchDetect.pitchDetect(wx, wy, wz, pitchCurve, CONTROLER.WIIMOTE);
this.midiOut.sendNote(wiiChannel, inrange(wiiPitch+wiiTranspose), wiimoteAccDetect.getVelocity(velocityCurve));
wiiNoteOffTimer.Enabled = true;

}
if (nunchuckAccDetect.getFire() == 1)
{
nunchuckPitch = inrange(nunchuckPitch);
nunchuckPitch = nunchuckPitchDetect.pitchDetect(nx, ny, nz, pitchCurve, CONTROLER.NUNCHUCK);

this.midiOut.sendNote(nunchuckChannel, inrange(nunchuckPitch+nunchuckTranspose), nunchuckAccDetect.getVelocity(velocityCurve));
nunchuckNoteOffTimer.Enabled=true;
//this.midiOut.sendNoteOff(nunchuckChannel, nunchuckPitch);
}
if (ws.ButtonState.B)
// tw.Close();

//Put old wiimote stare into storage for comparrison next time

}
//Called when nunchuck is inserted
void wm_WiimoteExtensionChanged(object sender, WiimoteExtensionChangedEventArgs args)
{
if (args.Inserted)
wm.SetReportType(InputReport.ExtensionAccel, true);
else
wm.SetReportType(InputReport.ButtonsAccel, true);
}
//Timers for turning the notes off
private void wiiNoteOffTimerOut(object source, ElapsedEventArgs e)
{
midiOut.sendNoteOff(wiiChannel, inrange(wiiPitch+wiiTranspose));
}

private void nunchuckNoteOffTimerOut(object source, ElapsedEventArgs e)
{
midiOut.sendNoteOff(nunchuckChannel, inrange(nunchuckPitch+nunchuckTranspose));
}
//Sets LEDs on wiimote
public void SetLEDs(bool a, bool b, bool c, bool d)
{
wm.SetLEDs(a, b, c, d);
}
//Sets and gets the MIDI output device in the MIDIout class
{
get { return outputDeviceId; }
set
{
outputDeviceId = value;
midiOut.OutputDeviceId = outputDeviceId;
}
}
//Sets and gets
public int WiiChannel
{
get { return wiiChannel; }
set
{
wiiChannel = value;
}
}
public int NunchuckChannel
{
get { return nunchuckChannel; }
set
{
nunchuckChannel = value;
}
}
public int WiiTranspose
{
get {return wiiTranspose;}
set
{
wiiTranspose=value;
}
}
public int NunchuckTranspose
{
get { return nunchuckTranspose; }
set { nunchuckTranspose = value; }
}
{
get { return velocityCurve; }
set { velocityCurve = value;
wiimoteAccDetect.VelocityCurve = value;
}
}
public int PitchCurve
{
get { return pitchCurve; }
set { pitchCurve = value; }
}
public void wiiReset(Wiimote _wm)
{

//Reset events
wm.WiimoteChanged -= wm_WiimoteChanged;
wm.WiimoteExtensionChanged -= wm_WiimoteExtensionChanged;
//See if we already have a wiimote open, if so. disconnect.
wm.Disconnect();
wm.Dispose();
wm = _wm;

//setup the event to handle state chages
wm.WiimoteChanged += wm_WiimoteChanged;
//setup the event to handle ins/rm extensions
wm.WiimoteExtensionChanged += wm_WiimoteExtensionChanged;
//If not already connected,
//Connect to wiimote
try { wm.Connect(); }
catch
{
wiimoteNotFoundException();
}

//See if we have an extension:
if (wm.WiimoteState.Extension)
else
//Set LEDs to make sure we're connected
this.SetLEDs(true, false, false, true);
//Display Battery level
Console.WriteLine("Bat: {0}", wm.WiimoteState.Battery);

}
WiimoteNotFoundException wiimoteNotFoundException()
{
WiimoteNotFoundException wiimoteNotFound = new WiimoteNotFoundException();
return (wiimoteNotFound);
}
int inrange(int i)
{
if (i < 0)
i = 0;
if (i > 127)
i = 127;
return i;
}
}
}
using System;
namespace TheSphere
{
enum VELOCITY_CURVE
{LINEAR = 1,
SQUARED,
S_CURVE
}
enum PITCH_CURVE
{
//Min Pent
NUMERICAL =1,
ANGULAR,
FLICK,
THUMB_FLICK_5,
THUMB_FLICK_7,
//Other
DUAL_12,
TWO_TIER,
TWO_TIER_2CHAN,
}
enum CONTROLER
{WIIMOTE, NUNCHUCK
}

public enum OPP_CODE {TRANS_UP, TRANS_DOWN}

public struct OldWmState
{
public bool A;
public bool B;
public bool UP;
public bool DOWN;
public bool LEFT;
public bool RIGHT;
public bool MINUS;
public bool PLUS;
public bool HOME;
public bool ONE;
public bool TWO;
public double accX;
public double accY;
public double accZ;
}
}
using System;
using System.Linq;
using System.Text;
namespace TheSphere
{
/// <summary>
/// Class to find the angle that the wiimote is held at to establish the pitch.
/// </summary>
class PitchDetect
{
int pitch;

//Filter values
//For note detection
double[] x = new double[3]{0,0,0};
double[] y = new double[3]{0,0,0};
//For tilt switch
double[] yx = new double[3] { 0, 0, 0 };
double[] yy = new double[3] { 0, 0, 0 };

//Constructor
public PitchDetect()
{
}

public float butterworth(float x2f)
{
double gain;
float y2f;
for (int i = 0; i < 3; i++)
{
if ((x[i] > 50)||(x[i]<-50))
x[i] = 0;
if ((y[i] > 50) || (y[i] < -50))
y[i] = 0;
}
//10Hz gain
//gain = 14.82463775;

//5Hz gain
//gain = 49.79245121
//3Hz gain
gain = 127.8738361;
//1.5Hz Gain
gain = 480;
x[0] = x[1];
x[1] = x[2];
x[2] = (x2d / gain);
y[0] = y[1];
y[1] = y[2];
//y[2] = x[0] + 2 * x[1] + x[2] - (0.4128015981 * y[0]) + (1.1429805025 * y[1]);

//y[2] = x[0] + 2 * x[1] + x[2] + (-0.6413515381 * y[0]) + (1.5610180758 * y[1]);
// y[2] = x[0] + 2 * x[1] + x[2] + (-0.7660066009 * y[0]) + (1.7347257688 * y[1]);
//Recursion expression 2nd order C=1.5 Hz
y[2]= x[0] + 2*x[1] + x[2] + (-0.8752145483 * y[0]) + (1.8668922797 * y[1]);
y2f = (float)y[2];
return y2f;
}
public float yButterworth(float x2f)
{
double gain;
float y2f;
for (int i = 0; i < 3; i++)
{
if ((yx[i] > 50) || (yx[i] < -50))
x[i] = 0;
if ((yy[i] > 50) || (yy[i] < -50))
yy[i] = 0;
}
//3 Hz
/*
//3Hz gain
gain = 127.8738361;
yx[0] = yx[1];
yx[1] = yx[2];
yx[2] = (x2d / gain);
yy[0] = yy[1];
yy[1] = yy[2];
yy[2] = yx[0] + 2 * yx[1] + yx[2] + (-0.7660066009 * yy[0]) + (1.7347257688 * yy[1]);
y2f = (float)yy[2];
*/
//1 Hz Gain
gain = 1058.546241;
yx[0] = yx[1];
yx[1] = yx[2];
yx[2] = (x2d/gain);
yy[0] = yy[1];
yy[1] = yy[2];
yy[2] = (yx[0] + yx[2]) + 2 * yx[1] + (-0.9149758348 * yy[0]) + (1.9111970674 * yy[1]);

y2f = (float)yy[2];

return y2f;
}
public int pitchDetect(float x, float y, float z, int pitchCurve, CONTROLER cONTROLLER)

x=calibrate(x, y);

switch(pitchCurve)
{

case ((int)PITCH_CURVE.ANGULAR):
{
if (x < -0.8)
pitch = 60;
else if ((x >= -0.8) && (x < -0.5))
pitch = 63;
else if ((x >= -0.5) && (x < 0))
pitch = 65;
else if ((x >= 0) && (x < 0.5))
pitch = 67;
else if ((x >= 0.5) && (x < 0.8))
pitch = 70;
else if (x >= 0.8)
pitch = 72;
break;
}
case ((int)PITCH_CURVE.NUMERICAL):
{
if (x < -0.66)
pitch = 60;
else if ((x >= -0.66) && (x < -0.33))
pitch = 63;
else if ((x >= -0.33) && (x < 0))
pitch = 65;
else if ((x >= 0) && (x < 0.33))
pitch = 67;
else if ((x >= 0.33) && (x < 0.66))
pitch = 70;
else if (x >= 0.66)
pitch = 72;
break;
}
case ((int)PITCH_CURVE.FLICK):
{
if (x < -1)
pitch = 60;
else if ((x >= -1) && (x < -0.7))
pitch = 63;
else if ((x >= -0.7) && (x < 0))
pitch = 65;
else if ((x >= 0) && (x < 0.7))
pitch = 67;
else if ((x >= 0.7) && (x < 1))
pitch = 70;
else if (x >= 1)
pitch = 72;
break;
}
{
if (x < -1)
pitch = 67;
else if ((x >= -1) && (x < -0.5))
pitch = 63;
else if ((x >= -0.5) && (x < 0.5))
pitch = 60;
else if ((x >= 0.5) && (x < 1))
pitch = 65;
else if (x >= 1)
pitch = 70;
break;

}
{
if (x < -1)
pitch = 72;
else if ((x >= -1) && (x < -0.81))
pitch = 67;
else if ((x >= -0.81) && (x < -0.31))
pitch = 63;
else if ((x >= -0.31) && (x < 0.31))
pitch = 60;

else if ((x >= 0.31) && (x < 0.81))
pitch = 65;
else if ((x >= 0.81) && (x < 1))
pitch = 70;

else if (x >= 1)
pitch = 75;
break;
}

case ((int)PITCH_CURVE.DUAL_12):
{
switch (cONTROLLER)
{
case (CONTROLER.NUNCHUCK):
{
if (x < -0.8)
pitch = 68;
else if ((x >= -0.8) && (x < -0.5))
pitch = 65;
else if ((x >= -0.5) && (x < 0))
pitch = 63;
else if ((x >= 0) && (x < 0.5))
pitch = 62;
else if ((x >= 0.5) && (x < 0.8))
pitch = 66;
else if (x >= 0.8)
pitch = 71;
break;
}
case(CONTROLER.WIIMOTE):
{
if (x < -0.8)
pitch = 70;
else if ((x >= -0.8) && (x < -0.5))
pitch = 64;
else if ((x >= -0.5) && (x < 0))
pitch = 61;
else if ((x >= 0) && (x < 0.5))
pitch = 60;
else if ((x >= 0.5) && (x < 0.8))
pitch = 77;
else if (x >= 0.8)
pitch = 79;

break;
}

}

break;
}
case ((int)PITCH_CURVE.TWO_TIER):
{//Lower Tier
if (y < 0.15)
{
if (x < -1)
pitch = 60;
else if ((x >= -1) && (x < -0.7))
pitch = 63;
else if ((x >= -0.7) && (x < 0))
pitch = 65;
else if ((x >= 0) && (x < 0.7))
pitch = 67;
else if ((x >= 0.7) && (x < 1))
pitch = 68;
else if (x >= 1)
pitch = 70;

}
//Upper Tier
else if (y > 0.3)
{
x = calibrate(x, y);
if (x < -1)
pitch = 72;
else if ((x >= -1) && (x < -0.7))
pitch = 75;
else if ((x >= -0.7) && (x < 0))
pitch = 77;
else if ((x >= 0) && (x < 0.7))
pitch = 79;
else if ((x >= 0.7) && (x < 1))
pitch = 80;
else if (x >= 1)
pitch = 82;
}
break;
}
}
if ((cONTROLLER == CONTROLER.NUNCHUCK) && (pitchCurve != (int)PITCH_CURVE.DUAL_12))
pitch = pitch - 12;

Console.WriteLine("x:{0}, P:{1}", x, pitch);
return pitch;
}
public int getPitch()
{
return pitch;
}
{
return x;
}
}
}
using System;
using System.Linq;
using System.Text;
namespace TheSphere
{
//Z has been flipped so you are looking for a +ive peak then trough in a very short amount of time
//Now using 'hoop' method to ensure snug tracking
//Reaches peak in around 3-4 cycles (from stationary)
//Therefor must jump through 3 up hoops to gurantee peak
//Class for finding peaks in accelerometer and giving back a 'go' signal
//Does not filter noise
//Does not do angle (for the maths class - experimentation needed)
//Don't have any timers not, use 3 counts.
//Need one more up to down detect to make sure we don't trigger twice.
class AccDetect
{
float count, ocount, precount, zo;
public double hoopHeight;
int fire, velocity, velocityCurve;
bool upMode, downMode, up2Mode, firstrun;
int upCount, up2Count;
//Constructor
public AccDetect()
{

hoopHeight = 0.1;
firstrun = true;
reset();
velocityCurve=0;

}

public int peakDetect(float z)
{
fire = 0;
if (firstrun)
{
zo = z;
firstrun = false;
return (0);
}

//Look for upward trend if in steady state
if((!upMode)&&(!downMode)&&(!up2Mode))
{
fire = 0;
//If this looks like an upward trend, start the hoop jumping, if we can jump through enough hoops
//then we're on a peak
if ((z > zo) && (upCount==0))
{
precount = count;
ocount = count;
upCount++;
zo = z;
return (0);
}
//If we have started to look for an upward trend, keep counting until we have jumped through enough hoops.
//If we don't jump through enough hoops then just reset(); and return(0);
else if ((upCount > 0) && (upCount < 4))
{

//We still going up?
if (count > (ocount + hoopHeight))
{
upCount++;
ocount = count;
zo = z;
// Console.WriteLine("UC: {0}", upCount);
return (0);
}
//If not - TOO SLOW! reset();
else
{
reset();
return (0);
}

}

//If we've made it through 3 hoops then we're in a peak.
//Set up mode
else if (upCount == 4)
{
// Console.WriteLine("upZ:{0}",z);
zo = z;
upMode = true;
upCount = 0;
return (0);
}

//If we're not finding anything good then reset and try again
else
{
reset();
zo = z;
return (0);
}

}// end IF nothing yet
//We're in an upward spike, so we'll be going down very soon
//Just start looking for the down.
if (upMode)
{
//If we're still going up, carry on
if (z > zo)
{
zo = z;
return (0);
}
//If we are the downward peak, set down mode
if (z < zo)
{
velocity = countToVelocity(count);
downMode = true;
upMode = false;
zo = z;
return (0);
}

}

//Got to look for the bottom now - just wait till zo<z
if (downMode)
{

//If zo>z we're still going down, keep cycling
if (zo > z)
{
zo = z;
return (0);
}
//We've reched the bottom, send fire note
//To make sure we're not sending ghost notes,
//wait 2 cycles so as not to get the upward spike
if (zo < z)
{
// Console.WriteLine("Dn z:{0}, count:{1}", z, count);
up2Mode = true;
downMode = false;
if(count<0)
fire = 1;

return (1);
}

}
//Having an up2mode to wait to finish is a bit much really
//For now, just wait a cycle, should calm it down a bit
if (up2Mode)
{
if(up2Count< 0)
{
up2Count++;
return (0);
}

else
{
reset();
}

}
//Set up zo for next run
zo = z;
return (0);
}
private int countToVelocity(float countIn)
{
double fVelocity;
countIn = Math.Abs(countIn);

switch (velocityCurve)
{

case ((int)VELOCITY_CURVE.LINEAR):
{
break;
}

case ((int)VELOCITY_CURVE.SQUARED):
{
countIn=(countIn/11)*(countIn/11);
break;
}
case ((int)VELOCITY_CURVE.S_CURVE):
{
countIn = (float)((Math.Pow(((countIn / 2) - 30), 3))/50000)*127+64;
break;
}
}
fVelocity = ((countIn / 6) * 127);
velocity = (int)fVelocity;
if (velocity > 127)
velocity = 127;
if (velocity < 0)
velocity = 0;
Console.WriteLine("Count: {0}, fVel: {1}, iVel:{2}", countIn, fVelocity, velocity);
return (velocity);
}
private void reset()
{
count = 0;
ocount = 0;
precount = 0;
upMode = false;
downMode = false;
up2Mode = false;
fire = 0;
upCount = 0;
up2Count = 0;
velocity = 0;
}
//Need a query method to send data to
public int getFire()
{
return fire;
}
public void setHoopHeight(double height)
{
hoopHeight = height;
}
public int getVelocity(int velocityCurve)
{

return (velocity);
}
{
get{return (velocityCurve);}
set{velocityCurve=value;}
}
}
}
using System;
using System.Linq;
using System.Text;
namespace TheSphere
{

{

//Constructor
{

{
throw System.Exception("New Devices Found!");
}
else
{
}
}

{
builder.Build();

}
{
builder.Data2 = 0;
builder.Build();
}
{
builder.Command = ChannelCommand.Controller;
builder.Data1 = cc;
builder.Build();
}
/// <summary>
/// </summary>
{
set
{
}
}

}
}
using System;
using System.Linq;
using System.Text;
using System.IO;
using WiimoteLib;
namespace TheSphere
{
{
int wiiChannel, nunchuckChannel;
int outputDeviceId;
int wiiPitch, nunchuckPitch, wiiTranspose, nunchuckTranspose;
int velocityCurve, pitchCurve;
AccDetect wiimoteAccDetect = new AccDetect();
AccDetect nunchuckAccDetect = new AccDetect();
PitchDetect wiiPitchDetect = new PitchDetect();
PitchDetect nunchuckPitchDetect = new PitchDetect();
Timer wiiNoteOffTimer = new Timer();
Timer nunchuckNoteOffTimer = new Timer();

bool firstrun;
//Constructor
{
wiiChannel = 0;
nunchuckChannel = 0;
outputDeviceId = 2;
firstrun = true;
wiiReset(wm);
//Set up note off timers
wiiNoteOffTimer.Elapsed += new ElapsedEventHandler(wiiNoteOffTimerOut);
nunchuckNoteOffTimer.Elapsed += new ElapsedEventHandler(nunchuckNoteOffTimerOut);
wiiNoteOffTimer.Interval = 50;
nunchuckNoteOffTimer.Interval = 50;
wiiNoteOffTimer.AutoReset = false;
nunchuckNoteOffTimer.AutoReset = false;
WiiTranspose=0;
NunchuckTranspose=0;
pitchCurve = 0;
velocityCurve = 0;
}
{

nx = -ws.NunchukState.AccelState.Values.X;
{
firstrun = false;

}

if ((ws.ButtonState.A) != (ows.ButtonState.A))
{
if (ws.ButtonState.A == true)
{

}
else
{

}
}

if((ws.NunchukState.Z) != (ows.NunchukState.Z))
{
if (ws.NunchukState.Z == true)
{
Console.WriteLine("NUNCHUCK DOWN OCT");
nunchuckTranspose = nunchuckTranspose - 12;
}
else
{
Console.WriteLine("Z Off");
}
}

if ((ws.NunchukState.C) != (ows.NunchukState.C))
{
if (ws.NunchukState.C == true)
{
Console.WriteLine("NUNCHUCK UP OCT");
nunchuckTranspose = nunchuckTranspose + 12;
}
else
{
Console.WriteLine("C Off");
}
}
if (((ws.NunchukState.Joystick.X) != (ows.NunchukState.Joystick.X))&&ws.ButtonState.B)
{
double joyx = ws.NunchukState.Joystick.X;
int joyMidiOut = (int)((joyx + 0.5) * 127);
joyMidiOut = 127;
if (joyMidiOut < 0)
joyMidiOut = 0;

}
if (((ws.NunchukState.Joystick.Y) != (ows.NunchukState.Joystick.Y)) && ws.ButtonState.B)
{
double joyy = ws.NunchukState.Joystick.Y;
int joyMidiOut = (int)((joyy + 0.5) * 127);
joyMidiOut = 127;
if (joyMidiOut < 0)
joyMidiOut = 0;
}
//Up
if (ws.ButtonState.Up != ows.ButtonState.Up)
{
if (ws.ButtonState.Up == true)
{
}
else
{
}
}
if ((ws.ButtonState.Down) != (ows.ButtonState.Down))
{
if (ws.ButtonState.Down == true)
{
}
else
{
}
}
//Transpose
if (((ws.ButtonState.Minus) != (ows.ButtonState.Minus))||((ws.ButtonState.Plus) != (ows.ButtonState.Plus)))
{
{
wiiTranspose--;
nunchuckTranspose--;
//OppButtonPress(this, OPP_CODE.TRANS_DOWN);
Console.WriteLine("WT:{0} NT:{1}", wiiTranspose, nunchuckTranspose);
}
{
wiiTranspose++;
nunchuckTranspose++;
// OppButtonPress(this, OPP_CODE.TRANS_UP);
}
}
//send z axis data for sample by sample analysis - it will report back when WMD should send a fire note
//We need to up scale the nunchuck as it only goes to +- 3g
nz = (nz / 3) * 5;
wiimoteAccDetect.peakDetect(wz);
nunchuckAccDetect.peakDetect(nz);
//send x and y data for sample by sample analysis to LPF the x axis
wx = wiiPitchDetect.butterworth(wx);
nx = nunchuckPitchDetect.butterworth(nx);
wy = wiiPitchDetect.yButterworth(wy);
ny = wiiPitchDetect.yButterworth(ny);
// tw.WriteLine("{0} {1}",ws.AccelState.Values.X , wx);

if (wiimoteAccDetect.getFire() == 1)
{
if (wiiPitch < 0)
wiiPitch = 0;
if (wiiPitch > 127)
wiiPitch = 127;
wiiPitch = inrange(wiiPitch);
wiiPitch = wiiPitchDetect.pitchDetect(wx, wy, wz, pitchCurve, CONTROLER.WIIMOTE);
this.midiOut.sendNote(wiiChannel, inrange(wiiPitch+wiiTranspose), wiimoteAccDetect.getVelocity(velocityCurve));
wiiNoteOffTimer.Enabled = true;

}
if (nunchuckAccDetect.getFire() == 1)
{
nunchuckPitch = inrange(nunchuckPitch);
nunchuckPitch = nunchuckPitchDetect.pitchDetect(nx, ny, nz, pitchCurve, CONTROLER.NUNCHUCK);

this.midiOut.sendNote(nunchuckChannel, inrange(nunchuckPitch+nunchuckTranspose), nunchuckAccDetect.getVelocity(velocityCurve));
nunchuckNoteOffTimer.Enabled=true;
//this.midiOut.sendNoteOff(nunchuckChannel, nunchuckPitch);
}
// tw.Close();


}
{
if (args.Inserted)
else
}
//Timers for turning the notes off
private void wiiNoteOffTimerOut(object source, ElapsedEventArgs e)
{
midiOut.sendNoteOff(wiiChannel, inrange(wiiPitch+wiiTranspose));
}

private void nunchuckNoteOffTimerOut(object source, ElapsedEventArgs e)
{
midiOut.sendNoteOff(nunchuckChannel, inrange(nunchuckPitch+nunchuckTranspose));
}
{
}
{
set
{
outputDeviceId = value;
}
}
//Sets and gets
public int WiiChannel
{
get { return wiiChannel; }
set
{
wiiChannel = value;
}
}
public int NunchuckChannel
{
get { return nunchuckChannel; }
set
{
nunchuckChannel = value;
}
}
public int WiiTranspose
{
get {return wiiTranspose;}
set
{
wiiTranspose=value;
}
}
public int NunchuckTranspose
{
get { return nunchuckTranspose; }
set { nunchuckTranspose = value; }
}
{
get { return velocityCurve; }
set { velocityCurve = value;
wiimoteAccDetect.VelocityCurve = value;
}
}
public int PitchCurve
{
get { return pitchCurve; }
set { pitchCurve = value; }
}
{

//Reset events
wm.Disconnect();
wm.Dispose();
wm = _wm;

catch
{
}

else
this.SetLEDs(true, false, false, true);

}
{
}
int inrange(int i)
{
if (i < 0)
i = 0;
if (i > 127)
i = 127;
return i;
}
}
}
using System;
namespace TheSphere
{
enum VELOCITY_CURVE
{LINEAR = 1,
SQUARED,
S_CURVE
}
enum PITCH_CURVE
{
//Min Pent
NUMERICAL =1,
ANGULAR,
FLICK,
THUMB_FLICK_5,
THUMB_FLICK_7,
//Other
DUAL_12,
TWO_TIER,
TWO_TIER_2CHAN,
}
enum CONTROLER
{WIIMOTE, NUNCHUCK
}

public enum OPP_CODE {TRANS_UP, TRANS_DOWN}

public struct OldWmState
{
public bool A;
public bool B;
public bool UP;
public bool DOWN;
public bool LEFT;
public bool RIGHT;
public bool MINUS;
public bool PLUS;
public bool HOME;
public bool ONE;
public bool TWO;
public double accX;
public double accY;
public double accZ;
}
}
176

PAA-/0$N " @'+,1- G'0- #$%&$/*
2. The Cello
using System;
using System.Linq;
using System.Text;
using System.IO;
using WiimoteLib;
namespace Cello
{
{
MotionDetect motionDetect = new MotionDetect();
PitchDetect pitchDetect = new PitchDetect();
Timer startTimer = new Timer();
int outputDeviceId, joystickFunction, pitchMovement, sampleSelect, sampleSelect2;
int basePitch, pitch, _pitch, channel, transpose, tuning, xvel, zvel;
const double PI = 3.14159265358979323846;

bool firstrun, startBool;

//Constructor
{
outputDeviceId = 0;
joystickFunction = (int)JOYSTICK_MODE.POLY;
firstrun = true;
startBool = false;
transpose = 0;
wiiReset(wm);
Range = 7;
Tuning = 7;
sampleSelect = 0;
sampleSelect2 = 0;
startTimer.Elapsed += new ElapsedEventHandler(startTimerUp);
startTimer.Interval = 2000;
startTimer.Start();
}
public void startTimerUp(object source, ElapsedEventArgs e)
{
this.startBool = true;
}
{
nx = ws.NunchukState.AccelState.Values.X;

{
firstrun = false;

}
//Send x and zto filters to determin angle of nunchuck
pitchDetect.nunchuckXFilter(nx);
pitchDetect.nunchuckZFilter(nz);

//send all accel for filtering
motionDetect.motionProcessing(wx, wy, wz);
motionDetect.nyProc(ny);

if(startBool)
{
int t;
t=inrange((int)(((Math.Asin(nx)+(PI/2))/PI)*127));
midiOut.sendControlChange(channel, (int)MIDI_CC.NUN_TILT, t);
}

//****B Note ON+OFF*********************************************************************************************************

//Send MIDI note on when B is just pressed
if ((ws.ButtonState.B) != (ows.ButtonState.B))
{

//Get velocity
xvel = (int)(motionDetect.XExcitation * 127 / 10);
zvel = (int)(motionDetect.ZExcitation * 127 / 10);
{
//Note on, first look for the actually note we're sending by asking pitchDetect
basePitch = pitchDetect.angleToNote();
//Now find out the state of the nunchuck buttons to determine 'string'
pitch = checkCZ(ws.NunchukState.C, ws.NunchukState.Z, basePitch);

pitch=inrange(pitch+60+transpose);
_pitch = pitch;
//Send Note Ons
midiOut.sendNote(channel, pitch, 127);

}

//B released, Send Note off
else
{
midiOut.sendNoteOff(channel, pitch);
}

}
//*********B Note manipulation******************************************************************************************************
{
xvel = inrange(xvel);
zvel = inrange(zvel);

//If we're in continuous mode, send PB
// if (pitchMovement == (int)PITCH_MOVEMENT.CONT)
if(ws.ButtonState.A)
{
midiOut.sendPitchBend(channel, pitchDetect.angleToPitchbend());
//Find out if we have changed 'string'
if ((ws.NunchukState.C != ows.NunchukState.C) || (ws.NunchukState.Z != ows.NunchukState.Z))
{
pitch = checkCZ(ws.NunchukState.C, ws.NunchukState.Z, basePitch) + 60 + transpose;
}
}
//IF just gone to discrete mode form pitch bend, will need to reset pb to stop whole
// thing from being out of tune
if (ows.ButtonState.A && !ws.ButtonState.A)
midiOut.sendPitchBend(channel, 8191);
//If we're in discrete mode, check pitch again, see if our note has changed. If so, send note off/on
//if (pitchMovement == (int)PITCH_MOVEMENT.DIS)
if(!ws.ButtonState.A)
{
if (_pitch != pitch)
{
midiOut.sendNoteOff(channel, _pitch);
_pitch = pitch;
}
}

//Send Vel
midiOut.sendControlChange(channel, (int)MIDI_CC.XVEL, xvel);
midiOut.sendControlChange(channel, (int)MIDI_CC.ZVEL, zvel);

//OSC 1
//Harmonics for additive synth
midiOut.sendControlChange(channel, (int)MIDI_CC.HAR1, inrange((int)(Math.Sqrt((Math.Pow(Math.E, ((xvel - 6) / 8))) / 10)) / 2));
midiOut.sendControlChange(channel, (int)MIDI_CC.HAR2, zvel);
midiOut.sendControlChange(channel, (int)MIDI_CC.HAR4, inrange((1 - zvel *zvel)-1));
if (zvel > 0)
midiOut.sendControlChange(channel, (int)MIDI_CC.HAR5, inrange((int)0.1 * zvel));
if (zvel > 120)
if (zvel < 120)
midiOut.sendControlChange(channel, (int)MIDI_CC.HAR6, 0);

//Set rumble
if ((xvel > 100) || (zvel > 100))
{
wm.SetRumble(true);
}
else { wm.SetRumble(false); }
//Send Joy
midiOut.sendControlChange(channel, (int)MIDI_CC.JOYLEV, joyToLevel(ws.NunchukState.Joystick.X, ws.NunchukState.Joystick.Y));
midiOut.sendControlChange(channel, (int)MIDI_CC.JOYPB, joyToPB(ws.NunchukState.Joystick.X, ws.NunchukState.Joystick.Y));
//Send Y
midiOut.sendControlChange(channel, (int)MIDI_CC.Y_PITCH, inrange(motionDetect.yMidiPitch()));
//Send ny
midiOut.sendControlChange(channel, (int)MIDI_CC.NUNY, inrange(motionDetect.nyMidiPitch()));

}

//*****Transpose***************************************************************************************************************
if (((ws.ButtonState.Minus) != (ows.ButtonState.Minus)) || ((ws.ButtonState.Plus) != (ows.ButtonState.Plus)))
{
transpose--;
{
transpose++;
}
}
//If Up, send prev sample select
if ((ws.ButtonState.Up != ows.ButtonState.Up) && (ws.ButtonState.Up))
{
sampleSelect--;
if (sampleSelect < 0)
sampleSelect = 0;
midiOut.sendControlChange(channel, (int)MIDI_CC.SAMPLE_SELECT, sampleSelect);
}
if ((ws.ButtonState.Down != ows.ButtonState.Down) && (ws.ButtonState.Down))
{
sampleSelect++;
if(sampleSelect>127)
sampleSelect=127;
}
if ((ws.ButtonState.Left != ows.ButtonState.Left) && (ws.ButtonState.Left))
{
sampleSelect2--;
if (sampleSelect2 < 0)
sampleSelect2 = 0;
midiOut.sendControlChange(channel, (int)MIDI_CC.SAMPLE_SELECT2, sampleSelect2);
}
if ((ws.ButtonState.Right != ows.ButtonState.Right) && (ws.ButtonState.Right))
{
sampleSelect2++;
if (sampleSelect2 > 127)
sampleSelect2 = 127;
}
//If not B turn off rumble
if ((!ws.ButtonState.B)&&(ws.Rumble))
wm.SetRumble(false);


}

{
int i;
i = (int)((j + 0.5) * 127);

i = inrange(i);

return i;
}

{
if (args.Inserted)
else
}

{
}
{
set {outputDeviceId = value;
}
}
//Sets and gets
public int Channel
{
get { return channel; }
set { channel = value;}
}
public int Transpose
{
get { return transpose; }
set {transpose = value; }
}
public float Resistivity
{
get { return motionDetect.Resistivity; }
set { motionDetect.Resistivity = value; }
}
public int Range
{
get { return pitchDetect.Range; }
set { pitchDetect.Range = value; }
}
public int Tuning
{
get { return tuning; }
set { tuning = value; }
}
public int PitchMovement
{
get { return pitchMovement; }
set { pitchMovement = value; }
}
public int JoystickFunction
{set{joystickFunction = value;}
get { return joystickFunction; }
}

{
//Reset events
wm.Disconnect();
wm.Dispose();
wm = _wm;
catch
{
}
else
this.SetLEDs(false, true, true, false);
}
//
int checkCZ(bool c, bool z, int b)
{
int p;

if (c && z)
p = b + (2 * tuning);
else if (c)
p = b + (3 * tuning);
else if (z)
p = b + tuning;
else p = b;
return p;
}
int joyToPB(float x, float y)
{

double theta = Math.Abs(Math.Atan(y/x));
int pb =0;
//right quadrant
if(x>=0)
{
//Top right
if(y>=0)
theta=(2*PI)-theta;
//bottom right
else { theta = Math.Abs(theta); }
}
//left quadrant
if(x<0)
{
//top left
if(y>=0)
theta=PI+theta;
else{theta=PI-theta;}
}

//Shift theta round to make the angle to PB translation easier
theta=theta+(PI/2);
if(theta>(2*PI))
theta=theta-(2*PI);
// Console.WriteLine((theta/PI)*180);
//base to min 3rd - error in wiimote, need to compensate
if ((theta>=0)&&(theta<(PI/4)))
{
pb = (int)((theta/(PI/4)) *31.75);
}

//min 3rs to 4th
else if((theta>=(PI/4))&&(theta<(3*PI/4)))
{
theta = theta - (PI / 4);

pb=(int)(31.75+ ((theta/(PI/2))*21.16));
}
//4th to 6th
else if((theta>=(3*PI/4))&&(theta<(5*PI/4)))
{
theta = theta - (3 * PI / 4);
pb = (int)(52+((theta / (PI/2)) * 42.3));
}
//6th to oct
else if((theta>=(5*PI/4))&&(theta<2*PI))
{
pb = (int)(95+((theta / (PI/2)) * 21.16));
}
pb = inrange(pb);

return pb;
}
{
return inrange((int)((Math.Sqrt(x * x + y * y)) * 254));
}
int inrange(int i)
{
if (i < 0)
i = 0;
if (i > 127)
i = 127;
return i;
}
{
}

}
}
using System;
using System.Linq;
using System.Text;
namespace Cello
{
class PitchDetect
{
int pitch, pitchBend;
int range;
float z;
double noteOnXValue;
const double PI = 3.14159265358979323846;
LowPassFilt zPitchFilt = new LowPassFilt();
//Filter values
double[] x = new double[3] { 0, 0, 0 };
double[] y = new double[3] { 0, 0, 0 };
public float nunchuckXFilter(float x2f)
{
double gain;
float y2f;
for (int i = 0; i < 3; i++)
{
if ((x[i] > 50) || (x[i] < -50))
x[i] = 0;
if ((y[i] > 50) || (y[i] < -50))
y[i] = 0;
}
//10Hz gain
//gain = 14.82463775;
//5Hz gain
gain = 49.79245121;
//3Hz gain
// gain = 127.8738361;
x[0] = x[1];
x[1] = x[2];
x[2] = (x2d / gain);
y[0] = y[1];
y[1] = y[2];
//y[2] = x[0] + 2 * x[1] + x[2] - (0.4128015981 * y[0]) + (1.1429805025 * y[1]);
y[2] = x[0] + 2 * x[1] + x[2] + (-0.6413515381 * y[0]) + (1.5610180758 * y[1]);
// y[2] = x[0] + 2 * x[1] + x[2] + (-0.7660066009 * y[0]) + (1.7347257688 * y[1]);
y2f = (float)y[2];
return y2f;
}
public float nunchuckZFilter(float z2f)
{
z2f = zPitchFilt._2HzLPF(z2f);
z = z2f;
return z2f;
}
public int angleToNote()
{
double ang, angb;

int i =0;
noteOnXValue = y[2];
double noteDiv = PI/range;
if (noteOnXValue > 1)
noteOnXValue = 1;
if (noteOnXValue < -1)
noteOnXValue = -1;
//All angles in rad
ang=Math.Asin(noteOnXValue);
angb = (-(PI / 2) + noteDiv);
while ((ang > angb)||(ang == angb))
{
i++;

angb = (i * noteDiv - (PI / 2) + noteDiv);

}
//Stabilise LH
if (z > -0.1)
{
if(ang<(PI/7))
i = 0;
if (ang > 6 * PI / 7)
i = 6;
}

return i;
}
//Takes the new note post 'note on' and send out the pitch bend.
public int angleToPitchbend()
{
// Console.WriteLine(((y[2] - noteOnXValue)/2) * 8191 + 8191);
int pb;
pb = (int)(((y[2] - noteOnXValue) / 2) * 8191 + 8191);
if(pb<0)
pb=0;
return pb;
}

{
return x;
}
public int Range
{
set { range = value; }
get { return range; }
}
}
}
using System;
using System.Linq;
using System.Text;
namespace Cello
{
class MotionDetect
{

HighPassFilt xHPF = new HighPassFilt();
HighPassFilt zHPF = new HighPassFilt();
LowPassFilt yLPF = new LowPassFilt();
LowPassFilt nyLPF = new LowPassFilt();
const double PI = 3.14159265358979323846;

float resistivity=0.01F;
float _xExcitation=0 , _zExcitation , yfilt, yPitch=0, nyfilt=0;
//Method for dealing with right hand data
public float motionProcessing(float x, float y, float z)
{
float xhp = xHPF._05HzHPF(x);
float zhp = zHPF._05HzHPF(z);
//Rectify both x and z then send to filt to get sensible values
float xExcitation = Math.Abs(xhp);
float zExcitation = Math.Abs(zhp);
_xExcitation = latchAndHold(xExcitation, _xExcitation);
_zExcitation = latchAndHold(zExcitation, _zExcitation);
yfilt= yLPF._2HzLPF(y);

return 0;
}
public float nyProc(float ny)
{
nyfilt = nyLPF._2HzLPF(ny);
return nyfilt;
}

//***/latch and hold**************************************************************************
{
if((_excitation<excitation)||(_excitation==excitation))
{
_excitation=excitation;
return (excitation);
}
//On downward run, trigger hold
else if(_excitation>excitation)
{

_excitation = _excitation - resistivity;

if (_excitation <= 0)
_excitation = 0;

return(_excitation);
}
return (_excitation);
}
public float XExcitation
{get{return _xExcitation;}
set { _xExcitation = value; }
}
public float ZExcitation
{
get { return _zExcitation; }
set { _zExcitation = value; }
}
{
get { return resistivity; }
set { resistivity = value; }
}
public int yMidiPitch()
{

float y2f;
y2f=yfilt;
if (y2f > 1)
y2f = 1;
if (y2f < -1)
y2f = -1;

return (int)(((Math.Asin(y2f) + (PI / 2)) / PI) * 127);
}
public int nyMidiPitch()
{
float ny2f;
ny2f = nyfilt;
if (ny2f > 1)
ny2f = 1;
if (ny2f < -1)
ny2f = -1;

return (int)(((Math.Asin(ny2f) + (PI / 2)) / PI) * 127);
}

}
}
using System;
using System.Linq;
using System.Text;
using Sanford.Threading;
namespace Cello
{
{
//Constructor
{
{
// throw System.Exception("New Devices Found!");
}
else
{
}
}
{
builder.Build();
}
{
builder.Data2 = 0;
builder.Build();

}
{
ChannelMessageBuilder cbuilder = new ChannelMessageBuilder();
cbuilder.Command = ChannelCommand.Controller;
cbuilder.MidiChannel = channel;
cbuilder.Data1 = cc;
cbuilder.Data2 = value;
cbuilder.Build();
outputDevice.Send(cbuilder.Result);
}
//Need a second one as too slow otherwise
public void sendControlChange2(int channel, int cc, int value)
{
ChannelMessageBuilder c2builder = new ChannelMessageBuilder();
c2builder.Command = ChannelCommand.Controller;
c2builder.MidiChannel = channel;
c2builder.Data1 = cc;
c2builder.Data2 = value;
c2builder.Build();
outputDevice.Send(c2builder.Result);
}
public void sendChannelPressure(int channel, int pitch, int value)
{
builder.Command = ChannelCommand.ChannelPressure;
//builder.Data2 = value;
builder.Build();
}
public void sendPitchBend(int channel, int pitchbend)
{
byte LSB, MSB;
int x;

ChannelMessageBuilder bbuilder = new ChannelMessageBuilder();
bbuilder.Command = ChannelCommand.PitchWheel;
bbuilder.MidiChannel = channel;

LSB = (byte)(pitchbend & 0x007F);
x = (pitchbend & 0x3F80);
x = x >> 7;
MSB = (byte)x;
bbuilder.Data1 = LSB;
bbuilder.Data2 = MSB;
bbuilder.Build();
outputDevice.Send(bbuilder.Result);
}
/// <summary>
/// </summary>
{
set
{
}
}
}
}
using System;
using System.Linq;
using System.Text;
namespace Cello
{
class LowPassFilt
{
double[] xlp = new double[3] { 0, 0, 0 };
double[] ylp = new double[3] { 0, 0, 0 };
public float _1HzLPF(float x2)
{
double lpgain = 1058.546241;
xlp[0] = xlp[1];
xlp[1] = xlp[2];
xlp[2]=(double)x2 / lpgain;
//Make sure filter doesn't run away with itself at start up
if ((xlp[2] > 70) || (xlp[2] < -70))
xlp[2] = 0;
ylp[0] = ylp[1];
ylp[1] = ylp[2];
ylp[2] =(xlp[0] + xlp[2]) + 2 * xlp[1] + (-0.9149758348 * ylp[0]) + (1.9111970674 * ylp[1]);
return (float)ylp[2];
}
{
xlp[0] = xlp[1];
xlp[1] = xlp[2];
xlp[2] = (double)x2 / lpgain;
if ((xlp[2] > 110) || (xlp[2] < -110))
xlp[2] = 0;
ylp[0] = ylp[1];
ylp[1] = ylp[2];
ylp[2] = (xlp[0] + xlp[2]) + 2 * xlp[1] + (-0.8371816513 * ylp[0]) + (1.8226949252 * ylp[1]);
}

}
}
using System;
using System.Linq;
using System.Text;
namespace Cello
{
class HighPassFilt
{
double[] xhp = new double[3] { 0, 0, 0 };
double[] yhp = new double[3] { 0, 0, 0 };

//Constructor
public HighPassFilt()
{
}
public float _05HzHPF(float x2)
{
//HP filt********************************************************************

double hpgain = 1.022463023;
// double lpgain = 5458.037903;
xhp[0] = xhp[1];
xhp[1] = xhp[2];
xhp[2] = (double)x2 / hpgain;
if ((xhp[2] > 70) || (xhp[2] < -70))
xhp[2] = 0;
yhp[0] = yhp[1];
yhp[1] = yhp[2];

// Rec Expression for 0.5 Hz HPF
yhp[2] = (xhp[0] + xhp[2]) - 2 * xhp[1] + (-0.9565436765 * yhp[0]) + (1.9555782403 * yhp[1]);
return ((float)yhp[2]);
}
}
}
/* //HP filt********************************************************************
//Gain for 0.4 - 1 Hz BPF
// double gain = 2886.793836;
//Gain for 0.5 Hz HPF
// double lpgain = 5458.037903;
xhp[0] = xhp[1];
xhp[1] = xhp[2];
xhp[2] = (double)recxz / hpgain;
if ((xhp[2] > 70) || (xhp[2] < -70))
xhp[2] = 0;
yhp[0] = yhp[1];
yhp[1] = yhp[2];
//Recurrence expression 0.4 - 1Hz BPF
// yhp[4] = (xhp[0] + xhp[4]) - 2 * xhp[2]
// + ( -0.9480817061 * yhp[0]) + ( 3.8398267915 * yhp[1])
// + ( -5.8353289077 * yhp[2]) + ( 3.9435813943 * yhp[3]);
//HP Filt end**************************************************************************
//LP Filt *****************************************************************************
/*
xlp[0] = xlp[1];
xlp[1] = xlp[2];
xlp[2] = xlp[3];
xlp[3] = xlp[4];
xlp[4] = yhp[2] / lpgain;
ylp[0] = ylp[1];
ylp[1] = ylp[2];
ylp[2] = ylp[3];

ylp[4] = (xlp[0] + xlp[4]) + 4 * (xlp[1] + xlp[3]) + 6 * xlp[2]
+ ( -0.5174781998 * ylp[0]) + ( 2.4093428566 * ylp[1])
+ ( -4.2388639509 * ylp[2]) + ( 3.3440678377 * ylp[3]);

return((float)ylp[4]);
*/
/* tEMP_FILT_OUT = (float)Math.Abs(yhp[2]);
*/
using System;
using System.Data;
using System.Linq;
using System.Text;
using WiimoteLib;
namespace Cello
{
{
WiiMidiDelegate wiiMidiDelegate;
public Form1()
{
populateMidiChannelCombo();

}
{
//Instantiate new WiiMidiDelegate on Loadq
wiiMidiDelegate = new WiiMidiDelegate();
wiiMidiDelegate.Resistivity = resistivityTrackbar.Value;
wiiMidiDelegate.OutputDeviceId = outDeviceListCombo.SelectedIndex;
wiiMidiDelegate.Range = 7;
rangeTrackbar.Value = 7;
rangeLabel.Text = (rangeTrackbar.Value.ToString());

}
{

int i;
catch
{
Console.WriteLine("No Wiimotes");
Application.Exit();
noWiimotesLabel.Visible = true;
}
{
}
}
{
{
{
}
}
}

private void populateMidiChannelCombo()
{
midiChannelCombo.Items.Clear();
for (int i = 0; i < 15; i++)
{
midiChannelCombo.Items.Add(i);
}
}

{
wiiMidiDelegate.wiiReset(wc[wiimoteListCombo.SelectedIndex]);
}
private void midiChannelCombo_SelectedIndexChanged(object sender, EventArgs e)
{
wiiMidiDelegate.Channel = midiChannelCombo.SelectedIndex;
System.Console.WriteLine("Wiimote Chanel: {0}", wiiMidiDelegate.Channel);
}
private void resistivityTrackbar_Scroll(object sender, EventArgs e)
{
wiiMidiDelegate.Resistivity = (float)resistivityTrackbar.Value/50;
Console.WriteLine(wiiMidiDelegate.Resistivity);
}
private void outDeviceListCombo_SelectedIndexChanged(object sender, EventArgs e)
{
System.Console.WriteLine("Output Device Change: {0}", wiiMidiDelegate.OutputDeviceId);
}
private void rangeTrackbar_Scroll(object sender, EventArgs e)
{
wiiMidiDelegate.Range = rangeTrackbar.Value;
Console.WriteLine("Range: {0}", rangeTrackbar.Value);
rangeLabel.Text = ( rangeTrackbar.Value.ToString());
}

}
}
public enum MIDI_CC
{
XVEL =1,
ZVEL=2,
JOYX =3,
JOYY=4,
JOYPB=5,
JOYLEV=6,
//Additive stuff
HAR1 =7,
HAR2=8,
HAR3=9,
HAR4=10,
HAR5=11,
HAR6=12,
Y_PITCH = 13,
SAMPLE_SELECT = 14,

NUN_TILT =15,
SAMPLE_SELECT2 = 16,
NUNY = 17
}
public enum PITCH_MOVEMENT
{
CONT,DIS,AIDED
}
public enum JOYSTICK_MODE
{
POLY,
CCS
}
using System;
using System.Data;
using System.Linq;
using System.Text;
using WiimoteLib;
namespace Cello
{
{
WiiMidiDelegate wiiMidiDelegate;
public Form1()
{
populateMidiChannelCombo();

}
{
//Instantiate new WiiMidiDelegate on Loadq
wiiMidiDelegate = new WiiMidiDelegate();
wiiMidiDelegate.Resistivity = resistivityTrackbar.Value;
wiiMidiDelegate.Range = 7;
rangeTrackbar.Value = 7;
rangeLabel.Text = (rangeTrackbar.Value.ToString());

}
{

int i;
catch
{
Console.WriteLine("No Wiimotes");
Application.Exit();
noWiimotesLabel.Visible = true;
}
{
}
}
{
{
{
}
}
}

private void populateMidiChannelCombo()
{
midiChannelCombo.Items.Clear();
for (int i = 0; i < 15; i++)
{
midiChannelCombo.Items.Add(i);
}
}

{
wiiMidiDelegate.wiiReset(wc[wiimoteListCombo.SelectedIndex]);
}
private void midiChannelCombo_SelectedIndexChanged(object sender, EventArgs e)
{
wiiMidiDelegate.Channel = midiChannelCombo.SelectedIndex;
System.Console.WriteLine("Wiimote Chanel: {0}", wiiMidiDelegate.Channel);
}
private void resistivityTrackbar_Scroll(object sender, EventArgs e)
{
wiiMidiDelegate.Resistivity = (float)resistivityTrackbar.Value/50;
Console.WriteLine(wiiMidiDelegate.Resistivity);
}
private void outDeviceListCombo_SelectedIndexChanged(object sender, EventArgs e)
{
System.Console.WriteLine("Output Device Change: {0}", wiiMidiDelegate.OutputDeviceId);
}
private void rangeTrackbar_Scroll(object sender, EventArgs e)
{
wiiMidiDelegate.Range = rangeTrackbar.Value;
Console.WriteLine("Range: {0}", rangeTrackbar.Value);
rangeLabel.Text = ( rangeTrackbar.Value.ToString());
}

}
}
using System;
using System.Linq;
using System.Text;
using System.IO;
using WiimoteLib;
namespace Cello
{
{
MotionDetect motionDetect = new MotionDetect();
PitchDetect pitchDetect = new PitchDetect();
Timer startTimer = new Timer();
int outputDeviceId, joystickFunction, pitchMovement, sampleSelect, sampleSelect2;
int basePitch, pitch, _pitch, channel, transpose, tuning, xvel, zvel;
const double PI = 3.14159265358979323846;

bool firstrun, startBool;

//Constructor
{
outputDeviceId = 0;
joystickFunction = (int)JOYSTICK_MODE.POLY;
firstrun = true;
startBool = false;
transpose = 0;
wiiReset(wm);
Range = 7;
Tuning = 7;
sampleSelect = 0;
sampleSelect2 = 0;
startTimer.Elapsed += new ElapsedEventHandler(startTimerUp);
startTimer.Interval = 2000;
startTimer.Start();
}
public void startTimerUp(object source, ElapsedEventArgs e)
{
this.startBool = true;
}
{
nx = ws.NunchukState.AccelState.Values.X;

{
firstrun = false;

}
//Send x and zto filters to determin angle of nunchuck
pitchDetect.nunchuckXFilter(nx);
pitchDetect.nunchuckZFilter(nz);

//send all accel for filtering
motionDetect.motionProcessing(wx, wy, wz);
motionDetect.nyProc(ny);

if(startBool)
{
int t;
t=inrange((int)(((Math.Asin(nx)+(PI/2))/PI)*127));
midiOut.sendControlChange(channel, (int)MIDI_CC.NUN_TILT, t);
}

//****B Note ON+OFF*********************************************************************************************************

//Send MIDI note on when B is just pressed
if ((ws.ButtonState.B) != (ows.ButtonState.B))
{

//Get velocity
{
//Note on, first look for the actually note we're sending by asking pitchDetect
//Now find out the state of the nunchuck buttons to determine 'string'
pitch = checkCZ(ws.NunchukState.C, ws.NunchukState.Z, basePitch);

pitch=inrange(pitch+60+transpose);
_pitch = pitch;
//Send Note Ons

}

//B released, Send Note off
else
{
}

}
//*********B Note manipulation******************************************************************************************************
{
xvel = inrange(xvel);
zvel = inrange(zvel);

//If we're in continuous mode, send PB
// if (pitchMovement == (int)PITCH_MOVEMENT.CONT)
if(ws.ButtonState.A)
{
midiOut.sendPitchBend(channel, pitchDetect.angleToPitchbend());
//Find out if we have changed 'string'
if ((ws.NunchukState.C != ows.NunchukState.C) || (ws.NunchukState.Z != ows.NunchukState.Z))
{
}
}
//IF just gone to discrete mode form pitch bend, will need to reset pb to stop whole
// thing from being out of tune
if (ows.ButtonState.A && !ws.ButtonState.A)
midiOut.sendPitchBend(channel, 8191);
//If we're in discrete mode, check pitch again, see if our note has changed. If so, send note off/on
//if (pitchMovement == (int)PITCH_MOVEMENT.DIS)
if(!ws.ButtonState.A)
{
if (_pitch != pitch)
{
midiOut.sendNoteOff(channel, _pitch);
_pitch = pitch;
}
}

//Send Vel
midiOut.sendControlChange(channel, (int)MIDI_CC.XVEL, xvel);
midiOut.sendControlChange(channel, (int)MIDI_CC.ZVEL, zvel);

//OSC 1
//Harmonics for additive synth
midiOut.sendControlChange(channel, (int)MIDI_CC.HAR4, inrange((1 - zvel *zvel)-1));
if (zvel > 0)
midiOut.sendControlChange(channel, (int)MIDI_CC.HAR5, inrange((int)0.1 * zvel));
if (zvel > 120)
if (zvel < 120)
midiOut.sendControlChange(channel, (int)MIDI_CC.HAR6, 0);

//Set rumble
if ((xvel > 100) || (zvel > 100))
{
wm.SetRumble(true);
}
else { wm.SetRumble(false); }
//Send Joy
midiOut.sendControlChange(channel, (int)MIDI_CC.JOYLEV, joyToLevel(ws.NunchukState.Joystick.X, ws.NunchukState.Joystick.Y));
midiOut.sendControlChange(channel, (int)MIDI_CC.JOYPB, joyToPB(ws.NunchukState.Joystick.X, ws.NunchukState.Joystick.Y));
//Send Y
midiOut.sendControlChange(channel, (int)MIDI_CC.Y_PITCH, inrange(motionDetect.yMidiPitch()));
//Send ny
midiOut.sendControlChange(channel, (int)MIDI_CC.NUNY, inrange(motionDetect.nyMidiPitch()));

}

//*****Transpose***************************************************************************************************************
if (((ws.ButtonState.Minus) != (ows.ButtonState.Minus)) || ((ws.ButtonState.Plus) != (ows.ButtonState.Plus)))
{
transpose--;
{
transpose++;
}
}
if ((ws.ButtonState.Up != ows.ButtonState.Up) && (ws.ButtonState.Up))
{
sampleSelect--;
if (sampleSelect < 0)
sampleSelect = 0;
}
if ((ws.ButtonState.Down != ows.ButtonState.Down) && (ws.ButtonState.Down))
{
sampleSelect++;
if(sampleSelect>127)
sampleSelect=127;
}
if ((ws.ButtonState.Left != ows.ButtonState.Left) && (ws.ButtonState.Left))
{
sampleSelect2--;
if (sampleSelect2 < 0)
sampleSelect2 = 0;
}
if ((ws.ButtonState.Right != ows.ButtonState.Right) && (ws.ButtonState.Right))
{
sampleSelect2++;
if (sampleSelect2 > 127)
sampleSelect2 = 127;
}
//If not B turn off rumble
if ((!ws.ButtonState.B)&&(ws.Rumble))
wm.SetRumble(false);


}

{
int i;
i = (int)((j + 0.5) * 127);

i = inrange(i);

return i;
}

{
if (args.Inserted)
else
}

{
}
{
set {outputDeviceId = value;
}
}
//Sets and gets
public int Channel
{
get { return channel; }
set { channel = value;}
}
public int Transpose
{
get { return transpose; }
set {transpose = value; }
}
{
get { return motionDetect.Resistivity; }
set { motionDetect.Resistivity = value; }
}
public int Range
{
get { return pitchDetect.Range; }
set { pitchDetect.Range = value; }
}
public int Tuning
{
get { return tuning; }
set { tuning = value; }
}
public int PitchMovement
{
get { return pitchMovement; }
set { pitchMovement = value; }
}
public int JoystickFunction
{set{joystickFunction = value;}
get { return joystickFunction; }
}

{
//Reset events
wm.Disconnect();
wm.Dispose();
wm = _wm;
catch
{
}
else
this.SetLEDs(false, true, true, false);
}
//
int checkCZ(bool c, bool z, int b)
{
int p;

if (c && z)
p = b + (2 * tuning);
else if (c)
p = b + (3 * tuning);
else if (z)
p = b + tuning;
else p = b;
return p;
}
int joyToPB(float x, float y)
{

double theta = Math.Abs(Math.Atan(y/x));
int pb =0;
//right quadrant
if(x>=0)
{
//Top right
if(y>=0)
theta=(2*PI)-theta;
//bottom right
else { theta = Math.Abs(theta); }
}
//left quadrant
if(x<0)
{
//top left
if(y>=0)
theta=PI+theta;
else{theta=PI-theta;}
}

//Shift theta round to make the angle to PB translation easier
theta=theta+(PI/2);
if(theta>(2*PI))
theta=theta-(2*PI);
// Console.WriteLine((theta/PI)*180);
//base to min 3rd - error in wiimote, need to compensate
if ((theta>=0)&&(theta<(PI/4)))
{
pb = (int)((theta/(PI/4)) *31.75);
}

//min 3rs to 4th
else if((theta>=(PI/4))&&(theta<(3*PI/4)))
{
theta = theta - (PI / 4);

pb=(int)(31.75+ ((theta/(PI/2))*21.16));
}
//4th to 6th
else if((theta>=(3*PI/4))&&(theta<(5*PI/4)))
{
pb = (int)(52+((theta / (PI/2)) * 42.3));
}
//6th to oct
else if((theta>=(5*PI/4))&&(theta<2*PI))
{
pb = (int)(95+((theta / (PI/2)) * 21.16));
}
pb = inrange(pb);

return pb;
}
{
return inrange((int)((Math.Sqrt(x * x + y * y)) * 254));
}
int inrange(int i)
{
if (i < 0)
i = 0;
if (i > 127)
i = 127;
return i;
}
{
}

}
}
using System;
using System.Linq;
using System.Text;
namespace Cello
{
class PitchDetect
{
int pitch, pitchBend;
int range;
float z;
double noteOnXValue;
const double PI = 3.14159265358979323846;
LowPassFilt zPitchFilt = new LowPassFilt();
//Filter values
double[] x = new double[3] { 0, 0, 0 };
double[] y = new double[3] { 0, 0, 0 };
public float nunchuckXFilter(float x2f)
{
double gain;
float y2f;
for (int i = 0; i < 3; i++)
{
if ((x[i] > 50) || (x[i] < -50))
x[i] = 0;
if ((y[i] > 50) || (y[i] < -50))
y[i] = 0;
}
//10Hz gain
//gain = 14.82463775;
//5Hz gain
gain = 49.79245121;
//3Hz gain
// gain = 127.8738361;
x[0] = x[1];
x[1] = x[2];
x[2] = (x2d / gain);
y[0] = y[1];
y[1] = y[2];
//y[2] = x[0] + 2 * x[1] + x[2] - (0.4128015981 * y[0]) + (1.1429805025 * y[1]);
y[2] = x[0] + 2 * x[1] + x[2] + (-0.6413515381 * y[0]) + (1.5610180758 * y[1]);
// y[2] = x[0] + 2 * x[1] + x[2] + (-0.7660066009 * y[0]) + (1.7347257688 * y[1]);
y2f = (float)y[2];
return y2f;
}
public float nunchuckZFilter(float z2f)
{
z2f = zPitchFilt._2HzLPF(z2f);
z = z2f;
return z2f;
}
public int angleToNote()
{
double ang, angb;

int i =0;
noteOnXValue = y[2];
double noteDiv = PI/range;
if (noteOnXValue > 1)
noteOnXValue = 1;
if (noteOnXValue < -1)
noteOnXValue = -1;
//All angles in rad
ang=Math.Asin(noteOnXValue);
angb = (-(PI / 2) + noteDiv);
while ((ang > angb)||(ang == angb))
{
i++;

angb = (i * noteDiv - (PI / 2) + noteDiv);

}
//Stabilise LH
if (z > -0.1)
{
if(ang<(PI/7))
i = 0;
if (ang > 6 * PI / 7)
i = 6;
}

return i;
}
//Takes the new note post 'note on' and send out the pitch bend.
public int angleToPitchbend()
{
// Console.WriteLine(((y[2] - noteOnXValue)/2) * 8191 + 8191);
int pb;
pb = (int)(((y[2] - noteOnXValue) / 2) * 8191 + 8191);
if(pb<0)
pb=0;
return pb;
}

{
return x;
}
public int Range
{
set { range = value; }
get { return range; }
}
}
}
using System;
using System.Linq;
using System.Text;
namespace Cello
{
class MotionDetect
{

HighPassFilt xHPF = new HighPassFilt();
HighPassFilt zHPF = new HighPassFilt();
LowPassFilt yLPF = new LowPassFilt();
LowPassFilt nyLPF = new LowPassFilt();
const double PI = 3.14159265358979323846;

float resistivity=0.01F;
float _xExcitation=0 , _zExcitation , yfilt, yPitch=0, nyfilt=0;
//Method for dealing with right hand data
public float motionProcessing(float x, float y, float z)
{
float xhp = xHPF._05HzHPF(x);
float zhp = zHPF._05HzHPF(z);
//Rectify both x and z then send to filt to get sensible values
float xExcitation = Math.Abs(xhp);
float zExcitation = Math.Abs(zhp);
_xExcitation = latchAndHold(xExcitation, _xExcitation);
_zExcitation = latchAndHold(zExcitation, _zExcitation);
yfilt= yLPF._2HzLPF(y);

return 0;
}
public float nyProc(float ny)
{
nyfilt = nyLPF._2HzLPF(ny);
return nyfilt;
}

//***/latch and hold**************************************************************************
{
if((_excitation<excitation)||(_excitation==excitation))
{
_excitation=excitation;
return (excitation);
}
//On downward run, trigger hold
else if(_excitation>excitation)
{

_excitation = _excitation - resistivity;

if (_excitation <= 0)
_excitation = 0;

return(_excitation);
}
return (_excitation);
}
public float XExcitation
{get{return _xExcitation;}
set { _xExcitation = value; }
}
public float ZExcitation
{
get { return _zExcitation; }
set { _zExcitation = value; }
}
{
get { return resistivity; }
set { resistivity = value; }
}
public int yMidiPitch()
{

float y2f;
y2f=yfilt;
if (y2f > 1)
y2f = 1;
if (y2f < -1)
y2f = -1;

return (int)(((Math.Asin(y2f) + (PI / 2)) / PI) * 127);
}
public int nyMidiPitch()
{
float ny2f;
ny2f = nyfilt;
if (ny2f > 1)
ny2f = 1;
if (ny2f < -1)
ny2f = -1;

return (int)(((Math.Asin(ny2f) + (PI / 2)) / PI) * 127);
}

}
}
using System;
using System.Linq;
using System.Text;
namespace Cello
{
class LowPassFilt
{
double[] xlp = new double[3] { 0, 0, 0 };
double[] ylp = new double[3] { 0, 0, 0 };
{
xlp[0] = xlp[1];
xlp[1] = xlp[2];
xlp[2]=(double)x2 / lpgain;
if ((xlp[2] > 70) || (xlp[2] < -70))
xlp[2] = 0;
ylp[0] = ylp[1];
ylp[1] = ylp[2];
ylp[2] =(xlp[0] + xlp[2]) + 2 * xlp[1] + (-0.9149758348 * ylp[0]) + (1.9111970674 * ylp[1]);
}
{
xlp[0] = xlp[1];
xlp[1] = xlp[2];
xlp[2] = (double)x2 / lpgain;
if ((xlp[2] > 110) || (xlp[2] < -110))
xlp[2] = 0;
ylp[0] = ylp[1];
ylp[1] = ylp[2];
ylp[2] = (xlp[0] + xlp[2]) + 2 * xlp[1] + (-0.8371816513 * ylp[0]) + (1.8226949252 * ylp[1]);
}

}
}
using System;
using System.Linq;
using System.Text;
namespace Cello
{
class HighPassFilt
{
double[] xhp = new double[3] { 0, 0, 0 };
double[] yhp = new double[3] { 0, 0, 0 };

//Constructor
public HighPassFilt()
{
}
public float _05HzHPF(float x2)
{
//HP filt********************************************************************

// double lpgain = 5458.037903;
xhp[0] = xhp[1];
xhp[1] = xhp[2];
xhp[2] = (double)x2 / hpgain;
if ((xhp[2] > 70) || (xhp[2] < -70))
xhp[2] = 0;
yhp[0] = yhp[1];
yhp[1] = yhp[2];

}
}
}
/* //HP filt********************************************************************
//Gain for 0.4 - 1 Hz BPF
// double gain = 2886.793836;
//Gain for 0.5 Hz HPF
// double lpgain = 5458.037903;
xhp[0] = xhp[1];
xhp[1] = xhp[2];
xhp[2] = (double)recxz / hpgain;
if ((xhp[2] > 70) || (xhp[2] < -70))
xhp[2] = 0;
yhp[0] = yhp[1];
yhp[1] = yhp[2];
//Recurrence expression 0.4 - 1Hz BPF
// yhp[4] = (xhp[0] + xhp[4]) - 2 * xhp[2]
// + ( -0.9480817061 * yhp[0]) + ( 3.8398267915 * yhp[1])
// + ( -5.8353289077 * yhp[2]) + ( 3.9435813943 * yhp[3]);
public enum MIDI_CC
{
XVEL =1,
ZVEL=2,
JOYX =3,
JOYY=4,
JOYPB=5,
JOYLEV=6,
//Additive stuff
HAR1 =7,
HAR2=8,
HAR3=9,
HAR4=10,
HAR5=11,
HAR6=12,
Y_PITCH = 13,
SAMPLE_SELECT = 14,

NUN_TILT =15,
SAMPLE_SELECT2 = 16,
NUNY = 17
}
public enum PITCH_MOVEMENT
{
CONT,DIS,AIDED
}
public enum JOYSTICK_MODE
{
POLY,
CCS
}
//HP Filt end**************************************************************************
//LP Filt *****************************************************************************
/*
xlp[0] = xlp[1];
xlp[1] = xlp[2];
xlp[2] = xlp[3];
xlp[3] = xlp[4];
xlp[4] = yhp[2] / lpgain;
ylp[0] = ylp[1];
ylp[1] = ylp[2];
ylp[2] = ylp[3];

ylp[4] = (xlp[0] + xlp[4]) + 4 * (xlp[1] + xlp[3]) + 6 * xlp[2]
+ ( -0.5174781998 * ylp[0]) + ( 2.4093428566 * ylp[1])
+ ( -4.2388639509 * ylp[2]) + ( 3.3440678377 * ylp[3]);

return((float)ylp[4]);
*/
/* tEMP_FILT_OUT = (float)Math.Abs(yhp[2]);
*/
177

PAA-/0$N " @'+,1- G'0- #$%&$/*
S. The Combination Instiument

using System;
using System.Linq;
using System.Text;
namespace Combo
{
public class MidiListener
{
public void OnMidi(object sender, MidiEventArgs e)
{
Console.WriteLine("EVENT!", "-", e.channel, ":", e.data1, ":", e.data2);
}
}
}
using System;
using System.Linq;
using System.Text;
namespace Combo
{
public class MidiEventArgs : EventArgs
{
public readonly int message;
public readonly int channel;
public readonly int data1;
public MidiEventArgs(int mes, int chan, int d1, int d2)
{
message = mes;
channel = chan;
data1 = d1;
data2 = d2;
}
}
}
using System;
using System.Data;
using System.Linq;
using System.Text;
using WiimoteLib;
namespace Combo
{
{
Controller controller;
public Form1()
{
}
{
controller = new Controller();
}
private void outDeviceIdCombo_SelectedIndexChanged(object sender, EventArgs e)
{
controller.StartBool = false;
controller.OutDeviceId = outDeviceIdCombo.SelectedIndex;
controller.StartBool = true;
}
{
outDeviceIdCombo.Items.Clear();
{
{
outDeviceIdCombo.Items.Add(OutputDevice.GetDeviceCapabilities(i).name);
}
}
}
}
}
using System;
using System.Linq;
using System.Text;
using System.Windows;
using WiimoteLib;
namespace Combo
{
public delegate void MidiEventHandler(object sender, MidiEventArgs e);
public class Controller
{
Sphere sphere;
Cello cello;
MidiOut midiOut;
public Controller()
{
bool wiiexists = true;
midiOut = new MidiOut(0);
catch
{ Console.WriteLine("No Wiimotes");
wiiexists = false;
}
if (wiiexists)
{
cello = new Cello();
sphere = new Sphere();
cello.wiiReset(wc[1]);
sphere.wiiReset(wc[0]);
}
MidiListener listener1 = new MidiListener();
sphere.midiHappened += new MidiEventHandler(sphere_midiHappened);
cello.midiHappenedCello += new MidiEventHandler(cello_midiHappenedCello);
}
{
if (e.message == (int)MIDICOMMAND.NOTE_ON)
{
midiOut.sendNote(e.channel, e.data1, e.data2);
}
else if (e.message == (int)MIDICOMMAND.NOTE_OFF)
{
midiOut.sendNoteOff(e.channel, e.data1);
}
else if (e.message == (int)MIDICOMMAND.CC)
{
midiOut.sendControlChange(e.channel, e.data1, e.data2);
}
}
void cello_midiHappenedCello(object sender, MidiEventArgs e)
{
{
}
{
}
{
}
else if (e.message == (int)MIDICOMMAND.PITCHBEND)
{
midiOut.sendPitchBend(e.channel, e.data1);
}
}
public int OutDeviceId
{
set { midiOut.OutputDeviceId = value; }
}
public bool StartBool
{
set { cello.StartBool = value; }
get { return cello.StartBool; }
}

}
}
using System;
using System.Data;
using System.Linq;
using System.Text;
using WiimoteLib;
namespace Combo
{
{
Controller controller;
public Form1()
{
}
{
controller = new Controller();
}
private void outDeviceIdCombo_SelectedIndexChanged(object sender, EventArgs e)
{
controller.StartBool = false;
controller.OutDeviceId = outDeviceIdCombo.SelectedIndex;
controller.StartBool = true;
}
{
outDeviceIdCombo.Items.Clear();
{
{
outDeviceIdCombo.Items.Add(OutputDevice.GetDeviceCapabilities(i).name);
}
}
}
}
}
using System;
using System.Linq;
using System.Text;
using System.Windows;
using WiimoteLib;
namespace Combo
{
public delegate void MidiEventHandler(object sender, MidiEventArgs e);
public class Controller
{
Sphere sphere;
Cello cello;
MidiOut midiOut;
public Controller()
{
bool wiiexists = true;
midiOut = new MidiOut(0);
catch
{ Console.WriteLine("No Wiimotes");
wiiexists = false;
}
if (wiiexists)
{
cello = new Cello();
sphere = new Sphere();
cello.wiiReset(wc[1]);
sphere.wiiReset(wc[0]);
}
MidiListener listener1 = new MidiListener();
sphere.midiHappened += new MidiEventHandler(sphere_midiHappened);
cello.midiHappenedCello += new MidiEventHandler(cello_midiHappenedCello);
}
{
{
}
{
}
{
}
}
void cello_midiHappenedCello(object sender, MidiEventArgs e)
{
{
}
{
}
{
}
else if (e.message == (int)MIDICOMMAND.PITCHBEND)
{
midiOut.sendPitchBend(e.channel, e.data1);
}
}
public int OutDeviceId
{
set { midiOut.OutputDeviceId = value; }
}
public bool StartBool
{
set { cello.StartBool = value; }
get { return cello.StartBool; }
}

}
}
using System;
using System.Linq;
using System.Text;
namespace Combo
{
public class MidiEventArgs : EventArgs
{
public readonly int message;
public readonly int channel;
public MidiEventArgs(int mes, int chan, int d1, int d2)
{
message = mes;
channel = chan;
data1 = d1;
data2 = d2;
}
}
}
using System;
using System.Linq;
using System.Text;
namespace Combo
{
public class MidiListener
{
public void OnMidi(object sender, MidiEventArgs e)
{
Console.WriteLine("EVENT!", "-", e.channel, ":", e.data1, ":", e.data2);
}
}
}

Wii Musical Interfaces - Max Leonard

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Wii Musical Interfaces - Max Leonard

Uploaded by

Copyright:

Available Formats

Wii Nusical Inteifaces

You might also like