CSIM RTI SJorda03 InteractiveMusic LowRes PDF

Real-Time Interaction Module
Interdisciplinary Master in Cognitive Systems

and Interactive Media
Session 3
Interactive Music
Prof. Sergi Jordà

sergi.jorda@upf.edu
Sergi Jordà, Music Technology Group

IUA, UPF – Barcelona, 2010
Interactivity Perspectives
! The study of interactivity can neither hardly be made from a generic
perspective. It has been mostly made from the perspective of
! WIMP, GUIs and Internet
! Virtual reality
! Videogames
! Media Art
! Music performance
! The discourse of the last two has been mostly kept (and still is today)
outside of the mainstream / unknown to mainstream HCI scholars, e.g.
Jonathan Grudin (2011) A moving target: The evolution of HCI, Taylor
& Francis (http://research.microsoft.com/apps/pubs/default.aspx?
id=138583)

Real-time Interaction (CSIM/SMC), UPF – Barcelona, 2011
Interactive Music
! Because
! it is mostly unknown to the mainstream
! it is well known to me
! it shows huge doses of uncertainty, complexity, content
creation and manipulation…
! We will have a quick overview at the history of Interactive Music

Interactive Music?

L’Instrument philosophe
L’instrument philosophe est

sensible; il est en même temps le
musicien et l’instrument. Comme
sensible, il a la conscience
momentanée du son qu’il rend; comme
l’animal il en a la mémoire. Cette
faculté organique, en liant les sons en
lui-même, y produit et conserve la
mélodie. Supposez au clavecin de la
sensibilité et de la mémoire, et dites-
moi s’il ne se répétera pas de lui-
même les airs que vous aurez exécutés
sur ces touches. Nous sommes des
instruments doués de sensibilité et de
mémoire (Diderot à D’Alembert
~1770).
AUTOMATED CONTROL
PROCESSES
Hammond+Leslie
NON-HUMAN
EXCITATION
ENERGY
Analog modular
synthesizer

Some Interactive Music Definitions/Metaphors
Joel Chadabe (2004)

! Sailing a boat on a windy day and through stormy seas (e.g.
G.Lewis’ Voyager)
! The net complexity or the conversational model (e.g. SalMar,
League Automatic Composers)
! The powerful gesture expander (Groove, Music Mouse…)
Jeff Pressing (1990)

! playing a musical instrument
! conducting an orchestra
! playing together (ensemble) with a machine
! acting as a one-man band

Electronic Music Performance
1960 1970 1980 1990
J.Cage “cartridge First hybrid systems Early MIDI

music” (1961) Mathew’s GROOVE, K.C.S., Music Mouse,
K.Stockhaussen J.Chadabe, M & Jam Factory,
(1964) S.Martirano HMSL…
Live electronics Microcomputers

MEV, D.Tudor, The League,
G.Mumma, G.Lewis,
D.Buchla … M.Waisvisz…

From ‘scored’ instruments (Mathews GROOVE)
to ‘intelligent’ instruments (L.Spiegel)
It …seemed to me that there were a number of aspects of
music where the performer didn’t have any freedom of choice.
The sequence of notes to be played, for example, was an area in
which the computer could legitimately help a performer in
making it easier to do things correctly (Roads 1989, interview
with Max Mathews).
Intelligent Musical Instruments are instruments that first

sensed its performer’s intent by the type and context of the
performer’s actions and then extended the performer’s controls
to execute the music automatically. Intelligent instruments let
people play the music on a compositional level (Gagne 1993,
interview with Laurie Spiegel).

Joel Chadabe
Macro vs. MicroControl
In 1977, for his piece Solo, he

connected two Theremin
antennas to a Synclavier and
controlled though the antennas,
the tempo and the timbre of
several simultaneous but
independent sequences. Moving
his hands in the air allowed him
to “conduct” an improvising
orchestra. As a result of these
experiments, he coined the term
‘interactive composing’.
Several years later, Chadabe founded the company Intelligent Music,
which in 1986 released the programs M and Jam Factory (Zicarelli 1987),
which constituted (together with the aforementioned Music Mouse) the
first commercial interactive music software (Chadabe 1981, 2004)

Salvatore Martirano
Interaction vs. Control ?
The SalMar Construction (1972)
included 291 touch-sensitive switches,
that could affect in real time four
concurrently running programs that drove
the four voices of a custom analog
synthesizer. The fact that the four
processes run in parallel, concurrently
modifying parameters such as pitches,
loudness, timbre, envelopes, etc. made
impossible to fully predict the system’s
behavior (Martirano 1971; Franco 1974;
Walker et al. 1992).
“performing with the system was too
complex to analyze… Control was an
illusion. But I was in the loop. I was
trading swaps with the logic… It was like
driving a bus” (from Chadabe 1997:291)

The League of Automatic
Composers
When the elements of the network are not connected the music sounds like three
completely independent processes, but when they are interconnected the music
seems to present a “mindlike” aspect. Why this is so or why we can perceive
some but not all activities as the product of an artificial intelligence is not
understood (Bischoff et al. 1978).
Musical Instruments Digital Interface (MIDI)
! http://en.wikipedia.org/wiki/Midi
! http://en.wikipedia.org/wiki/The_MIDI_1.0_Protocol

MIDI Messages
Message Name Binary Hex Data1 Data2
Note Off 1000 nnnn 8N pitch velocity
Note On 1001 nnnn 9N pitch velocity
Poly. Aftertouch 1010 nnnn AN pitch pressure
Control Change 1011 nnnn BN control intensity

number
Program Change 1100 nnnn CN program
Channel Aftertouch 1101 nnnn DN pressure
Pitch Bend 1110 nnnn EN MSByte LSByte
System Message 1111 xxxx FX

George Lewis
Machine Listening / Decontrol?
Modeling a human listener is part of it… I don’t like playing solo

trombone concerts. I would rather play with someone else. I wanted
to make a program to see what sorts of things an improviser has to
know in order to play successfully with another improviser (Lewis
interviewed in Roads 1985b:83).
The system is designed to avoid the kind of uniformity where the

same kind of input routinely leads to the same result. Voyager’s
aesthetic of variation and difference is at variance with the
information retrieval and control paradigm that late capitalism has
found useful in framing its preferred approach to the encounter with
computer technology. As I have observed elsewhere, interactivity has
gradually become a metonym for information retrieval rather than
dialogue, posing the danger of commodifying and ultimately reifying
the encounter with technology (Lewis 2000).

PITEL machine listening & improvisation
(1989-91)
Jordà, S. (1991). A Real-Time MIDI

Composer and Interactive Improviser
by Means of Feedback Systems.
Proceedings of the 1991
International Computer Music
Conference, 463–466.

JoAn, l’Home de Carn http://www.dtic.upf.edu/~sergi/
IUA_WEB/articles/JoAn93.pdf
Jordà & Antúnez 93

Machine Listening / Extreme Control
Interactivity?
! If the aforementioned examples, can be seen as the

paradigm of “out-of-control-interactivity”, the same
technique (Machine Listening) can also be applied in a
complete opposite way, and used in a quite non-
interactive manner…

Machine Listening at IRCAM
Score Following & Interactive Signal Processing
! Problems with tape music + 1

! Miller Puckette, early MAX (Vercoe & Puckette 1985) …
! Score following (Dannenberg 1984, 1988, 1989)
! Interactive Signal Processing (Winkler 1991, Lippe 1996)
…

Score Following & Interactive Signal Processing
! Interactive ?
! Problems ?
! The existence of an apparent 2-way communication
does not garantee interaction.
Tempo or discrete events
from the performer,
determine the sound
effects to be applied,
which on its turn, modify
the sound emitted by the
performer, but not
necessarily the tempo or
the future discrete events

Score Following & Interactive Signal Processing (ISP)
! Problems with tape music + 1

! Miller Puckette, early MAX …
! Score following
! Interactive Signal Processing …
This is more ISP !!

Some Interactive Music Definitions/Metaphors
Joel Chadabe (2004)

! Sailing a boat on a windy day and through stormy seas (e.g.
G.Lewis’ Voyager)
! The net complexity or the conversational model (e.g. SalMar,
League Automatic Composers)
! The powerful gesture expander (Groove, Music Mouse…)
Jeff Pressing (1990)

! playing a musical instrument
! conducting an orchestra
! playing together (ensemble) with a machine
! acting as a one-man band

Traditional vs.
Computer-based
Instrument Paradigm
One of the best assets

of new digital
instruments is the
possibility to run several
multiple and parallel
musical processes in a
shared control between
the instrument and the
performer

Multithreaded instruments & shared control
(Jordà 2005)
! In traditional instruments the performer is responsible for
controlling every smallest detail, leaving nothing to the instrument
responsibility
! One of the best assets of new digital instruments is the possibility
to run several multiple and parallel musical processes in a shared
control between the instrument and the performer
! These instruments’ ‘intelligence’ may be partially responsible for
one or more musical processes, the control of which may be
entirely left to the instrument’s responsibility or may be shared in
different ways with the performer
! When performers delegate some control to the instrument, their
role is also approaching that of composers, who do have to delegate
control of their work to instrumentalists

Some multithread &
shared control corollaries
These systems (instruments) can

! surpass the one gesture to one acoustic event’ paradigm
! go beyond the sound and note control level
! current systems (i.e. after MIDI) allow in fact to bridge a
continuum between micro (i.e. timbre) & macrostructure, micro
& macrocontrol (e.g. sound processing " machine listening
continuum)
! run multiple and parallel musical processes in a shared control
between the instrument and the performer # the possibility to
have multiple performers seems as a logical and promising
extension

Some musical interaction questions,
problems or paradoxes
! Interaction (‘surprise’) vs. instrument control?

! Interactive or merely reactive ? (e.g. score
following)
! Interaction and no improvisation? Is that possible?
! What’s the role of musical performance “in the
age of digital reproduction”?
! On the next class we will discuss some related

issues, from the opposite perspective, that of
control (or trying to deal with complexity)

Machine listening / Some more examples
! Gordon Mumma’s Hornpipe (1967)

! George Lewis’ Voyager (1984-)
! Sergi Jordà’s PITEL (1989-92)
! François Pachet’s Continuator (2003-05)
http://www.csl.sony.fr/~pachet/Continuator/
! Lemur’s Robots (2004-)
! Lemur’s Robots with Pat Metheny (Orchestrion) 2009
(http://www.youtube.com/watch?v=cCMnPb0qLIU)

Some additional early (& analog) Electronic
Music Performance refs.
In case you are interested in this topic (which we will not cover)
! Mumma, G. (1975). Live-electronic music. In J. Appleton and R.

Perera, eds. The Development of Electronic Music. Englewood
Cliffs: Prentice Hall., 286-335
! Davies, H. (1984). Electronic instruments. In Stanly Sadie, ed., The
New Grove Dictionary of Musical Instruments, London: Macmillan
Press.
! Chadabe, J. (1997). Electric Sound: The Past and Promise of
Electronic Music. Upper Saddle River, New Jersey: Prentice Hall.
! Holmes, T. B. (2002). Electronic and Experimental Music, 2nd
edition. NY: Routledge.
! Collins, N. (2007). Live electronic music. In N. Collins & J.
d’Escribán, eds., Cambridge Companion to Electronic Music.
Cambridge: Cambridge University Press.

Interactive music/instruments
definitions, metaphors, taxonomies
In case you want to read more about interactive music related
concepts. These references discuss concepts (not implementations) …
often leading to contradictory statements…
! John Bischoff et al. (1978)
! Joel Chadabe (1984)
! George Lewis (1985)
! Michel Waisvisz (1985)
! Jeff Pressing (1990)
! David Wessel (1991)
! Miller Puckette (1991)
! Laurie Spiegel (1992)
! Robert Rowe (1993)
! Todd Winkler (1998)
! Jordà (2005, 2007)
! Magnusson (2005, 2006)

Further Readings (1/2)
! Bischoff, J., Gold, R. & Horton, J. (1978). Music for an interactive Network of
Computers. Computer Music Journal, 2(3), 24-29.
! Chadabe, J (1984). Interactive Composing: An Overview. Computer Music Journal,
8(1), 22-28. Republished in Curtis Roads, ed., The Music Machine (Cambridge, MA:
MIT Press, 1989).
! Chadabe, J. (1997). Electric Sound: The Past and Promise of Electronic Music.
Upper Saddle River, New Jersey: Prentice Hall.
! Jordà, S. (2005). Digital Lutherie: Crafting musical computers for new musics’
performance and improvisation. Ph.D. dissertation, Dept. of Computer
Engineering, Universitat Pompeu Fabra, Barcelona (chaps. 2, 4, 5)
! Jordà, S. (2007). Interactivity and Live Computer Music, in "The Cambridge
Companion to Electronic Music", Edited by Nick Collins and Julio d’Escrivan.
Cambridge University Press, UK.
! Lewis, G. E. (2000). Too Many Notes: Computers, Complexity and Culture in
Voyager. Leonardo Music Journal, 10, 33–39.
! Magnusson, T. (2006). Screen-Based Musical Interfaces as Semiotic Machines, in
Proc. NIME 2006, Paris, France, 2006, pp. 162–167.
! Pressing, J. (1990). Cybernetic Issues in Interactive Performance Systems.
Computer Music Journal, 14(1), 12-25.
Further Readings (2/2)
! Puckette, M. & Settel, Z. (1993). Non-obvious roles for electronics in performance
enhancement. In Proceedings of the 1993 International Computer Music
Conference. San Francisco, CA: International Computer Music Association :
134-137.
! Rowe, R. (1993). Interactive Music Systems: Machine Listening and Composing.
Cambridge, MA: The MIT Press.
! Rowe, R. (2001). Machine Musicianship. Cambridge, MA: The MIT Press.
! Ryan, J. (1991). Some Remarks on Musical Instrument Design at STEIM.
Contemporary Music Review, 6(1), 3-17.
! Settel, Z. & Lippe, C. (2003). Convolution Brother’s Instrument Design. In
Proceedings of the 2003 Conference on New Interfaces for Musical Expression
(NIME-03), 197-200. Montreal.
! Spiegel, L. (1992). An Alternative to a Standard Taxonomy for Electronic and
Computer Instruments. Computer Music Journal, 16(3).
! Winkler, T. (1998). Composing Interactive Music: Techniques and Ideas Using Max.
Cambridge, MA: The MIT Press.

CSIM RTI SJorda03 InteractiveMusic LowRes PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

CSIM RTI SJorda03 InteractiveMusic LowRes PDF

Uploaded by

Copyright:

Available Formats

Real-Time Interaction Module

Interdisciplinary Master in Cognitive Systems

Prof. Sergi Jordà

Sergi Jordà, Music Technology Group

Sergi Jordà, Music Technology Group

! We will have a quick overview at the history of Interactive Music

Sergi Jordà, Music Technology Group

Sergi Jordà, Music Technology Group

L’instrument philosophe est

Sergi Jordà, Music Technology Group

Joel Chadabe (2004)

Jeff Pressing (1990)

Sergi Jordà, Music Technology Group

J.Cage “cartridge First hybrid systems Early MIDI

Live electronics Microcomputers

Sergi Jordà, Music Technology Group

Intelligent Musical Instruments are instruments that first

Sergi Jordà, Music Technology Group

In 1977, for his piece Solo, he

Sergi Jordà, Music Technology Group

Sergi Jordà, Music Technology Group

Sergi Jordà, Music Technology Group

Message Name Binary Hex Data1 Data2

Note Off 1000 nnnn 8N pitch velocity

Note On 1001 nnnn 9N pitch velocity

Poly. Aftertouch 1010 nnnn AN pitch pressure

Control Change 1011 nnnn BN control intensity

Channel Aftertouch 1101 nnnn DN pressure

Pitch Bend 1110 nnnn EN MSByte LSByte

System Message 1111 xxxx FX

Sergi Jordà, Music Technology Group

Modeling a human listener is part of it… I don’t like playing solo

The system is designed to avoid the kind of uniformity where the

Sergi Jordà, Music Technology Group

Jordà, S. (1991). A Real-Time MIDI

Sergi Jordà, Music Technology Group

Sergi Jordà, Music Technology Group

! If the aforementioned examples, can be seen as the

Sergi Jordà, Music Technology Group

! Problems with tape music + 1

Sergi Jordà, Music Technology Group

Sergi Jordà, Music Technology Group

! Problems with tape music + 1

This is more ISP !!

Sergi Jordà, Music Technology Group

Joel Chadabe (2004)

Jeff Pressing (1990)

Sergi Jordà, Music Technology Group

One of the best assets

Sergi Jordà, Music Technology Group

Sergi Jordà, Music Technology Group

These systems (instruments) can

Sergi Jordà, Music Technology Group

! Interaction (‘surprise’) vs. instrument control?

! On the next class we will discuss some related

Sergi Jordà, Music Technology Group

! Gordon Mumma’s Hornpipe (1967)

Sergi Jordà, Music Technology Group

! Mumma, G. (1975). Live-electronic music. In J. Appleton and R.

Sergi Jordà, Music Technology Group

Sergi Jordà, Music Technology Group

You might also like