Freya Bailes and Roger T.
Dean
MARCS Auditory Laboratories
University of Western Sydney, Australia
Locked Bag 1797
Penrith South DC NSW 1797, Australia
{f.bailes, roger.dean}@uws.edu.au
NoiseSpeech is a compositional device in which
sound is digitally manipulated with the intention of
evoking the sound qualities of unintelligible speech
(Dean 2005). Speech is characterized by rapidly
changing broadband sounds (Zatorre, Belin, and
Penhune 2002), whereas musicparticularly tonal
musicchanges more slowly and narrowly in
frequency content. As Zatorre and colleagues argue,
this distinction may be reflected in better temporal
resolution in the left auditory cortex and better
spectral resolution in the right, so that perception
is adapted to both ranges and extremes of sonic
stimuli. NoiseSpeech is constructed either by
applying the formant structure (that is, spectral
peak content) of speech to noise or other sounds, or
by distorting speech sounds such that they no longer
form identifiable phonemes or words. The resultant
hybrid is an artistic device that, we argue, may owe
its force to an encapsulation of the affective qualities
of human speech, while intentionally stripping the
sounds of any semantic content. In this article,
we present an empirical investigation of listener
perceptions of NoiseSpeech, demonstrating that
non-specialist listeners hear such sounds as similar
to each other and to unaltered speech.
NoiseSpeech is ambiguous in evoking the iden-
tification of an everyday source of soundhuman
speechwithin the musical context of sound art.
Arguably, it could be said to blur the distinction
described by Gaver (1993a, 1993b) between two
types of listening: the everyday and the mu-
sical. When NoiseSpeech occurs in the context
of a composition or performance, what form of
listening does a listener employ? The context is one
of musical listening, yet the identification of the
sounds with the human generation of speech is an
everyday listening concern. The propensity to iden-
tify the source of a sound is a question of interest to
both cognitive and ecological approaches to percep-
tion. Handel (1989) posits that cognizing sound in
Computer Music Journal, 33:1, pp. 5767, Spring 2009
c
!2009 Massachusetts Institute of Technology.
When Is Noise Speech? A
Survey in Sonic Ambiguity
terms of sound-causing events may override a more
bottomup sensory perception. Concordantly, Ballas
(1993) explores how associations are formed between
environmental sound and sound source, listing ex-
posure to particular sounds in everyday life, being
able to visualize the sound-producing event, and the
similarity of the sound to a mental stereotype.
Dean (2005) proposed that a hybrid of noise and
speech may not only invent a new language but
more importantly may present a new message.
NoiseSpeech seems to escape commodification,
and in this respect fits Attalis (1985) concept of
composition, yet it is not devoid of connotation
or expression. We argue that NoiseSpeech is likely
to evoke affective responses from a listener through
its association with the affective expression of
human speech (Dean and Bailes 2006). Traditionally,
affect has been conceptualized in terms of valence
(positive and negative) and arousal (active and
passive) dimensions. (See for example Leman et al.
2005). Here, we distinguish affect from emotional
connotations that are more concerned with top-
down cognitive associations, such as comfort
or annoyance. According to this distinction, a
certain familiarity with a sound is necessary for
emotion to be evoked, but not for the perception
of affect. Where a sound is identified as familiarly
speech-like, higher-order cognitive processes may
be involved, associated with the perception of an
emotion. However, when sounds are either strongly
distorted through processing or are of ambiguous
origin, listeners may perceive this on an affective
level as valence and arousal.
Vocal affect expression has been studied through-
out history (Banse and Scherer 1996). Links have
been made in speech between emotion and altered ar-
ticulation, respiration, and phonation. In particular,
these effects are believed to be quantifiable in terms
of acoustic variables such as spectral energy distri-
bution, fundamental frequency, and speech rate. For
example, Banse and Scherer (1996) examined the por-
trayal of different emotions by professional actors,
comparing human listener recognition rates with the
Bailes and Dean 57
results of digital acoustic analyses. Relevant to our
concern with extra-semantic affect, the expressions
the actors were required to speak were nonsense sen-
tences, albeit composed of recognizable phonemes.
Listeners recognized emotion based on acoustic
features without semantic clues. We used a related
approach in previous artistic works using artificially
constructed languages, mainly comprising unrecog-
nizable phonemes, also believing that listeners will
identify affective changes in spite of the total lack of
comprehensible words (e.g., Smith and Dean 1996).
Auditory psychology assumes commonality in
the experiences of a given population of listeners.
With respect to the perception of speech, research
has inevitably focused on parsing sounds and con-
structing linguistic meaning, the primary goal being
unambiguous communication. However, speech
can be powerful on a number of extra-linguistic
levels, including musical levels. Ambiguity of mes-
sage or meaning and freedom of interpretation are
useful within the context of sound art. Windsor
(1997) reminds us that context shapes a listeners
interpretation of sound:
Speech sounds, for example, could be organized
into a familiar sequence of fundamental frequen-
cies, encouraging us to hear them as a melody,
or as a sequence of linguistically meaningful
speech events, encouraging us to hear them as
talking. They could be organized in such as way
as to combine them, through cross-synthesis or
juxtaposition, with other sounds, encouraging
us to think of them in relation to other sounds
and their possible meanings, or their spectral
structure could be accentuated or altered to pro-
duce harmonic progressions or to focus our
attention upon their non-speech-like qualities.
Their provenance might be concealed, or they
might in fact be simulacra: we might hear a vir-
tual source other than speech despite the source
material, or hear speech where none exists.
To produce NoiseSpeech and reliably evoke the
impression of speech-derived sound, it is first nec-
essary to identify the perceptual markers of speech.
Expressed in terms of ecological theories of percep-
tion, it is necessary to identify the invariance of
speech (Windsor 1997; Clarke 2005). That is, what
58
properties of speech are systematically interpreted
as such by a listener? In fact, speech is highly com-
plex, typically integrating rhythm, accentuation,
intonation, and affective expression, not to mention
the semantic interpretation arising from parsing the
phonemic components. Speech is characterized by
particular groups of spectral peaks, called formants.
Whereas the precise frequency and spectral shape
of formants tend to distinguish each individual,
and are modulated according to whether a person is
whispering, speaking softly, shouting, or singing, a
discernible pattern of spectral peaks is nevertheless
associated with male or female human speech. Op-
eratic singers superimpose a characteristic singers
formant centered around 24 kHz. Boersma and
Kovacic (2006) describe the long-term audio spectral
characteristics of Croatian folk singing, illustrating
its lack of a singers formant, but they do note the
selective exploitation in some styles of a distinct
pattern in the 3.5 kHz range, which they call the
high shouters formant.
Other characteristics of speech include the
phenomenon of speech declination, which
describes the common large drop in frequency
(and perceived pitch) during a spoken utterance
(Pierrehumbert 1979). Although intensity changes
are modest during declination (generally less than
4 dB), there is a cognitive compensation for the
frequency drop that is influenced by intensity.
Perception of pitch in speech (like that in music)
is complex (de Cheveigne 2005) but possibly uses
pattern-matching of spectral components. This
may operate with speech somewhat differently than
with musical pitchcertainly with musical pitches
largely comprising harmonic tonesbecause speech
formants have central frequencies that often bear
non-harmonic relations to each other.
Sound source identification is important in audi-
tory perception, but to what extent is the tendency
to attribute sound to a particular source mal-
leable? NoiseSpeech artificially imposes formants
on sounds to evoke a sense of vocal human agency.
Alternatively, NoiseSpeech takes existing speech
and filters the sound, in our case taking care to
maintain at least those formants characteristic of
speech that were present in the frequency range
preserved. A few experimental psychologists have
Computer Music Journal
used non-speech control stimuli that were derived
from speech sounds. For example, some have re-
ferred to pseudospeech. This term has been used
inconsistently, but in one application consists of
auditory patterns generated by amplitude and fre-
quency modulation using a base frequency of 200 Hz
and an amplitude envelope which resembles the real
spoken sentences used in the delivery of a two-
minute story (Hermann et al. 2002). This application
suggested that more long-range couplings (between
EEG signals from different brain areas) occur in the
speech condition, as compared to the pseudospeech
condition (p. 3). Although this might suggest a
distinction between processing of speech and pseu-
dospeech, other evidence shows that speechlike
sounds may be comparably lateralized to speech:
in this case, the sounds studied had speechlike
rapid acoustic changes. . . . Their temporal structure
was identical to that of consonant-vowel-consonant
speech syllables, with the very rapid frequency
changes of the spectral peaks, or rapid formant tran-
sitions, that characterize consonants (Belin et al.
1998, p. 536). Overall, this evidence allows the possi-
bility that speech-like sounds may well be identified
as such, as also claimed in the early studies of Remez
et al. (1981), who synthesized what to some seemed
to be largely unintelligible speech, to others science
fiction sounds, music, or whistles, from a min- NoiseSpeech is a more arbitrary one, because our
imum set of three sinusoids. The responses of his par-
ticipants were greatly influenced by the instructions
they received, whether pointing to speech or not.
Features of speech that might be exploited in
sound art composition include rhythm, accentu-
ation, intonation, and other prosodic parameters.
Of course, these sonic features are already germane
to music in general, but their application to noise
was more unusual until the 1990s generation of
real-time laptop artists, and certainly less well
understood. To our knowledge, there has been no
empirical investigation of the extent to which
listeners are sensitive to speech characteristics
transmitted through noise (as distinct from the
many published experiments in which speech or
phonemic content is superimposed upon noise).
If NoiseSpeech can be effective as a different and
perhaps more emotive compositional device than
other digital manipulations of sound, it would be
necessary for listeners to recognize that such sounds
are related to each other. It would also be interesting
to study whether they associate the device with
human speech. This article presents a survey of
the extent to which listeners define NoiseSpeech
samples as being derived from actual speech.
Survey
Our survey aimed to determine whether listeners
would perceive NoiseSpeech samples of various sorts
as being derived from speech. NoiseSpeech samples
varied with respect to their composition (see stimuli
below), ranging from those clearly derived from
speech to highly processed and consequently distant
speech derivation. Listeners were asked to rate
each sample on a yes-or-no basis as being derived
from speech and/or piano, drums, or water. Water
was the dummy variable, in that no samples were
constructed from water sounds. Respondents were
also asked to indicate whether they were sureyes
or nofor each of the four listed source derivations.
The use of a dummy variable was important to es-
tablish the coherence and reliability of assessments.
On the other hand, the choice of instrumental
sounds with which to contrast the speech and
interest here is in whether the latter cluster together
in perception, rather than in how that clustering
relates to perception of other musical sounds.
Participants
Fifteen women and six men (N = 21), ages 1820,
participated in this survey. They did not classify
themselves as musicians, and none had received any
phonetics training. Volunteers were undergraduate
psychology students at the University of Canberra,
receiving 45 minutes of course credit for their
involvement in this survey.
Stimuli
All stimuli were six seconds in duration. Forty items
were selected to represent a variety of NoiseSpeech
Bailes and Dean 59
type. An additional item was used as a practice
trial. Stimuli can be categorized according to the
origin of the sound. One group of stimuli applied
a modification of the processing patch by Atau
Tanaka (provided with the Max/MSP programming
platform) to drum sounds. Another applied the
same patch to a female voice speaking the words
everything is at least double. The Tanaka patch
segments the item and then reassembles a sound
by randomized reordering of the segments. A third
group is another female voice speaking is that you,
included with the Max/MSP software distribution,
after subsequent digital filtering. A fourth group of
stimuli is based on a group of noise sounds, while
another is based on noise alone with stable formants
superimposed. The formants used were centered on
784, 1175, and 2784 Hz, being taken from a female
voice speaking the vowel /a/. Finally, a small group
of sounds presents processed piano phrases.
These groups can be further linked according to
whether they are modified speech or non-speech
with speech-like characteristics. It should be noted
that modified speech stimuli were not intended
to represent the comprehensive phonetic content
of English. Rather, modified speech stimuli were
selected on a compositional basis as examples of
the result of NoiseSpeech generation principles
used in works by Roger Dean and others. Table 1
summarizes these stimuli.
Procedure
Participants rated the stimuli during group listening
sessions, although one participant completed the
listening session alone with the experimenter.
During group sessions, respondents were spaced
around the room such that they would not be
influenced by the responses of others yet could
benefit from stereo presentation of the sounds.
Stimuli were presented using QuickTime and
played through Yamaha MSP5A speakers connected
to an iBook 900HMz G3 laptop computer. The
following instructions were given:
You are asked to decide whether the passages
of sound you hear are derived from drums,
60
Table 1. Description of Stimuli
Processed drum sounds (shuffling segments, time
stretching, frequency/rhythm shifting)
AtDm01 Unaltered, normal drum lick, very
slightly more than 2 bars
AtDm02 Shuffled, slightly more than two bars
AtDm03 Shuffled, slightly more than two bars
AtDm04 Shuffled, slightly more than two bars
AtDm05 Frequency shifted down, shuffled, but
tempo unchanged, so still slightly more
than two bars
AtDm06 Frequency-shifted upward, shuffled, but
tempo unchanged as AtDm05
AtDm07 Tempo slow, frequency-shifted down
AtDm08 Tempo even slower, frequency-shifted
down
AtDm09 Tempo slower still, frequency-shifted
down; really one event for the whole six
seconds
AtDm10 Tempo slower again, frequency-shifted
down, less than one complete event
AtDm11 Frequency-shifted up, slowed somewhat,
like AtDm08
AtDm12 Frequency-shifted up, slowed like AtDm07
Processed sample of female speaking everything is at
least double
AtHs01 Almost five renderings of the phrase,
normal speed
AtHs02 Shuffled, same speed, almost five
renderings of the shuffled version
AtHs03 Shuffled differently, same speed, almost
five renderings
AtHs04 Shuffled differently again, same speed,
almost five renderings
AtHs05 Another shuffle, like AtHs02-04
AtHs06 Slowed down, lower frequency, less than
one cycle
AtHs07 AtHs06 slowed further
AtHs08 AtHs06 slowed even further
AtHs09 AtHs06 slowed even further still
NoiseSpeech series: is that you, a female voice from
the Max/MSP distribution
NsSp01 Normal speed, almost eight renderings
NsSp02 Normal speed, high-pass filtered
NsSp03 Normal speed, high-pass filtered with a
higher cutoff frequency
Computer Music Journal
Table 1. Continued
NsSp04 Normal speed, high-pass filtered with an
even higher cutoff frequency
NsSp05 Normal speed, high-pass filtered with an
even higher cutoff frequency than NsSp04
NsSp06 Normal speed, low-pass filtered
NsSp07 Normal speed, low-pass filtered but with a
greater bandwidth to include more middle
frequencies
NsSp14 High-pass filtered, and with feedback
NoiseSpeech based on a group of noise sounds which
move in spectral center toward higher frequencies,
with a superimposed formant
NsSp08 Original with formant superimposed
NsSp09 High-pass filtered
NsSp10 High-pass filtered with a higher cutoff
frequency
NsSp11 High-pass filtered with an even higher
cutoff frequency
NsSp12 Low-pass filtered
NoiseSpeech: Noise alone with formant
NsSp15 Noise with high formant superimposed
NsSp16 Noise with broader lower formant
superimposed
NsSp17 Noise with broader lower formant still
NsSp18 Like NsSp16 with less of the lower
frequencies
PnPr series: processed piano phrases
PnPr01 High-pass filtered
PnPr02 Slowed phrase, high-pass filtered
PnPr03 Phrase slowed even more, low-pass filtered
piano, speech, and/or water, indicating whether
or not you are confident for each of the listed
possibilities. When the passage has finished
playing, you will be invited to indicate your
answers by ticking the appropriate boxes on
your answer sheet, either yes or no. Please although it does allow them to become aware of the
note that you may indicate several different
sources for the same passage, or you may think
that none of the possible sources are related to
what you hear.
Respondents were asked to consider sound
derivation rather than what the stimuli sound like,
because identifying the source of a sound is central
to our hypothesized increase in affective response
outlined herein.
The answer form for a given trial was as follows:
Example 1
Sound derived from this? Are you sure?
Drums Yes No Yes No
Piano Yes No Yes No
Speech Yes No Yes No
Water Yes No Yes No
Before the survey began, a practice stimulus was
presented with feedback from the researcher empha-
sizing the focus on possible sound derivation (but
without providing cues as to how this might be as-
sessed). Respondents were given the opportunity to
ask questions before proceeding to the main survey.
The forty stimuli were presented with an interval
of approximately five seconds between them. The
stimuli were played in a random order, and a
full presentation of all forty stimuli constituted
a block. Three blocks were presented (with
different randomization for each block) for the
complete survey. Each group of participants heard a
different randomization. As there were six different
groups of participants, this entailed six different
orders of stimuli presentation. Randomization
of stimulus order ensures that any given sound
is heard randomly spread across the duration
of the experimental session. This can remove
what might otherwise be effects of progressive
familiarization with the range of stimuli during
the data gathering, and in our design this was also
avoided by the use of an initial complete exposure
block (block 1), whose response data were not
used. Note that this initial block is unlikely to
impact on listeners longer-term experience with
familiar sounds such as speech, drums, and piano,
range of unfamiliar sounds to which they are being
exposed.
A break was introduced halfway through the
session, during which time participants filled out a
questionnaire asking for demographic information,
details of musical background, and whether they
had ever received any phonetics training.
Bailes and Dean 61
Table 2. Correlations Between Mean Ratings of the A cluster analysis was performed to investigate
Four Possible Sound Derivations of the Stimuli whether there were statistically reliable sub-groups
in listener ratings of the items. The advantage of
Drums Piano Speech Water
this technique is that it uses multivariate data
Drums / 0.25 0.49 0.49 (in this case, the separate ratings of drums, piano,
Piano / / 0.45 0.12 speech, and water) to form clusters of items based
Speech / / / 0.40 on statistical distance. Although the analysis is
Water / / / / exploratory, identifying clusters in this study is
p < 0.5. informative because the focus is on timbre as a
p < 0.001. rich and multidimensional phenomenon. Grouping
sounds according to multiple variables provides a
more realistic description of listener perceptions
Analysis than examining speech ratings alone. Item data were
averaged across respondent and block. The variables
Raw yes and no responses for possible sound drums, piano, speech, and water were
source and confidence were coded as ordinal data entered into a hierarchical cluster analysis in SPSS
such that no, sure = 1, no, unsure = 2, yes, 11.0. Three clusters emerged in the final solution
unsure = 3, and yes, sure = 4. Data from block 1 (see Figure 1). As a point of interest, although we
were disregarded, as it was intended simply to report a solution based on data from blocks 2 and 3,
provide an initial exposure to the full range of this is hardly changed when data from block 1 are
stimuli. Using data from blocks 2 and 3, we first included. There were 9 items in cluster 1, 14 items
checked for order effects across blocks, and then in cluster 2, and 17 items in cluster 3. Descriptive
tested for correlations between speech and the statistics for each cluster are shown in Table 3.
other response categories. The primary analysis Cluster 1 (N = 9) is characterized by high ratings
was a cluster analysis of the stimuli based on for drums. Indeed, the items constituting this cluster
the mean ratings for the four variables. Resulting are AtDm01AtDm08 and AtDm11. AtDm09 and
clusters were characterized, and the stimuli were AtDm10 are similarly derived from drum sounds,
additionally ranked in order of speech-like quality. but these belong to cluster 2; it is worth noting
that they represent the most processed of the drum
stimuli with respect to a slowing down of the sound.
Cluster 2 (N = 14) is characterized by higher-
Results than-average ratings for water. None of the sounds
were water-derived, but those that were rated
A repeated measures analysis of variance (ANOVA) as such include the entire noise-derived series,
of mean ratings per item (within-subjects factors of where either groups of noise (NsSp08NsSp12) or
block and source) revealed no significant difference noise alone (NsSp15NsSp18) were filtered and
in ratings across block. Consequently, data for superimposed with formants. The entire processed
blocks 2 and 3 were collapsed for subsequent piano series falls within cluster 2, raising the average
analyses. piano rating of the cluster. The two slowest drum
It was found that the four variables (drums, stimuli (AtDm09 and AtDm10) were also within
piano, speech, and water) were negatively correlated this cluster.
with each other except for piano and drum ratings, Cluster 3 (N = 17) is typified by high ratings for
which were positively correlated, and piano and speech. All items within this cluster are indeed
water, which were uncorrelated. (See the correlation derived from recordings of speech that have been
matrix in Table 2.) The positive correlation between subsequently processed, or as formant superimpo-
piano and drums probably reflects their shared sitions on noise or groups of noise. For instance,
attributes as percussion instruments. AtHs01AtHs09 are a processed sample of a female

62 Computer Music Journal

Figure 1. Dendrogram of
the cluster analysis of 40
items.
voice speaking everything is at least double, and
NsSp01NsSp14 constitute a female voice asking,
Is that you? As noted previously, two of the
items (NsSp01 and AtHs01) are unaltered speech;
this data shows that putative NoiseSpeech items
indeed cluster with genuine speech. One remarkable
feature that deserves future investigation is that
NsSp15NsSp18 cluster with the other speech and
NoiseSpeech sounds (though with generally lower
speech ratings), even though they involve sta-
ble (unchanging) formants superimposed on noise,
whereas speech itself comprises rapidly changing
formant spectra.
Taken together, the clusters seem to describe a
continuum, with the most clearly identifiable sound
sourcesdrums (cluster 1) and speech (cluster 3)at
Bailes and Dean 63
Table 3. Mean (M) and Standard Deviation (SD) for Table 4. Items Ranked by Mean Speech Rating
Blocks 2 and 3 by Cluster across Participants and Block (for Blocks 2 and 3)
Cluster Mean Mean
Speech Speech
Total items 1 (N = 9) 2 (N = 14) 3 (N = 17) Item Name Rating Cluster Item Name Rating Cluster
Clustering
Variable M SD M SD M SD M SD
AtHs01 4.00 3 AtDm10 1.40 2
Drums 1.82 1.05 3.68 0.30 1.29 0.29 1.28 0.25 NsSp02 4.00 3 NsSp08 1.33 2
Piano 1.29 0.32 1.38 0.24 1.43 0.43 1.13 0.12 NsSp03 4.00 3 NsSp12 1.33 2
Speech 2.26 1.26 1.12 0.08 1.29 0.15 3.66 0.47 AtHs04 3.98 3 NsSp18 1.33 2
Water 1.85 0.82 1.21 0.09 2.75 0.69 1.45 0.35 AtHs02 3.95 3 AtDm08 1.28 1
AtHs03 3.95 3 NsSp09 1.23 2
NsSp14 3.95 3 NsSp17 1.20 2
the extremes, passing through cluster 2, which NsSp01 3.90 3 AtDm02 1.17 1
NsSp04 3.85 3 AtDm01 1.15 1
groups the most processed sounds together. These
NsSp07 3.80 3 AtDm04 1.15 1
seem to have been the most ambiguous items for
AtHs09 3.78 3 NsSp10 1.15 2
listeners and were rated highly for water, even AtHs05 3.72 3 NsSp11 1.15 2
though water sounds were never sampled. A two- NsSp06 3.63 3 NsSp15 1.15 2
cluster reading combines clusters 2 and 3 and AtHs06 3.43 3 PnPr01 1.15 2
contains all speech and NoiseSpeech stimuli, joined AtHs08 3.03 3 PnPr02 1.15 2
by only five other sounds. NsSp05 2.73 3 AtDm05 1.10 1
Table 4 shows the ranking of all items for AtHs07 2.50 3 AtDm11 1.10 1
their average rating as speech-like, disregarding AtDm09 1.65 2 AtDm03 1.08 1
other response categories. Again notable is the fact NsSp16 1.45 2 AtDm12 1.05 1
that only one of the unprocessed speech samples PnPr03 1.45 2 AtDm06 1.03 1
(AtHs01) attracted the highest possible score for
speech quality, and the rating of two NoiseSpeech the most speech-like, both in terms of listener
items equaled this, whereas the speech ratings of perceptions and in actual sound source: here, speech
several other NoiseSpeech items exceeded that samples were processed to varying extents.
for the other unprocessed speech stimuli. Thus, The relationship between sound-source ratings,
listeners intermingled speech and NoiseSpeech in which exhibits a pattern of correlation as predicted,
their categorical responses. merits discussion. The only positive correlation was
found between ratings for piano and drums: these
two sound sources are both percussion instruments.
Discussion Other correlations were negative or non-existent
(water and piano). The inclusion of water as a
Results of the cluster analysis can be seen as dummy response was intentional. Had we used
describing differing degrees of speech-like quality, the category of other, we might have expected
moderated by timbral complexities of the sound quite different results, with listeners tending to
qualities evoked by water, drums, and piano. attribute sounds about which they were unsure in
Cluster 1 can be considered the least speech- this bracket. At the opposite extreme, replacing
like, with a clear grouping based on high ratings the dummy response water with noise as a
for drums. Cluster 2 is more ambiguous, with possible sound derivation would certainly have led
a mismatch between sound source and listener to a tighter grouping of items in cluster 2, which
perception insofar as these items were rated as constitutes the noisy items.
being water-derived, when in fact water was never In the context of the current survey, it is difficult
sampled or intentionally evoked. Cluster 3 is to determine to what extent hearing the full range

64 Computer Music Journal

of NoiseSpeech stimuli in block 1 subsequently
strengthened the listeners perceptions of and as-
sociations with speech or with other unmodified
sounds. Listeners may well have formed an associa-
tion between the few undisguised items and stimuli
generated from these. This is not unlike an artis-
tic listening experience in which the inclusion of
variously speech-like NoiseSpeech within one work
provides a listener with an increasingly relative
and shifting context of cues through time. It is also
related to many published speech-perception experi-
ments that require phoneme identification among a
set of stimuli varying along a subtle transformation
gradient. Would an isolated stimulus be rated as
more speech-like in a different context of stimuli
that are in no way reminiscent of speech? If this
were the case, it would have interesting implications
for theories that posit the immediate perception of
environmentally significant information, as with
the human voice.
It is worth considering further the psychological
background to the open question of the affective
impact of NoiseSpeech. One reason for proposing it
as a viable form for expressive composition is the
longstanding claim that speech engages different
modes of perception and cognition than do other
sounds. For example, early studies from the Haskins
laboratory and elsewhere, critically reviewed by
Repp (1982, p. 81), argued that speech cannot be
reduced to a combination of auditory processes
involved also in perceiving and interpreting non-
speech sounds. A few of the component arguments
can be summarized briefly. The left-hemisphere
lateralization of speech processing is unassailable
(p. 85) but shared with certain other sound stimuli.
Duplex perception, in which formant transitions
are removed from a synthetic syllable and presented
to one ear while the rest of the speech pattern is
presented to the other ear (p. 86), creates both a
fused percept of the original syllable and a whistle
or chirp in one ear, and is argued to suggest the
simultaneous use of speech and nonspeech modes
of perception (p. 86). Phonetic trading is the
phenomenon in which a change in the setting of
one cue . . . can be offset by an opposed change in the
setting of another cue so as to maintain the original
phonetic percept. Repp argues that certain aspects
of this trading only function when the speech
perception mode is engaged, citing in particular
the work of Best, Morrongiello, and Robson (1981)
in using analogous sine waves imitating the
formant trajectories of (voiced) speech signals as
a comparison with genuine speech signals. Those
listeners who engaged only an auditory rather than
speech mode failed to demonstrate such trading
relations, and they heard some of the sounds solely
as whistles. However, once instructed to listen for
speech, this could be overcome.
More recent evidence has questioned most of
these arguments. For example, Diehl, Lotto and
Holt (2004) and colleagues review speech perception
from three major perspectives. Two of these, the
motor theory and the direct-realist theory, implicate
the articulatory events of the tongue, lips, and vocal
folds as key gestural objects of speech perception
(Diehl, Lotto, and Holt 2004). In contrast, the third
perspective, that of general-auditory (GA) theories,
makes no such assumption, and partly as a result,
does not need to invoke a special speech-mode of
perception. In this GA perspective, speech may still
be uniquely important during learning processes:
speech may be a particularly salient signal for
infants, and learning processes may be biased
to pick up just the kind of information that is
important for speech categories (Diehl, Lotto, and
Holt 2004, p. 164).
The issue of a speech-specific mode of perception
thus remains to be resolved, but the likelihood that
the engagement of perceptual or cognitive processes
focused on speech can affect listeners responses
remains unchallenged by this literature. In our
study, NoiseSpeech sounds are clustered in sonic
character with speech. Thus they are potentially
usable semiotic tools within the compositional
process; and it is remarkable that some of the
NoiseSpeech samples we have analyzed were
indeed derived solely from noise and formant
characteristics (and not from preexistent speech
sounds). It remains to be established whether they
have distinct affective impacts by virtue of their un-
conscious or conscious identification as speech-like
and their engagement of some modes of perception
and cognition that are shared by speech. Moreover,
there are many cultural associations with sounds
Bailes and Dean 65
that may override the impact of human agency. manipulation, particularly with sounds involving
For example, the Australian listeners in this study substantial components of noisy spectra, our results
were surveyed during a serious drought. Had they indicate that superimposition of certain sonic fea-
been asked to rate the emotional impact of sounds tures from speech could again transform the axes
they deemed to be derived from water, this could upon which listeners attend to the music; this may
well have been greater than for sounds perceived as be an effective device for the laptop improviser, as
speech-derived. we originally proposed (Dean 2005).
Our findings provide an opportunity to reflect
on how well a computer would fare in detecting
NoiseSpeech. A range of information-retrieval
methods for machine discrimination of speech from
Future Work
music has been developed, often using cepstral
The compositional characteristics underlying the
coefficients and Hidden Markov Models. Other
clustering of NoiseSpeech sounds surveyed in this
methods have been developed that use, for example,
study raise a number of questions for future research.
geometric properties of spectrograms as objects for
First, the clustering of items comprising stable
visual identification (Casagrande, Eck, and Kegl
formants with those in which the spectrum change
2005). Casagrande and colleagues wrote a Winamp
over time is of considerable theoretical interest,
plug-in that trained an AdaBoost model. On the
because spectral and temporal sound dimensions
basis of our present results, it can be predicted
may be important features distinguishing music
that a machine will probably identify NoiseSpeech
and speech perception in general, as indicated at
roughly as do human listeners, and informal use
the outset of this article. Second, clusters 2 and 3
of this Winamp plugin supports this view. In
are separable along compositional lines: cluster 2
addition, such automated detection of NoiseSpeech
includes sounds that are shaped white noise, whereas
in line with human perceptions could be a useful
cluster 3 comprises sounds that are native or
compositional or performance tool in generative
processed speech samples. We speculate that the
sound art. Of course, the extent to which ambiguity
items with the character of cluster 2, in which
in listener perceptions is deliberately fostered is an
noise is the predominant sound (indicated by their
artistic question. Results from our survey can only
attribution to a water source) will generally evoke
serve to inform sound artists of possible listener
a more negative affect than the items in cluster 3,
responses.
which are heard as speech-like. This hypothesis
In the broadest terms, to consider sonic construc-
can be tested using NoiseSpeech stimuli, taken
tion in terms of multidimensional spaces of timbres
in conjunction with the results from the current
is worthwhile, because composers can assert or
survey. Further empirical investigation would not
emphasize such relationships systematically if they
only inform computer-music composition, but it
choose. In sound-sculpted work that is entirely fixed
would also provide a greater understanding of the
before presentation, this implies that procedures
role of familiarity in the perception of emotional
advocated by spectromorphologic composers and
or affective expression. Future research should also
by timbre-space empiricists can indeed have use-
further investigate different listener perceptions
ful application. For example, transformations or
when NoiseSpeech is heard in either a musical
cross-morphing of speech sound, as practiced by
setting or in an isolated hearing.
composers such as Charles Dodge, Paul Lansky,
Trevor Wishart, Wende Bartley, Tristan Murail, and
others, may pull a listener away from semantic
listening (that is, attending to verbal content) to Acknowledgments
speech-oriented yet non-semantic listening, and
eventually to musical listening without concern for Our thanks go to Michael Bylstra for his assistance
the physical source of the sound. In real-time sonic in encoding data.

66 Computer Music Journal

References
Attali, J. 1985. Noise: The Political Economy of Music,
trans. B. Massumi. Manchester: Manchester University
Press.
Ballas, J. A. 1993. Common Factors in the Identification
of an Assortment of Brief Everyday Sounds. Journal
of Experimental Psychology: Human Perception and
Performance 19:250267.
Banse, R., and K. R. Scherer. 1996. Acoustic Profiles in
Vocal Emotion and Expression. Journal of Personality
and Social Psychology 70:614636.
Belin, P., et al. 1998. Lateralization of Speech and
Auditory Temporal Processing. Journal of Cognitive
Neuroscience 10(4):536540.
Best, C. T., B. Morrongiello, and R. Robson. 1981.
Perceptual Equivalence of Acoustic Cues in Speech and
Nonspeech Perception. Perception and Psychophysics
29:191211.
Boersma, P., and G. Kovacic. 2006. Spectral Characteris-
tics of Three Styles of Croatian Folk Singing. Journal
of the Acoustical Society of America 119(3):18051816.
Casagrande, N., D. Eck, and B. Kegl. 2005. Frame-
Level Speech/Music Discrimination Using AdaBoost.
Proceedings of the 2005 International Society for
Music Information Retrieval. London: Queen Mary,
University of London, pp. 345350.
Clarke, E. F. 2005. Ways of Listening: An Ecological
Approach to the Perception of Musical Meaning. New
York: Oxford University Press.
Dean, R. T. 2005. NoiseSpeech, a Noise of Living
Bodies: Towards Attalis Composition. Journal of
New Media and Culture 3(1). Available online at
www.ibiblio.org/nmediac/winter2004/NoiseSpc.htm.
Dean, R. T., and F. Bailes. 2006. NoiseSpeech. Perfor-
mance Research 11(3):8586.
de Cheveigne, A. 2005. Pitch Perception Models. In C. J.
Plack, et al., eds. Pitch: Neural Coding and Perception.
New York: Springer.
Diehl, R. L., A. J. Lotto, and L. L. Holt. 2004. Speech
Perception. Annual Review of Psychology 55:149
179.
Gaver, W. W. 1993a. How Do We Hear in the World?
Explorations in Ecological Acoustics. Ecological
Psychology 5(4):285313.
Gaver, W. W. 1993b. What in the World Do We Hear? An
Ecological Approach to Auditory Event Perception.
Ecological Psychology 5(1):129.
Handel, S. 1989. Listening: An Introduction to the Per-
ception of Auditory Events. Cambridge, Massachusetts:
MIT Press.
Hermann, T., et al. 2002. Sonifications for EEG
Data Analysis. Paper presented at the 2002
International Conference on Auditory Dis-
play, Kyoto, Japan, July. Available online at
www.icad.org/websiteV2.0/Conferences/ICAD2002/
proceedings/22 Thomas Hermann EEG.pdf.
Leman, M., et al. 2005. Prediction of Musical Affect
Using a Combination of Acoustic Structural Cues.
Journal of New Music Research 34(1):3967.
Pierrehumbert, J. 1979. The Perception of Fundamental
Frequency Declination. Journal of the Acoustical
Society of America 66(2):363369.
Remez, R. E., et al. 1981. Speech Perception with-
out Traditional Speech Cues. Science 212:947
950.
Repp, B. H. 1982. Phonetic Trading Relations and Context
Effects: New Experimental Evidence for a Speech
Mode of Perception. Psychological Bulletin 92(1):81
110.
Smith, H., and R. T. Dean. 1996. Nuraghic Echoes. Audio
compact disc. Rufus Records RF 025.
Windsor, L. W. 1997. Frequency Structure in Electroa-
coustic Music: Ideology, Function and Perception.
Organised Sound 2(2):7782.
Zatorre, R. J., P. Belin, and V. B. Penhune. 2002. Structure
and Function of Auditory Cortex: Music and Speech.
Trends in Cognitive Sciences 6(1):3746.
Bailes and Dean 67