This article was downloaded by: [41.235.86.

174]
On: 10 July 2011, At: 12:24
Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,
37-41 Mortimer Street, London W1T 3JH, UK

Journal of New Music Research

Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/nnmr20

Perceptual Evaluation of Inter-song Similarity in

Western Popular Music
a b c a b
Alberto Novello , Martin M.F. McKinney & Armin Kohlrausch
a
Technische Universiteit Eindhoven, The Netherlands
b
Philips Research Laboratories, The Netherlands
c
Starkey Laboratories, USA

Available online: 31 Mar 2011

To cite this article: Alberto Novello, Martin M.F. McKinney & Armin Kohlrausch (2011): Perceptual Evaluation of Inter-song
Similarity in Western Popular Music, Journal of New Music Research, 40:1, 1-26

To link to this article: http://dx.doi.org/10.1080/09298215.2010.523470

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-conditions

This article may be used for research, teaching and private study purposes. Any substantial or systematic
reproduction, re-distribution, re-selling, loan, sub-licensing, systematic supply or distribution in any form to
anyone is expressly forbidden.

The publisher does not give any warranty express or implied or make any representation that the contents
will be complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should
be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims,
proceedings, demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in
connection with or arising out of the use of this material.
 Journal of New Music Research
2011, Vol. 40, No. 1, pp. 1–26

Perceptual Evaluation of Inter-song Similarity in Western Popular

Music

Alberto Novello1,2, Martin M.F. McKinney3, and Armin Kohlrausch1,2

1
Technische Universiteit Eindhoven, The Netherlands; 2Philips Research Laboratories, The Netherlands; 3Starkey
Laboratories, USA

Abstract
Downloaded by [41.235.86.174] at 12:24 10 July 2011

We describe and test the methodological set up for a web-
based listening experiment that assesses the perception of
inter-song similarity optimizing the trade-off between
stimulus coverage and experimental time. The experiment
used a relatively large set of stimuli of Western popular
music: 78 song excerpts selected from 13 genres, involving
78 participants. The experiment used triadic comparisons
of song excerpts to present the participants with a low-
complexity task, and a partially balanced incomplete block
design (PBIBD) to reduce the number of stimulus
comparisons with the consequent possibility of extending
the stimulus set. The three control variables used in the
excerpt selection, genre, tempo and timbre, showed
statistically significant saliency and a hierarchical degree
of impact on participants’ pair rankings (genre 4 tempo 4
timbre). We investigated the participants’ perceptual space
using a combination of numerical and analytical methods
that help to reduce and represent the dimensionality of the
data. We used a combination of scaling and discriminant
functions to gain insight into the important factors
underlying the organization of the participants’ perceptual
space. In the perceptual space calculated through multi-
dimensional scaling, we used quadratic discriminant
analysis to search for axes that maximized the separation
of the excerpt classes. We identified three axes that were a
posteriori labelled as ‘slow–fast’, ‘vocal–non-vocal’, and
‘synthetic–acoustic’. We found a high correlation between
the excerpt tempo in beats per minute and the excerpt
projections on the slow–fast axis. A final analysis showed
that the relevance of the factors responsible for the
grouping of excerpt subsets is context dependent.
1. Introduction
It is a common phenomenon for music listeners to detect
similarity between and within pieces of music. Within a
piece of music, listeners spontaneously identify musical
segments with similar functions (e.g. choruses, verses,
bridges), deriving a structure for the summarization or
description of the song. Similarity between pieces is used
by listeners for comparison of one piece to another, for
categorization into styles and genres, and organization of
songs into collections and play-lists.
Music similarity is an ill-defined concept in the
cognitive and perceptual domains because it is context-
dependent (Cambouropoulos, 2009), there is no defini-
tion of which musical dimensions influence listeners’
perception and how music proximity can be objectively
measured (Orpen & Huron, 1992). Nevertheless, in
perceptual experiments, participants can easily decide
without a formal definition how similar two music pieces
are (Chupchik, Rickert, & Mendelson, 1982; McAdams,
Vieillard, Houix, & Reynolds, 2004), and they can do it
consistently (Logan & Salomon, 2001; Pampalk, 2006).
This fact suggests that although listeners’ perception of
music similarity depends on various complex phenom-
ena, such as timbre, rhythm, culture, social context, and
personal history, listeners can, and do, intuitively
interpret the meaning of similarity consistently. When
asked to describe the motivation for their perceived
music similarity, listeners often refer to surface features,
e.g. prominent music elements of a piece of music such as
dynamics, texture, loudness, tempo, and timbre (Lamont
& Dibben, 2001; McAdams et al., 2004).

Correspondence: Alberto Novello, Philips Research, DSP Group, High Tech Campus 36, Eindhoven, 5656 AE Netherlands.
E-mail: jestern@libero.it

DOI: 10.1080/09298215.2010.523470 Ó 2011 Taylor & Francis

 2 Alberto Novello et al.
These findings support the theoretical model proposed
by Deliège (2001) to explain the perception of music
similarity across song excerpts: during the listening
process, music listeners extract musical cues between
music-segment boundaries, unconsciously building a
mental description of each song segment; the musical
surface features are utilized to compare different
segments of the song, and, based on cue proximity,
music similarity is evaluated. It is possible to extend the
validity of Deliège’s conclusions to the case of across-
piece similarity, comparing the surface features of two
different pieces of music. Deliège’s model can then be
applied to predict within- and across-piece similarity.
Two studies support the extension of Deliège’s
hypothesis on the influence of musical surface features
to the case of across-piece similarity (Chupchik et al.,
1982; Eerola, Jarvinen, Louhivuori, & Toiviainen, 2001).
Eerola et al. (2001) found rhythm and pitch to be both
Downloaded by [41.235.86.174] at 12:24 10 July 2011
important factors on perceived similarity between folk
melodies synthesized from MIDI representation. Chup-
chik et al. (1982) performed two experiments to
investigate across-song music similarity between audio
excerpts and found that tempo, dominant instrument,
and articulation were the main musical features used by
participants for their ratings of similarity among jazz
improvisations. In comparisons between Classical, Jazz,
and Pop-Rock excerpts, the most relevant dimension
found was ‘Classical’ versus ‘Contemporary’.
Despite the consensus of several perceptual experi-
ments on the influence of music surface features on
perception of music similarity as hypothesized by
Deliège, there is no agreement across studies on which
music features are relevant in the case of music similarity
(Chupchik et al., 1982; Eerola et al., 2001; Lamont &
Dibben, 2001; McAdams et al., 2004). From the
comparison of the experimental results of a few studies
(Chupchik et al., 1982; Lamont & Dibben, 2001;
McAdams et al., 2004) we hypothesize that this lack of
agreement can be due to the context of stimuli used, i.e.
each individual stimulus subset could be perceptually
organized by a specific set of control variables. Because
of the limited number of genres or songs used in the
perceptual experiments, the experimental results can be
effectively used to verify the impact of individual musical
dimensions on a very particular music context, but have
limited representation of the perceptual space of the
listener in the context of other genres. This fact is
essential for the development of most algorithmic
applications based on music similarity.
Because of the lack of a commonly agreed database of
music similarity for a standard evaluation of the
algorithm performance (Logan, Ellis, & Berenzweig,
2003), several authors based the training and testing of
their applications on different sources (Aucoutourier &
Pachet, 2002; Berenzweig, Logan, Ellis, & Whitman,
2003; Logan et al., 2003) such as metadata annotations
and web-texts, or relying on the delicate assumption that
two songs are similar if they belong to the same artist,
album, or play-list. The use of different test data makes it
difficult to compare performance between algorithms.
Moreover, because all previous sources were not
collected by explicitly asking listeners to rate acoustical
similarity in a controlled experiment, they might not
reliably represent the actual perceived music similarity.
Several studies have run listening experiments to evaluate
algorithms for music similarity comparing participant
results and computer predictions (Aucoutourier &
Pachet, 2002; Herre, Allamanche, & Ertel, 2003; Mirex,
2006). The reported listening experiments are rather
time-consuming, and, using only few participants, have
limited validity as perceptual data. Overall, there is
relatively little attention paid in the computational
domain to the accurate modelling of perceptual music
similarity. In a recent article, Pampalk, Flexer, and
Widmer (2005) suggest the possibility that embedding a
human perceptual and cognitive model into the algo-
rithms could help overcome the performance ceiling
observed recently for the pure feature-based algorithms
(Berenzweig et al., 2003; Logan et al., 2003; Aucoutour-
ier & Pachet, 2004; Pampalk, 2004). Perceptual data on
music similarity collected for a large set of Western
popular music would be beneficial for the verification of
theoretical models, as reference for perceptual experi-
ments, and as training/testing material for algorithmic
applications. Collecting such a large data-base can be a
time-consuming and fatiguing operation for the partici-
pants because of the large number of stimulus compar-
isons required.
To collect such perceptual music-similarity data, we
developed an experimental method optimizing stimulus
coverage, experimental time, and simplicity of the task for
the participant. In this article, we present our methodol-
ogy and the results of a large-scale perceptual experiment
using 78 song excerpts selected from 13 genres of Western
popular music. A major problem in collecting such data is
related to the trade-off between the number of stimuli and
experimental time: even with a small set of stimuli, the
number of necessary comparisons can require long
experiments. In the literature three methods have been
used in perceptual experiments to assess similarity among
auditory objects: pair-rating (Lamont & Dibben, 2001),
pair-ranking (Levelt, van de Geer, & Plomp, 1966;
MacRae, Howgate, & Geelhoed, 1990), and object-
grouping (McAdams et al., 2004). In pair-scaling, the
participant chooses a value of similarity for a pair of song
excerpts on a numerical rating scale. Pair-ranking is an
ordinal procedure that asks participants to rank pairs of
objects depending on similarity. In an object-grouping
task, the participant is presented with a number of stimuli
and has to group them depending on similarity. Two
studies have shown the difficulty for the participants
and the possible data bias related to the widely used
 Perceptual evaluation of inter-song similarity in Western popular music
pair-rating paradigm and proved the easiness and
robustness of an ordinal task such as pair-ranking
(Burton & Nerlove, 1976; MacRae et al., 1990). Although
simple and solid in its conception, the grouping paradigm
could be applied only when the number of stimuli is small
due to the memory demands for the participant.
An advantage of both pair-ranking and pair-rating is
that under particular conditions (i.e. assuming the
symmetrical and transitive properties of stimulus simi-
larity) some similarity measures can be inferred from
previous comparisons. This fact allows the experimenter
to deduce the organization of the perceptual space from a
subset of the whole possible set of comparisons (Levelt
et al., 1966). As often noted in psychology and cognition
(Tversky, 1977; Tversky & Gati, 1982), similarity does not
in general follow a symmetric property: e.g. R.E.M. can
be similar to the Beatles more than the Beatles are similar
to the R.E.M, nor the transitive property, e.g. a pair of
Downloaded by [41.235.86.174] at 12:24 10 July 2011
scissors can be similar to a knife, and a knife similar to a
fork, but the perceptual similarity distance between a fork
and a pair of scissors is likely greater than the sum of these
two similarity distances. Thus, a thorough examination of
similarity would include pairwise comparisons for all
pairs of stimuli in an experimental setup. However, due to
the practical limitations of collecting data for all possible
pairs for a large set of stimuli, it can still be useful to relax
the assumptions of non-symmetry and non-transitivity
and work with partial but balanced block designs in
studying music similarity (Levelt et al., 1966; Grey, 1977;
MacRae et al., 1990).
Because no previous theoretical model advanced
hypotheses on listener concordance in judging music
similarity and previous experiments have investigated
only across-participant concordance with user-tests
involving a small number of participants and stimuli
(Logan & Salomon, 2001; Pampalk, 2006), we conceived
the experimental method to measure the within- and
across-participant concordance with an extended experi-
ment. The within-participant measure of concordance
evaluates the stability of the perception of music
similarity of each participant; the across-participant
measure of concordance evaluates the common percep-
tion of similarity of a group of listeners. A high degree of
within-participant concordance is a necessary property to
reveal whether the perception of music similarity is a
stable phenomenon. A high degree of across-participant
concordance would support the possibility for the
development of a global perceptual model of music
similarity.
Our goal is to identify the most relevant factors
underlying the organization of the complex experimental
data. Such an organized structure can be searched if there
is evidence of its existence from the analysis of participant
concordance. The choice of using a large stimulus-set
implies having an experiment with numerous inter-song
comparisons and results in a relatively large set of data
3
with no direct interpretation. The difficulty of the
analytical task resides in several factors:
. the complexity of representation: data are visualized
using multidimensional plots commonly interpreted
as the participants’ perceptual space, whose dimen-
sions are often not easily defined,
. the complex interaction of the relevant music dimen-
sions: the relevant structural factors controlling the
multidimensional data organization have simulta-
neous global and contextual influence,
. the definition and measurement of similarity: several
approaches can be followed, such as the Euclidean
distance between two items in the multidimensional
perceptual space, or a binary value representing the
membership of an excerpt to a specific spatial region
(e.g. a cluster). In this latter case, however, the
additional problem arises how to define the cluster-
discriminant function.
Several methods have been described in the literature
to visualize and analyse the data collected in a similarity
experiment. The most common is multidimensional
scaling (MDS), which operates on similarity ratings of
song pairs (Chupchik et al., 1982; Eerola et al., 2001;
Lamont & Dibben, 2001; Bella & Peretz, 2005). MDS
helps to reduce the dimensionality of the original data,
converts non-metrical (ordinal) measures into metrical
distances, and calculates the relevant dimensions of the
participants’ perceptual space (Nosofsky, 1992). When
forcing high-dimensional data to be fitted onto a lower
space, the MDS calculation introduces a distortion, and
the distances between items are altered. The distortion is
represented by the RSQ value, which is the squared
correlation between the original distances and the
induced alteration. Furthermore, when a solution is
found, the interpretation of the axes is not always clear as
the axes rotation is automatically chosen to maximize
data variance: it is up to the experimenter, by observa-
tion of the data distribution, to determine the final
rotation of the axes and their contextual interpretation
(Kruskall & Wish, 1978; Young & Lewyckyj, 1996).
Another method to observe stimulus grouping in the
perceptual space is hierarchical clustering (McAdams
et al., 2004): from the relative distance of each item,
groupings are calculated hierarchically, creating a tree of
possible clusters. This method has the advantage of
displaying, in one plot, the whole complexity of the
multidimensional space without distortion, but leaves
to the experimenter the choice of the threshold that
determines the number of clusters and finalizes the
grouping.
In the experiment presented in this paper, we
measured similarity in a constrained environment: using
triadic comparisons of short music excerpts with limited
vocals in order to narrow the focus and the dimensions
 4 Alberto Novello et al.
along which similarity can be judged. In this way we
attempt to limit the asymmetry and non-transitivity of
pairwise similarity rankings in our database. In the
experimental design we use a partially balanced incom-
plete block design to reduce the number of required
comparisons over a relatively large stimulus set. In the
analysis we use the MDS method, which has been used
successfully in the past for other perceptual similarity
tasks, such as timbre (Grey, 1977), music intervals
(Levelt et al., 1966), and scent (MacRae et al., 1990), in
an attempt to reveal the number of dimensions in the
perceptual space. In the data analysis presented here, we
use a combination of scaling and discriminant functions
to gain more insight into the relevant dimensions
underlying participants’ rankings of music similarity. In
this way, we obtain a quantitative measure of the
relevance of control variables in similarity rankings to
distil the data and discover the important factors in
Downloaded by [41.235.86.174] at 12:24 10 July 2011
similarity rankings.
2. The experimental setup
We selected a set of 78 song excerpts in order to verify
and assess the influence of three control variables—
genre, tempo, and primary instrument—on music
similarity rankings. Because of the large set of stimuli,
we applied a high level of comparison reduction in the
experimental design. We used a partially balanced
incomplete block design (PBIBD) consisting of two
nested balanced incomplete block designs (BIBDs) to
reduce the number of comparisons without introducing
distortions and to allow for cross-checking of data
(Levelt et al., 1966; Burton & Nerlove, 1976; MacRae
et al., 1990). To reach a wide variety of participants, the
experiment was conducted on the Internet via a web
interface.
2.1 The block design
A complete block design (CBD) of n stimuli and k items
per trial (with k 5n), consists of all possible sets of k
items selected out of the n total stimuli while avoiding the
possible within-trial permutations (e.g. ABC, ACB,
BAC, etc.). In the case of k ¼ 3 , the number b of trials
in a CBD is given by the formula:
nðn  1Þðn  2Þ
b¼ : ð1Þ
6
In a BIBD, all possible pair-wise comparisons of stimuli
occur l times (Burton & Nerlove, 1976) and every pair of
stimuli is presented equally often in the whole design,
providing an equal amount of information from which to
compute a similarity measure for each stimulus pair.
Thus, if k is the number of stimuli per trial, and n the
total number of stimuli, the total number of trials, b, in a
BIBD is:
lnðn  1Þ
b¼ : ð2Þ
kðk  1Þ
For k ¼ 3 (only) the BIBD reduces the number of
comparisons of the complete design by a factor (n72)/
l. The BIBD method and the choice of the appropriate
reduction factor l has been tested in previous perceptual
experiments. Levelt et al. (1966) used triadic comparisons
and a BIBD for the comparison of musical intervals and
found reliable results with l ¼ 2.
Previous papers (Burton & Nerlove, 1976; MacRae
et al., 1990) tested the reliability of the BIBD data in
comparison to the complete design case for different l
values. Both studies found that a value of l  2 leads
generally to reliable results while the use of l ¼ 1 leads to
distortion of the data.
In our experiment, we generated a PBIBD by nesting
two BIBDs: one to create an incomplete but overlapping
set of genres for each participant, and another to create a
set of triadic comparisons of songs within each genre set.
The outside (genre) BIBD divided the 13 genres into sets
of four for each participant, and the inside (excerpt)
BIBD divided the 24 excerpts (6 per genre) into sets of
triads. The outside BIBD had n ¼ 13 genres in total, k ¼ 4
(quadratic comparisons, each participant examined four
genres) and l ¼ 3 (three participants examined each genre
pair): Equation 2 gives b ¼ 39. This is the number of
participants required to complete the whole experiment
once; we ran the complete PBIBD twice, doubling the
number of participants needed, to collect two sets of
independent data for every triad in the experimental
design. The inside BIBD had n ¼ 24 songs per partici-
pant, l ¼ 2 (every song pair was presented twice in the
participant design) and k ¼ 3 (triadic comparisons):
Equation 2 gives 184 triadic comparisons per participant.
Both BIBDs were computed using R software with the
AlgDesign package (R-project, 2008).
We finally added 10 identical triads to each partici-
pant’s experimental design to evaluate across-participant
concordance, and repeated them to assess within-
participant concordance. Thus, the total amount of
triadic comparisons per participant was 204. For every
participant the design was randomized on triadic
comparisons and on song order inside each triadic
comparison to reduce potential order effects. With 204
triads, normal listening experimental time was estimated
to be about 2.3 h per participant.
2.2 Stimuli
Based on genre classification of the All Music website (All
Music, 2008), we chose 13 genres to cover a large range of
Western music, primarily from popular genres: Afro-Pop,
 Perceptual evaluation of inter-song similarity in Western popular music
Blues, Classical, Country, Folk, Heavy Metal, Hip-Hop,
Jazz, Latin, Pop, Reggae, R&B, and Rock. We selected
six song excerpts from each genre to systematically vary
tempo and primary instrument within each genre.
We chose genre as a selection criterion to obtain a
variable degree of similarity between song excerpts: we
intended to observe if songs within the same genre were
perceived to be more similar to each other than songs
from two different genres, i.e. to test if genre was a
stronger attractor than the other two control variables:
tempo and primary instrument. Tempo and timbre (in
the form of primary instrument) were used in the
selection because they were relevant in the perception
of similarity in the context of previous experimental
studies (Chupchik et al., 1982; McAdams & Matzin,
2001). While tempo is an objective and quantitatively
measurable property, timbre is a perceptual phenomenon
involving several musical dimensions. In the song
Downloaded by [41.235.86.174] at 12:24 10 July 2011
selection of the experiment, we represented the timbre
of each song by the primary instrument in the excerpt;
inside each genre, we selected two excerpts from each of
three primary-instrument categories (vocal, piano, and
guitar). For each primary-instrument category we
selected one excerpt from each of two tempo categories:
slow (5100 beats per minute for a quarter note) and fast
(4140 beats per minute for a quarter note). Inside each
category, we aimed at selecting prototypical and popular
examples, to produce a collection of songs as common
and varied as possible. Regarding timbre, we chose songs
with the presence of drums, consistent with the majority
of Western popular music. We selected only studio
recordings to minimize concert-related uncontrolled
factors, such as audience noise and recording quality.
The songs satisfying the previous criteria were selected
from an expert-composed database or purchased from
the ‘iTunes Store’ (iTunes, 2008). If the selected criteria
were still satisfied, the excerpts were extracted from the
beginning of the chorus as it is typically considered to be
the most representative part of the song. All excerpts
were 15 s long, monophonic, normalized in loudness and
faded in and out over one second. The stimuli were
converted to the MPEG-1 Audio Layer 3 (320 Kb s 71)
file format and were completely downloaded on the
participants’ computer before being played. All selected
excerpts are listed in Appendix A.1
2.3 Participants
We conducted the large-scale experiment on the Internet
because of the large number of participants required. The
experiment was run via connection to a server computer
1
The excerpt database (converted to 192 kbps MP3 format)
of the experiment can be obtained from the author,
Alberto Novello, by sending a request mail to:
jestern77@yahoo.it
5
hosted in the Philips Research Laboratories in Eindhoven
(The Netherlands) between December 2006 and March
2007. We recruited the participants by sending an
invitation with explanation of our experiment through
mailing-lists, on-line forums and universities in different
countries to have variability among the participants. Each
participant was explicitly instructed to test the computer
and audio settings to adjust the listening intensity to a
comfortable level. We asked participants to fill out a
questionnaire to record their musical background, musical
training, and listening conditions. The participants were
59 males and 19 females. Their age ranged from 18 to 72
years and their average age was 28 years. Most participants
declared to have had few years of music practice as
obligatory training in the lower or middle part of the
school system of their respective countries. Assuming that
these early years of practice were not enough to consider a
participant as a musician, we defined participants having
had fewer than four years of musical training as non-
musicians, and participants having had more than six
years of musical training as musicians. According to these
boundaries, 37 participants were considered musicians
and 41 were considered non-musicians. Loudspeakers
were used by 53 participants, while the other 25 used
headphones. The average actual experimental time for the
participants was about 2 h.
2.4 Procedure
The experiment was conducted via a graphical web
interface available in four languages: Dutch, French,
English, and Italian. The first page of the web interface
introduced participants to the purpose and procedure of
the experiment. After acceptance of participation, each
participant was given a brief questionnaire with some
general questions: name, gender, age, years of musical
training, the listening setting during the experiment
(headphones or loudspeakers), and favourite musical
genre. The participants were then presented with the
experiment page and given the following instructions:
‘Please listen to three songs by pushing the buttons next
to the letters, then push MOST for the Most similar pair
and LEAST for the Least similar pair; push the NEXT
button when you are ready to go to the next triadic
comparison’. The three song excerpts of each triad where
presented on the graphical user interface on the corners
of an equilateral triangle to reduce positioning bias. The
participant had to listen to each of them, one at a time,
before ranking them. This procedure was repeated until
the experiment was completed (204 triadic comparisons).
The participants could stop at any time and continue the
experiment at a later time, completing it in as many
sessions as desired. Each participant was asked not to
discuss the experiment with anybody until the experiment
was completed to guarantee independence of data. The
participants received 15 Euros for their participation. No
 6 Alberto Novello et al.

participant reported any problems during the experi-

mental procedure. To assess similarity holistically, we let
the participants spontaneously choose the perceptual
criterion for their rankings and we explicitly did not
provide the participants with any definition of ‘similar’.
Ten participants explicitly asked which dimensions they
had to consider while ranking pair similarity. No
indication was provided. At the end of the experiment,
each participant was asked to report which music
properties were used when ranking similarity.

3. Preliminary results and control for

methodological robustness
3.1 Participant concordance
We measured participant consistency in two ways, as Fig. 1. Within-participant concordance. For each repeated triad
Downloaded by [41.235.86.174] at 12:24 10 July 2011

within- and across-participant concordance. The first
assesses the reliability of each participant, by measuring
the stability of its perception over time, while the second
assesses the common perception of similarity of a group
of listeners. To measure both the within- and across-
participant concordance, we used Kendall’s coefficient of
concordance (Kendall, 1975):
12S
W¼ ; ð3Þ
m2 ðn3  nÞ
where m is the number of participants, n ¼ 3 the number
of song excerpts in a triad, and S is the variance of the
sum of ranks per excerpt:
X
n
S¼  2;
ðRj  RÞ ð4Þ
j¼1
where Rj is the sum of participants’ rankings for the
jth pair of song excerpts, and R is the expected value if
the rankings by each participant were unrelated, i.e. the
null hypothesis (equal to 12 mðn þ 1Þ). In the case of
within-participant concordance, for each participant we
calculated 10 concordance values using the duplicate
rankings (m ¼ 2) on the 10 repeated triads (n ¼ 3). In the
case of across-participant concordance, we calculated
one concordance value for each of the 102 triads (n ¼ 2)
using the rankings of all 36 participants (m ¼ 36) across
each triad.
the value of the Kendall’s coefficient of concordance is
displayed. The horizontal line is the significance level
of concordance at p ¼ 0.05. The mean concordance value for
all participants is 0.82. Appendix B indicates the excerpt-
composition for each triad.
one had very low concordance values and three were very
close to significance.
We calculated the mean within-participant concor-
dance per triad. On all 10 repeated triads participants
were significantly concordant. The two triads that
showed lowest concordance are numbers 2 and 6.
Excluding these two triads from the calculation, five of
the participants that scored low on within-participant
concordance reached the significant level.2 Because
participants 16, 20, and 74 were inconsistent even on
triads on which other participants were highly coherent,
we removed their data from the rest of the analysis. We
calculated a two-sample t-test between distributions of
concordance for the musicians and the non-musicians for
each of the 10 repeated triads. We found no statistically
significant difference. The calculated p values ranged
from 0.08 to 0.67.
3.1.2 Across-participant concordance
The across-participant concordance was significant
on all ten common triads (Figure 2). The signi-
ficance level was calculated using a Chi-squared

3.1.1 Within-participant concordance 2

We assessed within-participant concordance comparing
the rankings on the ten repeated triads common to every
participant’s design (the list of songs is given in
Appendix B). The results are shown in Figure 1. Out of
78 participants, 70 showed a significant level of within-
participant concordance. Among the other participants,
The estimation of the 0.05 significance level for n W values is
computed assuming a normal distribution of the W values
with an expected value of 0.5 and a standard deviation of [1/
(n * 8)]1/2 (ranking three items gives four possible outcomes of
W). In the case of the 10 repeated triads per participant n ¼ 10
and the standard deviation is 0.11, in the case of the within-
participant concordance per triad n ¼ 78 and the standard
deviation is 0.04.
 Perceptual evaluation of inter-song similarity in Western popular music
Fig. 2. Across-participant concordance on 10 repeated triads.
Downloaded by [41.235.86.174] at 12:24 10 July 2011
The crosses mark the Kendall’s concordance value of all
subjects on each of the 10 added triads. The circles mark the
values of concordance for the second presentation of the 10
added triads. The horizontal bar is the significance level of
concordance at p ¼ 0.05. The mean across-concordance value
across the 10 triads is 0.51 + 0.06. Appendix B indicates the
excerpt-composition for each triad.
approximation.3 The two triads which showed the lowest
concordance values are number 2 and 6, the same that
scored low in the within-concordance analysis. We com-
pared across-participant concordance for musicians and
non-musicians on the ten repeated triads using a sign test.
We found no statistically significant difference (p ¼ 0.11).
3.2 Saliency of control variables
We investigated the salience of each control variable used
to select the stimuli (genre, tempo, and primary instru-
ment) on the participant rankings. We used two
approaches: an absolute measure, observing only one
control variable at a time, and a relative measure,
comparing two control variables. In the first case (the
absolute measure), we selected all triads in the PBIBD
with exactly two stimuli belonging to the same class of a
given control variable (e.g. fast–fast–slow for tempo or
piano–piano–vocal for primary instrument) and exam-
ined the number of occurrences in which that pair of
stimuli was selected as the most similar, least similar, or
intermediate. Figure 3 shows the occurrences of each of
the three rankings for each control variable normalized
3
The measure of W is related to the Friedman test statistic,
which can be approximated with a Chi-squared distribution in
the case of more than two judges. The significance level is then
calculated from a Chi-squared distribution with two degrees of
freedom, normalized by the number of participants and degrees
of freedom (Kendall, 1975).
7
by the total number of selected triads in each case: with a
fast-tempo pair (panel (a)), with a slow-tempo pair (panel
(b)), with a same-genre pair (panel (c)), and with a same-
primary instrument pair (panel (d)). In all triad
topologies examined, the pair with stimuli belonging to
the same class of the selected control variable is chosen
more often to be the most similar than to be least similar
or intermediate (p 50.01). Statistical significance was
derived using a bootstrap method in which the complete
collection of similarity rankings was resampled on a triad
basis (Efron & Tibshirani, 1993).
In the second case (the relative measure), in order to
rank the relative effect of each control variable, we
compared control variable saliency inside crossed triads,
i.e. triads that contained two pairs of excerpts belonging
to different control variables (e.g. two stimuli belonging
to fast tempo while one of these belonging to the same
genre of the third stimulus, see template in Table 1). We
selected crossed triads for each combination of two
control variables and counted the rankings for each pair.
The three panels at the bottom of Figure 3 display the
occurrence of ‘most similar’ rankings for each of the two
pairs belonging to the two control-variable categories in
the crossed triads.
When analysing genre and primary-instrument
crossed triads (panel (e)), the pair with stimuli belonging
to the same genre is chosen more often as most similar.
In the case of tempo and primary-instrument crossed
triads (panel (f)), the tempo pair is chosen more often,
but the influence of the two control variables is more
similar than for the conditions presented in panels (e)
and (g). In the case of genre and tempo crossed triads
(panel (g)), the pair with same genre is chosen more often
as most similar. All these differences have statistical
significance (p 50.01) calculated in a two-sample t-test of
the bootstrapped distributions.
3.3 Solidity of the experimental design
To cross-check the nesting-design concordance we
examined the rankings of the pairs for participants who
had the same experimental design. As described before,
we doubled the number of participants in the whole
experiment to have two participants ranking the same
triads. Observing each one of the 39 participant pairs, we
found just one pair whose participants (number 35 and
74) were not significantly concordant. We attribute the
low concordance of the pair to participant 74 who
performed very low in the within-participant concor-
dance and was removed from the rest of the analysis.
3.3.1 Control experiment 1: triadic comparison versus
grouping paradigm
To evaluate the methodological solidity of our experi-
mental design, we compared the experimental results of
 8 Alberto Novello et al.
Downloaded by [41.235.86.174] at 12:24 10 July 2011

Fig. 3. Analysis of the saliency of the three control variables. Panels a to d show the mean number of occurrences of each of the three
possible rankings for each pair in the whole set of triads: a—for the fast pair in the two-fast-song triads, b—for the slow pair in the
two-slow-song triads, c—for the same primary instrument in the two-primary-instrument-song triads, and d—for the same genre pair
in the two-genre-song triads. The three plots at the bottom show the comparison of saliency for each control variable on participants’
rankings. For the two pairs belonging to the two control variables in the crossed triad, we calculated the mean number of occurrences
of most-similar rankings. This operation was performed in the case of e—genre and primary-instrument crossed triad, f—tempo and
primary-instrument crossed triad, and g—genre and tempo crossed triad. In all seven plots we omitted to display the standard error of
the mean that ranged between 0.1% and 2%.

Table 1. Scheme of ‘crossed’ control-variables triads.
Stimulus 1 Stimulus 2 Stimulus 3
Control variable 1 class A A
Control variable 2 class b B
two side experiments: the first using a triadic comparison
paradigm and the second using a grouping paradigm
(MacRae et al., 1990). For both we selected a new
stimulus set with 18 song excerpts (informations on the
selected excerpts are listed in Appendix D). The stimuli
were 10-second excerpts from 18 songs covering nine
genres of Western music, primarily from popular genres:
Blues, Classic, Country, Funk, Heavy Metal, Hip-Hop,
Jazz, Pop, and Rock. For each genre, we chose two
songs, one with a slow tempo (5100 beats per minute for
a quarter note) and one with a fast tempo (4 140 beats
a per minute for a quarter note).
B A total of 36 listeners participated in the triadic
comparisons experiment, of which 18 were considered
musicians, having more than seven years of practical
music training and 18 were non-musicians, having less
than one year of music training. We used these limits in
the participants selection to have a neat separation
between the two groups of participants based on music
practice. Of the participants, 25 were male and 11 were
female, and the average age was 28 years. The experiment
followed the typical triadic comparison procedure de-
scribed above.
 Perceptual evaluation of inter-song similarity in Western popular music
In the grouping paradigm 18 new participants took
part, nine of which were musicians. The average age was
24 years. The participants were presented with a graphical
user interface that showed all song excerpts in the form of
sound icons, randomly placed on the computer screen.
Each participant was asked to rearrange the items into
groups depending on similar characteristics. The partici-
pants were free to choose the number of necessary groups
and were asked to perform the task twice (with at least
one week break between the two sessions).
When the number of items is relatively small and if the
two-dimensional space used to display the objects is not
too different from the perceptual space of the participant,
the grouping method is considered more efficient, user-
friendly and intuitive compared to other methodologies,
such as pair rating and pair ranking (Goldstone, 1994).
We ran this experiment to check if grouping and ranking
provided similar results and to see if the stimulus-
Downloaded by [41.235.86.174] at 12:24 10 July 2011
presentation order (imposed in the case of triad
comparisons, and free in the case of grouping) influenced
the final results.
We then compared the items’ distance derived from
all participants’ data, collected both in the triadic
comparisons and the grouping paradigm. From the
triadic comparisons, we built a 18 6 18 dissimilarity
matrix, assigning two points to every pair ranked as
least similar by participants, one to the intermediate,
and zero to the most similar pair (Levelt et al., 1966).
Because we used a non-parametric multi-dimensional
scaling method (ALSCAL), the specific values assigned
to the three similarity types are not crucial as long as
they are monotonic and rank order is preserved. We
also built an 18 6 18 dissimilarity matrix from the
grouping paradigm using the two sets of experimental
data for each participant, assigning, for each session,
one point to all song pairs that were not grouped
together and a zero value for song pairs grouped
together (MacRae et al., 1990). The values of the two
grouping sessions were added. We compared distances
of the same pairs of song excerpts in the triadic
comparisons and in the grouping paradigm, and found
a statistically significant correlation (r ¼ 0.67, df ¼ 152,
p 50.01). In line with the finding of MacRae et al.
(1990), this result supports the hypothesis that pair
ordering and stimulus grouping are paradigms that
provide correlated results, that can be used for a
common perceptual understanding of music similarity.
Although the grouping procedure is easier for a small
set of stimuli, pair ordering may be preferable with a
large number of stimuli.
3.3.2 Control experiment 2: complete block design versus
balanced incomplete block design
We ran a second control experiment to estimate the
reliability of the reduction of comparisons introduced by
9
a BIBD with l ¼ 2. To evaluate the influence of triad
reduction, 12 new participants performed a triadic
comparison experiment with a complete block design
(CBD) using nine song excerpts (see Appendix C for the
selected excerpts). The CBD was presented in three
laboratory sessions. Ten triads were repeated in each
design to evaluate participant concordance.
All participants showed significant within-participant
concordance on all repeated triads. From the complete
set of participants’ rankings (84 triads), we extracted the
rankings of the triads belonging to the BIBD (24 triads).
We built two 9 6 9 similarity matrices adding the
rankings of all participants: one for the CBD and one
for the BIBD, and calculated the correlation between the
values of the two matrices. A highly significant average
correlation was found between the two matrices
(r ¼ 0.92, df ¼ 36, p 50.01). We also computed the
correlation between the CBD and BIBD matrices built
using the rankings of each participant. In this case we
found again a highly significant mean correlation
between the two matrices across participants
(r ¼ 0.91 + 0.02, df ¼ 36, p 50.01). These results suggest
the reliability of applying a BIBD in substitution of a
CBD in this perceptual context.
4. Exploration of the perceptual space
By combining the similarity-based pair-rankings for each
triad from all participants, we constructed a grand
78 6 78 dissimilarity matrix, in which each cell represents
the similarity between two independent excerpts. We
assigned two points to each song pair that was chosen to
be most similar, zero for the least similar and one for the
intermediate pair and added each participant’s results to
the matrix. In this respect, we tried several combinations
of values and found no relevant difference in the
successive analysis. Because our goal is to preserve the
ranking, changing these values (e.g. proportionally)
would not sensibly alter the perceptual space. Finally,
not all matrix cells were judged the same number of
times; we normalized each cell value by its number of
presentations in the perceptual experiment (which ranged
from a minimum of 12 to a maximum of 48).
In the data analysis presented here, the experimental
data in the form of a 78 6 78 matrix is used through non-
metrical multidimensional scaling (MDS) to estimate the
‘participant perceptual space’, which is a topological
representation of the perceived similarity between songs
for visualization and analysis, with the main objective of
quantitatively investigating the influence of aggregators
such as tempo and primary instrument on the organiza-
tion of the participant perceptual space.
MDS has been used successfully in the past for other
perceptual similarity tasks, such as timbre (Grey, 1977),
music intervals (Levelt et al., 1966), and scent (MacRae
 10 Alberto Novello et al.
et al., 1990). To narrow the focus and the dimensions
along which similarity can be judged, we measured
similarity in a very constrained environment (triadic
comparisons, short excerpts, limited vocals, limited
control variables: genre-tempo and primary instrument).
Our intention is to test from different perspectives our
hypotheses that specific attributes of our stimuli are used
by listeners in similarity rankings. Using several techni-
ques, we search for a clear ordering of our attributes
along any arbitrary dimension in the MDS space, to
provide support for our hypothesis.
4.1 Analytical methodology
To assess the influence of each control variable on the
distribution of song excerpts in the perceptual space, it is
first necessary to have a representation of the topology of
such a space, i.e. compute the coordinates for each song
Downloaded by [41.235.86.174] at 12:24 10 July 2011
excerpt. MDS is a set of statistical techniques largely
used in literature for visualization of complex data to
reduce data dimensionality (Shepard, 1962a, 1962b;
Chupchik et al., 1982; Eerola et al., 2001; Lamont &
Dibben, 2001). MDS takes as input a matrix of pairwise
item similarity scores (such as the 78 6 78 matrix of
perceptual data) and assigns a location of each item in a
lower-dimensional space. MDS can transform metrical
or non-metrical data, such as as rankings, into coordi-
nates. We used the ALSCAL algorithm (Young &
Lewyckyj, 1996) for the non-metrical MDS analysis.
The data distortion introduced by an MDS algorithm
when attempting to reduce the number of dimensions is
represented by the Pearson’s correlation coefficient
between the dissimilarity matrix (input) and the distances
between coordinates in the MDS-calculated space (out-
put). Every violation of the ordering of the data in the
dissimilarity matrix is a distortion of the original data
and decreases the Pearson’s correlation coefficient value:
the higher the Pearson’s correlation coefficient, the more
reliable the representation of the original data. One
important parameter that the experimenter has to
carefully define in the MDS computation, is thus the
number of dimensions to keep a low space distortion and
easiness of visualization of the experimental data.
We first investigated the variation of the Pearson’s
correlation coefficient value with the number of dimen-
sions (Shepard, 1962a, 1962b) to decide on the optimal
number of final dimensions in the MDS output. We then
extracted the coordinates of each stimulus in the multi-
dimensional space that we interpret as the participants’
perceptual space. One difficulty of the MDS method
resides in the fact that the new coordinates are referred to
a set of axes chosen by the algorithm to maximize the
data variance but may have no explicit connection to
physical properties of the stimuli; it is up to the user, by
observation and extra analysis of the data distribution, to
determine the final rotation of the axes and their
contextual interpretation (Shepard, 1986). Because of
this fact, in our analysis we used non-metrical MDS to
scan the resulting space for any potential dimension
correlates, not assuming any particular dimension
returned by the MDS algorithm.
If the perceptual space requires high dimensionality
for display, genre topology and hierarchical clustering
are complementary perspectives for a preliminary sim-
plified representation of the perceptual space: the first
represents the spatial distribution of clusters and the
second represents the distance among individual ex-
cerpts. A combination of both techniques provided
in our case an efficient preliminary approach for a
qualitative investigation of the features of the perceptual
space.
To quantitatively verify the existence of features
observed in the previous analysis stages and measure
the influence of the control variables, tempo and primary
instrument, used for the stimuli selection on the
organization of the perceptual space, we use quadratic
discriminant analysis (QDA). QDA is a classification
method used to separate two or more classes of objects
through the use of quadratic functions of the objects’
features (Duda, Hart, & Stork, 2001). In the present
article, we employ QDA in a novel fashion using the
control-variable assigned to each song excerpt (genre,
tempo (slow and fast), and primary instrument (guitar,
piano, vocal)) as classification classes and the MDS
coordinates for each song excerpt as features. In this
way, we obtain a quantitative measure of the influence of
each specific control variable, expressed as a measure of
spatial separation of stimulus classes, and the axis
interpretation, as the control variable correlated with
the separation. Based on our observation that the
stimulus classes in the MDS space do not appear to be
of equivalent size, we chose to use QDA, rather than a
linear approach where it is assumed that all classes have
equivalent variance.
4.2 Dimension calculation
As a first step we generated a 2D MDS estimate of
the perceptual similarity space. The results show a low
degree of correlation between MDS representation and
the data points (Pearson’s correlation coefficient of
0.63). The resulting plot of excerpts in this 2D space
(Figure 4) shows the typical circular shape of a data
set whose dimensionality is under-represented. It is
clear from this analysis that the similarity data
requires more than two dimensions to be modelled
accurately.
We calculated the plot of Pearson’s correlation
coefficient value versus the number of dimensions
(Figure 5), to estimate the number of dimensions required
to accurately model the participants’ perceptual space.
Because the Pearson correlation coefficient saturates at
 Perceptual evaluation of inter-song similarity in Western popular music 11

Fig. 4. Two-dimensional representation of the perceptual space

of the 78-song excerpts. The points represent the position of Fig. 5. Pearson’s correlation coefficient value versus the number
Downloaded by [41.235.86.174] at 12:24 10 July 2011

songs in the perceptual space computed in the MDS calculation of dimensions: the points represent the Pearson correlation
(as performed by the ALSCAL algorithm (Young & Lewyckyj, coefficient value corresponding to the final number of dimen-
1996)). The 13 different markers represent the belonging to the sions chosen for the MDS calculation. The Pearson correlation
13 genres used in the stimulus selection. The number next to coefficient saturates at a relatively high value around 0.81.
each song refers to the songs properties reported in the
Appendix A. In the MDS calculation of this plot, the Pearson
correlation coefficient has a value of 0.6.

six dimensions with a value around 0.81, we chose six

dimensions to represent the participants’ data. Applying
the ALSCAL algorithm on the 78 6 78 dissimilarity
matrix, we obtained the coordinates for each song excerpt
in this six-dimensional space.

4.3 Genre topology

When the perceptual space requires high dimensionality
(i.e. more than three dimensions), its visualization might
become complex; in our case, we additionally reduced the
dimensionality of the representation in a controlled
manner to achieve a preliminary visualization of the
perceptual space by computing a two-dimensional dis-
play of the ‘genre topology’. We first computed the 13
genre centroids by averaging the coordinates of the six
song excerpts within each genre. We then constructed a
Fig. 6. Greyscale image of the Euclidean distances between
13 6 13 genre dissimilarity matrix from the Euclidean genre centroids. On the x- and y-axis the labels refer to the 13
distances between genre centroids. genres of the stimuli. Each square represents how close two
Figure 6 displays the dissimilarity matrix between the genre centroids are in the six-dimensional perceptual-space. The
13 genre centroid as a greyscale image. Complete darker the square the closer the centroids. The legend on the
similarity is represented as black, complete dissimilarity side gives a magnitude reference for the relative Euclidean
as white. We observe a dark diagonal, which is expected distances in the six-dimensional space.
as songs belonging to the same genre tend to be perceived
as similar to each other. Some genre, however, present a
less than perfect intra-genre similarity, e.g. Latin, whose Hop, and the relatively large distance between Classical
songs are perceived similar to other genres. The plot and Rock, R&B, and Reggae. However, with this plot we
shows some musicologically-expected results, such as the cannot deduce the distribution of the genres in the
proximity of Country and Folk, and of Electro and Hip- perceptual space.
 12 Alberto Novello et al.
To compute the organization of the genres in the
perceptual space we use the distances between genre
centroids as input for the non-parametric MDS algo-
rithm (ALSCAL): two dimensions provided an accep-
table compromise between the Pearson correlation
coefficient value (0.86) and ease of display. Figure 7
displays the topology of the 13 genre centroids.
Figure 7 confirms the results found in Figure 6 in
terms of distance and proximity of genre centroids, and
adds information on their relative position. In the plot
Reggae is positioned half-way between Afro-Pop and
Rock, which makes sense musicologically. We drew an
axis in the figure labelled ‘Synthetic vs. acoustic’ to
illustrate the distribution along the axis of genres using
predominantly acoustic instruments and those using
predominantly synthesized sounds: electronic drums,
distorted guitars, synthesizers and effects. Classical,
Latin, Jazz, Country, Folk are positioned toward the
Downloaded by [41.235.86.174] at 12:24 10 July 2011
acoustic side of the axis, while Electro, Hip-Hop, and
Rock are positioned toward the synthetic side. This
topology suggests that the acoustic-synthetic timbral
quality is a relevant factor in listeners’ judgments of
music. The genre topology has the advantage of reducing
the complexity of the output data by displaying genre
centroids in two dimensions with a relatively high
correlation coefficient (0.86) at the price of losing
information of the excerpt-spread within each genre.
Fig. 7. Genre topology: two-dimensional MDS plot, calculated
from the 13 6 13 dissimilarity matrix constructed from the
relative distances of the 13 genre centroids (Pearson correlation
coefficient ¼ 0.86). The arrows drawn represent the separation
between prevalently acoustic and synthetic genres. The values
on the axes refer to the point distances in the two-dimensional
MDS-space.
4.4 Clustering interpretation
To observe the grouping of song excerpts in the
participants’ perceptual space without losing the infor-
mation of each individual song, we calculated the k-
means hierarchical-clustering from the six-dimensional
coordinates of all song excerpts. In hierarchical cluster-
ing, a series of partitions of the data takes place,
depending on the distances between items. The partition-
ing runs hierarchically creating a tree of possible clusters.
The limit of this approach is the positioning of song
excerpts in the space: the relative distance between
stimuli is visualized but not their spatial distribution.
Figure 8 displays the dendrogram calculated from the
hierarchical clustering of the distances in the six-
dimensional space between the song excerpts.
Our first observation from the dendrogram is the
spread of each genre’s excerpt set. In the case of Folk, for
example, the excerpts are scattered in the two farthest
clusters (at the top and bottom of Figure 8): a relatively
large distance is thus needed to group them all together.
On the other hand, in the case of Classical, we can use a
smaller distance to group all excerpts. The genre-
topology plot and the hierarchical-clustering dendro-
gram provide complementary information and together
help distinguish different distribution types: Folk and
Country centroids are near each other in the genre
topology plot, but their song-position spread in the
dendrogram is quite large, while the Classical centroid is
isolated from other genres in the topology plot, and the
songs are tightly clustered in the dendrogram. The genre
topology and the clustering dendrogram might be
integrated in one analytical method for the visualization
of the perceptual space. In such an approach, one could
normalize the Euclidean distance between two genres
(from the genre topology) by an estimate of the metrical
spread of the two genres (from the dendrogram) to
display the overlap of two genre clusters while keeping a
relatively low dimensional space. Another observation
from the dendrogram in Figure 8 concerns the char-
acteristic of clusters. At the highest hierarchical level
(right most in the figure), the clusters are split into two
groups differentiated by the tempo and rhythm of the
song excerpts they contain; the bottom-most cluster
contains mainly excerpts belonging to the slow-tempo
class, while the top-most cluster contains excerpts
belonging to the fast-tempo class or slow songs belonging
to rhythmically complex genres, such as Afro-Pop, Hip-
Hop and Reggae. As a measure of cluster separation, we
calculated the discriminability, or d 0 of the distribution of
BPM values of the song excerpts for the two clusters (see
Appendix A for relative values):
difference between means m 1  m2
d0 ¼ ¼ ; ð5Þ
spread ðs1  s2 Þ1=2
 Perceptual evaluation of inter-song similarity in Western popular music 13
Downloaded by [41.235.86.174] at 12:24 10 July 2011

Fig. 8. Dendrogram of the hierarchical clustering of song excerpts. On the y-axis the individual songs are represented by: the genre,
followed by two subscripts for the fast- (F) and slow- (S) tempo class, and for the guitar- (G), piano- (P), and vocal- (V) primary-
instrument class; the excerpt reference number corresponds to Appendix B. The length of the horizontal lines indicates the distance
between two clusters in the six-dimensional space derived through MDS. At the highest hierarchical level (right-most in the figure) the
clusters are split into two groups: the fast-rhythmically complex group (top of the figure), the slow group (bottom of the figure). The
vertical line is the threshold chosen to separate the ten sub-clusters, six of which are labelled in the figure.

where m1 ¼ 150 BPM and m2 ¼ 86 BPM, the means of the
two tempo distributions and s1 ¼ 58 BPM and s2 ¼ 28
BPM, their standard deviations. This results in a d 0 ¼ 1.5,
which is greater than the value (1.0) typically used as
the threshold for perceptual discriminability (Green &
Swets, 1966/1988). A double sample t-test confirmed the
statistically significant separation of the two distributions
(p 50.01).
From the analysis of the composition of the two main
clusters, we set a threshold to distinguish ten sub-clusters.
The excerpt characteristics inside each sub-cluster sug-
gests a preliminary labelling for six of them (from top to
bottom in Figure 8): Instrumental fast, Vocal fast, Vocal
rhythmically complex (Afro-Pop, Reggae, Hip-Hop),
Classical, Instrumental slow, Vocal slow. Although still
unverified at this stage, this labelling suggests the possible
interpretation of some of the MDS axes: we see in fact
excerpt clustering depending on tempo (slow–fast), genre
(Classical, rhythmically complex genres), and primary
instrument (vocal versus non-vocal music).
 14 Alberto Novello et al.
4.5 Identification of dimensions
We employed quadratic discriminant analysis (QDA) to
quantitatively verify the interpretations drawn from
hierarchical clustering and genre topology mapping,
and to find a new set of axes in the perceptual space
that could maximize the variance of each control
variable, expressed as spatial separation between stimuli
stemming from different classes of a control variable. In
our QDA computation, we employed the tempo and
primary-instrument classes assigned to each song excerpt
as classification classes, and the six-dimensional coordi-
nates of each song excerpt as features. The QDA training
process computed the best quadratic discriminant func-
tion to separate the stimuli of each class. In addition, our
QDA implementation provides an estimate of classifica-
tion performance as percentage correct, and a weighting
matrix that can be used to estimate the relative
Downloaded by [41.235.86.174] at 12:24 10 July 2011
importance of each dimension (feature) separating the
excerpts into each class (control variable). Thus, QDA
was used to determine which MDS axes were most
relevant in maximizing the separation between the two
classes of a given control variable: for tempo, the
separation of fast and slow pieces; for primary instru-
ment, the separation of vocal and non-vocal (guitar and
piano excerpts were grouped together); and for the
separation of synthetic and acoustic excerpts. In the
general case, more than one MDS axis is relevant in
the maximization of separation of stimuli classes for a
given control variable: a linear combination of the
relevant MDS axes provides then a new axis candidate
that alone best separates the classes.
In the QDA analysis, to verify the separation of
synthetic versus acoustic genres hypothesized in the
observation of the genre topology, we added a new post-
hoc control variable called ‘Synthetic-acoustic’ manually
labelling the songs one by one: song excerpts containing
primarily timbres of distorted or deeply effected guitars,
electronic drums and synthesizer pads were assigned
to the synthetic class, while song excerpts using
acoustic instruments or unprocessed amplified instru-
ments (clean) were assigned to the acoustic class. In
Appendix A the reader finds the assigned class for each
song excerpt.
In the case of the slow and fast classes, the MDS
output was found to be already optimal in the QDA
analysis: one of the six MDS coordinates could alone
produce the highest separation between stimuli classes
with a correct classification mean of 82.8 + 5.7%. In the
following part of this article, we will refer to the axis
identified by this coordinate as the ‘slow–fast axis’
because of its salience in separating the tempo classes.
Because the calculation of the QDA performance
relies only on the number of stimuli correctly classified,
the d 0 computation adds complementary information
because it estimates the classes discriminability using the
distributions of the two classes of song positions on the
new axis. In the case of the slow–fast axis, we found
d 0 ¼ 2.0. This value is close to the discriminability we
calculated using the BPM value of each excerpt belong-
ing to the two main clusters in the hierarchical clustering.
To further investigate the slow–fast axis, we calcu-
lated the correlation between the annotated BPM values
of each song excerpt and the excerpt position on the
slow–fast axis. We found a significant correlation
(r ¼ 0.66, df ¼ 77, p 50.01) between the logarithm of
the BPM of each excerpt and the position on the
selected axis, indicating the presence of a BPM-based
excerpt-sorting in the perceptual similarity space of the
participants.
In the case of the primary-instrument control variable,
we first analysed the separation between stimulus
position and the three primary-instrument classes: piano,
guitar and vocal. In this case three out of the six MDS
coordinates were necessary to create the best stimulus
separation, with a correct classification of 75.2 + 7.9%.
The highest confusion occurs between piano and guitar
classes with an average misclassification of 23.2%.
Grouping piano and guitar classes in one new class
named ‘non-vocal’, and comparing vocal and non-vocal
classes with QDA improved classification performance to
95.9 + 2.4%. In this new case a combination of just two
MDS coordinates is sufficient to achieve optimal
classifier performance.
These results suggest that in the presence of vocal
stimuli, the timbre difference between excerpts with
guitar and piano as primary instrument is not a strong
factor in determining music similarity perception. We
determined the axis that best separates vocal and non-
vocal classes as a linear combination of the two MDS
coordinates sufficient to achieve optimal classifier per-
formance. In the following part of the article, we will
refer to this new axis as the ‘vocal–non-vocal axis’. We
calculated the discriminability of the two distributions of
song-excerpt positions on the vocal–non-vocal axis and
found a d 0 ¼ 2.3.
In the case of synthetic–acoustic timbre classes, a
combination of two MDS coordinates was necessary and
sufficient to achieve the top performance of the classifier:
97.2 + 1.3%. One of these two dimensions was the
tempo axis, used previously for the separation of slow
and fast classes. We defined a new axis, the ‘synthetic–
acoustic axis’, as a linear combination of these two MDS
coordinates. Because of this definition, the synthetic–
acoustic axis is not orthogonal with respect to the slow–
fast axis, with a 368 separation between the fast and the
synthetic direction and between the slow and the acoustic
directions. We calculated the discriminability of the two
distributions of song-excerpt positions on the synthetic–
acoustic axis and found d 0 ¼ 2.4.
Figure 9 displays the projection of the MDS positions
of the 78 song excerpts on the newly defined axes: the
 Perceptual evaluation of inter-song similarity in Western popular music 15

slow–fast, the vocal–non-vocal and synthetic–acoustic
axes. Different symbols represent excerpts belonging to
different control variable classes. All three plots outline
the clear clustering of songs on each of the three
calculated axes. While Figures 9(a) and (c) show a
homogeneous distribution of songs, Figure 9(b) shows
the song positions distributed in a more diagonal
fashion. This distribution reflects the non-orthogonality
of the synthetic–acoustic axis and the slow–fast axis, the
consequence of the definition of the synthetic–acoustic
axis as a linear combination of the slow–fast axis and one
of the axes resulting from the MDS calculation.
From our stimulus selection, we are not in a position
of hypothesizing if the distribution of songs along the
diagonal of Figure 9(b) could be a general behaviour: one
cause could be that the majority of songs using synthetic
timbres have fast tempi, and the majority of slow songs
use acoustic timbres; however, the diagonal distribution
Downloaded by [41.235.86.174] at 12:24 10 July 2011
stimuli). Further research is needed to test the presence of
such an effect connecting timbre and tempo in a larger set
of stimuli.
4.6 Contextual dependency of similarity factors
Although the QDA analysis showed the global influence
of tempo and timbre (the latter in the form of two axes:
vocal–non-vocal and synthetic–acoustic), it is reasonable
to suppose the magnitude of the influence on perceived
music similarity of each possible musical dimension to be
context dependent (Cambouropoulos, 2009). To analyse
this effect, we iteratively selected three genres (18
excerpts); from all experimental triads, we selected those
that contained only stimuli belonging to the defined
subset, and used their participant rankings (assigning
two points to the least similar pair, one to the
intermediate and zero to the most similar) to build an

in Figure 9(b) might also be caused by the unequal
numbers of stimuli in the synthetic and acoustic classes,
which cannot be balanced because of the a-posteriori
definition of the control variable: there are fewer slow-
synthetic excerpts (11 stimuli) than fast-synthetic ex-
cerpts (18 stimuli) and there are fewer fast-acoustic
excerpts (21 stimuli) than slow-acoustic excerpts (29
18 6 18 dissimilarity matrix representing the relative
distances between all stimuli in the subset. We then used
MDS to compute the two-dimensional representation of
the perceptual space of the specific stimulus subset.
In Figure 10(a), we show the MDS plots of data from
18 excerpts selected from three rather different genres:
Afro-Pop, Classical, and Rock. We observe three tight

Fig. 9. Projection of the six-dimensional coordinates of the 78 song excerpts on the three axes selected from the QDA analysis. The three
panels show two-dimensional representations for each pair of axes: a) Primary instrument versus tempo axes, b) Synthetic-acoustic versus
tempo axes, c) Synthetic-acoustic versus primary-instrument axes. In the three plots, to outline the clustering, song excerpts belonging to
different control variable classes are represented with different symbols explained in the legends.
 16 Alberto Novello et al.

genre-dependent clusters. Substituting Afro-Pop with
Latin, in Figure 10(b), we observe again a genre-based
clustering and the emergence of a tempo separation
within genres. Figure 10(c) displays the MDS results for
the excerpts from Afro-Pop, Country, and Latin genres.
In this case, the genre separation is less evident, especially
for the Latin and Afro-Pop excerpts; on the x axis, we
observe a global effect of tempo separation with slow
excerpts towards the negative values of the x axis and fast
excerpts towards the positive values; on the y axis, we
interpret the genre distribution to be influenced by
rhythmic complexity: relatively simple and steady rhyth-
mic structures (prevalent in Country) towards positive
values of the y axis and more rhythmically complex
structures (prevalent in Latin and Afro-Pop) towards
negative values of the y axis. The musicological proximity
of Latin and Afro-pop supports this interpretation.
The results shown in Figure 10 support previous
Downloaded by [41.235.86.174] at 12:24 10 July 2011
as Rock, Classical and Afro-Pop, the genre clustering
might mask the effect of other less salient musical
dimensions such as tempo or timbre. On the other hand,
in the case of excerpts selected from less musicologically
distant genres, e.g. Folk and Country, or Latin and Afro-
Pop, the effect of genre might be less strong and the effect
of tempo, rhythm, and timbre can have a stronger
influence on the perceived similarity. The effect of
context can be linked to two different causes. Consider-
ing the lower triad level, changing one song can alter the
perception of the triad: if all three songs have slow
tempo, for example, tempo does not play a strong role; if
we have one fast song and two slow songs, then the effect
of tempo on similarity is different. However, this
perceptual ‘switch’ can be due to the listeners not making
use of tempo in a specific context, or because there is not
enough variability of tempo inside the stimuli. In
conclusion, from the previous analysis it is not clear

experimental hypotheses (Lamont & Dibben, 2001;
Eerola & Bregman, 2007) suggesting that factors
responsible for the grouping of excerpt-subsets are likely
to be context dependent. In particular it seems reason-
able to hypothesize that in the presence of excerpts
selected from three musicologically distant genres, such
whether the observed variation of the relevance of the
control variables with the stimulus subset reflects a
context-dependent perception of the participants, or that
in a subpart of the perceptual space some control
variables have less variation causing one dimension to
emerge as dominant.

Fig. 10. Example of different contextual-dependent factors in the interpretation of excerpts-subsets grouping: a) Classical/Latin/Rock,
excerpts are grouped by genres; b) Afro-Pop/Classical/Rock, we observe the influence of tempo in the distribution (along the drawn
axis); c) Afro-Pop/Country/Latin, we interpret the x axis as ‘Tempo’, and y axis as ‘Rhythmic complexity’. Each marker identifies
excerpt positions of the genres in the legend. The letters identify the control variable classes: ‘F’ for fast, ‘S’ for slow, ‘V’ for vocal, ‘G’
for guitar, and ‘P’ for piano.
 Perceptual evaluation of inter-song similarity in Western popular music
5. Discussion
Despite the explicit choice of not providing the partici-
pants with a definition of similarity, to control the
perceptual dimensions used by the participant in the
ranking task, the experimental results show high
participant concordance, suggesting that listeners do
interpret similarity consistently and can perform a
similarity task. When asked which music properties were
used in ranking similarity, participants often referred
indirectly to the timbre properties.
We described an experimental method that assesses
similarity perception of Western popular music using
triadic comparisons of song excerpts to present the
participants with a low-complexity task, and partially
balanced incomplete block design (PBIBD) to reduce the
number of stimulus comparisons with the consequent
possibility of extending the stimulus set. The listening
Downloaded by [41.235.86.174] at 12:24 10 July 2011
experiment was conducted through the Internet and used
78 music excerpts and involved 78 participants with
various music backgrounds.
The initial analysis of the experimental results was
conducted to assess the existence of commonly-perceived
inter-song music similarity and test the solidity of the
methodology used. We found significant within-partici-
pant concordance for 90% of participants and significant
across-participant concordance for 100% of the tested
triads. The high within-participant concordance con-
firmed the existence of a clear and stable interpretation of
music similarity for most listeners. The high across-
participant concordance suggests the existence of an
underlying common perception of music similarity across
listeners, which could be formalized into a theoretical
model able to represent and predict the perception of
music similarity. Despite the different experimental
contexts, these results support the findings on across-
participant concordance of the two user tests run by
Logan and Salomon (2001) with 20 participants and
Pampalk (2006) with 25 participants. Our study extends
their conclusions on the existence of a common percep-
tion of music similarity to a large set of songs and genres
of Western popular music, with a larger number of
participants and with several cross-checks. In the large-
scale experiment we did not find any statistically
significant difference between musicians and non-musi-
cians in within- and across-participant concordance. This
result is in line with previous studies (on different song
databases) that found little or no statistically significant
difference between musicians and non-musicians in the
dimensions used for their judgments of music similarity
(Chupchik et al., 1982; Lamont & Dibben, 2001;
McAdams et al., 2004).
We tested the robustness of the experimental method
with two control experiments. The high correlation
found comparing the similarity rankings of a complete
block design against the rankings of a balanced
17
incomplete block design (BIBD) supports the reliability
of reducing the number of triads with l ¼ 2 in a
perceptual experiment as suggested by previous literature
(Levelt et al., 1966; MacRae et al., 1990). Results
obtained using a grouping paradigm and triadic compar-
isons in a BIBD showed high correlation, confirming the
solidity of our methodology and the interchangeability of
both paradigms as suggested by MacRae et al. (1990).
The efficiency of our experimental design and low
complexity of the task for the participants is indicated by
the actual average experimental time per participant:
about 2 h (based on informal feedback from a number of
participants). The high within- and across-participant
concordance of the results confirm the inner coherence
and robustness of the experimental method.
An exploration of the control variables used in the
selection of the stimulus material showed significant
saliency and a hierarchical degree of impact on partici-
pants’ rankings in our stimulus context: genre 4
tempo 4 primary instrument. This hierarchy is in
agreement with the findings of previous studies that used
different experimental methodologies and stimulus con-
texts (Chupchik et al., 1982; Lamont & Dibben, 2001;
McAdams & Matzin, 2001), and extends the validity of
the previous results to other genres of Western popular
music. Chupchik et al. (1982) found that tempo, primary
instrument, and articulation were the main musical
features used by participants for similarity pair-rating
of Jazz improvisations. In a second experiment using
pair-rating on excerpts selected from three different
genres, the most relevant dimensions found were genre
and tempo. McAdams and Matzin (2001) used a group-
ing paradigm and excerpts of contemporary musical
material and found that tempo and timbre influenced
participants’ judgments.
The predominance of genre in influencing partici-
pants’ judgments is not surprising: genre is a collection
of attributes encompassing several musical dimensions.
Genre is in fact not necessarily orthogonal to primary
instrument and tempo: e.g. Heavy Metal can be
identified by the sound of distorted guitars, and
Heavy-Metal songs tend to have high tempi. Given the
complexity of genre, it seems necessary to extend the
validity of our conclusions by exploring each genre in
detail with a specific experiment aimed at determining
relevant dimensions inside each genre: the six songs per
genre that we selected are not sufficient in our opinion to
cover the musical variety of a musical genre. Further-
more it seems reasonable to expect that the relevance of
the control variables might vary for different genres.
Between the two other control variables, tempo can be
objectively and quantitatively defined by the BPM value
on the quarter note. Timbre, on the other hand, is a
perceptual phenomenon, and can be described using
several musical dimensions, such as frequency spectrum,
music instruments in the piece, and recording techni-
 18 Alberto Novello et al.
ques. Our choice of representing timbre by the pre-
dominant instrument is limited and would need further
investigation, for which the same experimental design
could be used.
To quantitatively explore the perceptual space of the
participants, we used a combination of numerical and
analytical methods. Through multidimensional scaling
(MDS), we found that six dimensions were optimal to
represent the participants’ perceptual space obtained
from their similarity rankings in our experimental
context. By using QDA to model the excerpt positions
in the participants’ perceptual space, we selected and
labelled two axes that maximized the discriminability
(and the classifier performance) between the two excerpt
classes of each control variable: ‘slow–fast’ (d 0 ¼ 2.0) and
‘vocal–non-vocal’ (d 0 ¼ 2.3). Interpretations based on the
hierarchical clustering and genre topology analyses
suggested the influence of synthetic versus acoustic
Downloaded by [41.235.86.174] at 12:24 10 July 2011
timbres on the perceived similarity. We defined a
posteriori a third control variable manually assigned
based on excerpt timbre, that provided a third relevant
axis: ‘synthetic–acoustic’ (d 0 ¼ 2.4). The high correlation
(r ¼ 0.66 , df ¼ 77, p 50.01) between projected positions
of excerpts on the slow–fast axis and the logarithm of
each excerpt BPM value adds further evidence that the
selected slow–fast axis indeed relates to tempo. Perhaps
because of the complexity of timbre perception, we could
not find a unique objective variable to perform a similar
measure of axis-correlation in the case of the vocal–non-
vocal, and the synthetic–acoustic axes.
It can be argued that the synthetic–acoustic property
represents just a form of genre similarity, because of the
presence of genres such as electronica or classical that
have the majority of songs in one of the categories.
However our stimulus selection also comprises genres in
which the songs are split into the two classes, such as
R&B and Jazz. Further research is needed to verify the
influence of this ad hoc dimension.
Plots obtained projecting the MDS excerpt positions
on the three selected axes show clean clustering,
confirming the relevance of the chosen control variable
on participants’ rankings. These results are in line with
previous experimental studies that found a qualitative
influence of tempo, timbre and genre on participants’
judgments using different experimental contexts (Chup-
chik et al., 1982; McAdams et al., 2004). Our analyses
extend their validity to a larger set of stimuli and genres
of Western popular music, with quantitative verification
through different analytical tools. The results are
consistent with the hypothesis proposed by Deliège’s
‘cue abstraction’ model: participants determine excerpt
similarity by extracting relevant music cues during the
listening process, and extend the theoretical framework
selecting the most salient cues: tempo, and primary
instrument in the form of vocal–non-vocal and acoustic-
synthetic.
The final analysis showed that factors responsible for
the grouping of excerpt subsets are likely to be context
dependent. However, our experimental design does not
allow us to determine whether the varying relevance of
different music dimensions is due to the different
cognitive weight that they assume in the mind of the
listener, or if it is due to the limited variability of a
specific dimension within the stimuli in a region of the
perceptual space. In the song selection of our experi-
mental design, we used prototypical songs from several
genres to extend the context of our results and find
common dimensions that can be used in the majority of
applicative scenarios, i.e. a database with different genres
of Western popular music. However, because music
similarity is context-based, these results may not be easily
extended to the case of specific music databases: for
example, within a collection of jazz music, the perceptual
dimensions defining similarity may have completely
different weights from the ones found in this study.
Despite all verification used in the experimental and
analytical methodology, because of the subjectivity of
our song selection, it is not certain also that the
perceptual space we found will be similar in another
experiment using the same number of genres and songs
but selected from a different database by another
experimenter. Future research using a similar paradigm
but different song sets is needed to verify how much the
present results can be generalized to predict inter-song
similarity using several genres of Western popular music.
This observation about context raises the problem of
how to formally integrate the global and contextual
influence of each control variable in a theoretical model
or algorithmic application. Although it seems difficult to
integrate the various contextual factors in a generally
valid set of similarity descriptors, these results suggest
that algorithms should take into account contextual
information in order to provide results able to simulate
the judgment of human listeners. Future research might
test the validity of the similarity distance proposed by
Krumhansl (1978), which considers the perceptual
distance to be dependent on the local contextual density
of objects.
Our analyses leave some unanswered questions, show
the experimental limitations of the methodological
approach, and provide possible suggestions for future
experiments. While we identified and labelled three axes
in the six-dimensional space (slow–fast, vocal–non-vocal,
and synthetic–acoustic), we found no interpretation for
the three remaining dimensions. Genre and timbre, being
multidimensional concepts, could involve other music
aspects, such as texture, dynamics, that might explain the
interpretation of the three axes. The choice of control
variables, across which we distribute stimuli, is a pre-
experimental limiting factor. A follow up experiment
selecting a larger number of well-chosen control variables
(e.g. rhythm complexity, timbre brightness, harmonic
 Perceptual evaluation of inter-song similarity in Western popular music
stability, etc.), could investigate the interpretation of the
unexplored dimensions using similar experimental and
analytical methodology.
In conclusion, the combination of experimental and
analytical methods presented here provided a quantita-
tive estimation of the saliency for three musical
dimensions that have a statistically significant influence
on similarity perception on a large set of songs of
Western popular music: song tempo, presence/absence
of vocal parts, and use of synthetic/acoustic timbres.
Together with the previously reported influence of
genre, these results can be used to estimate the optimal
weight of the different music dimensions obtained
through music descriptors such as metadata labels or
audio features to finally derive a formal model for
music similarity incorporating perceptual information in
its predictions.
Downloaded by [41.235.86.174] at 12:24 10 July 2011
Acknowledgements
This work is performed as part of a Marie Curie Early
Stage Training grant (MEST-CT-2004–8201). We would
like to thank J. Engel for the statistical support provided
during the analysis of the data, and Dr H. Honing for
the analysis suggestions and constructive criticism of the
experimental method, and Janto Skowronek for the
advice on quadratic discriminant analysis.
References
All Music. (2008). Retrieved from http://www.allmusic.com/
Aucoutourier, J.J., & Pachet, F. (2002). Finding songs that
sound the same. In Proceedings of the IEEE Benelux
Workshop on Model based Processing and Coding of
Audio (MPCA-2002), Leuven, Belgium.
Aucoutourier, J.J., & Pachet, F. (2004). Improving timbre
similarity: How high’s the sky? Journal of Negative
Research Results in Speech and Audio Sciences, 1(1),
1–13.
Bella, S.D., & Peretz, I. (2005). Differentiation of classical
music requires little learning but rhythm. Cognition, 96B,
65–78.
Berenzweig, A., Logan, B., Ellis, D.P.W., & Whitman, B.
(2003). A large-scale evaluation of acoustic and sub-
jective music similarity measures. Computer Music
Journal, 28, 63–76.
Burton, M.L., & Nerlove, S.B. (1976). Balanced designs for
triads tests: Two examples from English. Social Science
Research, 5, 247–267.
Cambouropoulos, E. (2009). How similar is similar?
Musicae Scientiae – Discussion Forum, 4B, 7–24.
Chupchik, G.C., Rickert, M., & Mendelson, J. (1982).
Similarity and preference judgements of musical stimuli.
Scandinavian Journal of Psychology, 23, 273–282.
Deliège, I. (2001). Similarity perception – categorization –
cue abstraction. Music Perception, 18, 233–243.
19
Duda, R.O., Hart, P.E., & Stork, D.G. (2001). Pattern
classification. New York: Wiley.
Eerola, T., & Bregman, M. (2007). Melodic and contextual
similarity of folk song phrases. Musicae Scientiae –
Discussion Forum, 4A, 211–233.
Eerola, T., Jarvinen, T., Louhivuori, J., & Toiviainen, P.
(2001). Statistical features and perceived similarity of folk
melodies. Music Perception, 18, 275–296.
Efron, B., & Tibshirani, R.J. (1993). An Introduction to the
Bootstrap. Boca Raton, FL: Chapman & Hall.
Goldstone, R.L. (1994). An efficient method for obtaining
similarity data. Behavior Research Methods, Instruments,
& Computers, 26(4), 381–386.
Green, D.M., & Swets, J.A. (1966/1988). Signal Detec-
tion Theory and Psychophysics. Los Altos, CA:
Peninsula Publishing.
Grey, J. (1977). Multidimensional perceptual scaling of
musical timbres. Journal of the Acoustical Society of
America, 61, 1270–1277.
Herre, J., Allamanche, E., & Ertel, C. (2003). How similar
do songs sound? Towards modeling human perception
of musical similarity. In Proceedings of the IEEE
Workshop on Applications of Signal Processing to Audio
and Acoustics (WASPAA), New Paltz, NY, USA.
iTunes. (2008). Retrieved from http://www.apple.com/
itunes/
Kendall, M.G. (1975). Rank Correlation Methods. London:
Charles Griffin.
Krumhansl, C.L. (1978). Concerning the applicability of
geometric models to similarity data: The interrelationship
between similarity and spatial density. Psychological
Review, 85, 445–463.
Kruskall, J.B., & Wish, M. (1978). Multidimensional Scaling,
Quantitative Applications in the Social Sciences. London:
Sage.
Lamont, A., & Dibben, N. (2001). Motivic structure
and the perception of similarity. Music Perception, 18,
245–274.
Levelt, W.J.M., van de Geer, J.P., & Plomp, R. (1966).
Triadic comparisons of musical intervals. The British
Journal of Mathematical and Statistical Psychology, 19,
163–179.
Logan, B., Ellis, D.P.W., & Berenzweig, A. (2003).
Toward evaluation techniques for music similarity. In
Proceedings of the 3rd International Symposium on
Music Information Retrieval (ISMIR 2002), Paris,
France.
Logan, B., & Salomon, A. (2001). A music similarity
function based on signal analysis. In Proceedings of the
IEEE International Conference on Multimedia and Expo
(ICME 2001), Tokyo, Japan.
MacRae, A.W., Howgate, P., & Geelhoed, E. (1990).
Assessing the similarity of odours by sorting and
by triadic comparisons. Chemical Senses, 15, 691–
699.
McAdams, S., & Matzin, D. (2001). Similarity, invariance
and musical variation. Annals of the New York Academy
of Sciences, 930, 62–76.
 20 Alberto Novello et al.
McAdams, S., Vieillard, S., Houix, O., & Reynolds, R.
(2004). Perception of musical similarity among contem-
porary thematic materials in two instrumentations.
Music Perception, 22, 207–237.
Mirex. (2006). Mirex website. Retrieved from http://
www.music-ir.org/mirex2006
Nosofsky, R.M. (1992). Similarity scaling and cognitive
process models. Annual Review of Psychology, 43,
25–54.
Orpen, K.S., & Huron, D. (1992). Measurements of
similarity in music: A quantitative approach for
non parametric representations. Computers in Music
Research, 4, 1–44.
Pampalk, E. (2004). A Matlab toolbox to compute
music similarity from audio. In Proceedings of the 5th
International Symposium on Music Information Retrieval
(ISMIR 2004), Barcelona, Spain.
Pampalk, E. (2006). Computational models of music
similarity and their application in music information
Downloaded by [41.235.86.174] at 12:24 10 July 2011
retrieval (PhD thesis). Technische Universität Wien,
Austria.
Pampalk, E., Flexer, A., & Widmer, G. (2005). Improvements
of audio-based music similarity and genre classification. In
Proceedings of the 6th International Symposium on Music
Information Retrieval (ISMIR 2005), London, UK.
R-project. (2008). Retrieved from http://www.r-project.org/
Shepard, R.N. (1962a). The analysis of proximities: Multi-
dimensional scaling with unknown distance function I.
Psychometrika, 27(2), 125–140.
Shepard, R.N. (1962b). The analysis of proximities: Multi-
dimensional scaling with unknown distance function II.
Psychometrika, 27(3), 219–246.
Shepard, R.N. (1986). Discrimination and generalization
in identification and classification: Comment on Nosofs-
ky. Journal of Experimental Psychology, 115(1), 58–61.
Tversky, A. (1977). Features of similarity. Psychological
Review, 84, 327–352.
Tversky, A., & Gati, I. (1982). Similarity, separability, and the
triangle inequality. Physchological Review, 89, 123–154.
Young, F.W., & Lewyckyj, R. (1996). ALSCAL User’s
Guide (5th ed.). Chapel Hill: Psychometric Laboratory,
University of North Carolina.
 Downloaded by [41.235.86.174] at 12:24 10 July 2011

Appendix A. Excerpt selection.

Synthetic/
Number Genre Tempo BPM Timbre Acoustic Author Title Album Year Publisher Begin/End

1 Afro-Pop fast 112 guitar acoustic Salif Keita Tekere Folon 1995 Mango 0:10–0:25
2 Afro-Pop fast 130 piano acoustic Don Pullen & African Yebino spring Live . . . again 1995 Blue Note 7:30–7:45
Brazilian Records
Connection
3 Afro-Pop fast 170 vocal acoustic ARC Music Tongoyo Traditional songs and 2003 ARC Music 0:35–0:50
Productions dance from Africa Productions
4 Afro-Pop slow 89 guitar acoustic Chief Ebenezer Obey Operation feed the Ju ju jubilation 1998 EMI Records 2:30–2:45
nation Ltd
5 Afro-Pop slow 88 piano acoustic Abdullah Ibrahim Hajj (the journey) The journey 1978 Chiaroscuro 1:12–1:27
Records
6 Afro-Pop slow 82 vocal acoustic Youssou N’Dour Macoy The lion 1989 Virgin Records 1:40–1:55
7 Blues fast 220 guitar synthetic Little Charlie & The Percolatin’ Crucial guitar blues 2003 Alligator 1:22–1:37
Nightcats Records
8 Blues fast 170 piano acoustic Roosevelt Skies Hot pants Music is my busyness 2001 Corazong 0:53–1:08
Records
9 Blues fast 220 vocal acoustic S.R. Vaughan Give me back my wig Martin Scorsese presents 1986 Montreux 1:20–1:35
Stevie Ray Vaughan Sounds S.A.
10 Blues slow 63 guitar synthetic Johnny Winter I smell trouble Crucial guitar blues 2003 Alligator 0:02–0:17
Records
11 Blues slow 60 piano acoustic Big Mama Thorton I feel the way I feel Big Mama Thornton & 2004 Arhoolie 0:20–0:35
Muddy Waters Blues Records
Band - 1966
12 Blues slow 61 vocal acoustic Johnny B. Moore Back door friend Live at Blue Chicago 1996 Delmark 1:35–1:50
13 Classical fast 162 guitar acoustic J.S. Bach/Eduardo Prelude suite no.4 Johann Sebastian Bach: 2004 Oehms Classics 1:25–1:40
Fernandez 4 Suites for Lute
14 Classical fast 158 piano acoustic F. Chopin Piano concerto 2 Chopin: Piano 1999 EMI Classics 1:15–1:30
Op.21 III Concertos nos. 1 & 2,
Dutoit, Argerich
15 Classical slow 80 vocal acoustic C. Bartoli & G. Paisiello /Chi vuol la Se tu m’ami - Arie 1992 Decca Music 1:10–1:20
Fischer zingarella antiche Group Ltd
16 Classical slow 80 guitar acoustic J. Rodrigo Adagio Concierto de Aranjuez 1994 Philips 9:25–9:40
Fantası́a para un
Gentilhombre
17 Classical slow 80 piano acoustic E. Satie Gymnopédies Gymnopédies 2000 EMI Classics 0:10–0:25
Perceptual evaluation of inter-song similarity in Western popular music

18 Classical slow 80 vocal acoustic W.A. Mozart Requiem - Introitus W.A. Mozart/ 1999 Virgin Records 2:25–2:40
Norrington, London
Classical Players, et
al.
19 Country fast 120 guitar acoustic Chet Atkins Yakety axe The essential Chet 1972 BMG 1:20–1:35
Atkins Entertainment
20 Country fast 150 piano acoustic Floyd Cramer On the rebound 20 greatest hits 207 Gusto Records 0:35–0:50

(continued)
21
 Downloaded by [41.235.86.174] at 12:24 10 July 2011

22
Appendix A. (Continued).

Synthetic/
Number Genre Tempo BPM Timbre Acoustic Author Title Album Year Publisher Begin/End

21 Country fast 282 vocal acoustic Statler Brothers Flowers on the wall Pulp Fiction - collector’s 1994 MCA Records 0:07–0:23
edition
22 Country slow 105 guitar acoustic Willie Nelson Blue eyes crying in All the songs I’ve loved 2002 Special 1:10–1:25
the rain before Marketing
23 Country slow 67 piano acoustic Hargus Robbins I’m hurting Belly up to the bar: 2005 Time Records 0:20–0:35
classic country and
western
24 Country slow 87 vocal acoustic George Jones The grand tour George Jones: The 2003 Epic 0:24–0:39
definitive country
collection
25 Electronica fast 112 guitar synthetic Daft Punk Robot rock Human after all 2005 Virgin Records 0:25–0:40
Ltd
26 Electronica fast 160 piano synthetic Goldie Crystal clear Saturnz return 1998 Ffrr Records Ltd 4:01–4:16
27 Electronica fast 144 vocal synthetic Prodigy No good (start the Music for the jilted 1995 Mute 5:40–5:55
dance) generation
28 Electronica slow 50 guitar acoustic Chemical Brothers Where do I begin Dig your own hole 1997 Astralwerks 0:15–0:30
29 Electronica slow 70 piano synthetic Robert Miles Children (dream Dreamland 1996 Arista 0:54–1:09
version)
30 Electronica slow 90 vocal synthetic Kraftwerk The man-machine The man-machine 1978 Capitol 1:26–1:41
31 Folk fast 122 guitar acoustic Sarah Bolen Fantasy Naked on the inside 2001 Sarah Bolen 3:15–3:30
32 Folk fast 122 piano acoustic Jeff Little Grassy creek Piano man from blue 2003 Jeff Little 1:20–1:35
ridge
Alberto Novello et al.

33 Folk fast 126 vocal acoustic The Seeger Family Muskrat Animal folk songs for 1992 Rounder 0:38–0:53
children and other
people!
34 Folk slow 82 guitar acoustic Cheryl Wheeler But the days and Sylvia hotel 1999 Philo/Umgd 1:52–2:07
nights are long
35 Folk slow 78 piano acoustic Gary Remal Malkin Appalachian sunrise The music of the great 1996 Real Music 1:18–1:33
smoky mountains
36 Folk slow 66 vocal acoustic P. Seeger My name is Liza Waist deep in the big 1967 Sony 1:35–1:50
Kalvelage muddy and other love
songs
37 Hip-Hop fast 100 guitar synthetic Public Enemy New whirl odor New whirl odor 2005 Slam Jamz 9:30–9:45
Records
38 Hip-Hop fast 96 piano synthetic Outkast Mrs Jackson Stankonia 2000 La Face 3:57–4:13
39 Hip-Hop fast 136 vocal synthetic Busta Rhymes Gimme some more The best of Busta 2001 Elektra/Wea 1:00–1:15
Rhymes
40 Hip-Hop slow 85 guitars synthetic Cypress Hill Amplified Stoned raiders 2001 Sony 0:08–0:23

(continued)
 Downloaded by [41.235.86.174] at 12:24 10 July 2011

Appendix A. (Continued).

Synthetic/
Number Genre Tempo BPM Timbre Acoustic Author Title Album Year Publisher Begin/End

41 Hip-Hop slow 80 piano synthetic Beastie Boys Ricky’s theme The in sound from way 1996 Capitol 0:08–0:23
out!
42 Hip-Hop slow 80 vocal synthetic Arrested Development People everyday 3 Years 5 months & 2 1992 Capitol 0:27–0:43
days in the Life of
43 Jazz fast 260 guitar synthetic Mike Stern Good question Who let the cats out? 2006 Heads Up 1:30–1:45
44 Jazz fast 270 piano acoustic Herbie Hancock One finger snap Empyrean isles 1964 Blue Note 2:43–2:58
45 Jazz fast 220 vocal acoustic E. Fitzgerald and L. I’ve got my love to Verve jazz masters 1957 Polygram 0:57–1:07
Armstrong keep me warm Records
46 Jazz slow 102 guitar acoustic Bireli Lagraine Insensatez Bireli Lagrene: 1992 Blue Note 1:12–1:27
Standards Records
47 Jazz slow 66 piano acoustic Frank Morgan Mood indigo Mood indigo 1990 Polygram 4:04–4:19
Records
48 Jazz slow 64 vocal acoustic Billie Holiday Good morning The Billie Holiday 1952 Polygram 0:20–0:35
heartache songbook Records
49 Latin fast 220 guitar acoustic Vieja Trova Cuida eso La manigua 1998 Virgin Records 0:00–0:15
Santiaguera
50 Latin fast 270 piano acoustic Jesus Alemany Tumbao de coqueta Cubanismo! 1996 Hannibal 2:37–2:52
51 Latin fast 167 vocal synthetic Manu Chao La marea Proxima estación: 2001 Virgin 0:00–0:15
esperanza
52 Latin slow 101 guitar acoustic Compay Secundo Es mejor vivir ası̀ Lo mejor de la vida 1998 Nonesuch 0:02–0:17
53 Latin slow 105 piano acoustic Buena Vista Social Buena Vista Social Buena Vista Social Club 1997 Nonesuch 0:28–0:43
Club Club
54 Latin slow 81 vocal acoustic Ibrahim Ferrer Mil congojas Buenos hermanos 2003 Nonesuch 1:15–1:30
55 Pop fast 207 guitar synthetic The Bangles Walk like an Greatest hits 1990 Sony 1:31–1:46
Egyptian
56 Pop fast 135 piano acoustic Tori Amos Cornflake girl Under the pink 1994 Atlantic/WEA 4:05–4:20
57 Pop fast 190 vocal acoustic The Housemartins Happy hour Now thats what I Call 1988 Go! Disc Ltd 0:33–0:48
quite good
58 Pop slow 79 guitar acoustic Eric Clapton Tears in heaven The best of Eric Clapton 1999 Reprise/Wea 2:32–2:47
59 Pop slow 67 piano acoustic Lionel Ritchie Easy The definitive collection 2003 Motown 0:00–0:15
60 Pop slow 68 vocal acoustic Elton John Rocket man Rocket man: The 2007 Mercury 1:30–1:40
definitive hits Records
61 R&B fast 135 guitar synthetic L. Buksbaum and S. P. Can’t catch me Sports highlights vol. 2 2006 Freeplaymusic, 0:30–0:45
Perceptual evaluation of inter-song similarity in Western popular music

Schreer BMI
62 R&B fast 134 piano synthetic James Taylor’s Splat Message from the 2001 Ubiquity 0:35–0:50
Quartet godfather Recordings
63 R&B fast 133 vocal acoustic Oliver Morgan Roll call (New Saturday night fish fry 2001 Soul Jazz 0:40–0:55
Orleans funk and
soul)

(continued)
23
 Downloaded by [41.235.86.174] at 12:24 10 July 2011

24

Appendix A. (Continued).

Synthetic/
Number Genre Tempo BPM Timbre Acoustic Author Title Album Year Publisher Begin/End

64 R&B slow 85 guitar synthetic P. Calandra & S.P. Blusey Funk Blues vol. 3 2006 Pecamusic/BMI/ 0:00–0:15
Schreer Freeplaymusic
65 R&B slow 84 piano synthetic Niels Landgren Calvados Fonk da world 2001 ACT 1:12–1:27
66 R&B slow 70 vocals acoustic Aretha Franklin Something he can Respect: The very best 2002 Warner/BMG 0:47–1:02
feel of Aretha Franklin
67 Reggae fast 140 guitar synthetic Skavoovie & The Nut monkey Fat footin’ 1996 Moon Ska/ 1:43–1:58
Epitones Caroline
68 Reggae fast 160 piano synthetic Pannonia allstars Balkan fever Budapest ska mood 2002 Megalith 0:50–1:05
Records
69 Reggae fast 155 vocal synthetic Gentleman Face off Confidence 2004 Four Music 1:14–1:29
Productions
70 Reggae slow 80 guitar acoustic Ernest Ranglin Below the bassline Below the bassline 1996 Island 1:18–1:33
71 Reggae slow 80 piano synthetic Bob Marley No woman no cry Legend - The best of 2002 Def Jam 0:05–0:20
Bob Marley and The
Wailers
Alberto Novello et al.

72 Reggae slow 70 vocal acoustic Peter Tosh Legalize it Legalize it 1999 Sony 0:30–0:45
73 Rock fast 200 guitar synthetic AC/DC Whole lotta rosie AC/DC box set 2006 Sony Bmg 2:13–2:28
74 Rock fast 185 piano synthetic Supertramp School The very best of 2001 A&M 3:12–3:27
Supertramp
75 Rock fast 162 vocals synthetic Black Sabbath Paranoid Paranoid 1990 Warner Bros./ 0:36–0:51
WEA
76 Rock slow 68 guitar synthetic Jimi Hendrix Little wing Experience Hendrix - 2000 Experience 1:58–2:13
The best of Jimi Hendrix
Hendrix
77 Rock slow 85 piano synthetic Faith No More Epic The real thing 1989 Reprise/WEA 4:05–4:20
78 Rock slow 80 vocals synthetic Alanis Morisette Right through you Jagged little pill 1995 Maverick 0:36–0:51
 Perceptual evaluation of inter-song similarity in Western popular music 25

Appendix B. Excerpt composition per repeated triad.

Triad number First excerpt Second excerpt Third excerpt

triad 1 55 - Pop - Fast - Guitar 56 - Pop - Fast - Piano 57 - Pop - Fast - Vocal
triad 2 52 - Latin - Slow - Guitar 53 - Latin - Slow - Piano 54 - Latin - Slow - Vocal
triad 3 43 - Jazz - Fast - Guitar 44 - Jazz - Fast - Piano 48 - Jazz - Slow - Vocal
triad 4 64 - R&B - Slow - Guitar 65 - R&B - Slow - Piano 66 - R&B - Slow - Vocal
triad 5 31 - Folk - Fast - Guitar 32 - Folk - Fast - Piano 39 - Hip-Hop - Fast - Vocal
triad 6 26 - Electro - Fast - Piano 29 - Electro - Slow - Piano 23 - Country - Slow - Piano
triad 7 7 - Blues - Slow - Guitar 10 - Blues - Fast - Guitar 13 - Classical - Fast - Guitar
triad 8 73 - Rock - Fast - Guitar 68 - Reggae - Fast - Piano 63 - R&B - Fast - Vocal
triad 9 48 - Jazz - Slow - Vocal 54 - Latin - Slow - Vocal 57 - Pop - Fast - Vocal
triad 10 27 - Electro - Fast - Vocal 36 - Folk - Slow - Vocal 42 - Hip-Hop - Slow - Vocal

Appendix C. CBD versus BIBD control experiment: excerpt selection (see Appendix D for excerpt detail).
Downloaded by [41.235.86.174] at 12:24 10 July 2011

Number as
Number in Appendix D Genre Tempo Author Title

1 3 Classical slow C. Bartoli & G. Fischer Paisiello/Chi vuol la zingarella

2 4 Classical fast L. V. Beethoven Symphony no. 9, Op. 125, 4th movement
3 5 Rock slow Jimi Hendrix Foxey lady
4 6 Rock fast Queen Headlong
5 10 Heavy Metal slow Paradise Lost True belief
6 13 Jazz slow F. Sinatra & E. Ellington I like the sunrise
7 14 Jazz fast E. Fitzgerald & L. Armstrong I’ve got my love to keep me warm
8 17 Country slow J. Reeves He’ll have to go
9 18 Country fast Cumberland Highlanders Cumberland mountain home
 Downloaded by [41.235.86.174] at 12:24 10 July 2011

26

Appendix D. Control experiment I: triadic comparison vs. grouping paradigm, the excerpt selection.

Number Genre Tempo BPM Author Title Album Year Publisher Begin/End

1 Blues fast 220 S.R. Vaughan Give me back Martin Scorsese 1986 Montreux 1:20–1:30
my wig presents Stevie Sounds S.A.
Ray Vaughan
2 Blues slow 75 B.B. King I need you so Reflections 2003 Geffen Records 0:15–0:25
3 Classical fast 140 L. V. Beethoven/ 4th Mouvement Symphony no.9 2005 EMI Records 1:20–1:30
London Classical Prestissimo Op. 125
Players & Schutz
choir of Norrington
4 Classical slow 80 C. Bartoli & G. Fischer Paisiello/Chi vuol Se tu m’ami - 1992 Decca 1:10–1:20
la zingarella Arie antiche Music Group Ltd
5 Country fast 135 Cumberland Cumberland Cumberland 2000 Rural Rhythm 0:40–0:50
Highlanders mountain home mountain home
6 Country slow 85 J. Reeves He’ll have to go Greatest hits 1972 BMG Entertainment 0:30–0:40
7 Funk fast 115 Don Covay Overtime man So soulful 70’s 1999 Kent 0:10–0:20
8 Funk slow 90 B. Collins Hollywood squares Back in the day: 1976 Warner Bros. 1:10–1:20
The best of Bootsy Collins
9 Heavy Metal fast 168 Metallica The prince Garage inc. 1998 Elektra/Wea 1:30–1:40
Alberto Novello et al.

10 Heavy Metal slow 60 Paradise Lost True belief Reflection 1998 Pid 0:50–1:00
11 Hip-Hop fast 134 Dr. Dre and Eminem Forgot about Dre 2001 1999 Aftermath Ent. 1:19–1:29
Interscope Records
12 Hip-Hop slow 80 Coolio Gangsta’s paradise Gangsta’s paradise 1995 Warner Strategic Market 0.51–0.61
13 Jazz fast 220 E. Fitzgerald & I’ve got my love Verve jazz masters 1957 Polygram Records 0:57–1:07
L. Armstrong to keep me warm
14 Jazz slow 67 F. Sinatra & E. Ellington I like the sunrise Francis A. Sinatra and 1967 Warner Bros./Wea 0:41–0:51
Edward K. Ellington
15 Pop fast 190 The Housemartins Happy hour Now that’s what I 1988 Go! Disc Ltd 0:33–0:43
call quite good
16 Pop slow 68 Elton John Rocket man Rocket man: The 2007 Mercury Records 1:30–1:40
definitive hits
17 Rock fast 147 Queen Headlong Innuendo 1991 Queen Prods. 0:30–0:40
18 Rock slow 100 Jimi Hendrix Foxey lady Are you experienced? 1979 Experience Hendrix 0:56–1:06