You are on page 1of 18

Food Quality and Preference 28 (2013) 164181

Contents lists available at SciVerse ScienceDirect

Food Quality and Preference


journal homepage: www.elsevier.com/locate/foodqual

A summary of projective mapping observations The effect of replicates


and shape, and individual performance measurements
Helene Hopfer , Hildegarde Heymann
Department of Viticulture & Enology, University of California, Davis, One Shields Avenue, Davis, CA 95616, USA

a r t i c l e

i n f o

Article history:
Received 2 May 2012
Received in revised form 13 August 2012
Accepted 27 August 2012
Available online 14 September 2012
Keywords:
Projective mapping/Napping
Shape
Descriptive analysis
Comparison
Red wine blends
Multi-factor analysis

a b s t r a c t
Projective mapping (PM) or Napping are fast alternatives to traditional descriptive analysis (DA), and are
becoming more popular among sensory scientists to obtain a quick overview of (dis)similarities among a
certain sample set. Ideally, PM should be able to deliver similar results as a DA, and this aspect has been
studied extensively in the last years, also in comparison to other fast alternative descriptive methods.
Other aspects of research include the effect of replication and how to analyze the data.
Besides the two previously mentioned aspects (the effect of replicates and the comparability with DA),
we focused in this study on the effect of the provided PM space, and compared a square to a rectangular
space, and whether the obtained results would differ. In two consecutive studies, we compared a square
conguration to a horizontal and a vertical rectangle conguration. We found that the judges did position
their samples in a different way when confronted with a differently shaped space. These results suggest
that the PM product representation depends on the provided space.
A last aspect of this study was dealing with individual performance, how this could be measured, and
how large the effect of the individual judges on the overall solution is. Generally, judges that did not use
the usual Cartesian coordinate system to position their samples did not affect the consensus PM solution.
This can be most likely attributed to the large number of panelists in our studies. Additionally, we propose a people performance index (PPI) to measure the ability of an individual to place blind duplicated
samples close to each other.
2012 Elsevier Ltd. All rights reserved.

1. Introduction
In the recent past, alternatives to conventional and traditional
descriptive analysis methods (generic descriptive analysis (DA)
and its trademarked variations) have gained more and more interest. All these alternative methods try to overcome drawbacks from
DA, which are mainly (i) the longer time needed to obtain results
due to the need of panelist training, and (ii) the importance to obtain consensus on particular attributes, which sometimes induces a
difcult task when working with expert judges like wine professionals or chefs.

Abbreviations: PM, projective mapping; DA, descriptive analysis; UFP, ultra-ash


proling; FP, ash proling; FMS, free multiple sorting; GN, global Napping; PN,
partial Napping; CI, condence intervals; PCA, principal component analysis; MFA,
multi-factor analysis; HMFA, hierarchical multi-factor analysis; PMFA, procrustes
multi-factor analysis; CA, coordinate averaging; INDSCAL, individual difference
scaling; MDS, multi-dimensional scaling; GPA, generalized procrustes analysis;
RSQ, squared correlations; RG, repertory grid.
Corresponding author. Tel.: +1 530 752 9356; fax: +1 530 752 0382.
E-mail addresses: hhopfer@ucdavis.edu (H. Hopfer), hheymann@ucdavis.edu (H.
Heymann).
0950-3293/$ - see front matter 2012 Elsevier Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.foodqual.2012.08.017

One of these alternatives projective mapping (PM) was


developed in the early 90s by Risvik and colleagues (1994, 1997)
studying chocolates and blueberry soups. Panelists were asked to
place products on a two-dimensional space according to similarity.
Data analysis of the so created maps was facilitated using generalized procrustes analysis (GPA) and principal component analysis
(PCA), and results were compared to conventional DA using the
RV coefcient, a multi-variate equivalent to a Pearsons correlation
coefcient (Abdi, 2007, 2010).
A re-introduction and adaptation of the PM method under the
name Napping occurred in the mid 00s by Pags and coworkers
(Pags, 2005; Pags, Cadoret, & L, 2010; Pags & Husson, 2005;
Perrin et al., 2008), together with a new way to analyze the obtained maps by multi-factor analysis (MFA) (Escoer & Pags,
1994). MFA is a version of a principal component analysis (PCA)
on different datasets that were horizontally merged and
standardized.
The differences between PM and Napping are not very clear;
while some (Nestrud & Lawless, 2010) used both terms interchangeably to describe a multi-dimensional data collection, others
like Dehlholm, Brockhoff, Meinert, Aaslyng, and Bredie (2012)
distinguished strictly between the two techniques (see Fig. 1):

H. Hopfer, H. Heymann / Food Quality and Preference 28 (2013) 164181

Fig. 1. Differentiation between projective mapping (PM) and Napping according


to (Dehlholm et al., 2012).

according to them the Napping technique is a sub-form of PM, restricted to a rectangular space (60  40 cm), with no data scaling
prior to the analysis solely with MFA. Various sub-forms of Napping were identied based on (i) the holistic nature of the task (divided into global Napping, where the panelists are not restricted to
one sensory modality, and partial Napping, where judges are
asked to focus on a single sensory modality (e.g. aroma, or taste,
or texture, etc.)), and (ii) the descriptive attribute collection by
combining the Napping task with ultra-ash proling (UFP) or
simple sorting. In a sense the partial Napping can be seen as a
compromise in between the holistic global Napping and the atomistic DA (Perrin & Pags, 2009).
In contrast to Napping conventional PM is less restricted, and
can be carried out on a square space (e.g. 60  60 cm) or a rectangular space, using both structured and unstructured hedonic line
scales, and can be analyzed with other techniques than MFA (e.g.
GPA). A so-called structured PM is a PM with labeled axes on the
provided space: King and coworkers applied both methods in their
study on snack bars, and used the structured PM with the x-axis for
liking labeled from low to high, and the y-axis for the intended
use from treat to meal replacement (King, Cliff, & Hall, 1998).
That being said, as both methods use the same concept i.e. a
two-dimensional sorting procedure we believe that this strict
separation between PM and Napping needs to be further
discussed.
PM and Napping have been applied on a broad range of foods
and beverages (see Table 1): all these studies compared PM or Napping to other sensory methods, usually DA, but in a recent publication the authors compared PM to DA and other fast alternative
descriptive methods (Dehlholm et al., 2012), or to repertory grid
(RG) and ash proling (FP) (Veinand, Godefoy, Adam, & Delarue,
2011). With the exception of one study, all publications used 12
or fewer different samples, that were also rather easy to discriminate (Barcenas, Elortondo, & Albisu, 2004; Dehlholm et al., 2012;
Kennedy, 2010; Kennedy & Heymann, 2009; Nestrud & Lawless,
2010; Pags et al., 2010). An exception was the study by King
et al. who studied 18 quite different snack bars (King et al.,
1998). Three studies evaluated similar white and red wines from
the Loire region, differing in vintage, producer, appellation and/or
time spent in an oak barrel (Pags, 2005; Perrin & Pags, 2009;
Perrin et al., 2008).
From a methodological and developmental point of view, it
makes sense to test the feasibility of PM by using a relatively small
number of samples that are somewhat easy to differentiate. However, from a practitioners point of view, the applicability of a study
on 8 different fruit drinks or 10 apple varieties does not seem easily

165

transferable to more practical sample sets containing more and, often, quite similar samples.
In the literature, for the sorting task, a technique with similar
intent to PM, between 10 and 20 samples are recommended. On
the one hand this is a big enough sample set to study differences,
and on the other hand there are not so many samples that palate
fatigue and frustration with the task become an issue (Lawless &
Heymann, 2010). Pags (2005) suggested a limit of 12 samples
for Napping on his ndings for white wines from the Loire region.
In this study we used a higher number of samples (18) to test this
hypothesis and to see whether there is an upper limit for PM sample sizes.
Another point of discussion for PM and Napping is whether the
tasks should be replicated or not: while in some studies the products were positioned only once (Dehlholm et al., 2012; Kennedy &
Heymann, 2009; King et al., 1998; Nestrud & Lawless, 2008, 2010;
Pags, 2005; Pags et al., 2010; Perrin et al., 2008), other studies
did specically investigate the effect of replication on the obtained
product spaces: using 15 untrained consumers Kennedy (2010)
evaluated the effect of replication on the spatial product presentation, and found that the PM plots differed a great deal for each
judge from replicate to replicate. She speculated that these different arrangements might arise from a change in the arrangement
criteria. However, the consensus product maps did not differ much
from each other, indicating, that averaged over all judges similar
product arrangements were made. Similar results were found by
Risvik et al. (1994, 1997) using blueberry soups and chocolates: a
low similarity of the product maps over three replicates was found
for individual judges, but the solutions obtained by GPA for all
judges for each replicate were very similar. For ewes milk cheeses
(Barcenas et al., 2004) judges grouped some samples together in all
three replicates, while two cheeses showed a low repeatability:
The authors speculated that variability in the cheese production resulted in different cheeses in the three replicates. An alternative to
true sensory replication could be the use of blind duplicates within
the product set, i.e. when one or two or all products are presented
twice. This procedure allows you also to check how capable the
judges are to actually pick up similarities. In two studies conducted
by Nestrud and Lawless (2008, 2010) the judges were able to position the blind duplicates close to each other, a similar result that
was reported as well by Veinand et al. (2011) for lemon ice teas
using untrained consumers.
It was found that about 615% of the judges have problems with
the PM task (Nestrud & Lawless, 2008; Pags, 2005; Veinand et al.,
2011) i.e. were unable to create a plane product representation
map. One way to overcome this constraint is a short introduction
or training prior to the PM task as done by Risvik et al. (1994,
1997) and Barcenas et al. (2004) who both used the example of
intercity distances from Kruskal and Wish (1978), or handing out
a brochure with an explained example (Veinand et al., 2011). Similarly, we included a short training exercise in our study using various shapes differing in color and size.
When compared to other sensory methods, good agreement can
usually be found between the PM or Napping product spaces and
conventional DA (Barcenas et al., 2004; Dehlholm et al., 2012;
Pags, 2005; Perrin et al., 2008; Risvik et al., 1994, 1997). When
using both untrained consumers and trained judges for the PM task
Barcenas et al. (2004) found a better correlation with the DA for the
PM with the trained judges. Similarly, untrained consumers in a
PM detected the major differences among the samples similarly
to trained judges in a DA, but did not pick up the smaller differences (Risvik et al., 1997). An extensive comparison with other fast
descriptive methods was recently published (Dehlholm et al.,
2012): similar product maps were obtained with all methods, but
the highest similarity was found for DA and the partial Napping.
Similarly, using untrained consumers and comparing PM to FP

166

Table 1
Summary of projective mapping (PM) and Napping studies comparing evaluated products, applied methods, used statistical techniques and major ndings.
Products

Sensory method

PM
replicates

PM space

Blind
duplicates

Descriptors

PM statistical analysis

Dehlholm et al. (2012)

9 Liver pates

Comparison of DA, FP, FMS, PN, GN; 2 professional


panels (9 judges each) doing all analyses (only one
panel for FP)

No

60  40 cm

No

From UFP

MFA
RV to compare methods
bootstrapping for CI

 Similar maps in
all methods
 DA and PN highest
similarity
within
and
between panels

King et al. (1998)

18 Commercial snack
bars

Sorting, structured & unstructured PM;


2 untrained panels (24 judges, 1 for sorting, 1 for
PM)

No

60  60 cm

No

No

GPA and MDS for unstructured


PM
CA on structured PM
RV to compare methods

 CA less effective
than MDS or GPA
 With
unstructured PM additional information
was
recovered

Risvik et al. (1994)

5 Commercial
chocolates

DA, PM and dissimilarity scaling for pairwise


comparison; 1 panel (9 judges): PM training with
towns map of Kruskal and Wish (1978)

21  29.7 cm
(A4) marked
with crossed
axes

No

No

GPA
RV to compare methods

 PM is able to link
consumer
and
sensory studies
 High similarity
between DA and
PM for replicate
session 2 and 3
with RV over 0.7

Risvik et al. (1997)

7 commercial
blueberry soups

DA (12 judges) and PM (8 consumers); hedonic


rating of consumers after last PM session; PM
training with towns map of Kruskal and Wish
(1978)

29.7 x 42 cm
(A3)

No

No

PCA on normalized PM data


RV to compare methods

 High agreement
between DA and
PM in the rst
dimension only
 Conclude
that
consumers pull
out major differences similar to
DA, but not smaller ones
 For each PM replicate large individual
differences were found

Pags et al. (2010)

8 smoothies 4
avors from 2 brands

sorted Napping = PM combined with sorting,; 1


panel (24 judges); black glasses and red light

No

No, but
considered
it

from UFP

HMFA

 Expand the PM
task with categorization using the
sorting task to
group
similar
samples
 Analyze then by
HMFA

Nestrud and Lawless


(2008)

11 Citrus juices from


different varieties

PM; 2 panels (14 chefs and 16 consumers); scaling


of attributes; under red light

No

Yes, 2
juices

Yes, collected
after 1st
session for
scaling
exercise in
session 2

MFA and GPA on PM


coordinates
RV to compare methods

 Consumer panel
produced similar
maps in PM and
scaling,
while
chefs did not!
 2 out of 30
judges had difculties with the
PM task

60  60 cm

Major ndings

H. Hopfer, H. Heymann / Food Quality and Preference 28 (2013) 164181

Publication

10 Cheddar cheeses,
10 apple varieties

PM and Sorting; 1 untrained panel for each


product (19 judges for apples, 21 judges for
cheeses); under red light

No

60  60 cm

Yes, 2 for
each study

From UFP

MFA
RV to compare methods
Hierarchical clustering on
coordinates from MFA; multiple
regression with coordinates of
common maps to predict
attributes

 PM better than
sorting due to
better
dened
clusters in PM
 Judges had more
difculties with
apples
than
cheeses

Kennedy (2010)

8 Granola bars

PM; 1 untrained panel (15 judges)

60  60 cm

No

From UFP

MFA, HMFA and PMFA


RV to compare results;
descriptors as supplementary
not categorical variables in
analyses

 Little similarity
for
individual
judge
between
replicate sessions
 Speculates if they
changed their PM
criteria
 Overall consensus map was stable over replicate
sessions
 Similar
maps
with all analytical methods

Kennedy and Heymann


(2009)

1112 chocolates
with 9 common in all
three panels

PM and DA; 3 separate panels (89 judges each)

No

60 x 60 cm

No

From UFP

MFA for PM
RV to compare methods and
panels

 Good correlation
between PM and
DA
 Good correlation
between panels
 Product separation
mostly
based on cacao
content
 Low
RV
for
judges who used
a different separation criterion

Barcenas et al. (2004)

8 Ewes milk cheeses


differing in ripeness
level and type

PM (12 consumers) and DA (8 experienced


judges); training similar to Risvik et al. (1994),
Risvik et al. (1997) using towns map (Kruskal and
Wish, 1978); instructed PM panel to use odor,
taste and texture attributes

29.7  42 cm
(A3)

No

No

INDSCAL with distances


between products and over all
replicates
Correlation coefcient between
replicate sessions
stress and RSQ

 Consumers created very different


product
maps with little
agreement
 Lower
correlation for consumers than for DA
panel

Pags (2005)

5 Vouvray Chenin
Blancs and 5 Touraine
Sauvignon Blancs

DA (8 trained judged) and PM (11 wine


professionals)

No

40  60 cm

From DA
panel

MFA

 1015% of the
judges had problems with the
PM task
 Recommends DA
to
obtain
attributes
 Recommends to
limit number of
products to 12
wines

167

(continued on next page)

H. Hopfer, H. Heymann / Food Quality and Preference 28 (2013) 164181

Nestrud and Lawless


(2010)

168

Table 1 (continued)
Products

Sensory method

PM
replicates

PM space

Blind
duplicates

Descriptors

PM statistical analysis

Major ndings

Perrin et al. (2008)

10 Loire Chenin
Blancs differing in
vintage and oak
ageing

DA (17 judges), PM (12 wine professionals) and FP


(12 wine professionals)

No

40  60 cm

No

From UFP

HMFA on all three methods


MFA on PMdescriptors as
supplementary not categorical
variables in analyses

 Similar product
maps from all 3
methods

Perrin and Pags


(2009)

10 Loire red wines


from 5 producers and
two designations
(Anjou Rouge, Anjou
Village Brissac)

PM with UFP (14 wine professionals) and DA (8


wine professionals); no formal training for DA

40  60 cm

No

From UFP

CA and PCA on attributes


MFA on coordinates
RV to compare methods

 Good agreement
between
maps
from UFP and DA

 Napping procedure
pushes
judges to nd discriminating
attributes

Veinand et al. (2011)

8 Lemon ice teas from


Italy Switzerland and
France

PM with UFP (30 consumers), FP (43 consumers),


RG (42 consumers)

No

29.7  42 cm
(A3)

Yes, one
product

From UFP

GPA followed by agglomerative


hierarchical clustering on
coordinates, RV to compare
methods, MANOVA for quality
of product separation

 Some consumers
had
problems
with
map
creation
 Suggest to limit
PM to expert
judges
 Difculties
to
compare
PM
with RG and FP
due to different
numbers
of
dimensions
 Similar product
maps from all 3
methods
 Consumers were
able to detect
the
duplicated
samples

PM, projective mapping; DA, descriptive analysis; UFP, ultra-ash proling; FP, ash proling; FMS, free multiple sorting; GN, global Napping; PN, partial Napping; CI, condence intervals; PCA, principal component analysis;
MFA, multi-factor analysis; HMFA, hierarchical multi-factor analysis; PMFA, procrustes multi-factor analysis; CA, coordinate averaging; INDSCAL, individual difference scaling; MDS, multi-dimensional scaling; GPA, generalized
procrustes analysis; RV, regression vector; RSQ, squared correlations; RG, repertory grid.

H. Hopfer, H. Heymann / Food Quality and Preference 28 (2013) 164181

Publication

H. Hopfer, H. Heymann / Food Quality and Preference 28 (2013) 164181

and RG, Veinand et al. (2011) found similar product maps for the
studied lemon ice teas. All these results indicate that with PM
and Napping products are similarly characterized as in a DA, however, a product map alone is not the complete story until the obtained differences among the samples can be explained by
descriptive terms.
For this reason, PM and Napping are usually combined with an
attribute collection exercise, either from an ultra ash proling
(UFP) or descriptors from a conventional DA. While the former allows the judges to freely create and add descriptive terms for each
product or product group, is the latter more specic in terms of
denition. For each descriptor in a DA at least a verbal denition
or explanation, most often an actual reference standard is available
and can be tied to the same sensation, which might be a huge issue
in the UFP: Two judges might come up with the same word for the
descriptor, but mean it in a very different way.
Based on all these ndings we felt that there were still open
questions about PM and Napping, mainly whether the provided
space and the shape of this space would inuence the product
arrangement. Therefore, we designed a PM experiment as follows:
in two studies we investigated the shape inuence on the PM task
and compared these ndings to conventional DA. All tasks (PM and
DA) were carried out in triplicate on the same sample set of 18 red
wines (three monovarietal and 15 blends made thereof). The provided spaces differed in the shape and had roughly the same area.

2. Materials and methods


2.1. Samples
In both studies 18 red wines were used including three monovarietal wines and 15 two- and three-wine blends made thereof.
The wines were part of another study to evaluate the effect of wine
blending (Hopfer, Ebeler, & Heymann, 2012). In both studies PM
panelists were presented with 20 glasses of wines i.e. 18 wines
with two blind replicates to check for individual consistency similarly to Nestrud and Lawless (2008, 2010) (Table 2). The wines
were bottled about 2 weeks before study 1, and had been bottled
for about 1 year when study 2 took place. Wines were stored in a
dark space with controlled temperature (20 C) and humidity
(50% rel. humidity) for the year.

Table 2
Wine composition of the used samples in both PM studies.

a
b

No.

Label

Cabernet Sauvignon
(S) [v%]

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

S
M
F
SM11
SM91
SM82
SF11b
SF91a
SF82
MS91b
MS82
MF11
MF91
MF82
SMF111a
SMF811
MSF811
Blend

100.0

Merlot (M)
[v%]

Cabernet Franc (F)


[v%]

100.0
100.0
49.6
90.1
80.2
49.4
90.1
80.3
9.9
19.7

33.4
80.0
10.1
30.4

Duplicated wines in study 1.


Duplicated wines in study 2.

50.4
9.9
19.8
50.6
9.9
19.7
90.1
80.3
50.7
90.1
80.0
33.2
10.0
79.9
14.9

49.3
9.9
20.0
33.4
10.0
10.1
54.8

169

2.2. Projective mapping panels


The UC Davis Institutional Review Board approved all studies
(Protocol No. 201018550-1) and all judges gave oral informed consent. No monetary compensation was provided, but snacks were
provided after each session.
2.2.1. Training session
In order to ensure everybody understood the concept of PM we
gave all judges a short introduction followed by a training exercise.
In the training exercise the panelists were given paper shapes differing in color (red, blue, green, orange, pink, purple, white), shape
(cubes, cuboids, pyramids, prisms, cones, cylinders) and size (small
and large). They were asked to position these gures on a provided
space, and had to briey explain which criteria they used for their
PM conguration. When we provided the training samples we
made sure to include an odd sample (i.e. a single color or a single
shape) to challenge the judges how to t that one into their
arrangements. If needed, we pointed out that PM is a two-dimensional arrangement task while sorting is a one-dimensional one,
from a strictly geometrical point of view: While in a sorting task
judges only need to form groups, and do not need to indicate
how similar these groups are to each other, in a PM task judges rst
need to nd similar samples and group them together and secondly, need to relate these different groups to each other in terms
of (dis) similarity. One might argue that the sorting task is then not
a one-dimensional task, as due to the independent group formation
no restriction is made on the dimensionality of the sample space.
However, in most cases, one is interested in how different or
how similar the studied samples are to each other, and by using
PM judges are delivering this information, while focusing on the
most differentiating attributes present in the sample set. All judges
in both panels successfully completed the training. During the
instruction and training phase we took care not to put too many
constraints on the judges approach to PM: e.g. most judges used
color variation as one axis, and while most arranged the colors
from light to dark (i.e. yellow orange red violet green
blue), one judge ordered the colors following the rainbow order
(i.e. red orange yellow green blue indigo violet). Upon
completion of the training exercise the panelists commenced the
actual PM of the wines.
2.2.2. Booth set-up
All studies took place in individual tasting booths under red
light (study 1) or white light (study 2) (Lawless & Heymann,
2010). The provided spaces for the PM task were delimited by commercial blue painters tape (ScotchBlue, 3 M, St. Paul, Minnesota),
and judges were explicitly told to stay within the provided space.
In previous studies in our lab some judges failed to stay within
the border and positioned samples outside the provided space
(Sanchez Gavito Sanchez, 2011).
Approx. 25 ml aliquots of all the wines were served in pear
shaped transparent ISO glasses (International Organization for
Standardization, 1977) in study 1 and pear shaped black ISO
glasses in study 2, labeled with three-digit random numbers. All
wine samples as well as water (Arrowhead, Nestle Waters America, Stamford, CT, USA) and unsalted crackers (Nabisco unsalted top
premium saltine crackers, Kraft Foods, Northeld, IL, USA) were
placed next to the provided space inside the booth before the judge
entered the booth. All wine samples had to be expectorated. Judges
were encouraged to provide descriptive terms for the groups or
individual wines but were not forced to do so. In our opinion too
many restrictions limit the judges in freely creating their product
spaces. Additionally, since PM is a technique that supposedly
works with people unfamiliar with DA, the generation of descriptive terms could be burdensome. Judges were also told to not

170

H. Hopfer, H. Heymann / Food Quality and Preference 28 (2013) 164181

second guess themselves during the task to prevent frustration.


Judges usually took longest for the rst session (up to 1 h) but
signicantly reduced their PM time in the consecutive sessions to
2030 min. Instructions were given like this:
Please evaluate the wines in front of you and place them on the
provided space according to how similar or dissimilar they are
for you. The more similar the wines are the closer they should be
positioned to each other, the more dissimilar they are the further
apart they should be positioned. Please fell free to form as many
groups of similar wines as you want, but a minimum of two and
a maximum of 19 (i.e. group at least two wines together). If you
want you could write down your criteria and/or descriptive terms
for your groups. There is no right or wrong answer to that. Have
fun and thanks for your participation!

2.2.3. Study design


In the rst PM study 12 judges (ve females; aged 2145 years)
and in the second PM study 22 judges (12 females; aged 21
68 years) were recruited. All judges were students, staff or retirees
of the Departments of Viticulture & Enology or Food Science &
Technology, and were recruited via emails. Some of the judges
had had previous DA experience, while some were sensory analysis
novices. They were asked to come in the sensory lab for three sessions over a period of 4 days with the possibility of doing two sessions per day with a minimum 4-h break in between the two
sessions. All PM tasks were carried out in individual temperature-controlled tasting booths with a positive airow. In study 1
the judges were randomly assigned to two groups (i.e. the square
and the rectangle group). The square group performed their
PM on a square-shaped space in triplicate (dimensions:
27.5  27.5 in/69.9  69.9 cm), while the rectangle group performed the three replicates on a long horizontal rectangle-shaped
space (dimensions: 32  22 in/81.3  55.9 cm). For this study
wines 8 (SF91) and 15 (SMF111) were presented as blind duplicate.
In study 2 all judges positioned the same wines in the three sessions on three differently shaped spaces, i.e. a square conguration (dimensions 24.5  24.5 in/62.2  62.2 cm), a long vertical
rectangular lore conguration (dimensions 20  30 in/
50.8  76.2 cm) and a long horizontal rectangular shore conguration (dimensions 30  20 in/76.2  50.8 cm). Judges were randomly assigned to a shape sequence. No judge was aware of the
purpose of the study, but some noticed the changes in the shapes
over the three sessions in the exit talk with the experimenter.
Wines 7 (SF11) and 10 (MS91) were presented in blind duplicate.
2.2.4. Data collection
All product positions (x- and y-coordinates) were measured by
hand using a commercial sewing measurement tape from the bottom left corner to the center of the glass. For any further data analyses all coordinates obtained were standardized and expressed as
relative coordinates to the maximum value of the respective shape
dimension. If judges provided descriptive terms for wines and/or
wine groups these terms were collected and summarized in a frequency table. Similar terms given by different judges or in different
sessions were combined to one (e.g. red fruit and red berry).
For the statistical evaluation descriptors terms were used which
were provided at least 9 times. Terms that were constructed of different words (e.g. low fruit) were split and added to the low
group as well as the to the fruit attribute.
2.3. Descriptive analysis
A more extensive description of the DA can be found in (Hopfer
et al., 2012). Therefore, we will provide only a short summary. The

18 wines were evaluated about 3 months after bottling using the


methodology described in detail in Lawless and Heymann (2010).
The DA panel consisted of 14 people (four females, 10 males, age
2435 years), and all panelists had had previous experiences with
DA. Panelists completed six 1-h long training sessions over a period
of 2 weeks, where they were exposed to subsets of the wines, and
asked to generate attributes to describe differences among the
wines. Each wine was blindly presented twice during these training sessions. Aroma, taste and mouthfeel attributes were generated
and reference standards for these attributes were provided to anchor and obtain consensus on the attributes. In total, the 23 attributes and their reference standards were chosen by the panel and
used in the wine evaluation. All judges completed nine wine evaluations tasting six wines in each session. Wines were served in a
randomized William Latin Square incomplete block design to control for possible carry-over effects. The wines were evaluated in
individual temperature-controlled tasting booths with positive airow. Water (Arrowhead, Nestle Waters America, Stamford, CT,
USA) and unsalted crackers (Nabisco unsalted top premium saltine
crackers, Kraft Foods, Northeld, IL, USA) were provided for palate
cleansing. Prior to each assessment session panelists smelled the
aroma standards to refresh their memories. Between each wine a
1 min break and an additional 3 min break between the third
and fourth wine were added to decrease fatigue. During each break
panelists chewed on a cracker and then rinsed their mouths with
water to clean their palates. The judges rated the aroma, taste
and mouthfeel attributes on a 10 cm unstructured line scale anchored from low to high provided by FIZZ sensory analysis
software (version 2.54A, Biosystmes, Couternon, France). The
wines (25 ml aliquots) were presented in clear pear-shaped ISO
glasses (ISO 3591:1977) labeled with three digits random number
codes under red light to prevent color bias. All judges rated each
wine in triplicate during the 3-week evaluation period. Judges
did not know how many samples or replicates they were
evaluating.
2.4. Data analysis
For all statistical data analyses we used R (R Foundation for Statistical Computing., 2011). Canonical Variate Analysis (CVA) implemented into R in the candisc package (Friendly & Fox, 2010) after
multi and univariate Analysis of Variances ((M)ANOVA) on the
DA data were used to determine the signicant attributes and to
plot the product map. Further details for the statistical data analyses of the DA data can be found in (Hopfer et al., 2012).
For the PM data multi-factor-analysis (MFA) as described by
Abdi and Valentin (2007), Escoer and Pags (1994) and BcueBertaut and Pags (2008) was used to produce the product plots.
MFA is implemented into R in the FactoMineR package (Husson,
Josse, Le, & Mazet, 2012; Le, Josse, & Husson, 2008). In addition,
hierarchical cluster analysis on the MFA dimensions was performed using the HCPC() function in the FactoMineR package to
nd similarities among the samples. Euclidean distances and
Wards linkage were used.
Judge consistency was evaluated using Bertucciolis idea to calculate a people performance index (PPI) (Bertuccioli, 2011). The
PPI is the ratio of the Euclidean distance between two replicated
products and the maximum Euclidean distance between two different products in the PM plot. The PPI ranges between 0 and 1 and
gets smaller the closer the judge places identical samples together.
The people performance index (PPI) for a given judge and replicate
is dened as the ratio of the Euclidean distance between the duplicated samples and the maximum Euclidean distance between all
samples:

PPI dduplicate =max dp; q

H. Hopfer, H. Heymann / Food Quality and Preference 28 (2013) 164181

The RV coefcients a coefcient between two variable vectors


implemented in the FactoMineR package were calculated to express similarities between the various congurations and the
goodness-of-t of the consensus product positions. The RV coefcient was introduced by Escouer as . . . a measurement of similarity between squared symmetric matrices . . . (Abdi, 2007).
Different authors consider an RV coefcient of about 0.7 a good level of agreement (Cartier et al., 2006; Faye et al., 2004; Nestrud &
Lawless, 2008; Tang & Heymann, 2002). However, one is tempted
to believe, that a high RV coefcient indicates also statistical significance, but as pointed out by Josse, Pags, and Husson (2008) the
RV value depends on the number of products and variables, and
the structure of the specic data set. They also discuss signicance
testing for the RV coefcient (i.e. to measure whether the RV coefcient could have been obtained by chance alone) with the use of
permutation tests (Abdi, 2010). MFA and RV coefcients were also
used to compare the results from the DA and the two PM studies.
2.4.1. PM study 1
Two data matrices were collected: (i) the x- and y-coordinates for
each judge, replicate and shape, and (ii) a frequency table with all
descriptive terms used by the judges in all replicates for both shapes.
We analyzed each shape (square and rectangle) and replicate separately to study the differences in the overall consensus product plots
(i) among the three replicates and (ii) between the two provided
spaces using MFA and calculated the RV coefcients. To analyze if
the results differed depending on the spatial shape (square vs.
rectangle) we performed a xed-effect ANOVA on the x- and ycoordinates using wine, replicate and shape as main effects with
the judges being nested within the shape or averaged over the judges
within one shape as well as all two-way interactions. The ANOVA
was also performed to study whether judges use the x- and y-coordinates in a similar way, independent of what attributes they use for
their sample separation along the two axes. It is true that each judge
uses the x- and y-coordinates in a different way (i.e. different attributes a.k.a. separation criteria), however, the ANOVA allows us to
check for different use of the horizontal or vertical axes, and whether
this effect can be attributed to the different lengths of the two axes
when using a rectangular space compared to a squared space.
2.4.2. PM study 2
In the second study each of the 22 judges positioned the wines
on three different shapes. Similar to study 1 we had two matrices:
Table 3
People performance indices (PPI) for all judges in PM study 1 for both shapes. Italic
values represent the minimum value for that column.
Replicate 1

PPI1a

Replicate 2

Replicate 3

PPI2a

PPI1a

PPI2a

PPI1a

PPI2a

Rectangle
gh
0.15
cd
0.89
ah
0.35
md
0.27
tj
0.14
sl
0.31

0.20
0.34
0.33
0.36
0.38
0.75

0.10
0.80
0.33
0.51
0.55
0.39

0.44
0.09
0.58
0.44
0.53
0.58

0.30
0.55
0.65
0.08
0.29
0.24

0.89
0.28
0.09
0.28
0.55
0.41


xs

0.4 0.28

0.4 0.18

0.5 0.23

0.4 0.19

0.4 0.21

0.4 0.28

Square
kc
sj
ac
tc
mk
bl

0.09
0.54
0.44
0.39
0.70
0.37

0.09
0.39
0.82
0.43
0.81
0.15

0.49
0.22
0.22
0.25
0.38
0.35

0.20
0.11
0.39
0.12
0.59
0.46

0.25
0.63
0.77
0.58
0.73
0.11

0.56
0.32
0.30
0.39
0.65
0.21


xs

0.4 0.21

0.5 0.31

0.3 0.11

0.3 0.20

0.5 0.27

0.4 0.17

Judge

PPI1, PPI calculated for the SF91 blend; PPI2, PPI calculated for the SMF111
blend.

171

(i) the x- and y-coordinates for each judge wine and shape and (ii) a
frequency table with all descriptive terms given by the judges over
the three sessions for all shapes. We analyzed each of the three
shapes separately to study the differences in the overall consensus
product plots using MFA. RV coefcients were calculated to express similarities among the various congurations and the goodness-of-t of the consensus product positions.
To analyze if the obtained results differed depending on the
shapes (square vs. lore vs. shore) we performed a xed-effect
ANOVA on the x- and y-coordinates using wine, judge and shape as
main effects with all two-way interactions. Pairwise t-tests were
performed on the coordinates to see whether judges used the axes
similarly in the different congurations.
3. Results and discussion
3.1. People performance index (PPI)
For both studies two wines were present in duplicate in the
sample set to check whether the judges were able to position them
together. People performance indices (PPI) were calculated as described in Section 2. For the rst PM study the same judges evaluated the wines in triplicate, thus, allowing us to detect replicate
variations. Tables 3 and 4 show the calculated PPI for both studies:
in PM study 1 (Table 3) the lowest PPI values (0.08 and 0.09) were
found in both groups for judge md in the third replicate for PPI1,
for judge ah in the third replicate for PPI2, and for judge kc for both
PPI1 and PPI2 in replicate 1. The square group showed lower overall mean PPI values with a smaller standard deviation. However,
the reported PPI values did vary greatly for each judge over the
three replicates and also among the judges. Two judges from each
shape group improved from the rst to the third replicate (judge
cd, sl, md and ac), but most judges did not improve over the three
replicate sessions. Judging from these results, the positioning of
these wine samples created a difcult task for the judges.
In the second PM study (Table 4) the lowest PPI value was found
in the long vertical rectangle lore (0.10), closely followed by the
two other shapes (0.11). Overall, the PPI means were between 0.4
and 0.6 with relative standard deviations between 39% and 50%,
and with lower PPI values for the second wine duplicate (MS91).
Generally, the PPI values were higher in the second PM study:
The highest overall means were found for PPI1 in the short vertical
rectangular shore conguration, however, the observed differences among the three congurations were not signicantly different due to the large standard deviation values of up to 50%.
We believe that the inclusion of blind duplicated samples in the
PM sample set is a useful tool to check for individuals variability
and for difculty of the PM task. More studies have to be conducted
to establish guidelines what a good or useful PPI value might be.
3.2. How different are individual judges and how much does individual
oddity inuence the overall panel solution?
We were interested in individual PM techniques and found that
in PM2 some judges arranged their samples in unique ways. Most
judges used a Cartesian system with a horizontal and a vertical
dimension, with either aroma or taste-related differentiation and
grouping criteria, and they arranged their groups in a rather
unstructured way. However, we were able to identify three judges
who did something quite different (see Fig. 2): The rst example
(Fig. 2a) shows how judge mc arranged the samples in a circular
fashion, similar to a polar coordinate system, with the polar coordinate being intensity ranging from low in the middle to high on
the outside. The second dimension (i.e. the angular coordinate)
ranges from bitter in the top right corner over increasing smooth

172

H. Hopfer, H. Heymann / Food Quality and Preference 28 (2013) 164181

Table 4
People performance indices (PPI) for all judges in PM study 2 for all three shapes. Italic values represent the minimum value for that column.
Short vertical rectangle shore

Judge

PPI1

ac
ah
aj
dw
ek
hs
ht
hy
jb
lb
mc
mh
ms
mw
nd
pb
sf
sl
sv
tc
tfo
tfr

xs

Long vertical rectangle lore


a

Square squ
PPI1a

PPI2a

0.17
0.13
0.96
0.53
0.20
0.36
0.38
0.63
0.74
0.14
0.48
0.67
0.62
0.12
0.58
0.11
0.65
0.34
0.38
0.40
0.10
0.50

0.63
0.22
0.77
0.37
0.22
0.65
0.57
0.29
0.72
0.32
0.56
0.79
0.35
0.72
0.36
0.47
0.26
0.61
0.60
0.50
0.28
0.32

0.25
0.67
0.52
0.16
0.85
0.11
0.42
0.49
0.50
0.68
0.41
0.37
0.51
0.68
0.18
0.29
0.31
0.37
0.13
0.29
0.81
0.34

0.4 0.24

0.5 0.19

0.4 0.21

PPI2

PPI1

PPI2

0.38
0.64
0.63
0.91
0.59
0.65
0.44
0.73
0.41
0.85
0.93
0.80
0.36
0.80
0.39
0.49
0.15
0.43
0.17
0.19
0.83
0.52

0.68
0.26
0.31
0.35
0.59
0.49
0.78
0.30
0.10
0.76
0.46
0.56
0.34
0.44
0.19
0.41
0.58
0.19
0.58
0.42
0.11
0.40

0.64
0.13
0.41
0.41
0.76
0.63
0.29
0.35
0.57
0.59
0.63
0.39
0.81
0.75
0.12
0.36
0.30
0.21
0.19
0.50
0.50
0.35

0.6 0.24

0.4 0.19

0.5 0.20

PPI1, PPI calculated for the MS91 blend; PPI2, PPI calculated for the SF11 blend.

sweetness in the top middle of the plot to musty in the bottom left
corner. In the second example (Fig. 2b) judge mh tried to come up
with different separation criteria for each session, and chose in the
third session to arrange the samples along the vertical dimension
from low to high label numbers, after they had been grouped
according to their complexity along the horizontal axis. Judge tfr
(Fig. 2c) divided the provided space into increments and positioned
the samples belonging to one group in one of these increments. It
seems that along the horizontal axis the samples were separated
according to their fruity character with high red fruit and cherry
samples on the left side and green, herbal samples on the right
side. Unfortunately the vertical differentiation could not be recovered from the judges notes. Likely, this judge did not separate the
samples along the vertical axis, thus the judge sorted the wines in
rather than using two-dimensional PM. We did not exclude any of
these judges from the data analysis nor changed the obtained coordinates in any way (e.g. converting them to true Cartesian coordinates via trigonometric functions). Removal of these judges from
the data set did not seem right as all of them followed the instructions, positioned the samples in two dimensions, and were able to
put the duplicated samples close to each other (see PPI values in
Table 4). In addition, we re-ran the MFA of all three shapes without
these three judges one at a time to check the individual impact of
these unusual PM techniques on the overall PM solution. No
changes in the RV coefcients were observed by the removal of
any of the three judges; for judge tfr and mc we ran the MFA
removing all three shapes as both did the incremental or circular PM for all three provided spaces, for judge mh we only removed the long vertical rectangle lore data as this was the only
shape where the random label numbers were used as a separation
criterion.
Based on these ndings we assume that the PM solution from
PM study 2 is rather stable and robust against out-of-the ordinary
PM arrangements. However, we also think that this robustness is
most likely due to the high number of judges (21), and speculate
that with a lower number of judges and/or more unique PM techniques the overall MFA solution will change. PM experimenters
need to keep this fact in mind.

3.3. The effect of replicates


We investigated the effect of replicates on the PM solution of
study 1 running (i) a three-way xed effect ANOVA with judge,
wine and replicate for the two shapes separately, (ii) a three-way
xed effect ANOVA on wine, replicate and shape with judges being
nested within the shape, and (iii) a three-way xed effect ANOVA
on wine, replicate and shape averaged over the judges (Supplementary Tables 1 and 2). No signicant replicate effects were observed for the x- and y-coordinates (P 6 0.05).
In Fig. 3af we present the product plots from the rst PM study
for each replicate for both shapes next to each other including the
results from the hierarchical clustering: In all plots the three base
wines (Cabernet Sauvignon S, Merlot M and Cabernet Franc F) are
positioned far from each other, and in most cases the three base
wines are grouped in different clusters with the exception of the
rst and second replicate of the square shape (Fig. 3b and d), the
third replicate of the rectangle shape (Fig. 3f). In most plots the
Cabernet Sauvignon-based blends (SF91, SM91, SM82, SMF811,
SF82) and Merlot-based blends (MS91, MS82, MF91, MF82) were
located close, and clustered together with their major monovarietal wine Cabernet Sauvignon S, and Merlot M respectively.
Similar to previous studies (Barcenas et al., 2004; Kennedy,
2010; Risvik et al., 1994) the product plots differed drastically from
one replicate to the other. However, similarly to the results of Kennedy on granola bars (Kennedy, 2010), the overall similarities and
dissimilarities among the samples were conserved over the triplicate evaluation. We highly recommend replicated PM studies to
ensure that the observed similarities and dissimilarities can be
repeatedly picked up by the judges (Table 5).

3.4. The effect of shapes


3.4.1. Square vs. rectangle (PM study 1)
The differences in the shapes were studied in three ways: comparing (i) the explained variances of the MFA solutions for the rectangle and the square group, (ii) the two MFA product plots in

H. Hopfer, H. Heymann / Food Quality and Preference 28 (2013) 164181


20

(a)
MS91b

SM11

MS91a

SMF811

SF11a

SF82
SF91

Blend

MF11

SMF111
MF82

SM82
M

10

MF91

MS82

SF11b

SM91

MSF811

sweet

bitter
low

musty
high
0
0
20

10

20

30

(b)
SF11a

SM82

MS91b

MF11

574

648

734

795

MSF811

417
MS91a
10

SF91

797

810

933

SF82

MS82

SM91

533

644

747

SF11b

MF82

193

359

449

SMF111 SMF811

429

544

566

SM11

Blend

MF91

015

075

256

high number on label


less complex

more complex

low number on label


0
0

10

20

MF82

SF82

cherries,
some
wood, no
veggies,
low
pepper,
no tannin,
low sugar

10

MSF811

20

SM11

Blend

SMF811

MS82

SM91

SF91

MF91

MS91b

SF11b
SM82

herbal,
veggie,
little fruit,
viscous,
medium
pepper,
mushroom

SMF111
SF11a

red fruit, low


alcohol,
herbal, med
pepper,
viscous,
med tannin

higher alc,
low fruit,
veg
strong,
watery,
low
pepper,
chocolate

30

cooked
fruit,
cooked
veg, salt/
pepper,
tannin,
earthy,
oak

MS91a

MF11

tannin, medium
alc, some wood,
low sugar

(c)
0
0

10

20

30

Fig. 2. Examples of individual PM techniques: (a) circular PM, (b) using random
three-digit label numbers as separation criterion, and (c) incremental PM.

combination with the descriptor plots and the results from the
hierarchical clustering, and (iii) the results from the ANOVAs.
As described in Section 3.3, the square conguration showed
higher variability from replicate to replicate and therefore, it is
not surprising that the overall solution (using all data from all
three replicates) for the square conguration showed a smaller explained variance within the rst two dimensions than the rectangle one (25% vs. 34%) (Table 6).
In the rectangle conguration 34% of the variance could be explained within the rst two dimensions with 20% accounting for
the rst one. In the product plot shown in Fig. 4a the Cabernet
Franc F is clearly separated from the rest of the samples in the
top right corner sharing the same quadrant as MF11, the winemakers blend, SF11 and SMF111A. With the exception of the last

173

wine, all these wines were also clustered together. These wines
were primarily described with vegetative, berry, vinegar and alcohol avors and sour taste. The second monovarietal wine Merlot
M is located in the bottom right corner and shares its quadrant
with Merlot-based blends (MF82, MS82, MS92, MF91, MSF811)
and the ternary mixture SMF111B. All these wines and the
SMF111A share one cluster. Descriptions of various berries, butter
and fried fruits were collected from the judges for these wines.
The whole negative horizontal area are occupied by the Cabernet Sauvignon S (along the negative x-axis) and various Cabernet
Sauvignon-based blends (SM82, SF82, SF91A & B, SF82, SM91,
SMF811) and the binary mixture of the two Cabernets (SF11),
which were all grouped together by hierarchical clustering. This
side of the product plot was correlated to smoky, oaky, spicy, sulfury and chocolates avors and high astringency mouthfeel.
In the cluster analysis both duplicated wines (SMF111 and
SF91) were clustered together (SMF111 in cluster 2, SF91 in cluster
1), an indicator that over all three replicates and all judges the similarity of these wines was detected.
In contrast to the rectangle product plot stands the square one
(see Fig. 4b): In this product plot both monovarietal Cabernets (S
and F) are clearly separated from the remaining samples and drive
the differentiation along the rst dimension accounting for about
13% of the total variance with chemical, plum, fruit and astringency correlating to the Cabernet Sauvignon S and dirt, burnt, pepper and sour correlating to the Cabernet Franc F. All remaining
wines are located in between those two wines. In the lower middle
part of the product map are all Cabernet Sauvignon-based blends
located (SM82, SM91, SMF811, SF91A & B, SF82, SF11), which were
clustered into cluster 3 except SM91 and SF11 (cluster 4). Closest to
the Cabernet Franc F is the binary mixture of this monovarietal
wine with Merlot (MF11) these two wines were also put into
one cluster (cluster 5). The latter wine was highly correlated to veggie, while the Cabernet Sauvignon-based blends were mainly characterized by perfume, Brett1, oxidized, oak and leather avors and
tannic mouthfeel. The upper part of the plot is dominated by the third
monovarietal wine (Merlot M), which is grouped together with the
Merlot-based blends (MF91, MS82, MF82, MS91, MSF811) and the
binary mixture SM11 (cluster 2). The ternary mixture blend SMF111A
& B were located around the plot center and clustered together with
SM91 and SF11 in cluster 4. These wines showed descriptors ranging
from jammy, butter, spice and vanilla to dark fruits.
Similarly to the rectangle product plot, the two duplicated
wines (SMF111 in cluster 4 and SF91 in cluster 3) are both located
close to each other indicating that over all three replicates the six
judges in the square group experienced the wines very similar. The
separation along the x-axis along the two Cabernet monovarietal
wines was found in the DA too (see CVA product plot in Fig. 6);
however, all other wines were not similarly positioned. This nding is in good agreement with Risvik et al. (1994), who found that
major differences were detected by trained DA judges and untrained PM consumers in a similar way, but only the DA panel
was able to nd subtle and smaller differences among the samples.
Using both congurations (rectangle and square) and performing a xed effect ANOVA on wine, replicate and shape with the
judges being nested in the shape, no signicant effect of the shape
was found. However, using all data averaged over the judges the xcoordinates showed a signicant shape effect indicating that the
position of the wines differed between the two shapes along the
horizontal x-axis (P 6 0.05). This is an interesting nding as the difference between the two shapes for both axes was about the same
1
Brett refers to the volatile compounds produced by the yeast Brettanomyces
bruxellensis that is generally considered a spoilage yeast in wine production. The
aroma can be described as sweaty horse, sweaty leather saddle, barnyard, leather or
Band-Aid.

174

H. Hopfer, H. Heymann / Food Quality and Preference 28 (2013) 164181

(a)

(b)

(c)

(d)

(e)

(f)

Fig. 3. MFA product plots from the rst PM study for the three replicates using the rectangular shape (a, c, e) and the three replicates for square shape (b, d, f), with added
hierarchical clusters (cluster 1, solid black circle; cluster 2, solid red triangles; cluster 3, solid blue squares; cluster 4, solid purple diamonds; and cluster 5, cyan stars).
Clusters were partitioned after a high relative inertia loss. (For interpretation of the references to colour in this gure legend, the reader is referred to the web version of this
article.)

(5 inches) the square conguration was 27.5 inches in both axes


while the rectangle was 32 inches long on the x-axis and 22 inches
on the y-axis but the judges seemed to respond only to the
changes along the x-axis (Supplementary Table 2).
3.4.2. Square vs. long vertical rectangle (lore) vs. short vertical
rectangle (shore) (PM study 2)
The differences in the shapes of the three different spaces were
studied in three ways: comparing (i) the explained variances from

the MFA solution for the three provided spaces, (ii) the three product plots in combination with the descriptor plots with the results
of the hierarchical clustering, and (iii) the results from the
ANOVAs.
In Table 7 the eigenvalues and explained variances for the three
congurations are shown for the rst three dimensions. We ran the
MFAs with and without the descriptor matrix to study the effect of
the descriptive terms on the solution, and found that for all
analyses the explained variances and the eigenvalues were slightly

H. Hopfer, H. Heymann / Food Quality and Preference 28 (2013) 164181


Table 5
RV coefcients for the overall MFA solution from PM study 1 (coordinate matrix c,
descriptor matrix w, rectangle r, square s, replicate 1, 2, 3).

c_r2
c_r3
w_r
c_s1
c_s2
c_s3
w_s
MFA

c_r1

c_r2

c_r3

w_r

c_s1

c_s2

c_s3

w_s

0.39
0.53
0.62
0.29
0.43
0.41
0.58
0.72

1.00
0.53
0.45
0.29
0.40
0.46
0.60
0.72

1.00
0.51
0.35
0.42
0.50
0.56
0.77

1.00
0.24
0.50
0.37
0.55
0.71

1.00
0.36
0.41
0.38
0.59

1.00
0.42
0.50
0.69

1.00
0.50
0.73

1.00
0.81

Table 6
Eigenvalues and explained variances for the rectangle (rect), square (squ) and both
shapes (all) in PM study 1 for the rst three dimensions (coordinate matrix c,
descriptor matrix w).
Dim. 1

Dim. 2

Dim. 3

Expl. Var.

Expl. Var.

Expl. Var.

rect_r1
rect_r2
rect_r3

1.7
1.6
1.7

30.9
20.1
22.2

1.0
1.3
1.3

18.4
15.4
17.7

0.6
1.0
1.0

11.6
12.2
12.4

rect
squa_r1
squa_r2
squa_r3

3.0
1.3
1.4
1.5

20.2
14.8
19.4
18.2

2.0
1.3
1.3
1.3

14.0
14.1
17.1
15.5

1.4
1.1
0.9
1.0

9.4
12.6
12.5
11.8

squa
all

2.3
4.9

12.8
15.2

2.2
3.7

12.2
11.5

2.0
3.0

10.9
9.2

175

higher if we included the descriptor matrix (data not shown). All


three congurations explained about the same of the total variance
with about 1315% in the rst dimension. The long vertical rectangular lore shape could explain the highest degree of variance
within the rst two dimensions (26.4%).
Fig. 5ac shows the MFA plots for the three different shapes. For
both rectangle congurations (shore (Fig. 5a) and lore (Fig. 5b))
around 26% of the total variance could be explained within the rst
two dimensions, while using the square conguration (Fig. 5c) 2%
less could be explained. For the two rectangle congurations the
positions of the three monovarietal base wines (Cabernet Sauvignon S, Merlot M, Cabernet Franc F) in the product plots were similar: In contrast to the results of the DA and the rst PM study the
monovarietals did not express the extremes of the product spaces
(i.e. being located on the far ends of the plot), but were located near
the center or at least in a middle area near the center. Additionally,
in the shore and lore conguration the monovarietal Merlot
and Cabernet Franc (M and F) were clustered together in cluster
1, while the monovarietal Cabernet Sauvignon S was in a separate
cluster (cluster 4, and 3, respectively). The exception was the
square conguration, where the Merlot M and the Cabernet Franc
F were separated along one vertical line on the negative rst
dimension in two different clusters (cluster 1 (M) and 2 (F)), but
the Cabernet Sauvignon (S) was in the same cluster as F.
For the shore conguration (Fig. 5a) the product plot spans
from the binary Cabernet blend SF11B (cluster 1) to the positively
loaded winemakers blend (cluster 3), and is characterized along
the rst dimension by the green, vegetal, herbal, pepper, medicinal
and red fruit descriptors on the negative side and bitter, vanilla,
oak-toast and fruit attributes on the positive rst dimension.

(a)

(b)

Fig. 4. Overall MFA solutions with marked hierarchical clusters for the rst PM study separated for the two shapes: (a) rectangle PM space (cluster 1, solid black circles;
cluster 2, solid red triangles; cluster 3, solid blue squares), (b) square PM space (cluster 1, solid black circles; cluster 2, solid red triangles; cluster 3, solid blue squares; cluster
4, solid purple diamonds; cluster 5, cyan stars). Clusters were partitioned after a high relative inertia loss. (For interpretation of the references to colour in this gure legend,
the reader is referred to the web version of this article.)

176

H. Hopfer, H. Heymann / Food Quality and Preference 28 (2013) 164181

Similarly, is the negative second dimension dominated by sweet,


veg and canned veg descriptors, while the positive second dimension is correlated to Brett, wood, sulfur and tannin. The second
dimension spans from the positively loaded blends MF82 and
SM82 (both cluster 3) to the negatively loaded MS91B and
SMF811 wines (both cluster 2).
Similar to the short vertical rectangle shore conguration, in
the long vertical rectangle lore conguration (Fig. 5b) all three

monovarietal base wines were not located on the outer sides of


the product plot, indicating that blends rather than the monovarietals represented the extremes of the product space. For the lore
congurations these extremes ranged along the rst dimensions
from the negatively loaded SF11B (in cluster 2), blend and MS82
(both in cluster 1) to the positively loaded SM82 and MF11 (both
cluster 3), and along the second dimension from the negatively
loaded SF11B (cluster 2) and SM82 (cluster 3) to the positively

(a)

(b)

(c)

Fig. 5. Overall MFA solutions (product maps and descriptor plots) with marked hierarchical clusters for the second PM study separated for the three shapes: (a) short vertical
rectangle shore, (b) long vertical rectangle lore, (c) square squ. Descriptors for the shore and the squ space have a _s and _sq appendix respectively (cluster 1,
solid black circles; cluster 2, solid red triangles; cluster 3, solid blue squares), (b) square PM space (cluster 1, solid black circles; cluster 2, solid red triangles; cluster 3, solid
blue squares; cluster 4, solid purple diamonds). (For interpretation of the references to colour in this gure legend, the reader is referred to the web version of this article.)

H. Hopfer, H. Heymann / Food Quality and Preference 28 (2013) 164181

(a)

177

(b)

Fig. 6. MFA plots from PM study 1 (a) compared to the CVA plots from the DA (b). Shown are the product plot (top), the variable plot (middle), and the group representation
(bottom), with the latter one not available for the DA data.

loaded MS82, MF82 (both cluster 1) and MF11 (cluster 3). Correspondingly, the rst dimension was correlated to chemical, fruit
and smooth on the negative axis and smoke, Brett2 and astringency
on the positive axis. On the second dimension the samples were
separated along the negatively loaded sour attribute and the positively loaded sulfur, pepper and veg attributes. In addition, loadings on the 45-degree angle differentiated between smooth (top
left quadrant) and the oppositely loaded harsh descriptor (bottom
right quadrant).
In the square conguration (Fig. 5c) three wine clusters could be
identied, such as cluster 1 consisting of Merlot M and other Merlot

based blends (MS91B, MSF811) and SF11B, showing a positive correlation to darkfruit, green and wood. Cluster 2 included F, SM11,
MF11, SF82, SF11A, SMF111, SM91, which showed a high positive
correlation to the descriptors smoky and dark fruit, with the Cabernet Franc F mainly characterized by veg and pepper attributes. Three
additional members of cluster 2 (SM91, Blend and S) were positively
correlated to caramel, oral and oak aromas and sweet taste. The
very same attributes also characterized the Cabernet Sauvignon
based blends SM82, SMF811 and MF82, which were clustered in
cluster 3, together with SF91 and MS91a, which were described by
wood, green, oak, astringency and fruit descriptive terms.

178

H. Hopfer, H. Heymann / Food Quality and Preference 28 (2013) 164181

Table 7
Eigenvalues and explained variances for the square (squ), the long vertical rectangle
(lore), the short vertical rectangle (shore) and all three shapes (all) in PM study 2 for
the rst three dimensions (coordinate matrix c, descriptor matrix w).
Dim. 1

lore_c
lore_cw
shore_c
shore_cw
squ_c
squ_cw
all_c
all_cw

Dim. 2

Dim. 3

Expl. Var.

Expl. Var.

Expl. Var.

4.3
1.5
4.8
1.4
4.2
1.5
9.6
3.4

13.1
14.9
14.1
14.0
12.3
12.9
9.7
10.6

3.5
1.2
4.1
1.2
3.7
1.3
9.0
2.8

10.7
11.5
11.9
11.9
10.9
11.6
8.8
8.8

3.1
1.0
3.3
1.0
3.6
1.1
7.3
2.4

9.3
9.6
9.6
9.7
10.5
9.8
7.2
7.6

Judging from the MFA product and variable plots together with
the hierarchical clustering, panelists positioned their wines differently depending on the provided space: While the two rectangle
congurations showed more similarity to each other (e.g. in both
cases the monovarietal Merlot and Cabernet Franc was clustered
together), was the opposite observed for the square space, where
the two monovarietal Cabernets shared the same cluster 2.
A xed effect ANOVA with two-way interactions showed a signicant shape and judge effect for the y-coordinates but not for the
x-coordinates (P 6 0.05) (Supplementary Table 3). We were interested whether our judges positioned the wines differently depending on the provided space (square vs. long vertical rectangle lore
vs. short vertical rectangle shore) and ran paired t-tests on the xand y-coordinates of the three congurations (Table 8). It seems
that the judges in the second study changed their arranging pattern according to the provided space: They used a long horizontal
dimension in the short vertical rectangle shore conguration
similarly to the long vertical dimension in the long vertical lore
rectangle conguration, i.e. they used the larger of the two dimensions similarly, independent of the orientation. The short vertical
dimension in the short vertical rectangle shore conguration
was used in the same way as the vertical dimension in the square
space. However, for the long vertical rectangle lore no signicant
difference could be found between the horizontal and vertical
dimension, and both were used similarly to the horizontal dimension of the square. One explanation for this could be that not all
judges used the whole length of the vertical dimension and used
a square space for their PM task. In both PM studies, a clear shape
effect on the sample positioning was found. For future PM studies,
the experimenter should be aware that the shape of the PM space
does have an impact on the outcome of the study.
3.5. How well does it t? Comparison to the DA
The last remaining question we tried to answer was how well
the MFA solutions of the PM studies related to the CVA we obtained from the DA data. Due to the 1 year bottle ageing in between
the PM1 and the DA to the second PM study, we only compare the
results from PM study 1 to the CVA on the DA data (Fig. 6a and b):

Table 8
Paired t-test values (a at 5%) for the horizontal (X) and vertical (Y) dimensions of the
three shapes in PM study 2 (short vertical rectangle shore, long vertical rectangle
lore, square squ).
shore_X
shore_Y
lore_X
lore_Y
squ_X
squ_Y

shore_Y

lore_X

lore_Y

NS
NS

NS

squ_X

NS

NS

From the PM study we obtained three plots, namely (i) a MFA


product plot, (ii) a MFA variables plot, and (iii) a group representation plot (Fig. 6 left and middle column). In a group representation
plot individual data matrices (coordinates from the different replicates and shapes, and descriptors for both shapes) are plotted together to express their (dis) similarity in a graphical way; the
closer two data matrices are to each other the more similar they
are.
In PM study 1 the highest similarity according to the group representation plot (Fig. 6a bottom) was found for the coordinates of
the rst and third replicate and the provided descriptors in the
rectangle group. Within the rst two dimensions about 26.7% of
the total variance could be explained. Similar to the previous product plots for PM study 1 we found the three monovarietal base
wines positioned in three different quadrants with their respective
blends located around them in four different clusters (Fig. 6a top).
The middle plot in Fig. 6a shows the loadings of the descriptive
terms for the rectangle group and the square group. Similar
descriptive terms were chosen in both groups, and we observed
some overlap of descriptors e.g. veggie in the square group and
veg in the rectangle group.
In order to compare the ndings of the PM study we also conducted a DA on the wines shown in Fig. 6b: Similar to the PM
the three monovarietal base wines (Cabernet Sauvignon S, Merlot
M, Cabernet Franc F) are located in three separate quadrants, often
representing the extremes of the product space. A signicant higher portion of the total variance ratio could be explained within the
rst two dimensions by the DA panel (52.5% within the rst two
dimensions). The Cabernet Sauvignon (S) located in the bottom left
corner was mainly characterized by high loadings of the signicant
attributes soysauce, sulfur, astringency and sour, and the absence
of fruit, black pepper, sherry (all signicant at 5%), canned veg,
fresh veg and lactic (all not signicant (P 6 0.05)). All Cabernet
Sauvignon based blends grouped together in the top left quadrant,
showing a high correlation to medicinal, oak, earthy and the not
signicant watery attributes.
The Merlot and all Merlot based blends grouped in the bottom
right corner showing a high similarity to each other. These wines
were mainly described as being high in chemical, oral, sweet
and hot mouthfeel attributes. In the top right corner the third
monovarietal wine (Cabernet Franc F) was grouped with its respective blends (winemakers Blend, SF11, SMF111 and SM11). These
wines were highly correlated to the signicant fruit, black pepper
and sherry descriptors, as well to the not signicant attributes
fresh and canned veggie, lactic and vanilla.
Comparing the product plots (PM MFA plot and DA CVA plot)
some differences and some similarities could be observed: The plot
from the PM and the DA study are quite similar to each other in
both plots the three monovarietal wines (S, M, F) represent the extremes of the product space. All blends are positioned in between
these extremes. Comparing the descriptive terms used in the DA
and given by the judges in the PM study (Table 9), it seems that
all judges picked up similar sensory impressions as all major DA
attributes (i.e. the ones that were signicantly different
(P 6 0.05)) were chosen in the PM study as well). Interestingly,
attributes associated with intensity such as low, mild or smooth
in contrast to harsh were chosen by the judges in the PM. This
might be an indicator that some kind of intensity was chosen as
one of the separating dimensions, and that PM judges intuitively
do what is part of a DA scoring the intensities of attributes. As
the DA terms often incorporated several single impressions (e.g.
the term fruit included red and dark fruit impressions) all the
descriptors from the PM studies describing some kind of fruit characteristics were summarized. For a more detailed description of all
the DA attributes we refer you to the original DA work (Hopfer
et al., 2012).

179

H. Hopfer, H. Heymann / Food Quality and Preference 28 (2013) 164181


Table 9
Descriptors used in the PM studies and in the DA (descriptors in bold were signicant at an a level of 5% in the DA).
DA

Fruit

Floral
Oak

PM1
Rectangle

Square

Square

Shore

Lore

Fruit
Black fruit
Red berry
Berries

Fruit
Dark fruit
Plum

Fruit
Dark fruit

Fruit
Red fruit

Fruit
Dark fruit

Perfume
Oak
Burnt
Brett
Leather

Floral
Oak toast
Smoky

Oak toast

Oak
Smoky

Medicinal

Soysauce
Earthy
Chemical

Sulfur

Sherry
Black pepper
Sweet
Sour
Astringent

PM2

Alcohol
Vinegar

Dirt
Chemical

Sulfur

Dried fruit

Oxidized

Sour
Astringent

Sour
Astringent

Smoke
Medicinal
Brett
Barnyard
Phenolic
Animal

Alcohol

Sulfur
H2S
Rubber
SO2

Sulfur
H2S
Rubber
Reduced

Sweet

Sweet sugar

Astringent
Dry
Tannin

Hot
Fresh veggie

Canned veg
Menthol-cedar
Butter creamy chocolate
Lactic
Vanilla caramel
Spice
Bitter
Watery

Veggie
Pepper

Veggie
Pepper
Stemy

Cooked veggie

Veggie
Pepper
Green
Herbal
Grassy
Canned veg

Wood
Chocolate
Butter

Butter

Spice

Vanilla
Spice

Veggie
Pepper
Green
Herbal

Brett
Barnyard
Phenolic

Chemical
Solvent
Plastic
Cardboard
H2S
Rubber

Sweet
Acidic sour
Astringent
Dry
Tannin
Hot
Veggie
Pepper
Celery

Canned veg
Wood resin

Chocolate

Caramel
Spice

Low
Jam

A third way to compare how similar the MFA product plots from
the PM study are to the CVA product plot on the DA data is by calculating RV coefcients between the product positions in the individual MFA and CVA solutions. These results are shown in Fig. 7: In
the graph all individual PM solutions were compared to the DA
solution and the RV coefcients are plotted for the rst four dimensions. All MFAs from the rst study (PM1) showed a high similarity
to the DA solution, while none of the MFAs in the second study
(PM2) did. The reason for this probably is the 1-year bottle ageing
of the wines between PM1 and the DA, and the PM2. Interestingly,
the RV coefcients for the rst study decreased with increasing
model dimensions and the rst two dimensions showed the highest similarity to the DA solution for all three studied congurations
(square, rectangle and both). A similar result was reported by Risvik et al. (1994): The highest correlation between the DA and the
PM was found for the rst dimension, which the authors speculated resulted from a similar detection of major sample differences
in both sensory methods. The opposite was observed for the second PM study where with increasing model dimensions also the
RV coefcient increased.

Vanilla
Spice
Bitter
Flat
Low mild
Jam cooked fruit syrup
Bubblegum

Smooth mild
Harsh

In this study the main differences and characteristics of the


wines were picked up by the PM panel in a similar way to the
DA one. The obtained PM consensus map resembled the CVA product plot of the DA, and in both panels judges picked similar sensory
attributes. In addition, the PM panelists often used intensity terms
to express whether the particular attribute was weak or strong,
which is very similar to the scoring of attributes in a DA.
4. Conclusions
In this work we summarized different observations from two
projective mapping (PM) studies on red wine blends, i.e. the effect
of differently shaped PM spaces with the same area, namely
squares versus rectangles, the effect of replicated PM sessions on
the created product space, how well the obtained PM solution ts
to a conventional DA solution, and give examples of out-of-theordinary PM techniques, and how these techniques of individual
judges affect the overall consensus solution. Additionally, we propose a people performance index (PPI) based on individuals ability
to position blind duplicated samples close to each other.

H. Hopfer, H. Heymann / Food Quality and Preference 28 (2013) 164181

1.0

180

squPM1
rectPM1

allPM1
squPM2

lorePM2
shorePM2

allPM2

0.5

RV coefficient

Acknowledgements

0.0

axis is the longer or the shorter side of the rectangle. Experimenter


need to be aware that the provided PM space will inuence the
product representation, and might choose either a square or a rectangle space depending on their study goal: Large differences will
probably show more clearly and more separated when judges are
presented with a large horizontal dimension while choosing a
square conguration might discriminate less along the horizontal
axis and results will probably show also smaller differences among
the samples.

Dimension
Fig. 7. Comparison of the RV coefcients of both PM studies to the DA data for the
rst four model dimensions (PM 1 values are in dashed lines, PM 2 values are in
solid lines).

We found a signicant shape effect between a rectangle and a


square space in our rst study (P 6 0.05) for the x-coordinate, indicating that our judges made different use of the horizontal axis
when confronted with differently shaped PM spaces. However,
for the vertical axis this effect could not be observed.
In a second PM study we extended the shapes to three, namely a
square, a long vertical and a short vertical rectangle. Again, a significant shape effect was only found for one of the axes, this time for
the vertical y-axis (P 6 0.05). Subsequent paired t-tests revealed
that the longer axis in the rectangles was used similarly when horizontal, e.g. the long horizontal axis in the short vertical rectangle
was not signicantly different from the long vertical axis in the
long vertical rectangle. Generally, the two axes in one space were
used differently with the exception of the long vertical rectangle,
where no signicant difference was found. This implied that the
judges did not make use of the longer vertical space and restricted
themselves to a rather square space.
The results from the rst PM study were compared to the DA, as
these two studies were conducted within 3 months of each other
on the same set of samples. Similar product maps and similar
descriptors were created and used by the PM judges compared to
the DA. All monovarietal wines (S, M and F) represented the product extremes, and the PM judges also picked up all signicant DA
attributes. In addition, most PM judges included some kind of
intensity scoring of their descriptors.
A strong replicate effect was found in PM study 1 for the individual judges, but not for the overall consensus product maps. This
observation is in good agreement with previous reports, implying
that individuals position the samples differently every time, but
conserve relative differences and similarities over the replicates.
The calculation of the people performance index (PPI) revealed
that individual judges showed a high variability in their ability to
position identical samples close to each other varying from replicate to replicate and from one duplicated sample to the other one.
This can be attributed to several reasons, such as very similar samples, a high number of samples or no detectable differences among
the samples. We suggest further studies on this aspect, using a
smaller number and/or more differing samples.
Overall, we could show that the shape of the provided PM space
does matter: judges performed different PM arrangements on the
three different spaces. It seems that judges use the horizontal axis
more than the vertical one, independently whether the horizontal

Jerry Lohr is gratefully acknowledged for providing nancial


postdoctoral support. We are indebted to all sensory judges who
put their time and effort in participating in the studies. A special
thank you to Greg Hirson for the initial project idea. Line Holler
Mielby and Joerg Meier are greatly acknowledged for helpful discussions about PM during various coffee breaks. Vince S. Buffalo
helped with data analysis and taming R graphics. We are thankful for the very helpful comments and input from two anonymous
reviewers.

Appendix A. Supplementary data


Supplementary data associated with this article can be found, in
the online version, at http://dx.doi.org/10.1016/j.foodqual.2012.
08.017.
References
Abdi, H. (2007). RV coefcient and congruence coefcient. In N. Salkind (Ed.),
Encyclopedia of measurement and statistics. Thousand Oaks, USA: Sage.
Abdi, H. (2010). Congruence: Congruence coefcient, RV-coefcient and mantel
coefcient. In N. Salkind (Ed.), Encyclopedia of research design. Thousand Oaks,
USA: Sage.
Abdi, H., & Valentin, D. (2007). Multiple factor analysis (MFA). In N. Salkind (Ed.),
Encyclopedia of measurements and statistics (pp. 657663). Thousand Oaks, USA:
Sage.
Barcenas, P., Elortondo, F. J. P., & Albisu, M. (2004). Projective mapping in sensory
analysis of ewes milk cheeses: A study on consumers and trained panel
performance. Food Research International, 37(7), 723729.
Bcue-Bertaut, M., & Pags, J. (2008). Multiple factor analysis and clustering of a
mixture of quantitative, categorical and frequency data. Computational Statistics
& Data Analysis, 52(6), 32553268.
Bertuccioli, M. (2011). The role of sensory analysis to identify factors contributing to
wine performance. Graz, Austria: Keynote lecture at the In Vino Analytica
Scientia.
Cartier, R., Rytz, A., Lecomte, A., Poblete, F., Krystlik, J., Belin, E., et al. (2006). Sorting
procedure as an alternative to quantitative descriptive analysis to obtain a
product sensory map. Food Quality and Preference, 17(78), 562571.
Dehlholm, C., Brockhoff, P. B., Meinert, L., Aaslyng, M. D., & Bredie, W. L. P. (2012).
Rapid descriptive sensory methods Comparison of free multiple sorting,
partial napping, napping, ash proling and conventional proling. Food Quality
and Preference, 26(2), 267277.
Escoer, B., & Pags, J. (1994). Multiple factor analysis (AFMULT package).
Computational Statistics & Data Analysis, 18(1), 121140.
Faye, P., Brmaud, D., Durand Daubin, M., Courcoux, P., Giboreau, A., & Nicod, H. (2004).
Perceptive free sorting and verbalization tasks with naive subjects: An alternative
to descriptive mappings. Food Quality and Preference, 15(78), 781791.
Friendly, M., & Fox, J. (2010). candisc: Generalized Canonical Discriminant Analysis.
R package (Version 0.5-19). Retrieved from http://CRAN.R-project.org/
package=candisc.
Hopfer, H., Ebeler, S. E., & Heymann, H. (2012). How blending affects the sensory
and chemical properties of red wine. American Journal of Enology and Viticulture,
63(3).
Husson, F., Josse, J., Le, S., & Mazet, J. (2012). FactoMineR: Multivariate Exploratory
Data Analysis and Data Mining with R (Version 1.18).
International Organization for Standardization (1977). ISO 3591: Sensory Analysis
Apparatus Wine tasting glass.
Josse, J., Pags, J., & Husson, F. (2008). Testing the signicance of the RV coefcient.
Computational Statistics & Data Analysis, 53(1), 8291.
Kennedy, J. (2010). Evaluation of replicated projective mapping of granola bars.
Journal of Sensory Studies, 25, 672684.
Kennedy, J., & Heymann, H. (2009). Projective mapping and descriptive analysis of
milk and dark chocolate. Journal of Sensory Studies, 24(2), 220233.

H. Hopfer, H. Heymann / Food Quality and Preference 28 (2013) 164181


King, M. C., Cliff, M. A., & Hall, J. W. (1998). Comparison of projective mapping and
sorting data collection and multivariate methodologies for identication of
similarity-of-use of snack bars. Journal of Sensory Studies, 13(3), 347358.
Kruskal, J. B., & Wish, M. (1978). Multidimensional scaling. Beverly Hills, California,
USA: Sage Publications.
Lawless, H. T., & Heymann, H. (2010). Sensory evaluation of food: Principles and
practices. New York: Springer.
Le, S., Josse, J., & Husson, F. (2008). FactoMineR: An R Package for Multivariate
Analysis. Journal of Statistical Software, 25(1), 18.
Nestrud, M. A., & Lawless, H. T. (2008). Perceptual mapping of citrus juices using
projective mapping and proling data from culinary professionals and
consumers. Food Quality and Preference, 19(4), 431438.
Nestrud, M. A., & Lawless, H. T. (2010). Perceptual mapping of apples and
chesses using projective mapping and sorting. Journal of Sensory Studies, 25,
309324.
Pags, J. (2005). Collection and analysis of perceived product inter-distances using
multiple factor analysis: Application to the study of 10 white wines from the
Loire valley. Food Quality and Preference, 16(7), 642649.
Pags, J., Cadoret, M., & L, S. (2010). The sorted Napping: A new holistic approach in
sensory evaluation. Journal of Sensory Studies, 25(5), 637658.
Pags, J., & Husson, F. (2005). Multiple factor analysis with condence ellipses: A
methodology to study the relationships between sensory and instrumental
data. Journal of Chemometrics, 19, 138144.

181

Perrin, L., & Pags, J. (2009). Construction of a product space from the ultra-ash
proling method: Application to 10 red wines from the Loire valley. Journal of
Sensory Studies, 24(3), 372395.
Perrin, L., Symoneaux, R., Matre, I., Asselin, C., Jourjon, F., & Pags, J. (2008).
Comparison of three sensory methods for use with the Napping procedure:
Case of ten wines from Loire valley. Food Quality and Preference, 19(1), 111.
R Foundation for Statistical Computing (2011). R: A language and environment for
statistical computing: R Foundation for Statistical Computing, Vienna, Austria.
Retrieved from http://www.R-project.org/.
Risvik, E., McEwan, J. A., Colwill, J. S., Rogers, R., & Lyon, D. H. (1994). Projective
mapping: A tool for sensory analysis and consumer research. Food Quality and
Preference, 5(4), 263269.
Risvik, E., McEwan, J. A., & Rdbotten, M. (1997). Evaluation of sensory proling and
projective mapping data. Food Quality and Preference, 8(1), 6371.
Sanchez Gavito Sanchez, J. V. (2011). Comparison of Descriptive Analysis and
Projective Mapping Techniques in the Aroma Evaluation of the Distilled Spirits,
Gin and Tequila. (M.S.). Davis, United States, Davis, California: University of
California [AAT 1502270].
Tang, C., & Heymann, H. (2002). Multidimensional sorting, similarity scaling and
free-choice proling of grape jellies. Journal of Sensory Studies, 17, 493509.
Veinand, B., Godefoy, C., Adam, C., & Delarue, J. (2011). Highlight of important
product characteristics for consumers. Comparison of three sensory descriptive
methods performed by consumers. Food Quality and Preference, 22(5), 474485.

You might also like