Group Decision and Negotiation 15: 127140, 2006.

DOI: 10.1007/s10726-006-9024-z


C Springer 2006

Comparative Analysis of Text Data in Successful

Face-to-Face and Electronic Negotiations
MARINA SOKOLOVA
School of Information Technology and Engineering, University of Ottawa, Ottawa, Canada
(E-mail: sokolova@site.uottawa.ca)

MOHAK SHAH
School of Information Technology and Engineering, University of Ottawa, Ottawa, Canada
(E-mail: mshah@site.uottawa.ca)

STAN SZPAKOWICZ
School of Information Technology and Engineering, University of Ottawa, Ottawa, Canada;
Institute of Computer Science, Polish Academy of Sciences, Warsaw, Poland
(E-mail: szpak@site.uottawa.ca)

Abstract
Various combination of Natural Language Processing and Machine Learning methods offer ample opportunities
wherever texts are an important element of an application or a research area. Such methods discover patterns and
regularities in the data, seek generalization and in effect learn new knowledge. We have employed such methods
in learning from a large amount of textual data. Our application is electronic negotiations. The genre of texts
found in electronic negotiations may seem limited. It is an important research question whether our methods and
findings apply equally well to texts that come from face-to-face negotiations. In order to confirm such more general
applicability, we have analyzed comparable collections of texts from electronic and face-to-face negotiations. We
present our findings on the extent of similarity between these two related but distinct genres. In this study we have
analyzed similarities in the text data of electronic and face-to-face negotiations. The results show that in certain
conditions vocabulary richness, language complexity and text predictability are similar.
Key words: Electronic negotiations, face-to-face negotiations, communication process, text data, corpus analysis

1. Introduction
Natural Language Processing (NLP) and Machine Learning (ML) techniques have been
applied in a wide variety of fields. They are, however, a new tool for the study of negotiation
or, more specifically, texts found in negotiation. NLP and ML methods work well with large
amounts of textual data. This makes them desirable research tools in the fields where the
volume of information to process grows rapidly. Bioinformatics and medicine are among

This is an expanded version of a paper published in the Proceedings of FINEXIN 2005 (Workshop on the Analysis
of Formal and Informal Information Exchange during Negotiations), 3142, Ottawa, Canada, May 2005.

128

SOKOLOVA, SHAH AND SZPAKOWICZ

the recent successful applications of NLP and ML that lead to the discovery of useful and
interesting patterns.
Learning from large amounts of negotiation-related information is becoming a necessity: we note a substantial increase of the volume of electronic negotiations (e-negotiations).
Traditionally, negotiation has been understood as interaction between two negotiating parties with at least oral contact (face-to-face, by telephone, and so on). The pervasiveness of
information technology, however, has made e-negotiations a new reality. They take advantage of electronic means, not necessarily limited to email or the Internet. They also include
other electronic assistance such as negotiation support systems or online analysis software
(Schoop, 2003).
E-negotiations restrict communication between the participants to the exchange of formal
or informal messages. Formal messages represent offers, positions and preferences unambiguously, but tend to mask the more interesting aspects of the negotiators personality.
Informal messages are unrestricted texts in which negotiators confirm decisions reflected
in a formal representation, and may also establish a personal relationship. When informal
messages are allowed, their language may affect the negotiation outcome, strategies that
participants use, and roles in negotiations. These results come from analyzing the data
gathered in e-negotiations (Sokolova et al. 2005b).
We want to keep our ML/NLP toolkit general and to apply it to problems other than
electronic negotiations. We study the data of face-to-face negotiations. We must identify
the differences and similarities between the two types of negotiation processes, and find out
how this affects the language in textual data. If the data of face-to-face negotiations exhibit
the characteristics similar to those of e-negotiations data, we can expect that the learning
methods applied to e-negotiation data will also work with face-to-face negotiation data.
We seek methods that capture similarities between the text data of face-to-face and electronic negotiations. We study the richness of the vocabulary, word distribution, predictability
of data and common sets of features. We have applied statistical analysis and modelling.
We show that these methods, if based on word frequencies, find common characteristics
in the language of negotiations. We apply the statistical analysis and modelling methods
to the language in face-to-face negotiations, and compare the results with those for the
corresponding e-negotiation data.
The face-to-face textual data that we analyze here come from the Cartoon negotiations
(Brett, 2001). The texts are part of manual transcripts with non-textual elements removed
(those are the elements that pertain to body language, gestures, pauses and so on). We
compare these data with the textual e-negotiation data collected by the Inspire system
(Kersten and Zhang, 2003). Both data sets arose in the context of the training of negotiators,
but they find good use in research. In addition to showing that the two domains resemble
each other closely with respect to word distribution, we look at whether process-specific
information can help characterize them, given their statistical similarity.
The remainder of the paper is organized as follows. Section 2 reviews elements of
negotiations in general and introduces the data from the Cartoon and Inspire negotiations.
Section 3 reports the results of the data analysis. Section 4 presents the role of the processspecific information in the two contexts and explores its distribution. In the concluding
Section 5 we emphasize the central aspects of our work and suggest possible future work.

129

COMPARATIVE ANALYSIS OF TEXT DATA

2. Problem Statement
In this work, we focus on the data gathered in business negotiations. Rojot (1991) defines
negotiation as a process where two or more parties aim to settle what each shall give and
take in a transaction between them.
The Cartoon and Inspire negotiations are training negotiations. Both represent the class of
bilateral integrative business negotiations. One participant is a buyer and the other is a seller.
The goal of a negotiation is to make a virtual purchase. A negotiation is successful if the
virtual purchase has taken place. The language of communication during these negotiations
is English.
Despite the similarities between the Cartoon and Inspire negotiations, there are substantial differences in their setting. The Cartoon negotiations are conducted face-to-face (Brett,
2001). The issue is the virtual purchase of a TV program. The whole negotiation process
takes place during one uninterrupted meeting. The Inspire system supports e-negotiations
(Kersten, 2003). The issue is the virtual purchase of bicycle parts. The process lasts up
to three weeks, during which participants exchange freely written messages and formal
numerical offers. Table 1 compares the key elements of Cartoon and Inspire negotiations.
We have access to the tape record transcripts1 of 40 Cartoon negotiations, all of them
successful. 20 negotiations were held between a Japanese buyer and a Japanese seller (the
Cartoon Japanese data), the other 20 between a US buyer and a US seller (the Cartoon US
data). The text data contain 157,253 word tokens distributed among 4,244 word types, with
1686 rare word types, appearing only once in the data (Oakes, 1998). As to the Inspire data,
in the experiments described in this paper we have worked with all text messages exchanged
in 1427 successful negotiations. Negotiations are intercultural: the participants come from
50 countries. The text data contain 1,073,398 word tokens distributed among 25,085 word
types, with 12,897 rare word types. A more detailed analysis of the Inspire language data
appears in (Sokolova et al., 2005a).
This comparison highlights the differences between the Cartoon data and the successful Inspire negotiation data. One is a small corpus containing recorded, transcribed and

Table 1. Cartoon and Inspire negotiations.

Elements

Cartoon

Inspire

Environment
Means
Topic
Goal
Rules
Communication
# of negotiators
Roles
Protocol

Business
Face-to-face meeting
Purchase of a TV program
To make a purchase

Business
Electronic system
Purchase of goods
To make a purchase

Synchronous
two
buyer and seller
imposed by Cartoon

Asynchronous
two
buyer and seller
imposed by Inspire

130

SOKOLOVA, SHAH AND SZPAKOWICZ

post-edited speech data, the other is a relatively large corpus of unedited written texts. We
seek the similarities between the Cartoon data and the successful Inspire negotiations data.
We are interested in similarities due to the common origin of the data communication
during a negotiation and to the process characteristics business, bilateral, integrative,
and successful.
Our research problem is to identify a class of methods that will detect this type of
similarities. In the following sections we present such methods of revealing similarities
between two data sets. It is worth noting that finding similarities between corpora is one of
the more complex tasks in corpus analysis (Kilgarriff, 2001).

3. Data Analysis
We first employ corpus-linguistic methods to analyze and compare corpora. Section 3.1 explores lexical richness by studying vocabulary growth, homogeneity and variety of words
and simple expressions in both corpora. It also investigates how two corpora are close
statistically. Section 3.2 models the Cartoon and the Inspire data to evaluate their lexical complexity and predictability. The reliability of word frequencies makes the results a
trustworthy measure of corpus comparison.

3.1. Corpus comparison

We start with the descriptive analysis of corpora. We calculate the type-token ratio T T (N )
where N is the number of tokens. We add information on rare words based on hapax
legomena (V (1, N )) and dis legomena (V (2, N )) (Tweedie and Baayen, 1998); V (i, N ) is
the number of types that occur i times in the text with N tokens. In both data T T (N ) is lower
than in a benchmark Brown corpus (Francis and Kucera, 1979). We explain the somehow
restricted vocabulary by the fixed topic of discussion. We calculate the growth rate of the
vocabulary P(N )
P(N ) =

V (1, N )
;
N

(1)

and Sichels characteristic S(N )

S(N ) =

V (2, N )
N

(2)

P(N ) is a common measure of the richness of the vocabulary of a text (Sichel, 1986). We
also calculate the ratio of bigrams bi(N )
bi(N ) =

# of bigrams
N

(3)

131

COMPARATIVE ANALYSIS OF TEXT DATA

Table 2. The data statistics.

Cartoon

Inspire-successful

T T (N )

0.027

0.023

P(N )
S(N )
bi(N )
tri(N )

0.4
0.076
0.289
0.682

0.5
0.002
0.198
0.486

Statistics

and the ratio of trigrams tri(N )

tri(N ) =

# of trigrams
N

(4)

See Table 2 for the results.

The typical P(N ) is approximately 0.5 (Oakes, 1998). A larger P(N ) denotes a richer
vocabulary. The empirical values of P(N ) show that the Inspire corpus exhibits traits of a
richer vocabulary when compared to the Cartoon corpus. We can explain this by the cultural
diversity of the Inspire data contributors and by the longer duration of Inspire negotiations.
The latter allows more interaction and makes it possible to establish social contact between
negotiators; the former introduces a variety of discussion topics, which may lead to a
larger vocabulary. Notice that other reported corpus statistics depend on size more than
P(N ) (Tweedie and Baayen, 1998). S(N ), which is another measure of vocabulary richness
(Thomas et al., 2004), is in inverse proportion to the size of the corpus. S(N ) essentially
tells us whether the corpus is homogeneous by converging, or not converging, to a limit;
convergence is calculated on samples of increasing sizes.
Nouns are among the most frequent words of e-negotiation data. This differs from the
typical corpora, where the most frequent words are function words, for example, prepositions, determiners, and words such as pronouns or interjections. Table 3 lists 10 most
frequent words from the Inspire data, all Cartoon data and their Japanese and US subsets.
We compare our data with face-to-face conversations of upper-level college students (Dialogues) (Allen and Guy, 1974), with the Brown corpus, and the Wall Street Journal (WSJ)
corpus (Paul and Baker, 1992). We exemplify the commonalities and differences using the
10 most common types in these corpora; see Table 3.
The similarity of the results in columns 2 (Cartoon US) and 3 (Cartoon All) is due to the
imbalance between Cartoon US and Cartoon Japanese data. The former are five times as
large as the latter, that is, the Cartoon All characteristics are dominated by those of Cartoon
US.
The different sizes of the comparable corpora suggest the use of rank-based methods
such as the Spearmans Rank Correlation Coefficient (SRCC)

6 d2
=1
N (N 2 1)

(5)

132

SOKOLOVA, SHAH AND SZPAKOWICZ

Table 3. 10 most frequent words in Cartoon, Inspire, Dialogues, Brown, WSJ.

Cartoon
Japanese

Cartoon
US

Cartoon
All

Insp-succ

Dialogues

Brown

WSJ

i
you
to

the
to
that

the
to
you

to
i
you

i
you
and

the
of
and

the
of
to

the
million
runs
a

you
i
and
we

i
that
and
we

the
we
and
a

the
to
ah
a

to
a
in
that

a
and
in
that

for
we

a
of

a
of

your
offer

it
in

is
was

for
one

and

is

is

for

know

he

is

Table 4. Spearmans coefficient.

Data

Cartoon Japanese

Cartoon US

Insp-succ

Dialogue

Cartoon Japanese
Cartoon US

0.45

0.45

0.53
0.57

0.42
0.39

N is the number of words, and d is the difference in the ranks of the same word in two
corpora (Oakes, 1998). SRCC shows accurate results in comparing corpora of different sizes
(superior to other methods) and is a reliable indicator of the corpus similarity (Kilgarriff,
2001). We avoid SRCCs bias towards low-frequency words it tends to overemphasize
such words by calculating it on the most frequent words where its performance is balanced.
Spearmans coefficient, calculated on the most common words, has shown that the
Japanese data are closer to the Dialogue data than to the Cartoon US data. Same applies
to the US data: they are closer to the Dialogue data than to the Cartoon Japanese data; see
Table 4.
The results show that similarities between the Cartoon and the Inspire data are statistically
significant with the 0.80 significance level.

3.2. Statistical modelling

Statistical modelling is an important tool in data comparison (Rosenfeld, 2000). A model
tells us about word distribution, dependences between words, and the level of data predictability. In this section we compare the statistical models of the Cartoon and the Inspire
data. Corpus analysis shows that the Cartoon data indeed consist of two easily distinguishable subsets: the Cartoon Japanese data and the Cartoon US data. For more detailed results,
we model these data separately. To make the results comparable, we sample the successful

133

COMPARATIVE ANALYSIS OF TEXT DATA

Inspire negotiation data instead of working with the whole set. That is because statistical
modelling results depend on the size of the data set (Chen and Goodman, 1996).
3.2.1. Sampling
We consider the size in tokens and the number of negotiations as the main parameters for
choosing samples. The average length of a successful Inspire negotiation is 750 tokens.
We have found a subclass of the Inspire negotiations where the average negotiation length
is 1,160 words. Both average lengths are closer to the Cartoon Japanese negotiations in
which the average length is 1,340 than to the Cartoon US negotiations with 6,250 tokens on
average. We construct two samples from the successful Inspire negotiation data. InspGen is
selected from all successful Inspire negotiations. InspEmail is selected from longer successful negotiations (as measured by the number of word tokens) where negotiators exchange
personal email addresses. For both samples we choose negotiations uniformly distributed
in the original population.
The results show that the number of tokens and the number of negotiations of the samples
correspond to those of the Cartoon Japanese data and differ from those of Cartoon US data,
while the word distribution preserves characteristics of the successful Inspire negotiation
data; see Table 5.
We calculate the sample statistics; see Table 6. The size dependencies of S(N ), bi(N ),
tri(N ) are clearly seen if we compare the new results with the results reported in Table 2.
Finally, we report 10 most frequent words from the samples; see Table 7.
The results confirm that the Cartoon corpus is heterogenous whereas the Inspiresuccessful corpus is homogenous.
The most frequent words show that the common type of texts the negotiation messages
makes the Cartoon and Inspire corpora similar.
Table 5. Word distribution in four collections.
Cartoon Japanese

Characteristics
# of negotiations
N
T

InspEmail

InspGen

Cartoon US

20

25

33

20

26795
1412

26987
2764

27218
2539

130458
3702

534
194

1333
390

1180
361

1357
519

V(1,N)
V(2,N)

Table 6. The sample statistics.

Statistics

Cartoon Japanese

InspEmail

InspGen

Cartoon US

T T (N )
P(N )
S(N )

0.053
0.020
0.007

0.102
0.049
0.014

0.093
0.043
0.013

0.028
0.011
0.003

bi(N )
tri(N )

0.377
0.714

0.510
0.827

0.337
0.529

0.303
0.699

134

SOKOLOVA, SHAH AND SZPAKOWICZ

Table 7. 10 most common words in the samples.

rank

Cartoon Japanese

InspEmail

InspGen

rank

Cartoon Japanese

InspEmail

InspGen

you

runs

and

2
3
4
5

you
to
the
million

the
to
you
and

to
i
the
a

7
8
9
10

a
for
we
and

we
is
for
your

we
for
your
offer

3.2.2. Modelling
The goal of statistical modelling is to find a way of approximating the data well. For textual
data, a model predicts which words, and with what probabilities, can appear after a given
sequence of words. The criterion of the models quality is the value of cross-entropy:

n
1
log(P(wi ))
n i=0

(6)

n is the number of words in the test set, P(wi ) is the probability (calculated by the model)
of the appearance of the word wi in the test data (Oakes, 1998). A models cross-entropy
is calculated when it is built from training data, or training and validation data, and tested
on previously unseen test data (Chen and Goodman, 1996). A model with the lower crossentropy models the data better. With respect to language, cross-entropy is one of the estimators of its complexity or predictability. Lower cross-entropy means that the data are
predictable, higher cross-entropy indicates more uncertainty in the data.
For text data, N -gram models are very widely used. An N -gram is a sequence of N
consecutive words w1 , . . . , w N . An N -gram model approximates the probability of the
appearance of word w N after the sequence of words w1 , . . . , w N 1 . This is expressed as
conditional probability P(w N |w1N 1 ). The approximation is as follows:




1
P w N |w1N 1 P w N |w NN k
The values of k are usually 1, 2 and 3. N -gram models for N > 4 almost do not appear
in practice, quadrigram models are rare, and trigram models tend to slightly outperform
bigram models. The probabilities of N -grams are calculated on training data and then the
same probability applies to each N -gram appearing in the test data. In our study we employ
the Kneser-Ney and Good-Turing trigram models; for details and formulae see (Chen and
Goodman, 1996; Katz, 1987). Here is why we use these models: their performance is high
compared with other models; they require only training and test data; they are easy to
implement and to apply, for example, have no adjustable parameters.
From the language point of view, the models explore different data properties. The
Kneser-Ney probability of a word depends on the number of preceding word types, that
is, the number of different words. A previously unseen trigram may contain a collocation

135

COMPARATIVE ANALYSIS OF TEXT DATA

Table 8. Cross-entropy results.

Model

Cartoon Japan.

InspEmail

InspGen

Cartoon US

GTK

10.06

11.8

11.04

12.21

KN

6.67

7.15

7.27

7.79

(a noticeable arrangement of adjacent words). This often happens in negotiation data with
their restricted topic. The Kneser-Ney model treats such trigrams differently from trigrams
that do not contain collocations. On the other hand, the Good-Turing model calculates the
frequency of N-grams in the training set, so it heavily depends on the sizes of the training
and test sets.
An important concern in such modelling is smoothing that helps account in advance for
previously unseen N -grams. The Kneser-Ney model has smoothing built-in: the probability
of an unseen N -gram is substituted by the probability of the (N 1)-gram. The probability
of a previously unseen word is N1 , where N is the number of tokens. For the Good-Turing
model we use Katz smoothing that generally performs well on data with a high token-type
ratio (Katz, 1987).
We employ five-fold cross-validation to train and test statistical models. In K -fold cross
validation, the examples are randomly divided into K mutually disjoint sets of approximately
equal sizes. The concept is learned from the examples in K 1 sets, and is tested on the
examples from the remaining set. This is repeated K times, once for each set (that is, each set
is used once as a test set). The average error rate over all K sets is the cross-validated error
rate. Table 8 reports the average five-fold cross-validation results for all the experiments.
GTK stands for the Good-Turing model with Katz smoothing, KN for the Kneser-Ney
model.
A cross-entropy reduction of more than 0.16 is noteworthy (Rosenfeld, 2000). The results
show that the predictability of the Cartoon Japanese data is closer to the predictability of the
Inspire samples than to the predictability of the Cartoon US data. At the same time, for all
four data sets the KN model is significantly better than the GTK model. This corresponds
to the KN and GTK performance results that Chen and Goodman (1996) reported for the
Switchboard corpus. We cite the results here because the Switchboard corpus contains
transcripts of spoken dialogues between a telephone operator and a client. While it is not a
business negotiation, such conversation sometimes is referred to as collaborative negotiation
(Chu-Carroll and Carberry, 2000).
Because the models depend on the sizes of the data sets, we can explain this by fact that
the KN model performs better on small data than the GTK model. Another conclusion is
related to the smoothing used in the KN model: in our data the number of different words
preceding word wk is a better indicator of wk than the N -gram frequencies.
The following question arises naturally: how well can one data set predict a word from
another set? Put differently, how closely can the Cartoon and Inspire samples approximate
each other with respect to the models? To answer this question, we have built cross-models
of the data for Cartoon Japanese and US samples and InspEmail and InspGen samples. For
cross-modelling we use a training set from one data class and a test set from another data

136

SOKOLOVA, SHAH AND SZPAKOWICZ

Table 9. Cross-modelling results.

Model

Cartoon Japan
vs
Cartoon US

InspEmail
vs
InspGen

InspGen
vs
InspEmail

Cartoon US
vs
Cartoon Japan

GTK
KN

9.84
7.35

11.33
7.14

11.49
7.39

12.30
7.62

class. Table 9 presents the results of cross-modelling. X vs Y means that X is a training set
and Y is a test set. To make a fair comparison we use the same training and test sets as in
the previous modelling experiments.
Prediction of a word becomes harder when a model is trained on InspGen and Cartoon
US data and tested on InspEmail and Cartoon Japanese data respectively. The same holds
for a KN model trained on Cartoon Japanese and tested on Cartoon US data. The GTK
model, however, improves its fitness for Cartoon US data when trained on Cartoon Japanese
data. These results are intriguing but inconclusive. Their confirmation and interpretation
are a matter of future work.
Other improvements of the model fitness are due to the sample data. Cross-entropy
reported in InspEmail vs InspGen is smaller than in InspEmail. Note that the same training
data were used in both cases. The difference in cross-entropy results signals that the InspGen
test set has less unseen data than the InspEmail test set. This supports the previously stated
hypothesis that people use diversified vocabulary when they exchange personal information.
Having shown a considerable similarity of the Cartoon and the Inspire data, we now
investigate whether process-specific information can be used to represent and characterize
them.

4. Process-Specific Information in Negotiation Data

As a special type of human communication, negotiations have the following qualities of a
process (Hargie and Dickson, 2004): they are dynamic, multi-dimensional, irreversible and
purposeful. In this study we explore how the purpose of business negotiations is reflected in
the textual data. If this purpose to make a deal is fulfilled, the negotiation is successful
and its outcome is positive. We show the presence in the Cartoon data of the characteristics
of the Inspire text data that help predict negotiation outcomes with high accuracy.
In the case of e-negotiations, words related to the negotiation process appear among the
most frequent words. This is partially due to the closed topic of discussion and partially to
the use of electronic means of communication. Shah et al. (2004) have shown the importance
of negotiation-related information in representing and characterizing the data. It is strongly
related to the process of negotiation, so we refer to it as process-specific information.
A representation that uses process-specific information is especially useful since textual
data of this kind usually contain a very high number of word types (usually in the order
of thousands). Word types are the features used most often to represent text data collected

137

COMPARATIVE ANALYSIS OF TEXT DATA

through electronic means. On the other hand, the largest available collection of negotiation
texts consists merely of some 2500 negotiations. The number of available examples is
small compared with the number of features word types. This makes it necessary to
look for a much smaller number of features that can still capture the most important data
characteristics. Process-specific information helps do just that. We show that when it is used,
our textual e-negotiation data can be represented by merely 124 word types/features. If this
can be generalized, we will indeed be able to deal successfully with negotiation data at the
process level. In order to obtain process-specific information, we apply a semi-automatic
procedure discussed (Shah et al., 2004). Very briefly, the procedure takes a set of texts and
a communication model (Hargie and Dickson, 2004); identifies process-specific elements
of the model; builds a lexicon and the bigram and trigram models of the data; performs
simple word-sense disambiguation; constructs a set of semantic categories, among them
process-specific categories, and tags all word types in the lexicon with these categories;
finally, represents a text using only word types tagged as process-specific.
We have found the distribution of the process-specific words in the Cartoon data. In order
to obtain process-specific information, we use the same procedure as described before. The
results support those produced from the Inspire data; see Table 10. Table 11 reports the most
frequent bigrams used in the process-specific context to support that the process-specific
words play important role in the negotiation data. Recall that usually the most frequent
bigrams are prepositions, determiners, and words such as pronouns or interjections.
Here are a few detailed findings after applying the procedure to our data. A noun appears
among 10 most frequent unigrams in the Cartoon Japanese data usually, all top unigrams
are function words. The noun million is specific to the Cartoon negotiations. The most
Table 10. Percentage of process-specific words in 100 most
frequent unigrams (excluding function words).
Cartoon Japan

Cartoon US

Cartoon All

Insp-succ

64.8

51.1

56.6

57.9

Table 11. 10 most common bigrams in various negotiation datasets.

Cartoon Japan

Cartoon US

Cartoon All

InspEmail

InspGen

Insp-succ

5 million
want to
ultra ranger
i see
down payment
8 runs
the program
ok
if you
how about

you know
i think
going to
per episode
of the
okay. . .
we can
if we
i mean
would be

you know
i think
per episode
going to
we can
of the
okay. . .
want to
if we
in the

i am
the price
i think
if you
for you
in the
we are
and i
i hope
that we

i am
your offer
for you
the price
we are
i have
thank you
if you
and i
i hope

i am
we are
your offer
the price
i hope
for your
we can
that we
if you
i have

138

SOKOLOVA, SHAH AND SZPAKOWICZ

frequent noun in the Cartoon US data appears among the top 40 words. The noun episode
is also specific to the Cartoon negotiations.

5. Conclusion and Future Work

In this work, we have reported the results that show similarities between the text data
gathered in electronic negotiation and in face-to-face negotiations. We applied statistical
corpus analysis based on word frequencies. For the electronic negotiations data, we use
the negotiation transcripts from the Inspire system, while the face-to-face negotiations
are represented by the Cartoon negotiations transcripts. We have shown that the textual
components of the two negotiations are considerably close statistically. Moreover, following
Shah et al. (2004), we have confirmed that such data can be successfully characterized by
a specific word distribution using what we refer to as process-specific information. As the
main result, we have identified a class of analysis methods that extracts similarities from
the text data of face-to-face and electronic negotiations. In Table 12 we have summarized
the methods used in this work, the measures they produce, the meaning of these measures
in terms of the data, and the results for the Cartoon and the Inspire-successful data.2
Many extensions may arise from the insights gained in this work. One of possible directions is to understand how much information we can get from one data set about another.
It can be done via statistical modelling, for example, by cross-calculating the perplexity of
the Cartoon corpus under a trigram model learned from the Inspire corpus, and vice versa.
Our next step is to understand how much we can learn from one data set about the other
one. If we can characterize successful negotiations using process-specific information, we
should be able to assist in predicting negotiation outcomes from text data in both the electronic and face-to-face scenarios. This can be done by representing Cartoon negotiations and
successful Inspire with process-specific features and then solving the binary classification
problem where negotiations from one corpus and negotiations from another corpus act as
two classes.
Table 12. The data statistics.
Cartoon

Inspire

hapax legomena
dis legomena
unigram token ratio

0.4
0.076
0.027

0.5
0.002
0.023

bigram token ratio

trigram token ratio
cross-entropy (KN)
cross-entropy (GTK)
Spearman coefficient
negotiation-related
features

0.289
0.682
6.98
11.12
significantly
58.0

0.198
0.486
7.47
11.42
similar
57.3

Method

Corpus characteristics

Measure

Analysis

lexical richness

Modelling

lexical complexity

Comparison
Feature selection

similarity between corpora

learning of
the unseen texts

COMPARATIVE ANALYSIS OF TEXT DATA

139

This knowledge can in the future be useful in assisting negotiation support systems, since
with time more and more negotiations will take place online. In addition, the approach is
independent of the particular electronic means that are employed for electronic negotiations.
This allows the results to be generalized beyond some specific negotiation support system
or other electronic assistance. Finally, representations that use process-specific information
have the potential to be extended to other types of human communication.

Acknowledgments
This work has been partially supported by the Social Sciences and Humanities Research
Council of Canada and by the Natural Sciences and Engineering Research Council of
Canada. We thank Jeanne M. Brett and Wendi L. Adair for making the Cartoon data
available.

Notes
1. Some transcripts skip the introductory part of a negotiation.
2. The Modelling and Feature Selection sections average results from Table 8.

References
Allen, D. E. and Guy, R. F. (1974). Conversation Analysis: The Sociology of Talk, Mouton, The Hague.
Brett, J. M. (2001). Negotiating Globally, Jossey-Bass, San Francisco.
Francis, W. N. and Kucera, H. (1979). Brown Corpus Manual, http://helmer.aksis.uib.no/icame/brown/bcm.
html.
Cellich, C. and Jain, S. C. (2004). Global Business Negotiations: A Practical Guide, Thomson, South-Western.
Chen, S. F. and Goodman, J. (1996). An Empirical Study of Smoothing Techniques for Language Modeling,
Proceedings of the 34th Annual Meeting of the Association for Computational Linguistics, 310318.
Chu-Carroll, J. and Carberry, S. (2000). Conflict Resolution in Collaborative Planning Dialogues, International
Journal of Human-Computer Studies 53(6), 9691015.
Hargie, O. and Dickson, D. (2004). Skilled Interpersonal Communication: Research, Theory and Practice,
Routledge.
Katz, S. M. (1987). Estimation of Probabilities from Sparse Data for the Language Model Component of a Speech
Recognizer, IEEE Transactions on Acoustics, Speech and Signal Processing (ASSP) 35(3), 400401.
Kersten, G. E. (2003). The Science and Engineering of E-negotiation: An Introduction, Proceedings of 36th
Hawaii International Conference on System Sciences (HICSS 2003), 27 p.
Kersten, G. E. and Zhang, G. (2003). Mining Inspire Data for the Determinants of Successful Internet Negotiations, Central European Journal of Operational Research 11(3), 297316.
Kilgarriff, A. (2001). Comparing Corpora, International Journal of Corpus Linguistics 6(1), 97133.
Oakes, M. P. (1998). Statistics for Corpus Linguistics, Edinburg University Press.
Paul, D. B. and Baker, J. M. (1992). The Design for the Wall Street Journal-Based CSR Corpus, Proceedings of
ICSLP92, 357361.
Rojot, J. (1991). Negotiation: From Theory to Practice, Macmillan Academic and Professional.
Rosenfeld, R. (2000). Two Decades of Statistical Language Modeling: Where Do We Go from There? Proceedings of IEEE 88, 12701278.

140

SOKOLOVA, SHAH AND SZPAKOWICZ

Schoop, M. (2003). A Language-Action Approach to Electronic Negotiations, Proceedings of LAP 2003, 143
160.
Sebastiani, F. (2002). Machine Learning in Automated Text Categorization, ACM Computing Surveys, 147.
Sichel, H. (1986). Word Frequency Distributions and Type-Token Characteristics, Mathematical Scientist 11,
4572.
Shah, M., Sokolova, M., and Szpakowicz, S. (2004). The Role of Domain-Specific Knowledge in Classifying the
Language of E-negotiations, Proceedings of 3rd International Conference on Natural Language Processing
(ICON 2004), Allied Publishers, India, 99108.
Sokolova, M., Nastase, V., Szpakowicz, S., and Shah, M. (2005a). Analysis and Models of Language in Electronic
Negotiations, in M. Draminski, P. Grzegorzewski, K. Trojanowski and S. Zadrozny (Eds.) Issues in Intelligent
Systems, Models and Techniques, EXIT, Warszawa, 197211.
Sokolova, M., Nastase, V., Shah, M., and Szpakowicz, S. (2005b). Feature Selection in Electronic Negotiation
Texts, Proceedings of Recent Advances in Natural Language Processing (RANLP2005), 518524.
Strobel, M. (2000). Effects of Electronic Markets on Negotiation Processes, Proceedings of ECIS 2000, 445452.
Thomas, L., Pfister, H. P., and Peterson, P. (2004). Issues Related to the Construction of a Purpose-Built DomainSpecific Word Corpus, Australian Journal of Educational and Developmental Psychology 4, 1328.
Thompson, L. and Nadler, J. (2002). Negotiating Via Information Technology: Theory and Application, Journal
of Social Issues 58(1), 109124.
Tweedie, F. J. and Baayen, R. H. (1998). How Variable May a Constant be? Measures of Lexical Richness in
Perspective, Computers and the Humanities 32, 323352.