You are on page 1of 9

Invariants and Variability of Synonymy Networks:

Self Mediated Agreement by Confluence

Benoît Gaillard Bruno Gaume Emmanuel Navarro


CLLE-ERSS, CNRS CLLE-ERSS, CNRS IRIT
University of Toulouse University of Toulouse University of Toulouse
Toulouse, France Toulouse, France Toulouse, France
benoit.gaillard@univ-tlse2.fr bruno.gaume@univ-tlse2.fr navarro@irit.fr

Abstract growth is that of comparing various resources. A


lexical resource is usually based on semantic judge-
Edges of graphs that model real data can be ments about lexical elements (a human judgement
seen as judgements whether pairs of objects performed by a lexicographer, or a machine-based
are in relation with each other or not. So,
judgement in the case of automatically built re-
one can evaluate the similarity of two graphs
with a measure of agreement between judges sources). Often, two independently built resources
classifying pairs of vertices into two cate- that describe the same linguistic reality only show a
gories (connected or not connected). When weak agreement even when based on human judge-
applied to synonymy networks, such measures ments under the same protocol (Murray and Green,
demonstrate a surprisingly low agreement be- 2004).
tween various resources of the same language. Many of such resources, such as WordNet (Fell-
This seems to suggest that the judgements
baum, 1998) or Wiktionary1 (Zesch et al., 2008;
on synonymy of lexemes of the same lexi-
con radically differ from one dictionary ed-
Sajous et al., 2010) can be modelled as graphs. A
itor to another. In fact, even a strong dis- graph encodes a binary relation on a set V of ver-
agreement between edges does not necessarily tices. A graph G = (V, E) is therefore defined by
mean that graphs model a completely differ- a finite, non empty set of n = |V | vertices and by
ent reality: although their edges seem to dis- a set E ⊆ V × V of m = |E| couples of vertices
agree, synonymy resources may, at a coarser (edges). In the linguistic field, vertices can be vari-
grain level, outline similar semantics. To in- ous elements of the lexicon: lemmas, word senses,
vestigate this hypothesis, we relied on shared
syntactic frames... and edges can describe various
common properties of real world data net-
works to look at the graphs at a more global relations: synonymy, hyperonymy, translation, co-
level by using random walks. They enabled occurrence... Edges between two vertices can be
us to reveal a much better agreement between seen as judgements that decide whether the consid-
dense zones than between edges of synonymy ered relation applies to this pair. For example, in a
graphs. These results suggest that although synonymy graph, an edge exists between two words
synonymy resources may disagree at the level if they were judged to be synonyms by the lexicogra-
of judgements on single pairs of words, they
pher who was compiling the dictionary. So, different
may nevertheless convey an essentially simi-
lar semantic information.
graphs that model dictionaries of synonyms are built
according to the judgements of various “judges”.
We first illustrate, in section 2, how various stan-
1 Introduction dard synonymy resources of English and French
share common structural properties: they all are Hi-
More and more resources exist, built with various
erarchical Small Worlds (HSW). However, we then
approaches and methods and with many different
1
aims and intended uses. A new issue raised by this http://www.wiktionary.org/
show that the synonymy judgements they describe 2.1 Invariants : similar structural properties
seem to disagree: the Kappa (Cohen, 1960) between Most lexical networks, as most field networks5 ,
the edges of any two such resources remains surpris- are Hierarchical Small World (HSW) Networks that
ingly low. In the third section, we analyse this appar- share similar properties (Watts and Strogatz, 1998;
ent disagreement and in section 4, we address it by Albert and Barabasi, 2002; Newman, 2003; Gaume
proposing an alternative view of the networks, based et al., 2010; Steyvers and Tenenbaum, 2005). They
on random walks. This more global view enables us exhibit a low density (not many edges), short paths
to assess if disagreeing synonymy networks never- (the average number of edges L on the shortest
theless concord at a more global level, because they path between two vertices is low), a high clustering
model the same linguistic reality. Beyond the usual rate C (locally densely connected subgraphs can be
Kappa agreement measure, which is based on the lo- found whereas the whole graph is globally sparse in
cal comparison of two category judgements (a pair is edges), and the distribution of their degrees follows
or is not a pair of synonyms), we can show that syn- a power law. All graphs in our sample exhibit the
onymy judgements do not essentially diverge on the HSW properties. For example, Table 1 shows the
lexical semantic structure that emerges from them. pedigrees of synonymy graphs of verbs(for space
In the fifth section, we conclude by outlining possi- reasons we only show results for verbs, results are
ble applications and perspectives of this work. similar for the two other parts of speech). In this ta-
ble, n and m are the number of vertices and edges,
hki is the average degree of vertices, and λ is the
2 Graph modelling of various synonymy coefficient of the power law that fits the distribution
resources of degrees, with a correlation coefficient r2 . nlcc
and Llcc are the number of vertices and the aver-
In order to study the similarities and variations of age path length measured on the largest connected
lexical resources, let us study a sample of graphs that component. Even if n and hki vary across dictionar-
model several standard synonymy resources. We ies, Llcc is always small, C is always higher than for
analyse five standard, general purpose, paper dictio- equivalent random graphs (Newman, 2003) and the
naries of French synonyms2 : Bailly (Bai), Benac distribution of degrees remains close to a power law
(Ben), Bertaud du Chazaut (Ber), Larousse (Lar), with a good correlation coefficient.
Robert (Rob). We also study synonymy relations ex-
tracted from the Princeton Word Net (P W N ) and Table 1: Pedigrees of seven synonymy graphs (verbs).
from the English Wiktionary (W ik). The P W N n m hki nlcc mlcc C Llcc λ r2
synonymy network was built according to the fol- BaiV 3082 3648 2.46 2774 3417 0.04 8.24 -2.33 0.94
BenV 3549 4680 2.73 3318 4528 0.03 6.52 -2.10 0.96
lowing rule: an edge is drawn between any two BerV 6561 25177 7.71 6524 25149 0.13 4.52 -1.88 0.93
words that belong to the same synset. The Wik- LarV 5377 22042 8.44 5193 21926 0.17 4.61 -1.94 0.88
RobV 7357 26567 7.48 7056 26401 0.12 4.59 -2.01 0.93
tionary synonymy network was extracted from Wik- P W NV 11529 23019 6.3 6534 20806 0.47 5.9 -2.4 0.90
tionary dumps3 by methods exposed in (Sajous et W ikV 7339 8353 2.8 4285 6093 0.11 8.9 -2.4 0.94
al., 2010). Each of these resources is split4 by parts
of speech (Nouns, Verbs, Adjectives) resulting in
three different synonymy graphs, designated, for ex- 2.2 Variability : a low agreement between
ample for the Robert dictionary, as follows: RobN , edges
RobV , RobA . Although all these graphs are HSW, Table 1 shows
that the lexical coverage (n) and the number of syn-
onymy links (m) significantly vary across graphs.
2
Synonymy relations from each of these dictionaries were Given two graphs G1 = (V1 , E1 ) and G2 =
extracted by the INALF/ATILF Research Unit and corrected by
5
the CRISCO Research Unit. Field networks are networks that model real data gathered
3
http://redac.univ-tlse2.fr/lexiques/wiktionaryx.html by field work, for example in sociology, linguistics or biol-
4
Note that splitting is not necessary. The following work ogy. They contrast with artificial networks (deterministic or
would apply similarly to whole resources. random).
• G02 = V 0 = (V1 ∩ V2 ), E20 = E2 ∩ (V 0 × V 0 ) ;

(V2 , E2 ), in order to compare their lexical cover-
ages, we compute the Recall (R• ), Precision (P• )
and F-score (F• ) of their vertex sets: For each pair of vertices (a, b) ∈ (V 0 × V 0 ), four
|V1 ∩V2 | |V1 ∩V2 | cases are possible:
R• (G1 , G2 ) = |V2 | P• (G1 , G2 ) = |V1 |
• (a, b) ∈ E10 E20 : agreement on pair (a, b),
T

F• (G1 , G2 ) = 2. RR••(G
(G1 ,G2 ).P• (G1 ,G2 ) (a, b) is synonymous for G01 and for G02 ;
1 ,G2 )+P• (G1 ,G2 )

F-scores of pairs of comparable graphs (same lan- • (a, b) ∈ E10 E20 : agreement on pair (a, b),
T
guage and same part of speech) of our sample re- (a, b) is neither synonymous for G01 nor for G02 ;
main moderate. Table 2 illustrates these measures on
• (a, b) ∈ E10 E20 : disagreement on pair (a, b),
T
the eleven pairs of graphs involving the five French
synonymy graphs (verbs) and the two English ones. (a, b) is synonymous for G01 but not for G02 ;
It shows that the lexical coverages of the various
• (a, b) ∈ E10 E20 : disagreement on pair (a, b),
T
synonymy graphs do not perfectly overlap.
(a, b) is synonymous for G02 but not for G01 ;

Table 2: Precision, Recall and F-score of vertex sets of The agreement between the two synonymy judge-
eleven pairs of graphs. G1 in rows, G2 in cols. ments of G1 and G2 is measured by Kl (G01 , G02 ), the
BenV BerV LarV RobV W ikV Kappa between the two sets of edges E10 and E20 :
R• 0.66 0.45 0.51 0.40
BaiV P• 0.76 0.96 0.90 0.95 (p0 − pe )
F• 0.71 0.61 0.65 0.56 Kl (G01 , G02 ) = (1)
R• 0.52 0.58 0.45 (1 − pe )
BenV P• 0.96 0.88 0.93
F• 0.68 0.70 0.60 where:
R• 0.85 0.73 1
BerV P• 0.70 0.82 p0 = .(|E10 ∩ E20 | + |E10 ∩ E20 |) (2)
F• 0.77 0.77 ω
R• 0.68 is the relative observed agreement between vertex
LarV P• 0.92
F• 0.78 pairs of G01 and vertex pairs of G02 , where ω is the
R• 0.49 number of possible edges6 ω = 12 .|V 0 |.(|V 0 | − 1).
P W NV P• 0.31
F• 0.38 1
pe = .(|E10 |.|E20 | + |E10 |.|E20 |) (3)
ω2
The value of F• (G1 , G2 ) measures the relative
is the hypothetical probability of chance agreement,
lexical coverage of G1 and G2 but does not eval-
assuming that judgements are independent7 .
uate the agreement between the synonymy judge-
The value of agreement on synonymy judgements
ments modelled by the graphs’ edges. The Kappa
Kl (G01 , G02 ) varies significantly across comparable
of Cohen (Cohen, 1960) is a common measure of
dictionary pairs of our sample, however it remains
agreement between different judges who categorize
quite low. For example: Kl (Rob0V , LarV0 ) = 0.518
the same set of objects. In the case of graphs, the
and Kl (P W NV0 , W ikV0 ) = 0.247 (cf. Table 3). On
judgements are not applied to simple entities but to
the whole sample studied in this work this agreement
relations between pairs of entities. Two synonymy
value ranges from 0.25 to 0.63 averaging to 0.39.
graphs G1 = (V1 , E1 ) and G2 = (V2 , E2 ) give two
This shows that, although standard dictionaries of
judgements on pairs of vertices. For example, if a
synonyms show similar structural properties, they
pair (u, v) ∈ V1 × V1 is judged as synonymous then
considerably disagree on which pairs of words are
(u, v) ∈ E1 , else (u, v) ∈ E1 . To measure the
synonymous.
agreement between edges of G1 and G2 , one first
6
has to reduce the two graphs to their common ver- Here, we do not consider reflexivity edges, that link ver-
tices: tices to themselves, as they are obviously in agreement across
graphs and are not informative synonymy judgements.
• G01 = V 0 = (V1 ∩ V2 ), E10 = E1 ∩ (V 0 × V 0 ) ;
 7
Note that Kl (G01 , G02 ) = Kl (G02 , G01 ).
3 Analysis of the disagreement between Since G is reflexive, each vertex has at least one
synonymy networks neighbour (itself) thus [G] is well defined. Further-
more, byPconstruction, [G] is a stochastic matrix:
When comparing two lexical resources built by lexi-
∀u ∈ V, v∈V gu,v = 1.
cographers, one can be surprised to find such a level
The probability PGt (u v) of a walker starting on
of disagreement on synonymy relations. This diver-
vertex u to reach a vertex v after t steps is:
gence in judgements can be explained by editorial
policies and choices (regarding, for example printed PGt (u v) = ([G]t )u,v (4)
size constraints, targeted audiences...). Furthermore, One can then prove (Gaume, 2004), with the
lexicographers also have their subjectivity. Since Perron-Frobenius theorem (Stewart, 1994), that if G
synonymy is more a continuous gradient than a dis- is connected8 (i.e. there is always at least one path
crete choice (Edmonds and Hirst, 2002), an alterna- between any two vertices), reflexive and undirected,
tive limited to synonym/not synonym leaves ample then ∀u, v ∈ V :
room for subjective interpretation. However, these dG (v)
justifications do not account for such discrepancies lim PGt (u v) = lim ([G]t )u,v = P
x∈V dG (x)
t→∞ t→∞
between resources describing the semantic relations (5)
of words of the same language. Therefore, we ex- It means that when t tends to infinity, the probability
pect that, if two words are deemed not synonyms of being on a vertex v at time t does not depend on
in one resource G1 , but synonyms in another G2 , the starting vertex but only on the degree of v. In the
they will nevertheless share many neighbours in G1 following we will refer to this limit as πG (v).
and G2 . In other words they will belong to the same
dense zones. Consequently the dense zones (or clus- 3.2 Confluence in synonymy networks
ters) found in G1 will be similar to those found in The dynamics of the convergence of random walks
G2 . Random walks are an efficient way to reveal towards the limit (Eq. (5)) is heavily dependent on
these dense zones (Gaume et al., 2010). So, to eval- the starting node. Indeed, the trajectory of the ran-
uate the hypothesis, let us begin by studying the sim- dom walker is completely governed by the topology
ilarity of random walks on various synonymy net- of the graph: after t steps, any vertex v located at a
works. distance of t links or less can be reached. The prob-
ability of this event depends on the number of paths
3.1 Random walks on synonymy networks between u and v, and on the structure of the graph
If G = (V, E) is a reflexive and undirected graph, around the intermediary vertices along those paths.
let us define dG (u) = |{v ∈ V /(u, v) ∈ E}| the The more interconnections between the vertices, the
degree of vertex u in graph G, and let us imagine a higher the probability of reaching v from u.
walker wandering on the graph G: For example, if we take G1 = RobV and
G2 = LarV , and choose the three vertices
• At a time t ∈ N, the walker is on one vertex u = éplucher (peel), r = dépecer (tear apart) and
u∈V; s = sonner (ring), which are such that:
• At time t + 1, the walker can reach any neigh- • u= éplucher (peel) and r= dépecer (tear apart)
bouring vertex of u, with uniform probability. are synonymous in RobV : (u, r) ∈ E1 ;
This process is called a simple random walk (Bol- • u= éplucher (peel) and r= dépecer (tear apart)
lobas, 2002). It can be defined by a Markov chain are not synonymous in LarV : (u, r) ∈
/ E2 ;
on V with a n × n transition matrix [G]:
• r= dépecer (tear apart) and s= sonner (ring)
[G] = (gu,v )u,v∈V have the same number of synonyms in G1 :
dG1 (r) = dG1 (s) = d1 ;
 1 if (u, v) ∈ E,

8
The graph needs to be connected for Eq. 5 to be valid but,
with gu,v = dG (u) in practice, the work presented here also holds on disconnected

0 else. graphs.
• r= dépecer (tear apart) and s= sonner (ring)
10-1
have the same number of synonyms in G2 : Pt (éplucher dépecer)t >1
dG2 (r) = dG2 (s) = d2 . Pt (éplucher sonner))t >1
Common asymptotical value
10-2

∗ )t >1
Then Equation (5) states that (PGt 1 (u r))1≤t and

Pt (éplucher
10-3
(PGt 1 (u s))1≤t converge to the same limit:

d1 10-4
πG1 (r) = πG1 (s) = P
x∈V1 dG1 (x)

as do (PGt 2 (u r))1≤t and (PGt 2 (u s))1≤t : 10-5 10 20 30 40 50


t : Length of random walk

d2 (a) G1 = RobV
πG2 (r) = πG2 (s) = P
x∈V2 dG2 (x)

10-2
Pt (éplucher dépecer)t >1
However the two series do not converge with the Pt (éplucher sonner)t >1
same dynamics. At the beginning of the walk, for t Common asymptotical value
small, one can expect that PGt 1 (u r) > PGt 1 (u s) 10-3

∗ )t ≥1
and PGt 2 (u r) > PGt 2 (u s) because éplucher is
semantically closer to dépecer than to sonner. In- Pt (éplucher

deed the number of short paths between éplucher


10-4
and dépecer is much greater than between éplucher
and sonner.
Figure 1(a) shows the values of PGt 1 (u r)
10-5
and PGt 1 (u s) versus t, and compares them 10 20 30
t : Length of random walk
40 50

to their common limit. Figure 1(b) shows


the values of PGt 2 (u r) and PGt 2 (u s) ver- (b) G2 = LarV
sus t, and compares them to their common limit.
These figures confirm our intuition that, since Figure 1: Confluences between éplucher (peel),
éplucher (peel) and dépecer (tear apart) are seman- dépecer (tear apart) and éplucher (peel), sonner (ring)
in RobV and LarV .
tically close, PGt 1 (u r) and PGt 2 (u r) decrease to
their limit. We call this phenomenon strong con-
fluence. It is worth noting that this remains true (2) weak in both graphs (confluence agreement),
even if éplucher (peel) and dépecer (tear apart)
are not synonyms in LarV . Conversely, since (3) strong in one graph, but weak in the other (con-
éplucher (peel) and sonner (ring) are semantically fluence disagreement).
distant, PGt 1 (u s) and PGt 2 (u s) increase to their To contrast cases (1) and (2) from case (3) we
asymptotic value. We call this phenomenon weak measure the correlation between the confluences of
confluence. disagreeing pairs of two synonymy networks G01
and G02 . We compare it to this same correlation on
3.3 Correlation of the confluence of
two reflexive and undirected random graphs RG01 =
disagreeing synonymy pairs
(V 0 , E1R ) and RG02 = (V 0 , E2R ) built such that:
When two graphs G1 and G2 disagree on a pair of
vertices (a, b) (a is a neighbour of b in one graph but |E1R ∩ E2R | = |E10 ∩ E20 |,
not in the other) there are three possible cases for the
strength of the confluence between vertices a and b: |E1R ∩ E2R | = |E10 ∩ E20 |,
(1) strong in both graphs (confluence agreement), |E1R ∩ E2R | = |E10 ∩ E20 |,
which means that the Kappa agreement between strong agreement is found between the two resulting
RG01 and RG02 is the same as between G01 and G02 . graphs.
For a given t > 1 and a set of vertex pairs X ⊆ Let G1 = (V1 , E1 ) and G2 = (V2 , E2 ) be two
V × V 0 , the correlation of confluences ΓX (G01 , G02 )
0 synonymy networks, both reflexive, undirected, con-
is defined by the Pearson’s linear correlation nected, and a given t ∈ N∗ . We define:
 coef-
ficient of the two value tables PGt 0 (u v) (u,v)∈X
• G01 = V 0 = (V1 ∩ V2 ), E10 = E1 ∩ (V 0 × V 0 )

1
and PGt 0 (u v) (u,v)∈X .

2
• G02 = V 0 = (V1 ∩ V2 ), E20 = E2 ∩ (V 0 × V 0 )

For all comparable pairs of our sample, we
see that disagreeing pairs tend to have a much (+G2 )
higher correlation of confluence than disagreeing • G1 = (V 0 , E1+ = E10 ∪ C1 ) where
pairs of equivalent random networks. As an ex- n o
(u, r) ∈ E10 ∩ E20 PG
 t
C1 = 0 (u r) > πG0 (r) (6)
ample, for G1 = RobV , G2 = LarV and 1 1

t = 3, we have ΓE 0 T E 0 (G01 , G02 ) = 0.41 and


1 2 (+G1 )
ΓE 0 T E 0 (G01 , G02 ) = 0.38, whereas in the case of • G2 = (V 0 , E2+ = E20 ∪ C2 ) where
1 2
the equivalent random graphs the same figures are n o
(u, r) ∈ E10 ∩ E20 PG
 t
C2 = 0 (u r) > πG0 (r) (7)
close to zero. 2 2

This suggests that even if graphs disagree on the


(+G ) (+G )
synonymy of a significant number of pairs, they nev- G1 2 and G2 1 are called accommodating
ertheless generally agree on the strength of their graphs. The construction of the accommodating
confluence. In other words, occurrences of cases (1) graphs may be metaphorically understood as a con-
and (2) are the majority whereas occurrences of case ciliation protocol by which two graphs accept pro-
(3) are rare. We propose in the next section an exper- posals of the other that they can reconsider. For ex-
(+G )
iment to verify if we can rely on confluence to find a ample, G1 2 is the graph G01 enriched by edges
greater agreement between two graphs that disagree (u, r) of G02 such that there is a strong confluence
at the level of synonymy links. between vertices u and r in G01
The following property is worth noticing:
4 Self mediated agreement by confluence Proposition 1. ∀t ∈ N∗ :
4.1 Hypothesis: Conciliation reveals structural
(E10 ∩ E20 ) ⊆ (E1+ ∩ E2+ ) ⊆ (E10 ∪ E20 ) (8)
similarity beyond disagreement of local
synonymy Proof. By definition, E1+ = E10 ∪ C1 and E2+ =
We saw in section 2.2 that the rate of agreement be- E20 ∪C2 , thus (E10 ∩E20 ) ⊆ (E1+ ∩E2+ ), furthermore,
tween edges of two standard synonymy networks G01 by definition, C1 ⊆ E10 ∩ E20 and C2 ⊆ E10 ∩ E20 thus
and G02 , Kl (G01 , G02 ), is usually low. However, we (E1+ ∩ E2+ ) ⊆ (E10 ∪ E20 ).
have noticed in Section 3.3 that the confluences of
4.2 Experimental protocol
pairs on which synonymy graphs disagree are sig-
nificantly more correlated (Γ ≈ 0.4) than the conflu- If, for any (G1 , G2 ) synonymy resources of the
(+G ) (+G )
ence of equivalent random networks (Γ ≈ 0). This same language, Kl (G1 2 , G2 1 ) is signifi-
suggests the following hypothesis: synonymy net- cantly greater than Kl (G01 , G02 ), then the hypothe-
works are in agreement at a level that is not taken sis is verified. The conciliation process depends on
into account by the Kappa measure on edges. confluence measures that depend on a given t, the
To verify this hypothesis, we try to make each pair number of steps of the random walk. For t = 1,
of graphs conciliate on the basis of confluence val- only vertices in the neighbourhood of the starting
ues. We propose a conciliation process by which vertex are reachable. Consequently only pairs of
a graph can accept the addition of another’s edges vertices that are edges have a non null confluence.
(+G ) (+G )
if they do not contradict its structure (i.e. there Thus Kl (G1 2 , G2 1 ) = Kl (G01 , G02 ) which
is a strong confluence value). We then assess if a does not help us to contrast conciliated graphs from
initial binary synonymy graphs. So we fix t = 2
Table 3: Kappa (ori.) and accommodating Kappa (acc.)
the shortest walk length that still yields informative values between French and English synonymy graphs (of
results. verbs), compared with the Kappa values between pairs of
We propose a control experiment that consists equivalent random graphs (“ori. r.” and “acc. r.”).
in applying the conciliation process to random net- Kl BenV BerV LarV RobV W ikV
works that have the same Kappa as the pairs of syn- ori. 0.583 0.309 0.255 0.288
acc. 0.777 0.572 0.603 0.567
onymy networks. The construction of these random BaiV
ori. r. 0.583 0.309 0.256 0.288
graphs is described above, in section 3.3. We mea- acc. r. 0.585 0.313 0.262 0.293
sure the agreement after conciliation of 20 different ori. 0.389 0.276 0.293
random graphs. With this control experiment we as- acc. 0.657 0.689 0.636
BenV
ori. r. 0.390 0.276 0.294
sess that the observed results are specific to graphs
acc. r. 0.392 0.283 0.301
describing the same resource, and not a mere bias of ori. 0.416 0.538
the protocol (let us imagine a protocol whereby one acc. 0.838 0.868
BerV
would add all the disagreeing edges to the graphs: ori. r. 0.417 0.539
acc. r. 0.434 0.549
not only the Kappa of the pseudo accommodating
ori. 0.518
synonymy graphs would be equal to one, but also the acc. 0.852
Kappa of pseudo accommodating random graphs, LarV
ori. r. 0.518
which would disqualify the protocol). acc. r. 0.529
ori. 0.247
acc. 0.540
4.3 Results P W NV
ori. r. 0.247
Table 3 summarizes Kappa and conciliated Kappa acc. r. 0.251
values on the pairs of synonymy graphs of verbs.
It shows a significant improvement of agreement So, when G1 and G2 are two synonymy graphs
after conciliation. For example, from a moder- of a given language, then they are able to address
ate Kappa (0.518) between graphs Rob0V and LarV0 their local synonymy disagreement and to reach a
(constructed by experts), the conciliation process significantly better agreement. On the other hand,
leads to an excellent Kappa (0.852). Conversely the the agreement of random networks does not really
random networks only increase their agreement by improve after conciliation. This proves that the syn-
0.01 (with a very low standard deviation σ < 0.001). onymy networks of the same language share specific
In English, from a poor (0.247) Kappa between similar structures that can be detected with the help
P W NV0 (constructed by experts) and W ikV0 (con- of confluence measures.
structed by the “crowds”), the conciliation process
leads to a moderate Kappa (0.530), whereas the ran- 5 Conclusion
dom networks only marginally increase their agree-
Although graphs that encode synonymy judgements
ment (0.004).
of standard semantic lexical resources share simi-
Results are similar for other parts of speech. This lar HSW properties they diverge on their synonymy
means that the conciliation process significantly im- judgements as measured by a low Kappa of edges.
proves the agreement between resources, even if So, one could wonder whether the notion of syn-
they are originally significantly diverge. onymy is well defined, or if synonymy judgements
It is interesting to notice that the most sim- are really independent. Without directly address-
ilar pairs in terms of edge agreement do not ing this question, we nevertheless have shown that
necessarily produce the most agreeing pairs strong confluence measures help two synonymy
of accommodating graphs. For example, the graphs accommodate each others’ conflicting edges.
pair(BaiV , RobV ) agrees more than the pair They reach a much better agreement, whereas ran-
(BaiV , LarV ), whereas for their accommodating dom graphs’ divergence is maintained. Since the
(+RobV ) (+BaiV )
graphs, the pair(BaiV , RobV ) agrees graphs are HSW, they draw clusters of synonyms
(+LarV ) (+BaiV )
less than the pair (BaiV , LarV ). in which pairs of vertices have a strong confluence.
This suggests two conclusions. First, different syn- • The conciliation process seems unbalanced in-
onymy resources that describe the same lexicon re- sofar as graphs only accept to add edges. It
veal dense zones that are much more similar across should be extended to a negotiating process
graphs than the binary synonymy categorisation (the where a graph could also accept to remove one
synonym/not synonym alternative). These dense edge if the other does not have it and its conflu-
zones convey information about the semantic organ- ence is weak.
isation of the lexicon. Second, random walks and
confluence measures seem an appropriate technique • The conciliation process could also be gen-
to detect and compare the dense zones of various eralised to graphs that have different vertices,
synonymy graphs. such as two synonymy networks of different
This theoretical work validates the random languages. In that case the issue is not anymore
walk/confluence approach as a potentially valid tool to reveal a deeper similarity, beyond a local
for detecting semantic similarities. This opens many disagreement, because one can not compare
perspectives for applications. For example, it can the graphs vertex by vertex or edge by edge.
be used to enrich resources as was done for the However, questioning whether the semantic
Wisigoth project (Sajous et al., 2010). It may also structures revealed by dense zones are similar
help to merge, or aggregate, resources. If we apply from one lexicon to another is an interesting
the conciliation process to two graphs G1 and G2 , line of research. One approach to compare two
(+G ) synonymy graphs of two different languages
obtaining two accommodating graphs G1 2 =
(+G ) would be to draw edges between vertices that
(V 0 , E1+ )) and G2 1 = (V 0 , E2+ )) then the graph are translations of each other. Random walks
G = (V 0 , E 00 = (E1+ ∩ E2+ )) could be a merged could then reach vertices of the two lexicons,
resource. Indeed, G’s set of edges, E 00 seems like so that the conciliation process could be
a good compromise because, according to the prop- generalised to accommodating two synonymy
erty 1, (E10 ∩ E20 ) ⊆ E 00 ⊆ (E10 ∪ E20 ). This new graphs via translation links.
aggregation method would need to be validated by
comparing the quality of the merged resource to the
results of the union or intersection. Acknowledgements
Furthermore, this work is a first step for defin-
ing a similarity measure between graphs, that could The research presented in this paper was supported
take into account the structural agreement rather by the ANR-NSC (France-Taiwan) bilateral project
than a simple edge-to-edge disagreement. Subse- M3 (Modeling and Measurement of Meaning). We
quent work should generalise the conciliation pro- would like to thank the reviewers for their insightful
cess along several axes: comments.

• The number of steps t was chosen as the short-


est possible for the confluence measures. It References
would be worthwhile to investigate the effect
[Albert and Barabasi2002] Réka Albert and Albert-
of the length of the walks on the agreement of
László Barabasi. 2002. Statistical Mechanics of
the accommodating graphs. Complex Networks. Reviews of Modern Physics,
74:74–47.
• Another line of research would be to alter the [Bollobas2002] Bela Bollobas. 2002. Modern Graph
conciliation ability of graphs, by increasing or Theory. Springer-Verlag New York Inc., October.
decreasing the criterion for strong confluence. [Cohen1960] Jacob Cohen. 1960. A coefficient of
One can for example introduce a k parameter in agreement for nominal scales. Educ. Psychol. Meas.,
the definition of C1 (resp. C2 ), in Equation 6: (20):27–46.
[Edmonds and Hirst2002] Philip Edmonds and Graeme
Hirst. 2002. Near-Synonymy and Lexical Choice.
PGt 0 (u r) > k.πG01 (r) (9)
1 Computational Linguistics, 28(2):105–144.
[Fellbaum1998] Christiane Fellbaum, editor. 1998.
WordNet: An Electronic Lexical Database. MIT Press.
[Gaume et al.2010] Bruno Gaume, Fabien Mathieu, and
Emmanuel Navarro. 2010. Building Real-World
Complex Networks by Wandering on Random Graphs.
I3: Information Interaction Intelligence, 10(1).
[Gaume2004] Bruno Gaume. 2004. Balades Aléatoires
dans les Petits Mondes Lexicaux. I3: Information In-
teraction Intelligence, 4(2).
[Murray and Green2004] G. Craig Murray and Rebecca
Green. 2004. Lexical Knowledge and Human Dis-
agreement on a WSD Task. Computer Speech & Lan-
guage, 18(3):209–222.
[Newman2003] M. E. J. Newman. 2003. The Structure
and Function of Complex Networks. SIAM Review,
45:167–256.
[Sajous et al.2010] Franck Sajous, Emmanuel Navarro,
Bruno Gaume, Laurent Prévot, and Yannick Chudy.
2010. Semi-automatic endogenous enrichment of col-
laboratively constructed lexical resources: Piggyback-
ing onto wiktionary. In Hrafn Loftsson, Eiríkur Rögn-
valdsson, and Sigrún Helgadóttir, editors, Advances in
NLP, volume 6233 of LNCS, pages 332–344. Springer
Berlin / Heidelberg.
[Stewart1994] G. W. Stewart. 1994. Perron-frobenius
theory: a new proof of the basics. Technical report,
College Park, MD, USA.
[Steyvers and Tenenbaum2005] Mark Steyvers and
Joshua B. Tenenbaum. 2005. The large-scale struc-
ture of semantic networks: Statistical analyses and
a model of semantic growth. Cognitive Science,
29(1):41–78.
[Watts and Strogatz1998] Duncan J. Watts and Steven H.
Strogatz. 1998. Collective Dynamics of Small-World
Networks. Nature, 393:440–442.
[Zesch et al.2008] Torsten Zesch, Christof Müller, and
Iryna Gurevych. 2008. Using wiktionary for comput-
ing semantic relatedness. In Proceedings of the 23rd
national conference on Artificial intelligence - Volume
2, pages 861–866, Chicago, Illinois. AAAI Press.

You might also like