Consider the case of a query q passed by a user u to retrieve a set of documents .

We
need to order by their relevance to the query for that particular user. We are given
some training data that built from clicks gathered from users. The data includes the set of
documents a user has visited from the list of displayed documents. This provides some
implicit information of that user's preferred ranking of the listed documents. We can
impose a partial ordering on the documents and use it for learning parameters for our
ranking model.
y

= |(I
]
> I
k
)|], k |I||
In summary, we are given the training data that contains queryuser tuples x

= (q

, u

)
and the corresponding partial ordering derived from their click data
listing all pairs I
]
, I
k
such that document I
]
should be ranked above document I
k
. !ow,
given a test input x
t
= (q
t
, u
t
) we wish to rank a set of documents .
Current state of the art uses losscalibrated ma"margin methods #or structured prediction
models$ to rank the documents. There are two kinds of loss functions. %ne that imposes a
ranking on each pair of documents that we refer to as pairwise loss. The other kind that
we call the listwise loss imposes a complete ordering on the complete list of documents.
The general framework of optimi&ation has the following form'
orgmin
]F
[F [
p
:
+ (

sub]cct to (x

, y

) > (x

, y
]
) + A(y

, y
]
) - (

v] = i,
onJ (

> u vi,
wheie is a pieuictoi fiom the paiametiic class of functions F the capacity of which
is contiolleu by an I
p
-noim penalty on the paiameteis. The slack vaiiable (

takes a
positive value if the coiiesponuing constiaint is voilateu anu is zeio otheiwise by
complimentaiy slackness. The loss function is uenoteu by A(y

, y
]
) takes two iankings
y

anu y
]
of the same set of uocuments anu ietuins a ieal numbei quantifying theii
uiffeience.
The class of pieuictois is usually the lineai functions. Theie aie as many constiaints
anu coiiesponuingly slacks as theie aie incoiiect iankings. Bowevei, this is veiy
expinsive given that theie will be exponentially laige numbei of costiaints. vaiious
methous like the cutting-plane algoiithm can be useu to finu the solution. Theie aie to
equivalent foimulations calleu the N-slack anu the 1-slack of which the lattei is easiei
to solve. We biiefly uiscuss these uetails latei in the uiaft.
Choice of A is known to be an impoitant factoi foi ianking peifoimance. Among
paiiwise losses the following weie uiscusseu by Chakiaboithi et. al. (2uu8):
Learning to rank
(aturday, )arch *+, ,-*. **'/0 1)
Learning to rank Page 1
1. Nean aveiage piecision
2. Aiea unuei ieceivei opeiating cuive
S. Noimalizei uiscounteu cumulative gain
4. Nean iecipiocal iank
S. Boosteu iegiession tiees
6. Biiect optimization of ianking measuie(..-not cleai to me)
paiiwise losses the following weie uiscusseu by Chakiaboithi et. al. (2uu8):
1. Accuiacy at the top by Boyu et. al. (2u11)
2. Infinite push by Agaiwal et. al. (2u11)
Also see Chao et. al. #,--2$. S.
Alteinatively one coulu choose a ianking measuie that scoies a completely oiueieu
list using a listwise loss as the following functions:
Theie is eviuence to claim that listwise losses peifoim bettei. Alteinative to the max-
maigin fiamewoik, one coulu use an exponential of some pieuiction scoie anu
maximize the likelihoou of tiaining uata, possibly with some iegulaiization, similai to
the logistic iegiession fiamewoik. The pieuiction scoie heie shoulu ieflect the
piopeities we wish to have in out iesulting ianking as uesciibeu by Chao et. al. (2uu7)
anu 3ali&adegan et. al. #,--0$ #44read this ref.$.
An oveiview of oui methou is as follows. We consiuei an s-uimensional latent space of
both queiy-usei tuples anu uocuments to be iankeu. We pioject them both into the
latent space compaie theii ielevance using a uot piouuct. The magnituue of the uot
piouucts ieveal the uegiee of ielevance of the uocument foi that queiy-usei
combination. The iesulting scoie imposes a ianking on the uocuments. We leain the
piojections of the featuies of queiy-usei tuples anu the uocuments inuepenuently into
the latent space.
0n the computational siue, this woulu have the uisauvantage of significantly high
numbei of paiameteis. Bowevei, we use a low iank constiaint on the piojection
matiices to ensuie that latent space assumption ieflects in the paiameteis. This will
bounu the numbei of paiameteis iequiieu to encoue the pioblem. Auuitionally, we
leveiage the latent space encouing to oveicome the classical colu-stait pioblem which
of significant piactical inteiest.
The closest woik to ouis is that of Lanciiet et. al. (2u1u). They have uiscusseu ianking
in ielation to metiic leaining to pioject obseivations into a space wheie expecteu
oiueiing among the uocuments is iespecteu. We uesciibe the noiation anu foimalize
oui methou in the following sections.
Consiuei a featuie encouing of the queiy-usei obseivation tuple into a J-uimensional
space by a mapping : x = (q, u) - R
d
. Let w

be a (J s) matiix that piojects anu

input featuie (x) to an s-uimensional space. Similaily, let () be an c-uimensional
encouing of a uocument . Now, let w

be an (c s) matiix that piojects the

uocument featuies into an s-uimensional subspace. Consiueiing a lineai set of
pieuectois the scoie is ielevance of an input x anu a uocument is
Learning to rank Page 2
o = (x)
1
w

w
o
().
The ianking of uocuments is given by soiting the uocuments by theii o scoies.
(. Chakrabarti, 5. 6hanna, 7. (awant, and C. 8hattacharyya. (tructured 9earning for
!on(mooth 5anking 9osses. In :roceedings of 6nowledge and ;ata ;iscovery, ,--<.
*.
,. (. 8oyd, C. Cortes, ). )ohri, and 1. 5adovanovic. 1ccuracy at the Top. In 1dvances of
the !eural Information :rocessing (ystems, ,-**.
(. 1garwal. The in=nite push' 1 new support vector ranking algorithm that directly
optimi&es accuracy at the absolute top of the list. In :roceedings of the (I1)
International Conference on ;ata )ining, ,-**.
/.
.. >. 3ali&adegan, 5. ?in, 5. @hang, and ?. )ao. 9earning to 5ank by %ptimi&ing !;CA
)easure. In 1dvances of the !eural Information :rocessing (ystems, ,--0.
@. Chao, T. Bin, TC. 9iu, )D. Tsai, and >. 9i. 9earning to 5ank' Drom :airwise 1pproach
to 9istwise 1pproach. In :roceedings of the International Conference on )achine
9earning, ,--2.
+.
References
Learning to rank Page 3