Learning to Rank Using Implicit Feedback andActive Exploration : Applying the Glicko Rating System to a People-Search Context

Abstract: Learning to rank using implicit feedback is a relatively new supervised machine learning method, in which training data is used in order to automatically calculate search engine document ranking parameters. This thesis describes the implementation and evaluation of a rank learning algorithm with active learning.The relevance judgments are collected from users in the form of click-through data, and the ranking parameters are learned incrementally over time.The online learner is implemented as a faculty search engine at the Royal Institute of Technology, and is evaluated both viahuman usage and user simulation experiments. The Kendall Tau Ranking Correlation Coecient is used in order to comparethe learned ranks with the corresponding true ranks of the experiments.Results of the evaluation show that the online learner fares marginally better in situations where users hold a common view of the relevance values of the documents in the document corpus.The results are not entirely conclusive, however. Hence, improvements of both the learner and the evaluation techniques are proposed for future researchers.