Title: The evaluation of information retrieval systems by Wessel Kraaij (TNO-ICT) Wednesday 4 October, 13.45 hour in LA-1812 This lecture provides the tools and methodology for comparing the effectiveness of two or more information retrieval systems in a meaningful way. Several aspects of information retrieval systems can be evaluated without consulting the potential users or customers of the system, such as for instance the query processing time (measured for instance in miliseconds per query) or the query throughput (measured for instance as the number of queries per second). This lecture, however, focuses on aspects of the system that in uence the quality of the retrieved results. In order to measure the quality of search results, one must at some point consult the potential user of the system. For, what are the correct results for the query “black jaguar”? Cars, or cats? Ultimately, the user has to decide….