Efficient Session Retrieval Using Topical Index Shards

by Gijs Hendriksen, Djoerd Hiemstra, and Arjen de Vries.

Retrieval is often considered one query at a time. However, in practice, queries regularly come in the context of sessions with coherent topics. By dividing a collection into topical index shards and matching the topical context of a session with the right shards, we may reduce the amount of resources required for answering each query. We consider two alternatives: (1) starting with exhaustive search and pruning unnecessary shards after each session turn, and (2) applying a resource selection algorithm to pre-select shards at the start of the session. We empirically evaluate our approaches on a conversational search dataset (CAsT), and compare effectiveness and resource usage against exhaustive retrieval. Our experiments show that both approaches reduce the number of postings necessary to fulfill a search request (by 50-80%), and in terms of effectiveness our systems are statistically indistinguishable from a system performing exhaustive retrieval.

To be presented at the European Conference on Information Retrieval (ECIR 2025) in Lucca, Italy on 6-10 April 2025.

[download pdf]

Leave a Reply

Your email address will not be published.