An Index Lake for Anserini
by Gijs Hendriksen, Djoerd Hiemstra, Jimmy Lin, and Arjen de Vries
We present an index lake for Anserini: a remote index representation of all inverted indexes from Anserini’s reproducibility suite. Each index is represented as a set of Parquet files on remote object storage, and retrieval is performed through SQL with DuckDB. Through an extensive benchmark of 239 experiments across 50 collections, we empirically confirm our results are consistent with Anserini’s. We analyze query latencies over the index lake and conclude that (1) most experiments can be run relatively efficiently, requiring only a couple of seconds per query; and (2) for large collections and learned sparse indexes, queries suffer from increased latency due to high overhead. We are also able to reduce total network transfer, since (1) index lakes are smaller than the corresponding prebuilt Lucene indexes; and (2) up to 90% of the index can be skipped by pushing filters on the postings table to the data lake. The index lake increases reproducibility and makes it easier to run new experiments on standard test collections and prebuilt indexes.
To be presented at the 35th ACM International Conference on Information and Knowledge Management (CIKM 2026) on November 7-11, 2026 in Rome, Italy
