Today, we kick-off our new EU project OpenWebSearch.eu. In the project, we develop a new architecture for search engines where many parts of the system will be decentralized. The key idea is to separate index construction from the search engines themselves, where the most expensive step to create index shards can be carried out on large clusters while the search engine itself can be operated locally.
We also envision an Open-Web-Search Engine Hub, where companies and individuals can share their specifications of search engines and pre-computed, regularly updated search indices. We think of this as a search engine mash-up, that would enable a new future of human-centric search without privacy concerns.
More information at: https://openwebsearch.eu/partners/radboud-university/
In reply to Open Web Search project kicked off by Djoerd Hiemstra
I’m looking forward to following this project, Djoerd! It sounds sort-of IndieWeb like. Where e.g. MicroSub decouples feed fetching from feed reading and Micropub writing from posting, this project decouples index building from the search. Within IndieWeb that allows the creation of a variety of personal tools, to read and write the web. I’ve long been musing about personal search engines and personal agents and crawlers without putting anything into action. I’m curious to see if this project will actually deliver some of the things I dreamt of over time, by enabling personal tools for search.
Nice comparison of OpenWebSearch.eu and the Indy Web! Yes, the IndyWeb session we did a few years back was inspiring and helped thinking about the goals of our project! 🙂
Many thanks for the shout out on your blog!