Somto Enendu graduates cum laude on labelling document images

Predicting Semantic Labels of Text Regions in Heterogeneous Document Images

by Somtochukwu Enendu

This MSc thesis describes the use of sequence labeling methods in predicting the semantic labels of extracted text regions of heterogeneous electronic documents, by utilizing features related to each semantic label. In this study, we construct a novel dataset consisting of real world documents from multiple domains. We test the performance of the methods on the dataset and offer a novel investigation into the influence of textual features on performance across multiple domains. The results of the experiments show that the Conditional Random Field method is robust, outperforming the neural network when limited training data is available. Regarding generalizability, our experiments show that the inclusion of textual features does not guarantee performance improvements.

[download pdf]

Ties de Kock graduates on visualization recommendation

Visualization recommendation in a natural setting

by Ties de Kock

Data visualization is often the first step in data analysis. However, creating visualizations is hard: it depends on both knowledge about the data and design knowledge. While more and more data is becoming available, appropriate visualizations are needed to explore this data and extract information. Knowledge of design guidelines is needed to create useful visualizations, that are easy to understand and communicate information effectively.
Visualization recommendation systems support an analyst in choosing an appropriate visualization by providing visualizations, generated from design guidelines implemented as (design) rules. Finding these visualizations is a non-convex optimization problem where design rules are often mutually exclusive: For example, on a scatter plot, the axes can often be swapped; however, it is common to have time on the x-axis.
We propose a system where design rules are implemented as hard criteria and heuristics encoded as soft criteria that do not need to be satisfied, that guide the system toward effective chart designs. We implement this approach in a visualization recommendation system named OVERLOOK , modeled as an optimization problem implemented with the Z3 Satisfiability Modulo Theories solver. Solving this multi-objective optimization problem results in a Pareto front of visualizations balancing heuristics, of which the top results were evaluated in a user study using an evaluation scale for the quality of visualizations as well as the low-level component tasks for which they can be used. In evaluation, we did not find a difference in performance between OVERLOOK and a baseline of manually created visualizations for the same datasets.
We demonstrated OVERLOOK, a system that creates visualization prototypes based on formal rules and ranks them using the scores from both hard- and soft criteria. The visualizations from OVERLOOK were evaluated in a user study for quality. We demonstrate that the system can be used in a realistic setting. The results lead to future work on learning weights for partial scores, given a low-level component task, based on the human quality annotations for generated visualizations.

[download pdf]

ECIR 2019 proceedings online

by Leif Azzopardi, Benno Stein, Norbert Fuhr, Philipp Mayr, Claudia Hauff, and Djoerd Hiemstra

The 41st European Conference on Information Retrieval (ECIR) was held in Cologne, Germany, during April 14–18, 2019, and brought together hundreds of researchers from Europe and abroad. The conference was organized by GESIS–Leibniz Institute for the Social Sciences and the University of Duisburg-Essen — in cooperation with the British Computer Society’s Information Retrieval Specialist Group (BCS-IRSG). These proceedings contain the papers, presentations, workshops, and tutorials given during the conference. This year the ECIR 2019 program boasted a variety of novel work from contributors from all around the world and provided new platforms for promoting information retrieval-related (IR) activities from the CLEF Initiative. In total, 365 submissions were fielded across the tracks from 50 different countries.
The final program included 39 full papers (23% acceptance rate), 44 short papers (29% acceptance rate), eight demonstration papers (67% acceptance rate), nine reproducibility full papers (75% acceptance rate), and eight invited CLEF papers. All submissions were peer reviewed by at least three international Program Committee members to ensure that only submissions of the highest quality were included in the final program. As part of the reviewing process we also provided more detailed review forms and guidelines to help reviewers identify common errors in IR experimentation as a way to help ensure consistency and quality across the reviews.
The accepted papers cover the state of the art in IR: evaluation, deep learning, dialogue and conversational approaches, diversity, knowledge graphs, recommender systems, retrieval methods, user behavior, topic modelling, etc., and also include novel application areas beyond traditional text and Web documents such as the processing and retrieval of narrative histories, images, jobs, biodiversity, medical text, and math. The program boasted a high proportion of papers with students as first authors, as well as papers from a variety of universities, research institutes, and commercial organizations.
In addition to the papers, the program also included two keynotes, four tutorials, four workshops, a doctoral consortium, and an industry day. The first keynote was presented by this year’s BCS IRSG Karen Sparck Jones Award winner, Prof. Krisztian Balog, On Entities and Evaluation, and the second keynote was presented by Prof. Markus Strohmaier, On Ranking People. The tutorials covered a range of topics from conducting lab-based experiments and statistical analysis to categorization and deeplearning, while the workshops brought together participants to discuss algorithm selection (AMIR), narrative extraction (Text2Story), Bibliometrics (BIR), as well as social media personalization and search (SoMePeAS). As part of this year’s ECIR we also introduced a new CLEF session to enable CLEF organizers to report on and promote their upcoming tracks. In sum, this added to the success and diversity of ECIR and helped build bridges between communities.
The success of ECIR 2019 would not have been possible without all the help from the team of volunteers and reviewers. We wish to thank all our track chairs for coordinating the different tracks along with the teams of meta-reviewers and reviewers who helped ensure the high quality of the program. We also wish to thank the demo chairs: Christina Lioma and Dagmar Kern; student mentorship chairs: Ahmet Aker and Laura Dietz; doctoral consortium chairs: Ahmet Aker, Dimitar Dimitrov and Zeljko Carevic; workshop chairs: Diane Kelly and Andreas Rauber; tutorial chairs: Guillaume Cabanac and Suzan Verberne; industry chair: Udo Kruschwitz; publicity chair: Ingo Frommholz; and sponsorship chairs: Jochen L. Leidner and Karam Abdulahhad. We would like to thank our webmaster, Sascha Schüller and our local chair, Nina Dietzel along with all the student volunteers who helped to create an excellent online and offline experience for participants and attendees.

Published as: Advances in Information Retrieval. Proceedings of the 41st European Conference on Information Retrieval Research (ECIR), Lecture Notes in Computer Science, volumes 11437 and 11438, Springer, 2019
[Part I] [Part II]

Goodbye everybody at U. Twente

(written for CS teaching mailing no. 16 of 11 July)

As of 1 July, I will leave the U. Twente after almost 30 years (first as student, then as PhD student, finally as staff member) for a new challenge at the Radboud University in Nijmegen. I am proud to announce that I will join Radboud University’s faculty of science as professor of Federated Search.

I was privileged to teach in a world that changed a lot since I became an assistant professor (in 2001). Today, university-level courses are no longer taught for the privileged few at universities in developed countries. They are now freely available to anyone online via platforms like Coursera, edX, FutureLearn and on social media, such as on YouTube. Over the last 18 years, I tried to stimulate students to find additional study material online. In return I tried to contribute to the online study material by publishing my teaching material for students to use and for colleagues to share (my Canvas courses are still entirely publicly available) and by using novel social media like UT Mastodon (

In my years at the UT, I enjoyed promoting critical thinking by letting students actively put theory to practice, instead of letting students passively absorb knowledge. I particularly enjoyed developing the MSc course Managing Big Data with Maarten Fokkinga and Robin Aly (later perfected by Doina Bucur) where students analysed terabytes of data on a large Hadoop cluster. I enjoyed developing the BSc module Data & Information with Klaas Sikkel, Maurice van Keulen and Luís Ferreira Pires, where we let students work in agile teams, including daily stand-ups, sprint review meetings, and sprint backlogs. I also very much liked running the MSc course Information Retrieval with Paul van der Vet, Theo Huibers and Dolf Trieschnigg, where students used open source search engines and actively contributed to our research. Some of that work was published, and in such cases, students presented their work at international workshops or conferences.

Saying goodbye to Twente is harder than I expected. But remember, Nijmegen is close by: Feel free to contact me. As for PhD students, I intend to continue to be an active contributor to the courses of the Dutch research school SIKS: I hope to see you there.

Goodbye everybody!

The influence of network structure and prosocial cultural norms on charitable giving

A multilevel analysis of Movember’s fundraising campaigns in 24 countries

by Tijs van den Broek, Ariana Need, Michel Ehrenhard, Anna Priante, and Djoerd Hiemstra

This study examines how the interplay between an online campaign’s network structure and prosocial cultural norms in a country affect charitable giving. We conducted a multilevel analysis that includes Twitter network and aggregated donation data from the 2013 Movember fundraising campaigns in 24 countries during 62 campaign days. Prosocial cultural norms did not affect the relationship between network size and average donations raised, nor did they affect the relationship between network centralization and average donation amount. Prosocial cultural norms did affect the relationship between network density and average donations raised. However, this effect was negative and contrary to our expectation.

Published in Social Networks 58, pages 128-135

[download pdf]

Beyond research and teaching: on the role of universities in our society

(a thread on Mastodon U. Twente.)

In the essay The Fragmentation of Truth danah boyd makes the following important point: To combat increasing polarisation in our society, we need to rely on organisations that actively and intentionally let people with fundamental differences work alongside one another.

Boyd mentions the military as an example of an organisation that brings together people from different social backgrounds and political views to work on a common goal. To “intentionally bridge gaps in the social graph, to intentionally connect people and communities.”

I see schools and universities as another major power to combat polarisation in our society. Our university brings together people from different backgrounds, politcal views and cultures. Creating a sense of common purpose and a sense of a university community is important to fight polarisation and populism in our society.

That’s why our campus, our study associations, our sport, cultural and other student associations, are so important. That’s also why we need democratic institutions and self-government. They do not only shape our university now, they shape our future society.

We need to work harder to shape our universty as a community. If international students feel disconnected, then we completely failed as a university, no matter how excellent our educational programs are. This U-Today story, International bachelors: psychological and social problems, breaks my heart: (“One in three non-European bachelors had study problems in the previous academic year due to psychological, medical or social circumstances.”)

Danah boyd discusses in depth how platforms like Youtube and Facebook harm our society; how they directly threaten the important role that schools and universities play in creating a peaceful society. From this view point it is clear: Youtube should not be the primary channel for our online lectures; Facebook should not be the primary channel for our events.

Finally, services like search engines may be harmful, however well-intended and well-implemented. I find this hard to say as an Information Retrieval researcher, but search is easily manipulated, and you might not want powerful search in some applications. Boyd’s concept of ‘data voids’ is really insightful. Maybe we should teach students about search engine optimization in our courses too… #FIR

The Recent Applications of Machine Learning in Rail Track Maintenance: A Survey

by Muhammad Nakhaee, Djoerd Hiemstra, Mariëlle Stoelinga, and Martijn van Noort

Railway systems play a vital role in the world’s economy and movement of goods and people. Rail tracks are one of the most critical components needed for the uninterrupted operation of railway systems. However, environmental conditions or mechanical forces can accelerate the degradation process of rail tracks. Any fault in rail tracks can incur enormous costs or even result in disastrous incidents such as train derailment. Over the past few years, the research community has adopted the use of machine learning (ML) algorithms for diagnosis and prognosis of rail defects in order to help the railway industry to carry out timely responses to failures. In this paper, we review the existing literature on the state-of-the-art machine learning-based approaches used in different rail track maintenance tasks. As one of our main contributions, we also provide a taxonomy to classify the existing literature based on types of methods and types of data. Moreover, we present the shortcomings of current techniques and discuss what research community and rail industry can do to address these issues. Finally, we conclude with a list of recommended directions for future research in the field.

To be presented at the International Conference on Reliability, Safety and Security of Railway Systems: Modeling, Analysis, Verification and Certification (RSSRail 2019) on 4-6 June 2019 in Lille, France.

[download pdf]

Wim Florijn graduates on Semantically Grouping Search Query Data

Information Retrieval by Semantically Grouping Search Query Data

by Wim Florijn

Query data analysis is a time-consuming task. Currently, a method exists where word (combinations) in queries are labelled by using an information collection consisting of regular expressions. Because the information collection does not contain regular expressions from never-before seen domains, the method heavily relies on manual work, resulting in decreased scalibility. Therefore, a machine-learning based method is proposed in order to automate the annotation of word (combinations) in queries. This research searches for the optimal configuration of a pre-processing method, word embedding model, additional data set and classifier variant. All configurations have been examined on multiple data sets, and appropriate performance metrics have been calculated. The results show that the optimal configuration consists of omitting pre-processing, training a fastText model and enriching word features using additional data in combination with a recurrent classifier. We found that an approach using machine learning is able to obtain excellent performance on the task of labelling word (combinations) in search queries.

[download pdf]

Anna Priante defends PhD thesis on online social movement campaigns

by Anna Priante

Social movement organizations widely use social media to organize collective action for social change, such as cancer awareness campaigns. However, little is known about how effective online social movement campaigns are at generating social change by translating online action into meaningful (offline) action. This dissertation examines the micro-mobilization dynamics at play that can explain the effectiveness of online social social movement campaigns. This book comprises seven chapters presenting research based on a multidisciplinary, mixed-method approach, combining theories and methods from sociology, social psychology, communication science, and computational social science. The findings show that, with mobilization dynamics of collective action, we can gain an important understanding of the mechanisms at work during online social movement campaigns and of the effectiveness of such campaigns in fostering communication processes related to the cause, obtaining important resources for the cause, developing a collective identity, and raising awareness.

[download pdf]

Anna’s defense was the 5000th PhD defense at the UT!

Marieke Graef graduates cum laude on the Analysis of HPV discussions on Twitter

Responses to HPV Vaccination Campaigns in The Netherlands: an analysis of discussions on Twitter

by Marieke Graef

Even though the human papillomavirus vaccine (HPV) is an effective and safe instrument to decrease HPV infections and cases of several types of cancer, the Dutch HPV vaccination rate has been suboptimal from the start and has even shown a decline in the last two years. This study sought to assess the determinants of HPV vaccination uptake in the Netherlands and how the vaccine and RIVM and GGDTwente messages were discussed on Twitter from 2011 till 2016. Method: All Dutch language tweets mentioning HPV from the years 2011 till 2016 were collected from a database, amounting to a total of 17319. A content analysis of all tweets was carried out manually. The content of the GGDTwente and RIVM tweets was examined as well as responses to these tweets. Furthermore, the tweets were analyzed for specific determinants of HPV vaccination uptake and general sentiments. Results: The GGDTwente and RIVM only became truly active on Twitter regarding the HPV vaccination program in 2015. The RIVM tweets received significantly more response, though this response mostly consisted of retweets. Nearly all GGDTwente tweets concerned vaccination schedules. By far the most common determinant of low vaccination uptake in tweets from the public was the fear of side-effects, with scare stories going viral in 2015 and 2016 especially. On the other hand, publications on the high number of HPV infections among women received a lot of attention as well. Overall, the general sentiment towards the HPV vaccine on Twitter was more positive than negative in the first years, but due to stories about side-effects turned more negative in 2015. Conclusions: The results show that the fear of side-effects is something that needs to be addressed by public health authorities. Additionally, more practical measures such as a school-based vaccination program may be a great way to help increase the vaccination rate.

[download pdf]