|FERRARI DACREMA MAURIZIO||Cycle: XXXII |
Section: Computer Science and Engineering
Tutor: AMIGONI FRANCESCO
Advisor: CREMONESI PAOLO Major Research topic
:An assessment of reproducibility and methodological issues in neural recommender systems researchAbstract:
The design of algorithms that generate personalized ranked item lists is a central topic of research in the field of recommender systems.
In recent years, in particular, the interest of the research community has moved towards approaches based on deep learning (neural) techniques, which have become dominant in the literature.
Since each of those publications claims substantial progress over the state-of-the-art, it seems logical to expect the research field to be on a steady trajectory of increased effectiveness. However, several studies indicated the existence of certain problems in today's research practice, e.g., with respect to the choice and optimization of the baselines used for comparison or to the design of the experimental protocol itself, raising questions about the published claims.
In order to assess the current level of reproducibility and the existence of issues in the current recommender systems research practice, this thesis attempts to reproduce recent results in the area of neural recommendation approaches based on collaborative filtering. The analysis in particular focuses on articles published at high level scientific conferences between 2015 and 2018.
The results is that out of 24 articles, only 12 can be reproduced and only 1 shows to be consistently competitive against simple methods, e.g., based on the nearest-neighbor heuristics or linear machine-learning.
In our analysis, we discuss this surprising result and trace it back to several common issues in today's research practice, which, despite the many papers that are published on the topic, have apparently led the field to a certain level of stagnation.