Section: Computer Science and Engineering
Tutor: AMIGONI FRANCESCO Abstract:
Physical Interactive Robogames (PIRG) are one of the most challenging task in robotics, as they involved complex interaction with both environment and people. They represent a good benchmark for home robotics applications, as they have limited and well defined domains (the game rules), but they allow to highlight most of the common problems that robots can have in domestic environments while they interact with people.
To obtain engaging interaction between the player and a robot, the robot must have the ability to adapt to multiple possible scenarios it can found, such as different game configurations and different kind of players. Furthermore a pure reactive agent is not well suited for the task, as the robot tends to act in a naive way, that can be easily exploited by the player, making the game too easy and boring the player.
Particularly for complex games, designing and implementing a complex robot policy, not a purely reactive one, is a hard task. The task becomes harder if the robot has to face novel situations and different types of players. The design of complex games that show advanced interaction between the robot and the players is strongly limited by the ability of the robot of act in a credible and reliable way.
Reinforcement leaning is useful to overcome these limitations and allows an easier adaptation of the robot to new environments. In particular, recent advances in policy search algorithms and inverse reinforcement learning have made possible for robots to learn complex tasks that are very hard to model and solve by classical approaches. However, policy search approaches may fail when the policy is too complex and general, when the number of robot behavior samples needed to learn the policy becomes intractable, and when the reward signal is sparse. Hierarchical reinforcement learning approaches decompose the task into subgoals, that are easier to learn. In this research we will focus on extending the Hierarchical reinforcement learning approaches in order to allow the learning of complex policies. The key idea behind this approach is to give to the robot a set of tasks of increasing complexity in order to make the robot incrementally learn skills useful to solve more complex tasks. We will research techniques to automatically discover subgoals and transfer subgoals policies to other task and different environments. Inverse reinforcement learning techniques will be used for learning the structural properties of a subgoal.
The ultimate goal is to have a robot that, given a sufficient set of skills, is able to engage effectively any kind of player, by learning just using its own experience and expert demonstrations in a simplified version of the game.
Advisor: Andrea Bonarini, Marcello Restelli
Semantic Monocular SLAM
Semantic Monocular SLAM
Recent research on the Simultaneous Localization and Mapping (SLAM) has focused on semantic visual SLAM. Semantic SLAM, differently from the classical visual approaches, that extract image features (such as points) to localize the robot in the environment, works a at a higher level and uses objects as map landmarks.
Using object gives some advantages: objects can be discriminated by their category and their position with respect to other objects, leading to strong data association and simple loop closure mechanism. Also a object-based map is useful for reasoning and planning.
Most of the literature of semantic visual SLAM is based on depth cameras, such as RGBD and stereo cameras. Our approach is based on monocular camera and an inertial measurement unit (IMU). The IMU is used to disambiguate the scale, as the scale cannot be estimated from monocular observation, and scale is very important to discriminate objects, e.g. a real door from a doll-house door.
We describe the objects in terms of structural points that are the points that describe the structure of the objects, e.g. the four corners of a rectangle-shaped object.
The SLAM algorithm is based on bundle adjustment techniques. We define a novel parametrization to describe objects in the estimation process as collection of structural points with structural geometrical constraints, and use objects directly to localize the robot.
Object detection is performed by a Fuzzy Tree Classifier, that exploits the relationships among extracted image features to discriminate objects. Object tracking and recognition is performed by long term tracking algorithms.
Most of the work of this research is focused on improving the visual object detection, tracking, and structural point extraction frontend, while for the pose and map estimation the research is focused on the parametrization used to represent objects in the map.
Advisor: Andrea Bonarini