Apprentissage d'agents multi-tâches sous une supervision minimale

Learning goal-oriented agents with limited supervision

• Auteur / Autrice : Lina Mezghani
• Direction : Karteek Alahari, Piotr Bojanowski
• Type : Thèse de doctorat
• Discipline(s) : Mathématiques et informatique
• Date : Soutenance le 03/07/2023
• Etablissement(s) : Université Grenoble Alpes
• Ecole(s) doctorale(s) : École doctorale Mathématiques, sciences et technologies de l'information, informatique (Grenoble ; 1995-....)
• Partenaire(s) de recherche : Laboratoire : Laboratoire Jean Kuntzmann (Grenoble)
• Partenaire(s) de recherche : CIFRE : Meta AI (Paris)
• Jury : Président / Présidente : Abhinav Gupta
• Jury : Examinateurs / Examinatrices : Pierre-Yves Oudeyer, Anne-Marie Spalanzani
• Jury : Rapporteurs / Rapporteuses : Abhinav Gupta, Olivier Sigaud

Mots clés

Mots clés contrôlés

Apprentissage par renforcement (intelligence artificielle)

Odométrie visuelle

Mots clés libres

Apprentissage par Renforcement

Apprentissage Auto-Supervise

Intelligence Artificielle

Vision par Ordinateur

Reinforcement Learning

Self-Supervised Learning

Artificial Intelligence

Computer Vision

Résumé

Non fourni par l'auteur

The development of intelligent agents has seen significant progress in the lastdecade, showing impressive capabilities in various tasks, such as video games orrobot navigation. These advances were made possible by the advent of deep reinforcementlearning (RL), which allows to train neural network-based policies,through interaction of the agent with its environment. However, in practice, theimplementation of such agents requires significant human intervention and priorknowledge on the task at hand, which can be seen as forms of supervision. In thisthesis, we tackle three different aspects of supervision in RL, and propose methodsto reduce the amount of human intervention required to train agents.We first investigate the impact of supervision on the choice of observations seen bythe agent. In robot navigation for example, the modalities of the environment observedby the agent are an important design choice that can have a significant impacton the difficulty of the task. To tackle this question, we focus on image-goal navigationin photo-realistic environments, and propose a method for learning to navigatefrom raw visual inputs, i.e., without relying on depth or position information.Second, we target the problem of reward supervision in RL. Standard RL algorithmsrely on the availability of a well-shaped reward function to solve a specific task.However, the design of such functions is often a difficult and time-consuming process,which requires prior knowledge on the task and environment. This limits thescalability and generalization capabilities of the designed approaches. To addressthis issue, we tackle the problem of learning state-reaching policies without rewardsupervision, and design methods that leverage intrinsic reward functions to learnsuch policies.Finally, we study the problem of learning agents offline, from pre-collected demonstrations,and question the availability of such data. Collecting expert trajectories isoften a difficult and time-consuming process, which can be more difficult than thedownstream task itself. Offline algorithms should therefore rely on existing data,and we propose a method for learning goal-conditioned agents from tutorial videos,which contains expert demonstrations aligned with natural language captions.