Discovering and solving subtasks individually is a common approach to tackle long-horizon tasks. However, the existing approaches for subtask discovery primarily aim to accelerate downstream task training and often fail to identify semantically meaningful subtasks. We aim to discover semantically meaningful subtasks, in a robotic dataset where the demonstrations complete subtasks in varying orders, to enable sequential execution of subtask policies. We first segment the demonstrations into subtask working and transition phases in a semantically meaningful way. We then employ a self-supervised approach that is agnostic to the number of labels to classify the trajectory segments to obtain distinct subtasks. We trained a set of independent policies solely on the corresponding subtask demonstration segments. The policy learns to decide when to switch to the next subtask policy. Our experiment showed that executing these subtask policies sequentially achieved 98% of the total rewards of a baseline policy trained on complete demonstrations. This result demonstrates that our method identifies semantically meaningful subtasks with accurate boundaries, ensuring the seamless execution of multiple independent subtask policies.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Trajectory Segmentation and Self-Supervised Classification for Subtask Discovery

  • Runze Tang,
  • Penny Sweetser

摘要

Discovering and solving subtasks individually is a common approach to tackle long-horizon tasks. However, the existing approaches for subtask discovery primarily aim to accelerate downstream task training and often fail to identify semantically meaningful subtasks. We aim to discover semantically meaningful subtasks, in a robotic dataset where the demonstrations complete subtasks in varying orders, to enable sequential execution of subtask policies. We first segment the demonstrations into subtask working and transition phases in a semantically meaningful way. We then employ a self-supervised approach that is agnostic to the number of labels to classify the trajectory segments to obtain distinct subtasks. We trained a set of independent policies solely on the corresponding subtask demonstration segments. The policy learns to decide when to switch to the next subtask policy. Our experiment showed that executing these subtask policies sequentially achieved 98% of the total rewards of a baseline policy trained on complete demonstrations. This result demonstrates that our method identifies semantically meaningful subtasks with accurate boundaries, ensuring the seamless execution of multiple independent subtask policies.