The last decade has increasingly witnessed the use of robots in varied applications, ranging from industrial automation to space exploration. A key component in these robotics systems is their ability to extract semantic information regarding the objects they interact with when deployed into the real world. The field of object recognition has seen significant advances in accuracy and robustness using new artificial intelli- gence methods and machine learning algorithms. However, these methods partially address the cost and sample efficiency issue for acquiring the information required to learn such algorithms successfully. The use of active learning becomes essential to address these issues.
Prior active learning methods assume offline access to the entire dataset ahead of time. However, in the context of mobile robotics, where robots learn continuously by interacting with new objects and environments, the study of active learning in a streaming setting is more relevant. Motivated by these observations, we develop a data-driven framework that combines reinforcement learning with active learning for streaming scenarios. We propose our framework in two steps: first, given the abundance of labeled datasets for object recognition, we learn an active learning policy from these datasets using reinforcement learning, and in the second step, we apply this learned policy to select the most informative samples encountered in a stream of unlabeled data.
In this work, we model the elements of reinforcement learning in a stream-based active learning scenario. We then systematically investigate the performance of our proposed data-driven framework using diverse state representations for our reinforce- ment learning agent. We further provide insights into the advantages and challenges of each representation and also compare our method with relevant active learning baselines. Finally, our proposed methods significantly improve the selection of the most informative data over the considered streaming baseline, however, demonstrate on-par performance with the pool-based baselines.