MIME (Multiple Interactions Made Easy) is a diverse, large-scale dataset of 8260 human-robot demonstrations over 20 different robotic tasks. The dataset consists of videos of human demonstrations and kinesthetic trajectories of robot demonstrations.
The data was recieved from the sensors at a fixed constant rate for each sensor, so scaling the data of the joint angles, grippers and video data to the value of the LCM of the length of each of these individually should align the data in time. It can then be subsampled to a smaller number as needed.
Data was collected using a Baxter robot in gravity compensated kinesthetic mode. The robot is equipped with a Kinect mounted on the robot’s head, and two SoftKinetic DS325 cameras, each mounted on the robot’s wrist. The head camera acts like an external camera observing the task on the table, while the wrist cameras act as eye in robot cameras that move as the arm moves. During every robot demonstration, the robot joint angles and the gripper positions are synchronized and stored.
Every data point consists of a human demonstration and a corresponding robot demonstration. Demonstrations performed by the participants are visually reviewed and either accepted or rejected by other participants. If a demonstration is rejected, the trajectories are checked and redone by the demonstrator. For improved robustness and to avoid bias, every participant gives multiple demonstrations for the same task using a variety of objects. After completing demonstrations for a particular task, the participant is assigned a new task. This ensures that each task has multiple demonstrators collecting data for it. The data collection pipeline was iteratively improved based on participant feedback.
After this process, all of the collected data was reviewed by other participants and incorrect demonstrations were removed from the dataset. An interesting observation is that the erroneous trajectories were mostly collected by newer participants. After a few trials, the participants developed the ability to accurately manoeuvre the robot, resulting in less errors and faster trajectories.
@article{DBLP:journals/corr/abs-1810-07121,
author = {Pratyusha Sharma and Lekha Mohan and Lerrel Pinto and Abhinav Gupta},
title = {Multiple Interactions Made Easy {(MIME):} Large Scale Demonstrations Data for Imitation},
journal = {CORR},
volume = {abs/1810.07121},
year = {2018}
}
