MimicDroid: In-Context Learning for Humanoid Robot Manipulation from Human Play Videos

Home page · Paper

This project builds on RoboCasa, a large-scale simulation framework for training generally capable robots to perform everyday tasks. Please cite RoboCasa if you use this codebase.

---

We introduce a benchmark built on RoboCasa, spanning 30 objects, 8 kitchen environments, and 8 hours of human play data for training. All the training environments are shown below:

Evaluation is structured into **three levels** with increasing difficulty and **4 tasks** in each level.

Level	Task Name	Abstract Embodiment	Humanoid Embodiment
L1 (Seen Objects, Seen Environment)	PnPSinkToRightCounterPlate
	PnPSinkToCabinet
	TurnOnFaucet
	CloseLeftCabinetDoor
L2 (Unseen Objects, Seen Environment)	PnPSinkToRightCounterPlateL2
	PnPSinkToCabinetL2
	CloseRightCabinetDoorL2
	CloseLeftCabinetDoorL2
L3 (Unseen Objects, Unseen Environment)	CloseLeftCabinetDoorL3
	PnPSinkToRightCounterPlateL3
	PnPSinkToMicrowaveTopL3
	TurnOnFaucetL3

---

Installation

MimicDroid builds on RoboCasa and works across major platforms. The easiest way to set up is via Anaconda. Follow the steps below.

• Set up conda environment

conda create -c conda-forge -n mimicdroid python=3.10
conda activate mimicdroid

• Clone and set up robosuite (use the abs_robot branc ShahRutav/robosuite)

git clone --branch=abs_robot https://github.com/ShahRutav/robosuite
cd robosuite
pip install -e .
cd ..

• Clone and set up RoboCasa

git clone --branch=latest https://github.com/UT-Austin-RPL/mimicdroid-robocasa
cd robocasa
pip install -e .
pip install pre-commit; pre-commit install    # Optional: code formatter

• (Optional) If you run into numba/numpy issues

conda install -c numba numba=0.56.4 -y

• Install the package and download assets

python robocasa/scripts/download_kitchen_assets.py   # ~5GB download
python robocasa/scripts/setup_macros.py              # Set up system variables

---

Dataset

Please see DATASET.md for dataset installation and visualization instructions.

Getting Started

Please see GETTING_STARTED.md for instructions.

Citation

@article{shah2025mimicdroid,
  title={MimicDroid: In-Context Learning for Humanoid Manipulation from Human Play Videos},
  author={Shah, Rutav and Liu, Shuijing and Wang, Qi and Jiang, Zhenyu and Kumar, Sateesh and Seo, Mingyo and Mart{\'\i}n-Mart{\'\i}n, Roberto and Zhu, Yuke},
  journal={arXiv preprint arXiv:2509.09769},
  year={2025}
}

@inproceedings{robocasa2024,
  title={RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots},
  author={Soroush Nasiriany and Abhiram Maddukuri and Lance Zhang and Adeet Parikh and Aaron Lo and Abhishek Joshi and Ajay Mandlekar and Yuke Zhu},
  booktitle={Robotics: Science and Systems},
  year={2024}
}