Biography
I am a PostDoc at UC Berkeley working with Prof. Sergey Levine. I received my PhD in Computer Science (summa cum laude) in 2023 from the Freiburg University supervised by Prof. Dr. Wolfram Burgard. My research focuses on robot learning, to enable robots to intelligently interact with both the physical world and humans, and improve themselves over time. These days, I am particularly interested in how we can build self-improving embodied foundation models that can generalize the same way humans do. My research has been nominated for (and received) several Best Paper Awards, including ICRA and RA-L. Previously, I also spent time at NVIDIA AI interning with Dieter Fox.
Interests
- Robot Learning
- Robot Foundation Models
- Language Grounding
Education
Postdoc in Computer Science, 2024
UC Berkeley
PhD in Computer Science, 2023
University of Freiburg
Recent Work
News
- 31 Oct 2024: New preprint on GHIL-Glue: Hierarchical Control with Filtered Subgoal Images.
- 30 Oct 2024: I gave a talk about my research at Stanford University.
- 25 Oct 2024: New CoRL paper on Scaling Robot Policy Learning via Zero-Shot Labeling with Foundation Models.
- 17 Oct 2024: New CoRL paper on Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance.
- 15 Oct 2024: New preprint on The Ingredients for Robotic Diffusion Transformers.
- 14 Oct 2024: I gave an invited talk at the Workshop on Environment Dynamics Matters Embodied Navigation to Movable Objects at IROS.
- 07 Oct 2024: New CoRL paper on LeLaN: Learning A Language-conditioned Navigation Policy from In-the-Wild Video.
- 18 Sep 2024: Honored to see our recent work on cross-embodiment learning featured on IEEE Spectrum!
- 29 Aug 2024: New preprint on Policy Adaptation via Language Optimization: Decomposing Tasks for Few-Shot Imitation.
- 22 Aug 2024: New preprint on Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviation.
- 30 July 2024: New preprint on Autonomous Improvement of Instruction Following Skills via Foundation Models.
- 15 July 2024: I gave an invited talk at the Workshop on Semantics for Robotics: From Environment Understanding and Reasoning to Safe Interaction at RSS.
- 11 July 2024: New preprint on Robotic Control via Embodied Chain-of-Thought Reasoning.
- 20 May 2024: Honored to have been invited to serve as an Area Chair for CoRL 2024 in Munich.
- 16 May 2024: Our community paper Open-X Embodiment won the ICRA 2024 Best Conference Paper Award!.
- 13 May 2024: One paper accepted at RSS 2024 Octo: An Open-Source Generalist Robot Policy.
- 09 May 2024: New preprint on Evaluating Real-World Robot Manipulation Policies in Simulation.
- 16 Apr 2024: I was interviewed on Basque/Spanish radio about my research and the future of embodied AI.
- 20 Mar 2024: Honored to have been invited to serve as an Associate Editor for the 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2024).
- 13 Feb 2024: New preprint on Vision-Language Models Provide Promptable Representations for Reinforcement Learning.
- 7 Feb 2024: I have been shortlisted for the George Giralt PhD Award, the most prestigious recognition for a robotics dissertation in Europe!
- 29 Jan 2024: One paper accepted at ICRA 2024, Open-X Embodiment.
- 01 Jan 2024: Honored to have been invited to serve as an Associate Editor for IEEE Transactions on Robotics (T-RO).
- 15 Dec 2023: New preprint on Octo: An Open-Source Generalist Robot Policy.
- 21 Nov 2023: Honored to have been invited to serve as an Associate Editor for IEEE Robotics and Automation Letters.
- 18 Oct 2023: I was invited to give a talk about my research by ML @ Toyota Research Institute in Los Altos.
- 3 Oct 2023: Excited to have contributed to the Open-X Embodiment world collaboration paper to advance generalist AI models.
- 18 Sep 2023: I was invited to give a Google Tech Talk about my research by Robotics @ Google DeepMind in Mountain View.
Past News
- 11 Sep 2023: One paper accepted at ISER 2023, Audio Visual Language Maps for Robot Navigation
- 1 Sep 2023: I have started my PostDoc at UC Berkeley with Prof. Sergey Levine.
- 28 Jun 2023: The german newspaper Badische Zeitung wrote a profile about my work.
- 20 Jun 2023: I was invited to give talks about my work by TU Munich and Volkswagen’s Machine Learning Research Lab.
- 15 Jun 2023: Honored to have been invited to join the ELLIS - European Laboratory for Learning and Intelligent Systems Society, the pan-European AI Laboratory!
- 2 Jun 2023: We won the 2022 IEEE Robotics and Automation Letters Best Paper Award for CALVIN: A Benchmark for Language-conditioned Policy Learning for Long-horizon Robot Manipulation Tasks!
- 26 May 2023: I successfully defended my PhD thesis (summa cum laude)!
- 10 May 2023: Our paper on Grounding Language with Visual Affordances over Unstructured Data has been nominated for the Best Paper Award in Robot Learning at ICRA 2023!
- 26 Apr 2023: I have received the 2023 AI Newcomer award by the German Federal Ministry of Education and Research and the German Informatics Society in the Technical and Engineering Sciences category ! + 26 Mar 2023: Our work on connecting foundation models <-> maps (via language) has been featured on the Google AI Blog.
- 15 Mar 2023: New preprint for our latest collaboration with Google Research, Audio Visual Language Maps for Robot Navigation.
- 27 Jan 2023: I have been nominated for the 2023 AI Newcomer award by the German Federal Ministry of Education and Research and the German Informatics Society! If you have time, please support me by voting via http://bit.ly/3XWVaF8
- 17 Jan 2023: Two papers accepted for ICRA 2023 in London, Grounding Language with Visual Affordances over Unstructured Data and Visual Language Maps for Robot Navigation.
- 13 Oct 2022: New preprint for our latest collaboration with Google Research, Visual Language Maps for Robot Navigation.
- 6 Oct 2022: New preprint for Grounding Language with Visual Affordances over Unstructured Data.
- 1 Oct 2022: I have started an internship at NVIDIA with the robotics team led by Dieter Fox.
- 12 Sep 2022: Our work on task agnostic offline reinforcement learning has been accepted for presentation at CoRL 2022!
- 10 Sep 2022: I am excited to co-organize the Workshop on Language and Robot Learning at CoRL. Check the website for our fantastic line of speakers! The submission deadline will be in October 2022!
- 16 Jul 2022: Two new papers accepted for RA-L with IROS 2022 presentation in Kyoto, Japan. What Matters in Language Conditioned Imitation Learning over Unstructured Data and CALVIN: A Benchmark for Language-conditioned Policy Learning for Long-horizon Robot Manipulation Tasks.
- 16 May 2022: I have been selected as part of the 2022 cohort for RSS Pioneers, which will be held in conjunction with Robotics: Science and Systems 2022. + 21 Apr 2022: I am excited to co-organize the 2nd Workshop on Scaling Robot Learning at RSS together with Andy Zeng, Jeannette Bohg, Wolfam Burgard, Dieter Fox, Serena Ivaldi, Danica Kragic and Stephanie Tellex.
- 14 Apr 2022: New preprint for What Matters in Language Conditioned Imitation Learning.
- 08 Feb 2022: I am co-organizing an ICRA workshop on Scaling Robot Learning together with Andy Zeng, Raia Hadsell, Roberto Calandra, Danica Kragic, Serena Ivaldi and Wolfram Burgard.
- 01 Feb 2022: Our paper on visual affordances for efficient policy learning has been accepted for presentation at ICRA 2022!
- 7 Dec 2021: New preprint for CALVIN: A Benchmark for Language-conditioned Policy Learning for Long-horizon Robot Manipulation Tasks.
- 30 Nov 2021: Our work on leveraging visual affordances for efficient policy learning has been accepted for presentation at the NeurIPS 2021 workshop on Robot Learning.
- 15 Mar 2021: I am happy to co-organize an ICRA workshop on Semantic Representations for Robotics through Continuous Interaction and Incremental Learning together with Tamim Asfour and Wolfram Burgard.
- 17 Feb 2021: New preprint for our upcoming ISER paper on Composing Pick-And-Place Tasks By Grounding Language.
- 04 Aug 2020: New preprint for our upcoming IROS paper on Hindsight for Foresight: Unsupervised Structured Dynamics Models from Physical Interaction, check out the video.
- 13 May 2020: Our paper on unsupervised skill learning from video has been nominated for the Best Paper Award in Cognitive Robotics at ICRA 2020!
- 09 Mar 2020: I am co-organizing an RSS workshop on Self-supervised Robot Learning together with Wolfram Burgard, Anelia Angelova, Joschka Boedecker and Abhinav Valada. The submission deadline will be in April 2020!
- 12 Feb 2020: Our work on Learning Object Placements For Relational Instructions by Hallucinating Scene Representations has been featured on TechXplore.
- 01 Feb 2020: Two new papers accepted for ICRA 2020. Adversarial Skill Networks: Unsupervised Robot Skill Learning from Video and Learning Object Placements For Relational Instructionsby Hallucinating Scene Representations.
- 12 Sep 2019: New preprint for our upcoming IROS paper on Self-supervised 3D Shape and Viewpoint Estimation from Single Images for Robotics, check out the video.
- 15 Aug 2019: I was interviewed on Spanish TV.
Experience
PostDoc
Advisor: Prof. Sergey Levine.
Robotics Research Intern
Advisor: Prof. Dieter Fox.
Ph.D Candidate in Computer Science
Doctoral advisor: Prof. Wolfram Burgard.
Student Research Assistant
Student Research Intern
Student Research Intern
Student Research Intern