Photo of Boyuan Chen

About Me

I'm Boyuan Chen (陈博远), an AI researcher and roboticist at MIT. I am currently a third year PhD student working with Prof. Vincent Sitzmann and Prof. Russ Tedrake. I am interested in foundation models for decision making. I work on building world models that allow AI agents to aquire skills via both search (like AlphaGo) and policy (like GPT). I am also interested in deploying these models on real-world robots, aka embodied intelligence. Outside research, building robots is my long-time hobby. Scroll down to see some of my robots!

Previosuly, I interned at Google Deepmind and Google X. I obtained my bachelor's degree in computer science and math at UC Berkeley, where I spent a signficant amount of time doing research at Berkeley Artificial Intelligence Research (BAIR) on deep reinforcement learning and unsupervised learning. I also spent a year studying philosophy during my undergrad. I am a big fan of chess, robots and boba.

My research

Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion
Boyuan Chen, Diego Marti Monso, Yilun Du, Max Simchowitz, Russ Tedrake, Vincent Sitzmann
NeurIPS 2024 (Conference of Neural Information Processing Systems)

website | paper | abstract | bibtex
@misc{chen2024diffusionforcingnexttokenprediction,
    title={Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion}, 
    author={Boyuan Chen and Diego Marti Monso and Yilun Du and Max Simchowitz and Russ Tedrake and Vincent Sitzmann},
    year={2024},
    eprint={2407.01392},
    archivePrefix={arXiv},
    primaryClass={cs.LG},
    url={https://arxiv.org/abs/2407.01392}, 
}
              

DittoGym: Learning to Control Soft Shape-Shifting Robots
Suning Huang, Boyuan Chen, Huazhe Xu, Vincent Sitzmann
ICLR 2024 (International Conference on Learning Representations)

website | paper | abstract | bibtex
@misc{huang2024dittogym,
  title={DittoGym: Learning to Control Soft Shape-Shifting Robots}, 
  author={Suning Huang and Boyuan Chen and Huazhe Xu and Vincent Sitzmann},
  year={2024},
  eprint={2401.13231},
  archivePrefix={arXiv},
  primaryClass={cs.RO}
}
              

SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities
Boyuan Chen, Zhuo Xu, Sean Kirmani, Brian Ichter, Danny Driess, Pete Florence, Dorsa Sadigh, Leonidas Guibas, Fei Xia
CVPR 2024 (Conference on Computer Vision and Pattern Recognition)

website | paper | abstract | bibtex
@InProceedings{Chen_2024_CVPR,
    author    = {Chen, Boyuan and Xu, Zhuo and Kirmani, Sean and Ichter, Brain and Sadigh, Dorsa and Guibas, Leonidas and Xia, Fei},
    title     = {SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2024},
    pages     = {14455-14465}
}
              

Self-Supervised Reinforcement Learning that Transfers using Random Features
Boyuan Chen, Chuning Zhu, Pulkit Agrawal, Kaiqing Zhang, Abhishek Gupta
NeurIPS 2023 (Conference of Neural Information Processing Systems)

website | paper | abstract | bibtex
@misc{chen2023selfsupervised,
  title={Self-Supervised Reinforcement Learning that Transfers using Random Features}, 
  author={Boyuan Chen and Chuning Zhu and Pulkit Agrawal and Kaiqing Zhang and Abhishek Gupta},
  year={2023},
  eprint={2305.17250},
  archivePrefix={arXiv},
  primaryClass={cs.LG}
}
              

Extraneousness-Aware Imitation Learning
Ray Chen Zheng, Kaizhe Hu, Zhecheng Yuan, Boyuan Chen, Huazhe Xu
ICRA 2023 (International Conference on Robotics and Automation)

website | paper | abstract | bibtex | talk video
@misc{https://doi.org/10.48550/arxiv.2210.01379,
  doi = {10.48550/ARXIV.2210.01379},
  url = {https://arxiv.org/abs/2210.01379},
  author = {Zheng, Ray Chen and Hu, Kaizhe and Yuan, Zhecheng and Chen, Boyuan and Xu, Huazhe},
  keywords = {Robotics (cs.RO), Artificial Intelligence (cs.AI), FOS: Computer and information sciences, FOS: Computer and information sciences},
  title = {Extraneousness-Aware Imitation Learning},
  publisher = {arXiv},
  year = {2022},
  copyright = {arXiv.org perpetual, non-exclusive license}
}
              

Open-vocabulary Queryable Scene Representations for Real World Planning
Boyuan Chen, Fei Xia, Brian Ichter, Kanishka Rao, Keerthana Gopalakrishnan, Michael S. Ryoo, Austin Stone, Daniel Kappler
ICRA 2023 (International Conference on Robotics and Automation)

website | paper | abstract | bibtex | talk video
@inproceedings{chen2022nlmapsaycan,
    title={Open-vocabulary Queryable Scene Representations for Real World Planning},
    author={Boyuan Chen and Fei Xia and Brian Ichter and Kanishka Rao and Keerthana Gopalakrishnan and Michael S. Ryoo and Austin Stone and Daniel Kappler}
    booktitle={arXiv preprint arXiv:2209.09874},
    year={2022}
}
              

Unsupervised Learning of Visual 3D Keypoints for Control
Boyuan Chen, Pieter Abbeel, Deepak Pathak
ICML 2021 (International Conference on Machine Learning)

website | paper | abstract | bibtex | code | talk video
@article{chen2021keypoint3D,
    author = {Chen, Boyuan and Abbeel, Pieter and Pathak, Deepak},
    title  = {Unsupervised Learning of Visual 3D Keypoints for Control},
    journal= {ICML},
    year   = {2021}
}
              

Zero-shot Policy Learning with Spatial Temporal Reward Decomposition on Contingency-aware Observation
Boyuan Chen*, Huazhe Xu*, Yang Gao and Trevor Darrell
ICRA 2021 (International Conference on Robotics and Automation)

website | paper | abstract | bibtex | code |
@article{DBLP:journals/corr/abs-1910-08143,
    author    = {Huazhe Xu and
                    Boyuan Chen and
                    Yang Gao and
                    Trevor Darrell},
    title     = {Scoring-Aggregating-Planning: Learning task-agnostic priors from interactions
                    and sparse rewards for zero-shot generalization},
    journal   = {CoRR},
    volume    = {abs/1910.08143},
    year      = {2019},
    url       = {http://arxiv.org/abs/1910.08143},
    eprinttype = {arXiv},
    eprint    = {1910.08143},
    timestamp = {Fri, 27 Nov 2020 15:04:16 +0100},
    biburl    = {https://dblp.org/rec/journals/corr/abs-1910-08143.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
    }
                            

Discovering Diverse Multi-agent Strategic Behavior via Reward Randomization
Zhenggang Tang, Chao Yu, Boyuan Chen, Huazhe Xu, Xiaolong Wang, Fei Fang, Simon Shaolei Du, Yu Wang, Yi Wu
ICLR 2021 (International Conference on Learning Representations)

website | paper | abstract | bibtex | code |
@misc{tang2021discovering,
    title={Discovering Diverse Multi-Agent Strategic Behavior via Reward Randomization}, 
    author={Zhenggang Tang and Chao Yu and Boyuan Chen and Huazhe Xu and Xiaolong Wang and Fei Fang and Simon Du and Yu Wang and Yi Wu},
    year={2021},
    eprint={2103.04564},
    archivePrefix={arXiv},
    primaryClass={cs.AI}
}
                            

MISC

  • Robots
  • Cooking
  • Teams
Robomooc Robotics Kit

I designed it with my friend, Kinsky. We sold it as an education kit to schools. You can ride on it!

Robomaster ICRA challenge

DJI robomaster robot for ICRA AI Challenge. During my undergrad, I was the captain of the team, leading the development of autonomous algorithms in the robot shooting challenge.

Autonomous Bogie Rover

My personal robot that can handle a variety of terrains. I did everything from machanical design, electronics to programming. It uses computer vision to autonomously follow me and avoid obstables.

FRC 2017 Robot

In 2017, I founded my high school's first FRC team. We didn't have the mentorship nor funding we need, but the team did amazing. I did the majority of the design.

PR2 in RLL

In 2021, I graduated from UC Berkeley, where I spent some amazing time doing research in robotics learning lab.

Autonomous Drone

An autonomous drone which I built and coded. I installed a camera a mini railgun on it to track and aim at the target I select.

FTC 2017 Robot

Our FTC competition robot in 2017, when I became the captain of the team. It's my team's first robot designed with CAD. The robot won the east China regional.

My first ftc robot

In 2016, I participanted in robotics competition for the first time. This is a super cool robot which marks the beginning of my robotics journey.

FRC 2018 Robot

After my graduation from high school, I continued mentoring the team. My successor Xinpei designed the robot under my mentorship.

Blog