Siddhant Haldar

I am currently working on a new company in AI/robotics. Before this, I received my PhD from the CILVR Lab at NYU Courant, where I was advised by Prof. Lerrel Pinto. My research during the Ph.D. focused on enabling data-efficient generalization for robot manipulation, spanning efficient real-world RL, architectures for offline learning, and observational priors for sample-efficient generalization. During my Ph.D., I did research internships at the NVIDIA Seattle Robotics Lab and the Nuro Behavior Team.

Before my Ph.D., I completed an Integrated Bachelor's and Masters Degree in Electrical Engineeing at Indian Institute of Technology (IIT), Kharagpur. My previous internships include working on multimodal policy learning at Nuro, causal inference at Microsoft Research and IBM Research, adversarial attacks with Prof. Atul Prakash at MIDAS, University of Michigan, and multimodal object detection with Prof. Zheng Liu at ISDPRL Lab, University of British Columbia. My Master's thesis was on the topic of action quality assessment (AQA) with Prof. Brendan Morris at RTIS Lab, University of Nevada, Las Vegas. I did my Bachelor's thesis on the topic of attention-guided video inpainting with Dr. Avisek Lahiri and Prof. Pabitra Mitra at IIT Kharagpur. I also worked on developing the perception and control system modules for self-driving cars at the Autonomous Ground Vehicle Lab, IIT Kharagpur under the supervision of Prof. Debashis Chakravarty. In my spare time, I enjoy hiking, reading and traveling.

Email / CV / Google Scholar / Twitter / Github

Research

* denotes equal contribution

	Point Bridge: 3D Representations for Cross Domain Policy Learning Siddhant Haldar, Lars Johannsmeier, Lerrel Pinto, Abhishek Gupta, Dieter Fox, Yashraj Narang, Ajay Mandlekar, ICRA 2026 Workshop on Synthetic Data for Robot Learning project page / arXiv / code
	Touch begins where vision ends: Generalizable policies for contact-rich manipulation Zifan Zhao, Siddhant Haldar, Jinda Cui, Lerrel Pinto, Raunaq Bhirangi RSS 2025 Workshop on Out-of-Distribution Generalization in Robotics RSS 2025 Workshop on Dexterous Manipulation project page / arXiv / code
	EgoZero: Robot Learning from Smart Glasses Vincent Liu, Ademi Adeniji, Haotian Zhan, Siddhant Haldar, Raunaq Bhirangi, Pieter Abbeel, Lerrel Pinto RSS 2025 Workshop on Learned Robot Representations* project page / arXiv / code
	Feel The Force: Contact-Driven Learning from Humans Ademi Adeniji, Zhuoran Chen, Vincent Liu, Venkatesh Pattabiraman, Siddhant Haldar, Raunaq Bhirangi, Pieter Abbeel, Lerrel Pinto RSS 2025 Workshop on Learned Robot Representations project page / arXiv / code
	Point Policy: Unifying Observations and Actions with Key Points for Robot Manipulation Siddhant Haldar, Lerrel Pinto, CoRL 2025 RSS 2025 Workshop on Out-of-Distribution Generalization in Robotics RSS 2025 Workshop on Reasoning for Robust Robot Manipulation in the Open World project page / arXiv / code / data
	Learning Precise, Contact-Rich Manipulation through Uncalibrated Tactile Skins Venkatesh Pattabiraman, Yifeng Cao, Siddhant Haldar, Lerrel Pinto, Raunaq Bhirangi ICRA 2025 ViTac Workshop 2025 (Best Poster Award) project page / arXiv / code
	P3-PO: Prescriptive Point Priors for Visuo-Spatial Generalization of Robot Policies Mara Levy, Siddhant Haldar, Lerrel Pinto, Abhinav Shrivastava ICRA 2025 project page / arXiv / code
	BAKU: An Efficient Transformer for Multi-Task Policy Learning Siddhant Haldar, Zhuoran Peng, Lerrel Pinto NeurIPS 2024 project page / arXiv / code
	DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Control Zichen Jeff Cui, Hengkai Pan, Aadhithya Iyer, Siddhant Haldar, Lerrel Pinto NeurIPS 2024 project page / arXiv / code
	OPEN TEACH: A Versatile Teleoperation System for Robotic Manipulation Aadhithya Iyer, Zhuoran Peng, Yinlong Dai, Irmak Guzey, Siddhant Haldar, Soumith Chintala, Lerrel Pinto CoRL 2024 project page / arXiv / code
	Open X-Embodiment: Robotic Learning Datasets and RT-X Models Open X-Embodiment Collaboration, ICRA 2024 (Best Paper Award) project page / arXiv / code
	PolyTask: Learning Unified Policies through Behavior Distillation Siddhant Haldar, Lerrel Pinto arXiv project page / arXiv / code
	Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrations Siddhant Haldar, Jyothish Pari, Anant Rai, Lerrel Pinto RSS 2023 (Best Student Paper Award) project page / arXiv / paper / code
	Watch and Match: Supercharging Imitation with Regularized Optimal Transport Siddhant Haldar, Vaibhav Mathur, Denis Yarats, Lerrel Pinto CoRL 2022 (Oral Presentation, Best Paper Award Finalist) project page / arXiv / paper / code
	Network Embedding of Distributional Thesaurus Combined with Word Vectors Leads to Better Representation Abhik Jana, Siddhant Haldar, Pawan Goyal ESWA 2022 paper
	Lightweight Modules for Efficient Deep Learning based Image Restoration Avisek Lahiri, Sourav Bairagya, Sutanu Bera, Siddhant Haldar, Prabir Kumar Biswas TCSVT 2020 arXiv / code
	Design and Implementation of Autonomous Ground Vehicle for Constrained Environments Siddhant Haldar et al IRC 2019 paper
	Ground Vehicle Odometry using a Non-Intrusive Inertial Speed Sensor Het Shah, Siddhant Haldar, Rohit Ner, Siddharth Jha, Debashish Chakravarty ICIT 2019 arXiv
	Off-Road Lane Detection Using Superpixel Clustering And RANSAC Curve Fitting Sanskar Agrawal, Indu Kant Deo, Siddhant Haldar, G. Rahul Krantikiran, Debashish Chakravarty ICARCV 2018 paper