Perception and Decision Making for Athletic Humanoid Robotics

Humanoid robots offer two unique advantages for general-purpose embodied intelligence. First, they are inherently generalist platforms, capable of performing a wide range of tasks in complex, human-centric environments. Second, the close embodiment alignment between humans and humanoids enables the transfer of skills from human data to robotic control.

However, unlike direct teleoperation, learning from off-embodiment human data introduces non-trivial cross-embodiment gaps, particularly at the level of dynamics and actuation. In this talk, we present a general recipe for learning humanoid skills from human data with physics grounding. Human data provides high-level intent and skill structure, while a physics-grounded layer translates them into motor-level actions. This approach enables versatile, adaptive, and robust skill learning across a range of loco-manipulation and dexterous manipulation tasks.

Artificial intelligence (AI) systems now challenge or surpass human experts in many computer games. Physical and real-time sports such as table tennis, however, remain a major open challenge because of their requirements for fast, precise and adversarial interactions near obstacles and at the edge of human reaction time. This talk presents Ace, to our knowledge the first real-world autonomous system competitive with elite human table tennis players. Ace addresses the challenges of physical real-time interaction through a new, high-speed perception system using event-based vision sensors, and a new control system based on model-free reinforcement learning, as well as state-of-the-art high-speed robot hardware. Evaluated in matches against elite and professional players under official competition rules, Ace achieved several victories and demonstrated consistent returns of high-speed, high-spin shots. These results highlight the potential of physical AI agents to perform complex, real-time interactive tasks, suggesting broader applications in domains requiring fast, precise human-robot interaction.

Learning skills on physical robots to solve tasks at human-level performance is a key enabling technology that can revolutionize everyone's lives. Much of today’s research focuses on developing learning algorithms, while designing the robotic body used for execution is often treated as a separate problem. Biological systems, and especially the human anatomy, show that generalization and high performance in difficult tasks are facilitated by the body itself. In this talk, I will illustrate what we have learned over the past years about how we can build robots to run learning algorithms more efficiently and safely.

Humanoid robots have achieved impressive capabilities in locomotion and manipulation, yet fast and precise interaction with moving objects remains a major challenge. In this talk, I will present a reinforcement-learning framework for humanoid badminton that learns a unified whole-body policy for footwork, stroke, and targeted shuttle placement. To improve both motion quality and task adaptability, multiple region-specific adversarial motion prior discriminators guide the robot toward different human-inspired striking styles across its reachable workspace. The desired return location is incorporated directly into policy training, allowing the controller to jointly determine how to move, where to strike, which stroke to execute, and where to return the shuttle, without relying on an upper-level planner.

I will also introduce an annealed curriculum that initially uses locomotion-related subtasks to reduce the exploration difficulty of whole-body learning. These auxiliary objectives are gradually annealed away as training progresses, reducing gradient conflicts between locomotion and striking. In addition, I will discuss a prediction-free variant that achieves competitive performance without explicit shuttlecock trajectory prediction, highlighting the potential for more reactive and streamlined dynamic interaction systems. The resulting policy is evaluated in simulation and on a real humanoid, demonstrating dynamic and coordinated whole-body badminton performance.