(WIP) Training a humanoid to kick a football using RL

Until I write a proper technical blog about it, here’s the LinkedIn/X post I shared about it:

I believe robotics and physical AI are two of the “next big things”, so I’ve spent some time brushing up on reinforcement learning and robotics. Watching the World Cup gave me the idea for the project - teaching a robot to play football! That was too ambitious for a side project, so I scoped it down to teaching the robot to walk and kick the ball.

I used PPO (Proximal Policy Optimization) and trained the behaviours in stages: first learning to stand still, then walking and turning, and finally kicking. The final system combines several learned policies with a controller that positions the robot and manages the handoffs between behaviours.

The video shows the training progress. It definitely couldn’t have helped Argentina win in the final 🤷

There are still plenty of limitations: the robot often falls after kicking the ball, it learned to walk by making micro-steps instead of a natural walking gait… However, seeing the complete sequence work was very satisfying.

A blog post and a repository with more technical details are coming soon™️.