Active Embodiment Identification: Teaching Robots to "Know Thyself" via RL

TL;DR

Researchers have developed a method that allows legged robots—ranging from quadrupeds to humanoids—to identify their own physical characteristics (like mass, link lengths, and inertia) through active movement. By rewarding a Reinforcement Learning agent for reducing prediction error, the robot learns specific "information-seeking" behaviors that reveal its unique embodiment parameters, even when those parameters are hidden or changed due to wear and tear.

Background: The Embodiment Gap

In the race to create general-purpose robots, we often rely on Domain Randomization. We train a single policy on thousands of simulated robots with slightly different masses and leg lengths so that the resulting controller is "robust." However, robustness is often a double-edged sword: it leads to conservative behavior. If a robot doesn't know its exact maximum torque or its precise center of mass, it won't push itself to its physical limits.

The core challenge is that real-world robots are rarely identical to their CAD models. Manufacturing tolerances, sensors, and aging make every robot unique. This paper asks: What if the robot could learn to identify itself through action?

Problem & Motivation

Current methods for system identification (SysId) are often:

1. Passive: They observe normal movement and try to guess parameters, which often fails because standard walking doesn't "excite" the dynamics enough to reveal things like rotor inertia.
2. Model-Based: They rely on rigid mathematical models that might not account for complex non-linearities.

The authors propose Active Embodiment Identification, framing self-discovery as an active perception problem. The robot isn't just trying to walk; it's trying to move in a way that makes its physical properties obvious to an internal observer.

Methodology: URMA and Active Perception

The architecture is built upon the Unified Robot Morphology Architecture (URMA), which allows a single neural network to handle robots with different numbers of joints.

The Three-Tier Architecture

• Identification Network: A combination of a joint encoder and a Gated Recurrent Unit (GRU) that looks at the history of movement. It has two prediction "heads": one for joint-level data (stiffness, damping) and one for global data (total mass, trunk dimensions).
• The Policy & Critic: These generate actions based on the current state and the robot's nominal (assumed) description.

The Identification Reward

Instead of rewarding the robot for moving forward, the reward function is purely mathematical: $r_{id}=\frac{1}{2}\left[\exp \left(-\frac{{\mathcal{L}}_{joint}}{au}\right)+\exp \left(-\frac{{\mathcal{L}}_{general}}{au}\right)\right]$ This forces the policy to find movements that minimize the Mean Squared Error (MSE) of its own self-predictions.

Fig 1: The framework was tested on diverse morphologies including Unitree humanoids and ANYmal quadrupeds.

Experimental Results: High-Precision Self-Discovery

The researchers trained the system for 2 billion steps across 4,096 parallel environments using the MJX physics engine.

Key Findings:

• Mass Accuracy: For a robot like the ANYmal C (approx. 50kg), the system could predict mass with an error of just 0.25 kg.
• Geometric Precision: Trunk dimensions and Center of Mass (COM) positions were identified with millimeter-level accuracy.
• The Difficulty of Limits: Parameters like "maximum torque" were harder to identify. This makes sense: to know your max torque, you have to push the motor to its limit, which might cause a fall. The trained policy preferred "stable standing" shakes and shifts, which revealed mass but not peak power.

Fig 2: Prediction errors across different parameters. Global parameters (bottom) show much higher relative accuracy than refined joint-level parameters (top).

Critical Analysis & Future Outlook

Takeaway

This research proves that "information-seeking" is a viable objective for robot learning. Instead of manually engineering "calibration routines," we can let RL discover the most efficient way to calibrate a robot's internal model.

Limitations

A notable limitation is the Exploration-Stability Trade-off. Because the robot is afraid of falling (which terminates the RL episode), it doesn't perform the high-energy maneuvers needed to identify limits like maximum velocity or stiction.

Future Work

The logical next step is Closed-loop Adaptation. Currently, the robot identifies itself, but it doesn't yet "use" that information to change its walking style in real-time. Integrating the identification output directly into a control policy could allow a robot to instantly adapt to a heavy payload or a damaged motor—realizing the true potential of "Embodiment-Aware" AI.