[CoRL 2025] LESSMIMIC: Breaking the Geometry Barrier in Humanoid Interaction

Executive Summary

TL;DR: LESSMIMIC is a breakthrough in humanoid embodied intelligence that moves away from rigid "motion-tracking" scripts. By conditioning a single whole-body policy on Distance Fields (DF), the robot "feels" the geometry of objects. This allows a single policy to pick up boxes of any size, sit on various chairs, and chain up to 40 different tasks together—all while being able to recover from human interference in real-time.

Background Positioning: This work addresses the "generalization gap" in humanoid robotics. Most SOTA models are either too rigid (Reference-based) or too specialized (Task-specific). LESSMIMIC sits at the intersection of Geometry-Aware Perception and Whole-Body Reinforcement Learning, providing a scalable blueprint for "Generalist" robots.

1. The Problem: The "Geometry Specialist" Trap

Why can't current humanoids just "pick up a box"?

1. The Reference-Based Failure: Current models like DeepMimic or ResMimic track human motion data. If the box in the real world is 10cm wider than the one in the training data, the hand trajectories miss the target or cause a collision. The robot is tracking a path, not an interaction.
2. The Reward Engineering Nightmare: Reference-free methods often require handcrafted rewards for every tiny movement. You can't easily tell a robot to "push, then pick up, then carry" because the rewards for "pushing" conflict with "carrying."

2. The Solution: Distance Fields (DF) as a Sense of Touch

The authors argue that the robot shouldn't care about absolute coordinates. It should care about its spatial relationship to the object surface.

The Physics Intuition

Using the Distance Field (Φ), the robot perceives:

• Surface Distance: How far is the hand from the box?
• Surface Gradient: What is the "normal" direction to the surface?
• Velocity Decomposition: Is the hand moving towards the surface (approach) or along the surface (sliding)?

By encoding these into a VAE (Variational Auto-Encoder), the robot gains a "interaction latent" that looks the same whether it's grabbing a small sphere or a large crate.

3. Methodology: Training the Generalist

LESSMIMIC uses a sophisticated Three-Stage Pipeline:

1. Pre-training (Stable Initialization): Uses Behavior Cloning to "teach" the robot basic movements from human data, but forces the robot to look only at DF-features, not the human references.
2. Adversarial Interaction Priors (AIP): This is the "secret sauce." An RL discriminator is trained to tell if an interaction "looks right" geometrically. If the robot's contact patterns match the style of valid interactions, it gets a reward. This allows the robot to adapt to unseen shapes without new labels.
3. Visual Distillation: To move away from expensive MoCap (Motion Capture), the "privileged" geometry data is distilled into a policy that only uses egocentric depth cameras.

4. Evidence of Breakthrough: Scaling and Composing

The experiments prove that LESSMIMIC isn't just a marginal improvement; it's a qualitative shift.

Geometric Robustness

While baselines like HDMI and ResMimic see their success rates drop to 0% when an object is scaled to 0.4x or 1.6x of its original size, LESSMIMIC maintains 80-100% success. It recognizes the local geometry of contact, making size irrelevant.

The "Marathon" of Tasks

Perhaps most impressively, the unified representation allows Long-Horizon Skill Composition. Because the policy is conditioned on the goal and the current geometry, it can transition from "Push" to "Pick" to "Carry" naturally. In tests, it completed sequences of 40 consecutive tasks without resetting—a feat previously impossible for single-policy humanoid models.

5. Critical Analysis & Future Outlook

Why it works: By projecting global velocities into local surface coordinates (Normal vs. Tangent), the authors have found a "Universal Language" for contact. It mimics how humans don't need to know the exact GPS coordinates of a coffee cup to grab it; we just respond to its surface as we get close.

Limitations:

• Articulated Objects: Currently handles rigid shapes. Opening doors or drawers (articulated) is the next frontier.
• Occlusion: In vision-only mode, if the robot's hands block the camera's view of the object, performance drops.

Final Takeaway: LESSMIMIC proves that for humanoid robots to leave the lab and enter the home, we must stop teaching them how to move and start teaching them how to interact with geometry.