TSM-Pose: Topology-Aware Learning with Semantic Mamba for Category-Level Object Pose Estimation

TSM-Pose: Bridging Persistent Homology and Mamba for Robust 6D Pose Estimation

TSM-Pose: Topology-Aware Learning with Semantic Mamba for Category-Level Object Pose Estimation

Summary

Problem

Method

Results

Takeaways

Abstract

TSM-Pose is a novel category-level object pose estimation framework that integrates a Topology Extractor with a Mamba-based Global Semantic Aggregator (MGSA). It achieves state-of-the-art (SOTA) performance on benchmarks like REAL275 and HouseCat6D by capturing deformation-insensitive structural features and modeling long-range dependencies among keypoints.

Executive Summary

TL;DR: TSM-Pose is a high-performance framework for category-level 6D object pose estimation. By combining the mathematical rigor of Persistent Homology with the efficient sequence modeling of Mamba, it solves the dual problem of intra-class shape deformation and semantic information loss. It sets new records on the REAL275 and HouseCat6D datasets, proving that capturing the "topology" of an object is often more valuable than just its "geometry."

Background Positioning: This work belongs to the shape-prior-free paradigm of Category-level Object Pose Estimation (COPE) and is one of the first to effectively integrate Topological Data Analysis (TDA) and State Space Models (SSM) in this domain.

Problem & Motivation: The Geometry Sensitivity Trap

In category-level tasks, we aren't just looking for one specific mug; we are looking for the "concept" of a mug. Traditional local feature extractors (like PointNet++) focus on local coordinate patches. However, these are highly sensitive to:

Intra-class variation: A bottle with a thin neck vs. a wide neck.
Noise and Occlusion: Real-world scans are never perfect.

Previous SOTA methods like AG-Pose attempted to use sparse keypoints to simplify the problem, but they faced Semantic Dilution—when you compress a dense point cloud into a few points, you lose the global "category" context needed for reasoning.

Methodology: Topology and Selective Sequences

1. The Topology Extractor (The "Invariant" Shield)

Instead of relying solely on local points, the authors use Persistent Homology (via Alpha Complexes). They calculate Betti curves and Topological Entropy. These features describe holes, cavities, and connectivity—features that don't change even if you stretch or slightly deform the object.

TSM-Pose Architecture

2. Mamba-based Global Semantic Aggregator (MGSA)

To fix semantic dilution, TSM-Pose introduces:

Semantic Injection: A CAT token (category-level prior) is prepended to the keypoint sequence.
TwinMamba: Since point clouds are unordered, they serialize keypoints using a Hilbert curve. To avoid bias toward the scan order, they use a "Twin" structure: one branch processes the standard sequence, while the other uses Channel Flipping (CF) to look at the features from a different perspective.

3. Feature Fusion

The model fuses visual (DINOv2), local geometric (PointNet++), and global topological features into a unified pool before keypoint detection. This ensures the "where" (keypoints) is informed by the "what" (semantics/topology).

Experiments & Results

TSM-Pose was tested on the most rigorous benchmarks in the field:

REAL275: Achieved 62.4% on the strict 5°2cm metric, a massive leap from the baseline AG-Pose (57.0%).
HouseCat6D: Outperformed CleanPose and SpherePose, proving its mettle in high-occlusion household scenes.

Ablation Highlights

The ablation study confirms that the "Topology Extractor" (TE) and "TwinMamba" (TM) are not just marginal additions:

Baseline (AG-Pose): 57.0% (5°2cm)
+ TwinMamba: 59.9%
+ Topology Extractor: 61.7%
+ Semantic Injection: 62.4%

Visual Comparison Visualizations show that TSM-Pose maintains tight bounding box alignment even for complex categories like 'Camera' where others fail.

Critical Analysis & Conclusion

Takeaway

TSM-Pose demonstrates that global structural stability (Topology) and efficient long-range dependency modeling (Mamba) are the new frontiers for 3D spatial reasoning. It successfully bridges the gap between high-level semantics and low-level geometry.

Limitations & Future Work

Inference Speed: TDA algorithms currently lack heavy GPU optimization. TSM-Pose runs at 15 FPS compared to 23 FPS for the baseline.
Computational Overhead: The addition of topological calculations adds latency that might hinder real-time deployment on low-power edge robots.
Future: The authors aim to parallelize the TDA process on GPUs and explore more multimodal strategies.

TSM-Pose is a significant step toward "Embodied Intelligence" that can truly understand the structure of the world, not just its surface.

Find Similar Papers

Try Our Examples

Find recent papers in 3D computer vision that utilize persistent homology or topological data analysis for robust feature representation.
Which 3D point cloud analysis papers first adapted the Mamba/State Space Model architecture, and how do they handle the lack of inherent order in point sets?
Explore current research on "category-level object pose estimation" that specifically focuses on handling heavy occlusions and high intra-class deformation in robotic grasping.

Contents

TSM-Pose: Bridging Persistent Homology and Mamba for Robust 6D Pose Estimation

1. Executive Summary

2. Problem & Motivation: The Geometry Sensitivity Trap

3. Methodology: Topology and Selective Sequences

3.1. 1. The Topology Extractor (The "Invariant" Shield)

3.2. 2. Mamba-based Global Semantic Aggregator (MGSA)

3.3. 3. Feature Fusion

4. Experiments & Results

4.1. Ablation Highlights

5. Critical Analysis & Conclusion

5.1. Takeaway

5.2. Limitations & Future Work