Can Biomimetic Attention and State Space Improve Point Cloud Representation Learning?

The tradeoff the paper attacks

Point cloud representation learning faces a persistent tension. Local attention networks compute attention only inside local neighbors or windows, which keeps cost linear in the number of input points but narrows the perceptual field and weakens long-range modeling between objects in a scene. Selective state space models such as S6 offer long-range modeling at linear complexity, and prior work brought them into point clouds, but the paper argues that bidirectional S6 still compresses all context into a history-hidden state, so locality learning stays insufficient.

PointLearner's answer is to stop treating these as alternatives. Each PointLearner block first applies point-focused attention for local geometric modeling, then a context-scan state space for long-range dependency interaction. The paper frames this as focus-then-context, and the rest of its evidence is organized around whether that ordering and those two components actually pay off.

Foveal vision as two competing branches

Point-focused attention has a dual-branch design. The local neighbor branch runs standard attention over the K nearest neighbors of each query point, giving fine-grained perception near the focus. The spatial downsampling branch attends from the query to a set of spatially downsampled features, giving coarse-grained awareness of global semantics. The analogy is explicit: high acuity at the focus, low acuity in the periphery.

The downsampled features are not produced by farthest point sampling. The paper introduces induced point pooling, in which a fixed number of trainable inducing-point vectors attend over the point features to produce M downsampled tokens. Because the number of inducing points is controlled and they interact directly with the point cloud, the pooling can adapt to the non-uniform distribution of points, which the paper says prevents simple average pooling from working as it does on 2D images.

The two branches are then fused competitively rather than by summation. Instead of adding the branch outputs, the model concatenates both branches' query-key scores into a single softmax, splits the resulting weights, and applies them to the respective values. The paper reports that this competitive normalization improves deep interaction between fine and coarse features without adding computational overhead, and its complexity analysis gives linear scaling in the number of points.

Saccades become a Hilbert-guided scan

The context-scan state space is the second half of the block. It uses a bidirectional S6: a forward S6 scans the point sequence in order and a backward S6 scans it in reverse, so each point in the sequence gains a global receptive field. The paper likens this back-and-forth scanning to how the eye infers information about an indistinct object.

The scan path is not arbitrary. Points are serialized along a Hilbert curve, which the paper selects for its spatial locality preservation, so consecutive positions in the sequence correspond to spatially adjacent regions. The ablation supports this choice: on ModelNet40, no serialization gives 91.34% overall accuracy, Hilbert gives 94.17%, Z-order gives 93.06%, Hilbert plus trans-Hilbert gives 93.78%, Hilbert plus Z-order gives 93.52%, and a learnable serialization gives 92.78%. All configurations use 7.36M parameters and 0.610G FLOPs except the learnable one, which uses 8.04M and 0.723G. Throughput differs: 219 FPS with no serialization, 163 FPS with Hilbert, 209 with Z-order, and 168 with learnable serialization.

The paper's explanation is that combining multiple serialization strategies can introduce conflicting spatial relationships and interfere with spatial consistency, while the Hilbert curve supplies a stronger geometry-specific inductive bias than either the Z-order curve or an adaptive learnable ordering. That is the authors' interpretation of the ablation, not a separately measured mechanism.

Where the accuracy gains land

On ModelNet40 classification, PointLearner reports 94.2% overall accuracy, the highest in the table it presents. The nearest attention-based entries include GAD at 93.8%, LFT-Net at 93.2%, and OctFormer at 92.7%; among state space models, PointMamba reaches 93.6%, PCM and Mamba3D 93.4%, and NIMBA 92.1%. Several of those baselines are marked as using pre-training, while PointLearner's row is not.

On ShapeNet part segmentation, PointLearner reports 86.9 instance mIoU, against 86.4 for ReCon, 86.3 for Point2Vec and MaskFeat3D, 86.2 for PointGPT and PointMamba, and 85.7 for PoinTramba. These are instance mIoU on a 16-category, 50-part dataset with 14,005 training and 2,873 test samples, so the number is not interchangeable with the scene-level mIoU used elsewhere in the paper.

The paper also evaluates on S3DIS, where area 5 is the test set and mean IoU over 13 semantic labels is the metric, and on the ScanObjectNN PB T50 RS variant with 15 categories and coordinates-only input. The abstract summarizes these as state-of-the-art performance across multiple point cloud tasks; the specific numbers above are the ones the packet lets a reader check.

Why induced point pooling matters at high sparsity

The most decision-relevant ablation concerns the downsampling branch. The paper compares induced point pooling with farthest point sampling across sampling rates, where the rate is expressed relative to the input number of points, and includes a baseline with no spatial downsampling branch at all.

At low sampling rates the two methods perform comparably, which the paper attributes to farthest point sampling achieving good spatial coverage when enough points are retained. As the downsampling rate increases, farthest point sampling degrades sharply, and at a rate of 1/256 its accuracy approaches the no-downsampling baseline, indicating it cannot capture critical semantics from non-uniform point clouds at high sparsity. Induced point pooling declines more gradually and still reaches 93.39% at 1/256. The paper reads this as evidence that trainable inducing points adaptively learn the point distribution and integrate global semantics more robustly.

This is the result that most changes how a reader should compare the two pooling choices: the ranking is not uniform across sampling budgets, and the advantage of induced point pooling is concentrated where the downsampled token set is small.

What the robustness claim covers

The paper's discussion states that PointLearner exhibits strong robustness to noise and varying sampling densities, and the abstract calls the robustness remarkable. The evidence packet supports the sampling-density part most directly through the induced point pooling ablation at rates down to 1/256, and through the ScanObjectNN variant that uses only coordinates as input.

Readers should keep the scope tight. The reported comparisons are on ModelNet40, ShapeNet, S3DIS, and ScanObjectNN, with recognition and segmentation heads; the packet does not report results for other task families or deployment conditions. The architecture follows a standard Point Transformer-style encoder-decoder with farthest point sampling for downsampling and linear interpolation for upsampling, so the biomimetic components sit inside a conventional backbone rather than replacing it.

The priority the paper establishes

The paper's distinctive claim is an ordering, not a single module. Point-focused attention supplies the fine local structure that bidirectional S6 loses when it compresses context into a hidden state, and the Hilbert-guided scan supplies the long-range structure that local attention cannot reach. The ablation pattern supports treating the serialization choice as a first-order decision: Hilbert ordering alone moves ModelNet40 accuracy from 91.34% to 94.17% at identical parameter and FLOP counts, a larger swing than most architecture-level differences in the comparison table.

For a reader deciding what to take from this work, the practical priority is therefore the pairing of a locality-preserving scan order with a downsampling branch that holds up when the token budget is small. The induced point pooling result at 1/256, where farthest point sampling has fallen back to the no-downsampling baseline while induced point pooling remains at 93.39%, is the clearest threshold in the paper for when that second choice starts to matter.

Sources used in this answer