Can a compact magnetociliary sensor give robots 3D force and surface sensing?

Why one Hall sensor makes 3D force hard

A magnetic tactile sensor works by letting contact deform a magnetized structure and reading how the local magnetic field changes. MagCilia's structure is a skin carrying a 9×13 array of magnetic cilia on a 1 mm base membrane, with 2.5 mm cilia, bonded to a PDMS buffer layer; a Hall sensor sits centered beneath the skin and samples at 246 Hz. The whole unit measures 28.00×24.00×11.70 mm and weighs 8.6 g.

The difficulty is coupling. Normal loading compresses the cilia and supporting layer, tangential loading bends the cilia and shears the buffer, and oblique loading does both. Each of these moves and rotates magnetic material relative to the sensing point, so a force along one axis changes more than one magnetic channel. The paper's sensitivity matrix makes this explicit: its off-diagonal terms describe how loading in one direction leaks into the other channels. That is why a channel-by-channel mapping cannot simply be inverted for 3D force.

Quasi-static finite element analysis in COMSOL supports the picture. Under normal indentation the field change appears mainly in the Bz component; under orthogonal shear the dominant transverse response reverses sign with loading direction; oblique shear changes both transverse components. Different directions therefore leave distinguishable but mixed signatures, which motivates reading the full magnetic vector jointly rather than one channel at a time.

How CHFR turns magnetic history into force

Causal history fusion regression (CHFR) is the reconstruction method. Its input is not just the current baseline-corrected three-axis magnetic readout but that readout plus its differences over the previous 40 ms and 120 ms, giving a nine-dimensional vector. The current value describes the present magnetic state; the two differences describe how it has recently evolved at two time scales. All three magnetic channels are included in each part.

CHFR then maps those nine features to Fx, Fy and Fz through two parallel branches whose predictions are averaged equally. One branch is an Extra Trees ensemble of 180 decision trees with at least 15 samples per leaf, which captures nonlinear relationships through local partitions. The other is kernel-approximation ridge regression, using standardized inputs, a radial basis function kernel approximated with 400 Nyström components, γ = 0.03 and α = 1, which provides a smooth nonlinear fit.

The evaluation is designed to test generalization to unseen loading records rather than to interpolate within one. Five independent calibration records supply 304,870 synchronized magnetic-field and reference-force sample pairs, covering −4.74–4.81 N for Fx, −6.64–4.74 N for Fy and −9.60–0.05 N for Fz. Five-fold cross-validation holds out one complete record per fold, and metrics are computed from pooled out-of-fold predictions.

Force errors depend on the axis

Across the five held-out calibration records, the coefficient of determination is 0.93 for Fx, 0.90 for Fy and 0.92 for Fz. The corresponding RMSEs are 0.40, 0.57 and 0.69 N, and the mean absolute errors are 0.25, 0.36 and 0.45 N. The normal component Fz is thus the least accurate of the three, and its error is largest in absolute newtons.

These numbers are aggregate errors over the calibrated load ranges and contact conditions, not a guarantee for any single contact. The paper states that they support joint reconstruction of normal and tangential forces within those ranges. A reader comparing sensors should also keep the metric type straight: RMSE and MAE are force errors in newtons, while R² measures agreement relative to the variation of the reference force, so a high R² and a large RMSE can coexist when the tested force range is wide.

Dynamic fingertip tests complement the aggregate figures by checking tracking over time. In one record, RMSEs were 0.05, 0.07 and 0.12 N for Fx, Fy and Fz. Across valid paired time points in four records, RMSEs ranged from 0.05 to 0.10 N for Fx, 0.07 to 0.12 N for Fy and 0.12 to 0.19 N for Fz. These are much smaller than the calibration errors, so they should not be read as a replacement for them: they describe repeated manual contacts of varying magnitude, direction and rate, not the full calibration envelope.

Tangential force decides grip direction

In the grasping demonstration, MagCilia is mounted on a Franka Research 3 gripper and CHFR supplies real-time forces to the controller. The reconstructed signals are low-pass filtered with an 80 ms time constant. The controller tracks the tangential component aligned with vertical loading, Ft, relative to its stable-grasp baseline Ft,0; the change passes through a deadband and a persistence criterion before any command is issued. Normal compression, defined as Fn = −Fz, is used to monitor contact and limit further closure.

The division of labor matters: tangential change selects whether to close or open, while normal force constrains how much compression is allowed. Sustained downward loading triggers additional closure; sustained upward support or pulling triggers opening so a person can take the object; when tangential force returns to baseline the gripper holds its current opening.

The recorded forces follow that logic. From 2 to 14 s during initial grasping, Fn stays near 1.1 N and tangential forces stay close to zero. With added external loading the controller tightens, and Fn rises to about 1.7 N during 18–24 s while Fx rises to about 0.7 N. Further loading produces a peak Fn of 3.91 N. During upward support and removal the gripper opens and all three components return near zero after release.

Two further tests probe smaller and differently directed loads. Adding weights to a grasped box in three cumulative steps produces settled Fx plateaus of 0.08, 0.11 and 0.17 N against a pre-load mean of 0.04 N, while Fn rises from 0.32 to 0.48 N and Fy stays in a relatively small range. Under lateral perturbation, Fy varies over −1.87–0.80 N; under vertical perturbation, Fx and Fy span −0.85–0.93 N and −0.73–0.77 N while Fn peaks at 2.41 N.

Surface identity lives in frequency bands

The surface-recognition demonstration reuses the same reconstructed forces rather than adding a second sensing modality. A robot moves MagCilia forward and back along each surface while maintaining cilia contact, and each forward-and-return scan counts as one trial. The dataset contains 328 trials across 52 acquisition sessions, covering acrylic, fabric, paper, PLA, rubber and sponge.

The six categories differ in both amplitude and temporal fluctuation. The peak magnitude of the median normal-force curve is 2.50 N for paper and 0.73 N for sponge, and the median Fx curves have peak-to-peak amplitudes of 1.18 and 0.18 N respectively. Those median traces summarize typical response; the paper also reports the range across repeated trials.

Classification uses the spectral shape of those signals. Scanning signals are resampled at 250 Hz and detrended, Welch's method estimates each force component's power spectrum in each scanning direction, and spectra are divided into 14 bands. Band power is normalized within each channel and log-transformed, so the feature describes relative spectral distribution rather than overall force amplitude. Concatenating two directions, three force components and 14 bands gives 84 features for multiclass logistic regression.

In three-fold cross-validation grouped by acquisition session, so that all trials from one session stay in the same fold, the classifier correctly labels 326 of 328 trials: 99.39% accuracy, 99.37% balanced accuracy and a macro-averaged F1 of 0.99. An online robotic demonstration additionally identified all six tested surfaces.

What the single-unit comparison shows

The paper's comparison table places MagCilia against other magnetic tactile designs on configuration and capability. A sinusoidally magnetized film with Hall sensing delivered normal force and one tangential component; stacked orthogonally magnetized films with a 3×3 Hall array extended a unit to three force components with a 28 N normal range and ±8 N tangential range; a magnetic-cilia array with a 2×4 sensor array addressed tangential force and direction at 0.2 mN resolution; soft-magnetic cilia with an induction coil targeted weak tangential sensing; segmented magnetic cilia addressed flow and texture; and magnetic cilia with a single Hall sensor reported normal force and tangential direction with normal sensitivity 0.015 N⁻¹ and R² = 0.97.

MagCilia's distinguishing entry is the combination: magnetic cilia with a single Hall sensor, 3D force sensing, and reported RMSEs of 0.40/0.57/0.69 N with R² of 0.93/0.90/0.92. The comparison is across different sensing tasks and reported metrics, so the numbers are not a like-for-like accuracy ranking. What it does establish is that quantitative three-component reconstruction is claimed here for a single contact unit and a single Hall sensor, rather than for a multi-sensor array.

Where the claims stop

The force accuracy figures are tied to the calibrated load ranges and contact conditions, and the calibration data came from a handheld indenter applying normal, lateral and combined loading. The surface result is bounded by six categories, 328 trials and 52 sessions, with offline accuracy from session-grouped cross-validation and an online demonstration that covered the same six surfaces. Neither result establishes performance on untested materials, uncalibrated load ranges or contact geometries outside the calibration set.

The authors also state the direction of future work: optimizing the magnetic composite and cilia geometry, and integrating a Hall-sensor array to improve force-reconstruction accuracy and spatial resolution. That framing is itself informative, because a single sensing point limits spatial resolution even when three force components are reconstructed.

The real claim is reuse of one signal

The most decision-relevant point is not that MagCilia reports 3D force or that it classifies surfaces, but that both come from the same reconstructed force signals of one compact unit. Grasping feedback uses the time-domain force components and their changes; surface recognition uses the frequency-domain distribution of those same components across two scanning directions. A reader weighing this design should therefore ask which requirement dominates: if quantitative force feedback within a calibrated range matters most, the Fz RMSE of 0.69 N and the record-level validation are the numbers to scrutinize; if surface discrimination matters most, the relevant evidence is the 99.39% session-grouped result over six categories and the fact that it depends on active scanning rather than a single touch.

Sources used in this answer