MOYA: Bridging the Gap Between Child Curiosity and Language Acquisition via AI Robotics

TL;DR

MOYA is an interactive robotic toy that uses computer vision to "see" what a child is pointing at and "speak" the name of the object. Built on a Raspberry Pi using MobileNet and ROS, it transforms the common childhood habit of pointing into an autonomous, multi-modal language learning experience.

Background & Motivation: The Pointing Gesture

Between the toddler and preschool years, pointing is a fundamental communicative act. It signals a demand for attention or a "What is that?" inquiry. Traditionally, this requires a parent to be present 24/7. The authors identified a gap: How can we create an autonomous companion that understands these spatial gestures and provides immediate linguistic feedback without heavy server-side computing?

Methodology: High-End Interaction on Low-End Hardware

The challenge with MOYA was fitting complex AI into an embedded form factor.

1. Vision Logic: MobileNet over ResNet

The researchers opted against heavy object detection pipelines. Instead, they focused on Image Classification using the MobileNet architecture.

• Why? Architectures like ResNet are too latent for Raspberry Pi. MobileNet uses depthwise separable convolutions to maintain speed while sacrificing minimal accuracy.
• Optimization: They modified the capture resolution via OpenCV to 480x720 to reduce the pixel processing load for the TensorFlow model.

2. Spatial Awareness: Hand Motion Tracking

To understand where the child is pointing, MOYA utilizes the RealSense SDK for skeletal extraction. By mapping the vector of the child's arm to the robot’s own coordinate frame, the robot can orient its camera toward the target object.

Figure 1: The conceptual design showcases the integration of the camera, screen, and mobile base.

3. Software Architecture: The ROS Framework

MOYA runs on Ubuntu Mate and leverages ROS (Robot Operating System). The system is split into four distinct nodes:

• GUI Node: Visual feedback and quizzes.
• Serial Node: Communication with the Arduino Mega (controlling wheels/speakers).
• Classification Node: The TensorFlow inference engine.
• Control Node: Managing robot mobility.

Experimental Results & Tasks

The study successfully implemented four core interactive functionalities:

1. Object Naming: Converting pointing gestures into text-to-speech and visual display.
2. Autonomous Pursuit: MOYA maintains a specific distance from the child to remain within the gesture-capture zone.
3. Educational Quiz: A recap mode where MOYA shows an image and the child must name it.
4. Multi-language Support: A user-selectable language toggle for diverse learning.

Figure 2: The skeletal extraction process used to identify pointing vectors.

Critical Analysis & Conclusion

MOYA represents a successful fusion of Human-Computer Interaction (HCI) and Edge AI.

Key Strengths:

• Inductive Bias: By limiting the scope to 7 common household objects, the researchers achieved high reliability on very weak hardware.
• Modular Design: Using ROS allows for easy future upgrades—for instance, replacing the Raspberry Pi with a Jetson Nano without rewriting the motor control logic.

Limitations:

• Hardware Scale: The authors noted that the Intel RealSense camera was difficult to integrate into the final physical shell due to size constraints.
• Classification Depth: Currently limited to a small subset of objects.

Future Outlook: The implications of MOYA extend beyond childcare. The same logic of "pointing to recall names" has significant potential in geriatric care, specifically helping elderly patients with early-stage dementia maintain their vocabulary through interactive environmental engagement.

Note: This work was presented at the 9th Augmented Human International Conference (AH2018).