Story

Have you ever felt that “I can't help slacking off when I'm alone...” while studying? Then “Study Alert” is perfect for you!

This small box-shaped robot watches over your hand and gently scolds you the moment you put down your pen. It will support your learning as a reassuring ally when you study alone.

System

Study Alert is a learning support device for people who study alone. This system uses CoreMP135, a device equipped with an LCD, speaker, and USB on the STM32MP135. The system configuration utilizes the performance of STM32MP135 to support the user's learning.

The Study Alert system configuration is as follows:

〇Determine whether you are studying:

The USB camera takes a picture of your hand.

The position of your hand is detected from the image and the shape of your hand is further analyzed.

It determines whether you are holding a pen and infers whether you are studying or not.

〇Interactive encouragement:

The LCD screen of the CoreMP135 device displays an expressive face. The right cheek of the face is also equipped with a function to display a USB camera that monitors the state of your hand.

If the analysis of the shape of your hand determines that you are not studying, the CoreMP135 speaker will speak to you and gently scold you.

〇Improve learning effectiveness:

It is a strong ally when studying alone and increases your motivation to study.

Design

Extract hand regions from an image from a USB camera using NanoDet for object recognition. ncnn, a deep learning framework, is used to call NanoDet.<br>

Class classification inference is performed from an image of a hand region. MobileNetV1, a CNN, is used to find the feature vectors in the class classification. The SEFR algorithm is a classifier algorithm that can be used even when device resources are limited.

 In the study alert, we have prepared the following classifications: Studying class, Not Studying class, and Smart Phone class. In advance, we collect about 20 images for each class, and learn them by offline processing inside STM32MP135.