1. Blog>
  2. Voice Recognition Robot Car

Voice Recognition Robot Car

by: Feb 28,2022 530 Views 0 Comments Posted in Engineering Technical

Stem Robot

Note: The content and the pictures in this article are contributed by the author. The opinions expressed by contributors are their own and not those of PCBWay. If there is any infringement of content or pictures, please contact our editor (steven@pcbway.com) for deleting.

Written by Vincent Kok


In this article, I will be making a voice-activated Robot Car with the in-built microphone on Wio Terminal to recognize the go, stop and background noise by TinyML.

Trained a wake-up word recognition model using Codecraft that is powered by Edge Impulse! 

In this detailed blog post, I will cover the following:

What's UART Serial Communication?

UART Serial Communication between Wio Terminal and uKit Explore

  • 1.0 Train an Embedded Machine Learning Model Using Codecraft
  • 2.0 Arduino Text Code Modification
  • 3.0 Program the Robot Car (uKit Explore - Arduino Mega 2560 based)
  • 4.0 Expected Result

What's UART Serial Communication?

In layman terms, UART allows an embedded device such as Arduino to send data over to another Arduino via the TX (transmit) and RX (receive) line as shown below.

Image source: https://learn.sparkfun.com/tutorials/serial-communication/all

Let me give you an example case study. 

Well, Arduino UNO has no built-in WiFi and hence it's not possible to do IOT related projects. With the understanding of the basic UART serial communication, I was able to leverage the ESP8266/ESP32 as the co-processor to the Arduino UNO so that the data collected from the sensors attached to Arduino UNO will be sent over to ESP8266/ESP32 to send over to the cloud platform such as web server, Blynk or FAVORIOT.

UART Serial Communication between Wio Terminal and uKit Explore

The TX and RX pins for Wio Terminal is available on pin 8 and 10.

TXD - Pin 8 

RXD - Pin 10

Whereas for uKit Explore, there are also TX/RX pins available on the pin D0 and D1. You can check the full uKit Explore pinout here: https://ubtechedu.gitbook.io/ukit-explore/v/english/ukit-explore-quick-start/pinout

RX0 - D0

TX0 - D1

For the hardware connection, basically, you will just need to connect as below:

Wio Terminal (Pin 8) to uKit Explore (pin 0) - TXD to RXD

Wio Terminal (Pin 10) to uKit Explore(pin 1) - RXD to TXD

I will explain more on the software programming how it is done on 3.0 Program the Robot Car (uKit Explore - Arduino Mega 2560 based).

1.0 Train an Embedded Machine Learning Model Using Codecraft

In this first section, our goal is to create an embedded machine learning model (voice recognition) using Codecraft platform.

There are 4 steps involved to train an embedded machine learning model using Codecraft.

1.Create the "Wake-Up Words Recognition (built-in microphone)" model

2.Data Acquisition (on-board)

3.Training and Deployment

4.Programming & Model Usage

STEP 1 Create the "Wake-Up Words Recognition (built-in microphone)" model

Go to https://ide.tinkergen.com/. Select "(for TinyML) Wio Terminal".

Click on "Model Creation" on the embedded machine learning box on the middle left. Then select the "Wake-Up Words Recognition (built-in microphone)" as shown below. Enter the name of the model according to the requirements.

Click Ok and the window will automatically switch to the "Data Acquisition" interface.

STEP 2: Data Acquisition (on-board)


There are 3 default labels (hi wio, background, and other words) that are automatically created for you.

You can use it without any changes unless you want to have different names for your labels. For my case, I changed two of the default labels as below:

hi wio changed to go

other words changed to stop

IMPORTANT: Now, you have to remember to change the labels on the default data acquisition program to reflect the correct modified labels:


Connect Wio Terminal and Upload Data Acquisition Program.

Note: You need to download “Codecraft Assistant” to be able to Connect and upload code on Codecraft online IDE.

Data Acquisition: In the upper right hyperlink, you will find a step-by-step introduction to data acquisition. Follow the instructions to collect data accordingly to your modified labels.

STEP 3: Training and Deployment

· Click on “Training & Deployment”, and you will be seeing the model training interface as shown below.



Select neural network and parameters. Select the suitable neural network size: small, medium, and large

Set parameters:

number of training cycles (positive integer),

learning rate (number from 0 to 1)

minimum confidence rating (number from 0 to 1)

The interface provides default parameter values of training cycles of 50, however, the accuracy was not very good. Hence I changed the training cycles to 100.


Click “Start training”. When you click “Start training”, the windows will display “Loading..”! Wait for the training to be done!

Observe the model performance to select the ideal model. In the “Model Training Report” window, you can observe the training result including the accuracy, loss, and performance of the model.


· In the “Model Training Report” window, click on “Model Deployment”. Once the deployment is completed, click “Ok” to go the “Programming” windows which is the last step before we deploy the model to the Wio Terminal.


STEP 4: Programming & Model Usage

Alright, so now we done trained the model and the fun part of integrating Artificial Intelligence (Machine Learning in this case) with robotics (a Robo Car) using the UART communication protocol.

This is the sample program that created from the block programming interface:


We use the if-else conditional statements to evaluate the confidence of the labels. 

If the confidence of "go" is greater than 0.8 (80%), I will print "1" on the serial terminal.

If the confidence of "stop" is greater than 0.8 (80%), I will print "2" on the serial terminal.

Otherwise, if the confidence of "background" is greater than 0.8 (80%), I will print "0" on the serial terminal.

Ok, so for now, just remember 3 different conditions:

"go" > 0.8, the command is '1'

"stop" > 0.8, the command is '2'

"background". the command is '0'

2.0 Wio Terminal Arduino Text Code Modification

Alright, so take note on the 2 important findings as below as it will be the critical part for our project!

  • The serial line you can access from the 40-pin header of Wio Temrinal is Serial1 instead of usual Serial which basically shows the output via Serial Terminal.
  • We have the TX/ RX pin available on the pin 8 and 10 that can be used to connect to another board (uKit Explore in this case).


If we have a look at the text coding for the corresponding blocks code, you will notice that the Serial.print is NOT using the Serial1 line. Hence, this leads us to our second step which is to continue our coding on Arduino IDE for customization.


Toggle to the Text Code area & Copy The Text Code

On the text code area, copy all the code by pressing on CTRL + A to select all the code.

Open Arduino IDE, create a new file, paste the code into the empty sketch by pressing on CTRL+V. Proceed to save the sketch with the desired name.




Copy the Edge Impulse TinyML Arduino Library

Navigate to C:\Users\<User_Name>\AppData\Local\Programs\cc-assistant\resources\compilers\Arduino\contents\libraries

Find the folder name that has the same number with the Edge Impulse header file on top of your Arduino text code (in my case it will be 47606).


Copy the entire ei-project_47606 folder and paste it onto C:\Users\<User_Name>\Documents\Arduino\libraries\


Modification on the Serial.println function

Modify the Serial.println function to Serial1.println instead.



Upload the Code

Make sure you have installed the Wio Terminal board support package. If not, please refer to the "Get Started with Wio Terminal" guide on Seeed's Wiki.

Make sure you selected the correct board and COM port before you upload the code to Wio Terminal.


3.0 Program the Robot Car (uKit Explore - Arduino Mega 2560 based)

We will be programming the Robot Car that is built on the uKit Explore.

Introduction to uKit Explore

Let me give you some understanding of what's this uKit Explore about.

uKit Explore is a robotic kit manufactured by UBTECH Robotics and it is based on the Arduino ecosystem (the chipset is Arduino 2560).

"The uKit Explore has a main control box equipped with Arduino open-source platform which offers an incredible amount of electronics and programming learning resources from contributions of users worldwide."

Program uKit Explore to Read UART Data

Right now, we will need to program the Robo Car to act accordingly to the different conditions as it receives the command '1', '2', and '0' from Wio Terminal.

The key concept here is that we will need to get the uKit Explore to keep listening to the serial line if it contains any data. And if it does contain serial data, serial read the data and assigned it to a variable. The final step is that we will compare the variable and have different actions.

For our case, when uKit Explore 

  • receives command '1' (go), the RoboCar will move forward; 
  • receives command '2'(stop), it will stop; 
  • receives command '0'(background), it will stop as well.



Upload the Code

Finally, we will upload the code onto the board. As always, please make sure you selected the correct board and COM port before the upload.

4.0 Expected Result

Check out the video for the Robot Car in action!

https://youtu.be/qKS5SAMgBHU 

As you can see, the Robo Car can be activated via voice 'go, go, go', and stop by either 'stopppps....' or the 'background'. We have achieve the objective we wanted to!

Conclusion

The Codecraft is a very easy tool to train a simple embedded machine learning model and integrate it with your own robotics system powered by technology enabled by Edge Impulse and Wio Terminal. Please let me know if you have suggestions on the article. Thanks!


GitHub Source Code: https://github.com/VincentK16/voicerecognitionrobocar



Join us
Wanna be a dedicated PCBWay writer? We definately look forward to having you with us.
  • Comments(0)
You can only upload 1 files in total. Each file cannot exceed 2MB. Supports JPG, JPEG, GIF, PNG, BMP
    View More
    Back to top