OpenBot – Your smartphone controls a robot car – Record training data

This article deals with the recording of training data in order to train an own neural network. First of all it is very interesting to train your own neural network from a technical point of view but it is much more exciting to see if the robot car will be able to operate better in its own environment from which the training data originates. This does not always have to be the case and depends very much on whether the training data is good or not. Only what is good training data and what is not is not easy to say. But more about that in this article which will guide you step by step how to train a neural network for the OpenBot.

Note: You will need some perseverance and exploratory spirit if you have never done anything with Deep Learning etc. It can be a bit challenging even when choosing the hardware and software for the training computer. But I can promise the endurance is worth it and the topic of artificial intelligence is super exciting.

Table of Contents

Recording training data

In my report, which was generally about the use of the Android app, I had already touched on the issue of which training data can be recorded. For the next steps I will use the default settings of the app and record all sensor data as well as the scaled down image files for the neural network. As suggested by the OpenBot developers, this should be sufficient to train a neural network. This is also in line with my experiences, so I have successfully trained the donkey cars with images with a resolution of about 160 x 120 pixels.

Here again the options that can be selected for the training data recording.

OpenBot App Screenshot collect trainings data

Now it’s important to keep in mind the functions of the Android app that are controlled by the PS4 controller. This is the only way to successfully control the robot car or the training data recording without increasing the frustration level.

Therefore I have tried to show the different functions of the PS4 controller in the following picture. It is important to see with which button exactly which function can be controlled.

OpoenBot PS4 Controller functions

So, now it starts with the recording of the training data. I suggest that you do a few practice laps to practice the driving itself so that the training data will be of high quality later. If the driving is smooth then the recording of the training data starts which will later be used for the training of the neural network.

Note: Not always a lot of data helps but the quality of the data must be right.

Here are a few examples of errors when creating training data from different trainings with Donkey Cars which I have already done at many locations. The picture shows a few donkey cars I used for these trainings.

Jetson Nano Donkey Cars

Example 1:
During a training with 40 participants and four donkey cars, one of the four groups always drove clockwise along the track. Their donkey car was then unable to run the course counterclockwise with the trained model. Exactly this is an example of a human BIAS which was given to the neural network via the training data.
Example 2:
In another training session, a participant was able to control the Donkey Car with the PS4 controller. This group recorded many 10,000 training data sets where the donkey car ran full throttle clockwise and counterclockwise along the test track. But since the training data only knew one position of the throttle lever, the neural network had not learned to start the car. When it came to driving autonomously the donkey car did not start. If you pushed it for a short time, it drove full throttle down the track but not for long because after a few laps it couldn’t handle the steering anymore.
Example 3:
A training with managers resulted in the participants sitting on chairs around the track when the training data was recorded. When the neural network was trained and the car was supposed to drive autonomously, nobody was sitting along the track anymore. The result was that the donkey car could no longer steer. When the participants sat down again, the donkey car or the neural network drove along the track without any problems.

These are just a few examples that show how difficult it is to record “good” training data. If the room where the training data is recorded and later the robot is supposed to drive the car autonomously, the time of day with the corresponding solar radiation plays a role as well.

Now let’s start practicing driving and recording training data that meet the requirements, i.e. are “good”…

Summary

In this article I have tried to describe how training data can be recorded. It is important to be clear what you want to teach the OpenBot or the neural network. A really exciting challenge is to bend into e.g. another room when the robot car drives down a corridor. You can use the blinkers to save exactly this intention while recording the training data. Accordingly, the neural network should be able to make the robot turn off the car.