Now I assume that enough training data has been recorded and that this data meets my own requirements to train the neural network. The training environment should ideally be set up on a computer with Ubuntu and a powerful GPU. I use my aging Lenovo notebook with i5 processor and without a strong graphics card, so without GPU support.

Setting up the virtual environment

First you set up a virtual environment with Conda which is called “openbot”. To do so, execute the following command from which you will create the environment “openbot” with all necessary additional modules.

Note: I recommend to create a folder named “openbot” under the Home folder and to change to this folder via the console.

Installation without GPU support:

Command: conda create -n openbot python=3.7 tensorflow=2.0.0 notebook=6.1.1 matplotlib=3.3.1 pillow=7.2.0

Installation with GPU support:

Command: conda create -n openbot python=3.7 tensorflow-gpu=2.0.0 notebook=6.1.1 matplotlib=3.3.1 pillow=7.2.0

Note: If you get an error immediately, that the call conda is not known or similar then it is possible that conda is not installed on your computer yet. How exactly Conda can be installed can be read under the following link (How to install conda). If Conda and in which version is installed you can find out with the following command.

Command: conda --version

You can always start the virtual openbot Conda environment from the console with the following command

Command: conda activate openbot

OpenBot repository clonen

Afterwards the OpenBot repository has to be stored as a clone on your computer.

To download or clone the repository execute the following command.

Command: git clone https://github.com/intel-isl/OpenBot.git

Note: If there is an error that says Git is not available then install your Git with the following command under Ubuntu

Command: sudo apt-get install git

Now the download via Git should work and the cloned OpenBot repository should be on your computer.

Run Jupyter Notebook

Now the virtual Conda environment with the name “openbot” is started via the console if not already done (command: conda activate openbot). Afterwards Jupyter is executed to run the Jupyter Notebook “policy_learning.ipynb” in the browser. The Jupyter Notebook “policy_learning.ipynb” contains all the logic for training the neural network for your OpenBot. The command to start Jupyter is as follows and you execute this command in the active Conda “openbot” environment in the folder openbot.

Command: jupyter notebook

The command to start directly the Jupyter policy_learning.ipynb notebook is as follow.

Command: jupyter notebook policy_learning.ipynb

Here is a screenshot of how the command should be executed in the console in the folder openbot where I cloned the repository before

Jupyter notebook active conda environment

Jupyter notebook active conda environment

Now a browser window opens on your computer and there you can see the folder structure in the OpenBot folder that was created when cloning the corresponding repository.

OpenBot - Active Jupyter Notebook

OpenBot – Active Jupyter Notebook

Now navigate in the browser or the Jupyter environment to the folder “/OpenBot/policy” and click on the “policy_learning.ipynb” notebook to execute it.

OpenBot active policy learning.ipynb notebook

OpenBot active policy learning.ipynb notebook

That was it so far and now it could start with the training of the neural network if the training data are already on our computer. In the following paragraph I will briefly describe where exactly the training data must be located.

Prepare training data

Now everything is prepared and the training data that is still on the smartphone is transferred to the training computer. To do this, connect the smartphone to the computer via cable, download the corresponding *.ZIP files and store them on the desktop. You can find the *.ZIP files on your smartphone directly on the internal memory in a folder named “OpenBot”.

Now simply unzip two of the ZIP files. Because the training should be started with only a few files to see if everything works technically. We will continue to work with these files and store them in the folder “policy”, so that the Jupyter Notebook “policy_learning.ipynb” can read the data.

OpenBot training data folder

OpenBot training data folder

Now place the first of the two unzipped folders in the following folder structure “~/openbot/OpenBot/policy/dataset/test_data/my_data_living_room/<the unzipped folder>”.

The folder structure must be created as described above. The name “my_data_living_room” can be chosen freely and should describe the data in which context it was recorded. In this folder you put the unzipped zip file.

So you need two folders and the structure looks like this:

  • ~/openbot/OpenBot/policy/dataset/train_data/my_data_living_room/<the folder with the training files>
  • ~/openbot/OpenBot/policy/dataset/test_data/my_data_living_room/<the folder with the training files>

The following picture shows once again the structure how to create it.

OpenBot folder structure

OpenBot folder structure

Now all preparations are done and the first test run of the training of a neural network can start.

Summary

With this article you should now be able to install the OpenBot environment on your PC and train your own neural network. If all this works technically, then comes the exciting and time consuming part of creating training data. With this data you can teach your OpenBot completely new behaviors like turning into other rooms or navigating in a corridor with side corridors or crossings. This makes this project really exciting to train a neural network that learns these skills.


Article Overview OpenBot robot cuto:

OpenBot – Your smartphone controls a robot car – Introduction
OpenBot – Your smartphone controls a robot car – needed parts part 1-2
OpenBot – Your smartphone controls a robot car – needed parts part 2-2
OpenBot – Your smartphone controls a robot car – constructing a chassis
OpenBot – Your smartphone controls a robot car – Wiring
OpenBot – Your smartphone controls a robot car – Flashing the Arduino firmware
OpenBot – Your smartphone controls a robot car – Android App and first test run
OpenBot – Your smartphone controls a robot car – Record training data
OpenBot – Your smartphone controls a robot car – Set up training environment
OpenBot – Your smartphone controls a robot car – train the neural network
OpenBot – Your smartphone controls a robot car – Setting up Android Studio and your Smartphone
OpenBot – Your smartphone controls a robot car – compile your own OpenBot Android app

Planned articles in this series:

OpenBot – Your smartphone controls a robot car – End-2-End run