Autonomously driving Raspberry Pi KI robot car – IMU Sensor 9-DOF absolute orientation

I have installed an IMU Sensor 9 DOF absolute orientation on my Donkey Car. The idea I’m following here is to train the neural network with the measured acceleration values instead of the values of the PS4 controller. For example, the neural network should be able to drive up a mountain or to react to an empty battery by sending commands from the motor control to the motor. Whether this is the case and whether it works at all will be shown by the tests still to come. In general, however, it should not hurt to collect more data about the vehicle, its driving behaviour and other characteristics than to do without them.

Dexter Industries BNO055 Sensor

Which sensor I have installed and how I have taken this generally under Raspbian in operation, I explain in the following post here on my blog: Dexter Industries IMU Sensor 9-DOF Absolute Orientation BNO055

Table of Contents

BNO055 Donkey Car Part

In order for the Donkey Car Framework to be able to use the BNO055 sensor, an extra part must be written for it. I wrote this part in Python and put it online. The part allows to write all data from the sensor into the JSON file per recorded image.

The part is available online on GitHub in its early version: donkeypart_bno055

Integration of the Donkey Car Part BNO055

The integration of the part is actually quite simple. The following line must be added to the manage.py file. This loads the new part bno055 when the framework is started.

from donkeycar.parts.imubno055 import bno055

After the part has been integrated, the sensor values to be saved must be defined. Therefore I have added the following lines after the “V.add(cam, out…. )” line and before the “if use_joystick or…”.

V.add(cam, outputs=['cam/image_array'], threaded=True)

# Ingmar Stapel 20190311
imu = bno055()
V.add(imu, outputs=['imu/acl_x', 'imu/acl_y', 'imu/acl_z', 'imu/acg_x', 'imu/acg_y', 
'imu/acg_z', 'imu/gyr_x', 'imu/gyr_y', 'imu/gyr_z', 'imu/temp', 'imu/cali_sys', 
'imu/cali_gyro', 'imu/cali_accel', 'imu/cali_mag'], threaded=True)  

if use_joystick or cfg.USE_JOYSTICK_AS_DEFAULT:

Now it is known which values are to be saved. In order to make sure that the meta-description of the JSON file is also included in the JSON file and understood by the other programs that process the files, the meta-description of the JSON file must be adapted or supplemented. To this end I have made the following change. I simply commented out the existing definition of inputs and types and added my new definition.

# add tub to save data
#inputs = ['cam/image_array', 'user/angle', 'user/throttle', 'user/mode', 'timestamp']
#types = ['image_array', 'float', 'float',  'str', 'str']

#Ingmar Stapel 20190311 add tub to save data
inputs = ['cam/image_array', 'user/angle', 'user/throttle', 'user/mode', 'timestamp',
          'imu/acl_x', 'imu/acl_y', 'imu/acl_z', 'imu/acg_x', 'imu/acg_y', 'imu/acg_z', 
          'imu/gyr_x', 'imu/gyr_y', 'imu/gyr_z', 'imu/temp', 'imu/cali_sys', 'imu/cali_gyro', 
          'imu/cali_accel', 'imu/cali_mag']
types = ['image_array', 'float', 'float',  'str', 'str', 'float', 'float', 'float', 'float', 
         'float', 'float', 'float', 'float', 'float', 'float', 'int', 'int', 'int', 'int']

Now the new part is available and the values measured by the BNO055 sensor are stored in the JSON file for each image.

Summary

With this addition, it is now possible to record and save the IMU values or acceleration values. In a first test the values looked quite good. I am still unsure whether the BNO055 sensor needs to be calibrated or whether the automatic calibration, which is basically active, is sufficient. Then we need to clarify if the neural network can now control the donkey car better using the IMU values if the steering and acceleration values of the PS4 controller are not used.