Autonomous Voice Activated Robot

This project is designed to integrate different robotics modules like stop sign detection, lane tracking, obstacle detection, and using voice commands to allow the robot to take actions accordingly.

The main objective of this project is to build and deploy various robotics modules which have been created for Qualcomm Robotics RB5 Development Kit.

EquipmentDescription
Robotics DragonBoard™ RB5 development boardRobotics platform from Qualcomm Technologies, Inc.
Power adapter12 V with 2500 mA required by the 96Boards specification
USB to Micro USB cable

USB to USB Type C cable
For serial console interface and ADB, Fast boot commands

For connecting the USB3.0 Type C port to the board and flashing images
Turtlebot Burger (TurtleBot3)Robot
DescriptionLink
Source Code: Github Link for source of RB5 integrated Robotic Module.https://github.com/globaledgesoft/AutonomousRB5
Resource TitleLink or File name (and provide File)
Qualcomm Robotics RB5
Development Kit bringup
https://developer.qualcomm.com/qualcomm-robotics-rb5-kit/quick-start-guide/qualcomm_robotics_rb5_development_kit_bring_up

Acronyms:

ROSRobot Operating System

Prerequisites:

  1. A Linux workstation with Ubuntu 18.04.
  2. Install Android Platform tools (ADB, Fastboot)
  3. Download and install the SDK Manager
  4. Flash the RB5 firmware image on to the board
  5. Setup the Network.
  6. Install third party libraries and test basic RoS application as instructed here.
  7. Turtlebot burger is assembled, operational and is connected to the Qualcomm Robotics RB5 development board.
  8. Make sure that the development kit has installed portaudio from source.

Steps to build and deploy RB5 Scene Detection Application

  1. Setup the project Directory in the Qualcomm Robotics RB5 development kit by cloning the project repository from the github.
    adb shell
    cd /home/integrated-robotic-module-rb5
    git clone https://github.com/globaledgesoft/integrated-robotic-module-rb5
  1. Installation of TurtleBot 3 Package

    For the setup we will be using the TurtleBot3 Burger, we need to install TurtleBot Packages for controlling the TurtleBot.

    1. Setup the necessary packages by executing the following commands.
      apt install cmake python3-argcomplete python3-pip libboost-system-dev build-essential
      pip3 install colcon-common-extensions
      apt get install libasound-dev portaudio19-dev libportaudio2 libportaudiocpp0
    2. Create a new directory for TurtleBot 3.
      mkdir -p ~/turtlebot3_ws/src && cd ~/turtlebot3_ws/src
    3. Clone the necessary repositories and then access TurtleBot Folder
      git clone -b dashing-devel https://github.com/ROBOTIS-GIT/hls_lfcd_lds_driver.git
      git clone -b dashing-devel https://github.com/ROBOTIS-GIT/turtlebot3_msgs.git
      git clone -b dashing-devel https://github.com/ROBOTIS-GIT/turtlebot3.git
      git clone -b dashing-devel https://github.com/ROBOTIS-GIT/DynamixelSDK.git
      cd ~/turtlebot3_ws/src/turtlebot3
    4. Remove the folders that are not required for the current project.
      rm -r turtlebot3_cartographer turtlebot3_navigation2
      cd ~/turtlebot3_ws/
    5. Source & Building the TurtleBot3 Setup file
      echo 'source /opt/ros/dashing/setup.bash' >> ~/.bashrc
      source ~/.bashrc
      colcon build --symlink-install --parallel-workers 1

      if you get the error
      use the following command

      pip install pytest-rerunfailures
      echo 'source ~/turtlebot3_ws/install/setup.bash' >> ~/.bashrc
      source ~/.bashrc
      echo 'export ROS_DOMAIN_ID=30 #TURTLEBOT3' >> ~/.bashrc
      echo 'export TURTLEBOT3_MODEL=burger' >> ~/.bashrc
      source ~/.bashrc
  2. Steps to Setup Common ML Packages:

    We need to set up python libraries such as numpy and opencv-python for our application to run. Instructions for setting up them are as follows:

    1. Setup opencv python
      pip3 install --upgrade pip
      Python3 -m pip install opencv-python
    2. Setup Numpy
      python3 -m pip install numpy
    3. Setup Sci-kit learn
      Apt install python3-sklearn
    4. Setup TensorFlow Lite runtime
      python3 -m pip install --index-url https://google-coral.github.io/py-repo/ tflite_runtime
  3. Steps to flash ROS2 firmware into OpenCR:

    The Default firmware supports ROS1. As we are using ROS Dashing (ROS2 version), we need to upgrade OpenCR firmware.

    1. Create a temp folder for Binaries
      mkdir /home/opencrbin/ && cd /home/opencrbin
    2. Download the latest binaries & unzip
      wget https://github.com/ROBOTIS-GIT/OpenCR-
      Binaries/raw/master/turtlebot3/ROS2/latest/opencr_update.tar.bz2
      tar -xjf ./opencr_update.tar.bz2
    3. Set the OpenCR port & TurtleBot Model. Before flashing, check whether the ttyACM0 port exists.
      export OPENCR_PORT=/dev/ttyACM0
      export OPENCR_MODEL=burger
    4. Upload the latest firmware by following command:
      dpkg --add-architecture armhf
      apt-get update
      apt-get install libc6:armhf
      cd /home/opencrbin/opencr_update && ./update.sh $OPENCR_PORT $OPENCR_MODEL.opencr

      You will see the following response on the terminal window of the Qualcomm Robotics RB5 development kit and hear the OpenCR board chime to confirm the update was successful.

  4. Steps to Setup USB Camera:

    1. Connect USB Camera to development kit.
    2. Attach the camera on Turtlebot3 Burger at 3rd layer from bottom, 20 degree facing down from vertical angle.

    Note: If you are using the build of October 2020 provided by Thundercomm, you may face the issue of coredump after attaching the USB Camera with mic. So before starting, stop pulse audio service.

  5. Steps to Setup LiDAR:

    1. Connect LIDAR Scanner to Qualcomm Robotics RB5 development kit using microUSB cable.
    2. After connection make sure the /dev/ttyUSB0 port is accessible.

Steps to Run Application

Environment setup instructions before running the application:

  1. In the terminal-1 run the TurtleBot Bringup Command
    ros2 launch turtlebot3_bringup robot.launch.py
  2. Open a second terminal, and we can launch our application from that. If you are using adb shell, don’t forget to run source ~/.bashrc command before running the application.

Steps to Run main application:


  • Go to the project directory
    cd <PROJECT_DIR_PATH> Generating Makefile using CMake tools
    cmake . Building the project
    make
  • Running the project
    ./WWD_Engine mic

Just say “Hey Globalite” and the Turtlebot starts driving. The demo is designed to store the video recording from the camera on the Qualcomm Robotics RB5 platform.

Qualcomm Robotics RB5 is a product of Qualcomm Technologies, Inc. and/or its subsidiaries.

NameTitle
Company
Rakesh Sankar
[email protected]
Sr, System Architect
Global Edge Software Ltd
Steven P
[email protected]
Project Manager
Global Edge Software Ltd
Ashish Tiwari
[email protected]
Architect
Global Edge Software Ltd
Arunraj A P
[email protected]
Software Engineer
Global Edge Software Ltd