Amazon Lex service on TurboX C610 Open Kit.

The main objective of this project is to get familiar with the Qualcomm® QCS610 platform, and then use the TurboX C610 Open kit to create a voice-based application. It is also designed to help developers get started with some AWS cloud services on the development kits.

About the Project
This project uses Amazon Lex service to create a voice-based application in order to access the camera feed, and to store the captured image in the cloud using Amazon s3 bucket service. In this project, boto3, the AWS SDK for Python is integrated with the TurboX C610 Open Kit. With boto3, one can invoke and run various Amazon cloud services on the QCS610 target board.

Dependencies

1. Ubuntu System 18.04 or above to function as a host system
2. Install adb tool (Android debugging bridge) on host system
3. Create an Amazon AWS user account
4. Install python3.5 or above on the host system

Prerequisites

1. Camera Environment configuration on QCS610 development kit

Configure the camera environment by following section 2.11.1 of the technical document- '[TC_C610LE_2320]_TurboX C610 OK_Software User Manual(for LE)_V2.0.pdf '

2. Create a Lex bot with intent and slot

Create a Lex bot by following the instructions from https://alexaworkshop.com/en/custom-skill/1.create-lex.html
Note: Make sure you have created the intent for "camera", "video" and "recognize". The response message of lexbot service needs to match with "capture camera", "record video", "recognize".
Once the setup lex bot is done, keep a note of the botName, bot alias, user-id.

3. Create the s3 bucket service on AWS website

In order to create the s3 bucket, follow the steps listed here - https://docs.aws.amazon.com/AmazonS3/latest/userguide/creating-bucket.html
Note: Once the bucket is created, keep a note of the bucket name.

4. Building the OpenCV library for the target board, TurboX C610

To install OpenCV library on the target board the required meta recipes for OpenCV are already present in folder “poky/meta-openembedded/meta-oe/ recipes-support/opencv/opencv_3.4.5.bb” file. Follow the below steps to build.

  1. Get into the yocto working directory

  2. Execute source command for environment setting

  The pop up menu will be open for available machines that select “qcs610-odk” and press ok. Then one more pop up window will be open for distribution selection in that we need to select “qti-distro-fullstack-perf”

  3. Run the bitbake command for installing packages.

  4. Once the build is complete the shared library and include file will be available in “./tmp-glibc/sysroots-components/armv7ahf-neon/opencv/usr”

  5. Push the opencv shared library to the target board

Note :

• For more references, refer to the “QCS610/QCS410 Linux Platform Development Kit Quick Start Guide document”.
• Also make sure to install the dependency library from the yocto build to the system (ex: libgphoto2, libv4l-utils)
• Recipes of above libraries are available inside meta-oe layer you can directly run bitbake command

5. Install boto3 library on board

To install the boto3 package on QCS610. You need to install the following python3 packages on yocto build.
Required package are,
  python3-boto3, python3-botocore, python3-jmespath, python3-s3transfer, html, date-util, multiprocessing, etc.

We can place the above mentioned bb recipe in the given folder name poky/poky/meta-openembedded/meta-python/python/. Meta recipes for these packages are available in the meta-recipe from source repository.

Afterwards, run the bitbake command for each library (steps are mentioned in previous section(OpenCV installation))

Once the build is complete for all recipes, create a libboto3 directory and copy all the required libraries to the folder, and push this same folder to /data/lex/ folder path of target board.

Steps to build and run the application:

1. Building the camera application:

Step-1: initialize the target board with root access.
$ adb root
$ adb remount
$ adb shell mount -o remount,rw /
Step-2: Push the source to the target board with adb command.
$ git clone https://github.com/globaledgesoft/AWS-Lex-C610
$ cd AWS-Lex-C610
$ adb push lex_inference.py /data/lex/
$ adb push config.py /data/lex/
$ adb push cascade/ /data/lex/
$ adb push recognizer/ /data/lex/
Note: In file config.py, you need to fill your aws security key details abs lex bot information

- aws_access_key_id,
- aws_secret_access_key
- region
- user-id
- botName
- botAlias and s3_bucket_name.

• For more information, please refer to this url https://docs.aws.amazon.com/powershell/latest/userguide/pstools-appendix-sign-up.html

2. Execute the python script in the target environment.

To start the application, run the below commands on the QCS610 board,
$ adb shell
/#
To enable Wi-Fi connectivity on target board
/# wpa_supplicant -Dnl80211 -iwlan0 -c /etc/misc/wifi/wpa_supplicant.conf -ddddt &
/# dhcpcd wlan0
Export the shared library to the LD_LIBRARY_PATH
/ # export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/data/lex/lib/
/# cd data/lex/
Step-4:Execute the python code on the target board,
/# python3 lex_inference.py -t "record the video"
It will start recording the video for a total duration of 5 sec.
/# python3 lex_inference.py -t "capture the image"
Camera will start to capture the image and store the same on s3 bucket.
/# python3 lex_inference.py -t "recognize the person"

• It will start to identify the person standing before the camera. Display it on the terminal.
• To see the captured image, you need to log into the s3 bucket console and download the image

Qualcomm QCS610 is a product of Qualcomm Technologies, Inc. and/or its subsidiaries.