Center Stage Light Show

Skill LevelArea of FocusOperating SystemCloud Services/PlatformPlatform/Hardware
IntermediateBluetooth, IoT, Embedded, Sensors, Smart CitiesLinux, RTOSAmazon AWS IoTDragonBoard 410c, QCA 402x WiFi/BLE/Zigbee

This project is designed to use the proximity sensor on the QCA4020 development board to gauge crowd engagement at a mock center stage of a music festival. This sensor value can be used to change color patterns on Bluetooth Low Energy (BLE) light bulbs to create an engaging atmosphere at the venue. The QCA4020 development board is also programmed to interface with the DragonBoard™ 410c from Arrow Electronics, which is designed to act as a smart gateway that runs a node.js based web application and notifies the event organizers when the crowd engagement drops.

The main objective of this project is to create an atmosphere at the center stage of a music festival that directly reflects the crowd engagement via an interactive lighting effect. Below are a few reasons why QCA4020 is ideal for use in IoT devices.

  • QCA4020 offers tri-mode radios, combining Wi-Fi, BLE 5 and 802.15.4 based technologies, including Zigbee and Thread
  • Cortex M4F for application processing, Cortex M0 for network stack processing, and a separate processor for Wi-Fi stack designed to provide a highly concurrent multiple radio solution
  • Supports advanced hardware-based security features to help improve device protection that include secure boot, hardware root of trust, trusted execution environment, hardware crypto engines, storage security, debug security with lifecycle control, key provisioning and wireless protocol security
  • Pre-integrated software for multiple IoT ecosystems (such as HomeKit and OCF specifications), the AWS IoT SDK, and the Microsoft Azure IoT Device SDK, which connects devices with the Azure IoT Hub

The desired outcome is to provide the event organizer with the data collected on crowd engagement using intelligent connectivity solutions such as QCA4020 multi-mode connectivity SoC.

Downloading the source code form GitHub

Download the source from the GitHub repository link mentioned above.

The repository has code for the DragonBoard410c and the QCA4020 development board.

The code for QCA4020 is under Center-Stage/QCA4020_sdk/target/ It has the same folder structure as the QCA4020 SDK and has all the files and tools that are needed to build and flash the application. The actual source for the QCA4020 application can be found at the following location.

Center-Stage/QCA4020_sdk/target/quartz/demo/ Music_Demo2/

Below changes are made to the SDK files to optimize the application.

Setting up the QCA4020 board

By default, the QCA4020 board only pairs with 4 BLE devices. For this project, we want the board to change color patterns on 5 BLE bulbs. You will need to change the NVM configuration at the following location.

<sdk_path>\target\quartz\nvm\config\2.0\4020\CDB\QCA4020_2p0.nvm

Change the “Maximum simultaneous BLE connections (byte 4)” in “BLE size parameters (Tag Number:1)” from 0x04 to 0x05. For more information on NVM configuration file parameters, please refer to the programmer’s guide.

We can also optimize the application by resizing the application memory.

QCA4020 has 328KB of RAM dedicated for customer applications. In addition to this, you can choose to place your code in XIP region. You can change the distribution between code and data RAM in 128KB increments.

The default distribution of application data and code RAM is as follows.

Code RAM: 232KB (RAM_FOM_APPS_RO_MEMORY region)
Data RAM: 64KB (RAM_FOM_APPS_DATA_MEMORY region)

To increase data memory (RAM_FOM_APPS_DATA_MEMORY) and decrease the code memory (RAM_FOM_APPS_RO_MEMORY) by the same amount. Make the following changes in “DefaultTemplateLinkerScript.ld” script.

Change the following two lines.

RAM_FOM_APPS_RO_MEMORY (Rx) : ORIGIN = 0x10046000, LENGTH = 0x3a000 RAM_FOM_APPS_DATA_MEMORY (W) : ORIGIN = 0x10080000, LENGTH = 0x10000toRAM_FOM_APPS_RO_MEMORY (Rx) : ORIGIN = 0x10046000, LENGTH = 0x1a000 RAM_FOM_APPS_DATA_MEMORY (W) : ORIGIN = 0x10060000, LENGTH = 0x30000

By default, the Data Execution Prevention (DEP) is enabled on QCA4020. To make changes to code and data memory regions, adjust the DEP configuration. Modify DevCfg_master_devcfg_out.xml file to adjust DEP configuration region. Application RAM regions (RAM_FOM_APPS_DATA_MEMORY, RAM_FOM_APPS_RO_MEMORY) belong to MPU region 2 (row highlighted in the code snippet)

<!-- All the data is in Little Endian Format -->
<!-- FORMAT: -->
<!-- DEP_region_start_address - 4 bytes -->
<!-- DEP_region_size - 4 bytes -->
<!-- DEP_region_index - 1 byte -->
<!-- DEP_sub_region_mask - 1 byte (set bit to '1' for disabling the sub-region) -->
<!-- DEP_access_control - 1 byte (0x6 for RO, 0x3 for RW) -->
<!-- XN - 1 byte -->
<!-- 64MB ROM region = --> 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x06, 0x00,
<!-- 1MB RAM region = --> 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x10, 0x00, 0x01, 0x00, 0x03, 0x01,
<!-- FOM Code and Data region = --> 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x10, 0x00, 0x02, 0xF1, 0x06, 0x00,
<!-- FOM APPS region = --> 0x00, 0x00, 0x08, 0x10, 0x00, 0x00, 0x01, 0x00, 0x03, 0xFF, 0x06, 0x00,
<!-- SOM Code and Data region = --> 0x00, 0x40, 0x00, 0x10, 0x00, 0x40, 0x00, 0x00, 0x04, 0x01, 0x06, 0x00,
<!-- SOM APPS and HEAP = --> 0x00, 0x80, 0x00, 0x10, 0x00, 0x80, 0x00, 0x00, 0x05, 0xE0, 0x06, 0x00,
<!-- MOM region = --> 0x00, 0x10, 0x00, 0x10, 0x00, 0x10, 0x00, 0x00, 0x06, 0xC7, 0x06, 0x00,
<!-- SBL region = --> 0x00, 0x00, 0x0A, 0x10, 0x00, 0x00, 0x01, 0x00, 0x07, 0x00, 0x3, 0x01,

Every 128 KB block that needs to be changed requires corresponding adjustment to DEP_sub_region_mask byte (highlighted in red). Each bit in DEP_sub_region_mask corresponds to one 128KB block. Setting a bit changes the block from code to data. Example: changing it to 0xF9 moves 128KB from RO to DATA, 0xFD moves 256KB from RO to DATA and so on. Heap starts at the end of APP data region

Once the changes are made, build and compile the application by following the instructions.

Change to the application working directory

cd <sdk_path>\target\quartz\demo\Music_Demo2\build\gcc>

Build the application

build.bat

Flash the application using Open OCD using the following command.

flashopenocd.bat x 4020 cdb

For more information on how to set up your OpenOCD environment, please follow the instructions listed on the getting started page.

Configuring Bulbs

  • Install Playbulb mobile app (any Android or IOS devices can be used).
  • Opening the app will list the bulbs.
  • Connect to each bulb and by clicking on that bulb will move to next screen.
  • Click settings icon (gear icon) will show the Product Rename.
  • Enter “PIR-20-MSCD” and click “Rename Confirm” button to change the name.
  • Repeat this step for all the 5 bulbs.

QCA4020 application details

So how does the code on QCA4020 work? For starters, you will need a jumper on pins 1 and 2 of J25 on the QCA4020 development board.

Jumper Setting on the QCA4020 development board.

Whenever there is a movement on PIR sensor, the J25 (PIR_OUT_LR) which is configured as DUAL_EDGE_TRIGGER will send an Interrupt signal. The code utilizes the number of interrupts as an indicator for crowd engagement.

Below snippet in <sdk_path>\target2.0\quartz\demo\Music_Demo2\src\sensors\sensors.c changes the color pattern based on the number of interrupts.

if (sig & PIR_THREAD_SIGNAL_INTR)
		{
			QCLI_Printf(qcli_sensors_group, "PIR sensor detected motion = %d\n", (motion_cnt + 1));
      motion_cnt++;

			if((motion_cnt > motion_frequency_threshold * 2) && (motion_announced < 2))
			{
				mot_rate = RAINBOW_FAST; 
				QCLI_Printf(qcli_sensors_group, "Fast motion detected : *****\n");
				mscd_write_callback();
				motion_announced = 2;
			}
			else if((motion_cnt <= (motion_frequency_threshold * 2)) && (motion_cnt >= motion_frequency_threshold) && !motion_announced)
			{
				mot_rate = RAINBOW_FAST;
				QCLI_Printf(qcli_sensors_group, "Mediam motion detected : **\n");
				mscd_write_callback();
				motion_announced = 1;
			}	
		}

The function mscd_write_motion_data() in <sdk_path>\target2.0\quartz\demo\Music_Demo2\src\spple\spple_demo.c sends the color patterns to all connected BLE bulbs.

void mscd_write_motion_data()
{
   uint16_t char_handle;
   int attr_len;
   int Result;
   int deviceIndex;
   char* val;
   DeviceInfo_t *DeviceInfo;

   val = mot_rate_func();
   attr_len = 8;
   //QCLI_Printf(ble_group, "Value = %s\n", val);
   for(deviceIndex = 0; deviceIndex < MSCD_NUM_BULBS; deviceIndex++)
   {
      if((DeviceInfo = MSCDGetDeviceInfo(deviceIndex)))
      {
         char_handle = mscd_get_write_char_handle(deviceIndex);
         //add delay here?
         Sleep(750);
         if(char_handle)
         {
            if((Result = qapi_BLE_GATT_Write_Without_Response_Request(BluetoothStackID, 
               DeviceInfo->ConnectionID, char_handle, attr_len, val)) > 0)
            {
               QCLI_Printf(ble_group, "mscd_write_motion_data write success = %u\n", Result);
            }
            else if (Result == QAPI_BLE_GATT_ERROR_INVALID_CONNECTION_ID)
            {
               BoardStr_t                   BoardStr;
               BD_ADDRToStr(DeviceInfo->RemoteAddress, BoardStr);
               QCLI_Printf(ble_group, "addr = %s\n", BoardStr);
               DisplayFunctionError("qapi_BLE_GATT_Write_Request", Result);
               mscd_reset_device_data(deviceIndex);
            }
            else
            {
               DisplayFunctionError("qapi_BLE_GATT_Write_Request", Result);
               QCLI_Printf(ble_group, "Conn_id = %d", DeviceInfo->ConnectionID);
            }
         }
      }
   }
}

Setting up the DragonBoard 410c

Install NodeJS

Execute the following command in the command line to setup the NodeJS base version.

$> curl -sL https://deb.nodesource.com/setup_9.x | sudo -E bash –

Install NodeJS by executing the following command.

$> sudo apt-get install nodejs

Install NodeJS process Manager

Install process manager module by executing the following command

$> npm install pm2 -g

Setup Bluetooth Speaker

Execute the following command in Linaro terminal to turn on the Bluetooth Control.

$> bluetoothctl

The command prompt will turn to bluetoothctl. Now, scan for the speaker.

[bluetoothctl]# scan on

Which will list something like below.

Scanning Bluetooth speakers on DragonBoard 410c

If you find your device, execute the following command to stop scanning. If not, it will continue to scan.

[bluetooth]#scan off

Execute the following command to connect your device (speaker) from the board.

[bluetooth]#connect <Mac Address of the device>Eg. [bluetooth]#connect aa:bb:cc:dd:ee:ff

If connection is success. You will see “Connection Successful” message.

Also, need to route the audio to Bluetooth channel using following steps.

Start -> Sound & Video -> Pulseaudio Volume Control which opens the below application.

Selecting Bluetooth speakers as the default output.

Please, select the device as shown in the above image to route the audio to the selected device.

Install Dependencies

Run the setup script to install the dependencies. Please, find the script in the application’s root path. (~/demo2/)

$> ./setup.sh

Configuration Changes

Below is the Configuration which you can find in the below path open it if you want to edit it.

~/demo2/qca-iiot-music-festival/iiot-music-festival/config/config.json

It contains 3 parts

  • SERIAL_COMM
  • AWS_IOT_CLIENT
  • APP_DB_CONFIG
{
        "SERIAL_COMM":{
                "serial_port":"/dev/ttyUSB1",
                "start_4020_app":"4 4 1",
                "start_music":"4 4 3",
                "stop_music": "4 4 2",
                "duration_in_sec" : "3",
                "frequency_threshold" : "3"
        },
        "AWS_IOT_CLIENT":{
                "host" : "abcxyzpqr123.iot.eu-west-X.amazonaws.com",
                "rootCAPath" : "root-CA.crt",
                "certificatePath" : "QCA4020_MUSIC_FTV.cert.pem",
                "privateKeyPath" : "QCA4020_MUSIC_FTV.private.key",
                "clientId" : "QCA_Music_410c_Client",
                "topic" : "pir_timestamp",
                "mode" : "publish"
        },
        "APP_DB_CONFIG" :{
                "dbConfig" : {
                        "dialect": "sqlite3",
                        "schemaName": "music_festival",
                        "connection": {
                                "filename": "./demo2.db"
                        }
                },
                "port" : "8000"
        }

We need to change the following things,

  • serial_port - usb port name which we noted in earlier step.
  • duration_in_sec – duration to observe number of movements.
  • frequency_threshold – number of movements detected by the PIR sensor. LED Bulb patterns will change based on this number.

Note: If the USB cable port is changed, you will need to restart the QCA4020 board and make sure the serial_port configuration is updated with correct USB port name. Once the setting is changed, restart the NodeJS application.

Adding Songs

Songs are stored in the below mentioned folder path

~/demo2/qca-iiot-music-festival/songs/

You will see 4 folders

  1. edm
  2. hiphop
  3. pop
  4. rock

Please, add one or more songs with mp3 format. The application will shuffle among the songs and it will play.

Launching the applications.

1.1. Make the hardware connections.

  • Mount the QCA4020 board on a stand
  • Make sure the power switch is set to off (towards USB connection)

QCA4020 Development Board switch in Off position.

1.2. Setup DragonBoard 410c connections:

  • Plug in USB 4-port hub to the DragonBoard 410c USB port at the far edge of the board
  • Connect the Touch Screen HDMI cable, plug the USB touch screen cable into the USB hub, and power on the monitor
  • Connect USB to Ethernet dongle, and connect ethernet cable
  • Plug in wireless keyboard USB dongle and power on the keyboard
  • Do not connect power to the DragonBoard410c yet

1.3. Setup QCA4020 connections:
There are two USB to Micro-USB cables with the QCA4020 kit. Connect both as below:

Micro USB connector on QCA4020USB connection on DragonBoard 410c
J6 (just below the power switch)Any port on the USB extension hub.
J85 (below the J6)USB port closest to the High Speed Expansion header.

1.4. Plug in DragonBoard 410c power cord and power on

1.5. Power on the Bluetooth speaker and make sure it is in BLE discovery mode, meaning it is available to connect to the DragonBoard 410c

1.6. On the DragonBoard 410c desktop, open a terminal window:
Left click on the icon in the lower left status bar, then System Tools -> UXTerm:

Open Terminal Emulator on DragonBoard 410c.

1.7. Enter the following commands in sequence:

bluetoothctl

(note: last character is L, not 1, as in bluetoothctl)

For the JBL speakers, use the following command

connect aa:bb:cc:dd:ee:ff

Once the Bluetooth is connected, type the command exit and hit the return key.

1.8. Turn on the QCA-4020 board (flip the power switch up)

Turn QCA4020 Development Board ON.

1.9. In the DragonBoard 410c Terminal window, enter the following commands:

cd demo2/qca-iiot-music-festival/iiot-music-festival/
python launch.py

Some commands will start to run for several seconds, and then a Chrome browser window will open.

1.10. Go to Chrome Menu and set to Full Screen (shortcut: F11 on keyboard)

To run through the demo sequence:

  1. Tap anywhere on the “Welcome” screen to start
  2. Tap Yes to help the DJ out
  3. Tap on a music genre
  4. Move around (or just move your hand) in front of the QCA4020 motion sensor.
    PIR Sensor on the QCA4020 Development Board.
  5. After 30 seconds, the interaction ends and the demo resets to the Welcome screen.

You can also tap on the Dashboard button (bottom left) to see metrics from AWS IoT for music genres selected and activity levels measured

NameCompany
Prathyusha YGlobalEdge Software
P N RamyaGlobalEdge Software
Steven PGlobalEdge Software