QCA4020 Modern UI Application Development with the Moddable SDK

Skill LevelArea of FocusOperating SystemSoftware Tools
IntermediateEmbedded, IoTRTOSQCA 402x Wi-Fi/BLE/ZigBee

This project introduces you to UI Application Development in JavaScript using the Moddable SDK on the QCA4020 System-On-Chip (SoC) Product Development Kit. The Moddable SDK is a combination of development tools and runtime software to create applications for embedded devices. The QCA4020 IoT solution offers Wi-Fi, Bluetooth low energy (BLE), and 802.15.4 capable radios in a single-chip package. This project showcases the weather example app, a JavaScript app from the Moddable SDK that issues HTTP requests to the OpenWeatherMap REST API and displays the weather forecast for various U.S. cities on a color 240 x 320 LCD. The application is built with the Moddable SDK Piu user interface framework, allowing developers to design and implement user interfaces with smooth, flicker-free animations and beautiful font rendering on resource-constrained SoC and microcontroller platforms.

For a detailed explanation of how to issue HTTP client requests to REST APIs with the Moddable SDK, please check out our QCA4020 HTTP client in JavaScript using the Moddable SDK project post here on the Qualcomm Developer Network.

The "weather" application issues HTTP requests to the OpenWeatherMap REST API to retrieve the current weather in five different U.S. cities. Each forecast is displayed on a SPI LCD driven by our integrated ILI9341 MIPI-compatible display driver. In this project you will learn:

  • How to use the Moddable SDK build tools to build and run applications on the QCA4020 SoC Product Development Kit
  • How to attach a SPI LCD that can be driven by our integrated ILI9341 MIPI-compatible display driver
  • How to integrate font and texture assets, define the UI layout, build timeline animations, screen transitions and render text using Piu
  • How the weather application uses Piu to display each weather forecast and implement time-based animation features
  • How to use xsbug, the JavaScript source-level debugger from the Moddable SDK

Note: Piu is a robust user interface application framework. This project highlights just a few of the available APIs and classes. Please refer to the comprehensive Piu reference document to learn more about all the available features. We also encourage you to explore our collection of Piu example apps, to jumpstart your Moddable UI app development.

The contents of the QCA4020 Development kit, as seen above, include:

Follow the one-time setup instructions in the Moddable SDK Getting started with QCA4020 development document to complete the following steps:

  • Linux Host Environment Setup. Moddable supports FreeRTOS on the QCA4020 Customer Development Board (CDB) using a Linux host build platform.
  • QCA4020 SDK Setup, including FreeRTOS, QuRT, and OpenOCD
  • ARM Toolchain Setup
  • Moddable Host Tools Build

QCA4020 CDB20 SPI display wiring for ILI9341 display driver

The Moddable SDK includes integrated support for the popular ILI9341 display driver. The following pinout table and image show the connections required to support a SPI display on the CDB20 board. This project assumes the CDB20 is driving a 240 x 320 SPI LCD.

Screen pinCDB20 header / pin numberGPIO
SDO (MISO)J5 / 827
LED (3.3V)J1 / 4N/A
SCKJ5 / 425
SDI (MOSI)J5 / 626
RESETJ5 / 1913
CSJ5 / 224
DCJ5 / 811
GNDJ5 / 40N/A
VCCJ1 / 4N/A

Once you have completed the setup instructions above and connected the display, you are ready to build the weather example app, following the instructions in the next section.

Usage Instructions

In this section we learn how to build and run the Moddable weather example app on the QCA4020 CDB.

Building the weather app for the QCA4020 CDB

  1. Open a terminal window on the Linux build host, navigate to the weather example app directory, and build the app:

    cd $MODDABLE/examples/piu/weather
    mcconfig -d -m -p qca4020/cdb ssid=<your SSID> password=<your password>
    

    The $MODDABLE environment variable was initialized and mcconfig command line host tool built as part of the Moddable SDK setup described above. In the mcconfig command line you specify the Wi-Fi credentials corresponding to the Wi-Fi access point you'd like the CDB to connect to. The Moddable runtime automatically connects the CDB to this access point before running the app.

    The mcconfig tool builds the Moddable XS runtime, application, modules, and any assets into an archive that will be included in the final build.

  2. Open another terminal window, navigate to the project build directory, and build the QCA4020 launcher application. The launcher application includes the archive built above and QCA4020 libraries:

    cd $MODDABLE/build/devices/qca4020/xsProj/build/gcc
    APP_NAME=weather DEBUG=1 make
    
  3. Navigate to the project build directory and flash the built application to the device:

    cd $MODDABLE/build/devices/qca4020/xsProj/build/gcc
    sh flash_openocd.sh
    
  4. Open another terminal window and launch the xsbug debugger and serial2xsbug tool. Both the xsbug debugger and serial2xsbug tool were built as part of the Moddable SDK setup. The xsbug debugger is used to debug the JavaScript portion of the application and monitor resources.

    xsbug &
    serial2xsbug /dev/ttyUSB1 115200 8N1
    

    The image below shows the xsbug window.

  1. Navigate to the project build directory and run gdb to launch the app:

    cd $MODDABLE/build/devices/qca4020/xsProj/build/gcc
    arm-none-eabi-gdb -x v2/quartzcdb.gdbinit
    

    At the (gdb) prompt type c to continue three times to launch the weather app. You can also set breakpoints, view source and variables, etc... The application will connect to the specified Wi-Fi access point, issue the first HTTP request and display the weather condition on the LCD. The application requests each subsequent weather forecast every five seconds, in round robin fashion, using animations and screen transitions to update the display. This video shows the application running on a 240 x 320 screen.

Building the weather app for the Desktop Simulator

The Moddable SDK includes a desktop simulator that can be used to develop and debug many Moddable apps without the QCA4020 CDB. The simulator can significantly streamline UI development, since the desktop builds do not require the native code make or flash steps, which can take a fair amount of time. Desktop builds complete and launch in a matter of seconds. Only two steps are required to build and launch the weather app for the simulator:

  1. Open a terminal window on the Linux build host and launch the xsbug debugger.

    xsbug &
    
  2. In the same terminal window, navigate to the weather example app directory, and build the app:

    cd $MODDABLE/examples/piu/weather
    mcconfig -d -m
    

The mcconfig tool automatically opens the desktop simulator and launches the app. Note that all Moddable applications are built and run following the same basic steps outlined above.

Application Walk-Through

In this section we take a closer look at how the weather JavaScript application incorporates assets and leverages the Piu user interface framework to layout, display and animate the weather forecast screens.

Assets

Assets for UI applications typically include fonts for rendering text and PNG images for UI elements. The weather app includes two fonts for displaying the city name and temperature. In addition, a PNG image strip includes icons corresponding to common weather conditions. The Moddable command line tools automatically compress fonts and process graphics for optimal rendering on the QCA4020 device. Fonts and graphics are rendered directly from Flash memory (XIP), freeing RAM for application use.

The resources section in the application manifest.json file defines the assets required by the app:

"resources":{
    "*-alpha": [
        "./assets/icons-alpha",
        "$(MODDABLE)/examples/assets/fonts/OpenSans-Regular-52",
        "$(MODDABLE)/examples/assets/fonts/OpenSans-Semibold-28",
    ],
}

The "*-alpha" manifest target instructs the build tools to convert the PNG image and fonts to memory efficient alpha bitmaps that can be tinted at render time. An Array of path names references each asset bundled into the application.

Font Assets

The Moddable SDK supports the BMFont format for embedded fonts. Font assets can be generated with non-contiguous glyph ranges and include only the characters required by the application, which can significantly reduce asset storage size. Further, anti-aliased fonts are supported and any character with a Unicode representation can be rendered, making it possible to render any glyph in any language.

The Moddable team uses Glyph Designer to create bitmap fonts in the BMFont format. There are other font editing tools available. The Moddable build tools convert the font glyph PNG file into a black mask BMP file for rendering. The image below shows the BMP file generated from the OpenSans 52 point font used in this application.

Integrating Font Assets

Fonts are wrapped by Style objects in Piu apps. An application sets the font property to the name of the FNT file generated by the BMFont tool. Other properties can be specified, including color, margins, alignment and so on. The weather app creates two Style objects, corresponding to each of the two fonts used:

const OpenSans52 = new Style({ font: "52px Open Sans" });
const OpenSans28 = new Style({ font: "semibold 28px Open Sans", color: WHITE });

Image Assets

The weather app includes a single PNG asset file that contains icon images corresponding to weather conditions. The icon images are arranged in a horizontal strip. Each icon is 150 pixels square:

As with the fonts, the PNG image strip is converted at build-time to a 1-bit mask graphic for rendering. Transparency is preserved at runtime, allowing application developers and graphic designers to build assets using the wide variety of authoring tools that support the PNG image file format.

Integrating Image Assets

Images are wrapped by Texture objects in Piu apps. The Texture simply references the original asset file name:

const iconTexture = new Texture("icons-alpha.png");

Textures are typically used in conjunction with Skin objects. A Skin fills a rectangular area with a color or a portion of an image. The weather app defines an iconSkin that fills an area with a portion of the iconTexture:

const iconSkin = new Skin({ 
    texture: iconTexture, 
    color: [BLACK, WHITE], 
    x: 0, y: 0, width: 150, height: 150, variants: 150
});

The Skin defined above renders a 150 x 150 square portion of the iconTexture image. The variants property defines the horizontal offset between each icon in the image strip. The icons-alpha.png asset shown above contains eight variants, i.e. eight individual icon images. The weather app specifies the variant to draw based on the weather condition.

The color property in a Skin can reference a single color or Array of colors. When an Array is provided, Piu will render a blended color based on the colors in the Array. We will cover additional rendering details in the discussion of animations below.

UI Layout

Graphical user interface elements in Piu applications are comprised of a hierarchy of content objects. Each content object corresponds to a rectangular screen area. The built-in application content object serves as the root of the container hierarchy.

An application defines UI layouts for each screen using content templates. The templates define the initial location, size and various properties of each content object. These templates define the container hierarchy that make up both individual UI elements and complete screens. Here is the template corresponding to the main weather app screen:

const MainCol = Column.template($ => ({
    left:0, top:0, right: 0, bottom: 0, skin: blackSkin,
    contents:[
        new AnimatedText({ hue: 0, string: $.city, style: OpenSans28, }, { name: "city", top: 15, height: 38, left: 240,  }),
        new AnimatedText({ hue: 90, string: $.temp, style: OpenSans52, }, { name: "temp", top: -4, height: 58, left: -240 }),
        Content($, { name: "icon", width: 150, top: 10, height:150, skin: iconSkin, variant: $.icon }),
        new AnimatedText({ hue: 270, string: $.condition, style: OpenSans28, }, { name: "condition", top: 3, height: 38, }),   
    ],
    Behavior: MainColBehavior
}));

The template defines a vertical column named MainCol, which uses the Piu Column object to layout the contained contents vertically in a single column. The MainCol content fills its container by setting the left, right, top, and bottom coordinates to 0. These coordinates are offsets from the parent container's edges. Each of the items in the contents array corresponds to a single UI element. The text elements include a style property that defines the font used when rendering the text. The Content corresponding to the weather condition icon includes properties to specify the skin used to fill the content. The property values with a $ sign indicate that the corresponding value is initialized from the data that is provided when instantiating the template for display. For example, $.city corresponds to the city name string returned from the HTTP request and passed as instantiating data to the template.

Piu supports a variety of built-in content object types, e.g. Label, Scroller, Row, etc... you can use to simplify UI application development by leveraging the pre-defined behaviors. Applications can also define templates that extend built-in content object types. The AnimatedText content is an application-defined template built on the Piu Port object. A Port object is useful when your application requires precise control over how the content is rendered:

const AnimatedText = Port.template($ => ({
    width:240,
    Behavior: AnimatedTextBehavior
}));

Each content can include an optional name property. The name property helps to make the container hierarchy easier to read and follow. Further, applications can also reference a content by name. The illustration below shows the MainCol elements referenced by name:

The weather application launches to a black screen that displays a Loading... message. The corresponding container layout uses the Application.template because this screen is displayed by the application object at launch, The layout uses a single Label object to display the message centered on the screen:

const WeatherApp = Application.template($ => ({
    left: 0, right: 0, top: 0, bottom: 0, skin: blackSkin, 
    contents: [
        Label($, {
            left: 0, right: 0, top: 0, bottom: 0, 
            skin: blackSkin, style: OpenSans28, string: "Loading...",
        }),
    ]
}));

When an HTTP weather request completes, the application "empties" the container hierarchy and adds a new instance of the MainCol template, passing the instantiating data:

onAddNextScreen(application) {
    application.empty();
    application.add(new MainCol(this.data));
}

UI Behaviors and Events

A Behavior object contains functions corresponding to events that get triggered by the Piu application framework or host application. Any content object can assign a behavior object that can be extended by the application. Behavior functions are used to initialize content properties or modify properties based on time or other event triggers. The weather app instantiates the MainColBehavior to manage MainCol container events. In the MainCol behavior, the built-in onCreate() function is called by Piu when a content is initially created. The onDisplaying() function is called by Piu after a content is created but before the content is displayed on the screen. All built-in behavior functions are passed the associated content as the first parameter:

class MainColBehavior extends Behavior {
    onCreate(column, data) {
        this.data = data;
        this.transitioningIn = 1;
    }
    onDisplaying(column) {
        this.onTransitionIn(column);
    }
    onTransitionIn(column) {
        this.timeline = (new Timeline)
    ...

In the application code above, the onCreate() function stores the instantiating data in the behavior instance and sets a local transitioningIn variable to 1. The onDisplaying() function calls an application-defined onTransitionIn() function to create the animation timeline used to transition the MainCol UI elements onto the screen.

A behavior effectively manages content events and separates the logic associated with applications from the look and feel, i.e. container hierarchy. Moddable app developers often build common behaviors that are used across different applications.

The weather application defines three behaviors:

  • The WeatherAppBehavior is the application behavior. This behavior issues the HTTP weather requests and triggers an application-defined onTransitionOut event when a response is received. The onTransitionOut event starts the transition that removes the current weather UI elements from the screen.
  • The MainColBehavior is the weather screen behavior. This behavior animates the UI weather elements on and off the screen. When the elements are transitioned off the screen, the onFinished behavior function is called and triggers the application-defined onAddNextScreen event, which sets up the next weather condition for display.
  • The AnimatedTextBehavior animates the color hue of the city, temperature and current condition UI text elements.

Content Clocks

Every content in the containment hierarchy can be used as a clock to trigger behavior events after the specified duration has completed or at a specified time interval. For example, the WeatherAppBehavior uses a content clock to issue HTTP requests once every five seconds.

Content clocks are also the simplest way to drive time-based animations. All the animations in the weather app are driven by content clocks. In this section we take a closer look at how the AnimatedTextBehavior uses a content clock to animate the text color hue.

Initialization

The onDisplaying() function is called after the content is created and added to the container hierarchy, but before the content is displayed. Here the AnimatedTextBehavior initializes instance variables used when the content is drawn and sets the animation interval to 50 milliseconds. Finally, the call to start() starts the animation clock. Because no animation duration was specified, this clock runs until the content (port) is removed from the container hierarchy:

onDisplaying(port) {
    this.labelSize = port.measureString(this.string, this.style);
    this.leftMargin = (port.width - this.labelSize.width)/2;
    port.interval = 50;
    port.start();
}

Updates

After the content clock is started, the onTimeChanged() behavior function is called once every 50 milliseconds. The onTimeChanged() function simply invalidates the port, which is required in order to trigger the onDraw() function:

onTimeChanged(port) {
    port.invalidate();
}

The onDraw() function generates a hexidecimal color based on the current hue value. The global hsl() function is a Piu helper function that converts a HSL color to a hexidecimal value. The text string is then drawn, centered in the port bounds, using the generated color and text style provided by the instantiating data. The hue value is then incremented for the next update. This function animates the color hue by increasing the hue value by one every animation interval:

onDraw(port, x, y, w, h) {
    let color = hsl(this.hue, 1, 0.75);
    port.drawString(this.string, this.style, color, this.leftMargin, 0, this.labelSize.width, port.height);
    this.hue++;
}

Each content instantiates a unique behavior instance. In this example an instance of AnimatedTextBehavior is created for each AnimatedText content. This application provides instantiating data for each AnimatedText content in the MainCol template. For example, the initial hue value of the "temp" content is 90:

new AnimatedText({ hue: 90, string: $.temp, style: OpenSans52, }, { name: "temp", top: -4, height: 58, left: -240 }),

This same basic technique can be used to animate any content property. For example, we can modify the AnimatedTextBehavior to blink the content at a fixed interval by animating the content visibility:

class AnimatedTextBehavior extends Behavior {
    onCreate(port, data) {
        this.string = data.string;
        this.style = data.style;
        this.color = hsl(data.hue, 1, 0.75);
    }
    onDisplaying(port) {
        this.labelSize = port.measureString(this.string, this.style);
        this.leftMargin = (port.width - this.labelSize.width)/2;
        port.interval = 50;
        port.start();
    }
    onDraw(port, x, y, w, h) {
        port.drawString(this.string, this.style, this.color, this.leftMargin, 0, this.labelSize.width, port.height);
    }
    onTimeChanged(port) {
        port.visible = !port.visible;
    }
};

And this video shows the application running with the text blinking animation described above. Note that changing the visible property on the Port content automatically triggers the onDraw() function.

Timeline Animations

In the previous section we learned how to use a content clock to animate a single UI element. Piu also provides a timeline object for animating a collection of elements/tweens. Timelines are often used to drive screen transitions. The weather app uses timelines to animate the weather forecast elements on and off the screen. In this section we learn how the onTransitionIn() function builds and runs the timeline animation that transitions the next weather forecast onto the screen.

Note: Refer to the Timeline Object section in the Piu reference document for a detailed description of timeline features.

Creation

The onTransitionIn() function is called by the MainColBehavior when the weather forecast UI elements are about to be displayed. The function creates a timeline that specifies individual tweening behaviors for the "city", "temp", "condition" and "icon" elements:

onTransitionIn(column) {
    this.timeline = (new Timeline)
        .to(column.content("city"), { x:0 }, 750, Math.backEaseOut, 0)
        .to(column.content("temp"), { x:0 }, 750, Math.backEaseOut, -500)
        .from(column.content("condition"), { y: 320 }, 400, Math.quadEaseOut, -750)
        .to(column.content("icon"), { state: 1 }, 500, Math.quadEaseOut, -250)
    column.duration = this.timeline.duration;
    column.time = 0;
    this.timeline.seekTo(0);
    column.start();
}

This timeline uses two types of tweens. A "to tween" animates the content properties from their original values to the specified values. A "from tween" animates the content properties from the specified values to the original values. The specified values are defined as a property list, passed to the to and from functions. Let's take a closer look at the "city" tween to better understand tween property settings:

    .to(column.content("city"), { x:0 }, 750, Math.backEaseOut, 0)
  • This is a "to tween" that animates the "city" content
  • The tween animates the content's x property. At the end of the tween, the x property value will be set to 0
  • The tween duration is 750 milliseconds
  • The tween uses a backEaseOut easing function
  • The tween starts immediately at time 0 (no delay)

The easing function deserves special attention. Piu supports Robert Penner’s open source easing equations (see the Easing section of Robert Penner’s Programming Macromedia Flash MX). When an easing function is specified, the running timeline maps the current time through the easing function, yielding animations that feel more natural.

Timing

Each tween in a timeline is configured separately and specifies its own duration. The delay parameter, used to control the sequencing of each tween in the collection, specifies the number of milliseconds to delay after the previous tween completes before starting. Setting the tween delay to 0 causes the tween to start immediately after the prior tween completes. A negative delay value starts a tween before the previous tween completes. This timeline starts each successive tween before the previous tween completes:

.to(column.content("city"), { x:0 }, 750, Math.backEaseOut, 0)
.to(column.content("temp"), { x:0 }, 750, Math.backEaseOut, -500)
.from(column.content("condition"), { y: 320 }, 400, Math.quadEaseOut, -750)
.to(column.content("icon"), { state: 1 }, 500, Math.quadEaseOut, -250)
  • The "city" tween runs for 750 ms and starts immediately
  • The "temp" tween runs for 750 ms and starts after the "city" tween has run for 250 ms
  • The "condition" tween runs for 400 ms and starts at the same time as the "temp" tween
  • The "icon" tween runs for 500 ms and starts after the "condition" tween has run for 150 ms

The duration and delay parameters provide UI animation developers complete control over the animation timing.

After the timeline is instantiated, the application uses a content clock to start the timeline animation:

column.duration = this.timeline.duration;
column.time = 0;
this.timeline.seekTo(0);
column.start();

The timeline automatically calculates the aggregate animation duration based on the individual tween durations and delays. The application sets the content duration, seeks to the beginning of the timeline and starts the animation.

Updates

The application animates all the tweens in a timeline simultaneously by seeking to a specific point in the timeline. The onTimeChanged() behavior function is called when the time changes in a content clock that has a specified duration:

onTimeChanged(column) {
    let time = column.time;
    this.timeline.seekTo(time);
}

The time corresponds to the current time of the animation in ms, relative to the animation duration. All the magic happens in the seekTo() function, which seeks each of the tweens to the specified time and sets each of the tween's object property values.

Conclusion

Congratulations! By following the steps outlined in this project, you've built, deployed and run a JavaScript app on the QCA4020/CDB that displays weather conditions delivered over HTTP, on an LCD, animating both text and graphics. You've also learned how quick and efficient embedded development can be in JavaScript using the Moddable SDK runtime software and development tools.

Further Reading

Availability

The Moddable SDK with Qualcomm QCA4020 support is available immediately as part of the Moddable SDK repository on GitHub. The Moddable SDK is available for use with either a FOSS (Free and Open Source Software) license or a commercial software license. Additional information on both licenses is provided on the Moddable website.

NameEmailTitle/Company
Brian Friedkinbrian@moddable.techPrincipal Engineer - Moddable Tech, Inc.