Edge Impulse integration

Edge Impulse is a development platform that can be used to enable embedded machine learning on nRF Connect SDK devices. You can use this platform to collect data from sensors, train machine learning model, and then deploy it to your Nordic Semiconductor’s device.

Integration prerequisites

Before you start the nRF Connect SDK integration with Edge Impulse, make sure that the following prerequisites are completed:

Solution architecture

Support for Edge Impulse in the nRF Connect SDK is centered around the Edge Impulse wrapper, which is used for integrating the Edge Impulse machine learning model into an nRF Connect SDK application. The usage of the wrapper is demonstrated by the Edge Impulse: Wrapper sample, while the nRF Machine Learning application demonstrates the complete out of the box reference design.

Integration overview

Before integrating the Edge Impulse machine learning model into an nRF Connect SDK application, you must prepare and deploy the machine learning model for your embedded device. This model is prepared using the Edge Impulse studio external web tool. It relies on sensor data that can be provided by different sources, for example data forwarder. Check the Edge Impulse: Data forwarder sample for a demonstration of how you can send sensor data to Edge Impulse studio using Edge Impulse’s data forwarder.

The machine learning model is distributed as a single zip archive that includes C++ library sources. This file is used by the nRF Connect SDK build system to build the Edge Impulse library.

Integration steps

Complete the following steps to generate the archive and add it to the build system:

  1. Preparing the machine learning model

  2. Building the machine learning model in nRF Connect SDK

Preparing the machine learning model

To prepare the machine learning model, use Edge Impulse studio and follow one of the tutorials described in Edge Impulse getting started guide. For example, you can try the Continuous motion recognition tutorial. This tutorial will guide you through the following steps:

  1. Collecting data from sensors and uploading the data to Edge Impulse studio.

    Note

    You can use one of the development boards supported directly by Edge Impulse or your mobile phone to collect the data. You can also modify the Edge Impulse: Data forwarder sample or nRF Machine Learning application and use it to forward data from a sensor that is connected to any board available in the nRF Connect SDK.

  2. Designing your machine learning model (an impulse).

  3. Deploying the machine learning model to use it on an embedded device. As part of this step, you must select the C++ library to generate the required zip file that contains the source files for building the Edge Impulse library in nRF Connect SDK.

Building the machine learning model in nRF Connect SDK

After preparing the zip archive, you can use the nRF Connect SDK build system to build the C++ library with the machine learning model. Complete the following steps to configure the building process:

  1. Make sure that the following Kconfig options are enabled:

    Note

    The CONFIG_FPU Kconfig option is implied by default if floating point unit (FPU) is supported by the hardware. Using FPU speeds up calculations.

    The Edge Impulse library requires full C++ standard library implementation. Because of that, the CONFIG_EDGE_IMPULSE option selects the CONFIG_REQUIRES_FULL_LIBCPP option.

  2. Make sure that the CONFIG_FP16 Kconfig option is disabled. The Edge Impulse library is not compatible with half-precision floating point support introduced in Zephyr.

  3. Enable building the downloaded library by setting the CONFIG_EDGE_IMPULSE Kconfig option. Setting this option also enables the Edge Impulse wrapper.

  4. Enable and specify the Uniform Resource Identifier (URI) in the CONFIG_EDGE_IMPULSE_URI Kconfig option. You can set it to one of the following values:

    • An absolute or relative path to a file in the local file system. For this variant, you must download the zip file manually and place it under path defined by the Kconfig option. The relative path is tracked from the application source directory (APPLICATION_SOURCE_DIR). CMake variables that are part of the path are expanded.

    • Any downloadable URI supported by CMake’s file(DOWNLOAD) command. For this variant, the nRF Connect SDK build system will download the zip file automatically during build. The zip file is downloaded into your application’s build directory.

      If the URI requires providing an additional API key in the HTTP header, you can provide it using the EI_API_KEY_HEADER CMake definition. The API key is provided using a format in which key_name is followed by key_value. For example, if the URI uses x-api_key for authentication, the EI_API_KEY_HEADER can be defined as follows: x-api-key:aaaabbbbccccdddd. The aaaabbbbccccdddd is a sample key_value. See Providing CMake options for more information about defining CMake options for command line builds and the nRF Connect for VS Code extension. See Downloading model directly from Edge Impulse studio for details about downloading model directly from the Edge Impulse studio.

Downloading model directly from Edge Impulse studio

As an example of downloadable URI, you can configure the nRF Connect SDK build system to download your model directly from the Edge Impulse studio. You can download a model from either a private or a public project.

Downloading from a private project

Complete the following steps to download the model from a private Edge Impulse project:

  1. Set CONFIG_EDGE_IMPULSE_URI to the URI from Edge Impulse studio:

    CONFIG_EDGE_IMPULSE_URI="https://studio.edgeimpulse.com/v1/api/XYZ/deployment/download?type=zip"

    Set XYZ to the project ID of your Edge Impulse project. You can check the project ID of your project in the Project info panel under Dashboard.

    Project ID in Edge Impulse studio dashboard

    Project ID in Edge Impulse studio dashboard

  2. Define the EI_API_KEY_HEADER CMake option (see Providing CMake options) as x-api-key:[ei api key] to provide the x-api-key associated with your Edge Impulse project. To check what to provide as the [ei api key] value, check your API keys under the Keys tab in the Edge Impulse project dashboard.

    API key under the Keys tab in Edge Impulse studio

    API key under the Keys tab in Edge Impulse studio

Downloading from a public project

Complete the following steps to download the model from a public Edge Impulse project:

  1. Check the ID of the public project:

    1. Check the project ID of your project in the Project info panel under Dashboard.

    2. Provide this project ID in the XYZ field in the following URL:

      https://studio.edgeimpulse.com/v1/api/XYZ/versions/public

    3. Paste the URL into your browser. The ID of the public project is returned as the value of the publicProjectId field. For example:

      {"success":true,"versions":[{"version":1,"publicProjectId":66469,"publicProjectUrl":"https://studio.edgeimpulse.com/public/66468/latest"}]}

      In this example, the XYZ project ID is 66468, while the publicProjectId equals 66469.

  2. Set CONFIG_EDGE_IMPULSE_URI to the following URI from Edge Impulse studio:

    CONFIG_EDGE_IMPULSE_URI="https://studio.edgeimpulse.com/v1/api/XYZ/deployment/download?type=zip&modelType=int8"

    Set the XYZ to the public project ID from previous step. Using the example above, this would be 66469.

    Note

    This URI includes the modelType=int8 parameter because from public Edge Impulse projects you can only download quantized models created with Edge Impulse’s EON Compiler.

Applications and samples

The following application uses the Edge Impulse integration in the nRF Connect SDK:

  • nRF Machine Learning - demonstrates a complete out of the box reference design. The nRF Machine learning application can also be used to collect data from devices (for example, it supports Thingy:53 with an out-of-the-box accelerometer).

The following samples demonstrate the Edge Impulse integration in the nRF Connect SDK:

Library support

The following nRF Connect SDK libraries support the Edge Impulse integration: