nRF Compression

The nRF Compression library in the nRF Connect SDK provides a streamlined API for data compression and decompression on devices.

Configuration 

You can enable different compression types by configuring the appropriate Kconfig options. See Configuring Kconfig for information about the different ways you can set Kconfig options in the nRF Connect SDK.

To enable this library, set the CONFIG_NRF_COMPRESS Kconfig option. For decompression, set the CONFIG_NRF_COMPRESS_DECOMPRESSION Kconfig option.

Note

The compression functionality is not currently supported.

Compression types configuration 

You can use the available compression types by enabling their respective Kconfig options:

Name	Kconfig options	Implementation details
LZMA version 1	`CONFIG_NRF_COMPRESS_LZMA` and `CONFIG_NRF_COMPRESS_LZMA_VERSION_LZMA1`	Exclusive: cannot be enabled with LZMA version 2. Fixed probability size of 14272 bytes. Fixed dictionary size of 128 KiB.
LZMA version 2	`CONFIG_NRF_COMPRESS_LZMA` and `CONFIG_NRF_COMPRESS_LZMA_VERSION_LZMA2`	Exclusive: cannot be enabled with LZMA version 1. Fixed probability size of 14272 bytes. Fixed dictionary size of 128 KiB.
ARM thumb filter	`NRF_COMPRESS_ARM_THUMB`	—

Memory allocation configuration options 

Compression and decompression can use a significant amount of memory. To manage this, use the following Kconfig options to choose between static and dynamic (malloc) allocations from the system heap:

CONFIG_NRF_COMPRESS_MEMORY_TYPE_STATIC: This is the default option that uses static buffers, ensuring their availability but preventing other uses of the memory.
CONFIG_NRF_COMPRESS_MEMORY_TYPE_MALLOC: The option uses dynamic memory allocation, requiring the heap to have sufficient contiguous free memory for buffer allocation upon initializing the compression type. This allows other parts of the application to utilize the memory when the compression system is not in use.

Other configuration options 

CONFIG_NRF_COMPRESS_CHUNK_SIZE: This option specifies the chunk size, which is the maximum amount of data that can be input to a compression library. It determines the size of buffers that are statically or dynamically allocated, unless the compression type has a different memory allocation due to how it works.
CONFIG_NRF_COMPRESS_CLEANUP: This option enables memory buffer cleanup upon calling the nrf_compress_deinit_func_t() function. It is performed to prevent possible leakage of sensitive data. If data security is not a concern, this option can be disabled to reduce flash usage.

Samples using the library 

The nRF Compression: MCUboot compressed update sample uses this library.

Application integration 

The following sections describe how to integrate the nRF Compression library in your application.

Implementing a compression type 

You can implement custom compression types by using a shim over the compression source files.

Note

Currently, due to the large memory requirements when compressing, the subsystem only supports data decompression.

Note

The function definitions include inst as the first argument, which is reserved for future use. It should be set to NULL when initializing the compression library.

Initialization and deinitialization 

The following initialization and deinitialization functions are necessary for managing the lifecycle of the compression library within your application or module:

nrf_compress_init_func_t() - The init function is used when an application or module initiates the use of the compression library. It sets up the required buffers and reset all internal variables to their default values.
nrf_compress_deinit_func_t() - The deinit function is used for cleaning up and releasing the buffers. If the CONFIG_NRF_COMPRESS_CLEANUP Kconfig option is enabled, it also ensures that all buffers are cleared prior to releasing them to prevent any possible data leakage.

Reset 

The nrf_compress_reset_func_t() function is used to reset the compression library if it is partially used. It resets the internal variables and buffers without performing deinitialization, which allows the compression library to be reused with a new file.

Decompression 

There are two functions for decompression:

nrf_compress_decompress_bytes_needed_t - This function determines the ideal amount of data to supply for decompression. It typically matches the value of the CONFIG_NRF_COMPRESS_CHUNK_SIZE Kconfig option, unless limited by a header or the final amount of data required is predetermined.
nrf_compress_decompress_func_t() - This function processes input data and, if decompressed output data is available, returns a buffer containing that data along with its size. Not all input data may be consumed when this function is called. The compression library might require complete blocks and might not process the final block if it is incomplete, especially if multiple blocks are provided. In such cases, the offset value will be updated to reflect the amount of data that was read from the input buffer The application or module must monitor the amount of data it intends to decompress. It will set the last_part value to true when submitting the final segment of the data stream for decompression. This is crucial as some compression libraries require this information.

Defining compression type 

Once the code is developed, the library must be defined in an iterable section using the NRF_COMPRESS_IMPLEMENTATION_DEFINE macro, located in the header file include/nrf_compress/implementation.h. There are following requirements depending on the library’s capabilities:

If a library only supports compression, the compression function must be defined, and two decompression functions must be set to NULL.
If a library only supports decompression, the two decompression functions must be defined, and the compression function must be set to NULL.
If a library supports both compression and decompression, all three functions must be defined.

All other functions are always mandatory and must always be defined. Additionally, you must define a unique ID for the compression library. It should be globally available, allowing applications or modules use it.

Integrating the compression subsystem 

To decompress data using the nRF Compression library, complete the following steps:

Use the nrf_compress_implementation_find() function with the ID of the desired compression type. If it returns NULL, the compression type is not supported.
Check the init, deinit and reset core function pointers. If any return NULL, there is an implementation issue with the compression type.
For decompression support, check the nrf_compress_decompress_bytes_needed_t and nrf_compress_decompress_func_t() function pointers. If either is NULL, there is an implementation issue with the compression type.
Call the nrf_compress_init_func_t() function to set up the compression library. If a non-zero error is returned, the library cannot be initialized, possibly due to insufficient memory or other issues.
Call the nrf_compress_decompress_bytes_needed_t function to determine how many bytes of data should be specified for decompression. If a non-zero error is returned, there is an issue with the compression library.
Call the nrf_compress_decompress_func_t() function with the requested amount of data (or the maximum available, if less is available). In case there is no more data to process, set the last_part value to true, otherwise set it to false. If a non-zero error is returned, the provided data might be invalid.
Ensure that if the offset updated value is not equal to the size of the input data provided, the next call to nrf_compress_decompress_func_t() function includes the unused bytes from the buffer at its start.
Verify if the output_size value is greater than 0. In such a case, the specified amount of data is available in the output buffer and you should copy it or move it to its intended destination.
Repeat the process of calling the nrf_compress_decompress_bytes_needed_t function followed by nrf_compress_decompress_func_t() until all the data has been processed.
Call the nrf_compress_deinit_func_t() function to clean up the compression library.

See the following figure for the overview of the decompression flow:

nRF Compression library decompression flowchart

API documentation 

Header file: include/nrf_compress/implementation.h

Source files: subsys/nrf_compress/src/

Compression/decompression subsystem