eIQ MCU Media Processing Pipeline API

MPP VERSION 3.3

  1. MPP API

Functions

  1. Detailed Description

This section provides the detailed documentation for the MCU Media Processing Pipeline API.

  1. Function Documentation

  2. mpp_api_init()

int mpp_api_init (

mpp_api_params_t ∗ params ) Pipeline initialization.

This function initializes the library and its data structures.

It must be called before any other function of the API is called.

Parameters

in

params

API global parameters

out

ret

return code (0 - success, non-zero - error)

Returns

Return_codes

2. mpp_create()

mpp_t mpp_create (

mpp_params_t ∗ params, int ∗ ret )

Basic pipeline creation.

This function returns a handle to the pipeline. Parameters

in

params

pipeline parameters

out

ret

return code (0 - success, non-zero - error)

Returns

handle to the pipeline if success, NULL if there is an error.

  1. mpp_camera_add()

int mpp_camera_add (

mpp_t mpp,

const char ∗ name, mpp_camera_params_t ∗ params )

Camera addition.

This function adds a camera to the pipeline.

Parameters

in

mpp

input pipeline

in

name

camera driver name

in

params

parameters to be configured on the camera

Returns

Return_codes

  1. mpp_static_img_add()

int mpp_static_img_add (

mpp_t mpp,

mpp_img_params_t ∗ params, void ∗ addr )

Static image addition. Parameters

in

mpp

input pipeline

in

params

static image parameters

in

addr

image buffer

Returns

Return_codes Precondition

  • Image buffer allocation/free is the responsibility of the user.

  1. mpp_display_add()

int mpp_display_add (

mpp_t mpp,

const char ∗ name, mpp_display_params_t ∗ params )

Display addition.

This function adds a display to the pipeline. Parameters

in

mpp

input pipeline

in

name

display driver name

in

params

parameters that are configured on the display

Returns

Return_codes

  1. **mpp_nullsink_add()**int mpp_nullsink_add (

mpp_t mpp )

Null sink addition.

This function adds a null-type sink to the pipeline.

After this call pipeline is closed and no further elements can be added. Input frames are discarded.

Parameters

in

mpp

input pipeline

Returns

Return_codes

  1. mpp_element_add()

int mpp_element_add (

mpp_t mpp,

mpp_element_id_t id, mpp_element_params_t ∗ params, mpp_elem_handle_t ∗ elem_h )

Add processing element (single input, single output) This function adds an element to the pipeline. Available elements are:

  • 2D image processing

  • ML inference engine

  • Labeled rectangle

  • Compositor

Parameters

in

mpp

input pipeline

in

id

element id

in

params

element parameters

out

elem← _h

element handle in pipeline

Returns

Return_codes

  1. mpp_split()

int mpp_split (

mpp_t mpp,

unsigned int num, mpp_params_t ∗ params, mpp_t ∗ out_list )

Pipeline multiplication. Parameters

in

mpp

input pipeline

in

num

number of output pipeline

in

params

split mpp parameters

out

out_list

list of output pipelines

Returns

Return_codes Precondition

  • out_list array must contain at least num elements.

  1. mpp_background()

int mpp_background (

mpp_t mpp,

mpp_params_t ∗ params, mpp_t ∗ out_mpp )

Put next elements processing in background. Parameters

in

mpp

input pipeline

in

params

new mpp parameters (exec_flag must be MPP_EXEC_PREEMPT)

out

out_mpp

output pipeline

Returns

Return_codes

  1. mpp_element_update()

int mpp_element_update (

mpp_t mpp,

mpp_elem_handle_t elem_h, mpp_element_params_t ∗ params )

Update element parameters.

MCU Media Processing Pipeline 2. MPP Types 29**

Parameters

in

mpp

input pipeline

in

elem_h

element handle in the pipeline.

in

params

new element parameters

Returns

Return_codes

  1. **mpp_start()**int mpp_start (

mpp_t mpp, int last )

Start pipeline.

When called with last=0, this function prepares the branch of the pipeline specified with mpp. When called with last!=0, this function starts the data flow of the pipeline.

Data flow should start after all the branches of the pipeline have been prepared.

Parameters

in

mpp

pipeline branch handle to start/prepare

in

last

if non-zero start pipeline processing. No further start call is possible thereafter.

Returns

Return_codes

  1. **mpp_stop()**int mpp_stop (

mpp_t mpp )

Stop a branch of the pipeline.

This function stops the data processing and peripherals of a pipeline branch. Parameters

in

mpp

pipeline branch to stop

Returns

Return_codes

  1. **mpp_stats_enable()**void mpp_stats_enable (

    mpp_stats_grp_t grp ) Enable statistics collection.

    This function enables statistics collection for a given group. Statistics collection is disabled by default after API initialization. Calling this function when stats are enabled has no effect.

    Parameters

in

grp

statistics group

Returns

  1. **mpp_stats_disable()**void mpp_stats_disable (

    mpp_stats_grp_t grp ) Disable statistics collection.

    This function disables statistics collection for a given group. Calling this function when stats are disabled has no effect. This function is used to ensure stats are not updated while application tasks use the stats structures.

    Parameters

[in}

grp statistics group

  1. **mpp_get_version()**char ∗ mpp_get_version (

void )

Get MPP version. Returns

pointer to the MPP version string

  1. MPP Types

Data Structures

Macros

Typedefs

Enumerations

  1. Detailed Description

This section provides the detailed documentation for the MCU Media Processing Pipeline types.

  1. Data Structure Documentation

  2. union mpp_stats_t Data Fields

struct mpp_stats_t.api

api

Global execution performance counters.

struct mpp_stats_t.mpp

mpp

Pipeline execution performance counters.

struct mpp_stats_t.elem

elem

Element execution performance counters.

  1. struct mpp_api_params_t Data Fields

mpp_stats_t

stats

API stats.

unsigned int

rc_cycle_min

minimum cycle duration for RC tasks (ms), 0: sets default value

unsigned int

rc_cycle_inc

time increment for RC tasks (ms), 0: sets default value

int

pipeline_task_max_prio

pipeline tasks maximum priority.

  1. struct mpp_params_t Pipeline creation parameters.

    Data Fields

  1. struct mpp_camera_params_t

Camera parameters. Data Fields

int

height

buffer height

int

width

buffer width

mpp_pixel_format_t

format

pixel format

int

fps

frames per second

bool

stripe

stripe mode

  1. struct mpp_img_params_t

Static image parameters. Data Fields

int

height

buffer height

int

width

buffer width

mpp_pixel_format_t

format

pixel format

bool

stripe

stripe mode

  1. struct mpp_display_params_t

Display parameters. Data Fields

int

height

buffer resolution: setting to 0 will default to panel physical resolution

int

width

buffer resolution: setting to 0 will default to panel physical resolution

int

pitch

buffer resolution: setting to 0 will default to panel physical resolution

int

left

active rect: setting to 0 will default to fullscreen

int

top

active rect: setting to 0 will default to fullscreen

int

right

active rect: setting to 0 will default to fullscreen

int

bottom

active rect: setting to 0 will default to fullscreen

mpp_rotate_degree_t

rotate

rotate degree

mpp_pixel_format_t

format

pixel format

bool

stripe

stripe mode

  1. struct mpp_tensor_dims_t Inference tensor dimensions.

    Data Fields

uint32_t

size

uint32_t

data[MAX_TENSOR_DIMS]

  1. struct mpp_inference_tensor_params_t

tensor parameters Data Fields

const uint8_t ∗

data

data address

mpp_tensor_dims_t

dims

tensor data dimensions

mpp_tensor_type_t

type

tensor data type

  1. struct mpp_inference_cb_param_t

Inference callback parameters. Data Fields

void ∗

user_data

callback will pass this pointer

mpp_inference_tensor_params_t

out_tensors[MPP_INFERENCE_MAX

_OUTPUTSoutput tensors] parameters

int

inference_time_ms

inference run time measurement - output to user

mpp_inference_type_t

inference_type

type of the inference

  1. union mpp_color_t

mpp color encoding Data Fields

uint32_t

raw

Raw color.

struct mpp_color_t.rgb

rgb

rgb color values RGB color

  1. struct mpp_labeled_rect_t

mpp labeled rectangle element structure Data Fields

uint8_t

label[64]

label to print

Data Fields

uint16_t

clear

clear rectangle

uint16_t

line_width

rectangle line thickness

mpp_color_t

line_color

rectangle line color

uint16_t

top

rectangle top position

uint16_t

left

rectangle left position

uint16_t

bottom

rectangle bottom position

uint16_t

right

rectangle right position

uint16_t

tag

labeled rectangle tag

uint16_t

reserved

pad for 32 bits alignment

bool

stripe

stripe mode

  1. struct mpp_area_t

Image area coordinates. Data Fields

int

top

int

left

int

bottom

int

right

  1. struct mpp_dims_t

Image dimensions. Data Fields

unsigned int

width

unsigned int

height

  1. struct mpp_position_t

Image position. Data Fields

int

top

int

left

  1. struct mpp_inference_params_t

Model parameters. Data Fields

uint64_t

constant_weight_MemSize

model constant weights memory size

uint64_t

mutable_weight_MemSize

Defines the amount of memory required both input & output data buffers.

uint64_t

activations_MemSize

Size of scratch memory used for intermediate computations needed by the model.

int

num_inputs

model’s number of inputs

int

num_outputs

model’s number of outputs

uint64_t

inputs_offsets[MPP_INFERENCE_MAX_IN

PUTSoffset]of each input

uint64_t

outputs_offsets[MPP_INFERENCE_MAX_O

UTPUTSoffset ofeach] output

inference_entry_point_t

model_entry_point

function called to perform the inference

mpp_tensor_type_t

model_input_tensors_type

type of input buffer

  1. struct mpp_element_params_t

Processing element parameters. Data Fields

union mpp_element_params_t.unnamed5

__unnamed__

mpp_stats_t

stats

  1. struct mpp_stats_t.api Data Fields

unsigned int

rc_cycle

run-to-completion (RC) cycle duration (ms)

unsigned int

rc_cycle_max

run-to-completion work deadline (ms)

unsigned int

pr_slot

available slot for preemptable (PR) work (ms)

unsigned int

pr_rounds

number of RC cycles required to complete one PR cycle (ms)

unsigned int

app_slot

remaining time for application (ms)

  1. struct mpp_stats_t.mpp Data Fields

mpp_t

mpp

unsigned int

mpp_exec_time

pipeline execution time (ms)

  1. struct mpp_stats_t.elem Data Fields

mpp_elem_handle_t

hnd

unsigned int

elem_exec_time

element execution time (ms)

  1. struct mpp_color_t.rgb

rgb color values Data Fields

uint8_t

R

Red byte.

uint8_t

G

Green byte.

uint8_t

B

Blue byte.

uint8_t

pad

padding byte

  1. union mpp_element_params_t.__unnamed5__ Data Fields

struct mpp_element_params_t.unnamed5.compose

compose

Compose element’s parameters - NOT IMPLEMENTED YET.

struct

mpp_element_params_t.unnamed5.labels

labels

Labeled Rectangle element’s parameters.

struct mpp_element_params_t.unnamed5.convert

convert

Convert element’s parameters.

struct

mpp_element_params_t.unnamed5.resize

resize

Resize element’s parameters.

struct mpp_element_params_t.unnamed5.color_conv

color_conv

Color convert element’s parameters.

struct

mpp_element_params_t.unnamed5.rotate

rotate

Rotate element’s parameters.

struct

mpp_element_params_t.unnamed5.test

test

Test element’s parameters.

struct mpp_element_params_t.unnamed5.ml_inferen

ml_inference

ce

ML inference element’s parameters.

  1. struct mpp_element_params_t.__unnamed5__.compose

Compose element’s parameters - NOT IMPLEMENTED YET. Data Fields

float

a

float

b

  1. struct mpp_element_params_t.__unnamed5__.labels

Labeled Rectangle element’s parameters. Data Fields

uint32_t

max_count

maximum number of rectangles

uint32_t

detected_count

detected rectangles

mpp_labeled_rect_t

rectangles

array of rectangle data

  1. struct mpp_element_params_t.__unnamed5__.convert

Convert element’s parameters. Data Fields

mpp_dims_t

out_buf

output buffer dimensions

mpp_pixel_format_t

pixel_format

new pixel format

mpp_rotate_degree_t

angle

rotation angle

mpp_flip_mode_t

flip

flip mode

mpp_area_t

crop

input crop area

mpp_position_t

out_window

output window position

mpp_dims_t

scale

scaling dimensions

mpp_convert_ops_t

ops

operation selector mask

const char ∗

dev_name

device name used for graphics

bool

stripe_in

input stripe mode

bool

stripe_out

output stripe mode

  1. struct mpp_element_params_t.__unnamed5__.resize

Resize element’s parameters. Data Fields

unsigned int

width

unsigned int

height

  1. struct mpp_element_params_t.__unnamed5__.color_conv

Color convert element’s parameters. Data Fields

mpp_pixel_format_t

pixel_format

  1. struct mpp_element_params_t.__unnamed5__.rotate

Rotate element’s parameters. Data Fields

mpp_rotate_degree_t

angle

  1. struct mpp_element_params_t.__unnamed5__.test

Test element’s parameters. Data Fields

_Bool

inp

unsigned int

width

unsigned int

height

mpp_pixel_format_t

format

  1. struct mpp_element_params_t.__unnamed5__.ml_inference

ML inference element’s parameters. Data Fields

const void ∗

model_data

pointer to model binary

mpp_inference_type_t

type

inference type

int

model_size

model binary size

float

model_input_mean

model ‘mean’ of input values, used for normalization

float

model_input_std

model ‘standard deviation’ of input values, used for normalization

mpp_tensor_order_t

tensor_order

model input tensor component order

mpp_inference_params_t

inference_params

model specific parameters used by the inference

  1. Macro Definition Documentation

  2. MPP_INFERENCE_MAX_OUTPUTS

#define MPP_INFERENCE_MAX_OUTPUTS

Maximum number of inference inputs and outputs. Maximum number of outputs supported by the pipeline

  1. MPP_INFERENCE_MAX_INPUTS

#define MPP_INFERENCE_MAX_INPUTS

Maximum number of inputs supported by the pipeline.

  1. MPP_INVALID

#define MPP_INVALID Invalid pipeline handle.

  1. MPP_EVENT_ALL

#define MPP_EVENT_ALL

Bit mask to receive all Events.

  1. MAX_TENSOR_DIMS

#define MAX_TENSOR_DIMS

Maximum number of dimensions for tensors.

  1. Typedef Documentation

  2. mpp_t

typedef void∗ mpp_t Pipeline handle type.

  1. mpp_elem_handle_t

typedef uintptr_t mpp_elem_handle_t Element handle type.

  1. mpp_evt_mask_t

typedef unsigned int mpp_evt_mask_t Event mask for pipeline creation.

  1. inference_entry_point_t

typedef int(∗ inference_entry_point_t) (uint8_t ∗, uint8_t ∗, uint8_t ∗) Bundle inference function type.

  1. Enumeration Type Documentation

  2. mpp_evt_t

enum mpp_evt_t

Pipeline generated events.

Enumerator

MPP_EVENT_INVALID

invalid event

MPP_EVENT_INFERENCE_OUTPUT_READY

inference out is ready

MPP_EVENT_INTERNAL_TEST_RESERVED

INTERNAL: DO NOT USE.

MPP_EVENT_NUM

DO NOT USE.

  1. mpp_exec_flag_t

enum mpp_exec_flag_t Execution parameters.

These parameters control the execution of the elements of an mpp.

The “mpps” created using the flag MPP_EXEC_RC are guaranteed to run up to the completion of all processing elements, while not being preempted by other “mpps”.

The “mpps” created using the flag MPP_EXEC_PREEMPT are preempted after a given time interval by “mpps” that will run-to-completion again.

The “mpps” created with the MPP_EXEC_INHERIT flag inherit the same execution flag as the parent(s) in case of split operation.

Note: It is not possible to request run-to-completion execution when spliting preemptable-execution “mpps”.

Enumerator

MPP_EXEC_INHERIT

inherit from parent(s)

MPP_EXEC_RC

run-to-completion

MPP_EXEC_PREEMPT

preemptable

  1. mpp_stats_grp_t

enum mpp_stats_grp_t Enumerator

MPP_STATS_GRP_API

API (global) stats.

MPP_STATS_GRP_MPP

mpp_t stats

MPP_STATS_GRP_ELEMENT

element stats

MPP_STATS_GRP_NUM

number of groups

  1. mpp_rotate_degree_t

enum mpp_rotate_degree_t Rotation value.

Enumerator

ROTATE_0

0 degree

ROTATE_90

90 degrees

ROTATE_180

180 degrees

ROTATE_270

270 degrees

  1. mpp_flip_mode_t

enum mpp_flip_mode_t Flip type. Enumerator

FLIP_NONE

no flip

FLIP_HORIZONTAL

horizontal flip

FLIP_VERTICAL

vertical flip

FLIP_BOTH

vertical and horizontal flip

  1. mpp_convert_ops_t

enum mpp_convert_ops_t

The convert operations selector flags. Enumerator

MPP_CONVERT_NONE

no frame conversion

MPP_CONVERT_ROTATE

frame rotation and flip

MPP_CONVERT_SCALE

scaling from input_frame toward output window

MPP_CONVERT_COLOR

frame color conversion

MPP_CONVERT_CROP

input frame crop

MPP_CONVERT_OUT_WINDOW

output window

  1. mpp_pixel_format_t

enum mpp_pixel_format_t Pixel format. Enumerator

MPP_PIXEL_ARGB

ARGB 32 bits.

MPP_PIXEL_BGRA

BGRA 32 bits.

MPP_PIXEL_RGBA

RGBA 32 bits.

Enumerator

MPP_PIXEL_RGB

RGB 24 bits.

MPP_PIXEL_RGB565

RGB 16 bits.

MPP_PIXEL_BGR

BGR 24 bits.

MPP_PIXEL_GRAY888

gray 3x8 bits

MPP_PIXEL_GRAY888X

gray 3x8 bits +8 unused bits

MPP_PIXEL_GRAY

gray 8 bits

MPP_PIXEL_GRAY16

gray 16 bits

MPP_PIXEL_YUV1P444

YUVX interleaved 4:4:4.

MPP_PIXEL_VYUY1P422

VYUY interleaved 4:2:2.

MPP_PIXEL_UYVY1P422

UYVY interleaved 4:2:2.

MPP_PIXEL_YUYV

YUYV interleaved 4:2:2.

MPP_PIXEL_DEPTH16

depth 16 bits

MPP_PIXEL_DEPTH8

depth 8 bits

MPP_PIXEL_YUV420P

YUV planar 4:2:0.

MPP_PIXEL_INVALID

invalid pixel format

  1. mpp_element_id_t

enum mpp_element_id_t Processing element ids. Enumerator

MPP_ELEMENT_INVALID

Invalid element.

MPP_ELEMENT_COMPOSE

Image composition - NOT IMPLEMENTED YET.

MPP_ELEMENT_LABELED_RECTANGLE

Labeled rectangle - bounding box.

MPP_ELEMENT_TEST

Test inplace element - NOT FOR USE.

MPP_ELEMENT_INFERENCE

Inference engine.

MPP_ELEMENT_CONVERT

Image conversion: resolution, orientation, color format.

MPP_ELEMENT_NUM

DO NOT USE.

  1. mpp_tensor_type_t

enum mpp_tensor_type_t Inference tensor type. Enumerator

MPP_TENSOR_TYPE_FLOAT32

floating point 32 bits

MPP_TENSOR_TYPE_UINT8

unsigned integer 8 bits

MPP_TENSOR_TYPE_INT8

signed integer 8 bits

MCU Media Processing Pipeline

  1. mpp_tensor_order_t

enum mpp_tensor_order_t Inference input tensor order. Enumerator

MPP_TENSOR_ORDER_UNKNOWN

order not set

MPP_TENSOR_ORDER_NHWC

order: Batch, Height, Width, Channels

MPP_TENSOR_ORDER_NCHW

order: Batch, Channels, Height, Width

  1. mpp_inference_type_t

enum mpp_inference_type_t Inference type. Enumerator

MPP_INFERENCE_TYPE_TFLITE

TensorFlow-Lite.

  1. Return_codes

Macros

  1. Detailed Description

MPP APIs return status definitions.

  1. Macro Definition Documentation

  2. MPP_SUCCESS

#define MPP_SUCCESS Success return code.

3.3 Return_codes 27

  1. MPP_ERROR

#define MPP_ERROR

A generic error occured.

  1. MPP_INVALID_ELEM

#define MPP_INVALID_ELEM Invalid element provided.

  1. MPP_INVALID_PARAM

#define MPP_INVALID_PARAM Invalid parameter provided.

  1. MPP_ERR_ALLOC_MUTEX

#define MPP_ERR_ALLOC_MUTEX Error occured while allocating mutex.

  1. MPP_INVALID_MUTEX

#define MPP_INVALID_MUTEX Invalid mutex provided.

  1. MPP_MUTEX_TIMEOUT

#define MPP_MUTEX_TIMEOUT Mutex timeout occured.

  1. MPP_MUTEX_ERROR

#define MPP_MUTEX_ERROR Mutex error occured.

  1. MPP_MALLOC_ERROR

#define MPP_MALLOC_ERROR Memory allocation error occured.