eIQ MCU Media Processing Pipeline API
MPP VERSION 3.5
1. MPP API
Functions
- int mpp_api_init (mpp_api_params_t ∗params) 
- mpp_t mpp_create (mpp_params_t ∗params, int ∗ret) 
- int mpp_camera_add (mpp_t mpp, const char ∗name, mpp_camera_params_t ∗params) 
- int mpp_static_img_add (mpp_t mpp, mpp_img_params_t ∗params, void ∗addr) 
- int mpp_display_add (mpp_t mpp, const char ∗name, mpp_display_params_t ∗params) 
- int mpp_element_add (mpp_t mpp, mpp_element_id_t id, mpp_element_params_t ∗params, mpp_elem_handle_t ∗elem_h) 
- int mpp_split (mpp_t mpp, unsigned int num, mpp_params_t ∗params, mpp_t ∗out_list) 
- int mpp_background (mpp_t mpp, mpp_params_t ∗params, mpp_t ∗out_mpp) 
- int mpp_element_update (mpp_t mpp, mpp_elem_handle_t elem_h, mpp_element_params_t ∗params) 
- int mpp_start (mpp_t mpp, int last) 
- char ∗mpp_get_version (void) 
1.1 Detailed Description
This section provides the detailed documentation for the MCU Media Processing Pipeline API.
1.1.1 Function Documentation
- mpp_api_init() 
int mpp_api_init ( mpp_api_params_t ∗ params )
Pipeline initialization. This function initializes the library and its data structures. It must be called before any other function of the API is called.
Parameters
| in/out | name | description | 
|---|---|---|
| in | params | API global parameters | 
| out | ret | return code (0 - success, non-zero - error) | 
Returns
2. mpp_create()
mpp_t mpp_create ( mpp_params_t ∗ params, int ∗ ret )
Basic pipeline creation. This function returns a handle to the pipeline.
Parameters
| in/out | name | description | 
|---|---|---|
| in | params | pipeline parameters | 
| out | ret | return code (0 - success, non-zero - error) | 
Returns
handle to the pipeline if success, NULL if there is an error.
- mpp_camera_add() 
int mpp_camera_add ( mpp_t mpp, const char ∗ name, mpp_camera_params_t ∗ params )
Camera addition.
This function adds a camera to the pipeline.
Parameters
| in/out | name | description | 
|---|---|---|
| in | mpp | input pipeline | 
| in | name | camera driver name | 
| in | params | parameters to be configured on the camera | 
Returns
int mpp_static_img_add ( mpp_t mpp, mpp_img_params_t ∗ params, void ∗ addr, mpp_elem_handle_t ∗ elem_h )
Static image addition.
Parameters
| in/out | name | description | 
|---|---|---|
| in | mpp | input pipeline | 
| in | params | static image parameters | 
| in | addr | image buffer | 
| out | elem_h | element handle in pipeline | 
Returns
Precondition
- Image buffer allocation/free is the responsibility of the user. 
- mpp_display_add() 
int mpp_display_add ( mpp_t mpp, const char ∗ name, mpp_display_params_t ∗ params )
Display addition.
This function adds a display to the pipeline.
Parameters
| in/out | name | description | 
|---|---|---|
| in | mpp | input pipeline | 
| in | name | display driver name | 
| in | params | parameters that are configured on the display | 
Returns
int mpp_nullsink_add ( mpp_t mpp )
Null sink addition.
This function adds a null-type sink to the pipeline.
After this call pipeline is closed and no further elements can be added. Input frames are discarded.
Parameters
| in/out | name | description | 
|---|---|---|
| in | mpp | input pipeline | 
Returns
- mpp_element_add() 
int mpp_element_add ( mpp_t mpp, mpp_element_id_t id, mpp_element_params_t ∗ params, mpp_elem_handle_t ∗ elem_h )
Add processing element (single input, single output) This function adds an element to the pipeline. Available elements are:
- 2D image processing 
- ML inference engine 
- Labeled rectangle 
- Compositor 
Parameters
| in/out | name | description | 
|---|---|---|
| in | mpp | input pipeline | 
| in | id | element id | 
| in | params | element parameters | 
| out | elem_h | element handle in pipeline | 
Returns
- mpp_split() 
int mpp_split ( mpp_t mpp, unsigned int num, mpp_params_t ∗ params, mpp_t ∗ out_list )
Pipeline multiplication.
Parameters
| in/out | name | description | 
|---|---|---|
| in | mpp | input pipeline | 
| in | num | number of output pipeline | 
| in | params | split mpp parameters | 
| out | out_list | list of output pipelines | 
Returns
Precondition
- out_list array must contain at least num elements. 
- mpp_background() 
int mpp_background ( mpp_t mpp, mpp_params_t ∗ params, mpp_t ∗ out_mpp )
Put next elements processing in background.
Parameters
| in/out | name | description | 
|---|---|---|
| in | mpp | input pipeline | 
| in | params | new mpp parameters (exec_flag must be MPP_EXEC_PREEMPT) | 
| out | out_mpp | output pipeline | 
Returns
- mpp_element_update() 
int mpp_element_update ( mpp_t mpp, mpp_elem_handle_t elem_h, mpp_element_params_t ∗ params )
Update element parameters.
Parameters
| in/out | name | description | 
|---|---|---|
| in | mpp | input pipeline | 
| in | elem_h | element handle in the pipeline. | 
| in | params | new element parameters | 
Returns
int mpp_start ( mpp_t mpp, int last )
Start pipeline.
When called with last=0, this function prepares the branch of the pipeline specified with mpp. When called with last!=0, this function starts the data flow of the pipeline.
Data flow should start after all the branches of the pipeline have been prepared.
Parameters
| in/out | name | description | 
|---|---|---|
| in | mpp | pipeline branch handle to start/prepare | 
| in | last | if non-zero start pipeline processing. No further start call is possible thereafter. | 
Returns
int mpp_stop ( mpp_t mpp )
Stop a branch of the pipeline.
This function stops the data processing and peripherals of a pipeline branch.
Parameters
| in/out | name | description | 
|---|---|---|
| in | mpp | pipeline branch to stop | 
Returns
void mpp_stats_enable ( mpp_stats_grp_t grp )
Enable statistics collection.
This function enables statistics collection for a given group. Statistics collection is disabled by default after API initialization. Calling this function when stats are enabled has no effect.
Parameters
| in/out | name | description | 
|---|---|---|
| in | grp | statistics group | 
Returns
void mpp_stats_disable ( mpp_stats_grp_t grp )
Disable statistics collection.
This function disables statistics collection for a given group. Calling this function when stats are disabled has no effect. This function is used to ensure stats are not updated while application tasks use the stats structures.
Parameters
| in/out | name | description | 
|---|---|---|
| in | grp | statistics group | 
char ∗ mpp_get_version ( void )
Get MPP version.
Returns
pointer to the MPP version string
2. MPP Types
Data Structures
- union mpp_stats_t 
- struct mpp_api_params_t 
- struct mpp_params_t 
- struct mpp_camera_params_t 
- struct mpp_img_params_t 
- struct mpp_display_params_t 
- struct mpp_tensor_dims_t 
- struct mpp_inference_cb_param_t 
- union mpp_color_t 
- struct mpp_labeled_rect_t 
- struct mpp_area_t 
- struct mpp_dims_t 
- struct mpp_position_t 
- struct mpp_inference_params_t 
- struct mpp_element_params_t 
- struct mpp_stats_t.api 
- struct mpp_stats_t.mpp 
- struct mpp_stats_t.elem 
- struct mpp_color_t.rgb 
Macros
- #define MPP_INFERENCE_MAX_OUTPUTS 
- #define MPP_INFERENCE_MAX_INPUTS 
- #define MPP_INVALID 
- #define MPP_EVENT_ALL 
- #define MAX_TENSOR_DIMS 
Typedefs
- typedef void ∗mpp_t 
- typedef uintptr_t mpp_elem_handle_t 
- typedef unsigned int mpp_evt_mask_t 
- typedef int(∗inference_entry_point_t) (uint8_t ∗, uint8_t ∗, uint8_t ∗) 
Enumerations
- enum mpp_evt_t { - MPP_EVENT_INFERENCE_OUTPUT_READY, 
- enum mpp_exec_flag_t { 
- enum mpp_stats_grp_t { 
- enum mpp_rotate_degree_t { 
- enum mpp_flip_mode_t { 
- enum mpp_convert_ops_t { 
- enum mpp_pixel_format_t { 
- enum mpp_element_id_t { 
- enum mpp_tensor_type_t { 
- enum mpp_tensor_order_t { 
- enum mpp_inference_type_t { 
2.1 Detailed Description
This section provides the detailed documentation for the MCU Media Processing Pipeline types.
2.1.1 Data Structure Documentation
Data Fields
| type | name | description | 
|---|---|---|
| struct mpp_stats_t.api | api | Global execution performance counters. | 
| struct mpp_stats_t.mpp | mpp | Pipeline execution performance counters. | 
| struct mpp_stats_t.elem | elem | Element execution performance counters. | 
Data Fields
| type | name | description | 
|---|---|---|
| stats | API stats. | |
| unsigned int | rc_cycle_min | minimum cycle duration for RC tasks (ms), 0: sets default value | 
| unsigned int | rc_cycle_inc | time increment for RC tasks (ms), 0: sets default value | 
| int | pipeline_task_max_prio | pipeline tasks maximum priority. | 
Pipeline creation parameters.
Data Fields
- int(∗evt_callback_f )(mpp_t mpp, mpp_evt_t evt, void ∗evt_data, void ∗user_data) 
- mpp_evt_mask_t mask 
- mpp_exec_flag_t exec_flag 
- void ∗cb_userdata 
- mpp_stats_t ∗stats 
Camera parameters.
Data Fields
| type | name | description | 
|---|---|---|
| int | height | buffer height | 
| int | width | buffer width | 
| format | pixel format | |
| int | fps | frames per second | 
| bool | stripe | stripe mode | 
Static image parameters.
Data Fields
| type | name | description | 
|---|---|---|
| int | height | buffer height | 
| int | width | buffer width | 
| format | pixel format | |
| bool | stripe | stripe mode | 
| int | compressed_size | size in bytes for compressed format | 
Display parameters.
Data Fields
| type | name | description | 
|---|---|---|
| int | height | buffer resolution: setting to 0 will default to panel physical resolution | 
| int | width | buffer resolution: setting to 0 will default to panel physical resolution | 
| int | pitch | buffer resolution: setting to 0 will default to panel physical resolution | 
| int | left | active rect: setting to 0 will default to fullscreen | 
| int | top | active rect: setting to 0 will default to fullscreen | 
| int | right | active rect: setting to 0 will default to fullscreen | 
| int | bottom | active rect: setting to 0 will default to fullscreen | 
| rotate | rotate degree | |
| format | pixel format | |
| bool | stripe | stripe mode | 
| void ∗ | handle | pointer to an lvgl image widget | 
Inference tensor dimensions.
Data Fields
| type | name | description | 
|---|---|---|
| uint32_t | size | |
| uint32_t | data[MAX_TENSOR_DIMS] | 
tensor parameters
Data Fields
| type | name | description | 
|---|---|---|
| const uint8_t ∗ | data | data address | 
| dims | tensor data dimensions | |
| type | tensor data type | 
Inference callback parameters.
Data Fields
| type | name | description | 
|---|---|---|
| void ∗ | user_data | callback will pass this pointer | 
| out_tensors[MPP_INFERENCE_MAX_OUTPUTS] | output tensors parameters | |
| int | inference_time_ms | inference run time measurement - output to user | 
| inference_type | type of the inference | 
mpp color encoding
Data Fields
| type | name | description | 
|---|---|---|
| uint32_t | raw | Raw color. | 
| struct mpp_color_t.rgb | rgb | rgb color values RGB color | 
mpp labeled rectangle element structure
Data Fields
| type | name | description | 
|---|---|---|
| uint8_t | label[64] | label to print | 
| uint16_t | clear | clear rectangle | 
| uint16_t | line_width | rectangle line thickness | 
| line_color | rectangle line color | |
| int16_t | top | rectangle top position | 
| int16_t | left | rectangle left position | 
| int16_t | bottom | rectangle bottom position | 
| int16_t | right | rectangle right position | 
| uint16_t | tag | labeled rectangle tag | 
| uint16_t | reserved | pad for 32 bits alignment | 
| bool | stripe | stripe mode | 
Image area coordinates.
Data Fields
| type | name | description | 
|---|---|---|
| int | top | |
| int | left | |
| int | bottom | |
| int | right | 
Image dimensions.
Data Fields
| type | name | description | 
|---|---|---|
| unsigned int | width | |
| unsigned int | height | 
Image position.
Data Fields
| type | name | description | 
|---|---|---|
| int | ||
| int | left | 
Model parameters.
Data Fields
| type | name | description | 
|---|---|---|
| uint64_t | constant_weight_MemSize | model constant weights memory size | 
| uint64_t | mutable_weight_MemSize | Defines the amount of memory required both input & output data buffers. | 
| uint64_t | activations_MemSize | Size of scratch memory used for intermediate computations needed by the model. | 
| int | num_inputs | model’s number of inputs | 
| int | num_outputs | model’s number of outputs | 
| uint64_t | inputs_offsets[MPP_INFERENCE_MAX_INPUTS] | offset of each input | 
| uint64_t | outputs_offsets[MPP_INFERENCE_MAX_OUTPUTS] | offset ofeach output | 
| model_entry_point | function called to perform the inference | |
| model_input_tensors_type | type of input buffer | 
Static image and Processing elements parameters.
Data Fields
| type | name | description | 
|---|---|---|
| __unnamed__ | ||
| stats | 
Data Fields
| type | name | description | 
|---|---|---|
| unsigned int | rc_cycle | run-to-completion (RC) cycle duration (ms) | 
| unsigned int | rc_cycle_max | run-to-completion work deadline (ms) | 
| unsigned int | pr_slot | available slot for preemptable (PR) work (ms) | 
| unsigned int | pr_rounds | number of RC cycles required to complete one PR cycle (ms) | 
| unsigned int | app_slot | remaining time for application (ms) | 
Data Fields
| type | name | description | 
|---|---|---|
| mpp | ||
| unsigned int | mpp_exec_time | pipeline execution time (ms) | 
Data Fields
| type | name | description | 
|---|---|---|
| hnd | ||
| unsigned int | elem_exec_time | element execution time (ms) | 
rgb color values
Data Fields
| type | name | description | 
|---|---|---|
| uint8_t | R | Red byte. | 
| uint8_t | G | Green byte. | 
| uint8_t | B | Blue byte. | 
| uint8_t | pad | padding byte | 
Data Fields
| type | name | description | 
|---|---|---|
| static_image | Static Image element’s parameters. | |
| compose | Compose element’s parameters - NOT IMPLEMENTED YET. | |
| struct | labels | Labeled Rectangle element’s parameters. | 
| convert | Convert element’s parameters. | |
| struct | resize | Resize element’s parameters. | 
| color_conv | Color convert element’s parameters. | |
| struct | rotate | Rotate element’s parameters. | 
| struct | test | Test element’s parameters. | 
| struct | decode | Decoder element’s parameters. | 
| ml_inference | ML inference element’s parameters. | 
Static Image element’s parameters.
Data Fields
| type | name | description | 
|---|---|---|
| img_params | static image parameters | |
| void ∗ | img_buffer | static image buffer address | 
Compose element’s parameters - NOT IMPLEMENTED YET.
Data Fields
| type | name | description | 
|---|---|---|
| float | ||
| float | b | 
Labeled Rectangle element’s parameters.
Data Fields
| type | name | description | 
|---|---|---|
| uint32_t | max_count | maximum number of rectangles | 
| uint32_t | detected_count | detected rectangles | 
| rectangles | array of rectangle data | 
Convert element’s parameters.
Data Fields
| type | name | description | 
|---|---|---|
| out_buf | output buffer dimensions | |
| pixel_format | new pixel format | |
| angle | rotation angle | |
| flip | flip mode | |
| crop | input crop area | |
| out_window | output window position | |
| scale | scaling dimensions | |
| ops | operation selector mask | |
| const char ∗ | dev_name | device name used for graphics | 
| bool | stripe_in | input stripe mode | 
| bool | stripe_out | output stripe mode | 
Resize element’s parameters.
Data Fields
| type | name | description | 
|---|---|---|
| unsigned int | width | |
| unsigned int | height | 
Color convert element’s parameters.
Data Fields
| type | name | description | 
|---|---|---|
Rotate element’s parameters.
Data Fields
| type | name | description | 
|---|---|---|
| angle | 
Test element’s parameters.
Data Fields
| type | name | description | 
|---|---|---|
| _Bool | inp | |
| unsigned int | width | |
| unsigned int | height | |
| format | 
Decoder element’s parameters.
Data Fields
| type | name | description | 
|---|---|---|
| const char ∗ | dev_name | device name used for decoder | 
| unsigned int | width | |
| unsigned int | height | |
| out_format | 
ML inference element’s parameters.
Data Fields
| type | name | description | 
|---|---|---|
| const void ∗ | model_data | pointer to model binary | 
| type | inference type | |
| int | model_size | model binary size | 
| float | model_input_mean | model ‘mean’ of input values, used for normalization | 
| float | model_input_std | model ‘standard deviation’ of input values, used for normalization | 
| tensor_order | model input tensor component order | |
| inference_params | model specific parameters used by the inference | 
2.1.2 Macro Definition Documentation
- MPP_INFERENCE_MAX_OUTPUTS 
#define MPP_INFERENCE_MAX_OUTPUTS
Maximum number of inference inputs and outputs. Maximum number of outputs supported by the pipeline
#define MPP_INFERENCE_MAX_INPUTS
Maximum number of inputs supported by the pipeline.
- MPP_INVALID 
#define MPP_INVALID Invalid pipeline handle.
- MPP_EVENT_ALL 
Bit mask to receive all Events.
- MAX_TENSOR_DIMS 
Maximum number of dimensions for tensors.
2.1.3 Typedef Documentation
- mpp_t 
typedef void∗ mpp_t Pipeline handle type.
- mpp_elem_handle_t 
typedef uintptr_t mpp_elem_handle_t Element handle type.
- mpp_evt_mask_t 
typedef unsigned int mpp_evt_mask_t Event mask for pipeline creation.
- inference_entry_point_t 
typedef int(∗ inference_entry_point_t) (uint8_t ∗, uint8_t ∗, uint8_t ∗) Bundle inference function type.
2.1.4 Enumeration Type Documentation
- mpp_evt_t 
enum mpp_evt_t
Pipeline generated events.
Enumerator
| label | description | 
|---|---|
| invalid event | |
| inference out is ready | |
| INTERNAL: DO NOT USE. | |
| DO NOT USE. | 
- mpp_exec_flag_t 
enum mpp_exec_flag_t
Execution parameters.
These parameters control the execution of the elements of an mpp.
The “mpps” created using the flag MPP_EXEC_RC are guaranteed to run up to the completion of all processing elements, while not being preempted by other “mpps”.
The “mpps” created using the flag MPP_EXEC_PREEMPT are preempted after a given time interval by “mpps” that will run-to-completion again.
The “mpps” created with the MPP_EXEC_INHERIT flag inherit the same execution flag as the parent(s) in case of split operation.
Note: It is not possible to request run-to-completion execution when spliting preemptable-execution “mpps”.
Enumerator
| label | description | 
|---|---|
| inherit from parent(s) | |
| run-to-completion | |
| preemptable | 
- mpp_stats_grp_t 
enum mpp_stats_grp_t
Enumerator
| label | description | 
|---|---|
| API (global) stats. | |
| mpp_t stats | |
| element stats | |
| number of groups | 
- mpp_rotate_degree_t 
enum mpp_rotate_degree_t
Rotation value.
Enumerator
| label | description | 
|---|---|
| 0 degree | |
| 90 degrees | |
| 180 degrees | |
| 270 degrees | 
- mpp_flip_mode_t 
enum mpp_flip_mode_t
Flip type.
Enumerator
| label | description | 
|---|---|
| no flip | |
| horizontal flip | |
| vertical flip | |
| vertical and horizontal flip | 
- mpp_convert_ops_t 
enum mpp_convert_ops_t
The convert operations selector flags.
Enumerator
| label | description | 
|---|---|
| no frame conversion | |
| frame rotation and flip | |
| scaling from input_frame toward output window | |
| frame color conversion | |
| input frame crop | |
| output window | 
- mpp_pixel_format_t 
enum mpp_pixel_format_t
Pixel format.
Enumerator
| label | description | 
|---|---|
| ARGB 32 bits. | |
| BGRA 32 bits. | |
| RGBA 32 bits. | |
| BGRX 32 bits. | |
| RGBX 32 bits. | |
| RGB 24 bits. | |
| RGB 16 bits. | |
| BGR 24 bits. | |
| gray 3x8 bits | |
| gray 3x8 bits +8 unused bits | |
| gray 8 bits | |
| gray 16 bits | |
| YUVX interleaved 4:4:4. | |
| VYUY interleaved 4:2:2. | |
| UYVY interleaved 4:2:2. | |
| YUYV interleaved 4:2:2. | |
| depth 16 bits | |
| depth 8 bits | |
| YUV planar 4:2:0. | |
| JPEG. | |
| invalid pixel format | 
- mpp_element_id_t 
enum mpp_element_id_t
Processing element ids.
Enumerator
| label | description | 
|---|---|
| Invalid element. | |
| Image composition - NOT IMPLEMENTED YET. | |
| Labeled rectangle - bounding box. | |
| Test inplace element - NOT FOR USE. | |
| Inference engine. | |
| Image conversion: resolution, orientation, color format. | |
| Image decompression: JPEG, PNG. | |
| DO NOT USE. | 
- mpp_tensor_type_t 
enum mpp_tensor_type_t
Inference tensor type.
Enumerator
| label | description | 
|---|---|
| unsigned integer 8 bits | |
| signed integer 8 bits | 
- mpp_tensor_order_t 
enum mpp_tensor_order_t
Inference input tensor order.
Enumerator
| label | description | 
|---|---|
| order not set | |
| order: Batch, Height, Width, Channels | |
| order: Batch, Channels, Height, Width | 
- mpp_inference_type_t 
enum mpp_inference_type_t
Inference type.
Enumerator
| label | description | 
|---|---|
| TensorFlow-Lite. | 
3. Return_codes
Macros
- #define MPP_SUCCESS 
- #define MPP_ERROR 
- #define MPP_INVALID_ELEM 
- #define MPP_INVALID_PARAM 
- #define MPP_ERR_ALLOC_MUTEX 
- #define MPP_INVALID_MUTEX 
- #define MPP_MUTEX_TIMEOUT 
- #define MPP_MUTEX_ERROR 
- #define MPP_MALLOC_ERROR 
3.1 Detailed Description
MPP APIs return status definitions.
3.1.1 Macro Definition Documentation
- MPP_SUCCESS 
Success return code.
- MPP_ERROR 
A generic error occured.
- MPP_INVALID_ELEM 
Invalid element provided.
- MPP_INVALID_PARAM 
Invalid parameter provided.
- MPP_ERR_ALLOC_MUTEX 
Error occured while allocating mutex.
- MPP_INVALID_MUTEX 
Invalid mutex provided.
- MPP_MUTEX_TIMEOUT 
Mutex timeout occured.
- MPP_MUTEX_ERROR 
Mutex error occured.
- MPP_MALLOC_ERROR 
Memory allocation error occured.