eIQ MCU Media Processing Pipeline API
MPP API
Functions
int mpp_api_init (mpp_api_params_t ∗params)
mpp_t mpp_create (mpp_params_t ∗params, int ∗ret)
int mpp_camera_add (mpp_t mpp, const char ∗name, mpp_camera_params_t ∗params)
int mpp_static_img_add (mpp_t mpp, mpp_img_params_t ∗params, void ∗addr)
int mpp_display_add (mpp_t mpp, const char ∗name, mpp_display_params_t ∗params)
int mpp_element_add (mpp_t mpp, mpp_element_id_t id, mpp_element_params_t ∗params, mpp_elem_handle_t ∗elem_h)
int mpp_split (mpp_t mpp, unsigned int num, mpp_params_t ∗params, mpp_t ∗out_list)
int mpp_background (mpp_t mpp, mpp_params_t ∗params, mpp_t ∗out_mpp)
int mpp_element_update (mpp_t mpp, mpp_elem_handle_t elem_h, mpp_element_params_t ∗params)
int mpp_start (mpp_t mpp, int last)
char ∗mpp_get_version (void)
This section provides the detailed documentation for the MCU Media Processing Pipeline API.
mpp_api_params_t ∗ params ) Pipeline initialization.
This function initializes the library and its data structures.
It must be called before any other function of the API is called.
Parameters
in |
params |
API global parameters |
---|---|---|
out |
ret |
return code (0 - success, non-zero - error) |
Returns
2. mpp_create()
mpp_t mpp_create (
mpp_params_t ∗ params, int ∗ ret )
Basic pipeline creation.
This function returns a handle to the pipeline. Parameters
in |
params |
pipeline parameters |
---|---|---|
out |
ret |
return code (0 - success, non-zero - error) |
Returns
handle to the pipeline if success, NULL if there is an error.
mpp_camera_add()
mpp_t mpp,
const char ∗ name, mpp_camera_params_t ∗ params )
Camera addition.
This function adds a camera to the pipeline.
Parameters
in |
mpp |
input pipeline |
---|---|---|
in |
name |
camera driver name |
in |
params |
parameters to be configured on the camera |
Returns
mpp_t mpp,
mpp_img_params_t ∗ params, void ∗ addr )
Static image addition. Parameters
in |
mpp |
input pipeline |
---|---|---|
in |
params |
static image parameters |
in |
addr |
image buffer |
Returns
Return_codes Precondition
Image buffer allocation/free is the responsibility of the user.
mpp_display_add()
mpp_t mpp,
const char ∗ name, mpp_display_params_t ∗ params )
Display addition.
This function adds a display to the pipeline. Parameters
in |
mpp |
input pipeline |
---|---|---|
in |
name |
display driver name |
in |
params |
parameters that are configured on the display |
Returns
mpp_t mpp )
Null sink addition.
This function adds a null-type sink to the pipeline.
After this call pipeline is closed and no further elements can be added. Input frames are discarded.
Parameters
in |
mpp |
input pipeline |
---|
Returns
mpp_element_add()
mpp_t mpp,
mpp_element_id_t id, mpp_element_params_t ∗ params, mpp_elem_handle_t ∗ elem_h )
Add processing element (single input, single output) This function adds an element to the pipeline. Available elements are:
2D image processing
ML inference engine
Labeled rectangle
Compositor
Parameters
in |
mpp |
input pipeline |
---|---|---|
in |
id |
element id |
in |
params |
element parameters |
out |
elem← _h |
element handle in pipeline |
Returns
mpp_split()
mpp_t mpp,
unsigned int num, mpp_params_t ∗ params, mpp_t ∗ out_list )
Pipeline multiplication. Parameters
in |
mpp |
input pipeline |
---|---|---|
in |
num |
number of output pipeline |
in |
params |
split mpp parameters |
out |
out_list |
list of output pipelines |
Returns
Return_codes Precondition
out_list array must contain at least num elements.
mpp_background()
mpp_t mpp,
mpp_params_t ∗ params, mpp_t ∗ out_mpp )
Put next elements processing in background. Parameters
in |
mpp |
input pipeline |
---|---|---|
in |
params |
new mpp parameters (exec_flag must be MPP_EXEC_PREEMPT) |
out |
out_mpp |
output pipeline |
Returns
mpp_element_update()
mpp_t mpp,
mpp_elem_handle_t elem_h, mpp_element_params_t ∗ params )
Update element parameters.
MCU Media Processing Pipeline 2. MPP Types 29**
Parameters
in |
mpp |
input pipeline |
---|---|---|
in |
elem_h |
element handle in the pipeline. |
in |
params |
new element parameters |
Returns
mpp_t mpp, int last )
Start pipeline.
When called with last=0, this function prepares the branch of the pipeline specified with mpp. When called with last!=0, this function starts the data flow of the pipeline.
Data flow should start after all the branches of the pipeline have been prepared.
Parameters
in |
mpp |
pipeline branch handle to start/prepare |
---|---|---|
in |
last |
if non-zero start pipeline processing. No further start call is possible thereafter. |
Returns
mpp_t mpp )
Stop a branch of the pipeline.
This function stops the data processing and peripherals of a pipeline branch. Parameters
in |
mpp |
pipeline branch to stop |
---|
Returns
**mpp_stats_enable()**void mpp_stats_enable (
mpp_stats_grp_t grp ) Enable statistics collection.
This function enables statistics collection for a given group. Statistics collection is disabled by default after API initialization. Calling this function when stats are enabled has no effect.
Parameters
in |
grp |
statistics group |
---|
Returns
**mpp_stats_disable()**void mpp_stats_disable (
mpp_stats_grp_t grp ) Disable statistics collection.
This function disables statistics collection for a given group. Calling this function when stats are disabled has no effect. This function is used to ensure stats are not updated while application tasks use the stats structures.
Parameters
[in} |
grp statistics group |
---|
void )
Get MPP version. Returns
pointer to the MPP version string
Data Structures
union mpp_stats_t
struct mpp_api_params_t
struct mpp_params_t
struct mpp_camera_params_t
struct mpp_img_params_t
struct mpp_display_params_t
struct mpp_tensor_dims_t
struct mpp_inference_cb_param_t
union mpp_color_t
struct mpp_labeled_rect_t
struct mpp_area_t
struct mpp_dims_t
struct mpp_position_t
struct mpp_inference_params_t
struct mpp_element_params_t
struct mpp_stats_t.api
struct mpp_stats_t.mpp
struct mpp_stats_t.elem
struct mpp_color_t.rgb
Macros
#define MPP_INFERENCE_MAX_OUTPUTS
#define MPP_INFERENCE_MAX_INPUTS
#define MPP_INVALID
#define MPP_EVENT_ALL
#define MAX_TENSOR_DIMS
Typedefs
typedef void ∗mpp_t
typedef uintptr_t mpp_elem_handle_t
typedef unsigned int mpp_evt_mask_t
typedef int(∗inference_entry_point_t) (uint8_t ∗, uint8_t ∗, uint8_t ∗)
Enumerations
enum mpp_evt_t {
MPP_EVENT_INVALID , MPP_EVENT_INFERENCE_OUTPUT_READY , MPP_EVENT_INTERNAL_TEST_RESERVED , MPP_EVENT_NUM }
enum mpp_exec_flag_t {
enum mpp_stats_grp_t {
MPP_STATS_GRP_API , MPP_STATS_GRP_MPP , MPP_STATS_GRP_ELEMENT , MPP_STATS_GRP_NUM }
enum mpp_flip_mode_t {
enum mpp_convert_ops_t { MPP_CONVERT_NONE , MPP_CONVERT_ROTATE , MPP_CONVERT_SCALE , MPP_CONVERT_COLOR , MPP_CONVERT_CROP , MPP_CONVERT_OUT_WINDOW }
enum mpp_pixel_format_t { MPP_PIXEL_ARGB ,
MPP_PIXEL_GRAY888 , MPP_PIXEL_GRAY888X , MPP_PIXEL_GRAY ,
MPP_PIXEL_GRAY16 , MPP_PIXEL_YUV1P444 , MPP_PIXEL_VYUY1P422 , MPP_PIXEL_UYVY1P422 , MPP_PIXEL_YUYV , MPP_PIXEL_DEPTH16 , MPP_PIXEL_DEPTH8 , MPP_PIXEL_YUV420P , MPP_PIXEL_INVALID }
enum mpp_element_id_t { MPP_ELEMENT_INVALID , MPP_ELEMENT_COMPOSE , MPP_ELEMENT_LABELED_RECTANGLE , MPP_ELEMENT_TEST , MPP_ELEMENT_INFERENCE ,
enum mpp_tensor_type_t { MPP_TENSOR_TYPE_FLOAT32 , MPP_TENSOR_TYPE_UINT8 , MPP_TENSOR_TYPE_INT8 }
enum mpp_tensor_order_t { MPP_TENSOR_ORDER_UNKNOWN , MPP_TENSOR_ORDER_NHWC , MPP_TENSOR_ORDER_NCHW }
This section provides the detailed documentation for the MCU Media Processing Pipeline types.
struct mpp_stats_t.api |
api |
Global execution performance counters. |
---|---|---|
struct mpp_stats_t.mpp |
mpp |
Pipeline execution performance counters. |
struct mpp_stats_t.elem |
elem |
Element execution performance counters. |
stats |
API stats. |
|
---|---|---|
unsigned int |
rc_cycle_min |
minimum cycle duration for RC tasks (ms), 0: sets default value |
unsigned int |
rc_cycle_inc |
time increment for RC tasks (ms), 0: sets default value |
int |
pipeline_task_max_prio |
pipeline tasks maximum priority. |
int(∗evt_callback_f )(mpp_t mpp, mpp_evt_t evt, void ∗evt_data, void ∗user_data)
mpp_evt_mask_t mask
mpp_exec_flag_t exec_flag
void ∗cb_userdata
mpp_stats_t ∗stats
Camera parameters. Data Fields
int |
height |
buffer height |
---|---|---|
int |
width |
buffer width |
format |
pixel format |
|
int |
fps |
frames per second |
bool |
stripe |
stripe mode |
Static image parameters. Data Fields
int |
height |
buffer height |
---|---|---|
int |
width |
buffer width |
format |
pixel format |
|
bool |
stripe |
stripe mode |
Display parameters. Data Fields
int |
height |
buffer resolution: setting to 0 will default to panel physical resolution |
---|---|---|
int |
width |
buffer resolution: setting to 0 will default to panel physical resolution |
int |
pitch |
buffer resolution: setting to 0 will default to panel physical resolution |
int |
left |
active rect: setting to 0 will default to fullscreen |
int |
top |
active rect: setting to 0 will default to fullscreen |
int |
right |
active rect: setting to 0 will default to fullscreen |
int |
bottom |
active rect: setting to 0 will default to fullscreen |
rotate |
rotate degree |
|
format |
pixel format |
|
bool |
stripe |
stripe mode |
uint32_t |
size |
|
---|---|---|
uint32_t |
data[MAX_TENSOR_DIMS] |
tensor parameters Data Fields
const uint8_t ∗ |
data |
data address |
---|---|---|
dims |
tensor data dimensions |
|
type |
tensor data type |
Inference callback parameters. Data Fields
void ∗ |
user_data |
callback will pass this pointer |
---|---|---|
∗ |
out_tensors[MPP_INFERENCE_MAX |
_OUTPUTSoutput tensors] parameters |
int |
inference_time_ms |
inference run time measurement - output to user |
inference_type |
type of the inference |
mpp color encoding Data Fields
uint32_t |
raw |
Raw color. |
---|---|---|
struct mpp_color_t.rgb |
rgb |
rgb color values RGB color |
mpp labeled rectangle element structure Data Fields
uint8_t |
label[64] |
label to print |
---|
Data Fields
uint16_t |
clear |
clear rectangle |
---|---|---|
uint16_t |
line_width |
rectangle line thickness |
line_color |
rectangle line color |
|
uint16_t |
top |
rectangle top position |
uint16_t |
left |
rectangle left position |
uint16_t |
bottom |
rectangle bottom position |
uint16_t |
right |
rectangle right position |
uint16_t |
tag |
labeled rectangle tag |
uint16_t |
reserved |
pad for 32 bits alignment |
bool |
stripe |
stripe mode |
Image area coordinates. Data Fields
int |
top |
|
---|---|---|
int |
left |
|
int |
bottom |
|
int |
right |
Image dimensions. Data Fields
unsigned int |
width |
|
---|---|---|
unsigned int |
height |
Image position. Data Fields
int |
||
---|---|---|
int |
left |
Model parameters. Data Fields
uint64_t |
constant_weight_MemSize |
model constant weights memory size |
---|---|---|
uint64_t |
mutable_weight_MemSize |
Defines the amount of memory required both input & output data buffers. |
uint64_t |
activations_MemSize |
Size of scratch memory used for intermediate computations needed by the model. |
int |
num_inputs |
model’s number of inputs |
int |
num_outputs |
model’s number of outputs |
uint64_t |
inputs_offsets[MPP_INFERENCE_MAX_IN |
PUTSoffset]of each input |
uint64_t |
outputs_offsets[MPP_INFERENCE_MAX_O |
UTPUTSoffset ofeach] output |
model_entry_point |
function called to perform the inference |
|
model_input_tensors_type |
type of input buffer |
Processing element parameters. Data Fields
__unnamed__ |
||
---|---|---|
stats |
unsigned int |
rc_cycle |
run-to-completion (RC) cycle duration (ms) |
---|---|---|
unsigned int |
rc_cycle_max |
run-to-completion work deadline (ms) |
unsigned int |
pr_slot |
available slot for preemptable (PR) work (ms) |
unsigned int |
pr_rounds |
number of RC cycles required to complete one PR cycle (ms) |
unsigned int |
app_slot |
remaining time for application (ms) |
mpp |
||
---|---|---|
unsigned int |
mpp_exec_time |
pipeline execution time (ms) |
hnd |
||
---|---|---|
unsigned int |
elem_exec_time |
element execution time (ms) |
rgb color values Data Fields
uint8_t |
R |
Red byte. |
---|---|---|
uint8_t |
G |
Green byte. |
uint8_t |
B |
Blue byte. |
uint8_t |
pad |
padding byte |
compose |
Compose element’s parameters - NOT IMPLEMENTED YET. |
|
---|---|---|
struct |
labels |
Labeled Rectangle element’s parameters. |
convert |
Convert element’s parameters. |
|
struct |
resize |
Resize element’s parameters. |
color_conv |
Color convert element’s parameters. |
|
struct |
rotate |
Rotate element’s parameters. |
struct |
test |
Test element’s parameters. |
ml_inference |
ML inference element’s parameters. |
Compose element’s parameters - NOT IMPLEMENTED YET. Data Fields
float |
||
---|---|---|
float |
b |
Labeled Rectangle element’s parameters. Data Fields
uint32_t |
max_count |
maximum number of rectangles |
---|---|---|
uint32_t |
detected_count |
detected rectangles |
rectangles |
array of rectangle data |
Convert element’s parameters. Data Fields
out_buf |
output buffer dimensions |
|
---|---|---|
pixel_format |
new pixel format |
|
angle |
rotation angle |
|
flip |
flip mode |
|
crop |
input crop area |
|
out_window |
output window position |
|
scale |
scaling dimensions |
|
ops |
operation selector mask |
|
const char ∗ |
dev_name |
device name used for graphics |
bool |
stripe_in |
input stripe mode |
bool |
stripe_out |
output stripe mode |
Resize element’s parameters. Data Fields
unsigned int |
width |
|
---|---|---|
unsigned int |
height |
Color convert element’s parameters. Data Fields
Rotate element’s parameters. Data Fields
angle |
---|
Test element’s parameters. Data Fields
_Bool |
inp |
|
---|---|---|
unsigned int |
width |
|
unsigned int |
height |
|
format |
ML inference element’s parameters. Data Fields
const void ∗ |
model_data |
pointer to model binary |
---|---|---|
type |
inference type |
|
int |
model_size |
model binary size |
float |
model_input_mean |
model ‘mean’ of input values, used for normalization |
float |
model_input_std |
model ‘standard deviation’ of input values, used for normalization |
tensor_order |
model input tensor component order |
|
inference_params |
model specific parameters used by the inference |
#define MPP_INFERENCE_MAX_OUTPUTS
Maximum number of inference inputs and outputs. Maximum number of outputs supported by the pipeline
#define MPP_INFERENCE_MAX_INPUTS
Maximum number of inputs supported by the pipeline.
MPP_INVALID
#define MPP_INVALID Invalid pipeline handle.
MPP_EVENT_ALL
Bit mask to receive all Events.
MAX_TENSOR_DIMS
Maximum number of dimensions for tensors.
typedef void∗ mpp_t Pipeline handle type.
mpp_elem_handle_t
typedef uintptr_t mpp_elem_handle_t Element handle type.
mpp_evt_mask_t
typedef unsigned int mpp_evt_mask_t Event mask for pipeline creation.
inference_entry_point_t
typedef int(∗ inference_entry_point_t) (uint8_t ∗, uint8_t ∗, uint8_t ∗) Bundle inference function type.
enum mpp_evt_t
Pipeline generated events.
Enumerator
invalid event |
|
---|---|
inference out is ready |
|
INTERNAL: DO NOT USE. |
|
DO NOT USE. |
mpp_exec_flag_t
enum mpp_exec_flag_t Execution parameters.
These parameters control the execution of the elements of an mpp.
The “mpps” created using the flag MPP_EXEC_RC are guaranteed to run up to the completion of all processing elements, while not being preempted by other “mpps”.
The “mpps” created using the flag MPP_EXEC_PREEMPT are preempted after a given time interval by “mpps” that will run-to-completion again.
The “mpps” created with the MPP_EXEC_INHERIT flag inherit the same execution flag as the parent(s) in case of split operation.
Note: It is not possible to request run-to-completion execution when spliting preemptable-execution “mpps”.
Enumerator
inherit from parent(s) |
|
---|---|
run-to-completion |
|
preemptable |
mpp_stats_grp_t
enum mpp_stats_grp_t Enumerator
API (global) stats. |
|
---|---|
mpp_t stats |
|
element stats |
|
number of groups |
mpp_rotate_degree_t
enum mpp_rotate_degree_t Rotation value.
Enumerator
0 degree |
|
---|---|
90 degrees |
|
180 degrees |
|
270 degrees |
mpp_flip_mode_t
enum mpp_flip_mode_t Flip type. Enumerator
no flip |
|
---|---|
horizontal flip |
|
vertical flip |
|
vertical and horizontal flip |
mpp_convert_ops_t
enum mpp_convert_ops_t
The convert operations selector flags. Enumerator
no frame conversion |
|
---|---|
frame rotation and flip |
|
scaling from input_frame toward output window |
|
frame color conversion |
|
input frame crop |
|
output window |
mpp_pixel_format_t
enum mpp_pixel_format_t Pixel format. Enumerator
ARGB 32 bits. |
|
---|---|
BGRA 32 bits. |
|
RGBA 32 bits. |
Enumerator
RGB 24 bits. |
|
---|---|
RGB 16 bits. |
|
BGR 24 bits. |
|
gray 3x8 bits |
|
gray 3x8 bits +8 unused bits |
|
gray 8 bits |
|
gray 16 bits |
|
YUVX interleaved 4:4:4. |
|
VYUY interleaved 4:2:2. |
|
UYVY interleaved 4:2:2. |
|
YUYV interleaved 4:2:2. |
|
depth 16 bits |
|
depth 8 bits |
|
YUV planar 4:2:0. |
|
invalid pixel format |
mpp_element_id_t
enum mpp_element_id_t Processing element ids. Enumerator
Invalid element. |
|
---|---|
Image composition - NOT IMPLEMENTED YET. |
|
Labeled rectangle - bounding box. |
|
Test inplace element - NOT FOR USE. |
|
Inference engine. |
|
Image conversion: resolution, orientation, color format. |
|
DO NOT USE. |
mpp_tensor_type_t
enum mpp_tensor_type_t Inference tensor type. Enumerator
unsigned integer 8 bits |
|
signed integer 8 bits |
MCU Media Processing Pipeline
mpp_tensor_order_t
enum mpp_tensor_order_t Inference input tensor order. Enumerator
order not set |
|
---|---|
order: Batch, Height, Width, Channels |
|
order: Batch, Channels, Height, Width |
mpp_inference_type_t
enum mpp_inference_type_t Inference type. Enumerator
TensorFlow-Lite. |
---|
Return_codes
#define MPP_SUCCESS
#define MPP_ERROR
#define MPP_INVALID_ELEM
#define MPP_INVALID_PARAM
#define MPP_ERR_ALLOC_MUTEX
#define MPP_INVALID_MUTEX
#define MPP_MUTEX_TIMEOUT
#define MPP_MUTEX_ERROR
#define MPP_MALLOC_ERROR
MPP APIs return status definitions.
#define MPP_SUCCESS Success return code.
3.3 Return_codes 27
MPP_ERROR
A generic error occured.
MPP_INVALID_ELEM
#define MPP_INVALID_ELEM Invalid element provided.
MPP_INVALID_PARAM
#define MPP_INVALID_PARAM Invalid parameter provided.
MPP_ERR_ALLOC_MUTEX
#define MPP_ERR_ALLOC_MUTEX Error occured while allocating mutex.
MPP_INVALID_MUTEX
#define MPP_INVALID_MUTEX Invalid mutex provided.
MPP_MUTEX_TIMEOUT
#define MPP_MUTEX_TIMEOUT Mutex timeout occured.
MPP_MUTEX_ERROR
#define MPP_MUTEX_ERROR Mutex error occured.
MPP_MALLOC_ERROR