Maestro sync example
Table of content
Overview
The Maestro sync example demonstrates the use of synchronous pipelines (Tx and Rx in this case) processing in a single streamer task on the ARM cortex core utilizing the Maestro Audio Framework library.
The application is controlled by commands from a shell interface using serial console.
The feature is useful for testing the latency of the pipeline or implementing algorithms requiring reference signals such as echo cancellation. The VoiceSeeker library available in this example is not featuring AEC (Acoustic Echo Cancellation), but NXP is offering it in the premium version of the library. More information about the premium version can be found at VoiceSeeker. page. The demo uses two pipelines running synchronously in a single streamer task:
Playback (Tx) pipeline:
Playback of audio data in PCM format stored in flash memory to the audio Line-Out connector (speaker).
Recording (Rx) pipline:
Record audio data using a microphone.
VoiceSeeker processing.
Wake words + voice commands recognition.
Save the VoiceSeeker output to the voiceseeker_output.pcm file on the SD card.
As shown in the table below, the application is supported on several development boards, and each development board may have certain limitations, some development boards may also require hardware modifications or allow to use of an audio expansion board. Therefore, please check the supported features and Hardware modifications or Example configuration sections before running the demo.
| Mode | Standard | ||||||
|---|---|---|---|---|---|---|---|
| Feature | Audio inputs [num of channels] | Audio outputs [num of channels] | Additional libraries | ||||
| On board codec | DMICs | aud-exp-42448 | On board codec | aud-exp-42448 | VIT | VoiceSeeker | |
| EVKC-MIMXRT1060 | 1 | X | X | 1 | X | OK | OK |
| EVKB-MIMXRT1170 | X | 1-2 | X | 1-2 | X | OK | OK |
| LPCXpresso55s69 | X | X | X | X | X | X | X |
| EVK-MCXN5XX | X | X | X | X | X | X | X |
| RW612BGA | X | X | X | X | X | X | X |
Limitations:
Addition labraries
VIT:
The VIT is supported only in the MCUXpresso IDE.
VoiceSeeker:
The VoiceSeeker is supported only in the MCUXpresso IDE.
Known issues:
No known issues.
More information about supported features can be found on the Supported features page.
Hardware requirements
Desired development board
Micro USB cable
Speaker with 3.5 mm stereo jack
Personal computer
Optional:
SD card for file output
Hardware modifications
Some development boards need some hardware modifications to run the application. If the development board is not listed here, its default setting is required.
EVKC-MIMXRT1060:
Please make sure resistors below are removed to be able to use SD-Card.
R368, R347, R349, R365, R363
Please Make sure J99 is installed.
Preparation
Connect a micro USB cable between the PC host and the debug USB port on the development board
Open a serial terminal with the following settings:
115200 baud rate
8 data bits
No parity
One stop bit
No flow control
Download the program to the target board.
Insert the speaker into the Line-Out connector (headphone jack) on the development board.
Optional: Insert an SD card into the SD card slot to record to the VoiceSeeker output.
Either press the reset button on your development board or launch the debugger in your IDE to begin running the demo.
Running the demo
When the example runs successfully, you should see similar output on the serial terminal as below:
*****************************
Maestro audio sync demo start
*****************************
Copyright 2022 NXP
[APP_SDCARD_Task] start
[APP_Shell_Task] start
>> [APP_SDCARD_Task] SD card drive mounted
Type help to see the command list. Similar description will be displayed on serial console:
>> help
"help": List all the registered commands
"exit": Exit program
"version": Display component versions
"start [nosdcard]": Starts a streamer task.
- Initializes the streamer with the Memory->Speaker pipeline and with
the Microphone->VoiceSeeker->VIT->SDcard pipeline.
- Runs repeatedly until stop command.
nosdcard - Doesn't use SD card to store data.
"stop": Stops a running streamer:
"debug [on|off]": Starts / stops debugging.
- Starts / stops saving VoiceSeeker input data (reference and microphone data)
to SDRAM.
- After the stop command, this data is transferred to the SD card.
Details of commands can be found here.
Example configuration
The example can be configured by user. Before configuration, please check the table to see if the feature is supported on the development board.
Enable the premium version of VoiceSeeker:
The premium version of the VoiceSeeker library with AEC is API compatible with this example.
To get the premium version, please visit VoiceSeeker page.
The following steps are required to run this example with the VoiceSeeker&AEC library.
Link the
voiceseeker.alibrary instead ofvoiceseeker_no_aec.a.Set the
RDSP_ENABLE_AECdefinition to1Uin thevoiceseeker.hfile
VIT model generation:
For custom VIT model generation (defining own wake words and voice commands) please use https://vit.nxp.com/
Functionality
The start <nosdcard> command calls the STREAMER_Create function from the app_streamer.c file that creates pipelines with the following elements:
Playback pipeline:
ELEMENT_MEM_SRC_INDEX
ELEMENT_SPEAKER_INDEX
Record pipeline:
ELEMENT_MICROPHONE_INDEX
ELEMENT_VOICESEEKER_INDEX
ELEMENT_VIT_PROC_INDEX
ELEMENT_FILE_SINK_INDEX (If the
nosdcardargument is not used)
Processing itself can be started with the STREAMER_Start function.
Each of the elements has several properties that can be accessed using the streamer_get_property or streamer_set_property function. These properties allow a user to change the values of the appropriate elements. The list of properties can be found in streamer_element_properties.h. See the example of setting property value in the following piece of code from the app_streamer.c file:
ELEMENT_PROPERTY_T prop;
MEMSRC_SET_BUFFER_T buf;
buf.location = (int8_t *)TESTAUDIO_DATA;
buf.size = TESTAUDIO_LEN;
prop.prop = PROP_MEMSRC_SET_BUFF;
prop.val = (uintptr_t)&buf;
if (STREAM_OK != streamer_set_property(handle->streamer, 0, prop, true))
{
return kStatus_Fail;
}
prop.prop = PROP_MEMSRC_SET_MEM_TYPE;
prop.val = AUDIO_DATA;
if (STREAM_OK != streamer_set_property(handle->streamer, 0, prop, true))
{
return kStatus_Fail;
}
prop.prop = PROP_MEMSRC_SET_SAMPLE_RATE;
prop.val = DEMO_SAMPLE_RATE;
if (STREAM_OK != streamer_set_property(handle->streamer, 0, prop, true))
{
return kStatus_Fail;
}
Some of the predefined values can be found in the streamer_api.h.
States
The application can be in 2 different states:
Idle
Running
Commands in detail
Legend for diagrams: