XAF Playback Example
Table of Content
Overview
The dsp_xaf_playback application demonstrates audio processing using the DSP core, the Xtensa Audio Framework (XAF) middleware library, and selected Xtensa audio codecs.
As shown in the table below, the application is supported on several development boards and each development board may have certain limitations, some development boards may also require hardware modifications or allow the use of an audio expansion board. Therefore, please check the supported features and Hardware modifications or Example configuration sections before running the demo.
| Mode | Standard | Multi-channel | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Feature | Decoder | Encoder | Sample rate converter | 32 bit pcm, 2 or 8 channels | |||||||
| AAC, FLAC, MP3, OPUS, VORBIS, SBC | LC3 | G.711, G.722, BSAC, DAB+, DAB/MP2, DRM | OPUS, SBC | LC3 | MP3 | G.711, G.722 | SSRC | ASRC | |||
| EVK-MIMXRT595 | OK | X | X | OK | X | OK | X | OK | OK | X | |
| EVK-MIMXRT685 | OK | X | OK | OK | X | X | OK | OK | OK | X | |
| MIMXRT685-AUD-EVK | OK | X | OK | OK | X | X | OK | OK | OK | OK | |
| MIMXRT700-EVK | HIFI4 | OK | OK | OK | OK | OK | OK | OK | OK | OK | X |
| HIFI1 | OK | OK | X | OK | OK | OK | X | OK | OK | X | |
Limitations:
MP3 encoder, G.711, G.722, BSAC, DAB+, DAB/MP2, DRM: Provided only as linked libraries but are not enabled in the example.
Functionality
The application includes the following main components:
ARM Core (CM33) - Handles user interface, SD card operations, and communicates with the DSP core
DSP Core - Processes audio data using the Xtensa Audio Framework (XAF)
The typical audio processing pipeline includes:
File source component (reads from SD card)
Decoder component (decodes compressed audio)
Renderer component (outputs to audio hardware)
When the file playback command is issued, the ARM core reads the file from SD card and sends data to the DSP, which processes it and outputs to the audio hardware.
Hardware Requirements
Development board (one of the following):
EVK-MIMXRT595 board
EVK-MIMXRT685 board
MIMXRT685-AUD-EVK board
MIMXRT700-EVK board
Micro USB cable
JTAG/SWD debugger
Headphones with 3.5 mm stereo jack
Personal Computer
SD card with audio files (for file playback feature)
Hardware Modifications
Some development boards need some hardware modifications to run the application.
EVK-MIMXRT595:
To enable the example audio using WM8904 codec, connect pins as follows:
JP7-1 <–> JP8-2
Note: The I3C Pin configuration in pin_mux.c is verified for default 1.8V, for 3.3V, need to manually configure slew rate to slow mode for I3C-SCL/SDA.
EVK-MIMXRT685:
To enable the example audio using WM8904 codec, connect pins as follows:
JP7-1 <–> JP8-2
MIMXRT685-AUD-EVK:
Set the hardware jumpers (Tower system/base module) to default settings.
Set hardware jumpers JP2 2<–>3, JP44 1<–>2 and JP45 1<–>2.
MIMXRT700-EVK:
Set the hardware jumpers to default settings.
Preparation
Connect headphones to Audio HP / Line-Out connector (J4).
EVK-MIMXRT595 - J4
EVK-MIMXRT685 - J4
MIMXRT685-AUD-EVK - J4, J50, J51, J52
MIMXRT700-EVK - J29
Connect a micro USB cable between the PC host and the debug USB port on the development board.
EVK-MIMXRT595 - J40
EVK-MIMXRT685 - J5
MIMXRT685-AUD-EVK - J5
MIMXRT700-EVK - J54
Open a serial terminal with the following settings:
115200 baud rate
8 data bits
No parity
One stop bit
No flow control
Download the program for CM33 core to the target board.
Launch the debugger in your IDE to begin running the demo.
If building release configuration, start the xt-ocd daemon and download the program for DSP core to the target board. If building debug configuration, launch the Xtensa IDE or xt-gdb debugger to begin running the demo.
Notes:
DSP image can only be debugged using J-Link debugger. See the document ‘Getting Started with Xplorer’ for your particular board for more information.
Example Configuration
The example can be configured by user. Before configuration, please check the table to see if the feature is supported on the development board.
MIMXRT700-EVK Decoder Configuration:
RT700 has limited RAM on Cortex-M33 core 1 which limits the available decoders. Only SBC decoder is enabled by default. In order to enable a different decoder/encoder, it is necessary to define the appropriate define on project level. Use one of the following define from the list of the supported decoders on the HiFi1 core:
XA_AAC_DECODER
XA_MP3_DECODER
XA_SBC_DECODER
XA_VORBIS_DECODER
XA_OPUS_DECODER
Running the Demo
The ARM application will power and clock the DSP, so it must be loaded prior to loading the DSP application. The DSP application can be built by the following tools: Xtensa Xplorer or Xtensa C Compiler. Application for Cortex-M33 can be built by the other toolchains listed in MCUXpresso SDK Release Notes.
The release configurations of the demo will combine both applications into one ARM image. With this, the ARM core will load and start the DSP application on startup. Pre-compiled DSP binary images are provided under dsp/binary/ directory. If you make changes to the DSP application in release configuration, rebuild ARM application after building the DSP application. If you plan to use MCUXpresso IDE for cm33 you will have to make sure that the preprocessor symbol DSP_IMAGE_COPY_TO_RAM, found in IDE project settings, is defined to the value 1 when building release configuration.
The debug configurations will build two separate applications that need to be loaded independently. DSP application can be built by the following tools: Xtensa Xplorer or Xtensa C Compiler. Required tool versions can be found in MCUXpresso SDK Release Notes for the board. Application for cm33 can be built by the other toolchains listed there. If you plan to use MCUXpresso IDE for cm33 you will have to make sure that the preprocessor symbol DSP_IMAGE_COPY_TO_RAM, found in IDE project settings, is defined to the value 0 when building debug configuration. The ARM application will power and clock the DSP, so it must be loaded prior to loading the DSP application.
In order to debug both the Cortex-M33 and DSP side of the application, please follow the instructions:
It is necessary to run the Cortex-M33 side first and stop the application before the DSP_Start function
Run the xt-ocd daemon with proper settings
Download and debug the DSP application
In order to get TRACE debug output from the XAF it is necessary to define XF_TRACE 1 in the project settings. It is possible to save the TRACE output into RAM using DUMP_TRACE_TO_BUF 1 define on project level. Please see the initialization of the TRACE function in the xaf_main_dsp.c file. For more details see XAF documentation.
When the demo runs successfully, the terminal will display the following output (example from MIMXRT700-EVK):
******************************
DSP audio framework demo start
******************************
[CM33 Main] Configure codec
[DSP_Main] Cadence Xtensa Audio Framework
[DSP_Main] Library Name : Audio Framework (Hostless)
[DSP_Main] Library Version : 3.5
[DSP_Main] API Version : 3.2
[DSP_Main] start
[DSP_Main] established RPMsg link
[CM33_Main] DSP image copied to DSP TCM
[CM33_Main][APP_SDCARD_Task] start
[CM33_Main][APP_DSP_IPC_Task] start
[CM33_Main][APP_Shell_Task] start
Copyright 2024 NXP
Type help to see the command list. Similar description will be displayed on serial console (If multi-channel playback mode is enabled, the description is slightly different. Available encoders/decoders may differ - refer to the table.):
"help": List all the registered commands
"exit": Exit program
"version": Query DSP for component versions
"file": Perform audio file decode and playback from SD card
USAGE: file [list|stop|<audio_file>]
list List audio files on SD card available for playback
<audio_file> Select file from SD card and start playback
"decoder": Perform decode on DSP and play to speaker.
USAGE: decoder [aac|mp3|opus|sbc|vorbis_ogg|vorbis_raw]
aac: Decode aac data
mp3: Decode mp3 data
opus: Decode opus data
sbc: Decode sbc data
vorbis_ogg: Decode OGG VORBIS data
vorbis_raw: Decode raw VORBIS data
"encoder": Encode PCM data on DSP and compare with reference data.
USAGE: encoder [opus|sbc]
opus: Encode pcm data using opus encoder
sbc: Encode pcm data using sbc encoder
"src" Perform sample rate conversion on DSP
"gain": Perform PCM gain adjustment on DSP
Xtensa IDE log when command is playing a file (mp3/aac/vorbis/… ):
File playback start, initial buffer size: 16384
[DSP Codec] Audio Device Ready
[DSP Codec] Decoder component started
[DSP Codec] Setting decode playback format:
Decoder : mp3_dec
Sample rate: 16000
Bit Width : 16
Channels : 2
[DSP Codec] Renderer component started
[DSP Codec] Connected XA_DECODER -> XA_RENDERER
[DSP_ProcessThread] start
[DSP_BufferThread] start
Xtensa IDE log when decoder command starts playback successfully:
[DSP_Main] Input buffer addr: 0x20020000, buffer size: 94276
[DSP Codec] Audio Device Ready
[DSP Codec] Decoder created
[DSP Codec] Decoder component started
[DSP Codec] Renderer component created
[DSP Codec] Connected XA_DECODER -> XA_RENDERER
[DSP_ProcessThread] start
[DSP_ProcessThread] Execution complete - exiting
[DSP_ProcessThread] exiting
[DSP Codec] Audio device closed
[CM33 CMD] [APP_DSP_IPC_Task] response from DSP, cmd: 0, error: 0
[CM33 CMD] Decode complete
MIMXRT685-AUD-EVK Multi-channel Support:
The MIMXRT685-AUD-EVK board supports multi-channel audio. When selecting audio files for playback, you can specify the number of channels:
```
file [list|stop|<audio_file> [<nchannel>]]
<nchannel> Select the number of channels (2 or 8 can be selected).
NOTE: Selected audio file must meet the following parameters:
- Sample rate: 96 kHz
- Width: 32 bit
```
Xtensa IDE log when command is playing a PCM file:
```
[DSP_FILE_REN] Audio Device Ready
[DSP_FILE_REN] post-proc/pcm_gain component started
[DSP_FILE_REN] post-proc/client_proxy component started
[DSP_FILE_REN] Connected post-proc/pcm_gain -> post-proc/client_proxy
[DSP_FILE_REN] renderer component started
[DSP_FILE_REN] Connected post-proc/client_proxy -> renderer
[DSP_BufferThread] start
[DSP_ProcessThread] start
[DSP_CleanupThread] start3
```
Known Issues
The “file stop” command doesn’t stop the playback for some small files (with low sample rate).
MIMXRT700-EVK: Has limited RAM on Cortex-M33 core 1 which limits the available decoders.