-
Notifications
You must be signed in to change notification settings - Fork 0
Hardware Reference Edge AI Board
Board Type: imx8mm-jaguar-sentai
Platform: NXP i.MX8MM (Cortex-A53 quad-core)
Description: AI audio processing platform with TAS2563 dual codec, environmental sensors, radar, and wireless tri-radio support
- Module: Quectel FGS060N (based on NXP IW612 chipset)
- Capabilities: WiFi 6 (802.11ax) + Bluetooth 5.4 + 802.15.4 tri-radio
- WiFi: 2.4/5 GHz dual-band 1x1 802.11ax
- Bluetooth: 5.4 with BLE support
- 802.15.4: For Zigbee, Thread, and Matter protocols
- Audio Codec: TAS2563 (I2C address 0x4C)
- Microphones: Dual PDM microphones
- Audio Processing: Support for STT (Speech-to-Text) and TTS (Text-to-Speech)
- Audio Interface: SAI3 (TDM Serial Audio Interface)
- Radar: BGT60TR13C presence detection sensor (SPI2 interface)
- Environmental: SHT40-AD1F-R2 temperature & humidity (I2C 0x44)
- Motion: LIS2DH12 accelerometer (I2C 0x19)
- Temperature: STTS22H precision temperature sensor (I2C 0x3F)
- LEDs: 6x RGBW LEDs via LP5024 driver (I2C 0x28)
- Button: Single tactile switch (GPIO4_IO6)
- USB-C PD: STUSB4500 controller (I2C 0x48) with programmable voltage selection
- Cellular: Quectel LTE modem with power control
- Primary: WiFi 6, Bluetooth 5.4, 802.15.4
- Cellular: Optional Quectel LTE modem (USB interface)
- Programming: USB-C with power delivery
| Function | i.MX8MM Pin | GPIO | Signal |
|---|---|---|---|
| Power Down | MX8MM_IOMUXC_NAND_DATA01_GPIO3_IO7 | GPIO3_IO7 | WIFI_PD# |
| SDIO Interrupt | MX8MM_IOMUXC_NAND_DATA00_GPIO3_IO6 | GPIO3_IO6 | SD1_INT |
| SDIO Clock | MX8MM_IOMUXC_SD1_CLK_USDHC1_CLK | - | USDHC1_CLK |
| SDIO Command | MX8MM_IOMUXC_SD1_CMD_USDHC1_CMD | - | USDHC1_CMD |
| SDIO Data[0-3] | MX8MM_IOMUXC_SD1_DATA[0-3]_USDHC1_DATA[0-3] | - | USDHC1_DATA[0-3] |
| WLAN Reset | MX8MM_IOMUXC_SD1_RESET_B_USDHC1_RESET_B | - | WLAN_RST |
| WLAN Wake Host | MX8MM_IOMUXC_SD1_DATA7_GPIO2_IO9 | GPIO2_IO9 | WL_WAKE_HOST |
| Host Wake WLAN | MX8MM_IOMUXC_SD1_DATA4_GPIO2_IO6 | GPIO2_IO6 | WL_WAKE_DEV |
| Function | i.MX8MM Pin | GPIO | Signal |
|---|---|---|---|
| UART TX | MX8MM_IOMUXC_UART1_TXD_UART1_DCE_TX | - | UART1_TX |
| UART RX | MX8MM_IOMUXC_UART1_RXD_UART1_DCE_RX | - | UART1_RX |
| UART CTS | MX8MM_IOMUXC_UART3_RXD_UART1_DCE_CTS_B | - | UART1_CTS |
| UART RTS | MX8MM_IOMUXC_UART3_TXD_UART1_DCE_RTS_B | - | UART1_RTS |
| BT Reset | MX8MM_IOMUXC_NAND_DATA03_GPIO3_IO9 | GPIO3_IO9 | BT_RST# |
| BT Wake Host | MX8MM_IOMUXC_NAND_DATA02_GPIO3_IO8 | GPIO3_IO8 | BT_WAKE_HOST |
| Host Wake BT | MX8MM_IOMUXC_SD1_DATA5_GPIO2_IO7 | GPIO2_IO7 | BT_WAKE_DEV |
| Function | i.MX8MM Pin | GPIO | Signal |
|---|---|---|---|
| SPI Interrupt | MX8MM_IOMUXC_SAI2_RXC_GPIO4_IO22 | GPIO4_IO22 | ZB_INT |
| SPI Chip Select | MX8MM_IOMUXC_ECSPI1_SS0_ECSPI1_SS0 | - | SPI1_SS0 |
| SPI Clock | MX8MM_IOMUXC_ECSPI1_SCLK_ECSPI1_SCLK | - | SPI1_SCLK |
| SPI MISO | MX8MM_IOMUXC_ECSPI1_MISO_ECSPI1_MISO | - | SPI1_MISO |
| SPI MOSI | MX8MM_IOMUXC_ECSPI1_MOSI_ECSPI1_MOSI | - | SPI1_MOSI |
| ZB Reset | MX8MM_IOMUXC_SD1_STROBE_GPIO2_IO11 | GPIO2_IO11 | ZB_RST# |
| Function | i.MX8MM Pin | GPIO | Signal |
|---|---|---|---|
| Master Clock | MX8MM_IOMUXC_SAI3_MCLK_SAI3_MCLK | - | SAI3_MCLK |
| Bit Clock | MX8MM_IOMUXC_SAI3_TXC_SAI3_TX_BCLK | - | SAI3_TXC |
| Frame Sync | MX8MM_IOMUXC_SAI3_TXFS_SAI3_TX_SYNC | - | SAI3_TXFS |
| TX Data | MX8MM_IOMUXC_SAI3_TXD_SAI3_TX_DATA0 | - | SAI3_TXD |
| RX Data | MX8MM_IOMUXC_SAI3_RXD_SAI3_RX_DATA0 | - | SAI3_RXD |
| Codec Shutdown | MX8MM_IOMUXC_SPDIF_RX_GPIO5_IO4 | GPIO5_IO4 | CODEC_SD# |
| Codec Interrupt | MX8MM_IOMUXC_SAI1_TXFS_GPIO4_IO10 | GPIO4_IO10 | CODEC_IRQ# |
| I2C Clock | MX8MM_IOMUXC_I2C2_SCL_I2C2_SCL | - | I2C2_SCL |
| I2C Data | MX8MM_IOMUXC_I2C2_SDA_I2C2_SDA | - | I2C2_SDA |
| Function | i.MX8MM Pin | GPIO | Signal |
|---|---|---|---|
| PDM Clock | MX8MM_IOMUXC_SAI1_MCLK_PDM_CLK | - | PDM_CLK |
| Mic 1 Data | MX8MM_IOMUXC_SAI1_RXD0_PDM_DATA0 | - | PDM_D0 |
| Mic 2 Data | MX8MM_IOMUXC_SAI1_RXD1_PDM_DATA1 | - | PDM_D1 |
| Function | i.MX8MM Pin | GPIO | Signal |
|---|---|---|---|
| SPI Clock | MX8MM_IOMUXC_ECSPI2_SCLK_ECSPI2_SCLK | - | SPI2_SCLK |
| SPI Chip Select | MX8MM_IOMUXC_ECSPI2_SS0_ECSPI2_SS0 | - | SPI2_SS0 |
| SPI MOSI | MX8MM_IOMUXC_ECSPI2_MOSI_ECSPI2_MOSI | - | SPI2_MOSI |
| SPI MISO | MX8MM_IOMUXC_ECSPI2_MISO_ECSPI2_MISO | - | SPI2_MISO |
| Radar Interrupt | MX8MM_IOMUXC_SAI3_RXC_GPIO4_IO29 | GPIO4_IO29 | RADAR_IRQ |
| Radar Reset | MX8MM_IOMUXC_SAI3_RXFS_GPIO4_IO28 | GPIO4_IO28 | RADAR_RST |
| Function | i.MX8MM Pin | GPIO | Signal |
|---|---|---|---|
| LED Enable | MX8MM_IOMUXC_SAI1_RXD7_GPIO4_IO9 | GPIO4_IO9 | LED_ENABLE |
| I2C Clock | MX8MM_IOMUXC_I2C3_SCL_I2C3_SCL | - | I2C3_SCL |
| I2C Data | MX8MM_IOMUXC_I2C3_SDA_I2C3_SDA | - | I2C3_SDA |
| Function | i.MX8MM Pin | GPIO | Signal | Device |
|---|---|---|---|---|
| Button Switch | MX8MM_IOMUXC_SAI1_RXD4_GPIO4_IO6 | GPIO4_IO6 | SLMB_0 | Switch |
| Accelerometer INT1 | MX8MM_IOMUXC_SAI1_RXD5_GPIO4_IO7 | GPIO4_IO7 | SLMB_1 | LIS2DH12 |
| Temperature INT | MX8MM_IOMUXC_SAI1_RXD6_GPIO4_IO8 | GPIO4_IO8 | SLMB_2 | STTS22H |
| I2C Clock | MX8MM_IOMUXC_I2C3_SCL_I2C3_SCL | - | I2C3_SCL | All I2C3 devices |
| I2C Data | MX8MM_IOMUXC_I2C3_SDA_I2C3_SDA | - | I2C3_SDA | All I2C3 devices |
| Function | i.MX8MM Pin | GPIO | Signal |
|---|---|---|---|
| USB Alert | MX8MM_IOMUXC_SAI2_RXFS_GPIO4_IO21 | GPIO4_IO21 | USB_ALERT# |
| USB Reset | MX8MM_IOMUXC_SAI1_TXC_GPIO4_IO11 | GPIO4_IO11 | USB_RESET |
| I2C Clock | MX8MM_IOMUXC_I2C1_SCL_I2C1_SCL | - | I2C1_SCL |
| I2C Data | MX8MM_IOMUXC_I2C1_SDA_I2C1_SDA | - | I2C1_SDA |
| Function | i.MX8MM Pin | GPIO | Signal |
|---|---|---|---|
| USB Data+ | USB2_DP | - | USB2_DP |
| USB Data- | USB2_DN | - | USB2_DN |
| Modem Reset | MX8MM_IOMUXC_SD1_DATA6_GPIO2_IO8 | GPIO2_IO8 | Modem Reset |
| Modem Power | MX8MM_IOMUXC_NAND_CE0_B_GPIO3_IO1 | GPIO3_IO1 | FUL_CARD_POWER_OFF# |
The TAS2563 audio codec is used for output. Currently we've back ported an older TI driver as a kernel module. In future we plan to migrate to a backport of the newer TAS2781 driver.
The driver downloads a pre-built audio firmware binary to the TAS2563 (/lib/firmware/tas2563_uCDSP.bin). There is also a calibration file which is not currently supported.
The firmware can be built with the TI graphical tool for Windows which can be found here.
Further resources can be found here.
The driver for this is a module which loads as snd_soc_tas2563 (use lsmod to view)
We blacklist automatic loading of audio drivers in /etc/modprobe.d/blacklist.conf as otherwise card IDs can change depending on load order. Instead a systemd service audio-driver runs on startup and executes /usr/bin/load-audio-drivers.sh to load in the relevant drivers
When loaded aplay -l can be executed to show device details
root@edge-ai-board:/var/rootdirs/home/fio# aplay -l
**** List of PLAYBACK Hardware Devices ****
card 1: tas2563audio [tas2563-audio], device 0: 30030000.sai-tas2563 ASI1 tas2563 ASI1-0 [30030000.sai-tas2563 ASI1 tas2563 ASI1-0]
Subdevices: 1/1
Subdevice #0: subdevice #0
To play a sample wav file the following command can be used
aplay -Dhw:1,0 -r 48000 -c 2 sample.wav
NOTE: Currently when using the hardware device directly only 2 channnels of 48kHz audio are supported.
We need to be running audio within docker containers. We seem to be able to use ALSA but some .NET code fails with ALSA. So we've added PulseAudio to the host OS mage and can use this instead of ALSA.
There are PulseAudio equivalents to ALSA record and playback utilities. For example
parecord file.wav
paplay file.wav
There is also a command line control utility, for example
pactl list sources
pactl list sinks
pactl list modules
NOTE: That the pulseaudio server runs as non-root fio user. It's not possible to interact with the server as the root user and this will fail.
The docker container needs to have access to the host os pulse audio socket and dbus. An example docker-compose.yml configuration to achieve this looks something like this
version: '2'
services:
Example:
build: .
image: hub.foundries.io/dynamic-devices/example:latest
devices:
- /dev/snd:/dev/snd
environment:
- PULSE_SERVER=unix:/tmp/pulseaudio.socket
- PULSE_COOKIE=/tmp/pulseaudio.cookie
volumes:
- "/tmp:/tmp"
- "/run/dbus/system_bus_socket:/run/dbus/system_bus_socket"
restart: always
privileged: true
TODO: We should support cookies for authentication but this is not yet implemented. Instead we allow unauthenticated to the host pulseaudio server
To provide the socket in a known place, /tmp/pulseaudio.socket we have a script that runs the following command on start-up
pactl load-module module-native-protocol-unix socket=/tmp/pulseaudio.socket auth-anonymous=true
Then you can enter a docker container configured as above and run the paplay commands or similar
SECURITY NOTE: That usually PulseAudio would run within a user session. This makes sense for desktop/laptop systems but is non-ideal for embedded systems. Instead we run PulseAudio in the systemwide configuration so we don't have to worry about user login. There are potentially some security and other issues with running in this configuration. These should be noted and can be found here.
To see current IW612 firmware version use
cat /proc/mwlan/adapter0/wlan0/info | grep version
driver_version = SDIW612---18.99.2.p66.10-MM6X18423.p6-GPL-(FP92)
firmware_major_version=18.99.2
Run ifconfig to view the wlan0 device (there is no wired ethernet)
Run the following to add a WiFi connection
nmcli con add type wifi con-name $CONNECTIONAME ssid "$SSID" 802-11-wireless-security.key-mgmt WPA-PSK 802-11-wireless-security.psk "$PASSWORD" ifname wlan0
To see connection status use:
nmcli
If a Quectel modem module is installed it is posssible to see the modem ID with:
mmcli -L
Then use this to get the modem status
mmcli -m $MODEM_ID
To test bluetooth is working turn on the device and perform a scan
bluetoothctl power on
bluetoothctl scan on
or
bluetoothctl scan le
TBD
TBD
We use a STUSB4500 to manage USB-C power. This allows us automatically to switch a PSU between different voltages e.g. 5V, 9V, 20V.
The datasheet is here
And the register map is here
We have a tool that allows us to program the NVM in this part to control which settings should be used, the "PDOS". For details on this tool see the repository here.
To write a binary file of 40 bytes of NVM settings
stusb4500 write --file $input_filename
We also have own own current NVM setting binary in the /firmware tree so to program this use
stusb4500 write --file /lib/firmware/stusb4500.dat
To display the current settings:
stusb4500 read
To store the current settings:
stusb4500 read --file $output_file
This tool can also be used to show the current voltage, current consumption and related information. Example usage:
stusb4500 status
The button interrupt is not currently implemented but to test via polling use
echo 102 > /sys/class/gpio/export
cat /sys/class/gpio/gpio102/value
State is 1 when not depressed and goes to 0 when depressed.
We've integrated the Infineon spi-lib driver which talks to the part here
You can run this for test and it will generate a log file with copious numbers of data-points
seamless_dev_spi spi.mode=landscape rec.file=test.dat
The RADAR-SDK analysis library implements core protocols and we have this build for aarch64 here. The SDK talks to Infineon microcontroller firmware using a protocol which seems to be named "stratula" or similar. It can use USB/Serial and other transports.
As we don't have a microcontroller on our boards we need to interface the SDK to the code that accesses the part over SPI. We have been provided with spi-lib which we have here
We've then made changes to spi-lib to implement the needed protocol to communicate with the SDK code and we do this via a virtual serial port implementation.
Currently spi-lib expects to open a /dev/ttySpiLib device which can be linked to a virtual serial port.
A way of creating a virtual serial port pair is to use
sudo socat -d -d pty,raw,echo=0,link=/dev/ttyAMA0 pty,raw,echo=0,link=/tmp/ttySpiLib
This will give two /dev/pts/X devices which you than then create symlinks to.
NOTE: The RADAR-SDK automatically searches on /dev/ttyUSB*, /dev/ttyAMA* so we can create a /dev/ttyAMA0 and the spi-lib is currently hard-coded to use /tmp/ttySpiLib
This is implemented as an I2C device using the SHT4X IIO driver. The lm-sensors framework is installed to the image. To see sensor details use sensors.
root@edge-ai-board:~# sensors
sht4x-i2c-2-44
Adapter: 30a40000.i2c
temp1: +32.0 C
humidity1: 33.0 %RH
root@edge-ai-board:~# cat /sys/bus/iio/devices/iio\:device0/name
lis2dh12_accel
root@edge-ai-board:~# echo 400 > /sys/bus/iio/devices/iio\:device0/sampling_frequency
root@edge-ai-board:~# cat /sys/bus/iio/devices/iio\:device0/in_accel_x_raw
336
root@edge-ai-board:~# cat /sys/bus/iio/devices/iio\:device0/in_accel_x_raw
-8
The STT22H driver is implemented within the IIO subsystem. To check the temperature you can use:
root@edge-ai-board:~# cat /sys/bus/iio/devices/iio\:device1/name
stts22h
root@edge-ai-board:~# cat /sys/bus/iio/devices/iio\:device1/in_temp_ambient_raw
3073
root@edge-ai-board:~# cat /sys/bus/iio/devices/iio\:device1/in_temp_ambient_scale
10.000000000
- Pulse RGBW LED intensity "heartbeat"
test-leds-hb.sh - Rotate R, G, B, W in ring
test-leds-rc.sh
This requires a specific image lmp-factory-image-ce to be programmed onto the test board.
Individual tests are implemented as systemd services which automatically start and can also be stopped and/or disabled.
| Test | Description |
|---|---|
| ce-audio-test.service | Plays a tune on loop via PulseAudio |
| ce-led-test.service | Sets LEDs to 50% green |
| ce-mic-test.service | Records via PulseAudio to /dev/null |
| ce-wifi-bt-test.service | TBD |
| ce-radar-test.service | Runs seamless_dev_spi outputting data to /dev/null |
To view status of a service
systemctl status ce-foo-test
To stop a service when running
sudo systemctl stop ce-foo-test
To disable a service from running automatically
sudo systemctl disable ce-foo-test
To look at output from a service (and "follow" output)
sudo journalctl -u ce-foo-test -f
There are also some scripts which can be run manually as needed
To turn off the modem use
sudo ce-modem-pwrdown.sh
Note that this will turn off the modem and it will disconnect from the USB bus but it then powers up and re-enumerates. This issue is addressed in the latest revision of boards.
To turn off the WiFi/BT
ifconfig wlan0 down
gpioset gpiochip2 7=0
On the AI board run
iperf3 -s
On the monitoring device run
iperf3 -c $BOARD_IP_ADDRESS -V -b 10M -t 50