You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/learning-paths/embedded-and-microcontrollers/cmsisdsp-dev-with-python/how-to-1.md
+13-7Lines changed: 13 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,17 +8,23 @@ layout: learningpathall
8
8
9
9
## What is CMSIS-DSP?
10
10
11
-
CMSIS-DSP is a general-purpose compute library with a focus on DSP. It was initially developed for Cortex-M processors and has recently been upgraded to also support Cortex-A.
11
+
CMSIS-DSP is a general-purpose computation library focused on digital signal processing (DSP).
12
12
13
-
On each processor, CMSIS-DSP is optimized for the architecture: DSP extensions on M4 and M7; Helium on M55 and M85; Neon on A55, etc.
13
+
Originally developed for Cortex-M processors, it now also supports Cortex-A.
14
14
15
-
## What is the CMSIS-DSP Python package?
15
+
The library is optimized for each architecture:
16
+
17
+
- DSP extensions on Cortex-M4 and M7.
18
+
- Helium on M55 and M85.
19
+
- Neon on Cortex-A55 and other Cortex-A cores.
16
20
17
-
The CMSIS-DSP Python package is a Python API for CMSIS-DSP. Its goal is to make it easier to develop a C solution using CMSIS-DSP by decreasing the gap between a design environment like Python and the final C implementation.
21
+
## What is the CMSIS-DSP Python package?
22
+
23
+
The CMSIS-DSP Python package provides a Python API for CMSIS-DSP. Its goal is to make it easier to develop a C solution using CMSIS-DSP by bridging the gap between a Python-based design environment and a final C implementation.
18
24
19
-
For this reason, the Python API is as close as possible to the C one.
25
+
The API is designed to closely mirror the C version in both function and structure.
20
26
21
-
Fixed-point arithmetic is rarely provided by Python packages, which generally focus on floating-point operations. The CMSIS-DSP Python package provides the same fixed-point arithmetic functions as the C version: Q31, Q15 and Q7. The package also provides floating-point functions and will also support half-precision floats in the future, like the C API.
27
+
Fixed-point arithmetic is rarely supported by Python libraries, which generally focus on floating-point operations. The CMSIS-DSP Python package includes the same fixed-point arithmetic functions as the C version: Q31, Q15 and Q7. Support for half-precision floats will be added in a future release, matching the C API.
22
28
23
-
Finally, the CMSIS-DSP Python package is compatible with NumPy and can be used with all other scientific and AI Python packages such as SciPy and PyTorch.
29
+
The package is compatible with NumPy and integrates well with scientific and AI libraries such as SciPy and PyTorch.
Copy file name to clipboardExpand all lines: content/learning-paths/embedded-and-microcontrollers/cmsisdsp-dev-with-python/how-to-2.md
+29-12Lines changed: 29 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,47 +1,64 @@
1
1
---
2
-
title: Install the Python packages
2
+
title: Set up environment
3
3
weight: 3
4
4
5
5
### FIXED, DO NOT MODIFY
6
6
layout: learningpathall
7
7
---
8
8
9
-
The application you will develop requires a few additional Python packages besides CMSIS-DSP. These need to be installed before you start writing code.
9
+
## Create a Python virtual environment
10
10
11
-
You should install the packages in a Python virtual environment. For example, you can use:
11
+
To follow this Learning Path, you'll need to install a few additional Python packages alongside CMSIS-DSP.
12
12
13
-
```
13
+
Start by installing the packages in a Python virtual environment. For example, you can use:
14
+
15
+
```bash
14
16
python -m venv cmsis-dsp-venv
15
17
```
18
+
If required, activate the environment.
19
+
20
+
## Install CMSIS-DSP
16
21
17
-
The first package to install is CMSIS-DSP:
22
+
Now install the required packages, starting with CMSIS-DSP:
18
23
19
24
```bash
20
25
pip install cmsisdsp
21
26
```
22
-
It will also install `NumPy`, which is a dependency of the CMSIS-DSP Python package.
27
+
This will also install `NumPy`, which is a dependency of the CMSIS-DSP Python package.
23
28
24
-
You'll be working with a Jupyter notebook, so the `jupyter` package must also be installed:
29
+
## Install Jupyter
30
+
31
+
You'll be working with a Jupyter notebook, so install the `jupyter` package:
25
32
26
33
```bash
27
34
pip install jupyter
28
35
```
36
+
## Install additional tools
29
37
30
-
Finally, you'll need packages to read sound files, play sound using widgets, and display plots:
38
+
Finally, you'll also need packages for reading sound files, playing sound using widgets, and displaying plots:
31
39
32
40
```bash
33
41
pip install soundfile ipywidgets matplotlib
34
42
```
35
43
44
+
## Launch Jupyter and set up your notebook
45
+
36
46
You can now launch the Jupyter notebook with the following command:
37
47
38
48
```bash
39
49
jupyter notebook
40
50
```
41
-
A browser window should open showing the source tree your terminal launched from. Create a new Jupyter notebook by clicking the `New` dropdown and selecting `Python 3 (ipykernel)`. The new notebook will be named `Untitled`. Rename it to something more descriptive, for example `cmsis-dsp`.
51
+
A browser window should open, displaying the source tree.
52
+
53
+
Create a new Jupyter notebook by clicking **New** and selecting **Python 3 (ipykernel)**. The new notebook is called `Untitled`. Rename it to something descriptive, for example `cmsis-dsp`.
54
+
55
+
Now import all the required packages by copying and running the following Python code into your notebook and run the cell (Shift+Enter).
56
+
57
+
{{% notice Note%}}
58
+
All the Python code blocks in this Learning Path are intended to be executed in the same Jupyter notebook.
59
+
{{% /notice %}}
60
+
42
61
43
-
You can now import all the required packages. Copy the following Python code into your notebook and run the cell (Shift+Enter).
44
-
All the Python code blocks in this learning path are intended to be executed in the same Jupyter notebook.
45
62
46
63
```python
47
64
import cmsisdsp as dsp
@@ -65,4 +82,4 @@ import soundfile as sf
65
82
from urllib.request import urlopen
66
83
```
67
84
68
-
You're now ready to move on to the next steps, where you will set up some audio files for processing.
85
+
You're now ready to move on and set up the audio files you'll use for processing.
You can use it to listen to the audio. You'll hear a sequence of the words "yes" and "no", with some noise between them. The goal of this learning path is to design an algorithm to remove the noise.
30
+
You'll hear a sequence of the words "yes" and "no" with background noise. The goal of this Learning Path is to design an algorithm that removes that noise.
31
31
32
-
Next, convert the audio into a NumPy array so that it can be processed using CMSIS-DSP:
32
+
### Convert the Audio to a NumPy array
33
+
34
+
Convert the audio into a NumPy array so it can be processed with CMSIS-DSP:
33
35
34
36
```python
35
37
data, samplerate = sf.read(io.BytesIO(filedata))
36
-
iflen(data.shape)>1:
37
-
data=data[:,0]
38
+
iflen(data.shape)>1:
39
+
data=data[:,0]
38
40
data = data.astype(np.float32)
39
-
data=data/np.max(np.abs(data))
41
+
data=data/np.max(np.abs(data))
40
42
dataQ15 = fix.toQ15(data)
41
43
```
42
44
43
45
The code above does the following:
44
-
- Converts the audio into a NumPy array
45
-
- If the audio is stereo, only one channel is kept
46
-
- Normalizes the audio to ensure no value exceeds 1
47
-
- Converts the audio to Q15 fixed-point representation to enable the use of CMSIS-DSP fixed-point functions
46
+
- Converts the audio to a NumPy array.
47
+
- Keeps only one channel if the audio is stereo.
48
+
- Normalizes the audio to ensure no value exceeds 1.
49
+
- Converts the audio to Q15 fixed-point representation to enable the use of CMSIS-DSP fixed-point functions.
50
+
51
+
### Plot the audio signal
48
52
49
53
Now, plot the audio waveform:
50
54
@@ -53,27 +57,30 @@ plt.plot(data)
53
57
plt.show()
54
58
```
55
59
56
-
You'll get the following output:
60
+
You'll see:
61
+
62
+
![audio signal alt-text#center](signal.png"Audio signal")
57
63
58
-
![audio signal alt-text#center](signal.png"Figure 2. Audio signal")
64
+
The waveform shows a sequence of words. Between the words, the signal is not zero; there is some noise.
59
65
60
-
In the picture, you can see a sequence of words. Between the words, the signal is not zero: there is some noise.
66
+
### Prepare for block-based processing
61
67
62
-
In a real application, you don't wait for the entire signal to be received. The signal is continuous. The samples are processed as they are received. Processing can either be sample-based or block-based. For this learning path, the processing will be block-based.
68
+
In real applications, audio streams are processed in real time. Processing can either be sample-based or block-based. For this Learning Path, the processing is block-based.
63
69
64
-
Before you can move to the next step, this signal must be split into blocks. The processing will occur on small blocks of samples of a given duration, known as windows.
70
+
Before you can move to the next step, this signal must be split into overlapping blocks. The processing will occur on small blocks of samples of a given duration, known as windows.
Refer to the [NumPy documentation](https://numpy.org/doc/stable/reference/generated/numpy.lib.stride_tricks.sliding_window_view.html) for details about `sliding_window_view`. It's not the most efficient function, but it is sufficient for this tutorial. The signal is split into overlapping blocks: each block reuses half of the samples from the previous block as defined by the `winOverlap` variable.
77
83
78
-
By running that last block, you have an audio signal that has been split into overlapping blocks. The next step is to do some processing on those blocks.
84
+
By running that last block, you have an audio signal that has been split into overlapping blocks.
79
85
86
+
The next step is to do some processing on those blocks.
Copy file name to clipboardExpand all lines: content/learning-paths/embedded-and-microcontrollers/cmsisdsp-dev-with-python/how-to-4.md
+15-11Lines changed: 15 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,11 +8,13 @@ layout: learningpathall
8
8
9
9
## Write a simple voice activity detection
10
10
11
-
To remove the noise between speech segments, you need to detect when voice is present. Voice activity detection (VAD) can be complex, but for this learning path, you'll implement a very simple and naive approach based on _energy_. The idea is that if the environment isn't too noisy, speech should have more energy than the noise.
11
+
To remove the noise between speech segments, you need to detect when voice is present. Voice activity detection (VAD) can be complex, but for this Learning Path, you'll implement a simple energy-based approach. The idea is that if the environment isn't too noisy, speech should have more energy than noise.
12
12
13
-
The detection will rely on a comparison with a threshold that must be manually tuned. That threshold will be hard-coded, which might not work in a real-life use-case, but is a sufficient solution for this learning path. You'll first implement a version of the VAD with NumPy, which will serve as a reference. Then you'll implement the same version using CMSIS-DSP with the Q15 fixed-point format.
13
+
The detection relies on a comparison with a threshold that must be manually tuned. That threshold is hard-coded, which might not work in a real-life usecase, but is a sufficient solution for this Learning Path.
14
14
15
-
Throughout this section, you should copy the code-blocks and run them in your Jupyter notebook.
15
+
You'll first implement a version of the VAD with NumPy, which serves as a reference. Then you'll implement the same version using CMSIS-DSP with the Q15 fixed-point format.
16
+
17
+
Throughout this section, you should copy the code blocks and run them in your Jupyter notebook.
16
18
17
19
### NumPy VAD
18
20
@@ -56,7 +58,7 @@ vad = np.array([[w]*(winLength-winOverlap) for w in cleaned]).flatten()
56
58
ax.plot(data)
57
59
ax.plot(vad)
58
60
```
59
-
![vad alt-text#center](vad.png"Figure 3. VAD reference implementation")
This plot shows you that the reference implementation works. The next step is to implement a similar graph using CMSIS-DSP.
62
64
@@ -65,13 +67,15 @@ This plot shows you that the reference implementation works. The next step is to
65
67
#### Energy
66
68
First, you need to compute the signal energy from audio in Q15 format using CMSIS-DSP.
67
69
68
-
If you look at the CMSIS-DSP documentation, you'll see that the `power` and `vlog` functions don't produce results in Q15 format. Tracking the fixed-point format throughout all lines of an algorithm can be challenging. In this example, this means that:
70
+
If you look at the CMSIS-DSP documentation, you'll see that the `power` and `vlog` functions don't produce results in Q15 format. Tracking the fixed-point format throughout all lines of an algorithm can be challenging.
71
+
72
+
In this example, this means that:
69
73
70
74
* Subtracting the mean to center the signal - as you did in the reference implementation - is handled in CMSIS-DSP by negating the mean and applying it as an offset to the window. Using CMSIS-DSP, `arm_negate_q15` is needed to avoid saturation issues that could prevent the value sign from changing (`0x8000` remaining unchanged as `0x8000`). In practice, the mean should be small, and there should be no difference between `-` and `dsp.arm_negate_q15`. However, it is good practice to avoid using `-` or `+` in a fixed-point algorithm when translating it to CMSIS-DSP function calls.
71
-
* The resulting `energy` and `dB` values are not in Q15 format because the `power` and `vlog` functions are used
72
-
* The multiplication by 10 from the reference implementation is missing
75
+
* The resulting `energy` and `dB` values are not in Q15 format because the `power` and `vlog` functions are used.
76
+
* The multiplication by 10 from the reference implementation is missing.
73
77
74
-
This means that the `signal_energy_q15` will have a different output than the above implementation. Instead of trying to determine the exact fixed-point format of the output and applying the necessary shift to adjust the output's fixed-point format, you will address it in the next step by tuning the threshold of the detection function.
78
+
This means that the `signal_energy_q15` will have a different output than the above implementation. Instead of trying to determine the exact fixed-point format of the output and applying the necessary shift to adjust the output's fixed-point format, you will address it in the next step by tuning the threshold of the detection function:
75
79
76
80
77
81
```python
@@ -91,7 +95,7 @@ def signal_energy_q15(window):
91
95
92
96
#### VAD
93
97
94
-
The comparison function is very similar to the NumPy reference, but the threshold is different:
98
+
The comparison function is similar to the NumPy reference, but the threshold is different:
95
99
96
100
```python
97
101
defsignal_vad_q15(window):
@@ -102,12 +106,12 @@ def signal_vad_q15(window):
102
106
```
103
107
104
108
{{% notice Note %}}
105
-
In C code, you would hard-code the output of `fix.toQ15(-0.38)`. `fix.toQ15` is a utility of the Python package to convert float to fixed-point, but it is not available in the CMSIS-DSP C implementation. CMSIS-DSP C has functions like `arm_float_to_q15` which work on arrays and are meant to be used at runtime. If you need a pre-computed constant, you can use a utility function like `fix.toQ15` during development and use the resulting value in the C code.
109
+
In C code, you can hard-code the output of `fix.toQ15(-0.38)`. `fix.toQ15` is a utility of the Python package to convert float to fixed-point, but it is not available in the CMSIS-DSP C implementation. CMSIS-DSP C provides functions like `arm_float_to_q15` which operate on arrays and are intended for use at runtime. To define a precomputed constant, use a utility function like `fix.toQ15` during development and copy the resulting value into your C code.
106
110
{{% /notice %}}
107
111
108
112
#### Plot the Q15 implementation
109
113
110
-
The clean VAD function is now the same for both the NumPy and Q15 versions. You can check whether the Q15 version is working by plotting the signal and the output of the Q15 VAD algorithm.
114
+
The clean VAD function is now the same for both the NumPy and Q15 versions. You can check whether the Q15 version is working by plotting the signal and the output of the Q15 VAD algorithm:
0 commit comments