Merge pull request #76 from zhodj/master

HLJU_Pathological_Voice_Diagnosis

Author: YuguoWH

arc_design_contest/2021/HLJU_Pathological_Voice_Diagnosis/README.md

@@ -0,0 +1,128 @@
+# Pathological Voice Diagnosis
+
+* [Introduction](#introduction)
+
+	* [Actual Picture](#actual-picture)
+	* [Function](#function)
+	* [System Architecture](#system-architecture)
+* [Hardware and Software Setup](#hardware-and-software-setup)
+	* [Required Hardware](#required-hardware)
+	* [Required Software](#required-software)
+	* [Hardware Connection](#hardware-connection)
+* [User Manual](#user-manual)
+	* [Before Running This Application](#before-running-this-application)
+	* [Run This Application](#run-this-application)
+
+## Introduction
+This application is based on the ARC IoT Development Kit which work for Pathological speech diagnosis at the edge.Firstly, the diagnostic model of pathological speech classification was trained based on lightweight RBK-NET network, and then the model was deployed using TensorFlow Lite for Microcontrollers (TFLM). Finally, the model was imported into ARC IoTDK development board through OSP.To achieve the Pathological speech diagnosis detection at the edge.
+
+DEMO VIDEO:[Video presentation](https://v.youku.com/v_show/id_XNTE5MjY4NTIzNg==.html)
+
+### Actual Picture
+
+![actual_picture](https://github.com/zhodj/embarc_applications/blob/master/arc_design_contest/2021/HLJU_Pathological_Voice_Diagnosis/images/actual_picture.jpeg)
+
+### Function
+
+- Pathological Voice Diagnosis。At present, healthy and pathological speech can be recognized
+
+- Display the current voice status
+
+### System Architecture
+
+![system_architecture](https://github.com/zhodj/embarc_applications/blob/master/arc_design_contest/2021/HLJU_Pathological_Voice_Diagnosis/images/system_architecture.png)
+
+## Hardware and Software Setup
+### Required Hardware
+- ARC IoT Development Kit
+
+	![ARC IoT Development Kit](https://github.com/zhodj/embarc_applications/blob/master/arc_design_contest/2021/HLJU_Pathological_Voice_Diagnosis/images/arc_iot_development_kit.jpg)
+
+- WM8978 CODEC module
+
+	![WM8978 CODEC module](https://github.com/zhodj/embarc_applications/blob/master/arc_design_contest/2021/HLJU_Pathological_Voice_Diagnosis/images/WM8978.png)
+
+- OLED MC096GX (0.96inch )
+
+	![OLED MC096GX](https://github.com/zhodj/embarc_applications/blob/master/arc_design_contest/2021/HLJU_Pathological_Voice_Diagnosis/images/oled.png)
+
+### Required Software
+- DesignWare ARC MetaWare Development Toolkit
+- TensorFlow Lite for Microcontrollers 
+- emcARC OSP (github master branch)
+- Serial port terminal, such as putty, minicom or cutecom
+
+### Hardware Connection
+
+
+1.Connect I2C pins of WM8978 to I2C0 of IoT DK.
+
+2.Connect I2S pins of WM8978 to I2S_RX of IoT DK.**The REF_CLK should not be connected untile the initialization of WM8978 is finished**.
+
+3.OLED MC096GX pins.
+
+|OLED MC096GX pins|IoT DK pins  |
+|-----------------|-------------|
+|GND              |GND          |
+|VCC              |3.3V         |
+|SCL              |I2C0_SCL     |
+|SDA              |I2C0_SDA     |
+
+4.I2C series connection.
+
+![I2C Series Connection](https://github.com/zhodj/embarc_applications/blob/master/arc_design_contest/2021/HLJU_Pathological_Voice_Diagnosis/images/i2c_series_connection.png)
+
+
+## User Manual
+
+### Before Running This Application
+- Download source code of Pathological Voice Diagnosis from github.
+- Make sure all the connections are correct.
+- The link script file has been modified to ensure the heap and stack are large enough and the addresses of program sections have also been redirected.Make sure the link script is correct.
+- Use MWDT compiler tool to compile the source code, which includes Tensorflow Lite and the RBK-NET binary model.
+
+### Run This Application
+
+- Open serial terminal and configure it to right COM port and 115200bps.
+- Download with USB-JTAG or use bootloader to boot the program.
+- Connect the MCLK(REF_CLK) wire when seeing "Please connet the mclk !"at serial terminal.
+- Speak to the Mic. After some seconds, you'll see pathological or healthy result shows on the OLED screen.
+
+#### Makefile
+
+- Selected MLI Lib here, then you can use MLI Lib API in your application:
+
+		LIB_SEL = embarc_mli
+
+- Target options about IoT DK and toolchain:
+
+		BOARD = iotdk
+		BD_VER = 10
+		TOOLCHAIN = mw
+
+- The middleware used in your application:
+
+		MID_SEL = common u8glib
+
+See [ embARC Example User Guide][40], **"Options to Hard-Code in the Application Makefile"** for more detailed information about **Makefile Options**.
+
+#### Flie structure
+
+|  file               |            Function                      |
+| ------------------- | -----------------------------------------|
+|  tensorflow         |  Tensorflow Lite Micro                   |
+|  third_party        |  Tensorflow Lite Micro third party       |
+|  main.cc            |  Initialization and main loop            |
+|  Makefile           |  Makefile                                |
+|  mfcc.cc/mfcc.h     |  Algorithm of extracting mfcc features   |
+|  codec.cc/codec.h   |  Recive and store audio frames(interrupt)|
+|  FFT.cc/FFT.h       |  Fast Fourier transform                  |
+|  wm8978.c/wm8978.h  |  Config wm8978 through I2C               |
+|  wm8978i2s.c/.h     |  I2S driver                              |
+
+[0]: ./images/System_Architecture.PNG           "system_architecture"
+[1]: ./images/ARC_IoT_Development_Kit.jpg       "ARC IoT Development Kit"
+[2]: ./images/WM8978_CODEC_module.jpg           "WM8978 CODEC module"
+[3]: ./images/OLED(MC096GX).jpg                 "OLED MC096GX"
+
+[40]: http://embarc.org/embarc_osp/doc/embARC_Document/html/page_example.html   " embARC Example User Guide"

arc_design_contest/2021/HLJU_Pathological_Voice_Diagnosis/images/WM8978.png

@@ -0,0 +1,154 @@
+#include "stdlib.h"
+#include "math.h"
+
+//#define __EXTERN_W__
+
+typedef struct
+{
+	float real;
+	float imag;
+} Complex;
+
+#ifdef __EXTERN_W__
+//========Paste the definitions for W here========
+const Complex FFT_W[15] = 
+{
+	{0.98078528040323,-0.195090322016128},
+	{0.923879532511287,-0.38268343236509},
+	{0.831469612302545,-0.555570233019602},
+	{0.707106781186547,-0.707106781186547},
+	{0.555570233019602,-0.831469612302545},
+	{0.38268343236509,-0.923879532511287},
+	{0.195090322016128,-0.98078528040323},
+	{5.55111512312578E-17,-1},
+	{-0.195090322016128,-0.98078528040323},
+	{-0.38268343236509,-0.923879532511287},
+	{-0.555570233019602,-0.831469612302545},
+	{-0.707106781186547,-0.707106781186547},
+	{-0.831469612302545,-0.555570233019602},
+	{-0.923879532511286,-0.38268343236509},
+	{-0.98078528040323,-0.195090322016128}
+};
+const Complex *w = FFT_W;
+//================================================
+#endif
+
+unsigned char fftx(Complex *d,int m)
+{
+#ifndef __EXTERN_W__
+	static Complex *w;
+	static int mw = 0;
+	float arg, w_real, w_imag, wr_real, wr_imag, wtemp;
+#endif
+	static int n = 1;
+	Complex temp, tmp1, tmp2;
+	Complex *di, *dip, *dj, *wp;
+	int i, j, k, l, le, wi;
+#ifndef __EXTERN_W__
+	if(m != mw)
+	{
+		if(mw != 0)
+			free(w);
+		mw = m;
+		if(m == 0)
+			return 0;
+		n = 1 << m;
+		le = n >> 1;
+		w = (Complex*)calloc(le - 1,sizeof(Complex));
+		if(!w)
+			return 0;
+		arg = 4.0 * atan(1.0) / le;
+		wr_real = w_real = cos(arg);
+		wr_imag = w_imag = -sin(arg);
+		dj = w;
+		for(j = 1; j < le; j++)
+		{
+			dj->real = (float)wr_real;
+			dj->imag = (float)wr_imag;
+			dj++;
+			wtemp = wr_real * w_real - wr_imag * w_imag;
+			wr_imag = wr_real * w_imag + wr_imag * w_real;
+			wr_real = wtemp;
+		}
+	}
+#else
+	n = 1 << m;
+#endif
+	le = n;
+	wi = 1;
+	for(l = 0; l < m; l++)
+	{
+		le >>= 1;
+		for(i = 0; i < n; i += (le << 1))
+		{
+			di = d + i;
+			dip = di + le;
+			temp.real = (di->real + dip->real);
+			temp.imag = (di->imag + dip->imag);
+			dip->real = (di->real - dip->real);
+			dip->imag = (di->imag - dip->imag);
+			*di = temp;                       
+		}
+		wp = (Complex*)w + wi - 1;
+		for(j = 1; j < le; j++)
+		{
+			tmp1 = *wp;
+			for(i = j; i < n; i += (le << 1))
+			{
+				di = d + i;
+				dip = di + le;
+				temp.real = (di->real + dip->real);
+				temp.imag = (di->imag + dip->imag);
+				tmp2.real = (di->real - dip->real);
+				tmp2.imag = (di->imag - dip->imag);
+				dip->real = (tmp2.real * tmp1.real - tmp2.imag * tmp1.imag);
+				dip->imag = (tmp2.real * tmp1.imag + tmp2.imag * tmp1.real);
+				*di = temp;
+			}
+			wp += wi;
+		}
+		wi <<= 1;
+	}
+	
+	for(i = 0; i < n; i++)
+	{
+		j = 0;
+		for(k = 0; k < m; k++)
+			j = (j << 1) | ((i >> k) & 1);
+		if(i < j)
+		{
+			di = d + i;
+			dj = d + j;
+			temp = *dj;
+			*dj = *di;
+			*di = temp;
+		}
+	}
+	
+	return 1;
+}
+
+void AmpSpectrum(Complex *pData,int m,float* pDCAmp,float* pDistortion)
+{
+   // Amp = Abs / (1.414213562 * N)
+	int i;
+	int N = 1 << (m - 1);
+	int N2 = 1 << m;
+	float BasePower;     //exactly BasePower*2*N*N;
+	float SyntPower = 0; //exactly SyntPower*2*N*N;
+	float ModelSquare;
+	*pDCAmp = sqrt(pData[0].real * pData[0].real + pData[0].imag * pData[0].imag) / (N << 1);
+	BasePower = pData[1].real * pData[1].real + pData[1].imag * pData[1].imag;
+	pData[N2 - 1].real = sqrt(BasePower);
+	pData[N2 - 1].imag = asin(pData[N2 - 1].imag / pData[N2 - 1].real);
+	pData[N2 - 1].real /= N;
+	for(i = 2; i < N; i++)
+	{
+		ModelSquare = pData[i].real * pData[i].real + pData[i].imag * pData[i].imag;
+		pData[N2 - i].real = sqrt(ModelSquare);
+		pData[N2 - i].imag = asin(pData[N2-i].imag/pData[N2-i].real);
+		pData[N2 - i].real /= N;
+		SyntPower += ModelSquare;
+	}
+	*pDistortion = sqrt(SyntPower / BasePower);
+}

arc_design_contest/2021/HLJU_Pathological_Voice_Diagnosis/images/actual_picture.jpeg

@@ -0,0 +1,10 @@
+#include "stdlib.h"
+#include "math.h"
+
+typedef struct
+{
+	float real;
+	float imag;
+} Complex;
+
+unsigned char fftx(Complex *x,int m);