|
17 | 17 | "source": [ |
18 | 18 | "## The Wiener Filter\n", |
19 | 19 | "\n", |
20 | | - "The [Wiener filter](https://en.wikipedia.org/wiki/Wiener_filter), named after [*Nobert Wiener*](https://en.wikipedia.org/wiki/Norbert_Wiener), aims at estimating an unknown random signal by filtering a noisy distorted observation of the signal. It has a wide range of applications in signal processing. For instance noise reduction, system identification, deconvolution and signal detection. The Wiener filter is frequently used as basic building block for algorithms that denoise audio signals, like speech, or remove noise from a image." |
| 20 | + "The [Wiener filter](https://en.wikipedia.org/wiki/Wiener_filter), named after [*Nobert Wiener*](https://en.wikipedia.org/wiki/Norbert_Wiener), aims at estimating an unknown random signal by filtering a noisy distorted observation of the signal. It has a wide range of applications in signal processing. For instance, noise reduction, system identification, deconvolution and signal detection. The Wiener filter is frequently used as basic building block for algorithms that denoise audio signals, like speech, or remove noise from a image." |
21 | 21 | ] |
22 | 22 | }, |
23 | 23 | { |
|
38 | 38 | "e[k] = y[k] - s[k]\n", |
39 | 39 | "\\end{equation}\n", |
40 | 40 | "\n", |
41 | | - "is introduced. The error $e[k]$ equals zero, if the output signal $y[k]$ perfectly matches $s[k]$. In general this goal cannot be achieved in a strict sense. As alternative one could aim at minimizing the linear average of the error $e[k]$. However, this quantity is not very well suited for optimization since it is signed. Instead, the quadratic average of the error $e[k]$ is used in the Wiener filter and other techniques. This measure is known as [*mean squared error*](https://en.wikipedia.org/wiki/Mean_squared_error) (MSE). It is defined as\n", |
| 41 | + "is introduced. The error $e[k]$ equals zero, if the output signal $y[k]$ perfectly matches $s[k]$. In general, this goal cannot be achieved in a strict sense. As alternative one could aim at minimizing the linear average of the error $e[k]$. However, this quantity is not very well suited for optimization since it is signed. Instead, the quadratic average of the error $e[k]$ is used in the Wiener filter and other techniques. This measure is known as [*mean squared error*](https://en.wikipedia.org/wiki/Mean_squared_error) (MSE). It is defined as\n", |
42 | 42 | "\n", |
43 | 43 | "\\begin{equation}\n", |
44 | 44 | "E \\{ |e[k]|^2 \\} = \\frac{1}{2 \\pi} \\int\\limits_{-\\pi}^{\\pi} \\Phi_{ee}(\\mathrm{e}^{\\,\\mathrm{j}\\,\\Omega}) \\,\\mathrm{d}\\Omega = \\varphi_{ee}[\\kappa] \\Big\\vert_{\\kappa = 0}\n", |
45 | 45 | "\\end{equation}\n", |
46 | 46 | "\n", |
47 | | - "the latter to equalities are deduced from the properties of the power spectral density (PSD) and the auto-correlation function (ACF). Above equation is referred to as [*cost function*](https://en.wikipedia.org/wiki/Loss_function). We aim at minimizing the cost function, hence minimizing the MSE between the original signal $s[k]$ and its estimate $y[k]$. The solution is referred to as [minimum mean squared error](https://en.wikipedia.org/wiki/Minimum_mean_square_error) (MMSE) solution." |
| 47 | + "the equalities are deduced from the properties of the power spectral density (PSD) and the auto-correlation function (ACF). Above equation is referred to as [*cost function*](https://en.wikipedia.org/wiki/Loss_function). We aim at minimizing the cost function, hence minimizing the MSE between the original signal $s[k]$ and its estimate $y[k]$. The solution is referred to as [minimum mean squared error](https://en.wikipedia.org/wiki/Minimum_mean_square_error) (MMSE) solution." |
48 | 48 | ] |
49 | 49 | }, |
50 | 50 | { |
|
71 | 71 | "source": [ |
72 | 72 | "**Example - Denoising of an Deterministic Signal**\n", |
73 | 73 | "\n", |
74 | | - "The following example considers the estimation of the original signal from a distorted observation. It is assumed that the original signal is a deterministic signal $s[k] = \\sin[\\Omega_0\\,k]$ which is distorted by an LTI system and additive normally distributed zero-mean white noise with PSD $N_0 = 0.1$. The PSDs $\\Phi_{sx}(\\mathrm{e}^{\\,\\mathrm{j}\\,\\Omega})$ and $\\Phi_{xx}(\\mathrm{e}^{\\,\\mathrm{j}\\,\\Omega})$ are estimated from $s[k]$ and $x[k]$ using the [Welch technique](../spectral_estimation_random_signals/welch_method.ipynb). The Wiener filter is applied to the observation $x[k]$ in order to compute the estimate $y[k]$ of $s[k]$. Note, the discrete signals have been illustrated by continuous lines for ease of illustration." |
| 74 | + "The following example considers the estimation of the original signal from a distorted observation. It is assumed that the original signal is a deterministic signal $s[k] = \\sin[\\Omega_0\\,k]$ which is distorted by an LTI system and additive normally distributed zero-mean white noise with PSD $N_0 = 0.1$. The PSDs $\\Phi_{sx}(\\mathrm{e}^{\\,\\mathrm{j}\\,\\Omega})$ and $\\Phi_{xx}(\\mathrm{e}^{\\,\\mathrm{j}\\,\\Omega})$ are estimated from $s[k]$ and $x[k]$ using the [Welch technique](../spectral_estimation_random_signals/welch_method.ipynb). The Wiener filter is applied to the observation $x[k]$ in order to compute the estimate $y[k]$ of $s[k]$. Note that the discrete signals have been illustrated by continuous lines for ease of illustration." |
75 | 75 | ] |
76 | 76 | }, |
77 | 77 | { |
|
175 | 175 | "* Take a look at the PSDs and the resulting transfer function of the Wiener filter. How does the Wiener filter remove the noise from the observed signal?\n", |
176 | 176 | "* Change the frequency `Om0` of the original signal $s[k]$ and the noise power `N0` of the additive noise. What changes?\n", |
177 | 177 | "\n", |
178 | | - "The cross-PSD $\\Phi_{sx}(e^{j \\Omega})$ has its maximum at the frequency $\\Omega_0$ of the original signal $s[k]$, besides this frequency the PSD $\\Phi_{xx}(e^{j \\Omega})$ has approximately the value of the PSD $\\Phi_{nn}(e^{j \\Omega}) = N_0$ of the additive noise. The resulting Wiener filter $G(e^{j \\Omega})$ has an attenuation of 0 dB at $\\Omega_0$ which increases off this frequency. It has the characteristics of a band-pass filter centered at $\\Omega_0$. By filtering the observed signal $x[k]$ with the Wiener filter, the major portion of the additive white noise is removed. However, around $\\Omega_0$ both the original signal $s[k]$ and noise pass the filter. This explains the remaining small deviations between the estimated signal $y[k]$ and the original signal $y[k]$. The adaption of the Wiener filter to the original signal and additive noise when changing its frequency and PSD." |
| 178 | + "The cross-PSD $\\Phi_{sx}(e^{j \\Omega})$ has its maximum at the frequency $\\Omega_0$ of the original signal $s[k]$, besides this frequency the PSD $\\Phi_{xx}(e^{j \\Omega})$ has approximately the value of the PSD $\\Phi_{nn}(e^{j \\Omega}) = N_0$ of the additive noise. The resulting Wiener filter $G(e^{j \\Omega})$ has an attenuation of 0 dB at $\\Omega_0$ which increases off this frequency. It has the characteristics of a band-pass filter centered at $\\Omega_0$. By filtering the observed signal $x[k]$ with the Wiener filter, the major portion of the additive white noise is removed. However, around $\\Omega_0$ both the original signal $s[k]$ and noise pass the filter. This explains the remaining small deviations between the estimated signal $y[k]$ and the original signal $y[k]$. The adaption of the Wiener filter to the original signal and additive noise changes when changing its frequency and PSD." |
179 | 179 | ] |
180 | 180 | }, |
181 | 181 | { |
|
223 | 223 | "\n", |
224 | 224 | "1. If there is no additive noise $\\Phi_{nn}(\\mathrm{e}^{\\,\\mathrm{j}\\,\\Omega}) = 0$, the bracketed expression is equal to 1. Hence, the Wiener filter is simply given as the inverse system to the distorting system $$H(\\mathrm{e}^{\\,\\mathrm{j}\\,\\Omega}) = \\frac{1}{G(\\mathrm{e}^{\\,\\mathrm{j}\\,\\Omega})}$$\n", |
225 | 225 | "\n", |
226 | | - "2. If the distorting system is just a pass through $G(\\mathrm{e}^{\\,\\mathrm{j}\\,\\Omega}) = 1$, the Wiener filter is given as\n", |
| 226 | + "2. If the distorting system is just a pass-through $G(\\mathrm{e}^{\\,\\mathrm{j}\\,\\Omega}) = 1$, the Wiener filter is given as\n", |
227 | 227 | "\n", |
228 | 228 | " $$H(\\mathrm{e}^{\\,\\mathrm{j}\\,\\Omega}) = \\frac{\\text{SNR}(\\mathrm{e}^{\\,\\mathrm{j}\\,\\Omega})}{\\text{SNR}(\\mathrm{e}^{\\,\\mathrm{j}\\,\\Omega}) + 1}=\\frac{\\Phi_{ss}(\\mathrm{e}^{\\,\\mathrm{j}\\,\\Omega})}{\\Phi_{ss}(\\mathrm{e}^{\\,\\mathrm{j}\\,\\Omega})+\\Phi_{nn}(\\mathrm{e}^{\\,\\mathrm{j}\\,\\Omega})}$$\n", |
229 | 229 | " Two extreme cases are considered\n", |
|
237 | 237 | "cell_type": "markdown", |
238 | 238 | "metadata": {}, |
239 | 239 | "source": [ |
240 | | - "**Example - Denoising of an deterministic signal with the Wiener deconvolution filter**\n", |
| 240 | + "**Example - Denoising of a deterministic signal with the Wiener deconvolution filter**\n", |
241 | 241 | "\n", |
242 | 242 | "The preceding example of the general Wiener filter will now be reevaluated with the Wiener deconvolution filter." |
243 | 243 | ] |
|
339 | 339 | "* Compare the transfer function of the Wiener deconvolution filter to the transfer function of the general Wiener filter from the preceding example.\n", |
340 | 340 | "* What is different compared to the general Wiener filter? Why?\n", |
341 | 341 | "\n", |
342 | | - "The transfer function $H(e^{j \\Omega})$ of the Wiener deconvolution filter is much smoother that the transfer function of the general Wiener filter from the preceding example. This is due to the knowledge of the transfer function $G(e^{j \\Omega})$ of the disturbing system. The general Wiener filter inherently estimates this from the noisy observations." |
| 342 | + "The transfer function $H(e^{j \\Omega})$ of the Wiener deconvolution filter is much smoother than the transfer function of the general Wiener filter from the preceding example. This is due to the knowledge of the transfer function $G(e^{j \\Omega})$ of the disturbing system. The general Wiener filter inherently estimates this from the noisy observations." |
343 | 343 | ] |
344 | 344 | }, |
345 | 345 | { |
|
355 | 355 | } |
356 | 356 | ], |
357 | 357 | "metadata": { |
| 358 | + "anaconda-cloud": {}, |
358 | 359 | "kernelspec": { |
359 | | - "display_name": "Python 3", |
| 360 | + "display_name": "Python [default]", |
360 | 361 | "language": "python", |
361 | 362 | "name": "python3" |
362 | 363 | }, |
|
0 commit comments