# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     language: python
#     name: python3
# ---

# + [markdown] pycharm={"metadata": false}
# # Chapter 5 - Image Filtering
#
# In this chapter, we're introducing the concept of Image Filtering. 
# Filters can be applied on 2D Image data either for various applications. We can broadly differenciate low-pass filters smooth images
# (retrain low-frequenciy components) and high-pass filters (retain contours / edges, e.g. high frequencies). 
#

# + [markdown] pycharm={"metadata": false}
# ## Low-pass filters
# Low pass filters are typically applied to reduce noise in images. Noise can be seen as random artifacts in an image. 
# For example, salt & pepper noise describes the random occurrence of black / white pixels in the image, while
# gaussian noise is a random increase/decrease in each pixel’s color value, following a gaussian distribution.
#
# ![Different noise types](https://github.com/joelbarmettlerUZH/PyVisualOdometry/raw/master/img/chapter_5/1_different_noise_types.png)
# *Figure 1: Different noise types. [source](http://rpg.ifi.uzh.ch/docs/teaching/2019/04_filtering.pdf)*
#

# + [markdown] pycharm={"metadata": false}
# Low-pass filters assume that the pixel variation in the image should be lower than perceived, e.g. pixels should have a
# color value close to their neighbours. Low-pass filters therefore replace each pixels value with an average of the values in 
# the pixels neighbourhood. The neighbours can either be weighted based on their distance to the center pixel, or equally. 
#

# + [markdown] pycharm={"metadata": false}
# Moving a filter over all possible positions in an image is called a *convolution*, the filter is called a *Kernel* or *Mask* 
# and denoted *H*. 
# When convoluting a filter over an image, we flip the kernel by 180° before performing at each position before computing the weighted 
# average between the filter values and the pixel. If we do not flip the kernel, we speak of a cross-correlation instead of a convolution.
# For symmetric filters like "Gaussian Filter" or "Median Filter", a convolution and a cross-correlation will of course produce
# the same results. 
#
# ![Convolution vs. Cross-corelation formula](https://github.com/joelbarmettlerUZH/PyVisualOdometry/raw/master/img/chapter_5/3_convolution_vs_cross-correlation.png)
# *Figure 2: Convolution vs. Cross-corelation formula. [source](http://rpg.ifi.uzh.ch/docs/teaching/2019/04_filtering.pdf)*

# + [markdown] pycharm={"metadata": false}
# In the following example, a smoothing averaging filter is applied to a 1D signal, bringing the neighbouring values significantly closer together and reducing outliers.
#
# ![Filter example, p. 15](https://github.com/joelbarmettlerUZH/PyVisualOdometry/raw/master/img/chapter_5/2_smoothed_signal.png)
# *Figure 3: World Coordinates -> Pixel Coordinates. [source](http://rpg.ifi.uzh.ch/docs/teaching/2019/04_filtering.pdf)*

# + [markdown] pycharm={"metadata": false}
# The same principle can be applied to 2D Data like images. In the following example, a 2D Filter with size 1/1 averages a neighbourhood of 9 pixels, overwriting the central pixel with an equally weighted average of all 9 pixels. 
# Such a filter is called a "box" filter.
#
# ![Box filter](https://github.com/joelbarmettlerUZH/PyVisualOdometry/raw/master/img/chapter_5/4_before_filtering_image_with_box_filter.png)
# *Figure 4: Illustration of a box filter over a black image with a white square. [source](http://rpg.ifi.uzh.ch/docs/teaching/2019/04_filtering.pdf)*
#
# The output is - of course - a blurred version of the very same image.
#
# ![Box filter output](https://github.com/joelbarmettlerUZH/PyVisualOdometry/raw/master/img/chapter_5/5_after_filtering_image_with_box_filter.png)
# *Figure 5: Output of a smoothed image using a box-filter. [source](http://rpg.ifi.uzh.ch/docs/teaching/2019/04_filtering.pdf)*

# + [markdown] pycharm={"metadata": false}
# While the box filter smoothens an image quite well, it produces horizontal and vertical aftifacts since the filter itself has
# sharp edges. These artifacts are also called "aliasing" and is caused by the high frequency components of the box filter.
# A better way to smooth an image is with a gaussian filter, a filter implementing the 2D gaussian function. 
# For perfect results, take a large gaussian filters with smooth edges, e.g. low standard derivation that ensures that the outermost
# values of the filter are close to 1 while preserving a smooth derivative. 
#
# ![Gaussian filter visualization](https://github.com/joelbarmettlerUZH/PyVisualOdometry/raw/master/img/chapter_5/7_gaussian_filter_comparison.png)
# *Figure 6: Gaussian filter visualization. [source](http://rpg.ifi.uzh.ch/docs/teaching/2019/04_filtering.pdf)*
#
# ![Gaussian Filter comparison, p. 26](https://github.com/joelbarmettlerUZH/PyVisualOdometry/raw/master/img/chapter_5/17_gaussian_filter_comparison.png)
# *Figure 7: Gaussian Filter comparison. [source](http://rpg.ifi.uzh.ch/docs/teaching/2019/04_filtering.pdf)*

# + [markdown] pycharm={"metadata": false}
# When we apply any filter on an image, the question remains how to deal with the image boundary. Since - in most cases - we don't
# want the resulting image to be smaller than the input image, we have to simulate additional boundary pixels. 
# There are different strategies with varying results, like zero-padding (surrounding black pixels), wrap-around (repeating the image),
# copy-edge (always use outermost pixel values) or reflect accross edge (mirroring around edge, gives best results). 
#

# + [markdown] pycharm={"metadata": false}
# ### Non-linear Low-pass filters
#
# Gaussian filters or box-filters do not denoise salt & pepper noise since they get influenced by outliers by a high degree. 
# That's where **median filters** come into play. They can not be interpreted as a classical convolution filter like a Gaussian
# filter, it rather takes the median pixel value from the neighbourhood. The median filter is therefore much less influenced
# by strong noise, while he also preserves edges much better than the linear smoothing filters. 
#
# ![Gaussian Filter comparison, p. 26](https://github.com/joelbarmettlerUZH/PyVisualOdometry/raw/master/img/chapter_5/8_median_filtered_image.png)
# *Figure 8: Median Filter removing Salt & Pepper Noise. [source](http://rpg.ifi.uzh.ch/docs/teaching/2019/04_filtering.pdf)*

# + [markdown] pycharm={"metadata": false}
# Another such filter is the **billateral filter**. It acts like a median filter with preserving edges even more by adapting the kernel
# locally to the intensitiy profile of the underlaying image. They only average pixels with similar brightness: Pixels that fall below
# a brightness difference compared to the center pixel. 
#
# ![Bilateral filer with mask, p. 26](https://github.com/joelbarmettlerUZH/PyVisualOdometry/raw/master/img/chapter_5/9_billateral_filter_demonstration.png)
# *Figure 9: Bilateral filer with mask. [source](http://rpg.ifi.uzh.ch/docs/teaching/2019/04_filtering.pdf)*
#
# The extend to which neighbouring pixels have to be similar to the central pixel is controlled via a factor *sigma*.
#

# + [markdown] pycharm={"metadata": false}
# ## High-pass filters
#
# High-pass filters are mainly used for edge detection since react to sharp change in pixel intensity. Edges are sharp changes in
# an image functions intensity value. Applying the first derivative on an image would leave us with an image where sharp edges 
# are shown. 
#
# ![Image derivative detecting edges](https://github.com/joelbarmettlerUZH/PyVisualOdometry/raw/master/img/chapter_5/10_image_first_derivative_demonstration.png)
# *Figure 10: Image derivative detecting edges. [source](http://rpg.ifi.uzh.ch/docs/teaching/2019/04_filtering.pdf)*
#
# We therefore construct a filter that acts like a derivative by approximating the image derivative 
# *dI(x,y) / dx ~ I(x+1, y) - I(x,y)* and *dI(x,y) / dy ~ I(x, y+1) - I(x,y)*. 
# So we essentially compare each pixel to its direct neighbour and take the difference as an output. 
#
# ![Partial derivative filter](https://github.com/joelbarmettlerUZH/PyVisualOdometry/raw/master/img/chapter_5/11_partial_derivative_filters.png)
# *Figure 11: Partial derivative filter. [source](http://rpg.ifi.uzh.ch/docs/teaching/2019/04_filtering.pdf)*

# + [markdown] pycharm={"metadata": false}
# More advanced filters are larger in size and therefore produce less artifacts. The sobel-filter is an example for a larger
# derivative filter:
#
# ![Prewitt & Sobel filter](https://github.com/joelbarmettlerUZH/PyVisualOdometry/raw/master/img/chapter_5/12_prewitt_sobel_filter.png)
# *Figure 12: Prewitt & Sobel filter. [source](http://rpg.ifi.uzh.ch/docs/teaching/2019/04_filtering.pdf)*
#
# The direction of the edge can be determined by calculating the pixel regions gradient, so the diretion of fastest intensity change.
# The gradient direction is given by *angle = arctan2(dI/dx, dI/dy)*, so the two dimensional arcus tangens of the image derivative
# values. The edge strenght is given by the gradients magnitude: *strength = sqrt((dI/dx)^2 + (dI/dy)^2).
#

# + [markdown] pycharm={"metadata": false}
# A big problem for high-pass filters is gaussian noise: there will always be a steep difference between two neighbouring pixels, caused
# by normal gaussian noise produced by the image sensor. It is therefore best practice to softly filter the image first with a 
# gaussian filter before applying a high-pass filter. 
#

# + [markdown] pycharm={"metadata": false}
# In the following graphic, we see the original image I, the kernel H, the resulting image when H is applied I*H as well as the derrived
# image d(I*H)/dx
#
# ![Process steps for edge detection](https://github.com/joelbarmettlerUZH/PyVisualOdometry/raw/master/img/chapter_5/13_individual_processing_steps_for_edge_detection.png)
# *Figure 13: Process steps for edge detection. [source](http://rpg.ifi.uzh.ch/docs/teaching/2019/04_filtering.pdf)*
#
# A way better approach is to directly include the smoothing in the filter itself, giving us the filter dH/dx as seen in
# the following image:
#
# ![Gaussian smoothing within a derivative filter](https://github.com/joelbarmettlerUZH/PyVisualOdometry/raw/master/img/chapter_5/14_gaussian_smoothing_and_derivative_filter.png)
# *Figure 13: Gaussian smoothing within a derivative filter. [source](http://rpg.ifi.uzh.ch/docs/teaching/2019/04_filtering.pdf)*

# + [markdown] pycharm={"metadata": false}
# This is called a "derivative of gaussian" filter: it multiplies a normal gaussian filters with a high-pass 2x1 derivative filter. 
#
# ![Difference of Gaussians filter](https://github.com/joelbarmettlerUZH/PyVisualOdometry/raw/master/img/chapter_5/15_difference_of_gaussians_filter.png)
# *Figure 14: Difference of Gaussians filter. [source](http://rpg.ifi.uzh.ch/docs/teaching/2019/04_filtering.pdf)*

# + [markdown] pycharm={"metadata": false}
# Since we deal with two partial derivatives, we'd need to filter the image twice. A solution to this is given by the 
# "Laplacian of Gaussian"-Filter, which finds the derivative in all directions simultaniously. It is constructed by
# subtracting a smaller radius gaussian Filter from a large radius gaussian filter.
#
# ![Laplacian of Gaussian](https://github.com/joelbarmettlerUZH/PyVisualOdometry/raw/master/img/chapter_5/18_laplacian_of_gaussian.png)
# *Figure 15: Laplacian of Gaussian. [source](http://rpg.ifi.uzh.ch/docs/teaching/2019/04_filtering.pdf)*

# + [markdown] pycharm={"metadata": false}
# ## Canny edge detection
#
# Canny edge detection uses partial gaussian derivative filters to find all corners in an image. It then sets all pixelsvalues to 0 that
# fall under a given threshold. Finally, Canny takes the local maximum of any corner along the gradient direction, e.g. it only
# takes the peak of a wide edge. 
#
# ![Canny edge detection](https://github.com/joelbarmettlerUZH/PyVisualOdometry/raw/master/img/chapter_5/16_canny_edge_detection.png)
# *Figure 16: Canny edge detection. [source](http://rpg.ifi.uzh.ch/docs/teaching/2019/04_filtering.pdf)*
#
#
# -

# # Overview summary
#
# Let me quickly illustrate the main differences of Smoothing and Derivaitve filters.
#
# Smoothing filters always contain positive filter values that sum to 1 to preserve the overall brightness of constant regions. They are constructed to remove high-frequency components. 
#
# In contrast, derivative filters have two regions with opposite signs to get a high response in regions of high contrast. Their components sum to 1 to create no response on images with constant color. They are created to highlight high-frequency, not to remove them. 

StarcoderdataJupyterScriptsDedupFiltered