P2-T3 — Filtering and Noise Reduction

Learning objectives

After this lesson, you should be able to:

• Apply a convolution kernel to an image by hand and state what a given kernel does.
• Compare mean, Gaussian and median filters for random noise and for isolated outliers.
• Estimate the noise reduction of an averaging filter and its cost at edges.
• Choose a filter and size for a stated perception task, and state the cost.

The idea: trust the neighbours

Neighbouring pixels usually see nearly the same thing. If one pixel differs sharply from all eight around it, it is probably noise. A filter formalises this: the new value of a pixel is computed from the old values in a small window around it. The two basic choices are to average the window or to take its median.

Two kinds of noise

Averaging treats a very wrong pixel as a very large vote. Taking the median ignores how wrong it is. That is why outliers need the median.

What every filter costs

The price of smoothing is blur. A mean or Gaussian filter replaces a sharp edge with a ramp as wide as the window, so edge positions become less certain and small objects fade. The median filter keeps straight edges sharp but removes thin lines and small details narrower than half the window. A filter you cannot afford to apply is one that removes the thing you are looking for.

The mathematics: convolution and its effect on noise

A linear filter replaces each pixel by a weighted sum of its neighbourhood. For a kernel K of size (2r + 1) × (2r + 1):

I(x, y)

the input image value at column x, row y.

K(i, j)

the kernel weights: how much the pixel at offset (i, j) counts. For a filter that preserves overall brightness the weights sum to 1.

I′(x, y)

the output value at the same position.

r

the window radius: r = 1 gives a 3 × 3 window, r = 2 a 5 × 5 window.

(Strictly, convolution flips the kernel; for the symmetric kernels used for smoothing it makes no difference, and image libraries mostly implement correlation under the name convolution.)

Three common filters

Mean (box) filter: all weights equal, 1/n² for an n × n window.

Gaussian filter: weights fall off with distance from the centre:

where σ sets the width: larger σ means more smoothing. Pixels near the centre count most, so edges blur less than with a box of the same size, and the kernel has no sharp cut-off. The kernel is truncated at about 3σ and normalised to sum to 1.

Median filter: not linear. Sort the window values and take the middle one. It cannot be written as a kernel.

How much noise does averaging remove?

If the noise at each pixel is independent with standard deviation σ_n, the mean of N pixels has

A 3 × 3 box (N = 9) divides the noise by 3; a 5 × 5 box (N = 25) by 5; a 9 × 9 box by 9. The price is blur: the window width grows with n and so does the width of the smeared edge, while the noise falls only as 1/n. Doubling the window size halves the noise again and doubles the edge blur, and each doubling costs four times as many operations. This trade is the reason to use the smallest filter that does the job.

Cost and separability

An n × n kernel costs n² multiplications per pixel. A Gaussian is separable: the 2D kernel is the product of two 1D kernels, so filtering rows and then columns costs 2n per pixel. For a 15 × 15 window that is 225 against 30 operations. A median needs a sort (or histogram tricks) per pixel and is slower for large windows. On an embedded robot these differences decide what runs at frame rate.

A worked example: a hot pixel and a step edge

Take the 3 × 3 patch of a flat dark region with one hot pixel in the centre:

The mean filter at the centre gives (8 × 10 + 90)/9 = 18.9: the hot pixel is reduced from 90 to 18.9, but a bright residue is left, and its value leaks into the eight neighbours: each of their 3 × 3 windows contains the hot pixel, so each of them also becomes 18.9, a patch of nine slightly bright pixels where there was one very bright one. The median filter sorts the nine values (eight 10s and one 90) and takes the fifth, which is 10: the outlier is removed entirely and the neighbours are untouched.

Now a step edge, from 60 to 180 with Gaussian noise of σ_n = 8, as in the figure. A 3-pixel mean reduces the noise from 8 to 8/√3 ≈ 4.6 on the flat parts, but the step becomes a ramp over three pixels, and each of the two outliers produces a bump in the output: the 255 on the dark side adds about (255 − 60)/3 = 65, and the 0 on the bright side takes away about (180 − 0)/3 = 60. The 3-pixel median leaves the step in one pixel, removes both outliers, and removes less of the Gaussian noise (the median of three Gaussian samples has a standard deviation about 0.67 σ_n, so about 5.4 here).

Comparing the three on the same task

In the second lab of this module, the failing pipeline used a 7 × 7 mean filter on salt-and-pepper noise and closed the narrow gap between two blocks; the repair was a 3 × 3 median.

Engineering implications and limitations

Name the noise before choosing the filter. Look at the noise: speckle (outliers) or fine grain (Gaussian)? For outliers use a median; for fine grain a Gaussian or mean. Applying a mean to outliers makes them worse in the sense that it spreads them.

Use the smallest filter that works. Each increase in window size removes less additional noise and removes more detail. Choose the size from the smallest feature you must keep: a window larger than the object’s smallest dimension removes it.

Filter before differentiating, not after. Gradients amplify noise (Lesson P2-T5). Smooth first, then compute edges. Filtering after thresholding is a different operation (morphology) that cleans binary masks.

Mind edge handling. At the image border the window leaves the image. Libraries pad with zeros, replicate the border, or reflect it; zero padding darkens the border and can create false edges there.

Accept the cost. A large median is expensive, and a large kernel on an embedded CPU can consume the frame budget. Prefer separable Gaussians, box filters implemented with running sums, or downsampling first, which is a cheap smoothing step in itself.

Do not filter what you will measure by. If the quantity of interest is the exact edge position, heavy smoothing moves and blurs it. Use only as much as needed, and consider sub-pixel edge methods.

Common misconceptions

Engineering takeaways

1. A filter replaces each pixel by a function of its neighbours; a convolution kernel is a set of weights, normally summing to 1.
2. Mean and Gaussian filters reduce independent Gaussian noise by √N but blur edges; the Gaussian blurs less for a given smoothing and is separable.
3. The median filter removes impulse noise completely and keeps straight edges, but it is nonlinear, slower and removes thin features.
4. Choose by the noise type and by what must survive; use the smallest window that works.
5. Smooth before taking gradients, handle borders deliberately, and count the cost in the frame budget.

References and further reading

Standard references; the figure was generated for this lesson from a seeded random signal.

• Gonzalez, R. C. and Woods, R. E. (2018). Digital Image Processing, 4th ed. Pearson: spatial filtering, order-statistic filters and noise models.
• Szeliski, R. (2022). Computer Vision: Algorithms and Applications, 2nd ed. Springer: linear filtering, separable kernels and median filters.
• Huang, T. S., Yang, G. J. and Tang, G. Y. (1979). A fast two-dimensional median filtering algorithm. IEEE Transactions on Acoustics, Speech and Signal Processing: efficient median filtering.

Knowledge check

Three conceptual questions. Write an answer, then reveal the explanation. Your answers stay in your browser.

What’s next

Next: segmentation: turning a cleaned image into regions, with thresholds and connected components.

Related content

Nothing below is required to finish this lesson.

• Before
  P2-T2: RGB, Grayscale and HSV

  Convert between colour spaces and explain why HSV helps with colour detection.

• Lab
  P2-L3: Diagnose a Failing Image-Processing Pipeline

  Find which stage of a filter-threshold-contour pipeline fails and repair it.