Ahmed Gad, profile picture
Uploaded byAhmed Gad
PDF, PPTX8,235 views

Computer Vision: Correlation, Convolution, and Gradient

The document provides an overview of computer vision techniques including correlation, convolution, and gradient filtering. It discusses how correlation can be used to match a template to an image region by calculating a similarity measure as the template is passed over the image. Convolution is explained as similar to correlation but with the signs of the variables flipped in the formula. Implementing these techniques in Python code is also covered generically for squared and odd-sized templates.

Embed presentation

Download as PDF, PPTX
Computer Vision: Correlation, Convolution, and Gradient MENOUFIA UNIVERSITY FACULTY OF COMPUTERS AND INFORMATION INFORMATION TECHNOLOGY COMPUTER VISION ‫المنوفية‬ ‫جامعة‬ ‫والمعلومات‬ ‫الحاسبات‬ ‫كلية‬ ‫المعلومات‬ ‫تكنولوجيا‬ ‫بالحاسب‬ ‫الرؤيا‬ ‫المنوفية‬ ‫جامعة‬ Ahmed Fawzy Gad ahmed.fawzy@ci.menofia.edu.eg Tuesday 3 October 2017
Index • Correlation • Python Implementation • Convolution • Python Implementation • Gradient • Python Implementation Correlation Convolution Gradient Filtering
CORRELATION
Correlation for 2D Image • Correlation is used to match a template to an image. • Can you tell where the following template exactly located in the image? • Using correlation we can find the image region that best matches the template. ?
How 2D Correlation Works? • Given a template, using correlation the template will pass through each image part and a similarity check take place to find how similar the template and the current image part being processed. • Starting by placing the template top-left corner on the top-left corner of the image, a similarity measure is calculated.
How 2D Correlation Works? • Given a template, using correlation the template will pass through each image part and a similarity check take place to find how similar the template and the current image part being processed. • Starting by placing the template top-left corner on the top-left corner of the image, a similarity measure is calculated. 𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗]
How 2D Correlation Works? • Given a template, using correlation the template will pass through each image part and a similarity check take place to find how similar the template and the current image part being processed. • Starting by placing the template top-left corner on the top-left corner of the image, a similarity measure is calculated.
Correlation for 2D Image 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 2 1 -4 3 2 5 -1 8 1
Correlation for 2D Image 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 2 1 -4 3 2 5 -1 8 1
Correlation for 2D Image 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 2 ∗ 1 + 1 ∗ 3 − 4 ∗ 4 + 3 ∗ 2 + 2 ∗ 7 + 5 ∗ 4 − 1 ∗ 6 + 8 ∗ 2 + 1 ∗ 5 = 𝟒𝟒
Correlation for 2D Image 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 44 Did you get why template sizes are odd?
Even Template Size 58 3 213 81 78 185 87 32 27 11 70 66 60 2 19 61 91 129 89 38 14 7 58 14 42 Even template sizes has no center. E.g. 2x3, 2x2, 5x6, … 𝟖𝟒 ???
Correlation for 2D Image 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 2 1 -4 3 2 5 -1 8 1 44
Correlation for 2D Image 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 2 ∗ 3 + 1 ∗ 4 − 4 ∗ 3 + 3 ∗ 7 + 2 ∗ 4 + 5 ∗ 1 − 1 ∗ 2 + 8 ∗ 5 + 1 ∗ 2 = 𝟕𝟐 44
Correlation for 2D Image 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 44 72 Based on the current step, what is the region that best matches the template? It is the one corresponding to the highest score in the result matrix.
Continue Correlating the Template 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 2 1 -4 3 2 5 -1 8 1 44 72
Continue Correlating the Template 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 2 1 -4 3 2 5 -1 8 1 44 72 This will cause the program performing correlation to fall into index out of bounds exception.
Padding by Zeros 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 44 72 0 0 0 0 It doesn`t affect multiplication. Why Zero?
Continue Correlating the Template 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 44 72 0 0 0 0 2 1 -4 3 2 5 -1 8 1
Continue Correlating the Template 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 44 72 0 0 0 0 2 ∗ 4 + 1 ∗ 3 − 4 ∗ 0 + 3 ∗ 4 + 2 ∗ 1 + 5 ∗ 0 − 1 ∗ 5 + 8 ∗ 2 + 1 ∗ 0 = 𝟑𝟔
Continue Correlating the Template 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 44 72 36 0 0 0 0 2 ∗ 4 + 1 ∗ 3 − 4 ∗ 0 + 3 ∗ 4 + 2 ∗ 1 + 5 ∗ 0 − 1 ∗ 5 + 8 ∗ 2 + 1 ∗ 0 = 𝟑𝟔
Continue Correlating the Template 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 2 1 -4 3 2 5 -1 8 1 44 72 36
Continue Correlating the Template 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 2 1 -4 3 2 5 -1 8 1 44 72 36
Pad by Zeros 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 44 72 36 0 0 0 0
Continue Correlating the Template 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 44 72 36 0 0 0 0 2 1 -4 3 2 5 -1 8 1
Continue Correlating the Template 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 44 72 36 0 0 0 0 2 ∗ 2 + 1 ∗ 5 − 4 ∗ 2 + 3 ∗ 6 + 2 ∗ 8 + 5 ∗ 9 − 1 ∗ 0 + 8 ∗ 0 + 1 ∗ 0 = 𝟖𝟎
Continue Correlating the Template 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 44 72 36 80 Continue till end.
• It depends on both the template size. • For 3x3 template, pad two columns and two rows. One column to the left and another to the right. One row at the top and one row at the bottom. • For 5x5? Pad 4 rows (two left & two right) and 5 columns (two top & two bottom). • Generally, for N*N template pad (N-1)/2 columns and rows at each side. How much Padding Required? 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 How to do this Programmatically.?
Implementing Correlation in Python – 3x3 Template
Implementing Correlation in Python – 141x141 Template
Implementing Correlation in Python – 141x141 Template
Implementing Correlation in Python – 141x141 Template
Implementing Correlation in Python – 141x141 Template Is this is the optimal results? No. But WHY?
Matching using Correlation • Matching depends on just multiplication. • The highest results of multiplication is the one selected as best match. • But the highest results of multiplication not always refers to best match. Template Result
Matching using Correlation • Make a simple edit on the image by adding a pure white rectangular area. • Use the previous template. • Apply the algorithm. Template
Matching using Correlation • The best matched region is the white region. • The reason is that correlation depends on multiplication. • What gives highest multiplication results is the best match. • Getting high results corresponds to multiplying by higher numbers. • The highest pixel value for gray images is 255 for white.
Implementing Correlation in Python – Generic Code – Squared Odd Sized Templates
Implementing Correlation in Python – Generic Code – Squared Odd Sized Templates
Implementing Correlation in Python – Generic Code – Squared Odd Sized Templates
Implementing Correlation in Python – Generic Code – Squared Odd Sized Templates
CONVOLUTION
What is Convolution? • The primary objective of mathematical convolution is combining two signals to generate a third signal. Convolution is not limited on digital image processing and it is a broad term that works on signals. Input Signal Impulse Response Output Signal System Convolution
Convolution Formula for 2D Digital Signal • Convolution is applied similarly to correlation. • Note how similar the formulas for correlation and convolution. The only difference is that in the signs of 𝒖 and 𝒗 variables. • Correlation formula has positive signs for 𝒖 and 𝒗 while correlation has negative signs for them. 𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗] Is just changing signs make sense?
Correlation Vs. Convolution 𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗] Correlation Convolution 13204 458047 09373 .2.4.1 .5.8.3 .9.5.7 .2.4.1 .5.8.3 .9.5.7
Correlation Vs. Convolution 𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗] Correlation Convolution 13204 458047 09373 .2.4.1 .5.8.3 .9.5.7 .2.4.1 .5.8.3 .9.5.7
Correlation Vs. Convolution 𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗] Correlation Convolution 13204 458047 09373 .2.4.1 .5.8.3 .9.5.7 .2.4.1 .5.8.3 .9.5.7
Correlation Vs. Convolution 𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗] Correlation Convolution 13204 458047 09373 .2.4.1 .5.8.3 .9.5.7 .2.4.1 .5.8.3 .9.5.7
Correlation Vs. Convolution 𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗] Correlation Convolution 13204 458047 09373 .2.4.1 .5.8.3 .9.5.7 .2.4.1 .5.8.3 .9.5.7
Correlation Vs. Convolution 𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗] Correlation Convolution 13204 458047 09373 .2.4.1 .5.8.3 .9.5.7 .2.4.1 .5.8.3 .9.5.7
Correlation Vs. Convolution 𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗] Correlation Convolution 13204 458047 09373 .2.4.1 .5.8.3 .9.5.7 .2.4.1 .5.8.3 .9.5.7
Correlation Vs. Convolution 𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗] Correlation Convolution 13204 458047 09373 .2.4.1 .5.8.3 .9.5.7 .2.4.1 .5.8.3 .9.5.7
Correlation Vs. Convolution 𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗] Correlation Convolution 13204 458047 09373 .2.4.1 .5.8.3 .9.5.7 .2.4.1 .5.8.3 .9.5.7
Correlation Vs. Convolution 𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗] Correlation Convolution 13204 458047 09373 .2.4.1 .5.8.3 .9.5.7 .2.4.1 .5.8.3 .9.5.7 -1, 1-1, 0-1, -1 0, 10, 00, -1 1, 11, 01, -1 1, -11, 01, 1 0, -10, 00, 1 -1, -1-1, 0-1, 1 How to simplify convolution?
Simplifying Convolution .2.4.1 .5.8.3 .9.5.7 1, -11, 01, 1 0, -10, 00, 1 -1, -1-1, 0-1, 1 * 13204 458047 09373 .7.5.9 .3.8.5 .1.4.2 13204 458047 09373= -1, 1-1, 0-1, -1 0, 10, 00, -1 1, 11, 01, -1 Just rotate the template with 180 degrees before multiplying by the image. * Applying convolution is similar to correlation except for flipping the template before multiplication.
Two Many Multiplications & Additions • Assume to remove noise from an image, it should be applied to two filters (filter1 and filter2). Each filter has size of 7x7 pixels and the image is of size 1024x1024 pixels. • A simple approach is to apply filter1 to the image then apply filter2 to the output of filter1. • Too many multiplications and additions due to applying two filters on the image. 1,024 x 1,024 x 49 multiplication and 1,000 x 1,000 x 49 addition for the single filter. 1024x1024 Image Filter1 * Temp. Result Filter2 * Result
Convolution Associative Property • Using the associative property of convolution, we can reduce the number of multiplications and additions. • To apply two filters f1 and f2 over an image, we can merge the two filters together then apply the convolution between the resultant new filter and the image. • Finally, there is only 1,024 x 1024 x 49 multiplication and addition. (Im * 𝒇 𝟏) * 𝒇 𝟐 = Im * (𝒇 𝟏 * 𝒇 𝟐) For 𝒇 𝟏 * 𝒇 𝟐 = 𝒇 𝟑 Result is Im * 𝒇 𝟑
Convolution Applications • Convolution is used to merge signals. • It is used to apply operations like smoothing and filtering images where the primary task is selecting the appropriate filter template or mask. • Convolution can also be used to find gradients of the image.
Implementing Convolution in Python • The implementation of convolution is identical to correlation except for the new command that rotates the template.
GRADIENT
Image Gradients • Image gradients can be defined as change of intensity in some direction. • Based on the way gradients were calculated and specifically based on the template/kernel used, gradients can be horizontal (X direction) or vertical (Y direction) in direction. -1-1-1 000 111 Horizontal 10-1 10-1 10-1 Vertical θ = 𝑡𝑎𝑛−1 [ 𝐺 𝑦 𝐺 𝑥 ] 𝐺 = 𝐺 𝑥 + 𝐺 𝑦
Image Gradient Calculation Steps • Calculate the image gradient in X direction at each pixel. • Calculate the image gradient in Y direction at each pixel. • Calculate the overall gradient for the pixel.
Implementing Horizontal Gradient in Python
Implementing Horizontal Gradient in Python
Implementing Vertical Gradient in Python
Implementing Vertical Gradient in Python
FILTERING
Example – Mean Filter •Mean Filter Kernel. 𝟏 𝟗 𝟏 𝟗 𝟏 𝟗 𝟏 𝟗 𝟏 𝟗 𝟏 𝟗 𝟏 𝟗 𝟏 𝟗 𝟏 𝟗 58 3 213 81 78 185 87 32 27 11 70 66 60 2 19 61 91 129 89 38 14 7 58 14 42 𝟏 𝟗 ∗ 58 + 𝟏 𝟗 ∗ 3 + 𝟏 𝟗 ∗ 213 + 𝟏 𝟗 ∗ 185 + 𝟏 𝟗 ∗ 87 + 𝟏 𝟗 ∗ 32 + 𝟏 𝟗 ∗ 70 + 𝟏 𝟗 ∗ 66 + 𝟏 𝟗 ∗ 60 = 85.67 ~ 86 Implement in Python (Deadline next week starting @ 7 Oct. 2017). 1. Mean Smoothing Filter 2. Gaussian Smoothing Filter 3. Sharpening Filter 4. Median Filter

More Related Content

DOCX
Template Matching - Pattern Recognition
byMustafa Salam
 
PPTX
Hit and-miss transform
byKrish Everglades
 
PPTX
Image Segmentation using Otsu's Method - Computer Graphics (UCS505) Project PPT
byAkshit Arora
 
PPTX
Chapter 9 morphological image processing
byAhmed Daoud
 
PPT
Chapter 6 Image Processing: Image Enhancement
byVarun Ojha
 
PPTX
Digital Image Processing
byPreethiPreethi28
 
PPTX
Basic Relationships between Pixels- Digital Image Processing
byMd.Shohel Rana ( M.Sc in CSE Khulna University of Engineering & Technology (KUET))
 
PDF
Mathematical tools in dip
byVisvesvaraya National Institute of Technology, Nagpur, Maharashtra, India
 
Template Matching - Pattern Recognition
byMustafa Salam
 
Hit and-miss transform
byKrish Everglades
 
Image Segmentation using Otsu's Method - Computer Graphics (UCS505) Project PPT
byAkshit Arora
 
Chapter 9 morphological image processing
byAhmed Daoud
 
Chapter 6 Image Processing: Image Enhancement
byVarun Ojha
 
Digital Image Processing
byPreethiPreethi28
 
Basic Relationships between Pixels- Digital Image Processing
byMd.Shohel Rana ( M.Sc in CSE Khulna University of Engineering & Technology (KUET))
 
Mathematical tools in dip
byVisvesvaraya National Institute of Technology, Nagpur, Maharashtra, India
 

What's hot

PDF
Image segmentation
byVisvesvaraya National Institute of Technology, Nagpur, Maharashtra, India
 
PDF
Morphological operations
byNational Institute of Technology Durgapur
 
PDF
Spatial filtering
byVisvesvaraya National Institute of Technology, Nagpur, Maharashtra, India
 
PDF
Digital Image Processing: Digital Image Fundamentals
byMostafa G. M. Mostafa
 
PPT
Data Redundacy
byPoonam Seth
 
PDF
Operations in Digital Image Processing + Convolution by Example
byAhmed Gad
 
PPTX
Mathematical operations in image processing
byAsad Ali
 
PDF
03 image transform
byRumah Belajar
 
PPTX
Edge detection
byIshraq Al Fataftah
 
PPTX
impulse noise filter
byyousef_
 
PPTX
Image Filtering in the Frequency Domain
byAmnaakhaan
 
PPTX
COM2304: Intensity Transformation and Spatial Filtering – I (Intensity Transf...
byHemantha Kulathilake
 
PDF
Histogram processing
byVisvesvaraya National Institute of Technology, Nagpur, Maharashtra, India
 
PPTX
Color Models
byMustafa Salam
 
PPTX
Depth Buffer Method
byUmmiya Mohammedi
 
PPTX
Ray tracing
byLinh Nguyen
 
PPTX
Unit3 dip
byImran Khan
 
PPTX
Chapter 3 image enhancement (spatial domain)
byasodariyabhavesh
 
PPTX
Image processing second unit Notes
byAAKANKSHA JAIN
 
PPTX
Convolution&Correlation
byVenkateshPrasadV1
 
Image segmentation
byVisvesvaraya National Institute of Technology, Nagpur, Maharashtra, India
 
Morphological operations
byNational Institute of Technology Durgapur
 
Spatial filtering
byVisvesvaraya National Institute of Technology, Nagpur, Maharashtra, India
 
Digital Image Processing: Digital Image Fundamentals
byMostafa G. M. Mostafa
 
Data Redundacy
byPoonam Seth
 
Operations in Digital Image Processing + Convolution by Example
byAhmed Gad
 
Mathematical operations in image processing
byAsad Ali
 
03 image transform
byRumah Belajar
 
Edge detection
byIshraq Al Fataftah
 
impulse noise filter
byyousef_
 
Image Filtering in the Frequency Domain
byAmnaakhaan
 
COM2304: Intensity Transformation and Spatial Filtering – I (Intensity Transf...
byHemantha Kulathilake
 
Histogram processing
byVisvesvaraya National Institute of Technology, Nagpur, Maharashtra, India
 
Color Models
byMustafa Salam
 
Depth Buffer Method
byUmmiya Mohammedi
 
Ray tracing
byLinh Nguyen
 
Unit3 dip
byImran Khan
 
Chapter 3 image enhancement (spatial domain)
byasodariyabhavesh
 
Image processing second unit Notes
byAAKANKSHA JAIN
 
Convolution&Correlation
byVenkateshPrasadV1
 

Similar to Computer Vision: Correlation, Convolution, and Gradient

PPT
Template matching03
byamitkhanna1991
 
PDF
Lecture07
byzukun
 
PPT
Convolution.ppt
byAditiSharma852340
 
PPTX
Template Matching Computer VisionAI.pptx
byitzjokerehere2002
 
DOCX
Digtial Image Processing Q@A
byChung Hua Universit
 
PPTX
03 cie552 image_filtering_spatial
byElsayed Hemayed
 
PPT
05_Spatial_Filtering.ppt
bypawankamal3
 
PPT
Computer Vision - Image Filters
byYoss Cohen
 
PPTX
Estado del Arte de la IA
byPlain Concepts
 
PDF
Basics of Image Processing using MATLAB
byvkn13
 
PDF
Lecture-02_DigitalImageEnhancemnet-02.pdf
byMuhammad Hanif
 
PDF
GRUPO 2 : convolution separable
byviisonartificial2012
 
PPTX
Matlab level 1.pptx
byAbanobGozef
 
PPTX
Agile Analysis 101: Agile Stats v Command & Control Maths
byAxelisys Limited
 
PPTX
Dip day1&2
bynakarthik91
 
PPTX
5. convolution and correlation of discrete time signals
byMdFazleRabbi18
 
DOCX
The Digital Image Processing Q@A
byChung Hua Universit
 
PDF
TEMPLATE MATCHING TECHNIQUE FOR SEARCHING WORDS IN DOCUMENT IMAGES
byIJCI JOURNAL
 
PDF
2024-master dityv5y65v56u4b6u64u46p 0318-25.pdf
bynatnaeltamirat6212
 
PDF
Vision systems_Image processing tool box in MATLAB
byHinna Nayab
 
Template matching03
byamitkhanna1991
 
Lecture07
byzukun
 
Convolution.ppt
byAditiSharma852340
 
Template Matching Computer VisionAI.pptx
byitzjokerehere2002
 
Digtial Image Processing Q@A
byChung Hua Universit
 
03 cie552 image_filtering_spatial
byElsayed Hemayed
 
05_Spatial_Filtering.ppt
bypawankamal3
 
Computer Vision - Image Filters
byYoss Cohen
 
Estado del Arte de la IA
byPlain Concepts
 
Basics of Image Processing using MATLAB
byvkn13
 
Lecture-02_DigitalImageEnhancemnet-02.pdf
byMuhammad Hanif
 
GRUPO 2 : convolution separable
byviisonartificial2012
 
Matlab level 1.pptx
byAbanobGozef
 
Agile Analysis 101: Agile Stats v Command & Control Maths
byAxelisys Limited
 
Dip day1&2
bynakarthik91
 
5. convolution and correlation of discrete time signals
byMdFazleRabbi18
 
The Digital Image Processing Q@A
byChung Hua Universit
 
TEMPLATE MATCHING TECHNIQUE FOR SEARCHING WORDS IN DOCUMENT IMAGES
byIJCI JOURNAL
 
2024-master dityv5y65v56u4b6u64u46p 0318-25.pdf
bynatnaeltamirat6212
 
Vision systems_Image processing tool box in MATLAB
byHinna Nayab
 

More from Ahmed Gad

PPTX
ICEIT'20 Cython for Speeding-up Genetic Algorithm
byAhmed Gad
 
PDF
NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...
byAhmed Gad
 
PDF
Python for Computer Vision - Revision 2nd Edition
byAhmed Gad
 
PDF
Multi-Objective Optimization using Non-Dominated Sorting Genetic Algorithm wi...
byAhmed Gad
 
PDF
M.Sc. Thesis - Automatic People Counting in Crowded Scenes
byAhmed Gad
 
PDF
Derivation of Convolutional Neural Network from Fully Connected Network Step-...
byAhmed Gad
 
PDF
Introduction to Optimization with Genetic Algorithm (GA)
byAhmed Gad
 
PDF
Derivation of Convolutional Neural Network (ConvNet) from Fully Connected Net...
byAhmed Gad
 
PDF
Avoid Overfitting with Regularization
byAhmed Gad
 
PDF
Genetic Algorithm (GA) Optimization - Step-by-Step Example
byAhmed Gad
 
PDF
ICCES 2017 - Crowd Density Estimation Method using Regression Analysis
byAhmed Gad
 
PDF
Backpropagation: Understanding How to Update ANNs Weights Step-by-Step
byAhmed Gad
 
PDF
Python for Computer Vision - Revision
byAhmed Gad
 
PDF
Anime Studio Pro 10 Tutorial as Part of Multimedia Course
byAhmed Gad
 
PDF
Brief Introduction to Deep Learning + Solving XOR using ANNs
byAhmed Gad
 
PDF
MATLAB Code + Description : Real-Time Object Motion Detection and Tracking
byAhmed Gad
 
PDF
MATLAB Code + Description : Very Simple Automatic English Optical Character R...
byAhmed Gad
 
PDF
Graduation Project - Face Login : A Robust Face Identification System for Sec...
byAhmed Gad
 
PDF
Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...
byAhmed Gad
 
PDF
Introduction to Digital Signal Processing (DSP) - Course Notes
byAhmed Gad
 
ICEIT'20 Cython for Speeding-up Genetic Algorithm
byAhmed Gad
 
NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...
byAhmed Gad
 
Python for Computer Vision - Revision 2nd Edition
byAhmed Gad
 
Multi-Objective Optimization using Non-Dominated Sorting Genetic Algorithm wi...
byAhmed Gad
 
M.Sc. Thesis - Automatic People Counting in Crowded Scenes
byAhmed Gad
 
Derivation of Convolutional Neural Network from Fully Connected Network Step-...
byAhmed Gad
 
Introduction to Optimization with Genetic Algorithm (GA)
byAhmed Gad
 
Derivation of Convolutional Neural Network (ConvNet) from Fully Connected Net...
byAhmed Gad
 
Avoid Overfitting with Regularization
byAhmed Gad
 
Genetic Algorithm (GA) Optimization - Step-by-Step Example
byAhmed Gad
 
ICCES 2017 - Crowd Density Estimation Method using Regression Analysis
byAhmed Gad
 
Backpropagation: Understanding How to Update ANNs Weights Step-by-Step
byAhmed Gad
 
Python for Computer Vision - Revision
byAhmed Gad
 
Anime Studio Pro 10 Tutorial as Part of Multimedia Course
byAhmed Gad
 
Brief Introduction to Deep Learning + Solving XOR using ANNs
byAhmed Gad
 
MATLAB Code + Description : Real-Time Object Motion Detection and Tracking
byAhmed Gad
 
MATLAB Code + Description : Very Simple Automatic English Optical Character R...
byAhmed Gad
 
Graduation Project - Face Login : A Robust Face Identification System for Sec...
byAhmed Gad
 
Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...
byAhmed Gad
 
Introduction to Digital Signal Processing (DSP) - Course Notes
byAhmed Gad
 

Recently uploaded

PDF
Multiple Myeloma , definition, etiology, PP, CM , DE and management
byNisha Borsa
 
PDF
M.Sc. Nonchordates Complete Syllabus PPT | All Important Topics Covered
byKNIPSS SULTANPUR
 
PDF
IMANI Africa files RTI request seeking full disclosure on 2026 SIM registrati...
bynservice241
 
PPTX
Partial Correlation of Coefficient - Values of r₁₂.₃, r₂₃.₁ & r₁₃.₂
bySundar B N
 
PPTX
Steve Hale - Developing Elite Young Goalkeepers 2024.pptx
bypjmfd
 
PDF
Cultivating Greatness Pune's Best Preschools and Schools.pdf
byWellington College
 
PPTX
4 G8_Q3_L4 (Evaluating opinion editorials for textual evidence and quality).pptx
byNorafeSahagun
 
PPTX
Partial Correlation - Values of r₁₂.₃, r₂₃.₁ & r₁₃.₂ r₁₂, r₁₃ and r₂₃
bySundar B N
 
PPTX
ATTENTION -PART 2.pptx Shilpa Hotakar for I semester BSc students
bySHILPA HOTAKAR
 
PPTX
Expected Revenue Report In Odoo 18 CRM
byCeline George
 
PPTX
How to Access Revenue Report in Odoo 18 Events
byCeline George
 
PDF
Blood Group Incompatibility: Rh Factor and Erythroblastosis Fetalis
bySudhandraKarthi
 
PDF
Advanced Process Flow and Micromachining Techniques for MEMS Fabrication: A D...
byGS Virdi
 
PPTX
UNIT 1: COMMUNICATION SKILLS, BARRIERS TO COMMUNICATION, PERSPECTIVES IN COMM...
byPOOJA KARVANDE
 
PPTX
Brahma_Muhurta_Presentation Ayurvedic Perspective
bySudha Sumathy
 
PDF
International Men's Day Event: Breaking the Silence: Embracing Vulnerability ...
byAssociation for Project Management
 
PDF
Madhumathi Endowment Quiz Finals by Sreejesh P S and Shyam Krishnan P - Keral...
bySreejesh P S
 
PPTX
Unit 3: Gender Issues and Provisions F.Y.BEd Course
byRejoshaRajendran
 
PPTX
How to Track a Link Using Odoo 18 SMS Marketing
byCeline George
 
PDF
Comprehensive Principles, Design Techniques, and Applications of Electronic O...
byGS Virdi
 
Multiple Myeloma , definition, etiology, PP, CM , DE and management
byNisha Borsa
 
M.Sc. Nonchordates Complete Syllabus PPT | All Important Topics Covered
byKNIPSS SULTANPUR
 
IMANI Africa files RTI request seeking full disclosure on 2026 SIM registrati...
bynservice241
 
Partial Correlation of Coefficient - Values of r₁₂.₃, r₂₃.₁ & r₁₃.₂
bySundar B N
 
Steve Hale - Developing Elite Young Goalkeepers 2024.pptx
bypjmfd
 
Cultivating Greatness Pune's Best Preschools and Schools.pdf
byWellington College
 
4 G8_Q3_L4 (Evaluating opinion editorials for textual evidence and quality).pptx
byNorafeSahagun
 
Partial Correlation - Values of r₁₂.₃, r₂₃.₁ & r₁₃.₂ r₁₂, r₁₃ and r₂₃
bySundar B N
 
ATTENTION -PART 2.pptx Shilpa Hotakar for I semester BSc students
bySHILPA HOTAKAR
 
Expected Revenue Report In Odoo 18 CRM
byCeline George
 
How to Access Revenue Report in Odoo 18 Events
byCeline George
 
Blood Group Incompatibility: Rh Factor and Erythroblastosis Fetalis
bySudhandraKarthi
 
Advanced Process Flow and Micromachining Techniques for MEMS Fabrication: A D...
byGS Virdi
 
UNIT 1: COMMUNICATION SKILLS, BARRIERS TO COMMUNICATION, PERSPECTIVES IN COMM...
byPOOJA KARVANDE
 
Brahma_Muhurta_Presentation Ayurvedic Perspective
bySudha Sumathy
 
International Men's Day Event: Breaking the Silence: Embracing Vulnerability ...
byAssociation for Project Management
 
Madhumathi Endowment Quiz Finals by Sreejesh P S and Shyam Krishnan P - Keral...
bySreejesh P S
 
Unit 3: Gender Issues and Provisions F.Y.BEd Course
byRejoshaRajendran
 
How to Track a Link Using Odoo 18 SMS Marketing
byCeline George
 
Comprehensive Principles, Design Techniques, and Applications of Electronic O...
byGS Virdi
 

Computer Vision: Correlation, Convolution, and Gradient

  • 1.
    Computer Vision: Correlation, Convolution,and Gradient MENOUFIA UNIVERSITY FACULTY OF COMPUTERS AND INFORMATION INFORMATION TECHNOLOGY COMPUTER VISION ‫المنوفية‬ ‫جامعة‬ ‫والمعلومات‬ ‫الحاسبات‬ ‫كلية‬ ‫المعلومات‬ ‫تكنولوجيا‬ ‫بالحاسب‬ ‫الرؤيا‬ ‫المنوفية‬ ‫جامعة‬ Ahmed Fawzy Gad [email protected] Tuesday 3 October 2017
  • 2.
    Index • Correlation • PythonImplementation • Convolution • Python Implementation • Gradient • Python Implementation Correlation Convolution Gradient Filtering
  • 3.
  • 4.
    Correlation for 2DImage • Correlation is used to match a template to an image. • Can you tell where the following template exactly located in the image? • Using correlation we can find the image region that best matches the template. ?
  • 5.
    How 2D CorrelationWorks? • Given a template, using correlation the template will pass through each image part and a similarity check take place to find how similar the template and the current image part being processed. • Starting by placing the template top-left corner on the top-left corner of the image, a similarity measure is calculated.
  • 6.
    How 2D CorrelationWorks? • Given a template, using correlation the template will pass through each image part and a similarity check take place to find how similar the template and the current image part being processed. • Starting by placing the template top-left corner on the top-left corner of the image, a similarity measure is calculated. 𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗]
  • 7.
    How 2D CorrelationWorks? • Given a template, using correlation the template will pass through each image part and a similarity check take place to find how similar the template and the current image part being processed. • Starting by placing the template top-left corner on the top-left corner of the image, a similarity measure is calculated.
  • 8.
    Correlation for 2DImage 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 2 1 -4 3 2 5 -1 8 1
  • 9.
    Correlation for 2DImage 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 2 1 -4 3 2 5 -1 8 1
  • 10.
    Correlation for 2DImage 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 2 ∗ 1 + 1 ∗ 3 − 4 ∗ 4 + 3 ∗ 2 + 2 ∗ 7 + 5 ∗ 4 − 1 ∗ 6 + 8 ∗ 2 + 1 ∗ 5 = 𝟒𝟒
  • 11.
    Correlation for 2DImage 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 44 Did you get why template sizes are odd?
  • 12.
    Even Template Size 583 213 81 78 185 87 32 27 11 70 66 60 2 19 61 91 129 89 38 14 7 58 14 42 Even template sizes has no center. E.g. 2x3, 2x2, 5x6, … 𝟖𝟒 ???
  • 13.
    Correlation for 2DImage 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 2 1 -4 3 2 5 -1 8 1 44
  • 14.
    Correlation for 2DImage 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 2 ∗ 3 + 1 ∗ 4 − 4 ∗ 3 + 3 ∗ 7 + 2 ∗ 4 + 5 ∗ 1 − 1 ∗ 2 + 8 ∗ 5 + 1 ∗ 2 = 𝟕𝟐 44
  • 15.
    Correlation for 2DImage 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 44 72 Based on the current step, what is the region that best matches the template? It is the one corresponding to the highest score in the result matrix.
  • 16.
    Continue Correlating theTemplate 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 2 1 -4 3 2 5 -1 8 1 44 72
  • 17.
    Continue Correlating theTemplate 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 2 1 -4 3 2 5 -1 8 1 44 72 This will cause the program performing correlation to fall into index out of bounds exception.
  • 18.
    Padding by Zeros 13 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 44 72 0 0 0 0 It doesn`t affect multiplication. Why Zero?
  • 19.
    Continue Correlating theTemplate 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 44 72 0 0 0 0 2 1 -4 3 2 5 -1 8 1
  • 20.
    Continue Correlating theTemplate 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 44 72 0 0 0 0 2 ∗ 4 + 1 ∗ 3 − 4 ∗ 0 + 3 ∗ 4 + 2 ∗ 1 + 5 ∗ 0 − 1 ∗ 5 + 8 ∗ 2 + 1 ∗ 0 = 𝟑𝟔
  • 21.
    Continue Correlating theTemplate 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 44 72 36 0 0 0 0 2 ∗ 4 + 1 ∗ 3 − 4 ∗ 0 + 3 ∗ 4 + 2 ∗ 1 + 5 ∗ 0 − 1 ∗ 5 + 8 ∗ 2 + 1 ∗ 0 = 𝟑𝟔
  • 22.
    Continue Correlating theTemplate 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 2 1 -4 3 2 5 -1 8 1 44 72 36
  • 23.
    Continue Correlating theTemplate 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 2 1 -4 3 2 5 -1 8 1 44 72 36
  • 24.
    Pad by Zeros 13 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 44 72 36 0 0 0 0
  • 25.
    Continue Correlating theTemplate 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 44 72 36 0 0 0 0 2 1 -4 3 2 5 -1 8 1
  • 26.
    Continue Correlating theTemplate 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 44 72 36 0 0 0 0 2 ∗ 2 + 1 ∗ 5 − 4 ∗ 2 + 3 ∗ 6 + 2 ∗ 8 + 5 ∗ 9 − 1 ∗ 0 + 8 ∗ 0 + 1 ∗ 0 = 𝟖𝟎
  • 27.
    Continue Correlating theTemplate 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 Template – 3x3 44 72 36 80 Continue till end.
  • 28.
    • It dependson both the template size. • For 3x3 template, pad two columns and two rows. One column to the left and another to the right. One row at the top and one row at the bottom. • For 5x5? Pad 4 rows (two left & two right) and 5 columns (two top & two bottom). • Generally, for N*N template pad (N-1)/2 columns and rows at each side. How much Padding Required? 1 3 4 3 2 7 4 1 6 2 5 2 13 6 8 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 How to do this Programmatically.?
  • 29.
    Implementing Correlation inPython – 3x3 Template
  • 30.
    Implementing Correlation inPython – 141x141 Template
  • 31.
    Implementing Correlation inPython – 141x141 Template
  • 32.
    Implementing Correlation inPython – 141x141 Template
  • 33.
    Implementing Correlation inPython – 141x141 Template Is this is the optimal results? No. But WHY?
  • 34.
    Matching using Correlation •Matching depends on just multiplication. • The highest results of multiplication is the one selected as best match. • But the highest results of multiplication not always refers to best match. Template Result
  • 35.
    Matching using Correlation • Makea simple edit on the image by adding a pure white rectangular area. • Use the previous template. • Apply the algorithm. Template
  • 36.
    Matching using Correlation •The best matched region is the white region. • The reason is that correlation depends on multiplication. • What gives highest multiplication results is the best match. • Getting high results corresponds to multiplying by higher numbers. • The highest pixel value for gray images is 255 for white.
  • 37.
    Implementing Correlation inPython – Generic Code – Squared Odd Sized Templates
  • 38.
    Implementing Correlation inPython – Generic Code – Squared Odd Sized Templates
  • 39.
    Implementing Correlation inPython – Generic Code – Squared Odd Sized Templates
  • 40.
    Implementing Correlation inPython – Generic Code – Squared Odd Sized Templates
  • 41.
  • 42.
    What is Convolution? •The primary objective of mathematical convolution is combining two signals to generate a third signal. Convolution is not limited on digital image processing and it is a broad term that works on signals. Input Signal Impulse Response Output Signal System Convolution
  • 43.
    Convolution Formula for2D Digital Signal • Convolution is applied similarly to correlation. • Note how similar the formulas for correlation and convolution. The only difference is that in the signs of 𝒖 and 𝒗 variables. • Correlation formula has positive signs for 𝒖 and 𝒗 while correlation has negative signs for them. 𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗] Is just changing signs make sense?
  • 44.
    Correlation Vs. Convolution 𝑮𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗] Correlation Convolution 13204 458047 09373 .2.4.1 .5.8.3 .9.5.7 .2.4.1 .5.8.3 .9.5.7
  • 45.
    Correlation Vs. Convolution 𝑮𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗] Correlation Convolution 13204 458047 09373 .2.4.1 .5.8.3 .9.5.7 .2.4.1 .5.8.3 .9.5.7
  • 46.
    Correlation Vs. Convolution 𝑮𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗] Correlation Convolution 13204 458047 09373 .2.4.1 .5.8.3 .9.5.7 .2.4.1 .5.8.3 .9.5.7
  • 47.
    Correlation Vs. Convolution 𝑮𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗] Correlation Convolution 13204 458047 09373 .2.4.1 .5.8.3 .9.5.7 .2.4.1 .5.8.3 .9.5.7
  • 48.
    Correlation Vs. Convolution 𝑮𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗] Correlation Convolution 13204 458047 09373 .2.4.1 .5.8.3 .9.5.7 .2.4.1 .5.8.3 .9.5.7
  • 49.
    Correlation Vs. Convolution 𝑮𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗] Correlation Convolution 13204 458047 09373 .2.4.1 .5.8.3 .9.5.7 .2.4.1 .5.8.3 .9.5.7
  • 50.
    Correlation Vs. Convolution 𝑮𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗] Correlation Convolution 13204 458047 09373 .2.4.1 .5.8.3 .9.5.7 .2.4.1 .5.8.3 .9.5.7
  • 51.
    Correlation Vs. Convolution 𝑮𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗] Correlation Convolution 13204 458047 09373 .2.4.1 .5.8.3 .9.5.7 .2.4.1 .5.8.3 .9.5.7
  • 52.
    Correlation Vs. Convolution 𝑮𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗] Correlation Convolution 13204 458047 09373 .2.4.1 .5.8.3 .9.5.7 .2.4.1 .5.8.3 .9.5.7
  • 53.
    Correlation Vs. Convolution 𝑮𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 − 𝒖, 𝒋 − 𝒗]𝑮 𝒊, 𝒋 = 𝒖=−𝒌 𝒌 𝒗=−𝒌 𝒌 𝒉 𝒖, 𝒗 𝑭[𝒊 + 𝒖, 𝒋 + 𝒗] Correlation Convolution 13204 458047 09373 .2.4.1 .5.8.3 .9.5.7 .2.4.1 .5.8.3 .9.5.7 -1, 1-1, 0-1, -1 0, 10, 00, -1 1, 11, 01, -1 1, -11, 01, 1 0, -10, 00, 1 -1, -1-1, 0-1, 1 How to simplify convolution?
  • 54.
    Simplifying Convolution .2.4.1 .5.8.3 .9.5.7 1, -11,01, 1 0, -10, 00, 1 -1, -1-1, 0-1, 1 * 13204 458047 09373 .7.5.9 .3.8.5 .1.4.2 13204 458047 09373= -1, 1-1, 0-1, -1 0, 10, 00, -1 1, 11, 01, -1 Just rotate the template with 180 degrees before multiplying by the image. * Applying convolution is similar to correlation except for flipping the template before multiplication.
  • 55.
    Two Many Multiplications& Additions • Assume to remove noise from an image, it should be applied to two filters (filter1 and filter2). Each filter has size of 7x7 pixels and the image is of size 1024x1024 pixels. • A simple approach is to apply filter1 to the image then apply filter2 to the output of filter1. • Too many multiplications and additions due to applying two filters on the image. 1,024 x 1,024 x 49 multiplication and 1,000 x 1,000 x 49 addition for the single filter. 1024x1024 Image Filter1 * Temp. Result Filter2 * Result
  • 56.
    Convolution Associative Property •Using the associative property of convolution, we can reduce the number of multiplications and additions. • To apply two filters f1 and f2 over an image, we can merge the two filters together then apply the convolution between the resultant new filter and the image. • Finally, there is only 1,024 x 1024 x 49 multiplication and addition. (Im * 𝒇 𝟏) * 𝒇 𝟐 = Im * (𝒇 𝟏 * 𝒇 𝟐) For 𝒇 𝟏 * 𝒇 𝟐 = 𝒇 𝟑 Result is Im * 𝒇 𝟑
  • 57.
    Convolution Applications • Convolutionis used to merge signals. • It is used to apply operations like smoothing and filtering images where the primary task is selecting the appropriate filter template or mask. • Convolution can also be used to find gradients of the image.
  • 58.
    Implementing Convolution inPython • The implementation of convolution is identical to correlation except for the new command that rotates the template.
  • 59.
  • 60.
    Image Gradients • Imagegradients can be defined as change of intensity in some direction. • Based on the way gradients were calculated and specifically based on the template/kernel used, gradients can be horizontal (X direction) or vertical (Y direction) in direction. -1-1-1 000 111 Horizontal 10-1 10-1 10-1 Vertical θ = 𝑡𝑎𝑛−1 [ 𝐺 𝑦 𝐺 𝑥 ] 𝐺 = 𝐺 𝑥 + 𝐺 𝑦
  • 61.
    Image Gradient CalculationSteps • Calculate the image gradient in X direction at each pixel. • Calculate the image gradient in Y direction at each pixel. • Calculate the overall gradient for the pixel.
  • 62.
  • 63.
  • 64.
  • 65.
  • 66.
  • 67.
    Example – MeanFilter •Mean Filter Kernel. 𝟏 𝟗 𝟏 𝟗 𝟏 𝟗 𝟏 𝟗 𝟏 𝟗 𝟏 𝟗 𝟏 𝟗 𝟏 𝟗 𝟏 𝟗 58 3 213 81 78 185 87 32 27 11 70 66 60 2 19 61 91 129 89 38 14 7 58 14 42 𝟏 𝟗 ∗ 58 + 𝟏 𝟗 ∗ 3 + 𝟏 𝟗 ∗ 213 + 𝟏 𝟗 ∗ 185 + 𝟏 𝟗 ∗ 87 + 𝟏 𝟗 ∗ 32 + 𝟏 𝟗 ∗ 70 + 𝟏 𝟗 ∗ 66 + 𝟏 𝟗 ∗ 60 = 85.67 ~ 86 Implement in Python (Deadline next week starting @ 7 Oct. 2017). 1. Mean Smoothing Filter 2. Gaussian Smoothing Filter 3. Sharpening Filter 4. Median Filter