Lecture 06 — Edge Detection
Lecture 6
Where Are the Object Boundaries?
Which contours separate the elephants from the background or from each other?
Would you also mark the wrinkles, shadows, and grass as object boundaries?
Human-Annotated Boundaries
Human annotations emphasize meaningful contours, while omitting much of the texture.
We use shape and context to interpret boundaries, even where the intensity contrast is weak.
Edge Detection
An edge detector responds to local changes in image intensity.
These changes can come from object boundaries, but also from texture, shadows, and surface markings.
Edges and Object Boundaries
Edge strength does not necessarily correspond to our perception of boundaries.
A strong intensity change may occur within an object, such as a wrinkle or shadow.
An important object boundary may have weak contrast when neighboring regions have similar brightness.
Where are the object boundaries?
Which result better matches the boundaries you perceive? Why?
Should every visible change in brightness count as an edge?
Boundaries Can Be Ambiguous — Even for Us!
Where does the road end, and where do the snow-covered surroundings begin?
You can recognize vehicles ahead, but can you trace their complete outlines?
Do the strongest edges identify the most important boundaries?
Applications of Edges and Boundaries
Tissue engineering
Identify and count blood vessels.
Autonomous vehicles
Detect lane markings and road boundaries.
Image inpainting
Define a region to remove and fill.
Behavioral genetics
Trace worm contours to study shape and motion.
Semantic scene segmentation
Delineate regions such as roads, vehicles, and trees.
Interactive image editing
Follow object boundaries for selection and editing.
What Causes an Edge?
Can you find edges caused by different physical properties of the scene?
What Are Image Edges?
Surface height represents grayscale intensity at each pixel.
Edges occur where intensity changes sharply over a short distance. Look for the steep transition along the mountain silhouette.
Edge Detection
Edge detection
Identifying locations in an image where intensity changes sharply over a short distance.
Detecting Edges
How could we detect sharp changes in image intensity?
Take derivatives.
- Slowly varying intensity gives a small derivative.
- Sharp changes give large positive or negative responses.
How do we differentiate a discrete image—or any other sampled signal?
Use finite differences.
For samples spaced one pixel apart:
\[ \left.\frac{dI}{dx}\right|_{x=n} \approx \frac{I[n+1]-I[n-1]}{2}. \]
Derivative as a Filter
- When the kernel is centered on 125, its zero coefficient means the output depends only on the left and right neighbors.
- Reversing the kernel reverses the response’s sign.
- The factor \(\tfrac12\) is often omitted in kernels. This doubles the response, but does not affect its sign or locations of extrema.
Image Derivative via Filtering
- Filtered image: −1 = dark, 0 = mid-gray, +1 = bright.
Reminder: Sobel Filters
- Sobel x measures changes across columns and emphasizes vertical edges.
- Sobel y measures changes across rows and emphasizes horizontal edges.
Decomposing the Sobel Filters
Each Sobel kernel is the product of a column vector and a row vector:
| Filter | Column vector | Row vector |
|---|---|---|
| Sobel x | Vertical weighted smoothing | Horizontal derivative |
| Sobel y | Vertical derivative | Horizontal weighted smoothing |
Sobel combines differentiation in one direction with smoothing in the perpendicular direction.
Several Derivative Filters
- Sobel, Scharr, and Prewitt combine a derivative with different perpendicular smoothing weights.
- Roberts measures intensity change along the down-right and down-left directions in a \(2\times2\) neighborhood.
How are these filters related? How could we build a derivative filter larger than \(3\times3\)?
Comparing Derivative Filters
- All responses use a common signed grayscale scale: zero is mid-gray.
- Sobel, Scharr, and Prewitt measure horizontal intensity change; this Roberts kernel measures change in the down-right direction.
- Roberts uses a smaller neighborhood, with its derivative estimate centered between pixels.
Step 1: Choose Derivative Filters
Sobel filters for intensity changes in the x and y directions:
- Using intensities 0 and 8 to keep math simple.
- The factor \(\tfrac18\) scales down Sobel response.
- Image coordinates: x increases to the right, and y increases downward.
Step 2: Filter the Image to Compute Derivatives
Filter the image with each kernel to obtain two derivative images: \(I_x\) and \(I_y\).
- Our boundary rule: repeat the nearest boundary pixel so both outputs remain \(3\times3\).
- Immediately left of the bright pixel, \(I_x=(2\times8)/8=2\) and \(I_y=0\).
- At the center, opposite contributions cancel: both derivatives are zero.
Step 3: Compute Gradient Magnitude
The two derivatives form a vector at every pixel:
\[ \nabla I= \begin{bmatrix} I_x\\ I_y \end{bmatrix}, \qquad M=\|\nabla I\|=\sqrt{I_x^2+I_y^2}. \]
- Magnitude measures how strongly intensity changes, regardless of direction.
- Magnitudes are nonnegative (displayed values are rounded).
Step 3: Compute Gradient Direction
Use both signed derivatives to determine the direction:
\[ \theta=\operatorname{atan2}(I_y,I_x). \]
- The gradient points toward the greatest increase in intensity, perpendicular to a local edge.
- With image coordinates: 0° = right, 90° = down, 180° = left, −90° = up.
- The center has zero magnitude, so its direction is undefined, shown as —.
Image Gradient Example
How does the gradient direction relate to the orientation of edges in the image?
How Do We Find the Edge of This Signal?
- The signal contains a large intensity transition and many smaller fluctuations.
How can we distinguish the edge from noise?
Differentiation Is Sensitive to Noise
- The derivative responds to rapid changes caused by both the edge and noise.
- Noise creates many positive and negative responses.
- A large derivative alone does not guarantee a meaningful edge.
What could we do before taking the derivative?
Smooth Before Differentiating
- Gaussian smoothing reduces rapid noise fluctuations before differentiation.
- How much should we smooth? Increasing \(\sigma\) reduces noise, but broadens transitions and can merge nearby edges.
Derivative of Gaussian
We can combine smoothing and differentiation into one filter.
\[ \frac{d}{dx}\bigl(G_\sigma * I\bigr) = \left(\frac{dG_\sigma}{dx}\right)*I, \qquad *\text{ denotes convolution}. \]
- This gives the same result as smoothing and saves computation.
Second Derivatives and the Laplacian
- A second derivative measures how the first derivative changes.
- With one-pixel spacing, compare the differences on either side of a pixel:
\[ d_{\mathrm{right}}=I[n+1]-I[n], \qquad d_{\mathrm{left}}=I[n]-I[n-1]. \]
\[ I''[n]\approx d_{\mathrm{right}}-d_{\mathrm{left}} = I[n+1]-2I[n]+I[n-1]. \]
- At a smooth peak of the first derivative, the second derivative changes sign.
- In 2D, the Laplacian adds the second derivatives in both directions:
\[ \nabla^2 I= \frac{\partial^2 I}{\partial x^2} + \frac{\partial^2 I}{\partial y^2}. \]
Laplacian of Gaussian: Zero Crossings
Smooth first, then take the second derivative—or combine both operations into one filter.
- A first-derivative peak corresponds to a zero crossing of the second derivative.
- The crossing (sign change) can occur between samples; no sample needs to equal zero exactly.
- Noise also produces zero crossings. We need sufficient response strength to identify useful edge candidates.
Gaussian Filter and Its Derivatives in 2D
\[ G_\sigma(x,y)= \frac{1}{2\pi\sigma^2} e^{-\frac{x^2+y^2}{2\sigma^2}} \]
\[ \frac{\partial G_\sigma}{\partial x} = -\frac{x}{\sigma^2}G_\sigma \]
\[ \nabla^2G_\sigma = \frac{x^2+y^2-2\sigma^2}{\sigma^4}G_\sigma \]
In the derivative kernel images, gray represents zero, white positive values, and black negative values.
Gaussian and Derivative Filter Outputs
- Gaussian smoothing reduces fine detail and noise.
- The x derivative responds to intensity changes along the horizontal direction.
- The Laplacian combines second derivatives in x and y. Its zero crossings are candidate edge locations.
Which boundaries stand out, and which texture details also produce responses?
Canny Edge Detection
How can we obtain thin, connected edges while suppressing noise?
Canny Edge Detection Algorithm
- Step 1: Filter the image with Gaussian derivative kernels.
- Step 2: Compute gradient magnitude and direction.
- Step 3: Thin the responses using non-maximum suppression.
- Step 4: Select and connect edge pixels using hysteresis thresholding.
The output of the Canny Edge Detection Algorithm is a binary edge map.
Step 1: Gaussian Derivative Filters
- Gaussian smoothing uses a user-provided \(\sigma\) value.
- The derivative-of-Gaussian filters measure intensity changes in x and y while suppressing fine-scale noise.
- Each kernel filters the original grayscale image directly. Both outputs have the same size.
Step 2: Gradient Magnitude and Direction
\[ M=\sqrt{I_x^2+I_y^2}, \qquad \theta=\operatorname{atan2}(I_y,I_x) \]
- Magnitude measures the strength of the local intensity change.
- Direction points across the edge, perpendicular to its local orientation.
Responses often form a thick ridge. Which pixels should represent the edge?
Step 3: Non-Maximum Suppression
Non-max supression keeps a pixel only if its gradient magnitude is a local maximum along the gradient direction.
- Here, the gradient points horizontally, so compare each pixel with its left and right neighbors.
- Keep \(9\) because it exceeds both \(6\) and \(5\). Suppress the smaller responses to zero.
- Compare across the edge, not along it.
The retained values are still gradient magnitudes, not a binary edge map.
Comparing Along an Arbitrary Direction
The comparison locations may fall between pixel centers:
\[ (x_\pm,y_\pm)=(x\pm\cos\theta,\ y\pm\sin\theta). \]
Bilinear interpolation estimates the magnitude from four neighboring pixels.
First, interpolate horizontally:
\[ \text{Top}=0.75(2)+0.25(6)=3 \]
\[ \text{Bottom}=0.75(4)+0.25(8)=5 \]
Then, interpolate vertically:
\[ M(0.25,0.5)=0.5(3)+0.5(5)=4 \]
It is a weighted average: closer pixels contribute more.
Non-Maximum Suppression on the Image
- The broad ridges become thin responses.
- Small responses from texture and noise can still remain.
Which of these responses should we keep?
Step 4: Two Thresholds
Apply user-provided low & high thresholds to the magnitude (after non-maximum suppression).
- For this example, say \(T_{\mathrm{low}}=3\) and \(T_{\mathrm{high}}=8\), and denote magnitude by \(N\).
- Strong: \(N\geq T_{\mathrm{high}}\). Keep as an edge seed.
- Weak: \(T_{\mathrm{low}}\leq N<T_{\mathrm{high}}\). Decide using connectivity.
- Rejected: \(N<T_{\mathrm{low}}\).
Should every weak pixel become an edge?
Hysteresis: Keep Weak Pixels Connected to Strong Ones
- Start from strong pixels and follow chains of weak pixels.
- Horizontal, vertical, and diagonal neighbors count as connected.
- Keep the entire connected chain, even when a weak pixel is several steps from a strong pixel.
- Remove weak-only groups. Do not bridge gaps of rejected pixels.
The output is a binary edge map: retained edge pixels are 1, and all other pixels are 0.
Why Use Hysteresis?
- A low threshold preserves weaker boundaries but also retains more unwanted responses.
- A high threshold rejects more responses but can break edge curves.
- Hysteresis retains weak sections when a chain connects them to a strong edge seed.
Connectivity provides evidence that a weak response belongs to an edge.
The Complete Canny Pipeline
Which detected edges correspond to object boundaries, and which come from surface texture or illumination?
Choosing the Smoothing Scale
The 2 thresholds are fixed across the examples above.
- Smaller \(\sigma\) preserves fine details and more texture.
- Larger \(\sigma\) suppresses fine detail and can merge nearby boundaries.
- Smoothing also changes gradient strength, so thresholds may need adjustment.
Which scale best matches the boundaries you want to detect?
Canny in OpenCV
# Each setting is: (sigma, low, high).
settings = [
(1.0, 20, 50),
(3.0, 20, 50),
(1.0, 40, 100)]
outputs = []
# I_u8 is the grayscale image in uint8 format.
for sigma, low, high in settings:
blurred = cv2.GaussianBlur(
I_u8, (0, 0), sigmaX=sigma
)
result = cv2.Canny(
blurred,
threshold1=low,
threshold2=high,
apertureSize=3,
L2gradient=True
)
outputs.append(result)
sigmacontrols the explicit Gaussian smoothing before Canny.apertureSize=3selects 3×3 Sobel derivatives;L2gradient=Trueuses \(\sqrt{I_x^2+I_y^2}\).- Thresholds apply to gradient strength, not pixel brightness.
- For tuning thresholds, note that OpenCV uses uint8 input, with values from 0 to 255.