Image Basics
This tutorial covers reading and writing images in Python, indexing into an image as a matrix, kernels and convolution, a few basic filters including Sobel gradients and the Laplacian, and resizing, rotation, brightness and perspective changes. It goes with the Image Retrieval 1 lecture (decks 2 and 3).
All the code is in one file, image_basics.py. Download it and the sample image, astronaut.png (a public-domain NASA photo), then run it:
$ pip install numpy pillow
$ python3 image_basics.py astronaut.png
shape: (512, 512) dtype: float32
wrote gray, box, gaussian, sharpen, sobel_x, sobel_y, sobel_mag, laplacian, log
wrote small_slice, small_lanczos, flip, rotate_nearest, rotate_pil, brighter, contrast, gamma, perspective
Image I/O
An image file is a compressed encoding (PNG, JPEG, ...). To work with the pixels, decode it with Pillow and turn it into a NumPy array:
import numpy as np
from PIL import Image
rgb = np.asarray(Image.open("astronaut.png"))
print(rgb.shape, rgb.dtype) # (512, 512, 3) uint8
A color image is a 3-D array of shape (height, width, 3): one 8-bit value from 0 to 255 for each of red, green and blue. Most of the filters in this course work on a single grayscale channel, so convert first. Use float32 so arithmetic doesn't overflow:
def load_gray(path):
"""Read an image file and return a 2-D float32 array in [0, 255]."""
return np.asarray(Image.open(path).convert("L"), dtype=np.float32)
To write an array back out, clip it to the 0–255 range and convert back to uint8:
def save(arr, path):
"""Clip an array to [0, 255] and write it as an 8-bit image."""
Image.fromarray(np.clip(arr, 0, 255).astype(np.uint8)).save(path)
Pitfall: uint8 arithmetic wraps around silently. np.array([200], dtype=np.uint8) + 100 gives [44], not 300. Convert to float32 before filtering, and clip only when you save.
Indexing an image matrix
The origin is the top-left corner. The first index is the row (y, increasing downward) and the second is the column (x, increasing to the right). So im[i, j] is the pixel i rows down and j columns across, which is the reverse of the usual (x, y) order.
rgb[0, 0] # top-left pixel: array([154, 147, 151], dtype=uint8)
rgb[0, 0, 0] # its red value: 154
im = load_gray("astronaut.png")
im.shape # (512, 512): (height, width)
im[100:105, 200:205] # a 5 x 5 patch: rows 100-104, columns 200-204
[[ 60 43 130 154 32]
[ 61 61 93 91 27]
[ 72 61 45 37 38]
[126 84 79 66 70]
[157 157 166 141 120]]
Slicing returns a view, not a copy, so assigning to a slice changes the original image:
crop = im[:256, 256:] # top-right quarter
im[:50, :50] = 0 # paint the top-left 50 x 50 block black
patch = im[10:20, 10:20].copy() # use .copy() when you need an independent array
Kernels and convolution
A kernel is a small matrix of weights, usually 3×3 or 5×5 and always odd-sized so it has a center. Applying a kernel means sliding it over every pixel: each output pixel is the weighted sum of the input pixels under the kernel, with the kernel's center k[0, 0] on the pixel being computed.
out[i, j] = Σu Σv k[u, v] · im[i + u, j + v] for u = −a … a, v = −b … b
for a (2a+1)×(2b+1) kernel. Strictly, this is cross-correlation; true convolution flips the kernel first. The two agree for symmetric kernels like blurs, and computer-vision libraries usually mean this formula when they say "filter".
Pixels near the border need neighbors that don't exist. A common fix is to pad the image by repeating its edge pixels (mode="edge"). Here is the formula written out directly:
def correlate_slow(im, k):
"""Apply kernel k to every pixel of im, one pixel at a time."""
a, b = k.shape[0] // 2, k.shape[1] // 2
padded = np.pad(im, ((a, a), (b, b)), mode="edge")
out = np.zeros_like(im)
for i in range(im.shape[0]):
for j in range(im.shape[1]):
total = 0.0
for u in range(-a, a + 1):
for v in range(-b, b + 1):
total += k[u + a, v + b] * padded[i + a + u, j + b + v]
out[i, j] = total
return out
That's four nested Python loops: about 2 seconds for one 512×512 image, which adds up fast across many images. Swap the loop order: for each kernel entry, shift the whole padded image by (u, v), multiply by the weight and add. Now NumPy does the per-pixel work and Python only loops over the 9 or 25 kernel entries. This version takes about 5 milliseconds, roughly 400× faster:
def correlate(im, k):
"""Same result as correlate_slow, but loops over the kernel, not the image."""
a, b = k.shape[0] // 2, k.shape[1] // 2
padded = np.pad(im, ((a, a), (b, b)), mode="edge")
height, width = im.shape
out = np.zeros_like(im)
for u in range(-a, a + 1):
for v in range(-b, b + 1):
out += k[u + a, v + b] * padded[a + u:a + u + height,
b + v:b + v + width]
return out
Basic filters
Blurring: box and Gaussian
A box filter replaces each pixel with the plain average of its neighborhood. A Gaussian filter takes a weighted average that favors nearby pixels, which blurs more smoothly. Both kernels sum to 1, so overall brightness is unchanged.
BOX = np.ones((5, 5), dtype=np.float32) / 25
def gaussian_kernel(size, sigma):
"""Return a size x size Gaussian kernel whose entries sum to 1."""
r = size // 2
x = np.arange(-r, r + 1, dtype=np.float32)
g = np.exp(-x**2 / (2 * sigma**2))
k = np.outer(g, g)
return k / k.sum()
blurred = correlate(im, gaussian_kernel(9, 2.0))
np.outer(g, g) builds the 2-D kernel from a 1-D one. A Gaussian is separable: blurring down and then across gives the same result as one 2-D pass.
Sharpening
The sharpen kernel adds to each pixel the difference between it and its four neighbors, exaggerating local contrast. Its weights also sum to 1.
| 0 | −1 | 0 |
| −1 | 5 | −1 |
| 0 | −1 | 0 |




Sobel gradients
An edge is where brightness changes quickly, so finding edges means estimating the image gradient. A plain central difference [−1, 0, 1] responds to noise as much as to edges, so Sobel combines it with smoothing: smooth down with [1, 2, 1] and take the difference across.
| −1 | 0 | 1 |
| −2 | 0 | 2 |
| −1 | 0 | 1 |
| −1 | −2 | −1 |
| 0 | 0 | 0 |
| 1 | 2 | 1 |
SOBEL_X (left) and SOBEL_Y (right, its transpose).
SOBEL_X = np.array([[-1, 0, 1],
[-2, 0, 2],
[-1, 0, 1]], dtype=np.float32)
SOBEL_Y = SOBEL_X.T
gx = correlate(im, SOBEL_X) # positive where it gets brighter to the right
gy = correlate(im, SOBEL_Y) # positive where it gets brighter downward
magnitude = np.hypot(gx, gy) # sqrt(gx**2 + gy**2): edge strength
direction = np.arctan2(gy, gx) # edge orientation, in radians
gx and gy can be negative, so to view them, shift zero to mid-gray (128 + gx / 8). gx picks out vertical edges and gy horizontal ones. The magnitude combines the two into one edge map.

gx
gy
Gradients are the building block for the rest of the lecture: Harris corners are built from sums of gx and gy products, and gradient histograms bin direction weighted by magnitude.
Laplacian
Sobel estimates the first derivative: how fast brightness is changing. The Laplacian estimates the second derivative, summed over both directions: how fast that change is itself changing. Its kernel compares each pixel with its four neighbors:
| 0 | 1 | 0 |
| 1 | −4 | 1 |
| 0 | 1 | 0 |
LAPLACIAN = np.array([[0, 1, 0],
[1, -4, 1],
[0, 1, 0]], dtype=np.float32)
lap = correlate(im, LAPLACIAN)
The weights sum to 0, so flat regions and smooth ramps give 0. Across an edge, the response swings positive on one side and negative on the other, and the edge itself sits at the zero crossing between them. Unlike Sobel, it is a single kernel with no direction: it treats edges at every angle the same way.
The sharpen kernel above is just identity − LAPLACIAN: sharpening subtracts the Laplacian from the image, which pushes each side of an edge further apart.
Second derivatives amplify noise even more than first derivatives, so in practice you smooth first. Blurring with a Gaussian and then applying the Laplacian is called the Laplacian of Gaussian (LoG). The larger sigma is, the coarser the structures it responds to, which makes LoG a classic detector for blobs of a given size.
log = correlate(correlate(im, gaussian_kernel(9, 2.0)), LAPLACIAN)


The plain Laplacian picks up fine texture and noise everywhere; the LoG version keeps only the larger outlines.
Resizing and transformations
Retrieval has to cope with the same scene photographed at a different size, angle or exposure. Each of those is a transformation you can apply yourself, which is also how you test whether a detector or descriptor is robust to it. These examples use Pillow on the grayscale image, pil = Image.fromarray(im.astype(np.uint8)), alongside NumPy versions where they're short.
Resizing
Shrinking an image by keeping every 4th pixel is one slice, im[::4, ::4], but it throws away the 15 pixels in between instead of averaging them. Fine detail that repeats faster than the new pixel spacing turns into jagged or speckled patterns. This is called aliasing. The fix is to blur first, for example with a Gaussian, and then sample. A good resampling filter like Lanczos does both in one step:
small = im[::4, ::4] # fast, but aliases
small = pil.resize((128, 128), Image.Resampling.LANCZOS) # smooths, then samples
big = pil.resize((1024, 1024), Image.Resampling.BICUBIC) # enlarging: interpolate
The size argument is (width, height), the reverse of NumPy's (height, width) shape. When enlarging, NEAREST repeats pixels as blocks, while BILINEAR and BICUBIC interpolate between them.

im[::4, ::4], top-left corner enlarged 6×
Flipping and rotating
Flips and quarter turns only move pixels, so they are exact: np.fliplr(im), np.flipud(im) and np.rot90(im). Any other angle lands output pixels between input pixels, which needs inverse mapping: loop over the output pixels and ask which input location each came from. Going the other way, pushing each input pixel forward, leaves holes in the output.
def rotate_nearest(im, degrees):
"""Rotate counterclockwise about the center, by inverse mapping:
for each output pixel, look up the input pixel it came from."""
height, width = im.shape
cy, cx = (height - 1) / 2, (width - 1) / 2
theta = np.deg2rad(degrees)
rows, cols = np.indices(im.shape)
y, x = rows - cy, cols - cx # output, relative to center
src_x = np.cos(theta) * x - np.sin(theta) * y + cx
src_y = np.sin(theta) * x + np.cos(theta) * y + cy
src_i, src_j = np.round(src_y).astype(int), np.round(src_x).astype(int)
inside = (src_i >= 0) & (src_i < height) & (src_j >= 0) & (src_j < width)
out = np.zeros_like(im) # black where nothing maps
out[inside] = im[src_i[inside], src_j[inside]]
return out
Rounding to the nearest input pixel gives the same result as pil.rotate(30, resample=Image.Resampling.NEAREST). BILINEAR instead blends the four input pixels around each source location, which avoids jagged edges. Corners that map from outside the input come out black. Pass expand=True to enlarge the canvas so nothing is cut off.

np.fliplr(im)
rotate_nearest(im, 30)
pil.rotate(30, BILINEAR)Brightness, contrast and gamma
These change pixel values but not positions, so they are just arithmetic on the array:
def adjust(im, contrast=1.0, brightness=0.0, gamma=1.0):
"""Scale contrast around mid-gray, add brightness, then apply gamma."""
out = np.clip(contrast * (im - 128) + 128 + brightness, 0, 255)
return 255 * (out / 255) ** gamma
- Brightness adds a constant to every pixel.
- Contrast stretches values away from mid-gray (above 1) or squeezes them toward it (below 1).
- Gamma bends the curve: below 1 brightens the shadows more than the highlights, and above 1 darkens them.
Clipping at 0 and 255 loses information: two pixels that both clip to 255 can no longer be told apart. Brightness and contrast changes keep the order of pixel values, except where clipping happens. That is why BRIEF, which only compares pairs of pixels, isn't affected by them.

brightness=60
contrast=1.6
gamma=0.5Perspective
Photographing a flat surface from an angle makes a rectangle look like a general four-sided shape. Parallel lines meet, and far parts shrink. The map from output (x, y) to input (X, Y) is a perspective transform (a homography) with 8 unknowns:
X = (a·x + b·y + c) / (g·x + h·y + 1) Y = (d·x + e·y + f) / (g·x + h·y + 1)
Each point correspondence gives two linear equations, so 4 points pin down all 8 unknowns. Multiply both sides by the denominator and move the unknowns to one side:
def perspective_coeffs(dst, src):
"""Solve for the 8 coefficients (a..h) of the perspective map taking
each output point (x, y) in dst to its input point (X, Y) in src:
X = (a x + b y + c) / (g x + h y + 1), Y = (d x + e y + f) / (g x + h y + 1)."""
rows, rhs = [], []
for (x, y), (big_x, big_y) in zip(dst, src):
rows.append([x, y, 1, 0, 0, 0, -big_x * x, -big_x * y])
rows.append([0, 0, 0, x, y, 1, -big_y * x, -big_y * y])
rhs += [big_x, big_y]
return np.linalg.solve(np.array(rows, dtype=float), np.array(rhs, dtype=float))
corners = [(0, 0), (511, 0), (511, 511), (0, 511)] # input corners, (x, y)
tilted = [(80, 60), (511, 0), (511, 511), (80, 451)] # where they should land
coeffs = perspective_coeffs(tilted, corners)
warped = pil.transform(pil.size, Image.Transform.PERSPECTIVE, coeffs,
Image.Resampling.BILINEAR)
Pillow, like rotate_nearest, maps each output point back to the input, so the coefficients go from tilted to corners. Squeezing the left edge makes the photo look turned away from the camera. With g = h = 0 the denominator is 1 and the map is affine, which covers scaling, rotation, shear and translation. OpenCV's cv2.getPerspectiveTransform solves the same system. Estimating these coefficients from matched keypoints is the core of geometric verification in the next lecture.

Using a library
Write it yourself once to understand it; in real code, use a library. These give the same results as correlate above:
from scipy import ndimage
out = ndimage.correlate(im, SOBEL_X, mode="nearest")
import cv2 # pip install opencv-python
out = cv2.filter2D(im, -1, SOBEL_X, borderType=cv2.BORDER_REPLICATE)
gx = cv2.Sobel(im, cv2.CV_32F, 1, 0, ksize=3, borderType=cv2.BORDER_REPLICATE)
blurred = cv2.GaussianBlur(im, (9, 9), 2.0, borderType=cv2.BORDER_REPLICATE)
lap = cv2.Laplacian(im, cv2.CV_32F, ksize=1, borderType=cv2.BORDER_REPLICATE)
lap = ndimage.laplace(im, mode="nearest")
The border options make the edge pixels match our mode="edge" padding; library defaults differ, which only changes a 1–4 pixel band around the border. Also note that OpenCV's cv2.imread returns color channels in BGR order, not RGB.
Try it
- Blur with Gaussians of increasing
sigmabefore running Sobel. What happens to the fine edges in the hair and the flag? - Check that a 9×9 Gaussian applied as one 2-D kernel matches a 9×1 pass followed by a 1×9 pass. Which is faster, and why?
- Find the zero crossings of the LoG image: pixels where the sign differs from the pixel to the right or below. Compare the result with the Sobel magnitude.
- Run Sobel on
small_sliceand onsmall_lanczos. Which edge map looks more like a shrunken version of the full-size one? - Threshold
magnitudeto get a black-and-white edge map. How does the threshold you choose change the result?