Computer Vision: From Pixels to Perception

Run a complete example · 01

Follow one channel

A list represents one known blue pixel. No image file or trained model is needed.

bgr = [255, 0, 0]
rgb = list(reversed(bgr))
value = 153
scaled = value / 255
standardized = (scaled - 0.5) / 0.25
print("RGB:", rgb)
print(f"scaled={scaled:.1f}; standardized={standardized:.1f}")
print("stored numbers:", 4 * 6 * 3)

python pixels-contracts-pixel-scale.py

RGB: [0, 0, 255]
scaled=0.6; standardized=0.4
stored numbers: 72

Read the code step by step

The first line stores blue, green, red. reversed changes their order and list stores the result. The next three lines scale 153, subtract 0.5, and divide by 0.25. The f-string prints one decimal place. The final multiplication counts 72 channel values, not 72 separate pixels. This checks data conventions; it does not recognize an object.

Run a complete example · 01

Test three known colours

Use ordinary RGB tuples and a transparent rule.

pixels = {"marker": (200, 40, 30), "paper": (220, 220, 220), "shade": (60, 20, 20)}
for name, (red, green, blue) in pixels.items():
    margin = red - max(green, blue)
    print(name, "margin:", margin, "selected:", margin > 50)

python colour-channels-red-channel-rule.py

marker margin: 160 selected: True
paper margin: 0 selected: False
shade margin: 40 selected: False

Read the code step by step

The dictionary links each name to three channel values. items returns name–value pairs, and unpacking names the channels. max chooses the stronger competing channel. Subtraction measures the gap, and >50 returns True or False. This is a hand-set colour filter, not a trained detector or a complete HSV conversion.

Run a complete example · 01

Average a small patch

Inspect a bright speck and a signed edge response. This is a hand-written filter, not a learned model.

patch = [10, 10, 10, 10, 19, 10, 10, 10, 10]
mean = sum(patch) / len(patch)
row = [2, 5, 8]
weights = [-1, 0, 1]
response = sum(value * weight for value, weight in zip(row, weights))
print(f"mean={mean:.1f}")
print("edge response:", response)

python filters-neighborhoods-local-filter.py

mean=11.0
edge response: 6

Read the code step by step

sum adds nine values and len counts them, giving 99/9 = 11. zip pairs each row value with a weight. The generator multiplies each pair; sum adds −2, 0 and 8 to get 6. The example computes only one valid location, so it avoids the separate padding question.

Run a complete example · 01

Remove an isolated speck

Use zero padding and a three-position neighbourhood.

mask = [0, 1, 0, 0, 1, 1, 1, 0]
def local_rule(values, operation):
    padded = [0] + values + [0]
    return [operation(padded[i:i + 3]) for i in range(len(values))]
eroded = local_rule(mask, min)
opened = local_rule(eroded, max)
print("eroded:", eroded)
print("opened:", opened)

python edges-morphology-binary-opening.py

eroded: [0, 0, 0, 0, 0, 1, 0, 0]
opened: [0, 0, 0, 0, 1, 1, 1, 0]

Read the code step by step

def names a reusable function. Adding lists supplies one zero at each end. range visits each original position; the slice i:i+3 takes three values and excludes its end index. Passing min performs erosion, while passing max performs dilation. Opening uses the eroded list as its next input. This is an exact toy morphology rule, not a learned image system.

Run a complete example · 01

Move both box corners

Scale first, then shift, using ordinary numbers.

corners = [(10, 20), (30, 50)]
moved = [(2 * x + 5, 2 * y - 3) for x, y in corners]
(x1, y1), (x2, y2) = moved
print("corners:", moved)
print("area:", (x2 - x1) * (y2 - y1))
print("clipped bottom:", min(y2, 80))

python geometry-augmentation-move-box.py

corners: [(25, 37), (65, 97)]
area: 2400
clipped bottom: 80

Read the code step by step

The list holds two coordinate pairs. The list comprehension applies the same two equations to each pair. Unpacking names the four new coordinates. Subtraction finds width 40 and height 60; multiplication gives area 2400. min clips the bottom coordinate to 80 but a real pipeline must also update visibility rules. This transforms annotations; it does not resample image pixels.

Run a complete example · 01

Compare two patch descriptions

Distances are computed from hand-set descriptors.

import math
query = (1, 2)
candidates = {"A": (1, 3), "B": (4, 6)}
def distance(a, b):
    return math.sqrt(sum((x - y) ** 2 for x, y in zip(a, b)))
ranked = sorted((distance(query, value), name) for name, value in candidates.items())
ratio = ranked[0][0] / ranked[1][0]
print("ranked:", ranked)
print(f"ratio={ratio:.2f}; accept={ratio < 0.8}")

python classical-features-descriptor-distance.py

ranked: [(1.0, 'A'), (5.0, 'B')]
ratio=0.20; accept=True

Read the code step by step

zip pairs entries, **2 squares their differences, sum adds them, and math.sqrt takes the square root. sorted orders (distance,name) pairs from smallest distance. ranked[0][0] selects the first pair’s distance; ranked[1][0] selects the next. The ratio is 0.20. This matches invented descriptors; it does not find image keypoints or implement SIFT.

Run a complete example · 01

Turn features into three scores

Hand-set features isolate the prediction head. This is not a trained image classifier.

import math
features = [1, 2]
weights = [[1, 0], [0, 1], [0, 0]]
scores = [sum(w * h for w, h in zip(row, features)) for row in weights]
exp_scores = [math.exp(s - max(scores)) for s in scores]
probabilities = [v / sum(exp_scores) for v in exp_scores]
print("scores:", scores)
print("probabilities:", [round(p, 4) for p in probabilities])
print(f"loss={-math.log(probabilities[1]):.4f}")

python recognition-transfer-classification-head.py

scores: [1, 2, 0]
probabilities: [0.2447, 0.6652, 0.09]
loss=0.4076

Read the code step by step

Each weight row makes one class score by multiplying and adding. Subtracting the largest score before exp keeps numbers smaller without changing softmax. Dividing by the total produces approximately 0.2447, 0.6652, 0.0900. Index 1 selects the second class. math.log computes its natural log and the minus sign makes loss 0.4076. The code does not train the backbone.

Run a complete example · 01

Measure two proposed boxes

Compute one suppression decision for two boxes of the same class.

a = (0, 0, 4, 4)
b = (1, 1, 5, 5)
width = max(0, min(a[2], b[2]) - max(a[0], b[0]))
height = max(0, min(a[3], b[3]) - max(a[1], b[1]))
intersection = width * height
area_a = (a[2] - a[0]) * (a[3] - a[1])
area_b = (b[2] - b[0]) * (b[3] - b[1])
iou = intersection / (area_a + area_b - intersection)
print(f"intersection={intersection}; IoU={iou:.4f}")
for threshold in [0.3, 0.5]:
    print(threshold, "suppress B" if iou > threshold else "keep B")

python detection-overlap-box-overlap.py

intersection=9; IoU=0.3913
0.3 suppress B
0.5 keep B

Read the code step by step

Indices 0 and 2 are left and right edges; 1 and 3 are top and bottom. min and max locate the shared part. The next lines calculate the two areas, subtract the shared area once, and divide. The for loop tests two thresholds; the inline if chooses the printed message. A is assumed to have the higher score. This example starts with boxes and does not detect objects in pixels.

Run a complete example · 01

Read three ranked detections

The inputs are checked match outcomes, not raw boxes.

matches = [True, False, True]
reference_count = 2
found = 0
ap = 0.0
for rank, correct in enumerate(matches, start=1):
    found += int(correct)
    precision = found / rank
    recall = found / reference_count
    if correct:
        ap += precision / reference_count
    print(f"rank={rank}: precision={precision:.4f}, recall={recall:.4f}")
print(f"toy AP={ap:.4f}")

python detection-average-precision-ranked-ap.py

rank=1: precision=1.0000, recall=0.5000
rank=2: precision=0.5000, recall=0.5000
rank=3: precision=0.6667, recall=1.0000
toy AP=0.8333

Read the code step by step

enumerate supplies a rank and the corresponding outcome. int turns True into 1 and False into 0. found counts cumulative matches. Each true match raises recall by 1/reference_count, so its contribution is precision/reference_count. += adds to the running total. The printed 0.8333 is this stated toy AP, not the full COCO metric.

Run a complete example · 01

Count foreground overlap

Tiny binary lists stand in for flattened masks.

prediction = [1, 1, 1, 0, 0, 0]
reference = [0, 1, 1, 1, 0, 0]
shared = sum(p == 1 and g == 1 for p, g in zip(prediction, reference))
union = sum(p == 1 or g == 1 for p, g in zip(prediction, reference))
iou = shared / union
dice = 2 * shared / (sum(prediction) + sum(reference))
print("shared:", shared, "union:", union)
print(f"IoU={iou:.4f}; Dice={dice:.4f}")

python segmentation-regions-mask-counts.py

shared: 2 union: 4
IoU=0.5000; Dice=0.6667

Read the code step by step

zip visits matching positions. and counts a position only when both masks select it; or counts it when either selects it. Python adds True as 1 and False as 0. There are two shared pixels and four union pixels. Each list selects three pixels, so Dice is 4/6. This evaluates given masks; it does not learn or predict them.

Run a complete example · 02

Compute segmentation IoU without counting ignored pixels

A road camera labels background, road and small objects. Some pixels have no reliable annotation. Mina first counts every pixel and gets a misleading score. She removes ignored pixels, builds one confusion matrix, and reads each class from its row and column. The small object now has its own score instead of disappearing inside the background total.

from math import isclose

def segmentation_report(truth, prediction, classes, ignore=255):
    if len(truth) != len(prediction) or classes <= 0:
        raise ValueError("aligned arrays and positive class count required")
    matrix = [[0] * classes for _ in range(classes)]
    for actual, predicted in zip(truth, prediction):
        if actual == ignore:
            continue
        if not (0 <= actual < classes and 0 <= predicted < classes):
            raise ValueError("unknown class")
        matrix[actual][predicted] += 1
    ious = []
    for c in range(classes):
        tp = matrix[c][c]
        actual_total = sum(matrix[c])
        predicted_total = sum(row[c] for row in matrix)
        union = actual_total + predicted_total - tp
        ious.append(tp / union if union else None)
    defined = [value for value in ious if value is not None]
    mean = sum(defined) / len(defined) if defined else None
    total = sum(map(sum, matrix))
    accuracy = sum(matrix[c][c] for c in range(classes)) / total if total else None
    return matrix, ious, mean, accuracy

truth = [0, 0, 0, 1, 1, 2, 2, 255]
prediction = [0, 0, 1, 1, 2, 2, 0, 1]
matrix, ious, mean, accuracy = segmentation_report(truth, prediction, 3)
assert matrix == [[2, 1, 0], [0, 1, 1], [1, 0, 1]]
assert isclose(mean, 7 / 18) and isclose(accuracy, 4 / 7)
assert segmentation_report([255], [1], 3)[2] is None
assert segmentation_report([0], [0], 3)[1] == [1.0, None, None]
print("rows = truth; columns = prediction:")
for row in matrix:
    print(row)
print("class IoU:", [round(value, 6) for value in ious])
print(f"mean IoU: {mean:.6f}")
print(f"pixel accuracy: {accuracy:.6f}")

python segmentation-regions-confusion-matrix-miou.py

rows = truth; columns = prediction:
[2, 1, 0]
[0, 1, 1]
[1, 0, 1]
class IoU: [0.5, 0.333333, 0.333333]
mean IoU: 0.388889
pixel accuracy: 0.571429

Follow the execution

1. Skip an ignored ground-truth pixel before updating any row or column. Its prediction contributes nothing.

2. Read TP from the diagonal. The row total counts actual pixels; the column total counts predicted pixels.

3. Union equals actual total plus predicted total minus TP. The subtraction avoids counting the intersection twice.

4. Average defined class IoUs equally. Return None for a class absent from both truth and prediction.

Read the code step by step

IoU_c=TP_c/(TP_c+FP_c+FN_c). For background, TP=2, FP=1 and FN=1, so IoU=2/4=0.5. Road and object each have IoU=1/3. Mean IoU is (1/2+1/3+1/3)/3=7/18, about 0.388889. Pixel accuracy is instead 4/7. These metrics answer different questions.

The confusion matrix pools the evaluated pixels. Pooling all images before computing IoU can differ from averaging per-image IoUs. State which method you use. This implementation excludes a class only when its union is zero; a class missing from truth but predicted somewhere has nonzero union and IoU zero. Other benchmarks choose different absent-class conventions.

An all-ignored image gives no defined score. It is not a perfect prediction. For P pixels and C classes, this straightforward implementation takes O(P+C squared) time and O(C squared) storage. Keep the ignore mask, class mapping, pooling rule and resize method identical across models. Never resize discrete label masks with a method that creates fractional class IDs.

Run a complete example · 01

Predict and gate one track

Use a single cart with equally spaced frames.

previous, current = 2, 5
velocity = current - previous
predicted = current + velocity
detections = [8, 15]
nearest = min(detections, key=lambda x: abs(x - predicted))
accepted = abs(nearest - predicted) <= 2
print("predicted:", predicted)
print("nearest:", nearest, "accepted:", accepted)

python tracking-identity-track-gate.py

predicted: 8
nearest: 8 accepted: True

Read the code step by step

Subtraction estimates speed; addition predicts the next centre. min searches the list. Its key is a small lambda function that returns absolute distance from the prediction, so it chooses the smallest distance rather than the smallest coordinate. <= tests the gate. The result accepts 8. This is a toy one-track association, not a trained tracker.

Run a complete example · 01

Count symbol mismatches

Compare invented binary symbol patterns; this is not Tesseract or a trained OCR model.

templates = {"H": [1, 0, 1, 1, 1, 0, 1], "I": [0, 1, 0, 1, 0, 1, 0]}
observed = [1, 0, 1, 1, 1, 0, 0]
distances = {}
for label, template in templates.items():
    distances[label] = sum(a != b for a, b in zip(observed, template))
print("distances:", distances)
print("closest:", min(distances, key=distances.get))

python ocr-reading-ocr-template.py

distances: {'H': 1, 'I': 5}
closest: H

Read the code step by step

The dictionary stores one list per label. The loop pairs observed and template positions with zip; != is True when they differ. sum counts these True values. distances.get supplies each label’s distance to min, which selects H. Equal lengths are assumed here; a real input check must enforce them. No text region detection is implemented.

Run a complete example · 01

Count patches and comparisons

Compare two patch sizes before considering a full model.

height, width, channels = 32, 32, 3
for patch_size in [8, 4]:
    patches = (height // patch_size) * (width // patch_size)
    values = patch_size * patch_size * channels
    tokens = patches + 1
    print(f"P={patch_size}: patches={patches}, values={values}, pairs={tokens ** 2}")

python visual-tokens-patch-budget.py

P=8: patches=16, values=192, pairs=289
P=4: patches=64, values=48, pairs=4225

Read the code step by step

The first line assigns three size variables. The loop repeats the same calculation for patch sizes 8 and 4. // is whole-number division; here both sizes divide evenly. Add one token for the class summary. **2 means multiply the token count by itself. We count 289 and 4225 pairs; we do not implement attention, learned projection or a trained transformer.

Run a complete example · 01

Locate the largest change

A four-position strip isolates the idea of a deviation map.

reference = [10, 10, 10, 10]
observed = [11, 9, 25, 10]
differences = [abs(x - r) for x, r in zip(observed, reference)]
flagged = [i for i, value in enumerate(differences) if value > 5]
print("differences:", differences)
print("flagged positions:", flagged)
print("image score:", max(differences))

python defect-inspection-difference-inspection.py

differences: [1, 1, 15, 0]
flagged positions: [2]
image score: 15

Read the code step by step

zip pairs aligned positions. abs removes the sign of each difference. The second list comprehension uses enumerate to retain position numbers and if to keep only values above 5. max gives score 15. A high difference identifies a candidate for review, not a proven defect. This is an untrained baseline with a hand-set reference.

Run a complete example · 01

Count alerts and misses

Calculate metrics from explicit counts for one held-out group.

true_positive, false_positive, false_negative = 15, 10, 5
precision = true_positive / (true_positive + false_positive)
recall = true_positive / (true_positive + false_negative)
f1 = 2 * true_positive / (2 * true_positive + false_positive + false_negative)
print(f"precision={precision:.2f}; recall={recall:.2f}; F1={f1:.4f}")
sessions = {"train": {"morning-A", "evening-A"}, "test": {"morning-B"}}
print("shared sessions:", len(sessions["train"] & sessions["test"]))

python evaluation-shift-evaluate-alerts.py

precision=0.60; recall=0.75; F1=0.6667
shared sessions: 0

Read the code step by step

The first line assigns the three error counts. Each division uses the population named by its metric. A dictionary maps split names to sets of capture-session names. & finds set intersection; len reports zero shared sessions. This verifies one chosen grouping rule, not every possible source of leakage. There is no model training here.

Run a complete example · 01

Inspect rounding and total delay

Use stated hypothetical times, not measurements.

values = [0.12, 0.26, 0.91]
step = 0.1
encoded = [round(value / step) for value in values]
restored = [number * step for number in encoded]
errors = [abs(a - b) for a, b in zip(values, restored)]
print("encoded:", encoded)
print("restored:", [round(x, 2) for x in restored])
print(f"largest error={max(errors):.2f}")
print("serial latency ms:", sum([4, 7, 2]))

python inference-pipelines-pipeline-budget.py

encoded: [1, 3, 9]
restored: [0.1, 0.3, 0.9]
largest error=0.04
serial latency ms: 13

Read the code step by step

The first comprehension divides and rounds each value. The second reconstructs approximate values by multiplication. zip pairs originals with reconstructions so abs can measure error. Rounding the displayed values to two decimals only affects presentation. sum adds the three assumed stage times. This toy quantizer does not convert or run a trained model.

Run a complete example · 02

Return a detection box to the original image

Mina fits a wide camera image into a square model input. She scales the image and adds padding. The model finds the object, but drawing its coordinates directly on the camera image moves the box. She writes down the actual resized dimensions and padding, reverses those operations, and tests the round trip before trusting the display.

from math import isclose

def metadata(width, height, side):
    if min(width, height, side) <= 0:
        raise ValueError("positive dimensions required")
    nominal = min(side / width, side / height)
    rw, rh = max(1, round(width * nominal)), max(1, round(height * nominal))
    return rw / width, rh / height, (side - rw) // 2, (side - rh) // 2

def forward(box, meta):
    sx, sy, px, py = meta
    x1, y1, x2, y2 = box
    return [sx * x1 + px, sy * y1 + py, sx * x2 + px, sy * y2 + py]

def restore(box, meta, width, height):
    sx, sy, px, py = meta
    x1, y1, x2, y2 = box
    restored = [(x1 - px) / sx, (y1 - py) / sy,
                (x2 - px) / sx, (y2 - py) / sy]
    clipped = [min(max(v, 0.0), limit)
               for v, limit in zip(restored, [width, height, width, height])]
    return clipped if clipped[0] < clipped[2] and clipped[1] < clipped[3] else None

original = [100.0, 50.0, 500.0, 300.0]
meta = metadata(800, 400, 640)
model_box = forward(original, meta)
restored = restore(model_box, meta, 800, 400)
assert all(isclose(a, b) for a, b in zip(original, restored))
odd_meta = metadata(853, 480, 640)
odd_back = restore(forward(original, odd_meta), odd_meta, 853, 480)
assert all(isclose(a, b) for a, b in zip(original, odd_back))
assert odd_meta[0] != odd_meta[1]  # Rounded height changes the actual y scale.
assert restore([0, 0, 20, 20], meta, 800, 400) is None
print("scale and padding:", meta)
print("model box:", model_box)
print("original box:", restored)
print("rounded-size round trip:", all(isclose(a, b) for a, b in zip(original, odd_back)))

python inference-pipelines-letterbox-coordinate-roundtrip.py

scale and padding: (0.8, 0.8, 0, 160)
model box: [80.0, 200.0, 400.0, 400.0]
original box: [100.0, 50.0, 500.0, 300.0]
rounded-size round trip: True

Follow the execution

1. An 800 by 400 image becomes 640 by 320. Add 160 pixels above it and 160 below it.

2. Map both corners. The box [100,50,500,300] becomes [80,200,400,400].

3. Subtract padding first, then divide by scale. Clip to the original continuous image boundaries.

4. Repeat with 853 by 480. Rounded resized dimensions can give slightly different x and y scales. Reject boxes that collapse inside padding.

Read the code step by step

For continuous box edges, x_prime=s_x*x+p_x and y_prime=s_y*y+p_y. Here s_x and s_y are actual resized width/original width and resized height/original height. Padding p_x and p_y records the left and top offsets. Reverse the transform with x=(x_prime-p_x)/s_x and y=(y_prime-p_y)/s_y.

In the first image both scales are 0.8. A model y-coordinate of 200 becomes (200-160)/0.8=50. For 853 by 480, rounding produces 640 by 360. Then s_x=640/853 while s_y=360/480. Using the nominal scale for both axes would introduce a small error. Store the actual transform used by preprocessing.

This example uses continuous box edges in [0,width] and [0,height], not integer pixel indices whose last valid values are width-1 and height-1. Real resize libraries can use different pixel-center and interpolation conventions. Cropping, rotation or other padding rules require their own recorded transforms. A round-trip test checks coordinate bookkeeping; it does not prove the detector is accurate.