Study Guide 5 — Lab 10: Data Augmentation + Full Exam Review
md9 min readSummary of AIDE500
Method: Read → run the notebook (try 2–3 transforms yourself) → Self-check out loud → then redo the exam.
LAB 10 — Data Augmentation (on CIFAR-10)
What is Data Augmentation?
- Definition (EXAM Q35): the process of creating new training examples by modifying existing ones.
- Key idea: introduce variety in a way that doesn't change the meaning of the data — the label stays the same (flipping a dog photo left/right is still a dog).
- Why it helps: models are sensitive to small details (a slight shift in color/shape can throw them off). Augmentation exposes the model to different versions of the same concept → it learns patterns that stay consistent → builds robustness for unseen data.
- The danger / balance (Q35 second half): rotate digit "6" too far → looks like "9"; crop too much → object cut out entirely. Good augmentations increase diversity without breaking the input–label connection. ⚠️ This exact "6→9" example is your model-answer hook for Q35.
- Where it happens: DA is done when defining the Dataset class, via the
transformsargument (usually aComposeof several functions) — i.e. exactly where Lab 8 put transforms. - ⚠️ EXAM Q8 (T/F): "augmentation is valid only if the label changes" → False (meaning must NOT change).
The 4 families (EXAM Q25 — you said Cutout-style methods were geometric; here's the fix)
| Family | What it changes | Examples |
|---|---|---|
| 1. Geometric | spatial layout — position, scale, orientation | HorizontalFlip, VerticalFlip, RandomRotation, RandomCrop, CenterCrop, Affine, Perspective |
| 2. Color & Appearance | lighting / sensor / pixel-level look | ColorJitter (brightness, contrast, saturation, hue), Grayscale, GaussianBlur, Noise, Posterize, Solarize |
| 3. Occlusion & Region-based | hides/removes parts → model must reason about incomplete data | Cutout, RandomErasing, Hide-and-Seek |
| 4. Sample Mixing | blends multiple images + labels (strong regularizers) | Mixup (linear mix), CutMix (patch pasting), FMix (frequency masks) |
⚠️ Q25 correction: RandomCrop = geometric (spatial layout). Cutout/RandomErasing = occlusion — they hide regions, they don't move the object. ColorJitter/Grayscale = color family. Mixup/CutMix = mixing family.
The code patterns used in the notebook
# 1. Geometric — RandomCrop (crops at random location, pads borders)
transforms.Compose([
transforms.RandomCrop((28,28), padding=(5,10), padding_mode='constant'),
transforms.ToTensor(),
])
# 2. Geometric — CenterCrop
transforms.Compose([transforms.CenterCrop((32,32)), transforms.ToTensor()])
# 3. Geometric — flips, with probability p
transforms.Compose([transforms.RandomHorizontalFlip(p=0.5), transforms.ToTensor()])
transforms.Compose([transforms.RandomVerticalFlip(0.5), transforms.ToTensor()])
# 4. Color — Grayscale ⚠️ EXAM Q45's answer
transforms.Compose([transforms.Grayscale(num_output_channels=1), transforms.ToTensor()])
# after this the batch shape is (4, 1, 32, 32) -> channel dim is 1
# 5. Mixing (note: transforms.v2 module)
import torchvision.transforms.v2 as transforms2
transforms.Compose([transforms2.MixUp(), transforms.ToTensor()])
# 6. Combined
transforms.Compose([
transforms.RandomCrop(25, padding=3),
transforms.RandomHorizontalFlip(),
transforms.RandomVerticalFlip(),
transforms.ToTensor(),
])- Every loader carries:
shuffle=False ### Shuffle to False so that we always see the same images→ third confirmation of EXAM Q10 (visualization needs reproducible order). - Rule from the notebook's Note: "you should not change the image size as a result of transformation" — a transform that alters H×W would break
fc1.in_features = 400(ties to Q37!). (RandomCrop uses padding to control this.) - Display trick:
data[0].swapaxes(0,2).swapaxes(0,1)converts CHW → HWC forimshow(same asnp.transpose(x,(1,2,0))from Lab 9). - Popular external package:
albumentations. - Caution (Note 2): don't blindly Mixup unrelated classes (dog + airplane) — mix same/related classes for sensible samples.
Self-check — Lab 10
- Define data augmentation in one sentence; why do labels stay valid? (Q35)
- The 4 families + 2 examples each. Which family is RandomCrop? Cutout? ColorJitter? Mixup? (Q25)
- Why is a "6" rotated too far a bad augmentation? (Q35 model answer)
- Write the Grayscale transform line. What batch shape results? (Q45)
- Why
shuffle=Falsein every visualization loader? (Q10) - Why must augmentation keep the image size (or pad back)? Link to
in_features. (Q37)
Answers: 1) new examples from existing ones; modification mustn't change meaning. 2) table above. 3) changes the label ("6"→"9") — breaks input–label link. 4) transforms.Grayscale(num_output_channels=1); (4,1,32,32). 5) so we always see the same images (stable comparison). 6) fc1 = Linear(16*5*5,120) assumes 32→…→5; a size change makes view/fc mismatch → runtime error.
FULL EXAM REVIEW — all 47 questions mapped to your guides
Legend: ✅ = you got it right in attempt 1 · ❌ = you missed it (study now) · ➖ = unanswered (model answer below)
Section 1 — True/False (1 pt each)
| Q | Fact | Your status | Where to study |
|---|---|---|---|
| 1 | np.empty does NOT guarantee zeros (garbage memory) |
✅ | G1 |
| 2 | subplot index moves left→right, top→bottom | ✅ | G1 |
| 3 | backward() computes grads; step() updates values → statement "backward changes params" = False |
✅ | G2 |
| 4 | torch.from_numpy shares memory → True |
✅ | G2 |
| 5 | Net outputs are logits ("energies"), no Softmax in forward → False | ❌ | G4 §loss/testing |
| 6 | ToTensor scales [0,255]→[0,1] → True |
✅ | G4 |
| 7 | linspace includes endpoint, arange excludes → True |
✅ | G1 |
| 8 | Good augmentation must NOT change meaning → True | ✅ | G5 above |
| 9 | You only define forward(); backward is automatic → False |
❌ | G3 §nn.Module |
| 10 | Visualization loaders use shuffle=False → False (statement says True) | ❌ | G4/G5 |
Section 2 — Multiple Choice (2 pts each)
| Q | Answer | Key fact | Status | Guide |
|---|---|---|---|---|
| 11 | B | 4th panel of 2×2 = bottom-right | ✅ | G1 |
| 12 | A | jagged plot → sample more points (step 0.01) | ✅ | G1 |
| 13 | C | linspace's 3rd arg = number of values, includes endpoint | ✅ | G1 |
| 14 | D | axis=1 reduces across rows: min(7,2)=2, min(3,4)=3, min(5,6)=5 → [2 3 5]; axis=0 → [3 2] |
❌ (you said axis=0) | G1 §axis |
| 15 | A | broadcasting: (2,)→(3,2), element-wise → [[8,3],[18,30],[24,39]] |
✅ | G1 |
| 16 | B | cat(dim=1): (2,3)+(2,3)→(2,6) |
✅ | G2 |
| 17 | B | torch.tensor() = deep copy; from_numpy() = shared |
❌ (you said tensor shares) | G2 §cheat-sheet |
| 18 | A | backward() stores grads in .grad of leaves, values unchanged |
✅ | G2 |
| 19 | B | requires_grad=False → no grad → w frozen, b updates |
✅ | G2 §freeze |
| 20 | C | ReLU = max(0,x) | ✅ | G3 |
| 21 | C | (32+0−5)/1+1 = 28 | ✅ | G4 §trace |
| 22 | A | batch = (64, 1, 28, 28), labels (64,) | ✅ | G4 |
| 23 | C | (x−0.5)/0.5 → [−1,1] | ❌ (you said [0,1]) | G4 §normalize |
| 24 | B | torch.max(outputs,1) returns argmax class indices (no probs — no Softmax anywhere) |
❌ (you said probabilities) | G4 §testing |
| 25 | D | RandomCrop = geometric; Cutout = occlusion family | ❌ (you said occlusion is geometric) | G5 table above |
Section 3 — Explain the Code (6 pts each) — rubrics from the key
- Q26 Broadcasting: (1) shape =
(3,2); (2) result[[8,3],[18,30],[24,39]]; (3) broadcasting expands[2,3]to 3 rows; (4) element-wise, not matmul. → You got (a)+(b): practice in G1 until you can write the 4 bullets cold. - Q27 Autograd: (a)
zhasgrad_fn(records ops), leaves haveNone; (b)backward()computes dL/dw →w.gradshape (5,3); (c) No — values unchanged; (d)step()appliesw ← w − lr·w.grad. → You had (b)/(d): memorize the (a)–(d) structure (G2). - Q28 Training loop: missing
optimizer.zero_grad(); without it gradients accumulate across batches → wrong oversized updates; correct order:zero_grad → forward → loss → backward → step. ❌ blank → G4 §loop. - Q29 CNN trace (batch 16):
[16,6,28,28] → [16,6,14,14] → [16,16,10,10] → [16,16,5,5] → [16,400] → [16,10]; 400 = 16·5·5 = fc1.in_features. → You had (a)/(c): drill the trace in G4 until automatic. - Q30 Transforms: (a) ToTensor: [0,255]→[0,1]; (b) Normalize: (x−0.5)/0.5 → [−1,1]; (c) Normalize doesn't accept PIL → ToTensor first; (d)
train=True= training split. → You missed (c)/(d): G4 §transforms. - Q31 NumPy bridge: (a)
t=[3,3,3,3,3],n=[3,3,3,3,3]; (b).numpy()is a view (shares memory); (c)t2 = tensor([2,2,2,2,2], dtype=torch.float64)(from_numpy keeps float64, shares memory); (d) trailing_= in-place op. → Your "deep copy" contradiction: G2 §bridge.
Section 4 — Open Questions (5 pts each) — YOU ANSWERED NONE. Model answers (study these — they're 30 of your 66 missing points):
- Q32 (Lab 2, plot smoothness): matplotlib connects points with straight-line segments → few points = coarse polygon. Fix: sample many closely spaced x-values, e.g.
X = [0.01*i for i in range(0,201)]over [0,2],Y = [x**2 for x in X]→ segments visually approximate a smooth curve. - Q33 (backward vs step):
backward()= reverse traversal of the graph, computes gradients into.gradofrequires_grad=Trueleaves, values unchanged;step()= appliesθ ← θ − lr·grad(actual learning). Only backward → gradients computed but never applied, no learning; only step → gradsNone/stale → no correct update. - Q34 (train/eval + no_grad):
model.train()while training (Dropout/BatchNorm training behaviour);model.eval()before testing (Dropout off, BatchNorm uses running stats);torch.no_grad()during inference → disables graph building → saves memory & compute. Forgetting it → wasted memory/slower eval and wrong layer behaviour (Dropout still dropping, BatchNorm updating stats) → unreliable metrics. - Q35 (augmentation): definition = creating new examples by modifying existing ones (Compose in the Dataset's
transform); why = variety without changing meaning, models are detail-sensitive, teaches focus on consistent patterns → robustness; good = diversity while preserving input–label link; harmful = rotating "6" too far → "9", or cropping the object out entirely. (Everything in G5's top section.) - Q36 (full pipeline, Lab 9): ① Data:
Compose([ToTensor(), Normalize((0.5,0.5,0.5),(0.5,0.5,0.5))])on CIFAR10; loaders batch 16 train / 4 test,num_workers=2. ② Net:Conv2d(3,6,5)→ReLU→pool→Conv2d(6,16,5)→ReLU→pool→view(-1,16*5*5)→Linear 400→120→84→10, ReLU on hidden only, no final activation. ③ Loss/opt:CrossEntropyLoss+SGD(lr=0.01, momentum=0.9); loop:zero_grad→forward→loss→backward→step, accumulateloss.item(), print every 2000 batches. ④ Save/load:torch.save(net.state_dict(), PATH); re-instantiateNet()thenload_state_dict. ⑤ Eval:with torch.no_grad(),torch.max(outputs,1)argmax,correct += (predicted==labels).sum().item(), accuracy = 55% vs 10% chance. - Q37 (conv size + in_features): formula
h_out = (h_in + 2p − D(k−1) − 1)/s + 1(D=1 →(h_in + 2p − k)/s + 1); pooling with k=2,s=2 halves. Trace: 32 →conv k=5→ 28 →pool→ 14 →conv k=5→ 10 →pool→ 5, channels 16. Flattened = 16·5·5 = 400 = fc1.in_features; a transform that changes image size breaks this fixed number.
Section 5 — Fill in the Blank (1 pt each) — your misses bolded
| Q | Blank(s) | Your status |
|---|---|---|
| 38 | Shift+Enter (run a cell) |
❌ you wrote "run all" |
| 39 | plt.legend() |
✅ |
| 40 | .shape attribute |
✅ |
| 41 | autograd (torch.autograd = PyTorch's automatic differentiation engine) |
❌ blank |
| 42 | epoch (one full pass of whole data through the model) |
❌ blank |
| 43 | (x − mean) / std |
✅? partial |
| 44 | nn.**CrossEntropyLoss**(), lr=**0.01** |
✅? |
| 45 | transforms.**Grayscale**(num_output_channels=1) |
✅? |
| 46 | squeeze (removes (1,28,28)→(28,28)) |
❌ you wrote slice[1:] |
| 47 | zero_grad → backward → step (3 blanks) |
❌ only got step |
Go to 00_STUDY_PLAN.md for the study schedule and how to resubmit.