Deep Learning with PyTorch
Deep learning earns its keep where features can't be hand-built: images, audio, text, and sequences whose structure lives in raw signal. On a spreadsheet, Part 6's boosted trees usually win. Know that boundary and you'll never reach for the flamethrower to light a candle. This part builds the mental model — tensors, autograd, the training loop — that every deep system, up to and including LLMs, is made of.
Three ideas, honestly stated
Tensors
N-dimensional arrays that live on CPU or GPU and remember how they were computed. If you know NumPy, you know 80% of the API; Apple Silicon runs them on the GPU via the mps device, CUDA elsewhere.
Autograd
A network is just a function with millions of knobs. Autograd records every operation in the forward pass and then computes, automatically, how much each knob contributed to the error. That gradient is the training signal; nothing about it is magic — it's the chain rule with excellent bookkeeping.
The training loop
import torch
from torch import nn
model = nn.Sequential(
nn.Linear(64, 128), nn.ReLU(),
nn.Linear(128, 2))
opt = torch.optim.AdamW(model.parameters(), lr=1e-3)
loss_fn = nn.CrossEntropyLoss()
for xb, yb in train_loader: # batches
opt.zero_grad()
loss = loss_fn(model(xb), yb) # forward
loss.backward() # gradients via autograd
opt.step() # nudge the knobsEvery architecture you will ever meet — CNN, transformer, diffusion model — is this loop with a different model and enough data. Own the loop, and no paper will scare you again.
The debugging catechism
Deep learning fails silently; discipline compensates:
- Overfit one batch first. A healthy model can memorize 32 examples to near-zero loss in seconds. If it can't, the bug is in your pipeline, not your hyperparameters.
- Watch train and validation loss together. Diverging curves mean overfitting (get more data, augment, or regularize); neither moving means the learning rate or the data plumbing.
- Loss ≈ ln(num_classes) at start for classification. Wildly off? The labels or the last layer are wrong.
- Fix seeds, log everything. Runs you can't reproduce are anecdotes.
Rule one of practice: don't train from scratch
Somebody with more GPUs already learned generic vision and language for you. Fine-tuning a pretrained model — freeze the backbone, replace the head, train briefly at low learning rate — beats scratch training on almost any realistic dataset size. Hubs like Hugging Face and torchvision make this a dozen lines. Training from zero is a research activity and a budget line, not a default.
When to say no
Deep learning costs you interpretability, training stability, and compute. Decline it when the data is tabular and small, when the baseline from Part 6 is within noise of the fancy model, or when the deployment target is a Raspberry Pi and the model is a memory hog. "We used a neural network" is an engineering decision, not a press release.
Checkpoint
Fine-tune a small pretrained image model on a dataset you assembled yourself (two classes, a few hundred photos). Hit the wall, diagnose with the catechism above, and write down the train/val curves and what they told you. The scars are the curriculum.