Deepfakes: How They Work and How to Spot Them
A deepfake is synthetic media — video, image, or audio — in which AI convincingly replaces or fabricates a person's likeness or voice. What began as a research curiosity and a novelty app is now a security problem: cloned-voice CEO fraud, fabricated evidence, non-consensual imagery, and disinformation at scale. Understanding how they're made explains both why they've become so good and how to defend against them.
How a deepfake is built
The classic pipeline has four stages:
- Data collection. Gather images, video, or audio of the target — historically a lot of it, though modern models need far less.
- Model training. A neural network learns the target's facial structure, expressions, and movements (or, for audio, their vocal timbre and cadence).
- Generation / face swap. The model synthesizes new frames, mapping the target's likeness onto another performance.
- Refinement. Lighting, expression, and lip-sync are tuned to erase the seams.
The engine underneath used to be Generative Adversarial Networks (GANs) — two networks in a contest, a generator making fakes and a discriminator trying to catch them, each improving the other until the output fools detection. Today the state of the art has largely shifted to diffusion models (the same family behind modern image generators), which produce higher fidelity with less source data. The trajectory is the point: quality up, data and skill required down.
Why "spot the glitch" advice is expiring
For years the guidance was to look for artifacts — unnatural blinking, weird teeth, mismatched lighting, blurry face edges, audio that doesn't quite sync. These tells are real but vanishing: each generation of models closes them. Betting your security on catching a rendering glitch is a losing race against the technology. It still helps to look — especially at hands, teeth, reflections, and the boundary where a face meets hair or background — but treat it as a weak signal, not a verdict.
The defenses that actually scale
Because you can't reliably detect a good fake by eye, real defense shifts from the artifact to the context and provenance:
- Verify through a second channel. The most robust defense, and the same one that beats phishing. An urgent video or voice request for money or secrets gets confirmed via a channel you initiate — call back on a known number. A deepfake can imitate a face; it can't answer your callback (see Phishing in the Age of AI).
- Family and corporate code words. A shared secret phrase defeats voice cloning instantly. Free, low-tech, effective.
- Content provenance (C2PA). The emerging industry answer: cryptographically signed "content credentials" that travel with media to attest how it was captured and edited. Verifying origin scales where spotting fakes does not.
- Detection tooling — AI classifiers that flag synthetic media exist and help at platform scale, but they're locked in the same arms race; use them as one input, not proof.
The mindset shift
Deepfakes end the era where "I saw it" or "I heard their voice" was sufficient proof. That's unsettling, but it's the same adjustment society made to Photoshop, only faster and for motion and sound. The practical response isn't paranoia — it's moving trust from the appearance of a message to its provenance and verification. Assume media can be fabricated; trust what you can independently confirm.
And a note on ethics: the technology has legitimate uses — film, accessibility, education, satire — but creating deceptive or non-consensual content is harmful and increasingly illegal. Capability is not permission.
Deepfakes are getting too good to catch by eye, so stop trying to spot the glitch and start verifying the source. A cloned face can't survive a callback, a code word, or a cryptographic provenance check — move your trust there.