The Discriminator
A classifier that learns to tell real from fake
By the end of this lesson you can
- Frame D(x) as the probability that x is real
- Derive binary cross-entropy as the discriminator's objective
- See how D's decision boundary moves as the fake samples move
The generator needs feedback, and the paper gets it from a second network: an ordinary binary classifier that looks at a sample and guesses whether it is real. Nothing about it is exotic. That is the point.
D(x) as a Probability
The paper defines a second multilayer perceptron D(x; θ_d) that outputs a single scalar. D(x) represents the probability that x came from the data rather than from p_g. In practice the last layer is a sigmoid, so the output lies in (0, 1):
- D(x) ≈ 1: "this looks real"
- D(x) ≈ 0: "this looks generated"
- D(x) = 1/2: "I cannot tell"
Labels: Real = 1, Fake = 0
Build a training set by mixing real examples with label y = 1 and generated examples G(z) with label y = 0. The discriminator is trained, in the paper's words, to maximise the probability of assigning the correct label to both training examples and samples from G.
Deriving Binary Cross-Entropy
Treat D(x) as the probability of a coin landing on "real". For a single example with label y, the likelihood of the correct label is a Bernoulli probability:
Take the log and flip the sign to get a loss to minimise:
That is binary cross-entropy (BCE). For a real example (y = 1) only the first term survives, −log D(x). For a fake example (y = 0) only the second survives, −log(1 − D(G(z))). Average over equal numbers of real and fake examples, flip the sign once more, and you get the quantity D maximises:
This is exactly the value function V(D, G) of the next lesson. The paper makes the same observation: the training objective for D can be interpreted as maximising the log-likelihood for estimating the conditional probability P(Y = y | x), where Y indicates whether x comes from p_data (y = 1) or from p_g (y = 0).
A Moving Target
Unlike a normal classifier, D's "class 0" keeps changing, because the fake samples move every time G updates. Picture the one-dimensional case:
- Fakes far from the data: D easily puts a sharp boundary between them. Accuracy near 100%, loss near 0.
- Fakes closer: the two clouds overlap. The boundary still sits between them, but D is less confident. Loss rises.
- Fakes on top of the data: no boundary helps. The best D can do is output 1/2 everywhere. Accuracy 50%, BCE = log 2 ≈ 0.693.
The paper's Figure 1 shows the same sequence: D (the blue dashed curve) is high over the data and low over the fakes, and flattens to 1/2 once p_g matches p_data.
Edit the code and press Ctrl + Enter to run it in your browser.
Let D(x) = σ(s) with s = f(x; θ_d). Show that the gradient of the binary cross-entropy loss with respect to the logit s is D(x) − y.
Real data is N(0, 1) and the generator outputs N(0, 3²): same mean, much wider spread. Can a discriminator of the form D(x) = σ(w·x + c) do better than chance? What about σ(w₁x + w₂x² + c)?
On one real example D outputs 0.9, and on one generated example it outputs 0.2. Compute the BCE loss for each. Then compute it for a discriminator that outputs 0.5 on both.
Key takeaway
The discriminator is an ordinary binary classifier trained with cross-entropy: real samples labelled 1, generated samples labelled 0.
Up next
Put the two networks against each other and you get a game.