Contents / Information Theory / Quantum States and Von Neumann Entropy

Chapter 16

Quantum States and Von Neumann Entropy

Qubits and no-cloning, density matrices, composite systems and the partial trace, measurement and POVMs, and von Neumann entropy with the inequalities it does and does not share with Shannon's.

Introduction

Qubits, density matrices and partial traces; what a measurement is; and the von Neumann entropy, whose properties agree with Shannon's in every respect but one.

16.1Qubits, Superposition and No-Cloning

Classical information theory starts from a random variable on a finite alphabet. Quantum information theory starts from a vector in a finite-dimensional complex Hilbert space, and almost every difference between the two theories can be traced to that one substitution.

Notation. A vector is written $∣ ψ ⟩$ (a ket); its conjugate transpose is the row vector $⟨ ψ ∣$ (a bra), so that $⟨ ϕ ∣ ψ ⟩$ is the inner product — conjugate-linear in the first slot — and $∣ ψ ⟩ ⟨ ϕ ∣$ is an operator of rank one. Throughout this chapter every Hilbert space is finite-dimensional, so "operator" means "matrix" and every Hermitian operator has an orthonormal eigenbasis. Logarithms are base $2$ unless stated otherwise, and $I$ is the identity.

Definition 16.1 (Qubit). A qubit is a unit vector in a two-dimensional complex Hilbert space with a fixed orthonormal computational basis ${∣0 ⟩, ∣1 ⟩}$ :

∣ ψ ⟩ = α ∣0 ⟩ + β ∣1 ⟩, α, β \in C, ∣ α ∣^{2} + ∣ β ∣^{2} = 1.

Two unit vectors differing by a global phase, $∣ ψ ⟩$ and $e^{iγ} ∣ ψ ⟩$ , describe the same physical state.

The amplitudes $α, β$ are not probabilities. They are complex numbers whose squared moduli are probabilities, and the phase they carry is what makes quantum information behave unlike a biased coin.

Definition 16.2 (Born rule, computational basis). Measuring $∣ ψ ⟩ = α ∣0 ⟩ + β ∣1 ⟩$ in the basis ${∣0 ⟩, ∣1 ⟩}$ returns the outcome $0$ with probability $∣ α ∣^{2}$ and $1$ with probability $∣ β ∣^{2}$ , and leaves the system in the corresponding basis vector. The general form of this rule is the subject of the section Measurement, POVMs and What Can Be Distinguished.

Global phase is invisible because $∣ e^{iγ} α ∣^{2} = ∣ α ∣^{2}$ in every basis. Relative phase is not: the states

∣ + ⟩ = \frac{1}{2} (∣0 ⟩ + ∣1 ⟩), ∣ - ⟩ = \frac{1}{2} (∣0 ⟩ - ∣1 ⟩)

give identical statistics in the computational basis — one bit of noise each — yet they are orthogonal, $⟨ + ∣ - ⟩ = 0$ , and so are told apart perfectly by a measurement in the basis ${∣ + ⟩, ∣ - ⟩}$ . A classical bit has no analogue of this: there is no pair of coins that look identical under one reading and are perfectly distinguishable under another.

Intuition. Think of the two amplitudes as an arrow in a plane rather than a pair of odds. Rotating the arrow can leave both shadows — the measurement probabilities in the computational basis — unchanged while pointing somewhere completely different. Information is stored in the direction of the arrow; a choice of basis decides which shadow you get to see.

Discarding the invisible global phase leaves exactly two real parameters, and they have a standard geometric name.

Definition 16.3 (Bloch representation of a pure qubit). Every pure qubit state can be written, up to global phase, as

∣ ψ ⟩ = cos \frac{θ}{2} ∣0 ⟩ + e^{i φ} sin \frac{θ}{2} ∣1 ⟩, θ \in [0, π], φ \in [0, 2 π),

so that pure states correspond to the points $(sin θ cos φ, sin θ sin φ, cos θ)$ on the unit sphere in $R^{3}$ — the Bloch sphere.

The poles are $∣0 ⟩$ and $∣1 ⟩$ ; the equator carries the equal-weight superpositions, with $∣ \pm ⟩$ at $φ = 0, π$ . Orthogonal states sit at antipodal points, not at right angles: the Hilbert-space angle is doubled by the map. That factor of two is worth remembering, because it says that a qubit's two perfectly distinguishable states are as far apart as the geometry allows.

Notation. The Pauli matrices

X = (0110), Y = (0 i - i 0), Z = (10 0 - 1)

together with $I$ form a basis of the real vector space of $2 \times 2$ Hermitian matrices. Each is Hermitian and unitary, squares to $I$ , is traceless, and anticommutes with the other two.

Example 16.4 (Reading a state off the sphere). Where does $∣ ψ ⟩ = \frac{1}{2} ∣0 ⟩ + \frac{3}{2} i ∣1 ⟩$ sit on the Bloch sphere, and what is the probability of the outcome $1$ ?

Solution. Match the standard form: $cos (θ /2) = 1/2$ gives $θ /2 = π /3$ , so $θ = 2 π /3$ . The phase of the second amplitude is $i = e^{iπ /2}$ , so $φ = π /2$ . The Bloch vector is therefore

(sin \frac{2 π}{3} cos \frac{π}{2}, sin \frac{2 π}{3} sin \frac{π}{2}, cos \frac{2 π}{3}) = (0, \frac{3}{2}, - \frac{1}{2}),

a point in the lower half of the sphere on the $y$ -axis side. By the Born rule $Pr (1) = ∣ 3 i /2 ∣^{2} = 3/4$ . □□

Now the first genuinely quantum limitation, and the one that reshapes every coding theorem in the two chapters that follow: quantum information cannot be duplicated.

Theorem 16.5 (No-cloning). Let $dim H \geq 2$ . There is no unitary $U$ on $H \otimes H$ and no fixed state $∣ b ⟩$ with

U (∣ ψ ⟩ \otimes ∣ b ⟩) = ∣ ψ ⟩ \otimes ∣ ψ ⟩ for every unit vector ∣ ψ ⟩ \in H .

Proof. Suppose such a $U$ existed and let $∣ ψ ⟩, ∣ ϕ ⟩$ be any two unit vectors. Unitaries preserve inner products, so pairing $U (∣ ψ ⟩ ∣ b ⟩)$ with $U (∣ ϕ ⟩ ∣ b ⟩)$ gives

⟨ ψ ∣ ϕ ⟩ ⟨ b ∣ b ⟩ = (⟨ ψ ∣ ϕ ⟩)^{2}, that is z = z^{2} with z = ⟨ ψ ∣ ϕ ⟩ .

Hence $z \in {0, 1}$ : any two states in the domain are either orthogonal or equal. In dimension at least two we may take $∣ ψ ⟩ = ∣0 ⟩$ and $∣ ϕ ⟩ = ∣ + ⟩$ , for which $z = 1/ 2$ , a contradiction.∎

The proof says exactly where the obstruction lies. A cloner for a known set of mutually orthogonal states is easy — measure, then re-prepare — and a cloner for one known state is trivial. What is impossible is a device that works on an unknown state drawn from a set containing two non-orthogonal members.

Corollary 16.6 (No perfect deterministic discrimination). Two states with $0 < ∣ ⟨ ψ ∣ ϕ ⟩ ∣ < 1$ cannot be distinguished with certainty by any measurement.

Proof. If some measurement identified them with certainty, a single run would produce the classical label, from which arbitrarily many copies could be prepared, contradicting Theorem No-cloning. A quantitative version, giving the optimal error probability, appears below as Theorem Helstrom bound, two pure states.∎

Pitfall. No-cloning does not say that quantum information cannot be moved, encoded or protected. Teleportation moves an unknown qubit perfectly, and quantum error-correcting codes spread one qubit across five without copying it. What is forbidden is producing two independent systems each carrying the original state — which is exactly what a classical repeater does, and exactly why quantum repeaters are hard.

Exercises for §16.1 →

16.2Density Matrices and Mixed States

A unit vector describes a system about which we have maximal information. That is rarely the situation: a source may emit $∣ ψ_{1} ⟩$ with probability $p_{1}$ and $∣ ψ_{2} ⟩$ with probability $p_{2}$ , or the system we hold may be half of a larger one. Both cases are handled by one object.

Definition 16.7 (Density operator). A density operator on $H$ is an operator $ρ$ that is

Hermitian, $ρ = ρ^{†}$ ;
positive semidefinite, $⟨ ϕ ∣ ρ ∣ ϕ ⟩ \geq 0$ for all $∣ ϕ ⟩$ ; and
of unit trace, $Tr ρ = 1$ .

It is pure if $ρ = ∣ ψ ⟩ ⟨ ψ ∣$ for some unit vector, and mixed otherwise. The density operators on a fixed $H$ form a convex set whose extreme points are exactly the pure states.

An ensemble ${p_{i}, ∣ ψ_{i} ⟩}$ gives rise to the density operator $ρ = \sum_{i} p_{i} ∣ ψ_{i} ⟩ ⟨ ψ_{i} ∣$ , and the Born rule extends to it by linearity: the probability of an outcome with projector $P$ is $Tr (Pρ)$ . Conditions 1–3 are precisely what is needed for that formula to return a probability distribution for every measurement.

Theorem 16.8 (Purity criterion). For a density operator $ρ$ , $Tr (ρ^{2}) \leq 1$ , with equality if and only if $ρ$ is pure.

Proof. Diagonalise $ρ = \sum_{i} λ_{i} ∣ i ⟩ ⟨ i ∣$ ; positivity and unit trace make the spectrum a probability distribution, $λ_{i} \geq 0$ and $\sum_{i} λ_{i} = 1$ . Then $Tr (ρ^{2}) = \sum_{i} λ_{i}^{2} \leq \sum_{i} λ_{i} = 1$ , because $λ_{i}^{2} \leq λ_{i}$ on $[0, 1]$ . Equality needs $λ_{i}^{2} = λ_{i}$ for every $i$ , so every eigenvalue is $0$ or $1$ ; with unit trace exactly one equals $1$ and $ρ = ∣ i ⟩ ⟨ i ∣$ is a rank-one projector.∎

The number $Tr (ρ^{2})$ is the purity. On a $d$ -dimensional system it runs from $1/ d$ , attained only by the maximally mixed state $I / d$ , up to $1$ for pure states.

Theorem 16.9 (Bloch ball). Every qubit density matrix can be written uniquely as

ρ = \frac{1}{2} (I + r_{x} X + r_{y} Y + r_{z} Z) = \frac{1}{2} (I + r \cdot σ), r \in R^{3},

and this is a density operator exactly when $∥ r ∥ \leq 1$ . Its eigenvalues are $\frac{1 \pm ∥ r ∥}{2}$ , its purity is $\frac{1 + ∥ r ∥ ^{2}}{2}$ , and it is pure exactly when $∥ r ∥ = 1$ .

Proof. ${I, X, Y, Z}$ is a basis of the Hermitian $2 \times 2$ matrices, orthogonal under $⟨ A, B ⟩ = Tr (A^{†} B)$ with each $Tr (σ_{μ}^{2}) = 2$ , so $ρ = \frac{1}{2} \sum_{μ} c_{μ} σ_{μ}$ with $c_{μ} = Tr (σ_{μ} ρ)$ real and unique. Unit trace forces $c_{0} = 1$ , since the Paulis are traceless. For the spectrum, the anticommutation relations give $(r \cdot σ)^{2} = ∥ r ∥^{2} I$ , so $r \cdot σ$ has eigenvalues $\pm ∥ r ∥$ and $ρ$ has eigenvalues $(1 \pm ∥ r ∥) /2$ . These are non-negative exactly when $∥ r ∥ \leq 1$ , and then

Tr (ρ^{2}) = (\frac{1 + ∥ r ∥}{2})^{2} + (\frac{1 - ∥ r ∥}{2})^{2} = \frac{1 + ∥ r ∥ ^{2}}{2} .

∎

Figure 16.10. A cross-section of the Bloch ball in the

r_{x}

–

r_{z}

plane. Pure states lie on the boundary circle, mixed states strictly inside, and the maximally mixed state

I /2

at the centre.

∣0 ⟩

and

∣1 ⟩

are antipodal because they are orthogonal, and

ρ = 0.75∣0 ⟩ ⟨ 0∣ + 0.25∣1 ⟩ ⟨ 1∣

lies on the segment joining them at Bloch radius

0.5

A mixed state is not a state in which the system "really is" one of the $∣ ψ_{i} ⟩$ while we merely fail to know which. The reason is that $ρ$ does not remember the ensemble that produced it.

Theorem 16.11 (Ensembles with the same density matrix). The ensembles ${\frac{1}{2}, ∣0 ⟩; \frac{1}{2}, ∣1 ⟩}$ and ${\frac{1}{2}, ∣ + ⟩; \frac{1}{2}, ∣ - ⟩}$ have the same density operator $I /2$ , and are therefore indistinguishable by any measurement whatsoever.

Proof. Directly, $\frac{1}{2} ∣0 ⟩ ⟨ 0∣ + \frac{1}{2} ∣1 ⟩ ⟨ 1∣ = \frac{1}{2} I$ . Since $∣ \pm ⟩ ⟨ \pm ∣ = \frac{1}{2} (I \pm X)$ , the second ensemble gives $\frac{1}{2} \cdot \frac{1}{2} (I + X) + \frac{1}{2} \cdot \frac{1}{2} (I - X) = \frac{1}{2} I$ as well. Every measurement statistic has the form $Tr (Eρ)$ , so it depends on the ensemble only through $ρ$ .∎

Intuition. Two different factories — one flipping a fair coin between $∣0 ⟩$ and $∣1 ⟩$ , the other between $∣ + ⟩$ and $∣ - ⟩$ — ship output that no experiment can tell apart. The density matrix is the complete inventory of what an experimenter can ever learn about a system. That is why every quantum information measure is a function of $ρ$ and never of the ensemble behind it.

Example 16.12 (Purity of a partially mixed qubit). Compute the Bloch vector, eigenvalues and purity of $ρ = 0.75 ∣0 ⟩ ⟨ 0∣ + 0.25 ∣1 ⟩ ⟨ 1∣$ .

Solution. In matrix form $ρ = (0.75 0 0 0.25)$ , already diagonal, so the eigenvalues are $0.75$ and $0.25$ . Reading off the Bloch decomposition, $r_{z} = Tr (Zρ) = 0.75 - 0.25 = 0.5$ and $r_{x} = r_{y} = 0$ , so $r = (0, 0, 0.5)$ with $∥ r ∥ = 0.5$ — halfway out of the ball, exactly as Theorem Bloch ball predicts from the eigenvalues $(1 \pm 0.5) /2$ . The purity is $0.7 5^{2} + 0.2 5^{2} = 0.625$ , matching $(1 + 0.25) /2 = 0.625$ . □□

Pitfall. A superposition is not a mixture. The pure state $∣ + ⟩$ and the mixture $\frac{1}{2} ∣0 ⟩ ⟨ 0∣ + \frac{1}{2} ∣1 ⟩ ⟨ 1∣$ both give $50/50$ statistics in the computational basis, but $∣ + ⟩$ has purity $1$ and is certain in the $X$ basis, while the mixture has purity $\frac{1}{2}$ and is $50/50$ in every basis. The off-diagonal entries of $ρ$ — the coherences — are what separate them.

Exercises for §16.2 →

16.3Composite Systems and the Partial Trace

Two classical variables are described by a joint distribution on the product alphabet; two quantum systems are described by a vector in the tensor product of their spaces. Dimension grows multiplicatively in both theories, but only in the quantum one does the joint state fail to be determined by the parts.

Definition 16.13 (Composite system, product and separable states). The state space of a system made of parts $A$ and $B$ is $H_{A} \otimes H_{B}$ , of dimension $d_{A} d_{B}$ . A state is a product state if it factorises as $∣ ψ ⟩_{A} \otimes ∣ ϕ ⟩_{B}$ (or $ρ_{A} \otimes ρ_{B}$ ), separable if it is a mixture $\sum_{i} p_{i} σ_{i}^{A} \otimes τ_{i}^{B}$ of product states, and entangled otherwise.

Definition 16.14 (Partial trace). The partial trace over $B$ is the linear map $Tr_{B}$ from operators on $H_{A} \otimes H_{B}$ to operators on $H_{A}$ determined by

Tr_{B} (M \otimes N) = M Tr (N) .

In components, for any orthonormal basis ${∣ j ⟩_{B}}$ ,

ρ_{A} = Tr_{B} (ρ_{A B}) = j \sum (I_{A} \otimes ⟨ j ∣_{B}) ρ_{A B} (I_{A} \otimes ∣ j ⟩_{B}) .

$ρ_{A}$ is the reduced state, or marginal, of $A$ .

The partial trace earns its place by being the only map with the right operational meaning: it is the state reproducing all statistics of measurements performed on $A$ alone.

Theorem 16.15 (The partial trace is the right marginal). For every operator $M$ on $H_{A}$ ,

Tr [(M \otimes I_{B}) ρ_{A B}] = Tr [M Tr_{B} (ρ_{A B})],

and $Tr_{B}$ is the unique linear map with this property.

Proof. Both sides are linear in $ρ_{A B}$ , so it is enough to check them on $ρ_{A B} = P \otimes Q$ : the left side is $Tr (MP) Tr (Q)$ and the right side is $Tr (MP Tr (Q))$ , the same number. For uniqueness, suppose a linear $Λ$ had the property too. Then $Tr [M (Λ (ρ) - Tr_{B} ρ)] = 0$ for every $M$ ; letting $M$ run over a basis of operators on $H_{A}$ forces $Λ (ρ) = Tr_{B} (ρ)$ , because the trace form $⟨ M, N ⟩ = Tr (M^{†} N)$ is non-degenerate.∎

The classical analogue is summation, $p_{A} (a) = \sum_{b} p (a, b)$ , and it behaves the same way — except in what it does to uncertainty. Classically the marginal of a point mass is a point mass. Quantum mechanically the marginal of a pure state need not be pure at all, and that single fact is entanglement.

Theorem 16.16 (Schmidt decomposition). Let $∣ ψ ⟩ \in H_{A} \otimes H_{B}$ be a unit vector. Then there are orthonormal families ${∣ a_{i} ⟩}$ in $H_{A}$ and ${∣ b_{i} ⟩}$ in $H_{B}$ and reals $λ_{i} > 0$ with $\sum_{i} λ_{i} = 1$ such that

∣ ψ ⟩ = i = 1 \sum k λ_{i} ∣ a_{i} ⟩ \otimes ∣ b_{i} ⟩ .

The number $k \leq min (d_{A}, d_{B})$ is the Schmidt rank and the $λ_{i}$ the Schmidt coefficients. Moreover $ρ_{A} = \sum_{i} λ_{i} ∣ a_{i} ⟩ ⟨ a_{i} ∣$ and $ρ_{B} = \sum_{i} λ_{i} ∣ b_{i} ⟩ ⟨ b_{i} ∣$ : the two marginals share their non-zero spectrum.

Proof. Write $∣ ψ ⟩ = \sum_{m, n} C_{mn} ∣ m ⟩ ∣ n ⟩$ in fixed bases, so $C$ is a $d_{A} \times d_{B}$ matrix with $Tr (C C^{†}) = 1$ . The singular value decomposition gives $C = U D V$ with $U, V$ unitary and $D$ diagonal with entries $s_{i} \geq 0$ . Substituting and setting $∣ a_{i} ⟩ = \sum_{m} U_{mi} ∣ m ⟩$ and $∣ b_{i} ⟩ = \sum_{n} V_{in} ∣ n ⟩$ — orthonormal because $U$ and $V$ are unitary — gives $∣ ψ ⟩ = \sum_{i} s_{i} ∣ a_{i} ⟩ ∣ b_{i} ⟩$ . Normalisation forces $\sum_{i} s_{i}^{2} = 1$ , so put $λ_{i} = s_{i}^{2}$ . Finally

ρ_{A} = Tr_{B} ∣ ψ ⟩ ⟨ ψ ∣ = i, j \sum s_{i} s_{j} ∣ a_{i} ⟩ ⟨ a_{j} ∣ ⟨ b_{j} ∣ b_{i} ⟩ = i \sum s_{i}^{2} ∣ a_{i} ⟩ ⟨ a_{i} ∣,

and symmetrically for $ρ_{B}$ .∎

Corollary 16.17 (Product states are exactly the Schmidt-rank-one states). A pure $∣ ψ ⟩_{A B}$ is a product state if and only if its Schmidt rank is $1$ , if and only if $ρ_{A}$ is pure.

Proof. Schmidt rank $1$ means $∣ ψ ⟩ = ∣ a_{1} ⟩ ∣ b_{1} ⟩$ , a product; conversely a product state has rank $1$ . By Theorem Schmidt decomposition the spectrum of $ρ_{A}$ is $(λ_{1}, \dots, λ_{k})$ , which is a point mass exactly when $k = 1$ , i.e. exactly when $ρ_{A}$ is pure.∎

Example 16.18 (The Bell state and its marginals). Compute $ρ_{A}$ for $∣ Φ^{+} ⟩ = \frac{1}{2} (∣00 ⟩ + ∣11 ⟩)$ and give its Schmidt rank.

Solution. The state is already in Schmidt form with $∣ a_{i} ⟩ = ∣ b_{i} ⟩ = ∣ i ⟩$ and $λ_{1} = λ_{2} = \frac{1}{2}$ , so the Schmidt rank is $2$ and

ρ_{A} = \frac{1}{2} ∣0 ⟩ ⟨ 0∣ + \frac{1}{2} ∣1 ⟩ ⟨ 1∣ = \frac{I}{2} .

By hand it comes out the same: $ρ_{A B} = \frac{1}{2} (∣00 ⟩ ⟨ 00∣ + ∣00 ⟩ ⟨ 11∣ + ∣11 ⟩ ⟨ 00∣ + ∣11 ⟩ ⟨ 11∣)$ , and tracing out $B$ kills the two cross terms because $⟨ 0∣1 ⟩ = 0$ . The joint state is pure and maximally informative; each half is maximally uninformative. □□

The converse of that example is a construction we shall use repeatedly.

Theorem 16.19 (Purification). For every density operator $ρ_{A}$ there is a space $H_{R}$ of dimension $rank ρ_{A}$ and a pure state $∣ ψ ⟩_{A R}$ with $Tr_{R} ∣ ψ ⟩ ⟨ ψ ∣ = ρ_{A}$ . Any two purifications of $ρ_{A}$ differ by a unitary acting on $R$ alone.

Proof. Diagonalise $ρ_{A} = \sum_{i} λ_{i} ∣ a_{i} ⟩ ⟨ a_{i} ∣$ over the non-zero eigenvalues, take $H_{R}$ with an orthonormal basis ${∣ r_{i} ⟩}$ of the same size, and set $∣ ψ ⟩_{A R} = \sum_{i} λ_{i} ∣ a_{i} ⟩ ∣ r_{i} ⟩$ . The computation in the proof of Theorem Schmidt decomposition gives $Tr_{R} ∣ ψ ⟩ ⟨ ψ ∣ = ρ_{A}$ . For uniqueness, any purification is in Schmidt form with the same coefficients $λ_{i}$ and the same $∣ a_{i} ⟩$ — these are fixed as the spectral data of $ρ_{A}$ — so two purifications differ only in the orthonormal family used on $R$ , and a unitary carries one orthonormal family to another.∎

Intuition. Every mixed state is a pure state with part of it hidden. Randomness in quantum theory can always be recast as entanglement with something you are not looking at: the environment, the other half of a pair, a machine that measured and will not tell you. Purification turns that slogan into a theorem, and it converts many statements about mixed states into statements about pure states on a larger space, where the Schmidt decomposition applies.

Pitfall. The partial trace is irreversible, and must not be read as "the state of $A$ given whatever $B$ turned out to be". There is no conditional state until a measurement on $B$ actually happens — and no measurement on $B$ changes $ρ_{A}$ at all, which is exactly what forbids signalling with entanglement.

Exercises for §16.3 →

16.4Measurement, POVMs and What Can Be Distinguished

Everything an experimenter learns arrives through a measurement, so the limits on measurement are the limits on quantum information. The general rule has two layers: what the outcome probabilities are, and what state is left behind.

Definition 16.20 (Projective measurement). A projective measurement is a family of orthogonal projectors ${P_{m}}$ with $P_{m} P_{m^{'}} = δ_{m m^{'}} P_{m}$ and $\sum_{m} P_{m} = I$ . Applied to $ρ$ it returns outcome $m$ with probability

Pr (m) = Tr (P_{m} ρ),

leaving the post-measurement state $P_{m} ρ P_{m} / Pr (m)$ . Equivalently one measures an observable $M = \sum_{m} m P_{m}$ , a Hermitian operator whose eigenvalues are the outcomes.

For $ρ = ∣ ψ ⟩ ⟨ ψ ∣$ and $P_{m} = ∣ m ⟩ ⟨ m ∣$ this is the Born rule of the first section, $Pr (m) = ∣ ⟨ m ∣ ψ ⟩ ∣^{2}$ . But projective measurements are not the most general possibility: one may couple the system to an ancilla and measure that, and the resulting statistics need not come from any projective measurement on the system alone.

Definition 16.21 (POVM). A positive operator-valued measure is a family ${E_{m}}$ of positive semidefinite operators with $\sum_{m} E_{m} = I$ . Outcome $m$ occurs with probability $Pr (m) = Tr (E_{m} ρ)$ .

Positivity makes the probabilities non-negative and completeness makes them sum to $1$ , so a POVM is exactly what is needed for $ρ \mapsto (Pr (m))_{m}$ to be a probability distribution for every $ρ$ . Projective measurements are the special case $E_{m} = P_{m}$ with $E_{m}^{2} = E_{m}$ ; in general a POVM element is not a projector, and the number of outcomes may exceed the dimension.

Note. Naimark's dilation theorem says every POVM on $H$ arises from a projective measurement on a larger space $H \otimes H_{anc}$ , so nothing is lost by taking POVMs as the primitive. We quote it: its proof is a spectral-theory argument outside this course, which assumes linear algebra but not operator theory. Note also that a POVM fixes the outcome probabilities but not the post-measurement state — several distinct physical processes share one POVM.

Example 16.22 (A POVM that no projective measurement can imitate). Alice sends either $∣0 ⟩$ or $∣ + ⟩$ , each with probability $\frac{1}{2}$ . Bob wants a measurement that never lies: it may output "don't know", but when it names a state it must be right. Show that

E_{0} = c ∣ - ⟩ ⟨ - ∣, E_{+} = c ∣1 ⟩ ⟨ 1∣, E_{?} = I - E_{0} - E_{+}

does this, find the largest admissible $c$ , and say how often Bob gets an answer.

Solution. Because $⟨ - ∣ + ⟩ = 0$ , the outcome $E_{0}$ has probability zero when the state is $∣ + ⟩$ ; so if Bob sees it, the state was $∣0 ⟩$ with certainty. Symmetrically $⟨ 1∣0 ⟩ = 0$ , so $E_{+}$ can occur only for $∣ + ⟩$ . The scheme is error-free for every $c > 0$ , and $c$ is taken as large as $E_{?} \geq 0$ allows.

Now $E_{0} + E_{+} = c (∣ - ⟩ ⟨ - ∣ + ∣1 ⟩ ⟨ 1∣)$ , which in the computational basis is

c (1/2 - 1/2 - 1/2 3/2),

a matrix of trace $2 c$ and determinant $c^{2} (3/4 - 1/4) = c^{2} /2$ , so its eigenvalues satisfy $μ^{2} - 2 c μ + c^{2} /2 = 0$ and are $μ = c (1 \pm \frac{1}{2})$ . We need the larger one to be at most $1$ , giving $c \leq (1 + 1/ 2)^{- 1} = 2 - 2 \approx 0.5858$ .

At that value the conclusive probability for input $∣0 ⟩$ is $Tr (E_{0} ∣0 ⟩ ⟨ 0∣) = c ∣ ⟨ - ∣0 ⟩ ∣^{2} = c /2 = 1 - \frac{1}{2} \approx 0.293$ , and the same for $∣ + ⟩$ by symmetry. So Bob learns the state with certainty about $29.3%$ of the time and learns nothing at all otherwise — the trade Corollary No perfect deterministic discrimination forces on him. □□

If Bob must guess every time instead, the optimal trade-off is a clean formula.

Theorem 16.23 (Helstrom bound, two pure states). Let $∣ ψ ⟩$ and $∣ ϕ ⟩$ be equally likely, with overlap $c = ∣ ⟨ ψ ∣ ϕ ⟩ ∣$ . The minimum probability of a wrong guess, over all POVMs, is

p_{err} = \frac{1}{2} (1 - 1 - c^{2}) .

Proof. Work in the span of the two states and set $Δ = \frac{1}{2} ∣ ψ ⟩ ⟨ ψ ∣ - \frac{1}{2} ∣ ϕ ⟩ ⟨ ϕ ∣$ , a Hermitian traceless operator, so its two eigenvalues are $\pm μ$ . For a two-outcome POVM ${E, I - E}$ the success probability is

\frac{1}{2} Tr (E ∣ ψ ⟩ ⟨ ψ ∣) + \frac{1}{2} Tr ((I - E) ∣ ϕ ⟩ ⟨ ϕ ∣) = \frac{1}{2} + Tr (E Δ) .

Since $0 \leq E \leq I$ , the quantity $Tr (E Δ)$ is largest when $E$ is the projector onto the positive eigenspace of $Δ$ , which gives $μ$ ; hence $p_{err} = \frac{1}{2} - μ$ . To find $μ$ , use $Tr Δ = 0$ and

Tr (Δ^{2}) = \frac{1}{4} (1 + 1 - 2 c^{2}) = \frac{1 - c ^{2}}{2},

so that $2 μ^{2} = \frac{1 - c ^{2}}{2}$ and $μ = \frac{1}{2} 1 - c^{2}$ .∎

Both endpoints are what intuition demands: orthogonal states ( $c = 0$ ) give $p_{err} = 0$ , identical states ( $c = 1$ ) give $p_{err} = \frac{1}{2}$ , the coin flip. Everything in between is a genuine information-theoretic cost, and it is why the Holevo bound of the next chapter falls strictly below $lo g d$ for non-orthogonal ensembles.

Pitfall. "The state collapses" is a statement about the projective rule, and only for a measurement performed and recorded. If a measurement is made and the outcome discarded, the correct description is the average $\sum_{m} P_{m} ρ P_{m}$ — a decohered state, generally mixed, and not any one of the $P_{m} ρ P_{m} / Pr (m)$ .

Exercises for §16.4 →

16.5Von Neumann Entropy

We can now define the quantity this chapter is named for. It is Shannon's entropy applied to the one distribution a quantum state canonically carries: its spectrum.

Definition 16.24 (Von Neumann entropy). The von Neumann entropy of a density operator $ρ$ is

S (ρ) = - Tr (ρ lo g ρ),

where $lo g ρ$ is defined through the spectral decomposition. Equivalently, if $ρ$ has eigenvalues $λ_{1}, \dots, λ_{d}$ then

S (ρ) = - i \sum λ_{i} lo g λ_{i} = H (λ),

the Shannon entropy of the spectrum, with the convention $0 lo g 0 = 0$ . Units are bits when the logarithm is base $2$ .

Never take a matrix logarithm literally: diagonalise, then evaluate $- \sum_{i} λ_{i} lo g λ_{i}$ . And because $S$ depends only on the spectrum it is invariant under every unitary, $S (U ρ U^{†}) = S (ρ)$ — the counterpart of the fact that Shannon entropy does not care how the alphabet is labelled.

Theorem 16.25 (Basic properties). For $ρ$ on a $d$ -dimensional space:

$S (ρ) \geq 0$ , with equality if and only if $ρ$ is pure;
$S (ρ) \leq lo g d$ , with equality if and only if $ρ = I / d$ ;
$S (U ρ U^{†}) = S (ρ)$ for every unitary $U$ ;
if $ρ$ is diagonal in some basis with entries $p_{i}$ , then $S (ρ) = H (p)$ .

Proof. All four reduce to $S (ρ) = H (λ)$ and the corresponding classical facts. (1) $H \geq 0$ with equality iff the distribution is a point mass, and by Theorem Purity criterion a point-mass spectrum means a pure state. (2) $H (λ) \leq lo g d$ with equality iff $λ$ is uniform, which says $ρ = I / d$ . (3) Conjugating by a unitary changes the eigenvectors and not the eigenvalues. (4) Immediate: a diagonal matrix's entries are its eigenvalues.∎

Notation. For a bipartite state $ρ_{A B}$ write $S (A) = S (ρ_{A})$ and $S (A B) = S (ρ_{A B})$ , and define

S (A ∣ B) = S (A B) - S (B), I (A; B) = S (A) + S (B) - S (A B),

so that $I (A; B) = S (A) - S (A ∣ B)$ , exactly as in the classical chain rules. There is no underlying joint distribution these are computed from; the formulas are the definitions.

Example 16.26 (Entropy from the Bloch radius). Give $S (ρ)$ for a qubit whose Bloch vector has length $r$ , and evaluate it at $ρ = 0.75∣0 ⟩ ⟨ 0∣ + 0.25∣1 ⟩ ⟨ 1∣$ .

Solution. By Theorem Bloch ball the eigenvalues are $(1 \pm r) /2$ , so

S (ρ) = h (\frac{1 + r}{2}), h_{2} (q) = - q lo g q - (1 - q) lo g (1 - q) .

For the given state $r = 0.5$ , so $S = h_{2} (0.75) = - 0.75 lo g_{2} 0.75 - 0.25 lo g_{2} 0.25 \approx 0.311 + 0.5 = 0.811$ bits. The function falls from $1$ bit at the centre of the ball to $0$ on its surface: entropy measures how far inside the Bloch ball a qubit sits. □□

Figure 16.27. Von Neumann entropy of a qubit against the Bloch radius

r

. The maximally mixed state at

r = 0

carries one full bit; the entropy falls to zero at the surface, where the state is pure. The curve has infinite slope at

r = 1

, which is why states close to the surface are still appreciably mixed — at

r = 0.5

there is still

0.81

of a bit.

Relative entropy transfers as well, and it is the engine behind almost every inequality in the next section.

Definition 16.28 (Quantum relative entropy). For density operators $ρ, σ$ ,

D (ρ ∥ σ) = Tr (ρ lo g ρ) - Tr (ρ lo g σ),

taken to be $+ \infty$ when the support of $ρ$ is not contained in the support of $σ$ .

Theorem 16.29 (Klein's inequality). $D (ρ ∥ σ) \geq 0$ , with equality if and only if $ρ = σ$ .

Proof. Let $ρ = \sum_{i} λ_{i} ∣ a_{i} ⟩ ⟨ a_{i} ∣$ and $σ = \sum_{j} μ_{j} ∣ b_{j} ⟩ ⟨ b_{j} ∣$ be spectral decompositions and put $P_{ij} = ∣ ⟨ a_{i} ∣ b_{j} ⟩ ∣^{2}$ , a doubly stochastic matrix because both families are orthonormal bases. Then $Tr (ρ lo g σ) = \sum_{i, j} λ_{i} P_{ij} lo g μ_{j}$ , and applying Jensen's inequality to the concave $lo g$ with the weights $P_{ij}$ , which sum to $1$ over $j$ ,

Tr (ρ lo g σ) \leq i \sum λ_{i} lo g (j \sum P_{ij} μ_{j}) = i \sum λ_{i} lo g ν_{i}, ν_{i} := j \sum P_{ij} μ_{j} .

Hence $D (ρ ∥ σ) \geq \sum_{i} λ_{i} lo g (λ_{i} / ν_{i}) = D (λ ∥ ν) \geq 0$ by the classical Gibbs inequality; note $\sum_{i} ν_{i} = 1$ because $P$ is doubly stochastic. Equality in Jensen forces the $μ_{j}$ to agree wherever $P_{ij} > 0$ , and equality in Gibbs forces $λ = ν$ ; together these give $ρ = σ$ .∎

Two consequences get used constantly. Taking $σ = I / d$ gives $D (ρ ∥ I / d) = lo g d - S (ρ) \geq 0$ , recovering the maximum-entropy bound; and for any $σ$ ,

S (ρ) \leq - Tr (ρ lo g σ),

the quantum form of "cross-entropy is at least entropy" — describing a state with the wrong model costs you bits.

Intuition. Von Neumann entropy is Shannon entropy asked in the state's own preferred basis. Measure $ρ$ in its eigenbasis and you get a classical variable of entropy exactly $S (ρ)$ ; measure it in any other basis and the classical entropy is strictly larger, because the doubly stochastic smearing in the proof of Klein's inequality can only spread a distribution out. So $S (ρ)$ is the minimum classical randomness extractable from the state — read the other way, the irreducible uncertainty in it.

Exercises for §16.5 →

16.6Properties of Von Neumann Entropy

Almost every classical entropy inequality has an exact quantum counterpart. The exception is important enough to organise this whole section: classically $H (X_{1}, \dots, X_{n})$ is never smaller than the entropy of any subcollection, and quantum mechanically that fails.

Theorem 16.30 (Subadditivity). For any bipartite state, $S (A B) \leq S (A) + S (B)$ , with equality if and only if $ρ_{A B} = ρ_{A} \otimes ρ_{B}$ .

Proof. Compute the relative entropy of $ρ_{A B}$ to the product of its own marginals. Since $lo g (ρ_{A} \otimes ρ_{B}) = lo g ρ_{A} \otimes I + I \otimes lo g ρ_{B}$ ,

D (ρ_{A B} ∥ ρ_{A} \otimes ρ_{B}) = - S (A B) - Tr [ρ_{A B} (lo g ρ_{A} \otimes I)] - Tr [ρ_{A B} (I \otimes lo g ρ_{B})] .

By Theorem The partial trace is the right marginal the second trace is $Tr (ρ_{A} lo g ρ_{A}) = - S (A)$ and the third is $- S (B)$ , so

D (ρ_{A B} ∥ ρ_{A} \otimes ρ_{B}) = S (A) + S (B) - S (A B) = I (A; B) .

Klein's inequality gives $I (A; B) \geq 0$ , which is the claim, with equality exactly when $ρ_{A B} = ρ_{A} \otimes ρ_{B}$ .∎

That identity — quantum mutual information is a relative entropy — is worth keeping. It gives $I (A; B) \geq 0$ at once, and it is the form in which the Holevo bound will be proved in the next chapter.

Theorem 16.31 (Araki–Lieb triangle inequality). For any bipartite state, $S (A B) \geq S (A) - S (B)$ .

Proof. Purify: by Theorem Purification there is a system $R$ and a pure $∣ ψ ⟩_{A BR}$ whose marginal on $A B$ is $ρ_{A B}$ . Because the global state is pure, Theorem Schmidt decomposition across the cut $A B ∣ R$ gives $S (A B) = S (R)$ , and across the cut $B ∣ A R$ gives $S (B) = S (A R)$ . Applying subadditivity to the pair $(A, R)$ ,

S (B) = S (A R) \leq S (A) + S (R) = S (A) + S (A B),

so $S (A B) \geq S (B) - S (A)$ . Exchanging $A$ and $B$ gives $S (A B) \geq S (A) - S (B)$ , and the two together give the absolute value.∎

Corollary 16.32 (Conditional entropy is bounded below). $S (A ∣ B) \geq - S (A)$ , and in particular $S (A ∣ B) \geq - lo g d_{A}$ .

Proof. $S (A ∣ B) = S (A B) - S (B) \geq - S (A)$ is Araki–Lieb rearranged, and $S (A) \leq lo g d_{A}$ by Theorem Basic properties.∎

Here is the break with the classical theory. Classically $H (X ∣ Y) \geq 0$ : knowing $Y$ cannot leave you with negative uncertainty about $X$ , because $H (X, Y) \geq H (Y)$ . The quantum statement is simply false.

Theorem 16.33 (Failure of monotonicity). There is a bipartite state with $S (A B) < S (A)$ ; equivalently $S (A ∣ B)$ can be negative. For the Bell state $∣ Φ^{+} ⟩$ one has $S (A B) = 0$ , $S (A) = S (B) = 1$ and $S (A ∣ B) = - 1$ bit, the most negative value Corollary Conditional entropy is bounded below permits for a qubit.

Proof. $∣ Φ^{+} ⟩$ is pure, so $S (A B) = 0$ . Its Schmidt coefficients are $\frac{1}{2}, \frac{1}{2}$ , so $ρ_{A} = ρ_{B} = I /2$ and $S (A) = S (B) = 1$ by Theorem Basic properties. Hence $S (A ∣ B) = 0 - 1 = - 1$ , while $- S (A) = - 1$ shows the bound is attained.∎

Intuition. Classically a joint state is a list of values, and the whole cannot be less uncertain than a part because the part is read off the whole. An entangled pair is not a list of values. The pair is in a perfectly definite state — "the two outcomes will agree" — while neither member has any definite state of its own. The negative bit in $S (A ∣ B)$ is not a bookkeeping accident: in the quantum state-merging protocol it counts Bell pairs returned to the parties, one ebit per negative bit.

The deeper inequality, from which most of quantum Shannon theory is derived, cannot be proved with the tools this course develops.

Theorem 16.34 (Strong subadditivity, quoted). For any tripartite state $ρ_{A BC}$ ,

S (A BC) + S (B) \leq S (A B) + S (BC),

equivalently $S (A ∣ BC) \leq S (A ∣ B)$ — discarding a system cannot decrease conditional entropy — equivalently again $I (A; BC) \geq I (A; B)$ .

Proof. This is a sketch, and the gap is named. Unlike subadditivity, the result does not follow from Klein's inequality: the natural argument would need $Tr exp$ to behave multiplicatively on non-commuting operators, and it does not. Lieb and Ruskai (1973) derive it from Lieb's concavity theorem, that $(A, B) \mapsto Tr (K^{†} A^{t} K B^{1 - t})$ is jointly concave in positive $A, B$ for $t \in [0, 1]$ — a matrix-analysis result about non-commuting operators requiring operator-convexity theory this course does not develop. An alternative route derives it from the monotonicity of relative entropy under quantum channels, stated in the next chapter; each of the two statements implies the other, so quoting one is quoting both.∎

Corollary 16.35 (Consequences of strong subadditivity). Strong subadditivity yields: $I (A; B) \leq I (A; BC)$ ; concavity of $S (A ∣ B)$ as a function of $ρ_{A B}$ ; and $S (A ∣ B) \geq 0$ for every separable $ρ_{A B}$ — so a negative conditional entropy certifies entanglement.

Proof. The first is a restatement. Granting the second, take a separable $ρ_{A B} = \sum_{i} p_{i} σ_{i}^{A} \otimes τ_{i}^{B}$ ; concavity gives

S (A ∣ B)_{ρ} \geq i \sum p_{i} S (A ∣ B)_{σ_{i} \otimes τ_{i}} = i \sum p_{i} S (σ_{i}) \geq 0,

using that a product state has $S (A B) = S (A) + S (B)$ and hence $S (A ∣ B) = S (A)$ . The concavity step is one of the standard equivalent forms of strong subadditivity and is quoted with it.∎

Finally, mixing. Classically, averaging distributions increases entropy, by at most the entropy of the mixing variable; both halves survive the passage to quantum states.

Theorem 16.36 (Mixing bounds). Let $ρ = \sum_{i} p_{i} ρ_{i}$ be a mixture of density operators. Then

i \sum p_{i} S (ρ_{i}) \leq S (ρ) \leq i \sum p_{i} S (ρ_{i}) + H (p),

and the right-hand inequality is an equality if and only if the $ρ_{i}$ have pairwise orthogonal supports.

Proof. For the lower bound — concavity of $S$ — expand the average relative entropy to $ρ$ :

i \sum p_{i} D (ρ_{i} ∥ ρ) = - i \sum p_{i} S (ρ_{i}) - Tr ((\sum_{i} p_{i} ρ_{i}) lo g ρ) = S (ρ) - \sum_{i} p_{i} S (ρ_{i}),

and every $D (ρ_{i} ∥ ρ) \geq 0$ by Klein's inequality. For the upper bound, adjoin a classical register: on an orthonormal family ${∣ i ⟩}$ put $ρ_{X A} = \sum_{i} p_{i} ∣ i ⟩ ⟨ i ∣ \otimes ρ_{i}$ . This is block diagonal, so its spectrum is the union of the spectra of the blocks $p_{i} ρ_{i}$ , giving

S (X A) = - i \sum k \sum p_{i} λ_{k}^{(i)} lo g (p_{i} λ_{k}^{(i)}) = H (p) + i \sum p_{i} S (ρ_{i}),

where $λ^{(i)}$ is the spectrum of $ρ_{i}$ . Also $S (X) = H (p)$ and $S (A) = S (ρ)$ . The claim is exactly $S (A) \leq S (X A)$ , that is $S (X ∣ A) \geq 0$ , which holds because $ρ_{X A}$ is separable — Corollary Consequences of strong subadditivity. Equality means $I (X; A) = H (p)$ , i.e. $X$ is perfectly recoverable from $A$ , which happens exactly when the $ρ_{i}$ occupy orthogonal subspaces and so are told apart by one projective measurement.∎

Example 16.37 (The mixing gap is the Holevo quantity). Compute both bounds for the ensemble ${\frac{1}{2}, ∣0 ⟩; \frac{1}{2}, ∣ + ⟩}$ and interpret the gap $S (ρ) - \sum_{i} p_{i} S (ρ_{i})$ .

Solution. Both members are pure, so $\sum_{i} p_{i} S (ρ_{i}) = 0$ and the lower bound is $0$ , while $H (p) = 1$ bit gives the upper bound. The average state is

ρ = \frac{1}{2} ∣0 ⟩ ⟨ 0∣ + \frac{1}{2} ∣ + ⟩ ⟨ + ∣ = \frac{1}{2} (1000) + \frac{1}{4} (1111) = (3/4 1/4 1/4 1/4) .

Its trace is $1$ and its determinant is $3/16 - 1/16 = 1/8$ , so the eigenvalues solve $λ^{2} - λ + 1/8 = 0$ and are $λ = \frac{1}{2} (1 \pm \frac{1}{2}) \approx 0.8536, 0.1464$ . Hence $S (ρ) = h_{2} (0.8536) \approx 0.601$ bits, strictly between the two bounds.

The gap is $0.601$ bits, and it is exactly the Holevo quantity $χ$ of this ensemble. By the theorem of the next chapter no measurement can extract more than $0.601$ bits about which state was sent — less than the $1$ bit that was encoded, because $∣0 ⟩$ and $∣ + ⟩$ are not orthogonal. □□

Pitfall. Do not import $H (X ∣ Y) \geq 0$ , $H (X) \leq H (X, Y)$ , or "a subsystem is less uncertain than the whole" into the quantum setting. The first two are false. The third, in the correct form $S (A ∣ BC) \leq S (A ∣ B)$ , is true but is strong subadditivity and is not cheap. When in doubt, test a claim on the Bell state: it is the counterexample to almost every false quantum entropy inequality.

Exercises for §16.6 →

Common Mistakes

Treating a qubit as a biased coin. A coin has one parameter, a qubit has two, and the extra one — relative phase — is what makes $∣ + ⟩$ and $∣ - ⟩$ perfectly distinguishable despite identical computational-basis statistics.
Confusing a superposition with a mixture. $∣ + ⟩$ is pure, with purity $1$ ; $\frac{1}{2} (∣0 ⟩ ⟨ 0∣ + ∣1 ⟩ ⟨ 1∣)$ is mixed, with purity $\frac{1}{2}$ . They agree in one basis and differ in every other.
Thinking the ensemble can be recovered from $ρ$ . It cannot: ${\frac{1}{2}, ∣0 ⟩; \frac{1}{2}, ∣1 ⟩}$ and ${\frac{1}{2}, ∣ + ⟩; \frac{1}{2}, ∣ - ⟩}$ both give $I /2$ , and no experiment separates them.
Reading the partial trace as conditioning. $ρ_{A} = Tr_{B} ρ_{A B}$ is the unconditional marginal. There is no "state of $A$ given $B$ " until $B$ is measured, and measuring $B$ never changes $ρ_{A}$ .
Assuming $S (A ∣ B) \geq 0$ . It is $- 1$ bit for a Bell pair. Any argument that silently uses a classical conditioning inequality on quantum states is wrong.
Using $S (A B) \geq S (A)$ . False, and the same Bell pair kills it. The correct lower bound is Araki–Lieb, $S (A B) \geq ∣ S (A) - S (B) ∣$ .
Taking the logarithm of $ρ$ entrywise. $lo g ρ$ is defined through the spectral decomposition; for a non-diagonal $ρ$ the entrywise logarithm is not even the right kind of object.
Expecting strong subadditivity to be a corollary of Klein's inequality. It is not. It rests on Lieb's concavity theorem, and every proof of it that works proves something of that depth along the way.

About the practice questions. They check that you can carry out this chapter's computations correctly, and each one is graded on a single answer. They are not proof exercises: working through them confirms the mechanics, not that you could prove the results yourself. For that, re-read the statements above and try to reconstruct their proofs with the page closed.

Practice this chapter