Теория информации / Cover T.M., Thomas J.A. Elements of Information Theory. 2006., 748p

It is not difficult to see that any point in C1 or C2 has x + y < 20, so any point in the convex hull of the union of C1 and C2 satisfies this property. Thus, the point (15,15), which is in C, is not in the convex hull of (C1 C2). This example also hints at the cause of the problem — in the definition for C1, the constraint x + y ≤ 100 is not active. If this constraint were replaced by a constraint x + y ≤ a, where a ≤ 20, the above result of the equality of the two regions would be true, as we now prove.

We restrict ourselves to the pentagonal regions that occur as components of the capacity region of a two-user multiple-access channel. In this case, the capacity region for a fixed p(x1)p(x2) is defined by three mutual informations, I (X1; Y |X2), I (X2; Y |X1), and I (X1, X2; Y ), which we shall call I1, I2, and I3, respectively. For each p(x1)p(x2), there is a corresponding vector, I = (I1, I2, I3), and a rate region defined by

CI = {(R1, R2) : R1 ≥ 0, R2 ≥ 0, R1 ≤ I1, R2 ≤ I2, R1 + R2 ≤ I3}.

(15.84)

Also, since for any distribution p(x1)p(x2), we have I (X2; Y |X1) =

H (X2|X1) − H (X2|Y, X1) = H (X2) − H (X2|Y, X1) = I (X2; Y, X1) = I (X2; Y ) + I (X2; X1|Y ) ≥ I (X2; Y ), and therefore, I (X1; Y |X2) + I (X2; Y |X1) ≥ I (X1; Y |X2)+ I (X2; Y ) = I (X1, X2; Y ), we have for all vectors I that I1 + I2 ≥ I3. This property will turn out to be critical for

the theorem.

Lemma 15.3.1 Let I1, I2 R3 be two vectors of mutual informations that define rate regions CI1 and CI2 , respectively, as given in (15.84).

For 0 ≤ λ ≤ 1, define Iλ = λI1 + (1 − λ)I2, and let CIλ be the rate region defined by Iλ. Then

Proof: We shall prove this theorem in two parts. We first show that any point in the (λ, 1 − λ) mix of the sets CI1 and CI2 satisfies the constraints Iλ. But this is straightforward, since any point in CI1 satisfies the inequalities for I1 and a point in CI2 satisfies the inequalities for I2, so the (λ, 1 − λ) mix of these points will satisfy the (λ, 1 − λ) mix of the constraints. Thus, it follows that

To prove the reverse inclusion, we consider the extreme points of the pentagonal regions. It is not difficult to see that the rate regions defined in (15.84) are always in the form of a pentagon, or in the extreme case

536 NETWORK INFORMATION THEORY

when I3 = I1 + I2, in the form of a rectangle. Thus, the capacity region CI can be also defined as a convex hull of five points:

Consider the region defined by Iλ; it, too, is defined by five points. Take any one of the points, say (I3(λ) − I2(λ), I2(λ)). This point can be written as

the (λ, 1 − λ) mix of the points (I3(1) − I2(1), I2(1)) and (I3(2) − I2(2), I2(2)), and therefore lies in the convex mixture of CI1 and CI2 . Thus, all extreme

points of the pentagon CIλ lie in the convex hull of CI1 and CI2 , or

In the proof of the theorem, we have implicitly used the fact that all the rate regions are defined by five extreme points (at worst, some of the points are equal). All five points defined by the I vector were within the rate region. If the condition I3 ≤ I1 + I2 is not satisfied, some of the points in (15.87) may be outside the rate region and the proof collapses.

As an immediate consequence of the above lemma, we have the following theorem:

Theorem 15.3.3 The convex hull of the union of the rate regions defined by individual I vectors is equal to the rate region defined by the convex hull of the I vectors.

These arguments on the equivalence of the convex hull operation on the rate regions with the convex combinations of the mutual informations can be extended to the general m-user multiple-access channel. A proof along these lines using the theory of polymatroids is developed in Han [271].

Theorem 15.3.4 The set of achievable rates of a discrete memoryless multiple-access channel is given by the closure of the set of all (R1, R2) pairs satisfying

for some choice of the joint distribution p(q)p(x1|q)p(x2|q)p(y|x1, x2) with |Q| ≤ 4.

Proof: We will show that every rate pair lying in the region defined in (15.89) is achievable (i.e., it lies in the convex closure of the rate pairs satisfying Theorem 15.3.1). We also show that every point in the convex closure of the region in Theorem 15.3.1 is also in the region defined in (15.89).

Consider a rate point R satisfying the inequalities (15.89) of the theorem. We can rewrite the right-hand side of the first inequality as

where m is the cardinality of the support set of Q. We can expand the other mutual informations similarly.

For simplicity in notation, we consider a rate pair as a vector and denote a pair satisfying the inequalities in (15.58) for a specific input product distribution p1q (x1)p2q (x2) as Rp1,p2 as Rq . Specifically, let Rq = (R1q , R2q ) be a rate pair satisfying

Then by Theorem 15.3.1, Rq = (R1q , R2q ) is achievable. Then since R satisfies (15.89) and we can expand the right-hand sides as in (15.91), there exists a setof pairs Rq satisfying (15.94) such that

q=1

Since a convex combination of achievable rates is achievable, so is R. Hence, we have proven the achievability of the region in the theorem. The same argument can be used to show that every point in the convex closure of the region in (15.58) can be written as the mixture of points satisfying (15.94) and hence can be written in the form (15.89).

Similarly, we have

i=1

nn

i=1

where

(a)follows from Fano’s inequality

(b)follows from the data-processing inequality

(c)follows from the chain rule

(d)follows from the fact that Yi depends only on X1i and X2i and is conditionally independent of everything else

(e)follows from the chain rule and removing conditioning

Hence, we have

542 NETWORK INFORMATION THEORY

The expressions in (15.114), (15.115), and (15.125) are the averages of the mutual informations calculated at the empirical distributions in column i of the codebook. We can rewrite these equations with the new variable Q, where Q = i {1, 2, . . . , n} with probability n1 . The equations become

whose distributions depend on Q in the same way as the distributions of X1i , X2i and Yi depend on i. Since W1 and W2 are independent, so are X1i (W1) and X2i (W2), and hence

Pr (X1i (W1) = x1, X2i (W2) = x2)

Hence, taking the limit as n → ∞, Pe(n) → 0, we have the following converse:

for some choice of joint distribution p(q)p(x1|q)p(x2|q)p(y|x1, x2). As in Section 15.3.3, the region is unchanged if we limit the cardinality of

Thus, the achievability of the region of Theorem 15.3.1 was proved in Section 15.3.1. In Section 15.3.3 we showed that every point in the region defined by (15.96) was also achievable. In the converse, we showed that the region in (15.96) was the best we can do, establishing that this is indeed the capacity region of the channel. Thus, the region in (15.58)