Теория информации / Cover T.M., Thomas J.A. Elements of Information Theory. 2006., 748p

Slepian –Wolf coding. Correlated sources X and Y can be described separately at rates R1 and R2 and recovered with arbitrarily low probability of error by a common decoder if and only if

Broadcast channels. The capacity region of the degraded broadcast channel X → Y1 → Y2 is the convex hull of the closure of all (R1, R2) satisfying

for some joint distribution p(u)p(x|u)p(y1, y2|x).

Relay channel. The capacity C of the physically degraded relay channel p(y, y1|x, x1) is given by

p(x,x1)

where the supremum is over all joint distributions on X × X1.

Source coding with side information. Let (X, Y ) p(x, y). If Y is encoded at rate R2 and X is encoded at rate R1, we can recover X with an arbitrarily small probability of error iff

for some distribution p(y, u) such that X → Y → U .

596 NETWORK INFORMATION THEORY

Rate distortion with side information. Let (X, Y ) p(x, y). The rate distortion function with side information is given by

where the minimization is over all functions f and conditional distributions p(w|x), |W| ≤ |X| + 1, such that

x w y

PROBLEMS

15.1Cooperative capacity of a multiple-access channel

(a) Suppose that X1 and X2 have access to both indices W1 {1, 2nR }, W2 {1, 2nR2 }. Thus, the codewords X1(W1,

W2), X2(W1, W2) depend on both indices. Find the capacity region.

(b) Evaluate this region for the binary erasure multiple access channel Y = X1 + X2, Xi {0, 1}. Compare to the noncooperative region.

15.2Capacity of multiple-access channels . Find the capacity region for each of the following multiple-access channels:

PROBLEMS 597

15.3Cut-set interpretation of capacity region of multiple-access channel . For the multiple-access channel we know that (R1, R2) is achievable if

for X1, X2 independent. Show, for X1, X2 independent that

I (X1; Y | X2) = I (X1; Y, X2).

Y

X2

S2

Interpret the information bounds as bounds on the rate of flow across cut sets S1, S2, and S3.

15.4Gaussian multiple-access channel capacity . For the AWGN multiple-access channel, prove, using typical sequences, the achievability of any rate pairs (R1, R2) satisfying

598 NETWORK INFORMATION THEORY

The proof extends the proof for the discrete multiple-access channel in the same way as the proof for the single-user Gaussian channel extends the proof for the discrete single-user channel.

15.5Converse for the Gaussian multiple-access channel . Prove the converse for the Gaussian multiple-access channel by extending the converse in the discrete case to take into account the power constraint on the codewords.

15.6Unusual multiple-access channel . Consider the following

multiple-access channel: X1 = X2 = Y = {0, 1}. If (X1, X2) = (0, 0), then Y = 0. If (X1, X2) = (0, 1), then Y = 1. If (X1, X2) =

(1, 0), then Y = 1. If (X1, X2) = (1, 1), then Y = 0 with probability 12 and Y = 1 with probability 12 .

(a)Show that the rate pairs (1,0) and (0,1) are achievable.

(b)Show that for any nondegenerate distribution p(x1)p(x2), we have I (X1, X2; Y ) < 1.

(c)Argue that there are points in the capacity region of this multiple-access channel that can only be achieved by timesharing; that is, there exist achievable rate pairs (R1, R2) that lie in the capacity region for the channel but not in the region defined by

for any product distribution p(x1)p(x2). Hence the operation of convexification strictly enlarges the capacity region. This channel was introduced independently by Csiszar´ and Korner¨

[149]and Bierbaum and Wallmeier [59].

15.7Convexity of capacity region of broadcast channel . Let C R2 be the capacity region of all achievable rate pairs R = (R1, R2) for the broadcast channel. Show that C is a convex set by using a time-sharing argument. Specifically, show that if R(1) and R(2) are achievable, λR(1) + (1 − λ)R(2) is achievable for 0 ≤ λ ≤ 1.

15.8Slepian–Wolf for deterministically related sources . Find and sketch the Slepian – Wolf rate region for the simultaneous data compression of (X, Y ), where y = f (x) is some deterministic function of x.

PROBLEMS 601

15.12Capacity points.

(a)For the degraded broadcast channel X → Y1 → Y2, find the points a and b where the capacity region hits the R1 and R2 axes.

R2

b

a R1

(b)Show that b ≤ a.

15.13Degraded broadcast channel . Find the capacity region for the degraded broadcast channel shown below.

15.14Channels with unknown parameters. We are given a binary

symmetric channel with parameter p. The capacity is C = 1 − H (p). Now we change the problem slightly. The receiver knows only that p {p1, p2} (i.e., p = p1 or p = p2, where p1 and p2 are given real numbers). The transmitter knows the actual value of p. Devise two codes for use by the transmitter, one to be used if p = p1, the other to be used if p = p2, such that transmission to the receiver can take place at rate ≈ C(p1) if p = p1 and at rate ≈ C(p2) if p = p2. (Hint: Devise a method for revealing p to the receiver without affecting the asymptotic rate. Prefixing the codeword by a sequence of 1’s of appropriate length should work.)

602 NETWORK INFORMATION THEORY

15.15Two-way channel . Consider the two-way channel shown in

Figure 15.6. The outputs Y1 and Y2 depend only on the current inputs X1 and X2.

(a)By using independently generated codes for the two senders, show that the following rate region is achievable:

for some product distribution p(x1)p(x2)p(y1, y2|x1, x2).

(b) Show that the rates for any code for a two-way channel with arbitrarily small probability of error must satisfy

for some joint distribution p(x1, x2)p(y1, y2|x1, x2).

The inner and outer bounds on the capacity of the two-way channel are due to Shannon [486]. He also showed that the inner bound and the outer bound do not coincide in the case of the binary multiplying channel X1 = X2 = Y1 = Y2 = {0, 1}, Y1 = Y2 = X1X2. The capacity of the two-way channel is still an open problem.

15.16Multiple-access channel . Let the output Y of a multiple-access channel be given by

Y = X1 + sgn(X2),

where X1, X2 are both real and power limited,

E(X12) ≤ P1,

E(X22) ≤ P2,

and sgn(x) =

1, x > 0, −1, x ≤ 0.

Note that there is interference but no noise in this channel.

(a)Find the capacity region.

(b)Describe a coding scheme that achieves the capacity region.

PROBLEMS 603

15.17Slepian–Wolf . Let (X, Y ) have the joint probability mass function p(x, y):

where β = 16 − α2 . (Note: This is a joint, not a conditional, probability mass function.)

(a)Find the Slepian – Wolf rate region for this source.

(b)What is Pr{X = Y } in terms of α?

(c)What is the rate region if α = 13 ?

(d)What is the rate region if α = 19 ?

15.18Square channel . What is the capacity of the following multipleaccess channel?

X1 {−1, 0, 1},

X2 {−1, 0, 1},

Y= X12 + X22.

(a)Find the capacity region.

(b)Describe p (x1), p (x2) achieving a point on the boundary of the capacity region.

15.19Slepian–Wolf . Two senders know random variables U1 and U2, respectively. Let the random variables (U1, U2) have the following joint distribution:

where α + β + γ = 1. Find the region of rates (R1, R2) that would allow a common receiver to decode both random variables reliably.

604 NETWORK INFORMATION THEORY

15.20Multiple access

(a)Find the capacity region for the multiple-access channel

Y = X1X2 ,

where

X1 {2, 4} , X2 {1, 2}.

(b) Suppose that the range of X1 is {1, 2}. Is the capacity region decreased? Why or why not?

15.21Broadcast channel . Consider the following degraded broadcast channel.

(a)What is the capacity of the channel from X to Y1?

(b)What is the channel capacity from X to Y2?

(c)What is the capacity region of all (R1, R2) achievable for this broadcast channel? Simplify and sketch.

15.22Stereo. The sum and the difference of the right and left ear signals are to be individually compressed for a common receiver. Let

Z1 be Bernoulli (p1) and Z2 be Bernoulli (p2) and suppose that Z1 and Z2 are independent. Let X = Z1 + Z2, and Y = Z1 − Z2.

(a)What is the Slepian – Wolf rate region of achievable (RX, RY )?

^ ^

Decoder (X, Y )