communication under normed uncertainties

Communication Under Normed Uncertainties

S. Z. DenicSchool of Information Technology and EngineeringUniversity of Ottawa, Ottawa, Canada

C. D. CharalambousDepartment of Electrical and Computer EngineeringUniversity of Cyprus, Nicosia, Cyprus

S. M. DjouadiDepartment of Electrical and Computer EngineeringUniversity of Tennessee, Knoxville, USA

Dec 9, 2004

Robust Capacity of White Gaussian Noise with Uncertainty

2

Overview

Importance of Uncertainty in Communications

Shannon’s Definition of Capacity

Review of Maximin Capacity

Paper Contributions

3

Overview

Main Results

Examples

Conclusion and Future Work

4

Importance of Communication Subject to Uncertainties

Channel measurement errors Network operating conditions Channel modeling Communication in presence of jamming Sensor networks Teleoperations

5

Shannon’s Definition of Capacity

6

Shannon’s Definition of Capacity

Model of communication system

wSource Encoder Decoder SinkChannel +

x

ny w

f g

M,...,1 M,...,1 wf wfh wfhg

M

ii

ne

i

MP

Misentiiifhg

n

MR

1

1

,...,1,|Pr

log

nM ,

8

Shannon’s Definition of Capacity

Discrete memoryless channel

Channel capacity depends on channel transition matrix Q(y|x) that is known

Xx YyXx

XP

xyQxP

xyQxyQxPQPI

QPIC

'

'|'

|log|,

,max

9

What if Q(y|x) is unknown ? Example: compound BSC

What is the channel capacity ?

Shannon’s Definition of Capacity

1-

1- 0 0

1 1

xy

xyxyQ

YX

,1

,;|

1,0,1,0

12

Additive Gaussian Channels Random Process Case

Shannon’s Definition of Capacity

+x y

fW

fH

n

13

Random process case derivation

Shannon’s Definition of Capacity

2

2

1sup log 1

2

;

x

x

S An

x x

S HC df

S W

A S S df P

14

Capacity of continuous time additive Gaussian channel

Shannon’s Definition of Capacity

df

WS

HC

n

2

2*

log2

1

PdfH

WSn

2

2

*

16

Water-filling

Shannon’s Definition of Capacity

*2

2

H

WSn

fBfBf

psd

17

Review of Maximin Capacity

18

Example: compound DMC

This result is due to Blackwell et. al. [6]. Also look at Csiszar [8], and Wolfowitz [21]

Blachman [5], and Dobrushin [12] were first to apply game theoretic approach in computing the channel capacity with mutual information as a pay-off function for discrete channels

Review of Minimax Capacity

;|,infmax

QPIC

XP

19

Review of Minimax Capacity

The existence of saddle point ?

For further references see Lapidoth, Narayan [18]

;|,;|,;|,

;|,maxinf;|,infmax

****

QPIQPIQPI

QPIQPICXPXP

23

Paper Contributions

24

Modeling of uncertainties in the normed linear spaces H∞, and L1

Explicit channel capacity formulas for SISO communication channels that depend on the sizes of uncertainty sets for uncertain channel, uncertain noise, and uncertain channel, and noise

Explicit water-filling formulas that describe optimal transmitted powers for all derived channel capacities formulas depending on the size of uncertainty sets

Paper Contributions

28

Main Results

29

Model

Communication system model

+x

n

y

fW

fH

31

Communication system model

Uncertainty models: additive and multiplicative

fWf 1

fGnom + fWf 1

fGnom +

32

Example

Communication system model

/(1-)

Re

Im

/(1+)

/

/,

2

fjfGnom

10,

1/2

ff

fj

ffG

p

p

33

The uncertainty set is described by the ball in frequency domain centered at and with radius of

Communication system model

fj

fWfGfG nom 21

fW1

fGnom

34

Channel capacity with uncertainty

Define four sets

,; 222 WWWHWA nom

1,,, 222

HWHHWnom

xxx PdffSfSA ;1

,; 113 WHHHHA nom

1,,, 111

HWHHHnom

35

Overall PSD of noise is and uncertainty is modeled by uncertainty of filter

or by the set A4

nnn PdffSfSA ;4

Channel capacity with uncertainty

2fWfSn

fWffWfW nom 22

37

Three problems could be defined Noise uncertainty

Channel uncertainty

df

WWS

HSC

nomn

x

AWASNU

x

2

22

2

1log2

1infsup

21

df

WS

WHSC

n

nomx

AHASCU

x

2

2

111log2

1infsup

31

Channel capacity with uncertainty I

38

Channel – noise uncertainty

df

WWS

WHSC

nomn

nomx

AHAWASCNU

x

2

22

2

111log2

1infinfsup

321

Channel capacity with uncertainty I

40

Channel capacity is given parametrically

df

WWS

WHC

nomn

nomCNU 2

2

21*

log2

1

x

nom

nomn PdfWH

WWS

2

1

2

2*

Channel capacity with uncertainty I

45

Maximization gives water – filling equation

*2

1

2

2*

WH

WWSS

nom

nomnx

Channel capacity with uncertainty I

46

Channel capacity with uncertainty I

Water – filling

*

21

2

2

WH

WWSn

fBfBf

psd

47

Jamming Noise uncertainty

Channel – noise uncertainty

Channel capacity with uncertainty II

df

WS

HSC

n

x

ASASNU

nx

2

2

1log2

1infsup

41

df

WS

WHSC

n

nomx

AHASASCNU

nx

2

2

111log2

1infinfsup

341

49

The lower value C- of pay-off function is defined as

and is given by Theorem 2. The upper value C+ is defined by

Channel capacity with uncertainty II

dfWS

HSCC

n

x

ASASNU

nx

2

2

1log2

1infsup

41

dfWS

HSC

n

x

ASASxn

2

2

1log2

1supinf

14

50

Channel capacity is given as

where are Lagrange multipliers

Channel capacity with uncertainty II

W

HRdfRCNU

;1log

2

1 2*2

*1

*

*2

*1

*2

2*1

*2

2*1*

2

1

2*2*

2

02

1

nx

xxnn

SR

RS

RSRSSS

0, *2

*1

52

Channel coding theorem

Define the frequency response of equivalent channel

with impulse response and ten sets

2/1

2

2

WS

HSfG

n

x

tg

3211 ,,; AHAWASGB x

55

Positive number Ri is called attainable rate for the set of channels Ki if there exists a sequence of codes such that when then uniformly over set Ki.

Theorem 1. The operational capacities Ci (supremum of all attainable rates Ri) for the sets of communication channels with the uncertainties Ki are given by corresponding computed capacity formulas.

Proof. Follows from [15], and [20] (see [11])

nnRT Te n ,,

nT 0n

Channel coding theorem

62

Uncertain channel, white noise Transfer function

Example 1

/,10,

1/2

1/2

/

1

ff

fj

ffH

fjfH

p

p

nom

1/2111

fjfWfWffHfH nom

63

Channel capacity

Example 1

3/16/1

13/16/1

49

tan4

91 cP

c

PCCU

otherwise

c

PcPc

fS fx

,0

4

9,

4

9|

3/16/12

3/12

2

220

12

Nc

64

P = 10-2 WN0 = 10-8 W/Hz = 1000 rad/s

1000

500

250

Example 1

65

Example 2

Uncertain noise Transfer function

Noise uncertainty description

22

2

222 nn

n

fjfjfH

/22 fj

fW

/,10,

1/2

2

ff

fj

ffW

p

p

66

sradn /700

sradn /1000

sradn /1300

srad

WP

/1000

1

2.0

01.0

Example 2

67

srad

srad

WP

n /1300

/1000

1

2.0

01.0

0

2.0

1.0

Example 2

68

Example 3

Uncertain channel, uncertain noise

Damping ration is uncertain

The noise uncertainty is modelled as in the Example 2

22

2

222 nn

n

fjfjfH

69

Example 3

70

Example 3

71

Currently generalizing these results to uncertain MIMO channels

Future work

72

References

[1] Ahlswede, R., “The capacity of a channel with arbitrary varying Gaussian channel probability functions”, Trans. 6th Prague Conf. Information Theory, Statistical Decision Functions, and Random Processes, pp. 13-31, Sept. 1971.

[2] Baker, C. R., Chao, I.-F., “Information capacity of channels with partially unknown noise. I. Finite dimensional channels”, SIAM J. Appl. Math., vol. 56, no. 3, pp. 946-963, June 1996.

73

[4] Biglieri, E., Proakis, J., Shamai, S., “Fading channels: information-theoretic and communications aspects,” IEEE Transactions on Information Theory, vol. 44, no. 6, pp. 2619-2692, October, 1998.

[5] Blachman, N. M., “Communication as a game”, IRE Wescon 1957 Conference Record, vol. 2, pp. 61-66, 1957.

[6] Blackwell, D., Breiman, L., Thomasian, A. J., “The capacity of a class of channels”, Ann. Math. Stat., vol. 30, pp. 1229-1241, 1959.

[7] Charalambous, C. D., Denic, S. Z., Djouadi, S. M. "Robust Capacity of White Gaussian Noise Channels with Uncertainty", accepted for 43th IEEE Conference on Decision and Control.

References

74

References

[8] Csiszar, I., Korner, J., Information theory: Coding theorems for discrete memoryless systems. New York: Academic Press, 1981.

[9] Csiszar, I., Narayan P., “Capacity of the Gaussian arbitrary varying channels”, IEEE Transactions on Information Theory, vol. 37, no. 1, pp. 18-26, Jan., 1991.

[10] Denic, S. Z., Charalambous, C. D., Djouadi, S.M., “Capacity of Gaussian channels with noise uncertainty”, Proceedings of IEEE CCECE 2004, Canada.

[11] Denic, S.Z., Charalambous, C.D., Djouadi, S.M., “Robust capacity for additive colored Gaussian uncertain channels,” preprint.

75

[12] Dobrushin, L. “Optimal information transmission through a channel with unknown parameters”, Radiotekhnika i Electronika, vol. 4, pp. 1951-1956, 1959.

[13] Doyle, J.C., Francis, B.A., Tannenbaum, A.R., Feedback control theory, New York: McMillan Publishing Company, 1992.

[14] Forys, L.J., Varaiya, P.P., “The -capacity of classes of unknown channels,” Information and control, vol. 44, pp. 376-406, 1969.

[15] Gallager, G.R., Information theory and reliable communication. New York: Wiley, 1968.

References

76

[16] Hughes, B., Narayan P., “Gaussian arbitrary varying channels”, IEEE Transactions on Information Theory, vol. 33, no. 2, pp. 267-284, Mar., 1987.

[17] Hughes, B., Narayan P., “The capacity of vector Gaussian arbitrary varying channel”, IEEE Transactions on Information Theory, vol. 34, no. 5, pp. 995-1003, Sep., 1988.

 [18] Lapidoth, A., Narayan, P., “Reliable communication under channel uncertainty,” IEEE Transactions on Information Theory, vol. 44, no. 6, pp. 2148-2177, October, 1998.

[19] Medard, M., “Channel uncertainty in communications,” IEEE Information Theory Society Newsletters, vol. 53, no. 2, p. 1, pp. 10-12, June, 2003.

References

77

[20] Root, W.L., Varaiya, P.P., “Capacity of classes of Gaussian channels,” SIAM J. Appl. Math., vol. 16, no. 6, pp. 1350-1353, November, 1968.

[21] Wolfowitz, Coding Theorems of Information Theory, Springer – Verlang, Belin Heildelberg, 1978.

[22] McElice, R. J., “Communications in the presence of jamming – An information theoretic approach, in Secure Digital Communications, G. Longo, ed., Springer-Verlang, New York, 1983, pp. 127-166.

[23] Diggavi, S. N., Cover, T. M., “The worst additive noise under a covariance constraint”, IEEE Transactions on Information Theory, vol. 47, no. 7, pp. 3072-3081, November, 2001.

References

78

[24] Vishwanath, S., Boyd, S., Goldsmith, A., “Worst-case capacity of Gaussian vector channels”, Proceedings of 2003 Canadian Workshop on Information Theory.

[25] Shannon, C.E., “Mathematical theory of communication”, Bell Sys. Tech. J., vol. 27, pp. 379-423, pp. 623-656,July, Oct, 1948

References