Reliability and Channel Coding

How fast can we send information over a link?

• Key channel properties: The bandwidth (B), single strength (S), and noise strength (N)– B limits the rate of transmissions– S and N limit how many signal levels we can

distinguish

Bandwidth B Signal S,Noise N

Nyquist Theorem

• The maximum symbol rate is 2B

• Thus, if there are K signal levels, ignoring noise, the maximum bit rate is:

0 1 0 1 0 1 0 1

The Effect of Noise on Communication

• In practice, the signal levels we can distinguish depends on S/N– Or SNR, the Signal-to-Noise Ratio– SNR often given on a log-scale in deciBels:• dB = 10log10(S/N)

• Shannon’s Theorem

Calculate Channel Capacity with S/N in dB

• The voice telephone system:– Signal-to-noise ratio: about 30 dB– An analog bandwidth: about 3 kHz

• Calculation– Step 1: Converting the S/N in dB into a simple

fraction: S/N = 10(dB/10)

• 30dB 1000

– Step 2: Applying Shannon's Theorem

about 30,000 bps

dB = 10log10(S/N)

Exercise• If two signal levels are used, what is

the data rate that can be sent over a coaxial cable that has an analog bandwidth of 6.2 MHz?

• If a system has an average power level of 100, an average noise level of 33.33, and a bandwidth of 100 MHz, what is the effective limit on channel capacity?

• If a telephone system can be created with a signal-to-noise ratio of 40 dB and an analog bandwidth of 3000 Hz, how many bits per second could be transmitted?

The Significance of Channel Capacity• Nyquist's work has provided an incentive to

explore complex ways to encode bits on signals.

• Shannon's Theorem is more fundamental because it represents an absolute limit derived from the laws of physics.

• Engineers recognize that faster transmission speeds will only be possible if the signal-to-noise ratio can be improved.

Errors Happen!

Illustration of a burst error that results from interference

Channel Coding: Strategies to Handle Channel Errors

• Forward Error Correction (FEC)– Adding informaion on the sender side– Detecting and correcting errors on the receiver side

• Automatic Repeat reQuest (ARQ)– Requiring cooperation and further message

exchanges between the sender and receiver.

Forward Error Correction

Using Error Codes

• Codeword consists of dataword D plus check bits R

D R=fn(D)

• Sender:– Compute check bits R based on the dataword D;

send the codeword D+R

Dataword Check bits

Using Error Codes

• Receiver:– Receive codeword D’+R’ with unknown errors

• The check bits could contain errors, too!

– Recompute R check bits based on the dataword D’; error if R does not match R’

D’ R’Dataword Check bits

R=fn(D’)=?

Example: Single Parity Checking

• Adding an extra bit to each byte to make the number of 1 bits even (Even Parity) or odd (Odd Parity)

How well does parity work?

• If one of the nine bits is changed, will the error be detected?

• How about if two bits are changed?

• What about larger errors (3, 4, 5,… bits are changed)?– Can detect if odd number of bits are changed

How to Choose Coding Schemes?

• An ideal coding scheme:– Any change of bits in a valid data leads to an invalid

result.

• In practice:– What is the minimum number of bits of a valid

codeword that must be changed to produce another valid codeword?

Hamming distance

• Distance between two codewords D and D’ is the number of bit flips needed the change D to D’

Hamming distance

• Hamming distance of a code is the minimum distance between any pair of codewords– Example: Single parity checking:

Hamming distance = 2

Hamming Distance

• Error detection:– For a code of distance d+1, up to d errors will

always be detected

• Error correction:– For a code distance 2d+1, up to d errors can

always corrected by mapping to the closest codeword

.

Error Correction with Row and Column (RAC) Parity

• If an error is detected, how do we know which bit went wrong?

• Adding bits for both rows and columns

.

Correcting a Single-Bit Error

Row and Column (RAC) Parity

• If one of the nine bits is changed, will the error be detected?

• How about if two bits are changed?

• What about larger errors (3, 4, 5,… bits are changed)?– Can detect if odd number of bits are changed

The 16-Bit Checksum Used in the Internet

• The Internet checksum treats data in a message as a series of 16-bit integers.– The size of a dataword can vary.– Adding 0s if the data is not an exact multiple of 16

bits.

The 16-Bit Checksum Used in the Internet

0100 1000 0110 01010110 1100 0110 1100

+ 0110 1111 0010 0001 1 0010 0011 1111 0010

0010 0011 1111 0010 + 1

0010 0011 1111 0011

1101 1100 0000 1100

Step 1: add 16-bit values

Step 2: add overflow

Step 3: invert result

Overflow

The 16-Bit Checksum Used in the Internet

• How well does the checksum work?– What is the Hamming distance of the code?– How many errors will it detect/correct?

• What about larger errors?– Correct all burst errors up to 16 bits!