channelcoding hafiz malik dept. of electrical & computer engineering the university of...
TRANSCRIPT
ChannelCoding
Hafiz MalikDept. of Electrical & Computer Engineering
The University of Michigan-Dearborn [email protected]
http://www-perosnal-engin.umd.umich.edu/~hafiz
Channel Coding • class of signal transformations designed to
improve communication performance by enabling the transmitted signals to better withstand channel distortions such as noise, interference, and fading.
• Channel coding can be divided into two major classes:
1. Waveform coding by signal design2. Structured sequences by adding redundancy
Waveform Coding– deals with transforming waveform into “better
waveform” robust to channel distortion hence improving detector performance.
• Examples:– Antipodal signaling– Orthogonal signaling– Bi-orthogonal signaling– M-ary signaling– Trellis-coded modulation
Structured Sequences– deals with transforming sequences into “better
sequences” by adding structured redundancy (or redundant bits). The redundant bits are used to detect and correct errors hence improves overall performance of the communication system.
• Examples:– Linear codes
• Hamming codes• BCH codes• Cyclic codes• Reed-Solomon codes
– Non-Linear codes• Convolution codes• Turbo codes
Waveform Coding
M-ary Communications• Send multiple, M, waveforms• Choose between one of M symbols instead of 1 or 0.• Waveforms differ by phase, amplitude, and/or
frequency• Advantage: Send more information at a time• Disadvantage: Harder to tell the signals apart or
more bandwidth needed
4-ary Amplitude• Each symbol sends 2 bits• Deciding which level is correct
gets harder due to fading and noise
• RCV needs better SNR to achieve accuracy
11
01
10
00
Orthogonal Signals
• Definition
• This means that the signals are perpendicular to each other in M-dim space
• For a correlative receiver, this means that each incoming signal can be compared with a model of the signal and the best match is the symbol that was sent
0 0
bT
i j
C i jp t p t dt
i j
Multi-Amplitude Shift Keying (MASK)
• Send multiple amplitudes to denote different signals
• Typical signal configuration:– +/- p(t)– +/- 3 p(t)
– +/- (M-1) p(t)
Multi-Phase• Binary Phase Shift Keying (BPSK)1: (t)= p(t) cos(ct)
0: (t)= p(t)cos(ct
• M-ary PSK
Re
Im
x x
2cosk cp t p t t k
M
Re
Im
x x
x x
x x
x
x
Quadrature Amplitude Modulation (QAM)
• Amplitude-phase shift keying (APK)
cos sin
cos
k k c k c
k c k
p t p t a t b t
p t r t
2 2 tan kk k k k
k
br a b
a
ri
i
Multitone Signaling (MFSK)
• M symbols transmitted by M orthogonal pulses of frequencies:
• Receiver is a bank of mixers, one at each frequency
• Higher M means wider bandwidth needed or tones are closer together
2 /k MN k T
M-ary Comments• As M increases, it is harder to make good
decisions, more power is used• But, more information is packed into a symbol
so data rates can be increased• Generally, higher data rates require more
power (shorter distances, better SNR) to get good results
How do we compare performance?• Symbols have different meanings, so what
does the probability of error, PE mean?
• If a detection error is made, then more than one bit is wrong
• DCS can be faster at the price of being less sensitive
15
Detect Error On Credit Card
16
Formula for detecting error
Let d2, d4, d6, d8, d10, d12, d14, d16 be all the even values in the credit card number.
Let d1, d3, d5, d7, d9, d11, d13, d15 be all the odd values in the credit card number.
Let n be the number of all the odd digits which have a value that exceeds four
Credit card has an error if the following is true:
(d1 + d3 + d5 + d7 + d9 + d11 + d13 + d15) x 2 + n +
(d2 + d4 + d6 + d8 + d10 + d12 + d14 + d16)
0 mod(10)
17
Detect Error On Credit Card
d1
d2 d3 … d15 d16
n = 3
18
Now the test
(4 + 4 + 8 + 1 + 3 + 5 + 7 + 9) = 41
(5 + 2 + 1 + 0 + 3 + 4 + 6 + 8) x 2 + 3 = 61
41 + 61 = 102 mod (10) = 2
3
19
Credit Card Summary
The test performed on the credit card number is called a parity check equation. The last digit is a function of the other digits in the credit card. This is how credit card numbers are generated by Visa and Mastercard. They start with an account number that is 15 digits long and use the parity check equation to find the value of the 16th digit.
“This method allows computers to detect 100% of single-position errors and about 98% of other common errors” (For All Practical Purposes p. 354).
Examples
• ISBN (international standard book number)• 0 – 20 – 1 – 36186 – 8
• UPC (universal product codes)– 12-digit sequence– 0 16000 66610 8
21
What is a code?
A code is defined as an n-tuple of q elements. Where q is any alphabet.
Ex. 1001 n=4, q={1,0}
Ex. 2389047298738904 n=16, q={0,1,2,3,4,5,6,7,8,9}
Ex. (a,b,c,d,e) n=5, q={a,b,c,d,e,…,y,z}
The most common code is when q={1,0}. This is known as a binary code.
22
The purposeA message can become distorted through a wide range of unpredictable errors.
• Humans• Equipment failure• Lighting interference• Scratches in a magnetic tape
23
Why error-correcting code?
To add redundancy to a message so the original message can be recovered if it has been garbled.
e.g. message = 10 code = 1010101010
24
Send a message
Message Encoder Channel Decoder Message
10 101010 noise 001010 10
25
Encoding
Naïve approach
Hamming codes
26
Take Naïve approach
Append the same message multiple times. Then take the value with the highest average.
Message:= 1001
Encode:= 1001100110011001
Channel:= 1001100100011001
Decode: = a1 = Average(1,1,0,1) = 1
a2 = Average(0,0,0,0) = 0 ... (a1,a2,a3,a4)
Message:= 1001
Naïve Solution: Example
Single CheckSum -• Truth table:
• General form:Data=[1 1 1 1]Message=[1 1 1 1 0]
• Repeats –Data = [1 1 1 1]Message= [1 1 1 1] [1 1 1 1] [1 1 1 1]
A B X-OR0 0 00 1 11 0 11 1 0
Why Naïve Solution is Inefficient?
ratio noise tosignal theis S/N
Capacity Channel raw isW
Capacity Channel is C
/1log2 NSWC
Shannon EfficiencyRepeat 3 times:•This divide W by 3•It divides overall capacity by at least a factor of 3x.
Single Checksum:•Allows an error to be detected but requires the message to be discarded and resent. •Each error reduces the channel capacity by at least a factor of 2 because of the thrown away message.
29
Hamming [7,4] Code
The seven is the number of digits that make the code.
E.g. 0100101
The four is the number of information digits in the code.
E.g. 0100101
30
Hamming [7,4] Encoding
Encoded with a generator matrix. All codes can be formed from row operations on matrix. The code generator matrix for this presentation is the following:
kknkk PIG
:
1111000
0110100
1010010
1100001
31
Hamming [7,4] Codes
1624 1000011010010100101100001111110011010101011001100011001101010100011001110100110010101111111011110000110010000000
12827 Codes
Possible codes
Multi-dimensional Codes
Code Space:
• 2-dimensional
• 5 element states
Circle packing makes more efficient use of the code-space
Cannon Balls
• http://wikisource.org/wiki/Cannonball_stacking• http://mathworld.wolfram.com/SpherePacking.html
Efficient Circle packing is the same as efficient 2-d code spacing
Efficient Sphere packing is the same as efficient 3-d code spacing
Efficient n-dimensional sphere packing is the same as n-code spacing
34
Minimum Weight Theorem
Definitions
Proof of Theorem
35
Definitions
The weight of a code is the number of nonzero components it contains.
e.g. wt(0010110) = 3
The minimum weight of Hamming codes is the weight of the smallest nonzero vector in the code.
e.g. d(G)= 3
1111000
0110100
1010010
1100001
G
36
Definitions
The distance between two codes u and v is the number of positions which differ
e.g. u=(1,0,0,0,0,1,1)
v=(0,1,0,0,1,0,1)
dist(u,v) = 4
Another definition of distance is wt(u – v) = dist(u,v).
37
Definitions
),(),(),( wvdistvudistwudist
),(),( uvdistvudist
0),( uudist
For any u, v, and w in a space V, the following three conditions hold:
38
Definitions
The sphere of radius r about a vector u is defined as:
}),(|{)( rvudistVvuSr
e.g. u=(1,0,0,0,0,1,1) (0,0,0,0,0,1,1)
(1,1,0,0,0,1,1)
(1,0,0,0,0,0,1)
(1,0,0,0,0,0,1)
(1,0,1,0,0,1,1)
(1,0,0,1,0,1,1) (1,0,0,0,1,1,1)
39
Minimum Weight Theorem
If d is the minimum weight of a code C, then C can correct t = [(d – 1)/2] or fewer errors, and conversely.
40
Proof
Want to prove that spheres of radius t = [(d – 1)/2] about codes are disjoint. Suppose for contradiction that they are not. Let u and w be distinct vectors in C, and assume that
)()( wSuSv tt
u wv
41
Proof
By triangle inequality
twvdistvudistwudist 2),(),(),(
u wv
Since spheres of radius t = [(d – 1)/2] so and this gives
But since
We have a contradiction. Showing the sphere of radius t about codes are disjoint.
42
Proof
12 dt12 dtdist(u,w)dist(u,v)dist(u,w)
dwuwtwudist )(),(
43
Result of Theorem
Since d(G) = 3 then for t = [(3 – 1)/2] = 1 or fewer errors, the received code is in a disjoint sphere about a unique code word.
1111000
0110100
1010010
1100001
G
Parity-Check Matrix• The parity check matrix is found by solving the generator matrix for
• For each generator matrix G, there exists an (n – k) x n matrix H, such that the rows of G are orthogonal to the rows of G; i.e.,
• where HT is the transpose of H, and 0 is an k x (n – k) all zeros matrix . • The matrix H is called the parity-check matrix, that can be used to decode
the received code words.
0TGH
3343
33
:
:
1001011
0101101
0011110
xx
T
IP
IP
H
0TGH
45
Channel Decoding
1. Brut force Approach: list all possible messages
2. Syndrome Decoding
46
Brute Force Approach:List all messages
This is done by generating a list of all the possible messages. For something small like the Hamming [7,4] codes the task is feasible, but for codes of greater length it is not. An example of a list is as follows:
Code words 1000011 0100101 0010110 … 0000011 0000101 0000110
Other 1000001 0100111 0010100Received 0010011 0001101 1010110Words 1100011 1100101 0110110
… … …
47
Brute Force Approach:List all messages
For example, if the received code was 0001101 then it would be decoded to 0100101 from the list.
Code words 1000011 0100101 0010110 … 0000011 0000101 0000110
Other 1000001 0100111 0010100Received 0010011 0001101 1010110Words 1100011 1100101 0110110
… … …
48
Syndrome DecodingConsider a transmitted code cm and y is the received sequence, y can be expressed as,
where e denotes binary error vector.
The decoder calculate product
(n – k)–dimensional vector s is called the syndrome of the error pattern. Or in other words, s contains the pattern of failure in parity checks.
ecy m
tyH
t
ttm
tm
t
eH
eHHc
Hec
yHs
Standard Array• Let us arrange the 2n n-tuples that represent all possible received
sequences in an array, such that first row contains all the codewords (2k) and first column contains all correctable error patterns. The standard array format of (k,n) code is as follows:
knkknknkn
knkknknkn
k
k
k
ececece
ececece
ececece
ececece
cccc
i
i
i
i
i
222212
1221212112
222212
121111
210
Standard Array• Note that codeword c0 is trivial codeword, that is, all-
zeros codeword. In addition, it is also error pattern of all-zeros.
• Each row in the standard array is called a coset.• An error pattern in the first column, called coset leader. • A coset leader is the vector with the minimum weight in
the coset.• Each coset consists of 2k n-tuples
there are 2n –k cosets.
51
Syndrome Decoding
The first step is to create a list of syndromes corresponding the coset leaders. The syndrome of each vector y is found by
THyysyn )(
When a code is received, the syndrome is computed and compared to the list of syndromes. Let the coset leader to the syndrome by e. Finally the code is decoded to x = y – e.
52
Syndrome example
1111000
0110100
1010010
1100001
G
1001011
0101101
0011110
H
Note that G=(I |P) and H = ( | I). TP
53
Syndrome example
Let x:= 1001100 be the original message
Message Encoder Channel Decoder Message
1001 1001100 noise 1000100 ?
Compute the syndrome of the received codeTHyysyn )(
1001011
0101101
0011110
H
1
0
1
0
0
0
1
1
1
1
54
Conclusion
A code of minimum weight d is called perfect if all the vectors in V are contained in the sphere of radius t = [(d – 1)/2] about the code-word.
The Hamming [7,4] code has eight vectors of sphere of radius one about each code-word, times sixteen unique codes. Therefore, the Hamming [7,4] code with minimum weight 3 is perfect since all the vectors (128) are contained in the sphere of radius 1.