predictive coding i

8/10/2019 Predictive Coding i

1/14

PREDICTIVE CODING I


2/14

Predictive CodingMakes use of the past history of the data being encoded.

Principally used for text and images

JPEG 2000 : Lossless method Predictive coding

In a message, if certain symbols occur with much higher probability than others (skewed) more compression can be achieved

One way transform the sequence into another sequencethat has the desired property- If the transformation is basedon the history of the sequence such schemes are called

predictive schemes.


3/14

Predictive Coding - Example1, 2, 5, 7, 2, -2, 0, -5, -3, -1, 1, -2 ,-7, -4, -2, 1, 3, 4

Entropy is 4 bits/symbolIf Huffman or arithmetic Min. 4 bits/symbol needed

If we form another sequence by adding 2 to the previousno. in the sequence and send the difference between thecurrent element in the sequence and this predicted value

Transmitted sequence will be

1, -1, 1, 0, -7, -6 , 0, -7, 0, 0, 0, -5, -7, 1, 0, 1, 0, -1

If this rule is known to the decoder, decoding can be donefrom the above residual sequence whose entropy is verymuch less than the original sequence. Hence more

compression can be achieved


4/14

Prediction SchemesUse the preceding symbols to predict the

probability model and with this probability model

(Cum.Distr.Function) arithmetic coding can bedone.

Adaptive Arithmetic Coding : Probability modelchanges while encoding process.

Preceding symbols form the context .


5/14

Prediction SchemesConsider the encoding of the word probability . Suppose wehave already encoded the first 4 letters, and we want to codethe fifth letter a

1. First order context for a b2. Second order context for a ob3. Third order context for a rob4. Fourth order context for a prob

Encode each letter using the probability of its occurrence

w.r.t a context of high orderBut we cannot find the contexts for all symbols of thealphabet at the beginning itself.We use a prediction with partial match (ppm)Prediction assigning the probability to a symbol based onits context


6/14

Prediction SchemesPPM (Prediction with Partial Match)

1. Best known context based algorithm2. Use large contexts to determine the prob. of the

symbol being encoded3. Estimates the probabilities as the coding

proceeds.4. Instead of storing all possible contexts, we store

those contexts that have occurred in thesequence being encoded.


7/14

Prediction with Partial Match (ppm)At the beginning we have to code lettersthat have not occurred previously in thiscontextTo handle this, source coder alphabetalways contains an escape symbol whichis used to signal that the letter to be encodedhas not seen in this context.


8/14

Prediction with Partial Match (ppm)BASIC ALGORITHM

The size of the largest context is fixed first.

If the symbol to be encoded is not previously encountered in thiscontext, an escape symbol is encoded.

Size of the context is reduced.

Continues until we obtain a context that has previously beenencountered with this symbol or we arrive at a conclusion that thesymbol has not been encountered previously in any context then

probability =1/M is assigned to encode that symbol where M is thesize of the alphabet.


9/14

ppm ExampleCoding a of probability Highest context =4First checks whether a had previously occurredwith the context prob If not, code Escape symbol and switch to nextlower context i.e., 3 checks whether a had

previously occurred after rob Therefore the development of probabilities w.r.tcontext is an adaptive process.


10/14

ExampleThe following table shows contexts andfrequency counts for orders 4 through 0after the 11 symbol string xyzzxyxyzzx has

been input and encoded.


11/14

Example

Building context table


12/14

Example Building context table

Let the 12th symbol is z. The updated table is shown below


13/14

Example

Encode the sequence assanissimassa

The table shows the contexts upto order 2collected while reading and encoding the 14symbol string


14/14

PPM - Encoding

predictive coding i

Documents