Image Processing and Coding 1 Image Rich info. from visual data Examples of images around us natural photographic images; artistic and engineering drawings scientific images (satellite, medical, etc.) Motion pictures => video movie, TV program; family video; surveillance and highway/ferry camera A picture is worth 1000 words. A video is worth 1000 sentences? 2 Image Processing Enhancement and restoration Remove artifacts and scratches from an old photo/movie Improve contrast and correct blurred images 3 Image Processing Composition (for magazines and movies), Display, Printing Transmission and storage images from oversea via Internet, or from a remote planet Information analysis and automated recognition Providing human vision to machines Face detection in CMU CS,4 An Example A Navigation system study for Accident Scene Analysis and Obstacle Avoidance Based on Mobile Multimedia sensor Platform 5 Image Processing Medical imaging for diagnosis and exploration Security, forensics and rights protection Encryption, hashing, digital watermarking, digital fingerprinting Visible Watermark invisible Watermark 6 Digital Image Exactness Perfect reproduction without degradation Perfect duplication of processing result Convenient & powerful computer-aided processing Can perform sophisticated processing through computer hardware or software Even kindergartners can do some! Easy storage and transmission 1 CD can store hundreds of family photos! Paperless transmission of high quality photos through network within seconds 7 Examples of Digital Image Processing Compression Manipulation and Restoration Restoration of blurred and damaged images Noise removal and reduction 8 Examples of Digital Image Processing Applications Face detection Visible and invisible watermarking Error concealment and resilience in video transmission 25% blocks in a checkerboard pattern are corrupted corrupted blocks are concealed via edge- directed interpolation (a) original lenna image (c) concealed lenna image (b) corrupted lenna image 9 International Conf. on Image Processing 10 What is An Image? 11 Grayscale image A grayscale image is a function I(x,y) of the two spatial coordinates of the image plane. I(x,y) is the intensity of the image at the point (x,y) on the image plane. I(x,y) takes non-negative values assume the image is bounded by a rectangle [0,a] [0,b] Color image Can be represented by three functions, R(x,y) for red, G(x,y) for green, and B(x,y) for blue. y x y x I(x, y) Matlab Example x=imread('lena.bmp'); imshow(x); [n,m]=size(x); x(5:40,5:40)=0; imshow(x) x(90:150, 90:150)=255; Imshow(x) 12 Different Ways to View an Image (More generally, to view a 2-D real- valued function) In 3-D (x,y, z) plot with z= I (x,y); red color for high value and blue for low Equal value contour in (x,y) plane Intensity visualization over 2-D (x,y) plane 13 Sampling and Quantization Computer handles discrete data. Sampling Sample the value of the image at the nodes of a regular grid on the image plane. A pixel (picture element) at (i, j) is the image intensity value at grid point indexed by the integer coordinate (i, j). Quantization Is a process of transforming a real valued sampled image to one taking only a finite number of distinct values. Each sampled value in a 256-level grayscale image is represented by 8 bits. 0 (black) 255 (white) 14 Image Compression Savings in storage and transmission Multimedia data (esp. image and video) have large data volume Difficult to send real-time uncompressed video over current network Accommodate relatively slow storage devices -they do not allow playing back uncompressed multimedia data in real time 1x CD-ROM transfer rate ~ 150 kB/s 320 x 240 x 24 fps color video bit rate ~ 5.5MB/s => 36 seconds needed to transfer 1-sec uncompressed video from CD 15 PCM coding How to encode a digital image into bits? Sampling and perform uniform quantization Pulse Coded Modulation (PCM) 8 bits per pixel ~ good for grayscale image/video bpp ~ needed for medical images Reduce # of bpp for reasonable quality via quantization Quantization reduces # of possible levels to encode Visual quantization: companding, contrast quantization, dithering, etc. Halftone use 1bpp but usually upsampling ~ saving less than 2:1 16 DiscussiononImprovingPCMDiscussion on Improving PCM Quantized PCM values may not be equally likely Can we do better than encode each value using same # bits? Example P(0 ) = 0.5, P(1) = 0.25, P(2) = 0.125, P(3) = If use same # bits for all values Need 2 bits to represent the four possibilities if treat If use less bits for likely values 0 ~ Variable Length Codes (VLC) 0 => [0], 1 => [10], 2 => [110], 3 => [111] Use 1.75 bits on average ~ saves 0.25 bpp! Bring probability into the picture Now use prob. distr. to reduce average # bits per quantized sample 17 Variable Length Code 18 If the probabilities of (a, b, c, d) were (1/2,1/4,1/8,1/8), the expected number of its used to represent a source symbol would be bits/per symbol: EntropycodingEntropy coding 19 E.g.ofEntropyCoding:HuffmanCodingE.g. of Entropy Coding: Huffman Coding 20 HuffmanCoding(contd)Huffman Coding (contd) 21 HuffmanCoding:Pros&ConsHuffman Coding: Pros & Cons Pro Simplicity in implementation (table lookup) For a given block size Huffman coding gives the best coding efficiency Con Need to obtain source statistics Improvement Code a group of symbols as a whole to allow fractional # bit per symbol Arithmetic coding Lempel-Ziv coding or LZW algorithm universal, no need to estimate source statistics 22 Discussion: Coding a Binary Image How to efficiently encode it? Run-length coding (RLC) Code length of runs of 0 between successive 1 run-length of 0 ~ # of 0 between 1 good if often getting frequent large runs of 0 and sparse 1 E.g., => (4) (0) (3) (1) (6) (0) (0) Assign fixed-length codeword to run-length Or use variable-length code like Huffman to further improve RLC Also applicable to general binary data sequence with sparse 1 (or 0) 23 24 Any Questions? 25 Things To Do Find your partners for the class project Feb. 11, Thursday ( me the team information) Each group has 3-4 members Check out the class website about lecture notes reading assignments Next Topics Image Compression-JPEG