1 jayanta mukhopadhyay department of computer science & engineering indian institute of...
TRANSCRIPT
![Page 1: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/1.jpg)
1
Jayanta MukhopadhyayDepartment of Computer Science & Engineering
Indian Institute of Technology, Kharagpur, 721302, India
Sanjit K. MitraMing Hsieh Dept. of Electrical Engineering
University of Southern California
Los Angeles, CA 90089, USA
COLOR ENHANCEMENT IN THE COMPRESSED DOMAIN
![Page 2: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/2.jpg)
2
Image Enhancement: An Example
(a) Original (b) Enhanced
![Page 3: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/3.jpg)
3
Enhancement Factors
• Dynamic Range of Intensity Values. Limited number of bit-planes.
• Varying illumination over the image space.
• Brightness
• Contrast
• Color
![Page 4: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/4.jpg)
4
Enhancement Techniques
• Spatial Domain Techniques.
Work with the pixels values. Histogram Stretching, Pixel Mapping, Image Sharpening
• Compressed Domain Techniques. Works with DCT coefficients
![Page 5: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/5.jpg)
5
Let x (m,n), 0 m,n N-1 be a 2-D image of size NxN. Its N-point DCT C(l,k), 0 l,k N-1 is defined as:
DCT: Definition
1 1
0 0
2 (2 1) (2 1)( , ) ( ) ( ) ( , ) cos cos
2 2
N N
m n
n l n kC l k l k x m n
N N N
otherwise 1
0for 2
1)(
pp
![Page 6: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/6.jpg)
6
Motivations
Computation with reduced storage. Avoid overhead of entropy decoding
and encoding. Exploit spectral factorization for
improving the quality of result and speed of computation.
![Page 7: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/7.jpg)
7
Existing Approaches
• Alpha Rooting Aghaglzadeh and Ersoy (1992), Opt.Engg• Multi Contrast Enhancement Tang, Peli and Acton (2003), IEEE SPL
A spectral band is defined with Yijs such that i+j=n; 0<n<15
Contrast measure (Hn)= Cumulative Energy Sum till nth band in original image ----------------------------------------------------------------------- Cumulative Energy Sum till nth band in enhanced image Each coefficient in a band is scaled by : Yij = λ Hn. Yij, , i+j=n
![Page 8: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/8.jpg)
8
Existing Approaches
• Multi-Contrast Enhancement with Dynamic Range Compression (S. Lee (2007), IEEE CSVT)
Modification of DC coefficients and AC coefficients (following similar strategy of multi-contrast enhancement).
Normalized DC coefficients (x) are modified as follows:
![Page 9: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/9.jpg)
9
Proposed Approach
• Adjust background illumination. Use DC coefficients of the Y component.
• Preserve Local Contrast. Scale AC coefficients of the Y component appropriately.
• Preserve Colors. Preserve Color Vectors in the DCT domain. DCT coefficients of Cb and Cr components.
![Page 10: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/10.jpg)
10
Let μ and σ denote the mean and standard deviation of
an image. Contrast ζ of an image is defined here as: .
Contrast : Definition
Weber Law:
where is the difference in luminance between a stimulus and its surround, and L is the luminance of the surround
L
L
L
![Page 11: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/11.jpg)
11
Theorem on Contrast Preservation in the DCT Domain
Let d be the scale factor for the DC coefficient and a a be the scale factor for the AC coefficients of a DCT block Y. The processed DCT block Ye is given by:
( , ), 0( , ),( , ) { d
a
Y i j i je Y i j otherwiseY i j
The contrast of the processed image then becomes a / d times of the contrast of the original image.
In this algorithm d = a = for preservation of the contrast.
![Page 12: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/12.jpg)
12
Preservation of Colours in the DCT Domain
Let U and V be the DCT coefficients of the Cb and Cr components, respectively. If the luminance component Y of an image is uniformly scaled by a factor , the colors of the processed image with Ye , Ue and Ve are preserved by the following operations:
( , )( ( 128)) 128, 0
( , ),( , ) {U i j
N i jN
e U i j otherwiseU i j
( , )( ( 128)) 128, 0
( , ),( , ) {V i j
N i jN
e V i j otherwiseV i j
![Page 13: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/13.jpg)
13
Enhancement by Scaling Coefficients
• Find the scale factor by mapping the DC coefficient with a monotonically increasing function.
• Apply scaling to all other coefficients in all the
components.
• For blocks having greater details, apply block
decomposition and re-composition strategy.
![Page 14: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/14.jpg)
14
Mapping functions for adjusting the local background illumination
( (0,0))
(0,0)
f Y
Y
(TW)
Mitra and Yu , CVGIP’87
(DRC)
Lee, CSVT’07
(SF)
De, TENCON’89
![Page 15: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/15.jpg)
15
Monotonic Mapping Functions
![Page 16: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/16.jpg)
16
Scaling only DC coefficients
![Page 17: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/17.jpg)
17
Scaling both DC and AC coefficients
![Page 18: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/18.jpg)
18
Preservation of Contrast and Color
original
![Page 19: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/19.jpg)
19
Block Decompos.
Smaller DCT blocks
8x8 block
Apply CES on smaller
blocks
Block Composition
Enhanced Block
Enhancement of Blocks with more details
![Page 20: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/20.jpg)
20
Removal of Blocking Artifacts
original
![Page 21: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/21.jpg)
21
Some Results
original AR MCE
MCEDRC TW-CES-BLK MSR
![Page 22: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/22.jpg)
22
Enhancement near Edges
AR MCE MCEDRC
TW-CES-BLK DRC-CES-BLK SF-CES-BLK
![Page 23: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/23.jpg)
23
Some Results
original AR MCE
MCEDRC TW-CES-BLK MSR
![Page 24: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/24.jpg)
24
Enhancement near edges
AR MCE MCEDRC
TW-CES-BLK DRC-CES-BLK SF-CES-BLK
![Page 25: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/25.jpg)
25
Some Results
original AR MCE
MCEDRC TW-CES-BLK MSR
![Page 26: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/26.jpg)
26
Enhancement near edges
AR MCE MCEDRC
TW-CES-BLK DRC-CES-BLK SF-CES-BLK
![Page 27: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/27.jpg)
27
Metrics for Comparison
2
2 2
4 xy
x y
x yQM
x y
Wang and Bovic (SPL, 2002)
JPEG Quality Metric (JPQM) Wang and Bovic (ICIP,2002)
2 2 2 20.3CM Susstrunk and Winkler (SPIE, 2004)
R G
2
R GB
![Page 28: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/28.jpg)
28
Other Approaches
• Alpha Rooting (AR) : Aghaglzadeh and Ersoy (1992), Opt.Engg.
• Multi-Contrast Enhancement (MCE): Tang, Peli and Acton (2003), IEEE SPL
• Multi-Contrast Enhancement with Dynamic Range Compression (MCEDRC):
S. Lee (2007), IEEE CSVT
• Contrast Enhancement by Scaling (CES): Proposed work
• Multi-Scale Retinex (MSR) (a reference spatial domain technique):
Jobson, Rahman and Woodell (1997), IEEE IP
![Page 29: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/29.jpg)
29
Average Performance Measures
Techniques JPQM CEF Y-QM
Cb-QM
Cr-QM
AR 8.58 0.97 0.80 0.67 0.67
MCE 7.00 0.94 0.76 0.67 0.67
MCEDRC 7.92 0.97 0.86 0.67 0.67
TW-CES-BLK
7.79 1.50 0.90 0.82 0.81
DRC-CES-BLK
8.16 1.18 0.86 0.76 0.76
SF-CES-BLK
8.13 1.25 0.89 0.78 0.77
![Page 30: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/30.jpg)
30
Computational Complexities
Techniques
Per Pixel Operations
AR 1E + 1M
MCE 2.19M+1.97A
MCEDRC 0.03E+3.97M+2A
TW-CES 0.02E+4.02M+1.05A
DRC-CES 0.05E+4M+1.08A
SF-CES 0.03E+4.02M+1.06A
MSR 18E+1866378M+8156703A
aE+bM+cA implies a Exponentiation, b Multiplication and c Addition operations.
![Page 31: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/31.jpg)
31
Iterative Enhancement
original
Iteration no.=1
Iteration no.=2
Iteration no.=3
Iteration no.=4
![Page 32: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/32.jpg)
32
Conclusion
• A novel algorithm for enhancement of color images in the block DCT domain.
• It takes care of adjustment of back-ground illumination, local contrast, as well as preservation of colours.
• Comparative study shows the superiority of the proposed scheme compared to some of the existing schemes.
![Page 33: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/33.jpg)
33
References
• [1] J. Mukherjee and S.K.Mitra, Enhancement of Color Images by
Scaling the DCT coefficients, IEEE trans. on Image Processing, vol. 17, no. 10, Oct., pp. 1783-1794, 2008. • [2] J. Mukherjee and S.K. Mitra, Color Enhancement in the Compressed Domain, Int. Conf. on Image Proc. (ICIP-2008), San Diego, USA, Oct. 12 -15, pp. 3144-3147, 2008 .
![Page 34: 1 Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.inay@cse.iitkgp.ernet.in](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e845503460f94b856e2/html5/thumbnails/34.jpg)
34
Thanks!