UNIVERSITI PUTRA MALAYSIA

TUNG LI QIAN

FK 2012 128

A MODIFIED CONTRAST LIMITED ADAPTIVE HISTOGRAM EQUALIZATION FOR HIGH DYNAMIC RANGE IMAGES

© CO

PYRI

GHT U

PM

A MODIFIED CONTRAST LIMITED ADAPTIVE HISTOGRAM

EQUALIZATION FOR HIGH DYNAMIC RANGE IMAGES

By

TUNG LI QIAN

Thesis Submitted to the School of Graduate Studies,

Universiti Putra Malaysia, in Fulfillment of the

Requirements for the Degree of Master of Science

February 2012

© CO

PYRI

GHT U

PM

ii

Abstract of thesis presented to the Senate of Universiti Putra Malaysia in fulfilment

of requirement of the degree of Master of Science

A MODIFIED CONTRAST LIMITED ADAPTIVE HISTOGRAM

EQUALIZATION FOR HIGH DYNAMIC RANGE IMAGES

By

TUNG LI QIAN

February 2012

Chair: Suhaidi Shafie, D.Eng

Faculty: Engineering

Tone mapping was originally used in the field of art where artists make use of a

limited palette to depict high contrast scenes. The purpose of tone mapping is to

display images or videos that correspond to the visual impression of an observer

when watching the original scene by reproducing contrast and brightness between the

original scene and the output device.

However, the dramatically reduction of contrast and brightness induce a loss of

contrast resulting in a loss of detail visibility. As a result, a fully automatic local tone

mapping algorithm was introduced to increase the local contrast and reduce the loss

of visual visibility. The algorithm was written by using MATLAB software.

The local tone mapping method is based on the modified contrast limited adaptive

histogram equalization (CLAHE) method. The tone mapping algorithm was first

© CO

PYRI

GHT U

PM

iii

compressed using global tone mapping method. The image was then segmented into

smaller parts using pixel-region-based image segmentation method. The segment

regions were processed to become masks using morphological method (opening and

closing method). Every mask’s clip limit was then decided according to the variance

and mean. This step alleviates the artifacts introduced by the original CLAHE

algorithm. Original CLAHE only have one clip limit and apply to each tile but the

modified CLAHE method applied different clip limit in different mask in order to

expand the dynamic range of the image. Lastly, the interpolation process was then

applied to the entire image to remove the artificially induced tile border.

For assessment, a subjective and an objective evaluation were conducted to evaluate

the performance of the tone mapping algorithm. For subjective evaluation a survey

was conducted to compare the fifteen images that had been processed by Reinhard’s,

Drago’s, Ward’s and proposed tone mapping operator. For objective evaluation,

visual difference predictor (VDP) was used to compare the difference between the

HDR image and the tone mapped image. The generated probability map of the

proposed tone mapped algorithm was then compared with the generated probability

map of Ward tone mapping algorithm. Both results show that the proposed tone

mapping algorithm produces images with good visual quality by retrieving more

detail and local contrast in the image if compare with other tone mapping operator.

© CO

PYRI

GHT U

PM

iv

Abstrak tesis yang dikemukakan kepada Senat Universiti Putra Malaysia sebagai

menemuhi keperluan untuk ijazah Master of Sains

SATU PENGUAHSUAI KONTRA TERHAD ADAPTIF HISTOGRAM

PENYAMAAN UNTUK JULAT DINAMIK TINGGI IMEJ

Oleh

TUNG LI QIAN

Februari 2012

Pengerusi: Suhaidi Shafie, D.Eng

Fakulti: Kejuruteraan

Pemetaan warna pada asalnya digunakan dalam bidang seni di mana artis-artis

menggunakan palet yang terhad untuk menggambarkan pemandangan kontras tinggi.

Tujuan pemetaan warna adalah untuk memaparkan imej atau video yang berpadanan

dengan kesan visual seseorang pemerhati semasa memerhatikan pemandangan asal

dengan menghasilkan semula kontra dan kecerahan di alat paparan yang bersamaan

dengan pemandangan asal. Walaubagaimanapun, pengurangan drastik kontra dan

kecerahan mendorong kepada kehilangan darjah penglihatan visual. Akibatnya,

logaritma pemetaan warna tempatan sepenuh automatik telah diperkenalkan untuk

meningkatkan kontra tempatan dan mengurangkan kehilangan penglihatan visual.

Algoritma ini ditulis dengan menggunakan perisian MATLAB.

Kaedah pemetaan warna tempatan adalah berdasarkan cara penyamaan histogram

penyesuaian kontras terhad (CLAHE) yang telah diubahsuai. Algoritma pemetaan

warna tersebut dimampatkan terlebih dahulu dengan menggunakan kaedah pemetaan

© CO

PYRI

GHT U

PM

v

warna global. Imej tersebut kemudian dibahagikan kepada bahagian yang lebih kecil

dengan menggunakan kaedah pembahagian imej berasas piksel-rantau. Rantau

bahagian telah diproses dengan menggunakan kaedah morfologi (pembukaan dan

penutupan). Had klip setiap topeng kemudiannya diputuskan mengikut varians dan

min. Langkah ini mengurangkan masalah artifak yang diperkenalkan oleh algoritma

CLAHE asal. CLAHE asal hanya mempunyai satu had klip dan digunakan atas jubin

masing-masing tetapi kaedah CLAHE yang diubahsuai menggunakan had klip yang

berbeza dalam topeng yang berbeza bagi memperluas julat dinamik imej. Akhir sekali,

proses penyelaan kemudiannya digunakan kepada seluruh imej untuk mengeluarkan

jubin sempadan buatan teraruh.

Untuk penilaian, satu penilaian subjektif and satu penilaian objektif telah dijalankan

untuk menilai prestasi algoritma pemetaan warna. Bagi penilaian subjektif, sebuah

kaji selidik telah dijalankan untuk membandingkan lima belas imej yang telah

diproses oleh Reinhard, Drago, Ward dan algoritma pemetaan warna yang

dicadangkan. Untuk penilaian objektif pula, peramal perbezaan visual telah

digunakan untuk membandingkan beza antara imej HDR and imej pemetaan warna.

Peta kebarangkalian yang dihasilkan oleh algoritma pemetaan warna yang

dicadangkan telah dibandingkan dengan peta kebarangkalian yang dihasilkan oleh

algoritma pemetaan warna Ward. Kedua-dua hasil menunjukkan algoritma pemetaan

warna yang dicadangkan menghasilkan imej dengan kualiti visual yang baik dengan

mengembalikan lebih banyak butiran and kontras tempatan dalam imej berbanding

dengan algoritma pemetaan warna yang lain.

© CO

PYRI

GHT U

PM

vi

ACKNOWLEDGEMENTS

During the course of my Master, many different persons that I have worked with

contributed to the outcome of this thesis. First would like to thank Dr Suhaidi Shafie

who took me as a Master student and then, I gave me the freedom to work during the

project. I would like to thank for his valuable guidance, support and encouragement

during my study. His professional review has helped this thesis to be improved.

Then, I would like to thank Prof Madya Dr M. Iqbal Saripan for his useful comment

and help and for his efforts in reading through the thesis. His technical review had

helped a lot to improve the thesis.

I would like to thanks my friends that work together in the KEE 008b Lab for the

excellent work atmosphere and for their help in all domains.

Besides, I would like to thank my family and friends who always encouraged me in

pursuing my studies. Finally, thanks to all those who contributed in one way or

another to the success of this research.

© CO

PYRI

GHT U

PM

vii

I certify that a Thesis Examination Committee has met on 22 February 2012 to conduct the final examination of Tung Li Qian on his (or her) thesis entitled " Modified Contrast-Limited Adaptive Histogram Equalization For High

Dynamic Range Images" in accordance with the Universities and University Colleges Act 1971 and the Constitution of the Universiti Putra Malaysia [P.U.(A) 106] 15 March 1998. The Committee recommends that the student be awarded the Master of Science.

Members of the Thesis Examination Committee were as follows:

Mohammad Hamiruce Marhaban, PhD

Prof. Madya Faculty of Engineering Universiti Putra Malaysia (Chairman) Maryam binti Mohd Isa, PhD Senior Lecturer

Faculty of Engineering Universiti Putra Malaysia (Internal Examiner) Dr Wan Zuha Wan Hasan, PhD Senior Lecturer

Faculty of Engineering Universiti Putra Malaysia (Internal Examiner) Md. Mamum Ibne Reaz, PhD

Prof. Madya Name of Department and/or Faculty Univesiti Kebangsaan Malaysia Malaysia (External Examiner)

SEOW HENG FONG, PhD

Professor and Deputy Dean School of Graduate Studies Universiti Putra Malaysia Date:

© CO

PYRI

GHT U

PM

viii

This thesis was submitted to the Senate of Universiti Putra Malaysia and has been

accepted as fulfilment of the requirement for the degree of Master of Science. The

members of the Supervisory Committee were as follows:

Suhaidi Shafie, D. Eng

Senior Lecturer

Faculty of Engineering

Universiti Putra Malaysia

(Chairman)

M. Iqbal SARIPAN, PhD Senior Lecturer

Faculty of Engineering

Universiti Putra Malaysia

(Member)

_______________________

BUJANG B KIM HUAT, PhD

Professor and Dean

School of Graduate Studies

Universiti Putra Malaysia

Date: 22 February 2012

© CO

PYRI

GHT U

PM

ix

DECLARATION

I declare that the thesis is my original work except for quotations and citations, which

have been duly acknowledged. I also declare that it has not been previously, and is

not concurrently, submitted for any other degree at Universiti Putra Malaysia or at

any other institution.

__________________________

TUNG LI QIAN

Date: 22 February 2012

© CO

PYRI

GHT U

PM

x

TABLE OF CONTENTS

Page

ABSTRACT ii

ABSTRAK iv

ACKNOWLEDGEMENTS vi

APPROVAL vii

DECLARATION ix

LIST OF TABLES xi

LIST OF FIGURES xii

LIST OF ABBREVIATIONS xvii

CHAPTER

1 INTRODUCTION 1

1.1 Tone Mapping 1

1.1.1 Human Visual System 2

1.1.2 The Dynamic Range of Scenes and Displays 5

1.2 The Global and Local Tone Mapping 6

1.3 Problem Statement 8

1.4 Project Overview 8

1.5 Scope of Work 9

1.6 Objectives 10

1.7 Thesis Contribution 11

1.8 Thesis Organization 11

1.9 Conclusion 12

2 LITERATURE REVIEW

2.1 Introduction 13

2.2 Simple Global Tone Mapping 14

2.2.1 Logarithm 14

2.2.2 Gamma Correction 16

2.3 Sophisticated Global Tone Mapping 22

2.4 Local Tone Mapping 22

2.5 Histogram 26

© CO

PYRI

GHT U

PM

xi

2.5.1 Histogram Equalization 28

2.5.2 Contrast Limited Adaptive Histogram Equalization 33

2.6 Histogram Equalization Based Tone Mapping Operator 37

2.7 Visual Different Predictor 42

2.8 Conclusion 45

3 METHODOLOGY

3.1 Overview 46

3.2 Global Tone Mapping 48

3.3 Modified CLAHE 54

3.3.1 Image Segmentation 54

3.3.2 Morphological Opening and Closing 60

3.3.3 Block Size and Padding 65

3.3.4 Clip Limit 70

3.3.5 Interpolation 73

3.4 Rendering 76

3.5 Conclusion 77

4 RESULTS AND DISCUSSIONS

4.1 Assessment of Tone Mapping Operator 78

4.2 Subjective Evaluation 81

4.2.1 Outdoor Night Scene 81

4.2.2 Outdoor Day Scene 90

4.2.3 Indoor Scene 99

4.2.4 Indoor with Outdoor/ Outdoor with Indoor Scene 108

4.3 Overall Subjective Evaluation Performance 115

4.4 Objective Evaluation 118

4.4.1 Outdoor Night Scene 119

4.4.2 Outdoor Day Scene 119

4.4.3 Indoor Scene 122

4.4.4 Outdoor with Indoor Scene 125

5 CONCLUSION AND FUTURE WORK

5.1 CONCLUSIONS AND FUTURE WORKS 128

REFERENCES 130

APPENDICES 134

© CO

PYRI

GHT U

PM

xii

LIST OF TABLE

Table Page

2.1: Summary of CLAHE process (MathWorks, Oct 2003) 38

2.2: Colour coding for probability detection 44

4.1: HDR image’s minimum luminance, maximum luminance

and the dynamic range for outdoor night scene 82

4.2: The statistic results of the psychophysical experiment for outdoor

night scene 88

4.3: HDR image’s minimum luminance maximum luminance and

the dynamic range for outdoor day scene 90

4.4: The statistic results of the psychophysical experiment for

Outdoor Day Scene 97

4.5: HDR image’s minimum luminance maximum luminance

and the dynamic range for indoor scene 99

4.6: The statistic results of the psychophysical experiment for Indoor Scene 106

4.7: HDR image’s minimum luminance maximum luminance and

the dynamic range for Indoor with Outdoor/ Outdoor with Indoor Scene 108 4.8: The statistic results of the psychophysical experiment for

Indoor with Outdoor /Outdoor with Indoor Scene 114 4.9: Summarizes of the result for the experiment 117

4.10: Colour coding for probability detection 119

© CO

PYRI

GHT U

PM

xiii

LIST OF FIGURES

Figure Page

1.1: Examples of scenes and their approximate level 2

1.2: The general plan of the human eye and the retina with

the names of their principal parts. 4

1.3: Perceptual match between a real world scene and displayed image. 4

1.4: Left: Image that is captured by the camera 4

2.1: Approximation of the luminances non-linear encoding performed

by the HVS. The was taken from 15

2.2 : Example of black level error (dotted red curves) versus

theoretical model (solid blue curve) from (MEYLAN, 2006) 17

2.3: Histogram for different key (low, normal and high key) with

the corresponding image 20

2.4: The application of gamma correction on an image with γ=1 (top),

γ=2.2(middle) and γ=3(bottom). 21

2.5: Four basic image types histogram (Woods, 2001) 30

2.6: (a) Original image (b) The histogram equalized result 34

2.7: Image histograms before and after equalization 35

2.8: Principle of contrast limiting as used with CLAHE. (a) Histogram of a

contextual region (b) Calculated cumulative histogram

(c) Clipped histogram. (d) Cumulative clipped histogram;

its maximum slope (equal to the contrast enhancement obtained) is equal 37

2.9: Comparison between the (a) standard histogram equalization

method with the (b) CLAHE result 39

2.10: Data flow diagram of the High Dynamic Range Visible

Difference Predictor (HDR VDP) 42

file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225358file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225358file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225359file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225360file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225360file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225361file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225361file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225362file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225362file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225363file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225363file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225364file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225365file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225366file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225367file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225367file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225367file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225367file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225368file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225368

© CO

PYRI

GHT U

PM

xiv

2.11: HDR VSP with two images as input (HDR image (Upper Left) and tone

mapped image (Lower Left)) and generated map of probability values (Right). 44

3.1: Framework of the proposed method 47

3.2 (a) Original image of the HDR image. (b) Luminance map 49

3.3: HDR mapped using Eq. 3.5 with different of τ. 52

3.4: Luminance map of the image after the logarithm compression 53

3.5: LDR image after the logarithm compression 53

3.6: (a) Partitioned image 3.6(b) corresponding quadtree 55

3.7: Image segmentation by a split and merge procedure.

(a) Original image (b) through (f) results of segmentation using function

splitmerge with values of mindim equal to 2, 4, 8, 16 and 32 respectively. 59

3.8: (a) Structuring element B “rolling” along the inner boundary of

A (the dot indicates the origin of B). (b) Structuring element B.

(c) The heavy line is the outer boundary of the opening.

(d) Complete opening (shaded) (González and Woods, 2002). 60

3.9: (a) Structuring element B “rolling” on the outer boundary of set A.

(b) The heavy line is the outer boundary of the closing.

(c) Complete closing (shaded). (González and Woods, 2002). 61

3.10: The structuring object for diamond shape. 62

3.11: The structuring object for square shape 63

3.12: Example of the image before operation of morphological

opening and closing 64

3.13: Example of the image after operation of morpological closing 64

3.14: Example of the image after the operation of morphological opening 65

3.15: The padding process 67

3.16: The example of padding 68

3.17: Regular masks 69

file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225371file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225372file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225373file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225374file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225375file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225376file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225377file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225377file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225377file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225378file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225378file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225378file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225378file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225379file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225379file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225379file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225380file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225381file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225382file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225382file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225383file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225384file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225385file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225386file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225387

© CO

PYRI

GHT U

PM

xv

3.18: The example of different mask corresponding to the global

tone mapped image 72

3.19: Image before the image interpolation process 74

3.20: Example of image interpolation 75

3.21: Image after the interpolation process 75

3.22: The output image after the tone mapping process 76

4.2: The HDR image of Montreal’s night scene that cannot be fully reproduced 83

4.1The HDR image of clockbui that cannot fully reproduce by LDR display 83

4.3: HDR image of the Venice’s Night scene that cannot be fully reproduced 84

4.4: Comparisons of the four different rendering of the Clockbui.

85

4.5: Comparisons of the four different rendering of the Montreal’s night

scene image. 86

4.6: Comparisons of the four different rendering of the Venice night.

87

4.7: Column charts of the statistic results for Outdoor Night Scene 89

4.8: The overall performance of tone mapping operator for Outdoor Night Scene 89

4.9: The HDR image of Mountain Tam West that cannot be fully

reproduced by LDR display 91

4.10: The HDR image of Peck Lake that cannot fully be reproduced

by LDR display 91

4.11: The HDR image of Spherion Price Western that cannot be fully

reproduced by LDR display 92

4.12: The HDR image Tree that cannot be fully reproduced by LDR display 92

4.13: Comparisons of the four different rendering of the

Mountain Tam West scene images. 93

4.14: Comparisons of the four different rendering of the Peck Lake scene images 93

file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225388file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225388file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225389file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225390file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225391file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225392file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225394file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225395file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225399file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225400file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225401file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225401file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225402file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225402file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225404file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225404

© CO

PYRI

GHT U

PM

xvi

4.15: Comparisons of the four different rendering of the Spherion

Price Western scene images. 94

4.16: Comparisons of the four different rendering of the Tree scene images. 95

4.17: Column charts of the statistic results for Outdoor Day Scene 97

4.18: The overall performance of tone mapping operator for outdoor day scene 98

4.19: The HDR image Dani Cathedral that cannot be fully reproduced

by LDR display 100

4.20: The HDR image Memorial that cannot be fully reproduced by LDR display 100

4.21: The HDR image Living Room that cannot be fully reproduced by

LDR display 101

4.22: The HDR image Rosette that cannot be fully reproduced by LDR display 101

4.23: Comparisons of the four different rendering of the Dani

Cathedral scene images. 102

4.24: Comparisons of the four different rendering of the Memorial scene images 103

4.25: Comparisons of the four different rendering of the Living

Room scene images. 104

4.26: Comparisons of the four different rendering of the Rosetta scene images 104

4.27: Column charts of the statistic results for Indoor Scene 106

4.28: The overall performance of tone mapping operator for indoor scene 107

4.30: The HDR image Foyer that cannot be fully reproduced by LDR display 109

4.29: The HDR image Display 1000 that cannot be fully reproduced

by LDR display 109

4.32: The HDR image Office that cannot be fully reproduced by LDR display 110

4.31: The HDR image Indoor that cannot be fully reproduced by LDR display 110

4.33: Comparisons of the four different rendering of the Display 1000

scene images. 111

4.34: Comparisons of the four different rendering of the Foyer scene images 111

4.35: Comparisons of the four different rendering of the Indoor scene images 112

file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225409file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225410file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225411file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225411file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225412file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225413file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225413file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225414file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225419file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225420file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225422file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225422file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225424

© CO

PYRI

GHT U

PM

xvii

4.36: Comparisons of the four different rendering of the Office scene images 112

4.37: Column charts of the statistic results for Indoor with

Outdoor /Outdoor with Indoor Scene 114

4.38: The overall performance of tone mapping operator for

Indoor with Outdoor /Outdoor with Indoor Scene 115

4.39: The overall performance of tone mapping operator for All Test Scene 118

4.40: (a) Ward tone mapped image (b) Proposed method tone mapped image

(c) Probability map generated from Ward tone mapped image

(d) Probability map generated from proposed method tone mapped image 121

4.41: (a) Ward tone mapped image (b) Proposed method tone mapped image(c)

Probability map generated from Ward tone mapped image (d) Probability map

generated from Proposed method tone mapped image 123

4.42: (a) Ward tone mapped image (b) Proposed method tone(c) Probability map

generated from Ward tone mapped image (d) Probability map generated from

Proposed method tone mapped image 124

4.43: (a) Ward tone mapped image (b) Proposed method tone mapped image

(c) Probability map generated from Ward tone mapped image

(d) Probability map generated from Proposed method tone mapped image 126

file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225429file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225429file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225430file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225430file:///C:/Users/user/Documents/correctedthesis.docx%23_Toc362225431

© CO

PYRI

GHT U

PM

xviii

LIST OF ABBREVIATIONS

CLAHE: Contrast Limited Adaptive Histogram Equalization

CRT: Cathode Ray Tube

HDR: High Dynamic Range

HVS: Human visual system

LCD: Liquid Crystal Display

LDR: Low Dynamic Range

LED: Light Emitting Diode

PPD: Panel display

SDR: Standard Dynamic Range

VDP: Visual Difference Predictor

© CO

PYRI

GHT U

PM

CHAPTER 1

INTRODUCTION

This chapter briefly introduces the tone mapping techniques which can be used to

reproduce the high dynamic range (HDR) images. Differences between the

standard digital image representation and the new high dynamic range imaging,

the dynamic range for scenes and display, and the global and local tone mapping

are explained too. The objectives of the research, problem statement and the thesis

organization are also described.

1.1 Tone Mapping

Tone mapping was originally designed for the field of art where artists make use

of a limited palette to depict high contrast scenes. Then this technique was later

developed for the use of photography and television (K. Devlin, 2002). The

purpose of the tone mapping is to display images or videos that correspond to the

visual impression of an observer watching the original scene. Tone mapping also

reproduce the contrast and brightness between the original scene and the output

device.

For good image reproduction, tone mapping must mimic the way the human

visual system (HVS) processes light information. The environment is full of wide

© CO

PYRI

GHT U

PM

2

range levels. HVS can have the ability to instantaneously and seamlessly adapt to

scenes with a HDR scene that can exceed from 0.0001 to more than 10000.

Examples of scenes and their approximate level is given in Figure 1.1 (Johnson,

2005).

Figure 1.1: Examples of scenes and their approximate level

1.1.1 Human Visual System

As mentioned in the previous section, the HVS system has almost 10 orders of

magnitude. Studies have shown that the retinal receptors have a dynamic range as

high as 10 orders of magnitude (Kuhna, 2011). Scene radiances are captured by

the rods and the cones in the retina and passed further to the visual pathway.

However, the optic nerves which transmit the visual signal only have a dynamic

range magnitude of 2 orders (Johnson, 2005). This discrepancy can be explicated

by a combination of visual phenomena processed by the HVS. The overall process

is often defined as human visual adaptation (E. Reinhard, 2005). HVS adapts the

illumination conditions by changing the diameter of the pupil depending on the

light that enters the eye. Besides, the cones adapt their sensitivities to the mean

luminance in the field of view. This is given by total retinal illumination and

finally the local adaption modulates local contrasts as HVS gaze visually scans the

© CO

PYRI

GHT U

PM

3

scene. Figure 1.2 gives a general plan of the human eye and the retina with the

names of their principal parts.

The image that had been captured is usually defined as linear with respect to

scene-referred coordinate because their pixels have a relation to the radiance in a

scene. Therefore, the main objective of tone mapping is to obtain an image from a

scene referred coordinate while maintaining a perceptual match between the real

scene and the displayed image, as shown in Figure 1.3. The figure shows the HVS

processes the scene radiances in a non-linear manner through different adaptation

processes. It forms a percept where all details are visible. Electronic devices

capture the scene radiances linearly. A tone mapping operator is necessary to non-

linearly encode the image as well as to map it to the display characteristics. This is

to ensure that the displayed image corresponds to our memory of the original

scene.

Figure 1.4 shows the difference between image captured by the camera and the

scene that the observer had seen. The image that the camera captured looks too

dark and loses massive amount of detail. Comparatively, the scene that the

observer had seen has more contrast in the shadow and preserves more detail.

© CO

PYRI

GHT U

PM

4

Figure 1.3: Perceptual match between a real world scene and displayed

image.

Match! Scene

Observer

Real-world

scene

Display Device

Observer

Tone mapping

Figure 1.4: Left: Image that is captured by the camera

Figure 1.2: The general plan of the human eye and the retina with the

names of their principal parts. (Original image of the eye courtesy of

Rhcastilhos, Wikimedia Commons.)

© CO

PYRI

GHT U

PM

5

1.1.2 The Dynamic Range of Scenes and Displays

Tone mapping can also be used to match the dynamic range scene with the display

device. Dynamic range of a digital image can be defined as the contrast ratio

between its brightest part and the darkest part while dynamic range of a scene is

the luminance ratio between the brightest and the darkest part in the scene (E.

Reinhard, 2005). Scene radiance can be measured using luminance, given in

candela per meter square (cd/m2). When the dynamic range of the digital image is

lower or higher than the dynamic range of a scene, tone mapping process will

either expand or compress the luminous intensity.

The dynamic range scene can be defined as the ratio of the luminance of the

brightest color (white) to that of the darkest color (black). The average contrast

ratio of those natural scenes is said to be 1:160. For scenes with fog, the contrast

ratio is lower than the average ratio but outdoor scenes can reach the ratio of

1:1000 (Hunt, 1995). The natural scene is considered being a low dynamic range

(LDR) when its dynamic range is lower than the output device. In order to view

the LDR scene in the output device, the input image should be expanded to fit the

output device. For the scene’s dynamic range to be approximately the same as the

output device, it can be defined as standard dynamic range (SDR). For the high

dynamic range (HDR) scene, the dynamic range of the input image far exceeds

the output device. As a result, certain parts of the displayed image will be clipped

during the capturing process. The most typical HDR scene is a view of indoor

room with visible scene outside the window.

http://en.wikipedia.org/wiki/Ratiohttp://en.wikipedia.org/wiki/Luminance

© CO

PYRI

GHT U

PM

6

Meanwhile, most of the conventional output display devices in the market are

plasma panel display (PPD), cathode ray tube (CRT), liquid crystal display (LCD)

and light emitting diode (LED) display. These output display devices are

classified as standard display and their dynamic range is standard as oppose to

HDR display which have a much larger dynamic range. Dynamic range of those

displays is generally from 1:100 to 1:1000. For current technologies, dynamic

range of the printer is typically from 1:50-1:1000. However, newly developed

HDR displays have a contrast ratio reaching 1:200,000, with the price cost of

$49,000.

1.2 The Global and Local Tone Mapping

There are two types of tone mapping algorithm: global (spatially uniform) and

local (spatially varying) (Barladian et al., 2004). For global operator, each pixel

applies the same transformation, for example; logarithmic function, gamma

function, and sigmoid function (see Chapter 2). Global tone mapping algorithm

applies the same function to all the pixels. Consequently, these global algorithms

tend to preserve the subjective perception of the scene. They have the advantage

of being fast and simple which makes them suitable for camera or video

processing. However, global tone mapping algorithm will cost the lost of image

detail information. (Venkata Lakshmi et al., 2012)

Local tone mapping operator applies a different scale to different parts of an

image. Different parts of the image will have different results which depend on

© CO

PYRI

GHT U

PM

7

the pixel position and on the surrounding pixel value. These methods makes the

local operator-based methods provide more details of the image than those global

operator based methods. However, they may also cause “halo” effects or ringing

artifacts in the reproduction.

When the dynamic range of the scene is approximate or lower than the output

display device, it is better to use global tone mapping. If the dynamic range of the

scene is far higher than the output display device, it is advised to use the global

tone mapping to compress the overall dynamic range of the HDR image. After the

compression, the image in this stage may lose a lot of contrast and detail of

visibility. Thus, the local tone mapping must be implemented to reproduce the

detail and increase the local contrast after the global tone mapping.

Recently, market starts to launch some monitors or televisions whose dynamic

range is close to that encountered in the real world. These types of HDR devices

make those HDR images able to display without much compression. As a result,

new tone mapping technique should be developed for rending of LDR, SDR and

HDR scenes to these displays. Moreover, for those images that had already tone-

mapped before this to suite the SDR display device, there is a need to re-tone map

them to the latest HDR display device. This can be done by reverse the operation

of the tone mapping.

© CO

PYRI

GHT U

PM

8

1.3 Problem Statement

The wide range of luminance available in high dynamic range images and videos

offers a high accuracy representation of natural scenes. However, the existing

display devices are insufficient to directly depict the exact contents. There are a

lot of industry field that really need high accuracy and visibility for images e.g.

Medical and Security Purpose. Therefore a tone mapping algorithm is required to

compress the wide range of luminance to display the reduced wide range of

luminance of HDR data in order to match the devices’ capabilities. The

dramatically reduction often reduce the quality of the original quality of HDR

contents. For that reason a successful tone mapping algorithm should produce a

LDR image which mimic the original scene of a HDR image with high accuracy

and at a minimal quality side effect.

1.4 Project Overview

In this thesis, we concentrate on the tone mapping of the HDR scene, i.e. scenes

with very wide luminance ratios. The HDR scene is normally of sunny outdoor

scenes or a view of indoor room with visible scene outside the window. These

HDR scenes are always facing difficulty to be displayed on SDR or LDR display

devices since their dynamic range is far higher than the display devices. As a

result, a local tone mapping algorithm needs to be introduced to increase the

contrast and reduce the loss of visual visibility caused by global tone mapping.

© CO

PYRI

GHT U

PM

9

1.5 Scope of Work

A local tone mapping method based on the modified contrast limited adaptive

histogram equalization (CLAHE) method which was originally developed for

medical imaging is used to render the HDR images to SDR displays. CLAHE has

effectively expanded the full dynamic range of display. Then, CLAHE method

was extended to become a new local tone mapping algorithm.

The tone mapping algorithm introduced was first compressed using global tone

mapping method. Then the image was segmented into smaller regions using

region splitting and merging. Next, morphological operation of opening and

closing were used to process the mask in order to apply the modified CLAHE in

different mask. After that is to decide every mask’s parameter clip limit. This step

alleviates the artifacts introduced by the original CLAHE algorithm. The original

CLAHE have only one clip limit and applied to each tile. The modified CLAHE

method applied different clip limit in different mask in order to expand the

dynamic range of the image. Software Matlab had been used to develop the

algorithms of the propose method. For performance evaluation, a subjective

evaluation and an objective evaluation are introduced. The subjective evaluation is

conducted by compare the proposed tone mapping operator with Drago, Reinhard

and Ward tone mapping algorithms. For objective evaluation, Ward (Larson et al.,

1997) and proposed tone mapping algorithm was compared with a reference scene

(HDR image) with HDR Visual Difference Predictor by Rafal Mantiuk. HDR

VDP can predict whether differences between two images are visible to the

human observer or not by generate probability map. The detection map shows

© CO

PYRI

GHT U

PM

10

how likely HVS will notice a difference between two images. Such metrics are

used for testing either visibility of information or visibility of noise. The two

generated probability maps was then compared to show is there any improvement

of proposed tone mapping algorithm with Ward's tone mapping algorithm. This is

due to the proposed tone mapping algorithm was the improved version of the

Ward’s tone mapping. The results shows that the proposed tone mapping

algorithm has successfully preserved the HDR image local contrast and detail.

1.6 Objectives

This research is to design a tone mapping operator that achieves the following

objectives:

To design and develop a local tone mapping algorithm to reproduce high

dynamic range image that match the display device.

To design and develop a fully automatic tone mapping algorithm without

deciding any input parameter.

To conduct an assessment to compare the images processed by proposed

tone mapping algorithm with other well known tone mapping operators.

© CO

PYRI

GHT U

PM

11

1.7 Thesis Contribution

The key contribution of this thesis is the development of a modified algorithm of

CLAHE. CLAHE which was originally developed for medical imaging used one

clip limit value and apply to each tile to enhance the picture. In this thesis, a

modified CLAHE algorithm was applied to expand the dynamic range of the

image after the global tone mapping. The modified CLAHE used morphological

operation of opening and closing to segment the image into smaller regions. Then,

the smaller regions used different clip limit value to increase the local contrast of

each region.

1.8 Thesis Organization

The thesis contains 5 chapters. Chapter 1 introduces an overall view of tone

mapping study. It addresses the research background, the purpose and objectives

of the work. Thesis organization is presented in the end of this chapter.

Chapter 2 reviews the global and local tone mapping methods. The basic global

tone mapping algorithms is described and some of the local tones mapping

techniques are discussed. The histogram equalization and CLAHE are also

introduced and lastly the tone mapping operator that based on histogram

equalization is reviewed.

© CO

PYRI

GHT U

PM

12

Chapter 3 presents the CLAHE-based local tone mapping method that we

developed to render HDR images to LDR displays. We first present the treatment

of the luminance channel that includes a global tone mapping processing followed

by a local processing based on CLAHE. The local processing uses region splitting

and merging to segment the image into smaller part and applied different clip

limit at different mask.

Chapter 4 presents the results and discussions. An assessment was conducted to

the fifteen images that had been processed by proposed tone mapping operator.

Comparison has also been made to the Reinhard’s, Drago’s and Ward’s tone

mapping operator and the result was discussed.

Chapter 5 presents the conclusions and future works. The summary of the

research is given and future work of the research is discussed.

1.9 Conclusion

The existing display devices in the market nowadays are insufficient to directly

depict the exact contents for HDR images. Therefore a tone mapping algorithm is

required to compress the wide range of luminance to display the reduced wide

range of luminance of HDR data to match the devices’ capabilities. In this chapter,

general information of why tone mapping process is important was addressed out.

The objectives and contribution of the proposed tone mapping were carried out.

© CO

PYRI

GHT U

PM

130

REFERENCE

Ashikhmin M. (2002) A tone mapping algorithm for high contrast images,

Proceedings of the 13th Eurographics workshop on Rendering,

Eurographics Association, Pisa, Italy. pp. 145-156.

Barladian B.K., Voloboi A.G., Galaktionov V.A., Kopylov E.A. (2004) An

Effective Tone Mapping Operator for High Dynamic Range Images.

Program. Comput. Softw. 30:266-272. DOI:

10.1023/b:pacs.0000043052.33460.48.

Chiu K., Herf M., Shirley P., Swamy S., Wang C., Zimmerman K. (1993)

Spatially Nonuniform Scaling Functions for High Contrast Images,

Graphics Interface. pp. 245-253.

Cohen J., Tchou C., Hawkins T., Debevec P.E. (2001) Real-Time High Dynamic

Range Texture Mapping, Proceedings of the 12th Eurographics Workshop

on Rendering Techniques, Springer-Verlag. pp. 313-320.

Dorsey F.D.a.J. (2002) Fast bilateral filtering for the display of high-dynamic-

range images, Proceedings of the 29th annual conference on Computer

graphics and interactive techniques, ACM, San Antonio, Texas. pp. 257-

266.

Drago F.A.M., Karol And Annen, Thomas And Chiba, Norishige. (2003)

Adaptive Logarithmic Mapping for Displaying High Contrast Scenes,

EUROGRAPHICS 2003, Blackwell, Granada, Spain. pp. 419-427.

Duan J., Bressan M., Dance C., Qiu G. (2010) Tone-mapping high dynamic range

images by novel histogram adjustment. Pattern Recognition 43:1847-1862.

DOI: 10.1016/j.patcog.2009.12.006.

E. Reinhard G.W., S. Pattanaik, and P. Debevec. ( 2005) High Dynamic Range

Imaging: Acquisition, Display, and Image-Based Lighting Morgan

Kaufmann Publishers Inc.

Fairchild G.J.B.a.M.D. (1999) Image Lightness Rescaling Using Sigmoidal

Contrast Enhancement Functions. Journal of Electronic Imaging

8:380-393.

Fattal R., Lischinski D., Werman M. (2002) Gradient domain high dynamic range

compression. ACM Trans. Graph. 21:249-256. DOI:

10.1145/566654.566573.

González R.C., Woods R.E. (1992) Digital image processing Addison-Wesley.

González R.C., Woods R.E. (2002) Digital image processing Prentice Hall.

© CO

PYRI

GHT U

PM

131

González R.C., Woods R.E., Eddins S.L. (2004) Digital Image processing using

MATLAB Pearson Prentice Hall.

Holm J. ( 1996) Photographic tone and colour reproduction goals, Proc.CIE

Expert Symposium'96 on Colour Standards for Image Technology, Vienna,

Austria. pp. 51-56.

Hunt R.W.G. ( 1995) The reproduction of colour, Fountain Press, England.

Jobson, D. J., Z. u. Rahman, et al. (1997). "A multiscale retinex for bridging the

gap between color images and the human observation of scenes." Image

Processing, IEEE Transactions on 6(7): 965-976.

Johnson G.M. (2005) Cares and concerns of CIE TC8-08: Spatial appearance

modeling and HDR rendering. Proc. IS&T/SPIE Electronic Imaging

5668:9.

Joohyun, L., J. Gwanggil, et al. (2009). "Piecewise tone reproduction for high

dynamic range imaging." Consumer Electronics, IEEE Transactions on

55(2): 911-918.

K. Devlin. (2002) A review of tone reproduction techniques. Technical Report

CSTR-02-005, Computer Science, Univ. of Bristol.

K. Matkovic L.N., and W. Purgathofer. (1997) A survey of tone mapping

techniques, 13th Spring Conference on Computer Graphics. pp. 163-170.

Khan I.R., Huang Z., Farbiz F., Manders C.M. (2009) HDR image tone mapping

using histogram adjustment adapted to human visual system, Proceedings

of the 7th international conference on Information, communications and

signal processing, IEEE Press, Macau, China. pp. 62-66.

Kuhna, M., M. Nuutinen, et al. (2011). Method for evaluating tone mapping

operators for natural high dynamic range images. Proc. SPIE 7876, Digital

Photography VII, 78760O-78760O.

Larson, G. W., H. Rushmeier, et al. (1997). "A visibility matching tone

reproduction operator for high dynamic range scenes." Visualization and

Computer Graphics, IEEE Transactions on 3(4): 291-306.

Ledda P., Chalmers A., Troscianko T., Seetzen H. (2005) Evaluation of tone

mapping operators using a High Dynamic Range display. ACM Trans.

Graph. 24:640-648. DOI: 10.1145/1073204.1073242.

MathWorks. (Oct 2003) Toolboxes: Image Processing Toolbox.

McNamara A. (2001) Visual Perception in Realistic Image Synthesis. Computer

Graphics Forum 20:211-224. DOI: 10.1111/1467-8659.00550.

© CO

PYRI

GHT U

PM

132

MEYLAN L. (2006) Tone Mapping for High Dynamic Range Images, Computer

and Communication Sciences, Ecole Polytechnique Federale de Lausanne

(EPFL), Lausanne. pp. 175.

N.J. Miller P.Y.N.a.D.D.M. (1984) The application of computer graphics in

lighting design. Journal of the IES 14:6-26.

Oppenheim A., Schafer R., Stockham T., Jr. (1968) Nonlinear filtering of

multiplied and convolved signals. Audio and Electroacoustics, IEEE

Transactions on 16:437-466.

Pattanaik S.N., Ferwerda J.A., Fairchild M.D., Greenberg D.P. (1998) A

multiscale model of adaptation and spatial vision for realistic image

display, Proceedings of the 25th annual conference on Computer graphics

and interactive techniques, ACM. pp. 287-298.

Poynton C. (2003 ) Digital video and HDTV, algorithms and interfaces, Morgan

Kaufmann publishers, San Francisco, CA. pp. Pages.

Qian T.L. (2011) High Dynamic Range Image Comparison.

Rafał Mantiuk S.D., Karol Myszkowski, Hans-Peter Seidel. (2005) Predicting

Visible Differences in High Dynamic Range Images - Model and its

Calibration, SPIE's 17th Annual Symposium on Electronic Imaging Proc.

of Human Vision and Electronic Imaging X. pp. 204-214.

Reinhard E., Stark M., Shirley P., Ferwerda J. (2002) Photographic tone

reproduction for digital images. ACM Trans. Graph. 21:267-276. DOI:

10.1145/566654.566575.

Stevens, S. S. S. a. J. C. (1960). "Brightness function: Parametric effects of

adaptation and contrast." journal of The Optical Society of America

53(1139).

Tumblin J., Rushmeier H. (1993) Tone reproduction for realistic images.

Computer Graphics and Applications, IEEE 13:42-48.

Tumblin J., Turk G. (1999) LCIS: a boundary hierarchy for detail-preserving

contrast reduction, Proceedings of the 26th annual conference on

Computer graphics and interactive techniques, ACM Press/Addison-

Wesley Publishing Co. pp. 83-90.

Tumblin J., Hodgins J.K., Guenter B.K. (1999) Two methods for display of high

contrast images. ACM Trans. Graph. 18:56-94. DOI:

10.1145/300776.300783.

Venkata Lakshmi S., Sujatha S., Janet J., Bellarmine T. (2012) Analysis of tone

mapping operators on high dynamic range images, Southeastcon, 2012

Proceedings of IEEE. pp. 1-6.

© CO

PYRI

GHT U

PM

133

Wen-Chung K., Chih-Hsiang H., Guan-Fan W., Chung-Yen S., Pei-Yung H.

(2010) Video recording in high dynamic range scenes, Green Circuits and

Systems (ICGCS), 2010 International Conference on. pp. 683-686.

Woods R.C.G.a.R.E. (2001) Digital Image Processing, Boston, MA, USA.

Zuiderveld K. (1994) Contrast limited adaptive histogram equalization, Graphics

gems IV, Academic Press Professional, Inc. pp. 474-485.

A MODIFIED CONTRAST LIMITED ADAPTIVE HISTOGRAM EQUALIZATION FOR HIGH DYNAMIC RANGE IMAGESABSTRACTTABLE OF CONTENTSCHAPTERSREFERENCE