Context-Aware Saliency DetectionUnder guidance of, Submitted by,Mr. Sanjay D.S., BE,Mtech , Apoorva A. Prabhu Asst. Professor , 4AI09CS009.Dept. of CS&E.

Outline

• Abstract• Introduction• Existing system

• Principles of context-aware saliency• Pseudo code• Results• Conclusion

Abstract

This Algorithm introduces a new method of saliency detection called Context aware Saliency detection. It differs from other saliency detection algorithms by that it not only identifies dominant object in the image but also its context. In some applications such as image retargeting and image summarization the context of the dominant object is as important as the dominant object itself.

Introduction

• This a new type of saliency algorithm contain not only the prominent objects but also the parts of the background that convey the context.

• The algorithm follows four psychological principles of human visual attention.• Local low-level considerations • Global considerations• Visual organization rules• High level factors

Existing system

Some existing algorithms are driven by low level stimulus such as color, contrast, orientation and texture. But they only succeed in finding some key fixation points rather than the regions of visual interest. Other approaches use global features and are based on finding regions in the image which have unique frequencies in the Fourier domain. They quickly locate the region of visual interest but they lack in detecting object boundaries accurately.

Principles of context-aware saliency

• 1. Local low-level considerations, including factors such as contrast and color. the distinctive color and other features area should obtain high

attention.

• 2. Global considerations, which suppress frequently occurring features, while maintaining features that deviate from the norm. redundant information should be suppressed and popping up the novelty part.

Principles of context-aware saliency

• 3. Visual organization rules, which state that visual forms may possess one or several centers of gravity about which the form is organized. the salient pixels should be grouped together, and not spread all over

the image.

• 4. High-level factors, such as human faces. implemented as post-processing.

Principles of context-aware saliency

(a)Local[24] (b)Global[7] (c)Local-global[13] (d)Context-aware

Comparing different approaches to saliency

Local-global single-scale saliency

• 1.

• 2.

We should not, however, look at an isolatedpixel, but rather at its surrounding patch.

Let dcolor(pi , pj) be the Euclidean distance between the vectorized patches pi and pj in CIE L*a*b colorspace, dposition(pi , pj) be the Euclidean distance between thepositions of patches pi and pj

It suffices to consider the K most similar patches

The single-scale saliency value of pixel i at scale r is defined as left . {qk} k=1 to K , K = 64 in our experiments

Multi-scale saliency enhancement

• 3.

• 4.

Background pixels (patches) are likely to have similar patches at multiple scales.

For a patch pi of scale r, we consider as candidate neighbors all the patches in the image whose scales are Rq = {r ,1/2r ,1/4r} .

Let R denote the set of patch sizes to be considered for pixel i.The saliency at pixel i is taken as the mean of its saliency at different scales

The larger Si is, the more salient pixel i is and the larger is its dissimilarity to the other patches.

Including the immediate context

• 1: A pixel is considered attended if its saliency value exceeds a certain threshold( Si > 0.8 in the examples

shown in this paper).

• 2: The saliency of a pixel is redefined as

Let dfoci(i) be the Euclidean positional distance between pixel i and the closest focus of attention pixel, normalized to the range [0,1]

Pseudo code• Step 1 : resize the image and convert the rgb to lab color

space.• Step 2 : The Euclidean distance and dissimilarities between

patches is found out, to find single local global saliency.• Step 3 : Multi-scale the image to further decrease saliency of

background pixels and also visual contextual effect is simulated.

• Step 4 : Center prior consideration is made (optional). • Step 5 : The high level factors can be considered now as a part

of pre-processing.

Steps

• The steps of our saliency estimation algorithm

High-level factors• Final , face detection algorithm

Si = Si , if Si > face(i)

face(i) , otherwise

face detection algorithm , which generates 1 for face pixels and 0 otherwise.

modified

Salient results (1)

From left to right input , result of [1] , result of [2], our result .

Compare to other methods with three cases image.

(a) a single object over an uninteresting Background.

(b) the immediate surroundings of the salient object is also salient.

(c) complex scenes.

Conclusion and Future Enhancements

• The proposed approach is new type of saliency – context aware saliency based on four principles observed in the psychological literature which detects the important parts of the scene.

• In the future we intend to learn the benefits of this saliency in more applications, such as image classification and thumb-nailing.

References

[1] Stas Goferman, Lihi Zelnik -Manor and Ayellet Tal .,” Context Aware Saliency Detection”, IEEE Transactions On Pattern Analysis and Machine Intelligence, Vol. 34 ,No. 10, October 2012.[2] D.Walther and C. Koch., “ Modelling attention to salient proto objects”, Neural Networks, 19(9):1395–1407, 2006.[3] X. Hou and L. Zhang. Saliency detection: A spectral residual approach. In CVPR, pages 1–8, 2007.

Thank you…