seam carving

Kshitij Padalkar13th Oct 2008

Contents: In one Line…Need

Solutions and Other WorkDefinition and Maths

Energy , Seam Pixel Removal Criteria

Other Energy Functions?Discrete Image Resizing

Aspect Ratio Change Optimal Seam Order

Image Enlarging Content Amplification Object Removal

Multi-size ImagesLimitationsLinks

In One Line…

Method for Content-Aware Image Resizing

Need

Different Displays-Different Resolutions-(Different Images??)

HTML allows automatic layout of the text, but not the images

Solutions and Other WorkCrop and ScaleSaliencySegmentation

Contents: In one Line…Need

Solutions and Other WorkDefinition and Maths

Energy , Seam Pixel Removal Criteria

Other Energy Functions?Discrete Image Resizing

Aspect Ratio Change Optimal Seam Order

Image Enlarging Content Amplification Object Removal

Multi-size ImagesLimitationsLinks

Definition and Maths…Goal is to remove unnoticeable pixels.Energy Function:

Remove Pixels with least energySort pixels by Energy, remove smallest pixels

(optimum)Remove Column with smallest total Energy

(column)Remove Seam with smallest total Energy (seam)

SeamVertical Seam:

an 8-connected path of pixels in the image from top to bottom, containing one, and only one, pixel in each row of the image

SeamEnergy of a Seam

Minimum Energy Seam

Pixel Removal CriteriaOptimal

Pixel Removal CriteriaPixel

Pixel Removal CriteriaColumn

Pixel Removal CriteriaSeam

Pixel Removal CriteriaComparison

Other Energy FunctionsGradientEntropyHOG (Histogram of Gradient)Saliency

Contents: In one Line…Need

Solutions and Other WorkDefinition and Maths

Energy , Seam Pixel Removal Criteria

Other Energy Functions?Discrete Image Resizing

Aspect Ratio Change Optimal Seam Order

Image Enlarging Content Amplification Object Removal

Multi-size ImagesLimitationsLinks

Aspect Ratio ChangeChange the image from (n X m) to (n X m’)

(assume m’ < m)Remove m-m’=c seams successively

Seam Carving

Aspect Ratio ChangeChange the image from (n X m) to (n X m’)

(assume m’ < m)Remove m-m’=c seams successively

Scaling

Optimal Seam OrderingChange the image from (n X m) to (n’ X m’)

(assume m’ < m and n’ < n)What to do? What is the correct order of seam

carving?Remove vertical seams first? Horizontal seams first? Or alternate between the two?

Optimal Seam OrderingWe optimize

where k = r+c, r = (m−m’), c = (n−n’) and i is used as a parameter that determine if at step i we remove a horizontal or vertical seam: = {0,1}

Optimal Seam OrderingTransport map

Matrix of size n X mEach element T(r,c) holds the minimal cost

needed to obtain an image of size n−r X m−c

Discrete Image ResizingAspect Ratio Change

Optimal Seam OrderImage EnlargingContent AmplificationObject Removal

Image Enlargingcompute the optimal vertical (horizontal) seam

‘s’ on I and duplicate the pixels of s by averaging them with their left and right neighbors (top and bottom in the horizontal case)

Image EnlargingTo reduce the width by ‘k’, compute first ‘k’

vertical seams and duplicate themAll seams duplicated, 50% enlargement. For

more do it in steps.

Discrete Image ResizingAspect Ratio Change

Optimal Seam OrderImage EnlargingContent AmplificationObject Removal

Content AmplificationScale the image, this will scale everything,

content as well as non-content.Shrink the scaled-image by seam carving, this

will carve out the non-content part.

Discrete Image ResizingAspect Ratio Change

Optimal Seam OrderImage EnlargingContent AmplificationObject Removal

Object RemovalUser marks the target object to be removed. Then seams are removed from the image until

all marked pixels are gone.To regain the original size, seam insertion is

employed.

Object RemovalFind the missing shoe…

Object RemovalObject marking to remove unwanted results

Contents: In one Line…Need

Solutions and Other WorkDefinition and Maths

Energy , Seam Pixel Removal Criteria

Other Energy Functions?Discrete Image Resizing

Aspect Ratio Change Optimal Seam Order

Image Enlarging Content Amplification Object Removal

Multi-size ImagesLimitationsLinks

Multi Size ImagesMethods mentioned so far are not real-time.We calculate each seam remove it and

calculate the next seam.

For real-time, we calculate-index map V of size n×m that encodes, for

each pixel, the index of the seam that removed it, i.e., V(i, j) = t means that pixel (i, j) was removed by the t-th seam removal.

Multi Size ImagesHorizontal Index map (H)Vertical Index map (V)

To get an image of width m’, we need to remove m-m’ pixels from each rowwe gather, in each row, all pixels with seam

index greater than or equal to m−m’.Same for changing the height.

Problem comes when we want to change both!

Contents: In one Line…Need

Solutions and Other WorkDefinition and Maths

Energy , Seam Pixel Removal Criteria

Other Energy Functions?Discrete Image Resizing

Aspect Ratio Change Optimal Seam Order

Image Enlarging Content Amplification Object Removal

Multi-size ImagesLimitationsLinks

Limitations

Too much contentNo space for seam to avoid content

Paper ftp://ftp1.idc.ac.il/Arik_shamir/SCweb/imret/vidret.pdf

Youtube demohttp://www.youtube.com/watch?v=6NcIJXTlugc

Search google for Seam Carving…. Many many blogs…

THANK YOU